2023-07-21T18:17:48.357 INFO:root:teuthology version: 0.0.1.dev153+g79491cb 2023-07-21T18:17:48.358 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_rishabh@teuthology --name rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} --archive /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864 -- /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/orig.config.yaml 2023-07-21T18:17:48.419 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T18:17:53.113 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864 branch: wip-rishabh-2023Jul13-m-quota description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} email: null first_in_suite: false job_id: '7346864' kernel: flavor: default kdb: true sha1: c685da6f56cc77f2cfb9c4221be3960522616caf last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-rishabh-2023Jul13-m-quota ceph: cephfs: max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - slow metadata IO - SLOW_OPS - slow request sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 subvols: create: 2 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 0 mountpoint: /tmp/cephfs check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount install: ceph: flavor: default sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 kclient: client.0: mntopts: - context=system_u:object_r:postgresql_db_t:s0 mount_subvol_num: 1 mountpoint: /tmp/cephfs selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-rishabh-2023Jul13-m-quota sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 owner: scheduled_rishabh@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 sleep_before_teardown: 0 suite: fs suite_branch: wip-rishabh-2023Jul13-m-quota suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi005.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac= smithi101.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi156.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - ceph-fuse: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - exec: client.0: - sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/ - sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres - sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data - sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?" - sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local - sudo restorecon -R -v /tmp/cephfs/postgres - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf - sudo -u postgres -- postgresql-setup --initdb - sudo ls -lZaR /tmp/cephfs/postgres/ - sudo systemctl start postgresql - sudo -u postgres -- pgbench -s 500 -i - sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900 - sudo systemctl stop postgresql - sudo ls -lZaR /tmp/cephfs/postgres/ teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 79491cb69dd37a1eafb93b3471998174bdfa2fd9 timestamp: 2023-07-21_17:58:19 tube: smithi user: rishabh verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.23459 2023-07-21T18:17:53.113 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa; will attempt to use it 2023-07-21T18:17:53.114 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks 2023-07-21T18:17:53.114 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-07-21T18:17:53.117 INFO:teuthology.task.internal:Checking packages... 2023-07-21T18:17:53.158 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '81cf785aa7781c4e08381c540f890cb9bc987ef9' 2023-07-21T18:17:53.158 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-07-21T18:17:53.159 INFO:teuthology.packaging:ref: None 2023-07-21T18:17:53.159 INFO:teuthology.packaging:tag: None 2023-07-21T18:17:53.159 INFO:teuthology.packaging:branch: wip-rishabh-2023Jul13-m-quota 2023-07-21T18:17:53.159 INFO:teuthology.packaging:sha1: 81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:17:53.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-rishabh-2023Jul13-m-quota 2023-07-21T18:17:53.498 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-4746.g81cf785a 2023-07-21T18:17:53.499 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-07-21T18:17:53.505 INFO:teuthology.task.internal:no buildpackages task found 2023-07-21T18:17:53.505 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-07-21T18:17:53.526 INFO:teuthology.task.internal:Saving configuration 2023-07-21T18:17:53.547 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-07-21T18:17:53.554 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T18:17:58.587 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.013420', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:18:08.623 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.010484', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:18:13.657 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.007635', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T18:18:13.658 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-07-21T18:18:13.664 INFO:teuthology.task.internal:roles: ubuntu@smithi005.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-07-21T18:18:13.665 INFO:teuthology.task.internal:roles: ubuntu@smithi101.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-07-21T18:18:13.665 INFO:teuthology.task.internal:roles: ubuntu@smithi156.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-07-21T18:18:13.665 INFO:teuthology.run_tasks:Running task console_log... 2023-07-21T18:18:30.048 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f7ebfbc9670>, signals=[15]) 2023-07-21T18:18:30.049 INFO:teuthology.run_tasks:Running task internal.connect... 2023-07-21T18:18:30.056 INFO:teuthology.task.internal:Opening connections... 2023-07-21T18:18:30.057 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T18:18:30.058 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:18:30.131 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi101.front.sepia.ceph.com 2023-07-21T18:18:30.132 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:18:30.200 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi156.front.sepia.ceph.com 2023-07-21T18:18:30.201 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:18:30.268 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-07-21T18:18:30.275 DEBUG:teuthology.orchestra.run.smithi005:> uname -m 2023-07-21T18:18:30.296 INFO:teuthology.orchestra.run.smithi005.stdout:x86_64 2023-07-21T18:18:30.297 DEBUG:teuthology.orchestra.run.smithi005:> cat /etc/os-release 2023-07-21T18:18:30.354 INFO:teuthology.orchestra.run.smithi005.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:ID="rhel" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:ID_LIKE="fedora" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:VERSION_ID="8.6" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:18:30.355 INFO:teuthology.orchestra.run.smithi005.stdout:ANSI_COLOR="0;31" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:18:30.356 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:18:30.357 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:18:30.357 INFO:teuthology.orchestra.run.smithi005.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:18:30.359 INFO:teuthology.lock.ops:Updating smithi005.front.sepia.ceph.com on lock server 2023-07-21T18:18:35.385 DEBUG:teuthology.orchestra.run.smithi101:> uname -m 2023-07-21T18:18:35.423 INFO:teuthology.orchestra.run.smithi101.stdout:x86_64 2023-07-21T18:18:35.424 DEBUG:teuthology.orchestra.run.smithi101:> cat /etc/os-release 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:ID="rhel" 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:ID_LIKE="fedora" 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:VERSION_ID="8.6" 2023-07-21T18:18:35.482 INFO:teuthology.orchestra.run.smithi101.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:ANSI_COLOR="0;31" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:18:35.483 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:18:35.484 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:18:35.484 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:18:35.484 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:18:35.484 INFO:teuthology.orchestra.run.smithi101.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:18:35.484 INFO:teuthology.lock.ops:Updating smithi101.front.sepia.ceph.com on lock server 2023-07-21T18:18:40.509 DEBUG:teuthology.orchestra.run.smithi156:> uname -m 2023-07-21T18:18:40.530 INFO:teuthology.orchestra.run.smithi156.stdout:x86_64 2023-07-21T18:18:40.530 DEBUG:teuthology.orchestra.run.smithi156:> cat /etc/os-release 2023-07-21T18:18:40.590 INFO:teuthology.orchestra.run.smithi156.stdout:NAME="Red Hat Enterprise Linux" 2023-07-21T18:18:40.590 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION="8.6 (Ootpa)" 2023-07-21T18:18:40.590 INFO:teuthology.orchestra.run.smithi156.stdout:ID="rhel" 2023-07-21T18:18:40.590 INFO:teuthology.orchestra.run.smithi156.stdout:ID_LIKE="fedora" 2023-07-21T18:18:40.590 INFO:teuthology.orchestra.run.smithi156.stdout:VERSION_ID="8.6" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:PLATFORM_ID="platform:el8" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:ANSI_COLOR="0;31" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:HOME_URL="https://www.redhat.com/" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-07-21T18:18:40.591 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:18:40.592 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-07-21T18:18:40.592 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-07-21T18:18:40.592 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-07-21T18:18:40.592 INFO:teuthology.orchestra.run.smithi156.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-07-21T18:18:40.592 INFO:teuthology.lock.ops:Updating smithi156.front.sepia.ceph.com on lock server 2023-07-21T18:18:45.616 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-07-21T18:18:45.627 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-07-21T18:18:45.643 INFO:teuthology.task.internal:Checking for old test directory... 2023-07-21T18:18:45.644 DEBUG:teuthology.orchestra.run.smithi005:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:18:45.647 DEBUG:teuthology.orchestra.run.smithi101:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:18:45.649 DEBUG:teuthology.orchestra.run.smithi156:> test '!' -e /home/ubuntu/cephtest 2023-07-21T18:18:45.667 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-07-21T18:18:45.714 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-07-21T18:18:45.714 DEBUG:teuthology.orchestra.run.smithi005:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:18:45.716 DEBUG:teuthology.orchestra.run.smithi101:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:18:45.718 DEBUG:teuthology.orchestra.run.smithi156:> test -z $(ls -A /var/lib/ceph) 2023-07-21T18:18:45.750 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-07-21T18:18:54.548 INFO:teuthology.run_tasks:Running task kernel... 2023-07-21T18:18:54.564 INFO:teuthology.task.kernel:normalize config orig: {'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'} 2023-07-21T18:18:54.565 INFO:teuthology.task.kernel:config {'mon.a': {'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'}, 'mon.b': {'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'}, 'mon.c': {'flavor': 'default', 'kdb': True, 'sha1': 'c685da6f56cc77f2cfb9c4221be3960522616caf'}}, timeout 300 2023-07-21T18:18:54.565 DEBUG:teuthology.orchestra.run.smithi005:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:18:54.566 DEBUG:teuthology.orchestra.run.smithi101:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:18:54.566 DEBUG:teuthology.orchestra.run.smithi156:> test -f /run/.containerenv -o -f /.dockerenv 2023-07-21T18:18:54.585 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:18:54.586 DEBUG:teuthology.task.kernel:sha1 for mon.c is c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.586 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "c685da6f56cc77f2cfb9c4221be3960522616caf"... 2023-07-21T18:18:54.587 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-07-21T18:18:54.587 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:18:54.588 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:18:54.589 DEBUG:teuthology.task.kernel:sha1 for mon.a is c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.589 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "c685da6f56cc77f2cfb9c4221be3960522616caf"... 2023-07-21T18:18:54.589 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-07-21T18:18:54.590 DEBUG:teuthology.task.kernel:sha1 for mon.b is c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.590 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "c685da6f56cc77f2cfb9c4221be3960522616caf"... 2023-07-21T18:18:54.590 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-07-21T18:18:54.646 INFO:teuthology.orchestra.run.smithi156.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T18:18:54.646 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.646 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-21T18:18:54.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.651 INFO:teuthology.orchestra.run.smithi005.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T18:18:54.652 INFO:teuthology.orchestra.run.smithi101.stdout:4.18.0-372.9.1.el8.x86_64 2023-07-21T18:18:54.652 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.652 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-21T18:18:54.652 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.654 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-372.9.1.el8.x86_64 vs c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.654 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-07-21T18:18:54.654 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:18:54.986 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum upgrade -y linux-firmware 2023-07-21T18:18:54.990 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum upgrade -y linux-firmware 2023-07-21T18:18:54.991 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum upgrade -y linux-firmware 2023-07-21T18:18:56.224 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-21T18:18:56.225 INFO:teuthology.orchestra.run.smithi156.stdout:Unable to read consumer identity 2023-07-21T18:18:56.245 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-21T18:18:56.246 INFO:teuthology.orchestra.run.smithi101.stdout:Unable to read consumer identity 2023-07-21T18:18:56.297 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-07-21T18:18:56.297 INFO:teuthology.orchestra.run.smithi005.stdout:Unable to read consumer identity 2023-07-21T18:18:57.842 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 9.6 kB/s | 3.5 kB 00:00 2023-07-21T18:18:57.843 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-07-21T18:18:57.848 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-21T18:19:03.242 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 2.9 MB/s | 16 MB 00:05 2023-07-21T18:19:06.684 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 310 kB/s | 24 kB 00:00 2023-07-21T18:19:07.308 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 1.7 MB/s | 16 MB 00:09 2023-07-21T18:19:07.464 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-21T18:19:07.465 INFO:teuthology.orchestra.run.smithi156.stdout:Nothing to do. 2023-07-21T18:19:07.465 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-21T18:19:07.598 INFO:teuthology.task.kernel:Downloading kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.c... 2023-07-21T18:19:07.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:19:07.824 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ 2023-07-21T18:19:07.824 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-21T18:19:07.825 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 1.6 MB/s | 16 MB 00:09 2023-07-21T18:19:07.827 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi156.front.sepia.ceph.com to complete... 2023-07-21T18:19:10.834 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 404 kB/s | 24 kB 00:00 2023-07-21T18:19:11.448 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 392 kB/s | 24 kB 00:00 2023-07-21T18:19:11.610 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-21T18:19:11.611 INFO:teuthology.orchestra.run.smithi101.stdout:Nothing to do. 2023-07-21T18:19:11.611 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-21T18:19:11.729 INFO:teuthology.task.kernel:Downloading kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.b... 2023-07-21T18:19:11.730 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:19:16.738 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-07-21T18:19:16.738 INFO:teuthology.orchestra.run.smithi005.stdout:Nothing to do. 2023-07-21T18:19:16.738 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-07-21T18:19:16.739 INFO:teuthology.task.kernel:Downloading kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.a... 2023-07-21T18:19:16.739 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F8%2Fx86_64&sha1=c685da6f56cc77f2cfb9c4221be3960522616caf 2023-07-21T18:19:22.057 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ 2023-07-21T18:19:22.058 DEBUG:teuthology.orchestra.run.smithi101:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-21T18:19:22.061 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi101.front.sepia.ceph.com to complete... 2023-07-21T18:19:22.141 INFO:teuthology.task.kernel:fetching, builder baseurl is https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ 2023-07-21T18:19:22.141 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/ --input-file=- 2023-07-21T18:19:22.144 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi005.front.sepia.ceph.com to complete... 2023-07-21T18:22:19.101 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21 18:22:19 URL:https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm [380683892/380683892] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-21T18:22:19.102 INFO:teuthology.orchestra.run.smithi156.stderr:FINISHED --2023-07-21 18:22:19-- 2023-07-21T18:22:19.102 INFO:teuthology.orchestra.run.smithi156.stderr:Total wall clock time: 3m 11s 2023-07-21T18:22:19.102 INFO:teuthology.orchestra.run.smithi156.stderr:Downloaded: 1 files, 363M in 3m 11s (1.90 MB/s) 2023-07-21T18:22:19.103 INFO:teuthology.task.kernel:Installing kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.c... 2023-07-21T18:22:19.104 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-21T18:22:20.712 INFO:teuthology.orchestra.run.smithi156.stdout:Verifying... ######################################## 2023-07-21T18:22:20.784 INFO:teuthology.orchestra.run.smithi156.stdout:Preparing... ######################################## 2023-07-21T18:22:20.888 INFO:teuthology.orchestra.run.smithi156.stdout:Updating / installing... 2023-07-21T18:22:58.797 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-6.5.0_rc1_gc685da6f56cc-1 ######################################## 2023-07-21T18:23:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21 18:23:12 URL:https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm [380683892/380683892] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-21T18:23:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:FINISHED --2023-07-21 18:23:12-- 2023-07-21T18:23:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:Total wall clock time: 3m 50s 2023-07-21T18:23:12.030 INFO:teuthology.orchestra.run.smithi005.stderr:Downloaded: 1 files, 363M in 3m 49s (1.58 MB/s) 2023-07-21T18:23:12.031 INFO:teuthology.task.kernel:Installing kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.a... 2023-07-21T18:23:12.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-21T18:23:13.705 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying... ######################################## 2023-07-21T18:23:13.767 INFO:teuthology.orchestra.run.smithi005.stdout:Preparing... ######################################## 2023-07-21T18:23:13.855 INFO:teuthology.orchestra.run.smithi005.stdout:Updating / installing... 2023-07-21T18:23:25.052 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi156.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-21T18:23:25.052 DEBUG:teuthology.orchestra.run.smithi156:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-21T18:23:25.099 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.099 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/config-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.099 INFO:teuthology.orchestra.run.smithi156.stdout:/boot/vmlinuz-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-21T18:23:25.100 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-21T18:23:25.101 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-21T18:23:25.102 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-21T18:23:25.103 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm.ko 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ansi_cprng.ko 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/anubis.ko 2023-07-21T18:23:25.104 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/arc4.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_memcpy.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_pq.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_tx.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_xor.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/raid6test.ko 2023-07-21T18:23:25.105 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authenc.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authencesn.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blake2b_generic.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_common.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_generic.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/camellia_generic.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast5_generic.ko 2023-07-21T18:23:25.106 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast6_generic.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast_common.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ccm.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cmac.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crc32_generic.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_engine.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_user.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cts.ko 2023-07-21T18:23:25.107 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/des_generic.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/dh_generic.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/echainiv.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/essiv.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/fcrypt.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/khazad.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4hc.ko 2023-07-21T18:23:25.108 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/md4.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/michael_mic.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcbc.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcrypt.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/rmd160.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/seed.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/serpent_generic.ko 2023-07-21T18:23:25.109 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tcrypt.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tea.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_common.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_generic.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/vmac.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/wp512.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xcbc.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xor.ko 2023-07-21T18:23:25.110 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xxhash_generic.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_pad.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/custom_method.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/ec_sys.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/platform_profile.ko 2023-07-21T18:23:25.111 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbs.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbshc.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/video.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/acard-ahci.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ahci_platform.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ata_generic.ko 2023-07-21T18:23:25.112 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/libahci_platform.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_acpi.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ali.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_amd.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_artop.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atiixp.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atp867x.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd640.ko 2023-07-21T18:23:25.113 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd64x.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cypress.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_efar.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt366.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt37x.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it8213.ko 2023-07-21T18:23:25.114 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it821x.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_jmicron.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_marvell.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_mpiix.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_netcell.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ninja32.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87410.ko 2023-07-21T18:23:25.115 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87415.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_oldpiix.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_opti.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_optidma.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pcmcia.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-21T18:23:25.116 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_piccolo.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_rdc.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sch.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_serverworks.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sil680.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sis.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sl82c105.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_triflex.ko 2023-07-21T18:23:25.117 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_via.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pdc_adma.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_inic162x.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_mv.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_nv.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_promise.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_qstor.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil.ko 2023-07-21T18:23:25.118 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil24.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sis.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_svw.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sx4.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_uli.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_via.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_vsc.ko 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay 2023-07-21T18:23:25.119 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay/ks0108.ko 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma/bcma.ko 2023-07-21T18:23:25.120 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe/aoe.ko 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/brd.ko 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd/drbd.ko 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/floppy.ko 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/loop.ko 2023-07-21T18:23:25.121 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/nbd.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk/null_blk.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/pktcdvd.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/rbd.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/virtio_blk.ko 2023-07-21T18:23:25.122 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkfront.ko 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram/zram.ko 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hangcheck-timer.ko 2023-07-21T18:23:25.123 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/via-rng.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-21T18:23:25.124 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/lp.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave/mwave.ko 2023-07-21T18:23:25.125 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ppdev.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tlclk.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-21T18:23:25.126 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/uv_mmtimer.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/virtio_console.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-21T18:23:25.127 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp.ko 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel 2023-07-21T18:23:25.128 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-aes.ko 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-sha.ko 2023-07-21T18:23:25.129 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca/dca.ko 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw 2023-07-21T18:23:25.130 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/amd64_edac.ko 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/e752x_edac.ko 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/edac_mce_amd.ko 2023-07-21T18:23:25.131 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3000_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3200_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5100_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5400_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7300_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7core_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i82975x_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/ie31200_edac.ko 2023-07-21T18:23:25.132 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/sb_edac.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/x38_edac.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/edd.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-ich.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-21T18:23:25.133 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-appleir.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-aureal.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-axff.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-dr.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elecom.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elo.ko 2023-07-21T18:23:25.134 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-emsff.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gaff.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gt683r.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gyration.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtekff.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-hyperv.ko 2023-07-21T18:23:25.135 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-icade.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-keytouch.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-kye.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lcpower.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-led.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lenovo.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-21T18:23:25.136 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-multitouch.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-ortek.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-penmount.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-petalynx.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-picolcd.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-pl.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-plantronics.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-primax.ko 2023-07-21T18:23:25.137 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-rmi.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-common.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-21T18:23:25.138 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-saitek.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-samsung.ko 2023-07-21T18:23:25.139 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sjoy.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sony.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-speedlink.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-steelseries.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sunplus.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-21T18:23:25.140 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tivo.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tmff.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-topseed.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-twinhan.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-uclogic.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-waltop.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-wiimote.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-xinmo.ko 2023-07-21T18:23:25.141 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zpff.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zydacron.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/uhid.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/wacom.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_balloon.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_utils.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_vmbus.ko 2023-07-21T18:23:25.142 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru3.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7414.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7418.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adc128d818.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1025.ko 2023-07-21T18:23:25.143 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1026.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1029.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1031.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm9240.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ads7828.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7410.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7411.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7462.ko 2023-07-21T18:23:25.144 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7470.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7475.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7x10.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/amc6821.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/applesmc.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asb100.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asc7621.ko 2023-07-21T18:23:25.145 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/atxp1.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/coretemp.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dme1737.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds1621.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds620.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc1403.ko 2023-07-21T18:23:25.146 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc6w201.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71805f.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71882fg.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f75375s.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fam15h_power.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fschmd.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g760a.ko 2023-07-21T18:23:25.147 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g762.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl518sm.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl520sm.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5500_temp.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5k_amb.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmaem.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmpex.ko 2023-07-21T18:23:25.148 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina209.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina2xx.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/it87.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k10temp.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k8temp.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lineage-pem.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm63.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm73.ko 2023-07-21T18:23:25.149 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm75.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm77.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm78.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm80.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm83.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm85.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm87.ko 2023-07-21T18:23:25.150 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm90.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm92.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm93.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95234.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95241.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95245.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc2945.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4151.ko 2023-07-21T18:23:25.151 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4215.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4222.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4245.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4260.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4261.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max16065.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1619.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1668.ko 2023-07-21T18:23:25.152 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max197.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6639.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6650.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6697.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/mcp3021.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6683.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775-core.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775.ko 2023-07-21T18:23:25.153 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct7802.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87360.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87427.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pcf8591.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-21T18:23:25.154 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-21T18:23:25.155 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/powr1220.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5627.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5636.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht15.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht21.ko 2023-07-21T18:23:25.156 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/shtc1.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sis5595.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47b397.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m1.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m192.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/thmc50.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp102.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp103.ko 2023-07-21T18:23:25.157 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp401.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp421.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via-cputemp.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via686a.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt1211.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt8231.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627ehf.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627hf.ko 2023-07-21T18:23:25.158 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83781d.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83791d.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83792d.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83793.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83795.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l785ts.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l786ng.ko 2023-07-21T18:23:25.159 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-21T18:23:25.160 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-21T18:23:25.161 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-21T18:23:25.162 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-dev.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-mux.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-21T18:23:25.163 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-smbus.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-stub.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_core.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-21T18:23:25.164 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4 2023-07-21T18:23:25.165 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-21T18:23:25.166 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-21T18:23:25.167 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/gameport.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/lightning.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/ns558.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/joydev.ko 2023-07-21T18:23:25.168 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/appletouch.ko 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/bcm5974.ko 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/cyapatp.ko 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-21T18:23:25.169 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/sermouse.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/altera_ps2.ko 2023-07-21T18:23:25.170 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/arc_ps2.ko 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/serio_raw.ko 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/sparse-keymap.ko 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds 2023-07-21T18:23:25.171 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-blinkm.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lm3530.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lp3944.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-ss4200.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-21T18:23:25.172 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache 2023-07-21T18:23:25.173 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache/bcache.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-bio-prison.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache-smq.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-crypt.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-delay.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-flakey.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-log-userspace.ko 2023-07-21T18:23:25.174 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-multipath.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-queue-length.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-raid.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-round-robin.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-service-time.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-switch.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-thin-pool.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-verity.ko 2023-07-21T18:23:25.175 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/faulty.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/linear.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/multipath.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid0.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid1.ko 2023-07-21T18:23:25.176 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid10.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid456.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptbase.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptctl.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptfc.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptlan.ko 2023-07-21T18:23:25.177 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptsas.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptscsih.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptspi.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_ich.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_sch.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/mfd-core.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/sm501.ko 2023-07-21T18:23:25.178 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/viperboard.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/vx855.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/wl1273-core.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds9802als.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds990x.ko 2023-07-21T18:23:25.179 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/bh1770glc.ko 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710/cb710.ko 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/at24.ko 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-21T18:23:25.180 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/max6875.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/enclosure.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/hpilo.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29003.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29020.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-21T18:23:25.181 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-me.ko 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-txe.ko 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei.ko 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-21T18:23:25.182 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_7xx1.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_core.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tsl2550.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_balloon.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-21T18:23:25.183 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding/bonding.ko 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-21T18:23:25.184 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dummy.ko 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/eql.ko 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-21T18:23:25.185 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-21T18:23:25.186 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-21T18:23:25.187 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-21T18:23:25.188 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom 2023-07-21T18:23:25.189 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-21T18:23:25.190 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-21T18:23:25.191 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco 2023-07-21T18:23:25.192 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-21T18:23:25.193 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dnet.ko 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex 2023-07-21T18:23:25.194 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ethoc.ko 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/fealnx.ko 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-21T18:23:25.195 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-21T18:23:25.196 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-21T18:23:25.197 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/jme.ko 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-21T18:23:25.198 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom 2023-07-21T18:23:25.199 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia 2023-07-21T18:23:25.200 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-21T18:23:25.201 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-21T18:23:25.202 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-21T18:23:25.203 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-21T18:23:25.204 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-21T18:23:25.205 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-21T18:23:25.206 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/geneve.ko 2023-07-21T18:23:25.207 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ifb.ko 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvlan.ko 2023-07-21T18:23:25.208 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvtap.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mii.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/net_failover.ko 2023-07-21T18:23:25.209 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/netconsole.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/nlmon.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/amd.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-21T18:23:25.210 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm87xx.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/broadcom.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/cicada.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/davicom.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/dp83640.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/et1011c.ko 2023-07-21T18:23:25.211 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/icplus.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/lxt.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/marvell.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/micrel.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/national.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/phylink.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/qsemi.ko 2023-07-21T18:23:25.212 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/realtek.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/smsc.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/ste10Xp.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/vitesse.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_async.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-21T18:23:25.213 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppoe.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppox.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pptp.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slhc.ko 2023-07-21T18:23:25.214 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slip.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/sungem_phy.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tap.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-21T18:23:25.215 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_random.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tun.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/veth.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/virtio_net.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan 2023-07-21T18:23:25.216 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan/vxlan.ko 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netfront.ko 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb/ntb.ko 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme 2023-07-21T18:23:25.217 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme-core.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_cs.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_pc.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_serial.ko 2023-07-21T18:23:25.218 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/xen-pcifront.ko 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/i82092.ko 2023-07-21T18:23:25.219 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/pd6729.ko 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-21T18:23:25.220 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acerhdf.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-21T18:23:25.221 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/hdaps.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel_ips.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-21T18:23:25.222 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-21T18:23:25.223 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi.ko 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-21T18:23:25.224 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps_parport.ko 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/pps_core.ko 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp.ko 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp_kvm.ko 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-21T18:23:25.225 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-21T18:23:25.226 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-em3027.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-21T18:23:25.227 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-max6900.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-21T18:23:25.228 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-x1205.ko 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi 2023-07-21T18:23:25.229 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-9xxx.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-sas.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-xxxx.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/BusLogic.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/a100u2w.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/advansys.ko 2023-07-21T18:23:25.230 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/am53c974.ko 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/atp870u.ko 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi 2023-07-21T18:23:25.231 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa/bfa.ko 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-21T18:23:25.232 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ch.ko 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-21T18:23:25.233 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dc395x.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dmx3191d.ko 2023-07-21T18:23:25.234 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esp_scsi.ko 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic/fnic.ko 2023-07-21T18:23:25.235 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hpsa.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hptiop.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hv_storvsc.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/initio.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ipr.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ips.ko 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci 2023-07-21T18:23:25.236 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci/isci.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc/libfc.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas 2023-07-21T18:23:25.237 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas/libsas.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-21T18:23:25.238 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvumi.ko 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pmcraid.ko 2023-07-21T18:23:25.239 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla1280.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/raid_class.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_debug.ko 2023-07-21T18:23:25.240 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ses.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/st.ko 2023-07-21T18:23:25.241 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/stex.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/virtio_scsi.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/wd719x.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb/ssb.ko 2023-07-21T18:23:25.242 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_file.ko 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_iblock.ko 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_mod.ko 2023-07-21T18:23:25.243 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_pscsi.ko 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_user.ko 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-21T18:23:25.244 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-21T18:23:25.245 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty 2023-07-21T18:23:25.246 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_gsm.ko 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_hdlc.ko 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/nozomi.ko 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-21T18:23:25.247 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/arc_uart.ko 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/synclink_gt.ko 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host 2023-07-21T18:23:25.248 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_aec.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_cif.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_pci_generic.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_sercos3.ko 2023-07-21T18:23:25.249 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-acm.ko 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usblp.ko 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usbtmc.ko 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-21T18:23:25.250 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/mdc800.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/microtek.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/adutux.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/appledisplay.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi26.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi62.ko 2023-07-21T18:23:25.251 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ezusb.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/idmouse.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/iowarrior.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ldusb.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/legousbtower.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-21T18:23:25.252 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usb3503.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usblcd.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/uss720.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/yurex.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy/phy-generic.ko 2023-07-21T18:23:25.253 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/aircable.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ark3116.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ch341.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cp210x.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cyberjack.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-21T18:23:25.254 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/empeg.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_ti.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipaq.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipw.ko 2023-07-21T18:23:25.255 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ir-usb.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mct_u232.ko 2023-07-21T18:23:25.256 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7720.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7840.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/navman.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/omninet.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/opticon.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/option.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/oti6858.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/pl2303.ko 2023-07-21T18:23:25.257 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcaux.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcserial.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/quatech2.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/safe_serial.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/sierra.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ssu100.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/symbolserial.ko 2023-07-21T18:23:25.258 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_debug.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/visor.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/whiteheat.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage 2023-07-21T18:23:25.259 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/uas.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-21T18:23:25.260 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-karma.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/usb-storage.ko 2023-07-21T18:23:25.261 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-21T18:23:25.262 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_net.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_scsi.ko 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video 2023-07-21T18:23:25.263 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/apple_bl.ko 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/lcd.ko 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio.ko 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_balloon.ko 2023-07-21T18:23:25.264 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_mmio.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_ring.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2482.ko 2023-07-21T18:23:25.265 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2490.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-21T18:23:25.266 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/wire.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-21T18:23:25.267 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/hpwdt.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/i6300esb.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ib700wdt.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ibmasr.ko 2023-07-21T18:23:25.268 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it87_wdt.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/machzwd.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/nv_tco.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-21T18:23:25.269 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/softdog.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/via_wdt.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/wdt_pci.ko 2023-07-21T18:23:25.270 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/xen_wdt.ko 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-evtchn.ko 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntalloc.ko 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntdev.ko 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback 2023-07-21T18:23:25.271 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-privcmd.ko 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p/9p.ko 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs 2023-07-21T18:23:25.272 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs/affs.ko 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs/befs.ko 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/binfmt_misc.ko 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs/btrfs.ko 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles/cachefiles.ko 2023-07-21T18:23:25.273 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph/ceph.ko 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda/coda.ko 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs/cramfs.ko 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm/dlm.ko 2023-07-21T18:23:25.274 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs/f2fs.ko 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/fat.ko 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/msdos.ko 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/vfat.ko 2023-07-21T18:23:25.275 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache/fscache.ko 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/cuse.ko 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/fuse.ko 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2/gfs2.ko 2023-07-21T18:23:25.276 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs/hfs.ko 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus/hfsplus.ko 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs/isofs.ko 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs/jfs.ko 2023-07-21T18:23:25.277 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd/lockd.ko 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix/minix.ko 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs/netfs.ko 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout 2023-07-21T18:23:25.278 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfs.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv3.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv4.ko 2023-07-21T18:23:25.279 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/grace.ko 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/nfs_acl.ko 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd/nfsd.ko 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2/nilfs2.ko 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls 2023-07-21T18:23:25.280 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-celtic.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-centeuro.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-croatian.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-cyrillic.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-gaelic.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-greek.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-iceland.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-inuit.ko 2023-07-21T18:23:25.281 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-roman.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-romanian.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-turkish.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1250.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1251.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1255.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp737.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp775.ko 2023-07-21T18:23:25.282 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp850.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp852.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp855.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp857.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp860.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp861.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp862.ko 2023-07-21T18:23:25.283 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp863.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp864.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp865.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp866.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp869.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp874.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp932.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp936.ko 2023-07-21T18:23:25.284 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp949.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp950.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_euc-jp.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-1.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-13.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-14.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-15.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-2.ko 2023-07-21T18:23:25.285 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-3.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-4.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-5.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-6.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-7.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-9.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-r.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-ru.ko 2023-07-21T18:23:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-u.ko 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_utf8.ko 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs 2023-07-21T18:23:25.287 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs/overlay.ko 2023-07-21T18:23:25.288 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore/ramoops.ko 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs/reiserfs.ko 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs/romfs.ko 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client 2023-07-21T18:23:25.289 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client/cifs.ko 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_arc4.ko 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_md4.ko 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs/squashfs.ko 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv/sysv.ko 2023-07-21T18:23:25.290 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf/udf.ko 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs/ufs.ko 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs/xfs.ko 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu 2023-07-21T18:23:25.291 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu/rcutorture.ko 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/torture.ko 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/asn1_encoder.ko 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc-itu-t.ko 2023-07-21T18:23:25.292 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc8.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libarc4.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libdes.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/libcrc32c.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lru_cache.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4_compress.ko 2023-07-21T18:23:25.293 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4hc_compress.ko 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math/cordic.ko 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6/raid6_pq.ko 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_bm.ko 2023-07-21T18:23:25.294 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_fsm.ko 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_kmp.ko 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd/zstd_compress.ko 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm/hwpoison-inject.ko 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan 2023-07-21T18:23:25.295 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/6lowpan.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_dest.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_fragment.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_hop.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_mobility.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_routing.ko 2023-07-21T18:23:25.296 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_udp.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/garp.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/mrp.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/p8022.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/psnap.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/stp.ko 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q 2023-07-21T18:23:25.297 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q/8021q.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_fd.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_rdma.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_virtio.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv/batman-adv.ko 2023-07-21T18:23:25.298 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/br_netfilter.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/bridge.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-21T18:23:25.299 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-21T18:23:25.300 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-21T18:23:25.301 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtables.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph/libceph.ko 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core 2023-07-21T18:23:25.302 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/failover.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/pktgen.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_diag.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv4.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv6.ko 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver 2023-07-21T18:23:25.303 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver/dns_resolver.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/dsa_core.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_brcm.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_dsa.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_none.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_trailer.ko 2023-07-21T18:23:25.304 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154.ko 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ah4.ko 2023-07-21T18:23:25.305 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/esp4.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/fou.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/gre.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/inet_diag.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_gre.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_tunnel.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_vti.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipcomp.ko 2023-07-21T18:23:25.306 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipip.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-21T18:23:25.307 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-21T18:23:25.308 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-21T18:23:25.309 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_bic.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_dctcp.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_diag.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_highspeed.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_htcp.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_hybla.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_illinois.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_lp.ko 2023-07-21T18:23:25.310 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_scalable.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_vegas.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_veno.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_westwood.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_yeah.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tunnel4.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_diag.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_tunnel.ko 2023-07-21T18:23:25.311 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ah6.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/esp6.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/fou6.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_tunnel.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_vti.ko 2023-07-21T18:23:25.312 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ipcomp6.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-21T18:23:25.313 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-21T18:23:25.314 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-21T18:23:25.315 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/sit.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/tunnel6.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key 2023-07-21T18:23:25.316 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key/af_key.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_core.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_eth.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip6.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_netlink.ko 2023-07-21T18:23:25.317 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ppp.ko 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc/llc.ko 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154/mac802154.ko 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls/mpls_gso.ko 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter 2023-07-21T18:23:25.318 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-21T18:23:25.319 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-21T18:23:25.320 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-21T18:23:25.321 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-21T18:23:25.322 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conncount.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-21T18:23:25.323 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-21T18:23:25.324 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_log_syslog.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_irc.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_sip.ko 2023-07-21T18:23:25.325 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_tables.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_log.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-21T18:23:25.326 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_chain_nat.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_compat.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_ct.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fib.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_hash.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_limit.ko 2023-07-21T18:23:25.327 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_log.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_masq.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_nat.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_numgen.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_queue.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_quota.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_redir.ko 2023-07-21T18:23:25.328 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject_inet.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_socket.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_tunnel.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_AUDIT.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-21T18:23:25.329 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CT.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_DSCP.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HL.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HMARK.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LED.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LOG.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-21T18:23:25.330 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NETMAP.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFLOG.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_RATEEST.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_SECMARK.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-21T18:23:25.331 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TEE.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TPROXY.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TRACE.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_addrtype.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_bpf.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cgroup.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cluster.ko 2023-07-21T18:23:25.332 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_comment.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connbytes.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlabel.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlimit.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connmark.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_conntrack.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cpu.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dccp.ko 2023-07-21T18:23:25.333 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_devgroup.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dscp.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ecn.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_esp.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hashlimit.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_helper.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hl.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipcomp.ko 2023-07-21T18:23:25.334 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_iprange.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipvs.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_l2tp.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_length.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_limit.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mac.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mark.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_multiport.ko 2023-07-21T18:23:25.335 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nat.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nfacct.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_osf.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_owner.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_physdev.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_pkttype.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_policy.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_quota.ko 2023-07-21T18:23:25.336 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_rateest.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_realm.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_recent.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_sctp.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_set.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_socket.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_state.ko 2023-07-21T18:23:25.337 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_statistic.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_string.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_tcpmss.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_time.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_u32.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink/netlink_diag.ko 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh 2023-07-21T18:23:25.338 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh/nsh.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/openvswitch.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-geneve.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-gre.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-vxlan.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet/af_packet_diag.ko 2023-07-21T18:23:25.339 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_rdma.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_tcp.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_csum.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_gact.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_ipt.ko 2023-07-21T18:23:25.340 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_mirred.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_nat.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_pedit.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_police.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_simple.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_skbedit.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_vlan.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_basic.ko 2023-07-21T18:23:25.341 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_bpf.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_flow.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_fw.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_route.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_u32.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_cmp.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_ipset.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_meta.ko 2023-07-21T18:23:25.342 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_nbyte.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_text.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_u32.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_choke.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_codel.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_drr.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_fq.ko 2023-07-21T18:23:25.343 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_gred.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hfsc.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hhf.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_htb.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_ingress.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio_lib.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_multiq.ko 2023-07-21T18:23:25.344 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_netem.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_pie.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_plug.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_prio.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_qfq.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_red.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfb.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfq.ko 2023-07-21T18:23:25.345 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_tbf.ko 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_teql.ko 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp.ko 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp_diag.ko 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-21T18:23:25.346 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/sunrpc.ko 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/diag.ko 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/tipc.ko 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix 2023-07-21T18:23:25.347 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix/unix_diag.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-21T18:23:25.348 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys/trusted.ko 2023-07-21T18:23:25.349 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib/irqbypass.ko 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias.bin 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.bin 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.modinfo 2023-07-21T18:23:25.350 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep.bin 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.devname 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.order 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.softdep 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols 2023-07-21T18:23:25.351 INFO:teuthology.orchestra.run.smithi156.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols.bin 2023-07-21T18:23:25.352 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.352 DEBUG:teuthology.orchestra.run.smithi156:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-21T18:23:25.380 INFO:teuthology.orchestra.run.smithi156.stdout:postinstall scriptlet (using /bin/sh): 2023-07-21T18:23:25.380 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc -a -r /boot/System.map-6.5.0-rc1-gc685da6f56cc ]; then 2023-07-21T18:23:25.381 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:23:25.381 INFO:teuthology.orchestra.run.smithi156.stdout:cp /boot/System.map-6.5.0-rc1-gc685da6f56cc /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:23:25.381 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.381 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/installkernel 6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:23:25.382 INFO:teuthology.orchestra.run.smithi156.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:23:25.382 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-07-21T18:23:25.382 INFO:teuthology.orchestra.run.smithi156.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-21T18:23:25.383 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-21T18:23:25.383 INFO:teuthology.orchestra.run.smithi156.stdout:new-kernel-pkg --remove 6.5.0-rc1-gc685da6f56cc --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-gc685da6f56cc.img 2023-07-21T18:23:25.383 INFO:teuthology.orchestra.run.smithi156.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-21T18:23:25.383 INFO:teuthology.orchestra.run.smithi156.stdout:kernel-install remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.384 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-07-21T18:23:25.384 INFO:teuthology.orchestra.run.smithi156.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-21T18:23:25.384 INFO:teuthology.orchestra.run.smithi156.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-21T18:23:25.384 INFO:teuthology.orchestra.run.smithi156.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.385 INFO:teuthology.orchestra.run.smithi156.stdout:fi 2023-07-21T18:23:25.386 DEBUG:teuthology.orchestra.run.smithi156:> sudo rpm -qi grub2-tools 2023-07-21T18:23:25.465 INFO:teuthology.orchestra.run.smithi156.stdout:Name : grub2-tools 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Epoch : 1 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Version : 2.02 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Release : 123.el8 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Architecture: x86_64 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Group : System Environment/Base 2023-07-21T18:23:25.466 INFO:teuthology.orchestra.run.smithi156.stdout:Size : 9265450 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:License : GPLv3+ 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Relocations : (not relocatable) 2023-07-21T18:23:25.467 INFO:teuthology.orchestra.run.smithi156.stdout:Packager : Red Hat, Inc. 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:Vendor : Red Hat, Inc. 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:Summary : Support tools for GRUB. 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:Description : 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-21T18:23:25.468 INFO:teuthology.orchestra.run.smithi156.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-21T18:23:25.469 INFO:teuthology.orchestra.run.smithi156.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-21T18:23:25.469 INFO:teuthology.orchestra.run.smithi156.stdout:hardware devices. 2023-07-21T18:23:25.469 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:23:25.469 INFO:teuthology.orchestra.run.smithi156.stdout:This subpackage provides tools for support of all platforms. 2023-07-21T18:23:25.470 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-21T18:23:25.470 INFO:teuthology.task.kernel:Updating grub on smithi156 to boot 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:25.470 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-21T18:23:27.993 INFO:teuthology.orchestra.run.smithi156.stderr:Generating grub configuration file ... 2023-07-21T18:23:30.040 INFO:teuthology.orchestra.run.smithi156.stderr:done 2023-07-21T18:23:30.049 DEBUG:teuthology.orchestra.run.smithi156:> mktemp 2023-07-21T18:23:30.067 INFO:teuthology.orchestra.run.smithi156.stdout:/tmp/tmp.K9kqsjhElG 2023-07-21T18:23:30.067 DEBUG:teuthology.orchestra.run.smithi156:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.K9kqsjhElG 2023-07-21T18:23:30.140 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 0666 /tmp/tmp.K9kqsjhElG 2023-07-21T18:23:30.292 DEBUG:teuthology.orchestra.remote:smithi156:/tmp/tmp.K9kqsjhElG is 6KB 2023-07-21T18:23:30.311 DEBUG:teuthology.orchestra.run.smithi156:> rm -fr /tmp/tmp.K9kqsjhElG 2023-07-21T18:23:30.328 DEBUG:teuthology.orchestra.run.smithi156:> sudo /bin/ls /boot/loader/entries || true 2023-07-21T18:23:30.398 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-21T18:23:30.399 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-21T18:23:30.399 INFO:teuthology.orchestra.run.smithi156.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc.conf 2023-07-21T18:23:30.400 DEBUG:teuthology.orchestra.run.smithi156:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:23:30.566 DEBUG:teuthology.orchestra.run.smithi156:> sudo shutdown -r now 2023-07-21T18:23:42.285 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21 18:23:42 URL:https://4.chacra.ceph.com/r/kernel/testing/c685da6f56cc77f2cfb9c4221be3960522616caf/centos/8/flavors/default/x86_64/kernel-6.5.0_rc1_gc685da6f56cc-1.x86_64.rpm [380683892/380683892] -> "/tmp/kernel.x86_64.rpm" [1] 2023-07-21T18:23:42.286 INFO:teuthology.orchestra.run.smithi101.stderr:FINISHED --2023-07-21 18:23:42-- 2023-07-21T18:23:42.286 INFO:teuthology.orchestra.run.smithi101.stderr:Total wall clock time: 4m 20s 2023-07-21T18:23:42.286 INFO:teuthology.orchestra.run.smithi101.stderr:Downloaded: 1 files, 363M in 4m 20s (1.40 MB/s) 2023-07-21T18:23:42.287 INFO:teuthology.task.kernel:Installing kernel c685da6f56cc77f2cfb9c4221be3960522616caf on mon.b... 2023-07-21T18:23:42.287 DEBUG:teuthology.orchestra.run.smithi101:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2023-07-21T18:23:43.886 INFO:teuthology.orchestra.run.smithi101.stdout:Verifying... ######################################## 2023-07-21T18:23:43.937 INFO:teuthology.orchestra.run.smithi101.stdout:Preparing... ######################################## 2023-07-21T18:23:44.043 INFO:teuthology.orchestra.run.smithi101.stdout:Updating / installing... 2023-07-21T18:23:54.329 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-6.5.0_rc1_gc685da6f56cc-1 ######################################## 2023-07-21T18:24:00.569 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2023-07-21T18:24:00.569 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:24:00.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:24:18.431 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi005.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-21T18:24:18.431 DEBUG:teuthology.orchestra.run.smithi005:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-21T18:24:18.483 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.484 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/config-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.484 INFO:teuthology.orchestra.run.smithi005.stdout:/boot/vmlinuz-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel 2023-07-21T18:24:18.484 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch 2023-07-21T18:24:18.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86 2023-07-21T18:24:18.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto 2023-07-21T18:24:18.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-21T18:24:18.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-21T18:24:18.485 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-21T18:24:18.486 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-21T18:24:18.487 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel 2023-07-21T18:24:18.488 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-21T18:24:18.489 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm.ko 2023-07-21T18:24:18.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto 2023-07-21T18:24:18.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ansi_cprng.ko 2023-07-21T18:24:18.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/anubis.ko 2023-07-21T18:24:18.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/arc4.ko 2023-07-21T18:24:18.490 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_memcpy.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_pq.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_tx.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_xor.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/raid6test.ko 2023-07-21T18:24:18.491 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authenc.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authencesn.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blake2b_generic.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_common.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_generic.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/camellia_generic.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast5_generic.ko 2023-07-21T18:24:18.492 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast6_generic.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast_common.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ccm.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cmac.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crc32_generic.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_engine.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_user.ko 2023-07-21T18:24:18.493 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cts.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/des_generic.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/dh_generic.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/echainiv.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/essiv.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/fcrypt.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/khazad.ko 2023-07-21T18:24:18.494 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4hc.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/md4.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/michael_mic.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcbc.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcrypt.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/rmd160.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/seed.ko 2023-07-21T18:24:18.495 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/serpent_generic.ko 2023-07-21T18:24:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tcrypt.ko 2023-07-21T18:24:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tea.ko 2023-07-21T18:24:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_common.ko 2023-07-21T18:24:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_generic.ko 2023-07-21T18:24:18.496 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/vmac.ko 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/wp512.ko 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xcbc.ko 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xor.ko 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xxhash_generic.ko 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi 2023-07-21T18:24:18.497 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_pad.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/custom_method.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/ec_sys.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/platform_profile.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbs.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbshc.ko 2023-07-21T18:24:18.498 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/video.ko 2023-07-21T18:24:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata 2023-07-21T18:24:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/acard-ahci.ko 2023-07-21T18:24:18.499 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ahci_platform.ko 2023-07-21T18:24:18.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ata_generic.ko 2023-07-21T18:24:18.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/libahci_platform.ko 2023-07-21T18:24:18.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_acpi.ko 2023-07-21T18:24:18.500 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ali.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_amd.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_artop.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atiixp.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atp867x.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd640.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd64x.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cypress.ko 2023-07-21T18:24:18.501 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_efar.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt366.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt37x.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it8213.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it821x.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_jmicron.ko 2023-07-21T18:24:18.502 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_marvell.ko 2023-07-21T18:24:18.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_mpiix.ko 2023-07-21T18:24:18.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_netcell.ko 2023-07-21T18:24:18.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ninja32.ko 2023-07-21T18:24:18.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87410.ko 2023-07-21T18:24:18.503 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87415.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_oldpiix.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_opti.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_optidma.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pcmcia.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-21T18:24:18.504 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_piccolo.ko 2023-07-21T18:24:18.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_rdc.ko 2023-07-21T18:24:18.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sch.ko 2023-07-21T18:24:18.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_serverworks.ko 2023-07-21T18:24:18.505 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sil680.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sis.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sl82c105.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_triflex.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_via.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pdc_adma.ko 2023-07-21T18:24:18.506 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_inic162x.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_mv.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_nv.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_promise.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_qstor.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil24.ko 2023-07-21T18:24:18.507 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sis.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_svw.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sx4.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_uli.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_via.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_vsc.ko 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay 2023-07-21T18:24:18.508 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay/ks0108.ko 2023-07-21T18:24:18.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base 2023-07-21T18:24:18.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap 2023-07-21T18:24:18.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-21T18:24:18.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-21T18:24:18.509 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma 2023-07-21T18:24:18.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma/bcma.ko 2023-07-21T18:24:18.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block 2023-07-21T18:24:18.510 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe 2023-07-21T18:24:18.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe/aoe.ko 2023-07-21T18:24:18.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/brd.ko 2023-07-21T18:24:18.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd 2023-07-21T18:24:18.511 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd/drbd.ko 2023-07-21T18:24:18.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/floppy.ko 2023-07-21T18:24:18.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/loop.ko 2023-07-21T18:24:18.512 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx 2023-07-21T18:24:18.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-21T18:24:18.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/nbd.ko 2023-07-21T18:24:18.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk 2023-07-21T18:24:18.513 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk/null_blk.ko 2023-07-21T18:24:18.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/pktcdvd.ko 2023-07-21T18:24:18.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/rbd.ko 2023-07-21T18:24:18.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/virtio_blk.ko 2023-07-21T18:24:18.514 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback 2023-07-21T18:24:18.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-21T18:24:18.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkfront.ko 2023-07-21T18:24:18.515 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram 2023-07-21T18:24:18.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram/zram.ko 2023-07-21T18:24:18.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char 2023-07-21T18:24:18.516 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hangcheck-timer.ko 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/via-rng.ko 2023-07-21T18:24:18.517 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-21T18:24:18.518 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/lp.ko 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave/mwave.ko 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ppdev.ko 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tlclk.ko 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm.ko 2023-07-21T18:24:18.519 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/uv_mmtimer.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/virtio_console.ko 2023-07-21T18:24:18.520 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq 2023-07-21T18:24:18.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-21T18:24:18.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-21T18:24:18.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-21T18:24:18.521 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-21T18:24:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-21T18:24:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-21T18:24:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto 2023-07-21T18:24:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp 2023-07-21T18:24:18.522 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp.ko 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-21T18:24:18.523 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-aes.ko 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-sha.ko 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca 2023-07-21T18:24:18.524 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca/dca.ko 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-21T18:24:18.525 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/amd64_edac.ko 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/e752x_edac.ko 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/edac_mce_amd.ko 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3000_edac.ko 2023-07-21T18:24:18.526 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3200_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5100_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5400_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7300_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7core_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i82975x_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/ie31200_edac.ko 2023-07-21T18:24:18.527 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/sb_edac.ko 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/x38_edac.ko 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/edd.ko 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-ich.ko 2023-07-21T18:24:18.528 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-appleir.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-aureal.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-axff.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-dr.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elecom.ko 2023-07-21T18:24:18.529 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elo.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-emsff.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gaff.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gt683r.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gyration.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-21T18:24:18.530 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtekff.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-hyperv.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-icade.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-keytouch.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-kye.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lcpower.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-led.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lenovo.ko 2023-07-21T18:24:18.531 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-multitouch.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-ortek.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-penmount.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-petalynx.ko 2023-07-21T18:24:18.532 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-picolcd.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-pl.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-plantronics.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-primax.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-rmi.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-21T18:24:18.533 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-common.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-21T18:24:18.534 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-saitek.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-samsung.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-21T18:24:18.535 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sjoy.ko 2023-07-21T18:24:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sony.ko 2023-07-21T18:24:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-speedlink.ko 2023-07-21T18:24:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-steelseries.ko 2023-07-21T18:24:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sunplus.ko 2023-07-21T18:24:18.536 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tivo.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tmff.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-topseed.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-twinhan.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-uclogic.ko 2023-07-21T18:24:18.537 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-waltop.ko 2023-07-21T18:24:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-wiimote.ko 2023-07-21T18:24:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-xinmo.ko 2023-07-21T18:24:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zpff.ko 2023-07-21T18:24:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zydacron.ko 2023-07-21T18:24:18.538 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/uhid.ko 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/wacom.ko 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_balloon.ko 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_utils.ko 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_vmbus.ko 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon 2023-07-21T18:24:18.539 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru3.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7414.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7418.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adc128d818.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1025.ko 2023-07-21T18:24:18.540 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1026.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1029.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1031.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm9240.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ads7828.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7410.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7411.ko 2023-07-21T18:24:18.541 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7462.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7470.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7475.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7x10.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/amc6821.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/applesmc.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asb100.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asc7621.ko 2023-07-21T18:24:18.542 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-21T18:24:18.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/atxp1.ko 2023-07-21T18:24:18.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/coretemp.ko 2023-07-21T18:24:18.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-21T18:24:18.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dme1737.ko 2023-07-21T18:24:18.543 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds1621.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds620.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc1403.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc6w201.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71805f.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71882fg.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f75375s.ko 2023-07-21T18:24:18.544 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fam15h_power.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fschmd.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g760a.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g762.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl518sm.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl520sm.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-21T18:24:18.545 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5500_temp.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5k_amb.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmaem.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmpex.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina209.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina2xx.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/it87.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k10temp.ko 2023-07-21T18:24:18.546 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k8temp.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lineage-pem.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm63.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm73.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm75.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm77.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm78.ko 2023-07-21T18:24:18.547 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm80.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm83.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm85.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm87.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm90.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm92.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm93.ko 2023-07-21T18:24:18.548 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95234.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95241.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95245.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc2945.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4151.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4215.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4222.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4245.ko 2023-07-21T18:24:18.549 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4260.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4261.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max16065.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1619.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1668.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max197.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6639.ko 2023-07-21T18:24:18.550 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6650.ko 2023-07-21T18:24:18.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6697.ko 2023-07-21T18:24:18.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/mcp3021.ko 2023-07-21T18:24:18.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6683.ko 2023-07-21T18:24:18.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775-core.ko 2023-07-21T18:24:18.551 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775.ko 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct7802.ko 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87360.ko 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87427.ko 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pcf8591.ko 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus 2023-07-21T18:24:18.552 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-21T18:24:18.553 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/powr1220.ko 2023-07-21T18:24:18.554 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5627.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5636.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht15.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht21.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/shtc1.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sis5595.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47b397.ko 2023-07-21T18:24:18.555 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m1.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m192.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/thmc50.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp102.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp103.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp401.ko 2023-07-21T18:24:18.556 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp421.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via-cputemp.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via686a.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt1211.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt8231.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627ehf.ko 2023-07-21T18:24:18.557 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627hf.ko 2023-07-21T18:24:18.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83781d.ko 2023-07-21T18:24:18.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83791d.ko 2023-07-21T18:24:18.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83792d.ko 2023-07-21T18:24:18.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83793.ko 2023-07-21T18:24:18.558 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83795.ko 2023-07-21T18:24:18.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l785ts.ko 2023-07-21T18:24:18.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l786ng.ko 2023-07-21T18:24:18.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c 2023-07-21T18:24:18.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos 2023-07-21T18:24:18.559 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-21T18:24:18.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-21T18:24:18.560 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses 2023-07-21T18:24:18.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-21T18:24:18.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-21T18:24:18.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-21T18:24:18.561 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-21T18:24:18.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-21T18:24:18.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-21T18:24:18.562 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-21T18:24:18.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-21T18:24:18.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-21T18:24:18.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-21T18:24:18.563 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-21T18:24:18.564 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-21T18:24:18.564 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-21T18:24:18.564 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-21T18:24:18.565 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-21T18:24:18.565 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-21T18:24:18.565 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-21T18:24:18.565 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-dev.ko 2023-07-21T18:24:18.566 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-mux.ko 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-smbus.ko 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-stub.ko 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-21T18:24:18.567 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_core.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw 2023-07-21T18:24:18.568 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca 2023-07-21T18:24:18.569 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-21T18:24:18.570 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp 2023-07-21T18:24:18.571 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-21T18:24:18.571 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt 2023-07-21T18:24:18.571 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-21T18:24:18.571 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input 2023-07-21T18:24:18.571 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/gameport.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/lightning.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/ns558.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/joydev.ko 2023-07-21T18:24:18.572 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/appletouch.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/bcm5974.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/cyapatp.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/sermouse.ko 2023-07-21T18:24:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio 2023-07-21T18:24:18.574 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/altera_ps2.ko 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/arc_ps2.ko 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/serio_raw.ko 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/sparse-keymap.ko 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd 2023-07-21T18:24:18.575 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-blinkm.ko 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lm3530.ko 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lp3944.ko 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-ss4200.ko 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger 2023-07-21T18:24:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-21T18:24:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache/bcache.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-bio-prison.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache-smq.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-crypt.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-delay.ko 2023-07-21T18:24:18.578 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-flakey.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-log-userspace.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-multipath.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-queue-length.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-raid.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-round-robin.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-service-time.ko 2023-07-21T18:24:18.579 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-switch.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-thin-pool.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-verity.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/faulty.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/linear.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/multipath.ko 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data 2023-07-21T18:24:18.580 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid0.ko 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid1.ko 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid10.ko 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid456.ko 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion 2023-07-21T18:24:18.581 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptbase.ko 2023-07-21T18:24:18.582 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptctl.ko 2023-07-21T18:24:18.582 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptfc.ko 2023-07-21T18:24:18.582 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptlan.ko 2023-07-21T18:24:18.582 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptsas.ko 2023-07-21T18:24:18.582 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptscsih.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptspi.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_ich.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_sch.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/mfd-core.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/sm501.ko 2023-07-21T18:24:18.583 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/viperboard.ko 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/vx855.ko 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/wl1273-core.ko 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds9802als.ko 2023-07-21T18:24:18.584 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds990x.ko 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/bh1770glc.ko 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710/cb710.ko 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/at24.ko 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-21T18:24:18.585 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/max6875.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/enclosure.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/hpilo.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29003.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29020.ko 2023-07-21T18:24:18.586 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d 2023-07-21T18:24:18.587 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-21T18:24:18.587 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-21T18:24:18.587 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei 2023-07-21T18:24:18.587 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-me.ko 2023-07-21T18:24:18.588 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-txe.ko 2023-07-21T18:24:18.588 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei.ko 2023-07-21T18:24:18.588 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru 2023-07-21T18:24:18.588 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-21T18:24:18.589 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp 2023-07-21T18:24:18.589 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-21T18:24:18.589 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-21T18:24:18.589 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-21T18:24:18.589 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_7xx1.ko 2023-07-21T18:24:18.590 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_core.ko 2023-07-21T18:24:18.590 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tsl2550.ko 2023-07-21T18:24:18.590 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_balloon.ko 2023-07-21T18:24:18.590 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci 2023-07-21T18:24:18.591 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-21T18:24:18.591 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net 2023-07-21T18:24:18.591 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding 2023-07-21T18:24:18.591 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding/bonding.ko 2023-07-21T18:24:18.592 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa 2023-07-21T18:24:18.592 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53 2023-07-21T18:24:18.592 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-21T18:24:18.592 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-21T18:24:18.593 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-21T18:24:18.593 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx 2023-07-21T18:24:18.593 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-21T18:24:18.593 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dummy.ko 2023-07-21T18:24:18.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/eql.ko 2023-07-21T18:24:18.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet 2023-07-21T18:24:18.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com 2023-07-21T18:24:18.594 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-21T18:24:18.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-21T18:24:18.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-21T18:24:18.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-21T18:24:18.595 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390 2023-07-21T18:24:18.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-21T18:24:18.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-21T18:24:18.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-21T18:24:18.596 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon 2023-07-21T18:24:18.597 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros 2023-07-21T18:24:18.598 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx 2023-07-21T18:24:18.599 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-21T18:24:18.600 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-21T18:24:18.601 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-21T18:24:18.602 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec 2023-07-21T18:24:18.603 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-21T18:24:18.604 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dnet.ko 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-21T18:24:18.605 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ethoc.ko 2023-07-21T18:24:18.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/fealnx.ko 2023-07-21T18:24:18.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel 2023-07-21T18:24:18.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-21T18:24:18.606 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-21T18:24:18.607 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf 2023-07-21T18:24:18.608 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-21T18:24:18.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-21T18:24:18.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-21T18:24:18.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-21T18:24:18.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-21T18:24:18.609 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/jme.ko 2023-07-21T18:24:18.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell 2023-07-21T18:24:18.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-21T18:24:18.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-21T18:24:18.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-21T18:24:18.610 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-21T18:24:18.611 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-21T18:24:18.612 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-21T18:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion 2023-07-21T18:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-21T18:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia 2023-07-21T18:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-21T18:24:18.613 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-21T18:24:18.614 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-21T18:24:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-21T18:24:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-21T18:24:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker 2023-07-21T18:24:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-21T18:24:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc 2023-07-21T18:24:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-21T18:24:18.617 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro 2023-07-21T18:24:18.618 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-21T18:24:18.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-21T18:24:18.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-21T18:24:18.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun 2023-07-21T18:24:18.619 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-21T18:24:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-21T18:24:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-21T18:24:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-21T18:24:18.620 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti 2023-07-21T18:24:18.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-21T18:24:18.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti 2023-07-21T18:24:18.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-21T18:24:18.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via 2023-07-21T18:24:18.621 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-21T18:24:18.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-21T18:24:18.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet 2023-07-21T18:24:18.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-21T18:24:18.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-21T18:24:18.622 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom 2023-07-21T18:24:18.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-21T18:24:18.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/geneve.ko 2023-07-21T18:24:18.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv 2023-07-21T18:24:18.623 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-21T18:24:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154 2023-07-21T18:24:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-21T18:24:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ifb.ko 2023-07-21T18:24:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan 2023-07-21T18:24:18.624 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-21T18:24:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvlan.ko 2023-07-21T18:24:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvtap.ko 2023-07-21T18:24:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio 2023-07-21T18:24:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio.ko 2023-07-21T18:24:18.625 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-21T18:24:18.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-21T18:24:18.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-21T18:24:18.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mii.ko 2023-07-21T18:24:18.626 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/net_failover.ko 2023-07-21T18:24:18.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/netconsole.ko 2023-07-21T18:24:18.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/nlmon.ko 2023-07-21T18:24:18.627 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs 2023-07-21T18:24:18.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-21T18:24:18.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-21T18:24:18.628 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy 2023-07-21T18:24:18.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/amd.ko 2023-07-21T18:24:18.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-21T18:24:18.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-21T18:24:18.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-21T18:24:18.629 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm87xx.ko 2023-07-21T18:24:18.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/broadcom.ko 2023-07-21T18:24:18.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/cicada.ko 2023-07-21T18:24:18.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/davicom.ko 2023-07-21T18:24:18.630 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/dp83640.ko 2023-07-21T18:24:18.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/et1011c.ko 2023-07-21T18:24:18.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/icplus.ko 2023-07-21T18:24:18.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/lxt.ko 2023-07-21T18:24:18.631 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/marvell.ko 2023-07-21T18:24:18.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/micrel.ko 2023-07-21T18:24:18.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/national.ko 2023-07-21T18:24:18.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/phylink.ko 2023-07-21T18:24:18.632 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/qsemi.ko 2023-07-21T18:24:18.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/realtek.ko 2023-07-21T18:24:18.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/smsc.ko 2023-07-21T18:24:18.633 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/ste10Xp.ko 2023-07-21T18:24:18.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/vitesse.ko 2023-07-21T18:24:18.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp 2023-07-21T18:24:18.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-21T18:24:18.634 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_async.ko 2023-07-21T18:24:18.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-21T18:24:18.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-21T18:24:18.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-21T18:24:18.635 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-21T18:24:18.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppoe.ko 2023-07-21T18:24:18.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppox.ko 2023-07-21T18:24:18.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pptp.ko 2023-07-21T18:24:18.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip 2023-07-21T18:24:18.636 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slhc.ko 2023-07-21T18:24:18.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slip.ko 2023-07-21T18:24:18.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/sungem_phy.ko 2023-07-21T18:24:18.637 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tap.ko 2023-07-21T18:24:18.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team 2023-07-21T18:24:18.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team.ko 2023-07-21T18:24:18.638 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-21T18:24:18.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-21T18:24:18.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-21T18:24:18.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_random.ko 2023-07-21T18:24:18.639 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-21T18:24:18.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tun.ko 2023-07-21T18:24:18.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/veth.ko 2023-07-21T18:24:18.640 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/virtio_net.ko 2023-07-21T18:24:18.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3 2023-07-21T18:24:18.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-21T18:24:18.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan 2023-07-21T18:24:18.641 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan/vxlan.ko 2023-07-21T18:24:18.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback 2023-07-21T18:24:18.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-21T18:24:18.642 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netfront.ko 2023-07-21T18:24:18.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb 2023-07-21T18:24:18.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb/ntb.ko 2023-07-21T18:24:18.643 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme 2023-07-21T18:24:18.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host 2023-07-21T18:24:18.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme-core.ko 2023-07-21T18:24:18.644 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme.ko 2023-07-21T18:24:18.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport 2023-07-21T18:24:18.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport.ko 2023-07-21T18:24:18.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_cs.ko 2023-07-21T18:24:18.645 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_pc.ko 2023-07-21T18:24:18.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_serial.ko 2023-07-21T18:24:18.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci 2023-07-21T18:24:18.646 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug 2023-07-21T18:24:18.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-21T18:24:18.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie 2023-07-21T18:24:18.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-21T18:24:18.647 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/xen-pcifront.ko 2023-07-21T18:24:18.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia 2023-07-21T18:24:18.648 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/i82092.ko 2023-07-21T18:24:18.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/pd6729.ko 2023-07-21T18:24:18.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-21T18:24:18.649 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-21T18:24:18.650 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acerhdf.ko 2023-07-21T18:24:18.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-21T18:24:18.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-21T18:24:18.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-21T18:24:18.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-21T18:24:18.651 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/hdaps.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-21T18:24:18.652 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel_ips.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-21T18:24:18.653 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps 2023-07-21T18:24:18.654 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps_parport.ko 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/pps_core.ko 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp.ko 2023-07-21T18:24:18.655 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp_kvm.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-21T18:24:18.656 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-em3027.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-21T18:24:18.657 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-max6900.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-21T18:24:18.658 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-21T18:24:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-x1205.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-9xxx.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-sas.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-xxxx.ko 2023-07-21T18:24:18.660 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/BusLogic.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/a100u2w.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/advansys.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-21T18:24:18.661 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/am53c974.ko 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/atp870u.ko 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa/bfa.ko 2023-07-21T18:24:18.662 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ch.ko 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-21T18:24:18.663 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dc395x.ko 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler 2023-07-21T18:24:18.664 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dmx3191d.ko 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r 2023-07-21T18:24:18.665 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-21T18:24:18.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esp_scsi.ko 2023-07-21T18:24:18.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe 2023-07-21T18:24:18.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-21T18:24:18.666 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic/fnic.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hpsa.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hptiop.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hv_storvsc.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/initio.ko 2023-07-21T18:24:18.667 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ipr.ko 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ips.ko 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci/isci.ko 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc 2023-07-21T18:24:18.668 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc/libfc.ko 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi.ko 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas/libsas.ko 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc 2023-07-21T18:24:18.669 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid.ko 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas 2023-07-21T18:24:18.670 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvumi.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pmcraid.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla1280.ko 2023-07-21T18:24:18.671 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/raid_class.ko 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_debug.ko 2023-07-21T18:24:18.672 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ses.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/st.ko 2023-07-21T18:24:18.673 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/stex.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/virtio_scsi.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/wd719x.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb/ssb.ko 2023-07-21T18:24:18.674 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_file.ko 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_iblock.ko 2023-07-21T18:24:18.675 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_mod.ko 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_pscsi.ko 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_user.ko 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal 2023-07-21T18:24:18.676 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-21T18:24:18.677 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-21T18:24:18.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-21T18:24:18.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-21T18:24:18.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-21T18:24:18.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-21T18:24:18.678 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-21T18:24:18.679 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_gsm.ko 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_hdlc.ko 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/nozomi.ko 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/arc_uart.ko 2023-07-21T18:24:18.680 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/synclink_gt.ko 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-21T18:24:18.681 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio.ko 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_aec.ko 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_cif.ko 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_pci_generic.ko 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_sercos3.ko 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class 2023-07-21T18:24:18.682 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-acm.ko 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usblp.ko 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usbtmc.ko 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image 2023-07-21T18:24:18.683 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/mdc800.ko 2023-07-21T18:24:18.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/microtek.ko 2023-07-21T18:24:18.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc 2023-07-21T18:24:18.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/adutux.ko 2023-07-21T18:24:18.684 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/appledisplay.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi26.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi62.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ezusb.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/idmouse.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/iowarrior.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-21T18:24:18.685 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ldusb.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/legousbtower.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usb3503.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usblcd.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-21T18:24:18.686 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/uss720.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/yurex.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy/phy-generic.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/aircable.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ark3116.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-21T18:24:18.687 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ch341.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cp210x.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cyberjack.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/empeg.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-21T18:24:18.688 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_ti.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipaq.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipw.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ir-usb.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan.ko 2023-07-21T18:24:18.689 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mct_u232.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7720.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7840.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/navman.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/omninet.ko 2023-07-21T18:24:18.690 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/opticon.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/option.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/oti6858.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/pl2303.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcaux.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcserial.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/quatech2.ko 2023-07-21T18:24:18.691 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/safe_serial.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/sierra.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ssu100.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/symbolserial.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_debug.ko 2023-07-21T18:24:18.692 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/visor.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/whiteheat.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/uas.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-21T18:24:18.693 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-karma.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-21T18:24:18.694 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/usb-storage.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-21T18:24:18.695 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio.ko 2023-07-21T18:24:18.696 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost.ko 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_net.ko 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_scsi.ko 2023-07-21T18:24:18.697 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/apple_bl.ko 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/lcd.ko 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio.ko 2023-07-21T18:24:18.698 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_balloon.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_mmio.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_ring.ko 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters 2023-07-21T18:24:18.699 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2482.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2490.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-21T18:24:18.700 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/wire.ko 2023-07-21T18:24:18.701 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/hpwdt.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/i6300esb.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-21T18:24:18.702 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ib700wdt.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ibmasr.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it87_wdt.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/machzwd.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/nv_tco.ko 2023-07-21T18:24:18.703 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/softdog.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/via_wdt.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-21T18:24:18.704 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/wdt_pci.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/xen_wdt.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-evtchn.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntalloc.ko 2023-07-21T18:24:18.705 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntdev.ko 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-privcmd.ko 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs 2023-07-21T18:24:18.706 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p/9p.ko 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs/affs.ko 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs/befs.ko 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/binfmt_misc.ko 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs 2023-07-21T18:24:18.707 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs/btrfs.ko 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles/cachefiles.ko 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph/ceph.ko 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda/coda.ko 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs 2023-07-21T18:24:18.708 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs/cramfs.ko 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm/dlm.ko 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs/f2fs.ko 2023-07-21T18:24:18.709 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/fat.ko 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/msdos.ko 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/vfat.ko 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache/fscache.ko 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse 2023-07-21T18:24:18.710 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/cuse.ko 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/fuse.ko 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2/gfs2.ko 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs/hfs.ko 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus/hfsplus.ko 2023-07-21T18:24:18.711 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs/isofs.ko 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs/jfs.ko 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd/lockd.ko 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix 2023-07-21T18:24:18.712 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix/minix.ko 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs/netfs.ko 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout 2023-07-21T18:24:18.713 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfs.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv3.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv4.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/grace.ko 2023-07-21T18:24:18.714 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/nfs_acl.ko 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd/nfsd.ko 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2/nilfs2.ko 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-celtic.ko 2023-07-21T18:24:18.715 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-centeuro.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-croatian.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-cyrillic.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-gaelic.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-greek.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-iceland.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-inuit.ko 2023-07-21T18:24:18.716 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-roman.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-romanian.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-turkish.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1250.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1251.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1255.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp737.ko 2023-07-21T18:24:18.717 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp775.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp850.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp852.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp855.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp857.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp860.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp861.ko 2023-07-21T18:24:18.718 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp862.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp863.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp864.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp865.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp866.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp869.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp874.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp932.ko 2023-07-21T18:24:18.719 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp936.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp949.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp950.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_euc-jp.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-1.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-13.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-14.ko 2023-07-21T18:24:18.720 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-15.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-2.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-3.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-4.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-5.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-6.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-7.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-9.ko 2023-07-21T18:24:18.721 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-r.ko 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-ru.ko 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-u.ko 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_utf8.ko 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm 2023-07-21T18:24:18.722 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-21T18:24:18.723 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs/overlay.ko 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore/ramoops.ko 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs/reiserfs.ko 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs/romfs.ko 2023-07-21T18:24:18.724 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client/cifs.ko 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_arc4.ko 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_md4.ko 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs 2023-07-21T18:24:18.725 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs/squashfs.ko 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv/sysv.ko 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf/udf.ko 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs/ufs.ko 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs 2023-07-21T18:24:18.726 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs/xfs.ko 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu/rcutorture.ko 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/torture.ko 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib 2023-07-21T18:24:18.727 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/asn1_encoder.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc-itu-t.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc8.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libarc4.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libdes.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/libcrc32c.ko 2023-07-21T18:24:18.728 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lru_cache.ko 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4_compress.ko 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4hc_compress.ko 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math/cordic.ko 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6 2023-07-21T18:24:18.729 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6/raid6_pq.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_bm.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_fsm.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_kmp.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd/zstd_compress.ko 2023-07-21T18:24:18.730 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm/hwpoison-inject.ko 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/6lowpan.ko 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_dest.ko 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_fragment.ko 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_hop.ko 2023-07-21T18:24:18.731 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_mobility.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_routing.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_udp.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/garp.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/mrp.ko 2023-07-21T18:24:18.732 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/p8022.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/psnap.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/stp.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q/8021q.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_fd.ko 2023-07-21T18:24:18.733 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_rdma.ko 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_virtio.ko 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv/batman-adv.ko 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/br_netfilter.ko 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/bridge.ko 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter 2023-07-21T18:24:18.734 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-21T18:24:18.735 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-21T18:24:18.736 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtables.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-21T18:24:18.737 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph/libceph.ko 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/failover.ko 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/pktgen.ko 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp 2023-07-21T18:24:18.738 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp.ko 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_diag.ko 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv4.ko 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv6.ko 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver/dns_resolver.ko 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa 2023-07-21T18:24:18.739 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/dsa_core.ko 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_brcm.ko 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_dsa.ko 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_none.ko 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_trailer.ko 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154 2023-07-21T18:24:18.740 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ah4.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/esp4.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/fou.ko 2023-07-21T18:24:18.741 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/gre.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/inet_diag.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_gre.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_tunnel.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_vti.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipcomp.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipip.ko 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter 2023-07-21T18:24:18.742 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-21T18:24:18.743 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-21T18:24:18.744 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-21T18:24:18.745 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_bic.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_dctcp.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_diag.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_highspeed.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_htcp.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_hybla.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_illinois.ko 2023-07-21T18:24:18.746 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_lp.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_scalable.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_vegas.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_veno.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_westwood.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_yeah.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tunnel4.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_diag.ko 2023-07-21T18:24:18.747 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_tunnel.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ah6.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/esp6.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/fou6.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_tunnel.ko 2023-07-21T18:24:18.748 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_vti.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ipcomp6.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-21T18:24:18.749 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-21T18:24:18.750 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-21T18:24:18.751 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/sit.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/tunnel6.ko 2023-07-21T18:24:18.752 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key/af_key.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_core.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_eth.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip.ko 2023-07-21T18:24:18.753 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip6.ko 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_netlink.ko 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ppp.ko 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc/llc.ko 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154/mac802154.ko 2023-07-21T18:24:18.754 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls/mpls_gso.ko 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set.ko 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-21T18:24:18.755 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-21T18:24:18.756 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-21T18:24:18.757 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-21T18:24:18.758 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conncount.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-21T18:24:18.759 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-21T18:24:18.760 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_log_syslog.ko 2023-07-21T18:24:18.761 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_irc.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_sip.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_tables.ko 2023-07-21T18:24:18.762 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_log.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_chain_nat.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_compat.ko 2023-07-21T18:24:18.763 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_ct.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fib.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_hash.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_limit.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_log.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_masq.ko 2023-07-21T18:24:18.764 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_nat.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_numgen.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_queue.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_quota.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_redir.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject_inet.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_socket.ko 2023-07-21T18:24:18.765 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_tunnel.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_AUDIT.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CT.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_DSCP.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HL.ko 2023-07-21T18:24:18.766 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HMARK.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LED.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LOG.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NETMAP.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFLOG.ko 2023-07-21T18:24:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_RATEEST.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_SECMARK.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TEE.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TPROXY.ko 2023-07-21T18:24:18.768 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TRACE.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_addrtype.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_bpf.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cgroup.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cluster.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_comment.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connbytes.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlabel.ko 2023-07-21T18:24:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlimit.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connmark.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_conntrack.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cpu.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dccp.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_devgroup.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dscp.ko 2023-07-21T18:24:18.770 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ecn.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_esp.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hashlimit.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_helper.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hl.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipcomp.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_iprange.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipvs.ko 2023-07-21T18:24:18.771 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_l2tp.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_length.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_limit.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mac.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mark.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_multiport.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nat.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nfacct.ko 2023-07-21T18:24:18.772 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_osf.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_owner.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_physdev.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_pkttype.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_policy.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_quota.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_rateest.ko 2023-07-21T18:24:18.773 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_realm.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_recent.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_sctp.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_set.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_socket.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_state.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_statistic.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_string.ko 2023-07-21T18:24:18.774 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_tcpmss.ko 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_time.ko 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_u32.ko 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink/netlink_diag.ko 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh/nsh.ko 2023-07-21T18:24:18.775 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/openvswitch.ko 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-geneve.ko 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-gre.ko 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-vxlan.ko 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet/af_packet_diag.ko 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds 2023-07-21T18:24:18.776 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_rdma.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_tcp.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_csum.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_gact.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_ipt.ko 2023-07-21T18:24:18.777 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_mirred.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_nat.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_pedit.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_police.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_simple.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_skbedit.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_vlan.ko 2023-07-21T18:24:18.778 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_basic.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_bpf.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_flow.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_fw.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_route.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_u32.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_cmp.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_ipset.ko 2023-07-21T18:24:18.779 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_meta.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_nbyte.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_text.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_u32.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_choke.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_codel.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_drr.ko 2023-07-21T18:24:18.780 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_fq.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_gred.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hfsc.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hhf.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_htb.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_ingress.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio_lib.ko 2023-07-21T18:24:18.781 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_multiq.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_netem.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_pie.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_plug.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_prio.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_qfq.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_red.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfb.ko 2023-07-21T18:24:18.782 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfq.ko 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_tbf.ko 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_teql.ko 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp.ko 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp_diag.ko 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss 2023-07-21T18:24:18.783 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/sunrpc.ko 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/diag.ko 2023-07-21T18:24:18.784 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/tipc.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix/unix_diag.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-21T18:24:18.785 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys 2023-07-21T18:24:18.786 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys/trusted.ko 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib/irqbypass.ko 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias.bin 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin 2023-07-21T18:24:18.787 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.bin 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.modinfo 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep.bin 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.devname 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.order 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.softdep 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols 2023-07-21T18:24:18.788 INFO:teuthology.orchestra.run.smithi005.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols.bin 2023-07-21T18:24:18.789 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.789 DEBUG:teuthology.orchestra.run.smithi005:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-21T18:24:18.818 INFO:teuthology.orchestra.run.smithi005.stdout:postinstall scriptlet (using /bin/sh): 2023-07-21T18:24:18.818 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc -a -r /boot/System.map-6.5.0-rc1-gc685da6f56cc ]; then 2023-07-21T18:24:18.818 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:18.818 INFO:teuthology.orchestra.run.smithi005.stdout:cp /boot/System.map-6.5.0-rc1-gc685da6f56cc /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/installkernel 6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-21T18:24:18.819 INFO:teuthology.orchestra.run.smithi005.stdout:new-kernel-pkg --remove 6.5.0-rc1-gc685da6f56cc --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-gc685da6f56cc.img 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:kernel-install remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.820 INFO:teuthology.orchestra.run.smithi005.stdout:fi 2023-07-21T18:24:18.821 DEBUG:teuthology.orchestra.run.smithi005:> sudo rpm -qi grub2-tools 2023-07-21T18:24:18.900 INFO:teuthology.orchestra.run.smithi005.stdout:Name : grub2-tools 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Epoch : 1 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Version : 2.02 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Release : 123.el8 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Architecture: x86_64 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-21T18:24:18.901 INFO:teuthology.orchestra.run.smithi005.stdout:Group : System Environment/Base 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Size : 9265450 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:License : GPLv3+ 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-21T18:24:18.902 INFO:teuthology.orchestra.run.smithi005.stdout:Relocations : (not relocatable) 2023-07-21T18:24:18.903 INFO:teuthology.orchestra.run.smithi005.stdout:Packager : Red Hat, Inc. 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:Vendor : Red Hat, Inc. 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:Summary : Support tools for GRUB. 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:Description : 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-21T18:24:18.904 INFO:teuthology.orchestra.run.smithi005.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-21T18:24:18.905 INFO:teuthology.orchestra.run.smithi005.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-21T18:24:18.905 INFO:teuthology.orchestra.run.smithi005.stdout:hardware devices. 2023-07-21T18:24:18.905 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:24:18.905 INFO:teuthology.orchestra.run.smithi005.stdout:This subpackage provides tools for support of all platforms. 2023-07-21T18:24:18.906 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-21T18:24:18.906 INFO:teuthology.task.kernel:Updating grub on smithi005 to boot 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:18.907 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-21T18:24:19.056 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-07-21T18:24:21.876 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-6.5.0_rc1_gc685da6f56cc-1 ######################################## 2023-07-21T18:24:22.382 INFO:teuthology.orchestra.run.smithi005.stderr:Generating grub configuration file ... 2023-07-21T18:24:24.394 INFO:teuthology.orchestra.run.smithi005.stderr:done 2023-07-21T18:24:24.396 DEBUG:teuthology.orchestra.run.smithi005:> mktemp 2023-07-21T18:24:24.418 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/tmp.6qjZ5HaBcj 2023-07-21T18:24:24.419 DEBUG:teuthology.orchestra.run.smithi005:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6qjZ5HaBcj 2023-07-21T18:24:24.492 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0666 /tmp/tmp.6qjZ5HaBcj 2023-07-21T18:24:24.637 DEBUG:teuthology.orchestra.remote:smithi005:/tmp/tmp.6qjZ5HaBcj is 6KB 2023-07-21T18:24:24.658 DEBUG:teuthology.orchestra.run.smithi005:> rm -fr /tmp/tmp.6qjZ5HaBcj 2023-07-21T18:24:24.681 DEBUG:teuthology.orchestra.run.smithi005:> sudo /bin/ls /boot/loader/entries || true 2023-07-21T18:24:24.748 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-21T18:24:24.749 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-21T18:24:24.749 INFO:teuthology.orchestra.run.smithi005.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc.conf 2023-07-21T18:24:24.750 DEBUG:teuthology.orchestra.run.smithi005:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:24.934 DEBUG:teuthology.orchestra.run.smithi005:> sudo shutdown -r now 2023-07-21T18:24:28.058 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:24:28.059 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:24:36.144 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-07-21T18:24:46.072 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi101.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2023-07-21T18:24:46.073 DEBUG:teuthology.orchestra.run.smithi101:> rpm -qlp /tmp/kernel.x86_64.rpm 2023-07-21T18:24:46.120 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.120 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/config-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/boot/vmlinuz-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto 2023-07-21T18:24:46.121 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-07-21T18:24:46.122 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crc32c-intel.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-avx2.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-07-21T18:24:46.123 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce 2023-07-21T18:24:46.124 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-amd.ko 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm-intel.ko 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/arch/x86/kvm/kvm.ko 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ansi_cprng.ko 2023-07-21T18:24:46.125 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/anubis.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/arc4.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_memcpy.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_pq.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_raid6_recov.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_tx.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/async_xor.ko 2023-07-21T18:24:46.126 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/async_tx/raid6test.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authenc.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/authencesn.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blake2b_generic.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_common.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/blowfish_generic.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/camellia_generic.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast5_generic.ko 2023-07-21T18:24:46.127 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast6_generic.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cast_common.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/ccm.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cmac.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crc32_generic.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_engine.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/crypto_user.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/cts.ko 2023-07-21T18:24:46.128 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/des_generic.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/dh_generic.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/echainiv.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/essiv.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/fcrypt.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/khazad.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4.ko 2023-07-21T18:24:46.129 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/lz4hc.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/md4.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/michael_mic.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcbc.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/pcrypt.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/rmd160.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/seed.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/serpent_generic.ko 2023-07-21T18:24:46.130 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tcrypt.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/tea.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_common.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/twofish_generic.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/vmac.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/wp512.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xcbc.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xor.ko 2023-07-21T18:24:46.131 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/crypto/xxhash_generic.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_ipmi.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/acpi_pad.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/custom_method.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/ec_sys.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/platform_profile.ko 2023-07-21T18:24:46.132 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbs.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/sbshc.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/acpi/video.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/acard-ahci.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ahci_platform.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/ata_generic.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/libahci_platform.ko 2023-07-21T18:24:46.133 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_acpi.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ali.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_amd.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_artop.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atiixp.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_atp867x.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd640.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cmd64x.ko 2023-07-21T18:24:46.134 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_cypress.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_efar.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt366.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt37x.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x2n.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_hpt3x3.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it8213.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_it821x.ko 2023-07-21T18:24:46.135 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_jmicron.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_marvell.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_mpiix.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_netcell.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ninja32.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87410.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_ns87415.ko 2023-07-21T18:24:46.136 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_oldpiix.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_opti.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_optidma.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pcmcia.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc2027x.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_piccolo.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_rdc.ko 2023-07-21T18:24:46.137 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sch.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_serverworks.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sil680.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sis.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_sl82c105.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_triflex.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pata_via.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/pdc_adma.ko 2023-07-21T18:24:46.138 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_inic162x.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_mv.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_nv.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_promise.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_qstor.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sil24.ko 2023-07-21T18:24:46.139 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sis.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_svw.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_sx4.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_uli.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_via.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ata/sata_vsc.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/auxdisplay/ks0108.ko 2023-07-21T18:24:46.140 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-i2c.ko 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/base/regmap/regmap-mmio.ko 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/bcma/bcma.ko 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe 2023-07-21T18:24:46.141 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/aoe/aoe.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/brd.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/drbd/drbd.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/floppy.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/loop.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-07-21T18:24:46.142 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/nbd.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/null_blk/null_blk.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/pktcdvd.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/rbd.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/virtio_blk.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-07-21T18:24:46.143 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/xen-blkfront.ko 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/block/zram/zram.ko 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hangcheck-timer.ko 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/amd-rng.ko 2023-07-21T18:24:46.144 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/intel-rng.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/via-rng.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/hw_random/virtio-rng.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-07-21T18:24:46.145 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_si.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/lp.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/mwave/mwave.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/ppdev.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tlclk.ko 2023-07-21T18:24:46.146 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_atmel.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_infineon.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_nsc.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/tpm/tpm_tis_core.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/uv_mmtimer.ko 2023-07-21T18:24:46.147 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/char/virtio_console.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/acpi-cpufreq.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/p4-clockmod.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/pcc-cpufreq.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/powernow-k8.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/cpufreq/speedstep-lib.ko 2023-07-21T18:24:46.148 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/ccp/ccp.ko 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common 2023-07-21T18:24:46.149 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-aes.ko 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/padlock-sha.ko 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca 2023-07-21T18:24:46.150 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dca/dca.ko 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/devfreq/governor_simpleondemand.ko 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/dw/dw_dmac.ko 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/dma/ioat/ioatdma.ko 2023-07-21T18:24:46.151 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/amd64_edac.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/e752x_edac.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/edac_mce_amd.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3000_edac.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i3200_edac.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5100_edac.ko 2023-07-21T18:24:46.152 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i5400_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7300_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i7core_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/i82975x_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/ie31200_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/sb_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/edac/x38_edac.ko 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware 2023-07-21T18:24:46.153 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/edd.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/firmware/iscsi_ibft.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-ich.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/gpio/gpio-viperboard.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-appleir.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-aureal.ko 2023-07-21T18:24:46.154 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-axff.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-dr.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elecom.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-elo.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-emsff.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gaff.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gt683r.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-gyration.ko 2023-07-21T18:24:46.155 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-kbd.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtek-mouse.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-holtekff.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-hyperv.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-icade.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-keytouch.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-kye.ko 2023-07-21T18:24:46.156 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lcpower.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-led.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-lenovo.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-dj.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-multitouch.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-ortek.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-penmount.ko 2023-07-21T18:24:46.157 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-petalynx.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-picolcd.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-pl.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-plantronics.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-primax.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-rmi.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-arvo.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-common.ko 2023-07-21T18:24:46.158 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-isku.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kone.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-konepure.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-lua.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-pyra.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-ryos.ko 2023-07-21T18:24:46.159 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat-savu.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-roccat.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-saitek.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-samsung.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sensor-hub.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sjoy.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sony.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-speedlink.ko 2023-07-21T18:24:46.160 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-steelseries.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-sunplus.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-thrustmaster.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tivo.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-tmff.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-topseed.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-twinhan.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-uclogic.ko 2023-07-21T18:24:46.161 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-waltop.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-wiimote.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-xinmo.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zpff.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/hid-zydacron.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/uhid.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hid/wacom.ko 2023-07-21T18:24:46.162 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_balloon.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_utils.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hv/hv_vmbus.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/abituguru3.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/acpi_power_meter.ko 2023-07-21T18:24:46.163 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7414.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ad7418.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adc128d818.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1025.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1026.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1029.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm1031.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adm9240.ko 2023-07-21T18:24:46.164 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ads7828.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7410.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7411.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7462.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7470.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7475.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/adt7x10.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/amc6821.ko 2023-07-21T18:24:46.165 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/applesmc.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asb100.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asc7621.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/asus_atk0110.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/atxp1.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/coretemp.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-07-21T18:24:46.166 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/dme1737.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds1621.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ds620.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc1403.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/emc6w201.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71805f.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f71882fg.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/f75375s.ko 2023-07-21T18:24:46.167 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fam15h_power.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/fschmd.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g760a.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/g762.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl518sm.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/gl520sm.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/hwmon-vid.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5500_temp.ko 2023-07-21T18:24:46.168 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/i5k_amb.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmaem.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ibmpex.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina209.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ina2xx.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/it87.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k10temp.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/k8temp.ko 2023-07-21T18:24:46.169 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lineage-pem.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm63.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm73.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm75.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm77.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm78.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm80.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm83.ko 2023-07-21T18:24:46.170 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm85.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm87.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm90.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm92.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm93.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95234.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95241.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/lm95245.ko 2023-07-21T18:24:46.171 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc2945.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4151.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4215.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4222.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4245.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4260.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/ltc4261.ko 2023-07-21T18:24:46.172 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max16065.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1619.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max1668.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max197.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6639.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6650.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/max6697.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/mcp3021.ko 2023-07-21T18:24:46.173 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6683.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775-core.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct6775.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/nct7802.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87360.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pc87427.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pcf8591.ko 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus 2023-07-21T18:24:46.174 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max16064.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max34440.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/max8688.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-07-21T18:24:46.175 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/powr1220.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5627.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch5636.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sch56xx-common.ko 2023-07-21T18:24:46.176 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht15.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sht21.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/shtc1.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/sis5595.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47b397.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m1.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/smsc47m192.ko 2023-07-21T18:24:46.177 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/thmc50.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp102.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp103.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp401.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/tmp421.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via-cputemp.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/via686a.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt1211.ko 2023-07-21T18:24:46.178 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/vt8231.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627ehf.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83627hf.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83781d.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83791d.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83792d.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83793.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83795.ko 2023-07-21T18:24:46.179 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l785ts.ko 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/hwmon/w83l786ng.ko 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-07-21T18:24:46.180 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-07-21T18:24:46.181 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-i801.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-isch.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-parport.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-07-21T18:24:46.182 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-via.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-dev.ko 2023-07-21T18:24:46.183 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-mux.ko 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-slave-eeprom.ko 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-smbus.ko 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/i2c/i2c-stub.ko 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_cm.ko 2023-07-21T18:24:46.184 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_core.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_umad.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/iw_cm.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_cm.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4 2023-07-21T18:24:46.185 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp 2023-07-21T18:24:46.186 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-07-21T18:24:46.187 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/fm801-gp.ko 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/gameport.ko 2023-07-21T18:24:46.188 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/lightning.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/gameport/ns558.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/joydev.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/keyboard/gpio_keys.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/appletouch.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/bcm5974.ko 2023-07-21T18:24:46.189 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/cyapatp.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/elan_i2c.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/sermouse.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/synaptics_usb.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/mouse/vsxxxaa.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/rmi4/rmi_core.ko 2023-07-21T18:24:46.190 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/altera_ps2.ko 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/arc_ps2.ko 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/serio/serio_raw.ko 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/input/sparse-keymap.ko 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd 2023-07-21T18:24:46.191 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/iommu/amd/iommu_v2.ko 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-blinkm.ko 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lm3530.ko 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-lp3944.ko 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/leds-ss4200.ko 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger 2023-07-21T18:24:46.192 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-07-21T18:24:46.193 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/bcache/bcache.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-bio-prison.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache-smq.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-cache.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-crypt.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-delay.ko 2023-07-21T18:24:46.194 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-flakey.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-log-userspace.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-multipath.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-queue-length.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-raid.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-round-robin.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-service-time.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-switch.ko 2023-07-21T18:24:46.195 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-thin-pool.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/dm-verity.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/faulty.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/linear.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/multipath.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-07-21T18:24:46.196 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid0.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid1.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid10.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/md/raid456.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptbase.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptctl.ko 2023-07-21T18:24:46.197 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptfc.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptlan.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptsas.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptscsih.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/message/fusion/mptspi.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_ich.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/lpc_sch.ko 2023-07-21T18:24:46.198 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/mfd-core.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/sm501.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/viperboard.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/vx855.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/mfd/wl1273-core.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-07-21T18:24:46.199 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds9802als.ko 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/apds990x.ko 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/bh1770glc.ko 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/cb710/cb710.ko 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/at24.ko 2023-07-21T18:24:46.200 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/eeprom/max6875.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/enclosure.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/hpilo.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29003.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/isl29020.ko 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d 2023-07-21T18:24:46.201 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-me.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei-txe.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/mei/mei.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-gru/gru.ko 2023-07-21T18:24:46.202 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xp.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpc.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/sgi-xp/xpnet.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_7xx1.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tifm_core.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/tsl2550.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_balloon.ko 2023-07-21T18:24:46.203 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/bonding/bonding.ko 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53 2023-07-21T18:24:46.204 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/b53/b53_common.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/bcm-sf2.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6060.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/dummy.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/eql.ko 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet 2023-07-21T18:24:46.205 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/8390.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-07-21T18:24:46.206 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/agere/et131x.ko 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-07-21T18:24:46.207 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx 2023-07-21T18:24:46.208 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-07-21T18:24:46.209 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet 2023-07-21T18:24:46.210 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-07-21T18:24:46.211 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb 2023-07-21T18:24:46.212 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-07-21T18:24:46.213 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/dnet.ko 2023-07-21T18:24:46.214 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ethoc.ko 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/fealnx.ko 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e100.ko 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000 2023-07-21T18:24:46.215 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf 2023-07-21T18:24:46.216 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-07-21T18:24:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/jme.ko 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/skge.ko 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox 2023-07-21T18:24:46.218 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-07-21T18:24:46.219 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion 2023-07-21T18:24:46.220 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen 2023-07-21T18:24:46.221 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-07-21T18:24:46.222 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/atp.ko 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/rocker/rocker.ko 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-07-21T18:24:46.223 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis190.ko 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sis/sis900.ko 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-07-21T18:24:46.224 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/cassini.ko 2023-07-21T18:24:46.225 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/niu.ko 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sungem.ko 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/ti/tlan.ko 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via 2023-07-21T18:24:46.226 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/geneve.ko 2023-07-21T18:24:46.227 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ieee802154/fakelb.ko 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ifb.ko 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ipvlan/ipvlan.ko 2023-07-21T18:24:46.228 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvlan.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/macvtap.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mdio/mdio-regmap.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/mii.ko 2023-07-21T18:24:46.229 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/net_failover.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/netconsole.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/nlmon.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs-lynx.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/amd.ko 2023-07-21T18:24:46.230 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm-phy-ptp.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm7xxx.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/bcm87xx.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/broadcom.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/cicada.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/davicom.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/dp83640.ko 2023-07-21T18:24:46.231 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/et1011c.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/icplus.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/lxt.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/marvell.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/micrel.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/national.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/phylink.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/qsemi.ko 2023-07-21T18:24:46.232 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/realtek.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/smsc.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/ste10Xp.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/phy/vitesse.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/bsd_comp.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_async.ko 2023-07-21T18:24:46.233 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_deflate.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_generic.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_mppe.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/ppp_synctty.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppoe.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pppox.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/ppp/pptp.ko 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip 2023-07-21T18:24:46.234 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slhc.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/slip/slip.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/sungem_phy.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tap.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_activebackup.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_broadcast.ko 2023-07-21T18:24:46.235 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_random.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/tun.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/veth.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/virtio_net.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-07-21T18:24:46.236 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/vxlan/vxlan.ko 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netback/xen-netback.ko 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/net/xen-netfront.ko 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ntb/ntb.ko 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme 2023-07-21T18:24:46.237 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme-core.ko 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/nvme/host/nvme.ko 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport.ko 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_cs.ko 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_pc.ko 2023-07-21T18:24:46.238 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/parport/parport_serial.ko 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/pcie/aer_inject.ko 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pci/xen-pcifront.ko 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia 2023-07-21T18:24:46.239 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/i82092.ko 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/pd6729.ko 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pcmcia/yenta_socket.ko 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86 2023-07-21T18:24:46.240 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acer-wmi.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/acerhdf.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/apple-gmux.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-laptop.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/asus-wmi.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/classmate-laptop.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-07-21T18:24:46.241 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/hdaps.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/intel_ips.ko 2023-07-21T18:24:46.242 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/msi-wmi.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/mxm-wmi.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-laptop.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/samsung-q10.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/topstar-laptop.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/toshiba_haps.ko 2023-07-21T18:24:46.243 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi-bmof.ko 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/platform/x86/wmi.ko 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_common.ko 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/powercap/intel_rapl_msr.ko 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-gpio.ko 2023-07-21T18:24:46.244 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps-ldisc.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/clients/pps_parport.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/pps/pps_core.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ptp/ptp_kvm.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq32k.ko 2023-07-21T18:24:46.245 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-bq4802.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1286.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1307.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1374.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1511.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1553.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1672.ko 2023-07-21T18:24:46.246 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds1742.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds2404.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-ds3232.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-em3027.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-fm3130.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl12022.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-isl1208.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m41t80.ko 2023-07-21T18:24:46.247 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t35.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-m48t59.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-max6900.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-msm6242.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf2127.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf85063.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8523.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8563.ko 2023-07-21T18:24:46.248 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-pcf8583.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rp5c01.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rs5c372.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8025.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-rx8581.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/rtc/rtc-x1205.ko 2023-07-21T18:24:46.249 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-9xxx.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-sas.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/3w-xxxx.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/BusLogic.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/a100u2w.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aacraid/aacraid.ko 2023-07-21T18:24:46.250 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/advansys.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/am53c974.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/atp870u.ko 2023-07-21T18:24:46.251 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bfa/bfa.ko 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i 2023-07-21T18:24:46.252 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ch.ko 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/csiostor/csiostor.ko 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i 2023-07-21T18:24:46.253 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dc395x.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2023-07-21T18:24:46.254 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/dmx3191d.ko 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esas2r/esas2r.ko 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/esp_scsi.ko 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/fcoe.ko 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic 2023-07-21T18:24:46.255 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/fnic/fnic.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hpsa.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hptiop.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/hv_storvsc.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/initio.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ipr.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ips.ko 2023-07-21T18:24:46.256 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/isci/isci.ko 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/iscsi_tcp.ko 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libfc/libfc.ko 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi.ko 2023-07-21T18:24:46.257 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libiscsi_tcp.ko 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/libsas/libsas.ko 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/lpfc/lpfc.ko 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid.ko 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-07-21T18:24:46.258 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvsas/mvsas.ko 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/mvumi.ko 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001 2023-07-21T18:24:46.259 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/pmcraid.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla1280.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-07-21T18:24:46.260 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/raid_class.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_debug.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_fc.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_sas.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_spi.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/scsi_transport_srp.ko 2023-07-21T18:24:46.261 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/ses.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/st.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/stex.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/virtio_scsi.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/vmw_pvscsi.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/scsi/wd719x.ko 2023-07-21T18:24:46.262 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ssb/ssb.ko 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/loopback/tcm_loop.ko 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_file.ko 2023-07-21T18:24:46.263 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_iblock.ko 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_mod.ko 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_pscsi.ko 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/target_core_user.ko 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal 2023-07-21T18:24:46.264 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-07-21T18:24:46.265 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-07-21T18:24:46.266 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_gsm.ko 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/n_hdlc.ko 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/nozomi.ko 2023-07-21T18:24:46.267 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/arc_uart.ko 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/serial/jsm/jsm.ko 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/tty/synclink_gt.ko 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs 2023-07-21T18:24:46.268 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/core/ufshcd-core.ko 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio.ko 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_aec.ko 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_cif.ko 2023-07-21T18:24:46.269 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_pci_generic.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/uio/uio_sercos3.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-acm.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/cdc-wdm.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usblp.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/class/usbtmc.ko 2023-07-21T18:24:46.270 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/host/sl811-hcd.ko 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/mdc800.ko 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/image/microtek.ko 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/adutux.ko 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/appledisplay.ko 2023-07-21T18:24:46.271 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi26.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/emi62.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ezusb.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/idmouse.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/iowarrior.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/isight_firmware.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/ldusb.ko 2023-07-21T18:24:46.272 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/legousbtower.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/trancevibrator.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usb3503.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usblcd.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/usbsevseg.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/uss720.ko 2023-07-21T18:24:46.273 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/misc/yurex.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/phy/phy-generic.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/aircable.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ark3116.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/belkin_sa.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ch341.ko 2023-07-21T18:24:46.274 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cp210x.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cyberjack.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/cypress_m8.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/digi_acceleport.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/empeg.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ftdi_sio.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/garmin_gps.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_edgeport.ko 2023-07-21T18:24:46.275 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/io_ti.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipaq.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ipw.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ir-usb.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/keyspan_pda.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kl5kusb105.ko 2023-07-21T18:24:46.276 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/kobil_sct.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mct_u232.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7720.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/mos7840.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/navman.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/omninet.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/opticon.ko 2023-07-21T18:24:46.277 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/option.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/oti6858.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/pl2303.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcaux.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/qcserial.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/quatech2.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/safe_serial.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/sierra.ko 2023-07-21T18:24:46.278 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/spcp8x5.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ssu100.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/symbolserial.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_debug.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/usb_wwan.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/visor.ko 2023-07-21T18:24:46.279 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/whiteheat.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/serial/xsens_mt.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/uas.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-alauda.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-cypress.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-datafab.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-07-21T18:24:46.280 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-freecom.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-isd200.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-karma.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-onetouch.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-realtek.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr09.ko 2023-07-21T18:24:46.281 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-sddr55.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/ums-usbat.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/storage/usb-storage.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-core.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/usbip-host.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio 2023-07-21T18:24:46.282 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/pci/vfio-pci.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_iotlb.ko 2023-07-21T18:24:46.283 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_net.ko 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/vhost/vhost_scsi.ko 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/apple_bl.ko 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/lcd.ko 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/video/backlight/platform_lcd.ko 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio 2023-07-21T18:24:46.284 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_balloon.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_mmio.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/virtio/virtio_ring.ko 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1 2023-07-21T18:24:46.285 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2482.ko 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/masters/ds2490.ko 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-07-21T18:24:46.286 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_smem.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/slaves/w1_therm.ko 2023-07-21T18:24:46.287 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/w1/wire.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim1535_wdt.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/alim7101_wdt.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/f71808e_wdt.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/hpwdt.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/i6300esb.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-07-21T18:24:46.288 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/iTCO_wdt.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ib700wdt.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ibmasr.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it8712f_wdt.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/it87_wdt.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/machzwd.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/nv_tco.ko 2023-07-21T18:24:46.289 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_pci.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/pcwd_usb.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sch311x_wdt.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/softdog.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/sp5100_tco.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/via_wdt.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-07-21T18:24:46.290 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83877f_wdt.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/w83977f_wdt.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/wdt_pci.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/watchdog/xen_wdt.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-acpi-processor.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-evtchn.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntalloc.ko 2023-07-21T18:24:46.291 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-gntdev.ko 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xen-privcmd.ko 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/drivers/xen/xenfs/xenfs.ko 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs 2023-07-21T18:24:46.292 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/9p/9p.ko 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/affs/affs.ko 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/befs/befs.ko 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/binfmt_misc.ko 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs 2023-07-21T18:24:46.293 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/btrfs/btrfs.ko 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cachefiles/cachefiles.ko 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ceph/ceph.ko 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/coda/coda.ko 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs 2023-07-21T18:24:46.294 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/cramfs/cramfs.ko 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/dlm/dlm.ko 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ecryptfs/ecryptfs.ko 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/f2fs/f2fs.ko 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat 2023-07-21T18:24:46.295 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/fat.ko 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/msdos.ko 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fat/vfat.ko 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fscache/fscache.ko 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/cuse.ko 2023-07-21T18:24:46.296 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/fuse/fuse.ko 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/gfs2/gfs2.ko 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfs/hfs.ko 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/hfsplus/hfsplus.ko 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs 2023-07-21T18:24:46.297 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/isofs/isofs.ko 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/jfs/jfs.ko 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/lockd/lockd.ko 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/minix/minix.ko 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs 2023-07-21T18:24:46.298 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/netfs/netfs.ko 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-07-21T18:24:46.299 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfs.ko 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv3.ko 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs/nfsv4.ko 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/grace.ko 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfs_common/nfs_acl.ko 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd 2023-07-21T18:24:46.300 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nfsd/nfsd.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nilfs2/nilfs2.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-celtic.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-centeuro.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-croatian.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-cyrillic.ko 2023-07-21T18:24:46.301 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-gaelic.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-greek.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-iceland.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-inuit.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-roman.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-romanian.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/mac-turkish.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1250.ko 2023-07-21T18:24:46.302 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1251.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp1255.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp737.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp775.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp850.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp852.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp855.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp857.ko 2023-07-21T18:24:46.303 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp860.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp861.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp862.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp863.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp864.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp865.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp866.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp869.ko 2023-07-21T18:24:46.304 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp874.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp932.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp936.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp949.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_cp950.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_euc-jp.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-1.ko 2023-07-21T18:24:46.305 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-13.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-14.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-15.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-2.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-3.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-4.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-5.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-6.ko 2023-07-21T18:24:46.306 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-7.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_iso8859-9.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-r.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-ru.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_koi8-u.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/nls/nls_utf8.ko 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster 2023-07-21T18:24:46.307 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-07-21T18:24:46.308 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/overlayfs/overlay.ko 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/pstore/ramoops.ko 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/reiserfs/reiserfs.ko 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs 2023-07-21T18:24:46.309 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/romfs/romfs.ko 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/client/cifs.ko 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_arc4.ko 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/smb/common/cifs_md4.ko 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs 2023-07-21T18:24:46.310 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/squashfs/squashfs.ko 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/sysv/sysv.ko 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/udf/udf.ko 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/ufs/ufs.ko 2023-07-21T18:24:46.311 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/fs/xfs/xfs.ko 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/rcu/rcutorture.ko 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/torture.ko 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/kernel/trace/ring_buffer_benchmark.ko 2023-07-21T18:24:46.312 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/asn1_encoder.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc-itu-t.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crc8.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libarc4.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/crypto/libdes.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/libcrc32c.ko 2023-07-21T18:24:46.313 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lru_cache.ko 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4_compress.ko 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/lz4/lz4hc_compress.ko 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/math/cordic.ko 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6 2023-07-21T18:24:46.314 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/raid6/raid6_pq.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/reed_solomon/reed_solomon.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_bm.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_fsm.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/ts_kmp.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/lib/zstd/zstd_compress.ko 2023-07-21T18:24:46.315 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/mm/hwpoison-inject.ko 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/6lowpan.ko 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_dest.ko 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_fragment.ko 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_hop.ko 2023-07-21T18:24:46.316 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_ipv6.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_mobility.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_routing.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/6lowpan/nhc_udp.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/garp.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/mrp.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/p8022.ko 2023-07-21T18:24:46.317 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/psnap.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/802/stp.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/8021q/8021q.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_fd.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_rdma.ko 2023-07-21T18:24:46.318 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/9p/9pnet_virtio.ko 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/batman-adv/batman-adv.ko 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/br_netfilter.ko 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/bridge.ko 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter 2023-07-21T18:24:46.319 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_among.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arp.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_limit.ko 2023-07-21T18:24:46.320 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_log.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_snat.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_stp.ko 2023-07-21T18:24:46.321 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/ebtables.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-07-21T18:24:46.322 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ceph/libceph.ko 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/failover.ko 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/core/pktgen.ko 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp.ko 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_diag.ko 2023-07-21T18:24:46.323 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv4.ko 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dccp/dccp_ipv6.ko 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dns_resolver/dns_resolver.ko 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/dsa_core.ko 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_brcm.ko 2023-07-21T18:24:46.324 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_dsa.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_none.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/dsa/tag_trailer.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ieee802154/ieee802154_socket.ko 2023-07-21T18:24:46.325 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ah4.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/esp4.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/fou.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/gre.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/inet_diag.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_gre.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_tunnel.ko 2023-07-21T18:24:46.326 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ip_vti.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipcomp.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/ipip.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arp_tables.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-07-21T18:24:46.327 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/iptable_security.ko 2023-07-21T18:24:46.328 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-07-21T18:24:46.329 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_bic.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_dctcp.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_diag.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_highspeed.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_htcp.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_hybla.ko 2023-07-21T18:24:46.330 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_illinois.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_lp.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_scalable.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_vegas.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_veno.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_westwood.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tcp_yeah.ko 2023-07-21T18:24:46.331 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/tunnel4.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_diag.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/udp_tunnel.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv4/xfrm4_tunnel.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ah6.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/esp6.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/fou6.ko 2023-07-21T18:24:46.332 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_tunnel.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ip6_vti.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/ipcomp6.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-07-21T18:24:46.333 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-07-21T18:24:46.334 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-07-21T18:24:46.335 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/sit.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/tunnel6.ko 2023-07-21T18:24:46.336 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/ipv6/xfrm6_tunnel.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/key/af_key.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_core.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_debugfs.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_eth.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip.ko 2023-07-21T18:24:46.337 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ip6.ko 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_netlink.ko 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/l2tp/l2tp_ppp.ko 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/llc/llc.ko 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mac802154/mac802154.ko 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls 2023-07-21T18:24:46.338 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/mpls/mpls_gso.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-07-21T18:24:46.339 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-07-21T18:24:46.340 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-07-21T18:24:46.341 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-07-21T18:24:46.342 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conncount.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-07-21T18:24:46.343 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_h323.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_irc.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sane.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_sip.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-07-21T18:24:46.344 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_dup_netdev.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_flow_table_inet.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_log_syslog.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_amanda.ko 2023-07-21T18:24:46.345 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_ftp.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_irc.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_sip.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_nat_tftp.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_synproxy_core.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nf_tables.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_acct.ko 2023-07-21T18:24:46.346 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_log.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_osf.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nfnetlink_queue.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_chain_nat.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_compat.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_ct.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_dup_netdev.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fib.ko 2023-07-21T18:24:46.347 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_fwd_netdev.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_hash.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_limit.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_log.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_masq.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_nat.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_numgen.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_queue.ko 2023-07-21T18:24:46.348 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_quota.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_redir.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_reject_inet.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_socket.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/nft_tunnel.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_AUDIT.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CHECKSUM.ko 2023-07-21T18:24:46.349 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CLASSIFY.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_CT.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_DSCP.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HL.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_HMARK.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_IDLETIMER.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LED.ko 2023-07-21T18:24:46.350 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_LOG.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_MASQUERADE.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NETMAP.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFLOG.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_NFQUEUE.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_RATEEST.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_REDIRECT.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_SECMARK.ko 2023-07-21T18:24:46.351 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPMSS.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TEE.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TPROXY.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_TRACE.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_addrtype.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_bpf.ko 2023-07-21T18:24:46.352 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cgroup.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cluster.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_comment.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connbytes.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlabel.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connlimit.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_connmark.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_conntrack.ko 2023-07-21T18:24:46.353 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_cpu.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dccp.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_devgroup.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_dscp.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ecn.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_esp.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hashlimit.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_helper.ko 2023-07-21T18:24:46.354 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_hl.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipcomp.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_iprange.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_ipvs.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_l2tp.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_length.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_limit.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mac.ko 2023-07-21T18:24:46.355 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_mark.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_multiport.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nat.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_nfacct.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_osf.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_owner.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_physdev.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_pkttype.ko 2023-07-21T18:24:46.356 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_policy.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_quota.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_rateest.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_realm.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_recent.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_sctp.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_set.ko 2023-07-21T18:24:46.357 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_socket.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_state.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_statistic.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_string.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_tcpmss.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_time.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netfilter/xt_u32.ko 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink 2023-07-21T18:24:46.358 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/netlink/netlink_diag.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/nsh/nsh.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/openvswitch.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-geneve.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-gre.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/openvswitch/vport-vxlan.ko 2023-07-21T18:24:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/packet/af_packet_diag.ko 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds.ko 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_rdma.ko 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/rds/rds_tcp.ko 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_csum.ko 2023-07-21T18:24:46.360 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_gact.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_ipt.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_mirred.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_nat.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_pedit.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_police.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_simple.ko 2023-07-21T18:24:46.361 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_skbedit.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/act_vlan.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_basic.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_bpf.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_flow.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_fw.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_route.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/cls_u32.ko 2023-07-21T18:24:46.362 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_cmp.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_ipset.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_meta.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_nbyte.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_text.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/em_u32.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_choke.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_codel.ko 2023-07-21T18:24:46.363 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_drr.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_fq.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_gred.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hfsc.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_hhf.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_htb.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_ingress.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio.ko 2023-07-21T18:24:46.364 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_mqprio_lib.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_multiq.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_netem.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_pie.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_plug.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_prio.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_qfq.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_red.ko 2023-07-21T18:24:46.365 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfb.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_sfq.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_tbf.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sched/sch_teql.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sctp/sctp_diag.ko 2023-07-21T18:24:46.366 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/sunrpc.ko 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc 2023-07-21T18:24:46.367 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/diag.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/tipc/tipc.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/unix/unix_diag.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock.ko 2023-07-21T18:24:46.368 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_diag.ko 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/vmw_vsock/vsock_loopback.ko 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/net/xfrm/xfrm_ipcomp.ko 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2023-07-21T18:24:46.369 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/security/keys/trusted-keys/trusted.ko 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/kernel/virt/lib/irqbypass.ko 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.alias.bin 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin 2023-07-21T18:24:46.370 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.bin 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.builtin.modinfo 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.dep.bin 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.devname 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.order 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.softdep 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols 2023-07-21T18:24:46.371 INFO:teuthology.orchestra.run.smithi101.stdout:/lib/modules/6.5.0-rc1-gc685da6f56cc/modules.symbols.bin 2023-07-21T18:24:46.372 DEBUG:teuthology.task.kernel:get_image_version: 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.372 DEBUG:teuthology.orchestra.run.smithi101:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2023-07-21T18:24:46.397 INFO:teuthology.orchestra.run.smithi101.stdout:postinstall scriptlet (using /bin/sh): 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/installkernel -a -r /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc -a -r /boot/System.map-6.5.0-rc1-gc685da6f56cc ]; then 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:cp /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:cp /boot/System.map-6.5.0-rc1-gc685da6f56cc /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:rm -f /boot/vmlinuz-6.5.0-rc1-gc685da6f56cc /boot/System.map-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/installkernel 6.5.0-rc1-gc685da6f56cc /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:rm -f /boot/.vmlinuz-6.5.0-rc1-gc685da6f56cc-rpm /boot/.System.map-6.5.0-rc1-gc685da6f56cc-rpm 2023-07-21T18:24:46.398 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:preuninstall scriptlet (using /bin/sh): 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/new-kernel-pkg ]; then 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:new-kernel-pkg --remove 6.5.0-rc1-gc685da6f56cc --rminitrd --initrdfile=/boot/initramfs-6.5.0-rc1-gc685da6f56cc.img 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:elif [ -x /usr/bin/kernel-install ]; then 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:kernel-install remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:postuninstall scriptlet (using /bin/sh): 2023-07-21T18:24:46.399 INFO:teuthology.orchestra.run.smithi101.stdout:if [ -x /sbin/update-bootloader ]; then 2023-07-21T18:24:46.400 INFO:teuthology.orchestra.run.smithi101.stdout:/sbin/update-bootloader --remove 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.400 INFO:teuthology.orchestra.run.smithi101.stdout:fi 2023-07-21T18:24:46.401 DEBUG:teuthology.orchestra.run.smithi101:> sudo rpm -qi grub2-tools 2023-07-21T18:24:46.475 INFO:teuthology.orchestra.run.smithi101.stdout:Name : grub2-tools 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Epoch : 1 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Version : 2.02 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Release : 123.el8 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Architecture: x86_64 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Install Date: Thu 14 Jul 2022 06:22:19 PM UTC 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Group : System Environment/Base 2023-07-21T18:24:46.476 INFO:teuthology.orchestra.run.smithi101.stdout:Size : 9265450 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:License : GPLv3+ 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Signature : RSA/SHA256, Thu 31 Mar 2022 07:24:35 PM UTC, Key ID 199e2f91fd431d51 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Source RPM : grub2-2.02-123.el8.src.rpm 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Build Date : Mon 28 Mar 2022 02:12:09 PM UTC 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Build Host : x86-vm-09.build.eng.bos.redhat.com 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Relocations : (not relocatable) 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Packager : Red Hat, Inc. 2023-07-21T18:24:46.477 INFO:teuthology.orchestra.run.smithi101.stdout:Vendor : Red Hat, Inc. 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:URL : http://www.gnu.org/software/grub/ 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:Summary : Support tools for GRUB. 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:Description : 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:customizable bootloader with modular architecture. It supports a rich 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:variety of kernel formats, file systems, computer architectures and 2023-07-21T18:24:46.478 INFO:teuthology.orchestra.run.smithi101.stdout:hardware devices. 2023-07-21T18:24:46.479 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:24:46.479 INFO:teuthology.orchestra.run.smithi101.stdout:This subpackage provides tools for support of all platforms. 2023-07-21T18:24:46.480 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-07-21T18:24:46.480 INFO:teuthology.task.kernel:Updating grub on smithi101 to boot 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:46.480 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-07-21T18:24:48.146 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:24:48.146 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:24:53.151 INFO:teuthology.orchestra.run.smithi101.stderr:Generating grub configuration file ... 2023-07-21T18:24:53.152 INFO:teuthology.orchestra.run.smithi101.stderr:done 2023-07-21T18:24:53.152 DEBUG:teuthology.orchestra.run.smithi101:> mktemp 2023-07-21T18:24:53.182 INFO:teuthology.orchestra.run.smithi101.stdout:/tmp/tmp.1ZPuQyhH76 2023-07-21T18:24:53.182 DEBUG:teuthology.orchestra.run.smithi101:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.1ZPuQyhH76 2023-07-21T18:24:53.260 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 0666 /tmp/tmp.1ZPuQyhH76 2023-07-21T18:24:53.422 DEBUG:teuthology.orchestra.remote:smithi101:/tmp/tmp.1ZPuQyhH76 is 6KB 2023-07-21T18:24:53.469 DEBUG:teuthology.orchestra.run.smithi101:> rm -fr /tmp/tmp.1ZPuQyhH76 2023-07-21T18:24:53.487 DEBUG:teuthology.orchestra.run.smithi101:> sudo /bin/ls /boot/loader/entries || true 2023-07-21T18:24:53.568 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-0-rescue.conf 2023-07-21T18:24:53.569 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-4.18.0-372.9.1.el8.x86_64.conf 2023-07-21T18:24:53.569 INFO:teuthology.orchestra.run.smithi101.stdout:5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc.conf 2023-07-21T18:24:53.570 DEBUG:teuthology.orchestra.run.smithi101:> sudo grub2-set-default 5a84457b7b52416e95d47d02e2e4a60f-6.5.0-rc1-gc685da6f56cc 2023-07-21T18:24:53.717 DEBUG:teuthology.orchestra.run.smithi101:> sudo shutdown -r now 2023-07-21T18:24:54.938 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-07-21T18:24:54.938 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:24:54.939 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:25:11.536 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.156 2023-07-21T18:25:18.448 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2023-07-21T18:25:23.724 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-07-21T18:25:23.724 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:25:23.725 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:25:28.729 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:25:28.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:25:33.733 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:25:33.734 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:25:39.068 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-07-21T18:25:39.511 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:25:39.511 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "6.5.0-rc1-gc685da6f56cc"... 2023-07-21T18:25:39.511 DEBUG:teuthology.orchestra.run.smithi156:> uname -r 2023-07-21T18:25:39.553 INFO:teuthology.orchestra.run.smithi156.stdout:6.5.0-rc1-gc685da6f56cc 2023-07-21T18:25:39.554 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-gc685da6f56cc vs 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:25:39.554 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-21T18:25:39.554 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-21T18:25:40.556 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2023-07-21T18:25:40.556 DEBUG:teuthology.orchestra.run.smithi156:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-21T18:25:40.702 INFO:teuthology.orchestra.run.smithi156.stdout:ttyS1 2023-07-21T18:25:40.729 DEBUG:teuthology.parallel:result is None 2023-07-21T18:25:47.120 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T18:25:56.122 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:25:56.122 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:26:01.125 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.5 2023-07-21T18:26:13.126 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:26:13.127 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:26:18.470 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-07-21T18:26:18.854 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:26:18.854 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "6.5.0-rc1-gc685da6f56cc"... 2023-07-21T18:26:18.855 DEBUG:teuthology.orchestra.run.smithi005:> uname -r 2023-07-21T18:26:18.899 INFO:teuthology.orchestra.run.smithi005.stdout:6.5.0-rc1-gc685da6f56cc 2023-07-21T18:26:18.900 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-gc685da6f56cc vs 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:26:18.900 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-21T18:26:18.900 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-21T18:26:19.632 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T18:26:19.901 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-07-21T18:26:19.902 DEBUG:teuthology.orchestra.run.smithi005:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-21T18:26:20.037 INFO:teuthology.orchestra.run.smithi005.stdout:ttyS1 2023-07-21T18:26:20.061 DEBUG:teuthology.parallel:result is None 2023-07-21T18:26:31.634 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:26:31.635 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:26:34.704 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.101 2023-07-21T18:26:49.706 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:26:49.706 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:26:55.075 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-21T18:26:55.466 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:26:55.466 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "6.5.0-rc1-gc685da6f56cc"... 2023-07-21T18:26:55.466 DEBUG:teuthology.orchestra.run.smithi101:> uname -r 2023-07-21T18:26:55.508 INFO:teuthology.orchestra.run.smithi101.stdout:6.5.0-rc1-gc685da6f56cc 2023-07-21T18:26:55.509 DEBUG:teuthology.task.kernel:current kernel version is 6.5.0-rc1-gc685da6f56cc vs 6.5.0-rc1-gc685da6f56cc 2023-07-21T18:26:55.509 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-07-21T18:26:55.509 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-07-21T18:26:56.510 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-07-21T18:26:56.511 DEBUG:teuthology.orchestra.run.smithi101:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-07-21T18:26:56.665 INFO:teuthology.orchestra.run.smithi101.stdout:ttyS1 2023-07-21T18:26:56.687 DEBUG:teuthology.parallel:result is None 2023-07-21T18:26:56.688 INFO:teuthology.run_tasks:Running task internal.base... 2023-07-21T18:26:56.696 INFO:teuthology.task.internal:Creating test directory... 2023-07-21T18:26:56.696 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:26:56.701 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:26:56.705 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-07-21T18:26:56.755 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-07-21T18:26:56.762 INFO:teuthology.run_tasks:Running task internal.archive... 2023-07-21T18:26:56.769 INFO:teuthology.task.internal:Creating archive directory... 2023-07-21T18:26:56.770 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:26:56.774 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:26:56.790 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-07-21T18:26:56.851 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-07-21T18:26:56.858 INFO:teuthology.task.internal:Enabling coredump saving... 2023-07-21T18:26:56.859 DEBUG:teuthology.orchestra.run.smithi005:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:26:56.863 DEBUG:teuthology.orchestra.run.smithi101:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:26:56.868 DEBUG:teuthology.orchestra.run.smithi156:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-07-21T18:26:56.936 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.936 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.951 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.964 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.965 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.978 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-07-21T18:26:56.981 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-07-21T18:26:56.988 INFO:teuthology.task.internal:Configuring sudo... 2023-07-21T18:26:56.988 DEBUG:teuthology.orchestra.run.smithi005:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:26:56.992 DEBUG:teuthology.orchestra.run.smithi101:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:26:56.997 DEBUG:teuthology.orchestra.run.smithi156:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-07-21T18:26:57.066 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-07-21T18:26:57.076 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-07-21T18:26:57.077 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:26:57.081 DEBUG:teuthology.orchestra.run.smithi101:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:26:57.085 DEBUG:teuthology.orchestra.run.smithi156:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-07-21T18:26:57.128 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:57.233 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:57.327 DEBUG:teuthology.orchestra.run.smithi005:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:57.399 DEBUG:teuthology.orchestra.run.smithi005:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:57.467 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:26:57.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:26:57.537 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:57.611 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:57.685 DEBUG:teuthology.orchestra.run.smithi101:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:57.761 DEBUG:teuthology.orchestra.run.smithi101:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:57.840 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:26:57.840 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:26:57.913 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:57.991 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-07-21T18:26:58.086 DEBUG:teuthology.orchestra.run.smithi156:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:58.163 DEBUG:teuthology.orchestra.run.smithi156:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-07-21T18:26:58.240 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:26:58.241 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-07-21T18:26:58.311 DEBUG:teuthology.orchestra.run.smithi005:> sudo service rsyslog restart 2023-07-21T18:26:58.316 DEBUG:teuthology.orchestra.run.smithi101:> sudo service rsyslog restart 2023-07-21T18:26:58.320 DEBUG:teuthology.orchestra.run.smithi156:> sudo service rsyslog restart 2023-07-21T18:26:58.438 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:26:58.440 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:26:58.477 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T18:26:58.907 INFO:teuthology.run_tasks:Running task internal.timer... 2023-07-21T18:26:58.915 INFO:teuthology.task.internal:Starting timer... 2023-07-21T18:26:58.915 INFO:teuthology.run_tasks:Running task pcp... 2023-07-21T18:26:58.926 INFO:teuthology.run_tasks:Running task selinux... 2023-07-21T18:26:58.934 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-07-21T18:26:58.935 DEBUG:teuthology.orchestra.run.smithi005:> sudo service auditd rotate 2023-07-21T18:26:59.055 INFO:teuthology.orchestra.run.smithi005.stdout:Rotating logs: 2023-07-21T18:26:59.058 DEBUG:teuthology.orchestra.run.smithi101:> sudo service auditd rotate 2023-07-21T18:26:59.173 INFO:teuthology.orchestra.run.smithi101.stdout:Rotating logs: 2023-07-21T18:26:59.177 DEBUG:teuthology.orchestra.run.smithi156:> sudo service auditd rotate 2023-07-21T18:26:59.305 INFO:teuthology.orchestra.run.smithi156.stdout:Rotating logs: 2023-07-21T18:26:59.309 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-07-21T18:26:59.309 DEBUG:teuthology.orchestra.run.smithi005:> /usr/sbin/getenforce 2023-07-21T18:26:59.365 INFO:teuthology.orchestra.run.smithi005.stdout:Permissive 2023-07-21T18:26:59.366 DEBUG:teuthology.orchestra.run.smithi101:> /usr/sbin/getenforce 2023-07-21T18:26:59.422 INFO:teuthology.orchestra.run.smithi101.stdout:Permissive 2023-07-21T18:26:59.422 DEBUG:teuthology.orchestra.run.smithi156:> /usr/sbin/getenforce 2023-07-21T18:26:59.477 INFO:teuthology.orchestra.run.smithi156.stdout:Permissive 2023-07-21T18:26:59.477 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi005.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi101.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi156.front.sepia.ceph.com': 'permissive'} 2023-07-21T18:26:59.478 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:26:59.550 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:26:59.551 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:26:59.622 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:26:59.623 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T18:26:59.696 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T18:26:59.697 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-07-21T18:26:59.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:26:59.767 DEBUG:teuthology.orchestra.run.smithi101:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:26:59.852 DEBUG:teuthology.orchestra.run.smithi156:> sudo /usr/sbin/setenforce permissive 2023-07-21T18:26:59.942 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-07-21T18:26:59.952 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-07-21T18:26:59.960 INFO:teuthology.repo_utils:Fetching main from origin 2023-07-21T18:27:05.163 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-07-21T18:27:05.186 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-07-21T18:27:05.188 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi005.front.sepia.ceph.com,smithi101.front.sepia.ceph.com,smithi156.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-07-21T18:48:16.097 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi005.front.sepia.ceph.com'), Remote(name='ubuntu@smithi101.front.sepia.ceph.com'), Remote(name='ubuntu@smithi156.front.sepia.ceph.com')] 2023-07-21T18:48:16.099 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:48:16.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi005.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:48:16.187 DEBUG:teuthology.orchestra.run.smithi005:> true 2023-07-21T18:48:16.293 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi005.front.sepia.ceph.com' 2023-07-21T18:48:16.294 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:48:16.294 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi101.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:48:16.381 DEBUG:teuthology.orchestra.run.smithi101:> true 2023-07-21T18:48:16.494 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi101.front.sepia.ceph.com' 2023-07-21T18:48:16.495 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:48:16.495 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi156.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-07-21T18:48:16.580 DEBUG:teuthology.orchestra.run.smithi156:> true 2023-07-21T18:48:16.691 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi156.front.sepia.ceph.com' 2023-07-21T18:48:16.692 INFO:teuthology.run_tasks:Running task clock... 2023-07-21T18:48:16.702 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-07-21T18:48:16.702 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:48:16.703 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:48:16.707 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:48:16.707 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:48:16.712 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-07-21T18:48:16.712 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T18:48:16.780 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:48:16.780 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:48:16.806 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-07-21T18:48:16.814 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:48:16.817 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:48:16.841 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-07-21T18:48:16.910 INFO:teuthology.orchestra.run.smithi101.stderr:sudo: ntpd: command not found 2023-07-21T18:48:16.912 INFO:teuthology.orchestra.run.smithi005.stderr:sudo: ntpd: command not found 2023-07-21T18:48:16.938 INFO:teuthology.orchestra.run.smithi101.stdout:506 Cannot talk to daemon 2023-07-21T18:48:16.942 INFO:teuthology.orchestra.run.smithi156.stderr:sudo: ntpd: command not found 2023-07-21T18:48:16.942 INFO:teuthology.orchestra.run.smithi005.stdout:506 Cannot talk to daemon 2023-07-21T18:48:16.972 INFO:teuthology.orchestra.run.smithi156.stdout:506 Cannot talk to daemon 2023-07-21T18:48:16.972 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:48:16.977 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:48:17.005 INFO:teuthology.orchestra.run.smithi101.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:48:17.006 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-07-21T18:48:17.012 INFO:teuthology.orchestra.run.smithi005.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:48:17.041 INFO:teuthology.orchestra.run.smithi156.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-07-21T18:48:17.106 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-07-21T18:48:17.114 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:48:17.114 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-07-21T18:48:17.114 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.115 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.115 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.115 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.125 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-07-21T18:48:17.131 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-07-21T18:48:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:48:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-07-21T18:48:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.133 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.134 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.138 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T18:48:17.139 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-07-21T18:48:17.139 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.139 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.139 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.140 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T18:48:17.141 INFO:teuthology.run_tasks:Running task pexec... 2023-07-21T18:48:17.150 INFO:teuthology.task.pexec:Executing custom commands... 2023-07-21T18:48:17.151 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T18:48:17.151 DEBUG:teuthology.orchestra.run.smithi005:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:48:17.151 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi101.front.sepia.ceph.com 2023-07-21T18:48:17.152 DEBUG:teuthology.orchestra.run.smithi101:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:48:17.152 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi156.front.sepia.ceph.com 2023-07-21T18:48:17.152 DEBUG:teuthology.orchestra.run.smithi156:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-07-21T18:48:17.631 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:17.638 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:17.667 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:18.607 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:50 ago on Fri 21 Jul 2023 06:46:28 PM UTC. 2023-07-21T18:48:18.649 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:11:00 ago on Fri 21 Jul 2023 06:37:18 PM UTC. 2023-07-21T18:48:18.659 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:10:39 ago on Fri 21 Jul 2023 06:37:39 PM UTC. 2023-07-21T18:48:20.431 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-21T18:48:20.432 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-21T18:48:20.432 INFO:teuthology.orchestra.run.smithi156.stdout: Package Architecture Version Repository Size 2023-07-21T18:48:20.432 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout:Resetting modules: 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout:================================================================================ 2023-07-21T18:48:20.433 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:20.452 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-21T18:48:20.453 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-21T18:48:20.453 INFO:teuthology.orchestra.run.smithi101.stdout: Package Architecture Version Repository Size 2023-07-21T18:48:20.453 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-21T18:48:20.453 INFO:teuthology.orchestra.run.smithi101.stdout:Resetting modules: 2023-07-21T18:48:20.453 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools 2023-07-21T18:48:20.454 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:20.454 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-21T18:48:20.454 INFO:teuthology.orchestra.run.smithi101.stdout:================================================================================ 2023-07-21T18:48:20.454 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:20.622 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-07-21T18:48:20.623 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-07-21T18:48:20.623 INFO:teuthology.orchestra.run.smithi005.stdout: Package Architecture Version Repository Size 2023-07-21T18:48:20.623 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-07-21T18:48:20.623 INFO:teuthology.orchestra.run.smithi005.stdout:Resetting modules: 2023-07-21T18:48:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools 2023-07-21T18:48:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:20.624 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-07-21T18:48:20.624 INFO:teuthology.orchestra.run.smithi005.stdout:================================================================================ 2023-07-21T18:48:20.624 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:21.672 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:53 ago on Fri 21 Jul 2023 06:46:28 PM UTC. 2023-07-21T18:48:21.691 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:11:03 ago on Fri 21 Jul 2023 06:37:18 PM UTC. 2023-07-21T18:48:21.878 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:10:42 ago on Fri 21 Jul 2023 06:37:39 PM UTC. 2023-07-21T18:48:24.408 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-21T18:48:24.758 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-21T18:48:24.869 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-07-21T18:48:25.012 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:25.362 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:25.490 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:25.954 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:11:07 ago on Fri 21 Jul 2023 06:37:18 PM UTC. 2023-07-21T18:48:26.288 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:01:57 ago on Fri 21 Jul 2023 06:46:28 PM UTC. 2023-07-21T18:48:26.467 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:10:46 ago on Fri 21 Jul 2023 06:37:39 PM UTC. 2023-07-21T18:48:27.736 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-21T18:48:27.835 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-21T18:48:27.835 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-07-21T18:48:27.835 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout:Installing group/module packages: 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:48:27.836 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout:Installing module profiles: 2023-07-21T18:48:27.837 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools/common 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout: container-tools rhel8 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout:============================================================================================================ 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout:Install 10 Packages 2023-07-21T18:48:27.838 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:27.839 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 18 M 2023-07-21T18:48:27.839 INFO:teuthology.orchestra.run.smithi156.stdout:Installed size: 65 M 2023-07-21T18:48:27.839 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-07-21T18:48:27.930 INFO:teuthology.orchestra.run.smithi156.stdout:(1/10): python3-pyxdg-0.25-16.el8.noarch.rpm 1.0 MB/s | 94 kB 00:00 2023-07-21T18:48:27.955 INFO:teuthology.orchestra.run.smithi156.stdout:(2/10): python3-pytoml-0.1.14-5.git7dea353.el8. 213 kB/s | 25 kB 00:00 2023-07-21T18:48:27.980 INFO:teuthology.orchestra.run.smithi156.stdout:(3/10): yajl-2.1.0-10.el8.x86_64.rpm 288 kB/s | 41 kB 00:00 2023-07-21T18:48:28.006 INFO:teuthology.orchestra.run.smithi156.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 654 kB/s | 49 kB 00:00 2023-07-21T18:48:28.039 INFO:teuthology.orchestra.run.smithi156.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.1 MB/s | 524 kB 00:00 2023-07-21T18:48:28.065 INFO:teuthology.orchestra.run.smithi156.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T18:48:28.090 INFO:teuthology.orchestra.run.smithi156.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T18:48:28.111 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-21T18:48:28.209 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout:Installing group/module packages: 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:48:28.210 INFO:teuthology.orchestra.run.smithi101.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:48:28.211 INFO:teuthology.orchestra.run.smithi101.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout:Installing module profiles: 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools/common 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling module streams: 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: container-tools rhel8 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-21T18:48:28.212 INFO:teuthology.orchestra.run.smithi101.stdout:============================================================================================================ 2023-07-21T18:48:28.213 INFO:teuthology.orchestra.run.smithi101.stdout:Install 10 Packages 2023-07-21T18:48:28.213 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:28.213 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 18 M 2023-07-21T18:48:28.213 INFO:teuthology.orchestra.run.smithi101.stdout:Installed size: 65 M 2023-07-21T18:48:28.214 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-07-21T18:48:28.232 INFO:teuthology.orchestra.run.smithi156.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 15 MB/s | 2.2 MB 00:00 2023-07-21T18:48:28.284 INFO:teuthology.orchestra.run.smithi101.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 349 kB/s | 25 kB 00:00 2023-07-21T18:48:28.309 INFO:teuthology.orchestra.run.smithi101.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 423 kB/s | 41 kB 00:00 2023-07-21T18:48:28.326 INFO:teuthology.orchestra.run.smithi101.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 827 kB/s | 94 kB 00:00 2023-07-21T18:48:28.352 INFO:teuthology.orchestra.run.smithi101.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 735 kB/s | 49 kB 00:00 2023-07-21T18:48:28.382 INFO:teuthology.orchestra.run.smithi156.stdout:(9/10): buildah-1.26.2-1.module+el8.6.0+16771+2 20 MB/s | 8.0 MB 00:00 2023-07-21T18:48:28.386 INFO:teuthology.orchestra.run.smithi101.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 6.8 MB/s | 524 kB 00:00 2023-07-21T18:48:28.411 INFO:teuthology.orchestra.run.smithi101.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.5 MB/s | 149 kB 00:00 2023-07-21T18:48:28.436 INFO:teuthology.orchestra.run.smithi101.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T18:48:28.474 INFO:teuthology.orchestra.run.smithi156.stdout:(10/10): skopeo-1.9.1-1.module+el8.6.0+16771+28 17 MB/s | 7.0 MB 00:00 2023-07-21T18:48:28.475 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:48:28.475 INFO:teuthology.orchestra.run.smithi156.stdout:Total 29 MB/s | 18 MB 00:00 2023-07-21T18:48:28.488 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-07-21T18:48:28.537 INFO:teuthology.orchestra.run.smithi101.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 22 MB/s | 2.2 MB 00:00 2023-07-21T18:48:28.594 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-07-21T18:48:28.594 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-07-21T18:48:28.594 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-07-21T18:48:28.594 INFO:teuthology.orchestra.run.smithi005.stdout:Installing group/module packages: 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: buildah x86_64 1:1.26.2-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 8.0 M 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: cockpit-podman noarch 49.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 524 k 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: crun x86_64 1.5-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 213 k 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: python3-podman noarch 4.0.1-1.module+el8.6.0+16837+85b82211 rhel-8-for-x86_64-appstream-rpms 149 k 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: skopeo x86_64 2:1.9.1-1.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 7.0 M 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: toolbox x86_64 0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 2.2 M 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout: udica noarch 0.2.6-4.module+el8.6.0+16771+28dfca77 rhel-8-for-x86_64-appstream-rpms 49 k 2023-07-21T18:48:28.595 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout: yajl x86_64 2.1.0-10.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout:Installing module profiles: 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools/common 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:28.596 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling module streams: 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout: container-tools rhel8 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout:============================================================================================================ 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout:Install 10 Packages 2023-07-21T18:48:28.597 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:28.598 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 18 M 2023-07-21T18:48:28.598 INFO:teuthology.orchestra.run.smithi005.stdout:Installed size: 65 M 2023-07-21T18:48:28.598 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-07-21T18:48:28.663 INFO:teuthology.orchestra.run.smithi005.stdout:(1/10): python3-pytoml-0.1.14-5.git7dea353.el8. 377 kB/s | 25 kB 00:00 2023-07-21T18:48:28.679 INFO:teuthology.orchestra.run.smithi101.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 26 MB/s | 7.0 MB 00:00 2023-07-21T18:48:28.688 INFO:teuthology.orchestra.run.smithi005.stdout:(2/10): yajl-2.1.0-10.el8.x86_64.rpm 448 kB/s | 41 kB 00:00 2023-07-21T18:48:28.716 INFO:teuthology.orchestra.run.smithi005.stdout:(3/10): python3-pyxdg-0.25-16.el8.noarch.rpm 791 kB/s | 94 kB 00:00 2023-07-21T18:48:28.741 INFO:teuthology.orchestra.run.smithi005.stdout:(4/10): udica-0.2.6-4.module+el8.6.0+16771+28df 634 kB/s | 49 kB 00:00 2023-07-21T18:48:28.770 INFO:teuthology.orchestra.run.smithi101.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 18 MB/s | 8.0 MB 00:00 2023-07-21T18:48:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:48:28.771 INFO:teuthology.orchestra.run.smithi101.stdout:Total 33 MB/s | 18 MB 00:00 2023-07-21T18:48:28.783 INFO:teuthology.orchestra.run.smithi005.stdout:(5/10): cockpit-podman-49.1-1.module+el8.6.0+16 5.4 MB/s | 524 kB 00:00 2023-07-21T18:48:28.807 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-07-21T18:48:28.808 INFO:teuthology.orchestra.run.smithi005.stdout:(6/10): python3-podman-4.0.1-1.module+el8.6.0+1 2.2 MB/s | 149 kB 00:00 2023-07-21T18:48:28.830 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-07-21T18:48:28.831 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-07-21T18:48:28.834 INFO:teuthology.orchestra.run.smithi005.stdout:(7/10): crun-1.5-1.module+el8.6.0+16771+28dfca7 4.1 MB/s | 213 kB 00:00 2023-07-21T18:48:28.942 INFO:teuthology.orchestra.run.smithi005.stdout:(8/10): toolbox-0.0.99.3-0.6.module+el8.6.0+167 20 MB/s | 2.2 MB 00:00 2023-07-21T18:48:29.103 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-07-21T18:48:29.125 INFO:teuthology.orchestra.run.smithi005.stdout:(9/10): skopeo-1.9.1-1.module+el8.6.0+16771+28d 22 MB/s | 7.0 MB 00:00 2023-07-21T18:48:29.126 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-07-21T18:48:29.126 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-07-21T18:48:29.226 INFO:teuthology.orchestra.run.smithi005.stdout:(10/10): buildah-1.26.2-1.module+el8.6.0+16771+ 16 MB/s | 8.0 MB 00:00 2023-07-21T18:48:29.227 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:48:29.227 INFO:teuthology.orchestra.run.smithi005.stdout:Total 29 MB/s | 18 MB 00:00 2023-07-21T18:48:29.337 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-07-21T18:48:29.342 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-07-21T18:48:29.568 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-07-21T18:48:29.592 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-07-21T18:48:29.593 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-07-21T18:48:29.635 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-07-21T18:48:29.640 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-07-21T18:48:29.900 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-07-21T18:48:30.066 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-07-21T18:48:30.071 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-07-21T18:48:30.127 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:48:30.154 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-07-21T18:48:30.352 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:48:30.364 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:30.523 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:30.625 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:48:30.773 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-07-21T18:48:30.801 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:48:30.900 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:48:30.965 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/10 2023-07-21T18:48:31.024 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:48:31.137 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:31.302 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:48:31.357 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 3/10 2023-07-21T18:48:31.434 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:48:31.604 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 4/10 2023-07-21T18:48:32.006 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 5/10 2023-07-21T18:48:32.186 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:48:32.263 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:48:32.907 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 6/10 2023-07-21T18:48:33.191 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:48:33.196 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:48:33.501 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:48:33.501 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:48:33.704 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:48:33.728 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:48:33.856 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 7/10 2023-07-21T18:48:33.911 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:33.935 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:34.098 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 8/10 2023-07-21T18:48:34.250 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 9/10 2023-07-21T18:48:34.342 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:48:34.343 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:48:34.344 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:48:34.344 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:48:34.344 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:48:34.345 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:34.345 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:48:34.345 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:48:34.346 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:48:34.600 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 10/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 1/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : yajl-2.1.0-10.el8.x86_64 2/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 3/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 4/10 2023-07-21T18:48:35.009 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca7 5/10 2023-07-21T18:48:35.010 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x 6/10 2023-07-21T18:48:35.010 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-podman-4.0.1-1.module+el8.6.0+16837+85b822 7/10 2023-07-21T18:48:35.010 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 8/10 2023-07-21T18:48:35.010 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86 9/10 2023-07-21T18:48:35.742 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:48:35.743 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:07 ago on Fri 21 Jul 2023 06:46:28 PM UTC. 2023-07-21T18:48:35.771 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:48:35.771 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:11:17 ago on Fri 21 Jul 2023 06:37:18 PM UTC. 2023-07-21T18:48:36.498 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77 10/10 2023-07-21T18:48:36.499 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:10:56 ago on Fri 21 Jul 2023 06:37:39 PM UTC. 2023-07-21T18:48:39.078 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-07-21T18:48:39.190 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.326 INFO:teuthology.orchestra.run.smithi101.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:48:39.327 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-21T18:48:39.412 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:39.412 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:48:39.413 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:48:39.414 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-21T18:48:39.546 DEBUG:teuthology.parallel:result is None 2023-07-21T18:48:39.647 DEBUG:teuthology.parallel:result is None 2023-07-21T18:48:39.930 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2023-07-21T18:48:40.159 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:40.160 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-07-21T18:48:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: buildah-1:1.26.2-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: cockpit-podman-49.1-1.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: crun-1.5-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:40.160 INFO:teuthology.orchestra.run.smithi005.stdout: python3-podman-4.0.1-1.module+el8.6.0+16837+85b82211.noarch 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: skopeo-2:1.9.1-1.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: toolbox-0.0.99.3-0.6.module+el8.6.0+16771+28dfca77.x86_64 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: udica-0.2.6-4.module+el8.6.0+16771+28dfca77.noarch 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: yajl-2.1.0-10.el8.x86_64 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:48:40.161 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-07-21T18:48:40.409 DEBUG:teuthology.parallel:result is None 2023-07-21T18:48:40.409 INFO:teuthology.run_tasks:Running task install... 2023-07-21T18:48:40.418 DEBUG:teuthology.task.install:project ceph 2023-07-21T18:48:40.419 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'}} 2023-07-21T18:48:40.419 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9'} 2023-07-21T18:48:40.420 INFO:teuthology.task.install:Using flavor: default 2023-07-21T18:48:40.425 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-07-21T18:48:40.426 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-07-21T18:48:40.426 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:48:40.427 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:48:40.429 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:48:40.429 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:48:40.430 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'tag': None, 'wait_for_package': False} 2023-07-21T18:48:40.431 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:48:40.765 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:48:40.765 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:48:40.770 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:48:40.770 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:48:40.773 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/ 2023-07-21T18:48:40.773 INFO:teuthology.task.install.rpm:Package version is 18.0.0-4746.g81cf785a 2023-07-21T18:48:41.040 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:48:41.040 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:48:41.040 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:48:41.047 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:48:41.048 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:48:41.048 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:48:41.051 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-07-21T18:48:41.051 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:48:41.051 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-07-21T18:48:41.110 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:48:41.110 DEBUG:teuthology.orchestra.run.smithi005:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:48:41.115 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:48:41.115 DEBUG:teuthology.orchestra.run.smithi156:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:48:41.120 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-07-21T18:48:41.120 DEBUG:teuthology.orchestra.run.smithi101:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/81cf785aa7781c4e08381c540f890cb9bc987ef9/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-07-21T18:48:41.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:48:41.226 DEBUG:teuthology.orchestra.run.smithi156:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:48:41.236 DEBUG:teuthology.orchestra.run.smithi101:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-07-21T18:48:41.326 DEBUG:teuthology.orchestra.run.smithi005:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:48:41.327 DEBUG:teuthology.orchestra.run.smithi156:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:48:41.336 DEBUG:teuthology.orchestra.run.smithi101:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-07-21T18:48:41.397 INFO:teuthology.orchestra.run.smithi005.stdout:check_obsoletes = 1 2023-07-21T18:48:41.398 INFO:teuthology.orchestra.run.smithi156.stdout:check_obsoletes = 1 2023-07-21T18:48:41.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum clean all 2023-07-21T18:48:41.401 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum clean all 2023-07-21T18:48:41.403 INFO:teuthology.orchestra.run.smithi101.stdout:check_obsoletes = 1 2023-07-21T18:48:41.407 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum clean all 2023-07-21T18:48:41.876 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:41.880 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:41.895 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:42.269 INFO:teuthology.orchestra.run.smithi101.stdout:44 files removed 2023-07-21T18:48:42.281 INFO:teuthology.orchestra.run.smithi156.stdout:44 files removed 2023-07-21T18:48:42.291 INFO:teuthology.orchestra.run.smithi005.stdout:44 files removed 2023-07-21T18:48:42.330 DEBUG:teuthology.orchestra.run.smithi101:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:48:42.341 DEBUG:teuthology.orchestra.run.smithi156:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:48:42.352 DEBUG:teuthology.orchestra.run.smithi005:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-07-21T18:48:42.821 INFO:teuthology.orchestra.run.smithi101.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:42.847 INFO:teuthology.orchestra.run.smithi156.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:42.860 INFO:teuthology.orchestra.run.smithi005.stdout:Updating Subscription Management repositories. 2023-07-21T18:48:44.396 INFO:teuthology.orchestra.run.smithi101.stdout:ceph packages for x86_64 143 kB/s | 81 kB 00:00 2023-07-21T18:48:44.441 INFO:teuthology.orchestra.run.smithi005.stdout:ceph packages for x86_64 137 kB/s | 81 kB 00:00 2023-07-21T18:48:44.457 INFO:teuthology.orchestra.run.smithi156.stdout:ceph packages for x86_64 146 kB/s | 81 kB 00:00 2023-07-21T18:48:44.889 INFO:teuthology.orchestra.run.smithi101.stdout:ceph noarch packages 21 kB/s | 9.4 kB 00:00 2023-07-21T18:48:44.950 INFO:teuthology.orchestra.run.smithi005.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-07-21T18:48:44.975 INFO:teuthology.orchestra.run.smithi156.stdout:ceph noarch packages 20 kB/s | 9.4 kB 00:00 2023-07-21T18:48:45.424 INFO:teuthology.orchestra.run.smithi101.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2023-07-21T18:48:45.493 INFO:teuthology.orchestra.run.smithi005.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2023-07-21T18:48:45.510 INFO:teuthology.orchestra.run.smithi156.stdout:ceph source packages 2.9 kB/s | 1.5 kB 00:00 2023-07-21T18:48:45.684 INFO:teuthology.orchestra.run.smithi101.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-07-21T18:48:45.794 INFO:teuthology.orchestra.run.smithi156.stdout:Copr repo for python3-asyncssh owned by ceph 13 kB/s | 3.5 kB 00:00 2023-07-21T18:48:45.853 INFO:teuthology.orchestra.run.smithi005.stdout:Copr repo for python3-asyncssh owned by ceph 10 kB/s | 3.5 kB 00:00 2023-07-21T18:48:51.549 INFO:teuthology.orchestra.run.smithi101.stdout:Extra Packages for Enterprise Linux 2.7 MB/s | 16 MB 00:05 2023-07-21T18:48:53.733 INFO:teuthology.orchestra.run.smithi156.stdout:Extra Packages for Enterprise Linux 2.0 MB/s | 16 MB 00:07 2023-07-21T18:48:56.034 INFO:teuthology.orchestra.run.smithi101.stdout:lab-extras 385 kB/s | 24 kB 00:00 2023-07-21T18:48:57.229 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 47 MB/s | 53 MB 00:01 2023-07-21T18:48:58.127 INFO:teuthology.orchestra.run.smithi156.stdout:lab-extras 358 kB/s | 24 kB 00:00 2023-07-21T18:48:59.363 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 MB/s | 47 MB 00:01 2023-07-21T18:49:01.661 INFO:teuthology.orchestra.run.smithi005.stdout:Extra Packages for Enterprise Linux 1.0 MB/s | 16 MB 00:15 2023-07-21T18:49:06.583 INFO:teuthology.orchestra.run.smithi005.stdout:lab-extras 42 kB/s | 24 kB 00:00 2023-07-21T18:49:06.810 INFO:teuthology.orchestra.run.smithi101.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 MB/s | 47 MB 00:01 2023-07-21T18:49:07.775 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-21T18:49:08.528 INFO:teuthology.orchestra.run.smithi156.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-07-21T18:49:17.285 INFO:teuthology.orchestra.run.smithi005.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 MB/s | 47 MB 00:01 2023-07-21T18:49:19.144 INFO:teuthology.orchestra.run.smithi101.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:19.145 INFO:teuthology.orchestra.run.smithi101.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:19.160 INFO:teuthology.orchestra.run.smithi101.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:19.161 INFO:teuthology.orchestra.run.smithi101.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:19.163 INFO:teuthology.orchestra.run.smithi101.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:49:19.164 INFO:teuthology.orchestra.run.smithi101.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:49:19.166 INFO:teuthology.orchestra.run.smithi101.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:49:19.170 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:49:19.171 INFO:teuthology.orchestra.run.smithi101.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:49:19.308 INFO:teuthology.orchestra.run.smithi101.stdout:Dependencies resolved. 2023-07-21T18:49:19.321 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-21T18:49:19.321 INFO:teuthology.orchestra.run.smithi101.stdout: Package Arch Version Repository Size 2023-07-21T18:49:19.321 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-21T18:49:19.321 INFO:teuthology.orchestra.run.smithi101.stdout:Installing: 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:49:19.322 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:49:19.323 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:49:19.324 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:49:19.325 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:49:19.326 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrading: 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout:Installing dependencies: 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:49:19.327 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:49:19.328 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:49:19.329 INFO:teuthology.orchestra.run.smithi101.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:49:19.330 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:49:19.331 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:49:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:49:19.333 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:49:19.334 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:49:19.335 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:49:19.336 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:49:19.337 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:49:19.338 INFO:teuthology.orchestra.run.smithi101.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout:Enabling module streams: 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql 10 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:49:19.339 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction Summary 2023-07-21T18:49:19.340 INFO:teuthology.orchestra.run.smithi101.stdout:=========================================================================================================================== 2023-07-21T18:49:19.340 INFO:teuthology.orchestra.run.smithi101.stdout:Install 128 Packages 2023-07-21T18:49:19.340 INFO:teuthology.orchestra.run.smithi101.stdout:Upgrade 3 Packages 2023-07-21T18:49:19.340 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:49:19.340 INFO:teuthology.orchestra.run.smithi101.stdout:Total download size: 205 M 2023-07-21T18:49:19.341 INFO:teuthology.orchestra.run.smithi101.stdout:Downloading Packages: 2023-07-21T18:49:19.913 INFO:teuthology.orchestra.run.smithi101.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-07-21T18:49:20.688 INFO:teuthology.orchestra.run.smithi101.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 848 kB 00:00 2023-07-21T18:49:20.761 INFO:teuthology.orchestra.run.smithi156.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:20.762 INFO:teuthology.orchestra.run.smithi156.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:20.778 INFO:teuthology.orchestra.run.smithi156.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:20.779 INFO:teuthology.orchestra.run.smithi156.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:20.780 INFO:teuthology.orchestra.run.smithi156.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:49:20.781 INFO:teuthology.orchestra.run.smithi156.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:49:20.784 INFO:teuthology.orchestra.run.smithi156.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:49:20.787 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:49:20.789 INFO:teuthology.orchestra.run.smithi156.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:49:20.872 INFO:teuthology.orchestra.run.smithi101.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 904 kB/s | 165 kB 00:00 2023-07-21T18:49:20.928 INFO:teuthology.orchestra.run.smithi156.stdout:Dependencies resolved. 2023-07-21T18:49:20.940 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout: Package Arch Version Repository Size 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout:Installing: 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:49:20.941 INFO:teuthology.orchestra.run.smithi156.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:49:20.942 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:49:20.943 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:49:20.944 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:49:20.945 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrading: 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:49:20.946 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout:Installing dependencies: 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:49:20.947 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:49:20.948 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:49:20.949 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:49:20.950 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:49:20.951 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:49:20.952 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:49:20.953 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:49:20.954 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:49:20.955 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:49:20.956 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:49:20.957 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:49:20.958 INFO:teuthology.orchestra.run.smithi156.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout:Enabling module streams: 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql 10 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:49:20.959 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction Summary 2023-07-21T18:49:20.960 INFO:teuthology.orchestra.run.smithi156.stdout:=========================================================================================================================== 2023-07-21T18:49:20.960 INFO:teuthology.orchestra.run.smithi156.stdout:Install 128 Packages 2023-07-21T18:49:20.960 INFO:teuthology.orchestra.run.smithi156.stdout:Upgrade 3 Packages 2023-07-21T18:49:20.960 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:49:20.960 INFO:teuthology.orchestra.run.smithi156.stdout:Total download size: 205 M 2023-07-21T18:49:20.961 INFO:teuthology.orchestra.run.smithi156.stdout:Downloading Packages: 2023-07-21T18:49:21.502 INFO:teuthology.orchestra.run.smithi156.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 22 kB/s | 6.4 kB 00:00 2023-07-21T18:49:21.922 INFO:teuthology.orchestra.run.smithi101.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.2 MB/s | 2.3 MB 00:01 2023-07-21T18:49:22.256 INFO:teuthology.orchestra.run.smithi101.stdout:(5/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 2.0 MB/s | 5.4 MB 00:02 2023-07-21T18:49:22.469 INFO:teuthology.orchestra.run.smithi156.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 878 kB/s | 848 kB 00:00 2023-07-21T18:49:22.498 INFO:teuthology.orchestra.run.smithi101.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 2.7 MB/s | 1.5 MB 00:00 2023-07-21T18:49:22.644 INFO:teuthology.orchestra.run.smithi156.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 945 kB/s | 165 kB 00:00 2023-07-21T18:49:23.281 INFO:teuthology.orchestra.run.smithi101.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 4.5 MB/s | 4.6 MB 00:01 2023-07-21T18:49:23.811 INFO:teuthology.orchestra.run.smithi156.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 2.0 MB/s | 2.3 MB 00:01 2023-07-21T18:49:24.911 INFO:teuthology.orchestra.run.smithi156.stdout:(5/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 1.5 MB 00:01 2023-07-21T18:49:25.286 INFO:teuthology.orchestra.run.smithi156.stdout:(6/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 1.3 MB/s | 5.4 MB 00:04 2023-07-21T18:49:25.415 INFO:teuthology.orchestra.run.smithi101.stdout:(8/131): ceph-common-18.0.0-4746.g81cf785a.el8. 3.8 MB/s | 22 MB 00:05 2023-07-21T18:49:25.498 INFO:teuthology.orchestra.run.smithi101.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 294 kB/s | 24 kB 00:00 2023-07-21T18:49:29.319 INFO:teuthology.orchestra.run.smithi156.stdout:(7/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 1.0 MB/s | 4.6 MB 00:04 2023-07-21T18:49:29.445 INFO:teuthology.orchestra.run.smithi005.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:29.446 INFO:teuthology.orchestra.run.smithi005.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-07-21T18:49:29.461 INFO:teuthology.orchestra.run.smithi005.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:29.463 INFO:teuthology.orchestra.run.smithi005.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-07-21T18:49:29.464 INFO:teuthology.orchestra.run.smithi005.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-07-21T18:49:29.465 INFO:teuthology.orchestra.run.smithi005.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-07-21T18:49:29.468 INFO:teuthology.orchestra.run.smithi005.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-07-21T18:49:29.471 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-07-21T18:49:29.472 INFO:teuthology.orchestra.run.smithi005.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-07-21T18:49:29.621 INFO:teuthology.orchestra.run.smithi005.stdout:Dependencies resolved. 2023-07-21T18:49:29.635 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-07-21T18:49:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: Package Arch Version Repository Size 2023-07-21T18:49:29.635 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-07-21T18:49:29.635 INFO:teuthology.orchestra.run.smithi005.stdout:Installing: 2023-07-21T18:49:29.635 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.4 k 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 5.4 M 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 848 k 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 165 k 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 1.5 M 2023-07-21T18:49:29.636 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 129 k 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 1.7 M 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 7.4 M 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 47 k 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 13 M 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 48 M 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 258 k 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 166 k 2023-07-21T18:49:29.637 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 258 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 18 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 31 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 772 k 2023-07-21T18:49:29.638 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 139 k 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 197 k 2023-07-21T18:49:29.639 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 370 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 364 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 107 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 94 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.4 M 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 179 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-07-21T18:49:29.640 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrading: 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: librados2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.7 M 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 3.6 M 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout:Installing dependencies: 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 22 M 2023-07-21T18:49:29.641 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 23 k 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 2.3 M 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 240 k 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 4.6 M 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 18 M 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts noarch 2:18.0.0-4746.g81cf785a.el8 ceph-noarch 15 k 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 24 k 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-07-21T18:49:29.642 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 137 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-07-21T18:49:29.643 INFO:teuthology.orchestra.run.smithi005.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 473 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2 x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 6.7 M 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-07-21T18:49:29.644 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-07-21T18:49:29.645 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 45 k 2023-07-21T18:49:29.646 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common x86_64 2:18.0.0-4746.g81cf785a.el8 ceph 119 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-07-21T18:49:29.647 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-07-21T18:49:29.648 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-07-21T18:49:29.649 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-07-21T18:49:29.650 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa noarch 4.8-1.el8 epel 61 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-07-21T18:49:29.651 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-07-21T18:49:29.652 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling module streams: 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql 10 2023-07-21T18:49:29.653 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:49:29.654 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction Summary 2023-07-21T18:49:29.654 INFO:teuthology.orchestra.run.smithi005.stdout:=========================================================================================================================== 2023-07-21T18:49:29.654 INFO:teuthology.orchestra.run.smithi005.stdout:Install 128 Packages 2023-07-21T18:49:29.654 INFO:teuthology.orchestra.run.smithi005.stdout:Upgrade 3 Packages 2023-07-21T18:49:29.654 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:49:29.655 INFO:teuthology.orchestra.run.smithi005.stdout:Total download size: 205 M 2023-07-21T18:49:29.655 INFO:teuthology.orchestra.run.smithi005.stdout:Downloading Packages: 2023-07-21T18:49:30.260 INFO:teuthology.orchestra.run.smithi005.stdout:(1/131): ceph-18.0.0-4746.g81cf785a.el8.x86_64. 20 kB/s | 6.4 kB 00:00 2023-07-21T18:49:31.193 INFO:teuthology.orchestra.run.smithi005.stdout:(2/131): ceph-fuse-18.0.0-4746.g81cf785a.el8.x8 909 kB/s | 848 kB 00:00 2023-07-21T18:49:31.485 INFO:teuthology.orchestra.run.smithi005.stdout:(3/131): ceph-immutable-object-cache-18.0.0-474 567 kB/s | 165 kB 00:00 2023-07-21T18:49:33.152 INFO:teuthology.orchestra.run.smithi005.stdout:(4/131): ceph-mds-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 2.3 MB 00:01 2023-07-21T18:49:34.302 INFO:teuthology.orchestra.run.smithi005.stdout:(5/131): ceph-base-18.0.0-4746.g81cf785a.el8.x8 1.2 MB/s | 5.4 MB 00:04 2023-07-21T18:49:34.502 INFO:teuthology.orchestra.run.smithi005.stdout:(6/131): ceph-mgr-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 1.5 MB 00:01 2023-07-21T18:49:35.065 INFO:teuthology.orchestra.run.smithi101.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 1.1 MB/s | 13 MB 00:11 2023-07-21T18:49:35.373 INFO:teuthology.orchestra.run.smithi101.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 837 kB/s | 258 kB 00:00 2023-07-21T18:49:35.473 INFO:teuthology.orchestra.run.smithi101.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 310 kB/s | 31 kB 00:00 2023-07-21T18:49:35.519 INFO:teuthology.orchestra.run.smithi005.stdout:(7/131): ceph-common-18.0.0-4746.g81cf785a.el8. 4.0 MB/s | 22 MB 00:05 2023-07-21T18:49:36.299 INFO:teuthology.orchestra.run.smithi101.stdout:(13/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 1.3 MB/s | 18 MB 00:13 2023-07-21T18:49:36.397 INFO:teuthology.orchestra.run.smithi101.stdout:(14/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 836 kB/s | 772 kB 00:00 2023-07-21T18:49:36.483 INFO:teuthology.orchestra.run.smithi101.stdout:(15/131): libcephsqlite-18.0.0-4746.g81cf785a.e 749 kB/s | 137 kB 00:00 2023-07-21T18:49:36.625 INFO:teuthology.orchestra.run.smithi101.stdout:(16/131): librados-devel-18.0.0-4746.g81cf785a. 612 kB/s | 139 kB 00:00 2023-07-21T18:49:36.841 INFO:teuthology.orchestra.run.smithi101.stdout:(17/131): libradosstriper1-18.0.0-4746.g81cf785 1.3 MB/s | 473 kB 00:00 2023-07-21T18:49:36.908 INFO:teuthology.orchestra.run.smithi101.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 673 kB/s | 45 kB 00:00 2023-07-21T18:49:36.994 INFO:teuthology.orchestra.run.smithi005.stdout:(8/131): ceph-mon-18.0.0-4746.g81cf785a.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-07-21T18:49:37.025 INFO:teuthology.orchestra.run.smithi101.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.0 MB/s | 119 kB 00:00 2023-07-21T18:49:37.061 INFO:teuthology.orchestra.run.smithi005.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 366 kB/s | 24 kB 00:00 2023-07-21T18:49:37.201 INFO:teuthology.orchestra.run.smithi101.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 1.1 MB/s | 197 kB 00:00 2023-07-21T18:49:37.426 INFO:teuthology.orchestra.run.smithi101.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.6 MB/s | 370 kB 00:00 2023-07-21T18:49:37.660 INFO:teuthology.orchestra.run.smithi101.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.5 MB/s | 364 kB 00:00 2023-07-21T18:49:37.802 INFO:teuthology.orchestra.run.smithi101.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 753 kB/s | 107 kB 00:00 2023-07-21T18:49:37.886 INFO:teuthology.orchestra.run.smithi101.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.1 MB/s | 94 kB 00:00 2023-07-21T18:49:38.195 INFO:teuthology.orchestra.run.smithi156.stdout:(8/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x86 1.4 MB/s | 18 MB 00:12 2023-07-21T18:49:38.286 INFO:teuthology.orchestra.run.smithi156.stdout:(9/131): ceph-selinux-18.0.0-4746.g81cf785a.el8 269 kB/s | 24 kB 00:00 2023-07-21T18:49:38.562 INFO:teuthology.orchestra.run.smithi005.stdout:(10/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 4.4 MB/s | 13 MB 00:03 2023-07-21T18:49:38.662 INFO:teuthology.orchestra.run.smithi005.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 2.5 MB/s | 258 kB 00:00 2023-07-21T18:49:38.737 INFO:teuthology.orchestra.run.smithi005.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 412 kB/s | 31 kB 00:00 2023-07-21T18:49:39.087 INFO:teuthology.orchestra.run.smithi005.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 2.2 MB/s | 772 kB 00:00 2023-07-21T18:49:39.094 INFO:teuthology.orchestra.run.smithi156.stdout:(10/131): ceph-common-18.0.0-4746.g81cf785a.el8 1.2 MB/s | 22 MB 00:17 2023-07-21T18:49:39.162 INFO:teuthology.orchestra.run.smithi005.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 1.8 MB/s | 137 kB 00:00 2023-07-21T18:49:39.254 INFO:teuthology.orchestra.run.smithi005.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 1.5 MB/s | 139 kB 00:00 2023-07-21T18:49:39.421 INFO:teuthology.orchestra.run.smithi005.stdout:(16/131): libradosstriper1-18.0.0-4746.g81cf785 2.8 MB/s | 473 kB 00:00 2023-07-21T18:49:39.428 INFO:teuthology.orchestra.run.smithi156.stdout:(11/131): cephfs-mirror-18.0.0-4746.g81cf785a.e 774 kB/s | 258 kB 00:00 2023-07-21T18:49:39.528 INFO:teuthology.orchestra.run.smithi156.stdout:(12/131): libcephfs-devel-18.0.0-4746.g81cf785a 311 kB/s | 31 kB 00:00 2023-07-21T18:49:39.994 INFO:teuthology.orchestra.run.smithi101.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.6 MB/s | 3.4 MB 00:02 2023-07-21T18:49:40.103 INFO:teuthology.orchestra.run.smithi156.stdout:(13/131): libcephfs2-18.0.0-4746.g81cf785a.el8. 1.3 MB/s | 772 kB 00:00 2023-07-21T18:49:40.136 INFO:teuthology.orchestra.run.smithi101.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.2 MB/s | 179 kB 00:00 2023-07-21T18:49:40.236 INFO:teuthology.orchestra.run.smithi101.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 231 kB/s | 23 kB 00:00 2023-07-21T18:49:40.270 INFO:teuthology.orchestra.run.smithi156.stdout:(14/131): libcephsqlite-18.0.0-4746.g81cf785a.e 824 kB/s | 137 kB 00:00 2023-07-21T18:49:40.454 INFO:teuthology.orchestra.run.smithi156.stdout:(15/131): librados-devel-18.0.0-4746.g81cf785a. 760 kB/s | 139 kB 00:00 2023-07-21T18:49:40.737 INFO:teuthology.orchestra.run.smithi101.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 258 kB/s | 129 kB 00:00 2023-07-21T18:49:40.787 INFO:teuthology.orchestra.run.smithi156.stdout:(16/131): libradosstriper1-18.0.0-4746.g81cf785 1.4 MB/s | 473 kB 00:00 2023-07-21T18:49:40.997 INFO:teuthology.orchestra.run.smithi005.stdout:(17/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 4.3 MB/s | 6.7 MB 00:01 2023-07-21T18:49:41.063 INFO:teuthology.orchestra.run.smithi005.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 677 kB/s | 45 kB 00:00 2023-07-21T18:49:41.139 INFO:teuthology.orchestra.run.smithi005.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 1.5 MB/s | 119 kB 00:00 2023-07-21T18:49:41.214 INFO:teuthology.orchestra.run.smithi005.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 2.6 MB/s | 197 kB 00:00 2023-07-21T18:49:41.331 INFO:teuthology.orchestra.run.smithi005.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 3.1 MB/s | 370 kB 00:00 2023-07-21T18:49:41.354 INFO:teuthology.orchestra.run.smithi156.stdout:(17/131): ceph-radosgw-18.0.0-4746.g81cf785a.el 1.1 MB/s | 13 MB 00:12 2023-07-21T18:49:41.413 INFO:teuthology.orchestra.run.smithi156.stdout:(18/131): python3-ceph-argparse-18.0.0-4746.g81 774 kB/s | 45 kB 00:00 2023-07-21T18:49:41.448 INFO:teuthology.orchestra.run.smithi005.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 3.1 MB/s | 364 kB 00:00 2023-07-21T18:49:41.532 INFO:teuthology.orchestra.run.smithi005.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 1.2 MB/s | 107 kB 00:00 2023-07-21T18:49:41.555 INFO:teuthology.orchestra.run.smithi156.stdout:(19/131): python3-ceph-common-18.0.0-4746.g81cf 840 kB/s | 119 kB 00:00 2023-07-21T18:49:41.624 INFO:teuthology.orchestra.run.smithi005.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 94 kB 00:00 2023-07-21T18:49:41.797 INFO:teuthology.orchestra.run.smithi156.stdout:(20/131): python3-cephfs-18.0.0-4746.g81cf785a. 815 kB/s | 197 kB 00:00 2023-07-21T18:49:41.945 INFO:teuthology.orchestra.run.smithi101.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.4 MB/s | 1.7 MB 00:01 2023-07-21T18:49:42.056 INFO:teuthology.orchestra.run.smithi156.stdout:(21/131): python3-rados-18.0.0-4746.g81cf785a.e 1.4 MB/s | 370 kB 00:00 2023-07-21T18:49:42.339 INFO:teuthology.orchestra.run.smithi156.stdout:(22/131): python3-rbd-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 364 kB 00:00 2023-07-21T18:49:42.489 INFO:teuthology.orchestra.run.smithi156.stdout:(23/131): python3-rgw-18.0.0-4746.g81cf785a.el8 712 kB/s | 107 kB 00:00 2023-07-21T18:49:42.512 INFO:teuthology.orchestra.run.smithi101.stdout:(30/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.1 MB/s | 6.7 MB 00:05 2023-07-21T18:49:42.574 INFO:teuthology.orchestra.run.smithi005.stdout:(25/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 3.6 MB/s | 3.4 MB 00:00 2023-07-21T18:49:42.590 INFO:teuthology.orchestra.run.smithi156.stdout:(24/131): rbd-fuse-18.0.0-4746.g81cf785a.el8.x8 947 kB/s | 94 kB 00:00 2023-07-21T18:49:42.682 INFO:teuthology.orchestra.run.smithi005.stdout:(26/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 1.6 MB/s | 179 kB 00:00 2023-07-21T18:49:42.766 INFO:teuthology.orchestra.run.smithi005.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 277 kB/s | 23 kB 00:00 2023-07-21T18:49:42.821 INFO:teuthology.orchestra.run.smithi101.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 781 kB/s | 240 kB 00:00 2023-07-21T18:49:42.866 INFO:teuthology.orchestra.run.smithi005.stdout:(28/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 1.3 MB/s | 129 kB 00:00 2023-07-21T18:49:42.913 INFO:teuthology.orchestra.run.smithi101.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 518 kB/s | 47 kB 00:00 2023-07-21T18:49:43.005 INFO:teuthology.orchestra.run.smithi101.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 158 kB/s | 15 kB 00:00 2023-07-21T18:49:43.330 INFO:teuthology.orchestra.run.smithi101.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 795 kB/s | 258 kB 00:00 2023-07-21T18:49:43.383 INFO:teuthology.orchestra.run.smithi005.stdout:(29/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 3.3 MB/s | 1.7 MB 00:00 2023-07-21T18:49:43.589 INFO:teuthology.orchestra.run.smithi101.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 645 kB/s | 166 kB 00:00 2023-07-21T18:49:44.131 INFO:teuthology.orchestra.run.smithi101.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 33 kB/s | 18 kB 00:00 2023-07-21T18:49:45.156 INFO:teuthology.orchestra.run.smithi101.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 299 kB/s | 306 kB 00:01 2023-07-21T18:49:45.290 INFO:teuthology.orchestra.run.smithi101.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2023-07-21T18:49:45.415 INFO:teuthology.orchestra.run.smithi156.stdout:(25/131): librgw2-18.0.0-4746.g81cf785a.el8.x86 1.5 MB/s | 6.7 MB 00:04 2023-07-21T18:49:45.423 INFO:teuthology.orchestra.run.smithi101.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 564 kB/s | 75 kB 00:00 2023-07-21T18:49:45.523 INFO:teuthology.orchestra.run.smithi156.stdout:(26/131): rbd-mirror-18.0.0-4746.g81cf785a.el8. 1.2 MB/s | 3.4 MB 00:02 2023-07-21T18:49:45.615 INFO:teuthology.orchestra.run.smithi156.stdout:(27/131): ceph-grafana-dashboards-18.0.0-4746.g 252 kB/s | 23 kB 00:00 2023-07-21T18:49:45.650 INFO:teuthology.orchestra.run.smithi005.stdout:(30/131): ceph-mgr-diskprediction-local-18.0.0- 3.3 MB/s | 7.4 MB 00:02 2023-07-21T18:49:45.682 INFO:teuthology.orchestra.run.smithi156.stdout:(28/131): rbd-nbd-18.0.0-4746.g81cf785a.el8.x86 673 kB/s | 179 kB 00:00 2023-07-21T18:49:45.699 INFO:teuthology.orchestra.run.smithi101.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-07-21T18:49:45.758 INFO:teuthology.orchestra.run.smithi156.stdout:(29/131): ceph-mgr-cephadm-18.0.0-4746.g81cf785 908 kB/s | 129 kB 00:00 2023-07-21T18:49:45.792 INFO:teuthology.orchestra.run.smithi005.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.7 MB/s | 240 kB 00:00 2023-07-21T18:49:45.807 INFO:teuthology.orchestra.run.smithi101.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-07-21T18:49:45.876 INFO:teuthology.orchestra.run.smithi005.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 569 kB/s | 47 kB 00:00 2023-07-21T18:49:45.949 INFO:teuthology.orchestra.run.smithi101.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-07-21T18:49:45.951 INFO:teuthology.orchestra.run.smithi005.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 195 kB/s | 15 kB 00:00 2023-07-21T18:49:46.084 INFO:teuthology.orchestra.run.smithi005.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.9 MB/s | 258 kB 00:00 2023-07-21T18:49:46.091 INFO:teuthology.orchestra.run.smithi101.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 233 kB/s | 33 kB 00:00 2023-07-21T18:49:46.168 INFO:teuthology.orchestra.run.smithi005.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 2.0 MB/s | 166 kB 00:00 2023-07-21T18:49:46.217 INFO:teuthology.orchestra.run.smithi101.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-07-21T18:49:46.243 INFO:teuthology.orchestra.run.smithi005.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 240 kB/s | 18 kB 00:00 2023-07-21T18:49:46.359 INFO:teuthology.orchestra.run.smithi101.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.2 MB/s | 173 kB 00:00 2023-07-21T18:49:46.526 INFO:teuthology.orchestra.run.smithi101.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.3 MB/s | 384 kB 00:00 2023-07-21T18:49:46.668 INFO:teuthology.orchestra.run.smithi101.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 574 kB/s | 81 kB 00:00 2023-07-21T18:49:46.793 INFO:teuthology.orchestra.run.smithi101.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 89 kB/s | 11 kB 00:00 2023-07-21T18:49:46.874 INFO:teuthology.orchestra.run.smithi156.stdout:(30/131): ceph-mgr-dashboard-18.0.0-4746.g81cf7 1.4 MB/s | 1.7 MB 00:01 2023-07-21T18:49:46.918 INFO:teuthology.orchestra.run.smithi101.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 141 kB/s | 18 kB 00:00 2023-07-21T18:49:47.083 INFO:teuthology.orchestra.run.smithi156.stdout:(31/131): ceph-mgr-modules-core-18.0.0-4746.g81 1.1 MB/s | 240 kB 00:00 2023-07-21T18:49:47.175 INFO:teuthology.orchestra.run.smithi156.stdout:(32/131): ceph-mgr-rook-18.0.0-4746.g81cf785a.e 519 kB/s | 47 kB 00:00 2023-07-21T18:49:47.202 INFO:teuthology.orchestra.run.smithi101.stdout:(50/131): ceph-mgr-diskprediction-local-18.0.0- 1.4 MB/s | 7.4 MB 00:05 2023-07-21T18:49:47.246 INFO:teuthology.orchestra.run.smithi101.stdout:(51/131): python3-kubernetes-11.0.0-6.el8.noarc 3.6 MB/s | 1.2 MB 00:00 2023-07-21T18:49:47.267 INFO:teuthology.orchestra.run.smithi156.stdout:(33/131): ceph-prometheus-alerts-18.0.0-4746.g8 158 kB/s | 15 kB 00:00 2023-07-21T18:49:47.380 INFO:teuthology.orchestra.run.smithi101.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 441 kB/s | 59 kB 00:00 2023-07-21T18:49:47.394 INFO:teuthology.orchestra.run.smithi005.stdout:(37/131): ceph-osd-18.0.0-4746.g81cf785a.el8.x8 1.4 MB/s | 18 MB 00:12 2023-07-21T18:49:47.459 INFO:teuthology.orchestra.run.smithi156.stdout:(34/131): ceph-volume-18.0.0-4746.g81cf785a.el8 1.3 MB/s | 258 kB 00:00 2023-07-21T18:49:47.514 INFO:teuthology.orchestra.run.smithi101.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2023-07-21T18:49:47.610 INFO:teuthology.orchestra.run.smithi156.stdout:(35/131): cephadm-18.0.0-4746.g81cf785a.el8.noa 1.1 MB/s | 166 kB 00:00 2023-07-21T18:49:47.656 INFO:teuthology.orchestra.run.smithi101.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.0 MB/s | 283 kB 00:00 2023-07-21T18:49:47.710 INFO:teuthology.orchestra.run.smithi156.stdout:(36/131): cephfs-top-18.0.0-4746.g81cf785a.el8. 180 kB/s | 18 kB 00:00 2023-07-21T18:49:47.731 INFO:teuthology.orchestra.run.smithi101.stdout:(55/131): python3-logutils-0.3.5-11.el8.noarch. 92 kB/s | 49 kB 00:00 2023-07-21T18:49:47.756 INFO:teuthology.orchestra.run.smithi101.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-07-21T18:49:47.940 INFO:teuthology.orchestra.run.smithi101.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 1.0 MB/s | 196 kB 00:00 2023-07-21T18:49:47.966 INFO:teuthology.orchestra.run.smithi101.stdout:(58/131): python3-repoze-lru-0.7-6.el8.noarch.r 144 kB/s | 34 kB 00:00 2023-07-21T18:49:48.066 INFO:teuthology.orchestra.run.smithi101.stdout:(59/131): python3-rsa-4.8-1.el8.noarch.rpm 491 kB/s | 61 kB 00:00 2023-07-21T18:49:48.141 INFO:teuthology.orchestra.run.smithi101.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 107 kB/s | 19 kB 00:00 2023-07-21T18:49:48.208 INFO:teuthology.orchestra.run.smithi101.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 169 kB/s | 24 kB 00:00 2023-07-21T18:49:48.308 INFO:teuthology.orchestra.run.smithi101.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 175 kB/s | 29 kB 00:00 2023-07-21T18:49:48.334 INFO:teuthology.orchestra.run.smithi101.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 219 kB/s | 27 kB 00:00 2023-07-21T18:49:48.602 INFO:teuthology.orchestra.run.smithi156.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 343 kB/s | 306 kB 00:00 2023-07-21T18:49:48.693 INFO:teuthology.orchestra.run.smithi101.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 701 kB/s | 251 kB 00:00 2023-07-21T18:49:48.810 INFO:teuthology.orchestra.run.smithi101.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 510 kB/s | 255 kB 00:00 2023-07-21T18:49:48.835 INFO:teuthology.orchestra.run.smithi101.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2023-07-21T18:49:48.877 INFO:teuthology.orchestra.run.smithi156.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 214 kB/s | 59 kB 00:00 2023-07-21T18:49:48.969 INFO:teuthology.orchestra.run.smithi101.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 172 kB/s | 23 kB 00:00 2023-07-21T18:49:48.994 INFO:teuthology.orchestra.run.smithi101.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 465 kB/s | 85 kB 00:00 2023-07-21T18:49:49.019 INFO:teuthology.orchestra.run.smithi101.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T18:49:49.052 INFO:teuthology.orchestra.run.smithi156.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 429 kB/s | 75 kB 00:00 2023-07-21T18:49:49.095 INFO:teuthology.orchestra.run.smithi101.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 624 kB/s | 47 kB 00:00 2023-07-21T18:49:49.136 INFO:teuthology.orchestra.run.smithi101.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-21T18:49:49.179 INFO:teuthology.orchestra.run.smithi101.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-21T18:49:49.237 INFO:teuthology.orchestra.run.smithi101.stdout:(73/131): thrift-0.13.0-2.el8.x86_64.rpm 6.5 MB/s | 1.7 MB 00:00 2023-07-21T18:49:49.271 INFO:teuthology.orchestra.run.smithi101.stdout:(74/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-07-21T18:49:49.319 INFO:teuthology.orchestra.run.smithi156.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 1.7 MB/s | 476 kB 00:00 2023-07-21T18:49:49.332 INFO:teuthology.orchestra.run.smithi101.stdout:(75/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 522 kB/s | 48 kB 00:00 2023-07-21T18:49:49.355 INFO:teuthology.orchestra.run.smithi101.stdout:(76/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.1 MB/s | 87 kB 00:00 2023-07-21T18:49:49.380 INFO:teuthology.orchestra.run.smithi101.stdout:(77/131): python3-requests-oauthlib-1.0.0-1.el8 852 kB/s | 43 kB 00:00 2023-07-21T18:49:49.406 INFO:teuthology.orchestra.run.smithi101.stdout:(78/131): python3-pycparser-2.14-14.el8.noarch. 2.1 MB/s | 109 kB 00:00 2023-07-21T18:49:49.428 INFO:teuthology.orchestra.run.smithi156.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 403 kB/s | 44 kB 00:00 2023-07-21T18:49:49.431 INFO:teuthology.orchestra.run.smithi101.stdout:(79/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-07-21T18:49:49.465 INFO:teuthology.orchestra.run.smithi101.stdout:(80/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.4 MB/s | 560 kB 00:00 2023-07-21T18:49:49.490 INFO:teuthology.orchestra.run.smithi101.stdout:(81/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 321 kB/s | 19 kB 00:00 2023-07-21T18:49:49.515 INFO:teuthology.orchestra.run.smithi101.stdout:(82/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.0 MB/s | 155 kB 00:00 2023-07-21T18:49:49.557 INFO:teuthology.orchestra.run.smithi101.stdout:(83/131): python3-cryptography-3.2.1-5.el8.x86_ 8.2 MB/s | 559 kB 00:00 2023-07-21T18:49:49.570 INFO:teuthology.orchestra.run.smithi156.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.3 MB/s | 185 kB 00:00 2023-07-21T18:49:49.583 INFO:teuthology.orchestra.run.smithi101.stdout:(84/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 268 kB/s | 18 kB 00:00 2023-07-21T18:49:49.616 INFO:teuthology.orchestra.run.smithi101.stdout:(85/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T18:49:49.675 INFO:teuthology.orchestra.run.smithi101.stdout:(86/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.5 MB/s | 238 kB 00:00 2023-07-21T18:49:49.703 INFO:teuthology.orchestra.run.smithi156.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 248 kB/s | 33 kB 00:00 2023-07-21T18:49:49.717 INFO:teuthology.orchestra.run.smithi101.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-21T18:49:49.759 INFO:teuthology.orchestra.run.smithi101.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-21T18:49:49.818 INFO:teuthology.orchestra.run.smithi101.stdout:(89/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 12 MB/s | 2.3 MB 00:00 2023-07-21T18:49:49.820 INFO:teuthology.orchestra.run.smithi156.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 140 kB/s | 16 kB 00:00 2023-07-21T18:49:49.843 INFO:teuthology.orchestra.run.smithi101.stdout:(90/131): elfutils-libelf-devel-0.186-1.el8.x86 716 kB/s | 60 kB 00:00 2023-07-21T18:49:49.868 INFO:teuthology.orchestra.run.smithi101.stdout:(91/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 1.9 MB/s | 98 kB 00:00 2023-07-21T18:49:49.894 INFO:teuthology.orchestra.run.smithi101.stdout:(92/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2023-07-21T18:49:49.919 INFO:teuthology.orchestra.run.smithi101.stdout:(93/131): libcom_err-devel-1.45.6-4.el8.x86_64. 771 kB/s | 39 kB 00:00 2023-07-21T18:49:49.961 INFO:teuthology.orchestra.run.smithi101.stdout:(94/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 9.4 MB/s | 644 kB 00:00 2023-07-21T18:49:49.986 INFO:teuthology.orchestra.run.smithi101.stdout:(95/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-21T18:49:49.996 INFO:teuthology.orchestra.run.smithi156.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 987 kB/s | 173 kB 00:00 2023-07-21T18:49:50.012 INFO:teuthology.orchestra.run.smithi101.stdout:(96/131): NetworkManager-initscripts-updown-1.3 2.7 MB/s | 138 kB 00:00 2023-07-21T18:49:50.054 INFO:teuthology.orchestra.run.smithi101.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T18:49:50.087 INFO:teuthology.orchestra.run.smithi101.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-07-21T18:49:50.129 INFO:teuthology.orchestra.run.smithi101.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T18:49:50.171 INFO:teuthology.orchestra.run.smithi156.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 2.1 MB/s | 384 kB 00:00 2023-07-21T18:49:50.180 INFO:teuthology.orchestra.run.smithi101.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-07-21T18:49:50.238 INFO:teuthology.orchestra.run.smithi101.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 669 kB/s | 39 kB 00:00 2023-07-21T18:49:50.289 INFO:teuthology.orchestra.run.smithi101.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-21T18:49:50.296 INFO:teuthology.orchestra.run.smithi156.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 652 kB/s | 81 kB 00:00 2023-07-21T18:49:50.339 INFO:teuthology.orchestra.run.smithi101.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 11 MB/s | 538 kB 00:00 2023-07-21T18:49:50.381 INFO:teuthology.orchestra.run.smithi101.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-21T18:49:50.413 INFO:teuthology.orchestra.run.smithi156.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-07-21T18:49:50.423 INFO:teuthology.orchestra.run.smithi101.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-21T18:49:50.474 INFO:teuthology.orchestra.run.smithi101.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-21T18:49:50.556 INFO:teuthology.orchestra.run.smithi156.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 124 kB/s | 18 kB 00:00 2023-07-21T18:49:50.666 INFO:teuthology.orchestra.run.smithi101.stdout:(107/131): python3-scipy-1.0.0-21.module+el8.5. 20 MB/s | 14 MB 00:00 2023-07-21T18:49:50.700 INFO:teuthology.orchestra.run.smithi101.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 803 kB/s | 27 kB 00:00 2023-07-21T18:49:50.741 INFO:teuthology.orchestra.run.smithi101.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.6 MB/s | 320 kB 00:00 2023-07-21T18:49:50.797 INFO:teuthology.orchestra.run.smithi101.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 788 kB/s | 44 kB 00:00 2023-07-21T18:49:50.884 INFO:teuthology.orchestra.run.smithi101.stdout:(111/131): python3-babel-2.5.1-7.el8.noarch.rpm 12 MB/s | 4.8 MB 00:00 2023-07-21T18:49:50.897 INFO:teuthology.orchestra.run.smithi156.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 3.4 MB/s | 1.2 MB 00:00 2023-07-21T18:49:50.934 INFO:teuthology.orchestra.run.smithi101.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 8.9 MB/s | 457 kB 00:00 2023-07-21T18:49:50.976 INFO:teuthology.orchestra.run.smithi101.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T18:49:51.018 INFO:teuthology.orchestra.run.smithi101.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-21T18:49:51.039 INFO:teuthology.orchestra.run.smithi156.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 344 kB/s | 49 kB 00:00 2023-07-21T18:49:51.093 INFO:teuthology.orchestra.run.smithi101.stdout:(115/131): python3-numpy-1.14.3-10.el8.x86_64.r 13 MB/s | 3.7 MB 00:00 2023-07-21T18:49:51.118 INFO:teuthology.orchestra.run.smithi101.stdout:(116/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 1.2 MB/s | 126 kB 00:00 2023-07-21T18:49:51.144 INFO:teuthology.orchestra.run.smithi101.stdout:(117/131): python3-jsonpointer-1.10-11.el8.noar 397 kB/s | 20 kB 00:00 2023-07-21T18:49:51.169 INFO:teuthology.orchestra.run.smithi101.stdout:(118/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.0 MB/s | 103 kB 00:00 2023-07-21T18:49:51.173 INFO:teuthology.orchestra.run.smithi156.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 441 kB/s | 59 kB 00:00 2023-07-21T18:49:51.211 INFO:teuthology.orchestra.run.smithi101.stdout:(119/131): libtool-2.4.6-25.el8.x86_64.rpm 10 MB/s | 709 kB 00:00 2023-07-21T18:49:51.253 INFO:teuthology.orchestra.run.smithi101.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-07-21T18:49:51.299 INFO:teuthology.orchestra.run.smithi156.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 482 kB/s | 60 kB 00:00 2023-07-21T18:49:51.412 INFO:teuthology.orchestra.run.smithi101.stdout:(121/131): openblas-threads-0.3.15-3.el8.x86_64 20 MB/s | 4.9 MB 00:00 2023-07-21T18:49:51.449 INFO:teuthology.orchestra.run.smithi156.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.8 MB/s | 283 kB 00:00 2023-07-21T18:49:51.454 INFO:teuthology.orchestra.run.smithi101.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-07-21T18:49:51.538 INFO:teuthology.orchestra.run.smithi101.stdout:(123/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-07-21T18:49:51.571 INFO:teuthology.orchestra.run.smithi101.stdout:(124/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-07-21T18:49:51.591 INFO:teuthology.orchestra.run.smithi156.stdout:(55/131): ceph-mgr-diskprediction-local-18.0.0- 1.3 MB/s | 7.4 MB 00:05 2023-07-21T18:49:51.616 INFO:teuthology.orchestra.run.smithi156.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 93 kB/s | 16 kB 00:00 2023-07-21T18:49:51.639 INFO:teuthology.orchestra.run.smithi101.stdout:(125/131): postgresql-contrib-10.21-2.module+el 12 MB/s | 808 kB 00:00 2023-07-21T18:49:51.714 INFO:teuthology.orchestra.run.smithi101.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 20 MB/s | 1.5 MB 00:00 2023-07-21T18:49:51.716 INFO:teuthology.orchestra.run.smithi156.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 269 kB/s | 34 kB 00:00 2023-07-21T18:49:51.806 INFO:teuthology.orchestra.run.smithi101.stdout:(127/131): postgresql-server-10.21-2.module+el8 19 MB/s | 5.1 MB 00:00 2023-07-21T18:49:51.842 INFO:teuthology.orchestra.run.smithi156.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 512 kB/s | 61 kB 00:00 2023-07-21T18:49:51.959 INFO:teuthology.orchestra.run.smithi156.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-07-21T18:49:52.076 INFO:teuthology.orchestra.run.smithi156.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 205 kB/s | 24 kB 00:00 2023-07-21T18:49:52.201 INFO:teuthology.orchestra.run.smithi156.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 234 kB/s | 29 kB 00:00 2023-07-21T18:49:52.260 INFO:teuthology.orchestra.run.smithi156.stdout:(62/131): python3-routes-2.4.1-12.el8.noarch.rp 305 kB/s | 196 kB 00:00 2023-07-21T18:49:52.318 INFO:teuthology.orchestra.run.smithi156.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 236 kB/s | 27 kB 00:00 2023-07-21T18:49:52.452 INFO:teuthology.orchestra.run.smithi156.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.8 MB/s | 251 kB 00:00 2023-07-21T18:49:52.465 INFO:teuthology.orchestra.run.smithi101.stdout:(128/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.8 MB/s | 48 MB 00:26 2023-07-21T18:49:52.477 INFO:teuthology.orchestra.run.smithi156.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.1 MB/s | 255 kB 00:00 2023-07-21T18:49:52.557 INFO:teuthology.orchestra.run.smithi101.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 8.6 MB/s | 798 kB 00:00 2023-07-21T18:49:52.603 INFO:teuthology.orchestra.run.smithi156.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 408 kB/s | 61 kB 00:00 2023-07-21T18:49:52.620 INFO:teuthology.orchestra.run.smithi156.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 602 kB/s | 85 kB 00:00 2023-07-21T18:49:52.745 INFO:teuthology.orchestra.run.smithi156.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 162 kB/s | 23 kB 00:00 2023-07-21T18:49:52.770 INFO:teuthology.orchestra.run.smithi156.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-07-21T18:49:53.029 INFO:teuthology.orchestra.run.smithi156.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 4.2 MB/s | 1.7 MB 00:00 2023-07-21T18:49:53.088 INFO:teuthology.orchestra.run.smithi156.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.9 MB/s | 110 kB 00:00 2023-07-21T18:49:53.230 INFO:teuthology.orchestra.run.smithi156.stdout:(72/131): python3-scipy-1.0.0-21.module+el8.5.0 30 MB/s | 14 MB 00:00 2023-07-21T18:49:53.255 INFO:teuthology.orchestra.run.smithi156.stdout:(73/131): python3-pytz-2017.2-9.el8.noarch.rpm 325 kB/s | 54 kB 00:00 2023-07-21T18:49:53.288 INFO:teuthology.orchestra.run.smithi156.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.4 MB/s | 259 kB 00:00 2023-07-21T18:49:53.314 INFO:teuthology.orchestra.run.smithi156.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 5.4 MB/s | 323 kB 00:00 2023-07-21T18:49:53.339 INFO:teuthology.orchestra.run.smithi156.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 769 kB/s | 39 kB 00:00 2023-07-21T18:49:53.373 INFO:teuthology.orchestra.run.smithi156.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 12 MB/s | 688 kB 00:00 2023-07-21T18:49:53.415 INFO:teuthology.orchestra.run.smithi156.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.0 MB/s | 538 kB 00:00 2023-07-21T18:49:53.440 INFO:teuthology.orchestra.run.smithi156.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-07-21T18:49:53.465 INFO:teuthology.orchestra.run.smithi156.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 3.6 MB/s | 187 kB 00:00 2023-07-21T18:49:53.499 INFO:teuthology.orchestra.run.smithi156.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-07-21T18:49:53.533 INFO:teuthology.orchestra.run.smithi156.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 800 kB/s | 27 kB 00:00 2023-07-21T18:49:53.599 INFO:teuthology.orchestra.run.smithi156.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 4.7 MB/s | 320 kB 00:00 2023-07-21T18:49:53.633 INFO:teuthology.orchestra.run.smithi156.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-07-21T18:49:53.725 INFO:teuthology.orchestra.run.smithi156.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 18 MB/s | 4.8 MB 00:00 2023-07-21T18:49:53.776 INFO:teuthology.orchestra.run.smithi156.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-07-21T18:49:53.818 INFO:teuthology.orchestra.run.smithi156.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-07-21T18:49:53.885 INFO:teuthology.orchestra.run.smithi156.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 15 MB/s | 3.7 MB 00:00 2023-07-21T18:49:53.910 INFO:teuthology.orchestra.run.smithi156.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 695 kB/s | 64 kB 00:00 2023-07-21T18:49:53.935 INFO:teuthology.orchestra.run.smithi156.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.5 MB/s | 126 kB 00:00 2023-07-21T18:49:53.960 INFO:teuthology.orchestra.run.smithi156.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-07-21T18:49:53.986 INFO:teuthology.orchestra.run.smithi156.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-07-21T18:49:54.007 INFO:teuthology.orchestra.run.smithi101.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 1.6 MB/s | 3.7 MB 00:02 2023-07-21T18:49:54.019 INFO:teuthology.orchestra.run.smithi156.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T18:49:54.093 INFO:teuthology.orchestra.run.smithi156.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 2.6 MB/s | 198 kB 00:00 2023-07-21T18:49:54.245 INFO:teuthology.orchestra.run.smithi156.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 19 MB/s | 4.9 MB 00:00 2023-07-21T18:49:54.287 INFO:teuthology.orchestra.run.smithi156.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T18:49:54.371 INFO:teuthology.orchestra.run.smithi156.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2023-07-21T18:49:54.405 INFO:teuthology.orchestra.run.smithi156.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.0 MB/s | 354 kB 00:00 2023-07-21T18:49:54.471 INFO:teuthology.orchestra.run.smithi156.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 12 MB/s | 808 kB 00:00 2023-07-21T18:49:54.555 INFO:teuthology.orchestra.run.smithi156.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-07-21T18:49:54.664 INFO:teuthology.orchestra.run.smithi156.stdout:(101/131): postgresql-server-10.21-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-07-21T18:49:54.689 INFO:teuthology.orchestra.run.smithi156.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 351 kB/s | 47 kB 00:00 2023-07-21T18:49:54.715 INFO:teuthology.orchestra.run.smithi156.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-07-21T18:49:54.740 INFO:teuthology.orchestra.run.smithi156.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 847 kB/s | 43 kB 00:00 2023-07-21T18:49:54.765 INFO:teuthology.orchestra.run.smithi156.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-07-21T18:49:54.790 INFO:teuthology.orchestra.run.smithi156.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 958 kB/s | 48 kB 00:00 2023-07-21T18:49:54.807 INFO:teuthology.orchestra.run.smithi156.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-21T18:49:54.824 INFO:teuthology.orchestra.run.smithi156.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.2 MB/s | 43 kB 00:00 2023-07-21T18:49:54.849 INFO:teuthology.orchestra.run.smithi156.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.6 MB/s | 109 kB 00:00 2023-07-21T18:49:54.875 INFO:teuthology.orchestra.run.smithi156.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.5 MB/s | 229 kB 00:00 2023-07-21T18:49:54.917 INFO:teuthology.orchestra.run.smithi156.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 8.2 MB/s | 560 kB 00:00 2023-07-21T18:49:54.942 INFO:teuthology.orchestra.run.smithi156.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 279 kB/s | 19 kB 00:00 2023-07-21T18:49:54.976 INFO:teuthology.orchestra.run.smithi156.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.6 MB/s | 155 kB 00:00 2023-07-21T18:49:55.009 INFO:teuthology.orchestra.run.smithi156.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.1 MB/s | 559 kB 00:00 2023-07-21T18:49:55.035 INFO:teuthology.orchestra.run.smithi156.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 307 kB/s | 18 kB 00:00 2023-07-21T18:49:55.068 INFO:teuthology.orchestra.run.smithi156.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-07-21T18:49:55.094 INFO:teuthology.orchestra.run.smithi156.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-07-21T18:49:55.144 INFO:teuthology.orchestra.run.smithi156.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-07-21T18:49:55.178 INFO:teuthology.orchestra.run.smithi156.stdout:(119/131): fuse-2.9.7-15.el8.x86_64.rpm 2.4 MB/s | 83 kB 00:00 2023-07-21T18:49:55.245 INFO:teuthology.orchestra.run.smithi156.stdout:(120/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 13 MB/s | 2.3 MB 00:00 2023-07-21T18:49:55.270 INFO:teuthology.orchestra.run.smithi156.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 651 kB/s | 60 kB 00:00 2023-07-21T18:49:55.295 INFO:teuthology.orchestra.run.smithi156.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.9 MB/s | 98 kB 00:00 2023-07-21T18:49:55.321 INFO:teuthology.orchestra.run.smithi156.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-07-21T18:49:55.346 INFO:teuthology.orchestra.run.smithi156.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 768 kB/s | 39 kB 00:00 2023-07-21T18:49:55.373 INFO:teuthology.orchestra.run.smithi101.stdout:(131/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.0 MB/s | 3.6 MB 00:03 2023-07-21T18:49:55.379 INFO:teuthology.orchestra.run.smithi101.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:49:55.380 INFO:teuthology.orchestra.run.smithi101.stdout:Total 5.7 MB/s | 205 MB 00:36 2023-07-21T18:49:55.380 INFO:teuthology.orchestra.run.smithi156.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-07-21T18:49:55.405 INFO:teuthology.orchestra.run.smithi156.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-07-21T18:49:55.422 INFO:teuthology.orchestra.run.smithi156.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-07-21T18:49:57.048 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction check 2023-07-21T18:49:57.181 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction check succeeded. 2023-07-21T18:49:57.181 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction test 2023-07-21T18:49:57.405 INFO:teuthology.orchestra.run.smithi156.stdout:(128/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 1.8 MB/s | 3.6 MB 00:01 2023-07-21T18:49:57.464 INFO:teuthology.orchestra.run.smithi156.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-21T18:49:58.014 INFO:teuthology.orchestra.run.smithi156.stdout:(130/131): librados2-18.0.0-4746.g81cf785a.el8. 1.4 MB/s | 3.7 MB 00:02 2023-07-21T18:49:58.737 INFO:teuthology.orchestra.run.smithi101.stdout:Transaction test succeeded. 2023-07-21T18:49:58.742 INFO:teuthology.orchestra.run.smithi101.stdout:Running transaction 2023-07-21T18:50:03.725 INFO:teuthology.orchestra.run.smithi101.stdout: Preparing : 1/1 2023-07-21T18:50:03.900 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:50:04.127 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:50:04.276 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:50:04.416 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:50:04.651 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:50:04.684 INFO:teuthology.orchestra.run.smithi005.stdout:(38/131): ceph-test-18.0.0-4746.g81cf785a.el8.x 1.7 MB/s | 48 MB 00:27 2023-07-21T18:50:05.118 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:50:05.376 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:50:05.816 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:50:05.992 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:50:06.124 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:50:06.317 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:50:06.347 INFO:teuthology.orchestra.run.smithi156.stdout:(131/131): ceph-test-18.0.0-4746.g81cf785a.el8. 1.7 MB/s | 48 MB 00:28 2023-07-21T18:50:06.352 INFO:teuthology.orchestra.run.smithi156.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:50:06.352 INFO:teuthology.orchestra.run.smithi156.stdout:Total 4.5 MB/s | 205 MB 00:45 2023-07-21T18:50:06.451 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:50:06.623 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:50:06.789 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T18:50:07.049 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:50:07.264 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:50:07.689 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:50:07.921 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:50:08.034 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction check 2023-07-21T18:50:08.100 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:50:08.169 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction check succeeded. 2023-07-21T18:50:08.170 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction test 2023-07-21T18:50:08.208 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:50:08.375 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:50:08.556 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:50:08.799 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T18:50:08.976 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T18:50:09.186 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T18:50:09.358 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:50:09.525 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:50:09.682 INFO:teuthology.orchestra.run.smithi156.stdout:Transaction test succeeded. 2023-07-21T18:50:09.683 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:50:09.687 INFO:teuthology.orchestra.run.smithi156.stdout:Running transaction 2023-07-21T18:50:10.378 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:50:10.550 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:50:11.236 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:50:11.399 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:50:12.224 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:50:12.499 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T18:50:12.701 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T18:50:12.953 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:50:13.140 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:50:13.305 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:50:13.555 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:50:13.758 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:50:14.687 INFO:teuthology.orchestra.run.smithi156.stdout: Preparing : 1/1 2023-07-21T18:50:14.902 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:50:15.114 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:50:15.380 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-07-21T18:50:15.570 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T18:50:15.700 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-07-21T18:50:15.818 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:50:15.962 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T18:50:16.435 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-07-21T18:50:16.678 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:50:16.999 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T18:50:17.165 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:50:17.354 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T18:50:17.394 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:50:17.547 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:50:17.639 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T18:50:17.803 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:50:17.925 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:50:17.934 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T18:50:18.149 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:50:18.169 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:50:18.370 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:50:18.427 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:50:18.590 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:50:18.700 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:50:18.773 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:50:18.925 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T18:50:19.134 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T18:50:19.178 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:50:19.332 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:50:19.427 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:50:19.457 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:50:19.619 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:50:19.726 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T18:50:19.739 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:50:19.930 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T18:50:19.985 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:50:20.081 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:50:20.098 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:50:20.221 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:50:20.243 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T18:50:20.407 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-07-21T18:50:20.443 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T18:50:20.594 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T18:50:20.628 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T18:50:20.824 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T18:50:21.028 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T18:50:21.193 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T18:50:21.324 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-07-21T18:50:21.397 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T18:50:21.560 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T18:50:21.606 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T18:50:21.840 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T18:50:22.239 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-07-21T18:50:22.297 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T18:50:22.502 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T18:50:22.664 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T18:50:22.872 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T18:50:23.142 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T18:50:23.334 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-07-21T18:50:23.383 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T18:50:23.684 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-07-21T18:50:23.723 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T18:50:23.916 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T18:50:23.953 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-07-21T18:50:24.120 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T18:50:24.234 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-07-21T18:50:24.367 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T18:50:24.453 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-07-21T18:50:24.555 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T18:50:24.746 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-07-21T18:50:25.030 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:50:25.054 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T18:50:25.259 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:50:25.395 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T18:50:25.423 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:50:25.693 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T18:50:25.953 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T18:50:26.209 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T18:50:26.374 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:50:26.543 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:50:26.546 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:50:26.855 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:50:26.948 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-07-21T18:50:27.090 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:50:27.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-07-21T18:50:27.286 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:50:27.435 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:50:27.598 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-07-21T18:50:27.644 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:50:27.834 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T18:50:27.871 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:50:28.069 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:50:28.250 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:50:28.437 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:50:28.659 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:50:28.944 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:50:29.120 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:50:29.373 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:50:29.631 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:50:29.807 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:50:29.895 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-07-21T18:50:30.028 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:50:30.327 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-07-21T18:50:30.571 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:50:30.840 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:50:31.093 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:50:31.338 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:50:31.412 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:50:31.517 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-07-21T18:50:31.700 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:50:31.712 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-07-21T18:50:31.961 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-07-21T18:50:31.994 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:50:32.139 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-07-21T18:50:32.344 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:50:32.423 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-07-21T18:50:32.632 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:50:32.675 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-07-21T18:50:32.833 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:50:33.177 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-07-21T18:50:33.524 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-07-21T18:50:33.688 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:50:33.756 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-07-21T18:50:33.855 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:50:33.930 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:50:34.063 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-07-21T18:50:34.139 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:50:34.352 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-07-21T18:50:34.728 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-07-21T18:50:35.001 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-07-21T18:50:35.208 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-07-21T18:50:35.460 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-07-21T18:50:35.690 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-07-21T18:50:35.888 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-07-21T18:50:36.294 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-07-21T18:50:36.664 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-07-21T18:50:36.973 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-07-21T18:50:37.258 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-07-21T18:50:37.475 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-07-21T18:50:37.712 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-07-21T18:50:37.941 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T18:50:38.039 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-07-21T18:50:38.353 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-07-21T18:50:38.592 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-07-21T18:50:38.782 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T18:50:38.884 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-07-21T18:50:39.046 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:39.143 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-07-21T18:50:39.313 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:39.377 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-07-21T18:50:39.584 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:50:39.816 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:50:39.985 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:40.111 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:50:40.247 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:50:40.381 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:50:40.382 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:50:40.563 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:50:40.762 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:50:40.963 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:50:41.224 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:50:41.387 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:50:41.439 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:50:41.627 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:50:41.814 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:50:42.068 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:50:42.340 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:50:42.523 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:50:42.790 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:50:43.008 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:50:43.175 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:50:43.441 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:50:44.782 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:50:45.052 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:50:45.304 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:50:45.680 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:50:46.008 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:50:46.213 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:50:47.088 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:50:47.256 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:50:47.352 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:50:47.556 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:50:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:53.048 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:53.723 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:50:53.991 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:50:54.128 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:50:54.330 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:51:01.555 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:51:01.555 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /sys 2023-07-21T18:51:01.555 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /proc 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /run 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /mnt 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /var/tmp 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /home 2023-07-21T18:51:01.556 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /tmp 2023-07-21T18:51:01.557 INFO:teuthology.orchestra.run.smithi101.stdout:skipping the directory /dev 2023-07-21T18:51:01.557 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:51:01.927 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:51:03.666 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:51:03.877 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:51:04.146 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:51:04.359 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:51:04.894 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:51:05.126 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:51:05.168 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:51:07.814 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:51:07.988 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:51:08.123 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:51:08.328 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:51:08.542 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:51:08.807 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:51:09.034 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:51:10.138 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:51:10.316 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:51:10.443 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:51:10.867 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:51:11.082 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:51:11.147 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:51:11.342 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:51:11.369 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:51:12.777 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:51:13.032 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:51:13.102 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:51:13.275 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:51:13.345 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:51:13.516 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:51:13.923 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:51:14.150 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:51:14.886 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:51:14.887 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /sys 2023-07-21T18:51:14.887 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /proc 2023-07-21T18:51:14.887 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-07-21T18:51:14.887 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /run 2023-07-21T18:51:14.887 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /mnt 2023-07-21T18:51:14.888 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /var/tmp 2023-07-21T18:51:14.888 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /home 2023-07-21T18:51:14.888 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /tmp 2023-07-21T18:51:14.888 INFO:teuthology.orchestra.run.smithi156.stdout:skipping the directory /dev 2023-07-21T18:51:14.888 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:51:15.277 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:51:17.018 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:51:17.203 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:51:17.475 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:51:17.643 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:51:18.179 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:51:18.418 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:51:18.460 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:51:19.112 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:51:20.372 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:51:20.744 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T18:51:20.937 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-21T18:51:21.230 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T18:51:21.251 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:51:21.407 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:51:21.423 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:51:21.575 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:51:21.657 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:51:21.806 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:51:21.895 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T18:51:22.042 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:51:22.070 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:51:22.249 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:51:22.307 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:51:22.527 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:51:23.645 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:51:23.791 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:51:23.920 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:51:24.319 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:51:24.581 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:51:24.648 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:51:24.833 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:51:24.862 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:51:25.516 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:25.823 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:25.861 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:26.065 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:51:26.278 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:51:26.282 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:51:26.503 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:51:26.576 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:51:26.606 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:51:26.775 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:51:26.847 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:51:26.995 INFO:teuthology.orchestra.run.smithi101.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:51:27.024 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:51:27.189 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:51:27.301 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:51:27.436 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:51:27.548 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:51:27.576 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:51:27.600 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:51:27.772 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:51:27.795 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:51:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T18:51:27.993 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:28.229 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:51:28.505 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:51:28.747 INFO:teuthology.orchestra.run.smithi101.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:29.010 INFO:teuthology.orchestra.run.smithi101.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:29.010 INFO:teuthology.orchestra.run.smithi101.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-21T18:51:29.010 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:51:29.643 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:51:29.644 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:51:29.645 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:51:29.646 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:51:29.647 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:51:29.654 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:51:29.655 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:51:29.656 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:51:29.657 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:51:29.658 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:51:29.659 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T18:51:29.660 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T18:51:29.661 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T18:51:29.662 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T18:51:29.663 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T18:51:29.664 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T18:51:29.665 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T18:51:29.666 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T18:51:29.667 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:29.668 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:51:32.352 INFO:teuthology.orchestra.run.smithi101.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:32.352 INFO:teuthology.orchestra.run.smithi101.stdout:Last metadata expiration check: 0:02:26 ago on Fri 21 Jul 2023 06:49:06 PM UTC. 2023-07-21T18:51:32.606 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:51:32.949 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:51:33.098 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T18:51:33.352 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-07-21T18:51:34.575 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-07-21T18:51:34.886 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-07-21T18:51:35.096 INFO:teuthology.orchestra.run.smithi101.stdout:Installed products updated. 2023-07-21T18:51:35.096 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 119/134 2023-07-21T18:51:35.312 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T18:51:35.339 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:51:35.339 INFO:teuthology.orchestra.run.smithi101.stdout:Upgraded: 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout:Installed: 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:51:35.340 INFO:teuthology.orchestra.run.smithi101.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.341 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.342 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.343 INFO:teuthology.orchestra.run.smithi101.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:51:35.344 INFO:teuthology.orchestra.run.smithi101.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:51:35.345 INFO:teuthology.orchestra.run.smithi101.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.346 INFO:teuthology.orchestra.run.smithi101.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:51:35.347 INFO:teuthology.orchestra.run.smithi101.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:51:35.348 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:51:35.349 INFO:teuthology.orchestra.run.smithi101.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:51:35.350 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:51:35.351 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:51:35.352 INFO:teuthology.orchestra.run.smithi101.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:51:35.353 INFO:teuthology.orchestra.run.smithi101.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.354 INFO:teuthology.orchestra.run.smithi101.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:51:35.355 INFO:teuthology.orchestra.run.smithi101.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:51:35.356 INFO:teuthology.orchestra.run.smithi101.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:51:35.357 INFO:teuthology.orchestra.run.smithi101.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:51:35.358 INFO:teuthology.orchestra.run.smithi101.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:51:35.358 INFO:teuthology.orchestra.run.smithi101.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:51:35.358 INFO:teuthology.orchestra.run.smithi101.stdout: 2023-07-21T18:51:35.359 INFO:teuthology.orchestra.run.smithi101.stdout:Complete! 2023-07-21T18:51:35.528 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:51:35.683 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:51:35.882 DEBUG:teuthology.parallel:result is None 2023-07-21T18:51:39.043 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:39.331 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:39.370 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:51:39.606 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:51:39.824 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:51:40.204 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:51:40.483 INFO:teuthology.orchestra.run.smithi156.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:51:40.697 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:51:40.720 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:51:40.925 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-07-21T18:51:41.138 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T18:51:41.251 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-07-21T18:51:41.463 INFO:teuthology.orchestra.run.smithi156.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T18:51:41.493 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-07-21T18:51:41.493 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:41.729 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:51:41.971 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:51:42.205 INFO:teuthology.orchestra.run.smithi156.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:42.479 INFO:teuthology.orchestra.run.smithi156.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:42.480 INFO:teuthology.orchestra.run.smithi156.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-21T18:51:42.480 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:51:43.117 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:51:43.118 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:51:43.119 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:51:43.120 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:51:43.121 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:51:43.122 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:51:43.122 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:51:43.129 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:51:43.130 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:51:43.131 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:51:43.132 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:51:43.133 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:51:43.133 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:51:43.133 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:51:43.134 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:51:43.134 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:51:43.134 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:51:43.134 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:51:43.135 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:51:43.135 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:51:43.135 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:51:43.136 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:51:43.136 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:51:43.136 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-07-21T18:51:43.137 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-07-21T18:51:43.138 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-07-21T18:51:43.139 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-07-21T18:51:43.140 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-07-21T18:51:43.141 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-07-21T18:51:43.142 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-07-21T18:51:43.143 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-07-21T18:51:43.144 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-07-21T18:51:43.145 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:51:43.145 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:51:43.145 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:51:43.145 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:51:43.145 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:51:45.837 INFO:teuthology.orchestra.run.smithi156.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:51:45.837 INFO:teuthology.orchestra.run.smithi156.stdout:Last metadata expiration check: 0:02:37 ago on Fri 21 Jul 2023 06:49:08 PM UTC. 2023-07-21T18:51:48.493 INFO:teuthology.orchestra.run.smithi156.stdout:Installed products updated. 2023-07-21T18:51:48.737 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:51:48.738 INFO:teuthology.orchestra.run.smithi156.stdout:Upgraded: 2023-07-21T18:51:48.738 INFO:teuthology.orchestra.run.smithi156.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:51:48.738 INFO:teuthology.orchestra.run.smithi156.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.738 INFO:teuthology.orchestra.run.smithi156.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout:Installed: 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.739 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.740 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.741 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.742 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:51:48.743 INFO:teuthology.orchestra.run.smithi156.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:51:48.744 INFO:teuthology.orchestra.run.smithi156.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:51:48.745 INFO:teuthology.orchestra.run.smithi156.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:51:48.746 INFO:teuthology.orchestra.run.smithi156.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:51:48.747 INFO:teuthology.orchestra.run.smithi156.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:51:48.748 INFO:teuthology.orchestra.run.smithi156.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:51:48.748 INFO:teuthology.orchestra.run.smithi156.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:51:48.748 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:51:48.748 INFO:teuthology.orchestra.run.smithi156.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:51:48.748 INFO:teuthology.orchestra.run.smithi156.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:51:48.749 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:51:48.749 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:51:48.749 INFO:teuthology.orchestra.run.smithi156.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:51:48.750 INFO:teuthology.orchestra.run.smithi156.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.751 INFO:teuthology.orchestra.run.smithi156.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:51:48.752 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:51:48.752 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:51:48.752 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:51:48.752 INFO:teuthology.orchestra.run.smithi156.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:51:48.753 INFO:teuthology.orchestra.run.smithi156.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:51:48.754 INFO:teuthology.orchestra.run.smithi156.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:51:48.754 INFO:teuthology.orchestra.run.smithi156.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:51:48.754 INFO:teuthology.orchestra.run.smithi156.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:51:48.754 INFO:teuthology.orchestra.run.smithi156.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:51:48.755 INFO:teuthology.orchestra.run.smithi156.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:51:48.755 INFO:teuthology.orchestra.run.smithi156.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:51:48.755 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:51:48.755 INFO:teuthology.orchestra.run.smithi156.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:51:48.756 INFO:teuthology.orchestra.run.smithi156.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:51:48.756 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:51:48.756 INFO:teuthology.orchestra.run.smithi156.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:51:48.756 INFO:teuthology.orchestra.run.smithi156.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:51:48.756 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:51:48.757 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:51:48.757 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:51:48.757 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:51:48.757 INFO:teuthology.orchestra.run.smithi156.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:51:48.757 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.758 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.758 INFO:teuthology.orchestra.run.smithi156.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:51:48.758 INFO:teuthology.orchestra.run.smithi156.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:51:48.758 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.758 INFO:teuthology.orchestra.run.smithi156.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:51:48.759 INFO:teuthology.orchestra.run.smithi156.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:51:48.759 INFO:teuthology.orchestra.run.smithi156.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:51:48.759 INFO:teuthology.orchestra.run.smithi156.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:51:48.759 INFO:teuthology.orchestra.run.smithi156.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:51:48.759 INFO:teuthology.orchestra.run.smithi156.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:51:48.760 INFO:teuthology.orchestra.run.smithi156.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:51:48.760 INFO:teuthology.orchestra.run.smithi156.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:51:48.760 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:51:48.760 INFO:teuthology.orchestra.run.smithi156.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:51:48.760 INFO:teuthology.orchestra.run.smithi156.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:51:48.761 INFO:teuthology.orchestra.run.smithi156.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:51:48.762 INFO:teuthology.orchestra.run.smithi156.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:51:48.762 INFO:teuthology.orchestra.run.smithi156.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:51:48.762 INFO:teuthology.orchestra.run.smithi156.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:51:48.762 INFO:teuthology.orchestra.run.smithi156.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:51:48.763 INFO:teuthology.orchestra.run.smithi156.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:51:48.763 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:51:48.763 INFO:teuthology.orchestra.run.smithi156.stdout:Complete! 2023-07-21T18:51:49.831 DEBUG:teuthology.parallel:result is None 2023-07-21T18:52:01.972 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] gperftools-libs-2.7-9.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/g/gperftools-libs-2.7-9.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T18:52:01.973 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] liboath-2.6.2-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/liboath-2.6.2-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T18:52:02.651 INFO:teuthology.orchestra.run.smithi005.stdout:(39/131): liboath-2.6.2-3.el8.x86_64.rpm 445 B/s | 59 kB 02:15 2023-07-21T18:52:03.085 INFO:teuthology.orchestra.run.smithi005.stdout:(40/131): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 kB/s | 306 kB 02:16 2023-07-21T18:52:18.298 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] libunwind-1.3.1-3.el8.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/l/libunwind-1.3.1-3.el8.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T18:52:19.008 INFO:teuthology.orchestra.run.smithi005.stdout:(41/131): libunwind-1.3.1-3.el8.x86_64.rpm 572 B/s | 75 kB 02:14 2023-07-21T18:52:19.408 INFO:teuthology.orchestra.run.smithi005.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 463 kB/s | 185 kB 00:00 2023-07-21T18:52:19.576 INFO:teuthology.orchestra.run.smithi005.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 198 kB/s | 33 kB 00:00 2023-07-21T18:52:19.700 INFO:teuthology.orchestra.run.smithi005.stdout:(44/131): python3-certifi-2018.10.15-7.el8.noar 130 kB/s | 16 kB 00:00 2023-07-21T18:52:20.500 INFO:teuthology.orchestra.run.smithi005.stdout:(45/131): python3-cheroot-8.5.2-1.el8.noarch.rp 216 kB/s | 173 kB 00:00 2023-07-21T18:52:21.193 INFO:teuthology.orchestra.run.smithi005.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 556 kB/s | 384 kB 00:00 2023-07-21T18:52:21.418 INFO:teuthology.orchestra.run.smithi005.stdout:(47/131): python3-google-auth-1.1.1-10.el8.noar 361 kB/s | 81 kB 00:00 2023-07-21T18:52:21.551 INFO:teuthology.orchestra.run.smithi005.stdout:(48/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-07-21T18:52:21.701 INFO:teuthology.orchestra.run.smithi005.stdout:(49/131): python3-jaraco-functools-2.0-4.el8.no 117 kB/s | 18 kB 00:00 2023-07-21T18:52:23.801 INFO:teuthology.orchestra.run.smithi005.stdout:(50/131): python3-kubernetes-11.0.0-6.el8.noarc 568 kB/s | 1.2 MB 00:02 2023-07-21T18:52:23.935 INFO:teuthology.orchestra.run.smithi005.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 365 kB/s | 49 kB 00:00 2023-07-21T18:52:24.077 INFO:teuthology.orchestra.run.smithi005.stdout:(52/131): python3-more-itertools-7.2.0-3.el8.no 415 kB/s | 59 kB 00:00 2023-07-21T18:52:24.210 INFO:teuthology.orchestra.run.smithi005.stdout:(53/131): python3-natsort-7.1.1-2.el8.noarch.rp 452 kB/s | 60 kB 00:00 2023-07-21T18:52:24.861 INFO:teuthology.orchestra.run.smithi005.stdout:(54/131): python3-pecan-1.3.2-9.el8.noarch.rpm 435 kB/s | 283 kB 00:00 2023-07-21T18:52:25.011 INFO:teuthology.orchestra.run.smithi005.stdout:(55/131): python3-portend-2.6-1.el8.noarch.rpm 104 kB/s | 16 kB 00:00 2023-07-21T18:52:25.161 INFO:teuthology.orchestra.run.smithi005.stdout:(56/131): python3-repoze-lru-0.7-6.el8.noarch.r 224 kB/s | 34 kB 00:00 2023-07-21T18:52:25.661 INFO:teuthology.orchestra.run.smithi005.stdout:(57/131): python3-routes-2.4.1-12.el8.noarch.rp 393 kB/s | 196 kB 00:00 2023-07-21T18:52:25.803 INFO:teuthology.orchestra.run.smithi005.stdout:(58/131): python3-rsa-4.8-1.el8.noarch.rpm 434 kB/s | 61 kB 00:00 2023-07-21T18:52:25.929 INFO:teuthology.orchestra.run.smithi005.stdout:(59/131): python3-simplegeneric-0.8.1-17.el8.no 150 kB/s | 19 kB 00:00 2023-07-21T18:52:26.062 INFO:teuthology.orchestra.run.smithi005.stdout:(60/131): python3-singledispatch-3.4.0.3-18.el8 180 kB/s | 24 kB 00:00 2023-07-21T18:52:26.196 INFO:teuthology.orchestra.run.smithi005.stdout:(61/131): python3-tempora-1.14.1-5.el8.noarch.r 219 kB/s | 29 kB 00:00 2023-07-21T18:52:26.330 INFO:teuthology.orchestra.run.smithi005.stdout:(62/131): python3-trustme-0.6.0-4.el8.noarch.rp 206 kB/s | 27 kB 00:00 2023-07-21T18:52:26.838 INFO:teuthology.orchestra.run.smithi005.stdout:(63/131): python3-waitress-1.4.3-1.el8.noarch.r 502 kB/s | 255 kB 00:00 2023-07-21T18:52:27.321 INFO:teuthology.orchestra.run.smithi005.stdout:(64/131): python3-webob-1.8.5-1.el8.1.noarch.rp 520 kB/s | 251 kB 00:00 2023-07-21T18:52:27.463 INFO:teuthology.orchestra.run.smithi005.stdout:(65/131): python3-websocket-client-0.56.0-5.el8 432 kB/s | 61 kB 00:00 2023-07-21T18:52:27.724 INFO:teuthology.orchestra.run.smithi005.stdout:(66/131): python3-webtest-2.0.33-1.el8.noarch.r 328 kB/s | 85 kB 00:00 2023-07-21T18:52:27.847 INFO:teuthology.orchestra.run.smithi005.stdout:(67/131): python3-zc-lockfile-2.0-2.el8.noarch. 187 kB/s | 23 kB 00:00 2023-07-21T18:52:31.647 INFO:teuthology.orchestra.run.smithi005.stdout:(68/131): thrift-0.13.0-2.el8.x86_64.rpm 467 kB/s | 1.7 MB 00:03 2023-07-21T18:52:31.681 INFO:teuthology.orchestra.run.smithi005.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.0 MB/s | 70 kB 00:00 2023-07-21T18:52:31.773 INFO:teuthology.orchestra.run.smithi005.stdout:(70/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 512 kB/s | 47 kB 00:00 2023-07-21T18:52:31.815 INFO:teuthology.orchestra.run.smithi005.stdout:(71/131): libselinux-devel-2.9-5.el8.x86_64.rpm 4.7 MB/s | 200 kB 00:00 2023-07-21T18:52:31.857 INFO:teuthology.orchestra.run.smithi005.stdout:(72/131): python3-jwt-1.6.1-2.el8.noarch.rpm 1.0 MB/s | 43 kB 00:00 2023-07-21T18:52:31.899 INFO:teuthology.orchestra.run.smithi005.stdout:(73/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 5.2 MB/s | 220 kB 00:00 2023-07-21T18:52:31.940 INFO:teuthology.orchestra.run.smithi005.stdout:(74/131): keyutils-libs-devel-1.5.10-9.el8.x86_ 1.1 MB/s | 48 kB 00:00 2023-07-21T18:52:31.982 INFO:teuthology.orchestra.run.smithi005.stdout:(75/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-07-21T18:52:32.024 INFO:teuthology.orchestra.run.smithi005.stdout:(76/131): python3-requests-oauthlib-1.0.0-1.el8 1.0 MB/s | 43 kB 00:00 2023-07-21T18:52:32.067 INFO:teuthology.orchestra.run.smithi005.stdout:(77/131): python3-pycparser-2.14-14.el8.noarch. 2.6 MB/s | 109 kB 00:00 2023-07-21T18:52:32.108 INFO:teuthology.orchestra.run.smithi005.stdout:(78/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 5.4 MB/s | 229 kB 00:00 2023-07-21T18:52:32.159 INFO:teuthology.orchestra.run.smithi005.stdout:(79/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 11 MB/s | 560 kB 00:00 2023-07-21T18:52:32.201 INFO:teuthology.orchestra.run.smithi005.stdout:(80/131): libaio-devel-0.3.112-1.el8.x86_64.rpm 451 kB/s | 19 kB 00:00 2023-07-21T18:52:32.243 INFO:teuthology.orchestra.run.smithi005.stdout:(81/131): python3-oauthlib-2.1.0-1.el8.noarch.r 3.6 MB/s | 155 kB 00:00 2023-07-21T18:52:32.331 INFO:teuthology.orchestra.run.smithi005.stdout:(82/131): python3-cryptography-3.2.1-5.el8.x86_ 6.2 MB/s | 559 kB 00:00 2023-07-21T18:52:32.376 INFO:teuthology.orchestra.run.smithi005.stdout:(83/131): libverto-devel-0.3.0-5.el8.x86_64.rpm 393 kB/s | 18 kB 00:00 2023-07-21T18:52:32.426 INFO:teuthology.orchestra.run.smithi005.stdout:(84/131): pcre2-devel-10.32-2.el8.x86_64.rpm 12 MB/s | 605 kB 00:00 2023-07-21T18:52:32.468 INFO:teuthology.orchestra.run.smithi005.stdout:(85/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 5.6 MB/s | 238 kB 00:00 2023-07-21T18:52:32.569 INFO:teuthology.orchestra.run.smithi005.stdout:(86/131): openssl-devel-1.1.1k-7.el8_6.x86_64.r 23 MB/s | 2.3 MB 00:00 2023-07-21T18:52:32.611 INFO:teuthology.orchestra.run.smithi005.stdout:(87/131): xfsprogs-devel-5.0.0-10.el8.x86_64.rp 1.7 MB/s | 72 kB 00:00 2023-07-21T18:52:32.652 INFO:teuthology.orchestra.run.smithi005.stdout:(88/131): fuse-2.9.7-15.el8.x86_64.rpm 1.9 MB/s | 83 kB 00:00 2023-07-21T18:52:32.695 INFO:teuthology.orchestra.run.smithi005.stdout:(89/131): elfutils-libelf-devel-0.186-1.el8.x86 1.4 MB/s | 60 kB 00:00 2023-07-21T18:52:32.736 INFO:teuthology.orchestra.run.smithi005.stdout:(90/131): libuuid-devel-2.32.1-35.el8.x86_64.rp 2.3 MB/s | 98 kB 00:00 2023-07-21T18:52:32.771 INFO:teuthology.orchestra.run.smithi005.stdout:(91/131): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 1.7 MB/s | 58 kB 00:00 2023-07-21T18:52:32.804 INFO:teuthology.orchestra.run.smithi005.stdout:(92/131): libcom_err-devel-1.45.6-4.el8.x86_64. 1.1 MB/s | 39 kB 00:00 2023-07-21T18:52:32.854 INFO:teuthology.orchestra.run.smithi005.stdout:(93/131): libgfortran-8.5.0-10.1.el8_6.x86_64.r 13 MB/s | 644 kB 00:00 2023-07-21T18:52:32.921 INFO:teuthology.orchestra.run.smithi005.stdout:(94/131): libquadmath-8.5.0-10.1.el8_6.x86_64.r 2.5 MB/s | 171 kB 00:00 2023-07-21T18:52:32.954 INFO:teuthology.orchestra.run.smithi005.stdout:(95/131): NetworkManager-initscripts-updown-1.3 4.0 MB/s | 138 kB 00:00 2023-07-21T18:52:33.296 INFO:teuthology.orchestra.run.smithi005.stdout:(96/131): python3-scipy-1.0.0-21.module+el8.5.0 40 MB/s | 14 MB 00:00 2023-07-21T18:52:33.338 INFO:teuthology.orchestra.run.smithi005.stdout:(97/131): python3-pyasn1-modules-0.3.7-6.el8.no 2.6 MB/s | 110 kB 00:00 2023-07-21T18:52:33.380 INFO:teuthology.orchestra.run.smithi005.stdout:(98/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-07-21T18:52:33.422 INFO:teuthology.orchestra.run.smithi005.stdout:(99/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.1 MB/s | 259 kB 00:00 2023-07-21T18:52:33.506 INFO:teuthology.orchestra.run.smithi005.stdout:(100/131): socat-1.7.4.1-1.el8.x86_64.rpm 3.8 MB/s | 323 kB 00:00 2023-07-21T18:52:33.539 INFO:teuthology.orchestra.run.smithi005.stdout:(101/131): python3-markupsafe-0.23-19.el8.x86_6 1.1 MB/s | 39 kB 00:00 2023-07-21T18:52:33.590 INFO:teuthology.orchestra.run.smithi005.stdout:(102/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-07-21T18:52:33.640 INFO:teuthology.orchestra.run.smithi005.stdout:(103/131): python3-jinja2-2.10.1-3.el8.noarch.r 10 MB/s | 538 kB 00:00 2023-07-21T18:52:33.682 INFO:teuthology.orchestra.run.smithi005.stdout:(104/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-07-21T18:52:33.724 INFO:teuthology.orchestra.run.smithi005.stdout:(105/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-07-21T18:52:33.774 INFO:teuthology.orchestra.run.smithi005.stdout:(106/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-07-21T18:52:33.916 INFO:teuthology.orchestra.run.smithi005.stdout:(107/131): python3-babel-2.5.1-7.el8.noarch.rpm 34 MB/s | 4.8 MB 00:00 2023-07-21T18:52:33.958 INFO:teuthology.orchestra.run.smithi005.stdout:(108/131): python3-jsonpatch-1.21-2.el8.noarch. 641 kB/s | 27 kB 00:00 2023-07-21T18:52:34.000 INFO:teuthology.orchestra.run.smithi005.stdout:(109/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-07-21T18:52:34.067 INFO:teuthology.orchestra.run.smithi005.stdout:(110/131): python3-prettytable-0.7.2-14.el8.noa 658 kB/s | 44 kB 00:00 2023-07-21T18:52:34.201 INFO:teuthology.orchestra.run.smithi005.stdout:(111/131): python3-numpy-1.14.3-10.el8.x86_64.r 28 MB/s | 3.7 MB 00:00 2023-07-21T18:52:34.259 INFO:teuthology.orchestra.run.smithi005.stdout:(112/131): python3-werkzeug-0.12.2-4.el8.noarch 7.7 MB/s | 457 kB 00:00 2023-07-21T18:52:34.301 INFO:teuthology.orchestra.run.smithi005.stdout:(113/131): python3-numpy-f2py-1.14.3-10.el8.x86 5.3 MB/s | 225 kB 00:00 2023-07-21T18:52:34.343 INFO:teuthology.orchestra.run.smithi005.stdout:(114/131): uuid-1.6.2-43.el8.x86_64.rpm 1.5 MB/s | 64 kB 00:00 2023-07-21T18:52:34.386 INFO:teuthology.orchestra.run.smithi005.stdout:(115/131): python3-pyasn1-0.3.7-6.el8.noarch.rp 3.0 MB/s | 126 kB 00:00 2023-07-21T18:52:34.418 INFO:teuthology.orchestra.run.smithi005.stdout:(116/131): python3-jsonpointer-1.10-11.el8.noar 604 kB/s | 20 kB 00:00 2023-07-21T18:52:34.460 INFO:teuthology.orchestra.run.smithi005.stdout:(117/131): python3-pyOpenSSL-19.0.0-1.el8.noarc 2.4 MB/s | 103 kB 00:00 2023-07-21T18:52:34.519 INFO:teuthology.orchestra.run.smithi005.stdout:(118/131): libtool-2.4.6-25.el8.x86_64.rpm 12 MB/s | 709 kB 00:00 2023-07-21T18:52:34.678 INFO:teuthology.orchestra.run.smithi005.stdout:(119/131): openblas-threads-0.3.15-3.el8.x86_64 31 MB/s | 4.9 MB 00:00 2023-07-21T18:52:34.719 INFO:teuthology.orchestra.run.smithi005.stdout:(120/131): libpq-13.5-1.el8.x86_64.rpm 4.7 MB/s | 198 kB 00:00 2023-07-21T18:52:34.853 INFO:teuthology.orchestra.run.smithi005.stdout:(121/131): openblas-0.3.15-3.el8.x86_64.rpm 35 MB/s | 4.7 MB 00:00 2023-07-21T18:52:34.895 INFO:teuthology.orchestra.run.smithi005.stdout:(122/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-07-21T18:52:34.937 INFO:teuthology.orchestra.run.smithi005.stdout:(123/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-07-21T18:52:35.087 INFO:teuthology.orchestra.run.smithi005.stdout:(124/131): postgresql-server-10.21-2.module+el8 34 MB/s | 5.1 MB 00:00 2023-07-21T18:52:35.138 INFO:teuthology.orchestra.run.smithi005.stdout:(125/131): postgresql-contrib-10.21-2.module+el 16 MB/s | 808 kB 00:00 2023-07-21T18:52:35.238 INFO:teuthology.orchestra.run.smithi005.stdout:(126/131): postgresql-10.21-2.module+el8.6.0+15 15 MB/s | 1.5 MB 00:00 2023-07-21T18:52:36.988 INFO:teuthology.orchestra.run.smithi005.stdout:(127/131): librados2-18.0.0-4746.g81cf785a.el8. 2.1 MB/s | 3.7 MB 00:01 2023-07-21T18:52:37.621 INFO:teuthology.orchestra.run.smithi005.stdout:(128/131): librbd1-18.0.0-4746.g81cf785a.el8.x8 5.6 MB/s | 3.6 MB 00:00 2023-07-21T18:52:37.680 INFO:teuthology.orchestra.run.smithi005.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-07-21T18:54:17.254 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-asyncssh-2.7.0-2.el8.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-asyncssh-2.7.0-2.el8.noarch.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T18:54:17.254 INFO:teuthology.orchestra.run.smithi005.stdout:[MIRROR] python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.es.its.nyu.edu/epel/8/Everything/x86_64/Packages/p/python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm [Failed to connect to mirror.es.its.nyu.edu port 80: Connection timed out] 2023-07-21T18:54:18.025 INFO:teuthology.orchestra.run.smithi005.stdout:(130/131): python3-bcrypt-3.1.6-2.el8.1.x86_64. 331 B/s | 44 kB 02:14 2023-07-21T18:54:18.767 INFO:teuthology.orchestra.run.smithi005.stdout:(131/131): python3-asyncssh-2.7.0-2.el8.noarch. 3.5 kB/s | 476 kB 02:16 2023-07-21T18:54:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:54:18.769 INFO:teuthology.orchestra.run.smithi005.stdout:Total 727 kB/s | 205 MB 04:49 2023-07-21T18:54:20.468 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction check 2023-07-21T18:54:20.614 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction check succeeded. 2023-07-21T18:54:20.614 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction test 2023-07-21T18:54:22.231 INFO:teuthology.orchestra.run.smithi005.stdout:Transaction test succeeded. 2023-07-21T18:54:22.235 INFO:teuthology.orchestra.run.smithi005.stdout:Running transaction 2023-07-21T18:54:27.323 INFO:teuthology.orchestra.run.smithi005.stdout: Preparing : 1/1 2023-07-21T18:54:27.540 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-07-21T18:54:27.724 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-07-21T18:54:27.888 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:54:28.029 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 3/134 2023-07-21T18:54:28.255 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:54:28.746 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/134 2023-07-21T18:54:28.954 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:54:29.434 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 5/134 2023-07-21T18:54:29.596 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:54:29.736 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:54:29.946 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:54:30.061 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:54:30.218 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:54:30.398 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/134 2023-07-21T18:54:30.659 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-07-21T18:54:30.860 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-07-21T18:54:31.314 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-07-21T18:54:31.584 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-07-21T18:54:31.729 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:54:31.859 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 14/134 2023-07-21T18:54:32.020 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:54:32.240 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 15/134 2023-07-21T18:54:32.470 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 16/134 2023-07-21T18:54:32.655 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 17/134 2023-07-21T18:54:32.874 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 18/134 2023-07-21T18:54:33.003 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:54:33.181 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 19/134 2023-07-21T18:54:33.387 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:54:34.131 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 20/134 2023-07-21T18:54:34.295 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:54:35.019 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 21/134 2023-07-21T18:54:35.228 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:54:36.097 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 22/134 2023-07-21T18:54:36.361 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 23/134 2023-07-21T18:54:36.560 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 24/134 2023-07-21T18:54:36.809 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-07-21T18:54:36.994 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 26/134 2023-07-21T18:54:37.109 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 27/134 2023-07-21T18:54:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 28/134 2023-07-21T18:54:37.578 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:54:39.766 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 29/134 2023-07-21T18:54:40.961 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 30/134 2023-07-21T18:54:41.298 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 31/134 2023-07-21T18:54:41.581 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 32/134 2023-07-21T18:54:41.801 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/134 2023-07-21T18:54:42.008 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rsa-4.8-1.el8.noarch 34/134 2023-07-21T18:54:42.252 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-07-21T18:54:42.444 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 36/134 2023-07-21T18:54:42.678 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 37/134 2023-07-21T18:54:42.821 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 38/134 2023-07-21T18:54:42.980 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 39/134 2023-07-21T18:54:43.152 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:54:43.264 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 40/134 2023-07-21T18:54:43.508 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 41/134 2023-07-21T18:54:43.701 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : automake-1.16.1-7.el8.noarch 42/134 2023-07-21T18:54:43.885 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:54:43.985 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 43/134 2023-07-21T18:54:44.198 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : jq-1.6-3.el8.x86_64 44/134 2023-07-21T18:54:44.399 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 45/134 2023-07-21T18:54:44.545 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 46/134 2023-07-21T18:54:44.722 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 47/134 2023-07-21T18:54:44.896 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : fuse-2.9.7-15.el8.x86_64 48/134 2023-07-21T18:54:45.075 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 49/134 2023-07-21T18:54:45.284 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 50/134 2023-07-21T18:54:45.494 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 51/134 2023-07-21T18:54:46.170 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 52/134 2023-07-21T18:54:46.477 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 53/134 2023-07-21T18:54:46.710 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 54/134 2023-07-21T18:54:46.935 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 55/134 2023-07-21T18:54:47.188 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 56/134 2023-07-21T18:54:47.552 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 57/134 2023-07-21T18:54:47.753 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 58/134 2023-07-21T18:54:47.968 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 59/134 2023-07-21T18:54:48.197 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 60/134 2023-07-21T18:54:48.434 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 61/134 2023-07-21T18:54:48.583 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 62/134 2023-07-21T18:54:48.903 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 63/134 2023-07-21T18:54:49.195 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 64/134 2023-07-21T18:54:49.437 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 65/134 2023-07-21T18:54:49.680 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 66/134 2023-07-21T18:54:49.828 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-07-21T18:54:49.993 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-07-21T18:54:50.304 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-07-21T18:54:50.501 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-07-21T18:54:50.673 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-07-21T18:54:50.848 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-07-21T18:54:51.034 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-07-21T18:54:51.227 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-07-21T18:54:51.450 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-07-21T18:54:51.604 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-07-21T18:54:51.733 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-07-21T18:54:52.002 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-07-21T18:54:52.215 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-07-21T18:54:52.383 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-07-21T18:54:52.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-07-21T18:54:52.852 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-07-21T18:54:53.011 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-07-21T18:54:53.269 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-07-21T18:54:54.663 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-07-21T18:54:54.948 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-07-21T18:54:55.200 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-07-21T18:54:55.550 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-07-21T18:54:55.863 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-07-21T18:54:56.106 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-07-21T18:54:56.979 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-07-21T18:54:57.147 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:54:57.230 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 92/134 2023-07-21T18:54:57.409 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 93/134 2023-07-21T18:55:02.497 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:55:02.716 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:55:03.436 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 94/134 2023-07-21T18:55:03.721 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:55:03.882 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 95/134 2023-07-21T18:55:03.982 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:55:25.289 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 96/134 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /sys 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /proc 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /run 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /mnt 2023-07-21T18:55:25.290 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /var/tmp 2023-07-21T18:55:25.291 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /home 2023-07-21T18:55:25.291 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /tmp 2023-07-21T18:55:25.291 INFO:teuthology.orchestra.run.smithi005.stdout:skipping the directory /dev 2023-07-21T18:55:25.291 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:55:25.671 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:55:27.529 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 97/134 2023-07-21T18:55:27.850 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:55:28.135 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 98/134 2023-07-21T18:55:28.354 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:55:28.923 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 99/134 2023-07-21T18:55:29.112 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:55:29.155 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 100/134 2023-07-21T18:55:32.010 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:55:32.170 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 101/134 2023-07-21T18:55:32.311 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 102/134 2023-07-21T18:55:32.618 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 103/134 2023-07-21T18:55:32.794 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:55:33.068 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 104/134 2023-07-21T18:55:33.274 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:55:34.460 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 105/134 2023-07-21T18:55:34.644 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:55:34.779 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 106/134 2023-07-21T18:55:35.151 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 107/134 2023-07-21T18:55:35.385 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:55:35.452 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 108/134 2023-07-21T18:55:35.636 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:55:35.665 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 109/134 2023-07-21T18:55:37.172 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 110/134 2023-07-21T18:55:37.361 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:55:37.430 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 111/134 2023-07-21T18:55:37.602 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:55:37.676 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 112/134 2023-07-21T18:55:37.819 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:55:38.262 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 113/134 2023-07-21T18:55:38.419 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:55:43.750 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 114/134 2023-07-21T18:55:45.068 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 115/134 2023-07-21T18:55:45.377 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 116/134 2023-07-21T18:55:45.549 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 117/134 2023-07-21T18:55:45.829 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 118/134 2023-07-21T18:55:45.984 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:55:46.250 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/134 2023-07-21T18:55:46.456 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 120/134 2023-07-21T18:55:46.615 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 121/134 2023-07-21T18:55:46.785 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-07-21T18:55:50.314 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:55:50.658 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:55:50.696 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-07-21T18:55:50.885 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 124/134 2023-07-21T18:55:51.095 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 125/134 2023-07-21T18:55:51.414 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 126/134 2023-07-21T18:55:51.803 INFO:teuthology.orchestra.run.smithi005.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-07-21T18:55:52.016 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:55:52.135 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 128/134 2023-07-21T18:55:52.333 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:55:52.362 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 129/134 2023-07-21T18:55:52.608 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:55:52.631 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 130/134 2023-07-21T18:55:52.795 INFO:teuthology.orchestra.run.smithi005.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 131/134 2023-07-21T18:55:52.796 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:55:52.998 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:55:52.998 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:55:53.316 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-07-21T18:55:53.501 INFO:teuthology.orchestra.run.smithi005.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:55:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:55:53.766 INFO:teuthology.orchestra.run.smithi005.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2023-07-21T18:55:53.766 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 1/134 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2/134 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 3/134 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 4/134 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-4746.g81cf7 5/134 2023-07-21T18:55:54.394 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 6/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 7/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 8/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 9/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 10/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 11/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 12/134 2023-07-21T18:55:54.395 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 13/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_ 14/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 15/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 16/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_6 17/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86 18/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 19/134 2023-07-21T18:55:54.396 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el 20/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-ceph-common-2:18.0.0-4746.g81cf785a.el8. 21/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_6 22/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 23/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 24/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 25/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 26/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 27/134 2023-07-21T18:55:54.397 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 28/134 2023-07-21T18:55:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a. 29/134 2023-07-21T18:55:54.404 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noa 30/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.n 31/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-4746.g81c 32/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el 33/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 34/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.e 35/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 36/134 2023-07-21T18:55:54.405 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 37/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 38/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-07-21T18:55:54.406 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-07-21T18:55:54.407 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-07-21T18:55:54.408 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-rsa-4.8-1.el8.noarch 60/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-07-21T18:55:54.409 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 72/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 73/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 74/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/134 2023-07-21T18:55:54.410 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 76/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 77/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 79/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 81/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 82/134 2023-07-21T18:55:54.411 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 83/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 84/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 85/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 86/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 87/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 88/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 89/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 90/134 2023-07-21T18:55:54.412 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 91/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 92/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 93/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 94/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 95/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 96/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 97/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 98/134 2023-07-21T18:55:54.413 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 99/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 100/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 101/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 102/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 103/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : bison-3.0.4-10.el8.x86_64 104/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 105/134 2023-07-21T18:55:54.414 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 106/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : automake-1.16.1-7.el8.noarch 108/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 109/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 110/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : flex-2.6.1-9.el8.x86_64 111/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 112/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 113/134 2023-07-21T18:55:54.415 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 114/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 115/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 116/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 117/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 118/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 119/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 120/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 121/134 2023-07-21T18:55:54.416 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : libpq-13.5-1.el8.x86_64 122/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 123/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : jq-1.6-3.el8.x86_64 124/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 125/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 126/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 127/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 128/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 129/134 2023-07-21T18:55:54.417 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-07-21T18:55:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 131/134 2023-07-21T18:55:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-07-21T18:55:54.418 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-07-21T18:55:57.258 INFO:teuthology.orchestra.run.smithi005.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-07-21T18:55:57.258 INFO:teuthology.orchestra.run.smithi005.stdout:Last metadata expiration check: 0:06:41 ago on Fri 21 Jul 2023 06:49:16 PM UTC. 2023-07-21T18:56:00.032 INFO:teuthology.orchestra.run.smithi005.stdout:Installed products updated. 2023-07-21T18:56:00.272 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:56:00.272 INFO:teuthology.orchestra.run.smithi005.stdout:Upgraded: 2023-07-21T18:56:00.272 INFO:teuthology.orchestra.run.smithi005.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-07-21T18:56:00.272 INFO:teuthology.orchestra.run.smithi005.stdout: librados2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.273 INFO:teuthology.orchestra.run.smithi005.stdout: librbd1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.273 INFO:teuthology.orchestra.run.smithi005.stdout:Installed: 2023-07-21T18:56:00.273 INFO:teuthology.orchestra.run.smithi005.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: automake-1.16.1-7.el8.noarch 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: bison-3.0.4-10.el8.x86_64 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-base-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.276 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-grafana-dashboards-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-immutable-object-cache-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mds-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-dashboard-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.277 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-diskprediction-local-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-modules-core-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mgr-rook-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-mon-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-osd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-prometheus-alerts-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-radosgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.278 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-selinux-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-test-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: ceph-volume-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: cephadm-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: cephfs-top-2:18.0.0-4746.g81cf785a.el8.noarch 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: flex-2.6.1-9.el8.x86_64 2023-07-21T18:56:00.279 INFO:teuthology.orchestra.run.smithi005.stdout: fuse-2.9.7-15.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: jq-1.6-3.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: libcephfs2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.280 INFO:teuthology.orchestra.run.smithi005.stdout: libcephsqlite-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: liboath-2.6.2-3.el8.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: libpq-13.5-1.el8.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-07-21T18:56:00.281 INFO:teuthology.orchestra.run.smithi005.stdout: librados-devel-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libradosstriper1-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: librgw2-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libtool-2.4.6-25.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libunwind-1.3.1-3.el8.x86_64 2023-07-21T18:56:00.282 INFO:teuthology.orchestra.run.smithi005.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-0.3.15-3.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-07-21T18:56:00.283 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: python3-babel-2.5.1-7.el8.noarch 2023-07-21T18:56:00.284 INFO:teuthology.orchestra.run.smithi005.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-argparse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-ceph-common-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cephfs-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-07-21T18:56:00.285 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-6.2-6.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-07-21T18:56:00.286 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-mako-1.0.6-13.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-07-21T18:56:00.287 INFO:teuthology.orchestra.run.smithi005.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-portend-2.6-1.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-07-21T18:56:00.288 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pycparser-2.14-14.el8.noarch 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-pytz-2017.2-9.el8.noarch 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rados-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-07-21T18:56:00.289 INFO:teuthology.orchestra.run.smithi005.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rgw-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-routes-2.4.1-12.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-rsa-4.8-1.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-07-21T18:56:00.290 INFO:teuthology.orchestra.run.smithi005.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-fuse-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.291 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-mirror-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: rbd-nbd-2:18.0.0-4746.g81cf785a.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: socat-1.7.4.1-1.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: thrift-0.13.0-2.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: uuid-1.6.2-43.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-07-21T18:56:00.292 INFO:teuthology.orchestra.run.smithi005.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-07-21T18:56:00.293 INFO:teuthology.orchestra.run.smithi005.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-07-21T18:56:00.293 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:56:00.293 INFO:teuthology.orchestra.run.smithi005.stdout:Complete! 2023-07-21T18:56:00.608 DEBUG:teuthology.parallel:result is None 2023-07-21T18:56:00.609 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:00.924 DEBUG:teuthology.orchestra.run.smithi005:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:56:00.979 INFO:teuthology.orchestra.run.smithi005.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:00.980 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:00.980 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:56:00.981 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:01.298 DEBUG:teuthology.orchestra.run.smithi101:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:56:01.359 INFO:teuthology.orchestra.run.smithi101.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:01.360 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:01.360 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:56:01.361 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:01.619 DEBUG:teuthology.orchestra.run.smithi156:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-07-21T18:56:01.681 INFO:teuthology.orchestra.run.smithi156.stdout:18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:01.682 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-4746.g81cf785a.el8 2023-07-21T18:56:01.683 INFO:teuthology.task.install:The correct ceph version 18.0.0-4746.g81cf785a is installed. 2023-07-21T18:56:01.683 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-07-21T18:56:01.684 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:56:01.685 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:56:01.774 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:56:01.775 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:56:01.869 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:56:01.869 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-07-21T18:56:01.963 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-07-21T18:56:01.964 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:56:01.964 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:56:02.034 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:56:02.142 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:56:02.142 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:56:02.214 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:56:02.323 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:56:02.323 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/daemon-helper 2023-07-21T18:56:02.395 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-07-21T18:56:02.504 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-07-21T18:56:02.505 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:56:02.505 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:56:02.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:56:02.683 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:56:02.683 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:56:02.751 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:56:02.857 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:56:02.858 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/adjust-ulimits 2023-07-21T18:56:02.928 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-07-21T18:56:03.037 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-07-21T18:56:03.037 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:56:03.038 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:56:03.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:56:03.215 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:56:03.216 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:56:03.283 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:56:03.390 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:56:03.390 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/usr/bin/stdin-killer 2023-07-21T18:56:03.459 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-07-21T18:56:03.566 INFO:teuthology.run_tasks:Running task cephadm... 2023-07-21T18:56:03.656 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'subvols': {'create': 2}} 2023-07-21T18:56:03.657 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:03.670 INFO:tasks.cephadm:Cluster fsid is 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:56:03.671 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-07-21T18:56:03.672 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.5', 'mon.b': '172.21.15.101', 'mon.c': '172.21.15.156'} 2023-07-21T18:56:03.673 INFO:tasks.cephadm:First mon is mon.a on smithi005 2023-07-21T18:56:03.673 INFO:tasks.cephadm:First mgr is x 2023-07-21T18:56:03.674 INFO:tasks.cephadm:Normalizing hostnames... 2023-07-21T18:56:03.675 DEBUG:teuthology.orchestra.run.smithi005:> sudo hostname $(hostname -s) 2023-07-21T18:56:03.749 DEBUG:teuthology.orchestra.run.smithi101:> sudo hostname $(hostname -s) 2023-07-21T18:56:03.820 DEBUG:teuthology.orchestra.run.smithi156:> sudo hostname $(hostname -s) 2023-07-21T18:56:03.890 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-07-21T18:56:03.891 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:04.292 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '81cf785aa7781c4e08381c540f890cb9bc987ef9', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/71509/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-4746-g81cf785a', 'node_name': '172.21.2.5+braggi05', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-4746.g81cf785a'}, 'url': 'https://1.chacra.ceph.com/r/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-07-18 00:38:12.274423', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-rishabh-2023Jul13-m-quota', 'chacra_url': 'https://1.chacra.ceph.com/repos/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2023-07-21T18:56:04.662 INFO:tasks.util.chacra:got chacra host 1.chacra.ceph.com, ref wip-rishabh-2023Jul13-m-quota, sha1 81cf785aa7781c4e08381c540f890cb9bc987ef9 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=81cf785aa7781c4e08381c540f890cb9bc987ef9 2023-07-21T18:56:04.663 INFO:tasks.cephadm:Discovered cachra url: https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm 2023-07-21T18:56:04.663 DEBUG:teuthology.orchestra.run.smithi005:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:05.495 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:56 /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:05.496 DEBUG:teuthology.orchestra.run.smithi101:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:06.406 INFO:teuthology.orchestra.run.smithi101.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:56 /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:06.407 DEBUG:teuthology.orchestra.run.smithi156:> curl --silent -L https://1.chacra.ceph.com/binaries/ceph/wip-rishabh-2023Jul13-m-quota/81cf785aa7781c4e08381c540f890cb9bc987ef9/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:07.445 INFO:teuthology.orchestra.run.smithi156.stdout:-rw-rw-r--. 1 ubuntu ubuntu 663791 Jul 21 18:56 /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:07.445 DEBUG:teuthology.orchestra.run.smithi005:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:07.496 DEBUG:teuthology.orchestra.run.smithi101:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:07.547 DEBUG:teuthology.orchestra.run.smithi156:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-07-21T18:56:07.612 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 on all hosts... 2023-07-21T18:56:07.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:56:07.618 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:56:07.622 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 pull 2023-07-21T18:56:09.073 INFO:teuthology.orchestra.run.smithi156.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:56:09.090 INFO:teuthology.orchestra.run.smithi101.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:56:12.236 INFO:teuthology.orchestra.run.smithi005.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:56:39.813 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T18:56:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:56:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:56:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "repo_digests": [ 2023-07-21T18:56:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:56:39.814 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T18:56:39.815 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T18:56:53.612 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-21T18:56:53.613 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:56:53.613 INFO:teuthology.orchestra.run.smithi156.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:56:53.613 INFO:teuthology.orchestra.run.smithi156.stdout: "repo_digests": [ 2023-07-21T18:56:53.613 INFO:teuthology.orchestra.run.smithi156.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:56:53.613 INFO:teuthology.orchestra.run.smithi156.stdout: ] 2023-07-21T18:56:53.614 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-21T18:56:54.135 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-21T18:56:54.135 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T18:56:54.136 INFO:teuthology.orchestra.run.smithi101.stdout: "image_id": "66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421", 2023-07-21T18:56:54.136 INFO:teuthology.orchestra.run.smithi101.stdout: "repo_digests": [ 2023-07-21T18:56:54.136 INFO:teuthology.orchestra.run.smithi101.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:9b4b7dd677a0f5b8e56a57eadd762824d684980cb80d319543e86b72e549ef9c" 2023-07-21T18:56:54.136 INFO:teuthology.orchestra.run.smithi101.stdout: ] 2023-07-21T18:56:54.136 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-21T18:56:54.162 DEBUG:teuthology.orchestra.run.smithi005:> sudo mkdir -p /etc/ceph 2023-07-21T18:56:54.240 DEBUG:teuthology.orchestra.run.smithi101:> sudo mkdir -p /etc/ceph 2023-07-21T18:56:54.311 DEBUG:teuthology.orchestra.run.smithi156:> sudo mkdir -p /etc/ceph 2023-07-21T18:56:54.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 777 /etc/ceph 2023-07-21T18:56:54.459 DEBUG:teuthology.orchestra.run.smithi101:> sudo chmod 777 /etc/ceph 2023-07-21T18:56:54.527 DEBUG:teuthology.orchestra.run.smithi156:> sudo chmod 777 /etc/ceph 2023-07-21T18:56:54.597 INFO:tasks.cephadm:Writing seed config... 2023-07-21T18:56:54.599 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-07-21T18:56:54.599 INFO:tasks.cephadm: override: [client] debug client = 20 2023-07-21T18:56:54.600 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-07-21T18:56:54.600 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-07-21T18:56:54.600 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-07-21T18:56:54.601 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-07-21T18:56:54.601 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-07-21T18:56:54.602 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-07-21T18:56:54.602 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-07-21T18:56:54.603 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-07-21T18:56:54.603 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-07-21T18:56:54.604 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-07-21T18:56:54.604 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-07-21T18:56:54.604 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-07-21T18:56:54.605 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-07-21T18:56:54.605 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-07-21T18:56:54.605 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-07-21T18:56:54.605 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-07-21T18:56:54.606 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-07-21T18:56:54.606 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-07-21T18:56:54.606 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-07-21T18:56:54.606 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-07-21T18:56:54.606 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-07-21T18:56:54.607 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-07-21T18:56:54.607 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-07-21T18:56:54.607 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-07-21T18:56:54.607 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-07-21T18:56:54.608 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-07-21T18:56:54.608 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-07-21T18:56:54.608 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-07-21T18:56:54.608 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-07-21T18:56:54.609 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-07-21T18:56:54.609 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-07-21T18:56:54.609 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-07-21T18:56:54.609 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-07-21T18:56:54.609 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-07-21T18:56:54.610 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-07-21T18:56:54.610 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-07-21T18:56:54.610 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-07-21T18:56:54.610 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-07-21T18:56:54.610 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-07-21T18:56:54.611 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-07-21T18:56:54.611 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-07-21T18:56:54.611 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2023-07-21T18:56:54.612 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:56:54.612 DEBUG:teuthology.orchestra.run.smithi005:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-07-21T18:56:54.656 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 3dc43aa4-27f8-11ee-9b34-001a4aab830c [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-07-21T18:56:54.656 DEBUG:teuthology.orchestra.run.smithi005:mon.a> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service 2023-07-21T18:56:54.700 DEBUG:teuthology.orchestra.run.smithi005:mgr.x> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x.service 2023-07-21T18:56:54.746 INFO:tasks.cephadm:Bootstrapping... 2023-07-21T18:56:54.747 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 -v bootstrap --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.5 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:56:54.768 INFO:journalctl@ceph.mon.a.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T18:56:54.817 INFO:journalctl@ceph.mgr.x.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T18:56:54.975 INFO:teuthology.orchestra.run.smithi005.stdout:-------------------------------------------------------------------------------- 2023-07-21T18:56:54.976 INFO:teuthology.orchestra.run.smithi005.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9', '-v', 'bootstrap', '--fsid', '3dc43aa4-27f8-11ee-9b34-001a4aab830c', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.5', '--skip-admin-label'] 2023-07-21T18:56:55.150 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:56:55.151 INFO:teuthology.orchestra.run.smithi005.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-07-21T18:56:55.151 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying podman|docker is present... 2023-07-21T18:56:55.322 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:56:55.323 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying lvm2 is present... 2023-07-21T18:56:55.323 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying time synchronization is in place... 2023-07-21T18:56:55.337 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T18:56:55.337 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T18:56:55.350 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T18:56:55.350 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:56:55.364 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2023-07-21T18:56:55.377 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2023-07-21T18:56:55.378 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2023-07-21T18:56:55.378 INFO:teuthology.orchestra.run.smithi005.stdout:Repeating the final host check... 2023-07-21T18:56:55.547 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 4.1.1 2023-07-21T18:56:55.548 INFO:teuthology.orchestra.run.smithi005.stdout:podman (/bin/podman) version 4.1.1 is present 2023-07-21T18:56:55.548 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl is present 2023-07-21T18:56:55.548 INFO:teuthology.orchestra.run.smithi005.stdout:lvcreate is present 2023-07-21T18:56:55.561 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-07-21T18:56:55.561 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-07-21T18:56:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-07-21T18:56:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:56:55.588 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout enabled 2023-07-21T18:56:55.600 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout active 2023-07-21T18:56:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:Unit chronyd.service is enabled and running 2023-07-21T18:56:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:Host looks OK 2023-07-21T18:56:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:Cluster fsid: 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:56:55.601 INFO:teuthology.orchestra.run.smithi005.stdout:Acquiring lock 140536240286184 on /run/cephadm/3dc43aa4-27f8-11ee-9b34-001a4aab830c.lock 2023-07-21T18:56:55.602 INFO:teuthology.orchestra.run.smithi005.stdout:Lock 140536240286184 acquired on /run/cephadm/3dc43aa4-27f8-11ee-9b34-001a4aab830c.lock 2023-07-21T18:56:55.602 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 3300 ... 2023-07-21T18:56:55.602 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying IP 172.21.15.5 port 6789 ... 2023-07-21T18:56:55.602 INFO:teuthology.orchestra.run.smithi005.stdout:Base mon IP(s) is [172.21.15.5:3300, 172.21.15.5:6789], mon addrv is [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2023-07-21T18:56:55.610 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp metric 100 2023-07-21T18:56:55.610 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.5 metric 100 2023-07-21T18:56:55.616 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-07-21T18:56:55.617 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2023-07-21T18:56:55.617 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium 2023-07-21T18:56:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-07-21T18:56:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-07-21T18:56:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T18:56:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2023-07-21T18:56:55.623 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1442/64 scope link 2023-07-21T18:56:55.624 INFO:teuthology.orchestra.run.smithi005.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-07-21T18:56:55.624 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2023-07-21T18:56:55.624 INFO:teuthology.orchestra.run.smithi005.stdout:Mon IP `172.21.15.5` is in CIDR network `172.21.0.0/20` 2023-07-21T18:56:55.625 INFO:teuthology.orchestra.run.smithi005.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-07-21T18:56:55.625 INFO:teuthology.orchestra.run.smithi005.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-07-21T18:56:55.626 INFO:teuthology.orchestra.run.smithi005.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:56:56.100 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stdout 66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-21T18:56:56.100 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9... 2023-07-21T18:56:56.100 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Getting image source signatures 2023-07-21T18:56:56.100 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:711183bdaeabd1b922e81a2784cd69d11b05a304985585795e1ae029bf69712a 2023-07-21T18:56:56.100 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying blob sha256:6022c611d474f2b306ddd1edfc8360123ece5266119096f25fde843d7967e636 2023-07-21T18:56:56.101 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Copying config sha256:66a4f7b015b53bd7c8d67b561c2d8f6525dccea2a3d3e0d20f2a1cd17a765421 2023-07-21T18:56:56.101 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Writing manifest to image destination 2023-07-21T18:56:56.101 INFO:teuthology.orchestra.run.smithi005.stdout:/bin/podman: stderr Storing signatures 2023-07-21T18:56:57.076 INFO:teuthology.orchestra.run.smithi005.stdout:ceph: stdout ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T18:56:57.077 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph version: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T18:56:57.077 INFO:teuthology.orchestra.run.smithi005.stdout:Extracting ceph user uid/gid from container image... 2023-07-21T18:56:57.958 INFO:teuthology.orchestra.run.smithi005.stdout:stat: stdout 167 167 2023-07-21T18:56:57.958 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial keys... 2023-07-21T18:56:58.837 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQD61LpkpsllFRAAGQBoa4aXv0Ranc3duLnh0w== 2023-07-21T18:56:59.612 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQD71Lpk2bhnDRAACJLPLUn9/a6jcW01iHNRpw== 2023-07-21T18:57:01.102 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph-authtool: stdout AQD81Lpk8lBgJRAAwxFcnzakWPrhF3yCG8Pq3Q== 2023-07-21T18:57:01.103 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial monmap... 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:monmaptool for a [v2:172.21.15.5:3300,v1:172.21.15.5:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-07-21T18:57:01.883 INFO:teuthology.orchestra.run.smithi005.stdout:setting min_mon_release = pacific 2023-07-21T18:57:01.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: set fsid to 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:01.884 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-07-21T18:57:01.884 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:57:01.884 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mon... 2023-07-21T18:57:02.658 INFO:teuthology.orchestra.run.smithi005.stdout:create mon.a on 2023-07-21T18:57:02.917 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-07-21T18:57:03.128 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-07-21T18:57:03.384 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target. 2023-07-21T18:57:03.384 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target → /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target. 2023-07-21T18:57:03.737 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a 2023-07-21T18:57:03.737 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service: Unit ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service not loaded. 2023-07-21T18:57:03.953 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target.wants/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service → /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@.service. 2023-07-21T18:57:04.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:03 smithi005 systemd[1]: Starting Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:57:04.750 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:04 smithi005 podman[125915]: 2023-07-21T18:57:04.751 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:04 smithi005 bash[125915]: b96fc1dd992f706720f52ebb218697e94c2a567db44d1538f5469979fcfd681a 2023-07-21T18:57:04.774 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:57:04.774 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-07-21T18:57:04.792 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:57:04.792 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:57:04.793 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-07-21T18:57:04.793 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T18:57:04.793 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon to start... 2023-07-21T18:57:04.793 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mon... 2023-07-21T18:57:05.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:04 smithi005 systemd[1]: Started Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:57:05.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:04 smithi005 ceph-mon[125955]: mkfs 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:05.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:04 smithi005 ceph-mon[125955]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:57:06.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:05 smithi005 ceph-mon[125955]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:57:06.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:05 smithi005 ceph-mon[125955]: monmap e1: 1 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0]} removed_ranks: {} 2023-07-21T18:57:06.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:05 smithi005 ceph-mon[125955]: fsmap 2023-07-21T18:57:06.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:05 smithi005 ceph-mon[125955]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T18:57:06.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:05 smithi005 ceph-mon[125955]: mgrmap e1: no daemons active 2023-07-21T18:57:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:12 smithi005 ceph-mon[125955]: from='client.? 172.21.15.5:0/362251265' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-07-21T18:57:12.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout cluster: 2023-07-21T18:57:12.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout id: 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:12.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-07-21T18:57:12.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout services: 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 7s) 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:12.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout data: 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout pgs: 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.284+0000 7f59709ff700 1 Processor -- start 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.284+0000 7f59709ff700 1 -- start start 2023-07-21T18:57:12.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.284+0000 7f59709ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.284+0000 7f59709ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c105b60 con 0x7f596c105210 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.285+0000 7f596a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57508/0 (socket says 172.21.15.5:57508) 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596a59c700 1 -- 172.21.15.5:0/792674245 learned_addr learned my addr 172.21.15.5:0/792674245 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596a59c700 1 -- 172.21.15.5:0/792674245 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c105ca0 con 0x7f596c105210 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596a59c700 1 --2- 172.21.15.5:0/792674245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f595c0098a0 tx=0x7f595c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ad755f17339c97e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:12.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596959a700 1 -- 172.21.15.5:0/792674245 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f595c004160 con 0x7f596c105210 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596959a700 1 -- 172.21.15.5:0/792674245 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c0042c0 con 0x7f596c105210 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.286+0000 7f596959a700 1 -- 172.21.15.5:0/792674245 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f595c0045a0 con 0x7f596c105210 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 -- 172.21.15.5:0/792674245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 msgr2=0x7f596c105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 --2- 172.21.15.5:0/792674245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f595c0098a0 tx=0x7f595c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 -- 172.21.15.5:0/792674245 shutdown_connections 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 --2- 172.21.15.5:0/792674245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c105620 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 -- 172.21.15.5:0/792674245 >> 172.21.15.5:0/792674245 conn(0x7f596c100900 msgr2=0x7f596c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:12.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 -- 172.21.15.5:0/792674245 shutdown_connections 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.287+0000 7f59709ff700 1 -- 172.21.15.5:0/792674245 wait complete. 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.288+0000 7f59709ff700 1 Processor -- start 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.288+0000 7f59709ff700 1 -- start start 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f59709ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f59709ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c198cf0 con 0x7f596c105210 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f596a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:12.900 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f596a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57512/0 (socket says 172.21.15.5:57512) 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f596a59c700 1 -- 172.21.15.5:0/362251265 learned_addr learned my addr 172.21.15.5:0/362251265 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.289+0000 7f596a59c700 1 -- 172.21.15.5:0/362251265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f595c009550 con 0x7f596c105210 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f596a59c700 1 --2- 172.21.15.5:0/362251265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f596c105f40 tx=0x7f595c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f595c011630 con 0x7f596c105210 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f595c011790 con 0x7f596c105210 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f595c011a30 con 0x7f596c105210 2023-07-21T18:57:12.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c198ef0 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c199310 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f595c022070 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.290+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f595c01c450 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.291+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c191ff0 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.294+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f595c01b450 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.369+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f596c02c850 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.370+0000 7f595b7fe700 1 -- 172.21.15.5:0/362251265 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+310 (secure 0 0 0) 0x7f595c01b5f0 con 0x7f596c105210 2023-07-21T18:57:12.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 msgr2=0x7f596c1987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 --2- 172.21.15.5:0/362251265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f596c105f40 tx=0x7f595c004750 comp rx=0 tx=0).stop 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 shutdown_connections 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 --2- 172.21.15.5:0/362251265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f596c105210 0x7f596c1987b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 >> 172.21.15.5:0/362251265 conn(0x7f596c100900 msgr2=0x7f596c18cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 shutdown_connections 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:12.371+0000 7f59709ff700 1 -- 172.21.15.5:0/362251265 wait complete. 2023-07-21T18:57:12.903 INFO:teuthology.orchestra.run.smithi005.stdout:mon is available 2023-07-21T18:57:12.904 INFO:teuthology.orchestra.run.smithi005.stdout:Assimilating anything we can from ceph.conf... 2023-07-21T18:57:13.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:13 smithi005 ceph-mon[125955]: from='client.? 172.21.15.5:0/1163717760' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T18:57:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:13 smithi005 ceph-mon[125955]: from='client.? 172.21.15.5:0/1163717760' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-07-21T18:57:13.978 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.5:3300,v1:172.21.15.5:6789] 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T18:57:13.979 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T18:57:13.980 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.406+0000 7fa969190700 1 Processor -- start 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.406+0000 7fa969190700 1 -- start start 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.406+0000 7fa969190700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.406+0000 7fa969190700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa964105b60 con 0x7fa964105210 2023-07-21T18:57:13.981 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.407+0000 7fa962d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.407+0000 7fa962d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57516/0 (socket says 172.21.15.5:57516) 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.407+0000 7fa962d9d700 1 -- 172.21.15.5:0/4008640458 learned_addr learned my addr 172.21.15.5:0/4008640458 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.407+0000 7fa962d9d700 1 -- 172.21.15.5:0/4008640458 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa964105ca0 con 0x7fa964105210 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa962d9d700 1 --2- 172.21.15.5:0/4008640458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6172f151fcba2f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa961d9b700 1 -- 172.21.15.5:0/4008640458 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa954004160 con 0x7fa964105210 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa961d9b700 1 -- 172.21.15.5:0/4008640458 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9540042c0 con 0x7fa964105210 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa961d9b700 1 -- 172.21.15.5:0/4008640458 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9540045a0 con 0x7fa964105210 2023-07-21T18:57:13.982 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 -- 172.21.15.5:0/4008640458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 msgr2=0x7fa964105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 --2- 172.21.15.5:0/4008640458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa9540098a0 tx=0x7fa954009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 -- 172.21.15.5:0/4008640458 shutdown_connections 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 --2- 172.21.15.5:0/4008640458 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa964105620 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 -- 172.21.15.5:0/4008640458 >> 172.21.15.5:0/4008640458 conn(0x7fa964100900 msgr2=0x7fa964102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.408+0000 7fa969190700 1 -- 172.21.15.5:0/4008640458 shutdown_connections 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.409+0000 7fa969190700 1 -- 172.21.15.5:0/4008640458 wait complete. 2023-07-21T18:57:13.983 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.410+0000 7fa969190700 1 Processor -- start 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.410+0000 7fa969190700 1 -- start start 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa969190700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa969190700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa96407c1a0 con 0x7fa964105210 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa962d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa962d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57526/0 (socket says 172.21.15.5:57526) 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa962d9d700 1 -- 172.21.15.5:0/1163717760 learned_addr learned my addr 172.21.15.5:0/1163717760 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.411+0000 7fa962d9d700 1 -- 172.21.15.5:0/1163717760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa954009550 con 0x7fa964105210 2023-07-21T18:57:13.984 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa962d9d700 1 --2- 172.21.15.5:0/1163717760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa964105f40 tx=0x7fa954004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa954011630 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954011790 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa954011a30 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa96407a2b0 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa96407a6d0 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa954022070 con 0x7fa964105210 2023-07-21T18:57:13.985 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.412+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa95401c450 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.413+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa964061b40 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.416+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fa95401b450 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.488+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fa964191fd0 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.492+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 7 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fa95401b5f0 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.492+0000 7fa94bfff700 1 -- 172.21.15.5:0/1163717760 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+427 (secure 0 0 0) 0x7fa954037520 con 0x7fa964105210 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.493+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 msgr2=0x7fa96407bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.493+0000 7fa969190700 1 --2- 172.21.15.5:0/1163717760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa964105f40 tx=0x7fa954004750 comp rx=0 tx=0).stop 2023-07-21T18:57:13.986 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.494+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 shutdown_connections 2023-07-21T18:57:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.494+0000 7fa969190700 1 --2- 172.21.15.5:0/1163717760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa964105210 0x7fa96407bc60 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.494+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 >> 172.21.15.5:0/1163717760 conn(0x7fa964100900 msgr2=0x7fa96418cc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.494+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 shutdown_connections 2023-07-21T18:57:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:13.494+0000 7fa969190700 1 -- 172.21.15.5:0/1163717760 wait complete. 2023-07-21T18:57:13.987 INFO:teuthology.orchestra.run.smithi005.stdout:Generating new minimal ceph.conf... 2023-07-21T18:57:14.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:14 smithi005 ceph-mon[125955]: from='client.? 172.21.15.5:0/2161178796' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:57:15.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.529+0000 7f69fc455700 1 Processor -- start 2023-07-21T18:57:15.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.529+0000 7f69fc455700 1 -- start start 2023-07-21T18:57:15.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fc455700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:15.062 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fc455700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f407a810 con 0x7f69f407bdd0 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fa1f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fa1f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57536/0 (socket says 172.21.15.5:57536) 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fa1f1700 1 -- 172.21.15.5:0/3585456888 learned_addr learned my addr 172.21.15.5:0/3585456888 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.530+0000 7f69fa1f1700 1 -- 172.21.15.5:0/3585456888 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f407a950 con 0x7f69f407bdd0 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69fa1f1700 1 --2- 172.21.15.5:0/3585456888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69f00098a0 tx=0x7f69f0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d583029dc239072c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69f91ef700 1 -- 172.21.15.5:0/3585456888 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f0004160 con 0x7f69f407bdd0 2023-07-21T18:57:15.063 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69f91ef700 1 -- 172.21.15.5:0/3585456888 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f69f00042c0 con 0x7f69f407bdd0 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69f91ef700 1 -- 172.21.15.5:0/3585456888 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f0011940 con 0x7f69f407bdd0 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69fc455700 1 -- 172.21.15.5:0/3585456888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 msgr2=0x7f69f407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69fc455700 1 --2- 172.21.15.5:0/3585456888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f69f00098a0 tx=0x7f69f0009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.531+0000 7f69fc455700 1 -- 172.21.15.5:0/3585456888 shutdown_connections 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.532+0000 7f69fc455700 1 --2- 172.21.15.5:0/3585456888 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f407a2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.532+0000 7f69fc455700 1 -- 172.21.15.5:0/3585456888 >> 172.21.15.5:0/3585456888 conn(0x7f69f4100900 msgr2=0x7f69f4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:15.064 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.532+0000 7f69fc455700 1 -- 172.21.15.5:0/3585456888 shutdown_connections 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.532+0000 7f69fc455700 1 -- 172.21.15.5:0/3585456888 wait complete. 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.533+0000 7f69fc455700 1 Processor -- start 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.533+0000 7f69fc455700 1 -- start start 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fc455700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fc455700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69f419b6b0 con 0x7f69f407bdd0 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fa1f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fa1f1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57548/0 (socket says 172.21.15.5:57548) 2023-07-21T18:57:15.065 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fa1f1700 1 -- 172.21.15.5:0/2161178796 learned_addr learned my addr 172.21.15.5:0/2161178796 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.534+0000 7f69fa1f1700 1 -- 172.21.15.5:0/2161178796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69f0009550 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69fa1f1700 1 --2- 172.21.15.5:0/2161178796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000c00 tx=0x7f69f0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f00036a0 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f69f0003c70 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f69f00193f0 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69f419b8b0 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69f419bcd0 con 0x7f69f407bdd0 2023-07-21T18:57:15.066 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f69f0019920 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.535+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f69f001a3d0 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.536+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69f410b4d0 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.539+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f69f0029060 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.612+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f69f419ece0 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.612+0000 7f69eb7fe700 1 -- 172.21.15.5:0/2161178796 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+173 (secure 0 0 0) 0x7f69f0033090 con 0x7f69f407bdd0 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.613+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 msgr2=0x7f69f419b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:15.067 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.613+0000 7f69fc455700 1 --2- 172.21.15.5:0/2161178796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f69f0000c00 tx=0x7f69f0011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:15.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.613+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 shutdown_connections 2023-07-21T18:57:15.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.613+0000 7f69fc455700 1 --2- 172.21.15.5:0/2161178796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f69f407bdd0 0x7f69f419b170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:15.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.613+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 >> 172.21.15.5:0/2161178796 conn(0x7f69f4100900 msgr2=0x7f69f4102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:15.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.614+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 shutdown_connections 2023-07-21T18:57:15.068 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:14.614+0000 7f69fc455700 1 -- 172.21.15.5:0/2161178796 wait complete. 2023-07-21T18:57:15.069 INFO:teuthology.orchestra.run.smithi005.stdout:Restarting the monitor... 2023-07-21T18:57:15.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:15 smithi005 systemd[1]: Stopping Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:57:15.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a[125931]: 2023-07-21T18:57:15.539+0000 7f71dec9a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T18:57:15.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a[125931]: 2023-07-21T18:57:15.539+0000 7f71dec9a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-07-21T18:57:16.272 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:15 smithi005 bash[126292]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a 2023-07-21T18:57:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:16 smithi005 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service: Succeeded. 2023-07-21T18:57:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:16 smithi005 systemd[1]: Stopped Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:57:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:16 smithi005 systemd[1]: Starting Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:57:17.132 INFO:teuthology.orchestra.run.smithi005.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2023-07-21T18:57:17.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:16 smithi005 podman[126420]: 2023-07-21T18:57:17.214 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 bash[126420]: 39dc741d85925d4034bf9aadfd9a68e8331c369eff06c58933aa0c539974310e 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 systemd[1]: Started Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: pidfile_write: ignore empty --pid-file 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: load: jerasure load: lrc 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:57:17.215 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Git sha 0 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: DB SUMMARY 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: DB Session ID: ZSQHQN15NWLR4CP2LJOJ 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-07-21T18:57:17.216 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 100002 ; 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:57:17.217 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.env: 0x55be97292720 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.info_log: 0x55be983bf280 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.statistics: (nil) 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.use_fsync: 0 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:57:17.218 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:57:17.219 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.db_log_dir: 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.wal_dir: 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:57:17.220 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.write_buffer_manager: 0x55be982e03c0 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:57:17.221 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.unordered_write: 0 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:57:17.222 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.row_cache: None 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.wal_filter: None 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:57:17.223 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.wal_compression: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:57:17.224 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:57:17.225 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:57:17.225 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:57:17.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:57:17.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:57:17.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:57:17.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:57:17.226 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:57:17.227 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_open_files: -1 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:57:17.228 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Compression algorithms supported: 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kZSTD supported: 0 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:57:17.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-07-21T18:57:17.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.merge_operator: 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_filter: None 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:57:17.231 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be983bf380) 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_type: 0 2023-07-21T18:57:17.232 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_index_type: 0 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_shortening: 1 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: checksum: 4 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: no_block_cache: 0 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache: 0x55be9837f1f0 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_options: 2023-07-21T18:57:17.233 INFO:journalctl@ceph.mon.a.smithi005.stdout: capacity : 536870912 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_shard_bits : 4 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: strict_capacity_limit : 0 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_cache_compressed: (nil) 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: persistent_cache: (nil) 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size: 4096 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_size_deviation: 10 2023-07-21T18:57:17.234 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_restart_interval: 16 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: index_block_restart_interval: 1 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: metadata_block_size: 4096 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: partition_filters: 0 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: use_delta_encoding: 1 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: filter_policy: bloomfilter 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: whole_key_filtering: 1 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: verify_compression: 0 2023-07-21T18:57:17.235 INFO:journalctl@ceph.mon.a.smithi005.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: format_version: 5 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_index_compression: 1 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: block_align: 0 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: max_auto_readahead_size: 262144 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: prepopulate_block_cache: 0 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:57:17.236 INFO:journalctl@ceph.mon.a.smithi005.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression: NoCompression 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.num_levels: 7 2023-07-21T18:57:17.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:57:17.238 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:57:17.239 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:57:17.240 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:57:17.241 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:57:17.242 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:57:17.243 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:57:17.244 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:57:17.245 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.ttl: 2592000 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:57:17.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:57:17.247 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: a9bb3d17-a679-4889-a4b0-00c8862e6074 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965837131214, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965837133919, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 94593, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 338, "table_properties": {"data_size": 92604, "index_size": 271, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13700, "raw_average_key_size": 49, "raw_value_size": 85127, "raw_average_value_size": 307, "num_data_blocks": 11, "num_entries": 277, "num_filter_entries": 277, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689965837, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "a9bb3d17-a679-4889-a4b0-00c8862e6074", "db_session_id": "ZSQHQN15NWLR4CP2LJOJ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:57:17.248 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965837134124, "job": 1, "event": "recovery_finished"} 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55be98418000 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: DB pointer 0x55be98404000 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:57:17.249 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** DB Stats ** 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:57:17.250 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: L0 2/0 94.21 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: Sum 2/0 94.21 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 60.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:57:17.251 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** Compaction Stats [default] ** 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 60.3 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.252 INFO:journalctl@ceph.mon.a.smithi005.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: Cumulative compaction: 0.00 GB write, 9.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: Interval compaction: 0.00 GB write, 9.61 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:57:17.253 INFO:journalctl@ceph.mon.a.smithi005.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache BinnedLRUCache@0x55be9837f1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: starting mon.a rank 0 at public addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] at bind addrs [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???) e1 preinit fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:17.254 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).mds e1 new map 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).mds e1 print_map 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: e1 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: legacy client fscid: -1 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout: No filesystems configured 2023-07-21T18:57:17.255 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mon.a is new leader, mons a in quorum (ranks 0) 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: monmap e1: 1 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0]} removed_ranks: {} 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: fsmap 2023-07-21T18:57:17.256 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: osdmap e1: 0 total, 0 up, 0 in 2023-07-21T18:57:17.257 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:17 smithi005 ceph-mon[126460]: mgrmap e1: no daemons active 2023-07-21T18:57:18.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.717+0000 7fe536af8700 1 Processor -- start 2023-07-21T18:57:18.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.718+0000 7fe536af8700 1 -- start start 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.718+0000 7fe536af8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.718+0000 7fe536af8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe530105b50 con 0x7fe530105200 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.719+0000 7fe534894700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.719+0000 7fe534894700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36844/0 (socket says 172.21.15.5:36844) 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.719+0000 7fe534894700 1 -- 172.21.15.5:0/3235379353 learned_addr learned my addr 172.21.15.5:0/3235379353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.719+0000 7fe534894700 1 -- 172.21.15.5:0/3235379353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe530105c90 con 0x7fe530105200 2023-07-21T18:57:18.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.719+0000 7fe534894700 1 --2- 172.21.15.5:0/3235379353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe524009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=934684f7a872093 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe52f7fe700 1 -- 172.21.15.5:0/3235379353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe524004160 con 0x7fe530105200 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe52f7fe700 1 -- 172.21.15.5:0/3235379353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe5240042c0 con 0x7fe530105200 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe52f7fe700 1 -- 172.21.15.5:0/3235379353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe524011940 con 0x7fe530105200 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 -- 172.21.15.5:0/3235379353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 msgr2=0x7fe530105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 --2- 172.21.15.5:0/3235379353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fe5240098a0 tx=0x7fe524009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 -- 172.21.15.5:0/3235379353 shutdown_connections 2023-07-21T18:57:18.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 --2- 172.21.15.5:0/3235379353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe530105610 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 -- 172.21.15.5:0/3235379353 >> 172.21.15.5:0/3235379353 conn(0x7fe530100890 msgr2=0x7fe530102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 -- 172.21.15.5:0/3235379353 shutdown_connections 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.720+0000 7fe536af8700 1 -- 172.21.15.5:0/3235379353 wait complete. 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.721+0000 7fe536af8700 1 Processor -- start 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe536af8700 1 -- start start 2023-07-21T18:57:18.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe536af8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe536af8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe53019b5b0 con 0x7fe530105200 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe534894700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe534894700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36850/0 (socket says 172.21.15.5:36850) 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.722+0000 7fe534894700 1 -- 172.21.15.5:0/2962366247 learned_addr learned my addr 172.21.15.5:0/2962366247 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe534894700 1 -- 172.21.15.5:0/2962366247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe524009550 con 0x7fe530105200 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe534894700 1 --2- 172.21.15.5:0/2962366247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe524011dc0 tx=0x7fe524011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe5240037d0 con 0x7fe530105200 2023-07-21T18:57:18.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fe52401b440 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe52401a560 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe53019b7b0 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.723+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe53019bbd0 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.724+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe52401aad0 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.724+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe5240193f0 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.724+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe530061b40 con 0x7fe530105200 2023-07-21T18:57:18.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.727+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fe524029060 con 0x7fe530105200 2023-07-21T18:57:18.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.800+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fe530107060 con 0x7fe530105200 2023-07-21T18:57:18.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.802+0000 7fe52dffb700 1 -- 172.21.15.5:0/2962366247 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fe530107060 con 0x7fe530105200 2023-07-21T18:57:18.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 msgr2=0x7fe53019b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:18.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 --2- 172.21.15.5:0/2962366247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fe524011dc0 tx=0x7fe524011ea0 comp rx=0 tx=0).stop 2023-07-21T18:57:18.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 shutdown_connections 2023-07-21T18:57:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 --2- 172.21.15.5:0/2962366247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe530105200 0x7fe53019b070 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 >> 172.21.15.5:0/2962366247 conn(0x7fe530100890 msgr2=0x7fe530101620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 shutdown_connections 2023-07-21T18:57:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:17.803+0000 7fe536af8700 1 -- 172.21.15.5:0/2962366247 wait complete. 2023-07-21T18:57:18.231 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote config to /etc/ceph/ceph.conf 2023-07-21T18:57:18.246 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:57:18.247 INFO:teuthology.orchestra.run.smithi005.stdout:Creating mgr... 2023-07-21T18:57:18.248 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 9283 ... 2023-07-21T18:57:18.248 INFO:teuthology.orchestra.run.smithi005.stdout:Verifying port 8765 ... 2023-07-21T18:57:18.615 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T18:57:18.616 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Failed to reset failed state of unit ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x.service: Unit ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x.service not loaded. 2023-07-21T18:57:18.823 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c.target.wants/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x.service → /etc/systemd/system/ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@.service. 2023-07-21T18:57:18.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:18 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2962366247' entity='client.admin' 2023-07-21T18:57:18.921 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:18 smithi005 systemd[1]: Starting Ceph mgr.x for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:57:19.612 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 podman[126687]: 2023-07-21T18:57:19.612 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 bash[126687]: 8d3ba823c2389dad04eb14a67c621dfde494bc03a6d10358b73016312b15dcd8 2023-07-21T18:57:19.635 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:57:19.636 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-07-21T18:57:19.653 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:57:19.653 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:57:19.653 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-07-21T18:57:19.653 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to enable service . firewalld.service is not available 2023-07-21T18:57:19.671 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:57:19.672 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-07-21T18:57:19.688 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:57:19.688 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:57:19.688 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-07-21T18:57:19.688 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-07-21T18:57:19.689 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr to start... 2023-07-21T18:57:19.689 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr... 2023-07-21T18:57:19.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 systemd[1]: Started Ceph mgr.x for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:57:19.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:19.716+0000 7f7ab1f11200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:57:19.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:19.828+0000 7f7ab1f11200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:57:20.328 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:19.951+0000 7f7ab1f11200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:57:20.328 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:20.085+0000 7f7ab1f11200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:57:20.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1503539830' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:57:20.670 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:20.323+0000 7f7ab1f11200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:57:20.672 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:57:20.673 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:57:20.674 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 3, 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:57:20.675 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:57:20.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:20.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:57:20.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:57:20.676 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:57:20.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:57:20.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:57:20.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:57:20.682 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:57:20.683 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:57:20.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:57:20.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:57:20.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:20.684 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:57:20.685 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:20.686 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:57:04.740231+0000", 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.207+0000 7fd790843700 1 Processor -- start 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.207+0000 7fd790843700 1 -- start start 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.207+0000 7fd790843700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:20.687 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.207+0000 7fd790843700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c07a810 con 0x7fd78c07bdd0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.208+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.208+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36876/0 (socket says 172.21.15.5:36876) 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.208+0000 7fd78a59c700 1 -- 172.21.15.5:0/1808779515 learned_addr learned my addr 172.21.15.5:0/1808779515 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.208+0000 7fd78a59c700 1 -- 172.21.15.5:0/1808779515 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c07a950 con 0x7fd78c07bdd0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.208+0000 7fd78a59c700 1 --2- 172.21.15.5:0/1808779515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098a0 tx=0x7fd77c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bfd08c732a41589 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.209+0000 7fd78959a700 1 -- 172.21.15.5:0/1808779515 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd77c004160 con 0x7fd78c07bdd0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.209+0000 7fd78959a700 1 -- 172.21.15.5:0/1808779515 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd77c0042c0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.688 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.209+0000 7fd78959a700 1 -- 172.21.15.5:0/1808779515 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd77c011940 con 0x7fd78c07bdd0 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 -- 172.21.15.5:0/1808779515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 msgr2=0x7fd78c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 --2- 172.21.15.5:0/1808779515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fd77c0098a0 tx=0x7fd77c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 -- 172.21.15.5:0/1808779515 shutdown_connections 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 --2- 172.21.15.5:0/1808779515 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c07a2d0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 -- 172.21.15.5:0/1808779515 >> 172.21.15.5:0/1808779515 conn(0x7fd78c100900 msgr2=0x7fd78c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:20.689 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 -- 172.21.15.5:0/1808779515 shutdown_connections 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.210+0000 7fd790843700 1 -- 172.21.15.5:0/1808779515 wait complete. 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.211+0000 7fd790843700 1 Processor -- start 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.211+0000 7fd790843700 1 -- start start 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd790843700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd790843700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c19b6b0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:20.690 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd78a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36888/0 (socket says 172.21.15.5:36888) 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd78a59c700 1 -- 172.21.15.5:0/1503539830 learned_addr learned my addr 172.21.15.5:0/1503539830 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.212+0000 7fd78a59c700 1 -- 172.21.15.5:0/1503539830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd77c009550 con 0x7fd78c07bdd0 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd78a59c700 1 --2- 172.21.15.5:0/1503539830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006af0 tx=0x7fd77c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd77c003bf0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd77c01b8b0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd77c0197e0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.691 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c19b8b0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c19bcd0 con 0x7fd78c07bdd0 2023-07-21T18:57:20.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.213+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd77c019d50 con 0x7fd78c07bdd0 2023-07-21T18:57:20.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.214+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd77c01a710 con 0x7fd78c07bdd0 2023-07-21T18:57:20.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.214+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c061b40 con 0x7fd78c07bdd0 2023-07-21T18:57:20.692 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.217+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7fd77c028060 con 0x7fd78c07bdd0 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.294+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fd78c19e710 con 0x7fd78c07bdd0 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.295+0000 7fd77b7fe700 1 -- 172.21.15.5:0/1503539830 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fd77c01bd70 con 0x7fd78c07bdd0 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.295+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 msgr2=0x7fd78c19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.295+0000 7fd790843700 1 --2- 172.21.15.5:0/1503539830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fd77c006af0 tx=0x7fd77c011a30 comp rx=0 tx=0).stop 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.296+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 shutdown_connections 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.296+0000 7fd790843700 1 --2- 172.21.15.5:0/1503539830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd78c07bdd0 0x7fd78c19b170 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.296+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 >> 172.21.15.5:0/1503539830 conn(0x7fd78c100900 msgr2=0x7fd78c102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:20.693 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.296+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 shutdown_connections 2023-07-21T18:57:20.694 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:20.296+0000 7fd790843700 1 -- 172.21.15.5:0/1503539830 wait complete. 2023-07-21T18:57:20.694 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (1/15)... 2023-07-21T18:57:21.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:20.758+0000 7f7ab1f11200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:57:21.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:20.878+0000 7f7ab1f11200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:57:21.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:21.044+0000 7f7ab1f11200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:57:21.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:21.173+0000 7f7ab1f11200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:57:22.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:21.737+0000 7f7ab1f11200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:57:22.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:22.549+0000 7f7ab1f11200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:57:22.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:22.767+0000 7f7ab1f11200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:57:23.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:23 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/119311989' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:57:23.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:23.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:23.761 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:57:23.762 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:57:23.763 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:57:23.764 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:57:23.770 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:57:23.770 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:57:23.770 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:57:23.770 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:57:23.771 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:57:23.772 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:57:23.773 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:57:04.740231+0000", 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:23.774 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.179+0000 7f9ed3b40700 1 Processor -- start 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.179+0000 7f9ed3b40700 1 -- start start 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.180+0000 7f9ed3b40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.180+0000 7f9ed3b40700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc07a830 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.180+0000 7f9ed18dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.775 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.180+0000 7f9ed18dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36894/0 (socket says 172.21.15.5:36894) 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.180+0000 7f9ed18dc700 1 -- 172.21.15.5:0/2731956181 learned_addr learned my addr 172.21.15.5:0/2731956181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.181+0000 7f9ed18dc700 1 -- 172.21.15.5:0/2731956181 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ecc07a970 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.181+0000 7f9ed18dc700 1 --2- 172.21.15.5:0/2731956181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098a0 tx=0x7f9ec8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8972d1f37d08b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.181+0000 7f9ed08da700 1 -- 172.21.15.5:0/2731956181 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec8004160 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.181+0000 7f9ed08da700 1 -- 172.21.15.5:0/2731956181 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9ec80042c0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.181+0000 7f9ed08da700 1 -- 172.21.15.5:0/2731956181 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec8011940 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.776 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 -- 172.21.15.5:0/2731956181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 msgr2=0x7f9ecc07a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 --2- 172.21.15.5:0/2731956181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9ec80098a0 tx=0x7f9ec8009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 -- 172.21.15.5:0/2731956181 shutdown_connections 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 --2- 172.21.15.5:0/2731956181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc07a2f0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 -- 172.21.15.5:0/2731956181 >> 172.21.15.5:0/2731956181 conn(0x7f9ecc100900 msgr2=0x7f9ecc102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 -- 172.21.15.5:0/2731956181 shutdown_connections 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.182+0000 7f9ed3b40700 1 -- 172.21.15.5:0/2731956181 wait complete. 2023-07-21T18:57:23.777 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.183+0000 7f9ed3b40700 1 Processor -- start 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.183+0000 7f9ed3b40700 1 -- start start 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed3b40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed3b40700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ecc19b6b0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed18dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed18dc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36898/0 (socket says 172.21.15.5:36898) 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed18dc700 1 -- 172.21.15.5:0/119311989 learned_addr learned my addr 172.21.15.5:0/119311989 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:23.778 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.184+0000 7f9ed18dc700 1 -- 172.21.15.5:0/119311989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ec8009550 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ed18dc700 1 --2- 172.21.15.5:0/119311989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:23.779 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec80036a0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f9ec8003c70 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ecc19b8b0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9ec80193f0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.780 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ecc19bcd0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9ec8019920 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.185+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9ec801a3d0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.186+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ecc10b510 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.189+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f9ec8029060 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.265+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f9ecc19ece0 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.265+0000 7f9ec2ffd700 1 -- 172.21.15.5:0/119311989 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f9ec8033090 con 0x7f9ecc07bdf0 2023-07-21T18:57:23.781 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.266+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 msgr2=0x7f9ecc19b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.266+0000 7f9ed3b40700 1 --2- 172.21.15.5:0/119311989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f9ec8000c00 tx=0x7f9ec8011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.267+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 shutdown_connections 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.267+0000 7f9ed3b40700 1 --2- 172.21.15.5:0/119311989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9ecc07bdf0 0x7f9ecc19b170 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.267+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 >> 172.21.15.5:0/119311989 conn(0x7f9ecc100900 msgr2=0x7f9ecc101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.267+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 shutdown_connections 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:23.267+0000 7f9ed3b40700 1 -- 172.21.15.5:0/119311989 wait complete. 2023-07-21T18:57:23.782 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (2/15)... 2023-07-21T18:57:24.148 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:23.785+0000 7f7ab1f11200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:57:24.148 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:23.908+0000 7f7ab1f11200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:57:24.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:24.136+0000 7f7ab1f11200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:57:24.799 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:24.508+0000 7f7ab1f11200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:57:25.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:24.788+0000 7f7ab1f11200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:57:25.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:24.986+0000 7f7ab1f11200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:57:25.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:25.342+0000 7f7ab1f11200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:57:26.590 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:26 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2332378115' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:57:26.591 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:26.319+0000 7f7ab1f11200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:57:26.591 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:26.512+0000 7f7ab1f11200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:57:26.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:26.848 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:57:26.849 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:57:26.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:57:26.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:26.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:57:26.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:57:26.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:26.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:57:26.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:57:26.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:57:26.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:57:26.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:57:26.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:57:26.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:57:26.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:57:26.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:57:26.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": false, 2023-07-21T18:57:26.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:26.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:57:04.740231+0000", 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:26.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.286+0000 7f0bf8215700 1 Processor -- start 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.286+0000 7f0bf8215700 1 -- start start 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.287+0000 7f0bf8215700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.287+0000 7f0bf8215700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf0105b60 con 0x7f0bf0105210 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.287+0000 7f0bf5fb1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.287+0000 7f0bf5fb1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36906/0 (socket says 172.21.15.5:36906) 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.287+0000 7f0bf5fb1700 1 -- 172.21.15.5:0/957666469 learned_addr learned my addr 172.21.15.5:0/957666469 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:26.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.288+0000 7f0bf5fb1700 1 -- 172.21.15.5:0/957666469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf0105ca0 con 0x7f0bf0105210 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.288+0000 7f0bf5fb1700 1 --2- 172.21.15.5:0/957666469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098a0 tx=0x7f0bec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f247b95fbc9b8068 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.288+0000 7f0bf4faf700 1 -- 172.21.15.5:0/957666469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec004160 con 0x7f0bf0105210 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.288+0000 7f0bf4faf700 1 -- 172.21.15.5:0/957666469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0bec0042c0 con 0x7f0bf0105210 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.288+0000 7f0bf4faf700 1 -- 172.21.15.5:0/957666469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec011940 con 0x7f0bf0105210 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 -- 172.21.15.5:0/957666469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 msgr2=0x7f0bf0105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 --2- 172.21.15.5:0/957666469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0bec0098a0 tx=0x7f0bec009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 -- 172.21.15.5:0/957666469 shutdown_connections 2023-07-21T18:57:26.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 --2- 172.21.15.5:0/957666469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf0105620 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 -- 172.21.15.5:0/957666469 >> 172.21.15.5:0/957666469 conn(0x7f0bf0100900 msgr2=0x7f0bf0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 -- 172.21.15.5:0/957666469 shutdown_connections 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.289+0000 7f0bf8215700 1 -- 172.21.15.5:0/957666469 wait complete. 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.290+0000 7f0bf8215700 1 Processor -- start 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.290+0000 7f0bf8215700 1 -- start start 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.290+0000 7f0bf8215700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:26.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.290+0000 7f0bf8215700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf019b640 con 0x7f0bf0105210 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0bf5fb1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0bf5fb1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36922/0 (socket says 172.21.15.5:36922) 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0bf5fb1700 1 -- 172.21.15.5:0/2332378115 learned_addr learned my addr 172.21.15.5:0/2332378115 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0bf5fb1700 1 -- 172.21.15.5:0/2332378115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009550 con 0x7f0bf0105210 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0bf5fb1700 1 --2- 172.21.15.5:0/2332378115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011dc0 tx=0x7f0bec011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec0037d0 con 0x7f0bf0105210 2023-07-21T18:57:26.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.291+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0bec01b440 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.292+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0bec01a560 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.292+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf019b840 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.292+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf019bc60 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.292+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0bec01a9c0 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.292+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0bec0193f0 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.293+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf0061b40 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.295+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72459 (secure 0 0 0) 0x7f0bec029060 con 0x7f0bf0105210 2023-07-21T18:57:26.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.372+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0bf0107070 con 0x7f0bf0105210 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.373+0000 7f0be6ffd700 1 -- 172.21.15.5:0/2332378115 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0bec01f070 con 0x7f0bf0105210 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 msgr2=0x7f0bf019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 --2- 172.21.15.5:0/2332378115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0bec011dc0 tx=0x7f0bec011ea0 comp rx=0 tx=0).stop 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 shutdown_connections 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 --2- 172.21.15.5:0/2332378115 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bf0105210 0x7f0bf019b100 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 >> 172.21.15.5:0/2332378115 conn(0x7f0bf0100900 msgr2=0x7f0bf0101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:26.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 shutdown_connections 2023-07-21T18:57:26.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:26.374+0000 7f0bf8215700 1 -- 172.21.15.5:0/2332378115 wait complete. 2023-07-21T18:57:26.870 INFO:teuthology.orchestra.run.smithi005.stdout:mgr not available, waiting (3/15)... 2023-07-21T18:57:26.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:26.776+0000 7f7ab1f11200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:57:27.229 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:26.932+0000 7f7ab1f11200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:57:27.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:27.217+0000 7f7ab1f11200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:57:28.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:28.320+0000 7f7ab1f11200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:57:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: Activating manager daemon x 2023-07-21T18:57:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: mgrmap e2: x(active, starting, since 0.00431602s) 2023-07-21T18:57:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:57:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: Manager daemon x is now available 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' 2023-07-21T18:57:28.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:57:28.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' 2023-07-21T18:57:28.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:28 smithi005 ceph-mon[126460]: from='mgr.14100 172.21.15.5:0/3859941443' entity='mgr.x' 2023-07-21T18:57:30.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:30.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:30.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "health": { 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-07-21T18:57:30.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 0 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "a" 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "monmap": { 2023-07-21T18:57:30.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-07-21T18:57:30.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-07-21T18:57:30.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-07-21T18:57:30.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-07-21T18:57:30.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-07-21T18:57:30.044 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.045 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-07-21T18:57:30.046 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modules": [ 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "iostat", 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "nfs", 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "restful" 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ], 2023-07-21T18:57:30.047 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "modified": "2023-07-21T18:57:04.740231+0000", 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "services": {} 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout }, 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-07-21T18:57:30.048 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:30.049 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.325+0000 7f0f1d748700 1 Processor -- start 2023-07-21T18:57:30.049 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.325+0000 7f0f1d748700 1 -- start start 2023-07-21T18:57:30.049 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.325+0000 7f0f1d748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:30.049 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.325+0000 7f0f1d748700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18105b60 con 0x7f0f18105210 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.326+0000 7f0f16ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.326+0000 7f0f16ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35164/0 (socket says 172.21.15.5:35164) 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.326+0000 7f0f16ffd700 1 -- 172.21.15.5:0/515433872 learned_addr learned my addr 172.21.15.5:0/515433872 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.326+0000 7f0f16ffd700 1 -- 172.21.15.5:0/515433872 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f18105ca0 con 0x7f0f18105210 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.327+0000 7f0f16ffd700 1 --2- 172.21.15.5:0/515433872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9150454d14accfdb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.331+0000 7f0f15ffb700 1 -- 172.21.15.5:0/515433872 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f08004160 con 0x7f0f18105210 2023-07-21T18:57:30.050 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.331+0000 7f0f15ffb700 1 -- 172.21.15.5:0/515433872 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0f080042c0 con 0x7f0f18105210 2023-07-21T18:57:30.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.331+0000 7f0f15ffb700 1 -- 172.21.15.5:0/515433872 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f08011940 con 0x7f0f18105210 2023-07-21T18:57:30.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 -- 172.21.15.5:0/515433872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 msgr2=0x7f0f18105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:30.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 --2- 172.21.15.5:0/515433872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0f080098a0 tx=0x7f0f08009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:30.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 -- 172.21.15.5:0/515433872 shutdown_connections 2023-07-21T18:57:30.051 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 --2- 172.21.15.5:0/515433872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18105620 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 -- 172.21.15.5:0/515433872 >> 172.21.15.5:0/515433872 conn(0x7f0f18100900 msgr2=0x7f0f18102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 -- 172.21.15.5:0/515433872 shutdown_connections 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.332+0000 7f0f1d748700 1 -- 172.21.15.5:0/515433872 wait complete. 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.333+0000 7f0f1d748700 1 Processor -- start 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.333+0000 7f0f1d748700 1 -- start start 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f1d748700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:30.052 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f1d748700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f18199440 con 0x7f0f18105210 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f16ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f16ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35168/0 (socket says 172.21.15.5:35168) 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f16ffd700 1 -- 172.21.15.5:0/310884563 learned_addr learned my addr 172.21.15.5:0/310884563 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f16ffd700 1 -- 172.21.15.5:0/310884563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f08009550 con 0x7f0f18105210 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.334+0000 7f0f16ffd700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f08011dc0 tx=0x7f0f08011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.335+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f080037d0 con 0x7f0f18105210 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.335+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0f0801b440 con 0x7f0f18105210 2023-07-21T18:57:30.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.335+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0f0801a560 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.335+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f18199640 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.335+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f18199a60 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.336+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 3) v1 ==== 45366+0+0 (secure 0 0 0) 0x7f0f0801b5b0 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.336+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f18061b40 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.336+0000 7f0efffff700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 0x7f0f0003aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.336+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0f0804c2f0 con 0x7f0f18105210 2023-07-21T18:57:30.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.337+0000 7f0f167fc700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 0x7f0f0003aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.337+0000 7f0f167fc700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 0x7f0f0003aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006fd0 tx=0x7f0f0c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.343+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0f08023590 con 0x7f0f18105210 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.629+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0f181070c0 con 0x7f0f18105210 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.629+0000 7f0efffff700 1 -- 172.21.15.5:0/310884563 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7f0f0801f070 con 0x7f0f18105210 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 msgr2=0x7f0f0003aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 0x7f0f0003aa90 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f0f0c006fd0 tx=0x7f0f0c006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:30.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 msgr2=0x7f0f18198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f0f08011dc0 tx=0x7f0f08011ea0 comp rx=0 tx=0).stop 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 shutdown_connections 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f0f000385e0 0x7f0f0003aa90 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 --2- 172.21.15.5:0/310884563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0f18105210 0x7f0f18198f00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 >> 172.21.15.5:0/310884563 conn(0x7f0f18100900 msgr2=0x7f0f18101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 shutdown_connections 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:29.632+0000 7f0f1d748700 1 -- 172.21.15.5:0/310884563 wait complete. 2023-07-21T18:57:30.056 INFO:teuthology.orchestra.run.smithi005.stdout:mgr is available 2023-07-21T18:57:30.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:30 smithi005 ceph-mon[126460]: mgrmap e3: x(active, since 1.00862s) 2023-07-21T18:57:30.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:30 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/310884563' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-07-21T18:57:31.219 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:31.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [global] 2023-07-21T18:57:31.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout fsid = 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:31.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-07-21T18:57:31.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-07-21T18:57:31.220 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [mgr] 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout [osd] 2023-07-21T18:57:31.221 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.526+0000 7fd1f63f3700 1 Processor -- start 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1f63f3700 1 -- start start 2023-07-21T18:57:31.222 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1f63f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1f63f3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f007a870 con 0x7fd1f007be30 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35170/0 (socket says 172.21.15.5:35170) 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.527+0000 7fd1effff700 1 -- 172.21.15.5:0/584324254 learned_addr learned my addr 172.21.15.5:0/584324254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.528+0000 7fd1effff700 1 -- 172.21.15.5:0/584324254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1f007a9b0 con 0x7fd1f007be30 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.528+0000 7fd1effff700 1 --2- 172.21.15.5:0/584324254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00098a0 tx=0x7fd1e0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9a15214d5a596bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:31.223 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.528+0000 7fd1eeffd700 1 -- 172.21.15.5:0/584324254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0004160 con 0x7fd1f007be30 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.528+0000 7fd1eeffd700 1 -- 172.21.15.5:0/584324254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd1e00042c0 con 0x7fd1f007be30 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.528+0000 7fd1eeffd700 1 -- 172.21.15.5:0/584324254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0011940 con 0x7fd1f007be30 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 -- 172.21.15.5:0/584324254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 msgr2=0x7fd1f007a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/584324254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd1e00098a0 tx=0x7fd1e0009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 -- 172.21.15.5:0/584324254 shutdown_connections 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/584324254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f007a330 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:31.224 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 -- 172.21.15.5:0/584324254 >> 172.21.15.5:0/584324254 conn(0x7fd1f0100900 msgr2=0x7fd1f0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 -- 172.21.15.5:0/584324254 shutdown_connections 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.529+0000 7fd1f63f3700 1 -- 172.21.15.5:0/584324254 wait complete. 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.530+0000 7fd1f63f3700 1 Processor -- start 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.530+0000 7fd1f63f3700 1 -- start start 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.530+0000 7fd1f63f3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.530+0000 7fd1f63f3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1f019b6f0 con 0x7fd1f007be30 2023-07-21T18:57:31.225 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35186/0 (socket says 172.21.15.5:35186) 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1effff700 1 -- 172.21.15.5:0/1929264720 learned_addr learned my addr 172.21.15.5:0/1929264720 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1effff700 1 -- 172.21.15.5:0/1929264720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1e0009550 con 0x7fd1f007be30 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1effff700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0006af0 tx=0x7fd1e0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e0003bf0 con 0x7fd1f007be30 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7fd1e001b8b0 con 0x7fd1f007be30 2023-07-21T18:57:31.226 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1f019b8f0 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.531+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd1e00197e0 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.532+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1f019bd10 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.533+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fd1e001ba20 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.533+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1f0061b40 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.533+0000 7fd1ed7fa700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 0x7fd1d803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.533+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd1e004c8b0 con 0x7fd1f007be30 2023-07-21T18:57:31.227 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.533+0000 7fd1ef7fe700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 0x7fd1d803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.534+0000 7fd1ef7fe700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 0x7fd1d803ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006fd0 tx=0x7fd1e4006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.538+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd1e0018530 con 0x7fd1f007be30 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.742+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fd1f019eb90 con 0x7fd1f007be30 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.743+0000 7fd1ed7fa700 1 -- 172.21.15.5:0/1929264720 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7fd1e0018350 con 0x7fd1f007be30 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 msgr2=0x7fd1d803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:31.228 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 0x7fd1d803ab70 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fd1e4006fd0 tx=0x7fd1e4006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 msgr2=0x7fd1f019b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd1e0006af0 tx=0x7fd1e0011a30 comp rx=0 tx=0).stop 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 shutdown_connections 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7fd1d80386c0 0x7fd1d803ab70 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 --2- 172.21.15.5:0/1929264720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd1f007be30 0x7fd1f019b1b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 >> 172.21.15.5:0/1929264720 conn(0x7fd1f0100900 msgr2=0x7fd1f0102020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:31.229 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 shutdown_connections 2023-07-21T18:57:31.230 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:30.746+0000 7fd1f63f3700 1 -- 172.21.15.5:0/1929264720 wait complete. 2023-07-21T18:57:31.230 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling cephadm module... 2023-07-21T18:57:31.464 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:31 smithi005 ceph-mon[126460]: mgrmap e4: x(active, since 2s) 2023-07-21T18:57:31.465 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:31 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1929264720' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-07-21T18:57:32.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2611866628' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-07-21T18:57:32.626 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setuser ceph since I am not root 2023-07-21T18:57:32.626 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setgroup ceph since I am not root 2023-07-21T18:57:32.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d61d2e700 1 Processor -- start 2023-07-21T18:57:32.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d61d2e700 1 -- start start 2023-07-21T18:57:32.901 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d61d2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:32.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d61d2e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c105b50 con 0x7f5d5c105200 2023-07-21T18:57:32.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:32.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35194/0 (socket says 172.21.15.5:35194) 2023-07-21T18:57:32.902 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d5b7fe700 1 -- 172.21.15.5:0/1763222998 learned_addr learned my addr 172.21.15.5:0/1763222998 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:32.905 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.712+0000 7f5d5b7fe700 1 -- 172.21.15.5:0/1763222998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d5c105c90 con 0x7f5d5c105200 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d5b7fe700 1 --2- 172.21.15.5:0/1763222998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098a0 tx=0x7f5d4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8884d36fee99c5f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d5a7fc700 1 -- 172.21.15.5:0/1763222998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d4c004160 con 0x7f5d5c105200 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d5a7fc700 1 -- 172.21.15.5:0/1763222998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5d4c0042c0 con 0x7f5d5c105200 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d5a7fc700 1 -- 172.21.15.5:0/1763222998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d4c011940 con 0x7f5d5c105200 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d61d2e700 1 -- 172.21.15.5:0/1763222998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 msgr2=0x7f5d5c105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:32.906 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/1763222998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c0098a0 tx=0x7f5d4c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d61d2e700 1 -- 172.21.15.5:0/1763222998 shutdown_connections 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/1763222998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c105610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.713+0000 7f5d61d2e700 1 -- 172.21.15.5:0/1763222998 >> 172.21.15.5:0/1763222998 conn(0x7f5d5c100890 msgr2=0x7f5d5c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.714+0000 7f5d61d2e700 1 -- 172.21.15.5:0/1763222998 shutdown_connections 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.714+0000 7f5d61d2e700 1 -- 172.21.15.5:0/1763222998 wait complete. 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d61d2e700 1 Processor -- start 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d61d2e700 1 -- start start 2023-07-21T18:57:32.907 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d61d2e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d61d2e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d5c1993b0 con 0x7f5d5c105200 2023-07-21T18:57:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35204/0 (socket says 172.21.15.5:35204) 2023-07-21T18:57:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.715+0000 7f5d5b7fe700 1 -- 172.21.15.5:0/2611866628 learned_addr learned my addr 172.21.15.5:0/2611866628 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:32.908 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d5b7fe700 1 -- 172.21.15.5:0/2611866628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009550 con 0x7f5d5c105200 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d5b7fe700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c011dc0 tx=0x7f5d4c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d4c0037d0 con 0x7f5d5c105200 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f5d4c01b440 con 0x7f5d5c105200 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d5c1995b0 con 0x7f5d5c105200 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5d4c01a560 con 0x7f5d5c105200 2023-07-21T18:57:32.909 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.716+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d5c1999d0 con 0x7f5d5c105200 2023-07-21T18:57:32.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.717+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f5d4c01b5b0 con 0x7f5d5c105200 2023-07-21T18:57:32.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.717+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d5c109690 con 0x7f5d5c105200 2023-07-21T18:57:32.910 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.718+0000 7f5d58ff9700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 0x7f5d4403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.718+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5d4c04c4a0 con 0x7f5d5c105200 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.718+0000 7f5d5affd700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 0x7f5d4403abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.719+0000 7f5d5affd700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 0x7f5d4403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006fd0 tx=0x7f5d50006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.723+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5d4c023590 con 0x7f5d5c105200 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:31.982+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f5d5c061b90 con 0x7f5d5c105200 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.336+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f5d4c00fda0 con 0x7f5d5c105200 2023-07-21T18:57:32.911 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.338+0000 7f5d58ff9700 1 -- 172.21.15.5:0/2611866628 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f5d4c0199c0 con 0x7f5d5c105200 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 msgr2=0x7f5d4403abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 0x7f5d4403abc0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f5d50006fd0 tx=0x7f5d50006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 msgr2=0x7f5d5c198e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d4c011dc0 tx=0x7f5d4c011ea0 comp rx=0 tx=0).stop 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 shutdown_connections 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f5d44038710 0x7f5d4403abc0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:32.912 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 --2- 172.21.15.5:0/2611866628 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5d5c105200 0x7f5d5c198e70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:32.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.340+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 >> 172.21.15.5:0/2611866628 conn(0x7f5d5c100890 msgr2=0x7f5d5c101620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:32.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.342+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 shutdown_connections 2023-07-21T18:57:32.913 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:32.342+0000 7f5d61d2e700 1 -- 172.21.15.5:0/2611866628 wait complete. 2023-07-21T18:57:32.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:32.614+0000 7f104e961200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:57:32.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:32.721+0000 7f104e961200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:57:32.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:32.839+0000 7f104e961200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:57:33.347 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:32.972+0000 7f104e961200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:57:33.347 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:33.206+0000 7f104e961200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:57:33.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2611866628' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-07-21T18:57:33.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-mon[126460]: mgrmap e5: x(active, since 4s) 2023-07-21T18:57:33.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:33.618+0000 7f104e961200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:57:33.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:33.739+0000 7f104e961200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:57:34.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:34.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-07-21T18:57:34.186 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:57:34.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T18:57:34.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T18:57:34.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:34.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.418+0000 7f306eb04700 1 Processor -- start 2023-07-21T18:57:34.187 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.418+0000 7f306eb04700 1 -- start start 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306eb04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306eb04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068105b60 con 0x7f3068105210 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306c8a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306c8a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35234/0 (socket says 172.21.15.5:35234) 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306c8a0700 1 -- 172.21.15.5:0/3491084529 learned_addr learned my addr 172.21.15.5:0/3491084529 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:34.188 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.419+0000 7f306c8a0700 1 -- 172.21.15.5:0/3491084529 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3068105ca0 con 0x7f3068105210 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f306c8a0700 1 --2- 172.21.15.5:0/3491084529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098a0 tx=0x7f305c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91983918fb3aad1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f30677fe700 1 -- 172.21.15.5:0/3491084529 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c004160 con 0x7f3068105210 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f30677fe700 1 -- 172.21.15.5:0/3491084529 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f305c0042c0 con 0x7f3068105210 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f30677fe700 1 -- 172.21.15.5:0/3491084529 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c011940 con 0x7f3068105210 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f306eb04700 1 -- 172.21.15.5:0/3491084529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 msgr2=0x7f3068105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.420+0000 7f306eb04700 1 --2- 172.21.15.5:0/3491084529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f305c0098a0 tx=0x7f305c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.189 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.421+0000 7f306eb04700 1 -- 172.21.15.5:0/3491084529 shutdown_connections 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.421+0000 7f306eb04700 1 --2- 172.21.15.5:0/3491084529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068105620 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.421+0000 7f306eb04700 1 -- 172.21.15.5:0/3491084529 >> 172.21.15.5:0/3491084529 conn(0x7f3068100900 msgr2=0x7f3068102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.421+0000 7f306eb04700 1 -- 172.21.15.5:0/3491084529 shutdown_connections 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.421+0000 7f306eb04700 1 -- 172.21.15.5:0/3491084529 wait complete. 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.422+0000 7f306eb04700 1 Processor -- start 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306eb04700 1 -- start start 2023-07-21T18:57:34.190 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306eb04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306eb04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068199440 con 0x7f3068105210 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306c8a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306c8a0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35242/0 (socket says 172.21.15.5:35242) 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.423+0000 7f306c8a0700 1 -- 172.21.15.5:0/120051997 learned_addr learned my addr 172.21.15.5:0/120051997 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f306c8a0700 1 -- 172.21.15.5:0/120051997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f305c009550 con 0x7f3068105210 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f306c8a0700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f305c011dc0 tx=0x7f305c011ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c0037d0 con 0x7f3068105210 2023-07-21T18:57:34.191 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f305c01b440 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068199640 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f305c01a560 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.424+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3068199a60 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.425+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3068061b40 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.426+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f305c01a7d0 con 0x7f3068105210 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.426+0000 7f3065ffb700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 0x7f305803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:34.192 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.426+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f305c04c740 con 0x7f3068105210 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.426+0000 7f3067fff700 1 -- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 msgr2=0x7f305803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.426+0000 7f3067fff700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 0x7f305803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.431+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f305c023590 con 0x7f3068105210 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.627+0000 7f3067fff700 1 -- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 msgr2=0x7f305803ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.627+0000 7f3067fff700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 0x7f305803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.691+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f30681070c0 con 0x7f3068105210 2023-07-21T18:57:34.193 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.692+0000 7f3065ffb700 1 -- 172.21.15.5:0/120051997 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f305c0233b0 con 0x7f3068105210 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 msgr2=0x7f305803ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 0x7f305803ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 msgr2=0x7f3068198f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f305c011dc0 tx=0x7f305c011ea0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 shutdown_connections 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f3058038760 0x7f305803ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.194 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 --2- 172.21.15.5:0/120051997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3068105210 0x7f3068198f00 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:34.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 >> 172.21.15.5:0/120051997 conn(0x7f3068100900 msgr2=0x7f3068101fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:34.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 shutdown_connections 2023-07-21T18:57:34.195 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:33.694+0000 7f306eb04700 1 -- 172.21.15.5:0/120051997 wait complete. 2023-07-21T18:57:34.195 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2023-07-21T18:57:34.195 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 5... 2023-07-21T18:57:34.347 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:33.911+0000 7f104e961200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:57:34.347 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:34.037+0000 7f104e961200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:57:34.633 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/120051997' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T18:57:34.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:34.622+0000 7f104e961200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:57:35.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:35.391+0000 7f104e961200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:57:35.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:35.602+0000 7f104e961200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:57:36.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:36.577+0000 7f104e961200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:57:36.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:36.701+0000 7f104e961200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:57:37.307 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:36.928+0000 7f104e961200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:57:37.586 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:37 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:37.297+0000 7f104e961200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:57:37.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:37 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:37.576+0000 7f104e961200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:57:37.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:37 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:37.774+0000 7f104e961200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:57:38.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:38.126+0000 7f104e961200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:57:39.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:39.077+0000 7f104e961200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:57:39.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:39.254+0000 7f104e961200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:57:39.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:39.505+0000 7f104e961200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:57:39.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:39.657+0000 7f104e961200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:57:40.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:39.937+0000 7f104e961200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:57:41.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:41.025+0000 7f104e961200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:57:41.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: Active manager daemon x restarted 2023-07-21T18:57:41.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: Activating manager daemon x 2023-07-21T18:57:41.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: osdmap e2: 0 total, 0 up, 0 in 2023-07-21T18:57:41.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: mgrmap e6: x(active, starting, since 0.00373084s) 2023-07-21T18:57:41.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:57:41.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:57:41.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:57:41.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:57:41.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:57:41.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: Manager daemon x is now available 2023-07-21T18:57:41.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:41.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:41.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:41.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:41 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:42.396 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:57:42.397 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-07-21T18:57:42.397 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T18:57:42.397 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:57:42.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.714+0000 7f6eb58ca700 1 Processor -- start 2023-07-21T18:57:42.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.714+0000 7f6eb58ca700 1 -- start start 2023-07-21T18:57:42.400 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eb58ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eb58ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb007a830 con 0x7f6eb007bdf0 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35248/0 (socket says 172.21.15.5:35248) 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eaeffd700 1 -- 172.21.15.5:0/1117833372 learned_addr learned my addr 172.21.15.5:0/1117833372 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.715+0000 7f6eaeffd700 1 -- 172.21.15.5:0/1117833372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6eb007a970 con 0x7f6eb007bdf0 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.716+0000 7f6eaeffd700 1 --2- 172.21.15.5:0/1117833372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098a0 tx=0x7f6ea0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd8c1c74bd4f0b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:42.401 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.716+0000 7f6eadffb700 1 -- 172.21.15.5:0/1117833372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0004160 con 0x7f6eb007bdf0 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.716+0000 7f6eadffb700 1 -- 172.21.15.5:0/1117833372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6ea00042c0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.716+0000 7f6eadffb700 1 -- 172.21.15.5:0/1117833372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ea0011940 con 0x7f6eb007bdf0 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 -- 172.21.15.5:0/1117833372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 msgr2=0x7f6eb007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/1117833372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6ea00098a0 tx=0x7f6ea0009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 -- 172.21.15.5:0/1117833372 shutdown_connections 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/1117833372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb007a2f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.402 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 -- 172.21.15.5:0/1117833372 >> 172.21.15.5:0/1117833372 conn(0x7f6eb0100900 msgr2=0x7f6eb0102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 -- 172.21.15.5:0/1117833372 shutdown_connections 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.717+0000 7f6eb58ca700 1 -- 172.21.15.5:0/1117833372 wait complete. 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.718+0000 7f6eb58ca700 1 Processor -- start 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.718+0000 7f6eb58ca700 1 -- start start 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.718+0000 7f6eb58ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.718+0000 7f6eb58ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eb019b6b0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.719+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:42.403 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.719+0000 7f6eaeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35262/0 (socket says 172.21.15.5:35262) 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.719+0000 7f6eaeffd700 1 -- 172.21.15.5:0/3504577199 learned_addr learned my addr 172.21.15.5:0/3504577199 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.719+0000 7f6eaeffd700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea0009550 con 0x7f6eb007bdf0 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.719+0000 7f6eaeffd700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0000c00 tx=0x7f6ea0011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.720+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ea00036a0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.720+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6ea0003c70 con 0x7f6eb007bdf0 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.720+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eb019b8b0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.404 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.720+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6ea00193f0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.720+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eb019bcd0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.721+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f6ea0019670 con 0x7f6eb007bdf0 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.721+0000 7f6eb48c8700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.721+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6eb0061b40 con 0x7f6e98038710 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.721+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6ea004c570 con 0x7f6eb007bdf0 2023-07-21T18:57:42.405 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.721+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.722+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.923+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:34.923+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:35.324+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:35.324+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T18:57:42.408 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:36.125+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:36.125+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:37.726+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:37.726+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:40.927+0000 7f6eae7fc700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/474183959 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:40.927+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:41.029+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7f6ea001b440 con 0x7f6eb007bdf0 2023-07-21T18:57:42.409 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:41.029+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 msgr2=0x7f6e9803abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:41.029+0000 7f6eb48c8700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.031+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f6ea004e2a0 con 0x7f6eb007bdf0 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.032+0000 7f6eb48c8700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 0x7f6e9803e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.032+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6eb0061b40 con 0x7f6e9803be30 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.032+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 0x7f6e9803e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.033+0000 7f6eae7fc700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 0x7f6e9803e210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4003a10 tx=0x7f6ea4009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:42.410 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.034+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f6eb0061b40 con 0x7f6e9803be30 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.041+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6eb019c4b0 con 0x7f6e9803be30 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.041+0000 7f6eb48c8700 1 -- 172.21.15.5:0/3504577199 <== mgr.14120 v2:172.21.15.5:6800/1672036156 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6eb019c4b0 con 0x7f6e9803be30 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 msgr2=0x7f6e9803e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 0x7f6e9803e210 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4003a10 tx=0x7f6ea4009290 comp rx=0 tx=0).stop 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 msgr2=0x7f6eb019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0000c00 tx=0x7f6ea0011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 shutdown_connections 2023-07-21T18:57:42.411 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6e9803be30 0x7f6e9803e210 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6eb007bdf0 0x7f6eb019b170 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 --2- 172.21.15.5:0/3504577199 >> [v2:172.21.15.5:6800/474183959,v1:172.21.15.5:6801/474183959] conn(0x7f6e98038710 0x7f6e9803abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 >> 172.21.15.5:0/3504577199 conn(0x7f6eb0100900 msgr2=0x7f6eb0101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 shutdown_connections 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.042+0000 7f6eb58ca700 1 -- 172.21.15.5:0/3504577199 wait complete. 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 5 is available 2023-07-21T18:57:42.412 INFO:teuthology.orchestra.run.smithi005.stdout:Setting orchestrator backend to cephadm... 2023-07-21T18:57:42.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:42 smithi005 ceph-mon[126460]: Found migration_current of "None". Setting to last migration. 2023-07-21T18:57:42.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:42 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:42.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:42 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:57:42.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:42 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:57:42.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:42 smithi005 ceph-mon[126460]: mgrmap e7: x(active, since 1.0068s) 2023-07-21T18:57:43.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:43 smithi005 ceph-mon[126460]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T18:57:43.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:43 smithi005 ceph-mon[126460]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T18:57:43.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:43 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:43.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:43 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:43.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.944+0000 7f1f8dfa7700 1 Processor -- start 2023-07-21T18:57:43.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f8dfa7700 1 -- start start 2023-07-21T18:57:43.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f8dfa7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:43.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f8dfa7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8807a830 con 0x7f1f8807bdf0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56362/0 (socket says 172.21.15.5:56362) 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.945+0000 7f1f877fe700 1 -- 172.21.15.5:0/4196873629 learned_addr learned my addr 172.21.15.5:0/4196873629 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.946+0000 7f1f877fe700 1 -- 172.21.15.5:0/4196873629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f8807a970 con 0x7f1f8807bdf0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.946+0000 7f1f877fe700 1 --2- 172.21.15.5:0/4196873629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8e406930149d9ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.946+0000 7f1f867fc700 1 -- 172.21.15.5:0/4196873629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78004160 con 0x7f1f8807bdf0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.946+0000 7f1f867fc700 1 -- 172.21.15.5:0/4196873629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1f780042c0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.946+0000 7f1f867fc700 1 -- 172.21.15.5:0/4196873629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f78011940 con 0x7f1f8807bdf0 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/4196873629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 msgr2=0x7f1f8807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/4196873629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098a0 tx=0x7f1f78009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/4196873629 shutdown_connections 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/4196873629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8807a2f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/4196873629 >> 172.21.15.5:0/4196873629 conn(0x7f1f88100890 msgr2=0x7f1f88102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/4196873629 shutdown_connections 2023-07-21T18:57:43.584 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.947+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/4196873629 wait complete. 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.948+0000 7f1f8dfa7700 1 Processor -- start 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.948+0000 7f1f8dfa7700 1 -- start start 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f8dfa7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f8dfa7700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8819b5d0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56366/0 (socket says 172.21.15.5:56366) 2023-07-21T18:57:43.585 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f877fe700 1 -- 172.21.15.5:0/3925980884 learned_addr learned my addr 172.21.15.5:0/3925980884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f877fe700 1 -- 172.21.15.5:0/3925980884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009550 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.949+0000 7f1f877fe700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.950+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780036a0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.950+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f1f78003c70 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.950+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8819b7d0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.950+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1f780193f0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.950+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8819bbf0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.586 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.951+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8810b500 con 0x7f1f8807bdf0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.951+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f1f7801b440 con 0x7f1f8807bdf0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.952+0000 7f1f84ff9700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 0x7f1f7003aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.952+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f1f7804c3d0 con 0x7f1f8807bdf0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.952+0000 7f1f86ffd700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 0x7f1f7003aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.953+0000 7f1f86ffd700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 0x7f1f7003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c006fd0 tx=0x7f1f7c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:42.957+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1f78022950 con 0x7f1f8807bdf0 2023-07-21T18:57:43.587 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.177+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f8819ea50 con 0x7f1f70038630 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.185+0000 7f1f84ff9700 1 -- 172.21.15.5:0/3925980884 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f1f8819ea50 con 0x7f1f70038630 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.187+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 msgr2=0x7f1f7003aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.187+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 0x7f1f7003aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c006fd0 tx=0x7f1f7c006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.187+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 msgr2=0x7f1f8819b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.187+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f78000c00 tx=0x7f1f78011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 shutdown_connections 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f1f70038630 0x7f1f7003aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:43.588 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 --2- 172.21.15.5:0/3925980884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1f8807bdf0 0x7f1f8819b090 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:43.589 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 >> 172.21.15.5:0/3925980884 conn(0x7f1f88100890 msgr2=0x7f1f88101610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:43.589 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 shutdown_connections 2023-07-21T18:57:43.589 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:43.188+0000 7f1f8dfa7700 1 -- 172.21.15.5:0/3925980884 wait complete. 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: [21/Jul/2023:18:57:43] ENGINE Bus STARTING 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: [21/Jul/2023:18:57:43] ENGINE Serving on https://172.21.15.5:7150 2023-07-21T18:57:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: [21/Jul/2023:18:57:43] ENGINE Serving on http://172.21.15.5:8765 2023-07-21T18:57:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: [21/Jul/2023:18:57:43] ENGINE Bus STARTED 2023-07-21T18:57:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:44 smithi005 ceph-mon[126460]: mgrmap e8: x(active, since 2s) 2023-07-21T18:57:44.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout value unchanged 2023-07-21T18:57:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.093+0000 7f6446451700 1 Processor -- start 2023-07-21T18:57:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.093+0000 7f6446451700 1 -- start start 2023-07-21T18:57:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f6446451700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f6446451700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644007a810 con 0x7f644007bdd0 2023-07-21T18:57:44.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f643ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:44.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f643ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56382/0 (socket says 172.21.15.5:56382) 2023-07-21T18:57:44.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f643ffff700 1 -- 172.21.15.5:0/3903216983 learned_addr learned my addr 172.21.15.5:0/3903216983 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:44.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.094+0000 7f643ffff700 1 -- 172.21.15.5:0/3903216983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644007a950 con 0x7f644007bdd0 2023-07-21T18:57:44.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f643ffff700 1 --2- 172.21.15.5:0/3903216983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64300098a0 tx=0x7f6430009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=508dada348d034fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:44.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f643effd700 1 -- 172.21.15.5:0/3903216983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6430004160 con 0x7f644007bdd0 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f643effd700 1 -- 172.21.15.5:0/3903216983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f64300042c0 con 0x7f644007bdd0 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f643effd700 1 -- 172.21.15.5:0/3903216983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6430011940 con 0x7f644007bdd0 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f6446451700 1 -- 172.21.15.5:0/3903216983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 msgr2=0x7f644007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f6446451700 1 --2- 172.21.15.5:0/3903216983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f64300098a0 tx=0x7f6430009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.095+0000 7f6446451700 1 -- 172.21.15.5:0/3903216983 shutdown_connections 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.096+0000 7f6446451700 1 --2- 172.21.15.5:0/3903216983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644007a2d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:44.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.096+0000 7f6446451700 1 -- 172.21.15.5:0/3903216983 >> 172.21.15.5:0/3903216983 conn(0x7f6440100900 msgr2=0x7f6440102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.096+0000 7f6446451700 1 -- 172.21.15.5:0/3903216983 shutdown_connections 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.096+0000 7f6446451700 1 -- 172.21.15.5:0/3903216983 wait complete. 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f6446451700 1 Processor -- start 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f6446451700 1 -- start start 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f6446451700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f6446451700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f644019b6b0 con 0x7f644007bdd0 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f643ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:44.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f643ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56394/0 (socket says 172.21.15.5:56394) 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.097+0000 7f643ffff700 1 -- 172.21.15.5:0/2178448747 learned_addr learned my addr 172.21.15.5:0/2178448747 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f643ffff700 1 -- 172.21.15.5:0/2178448747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6430009550 con 0x7f644007bdd0 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f643ffff700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64300036a0 con 0x7f644007bdd0 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f6430003c70 con 0x7f644007bdd0 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f644019b8b0 con 0x7f644007bdd0 2023-07-21T18:57:44.869 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f64300193f0 con 0x7f644007bdd0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.098+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f644019bcd0 con 0x7f644007bdd0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.099+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f644010b4d0 con 0x7f644007bdd0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.099+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f643001b440 con 0x7f644007bdd0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.100+0000 7f643d7fa700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 0x7f642803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.100+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f643004c590 con 0x7f644007bdd0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.100+0000 7f643f7fe700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 0x7f642803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.101+0000 7f643f7fe700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 0x7f642803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6434006fd0 tx=0x7f6434006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:44.870 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.105+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6430022950 con 0x7f644007bdd0 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.313+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f644019eb30 con 0x7f6428038760 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.314+0000 7f643d7fa700 1 -- 172.21.15.5:0/2178448747 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f644019eb30 con 0x7f6428038760 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 msgr2=0x7f642803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 0x7f642803ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6434006fd0 tx=0x7f6434006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 msgr2=0x7f644019b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6430000c00 tx=0x7f6430011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:44.871 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 shutdown_connections 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f6428038760 0x7f642803ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 --2- 172.21.15.5:0/2178448747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f644007bdd0 0x7f644019b170 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 >> 172.21.15.5:0/2178448747 conn(0x7f6440100900 msgr2=0x7f6440102000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.316+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 shutdown_connections 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:44.317+0000 7f6446451700 1 -- 172.21.15.5:0/2178448747 wait complete. 2023-07-21T18:57:44.872 INFO:teuthology.orchestra.run.smithi005.stdout:Generating ssh key... 2023-07-21T18:57:45.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-mon[126460]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:46.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: Generating public/private rsa key pair. 2023-07-21T18:57:46.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: Your identification has been saved in /tmp/tmp00h0ywjm/key. 2023-07-21T18:57:46.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: Your public key has been saved in /tmp/tmp00h0ywjm/key.pub. 2023-07-21T18:57:46.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: The key fingerprint is: 2023-07-21T18:57:46.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: SHA256:p8TuThdy+emJhjNxU8QKA9/V24nTWAgnsW2Ab/7onj4 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: The key's randomart image is: 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: +---[RSA 3072]----+ 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | .. .*+o. | 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | .oo .O... | 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | .oo+ o=o.| 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | . .+o+.o.| 2023-07-21T18:57:46.170 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | S B. . | 2023-07-21T18:57:46.171 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | o.=o+ . | 2023-07-21T18:57:46.171 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | ++..= | 2023-07-21T18:57:46.171 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | o+..E.o | 2023-07-21T18:57:46.171 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: | .o+==+ | 2023-07-21T18:57:46.171 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: +----[SHA256]-----+ 2023-07-21T18:57:46.245 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d807b1700 1 Processor -- start 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d807b1700 1 -- start start 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d807b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d807b1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7807a830 con 0x7f0d7807bdf0 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d7e54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d7e54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56400/0 (socket says 172.21.15.5:56400) 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.460+0000 7f0d7e54d700 1 -- 172.21.15.5:0/3797579642 learned_addr learned my addr 172.21.15.5:0/3797579642 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:46.246 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.461+0000 7f0d7e54d700 1 -- 172.21.15.5:0/3797579642 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7807a970 con 0x7f0d7807bdf0 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.461+0000 7f0d7e54d700 1 --2- 172.21.15.5:0/3797579642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098a0 tx=0x7f0d74009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=605a24847abfc53b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.461+0000 7f0d7d54b700 1 -- 172.21.15.5:0/3797579642 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d74004160 con 0x7f0d7807bdf0 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.461+0000 7f0d7d54b700 1 -- 172.21.15.5:0/3797579642 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0d740042c0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.461+0000 7f0d7d54b700 1 -- 172.21.15.5:0/3797579642 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d74011940 con 0x7f0d7807bdf0 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 -- 172.21.15.5:0/3797579642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 msgr2=0x7f0d7807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 --2- 172.21.15.5:0/3797579642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0d740098a0 tx=0x7f0d74009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:46.247 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 -- 172.21.15.5:0/3797579642 shutdown_connections 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 --2- 172.21.15.5:0/3797579642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7807a2f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 -- 172.21.15.5:0/3797579642 >> 172.21.15.5:0/3797579642 conn(0x7f0d78100900 msgr2=0x7f0d78102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 -- 172.21.15.5:0/3797579642 shutdown_connections 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.462+0000 7f0d807b1700 1 -- 172.21.15.5:0/3797579642 wait complete. 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.463+0000 7f0d807b1700 1 Processor -- start 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.463+0000 7f0d807b1700 1 -- start start 2023-07-21T18:57:46.248 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d807b1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d807b1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d7819b6b0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d7e54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d7e54d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56408/0 (socket says 172.21.15.5:56408) 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d7e54d700 1 -- 172.21.15.5:0/1380940209 learned_addr learned my addr 172.21.15.5:0/1380940209 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d7e54d700 1 -- 172.21.15.5:0/1380940209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d74009550 con 0x7f0d7807bdf0 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.464+0000 7f0d7e54d700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d74011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.465+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d740036a0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.249 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.465+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0d74003c70 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.465+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d740193f0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.465+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d7819b8b0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.465+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d7819bcd0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.466+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0d7401b440 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.466+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d7810b510 con 0x7f0d7807bdf0 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.466+0000 7f0d6f7fe700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 0x7f0d6403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:46.250 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.466+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0d7404c5d0 con 0x7f0d7807bdf0 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.467+0000 7f0d7dd4c700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 0x7f0d6403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.467+0000 7f0d7dd4c700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 0x7f0d6403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006fd0 tx=0x7f0d68006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.472+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0d74022950 con 0x7f0d7807bdf0 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.678+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f0d7819eb30 con 0x7f0d64038760 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.851+0000 7f0d6f7fe700 1 -- 172.21.15.5:0/1380940209 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0d7819eb30 con 0x7f0d64038760 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 msgr2=0x7f0d6403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:46.251 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 0x7f0d6403ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f0d68006fd0 tx=0x7f0d68006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 msgr2=0x7f0d7819b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000c00 tx=0x7f0d74011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 shutdown_connections 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0d64038760 0x7f0d6403ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 --2- 172.21.15.5:0/1380940209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d7807bdf0 0x7f0d7819b170 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 >> 172.21.15.5:0/1380940209 conn(0x7f0d78100900 msgr2=0x7f0d78101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:46.252 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 shutdown_connections 2023-07-21T18:57:46.253 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:45.854+0000 7f0d807b1700 1 -- 172.21.15.5:0/1380940209 wait complete. 2023-07-21T18:57:47.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:46 smithi005 ceph-mon[126460]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:47.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:46 smithi005 ceph-mon[126460]: Generating ssh key... 2023-07-21T18:57:47.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:46 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:47.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:46 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:47.424 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.783+0000 7f461fdbe700 1 Processor -- start 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.783+0000 7f461fdbe700 1 -- start start 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461fdbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461fdbe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461807a830 con 0x7f461807bdf0 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461db5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461db5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44952/0 (socket says 172.21.15.5:44952) 2023-07-21T18:57:47.425 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461db5a700 1 -- 172.21.15.5:0/3978621591 learned_addr learned my addr 172.21.15.5:0/3978621591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.784+0000 7f461db5a700 1 -- 172.21.15.5:0/3978621591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f461807a970 con 0x7f461807bdf0 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461db5a700 1 --2- 172.21.15.5:0/3978621591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f433b1028a893ed9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461cb58700 1 -- 172.21.15.5:0/3978621591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614004160 con 0x7f461807bdf0 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461cb58700 1 -- 172.21.15.5:0/3978621591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f46140042c0 con 0x7f461807bdf0 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461cb58700 1 -- 172.21.15.5:0/3978621591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4614011940 con 0x7f461807bdf0 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461fdbe700 1 -- 172.21.15.5:0/3978621591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 msgr2=0x7f461807a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:47.426 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461fdbe700 1 --2- 172.21.15.5:0/3978621591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f46140098a0 tx=0x7f4614009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461fdbe700 1 -- 172.21.15.5:0/3978621591 shutdown_connections 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461fdbe700 1 --2- 172.21.15.5:0/3978621591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461807a2f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.785+0000 7f461fdbe700 1 -- 172.21.15.5:0/3978621591 >> 172.21.15.5:0/3978621591 conn(0x7f4618100900 msgr2=0x7f4618102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.786+0000 7f461fdbe700 1 -- 172.21.15.5:0/3978621591 shutdown_connections 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.786+0000 7f461fdbe700 1 -- 172.21.15.5:0/3978621591 wait complete. 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461fdbe700 1 Processor -- start 2023-07-21T18:57:47.427 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461fdbe700 1 -- start start 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461fdbe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461fdbe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f461819b6b0 con 0x7f461807bdf0 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461db5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461db5a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44962/0 (socket says 172.21.15.5:44962) 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.787+0000 7f461db5a700 1 -- 172.21.15.5:0/1344408039 learned_addr learned my addr 172.21.15.5:0/1344408039 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f461db5a700 1 -- 172.21.15.5:0/1344408039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4614009550 con 0x7f461807bdf0 2023-07-21T18:57:47.428 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f461db5a700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46140036a0 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4614003c70 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f46140193f0 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f461819b8b0 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.788+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f461819bcd0 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.790+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f461401b440 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.790+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4618061b40 con 0x7f461807bdf0 2023-07-21T18:57:47.429 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.790+0000 7f460effd700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 0x7f460403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:47.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.790+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f461404c510 con 0x7f461807bdf0 2023-07-21T18:57:47.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.790+0000 7f461d359700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 0x7f460403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:47.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.791+0000 7f461d359700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 0x7f460403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:47.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:46.795+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4614022950 con 0x7f461807bdf0 2023-07-21T18:57:47.430 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.003+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f4618102c10 con 0x7f4604038760 2023-07-21T18:57:47.431 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.003+0000 7f460effd700 1 -- 172.21.15.5:0/1344408039 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f4618102c10 con 0x7f4604038760 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 msgr2=0x7f460403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 0x7f460403ac10 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f4608006fd0 tx=0x7f4608006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 msgr2=0x7f461819b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4614000c00 tx=0x7f4614011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 shutdown_connections 2023-07-21T18:57:47.432 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4604038760 0x7f460403ac10 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 --2- 172.21.15.5:0/1344408039 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f461807bdf0 0x7f461819b170 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 >> 172.21.15.5:0/1344408039 conn(0x7f4618100900 msgr2=0x7f4618101ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 shutdown_connections 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.006+0000 7f461fdbe700 1 -- 172.21.15.5:0/1344408039 wait complete. 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:Adding key to root@localhost authorized_keys... 2023-07-21T18:57:47.433 INFO:teuthology.orchestra.run.smithi005.stdout:Adding host smithi005... 2023-07-21T18:57:48.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:48 smithi005 ceph-mon[126460]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:49.313 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:49 smithi005 ceph-mon[126460]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:50.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:50 smithi005 ceph-mon[126460]: Deploying cephadm binary to smithi005 2023-07-21T18:57:51.814 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Added host 'smithi005' with addr '172.21.15.5' 2023-07-21T18:57:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.935+0000 7f024b24a700 1 Processor -- start 2023-07-21T18:57:51.815 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.935+0000 7f024b24a700 1 -- start start 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.935+0000 7f024b24a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.935+0000 7f024b24a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024407a830 con 0x7f024407bdf0 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.936+0000 7f0248fe6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.936+0000 7f0248fe6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44978/0 (socket says 172.21.15.5:44978) 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.936+0000 7f0248fe6700 1 -- 172.21.15.5:0/918873762 learned_addr learned my addr 172.21.15.5:0/918873762 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.936+0000 7f0248fe6700 1 -- 172.21.15.5:0/918873762 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024407a970 con 0x7f024407bdf0 2023-07-21T18:57:51.816 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.936+0000 7f0248fe6700 1 --2- 172.21.15.5:0/918873762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02380098a0 tx=0x7f0238009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa34a820c5454f53 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f02437fe700 1 -- 172.21.15.5:0/918873762 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0238004160 con 0x7f024407bdf0 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f02437fe700 1 -- 172.21.15.5:0/918873762 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f02380042c0 con 0x7f024407bdf0 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f02437fe700 1 -- 172.21.15.5:0/918873762 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0238011940 con 0x7f024407bdf0 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 -- 172.21.15.5:0/918873762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 msgr2=0x7f024407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 --2- 172.21.15.5:0/918873762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f02380098a0 tx=0x7f0238009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:51.817 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 -- 172.21.15.5:0/918873762 shutdown_connections 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 --2- 172.21.15.5:0/918873762 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024407a2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 -- 172.21.15.5:0/918873762 >> 172.21.15.5:0/918873762 conn(0x7f0244100890 msgr2=0x7f0244102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 -- 172.21.15.5:0/918873762 shutdown_connections 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.937+0000 7f024b24a700 1 -- 172.21.15.5:0/918873762 wait complete. 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.938+0000 7f024b24a700 1 Processor -- start 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f024b24a700 1 -- start start 2023-07-21T18:57:51.818 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f024b24a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f024b24a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024419b5d0 con 0x7f024407bdf0 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f0248fe6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f0248fe6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44988/0 (socket says 172.21.15.5:44988) 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f0248fe6700 1 -- 172.21.15.5:0/101857072 learned_addr learned my addr 172.21.15.5:0/101857072 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.939+0000 7f0248fe6700 1 -- 172.21.15.5:0/101857072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0238009550 con 0x7f024407bdf0 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f0248fe6700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:51.819 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02380036a0 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024419b7d0 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024419bbf0 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f0238003c70 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.940+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02380193f0 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.941+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0244061b40 con 0x7f024407bdf0 2023-07-21T18:57:51.820 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.941+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0238019550 con 0x7f024407bdf0 2023-07-21T18:57:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.942+0000 7f0241ffb700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 0x7f023403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.942+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f023804c770 con 0x7f024407bdf0 2023-07-21T18:57:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.942+0000 7f0243fff700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 0x7f023403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:51.821 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.943+0000 7f0243fff700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 0x7f023403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0230006fd0 tx=0x7f0230006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:51.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:47.947+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0238018b00 con 0x7f024407bdf0 2023-07-21T18:57:51.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:48.161+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi005", "addr": "172.21.15.5", "target": ["mon-mgr", ""]}) v1 -- 0x7f0244101d20 con 0x7f0234038760 2023-07-21T18:57:51.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.421+0000 7f0241ffb700 1 -- 172.21.15.5:0/101857072 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0244101d20 con 0x7f0234038760 2023-07-21T18:57:51.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 msgr2=0x7f023403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:51.822 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 0x7f023403ac10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0230006fd0 tx=0x7f0230006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:51.824 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 msgr2=0x7f024419b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:51.824 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f0238000c00 tx=0x7f0238011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:51.825 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 shutdown_connections 2023-07-21T18:57:51.825 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f0234038760 0x7f023403ac10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:51.825 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 --2- 172.21.15.5:0/101857072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f024407bdf0 0x7f024419b090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:51.825 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.424+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 >> 172.21.15.5:0/101857072 conn(0x7f0244100890 msgr2=0x7f0244101610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:51.825 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.425+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 shutdown_connections 2023-07-21T18:57:51.826 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:51.425+0000 7f024b24a700 1 -- 172.21.15.5:0/101857072 wait complete. 2023-07-21T18:57:51.826 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mon service... 2023-07-21T18:57:52.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:52 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:52.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:52 smithi005 ceph-mon[126460]: Added host smithi005 2023-07-21T18:57:52.689 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:52 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:57:52.887 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-07-21T18:57:52.891 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.315+0000 7f44550bc700 1 Processor -- start 2023-07-21T18:57:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f44550bc700 1 -- start start 2023-07-21T18:57:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f44550bc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f44550bc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445007a810 con 0x7f445007bdd0 2023-07-21T18:57:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f444ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:52.892 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f444ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44998/0 (socket says 172.21.15.5:44998) 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.316+0000 7f444ed9d700 1 -- 172.21.15.5:0/3391991782 learned_addr learned my addr 172.21.15.5:0/3391991782 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.317+0000 7f444ed9d700 1 -- 172.21.15.5:0/3391991782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f445007a950 con 0x7f445007bdd0 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.317+0000 7f444ed9d700 1 --2- 172.21.15.5:0/3391991782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44400098a0 tx=0x7f4440009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17b0b014c32dff06 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.317+0000 7f444dd9b700 1 -- 172.21.15.5:0/3391991782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4440004160 con 0x7f445007bdd0 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.317+0000 7f444dd9b700 1 -- 172.21.15.5:0/3391991782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f44400042c0 con 0x7f445007bdd0 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.317+0000 7f444dd9b700 1 -- 172.21.15.5:0/3391991782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4440011940 con 0x7f445007bdd0 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 -- 172.21.15.5:0/3391991782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 msgr2=0x7f445007a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:52.893 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 --2- 172.21.15.5:0/3391991782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f44400098a0 tx=0x7f4440009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 -- 172.21.15.5:0/3391991782 shutdown_connections 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 --2- 172.21.15.5:0/3391991782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445007a2d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 -- 172.21.15.5:0/3391991782 >> 172.21.15.5:0/3391991782 conn(0x7f4450100890 msgr2=0x7f4450102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 -- 172.21.15.5:0/3391991782 shutdown_connections 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.318+0000 7f44550bc700 1 -- 172.21.15.5:0/3391991782 wait complete. 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.319+0000 7f44550bc700 1 Processor -- start 2023-07-21T18:57:52.894 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.319+0000 7f44550bc700 1 -- start start 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.319+0000 7f44550bc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f44550bc700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f445019b5d0 con 0x7f445007bdd0 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f444ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f444ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45012/0 (socket says 172.21.15.5:45012) 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f444ed9d700 1 -- 172.21.15.5:0/496776863 learned_addr learned my addr 172.21.15.5:0/496776863 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f444ed9d700 1 -- 172.21.15.5:0/496776863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4440009550 con 0x7f445007bdd0 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.320+0000 7f444ed9d700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:52.895 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.321+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44400036a0 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.321+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f4440003c70 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.321+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f44400193f0 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.321+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f445019b7d0 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.321+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f445019bbf0 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.322+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f444001b440 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.322+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4450061b40 con 0x7f445007bdd0 2023-07-21T18:57:52.896 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.322+0000 7f4437fff700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 0x7f443803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.322+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f444004c5d0 con 0x7f445007bdd0 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.323+0000 7f444e59c700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 0x7f443803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.323+0000 7f444e59c700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 0x7f443803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4444006fd0 tx=0x7f4444006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.328+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4440022950 con 0x7f445007bdd0 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.546+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f4450101d60 con 0x7f4438038760 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.552+0000 7f4437fff700 1 -- 172.21.15.5:0/496776863 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f4450101d60 con 0x7f4438038760 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 msgr2=0x7f443803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:52.897 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 0x7f443803ac10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4444006fd0 tx=0x7f4444006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 msgr2=0x7f445019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4440000c00 tx=0x7f4440011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 shutdown_connections 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f4438038760 0x7f443803ac10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 --2- 172.21.15.5:0/496776863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f445007bdd0 0x7f445019b090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 >> 172.21.15.5:0/496776863 conn(0x7f4450100890 msgr2=0x7f4450101650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:52.898 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 shutdown_connections 2023-07-21T18:57:52.899 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:52.555+0000 7f44550bc700 1 -- 172.21.15.5:0/496776863 wait complete. 2023-07-21T18:57:52.899 INFO:teuthology.orchestra.run.smithi005.stdout:Deploying unmanaged mgr service... 2023-07-21T18:57:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:53 smithi005 ceph-mon[126460]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:53 smithi005 ceph-mon[126460]: Saving service mon spec with placement count:5 2023-07-21T18:57:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:53 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:54.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-07-21T18:57:54.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.458+0000 7f8285df6700 1 Processor -- start 2023-07-21T18:57:54.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.458+0000 7f8285df6700 1 -- start start 2023-07-21T18:57:54.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.458+0000 7f8285df6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.458+0000 7f8285df6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828007a830 con 0x7f828007bdf0 2023-07-21T18:57:54.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.459+0000 7f827f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.459+0000 7f827f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45020/0 (socket says 172.21.15.5:45020) 2023-07-21T18:57:54.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.459+0000 7f827f7fe700 1 -- 172.21.15.5:0/94155322 learned_addr learned my addr 172.21.15.5:0/94155322 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:54.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.459+0000 7f827f7fe700 1 -- 172.21.15.5:0/94155322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f828007a970 con 0x7f828007bdf0 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.459+0000 7f827f7fe700 1 --2- 172.21.15.5:0/94155322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f82700098a0 tx=0x7f8270009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e8a37bf8e7cc513 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f827e7fc700 1 -- 172.21.15.5:0/94155322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8270004160 con 0x7f828007bdf0 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f827e7fc700 1 -- 172.21.15.5:0/94155322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f82700042c0 con 0x7f828007bdf0 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f827e7fc700 1 -- 172.21.15.5:0/94155322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8270011940 con 0x7f828007bdf0 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 -- 172.21.15.5:0/94155322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 msgr2=0x7f828007a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 --2- 172.21.15.5:0/94155322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f82700098a0 tx=0x7f8270009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 -- 172.21.15.5:0/94155322 shutdown_connections 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 --2- 172.21.15.5:0/94155322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828007a2f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 -- 172.21.15.5:0/94155322 >> 172.21.15.5:0/94155322 conn(0x7f8280100890 msgr2=0x7f8280102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 -- 172.21.15.5:0/94155322 shutdown_connections 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.460+0000 7f8285df6700 1 -- 172.21.15.5:0/94155322 wait complete. 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.461+0000 7f8285df6700 1 Processor -- start 2023-07-21T18:57:54.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f8285df6700 1 -- start start 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f8285df6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f8285df6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f828019b5d0 con 0x7f828007bdf0 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f827f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f827f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45036/0 (socket says 172.21.15.5:45036) 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.462+0000 7f827f7fe700 1 -- 172.21.15.5:0/1895272795 learned_addr learned my addr 172.21.15.5:0/1895272795 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f827f7fe700 1 -- 172.21.15.5:0/1895272795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8270009550 con 0x7f828007bdf0 2023-07-21T18:57:54.283 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f827f7fe700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8270000c00 tx=0x7f8270011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82700036a0 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1036+0+0 (secure 0 0 0) 0x7f8270003c70 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f82700193f0 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f828019b7d0 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.463+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f828019bbf0 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.465+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f828010b500 con 0x7f828007bdf0 2023-07-21T18:57:54.284 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.465+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f827001b440 con 0x7f828007bdf0 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.465+0000 7f827cff9700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 0x7f826803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.465+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f827004c5d0 con 0x7f828007bdf0 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.465+0000 7f827effd700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 0x7f826803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.466+0000 7f827effd700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 0x7f826803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.470+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8270022950 con 0x7f828007bdf0 2023-07-21T18:57:54.285 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.697+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f828019ea50 con 0x7f8268038760 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.701+0000 7f827cff9700 1 -- 172.21.15.5:0/1895272795 <== mgr.14120 v2:172.21.15.5:6800/1672036156 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f828019ea50 con 0x7f8268038760 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 msgr2=0x7f826803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 0x7f826803ac10 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f8274006fd0 tx=0x7f8274006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 msgr2=0x7f828019b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8270000c00 tx=0x7f8270011e70 comp rx=0 tx=0).stop 2023-07-21T18:57:54.286 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 shutdown_connections 2023-07-21T18:57:54.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8268038760 0x7f826803ac10 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 --2- 172.21.15.5:0/1895272795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f828007bdf0 0x7f828019b090 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:54.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 >> 172.21.15.5:0/1895272795 conn(0x7f8280100890 msgr2=0x7f8280101610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:54.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 shutdown_connections 2023-07-21T18:57:54.287 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:53.704+0000 7f8285df6700 1 -- 172.21.15.5:0/1895272795 wait complete. 2023-07-21T18:57:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:54 smithi005 ceph-mon[126460]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:57:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:54 smithi005 ceph-mon[126460]: Saving service mgr spec with placement count:2 2023-07-21T18:57:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:54 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:54 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:55.569 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.856+0000 7f58093d6700 1 Processor -- start 2023-07-21T18:57:55.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.856+0000 7f58093d6700 1 -- start start 2023-07-21T18:57:55.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.856+0000 7f58093d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.570 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.856+0000 7f58093d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580407a850 con 0x7f580407be10 2023-07-21T18:57:55.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.857+0000 7f5802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.857+0000 7f5802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45050/0 (socket says 172.21.15.5:45050) 2023-07-21T18:57:55.571 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.857+0000 7f5802ffd700 1 -- 172.21.15.5:0/2356103907 learned_addr learned my addr 172.21.15.5:0/2356103907 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:55.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.857+0000 7f5802ffd700 1 -- 172.21.15.5:0/2356103907 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f580407a990 con 0x7f580407be10 2023-07-21T18:57:55.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.857+0000 7f5802ffd700 1 --2- 172.21.15.5:0/2356103907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57f40098a0 tx=0x7f57f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861fcc208aa7f804 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f5801ffb700 1 -- 172.21.15.5:0/2356103907 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f4004160 con 0x7f580407be10 2023-07-21T18:57:55.572 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f5801ffb700 1 -- 172.21.15.5:0/2356103907 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57f40042c0 con 0x7f580407be10 2023-07-21T18:57:55.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f5801ffb700 1 -- 172.21.15.5:0/2356103907 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f40118e0 con 0x7f580407be10 2023-07-21T18:57:55.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 -- 172.21.15.5:0/2356103907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 msgr2=0x7f580407a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.573 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 --2- 172.21.15.5:0/2356103907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f57f40098a0 tx=0x7f57f4009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 -- 172.21.15.5:0/2356103907 shutdown_connections 2023-07-21T18:57:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 --2- 172.21.15.5:0/2356103907 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580407a310 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 -- 172.21.15.5:0/2356103907 >> 172.21.15.5:0/2356103907 conn(0x7f5804100900 msgr2=0x7f5804102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.574 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.858+0000 7f58093d6700 1 -- 172.21.15.5:0/2356103907 shutdown_connections 2023-07-21T18:57:55.575 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.859+0000 7f58093d6700 1 -- 172.21.15.5:0/2356103907 wait complete. 2023-07-21T18:57:55.575 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.859+0000 7f58093d6700 1 Processor -- start 2023-07-21T18:57:55.575 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f58093d6700 1 -- start start 2023-07-21T18:57:55.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f58093d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f58093d6700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f580419b750 con 0x7f580407be10 2023-07-21T18:57:55.576 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f5802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f5802ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45064/0 (socket says 172.21.15.5:45064) 2023-07-21T18:57:55.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f5802ffd700 1 -- 172.21.15.5:0/3236033744 learned_addr learned my addr 172.21.15.5:0/3236033744 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:55.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.860+0000 7f5802ffd700 1 -- 172.21.15.5:0/3236033744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009550 con 0x7f580407be10 2023-07-21T18:57:55.577 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f5802ffd700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f40036a0 con 0x7f580407be10 2023-07-21T18:57:55.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57f4024440 con 0x7f580407be10 2023-07-21T18:57:55.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f580419b950 con 0x7f580407be10 2023-07-21T18:57:55.578 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f57f401a430 con 0x7f580407be10 2023-07-21T18:57:55.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.861+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f580419bd70 con 0x7f580407be10 2023-07-21T18:57:55.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.862+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58041951c0 con 0x7f580407be10 2023-07-21T18:57:55.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.862+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f57f4024820 con 0x7f580407be10 2023-07-21T18:57:55.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.863+0000 7f57ebfff700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 0x7f57ec03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:55.579 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.863+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f57f404c300 con 0x7f580407be10 2023-07-21T18:57:55.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.863+0000 7f58027fc700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 0x7f57ec03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:55.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.864+0000 7f58027fc700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 0x7f57ec03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006fd0 tx=0x7f57f8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:55.580 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:54.868+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f57f4023c20 con 0x7f580407be10 2023-07-21T18:57:55.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.076+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f580402cce0 con 0x7f580407be10 2023-07-21T18:57:55.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.079+0000 7f57ebfff700 1 -- 172.21.15.5:0/3236033744 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f57f4023810 con 0x7f580407be10 2023-07-21T18:57:55.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 msgr2=0x7f57ec03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.581 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 0x7f57ec03abc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f57f8006fd0 tx=0x7f57f8006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:55.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 msgr2=0x7f580419b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:55.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f57f4000c00 tx=0x7f57f40119a0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 shutdown_connections 2023-07-21T18:57:55.582 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f57ec038710 0x7f57ec03abc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 --2- 172.21.15.5:0/3236033744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f580407be10 0x7f580419b210 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:55.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 >> 172.21.15.5:0/3236033744 conn(0x7f5804100900 msgr2=0x7f5804102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:55.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 shutdown_connections 2023-07-21T18:57:55.583 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:55.083+0000 7f58093d6700 1 -- 172.21.15.5:0/3236033744 wait complete. 2023-07-21T18:57:56.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:55 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:56.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:55 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3236033744' entity='client.admin' 2023-07-21T18:57:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.089+0000 7f8cf7c2c700 1 Processor -- start 2023-07-21T18:57:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.089+0000 7f8cf7c2c700 1 -- start start 2023-07-21T18:57:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.090+0000 7f8cf7c2c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.090+0000 7f8cf7c2c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf0105b50 con 0x7f8cf0105200 2023-07-21T18:57:56.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.090+0000 7f8cf59c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.090+0000 7f8cf59c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45080/0 (socket says 172.21.15.5:45080) 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.090+0000 7f8cf59c8700 1 -- 172.21.15.5:0/3728893480 learned_addr learned my addr 172.21.15.5:0/3728893480 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.091+0000 7f8cf59c8700 1 -- 172.21.15.5:0/3728893480 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cf0105c90 con 0x7f8cf0105200 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.091+0000 7f8cf59c8700 1 --2- 172.21.15.5:0/3728893480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098a0 tx=0x7f8cec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e859e2c12908d44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.091+0000 7f8cf49c6700 1 -- 172.21.15.5:0/3728893480 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec004160 con 0x7f8cf0105200 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.091+0000 7f8cf49c6700 1 -- 172.21.15.5:0/3728893480 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cec0042c0 con 0x7f8cf0105200 2023-07-21T18:57:56.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.091+0000 7f8cf49c6700 1 -- 172.21.15.5:0/3728893480 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec0118e0 con 0x7f8cf0105200 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3728893480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 msgr2=0x7f8cf0105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3728893480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8cec0098a0 tx=0x7f8cec009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3728893480 shutdown_connections 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3728893480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf0105610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3728893480 >> 172.21.15.5:0/3728893480 conn(0x7f8cf0100890 msgr2=0x7f8cf0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3728893480 shutdown_connections 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.092+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3728893480 wait complete. 2023-07-21T18:57:56.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.093+0000 7f8cf7c2c700 1 Processor -- start 2023-07-21T18:57:56.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf7c2c700 1 -- start start 2023-07-21T18:57:56.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf7c2c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:56.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf7c2c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cf007c1a0 con 0x7f8cf0105200 2023-07-21T18:57:56.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf59c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:56.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf59c8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45084/0 (socket says 172.21.15.5:45084) 2023-07-21T18:57:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.094+0000 7f8cf59c8700 1 -- 172.21.15.5:0/3048512127 learned_addr learned my addr 172.21.15.5:0/3048512127 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.095+0000 7f8cf59c8700 1 -- 172.21.15.5:0/3048512127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cec009550 con 0x7f8cf0105200 2023-07-21T18:57:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.095+0000 7f8cf59c8700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003e30 tx=0x7f8cec0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:56.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.095+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec0036a0 con 0x7f8cf0105200 2023-07-21T18:57:56.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.095+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8cec01b440 con 0x7f8cf0105200 2023-07-21T18:57:56.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.095+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8cec023400 con 0x7f8cf0105200 2023-07-21T18:57:56.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.096+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cf007a2b0 con 0x7f8cf0105200 2023-07-21T18:57:56.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.096+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cf007a6d0 con 0x7f8cf0105200 2023-07-21T18:57:56.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.098+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cf0061b40 con 0x7f8cf0105200 2023-07-21T18:57:56.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.098+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f8cec01b5b0 con 0x7f8cf0105200 2023-07-21T18:57:56.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.099+0000 7f8ce6ffd700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 0x7f8cdc03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:56.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.099+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8cec04c770 con 0x7f8cf0105200 2023-07-21T18:57:56.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.099+0000 7f8cf51c7700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 0x7f8cdc03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:56.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.100+0000 7f8cf51c7700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 0x7f8cdc03ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006fd0 tx=0x7f8ce0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:56.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.103+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8cec02b9e0 con 0x7f8cf0105200 2023-07-21T18:57:56.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.315+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f8cf007aea0 con 0x7f8cf0105200 2023-07-21T18:57:56.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.319+0000 7f8ce6ffd700 1 -- 172.21.15.5:0/3048512127 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f8cec019ea0 con 0x7f8cf0105200 2023-07-21T18:57:56.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 msgr2=0x7f8cdc03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:56.865 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 0x7f8cdc03ac10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0006fd0 tx=0x7f8ce0006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:56.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 msgr2=0x7f8cf007bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:56.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f8cec003e30 tx=0x7f8cec0119a0 comp rx=0 tx=0).stop 2023-07-21T18:57:56.866 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 shutdown_connections 2023-07-21T18:57:56.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f8cdc038760 0x7f8cdc03ac10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:56.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 --2- 172.21.15.5:0/3048512127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8cf0105200 0x7f8cf007bc60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:56.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 >> 172.21.15.5:0/3048512127 conn(0x7f8cf0100890 msgr2=0x7f8cf0101670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:56.867 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 shutdown_connections 2023-07-21T18:57:56.868 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:56.322+0000 7f8cf7c2c700 1 -- 172.21.15.5:0/3048512127 wait complete. 2023-07-21T18:57:56.868 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling the dashboard module... 2023-07-21T18:57:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:57 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3048512127' entity='client.admin' 2023-07-21T18:57:58.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-mon[126460]: from='mgr.14120 172.21.15.5:0/312164044' entity='mgr.x' 2023-07-21T18:57:58.737 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2403577985' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-07-21T18:57:58.737 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setuser ceph since I am not root 2023-07-21T18:57:58.737 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setgroup ceph since I am not root 2023-07-21T18:57:58.738 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:58.725+0000 7f0109cef200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:57:58.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.503+0000 7ffab78a5700 1 Processor -- start 2023-07-21T18:57:58.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.503+0000 7ffab78a5700 1 -- start start 2023-07-21T18:57:58.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.504+0000 7ffab78a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:58.850 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.504+0000 7ffab78a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab0108180 con 0x7ffab0107830 2023-07-21T18:57:58.851 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.504+0000 7ffab5641700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:58.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.504+0000 7ffab5641700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47654/0 (socket says 172.21.15.5:47654) 2023-07-21T18:57:58.852 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.504+0000 7ffab5641700 1 -- 172.21.15.5:0/1385731744 learned_addr learned my addr 172.21.15.5:0/1385731744 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.505+0000 7ffab5641700 1 -- 172.21.15.5:0/1385731744 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffab01082c0 con 0x7ffab0107830 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.505+0000 7ffab5641700 1 --2- 172.21.15.5:0/1385731744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0098a0 tx=0x7ffaac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af6a9033bc9aca82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.505+0000 7ffaa7fff700 1 -- 172.21.15.5:0/1385731744 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffaac004160 con 0x7ffab0107830 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.505+0000 7ffaa7fff700 1 -- 172.21.15.5:0/1385731744 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffaac0042c0 con 0x7ffab0107830 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.505+0000 7ffaa7fff700 1 -- 172.21.15.5:0/1385731744 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffaac0118e0 con 0x7ffab0107830 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 -- 172.21.15.5:0/1385731744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 msgr2=0x7ffab0107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:58.853 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 --2- 172.21.15.5:0/1385731744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffaac0098a0 tx=0x7ffaac009bb0 comp rx=0 tx=0).stop 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 -- 172.21.15.5:0/1385731744 shutdown_connections 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 --2- 172.21.15.5:0/1385731744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab0107c40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 -- 172.21.15.5:0/1385731744 >> 172.21.15.5:0/1385731744 conn(0x7ffab007a8e0 msgr2=0x7ffab007ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 -- 172.21.15.5:0/1385731744 shutdown_connections 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.506+0000 7ffab78a5700 1 -- 172.21.15.5:0/1385731744 wait complete. 2023-07-21T18:57:58.854 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.507+0000 7ffab78a5700 1 Processor -- start 2023-07-21T18:57:58.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.507+0000 7ffab78a5700 1 -- start start 2023-07-21T18:57:58.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab78a5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:58.855 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab78a5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffab019b630 con 0x7ffab0107830 2023-07-21T18:57:58.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab5641700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:58.857 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab5641700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47670/0 (socket says 172.21.15.5:47670) 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab5641700 1 -- 172.21.15.5:0/2403577985 learned_addr learned my addr 172.21.15.5:0/2403577985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.508+0000 7ffab5641700 1 -- 172.21.15.5:0/2403577985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaac009550 con 0x7ffab0107830 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffab5641700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006af0 tx=0x7ffaac011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffaac0037d0 con 0x7ffab0107830 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffaac01b440 con 0x7ffab0107830 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffab019b830 con 0x7ffab0107830 2023-07-21T18:57:58.858 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffaac0193d0 con 0x7ffab0107830 2023-07-21T18:57:58.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.509+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffab019bc50 con 0x7ffab0107830 2023-07-21T18:57:58.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.510+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffab0061b40 con 0x7ffab0107830 2023-07-21T18:57:58.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.511+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ffaac01b5b0 con 0x7ffab0107830 2023-07-21T18:57:58.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.511+0000 7ffaa67fc700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 0x7ffa9c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:57:58.859 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.511+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ffaac04c5e0 con 0x7ffab0107830 2023-07-21T18:57:58.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.511+0000 7ffab4e40700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 0x7ffa9c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:57:58.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.512+0000 7ffab4e40700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 0x7ffa9c03abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0006fd0 tx=0x7ffaa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:57:58.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.517+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffaac02b530 con 0x7ffab0107830 2023-07-21T18:57:58.860 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:57.784+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7ffab019ecf0 con 0x7ffab0107830 2023-07-21T18:57:58.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.406+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7ffaac01b960 con 0x7ffab0107830 2023-07-21T18:57:58.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.407+0000 7ffaa67fc700 1 -- 172.21.15.5:0/2403577985 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7ffaac04cad0 con 0x7ffab0107830 2023-07-21T18:57:58.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 msgr2=0x7ffa9c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:58.861 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 0x7ffa9c03abc0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0006fd0 tx=0x7ffaa0006e40 comp rx=0 tx=0).stop 2023-07-21T18:57:58.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 msgr2=0x7ffab019b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:57:58.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ffaac006af0 tx=0x7ffaac011e80 comp rx=0 tx=0).stop 2023-07-21T18:57:58.862 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 shutdown_connections 2023-07-21T18:57:58.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7ffa9c038710 0x7ffa9c03abc0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:58.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 --2- 172.21.15.5:0/2403577985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffab0107830 0x7ffab019b0f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:57:58.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 >> 172.21.15.5:0/2403577985 conn(0x7ffab007a8e0 msgr2=0x7ffab0106060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:57:58.863 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 shutdown_connections 2023-07-21T18:57:58.864 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:58.411+0000 7ffab78a5700 1 -- 172.21.15.5:0/2403577985 wait complete. 2023-07-21T18:57:59.112 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:58.833+0000 7f0109cef200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:57:59.112 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:58.970+0000 7f0109cef200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:57:59.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:59.100+0000 7f0109cef200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:57:59.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:59.338+0000 7f0109cef200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:57:59.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2403577985' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-07-21T18:57:59.680 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-mon[126460]: mgrmap e9: x(active, since 17s) 2023-07-21T18:58:00.056 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:59.753+0000 7f0109cef200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:58:00.057 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:57:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:57:59.873+0000 7f0109cef200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:58:00.144 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:58:00.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-07-21T18:58:00.148 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "available": true, 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.367+0000 7f3df027a700 1 Processor -- start 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3df027a700 1 -- start start 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3df027a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:00.149 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3df027a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de807a850 con 0x7f3de807be10 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3dee016700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3dee016700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47690/0 (socket says 172.21.15.5:47690) 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.368+0000 7f3dee016700 1 -- 172.21.15.5:0/1374213315 learned_addr learned my addr 172.21.15.5:0/1374213315 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.369+0000 7f3dee016700 1 -- 172.21.15.5:0/1374213315 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de807a990 con 0x7f3de807be10 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.369+0000 7f3dee016700 1 --2- 172.21.15.5:0/1374213315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3de40098a0 tx=0x7f3de4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912820187e083105 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.369+0000 7f3ded014700 1 -- 172.21.15.5:0/1374213315 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de4004160 con 0x7f3de807be10 2023-07-21T18:58:00.150 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.369+0000 7f3ded014700 1 -- 172.21.15.5:0/1374213315 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3de40042c0 con 0x7f3de807be10 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.369+0000 7f3ded014700 1 -- 172.21.15.5:0/1374213315 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de40118e0 con 0x7f3de807be10 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 -- 172.21.15.5:0/1374213315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 msgr2=0x7f3de807a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 --2- 172.21.15.5:0/1374213315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3de40098a0 tx=0x7f3de4009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 -- 172.21.15.5:0/1374213315 shutdown_connections 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 --2- 172.21.15.5:0/1374213315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de807a310 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 -- 172.21.15.5:0/1374213315 >> 172.21.15.5:0/1374213315 conn(0x7f3de8100900 msgr2=0x7f3de8102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 -- 172.21.15.5:0/1374213315 shutdown_connections 2023-07-21T18:58:00.151 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.370+0000 7f3df027a700 1 -- 172.21.15.5:0/1374213315 wait complete. 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.371+0000 7f3df027a700 1 Processor -- start 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3df027a700 1 -- start start 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3df027a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3df027a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3de819b780 con 0x7f3de807be10 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3dee016700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3dee016700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47692/0 (socket says 172.21.15.5:47692) 2023-07-21T18:58:00.152 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3dee016700 1 -- 172.21.15.5:0/668296605 learned_addr learned my addr 172.21.15.5:0/668296605 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.372+0000 7f3dee016700 1 -- 172.21.15.5:0/668296605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de4009550 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3dee016700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de40119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de40036a0 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3de4024440 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3de819b980 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3de401a430 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.373+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3de819bda0 con 0x7f3de807be10 2023-07-21T18:58:00.153 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.374+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3de4003800 con 0x7f3de807be10 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.374+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3de81951c0 con 0x7f3de807be10 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.375+0000 7f3ddf7fe700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 0x7f3dd403ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.375+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3de404c2c0 con 0x7f3de807be10 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.375+0000 7f3ded815700 1 -- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 msgr2=0x7f3dd403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.375+0000 7f3ded815700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 0x7f3dd403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.380+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3de40239f0 con 0x7f3de807be10 2023-07-21T18:58:00.154 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.576+0000 7f3ded815700 1 -- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 msgr2=0x7f3dd403ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.576+0000 7f3ded815700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 0x7f3dd403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.644+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3de8061b40 con 0x7f3de807be10 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.644+0000 7f3ddf7fe700 1 -- 172.21.15.5:0/668296605 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3de40233a0 con 0x7f3de807be10 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 msgr2=0x7f3dd403ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 0x7f3dd403ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 msgr2=0x7f3de819b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3de4000c00 tx=0x7f3de40119a0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.155 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 shutdown_connections 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7f3dd40387b0 0x7f3dd403ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 --2- 172.21.15.5:0/668296605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3de807be10 0x7f3de819b240 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 >> 172.21.15.5:0/668296605 conn(0x7f3de8100900 msgr2=0x7f3de8102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 shutdown_connections 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:57:59.647+0000 7f3df027a700 1 -- 172.21.15.5:0/668296605 wait complete. 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for the mgr to restart... 2023-07-21T18:58:00.156 INFO:teuthology.orchestra.run.smithi005.stdout:Waiting for mgr epoch 9... 2023-07-21T18:58:00.370 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:00.045+0000 7f0109cef200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:58:00.370 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:00.177+0000 7f0109cef200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:58:00.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:00 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/668296605' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-07-21T18:58:01.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:00.761+0000 7f0109cef200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:58:01.920 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:01.530+0000 7f0109cef200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:58:01.920 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:01.743+0000 7f0109cef200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:58:03.087 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:02.723+0000 7f0109cef200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:58:03.087 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:02.847+0000 7f0109cef200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:58:03.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:03.075+0000 7f0109cef200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:58:03.748 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:03.448+0000 7f0109cef200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:58:04.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:03.736+0000 7f0109cef200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:58:04.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:03.935+0000 7f0109cef200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:58:04.668 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:04.291+0000 7f0109cef200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:58:05.668 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:05.248+0000 7f0109cef200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:58:05.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:05.428+0000 7f0109cef200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:58:06.128 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:05.682+0000 7f0109cef200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:58:06.129 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:05.834+0000 7f0109cef200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:58:06.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:06.117+0000 7f0109cef200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:58:07.669 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:58:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:58:07.216+0000 7f0109cef200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:58:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: Active manager daemon x restarted 2023-07-21T18:58:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: Activating manager daemon x 2023-07-21T18:58:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: osdmap e3: 0 total, 0 up, 0 in 2023-07-21T18:58:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: mgrmap e10: x(active, starting, since 0.00438455s) 2023-07-21T18:58:08.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: Manager daemon x is now available 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:08.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:08.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T18:58:08.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T18:58:08.650 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout { 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout "initialized": true 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout } 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.658+0000 7fab2a040700 1 Processor -- start 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.659+0000 7fab2a040700 1 -- start start 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.659+0000 7fab2a040700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.651 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.659+0000 7fab2a040700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2407a830 con 0x7fab2407bdf0 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47706/0 (socket says 172.21.15.5:47706) 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab237fe700 1 -- 172.21.15.5:0/2341930299 learned_addr learned my addr 172.21.15.5:0/2341930299 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab237fe700 1 -- 172.21.15.5:0/2341930299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab2407a970 con 0x7fab2407bdf0 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab237fe700 1 --2- 172.21.15.5:0/2341930299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab140098a0 tx=0x7fab14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b045399c60807ebd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.652 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab227fc700 1 -- 172.21.15.5:0/2341930299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab14004160 con 0x7fab2407bdf0 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab227fc700 1 -- 172.21.15.5:0/2341930299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab140042c0 con 0x7fab2407bdf0 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.660+0000 7fab227fc700 1 -- 172.21.15.5:0/2341930299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab140118e0 con 0x7fab2407bdf0 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 -- 172.21.15.5:0/2341930299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 msgr2=0x7fab2407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 --2- 172.21.15.5:0/2341930299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fab140098a0 tx=0x7fab14009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 -- 172.21.15.5:0/2341930299 shutdown_connections 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 --2- 172.21.15.5:0/2341930299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2407a2f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.653 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 -- 172.21.15.5:0/2341930299 >> 172.21.15.5:0/2341930299 conn(0x7fab24100900 msgr2=0x7fab24102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 -- 172.21.15.5:0/2341930299 shutdown_connections 2023-07-21T18:58:08.654 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.661+0000 7fab2a040700 1 -- 172.21.15.5:0/2341930299 wait complete. 2023-07-21T18:58:08.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.662+0000 7fab2a040700 1 Processor -- start 2023-07-21T18:58:08.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab2a040700 1 -- start start 2023-07-21T18:58:08.655 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab2a040700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.656 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab2a040700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2419b740 con 0x7fab2407bdf0 2023-07-21T18:58:08.656 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.656 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47710/0 (socket says 172.21.15.5:47710) 2023-07-21T18:58:08.657 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab237fe700 1 -- 172.21.15.5:0/2894695330 learned_addr learned my addr 172.21.15.5:0/2894695330 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:08.657 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.663+0000 7fab237fe700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab14009550 con 0x7fab2407bdf0 2023-07-21T18:58:08.657 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab237fe700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab14003e30 tx=0x7fab140119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab140036a0 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab1401b440 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2419b940 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fab14023400 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.664+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2419bd60 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.665+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fab14023630 con 0x7fab2407bdf0 2023-07-21T18:58:08.658 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.666+0000 7fab20ff9700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.666+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab24061b40 con 0x7fab0c0387b0 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.666+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fab1404c9c0 con 0x7fab2407bdf0 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.666+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.666+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.867+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:00.867+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T18:58:08.659 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:01.268+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:01.268+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:02.069+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:02.069+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:03.670+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:03.670+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T18:58:08.660 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:06.872+0000 7fab22ffd700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1672036156 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:06.872+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:07.220+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fab1402c910 con 0x7fab2407bdf0 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:07.220+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 msgr2=0x7fab0c03ac60 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:07.221+0000 7fab20ff9700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.223+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fab1402ad30 con 0x7fab2407bdf0 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.223+0000 7fab20ff9700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 0x7fab0c03e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:08.661 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.223+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fab24061b40 con 0x7fab0c03bfc0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.224+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 0x7fab0c03e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.225+0000 7fab22ffd700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 0x7fab0c03e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab18003a10 tx=0x7fab18009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.226+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fab24061b40 con 0x7fab0c03bfc0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.233+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fab2419c540 con 0x7fab0c03bfc0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.233+0000 7fab20ff9700 1 -- 172.21.15.5:0/2894695330 <== mgr.14152 v2:172.21.15.5:6800/1627438252 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fab2419c540 con 0x7fab0c03bfc0 2023-07-21T18:58:08.662 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 msgr2=0x7fab0c03e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 0x7fab0c03e3a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fab18003a10 tx=0x7fab18009270 comp rx=0 tx=0).stop 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 msgr2=0x7fab2419b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fab14003e30 tx=0x7fab140119a0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 shutdown_connections 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fab0c03bfc0 0x7fab0c03e3a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab2407bdf0 0x7fab2419b200 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.663 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 --2- 172.21.15.5:0/2894695330 >> [v2:172.21.15.5:6800/1672036156,v1:172.21.15.5:6801/1672036156] conn(0x7fab0c0387b0 0x7fab0c03ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:08.664 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 >> 172.21.15.5:0/2894695330 conn(0x7fab24100900 msgr2=0x7fab24102020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:08.664 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 shutdown_connections 2023-07-21T18:58:08.664 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:08.234+0000 7fab2a040700 1 -- 172.21.15.5:0/2894695330 wait complete. 2023-07-21T18:58:08.664 INFO:teuthology.orchestra.run.smithi005.stdout:mgr epoch 9 is available 2023-07-21T18:58:08.664 INFO:teuthology.orchestra.run.smithi005.stdout:Generating a dashboard self-signed certificate... 2023-07-21T18:58:09.620 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: [21/Jul/2023:18:58:08] ENGINE Bus STARTING 2023-07-21T18:58:09.620 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: mgrmap e11: x(active, since 1.00796s) 2023-07-21T18:58:09.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-07-21T18:58:09.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-07-21T18:58:09.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: [21/Jul/2023:18:58:08] ENGINE Serving on http://172.21.15.5:8765 2023-07-21T18:58:09.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: [21/Jul/2023:18:58:08] ENGINE Serving on https://172.21.15.5:7150 2023-07-21T18:58:09.621 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:09 smithi005 ceph-mon[126460]: [21/Jul/2023:18:58:08] ENGINE Bus STARTED 2023-07-21T18:58:10.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-07-21T18:58:10.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.243+0000 7fec23510700 1 Processor -- start 2023-07-21T18:58:10.033 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.243+0000 7fec23510700 1 -- start start 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.243+0000 7fec23510700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.243+0000 7fec23510700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c07a850 con 0x7fec1c07be10 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.244+0000 7fec212ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.244+0000 7fec212ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55694/0 (socket says 172.21.15.5:55694) 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.244+0000 7fec212ac700 1 -- 172.21.15.5:0/113843171 learned_addr learned my addr 172.21.15.5:0/113843171 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.244+0000 7fec212ac700 1 -- 172.21.15.5:0/113843171 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec1c07a990 con 0x7fec1c07be10 2023-07-21T18:58:10.034 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec212ac700 1 --2- 172.21.15.5:0/113843171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec180098a0 tx=0x7fec18009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891555e78b22a393 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec13fff700 1 -- 172.21.15.5:0/113843171 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec18004160 con 0x7fec1c07be10 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec13fff700 1 -- 172.21.15.5:0/113843171 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec180042c0 con 0x7fec1c07be10 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec13fff700 1 -- 172.21.15.5:0/113843171 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec180118e0 con 0x7fec1c07be10 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 -- 172.21.15.5:0/113843171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 msgr2=0x7fec1c07a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 --2- 172.21.15.5:0/113843171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fec180098a0 tx=0x7fec18009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 -- 172.21.15.5:0/113843171 shutdown_connections 2023-07-21T18:58:10.035 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 --2- 172.21.15.5:0/113843171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c07a310 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 -- 172.21.15.5:0/113843171 >> 172.21.15.5:0/113843171 conn(0x7fec1c100900 msgr2=0x7fec1c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.245+0000 7fec23510700 1 -- 172.21.15.5:0/113843171 shutdown_connections 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.246+0000 7fec23510700 1 -- 172.21.15.5:0/113843171 wait complete. 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec23510700 1 Processor -- start 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec23510700 1 -- start start 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec23510700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:10.036 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec23510700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec1c19b780 con 0x7fec1c07be10 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec212ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec212ac700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55704/0 (socket says 172.21.15.5:55704) 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.247+0000 7fec212ac700 1 -- 172.21.15.5:0/2892888205 learned_addr learned my addr 172.21.15.5:0/2892888205 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec212ac700 1 -- 172.21.15.5:0/2892888205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec18009550 con 0x7fec1c07be10 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec212ac700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec18000c00 tx=0x7fec180119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec180036a0 con 0x7fec1c07be10 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec18024440 con 0x7fec1c07be10 2023-07-21T18:58:10.037 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec1801a430 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec1c19b980 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.248+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec1c19bda0 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.249+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec1c1951c0 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.250+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fec18024820 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.250+0000 7fec127fc700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 0x7fec0803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.250+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fec1804c650 con 0x7fec1c07be10 2023-07-21T18:58:10.038 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.251+0000 7fec20aab700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 0x7fec0803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.251+0000 7fec20aab700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 0x7fec0803acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006fd0 tx=0x7fec0c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.255+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fec180245b0 con 0x7fec1c07be10 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.497+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fec1c060950 con 0x7fec08038800 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.584+0000 7fec127fc700 1 -- 172.21.15.5:0/2892888205 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fec1c060950 con 0x7fec08038800 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 msgr2=0x7fec0803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 0x7fec0803acb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fec0c006fd0 tx=0x7fec0c006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:10.039 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 msgr2=0x7fec1c19b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fec18000c00 tx=0x7fec180119a0 comp rx=0 tx=0).stop 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 shutdown_connections 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fec08038800 0x7fec0803acb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 --2- 172.21.15.5:0/2892888205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec1c07be10 0x7fec1c19b240 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 >> 172.21.15.5:0/2892888205 conn(0x7fec1c100900 msgr2=0x7fec1c102080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 shutdown_connections 2023-07-21T18:58:10.040 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:09.587+0000 7fec23510700 1 -- 172.21.15.5:0/2892888205 wait complete. 2023-07-21T18:58:10.041 INFO:teuthology.orchestra.run.smithi005.stdout:Creating initial admin user... 2023-07-21T18:58:10.287 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:10 smithi005 ceph-mon[126460]: mgrmap e12: x(active, since 2s) 2023-07-21T18:58:10.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:10 smithi005 ceph-mon[126460]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:10.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:10.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:10.288 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:11.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Jv3E2sXAsSAcqQIGZwAZZ.xaSU5fRixMtFKaxP3d1f4057HF36n46", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1689965891, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-07-21T18:58:11.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.566+0000 7f1d90edf700 1 Processor -- start 2023-07-21T18:58:11.520 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.566+0000 7f1d90edf700 1 -- start start 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.567+0000 7f1d90edf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.567+0000 7f1d90edf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c105b60 con 0x7f1d8c105210 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.567+0000 7f1d8a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.567+0000 7f1d8a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55708/0 (socket says 172.21.15.5:55708) 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.567+0000 7f1d8a59c700 1 -- 172.21.15.5:0/1583928537 learned_addr learned my addr 172.21.15.5:0/1583928537 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.568+0000 7f1d8a59c700 1 -- 172.21.15.5:0/1583928537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d8c105ca0 con 0x7f1d8c105210 2023-07-21T18:58:11.521 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.568+0000 7f1d8a59c700 1 --2- 172.21.15.5:0/1583928537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098a0 tx=0x7f1d7c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7633f81842fc0213 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.568+0000 7f1d8959a700 1 -- 172.21.15.5:0/1583928537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c004160 con 0x7f1d8c105210 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.568+0000 7f1d8959a700 1 -- 172.21.15.5:0/1583928537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d7c0042c0 con 0x7f1d8c105210 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.568+0000 7f1d8959a700 1 -- 172.21.15.5:0/1583928537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c0118e0 con 0x7f1d8c105210 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 -- 172.21.15.5:0/1583928537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 msgr2=0x7f1d8c105620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 --2- 172.21.15.5:0/1583928537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0098a0 tx=0x7f1d7c009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:11.522 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 -- 172.21.15.5:0/1583928537 shutdown_connections 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 --2- 172.21.15.5:0/1583928537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c105620 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 -- 172.21.15.5:0/1583928537 >> 172.21.15.5:0/1583928537 conn(0x7f1d8c100900 msgr2=0x7f1d8c102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 -- 172.21.15.5:0/1583928537 shutdown_connections 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.569+0000 7f1d90edf700 1 -- 172.21.15.5:0/1583928537 wait complete. 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.570+0000 7f1d90edf700 1 Processor -- start 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.570+0000 7f1d90edf700 1 -- start start 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d90edf700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:11.523 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d90edf700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c07c1a0 con 0x7f1d8c105210 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d8a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d8a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55724/0 (socket says 172.21.15.5:55724) 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d8a59c700 1 -- 172.21.15.5:0/2099679349 learned_addr learned my addr 172.21.15.5:0/2099679349 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.571+0000 7f1d8a59c700 1 -- 172.21.15.5:0/2099679349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d7c009550 con 0x7f1d8c105210 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d8a59c700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c003e30 tx=0x7f1d7c0119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c0036a0 con 0x7f1d8c105210 2023-07-21T18:58:11.524 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d7c01b440 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c07a2b0 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c023400 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.572+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c07a6d0 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.573+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f1d7c003800 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.574+0000 7f1d7b7fe700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 0x7f1d7403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.574+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c061b40 con 0x7f1d8c105210 2023-07-21T18:58:11.525 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.574+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f1d7c04c830 con 0x7f1d8c105210 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.574+0000 7f1d89d9b700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 0x7f1d7403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.575+0000 7f1d89d9b700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 0x7f1d7403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006fd0 tx=0x7f1d80006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.579+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1d7c0197f0 con 0x7f1d8c105210 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:10.833+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1d8c061960 con 0x7f1d74038760 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.080+0000 7f1d7b7fe700 1 -- 172.21.15.5:0/2099679349 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1d8c061960 con 0x7f1d74038760 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 msgr2=0x7f1d7403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 0x7f1d7403ac10 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1d80006fd0 tx=0x7f1d80006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:11.526 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 msgr2=0x7f1d8c07bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c003e30 tx=0x7f1d7c0119a0 comp rx=0 tx=0).stop 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 shutdown_connections 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1d74038760 0x7f1d7403ac10 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 --2- 172.21.15.5:0/2099679349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d8c105210 0x7f1d8c07bc60 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 >> 172.21.15.5:0/2099679349 conn(0x7f1d8c100900 msgr2=0x7f1d8c102060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 shutdown_connections 2023-07-21T18:58:11.527 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:11.083+0000 7f1d90edf700 1 -- 172.21.15.5:0/2099679349 wait complete. 2023-07-21T18:58:11.528 INFO:teuthology.orchestra.run.smithi005.stdout:Fetching dashboard port number... 2023-07-21T18:58:12.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:11 smithi005 ceph-mon[126460]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:12.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:12.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stdout 8443 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.074+0000 7fa9d6609700 1 Processor -- start 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.074+0000 7fa9d6609700 1 -- start start 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.074+0000 7fa9d6609700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.074+0000 7fa9d6609700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0105b50 con 0x7fa9d0105200 2023-07-21T18:58:13.053 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.075+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.075+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55730/0 (socket says 172.21.15.5:55730) 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.075+0000 7fa9cffff700 1 -- 172.21.15.5:0/810447392 learned_addr learned my addr 172.21.15.5:0/810447392 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.075+0000 7fa9cffff700 1 -- 172.21.15.5:0/810447392 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d0105c90 con 0x7fa9d0105200 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.075+0000 7fa9cffff700 1 --2- 172.21.15.5:0/810447392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00098a0 tx=0x7fa9c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6465ed942e0b1802 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9ceffd700 1 -- 172.21.15.5:0/810447392 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c0004160 con 0x7fa9d0105200 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9ceffd700 1 -- 172.21.15.5:0/810447392 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9c00042c0 con 0x7fa9d0105200 2023-07-21T18:58:13.054 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9ceffd700 1 -- 172.21.15.5:0/810447392 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c00118e0 con 0x7fa9d0105200 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 -- 172.21.15.5:0/810447392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 msgr2=0x7fa9d0105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 --2- 172.21.15.5:0/810447392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9c00098a0 tx=0x7fa9c0009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 -- 172.21.15.5:0/810447392 shutdown_connections 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 --2- 172.21.15.5:0/810447392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0105610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 -- 172.21.15.5:0/810447392 >> 172.21.15.5:0/810447392 conn(0x7fa9d0100890 msgr2=0x7fa9d0102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 -- 172.21.15.5:0/810447392 shutdown_connections 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.076+0000 7fa9d6609700 1 -- 172.21.15.5:0/810447392 wait complete. 2023-07-21T18:58:13.055 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.077+0000 7fa9d6609700 1 Processor -- start 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9d6609700 1 -- start start 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9d6609700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9d6609700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d0199450 con 0x7fa9d0105200 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55734/0 (socket says 172.21.15.5:55734) 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9cffff700 1 -- 172.21.15.5:0/998681133 learned_addr learned my addr 172.21.15.5:0/998681133 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:13.056 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.078+0000 7fa9cffff700 1 -- 172.21.15.5:0/998681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9c0009550 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9cffff700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006af0 tx=0x7fa9c0011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c00037d0 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa9c001b440 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d0199650 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9c0023400 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.079+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d0199a70 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.080+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa9c001b5b0 con 0x7fa9d0105200 2023-07-21T18:58:13.057 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.080+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d0061b40 con 0x7fa9d0105200 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.081+0000 7fa9cd7fa700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 0x7fa9b803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.081+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9c004c8c0 con 0x7fa9d0105200 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.081+0000 7fa9cf7fe700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 0x7fa9b803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.082+0000 7fa9cf7fe700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 0x7fa9b803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.085+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa9c001b8d0 con 0x7fa9d0105200 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.322+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa9d019c8d0 con 0x7fa9d0105200 2023-07-21T18:58:13.058 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.322+0000 7fa9cd7fa700 1 -- 172.21.15.5:0/998681133 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa9c000bbd0 con 0x7fa9d0105200 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 msgr2=0x7fa9b803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 0x7fa9b803ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9c4006fd0 tx=0x7fa9c4006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 msgr2=0x7fa9d0198f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0006af0 tx=0x7fa9c0011ed0 comp rx=0 tx=0).stop 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 shutdown_connections 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa9b8038760 0x7fa9b803ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 --2- 172.21.15.5:0/998681133 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa9d0105200 0x7fa9d0198f10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:13.059 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 >> 172.21.15.5:0/998681133 conn(0x7fa9d0100890 msgr2=0x7fa9d0101670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 shutdown_connections 2023-07-21T18:58:13.060 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:12.325+0000 7fa9d6609700 1 -- 172.21.15.5:0/998681133 wait complete. 2023-07-21T18:58:13.072 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-07-21T18:58:13.072 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout disabled 2023-07-21T18:58:13.089 INFO:teuthology.orchestra.run.smithi005.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-07-21T18:58:13.089 INFO:teuthology.orchestra.run.smithi005.stdout:systemctl: stdout inactive 2023-07-21T18:58:13.089 INFO:teuthology.orchestra.run.smithi005.stdout:firewalld.service is not enabled 2023-07-21T18:58:13.089 INFO:teuthology.orchestra.run.smithi005.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-07-21T18:58:13.091 INFO:teuthology.orchestra.run.smithi005.stdout:Ceph Dashboard is now available at: 2023-07-21T18:58:13.092 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:13.092 INFO:teuthology.orchestra.run.smithi005.stdout: URL: https://smithi005.front.sepia.ceph.com:8443/ 2023-07-21T18:58:13.092 INFO:teuthology.orchestra.run.smithi005.stdout: User: admin 2023-07-21T18:58:13.092 INFO:teuthology.orchestra.run.smithi005.stdout: Password: uhvymw9c4v 2023-07-21T18:58:13.092 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:13.093 INFO:teuthology.orchestra.run.smithi005.stdout:Saving cluster configuration to /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config directory 2023-07-21T18:58:13.093 INFO:teuthology.orchestra.run.smithi005.stdout:Enabling autotune for osd_memory_target 2023-07-21T18:58:13.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:13 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/998681133' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-07-21T18:58:14.274 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.606+0000 7f59149a8700 1 Processor -- start 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.606+0000 7f59149a8700 1 -- start start 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.607+0000 7f59149a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.607+0000 7f59149a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5910105b50 con 0x7f5910105200 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.607+0000 7f590e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.607+0000 7f590e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55746/0 (socket says 172.21.15.5:55746) 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.607+0000 7f590e59c700 1 -- 172.21.15.5:0/3349666328 learned_addr learned my addr 172.21.15.5:0/3349666328 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:14.275 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f590e59c700 1 -- 172.21.15.5:0/3349666328 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5910105c90 con 0x7f5910105200 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f590e59c700 1 --2- 172.21.15.5:0/3349666328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59000098a0 tx=0x7f5900009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6253077fe8d81674 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f590d59a700 1 -- 172.21.15.5:0/3349666328 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5900004160 con 0x7f5910105200 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f590d59a700 1 -- 172.21.15.5:0/3349666328 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59000042c0 con 0x7f5910105200 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f590d59a700 1 -- 172.21.15.5:0/3349666328 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59000118e0 con 0x7f5910105200 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.608+0000 7f59149a8700 1 -- 172.21.15.5:0/3349666328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 msgr2=0x7f5910105610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 --2- 172.21.15.5:0/3349666328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f59000098a0 tx=0x7f5900009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:14.276 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 -- 172.21.15.5:0/3349666328 shutdown_connections 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 --2- 172.21.15.5:0/3349666328 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f5910105610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 -- 172.21.15.5:0/3349666328 >> 172.21.15.5:0/3349666328 conn(0x7f5910100890 msgr2=0x7f5910102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 -- 172.21.15.5:0/3349666328 shutdown_connections 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.609+0000 7f59149a8700 1 -- 172.21.15.5:0/3349666328 wait complete. 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f59149a8700 1 Processor -- start 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f59149a8700 1 -- start start 2023-07-21T18:58:14.277 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f59149a8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f59149a8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591019b640 con 0x7f5910105200 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f590e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f590e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55762/0 (socket says 172.21.15.5:55762) 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.610+0000 7f590e59c700 1 -- 172.21.15.5:0/3608232132 learned_addr learned my addr 172.21.15.5:0/3608232132 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f590e59c700 1 -- 172.21.15.5:0/3608232132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5900009550 con 0x7f5910105200 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f590e59c700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5900003e30 tx=0x7f59000119a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:14.278 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f59000036a0 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f591019b840 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f590001b440 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5900023400 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.611+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f591019bc60 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.612+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5910061b40 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.612+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f590001b5b0 con 0x7f5910105200 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.613+0000 7f58ff7fe700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 0x7f58f803ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:14.279 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.613+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f590004c860 con 0x7f5910105200 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.613+0000 7f590dd9b700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 0x7f58f803ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.614+0000 7f590dd9b700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 0x7f58f803ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.618+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5900019c30 con 0x7f5910105200 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.823+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f591019eac0 con 0x7f5910105200 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.823+0000 7f58ff7fe700 1 -- 172.21.15.5:0/3608232132 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f5900019840 con 0x7f5910105200 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 msgr2=0x7f58f803ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:14.280 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 0x7f58f803ac60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5904006fd0 tx=0x7f5904006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 msgr2=0x7f591019b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5900003e30 tx=0x7f59000119a0 comp rx=0 tx=0).stop 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 shutdown_connections 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58f80387b0 0x7f58f803ac60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.825+0000 7f59149a8700 1 --2- 172.21.15.5:0/3608232132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5910105200 0x7f591019b100 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.826+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 >> 172.21.15.5:0/3608232132 conn(0x7f5910100890 msgr2=0x7f5910101630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:14.281 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.826+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 shutdown_connections 2023-07-21T18:58:14.282 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:13.826+0000 7f59149a8700 1 -- 172.21.15.5:0/3608232132 wait complete. 2023-07-21T18:58:15.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:15 smithi005 ceph-mon[126460]: mgrmap e13: x(active, since 6s) 2023-07-21T18:58:15.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:15 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/878896416' entity='client.admin' 2023-07-21T18:58:15.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.747+0000 7f32e5e04700 1 Processor -- start 2023-07-21T18:58:15.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.747+0000 7f32e5e04700 1 -- start start 2023-07-21T18:58:15.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.747+0000 7f32e5e04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:15.555 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.747+0000 7f32e5e04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e007a850 con 0x7f32e007be10 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32e4e02700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32e4e02700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55770/0 (socket says 172.21.15.5:55770) 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32e4e02700 1 -- 172.21.15.5:0/3048118990 learned_addr learned my addr 172.21.15.5:0/3048118990 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32e4e02700 1 -- 172.21.15.5:0/3048118990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e007a990 con 0x7f32e007be10 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32e4e02700 1 --2- 172.21.15.5:0/3048118990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32d40098a0 tx=0x7f32d4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52b3427b22d783fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.748+0000 7f32df7fe700 1 -- 172.21.15.5:0/3048118990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d4004160 con 0x7f32e007be10 2023-07-21T18:58:15.556 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32df7fe700 1 -- 172.21.15.5:0/3048118990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32d40042c0 con 0x7f32e007be10 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32df7fe700 1 -- 172.21.15.5:0/3048118990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d40118e0 con 0x7f32e007be10 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 -- 172.21.15.5:0/3048118990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 msgr2=0x7f32e007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 --2- 172.21.15.5:0/3048118990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f32d40098a0 tx=0x7f32d4009bb0 comp rx=0 tx=0).stop 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 -- 172.21.15.5:0/3048118990 shutdown_connections 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 --2- 172.21.15.5:0/3048118990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e007a310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 -- 172.21.15.5:0/3048118990 >> 172.21.15.5:0/3048118990 conn(0x7f32e01006c0 msgr2=0x7f32e0102ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:15.557 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 -- 172.21.15.5:0/3048118990 shutdown_connections 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.749+0000 7f32e5e04700 1 -- 172.21.15.5:0/3048118990 wait complete. 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.750+0000 7f32e5e04700 1 Processor -- start 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.750+0000 7f32e5e04700 1 -- start start 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e5e04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e5e04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e019b590 con 0x7f32e007be10 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e4e02700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:15.558 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e4e02700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55784/0 (socket says 172.21.15.5:55784) 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e4e02700 1 -- 172.21.15.5:0/878896416 learned_addr learned my addr 172.21.15.5:0/878896416 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.751+0000 7f32e4e02700 1 -- 172.21.15.5:0/878896416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32d4009550 con 0x7f32e007be10 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32e4e02700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006af0 tx=0x7f32d4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d4011990 con 0x7f32e007be10 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32d4003b30 con 0x7f32e007be10 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e019b790 con 0x7f32e007be10 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f32d401ad70 con 0x7f32e007be10 2023-07-21T18:58:15.559 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.752+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e019bbb0 con 0x7f32e007be10 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.753+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e0195000 con 0x7f32e007be10 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.753+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f32d40036a0 con 0x7f32e007be10 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.754+0000 7f32ddffb700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 0x7f32d003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.754+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f32d404c600 con 0x7f32e007be10 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.754+0000 7f32dffff700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 0x7f32d003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.754+0000 7f32dffff700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 0x7f32d003acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:15.560 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:14.758+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f32d4023b80 con 0x7f32e007be10 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.073+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f32e002cce0 con 0x7f32e007be10 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.076+0000 7f32ddffb700 1 -- 172.21.15.5:0/878896416 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) v1 ==== 153+0+0 (secure 0 0 0) 0x7f32d4023b80 con 0x7f32e007be10 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 msgr2=0x7f32d003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 0x7f32d003acb0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f32cc006fd0 tx=0x7f32cc006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 msgr2=0x7f32e019b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32d4006af0 tx=0x7f32d4011e80 comp rx=0 tx=0).stop 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 shutdown_connections 2023-07-21T18:58:15.561 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32d0038800 0x7f32d003acb0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 --2- 172.21.15.5:0/878896416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f32e007be10 0x7f32e019b050 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.078+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 >> 172.21.15.5:0/878896416 conn(0x7f32e01006c0 msgr2=0x7f32e0101df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.079+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 shutdown_connections 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr 2023-07-21T18:58:15.079+0000 7f32e5e04700 1 -- 172.21.15.5:0/878896416 wait complete. 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-07-21T18:58:15.562 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout:Or, if you are only running a single cluster on this host: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout:Please consider enabling telemetry to help improve Ceph: 2023-07-21T18:58:15.563 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: ceph telemetry on 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout:For more information see: 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:15.564 INFO:teuthology.orchestra.run.smithi005.stdout:Bootstrap complete. 2023-07-21T18:58:15.613 INFO:tasks.cephadm:Fetching config... 2023-07-21T18:58:15.614 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:58:15.614 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-07-21T18:58:15.698 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-07-21T18:58:15.698 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:58:15.698 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-07-21T18:58:15.782 INFO:tasks.cephadm:Fetching mon keyring... 2023-07-21T18:58:15.783 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:58:15.783 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/keyring of=/dev/stdout 2023-07-21T18:58:15.892 INFO:tasks.cephadm:Fetching pub ssh key... 2023-07-21T18:58:15.893 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:58:15.893 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-07-21T18:58:15.977 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-07-21T18:58:15.978 DEBUG:teuthology.orchestra.run.smithi005:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:58:16.124 INFO:teuthology.orchestra.run.smithi005.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:58:16.155 DEBUG:teuthology.orchestra.run.smithi101:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:58:16.259 INFO:teuthology.orchestra.run.smithi101.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:58:16.289 DEBUG:teuthology.orchestra.run.smithi156:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-07-21T18:58:16.394 INFO:teuthology.orchestra.run.smithi156.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDdZmvJoEevZ9O6Nt8ALZFXQcSxnbO3IHBTbO+MrXHaeCyUiAINEPLk6zHHfR/rriQU9qfr9LSav3pYd+LQs69u+8bNaVVYJxdvQpvVeAYL2Fp7HennRListRtCYBBgNcgq66N1pKzlYIcMkCjFyyHwWW2rML7Z8EOHDy0OAsRGbC8p+8SP3CrnrUDGEdBDTne8EzTpMArzPAtYp1WDNba+wwaKmC4aMgeiMbcvp58zDaK33IFKlN5vIRMkUVho8+lXeE5HMioXYwHiA/hfRogYeKVG78cd4w7yY8hfJoDv0m28ht6SpSfmncyOJjHtIqWFQacqPEgy1r4iQ+cikLftMtz8agIPU22mUouN3Y0+/w+MGp5ShuF0aFim20G2Kh3KdEJbG2UzziNOzK+yq2ZbGaUGJC0b/RVHeegdjn//1Q+R46jjjs5P9kBkHfWCSoTWwtITszvToKxllqrxNjq0UTdo/Y41WVfvsAtC8PwsPHOD4s+p7im6424NOjSMy7s= ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:58:16.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-07-21T18:58:17.396 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:18.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:18.561 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi005", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:58:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:18.562 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.783+0000 7f941ea76700 1 -- 172.21.15.5:0/1835272534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 msgr2=0x7f94180ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.783+0000 7f941ea76700 1 --2- 172.21.15.5:0/1835272534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f94180ff9f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f940c009910 tx=0x7f940c009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.783+0000 7f941ea76700 1 -- 172.21.15.5:0/1835272534 shutdown_connections 2023-07-21T18:58:18.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.784+0000 7f941ea76700 1 --2- 172.21.15.5:0/1835272534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f94180ff9f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:18.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.784+0000 7f941ea76700 1 -- 172.21.15.5:0/1835272534 >> 172.21.15.5:0/1835272534 conn(0x7f94180fab90 msgr2=0x7f94180fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:18.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.784+0000 7f941ea76700 1 -- 172.21.15.5:0/1835272534 shutdown_connections 2023-07-21T18:58:18.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.784+0000 7f941ea76700 1 -- 172.21.15.5:0/1835272534 wait complete. 2023-07-21T18:58:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.785+0000 7f941ea76700 1 Processor -- start 2023-07-21T18:58:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.785+0000 7f941ea76700 1 -- start start 2023-07-21T18:58:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.785+0000 7f941ea76700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.785+0000 7f941ea76700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9418194ba0 con 0x7f94180ff5e0 2023-07-21T18:58:18.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.785+0000 7f941c812700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:18.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f941c812700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59916/0 (socket says 172.21.15.5:59916) 2023-07-21T18:58:18.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f941c812700 1 -- 172.21.15.5:0/513913660 learned_addr learned my addr 172.21.15.5:0/513913660 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:18.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f941c812700 1 -- 172.21.15.5:0/513913660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f940c0095f0 con 0x7f94180ff5e0 2023-07-21T18:58:18.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f941c812700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f940c005950 tx=0x7f940c005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:18.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f940c00ba90 con 0x7f94180ff5e0 2023-07-21T18:58:18.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f940c00bbf0 con 0x7f94180ff5e0 2023-07-21T18:58:18.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9418194da0 con 0x7f94180ff5e0 2023-07-21T18:58:18.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.786+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f940c017440 con 0x7f94180ff5e0 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.787+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9418195240 con 0x7f94180ff5e0 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.787+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f940c00f460 con 0x7f94180ff5e0 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.788+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f941818e060 con 0x7f94180ff5e0 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.788+0000 7f9415ffb700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 0x7f940803ad00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.788+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f940c04c960 con 0x7f94180ff5e0 2023-07-21T18:58:18.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.788+0000 7f9417fff700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 0x7f940803ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:18.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.789+0000 7f9417fff700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 0x7f940803ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9404006fd0 tx=0x7f9404006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:18.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:18.793+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f940c02ad30 con 0x7f94180ff5e0 2023-07-21T18:58:19.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.002+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7f941802c850 con 0x7f94180ff5e0 2023-07-21T18:58:19.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.006+0000 7f9415ffb700 1 -- 172.21.15.5:0/513913660 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7f940c027030 con 0x7f94180ff5e0 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.008+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 msgr2=0x7f940803ad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.008+0000 7f941ea76700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 0x7f940803ad00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f9404006fd0 tx=0x7f9404006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.008+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 msgr2=0x7f9418194660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.008+0000 7f941ea76700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f940c005950 tx=0x7f940c005810 comp rx=0 tx=0).stop 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 shutdown_connections 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9408038850 0x7f940803ad00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:19.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 --2- 172.21.15.5:0/513913660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94180ff5e0 0x7f9418194660 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 >> 172.21.15.5:0/513913660 conn(0x7f94180fab90 msgr2=0x7f94180fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 shutdown_connections 2023-07-21T18:58:19.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:19.009+0000 7f941ea76700 1 -- 172.21.15.5:0/513913660 wait complete. 2023-07-21T18:58:19.377 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-07-21T18:58:19.378 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-07-21T18:58:20.104 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:20.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/513913660' entity='client.admin' 2023-07-21T18:58:20.349 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:20.350 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/2011178041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 msgr2=0x7ff2f00ff9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/2011178041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f00ff9f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec009910 tx=0x7ff2ec009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/2011178041 shutdown_connections 2023-07-21T18:58:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/2011178041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f00ff9f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/2011178041 >> 172.21.15.5:0/2011178041 conn(0x7ff2f00fab90 msgr2=0x7ff2f00fcfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/2011178041 shutdown_connections 2023-07-21T18:58:21.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.620+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/2011178041 wait complete. 2023-07-21T18:58:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f7c4b700 1 Processor -- start 2023-07-21T18:58:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f7c4b700 1 -- start start 2023-07-21T18:58:21.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f7c4b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f7c4b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f0194ba0 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f59e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f59e7700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59940/0 (socket says 172.21.15.5:59940) 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.622+0000 7ff2f59e7700 1 -- 172.21.15.5:0/178039793 learned_addr learned my addr 172.21.15.5:0/178039793 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2f59e7700 1 -- 172.21.15.5:0/178039793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec0095f0 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2f59e7700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005950 tx=0x7ff2ec005810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec00ba90 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2ec00bbf0 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff2ec017440 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f0194da0 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.623+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f0195240 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.624+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7ff2ec00f460 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.624+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2f018e060 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.625+0000 7ff2e6ffd700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 0x7ff2dc03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.625+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff2ec04c9c0 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.625+0000 7ff2f51e6700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 0x7ff2dc03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.626+0000 7ff2f51e6700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 0x7ff2dc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0006fd0 tx=0x7ff2e0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.630+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff2ec015350 con 0x7ff2f00ff5e0 2023-07-21T18:58:21.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.839+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff2f00609e0 con 0x7ff2dc0387b0 2023-07-21T18:58:21.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.842+0000 7ff2e6ffd700 1 -- 172.21.15.5:0/178039793 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff2f00609e0 con 0x7ff2dc0387b0 2023-07-21T18:58:21.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 msgr2=0x7ff2dc03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 0x7ff2dc03ac60 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff2e0006fd0 tx=0x7ff2e0006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 msgr2=0x7ff2f0194660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ff2ec005950 tx=0x7ff2ec005810 comp rx=0 tx=0).stop 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 shutdown_connections 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff2dc0387b0 0x7ff2dc03ac60 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 --2- 172.21.15.5:0/178039793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2f00ff5e0 0x7ff2f0194660 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.844+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 >> 172.21.15.5:0/178039793 conn(0x7ff2f00fab90 msgr2=0x7ff2f00fb920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.845+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 shutdown_connections 2023-07-21T18:58:21.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:21.845+0000 7ff2f7c4b700 1 -- 172.21.15.5:0/178039793 wait complete. 2023-07-21T18:58:22.291 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi101 2023-07-21T18:58:22.291 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:58:22.291 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/etc/ceph/ceph.conf 2023-07-21T18:58:22.339 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:58:22.340 DEBUG:teuthology.orchestra.run.smithi101:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:22.425 INFO:tasks.cephadm:Adding host smithi101 to orchestrator... 2023-07-21T18:58:22.426 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch host add smithi101 2023-07-21T18:58:23.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:23.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:23.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:23.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:23.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:23.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T18:58:23.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:22 smithi005 ceph-mon[126460]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:24.101 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:23 smithi005 ceph-mon[126460]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.700+0000 7fd05539c700 1 -- 172.21.15.5:0/2274034940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 msgr2=0x7fd050105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:24.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.700+0000 7fd05539c700 1 --2- 172.21.15.5:0/2274034940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd050105a00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd040009910 tx=0x7fd040009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.701+0000 7fd05539c700 1 -- 172.21.15.5:0/2274034940 shutdown_connections 2023-07-21T18:58:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.701+0000 7fd05539c700 1 --2- 172.21.15.5:0/2274034940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd050105a00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.701+0000 7fd05539c700 1 -- 172.21.15.5:0/2274034940 >> 172.21.15.5:0/2274034940 conn(0x7fd0500fb130 msgr2=0x7fd0500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.701+0000 7fd05539c700 1 -- 172.21.15.5:0/2274034940 shutdown_connections 2023-07-21T18:58:24.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.701+0000 7fd05539c700 1 -- 172.21.15.5:0/2274034940 wait complete. 2023-07-21T18:58:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.702+0000 7fd05539c700 1 Processor -- start 2023-07-21T18:58:24.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.702+0000 7fd05539c700 1 -- start start 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.702+0000 7fd05539c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.702+0000 7fd05539c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd05019ab70 con 0x7fd050105630 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.702+0000 7fd04effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd04effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59958/0 (socket says 172.21.15.5:59958) 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd04effd700 1 -- 172.21.15.5:0/4268834835 learned_addr learned my addr 172.21.15.5:0/4268834835 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd04effd700 1 -- 172.21.15.5:0/4268834835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0400095f0 con 0x7fd050105630 2023-07-21T18:58:24.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd04effd700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd040006010 tx=0x7fd040004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd04001d070 con 0x7fd050105630 2023-07-21T18:58:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd04000bdf0 con 0x7fd050105630 2023-07-21T18:58:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd040022d80 con 0x7fd050105630 2023-07-21T18:58:24.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.703+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd05019add0 con 0x7fd050105630 2023-07-21T18:58:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.704+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0501946b0 con 0x7fd050105630 2023-07-21T18:58:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.705+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fd04000b960 con 0x7fd050105630 2023-07-21T18:58:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.705+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd050061b40 con 0x7fd050105630 2023-07-21T18:58:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.705+0000 7fd037fff700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 0x7fd03803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:24.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.705+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd04004c8f0 con 0x7fd050105630 2023-07-21T18:58:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.705+0000 7fd04e7fc700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 0x7fd03803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:24.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.706+0000 7fd04e7fc700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 0x7fd03803acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:24.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.711+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd04002b590 con 0x7fd050105630 2023-07-21T18:58:24.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:24.923+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi101", "target": ["mon-mgr", ""]}) v1 -- 0x7fd050195170 con 0x7fd038038800 2023-07-21T18:58:25.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:25 smithi005 ceph-mon[126460]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:58:25.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:25.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:25.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:26.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:26 smithi005 ceph-mon[126460]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi101", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:27.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:27 smithi005 ceph-mon[126460]: Deploying cephadm binary to smithi101 2023-07-21T18:58:28.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.034+0000 7fd037fff700 1 -- 172.21.15.5:0/4268834835 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd050195170 con 0x7fd038038800 2023-07-21T18:58:28.045 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi101' with addr '172.21.15.101' 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.037+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 msgr2=0x7fd03803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.037+0000 7fd05539c700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 0x7fd03803acb0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd044006fd0 tx=0x7fd044006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.037+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 msgr2=0x7fd05019a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.037+0000 7fd05539c700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd040006010 tx=0x7fd040004ff0 comp rx=0 tx=0).stop 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 shutdown_connections 2023-07-21T18:58:28.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd038038800 0x7fd03803acb0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 --2- 172.21.15.5:0/4268834835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd050105630 0x7fd05019a630 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 >> 172.21.15.5:0/4268834835 conn(0x7fd0500fb130 msgr2=0x7fd0500fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 shutdown_connections 2023-07-21T18:58:28.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:28.038+0000 7fd05539c700 1 -- 172.21.15.5:0/4268834835 wait complete. 2023-07-21T18:58:28.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:28 smithi005 ceph-mon[126460]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:28.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:28.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:28.472 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T18:58:29.201 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:29.504 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:29 smithi005 ceph-mon[126460]: Added host smithi101 2023-07-21T18:58:29.505 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:29 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 -- 172.21.15.5:0/2532243232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 msgr2=0x7f0a10072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:30.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 --2- 172.21.15.5:0/2532243232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a10072c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0a04009910 tx=0x7f0a04009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 -- 172.21.15.5:0/2532243232 shutdown_connections 2023-07-21T18:58:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 --2- 172.21.15.5:0/2532243232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a10072c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 -- 172.21.15.5:0/2532243232 >> 172.21.15.5:0/2532243232 conn(0x7f0a100fb130 msgr2=0x7f0a100fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 -- 172.21.15.5:0/2532243232 shutdown_connections 2023-07-21T18:58:30.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.674+0000 7f0a17206700 1 -- 172.21.15.5:0/2532243232 wait complete. 2023-07-21T18:58:30.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.675+0000 7f0a17206700 1 Processor -- start 2023-07-21T18:58:30.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.675+0000 7f0a17206700 1 -- start start 2023-07-21T18:58:30.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a17206700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a17206700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a1019b5b0 con 0x7f0a100728b0 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a14fa2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a14fa2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50902/0 (socket says 172.21.15.5:50902) 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a14fa2700 1 -- 172.21.15.5:0/3608506186 learned_addr learned my addr 172.21.15.5:0/3608506186 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a14fa2700 1 -- 172.21.15.5:0/3608506186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040095f0 con 0x7f0a100728b0 2023-07-21T18:58:30.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.676+0000 7f0a14fa2700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a04004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.677+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a0401d070 con 0x7f0a100728b0 2023-07-21T18:58:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.677+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a0400bdf0 con 0x7f0a100728b0 2023-07-21T18:58:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.677+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1019b7b0 con 0x7f0a100728b0 2023-07-21T18:58:30.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.677+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a04022d80 con 0x7f0a100728b0 2023-07-21T18:58:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.677+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a1019bbd0 con 0x7f0a100728b0 2023-07-21T18:58:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.678+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a10061b40 con 0x7f0a100728b0 2023-07-21T18:58:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.678+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f0a04022570 con 0x7f0a100728b0 2023-07-21T18:58:30.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.679+0000 7f0a0dffb700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 0x7f0a0003acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.679+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f0a0404c8f0 con 0x7f0a100728b0 2023-07-21T18:58:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.679+0000 7f0a0ffff700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 0x7f0a0003acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:30.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.679+0000 7f0a0ffff700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 0x7f0a0003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09fc006fd0 tx=0x7f09fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:30.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.684+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a0402bb60 con 0x7f0a100728b0 2023-07-21T18:58:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.900+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0a10102580 con 0x7f0a00038800 2023-07-21T18:58:30.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.902+0000 7f0a0dffb700 1 -- 172.21.15.5:0/3608506186 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7f0a10102580 con 0x7f0a00038800 2023-07-21T18:58:30.912 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:30.912 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}] 2023-07-21T18:58:30.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 msgr2=0x7f0a0003acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 0x7f0a0003acb0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f09fc006fd0 tx=0x7f09fc006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 msgr2=0x7f0a1019b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f0a04006010 tx=0x7f0a04004f40 comp rx=0 tx=0).stop 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 shutdown_connections 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a00038800 0x7f0a0003acb0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 --2- 172.21.15.5:0/3608506186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a100728b0 0x7f0a1019b070 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:30.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.904+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 >> 172.21.15.5:0/3608506186 conn(0x7f0a100fb130 msgr2=0x7f0a100fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.905+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 shutdown_connections 2023-07-21T18:58:30.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:30.905+0000 7f0a17206700 1 -- 172.21.15.5:0/3608506186 wait complete. 2023-07-21T18:58:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:30 smithi005 ceph-mon[126460]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:30 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:31.482 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi156 2023-07-21T18:58:31.483 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:58:31.483 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.conf 2023-07-21T18:58:31.530 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:58:31.531 DEBUG:teuthology.orchestra.run.smithi156:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:31.616 INFO:tasks.cephadm:Adding host smithi156 to orchestrator... 2023-07-21T18:58:31.617 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch host add smithi156 2023-07-21T18:58:32.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:32.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:31 smithi005 ceph-mon[126460]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:58:32.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:31 smithi005 ceph-mon[126460]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:32.346 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 -- 172.21.15.5:0/923838038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 msgr2=0x7f9260105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:33.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 --2- 172.21.15.5:0/923838038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260105a00 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9250009910 tx=0x7f9250009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 -- 172.21.15.5:0/923838038 shutdown_connections 2023-07-21T18:58:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 --2- 172.21.15.5:0/923838038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260105a00 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 -- 172.21.15.5:0/923838038 >> 172.21.15.5:0/923838038 conn(0x7f92600fb130 msgr2=0x7f92600fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 -- 172.21.15.5:0/923838038 shutdown_connections 2023-07-21T18:58:33.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.850+0000 7f9266422700 1 -- 172.21.15.5:0/923838038 wait complete. 2023-07-21T18:58:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.851+0000 7f9266422700 1 Processor -- start 2023-07-21T18:58:33.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.851+0000 7f9266422700 1 -- start start 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.851+0000 7f9266422700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.851+0000 7f9266422700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9260074790 con 0x7f9260105630 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50926/0 (socket says 172.21.15.5:50926) 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925ffff700 1 -- 172.21.15.5:0/2134311492 learned_addr learned my addr 172.21.15.5:0/2134311492 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925ffff700 1 -- 172.21.15.5:0/2134311492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92500095f0 con 0x7f9260105630 2023-07-21T18:58:33.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925ffff700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f9250004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.852+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f925001d070 con 0x7f9260105630 2023-07-21T18:58:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.853+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925000bdf0 con 0x7f9260105630 2023-07-21T18:58:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.853+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9250022d80 con 0x7f9260105630 2023-07-21T18:58:33.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.853+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9260072d50 con 0x7f9260105630 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.853+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9260073250 con 0x7f9260105630 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.854+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9250022570 con 0x7f9260105630 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.854+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92601048d0 con 0x7f9260105630 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.854+0000 7f925d7fa700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 0x7f924803acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.854+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f925004c8a0 con 0x7f9260105630 2023-07-21T18:58:33.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.855+0000 7f925f7fe700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 0x7f924803acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:33.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.855+0000 7f925f7fe700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 0x7f924803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:33.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:33.860+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f925002a540 con 0x7f9260105630 2023-07-21T18:58:34.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:34.070+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}) v1 -- 0x7f92600008d0 con 0x7f9248038800 2023-07-21T18:58:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:34.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:58:34.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:34.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:35.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:35 smithi005 ceph-mon[126460]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:35.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:35 smithi005 ceph-mon[126460]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:58:36.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:36 smithi005 ceph-mon[126460]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:36 smithi005 ceph-mon[126460]: Deploying cephadm binary to smithi156 2023-07-21T18:58:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:36 smithi005 ceph-mon[126460]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:36 smithi005 ceph-mon[126460]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:37.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.287+0000 7f925d7fa700 1 -- 172.21.15.5:0/2134311492 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f92600008d0 con 0x7f9248038800 2023-07-21T18:58:37.299 INFO:teuthology.orchestra.run.smithi005.stdout:Added host 'smithi156' with addr '172.21.15.156' 2023-07-21T18:58:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 msgr2=0x7f924803acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 0x7f924803acb0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9254006fd0 tx=0x7f9254006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 msgr2=0x7f9260072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:37.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9250006010 tx=0x7f9250004ff0 comp rx=0 tx=0).stop 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 shutdown_connections 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9248038800 0x7f924803acb0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 --2- 172.21.15.5:0/2134311492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9260105630 0x7f9260072810 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 >> 172.21.15.5:0/2134311492 conn(0x7f92600fb130 msgr2=0x7f92600fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 shutdown_connections 2023-07-21T18:58:37.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:37.290+0000 7f9266422700 1 -- 172.21.15.5:0/2134311492 wait complete. 2023-07-21T18:58:37.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:37 smithi005 ceph-mon[126460]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:58:37.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:37.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:37.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:37.733 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch host ls --format=json 2023-07-21T18:58:38.460 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:38 smithi005 ceph-mon[126460]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:38 smithi005 ceph-mon[126460]: Added host smithi156 2023-07-21T18:58:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:39.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:39.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:39 smithi005 ceph-mon[126460]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 -- 172.21.15.5:0/797449199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 msgr2=0x7f9bac105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:39.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 --2- 172.21.15.5:0/797449199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac105a00 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c009910 tx=0x7f9b9c009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 -- 172.21.15.5:0/797449199 shutdown_connections 2023-07-21T18:58:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 --2- 172.21.15.5:0/797449199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac105a00 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 -- 172.21.15.5:0/797449199 >> 172.21.15.5:0/797449199 conn(0x7f9bac0fb130 msgr2=0x7f9bac0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 -- 172.21.15.5:0/797449199 shutdown_connections 2023-07-21T18:58:39.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.939+0000 7f9bb159c700 1 -- 172.21.15.5:0/797449199 wait complete. 2023-07-21T18:58:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.940+0000 7f9bb159c700 1 Processor -- start 2023-07-21T18:58:39.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.940+0000 7f9bb159c700 1 -- start start 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9bb159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9bb159c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bac19cda0 con 0x7f9bac105630 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9baaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46446/0 (socket says 172.21.15.5:46446) 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9baaffd700 1 -- 172.21.15.5:0/3078107560 learned_addr learned my addr 172.21.15.5:0/3078107560 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:39.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.941+0000 7f9baaffd700 1 -- 172.21.15.5:0/3078107560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0095f0 con 0x7f9bac105630 2023-07-21T18:58:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9baaffd700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006010 tx=0x7f9b9c004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:39.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c01d070 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9b9c00bdf0 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9b9c022d80 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bac19d000 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.942+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bac1968e0 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.943+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f9b9c022570 con 0x7f9bac105630 2023-07-21T18:58:39.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.943+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bac061b40 con 0x7f9bac105630 2023-07-21T18:58:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.944+0000 7f9b93fff700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 0x7f9b9403acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.944+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9b9c04c960 con 0x7f9bac105630 2023-07-21T18:58:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.944+0000 7f9baa7fc700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 0x7f9b9403acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:39.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.944+0000 7f9baa7fc700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 0x7f9b9403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0006fd0 tx=0x7f9ba0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:39.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:39.949+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9b9c02b590 con 0x7f9bac105630 2023-07-21T18:58:40.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.163+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9bac072e50 con 0x7f9b94038800 2023-07-21T18:58:40.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.164+0000 7f9b93fff700 1 -- 172.21.15.5:0/3078107560 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f9bac072e50 con 0x7f9b94038800 2023-07-21T18:58:40.175 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T18:58:40.175 INFO:teuthology.orchestra.run.smithi005.stdout:[{"addr": "172.21.15.5", "hostname": "smithi005", "labels": [], "status": ""}, {"addr": "172.21.15.101", "hostname": "smithi101", "labels": [], "status": ""}, {"addr": "172.21.15.156", "hostname": "smithi156", "labels": [], "status": ""}] 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.166+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 msgr2=0x7f9b9403acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.166+0000 7f9bb159c700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 0x7f9b9403acb0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9ba0006fd0 tx=0x7f9ba0006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.166+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 msgr2=0x7f9bac19c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.166+0000 7f9bb159c700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c006010 tx=0x7f9b9c004ff0 comp rx=0 tx=0).stop 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 shutdown_connections 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9b94038800 0x7f9b9403acb0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:40.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 --2- 172.21.15.5:0/3078107560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9bac105630 0x7f9bac19c860 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 >> 172.21.15.5:0/3078107560 conn(0x7f9bac0fb130 msgr2=0x7f9bac072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 shutdown_connections 2023-07-21T18:58:40.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:40.167+0000 7f9bb159c700 1 -- 172.21.15.5:0/3078107560 wait complete. 2023-07-21T18:58:40.635 INFO:tasks.cephadm:Setting crush tunables to default 2023-07-21T18:58:40.636 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd crush tunables default 2023-07-21T18:58:41.363 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:58:41.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:41.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:41 smithi005 ceph-mon[126460]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:58:42.123 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:42 smithi005 ceph-mon[126460]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/1085609478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 msgr2=0x7f7e14105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:42.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/1085609478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e14105a00 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f7e10009910 tx=0x7f7e10009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/1085609478 shutdown_connections 2023-07-21T18:58:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/1085609478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e14105a00 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/1085609478 >> 172.21.15.5:0/1085609478 conn(0x7f7e140fb130 msgr2=0x7f7e140fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/1085609478 shutdown_connections 2023-07-21T18:58:42.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.764+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/1085609478 wait complete. 2023-07-21T18:58:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.765+0000 7f7e1c2eb700 1 Processor -- start 2023-07-21T18:58:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.765+0000 7f7e1c2eb700 1 -- start start 2023-07-21T18:58:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1c2eb700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:42.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1c2eb700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e1419ab70 con 0x7f7e14105630 2023-07-21T18:58:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1a087700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1a087700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46468/0 (socket says 172.21.15.5:46468) 2023-07-21T18:58:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1a087700 1 -- 172.21.15.5:0/34564910 learned_addr learned my addr 172.21.15.5:0/34564910 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:58:42.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.766+0000 7f7e1a087700 1 -- 172.21.15.5:0/34564910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e100095f0 con 0x7f7e14105630 2023-07-21T18:58:42.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e1a087700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e10004f40 tx=0x7f7e10005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:42.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e1001d070 con 0x7f7e14105630 2023-07-21T18:58:42.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e100044b0 con 0x7f7e14105630 2023-07-21T18:58:42.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7e10003810 con 0x7f7e14105630 2023-07-21T18:58:42.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e1419add0 con 0x7f7e14105630 2023-07-21T18:58:42.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.767+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e141946b0 con 0x7f7e14105630 2023-07-21T18:58:42.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.768+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e14061b40 con 0x7f7e14105630 2023-07-21T18:58:42.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.768+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f7e1000b960 con 0x7f7e14105630 2023-07-21T18:58:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.769+0000 7f7e0b7fe700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 0x7f7e0003acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.769+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7e1004c6c0 con 0x7f7e14105630 2023-07-21T18:58:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.769+0000 7f7e19886700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 0x7f7e0003acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:42.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.769+0000 7f7e19886700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 0x7f7e0003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006fd0 tx=0x7f7e04006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:42.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.774+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7e100042e0 con 0x7f7e14105630 2023-07-21T18:58:42.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:42.983+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f7e14195bc0 con 0x7f7e14105630 2023-07-21T18:58:43.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.166+0000 7f7e0b7fe700 1 -- 172.21.15.5:0/34564910 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f7e1002a360 con 0x7f7e14105630 2023-07-21T18:58:43.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 msgr2=0x7f7e0003acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:43.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 0x7f7e0003acf0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f7e04006fd0 tx=0x7f7e04006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:43.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 msgr2=0x7f7e1419a630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f7e10004f40 tx=0x7f7e10005020 comp rx=0 tx=0).stop 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 shutdown_connections 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7e00038840 0x7f7e0003acf0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 --2- 172.21.15.5:0/34564910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e14105630 0x7f7e1419a630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 >> 172.21.15.5:0/34564910 conn(0x7f7e140fb130 msgr2=0x7f7e140fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 shutdown_connections 2023-07-21T18:58:43.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:58:43.168+0000 7f7e1c2eb700 1 -- 172.21.15.5:0/34564910 wait complete. 2023-07-21T18:58:43.181 INFO:teuthology.orchestra.run.smithi005.stderr:adjusted tunables profile to default 2023-07-21T18:58:43.540 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:43 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:58:43.542 INFO:tasks.cephadm:Adding mon.a on smithi005 2023-07-21T18:58:43.542 INFO:tasks.cephadm:Adding mon.b on smithi101 2023-07-21T18:58:43.542 INFO:tasks.cephadm:Adding mon.c on smithi156 2023-07-21T18:58:43.542 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch apply mon '3;smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c' 2023-07-21T18:58:44.289 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /etc/ceph/ceph.conf 2023-07-21T18:58:44.632 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:44.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:58:44.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:44.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:44.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:44 smithi005 ceph-mon[126460]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:58:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:45 smithi005 ceph-mon[126460]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:45.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:45 smithi005 ceph-mon[126460]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:46.140 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 -- 172.21.15.156:0/3928039176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 msgr2=0x7fc7a40ffa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:46.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 --2- 172.21.15.156:0/3928039176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a40ffa40 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc794009910 tx=0x7fc794009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:46.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 -- 172.21.15.156:0/3928039176 shutdown_connections 2023-07-21T18:58:46.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 --2- 172.21.15.156:0/3928039176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a40ffa40 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:46.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 -- 172.21.15.156:0/3928039176 >> 172.21.15.156:0/3928039176 conn(0x7fc7a40faf10 msgr2=0x7fc7a40fd320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:46.141 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 -- 172.21.15.156:0/3928039176 shutdown_connections 2023-07-21T18:58:46.142 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.130+0000 7fc7a359e700 1 -- 172.21.15.156:0/3928039176 wait complete. 2023-07-21T18:58:46.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.131+0000 7fc7a359e700 1 Processor -- start 2023-07-21T18:58:46.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.131+0000 7fc7a359e700 1 -- start start 2023-07-21T18:58:46.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.132+0000 7fc7a359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:46.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.132+0000 7fc7a359e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a419cd70 con 0x7fc7a40ff670 2023-07-21T18:58:46.143 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.132+0000 7fc7a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:46.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.132+0000 7fc7a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:56268/0 (socket says 172.21.15.156:56268) 2023-07-21T18:58:46.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.132+0000 7fc7a259c700 1 -- 172.21.15.156:0/824320079 learned_addr learned my addr 172.21.15.156:0/824320079 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T18:58:46.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7a259c700 1 -- 172.21.15.156:0/824320079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7940095f0 con 0x7fc7a40ff670 2023-07-21T18:58:46.144 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7a259c700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc794004f40 tx=0x7fc794005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:46.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc79401d070 con 0x7fc7a40ff670 2023-07-21T18:58:46.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7940044b0 con 0x7fc7a40ff670 2023-07-21T18:58:46.145 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a419cf70 con 0x7fc7a40ff670 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.133+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc794003860 con 0x7fc7a40ff670 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.134+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a419d330 con 0x7fc7a40ff670 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.135+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a4061b40 con 0x7fc7a40ff670 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.135+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7fc79400b960 con 0x7fc7a40ff670 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.135+0000 7fc7937fe700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 0x7fc78c03acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:46.146 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.135+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc79404c6c0 con 0x7fc7a40ff670 2023-07-21T18:58:46.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.136+0000 7fc7a1d9b700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 0x7fc78c03acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:46.147 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.136+0000 7fc7a1d9b700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 0x7fc78c03acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc798006fd0 tx=0x7fc798006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:46.150 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.141+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc79402a540 con 0x7fc7a40ff670 2023-07-21T18:58:46.352 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.342+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fc7a4103790 con 0x7fc78c038800 2023-07-21T18:58:46.359 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.349+0000 7fc7937fe700 1 -- 172.21.15.156:0/824320079 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fc7a4103790 con 0x7fc78c038800 2023-07-21T18:58:46.359 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mon update... 2023-07-21T18:58:46.361 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 msgr2=0x7fc78c03acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 0x7fc78c03acb0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc798006fd0 tx=0x7fc798006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 msgr2=0x7fc7a419c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc794004f40 tx=0x7fc794005020 comp rx=0 tx=0).stop 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 shutdown_connections 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc78c038800 0x7fc78c03acb0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:46.362 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 --2- 172.21.15.156:0/824320079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7a40ff670 0x7fc7a419c830 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:46.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.351+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 >> 172.21.15.156:0/824320079 conn(0x7fc7a40faf10 msgr2=0x7fc7a4102fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:46.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.352+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 shutdown_connections 2023-07-21T18:58:46.363 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:46.352+0000 7fc7a359e700 1 -- 172.21.15.156:0/824320079 wait complete. 2023-07-21T18:58:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:46 smithi005 ceph-mon[126460]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:58:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:46 smithi005 ceph-mon[126460]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:46.919 DEBUG:teuthology.orchestra.run.smithi101:mon.b> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.b.service 2023-07-21T18:58:46.925 DEBUG:teuthology.orchestra.run.smithi156:mon.c> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.c.service 2023-07-21T18:58:46.929 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:58:46.929 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:58:47.002 INFO:journalctl@ceph.mon.b.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T18:58:47.006 INFO:journalctl@ceph.mon.c.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T18:58:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c;count:3 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:58:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:47 smithi005 ceph-mon[126460]: Deploying daemon mon.c on smithi156 2023-07-21T18:58:48.028 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:48 smithi005 ceph-mon[126460]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:50.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.442+0000 7f3ddce4b700 1 -- 172.21.15.156:0/1631139735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 msgr2=0x7f3dd8072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:50.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.442+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/1631139735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd8072c80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8009910 tx=0x7f3dc8009c20 comp rx=0 tx=0).stop 2023-07-21T18:58:50.453 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.443+0000 7f3ddce4b700 1 -- 172.21.15.156:0/1631139735 shutdown_connections 2023-07-21T18:58:50.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.443+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/1631139735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd8072c80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:50.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.443+0000 7f3ddce4b700 1 -- 172.21.15.156:0/1631139735 >> 172.21.15.156:0/1631139735 conn(0x7f3dd80fb130 msgr2=0x7f3dd80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:50.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.443+0000 7f3ddce4b700 1 -- 172.21.15.156:0/1631139735 shutdown_connections 2023-07-21T18:58:50.454 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.443+0000 7f3ddce4b700 1 -- 172.21.15.156:0/1631139735 wait complete. 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.444+0000 7f3ddce4b700 1 Processor -- start 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.444+0000 7f3ddce4b700 1 -- start start 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.444+0000 7f3ddce4b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.444+0000 7f3ddce4b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd819b5b0 con 0x7f3dd80728b0 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.445+0000 7f3dd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:50.455 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.445+0000 7f3dd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:56286/0 (socket says 172.21.15.156:56286) 2023-07-21T18:58:50.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.445+0000 7f3dd659c700 1 -- 172.21.15.156:0/3073739754 learned_addr learned my addr 172.21.15.156:0/3073739754 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T18:58:50.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.445+0000 7f3dd659c700 1 -- 172.21.15.156:0/3073739754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc80095f0 con 0x7f3dd80728b0 2023-07-21T18:58:50.456 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3dd659c700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006010 tx=0x7f3dc8004f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:50.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc801d070 con 0x7f3dd80728b0 2023-07-21T18:58:50.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd819b7b0 con 0x7f3dd80728b0 2023-07-21T18:58:50.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3dc800bdf0 con 0x7f3dd80728b0 2023-07-21T18:58:50.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3dc8022d80 con 0x7f3dd80728b0 2023-07-21T18:58:50.457 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.446+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd819bbd0 con 0x7f3dd80728b0 2023-07-21T18:58:50.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.447+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f3dc8022570 con 0x7f3dd80728b0 2023-07-21T18:58:50.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.448+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd8061b40 con 0x7f3dd80728b0 2023-07-21T18:58:50.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.448+0000 7f3dc77fe700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 0x7f3dc003ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:50.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.448+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3dc804c930 con 0x7f3dd80728b0 2023-07-21T18:58:50.458 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.448+0000 7f3dd5d9b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 0x7f3dc003ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:50.459 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.449+0000 7f3dd5d9b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 0x7f3dc003ac00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006fd0 tx=0x7f3dcc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:50.464 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.454+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3dc802a540 con 0x7f3dd80728b0 2023-07-21T18:58:50.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:50 smithi005 ceph-mon[126460]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:50.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.758+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3dd8102510 con 0x7f3dd80728b0 2023-07-21T18:58:50.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.759+0000 7f3dc77fe700 1 -- 172.21.15.156:0/3073739754 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+732 (secure 0 0 0) 0x7f3dc8027070 con 0x7f3dd80728b0 2023-07-21T18:58:50.769 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:58:50.770 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":1,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:57:01.506340Z","created":"2023-07-21T18:57:01.506340Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-07-21T18:58:50.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 msgr2=0x7f3dc003ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:50.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 0x7f3dc003ac00 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3dcc006fd0 tx=0x7f3dcc006e40 comp rx=0 tx=0).stop 2023-07-21T18:58:50.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 msgr2=0x7f3dd819b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:50.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f3dc8006010 tx=0x7f3dc8004f40 comp rx=0 tx=0).stop 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 shutdown_connections 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3dc0038750 0x7f3dc003ac00 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 --2- 172.21.15.156:0/3073739754 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3dd80728b0 0x7f3dd819b070 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 >> 172.21.15.156:0/3073739754 conn(0x7f3dd80fb130 msgr2=0x7f3dd80fbd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 shutdown_connections 2023-07-21T18:58:50.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:50.762+0000 7f3ddce4b700 1 -- 172.21.15.156:0/3073739754 wait complete. 2023-07-21T18:58:50.774 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 1 2023-07-21T18:58:50.835 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:50 smithi156 systemd[1]: Starting Ceph mon.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:58:51.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:51 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/3073739754' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:58:51.876 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 podman[128276]: 2023-07-21T18:58:52.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 bash[128276]: 44be25cf07bf99b549d520ca69246e782210fb32e907af5753723b22640cb012 2023-07-21T18:58:52.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:58:52.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:58:52.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pidfile_write: ignore empty --pid-file 2023-07-21T18:58:52.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: load: jerasure load: lrc 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Git sha 0 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: DB SUMMARY 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: DB Session ID: KZURGDU286HZOK67T1UF 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:58:52.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:58:52.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.env: 0x55be5c17f720 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.info_log: 0x55be5d6b3280 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.statistics: (nil) 2023-07-21T18:58:52.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.use_fsync: 0 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:58:52.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:58:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:58:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:58:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:58:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.db_log_dir: 2023-07-21T18:58:52.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.wal_dir: 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:58:52.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.write_buffer_manager: 0x55be5d5d43c0 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:58:52.188 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.unordered_write: 0 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:58:52.189 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:58:52.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.row_cache: None 2023-07-21T18:58:52.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.wal_filter: None 2023-07-21T18:58:52.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:58:52.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:58:52.190 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:58:52.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:58:52.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.wal_compression: 0 2023-07-21T18:58:52.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:58:52.191 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:58:52.192 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:58:52.192 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:58:52.192 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:58:52.193 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:58:52.193 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:58:52.193 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:58:52.193 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:58:52.193 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:58:52.196 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:58:52.196 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:58:52.196 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:58:52.197 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:58:52.197 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:58:52.197 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:58:52.198 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:58:52.198 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:58:52.198 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:58:52.198 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:58:52.199 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:58:52.199 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:58:52.199 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:58:52.200 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_open_files: -1 2023-07-21T18:58:52.200 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:58:52.200 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:58:52.200 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:58:52.200 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:58:52.201 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:58:52.201 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Compression algorithms supported: 2023-07-21T18:58:52.201 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:58:52.201 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kZSTD supported: 0 2023-07-21T18:58:52.202 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:58:52.202 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:58:52.202 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:58:52.202 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:58:52.203 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:58:52.203 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:58:52.203 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:58:52.203 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:58:52.203 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-07-21T18:58:52.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:58:52.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:58:52.204 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.merge_operator: 2023-07-21T18:58:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_filter: None 2023-07-21T18:58:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:58:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:58:52.205 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:58:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:58:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55be5d6b3380) 2023-07-21T18:58:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:58:52.206 INFO:journalctl@ceph.mon.c.smithi156.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:58:52.207 INFO:journalctl@ceph.mon.c.smithi156.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:58:52.207 INFO:journalctl@ceph.mon.c.smithi156.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:58:52.207 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_type: 0 2023-07-21T18:58:52.207 INFO:journalctl@ceph.mon.c.smithi156.stdout: data_block_index_type: 0 2023-07-21T18:58:52.208 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_shortening: 1 2023-07-21T18:58:52.208 INFO:journalctl@ceph.mon.c.smithi156.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:58:52.208 INFO:journalctl@ceph.mon.c.smithi156.stdout: checksum: 4 2023-07-21T18:58:52.208 INFO:journalctl@ceph.mon.c.smithi156.stdout: no_block_cache: 0 2023-07-21T18:58:52.209 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache: 0x55be5d6731f0 2023-07-21T18:58:52.209 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:58:52.209 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_options: 2023-07-21T18:58:52.209 INFO:journalctl@ceph.mon.c.smithi156.stdout: capacity : 536870912 2023-07-21T18:58:52.209 INFO:journalctl@ceph.mon.c.smithi156.stdout: num_shard_bits : 4 2023-07-21T18:58:52.210 INFO:journalctl@ceph.mon.c.smithi156.stdout: strict_capacity_limit : 0 2023-07-21T18:58:52.210 INFO:journalctl@ceph.mon.c.smithi156.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:58:52.210 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_cache_compressed: (nil) 2023-07-21T18:58:52.210 INFO:journalctl@ceph.mon.c.smithi156.stdout: persistent_cache: (nil) 2023-07-21T18:58:52.210 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_size: 4096 2023-07-21T18:58:52.211 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_size_deviation: 10 2023-07-21T18:58:52.211 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_restart_interval: 16 2023-07-21T18:58:52.211 INFO:journalctl@ceph.mon.c.smithi156.stdout: index_block_restart_interval: 1 2023-07-21T18:58:52.211 INFO:journalctl@ceph.mon.c.smithi156.stdout: metadata_block_size: 4096 2023-07-21T18:58:52.212 INFO:journalctl@ceph.mon.c.smithi156.stdout: partition_filters: 0 2023-07-21T18:58:52.212 INFO:journalctl@ceph.mon.c.smithi156.stdout: use_delta_encoding: 1 2023-07-21T18:58:52.212 INFO:journalctl@ceph.mon.c.smithi156.stdout: filter_policy: bloomfilter 2023-07-21T18:58:52.213 INFO:journalctl@ceph.mon.c.smithi156.stdout: whole_key_filtering: 1 2023-07-21T18:58:52.213 INFO:journalctl@ceph.mon.c.smithi156.stdout: verify_compression: 0 2023-07-21T18:58:52.213 INFO:journalctl@ceph.mon.c.smithi156.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:58:52.213 INFO:journalctl@ceph.mon.c.smithi156.stdout: format_version: 5 2023-07-21T18:58:52.214 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_index_compression: 1 2023-07-21T18:58:52.214 INFO:journalctl@ceph.mon.c.smithi156.stdout: block_align: 0 2023-07-21T18:58:52.214 INFO:journalctl@ceph.mon.c.smithi156.stdout: max_auto_readahead_size: 262144 2023-07-21T18:58:52.214 INFO:journalctl@ceph.mon.c.smithi156.stdout: prepopulate_block_cache: 0 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression: NoCompression 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:58:52.215 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.num_levels: 7 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:58:52.216 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:58:52.217 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:58:52.218 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:58:52.219 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:58:52.220 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:58:52.221 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:58:52.222 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:58:52.223 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:58:52.224 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.ttl: 2592000 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:58:52.225 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:58:52.226 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: da3f42c9-1a39-44b2-8b66-4774f9b00765 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965931877992, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965931879273, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689965931, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "da3f42c9-1a39-44b2-8b66-4774f9b00765", "db_session_id": "KZURGDU286HZOK67T1UF", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965931879541, "job": 1, "event": "recovery_finished"} 2023-07-21T18:58:52.227 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55be5d70c000 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: DB pointer 0x55be5d6f8000 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** DB Stats ** 2023-07-21T18:58:52.228 INFO:journalctl@ceph.mon.c.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:58:52.229 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:58:52.229 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:58:52.229 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:58:52.229 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:58:52.229 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:58:52.230 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:58:52.230 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.230 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** Compaction Stats [default] ** 2023-07-21T18:58:52.230 INFO:journalctl@ceph.mon.c.smithi156.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:58:52.231 INFO:journalctl@ceph.mon.c.smithi156.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:58:52.231 INFO:journalctl@ceph.mon.c.smithi156.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:58:52.231 INFO:journalctl@ceph.mon.c.smithi156.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:58:52.231 INFO:journalctl@ceph.mon.c.smithi156.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:58:52.232 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.232 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** Compaction Stats [default] ** 2023-07-21T18:58:52.232 INFO:journalctl@ceph.mon.c.smithi156.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:58:52.233 INFO:journalctl@ceph.mon.c.smithi156.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:58:52.233 INFO:journalctl@ceph.mon.c.smithi156.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.2 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:58:52.233 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.233 INFO:journalctl@ceph.mon.c.smithi156.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:58:52.234 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.234 INFO:journalctl@ceph.mon.c.smithi156.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:58:52.234 INFO:journalctl@ceph.mon.c.smithi156.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: Cumulative compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:58:52.235 INFO:journalctl@ceph.mon.c.smithi156.stdout: Interval compaction: 0.00 GB write, 0.25 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout: Block cache BinnedLRUCache@0x55be5d6731f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1e-05 secs_since: 0 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-07-21T18:58:52.236 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: using public_addr v2:172.21.15.156:0/0 -> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: starting mon.c rank -1 at public addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] at bind addrs [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(???) e0 preinit fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 systemd[1]: Started Ceph mon.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).mds e1 new map 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).mds e1 print_map 2023-07-21T18:58:52.237 INFO:journalctl@ceph.mon.c.smithi156.stdout: e1 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout: legacy client fscid: -1 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout: 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout: No filesystems configured 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T18:58:52.238 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:58:52.239 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:58:52.240 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Deploying cephadm binary to smithi156 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.241 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Added host smithi156 2023-07-21T18:58:52.242 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:58:52.243 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.244 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.245 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c;count:3 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.246 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: Deploying daemon mon.c on smithi156 2023-07-21T18:58:52.247 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/3073739754' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expand map: {default=false} 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta from 'false' to 'false' 2023-07-21T18:58:52.248 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expanded map: {default=false} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expand map: {default=info} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta from 'info' to 'info' 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expanded map: {default=info} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expand map: {default=daemon} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expand map: {default=debug} 2023-07-21T18:58:52.249 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T18:58:52.250 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:51 smithi156 ceph-mon[128316]: expand_channel_meta expanded map: {default=debug} 2023-07-21T18:58:52.308 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:58:52.309 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:58:53.032 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:58:57.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:58:57.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: mon.a calling monitor election 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: mon.c calling monitor election 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: fsmap 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:58:57.421 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: mgrmap e13: x(active, since 49s) 2023-07-21T18:58:57.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T18:58:57.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:57.422 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:56 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:58:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: mon.a calling monitor election 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: mon.c calling monitor election 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: fsmap 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:58:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: mgrmap e13: x(active, since 49s) 2023-07-21T18:58:57.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T18:58:57.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:57.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:56 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:57.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 -- 172.21.15.156:0/606167984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 msgr2=0x7f6e90005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:57.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/606167984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6e90005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6ea4100bc0 tx=0x7f6ea000fe60 comp rx=0 tx=0).stop 2023-07-21T18:58:57.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 -- 172.21.15.156:0/606167984 shutdown_connections 2023-07-21T18:58:57.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/606167984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6e90005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:57.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 -- 172.21.15.156:0/606167984 >> 172.21.15.156:0/606167984 conn(0x7f6ea40fb210 msgr2=0x7f6ea40fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:57.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 -- 172.21.15.156:0/606167984 shutdown_connections 2023-07-21T18:58:57.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.578+0000 7f6eab7f4700 1 -- 172.21.15.156:0/606167984 wait complete. 2023-07-21T18:58:57.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.579+0000 7f6eab7f4700 1 Processor -- start 2023-07-21T18:58:57.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.579+0000 7f6eab7f4700 1 -- start start 2023-07-21T18:58:57.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6eab7f4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:57.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6eab7f4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:57.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6eab7f4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea4074790 con 0x7f6e900035c0 2023-07-21T18:58:57.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6eab7f4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea41a0c80 con 0x7f6ea4072d50 2023-07-21T18:58:57.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6ea9590700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:57.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6ea9590700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:45024/0 (socket says 172.21.15.156:45024) 2023-07-21T18:58:57.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.580+0000 7f6ea9590700 1 -- 172.21.15.156:0/116363229 learned_addr learned my addr 172.21.15.156:0/116363229 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea8d8f700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea8d8f700 1 -- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 msgr2=0x7f6ea40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea8d8f700 1 -- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 msgr2=0x7f6ea40731c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea8d8f700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea8d8f700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea9590700 1 -- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 msgr2=0x7f6ea40731c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea9590700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:57.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea9590700 1 -- 172.21.15.156:0/116363229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea000f8b0 con 0x7f6e900035c0 2023-07-21T18:58:57.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.581+0000 7f6ea9590700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0000c00 tx=0x7f6ea0013c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:57.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.582+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6ea0023070 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.582+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ea00041d0 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.582+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f6ea001cb60 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.582+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea41a0e80 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.582+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea41a1370 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.583+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f6ea001cd00 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.583+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea4061b40 con 0x7f6e900035c0 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.584+0000 7f6e9a7fc700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 0x7f6e8803aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:58:57.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.584+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6ea004d460 con 0x7f6e900035c0 2023-07-21T18:58:57.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.584+0000 7f6ea8d8f700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 0x7f6e8803aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:58:57.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.585+0000 7f6ea8d8f700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 0x7f6e8803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009850 tx=0x7f6e94008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:58:57.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.590+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6ea001bb90 con 0x7f6e900035c0 2023-07-21T18:58:57.871 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.860+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f6ea41a1950 con 0x7f6e900035c0 2023-07-21T18:58:57.872 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.861+0000 7f6e9a7fc700 1 -- 172.21.15.156:0/116363229 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1002 (secure 0 0 0) 0x7f6ea001bb90 con 0x7f6e900035c0 2023-07-21T18:58:57.872 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:58:57.872 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":2,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:58:51.920402Z","created":"2023-07-21T18:57:01.506340Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 msgr2=0x7f6e8803aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 0x7f6e8803aa90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6e94009850 tx=0x7f6e94008040 comp rx=0 tx=0).stop 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 msgr2=0x7f6ea4072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f6ea0000c00 tx=0x7f6ea0013c70 comp rx=0 tx=0).stop 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 shutdown_connections 2023-07-21T18:58:57.874 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6e880385e0 0x7f6e8803aa90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:57.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6e900035c0 0x7f6ea4072810 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:57.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 --2- 172.21.15.156:0/116363229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ea4072d50 0x7f6ea40731c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:58:57.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 >> 172.21.15.156:0/116363229 conn(0x7f6ea40fb210 msgr2=0x7f6ea418d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:58:57.875 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 shutdown_connections 2023-07-21T18:58:57.876 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:58:57.863+0000 7f6eab7f4700 1 -- 172.21.15.156:0/116363229 wait complete. 2023-07-21T18:58:57.876 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 2 2023-07-21T18:58:58.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:58.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:58:58.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:58.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: Deploying daemon mon.b on smithi101 2023-07-21T18:58:58.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:58.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/116363229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:58:58.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:58:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:58.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:58:58.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:58:58.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:58:58.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: Deploying daemon mon.b on smithi101 2023-07-21T18:58:58.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:58:58.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/116363229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:58:58.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:58:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:58:59.415 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-07-21T18:58:59.415 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph mon dump -f json 2023-07-21T18:59:00.144 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:59:00.550 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:00 smithi156 ceph-mon[128316]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:00.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:00 smithi005 ceph-mon[126460]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:00 smithi101 systemd[1]: Starting Ceph mon.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:59:01.426 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 podman[128096]: 2023-07-21T18:59:01.426 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 bash[128096]: 8ea5ffd7397fe528af6f57f3faf8c38badb5ffe98f68b763501b2861fc0f759a 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: set uid:gid to 167:167 (ceph:ceph) 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev), process ceph-mon, pid 7 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pidfile_write: ignore empty --pid-file 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: load: jerasure load: lrc 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: RocksDB version: 7.9.2 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Git sha 0 2023-07-21T18:59:01.427 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Compile date 2023-07-17 23:00:14 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: DB SUMMARY 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: DB Session ID: O4YVD2OLTE7FULHQRJ8N 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: CURRENT file: CURRENT 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: IDENTITY file: IDENTITY 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-07-21T18:59:01.428 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.error_if_exists: 0 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.create_if_missing: 0 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.paranoid_checks: 1 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.flush_verify_memtable_count: 1 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.env: 0x55cf464e8720 2023-07-21T18:59:01.429 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.fs: PosixFileSystem 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.info_log: 0x55cf46e39280 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_file_opening_threads: 16 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.statistics: (nil) 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.use_fsync: 0 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_log_file_size: 0 2023-07-21T18:59:01.430 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.log_file_time_to_roll: 0 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.keep_log_file_num: 1000 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.recycle_log_file_num: 0 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_fallocate: 1 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_mmap_reads: 0 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_mmap_writes: 0 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.use_direct_reads: 0 2023-07-21T18:59:01.431 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.create_missing_column_families: 0 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.db_log_dir: 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.wal_dir: 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.table_cache_numshardbits: 6 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.WAL_ttl_seconds: 0 2023-07-21T18:59:01.432 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.WAL_size_limit_MB: 0 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.is_fd_close_on_exec: 1 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.advise_random_on_open: 1 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.db_write_buffer_size: 0 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.write_buffer_manager: 0x55cf46d5a3c0 2023-07-21T18:59:01.433 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.use_adaptive_mutex: 0 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.rate_limiter: (nil) 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.wal_recovery_mode: 2 2023-07-21T18:59:01.434 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enable_thread_tracking: 0 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enable_pipelined_write: 0 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.unordered_write: 0 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.row_cache: None 2023-07-21T18:59:01.435 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.wal_filter: None 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_ingest_behind: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.two_write_queues: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.manual_wal_flush: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.wal_compression: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.atomic_flush: 0 2023-07-21T18:59:01.436 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.persist_stats_to_disk: 0 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.write_dbid_to_manifest: 0 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.log_readahead_size: 0 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.best_efforts_recovery: 0 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-07-21T18:59:01.437 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-07-21T18:59:01.438 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.allow_data_in_errors: 0 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.db_host_id: __hostname__ 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enforce_single_del_contracts: true 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_background_jobs: 2 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_background_compactions: -1 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_subcompactions: 1 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-07-21T18:59:01.439 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.delayed_write_rate : 16777216 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_total_wal_size: 0 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.stats_dump_period_sec: 600 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.stats_persist_period_sec: 600 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_open_files: -1 2023-07-21T18:59:01.440 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bytes_per_sync: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.wal_bytes_per_sync: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.strict_bytes_per_sync: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_readahead_size: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_background_flushes: -1 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Compression algorithms supported: 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kZSTD supported: 0 2023-07-21T18:59:01.441 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kXpressCompression supported: 0 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kLZ4HCCompression supported: 1 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kZlibCompression supported: 1 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kSnappyCompression supported: 1 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kLZ4Compression supported: 1 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: kBZip2Compression supported: 0 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Fast CRC32 supported: Supported on x86 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: DMutex implementation: pthread_mutex_t 2023-07-21T18:59:01.442 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.merge_operator: 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_filter: None 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_filter_factory: None 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.sst_partitioner_factory: None 2023-07-21T18:59:01.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.memtable_factory: SkipListFactory 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.table_factory: BlockBasedTable 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55cf46e39380) 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout: cache_index_and_filter_blocks: 1 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout: pin_top_level_index_and_filter: 1 2023-07-21T18:59:01.444 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_type: 0 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: data_block_index_type: 0 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_shortening: 1 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: data_block_hash_table_util_ratio: 0.750000 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: checksum: 4 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: no_block_cache: 0 2023-07-21T18:59:01.445 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache: 0x55cf46df91f0 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_name: BinnedLRUCache 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_options: 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: capacity : 536870912 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: num_shard_bits : 4 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: strict_capacity_limit : 0 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: high_pri_pool_ratio: 0.000 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_cache_compressed: (nil) 2023-07-21T18:59:01.446 INFO:journalctl@ceph.mon.b.smithi101.stdout: persistent_cache: (nil) 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_size: 4096 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_size_deviation: 10 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_restart_interval: 16 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: index_block_restart_interval: 1 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: metadata_block_size: 4096 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: partition_filters: 0 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: use_delta_encoding: 1 2023-07-21T18:59:01.447 INFO:journalctl@ceph.mon.b.smithi101.stdout: filter_policy: bloomfilter 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: whole_key_filtering: 1 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: verify_compression: 0 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: read_amp_bytes_per_bit: 0 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: format_version: 5 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: enable_index_compression: 1 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: block_align: 0 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: max_auto_readahead_size: 262144 2023-07-21T18:59:01.448 INFO:journalctl@ceph.mon.b.smithi101.stdout: prepopulate_block_cache: 0 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout: initial_auto_readahead_size: 8192 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout: num_file_reads_for_auto_readahead: 2 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.write_buffer_size: 33554432 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_write_buffer_number: 2 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression: NoCompression 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression: Disabled 2023-07-21T18:59:01.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.prefix_extractor: nullptr 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.num_levels: 7 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-07-21T18:59:01.450 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.window_bits: -14 2023-07-21T18:59:01.451 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.level: 32767 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.strategy: 0 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.enabled: false 2023-07-21T18:59:01.452 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.target_file_size_base: 67108864 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.target_file_size_multiplier: 1 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-07-21T18:59:01.453 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-07-21T18:59:01.454 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.arena_block_size: 1048576 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-07-21T18:59:01.455 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.disable_auto_compactions: 0 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-07-21T18:59:01.456 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.table_properties_collectors: 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.inplace_update_support: 0 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.inplace_update_num_locks: 10000 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-07-21T18:59:01.457 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.memtable_huge_page_size: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.bloom_locality: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.max_successive_merges: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.optimize_filters_for_hits: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.paranoid_file_checks: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.force_consistency_checks: 1 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.report_bg_io_stats: 0 2023-07-21T18:59:01.458 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.ttl: 2592000 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.periodic_compaction_seconds: 0 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enable_blob_files: false 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.min_blob_size: 0 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_file_size: 268435456 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_compression_type: NoCompression 2023-07-21T18:59:01.459 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.enable_blob_garbage_collection: false 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.blob_file_starting_level: 0 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-07-21T18:59:01.460 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7e54352f-6a00-46f0-b4c8-1120c4656109 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965941374046, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965941375159, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1689965941, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7e54352f-6a00-46f0-b4c8-1120c4656109", "db_session_id": "O4YVD2OLTE7FULHQRJ8N", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: EVENT_LOG_v1 {"time_micros": 1689965941375378, "job": 1, "event": "recovery_finished"} 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 systemd[1]: Started Ceph mon.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:59:01.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55cf46e92000 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: DB pointer 0x55cf46e7e000 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** DB Stats ** 2023-07-21T18:59:01.462 INFO:journalctl@ceph.mon.b.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.463 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** Compaction Stats [default] ** 2023-07-21T18:59:01.464 INFO:journalctl@ceph.mon.b.smithi101.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.465 INFO:journalctl@ceph.mon.b.smithi101.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: AddFile(Keys): cumulative 0, interval 0 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: Cumulative compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:59:01.466 INFO:journalctl@ceph.mon.b.smithi101.stdout: Interval compaction: 0.00 GB write, 0.23 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout: Block cache BinnedLRUCache@0x55cf46df91f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout: ** File Read Latency Histogram By Level [default] ** 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: using public_addr v2:172.21.15.101:0/0 -> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] 2023-07-21T18:59:01.467 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: starting mon.b rank -1 at public addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] at bind addrs [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(???) e0 preinit fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).mds e1 new map 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).mds e1 print_map 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout: e1 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T18:59:01.468 INFO:journalctl@ceph.mon.b.smithi101.stdout: legacy client fscid: -1 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout: 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout: No filesystems configured 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-07-21T18:59:01.469 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.470 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi101", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi156", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:59:01.471 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Deploying cephadm binary to smithi156 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.472 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.473 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Added host smithi156 2023-07-21T18:59:01.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:01.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/34564910' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi156", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:59:01.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:01.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Saving service mon spec with placement smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c;count:3 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:01.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Deploying daemon mon.c on smithi156 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/3073739754' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.a calling monitor election 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.c calling monitor election 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: monmap e2: 2 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: fsmap 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mgrmap e13: x(active, since 49s) 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T18:59:01.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: Deploying daemon mon.b on smithi101 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/116363229' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:59:01.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:01.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:01.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-07-21T18:59:01.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-07-21T18:59:01.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expand map: {default=false} 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta from 'false' to 'false' 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expanded map: {default=false} 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expand map: {default=info} 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta from 'info' to 'info' 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expanded map: {default=info} 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expand map: {default=daemon} 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta from 'daemon' to 'daemon' 2023-07-21T18:59:01.852 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expanded map: {default=daemon} 2023-07-21T18:59:01.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expand map: {default=debug} 2023-07-21T18:59:01.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta from 'debug' to 'debug' 2023-07-21T18:59:01.853 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:01 smithi101 ceph-mon[128136]: expand_channel_meta expanded map: {default=debug} 2023-07-21T18:59:06.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: mon.a calling monitor election 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: mon.c calling monitor election 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: mon.b calling monitor election 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:59:06.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: fsmap 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: mgrmap e13: x(active, since 59s) 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:59:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: mon.a calling monitor election 2023-07-21T18:59:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: mon.c calling monitor election 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: mon.b calling monitor election 2023-07-21T18:59:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: fsmap 2023-07-21T18:59:06.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: mgrmap e13: x(active, since 59s) 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: mon.a calling monitor election 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: mon.c calling monitor election 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: mon.b calling monitor election 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-07-21T18:59:06.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: monmap e3: 3 mons at {a=[v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0],b=[v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0],c=[v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0]} removed_ranks: {} 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: fsmap 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: osdmap e4: 0 total, 0 up, 0 in 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: mgrmap e13: x(active, since 59s) 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:06.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:07.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 -- 172.21.15.156:0/2142270264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4548006230 msgr2=0x7f4548008690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:07.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 --2- 172.21.15.156:0/2142270264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4548006230 0x7f4548008690 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f45601007f0 tx=0x7f455000fe20 comp rx=0 tx=0).stop 2023-07-21T18:59:07.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 -- 172.21.15.156:0/2142270264 shutdown_connections 2023-07-21T18:59:07.621 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 --2- 172.21.15.156:0/2142270264 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4548006230 0x7f4548008690 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 -- 172.21.15.156:0/2142270264 >> 172.21.15.156:0/2142270264 conn(0x7f45600fb210 msgr2=0x7f45600fd620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:07.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 -- 172.21.15.156:0/2142270264 shutdown_connections 2023-07-21T18:59:07.622 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.609+0000 7f4565a8e700 1 -- 172.21.15.156:0/2142270264 wait complete. 2023-07-21T18:59:07.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.611+0000 7f4565a8e700 1 Processor -- start 2023-07-21T18:59:07.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.611+0000 7f4565a8e700 1 -- start start 2023-07-21T18:59:07.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 0x7f456019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:07.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:07.623 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4560195060 0x7f4560195500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456019b5a0 con 0x7f456019af90 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601a1870 con 0x7f4548006230 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f4565a8e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45601a19b0 con 0x7f4560195060 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f455effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f455effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:34788/0 (socket says 172.21.15.156:34788) 2023-07-21T18:59:07.624 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.612+0000 7f455effd700 1 -- 172.21.15.156:0/3226606358 learned_addr learned my addr 172.21.15.156:0/3226606358 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T18:59:07.625 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455f7fe700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 0x7f456019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455ffff700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4560195060 0x7f4560195500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 -- 172.21.15.156:0/3226606358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4560195060 msgr2=0x7f4560195500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4560195060 0x7f4560195500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 -- 172.21.15.156:0/3226606358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 msgr2=0x7f456019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 0x7f456019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 -- 172.21.15.156:0/3226606358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455000f870 con 0x7f456019af90 2023-07-21T18:59:07.626 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455f7fe700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 0x7f456019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455effd700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f455400b780 tx=0x7f455400bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554004230 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4554004390 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45601a1c90 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.613+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554011a60 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.614+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45601a2190 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.614+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456018f130 con 0x7f456019af90 2023-07-21T18:59:07.627 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.615+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f45540036a0 con 0x7f456019af90 2023-07-21T18:59:07.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.615+0000 7f455cff9700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 0x7f454c03b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:07.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.615+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f455404ae10 con 0x7f456019af90 2023-07-21T18:59:07.628 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.616+0000 7f455f7fe700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 0x7f454c03b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:07.629 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.616+0000 7f455f7fe700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 0x7f454c03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4550004fd0 tx=0x7f455000ffa0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:07.632 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.621+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4554004500 con 0x7f456019af90 2023-07-21T18:59:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:07.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:07.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:07.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.909+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4560061b40 con 0x7f456019af90 2023-07-21T18:59:07.920 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.910+0000 7f455cff9700 1 -- 172.21.15.156:0/3226606358 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1272 (secure 0 0 0) 0x7f4554018e10 con 0x7f456019af90 2023-07-21T18:59:07.921 INFO:teuthology.orchestra.run.smithi156.stdout: 2023-07-21T18:59:07.921 INFO:teuthology.orchestra.run.smithi156.stdout:{"epoch":3,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","modified":"2023-07-21T18:59:01.425988Z","created":"2023-07-21T18:57:01.506340Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:3300","nonce":0},{"type":"v1","addr":"172.21.15.5:6789","nonce":0}]},"addr":"172.21.15.5:6789/0","public_addr":"172.21.15.5:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:3300","nonce":0},{"type":"v1","addr":"172.21.15.156:6789","nonce":0}]},"addr":"172.21.15.156:6789/0","public_addr":"172.21.15.156:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:3300","nonce":0},{"type":"v1","addr":"172.21.15.101:6789","nonce":0}]},"addr":"172.21.15.101:6789/0","public_addr":"172.21.15.101:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-07-21T18:59:07.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 msgr2=0x7f454c03b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:07.923 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 0x7f454c03b130 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4550004fd0 tx=0x7f455000ffa0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 msgr2=0x7f4560194b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f455400b780 tx=0x7f455400bb40 comp rx=0 tx=0).stop 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 shutdown_connections 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f454c038c80 0x7f454c03b130 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4560195060 0x7f4560195500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4548006230 0x7f456019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.924 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 --2- 172.21.15.156:0/3226606358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f456019af90 0x7f4560194b20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 >> 172.21.15.156:0/3226606358 conn(0x7f45600fb210 msgr2=0x7f45601013b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.912+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 shutdown_connections 2023-07-21T18:59:07.925 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:07.913+0000 7f4565a8e700 1 -- 172.21.15.156:0/3226606358 wait complete. 2023-07-21T18:59:07.926 INFO:teuthology.orchestra.run.smithi156.stderr:dumped monmap epoch 3 2023-07-21T18:59:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T18:59:08.633 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-07-21T18:59:08.633 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph config generate-minimal-conf 2023-07-21T18:59:08.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:08 smithi005 ceph-mon[126460]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:08.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/3226606358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:59:08.704 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 18:59:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T18:59:08.426+0000 7f00c3678700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-07-21T18:59:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:08 smithi101 ceph-mon[128136]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/3226606358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:59:08.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:08 smithi156 ceph-mon[128316]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:08.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/3226606358' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-07-21T18:59:09.403 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:59:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:09.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:09.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.851 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:09.852 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:09.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:09.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T18:59:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:59:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:59:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:10.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:10.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:10.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T18:59:10.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:59:10.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:10.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:10.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T18:59:10.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T18:59:10.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:10.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:10.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.001+0000 7f2f42996700 1 -- 172.21.15.5:0/2636874891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 msgr2=0x7f2f3c105a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.001+0000 7f2f42996700 1 --2- 172.21.15.5:0/2636874891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c105a00 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c009910 tx=0x7f2f2c009c20 comp rx=0 tx=0).stop 2023-07-21T18:59:11.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.001+0000 7f2f42996700 1 -- 172.21.15.5:0/2636874891 shutdown_connections 2023-07-21T18:59:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.001+0000 7f2f42996700 1 --2- 172.21.15.5:0/2636874891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c105a00 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.001+0000 7f2f42996700 1 -- 172.21.15.5:0/2636874891 >> 172.21.15.5:0/2636874891 conn(0x7f2f3c0fb130 msgr2=0x7f2f3c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.002+0000 7f2f42996700 1 -- 172.21.15.5:0/2636874891 shutdown_connections 2023-07-21T18:59:11.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.002+0000 7f2f42996700 1 -- 172.21.15.5:0/2636874891 wait complete. 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.003+0000 7f2f42996700 1 Processor -- start 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.003+0000 7f2f42996700 1 -- start start 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f3c19ada0 0x7f2f3c1948e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f3c194e20 0x7f2f3c1952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:11.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c19b3b0 con 0x7f2f3c105630 2023-07-21T18:59:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1a1680 con 0x7f2f3c19ada0 2023-07-21T18:59:11.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f42996700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f3c1a17c0 con 0x7f2f3c194e20 2023-07-21T18:59:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49910/0 (socket says 172.21.15.5:49910) 2023-07-21T18:59:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f3bfff700 1 -- 172.21.15.5:0/2177656570 learned_addr learned my addr 172.21.15.5:0/2177656570 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:59:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.004+0000 7f2f3b7fe700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f3c19ada0 0x7f2f3c1948e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:11.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f40f33700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f3c194e20 0x7f2f3c1952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 -- 172.21.15.5:0/2177656570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f3c194e20 msgr2=0x7f2f3c1952c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f3c194e20 0x7f2f3c1952c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 -- 172.21.15.5:0/2177656570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f3c19ada0 msgr2=0x7f2f3c1948e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f3c19ada0 0x7f2f3c1948e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 -- 172.21.15.5:0/2177656570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f2c0095f0 con 0x7f2f3c105630 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.005+0000 7f2f3bfff700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bcf0 tx=0x7f2f2c00bd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:11.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.006+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c01d070 con 0x7f2f3c105630 2023-07-21T18:59:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.006+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f2c004db0 con 0x7f2f3c105630 2023-07-21T18:59:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.006+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c1a1aa0 con 0x7f2f3c105630 2023-07-21T18:59:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.006+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f2c022bd0 con 0x7f2f3c105630 2023-07-21T18:59:11.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.006+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c1a1fa0 con 0x7f2f3c105630 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.007+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f2f2c00f460 con 0x7f2f3c105630 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.007+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f3c061b40 con 0x7f2f3c105630 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.007+0000 7f2f397fa700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 0x7f2f2403b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.007+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f2f2c04c9e0 con 0x7f2f3c105630 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.008+0000 7f2f3b7fe700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 0x7f2f2403b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:11.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.008+0000 7f2f3b7fe700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 0x7f2f2403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ad0 tx=0x7f2f30009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:11.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.013+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2f2c020960 con 0x7f2f3c105630 2023-07-21T18:59:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.227+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f2f3c195df0 con 0x7f2f3c105630 2023-07-21T18:59:11.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.228+0000 7f2f397fa700 1 -- 172.21.15.5:0/2177656570 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7f2f2c027030 con 0x7f2f3c105630 2023-07-21T18:59:11.238 INFO:teuthology.orchestra.run.smithi005.stdout:# minimal ceph.conf for 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:59:11.239 INFO:teuthology.orchestra.run.smithi005.stdout:[global] 2023-07-21T18:59:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: fsid = 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T18:59:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: mon_host = [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] 2023-07-21T18:59:11.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.230+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 msgr2=0x7f2f2403b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.230+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 0x7f2f2403b130 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009ad0 tx=0x7f2f30009210 comp rx=0 tx=0).stop 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.230+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 msgr2=0x7f2f3c19a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.230+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2f2c00bcf0 tx=0x7f2f2c00bd20 comp rx=0 tx=0).stop 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 shutdown_connections 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f24038c80 0x7f2f2403b130 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f3c194e20 0x7f2f3c1952c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f3c105630 0x7f2f3c19a860 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 --2- 172.21.15.5:0/2177656570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f3c19ada0 0x7f2f3c1948e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 >> 172.21.15.5:0/2177656570 conn(0x7f2f3c0fb130 msgr2=0x7f2f3c0fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 shutdown_connections 2023-07-21T18:59:11.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:11.231+0000 7f2f42996700 1 -- 172.21.15.5:0/2177656570 wait complete. 2023-07-21T18:59:11.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:11 smithi005 ceph-mon[126460]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:59:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:11 smithi005 ceph-mon[126460]: Reconfiguring daemon mon.a on smithi005 2023-07-21T18:59:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:11 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2177656570' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:11 smithi101 ceph-mon[128136]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:59:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:11 smithi101 ceph-mon[128136]: Reconfiguring daemon mon.a on smithi005 2023-07-21T18:59:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:11 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2177656570' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:11 smithi156 ceph-mon[128316]: Reconfiguring mon.a (unknown last config time)... 2023-07-21T18:59:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:11 smithi156 ceph-mon[128316]: Reconfiguring daemon mon.a on smithi005 2023-07-21T18:59:11.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:11 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2177656570' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:12.192 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-07-21T18:59:12.192 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:59:12.193 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:59:12.264 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:59:12.264 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:59:12.373 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:59:12.374 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:59:12.445 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:59:12.445 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:59:12.563 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:59:12.563 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.conf 2023-07-21T18:59:12.642 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:59:12.643 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-07-21T18:59:12.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:12.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:12.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:12.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.687 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:12.687 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:12.687 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:12.753 INFO:tasks.cephadm:Adding mgr.x on smithi005 2023-07-21T18:59:12.753 INFO:tasks.cephadm:Adding mgr.y on smithi101 2023-07-21T18:59:12.754 INFO:tasks.cephadm:Adding mgr.z on smithi156 2023-07-21T18:59:12.754 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch apply mgr '3;smithi005=x;smithi101=y;smithi156=z' 2023-07-21T18:59:12.854 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:12.854 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.854 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:12.854 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:12.854 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:12.855 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:13.510 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T18:59:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:59:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: Reconfiguring daemon mon.b on smithi101 2023-07-21T18:59:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:13.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:13.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:13.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: Reconfiguring daemon mon.b on smithi101 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:13.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:13.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: Reconfiguring mon.b (monmap changed)... 2023-07-21T18:59:13.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: Reconfiguring daemon mon.b on smithi101 2023-07-21T18:59:13.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:13.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-07-21T18:59:13.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-07-21T18:59:13.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:14 smithi101 ceph-mon[128136]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:59:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:14 smithi101 ceph-mon[128136]: Reconfiguring daemon mon.c on smithi156 2023-07-21T18:59:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:14 smithi101 ceph-mon[128136]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:14.901 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:14 smithi156 ceph-mon[128316]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:59:14.901 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:14 smithi156 ceph-mon[128316]: Reconfiguring daemon mon.c on smithi156 2023-07-21T18:59:14.901 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:14 smithi156 ceph-mon[128316]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:14.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:14 smithi005 ceph-mon[126460]: Reconfiguring mon.c (monmap changed)... 2023-07-21T18:59:14.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:14 smithi005 ceph-mon[126460]: Reconfiguring daemon mon.c on smithi156 2023-07-21T18:59:14.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:14 smithi005 ceph-mon[126460]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:15.296 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 -- 172.21.15.156:0/2669773378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc100d90 msgr2=0x7efcfc1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 --2- 172.21.15.156:0/2669773378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc100d90 0x7efcfc1011e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009990 tx=0x7efcf0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 -- 172.21.15.156:0/2669773378 shutdown_connections 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 --2- 172.21.15.156:0/2669773378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc101720 0x7efcfc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 --2- 172.21.15.156:0/2669773378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc0ffb90 0x7efcfc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 --2- 172.21.15.156:0/2669773378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc100d90 0x7efcfc1011e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 -- 172.21.15.156:0/2669773378 >> 172.21.15.156:0/2669773378 conn(0x7efcfc0fb1a0 msgr2=0x7efcfc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:15.297 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 -- 172.21.15.156:0/2669773378 shutdown_connections 2023-07-21T18:59:15.298 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.285+0000 7efd00b00700 1 -- 172.21.15.156:0/2669773378 wait complete. 2023-07-21T18:59:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.286+0000 7efd00b00700 1 Processor -- start 2023-07-21T18:59:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 -- start start 2023-07-21T18:59:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 0x7efcfc197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc100d90 0x7efcfc197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:15.299 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19f5c0 con 0x7efcfc101720 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19f700 con 0x7efcfc100d90 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efd00b00700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcfc19f890 con 0x7efcfc0ffb90 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efcfad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efcfad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:51762/0 (socket says 172.21.15.156:51762) 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efcfad9d700 1 -- 172.21.15.156:0/294919098 learned_addr learned my addr 172.21.15.156:0/294919098 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T18:59:15.300 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.287+0000 7efcf9d9b700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc100d90 0x7efcfc197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfa59c700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 0x7efcfc197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 -- 172.21.15.156:0/294919098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 msgr2=0x7efcfc197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 0x7efcfc197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 -- 172.21.15.156:0/294919098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc100d90 msgr2=0x7efcfc197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc100d90 0x7efcfc197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 -- 172.21.15.156:0/294919098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf00095f0 con 0x7efcfc101720 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfa59c700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 0x7efcfc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:59:15.301 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efcfad9d700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b9a0 tx=0x7efcf400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:15.302 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf400c930 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcfc19fb70 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.288+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf4014440 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.289+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf40075d0 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.289+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcfc1a00c0 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.290+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcfc192140 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.290+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7efcf40145b0 con 0x7efcfc101720 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.290+0000 7efceb7fe700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 0x7efce403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:15.303 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.290+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efcf4006e40 con 0x7efcfc101720 2023-07-21T18:59:15.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.291+0000 7efcfa59c700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 0x7efce403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:15.304 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.291+0000 7efcfa59c700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 0x7efce403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efcfc100bf0 tx=0x7efcec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:15.310 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.300+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efcf4013810 con 0x7efcfc101720 2023-07-21T18:59:15.515 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.504+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi101=y;smithi156=z", "target": ["mon-mgr", ""]}) v1 -- 0x7efcfc0609e0 con 0x7efce40389e0 2023-07-21T18:59:15.522 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.512+0000 7efceb7fe700 1 -- 172.21.15.156:0/294919098 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efcfc0609e0 con 0x7efce40389e0 2023-07-21T18:59:15.523 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mgr update... 2023-07-21T18:59:15.525 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.514+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 msgr2=0x7efce403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.514+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 0x7efce403ae90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efcfc100bf0 tx=0x7efcec009260 comp rx=0 tx=0).stop 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.514+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 msgr2=0x7efcfc19eff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.514+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7efcf400b9a0 tx=0x7efcf400bcb0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 shutdown_connections 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efce40389e0 0x7efce403ae90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efcfc101720 0x7efcfc19eff0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.526 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efcfc0ffb90 0x7efcfc197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 --2- 172.21.15.156:0/294919098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efcfc100d90 0x7efcfc197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:15.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 >> 172.21.15.156:0/294919098 conn(0x7efcfc0fb1a0 msgr2=0x7efcfc106740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:15.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 shutdown_connections 2023-07-21T18:59:15.527 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T18:59:15.515+0000 7efd00b00700 1 -- 172.21.15.156:0/294919098 wait complete. 2023-07-21T18:59:15.995 DEBUG:teuthology.orchestra.run.smithi101:mgr.y> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.y.service 2023-07-21T18:59:16.000 DEBUG:teuthology.orchestra.run.smithi156:mgr.z> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.z.service 2023-07-21T18:59:16.006 INFO:tasks.cephadm:Deploying OSDs... 2023-07-21T18:59:16.006 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T18:59:16.006 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:59:16.050 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:59:16.050 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_1 2023-07-21T18:59:16.075 INFO:journalctl@ceph.mgr.y.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T18:59:16.081 INFO:journalctl@ceph.mgr.z.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T18:59:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:59:16.134 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:16.134 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 449 Links: 1 2023-07-21T18:59:16.134 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:16.135 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:16.135 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-07-21 18:58:16.567368769 +0000 2023-07-21T18:59:16.135 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-07-21 18:38:45.099618184 +0000 2023-07-21T18:59:16.135 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-07-21 18:38:45.099618184 +0000 2023-07-21T18:59:16.135 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-07-21 18:38:45.075618655 +0000 2023-07-21T18:59:16.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:59:16.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:16.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:16.235 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi101=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: Saving service mgr spec with placement smithi005=x;smithi101=y;smithi156=z;count:3 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:16.236 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:16.237 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.241 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-07-21T18:59:16.242 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-07-21T18:59:16.242 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000227488 s, 2.3 MB/s 2023-07-21T18:59:16.244 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:59:16.289 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_2 2023-07-21T18:59:16.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:16.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:16.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi101=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: Saving service mgr spec with placement smithi005=x;smithi101=y;smithi156=z;count:3 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:16.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:16.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:16.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.374 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:59:16.374 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:16.374 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 455 Links: 1 2023-07-21T18:59:16.374 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:16.375 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:16.375 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-07-21 18:58:16.567368769 +0000 2023-07-21T18:59:16.375 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-07-21 18:38:45.919602085 +0000 2023-07-21T18:59:16.375 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-07-21 18:38:45.919602085 +0000 2023-07-21T18:59:16.375 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-07-21 18:38:45.896602537 +0000 2023-07-21T18:59:16.375 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:59:16.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:16.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi005=x;smithi101=y;smithi156=z", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: Saving service mgr spec with placement smithi005=x;smithi101=y;smithi156=z;count:3 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:16.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:16.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:16.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:16.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-07-21T18:59:16.480 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-07-21T18:59:16.481 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.00020062 s, 2.6 MB/s 2023-07-21T18:59:16.483 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:59:16.527 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_3 2023-07-21T18:59:16.611 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:59:16.612 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:16.612 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 461 Links: 1 2023-07-21T18:59:16.612 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:16.612 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:16.612 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-07-21 18:58:16.567368769 +0000 2023-07-21T18:59:16.613 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-07-21 18:38:46.753585712 +0000 2023-07-21T18:59:16.613 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-07-21 18:38:46.753585712 +0000 2023-07-21T18:59:16.613 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-07-21 18:38:46.730586163 +0000 2023-07-21T18:59:16.613 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:59:16.718 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-07-21T18:59:16.718 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-07-21T18:59:16.718 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000188765 s, 2.7 MB/s 2023-07-21T18:59:16.720 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:59:16.765 DEBUG:teuthology.orchestra.run.smithi005:> stat /dev/vg_nvme/lv_4 2023-07-21T18:59:16.849 INFO:teuthology.orchestra.run.smithi005.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:59:16.849 INFO:teuthology.orchestra.run.smithi005.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:16.849 INFO:teuthology.orchestra.run.smithi005.stdout:Device: 5h/5d Inode: 467 Links: 1 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout:Access: 2023-07-21 18:58:16.567368769 +0000 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout:Modify: 2023-07-21 18:38:47.578569515 +0000 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout:Change: 2023-07-21 18:38:47.578569515 +0000 2023-07-21T18:59:16.850 INFO:teuthology.orchestra.run.smithi005.stdout: Birth: 2023-07-21 18:38:47.555569967 +0000 2023-07-21T18:59:16.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:59:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records in 2023-07-21T18:59:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:1+0 records out 2023-07-21T18:59:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:512 bytes copied, 0.000188158 s, 2.7 MB/s 2023-07-21T18:59:16.958 DEBUG:teuthology.orchestra.run.smithi005:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:59:17.004 DEBUG:teuthology.orchestra.run.smithi101:> set -ex 2023-07-21T18:59:17.004 DEBUG:teuthology.orchestra.run.smithi101:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:59:17.051 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:59:17.052 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_1 2023-07-21T18:59:17.137 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 449 Links: 1 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-21 18:59:00.480983851 +0000 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-21 18:47:35.490995630 +0000 2023-07-21T18:59:17.138 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-21 18:47:35.490995630 +0000 2023-07-21T18:59:17.139 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-21 18:47:35.462996272 +0000 2023-07-21T18:59:17.139 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:59:17.238 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:16 smithi101 ceph-mon[128136]: Deploying daemon mgr.z on smithi156 2023-07-21T18:59:17.244 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-21T18:59:17.245 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-21T18:59:17.245 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000306343 s, 1.7 MB/s 2023-07-21T18:59:17.247 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:59:17.291 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_2 2023-07-21T18:59:17.326 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:16 smithi156 ceph-mon[128316]: Deploying daemon mgr.z on smithi156 2023-07-21T18:59:17.374 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 455 Links: 1 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-21 18:59:00.481983829 +0000 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-21 18:47:36.305976961 +0000 2023-07-21T18:59:17.375 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-21 18:47:36.305976961 +0000 2023-07-21T18:59:17.376 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-21 18:47:36.279977556 +0000 2023-07-21T18:59:17.376 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:59:17.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:16 smithi005 ceph-mon[126460]: Deploying daemon mgr.z on smithi156 2023-07-21T18:59:17.478 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-21T18:59:17.479 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-21T18:59:17.479 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000462732 s, 1.1 MB/s 2023-07-21T18:59:17.481 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:59:17.526 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_3 2023-07-21T18:59:17.610 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:59:17.610 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:17.610 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 461 Links: 1 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-21 18:59:00.481983829 +0000 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-21 18:47:37.137957902 +0000 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-21 18:47:37.137957902 +0000 2023-07-21T18:59:17.611 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-21 18:47:37.110958520 +0000 2023-07-21T18:59:17.612 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:59:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-21T18:59:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-21T18:59:17.715 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000300119 s, 1.7 MB/s 2023-07-21T18:59:17.717 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:59:17.763 DEBUG:teuthology.orchestra.run.smithi101:> stat /dev/vg_nvme/lv_4 2023-07-21T18:59:17.847 INFO:teuthology.orchestra.run.smithi101.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:59:17.848 INFO:teuthology.orchestra.run.smithi101.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:17.848 INFO:teuthology.orchestra.run.smithi101.stdout:Device: 5h/5d Inode: 467 Links: 1 2023-07-21T18:59:17.848 INFO:teuthology.orchestra.run.smithi101.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:17.848 INFO:teuthology.orchestra.run.smithi101.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:17.849 INFO:teuthology.orchestra.run.smithi101.stdout:Access: 2023-07-21 18:59:00.481983829 +0000 2023-07-21T18:59:17.849 INFO:teuthology.orchestra.run.smithi101.stdout:Modify: 2023-07-21 18:47:37.970938820 +0000 2023-07-21T18:59:17.849 INFO:teuthology.orchestra.run.smithi101.stdout:Change: 2023-07-21 18:47:37.970938820 +0000 2023-07-21T18:59:17.849 INFO:teuthology.orchestra.run.smithi101.stdout: Birth: 2023-07-21 18:47:37.943939438 +0000 2023-07-21T18:59:17.850 DEBUG:teuthology.orchestra.run.smithi101:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:59:17.952 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records in 2023-07-21T18:59:17.953 INFO:teuthology.orchestra.run.smithi101.stderr:1+0 records out 2023-07-21T18:59:17.953 INFO:teuthology.orchestra.run.smithi101.stderr:512 bytes copied, 0.000294819 s, 1.7 MB/s 2023-07-21T18:59:17.955 DEBUG:teuthology.orchestra.run.smithi101:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:59:18.000 DEBUG:teuthology.orchestra.run.smithi156:> set -ex 2023-07-21T18:59:18.001 DEBUG:teuthology.orchestra.run.smithi156:> dd if=/scratch_devs of=/dev/stdout 2023-07-21T18:59:18.048 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-07-21T18:59:18.049 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_1 2023-07-21T18:59:18.136 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-07-21T18:59:18.136 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 449 Links: 1 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-21 18:59:17.802702376 +0000 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-21 18:38:23.341002626 +0000 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-21 18:38:23.341002626 +0000 2023-07-21T18:59:18.137 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-07-21 18:38:23.316002968 +0000 2023-07-21T18:59:18.138 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-07-21T18:59:18.184 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:17 smithi156 systemd[1]: Starting Ceph mgr.z for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:59:18.253 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-21T18:59:18.253 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-21T18:59:18.253 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000211626 s, 2.4 MB/s 2023-07-21T18:59:18.255 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-07-21T18:59:18.307 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_2 2023-07-21T18:59:18.354 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-07-21T18:59:18.355 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:18.355 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 455 Links: 1 2023-07-21T18:59:18.355 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:18.355 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:18.356 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-21 18:59:17.803702361 +0000 2023-07-21T18:59:18.356 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-21 18:38:24.177991117 +0000 2023-07-21T18:59:18.356 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-21 18:38:24.177991117 +0000 2023-07-21T18:59:18.357 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-07-21 18:38:24.154991435 +0000 2023-07-21T18:59:18.357 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-07-21T18:59:18.424 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-21T18:59:18.425 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-21T18:59:18.425 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000165098 s, 3.1 MB/s 2023-07-21T18:59:18.427 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-07-21T18:59:18.472 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_3 2023-07-21T18:59:18.472 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:18 smithi156 ceph-mon[128316]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:18.558 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-07-21T18:59:18.558 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 461 Links: 1 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-21 18:59:17.803702361 +0000 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-21 18:38:25.020979448 +0000 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-21 18:38:25.020979448 +0000 2023-07-21T18:59:18.559 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-07-21 18:38:24.998979752 +0000 2023-07-21T18:59:18.560 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-07-21T18:59:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:18 smithi101 ceph-mon[128136]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:18 smithi005 ceph-mon[126460]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:18.675 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-21T18:59:18.675 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-21T18:59:18.675 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.000187908 s, 2.7 MB/s 2023-07-21T18:59:18.677 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-07-21T18:59:18.724 DEBUG:teuthology.orchestra.run.smithi156:> stat /dev/vg_nvme/lv_4 2023-07-21T18:59:18.747 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:18 smithi156 podman[130228]: 2023-07-21T18:59:18.770 INFO:teuthology.orchestra.run.smithi156.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-07-21T18:59:18.770 INFO:teuthology.orchestra.run.smithi156.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-07-21T18:59:18.770 INFO:teuthology.orchestra.run.smithi156.stdout:Device: 5h/5d Inode: 467 Links: 1 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout:Context: system_u:object_r:device_t:s0 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout:Access: 2023-07-21 18:59:17.803702361 +0000 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout:Modify: 2023-07-21 18:38:25.864967765 +0000 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout:Change: 2023-07-21 18:38:25.864967765 +0000 2023-07-21T18:59:18.771 INFO:teuthology.orchestra.run.smithi156.stdout: Birth: 2023-07-21 18:38:25.843968056 +0000 2023-07-21T18:59:18.772 DEBUG:teuthology.orchestra.run.smithi156:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-07-21T18:59:18.888 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records in 2023-07-21T18:59:18.889 INFO:teuthology.orchestra.run.smithi156.stderr:1+0 records out 2023-07-21T18:59:18.889 INFO:teuthology.orchestra.run.smithi156.stderr:512 bytes copied, 0.00059001 s, 868 kB/s 2023-07-21T18:59:18.892 DEBUG:teuthology.orchestra.run.smithi156:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-07-21T18:59:18.943 INFO:tasks.cephadm:Deploying osd.0 on smithi005 with /dev/vg_nvme/lv_4... 2023-07-21T18:59:18.943 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T18:59:19.048 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:18 smithi156 bash[130228]: 84611237cacf9b8e218a1702105816a2f53f1a7a377d61e90d606a90ab0dc57b 2023-07-21T18:59:19.048 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:18 smithi156 ceph-mgr[130412]: -- 172.21.15.156:0/521659983 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x563cccedb1e0 con 0x563ccceb0800 2023-07-21T18:59:19.049 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:18 smithi156 systemd[1]: Started Ceph mgr.z for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:59:19.049 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:18.931+0000 7f97a9477200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:59:19.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.037+0000 7f97a9477200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:59:19.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.151+0000 7f97a9477200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:59:19.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.268+0000 7f97a9477200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:59:19.678 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:59:19.888 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.485+0000 7f97a9477200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:59:20.163 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.877+0000 7f97a9477200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:19.992+0000 7f97a9477200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:20.152+0000 7f97a9477200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:20.165 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:20.165 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:20.165 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:20.165 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: Deploying daemon mgr.y on smithi101 2023-07-21T18:59:20.165 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:19 smithi156 ceph-mon[128316]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:20.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.229 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: Deploying daemon mgr.y on smithi101 2023-07-21T18:59:20.230 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:19 smithi005 ceph-mon[126460]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:20.311 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.311 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.311 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: Deploying daemon mgr.y on smithi101 2023-07-21T18:59:20.312 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:19 smithi101 ceph-mon[128136]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:20.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:20.273+0000 7f97a9477200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:59:21.180 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:20.809+0000 7f97a9477200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:59:21.342 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:21 smithi101 systemd[1]: Starting Ceph mgr.y for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T18:59:21.552 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T18:59:21.552 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T18:59:21.552 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-07-21T18:59:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-07-21T18:59:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0301231 s, 348 MB/s 2023-07-21T18:59:21.553 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 2023-07-21T18:59:21.553 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-07-21T18:59:21.930 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:21.584+0000 7f97a9477200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:59:21.930 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:21.791+0000 7f97a9477200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:59:21.988 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:21 smithi101 podman[129574]: 2023-07-21T18:59:21.988 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:21 smithi101 bash[129574]: f53b2c591fe58dc47ce01355edb787c5b1624c0104f334e4030df406a5506a21 2023-07-21T18:59:22.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi005:vg_nvme/lv_4 2023-07-21T18:59:22.271 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:21 smithi101 ceph-mgr[129614]: -- 172.21.15.101:0/2596913960 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b36bcd71e0 con 0x55b36bcacc00 2023-07-21T18:59:22.271 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 systemd[1]: Started Ceph mgr.y for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T18:59:22.271 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:22.156+0000 7fb1422a6200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T18:59:22.591 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:22.260+0000 7fb1422a6200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T18:59:22.591 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:22.381+0000 7fb1422a6200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T18:59:22.591 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:22.499+0000 7fb1422a6200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:22.797 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T18:59:23.091 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:22.727+0000 7fb1422a6200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T18:59:23.131 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:22.762+0000 7f97a9477200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:59:23.131 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:22.889+0000 7f97a9477200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:59:23.131 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:23.119+0000 7f97a9477200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:59:23.387 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:23 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:23.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:23 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:23.461 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:23 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:23.462 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:23.147+0000 7fb1422a6200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T18:59:23.462 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:23.269+0000 7fb1422a6200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T18:59:23.798 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:23.495+0000 7f97a9477200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:59:23.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:23.450+0000 7fb1422a6200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T18:59:23.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:23.576+0000 7fb1422a6200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T18:59:24.180 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:23.787+0000 7f97a9477200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:59:24.180 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:23.987+0000 7f97a9477200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:59:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 -- 172.21.15.5:0/450913989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00072f70 msgr2=0x7f5f000733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 --2- 172.21.15.5:0/450913989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00072f70 0x7f5f000733e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0009990 tx=0x7f5ef0009ca0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 -- 172.21.15.5:0/450913989 shutdown_connections 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 --2- 172.21.15.5:0/450913989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00107a60 0x7f5f00109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 --2- 172.21.15.5:0/450913989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00074540 0x7f5f000729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 --2- 172.21.15.5:0/450913989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00072f70 0x7f5f000733e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 -- 172.21.15.5:0/450913989 >> 172.21.15.5:0/450913989 conn(0x7f5f000fb1a0 msgr2=0x7f5f000fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:24.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 -- 172.21.15.5:0/450913989 shutdown_connections 2023-07-21T18:59:24.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.283+0000 7f5f07c72700 1 -- 172.21.15.5:0/450913989 wait complete. 2023-07-21T18:59:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.284+0000 7f5f07c72700 1 Processor -- start 2023-07-21T18:59:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.284+0000 7f5f07c72700 1 -- start start 2023-07-21T18:59:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.284+0000 7f5f07c72700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 0x7f5f001979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.284+0000 7f5f07c72700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f07c72700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00107a60 0x7f5f0019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f07c72700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0019f570 con 0x7f5f00074540 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f07c72700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0019f6b0 con 0x7f5f00072f70 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f07c72700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f0019f840 con 0x7f5f00107a60 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47832/0 (socket says 172.21.15.5:47832) 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 -- 172.21.15.5:0/300958612 learned_addr learned my addr 172.21.15.5:0/300958612 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0620f700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00107a60 0x7f5f0019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f05a0e700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 0x7f5f001979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 -- 172.21.15.5:0/300958612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00107a60 msgr2=0x7f5f0019efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00107a60 0x7f5f0019efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 -- 172.21.15.5:0/300958612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 msgr2=0x7f5f001979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 0x7f5f001979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f0520d700 1 -- 172.21.15.5:0/300958612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00095f0 con 0x7f5f00074540 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.285+0000 7f5f05a0e700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 0x7f5f001979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5f0520d700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0000c00 tx=0x7f5ef0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef001d070 con 0x7f5f00074540 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ef000bdb0 con 0x7f5f00074540 2023-07-21T18:59:24.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0019fac0 con 0x7f5f00074540 2023-07-21T18:59:24.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ef0022e90 con 0x7f5f00074540 2023-07-21T18:59:24.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.286+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0019fed0 con 0x7f5f00074540 2023-07-21T18:59:24.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.287+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 13) v1 ==== 45538+0+0 (secure 0 0 0) 0x7f5ef000b920 con 0x7f5f00074540 2023-07-21T18:59:24.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.287+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f00106110 con 0x7f5f00074540 2023-07-21T18:59:24.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.288+0000 7f5ef6ffd700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 0x7f5eec03ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:24.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.288+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5ef004c950 con 0x7f5f00074540 2023-07-21T18:59:24.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.288+0000 7f5f05a0e700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 0x7f5eec03ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:24.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.288+0000 7f5f05a0e700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 0x7f5eec03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009be0 tx=0x7f5efc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:24.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.293+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5ef0022600 con 0x7f5f00074540 2023-07-21T18:59:24.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:24 smithi005 ceph-mon[126460]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:24.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:24.514+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f5f00060950 con 0x7f5eec0389e0 2023-07-21T18:59:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:24 smithi101 ceph-mon[128136]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:24.591 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:24.156+0000 7fb1422a6200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T18:59:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:24 smithi156 ceph-mon[128316]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:24.681 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:24.359+0000 7f97a9477200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:59:25.308 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:24.958+0000 7fb1422a6200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T18:59:25.308 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:25.163+0000 7fb1422a6200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T18:59:25.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:25.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:25.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:25.582 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:25.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:25.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:25.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:25.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:25.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:25.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:25.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.680 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:25.302+0000 7f97a9477200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:59:25.681 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:25.473+0000 7f97a9477200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:59:25.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:25.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:25.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:25.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-07-21T18:59:25.684 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-07-21T18:59:25.685 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:26.145 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:25.715+0000 7f97a9477200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:59:26.146 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:25.862+0000 7f97a9477200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:59:26.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:26.134+0000 7f97a9477200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:59:26.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:26 smithi156 ceph-mon[128316]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:59:26.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:26 smithi156 ceph-mon[128316]: Reconfiguring daemon mgr.x on smithi005 2023-07-21T18:59:26.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:26 smithi156 ceph-mon[128316]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:26.443 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-mon[128136]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:59:26.444 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-mon[128136]: Reconfiguring daemon mgr.x on smithi005 2023-07-21T18:59:26.444 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-mon[128136]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:26.444 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:26.097+0000 7fb1422a6200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:59:26.444 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:26.215+0000 7fb1422a6200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T18:59:26.444 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:26.432+0000 7fb1422a6200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T18:59:26.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:26 smithi005 ceph-mon[126460]: Reconfiguring mgr.x (unknown last config time)... 2023-07-21T18:59:26.566 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:26 smithi005 ceph-mon[126460]: Reconfiguring daemon mgr.x on smithi005 2023-07-21T18:59:26.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:26 smithi005 ceph-mon[126460]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:27.078 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:26.791+0000 7fb1422a6200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T18:59:27.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:27.067+0000 7fb1422a6200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T18:59:27.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:27.259+0000 7fb1422a6200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T18:59:27.680 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T18:59:27.201+0000 7f97a9477200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:59:28.091 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:27.599+0000 7fb1422a6200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T18:59:28.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: Standby manager daemon z started 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:59:28.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:28.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:59:28.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:28.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:27 smithi101 ceph-mon[128136]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:28.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:28.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: Standby manager daemon z started 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:28.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:27 smithi005 ceph-mon[126460]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:28.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:28.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: Standby manager daemon z started 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T18:59:28.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/1616934815' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:28.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:27 smithi156 ceph-mon[128316]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:28.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:28.178+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 14) v1 ==== 90598+0+0 (secure 0 0 0) 0x7f5ef000f460 con 0x7f5f00074540 2023-07-21T18:59:28.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:28.513+0000 7fb1422a6200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T18:59:28.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:28.682+0000 7fb1422a6200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T18:59:28.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]: dispatch 2023-07-21T18:59:28.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]': finished 2023-07-21T18:59:28.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:59:28.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:28.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: mgrmap e14: x(active, since 80s), standbys: z 2023-07-21T18:59:28.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:59:29.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]: dispatch 2023-07-21T18:59:29.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]': finished 2023-07-21T18:59:29.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:59:29.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:29.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: mgrmap e14: x(active, since 80s), standbys: z 2023-07-21T18:59:29.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:59:29.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]: dispatch 2023-07-21T18:59:29.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3386170656' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6825a11-5019-461a-a2af-a0337a93556d"}]': finished 2023-07-21T18:59:29.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: osdmap e5: 1 total, 0 up, 1 in 2023-07-21T18:59:29.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:29.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: mgrmap e14: x(active, since 80s), standbys: z 2023-07-21T18:59:29.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T18:59:29.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:28.924+0000 7fb1422a6200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T18:59:29.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:29.070+0000 7fb1422a6200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T18:59:29.739 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:29.341+0000 7fb1422a6200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T18:59:30.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:29 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1748935996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:30.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:29 smithi101 ceph-mon[128136]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:30.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:29 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1748935996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:30.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:29 smithi005 ceph-mon[126460]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:30.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:29 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1748935996' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:30.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:29 smithi156 ceph-mon[128316]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:30.741 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T18:59:30.397+0000 7fb1422a6200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T18:59:30.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:30.740+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f5ef001f2d0 con 0x7f5f00074540 2023-07-21T18:59:31.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-mon[128136]: Standby manager daemon y started 2023-07-21T18:59:31.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:59:31.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:31.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:59:31.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:30 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:31.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:30 smithi005 ceph-mon[126460]: Standby manager daemon y started 2023-07-21T18:59:31.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:30 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:59:31.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:30 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:31.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:30 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:59:31.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:30 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:31.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:30 smithi156 ceph-mon[128316]: Standby manager daemon y started 2023-07-21T18:59:31.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:30 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T18:59:31.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:30 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T18:59:31.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:30 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T18:59:31.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:30 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/2559292986' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T18:59:32.073 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:31 smithi005 ceph-mon[126460]: mgrmap e15: x(active, since 83s), standbys: z, y 2023-07-21T18:59:32.073 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:59:32.074 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:31 smithi005 ceph-mon[126460]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:32.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:31 smithi101 ceph-mon[128136]: mgrmap e15: x(active, since 83s), standbys: z, y 2023-07-21T18:59:32.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:59:32.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:31 smithi101 ceph-mon[128136]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:32.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:31 smithi156 ceph-mon[128316]: mgrmap e15: x(active, since 83s), standbys: z, y 2023-07-21T18:59:32.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T18:59:32.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:31 smithi156 ceph-mon[128316]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:34.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:34 smithi005 ceph-mon[126460]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:34 smithi101 ceph-mon[128136]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:34 smithi156 ceph-mon[128316]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:36.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:36 smithi101 ceph-mon[128136]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:36.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:36 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:59:36.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:36 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:36.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:36 smithi101 ceph-mon[128136]: Deploying daemon osd.0 on smithi005 2023-07-21T18:59:36.634 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:36 smithi005 ceph-mon[126460]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:36.635 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:36 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:59:36.635 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:36 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:36.635 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:36 smithi005 ceph-mon[126460]: Deploying daemon osd.0 on smithi005 2023-07-21T18:59:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:36 smithi156 ceph-mon[128316]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:36 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-07-21T18:59:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:36 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:36.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:36 smithi156 ceph-mon[128316]: Deploying daemon osd.0 on smithi005 2023-07-21T18:59:38.580 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:38 smithi005 ceph-mon[126460]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:38 smithi101 ceph-mon[128136]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:38.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:38 smithi156 ceph-mon[128316]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:40.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:40 smithi005 ceph-mon[126460]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:40 smithi101 ceph-mon[128136]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:40 smithi156 ceph-mon[128316]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:42 smithi101 ceph-mon[128136]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:42.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:42 smithi005 ceph-mon[126460]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:42 smithi156 ceph-mon[128316]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:44.594 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:44 smithi101 ceph-mon[128136]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:44 smithi005 ceph-mon[126460]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:44 smithi156 ceph-mon[128316]: pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:45.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.186+0000 7f5ef6ffd700 1 -- 172.21.15.5:0/300958612 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5f00060950 con 0x7f5eec0389e0 2023-07-21T18:59:45.198 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 0 on host 'smithi005' 2023-07-21T18:59:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 msgr2=0x7f5eec03ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 0x7f5eec03ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5efc009be0 tx=0x7f5efc009260 comp rx=0 tx=0).stop 2023-07-21T18:59:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 msgr2=0x7f5f00197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0000c00 tx=0x7f5ef0005740 comp rx=0 tx=0).stop 2023-07-21T18:59:45.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 shutdown_connections 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5eec0389e0 0x7f5eec03ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5f00107a60 0x7f5f0019efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5f00072f70 0x7f5f001979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 --2- 172.21.15.5:0/300958612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5f00074540 0x7f5f00197f20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 >> 172.21.15.5:0/300958612 conn(0x7f5f000fb1a0 msgr2=0x7f5f000fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 shutdown_connections 2023-07-21T18:59:45.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T18:59:45.189+0000 7f5f07c72700 1 -- 172.21.15.5:0/300958612 wait complete. 2023-07-21T18:59:45.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:45.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:45.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.420 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:45.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:45.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:45.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:45.621 DEBUG:teuthology.orchestra.run.smithi005:osd.0> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.0.service 2023-07-21T18:59:45.625 INFO:tasks.cephadm:Deploying osd.1 on smithi101 with /dev/vg_nvme/lv_4... 2023-07-21T18:59:45.626 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T18:59:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:45.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T18:59:45.697 INFO:journalctl@ceph.osd.0.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T18:59:45.918 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 18:59:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T18:59:45.771+0000 7f8125bac640 -1 osd.0 0 log_to_monitors true 2023-07-21T18:59:46.355 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:59:46.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:46 smithi005 ceph-mon[126460]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:46.600 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:46 smithi005 ceph-mon[126460]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:59:46.663 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:46 smithi101 ceph-mon[128136]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:46.663 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:46 smithi101 ceph-mon[128136]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:59:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:46 smithi156 ceph-mon[128316]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:46.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:46 smithi156 ceph-mon[128316]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-07-21T18:59:47.669 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T18:59:47.340+0000 7f811b224700 -1 osd.0 0 waiting for initial osdmap 2023-07-21T18:59:47.669 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T18:59:47.345+0000 7f8115015700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T18:59:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-mon[126460]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:59:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-mon[126460]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:59:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:47 smithi005 ceph-mon[126460]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T18:59:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:47 smithi156 ceph-mon[128316]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:59:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:47 smithi156 ceph-mon[128316]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:59:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:47 smithi156 ceph-mon[128316]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T18:59:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:47 smithi101 ceph-mon[128136]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-07-21T18:59:47.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:47 smithi101 ceph-mon[128136]: osdmap e6: 1 total, 0 up, 1 in 2023-07-21T18:59:47.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:47.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:47 smithi101 ceph-mon[128136]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T18:59:48.620 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T18:59:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T18:59:48.621 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-21T18:59:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-21T18:59:48.621 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0404267 s, 259 MB/s 2023-07-21T18:59:48.621 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-21T18:59:48.660 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:48 smithi101 ceph-mon[128136]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:48.660 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:48 smithi101 ceph-mon[128136]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T18:59:48.660 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:48 smithi101 ceph-mon[128136]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:59:48.661 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:48.661 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:48 smithi005 ceph-mon[126460]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:48 smithi005 ceph-mon[126460]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T18:59:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:48 smithi005 ceph-mon[126460]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:59:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:48 smithi156 ceph-mon[128316]: pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:48.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:48 smithi156 ceph-mon[128316]: from='osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T18:59:48.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:48 smithi156 ceph-mon[128316]: osdmap e7: 1 total, 0 up, 1 in 2023-07-21T18:59:48.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:48.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:49.126 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_4 2023-07-21T18:59:49.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T18:59:49.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T18:59:49.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] boot 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi005 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:59:49.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 17272M 2023-07-21T18:59:49.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:49.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:49.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:49 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] boot 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:49.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi005 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 17272M 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:49.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:49.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T18:59:49.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T18:59:49.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: osd.0 [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] boot 2023-07-21T18:59:49.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: osdmap e8: 1 total, 1 up, 1 in 2023-07-21T18:59:49.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T18:59:49.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi005 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 17272M 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:49.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T18:59:49.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T18:59:49.839 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T18:59:50.685 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:50 smithi101 ceph-mon[128136]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:50.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:50 smithi101 ceph-mon[128136]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:59:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:50 smithi005 ceph-mon[126460]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:50 smithi005 ceph-mon[126460]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:59:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:50 smithi156 ceph-mon[128316]: pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-07-21T18:59:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:50 smithi156 ceph-mon[128316]: osdmap e9: 1 total, 1 up, 1 in 2023-07-21T18:59:51.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 -- 172.21.15.101:0/69918170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc101720 msgr2=0x7f9fbc107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:51.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 --2- 172.21.15.101:0/69918170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc101720 0x7f9fbc107fe0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9fb400b0a0 tx=0x7f9fb400b3b0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 -- 172.21.15.101:0/69918170 shutdown_connections 2023-07-21T18:59:51.293 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 --2- 172.21.15.101:0/69918170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc101720 0x7f9fbc107fe0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 --2- 172.21.15.101:0/69918170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 --2- 172.21.15.101:0/69918170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc100d90 0x7f9fbc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 -- 172.21.15.101:0/69918170 >> 172.21.15.101:0/69918170 conn(0x7f9fbc0fb1a0 msgr2=0x7f9fbc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T18:59:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 -- 172.21.15.101:0/69918170 shutdown_connections 2023-07-21T18:59:51.294 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.282+0000 7f9fc471e700 1 -- 172.21.15.101:0/69918170 wait complete. 2023-07-21T18:59:51.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.283+0000 7f9fc471e700 1 Processor -- start 2023-07-21T18:59:51.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.283+0000 7f9fc471e700 1 -- start start 2023-07-21T18:59:51.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:51.296 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc101720 0x7f9fbc19f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc19f690 con 0x7f9fbc101720 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc19f7d0 con 0x7f9fbc100d90 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc471e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fbc19f960 con 0x7f9fbc0ffb90 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:51.297 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:53072/0 (socket says 172.21.15.101:53072) 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 -- 172.21.15.101:0/1314213224 learned_addr learned my addr 172.21.15.101:0/1314213224 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc24ba700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc2cbb700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc101720 0x7f9fbc19f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 -- 172.21.15.101:0/1314213224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 msgr2=0x7f9fbc197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 -- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc101720 msgr2=0x7f9fbc19f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T18:59:51.298 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc101720 0x7f9fbc19f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc1cb9700 1 -- 172.21.15.101:0/1314213224 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fb4009d00 con 0x7f9fbc100d90 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.284+0000 7f9fc24ba700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fc1cb9700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d950 tx=0x7f9fac00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac009940 con 0x7f9fbc100d90 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9fac010460 con 0x7f9fbc100d90 2023-07-21T18:59:51.299 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fac00f620 con 0x7f9fbc100d90 2023-07-21T18:59:51.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fbc19fc40 con 0x7f9fbc100d90 2023-07-21T18:59:51.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.285+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fbc1a00f0 con 0x7f9fbc100d90 2023-07-21T18:59:51.300 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.286+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fbc192120 con 0x7f9fbc100d90 2023-07-21T18:59:51.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.288+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f9fac0105d0 con 0x7f9fbc100d90 2023-07-21T18:59:51.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.289+0000 7f9fb37fe700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 0x7f9fa80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T18:59:51.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.289+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7f9fac0cad00 con 0x7f9fbc100d90 2023-07-21T18:59:51.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.290+0000 7f9fc24ba700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 0x7f9fa80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T18:59:51.301 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.290+0000 7f9fc24ba700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 0x7f9fa80a3570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005950 tx=0x7f9fb80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T18:59:51.304 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.294+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9fac095e20 con 0x7f9fbc100d90 2023-07-21T18:59:51.508 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T18:59:51.497+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f9fbc060950 con 0x7f9fa80a10c0 2023-07-21T18:59:52.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:52 smithi101 ceph-mon[128136]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:52.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:52 smithi101 ceph-mon[128136]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:52.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:52 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:52.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:52 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:52.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:52 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:52 smithi005 ceph-mon[126460]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:52 smithi005 ceph-mon[126460]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:52 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:52 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:52 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:52.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:52 smithi156 ceph-mon[128316]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:52.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:52 smithi156 ceph-mon[128316]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T18:59:52.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:52 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T18:59:52.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:52 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T18:59:52.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:52 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T18:59:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:54 smithi101 ceph-mon[128136]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:54 smithi005 ceph-mon[126460]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:54 smithi156 ceph-mon[128316]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/441225181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]': finished 2023-07-21T18:59:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:59:55.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:59:55.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:55 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/1037846751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/441225181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]': finished 2023-07-21T18:59:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:59:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:59:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:55 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/1037846751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/441225181' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]: dispatch 2023-07-21T18:59:55.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "22a957d4-0669-466d-88ee-0f05ba0b0909"}]': finished 2023-07-21T18:59:55.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: osdmap e10: 2 total, 1 up, 2 in 2023-07-21T18:59:55.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T18:59:55.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:55 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/1037846751' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T18:59:56.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:56 smithi101 ceph-mon[128136]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:56.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:56 smithi005 ceph-mon[126460]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:56.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:56 smithi156 ceph-mon[128316]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:58.548 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 18:59:58 smithi101 ceph-mon[128136]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:58.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 18:59:58 smithi005 ceph-mon[126460]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T18:59:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 18:59:58 smithi156 ceph-mon[128316]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:00.526 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:00 smithi101 ceph-mon[128136]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:00.526 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:00 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T19:00:00.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:00 smithi005 ceph-mon[126460]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:00.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:00 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T19:00:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:00 smithi156 ceph-mon[128316]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:00 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T19:00:02.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:02 smithi101 ceph-mon[128136]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:02.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:02 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T19:00:02.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:02 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:02.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:02 smithi005 ceph-mon[126460]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:02.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:02 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T19:00:02.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:02 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:02 smithi156 ceph-mon[128316]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:02.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:02 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-07-21T19:00:02.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:02 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:03.477 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:03 smithi101 ceph-mon[128136]: Deploying daemon osd.1 on smithi101 2023-07-21T19:00:03.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:03 smithi005 ceph-mon[126460]: Deploying daemon osd.1 on smithi101 2023-07-21T19:00:03.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:03 smithi156 ceph-mon[128316]: Deploying daemon osd.1 on smithi101 2023-07-21T19:00:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:04 smithi101 ceph-mon[128136]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:04.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:04 smithi005 ceph-mon[126460]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:04 smithi156 ceph-mon[128316]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:06.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:06 smithi101 ceph-mon[128136]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:06.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:06 smithi005 ceph-mon[126460]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:06.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:06 smithi156 ceph-mon[128316]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:08.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:08 smithi101 ceph-mon[128136]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:08.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:08 smithi005 ceph-mon[126460]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:08 smithi156 ceph-mon[128316]: pgmap v60: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:09.570 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.570 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.570 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:09.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:10 smithi101 ceph-mon[128136]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:10 smithi005 ceph-mon[126460]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:10 smithi156 ceph-mon[128316]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:11 smithi101 ceph-mon[128136]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:11.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:11.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:11.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:11 smithi005 ceph-mon[126460]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:11.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:11.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:11.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:11 smithi156 ceph-mon[128316]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:11.933 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.923+0000 7f9fb37fe700 1 -- 172.21.15.101:0/1314213224 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f9fbc060950 con 0x7f9fa80a10c0 2023-07-21T19:00:11.934 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 1 on host 'smithi101' 2023-07-21T19:00:11.936 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 msgr2=0x7f9fa80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 0x7f9fa80a3570 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9fb8005950 tx=0x7f9fb80058e0 comp rx=0 tx=0).stop 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 msgr2=0x7f9fbc198040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9fac00d950 tx=0x7f9fac00dc60 comp rx=0 tx=0).stop 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 shutdown_connections 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9fa80a10c0 0x7f9fa80a3570 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:11.937 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9fbc101720 0x7f9fbc19f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:11.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9fbc0ffb90 0x7f9fbc197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:11.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.925+0000 7f9fc471e700 1 --2- 172.21.15.101:0/1314213224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9fbc100d90 0x7f9fbc198040 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:11.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.926+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 >> 172.21.15.101:0/1314213224 conn(0x7f9fbc0fb1a0 msgr2=0x7f9fbc106710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:11.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.926+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 shutdown_connections 2023-07-21T19:00:11.938 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:00:11.926+0000 7f9fc471e700 1 -- 172.21.15.101:0/1314213224 wait complete. 2023-07-21T19:00:12.365 DEBUG:teuthology.orchestra.run.smithi101:osd.1> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.1.service 2023-07-21T19:00:12.370 INFO:tasks.cephadm:Deploying osd.2 on smithi156 with /dev/vg_nvme/lv_4... 2023-07-21T19:00:12.371 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-07-21T19:00:12.442 INFO:journalctl@ceph.osd.1.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:00:13.109 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:00:13.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:13.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:12 smithi156 ceph-mon[128316]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:13.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:12 smithi156 ceph-mon[128316]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:13.219 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.220 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.220 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:13.220 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:12 smithi101 ceph-mon[128136]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:13.220 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:12 smithi101 ceph-mon[128136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:13.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:13.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:13.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:12 smithi005 ceph-mon[126460]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:13.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:12 smithi005 ceph-mon[126460]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-07-21T19:00:14.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T19:00:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T19:00:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-mon[128136]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:14.593 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1[133257]: 2023-07-21T19:00:14.198+0000 7f74b48f7700 -1 osd.1 0 waiting for initial osdmap 2023-07-21T19:00:14.593 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1[133257]: 2023-07-21T19:00:14.202+0000 7f74ac6e4700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:00:14.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T19:00:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T19:00:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:14 smithi005 ceph-mon[126460]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-07-21T19:00:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: osdmap e11: 2 total, 1 up, 2 in 2023-07-21T19:00:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: from='osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:00:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:14 smithi156 ceph-mon[128316]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:14.970 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-07-21T19:00:14.970 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-07-21T19:00:14.971 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-21T19:00:14.971 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-21T19:00:14.971 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0390748 s, 268 MB/s 2023-07-21T19:00:14.971 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-21T19:00:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:15 smithi156 ceph-mon[128316]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:00:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:15 smithi156 ceph-mon[128316]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T19:00:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.356 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:15 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:15.389 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_4 2023-07-21T19:00:15.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:15 smithi101 ceph-mon[128136]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:00:15.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:15 smithi101 ceph-mon[128136]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T19:00:15.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:15 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:15.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:15 smithi005 ceph-mon[126460]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:00:15.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:15 smithi005 ceph-mon[126460]: osdmap e12: 2 total, 1 up, 2 in 2023-07-21T19:00:15.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:15.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:15 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.115 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:00:16.497 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:00:16.498 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:00:16.498 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi101 2023-07-21T19:00:16.498 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:16.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-21T19:00:16.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] boot 2023-07-21T19:00:16.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T19:00:16.500 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:16.500 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.500 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:16.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:16.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.501 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:16 smithi156 ceph-mon[128316]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:16.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:00:16.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:00:16.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi101 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] boot 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T19:00:16.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:16.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:16.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:16.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:16 smithi101 ceph-mon[128136]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:16.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:00:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:00:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi101 2023-07-21T19:00:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 17267M 2023-07-21T19:00:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: osd.1 [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] boot 2023-07-21T19:00:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: osdmap e13: 2 total, 2 up, 2 in 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:16.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:16 smithi005 ceph-mon[126460]: pgmap v67: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 -- 172.21.15.156:0/945777860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc100d90 msgr2=0x7fa6bc1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 --2- 172.21.15.156:0/945777860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc100d90 0x7fa6bc1011e0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa6b0009990 tx=0x7fa6b0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 -- 172.21.15.156:0/945777860 shutdown_connections 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 --2- 172.21.15.156:0/945777860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc101720 0x7fa6bc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 --2- 172.21.15.156:0/945777860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 0x7fa6bc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.478 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 --2- 172.21.15.156:0/945777860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc100d90 0x7fa6bc1011e0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 -- 172.21.15.156:0/945777860 >> 172.21.15.156:0/945777860 conn(0x7fa6bc0fb1a0 msgr2=0x7fa6bc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:17.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 -- 172.21.15.156:0/945777860 shutdown_connections 2023-07-21T19:00:17.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.466+0000 7fa6c162b700 1 -- 172.21.15.156:0/945777860 wait complete. 2023-07-21T19:00:17.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 Processor -- start 2023-07-21T19:00:17.479 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 -- start start 2023-07-21T19:00:17.480 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 0x7fa6bc0742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 0x7fa6bc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc199670 con 0x7fa6bc0ffb90 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc073380 con 0x7fa6bc100d90 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.467+0000 7fa6c162b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6bc073540 con 0x7fa6bc101720 2023-07-21T19:00:17.481 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6baffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 0x7fa6bc0742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6ba7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 0x7fa6bc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:58870/0 (socket says 172.21.15.156:58870) 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 -- 172.21.15.156:0/2306774992 learned_addr learned my addr 172.21.15.156:0/2306774992 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 -- 172.21.15.156:0/2306774992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 msgr2=0x7fa6bc072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 0x7fa6bc072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.482 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 -- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 msgr2=0x7fa6bc0742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 0x7fa6bc0742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 -- 172.21.15.156:0/2306774992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b00095f0 con 0x7fa6bc101720 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6ba7fc700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 0x7fa6bc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.468+0000 7fa6bb7fe700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4007ee0 tx=0x7fa6b400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.469+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4017070 con 0x7fa6bc101720 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.469+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6b400f040 con 0x7fa6bc101720 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.469+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6b4005570 con 0x7fa6bc101720 2023-07-21T19:00:17.483 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.469+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6bc1a4040 con 0x7fa6bc101720 2023-07-21T19:00:17.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.469+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6bc1a4400 con 0x7fa6bc101720 2023-07-21T19:00:17.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.470+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6bc106b20 con 0x7fa6bc101720 2023-07-21T19:00:17.484 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.472+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7fa6b400db20 con 0x7fa6bc101720 2023-07-21T19:00:17.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.473+0000 7fa6a3fff700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 0x7fa6a40a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:17.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.473+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2309+0+0 (secure 0 0 0) 0x7fa6b40ca870 con 0x7fa6bc101720 2023-07-21T19:00:17.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.474+0000 7fa6baffd700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 0x7fa6a40a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:17.485 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.474+0000 7fa6baffd700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 0x7fa6a40a3570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc100bf0 tx=0x7fa6ac00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:17.488 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.478+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa6b4095770 con 0x7fa6bc101720 2023-07-21T19:00:17.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:17 smithi101 ceph-mon[128136]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T19:00:17.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:17 smithi005 ceph-mon[126460]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T19:00:17.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:17 smithi156 ceph-mon[128316]: osdmap e14: 2 total, 2 up, 2 in 2023-07-21T19:00:17.690 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:17.679+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6bc060950 con 0x7fa6a40a10c0 2023-07-21T19:00:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:18 smithi101 ceph-mon[128136]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:18 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:18.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:18 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:18.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:18 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:18 smithi005 ceph-mon[126460]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:18.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:18.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:18.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:18 smithi156 ceph-mon[128316]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:18.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:18 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:18.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:18 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:18.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:18 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:19.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:19 smithi156 ceph-mon[128316]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:19.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:19 smithi101 ceph-mon[128136]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:19.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:19 smithi005 ceph-mon[126460]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:20 smithi101 ceph-mon[128136]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:20.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:20 smithi005 ceph-mon[126460]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:20 smithi156 ceph-mon[128316]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:21 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/1117514028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:21 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:21 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]': finished 2023-07-21T19:00:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:21 smithi101 ceph-mon[128136]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T19:00:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:21.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:21 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/1117514028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:21 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:21 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]': finished 2023-07-21T19:00:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:21 smithi005 ceph-mon[126460]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T19:00:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:21 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/1117514028' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:21 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]: dispatch 2023-07-21T19:00:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:21 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6ba0786-1e76-44da-b171-81015c102bc4"}]': finished 2023-07-21T19:00:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:21 smithi156 ceph-mon[128316]: osdmap e15: 3 total, 2 up, 3 in 2023-07-21T19:00:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:22 smithi101 ceph-mon[128136]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:22 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/2239813922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:22 smithi005 ceph-mon[126460]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:22 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/2239813922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:22 smithi156 ceph-mon[128316]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:22 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/2239813922' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:24 smithi101 ceph-mon[128136]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:24.631 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:24 smithi156 ceph-mon[128316]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:24 smithi005 ceph-mon[126460]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:26.413 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:26 smithi156 ceph-mon[128316]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:26 smithi101 ceph-mon[128136]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:26.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:26 smithi005 ceph-mon[126460]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:28 smithi101 ceph-mon[128136]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T19:00:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:28 smithi005 ceph-mon[126460]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T19:00:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:28 smithi156 ceph-mon[128316]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-07-21T19:00:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:29.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:29 smithi156 ceph-mon[128316]: Deploying daemon osd.2 on smithi156 2023-07-21T19:00:29.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:29 smithi101 ceph-mon[128136]: Deploying daemon osd.2 on smithi156 2023-07-21T19:00:29.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:29 smithi005 ceph-mon[126460]: Deploying daemon osd.2 on smithi156 2023-07-21T19:00:30.504 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:30 smithi156 ceph-mon[128316]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:30 smithi101 ceph-mon[128136]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:30.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:30 smithi005 ceph-mon[126460]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:32.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:32 smithi156 ceph-mon[128316]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:32.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:32 smithi101 ceph-mon[128136]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:32.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:32 smithi005 ceph-mon[126460]: pgmap v77: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:34.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:34 smithi156 ceph-mon[128316]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:34.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:34 smithi101 ceph-mon[128136]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:34 smithi005 ceph-mon[126460]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:36.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:36 smithi156 ceph-mon[128316]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:36.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:36 smithi101 ceph-mon[128136]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:36.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:36 smithi005 ceph-mon[126460]: pgmap v79: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:37.764 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.753+0000 7fa6a3fff700 1 -- 172.21.15.156:0/2306774992 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa6bc060950 con 0x7fa6a40a10c0 2023-07-21T19:00:37.765 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 2 on host 'smithi156' 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.756+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 msgr2=0x7fa6a40a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.756+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 0x7fa6a40a3570 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fa6bc100bf0 tx=0x7fa6ac00b410 comp rx=0 tx=0).stop 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 msgr2=0x7fa6bc072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6b4007ee0 tx=0x7fa6b400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 shutdown_connections 2023-07-21T19:00:37.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa6a40a10c0 0x7fa6a40a3570 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6bc101720 0x7fa6bc072e40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6bc0ffb90 0x7fa6bc0742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 --2- 172.21.15.156:0/2306774992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6bc100d90 0x7fa6bc072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 >> 172.21.15.156:0/2306774992 conn(0x7fa6bc0fb1a0 msgr2=0x7fa6bc0fd4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 shutdown_connections 2023-07-21T19:00:37.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:00:37.757+0000 7fa6c162b700 1 -- 172.21.15.156:0/2306774992 wait complete. 2023-07-21T19:00:37.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:37.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:37.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:37.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:37.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:37.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:37.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:37.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:37.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:37 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:38.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:38.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:38.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:38.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:37 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:38.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:38.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:38.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:38.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:38.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:37 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:00:38.241 DEBUG:teuthology.orchestra.run.smithi156:osd.2> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.2.service 2023-07-21T19:00:38.246 INFO:tasks.cephadm:Deploying osd.3 on smithi005 with /dev/vg_nvme/lv_3... 2023-07-21T19:00:38.247 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T19:00:38.322 INFO:journalctl@ceph.osd.2.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:00:38.985 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:00:39.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:39 smithi156 ceph-mon[128316]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T19:00:39.534 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:39 smithi005 ceph-mon[126460]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T19:00:39.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:39 smithi101 ceph-mon[128136]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-07-21T19:00:40.562 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-mon[128316]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T19:00:40.562 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-mon[128316]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T19:00:40.562 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:40.562 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-mon[128316]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:00:40.562 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-mon[128316]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:40.563 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2[134150]: 2023-07-21T19:00:40.209+0000 7f5cf0b2c700 -1 osd.2 0 waiting for initial osdmap 2023-07-21T19:00:40.563 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2[134150]: 2023-07-21T19:00:40.214+0000 7f5cea91d700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:00:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:40 smithi101 ceph-mon[128136]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T19:00:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:40 smithi101 ceph-mon[128136]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T19:00:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:40.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:40 smithi101 ceph-mon[128136]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:00:40.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:40 smithi101 ceph-mon[128136]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:40 smithi005 ceph-mon[126460]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-07-21T19:00:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:40 smithi005 ceph-mon[126460]: osdmap e16: 3 total, 2 up, 3 in 2023-07-21T19:00:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:40 smithi005 ceph-mon[126460]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:00:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:40 smithi005 ceph-mon[126460]: pgmap v82: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2023-07-21T19:00:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T19:00:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T19:00:40.886 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-07-21T19:00:40.886 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-07-21T19:00:40.886 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0303036 s, 346 MB/s 2023-07-21T19:00:40.887 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-07-21T19:00:41.360 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi005:vg_nvme/lv_3 2023-07-21T19:00:41.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:00:41.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T19:00:41.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:41.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:41.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:41.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:00:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T19:00:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:41.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:41.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: osdmap e17: 3 total, 2 up, 3 in 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:00:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:41.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:41.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:00:41.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:42.095 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:00:42.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:00:42.500 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:00:42.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi156 2023-07-21T19:00:42.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 17267M 2023-07-21T19:00:42.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] boot 2023-07-21T19:00:42.501 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T19:00:42.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:42.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: pgmap v85: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:42.502 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:00:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:00:42.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi156 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 17267M 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] boot 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: pgmap v85: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:42.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:00:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi156 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 17267M 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: osd.2 [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] boot 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: osdmap e18: 3 total, 3 up, 3 in 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: pgmap v85: 0 pgs: ; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:42.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:43.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 -- 172.21.15.5:0/2805459415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 msgr2=0x7f59ac1011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:43.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 --2- 172.21.15.5:0/2805459415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac1011e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5994009990 tx=0x7f5994009ca0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 -- 172.21.15.5:0/2805459415 shutdown_connections 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 --2- 172.21.15.5:0/2805459415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 0x7f59ac107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 --2- 172.21.15.5:0/2805459415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 0x7f59ac0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 --2- 172.21.15.5:0/2805459415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac1011e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 -- 172.21.15.5:0/2805459415 >> 172.21.15.5:0/2805459415 conn(0x7f59ac0fb1a0 msgr2=0x7f59ac0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 -- 172.21.15.5:0/2805459415 shutdown_connections 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.538+0000 7f59b22d4700 1 -- 172.21.15.5:0/2805459415 wait complete. 2023-07-21T19:00:43.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.539+0000 7f59b22d4700 1 Processor -- start 2023-07-21T19:00:43.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 -- start start 2023-07-21T19:00:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 0x7f59ac197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:43.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 0x7f59ac19efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac19f5b0 con 0x7f59ac100d90 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac19f6f0 con 0x7f59ac101720 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b22d4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59ac19f880 con 0x7f59ac0ffb90 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59a3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44918/0 (socket says 172.21.15.5:44918) 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59a3fff700 1 -- 172.21.15.5:0/4003995914 learned_addr learned my addr 172.21.15.5:0/4003995914 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:00:43.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59b0871700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 0x7f59ac19efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.540+0000 7f59abfff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 0x7f59ac197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 -- 172.21.15.5:0/4003995914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 msgr2=0x7f59ac197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 0x7f59ac197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 -- 172.21.15.5:0/4003995914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 msgr2=0x7f59ac19efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 0x7f59ac19efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 -- 172.21.15.5:0/4003995914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59940095f0 con 0x7f59ac100d90 2023-07-21T19:00:43.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a3fff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5994000c00 tx=0x7f5994005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599401d070 con 0x7f59ac100d90 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59ac19fb00 con 0x7f59ac100d90 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f599400be70 con 0x7f59ac100d90 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994022e30 con 0x7f59ac100d90 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.541+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59ac19ff70 con 0x7f59ac100d90 2023-07-21T19:00:43.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.542+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59ac10b250 con 0x7f59ac100d90 2023-07-21T19:00:43.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.544+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 15) v1 ==== 135658+0+0 (secure 0 0 0) 0x7f599400b9e0 con 0x7f59ac100d90 2023-07-21T19:00:43.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.547+0000 7f59a9ffb700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 0x7f59980a35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:00:43.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.547+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3237+0+0 (secure 0 0 0) 0x7f59940cc930 con 0x7f59ac100d90 2023-07-21T19:00:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.547+0000 7f59abfff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 0x7f59980a35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:00:43.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.547+0000 7f59abfff700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 0x7f59980a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59ac100bf0 tx=0x7f599c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:00:43.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.551+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5994097520 con 0x7f59ac100d90 2023-07-21T19:00:43.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:43 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:43.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:43 smithi101 ceph-mon[128136]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T19:00:43.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:43 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:43.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:43 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:43.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:43 smithi005 ceph-mon[126460]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T19:00:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:43 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:43.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:43 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:43.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:43 smithi156 ceph-mon[128316]: osdmap e19: 3 total, 3 up, 3 in 2023-07-21T19:00:43.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:43 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-07-21T19:00:43.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:43.767+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f59ac107790 con 0x7f59980a1140 2023-07-21T19:00:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T19:00:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:44.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T19:00:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:44.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-07-21T19:00:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: osdmap e20: 3 total, 3 up, 3 in 2023-07-21T19:00:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:00:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:00:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:45.167 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:44 smithi005 sudo[141928]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:45.167 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:44 smithi005 sudo[141928]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:45.167 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:44 smithi005 sudo[141928]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:45.167 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:44 smithi005 sudo[141928]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:45.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:45 smithi005 ceph-mon[126460]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:45.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:45 smithi005 ceph-mon[126460]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T19:00:45.419 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:45 smithi005 sudo[141979]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:45.419 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:45 smithi005 sudo[141979]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:45.419 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:45 smithi005 sudo[141979]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:45.419 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:00:45 smithi005 sudo[141979]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:45.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:45 smithi101 ceph-mon[128136]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:45.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:45 smithi101 ceph-mon[128136]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T19:00:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:45 smithi156 ceph-mon[128316]: from='client.14304 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:00:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:45 smithi156 ceph-mon[128316]: osdmap e21: 3 total, 3 up, 3 in 2023-07-21T19:00:46.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:00:46.245+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f59940978f0 con 0x7f59ac100d90 2023-07-21T19:00:46.341 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:45 smithi101 sudo[135573]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:46.341 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:45 smithi101 sudo[135573]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:46.341 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:45 smithi101 sudo[135573]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:46.341 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:45 smithi101 sudo[135573]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:46 smithi005 ceph-mon[126460]: pgmap v90: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:46 smithi005 ceph-mon[126460]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T19:00:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:46 smithi156 ceph-mon[128316]: pgmap v90: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:46 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:46 smithi156 ceph-mon[128316]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T19:00:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:46 smithi101 ceph-mon[128136]: pgmap v90: 1 pgs: 1 unknown; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:46 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:46 smithi101 ceph-mon[128136]: mgrmap e16: x(active, since 2m), standbys: z, y 2023-07-21T19:00:46.841 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:46 smithi101 sudo[135576]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:46.842 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:46 smithi101 sudo[135576]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:46.842 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:46 smithi101 sudo[135576]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:46.842 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:00:46 smithi101 sudo[135576]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:47.180 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:46 smithi156 sudo[136410]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:47.180 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:46 smithi156 sudo[136410]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:47.180 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:46 smithi156 sudo[136410]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:47.180 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:46 smithi156 sudo[136410]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:47.680 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:47 smithi156 sudo[136413]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:47.680 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:47 smithi156 sudo[136413]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:47.680 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:47 smithi156 sudo[136413]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:47.681 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:00:47 smithi156 sudo[136413]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1911204873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]': finished 2023-07-21T19:00:47.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T19:00:47.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:00:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1911204873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]': finished 2023-07-21T19:00:47.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T19:00:47.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:00:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1911204873' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]: dispatch 2023-07-21T19:00:47.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "01f22961-e7d2-423a-8511-6b3ed03de822"}]': finished 2023-07-21T19:00:47.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: osdmap e22: 4 total, 3 up, 4 in 2023-07-21T19:00:47.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:00:48.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 sudo[142105]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:48.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 sudo[142105]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:48.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:47 smithi005 sudo[142105]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:48.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 sudo[142105]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:48.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:48.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:48 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:48.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 sudo[142108]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 sudo[142108]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 sudo[142108]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 sudo[142108]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:48 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:00:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:48 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:49.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136416]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:49.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136416]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:49.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136416]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:49.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136416]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:49.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:49.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:49.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:49.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1688150936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:49.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:49.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:49.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:49.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:49.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:49 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136419]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136419]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136419]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 sudo[136419]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1688150936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:49.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:49.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:49.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:49.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:49.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:49 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:50.340 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135579]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-07-21T19:00:50.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135579]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:50.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135579]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:50.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135579]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135582]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-07-21T19:00:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135582]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-07-21T19:00:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135582]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-07-21T19:00:50.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 sudo[135582]: pam_unix(sudo:session): session closed for user root 2023-07-21T19:00:50.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:50.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:50.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:50.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1688150936' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:50.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:50.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:50.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:50.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:50.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:50.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:00:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:50.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:00:50.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:00:50.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:00:51.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:51 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:51.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:51 smithi101 ceph-mon[128136]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:51.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:51 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:51.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:51 smithi005 ceph-mon[126460]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:51 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-07-21T19:00:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:51 smithi156 ceph-mon[128316]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-07-21T19:00:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:52 smithi101 ceph-mon[128136]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 297 KiB/s wr, 20 op/s 2023-07-21T19:00:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:52 smithi005 ceph-mon[126460]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 297 KiB/s wr, 20 op/s 2023-07-21T19:00:52.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:52 smithi156 ceph-mon[128316]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 6.8 KiB/s rd, 297 KiB/s wr, 20 op/s 2023-07-21T19:00:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:54 smithi101 ceph-mon[128136]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.7 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-07-21T19:00:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:54 smithi005 ceph-mon[126460]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.7 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-07-21T19:00:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:54 smithi156 ceph-mon[128316]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 4.7 KiB/s rd, 204 KiB/s wr, 14 op/s 2023-07-21T19:00:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:55 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T19:00:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:55 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:55 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T19:00:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:55 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:55 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-07-21T19:00:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:55 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:00:56.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:56 smithi101 ceph-mon[128136]: Deploying daemon osd.3 on smithi005 2023-07-21T19:00:56.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:56 smithi101 ceph-mon[128136]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 155 KiB/s wr, 10 op/s 2023-07-21T19:00:56.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:56 smithi005 ceph-mon[126460]: Deploying daemon osd.3 on smithi005 2023-07-21T19:00:56.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:56 smithi005 ceph-mon[126460]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 155 KiB/s wr, 10 op/s 2023-07-21T19:00:56.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:56 smithi156 ceph-mon[128316]: Deploying daemon osd.3 on smithi005 2023-07-21T19:00:56.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:56 smithi156 ceph-mon[128316]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 3.6 KiB/s rd, 155 KiB/s wr, 10 op/s 2023-07-21T19:00:58.358 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:00:58 smithi005 ceph-mon[126460]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2023-07-21T19:00:58.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:00:58 smithi101 ceph-mon[128136]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2023-07-21T19:00:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:00:58 smithi156 ceph-mon[128316]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.9 KiB/s rd, 126 KiB/s wr, 8 op/s 2023-07-21T19:01:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:00 smithi101 ceph-mon[128136]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:00.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:00 smithi005 ceph-mon[126460]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:00 smithi156 ceph-mon[128316]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:02 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:02 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:02 smithi101 ceph-mon[128136]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:02 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:02.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:02 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:02 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:02 smithi005 ceph-mon[126460]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:02.507 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:02 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:02 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:02 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:02.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:02 smithi156 ceph-mon[128316]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail; 2.5 KiB/s rd, 109 KiB/s wr, 7 op/s 2023-07-21T19:01:02.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:02 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:04.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:04.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:04 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:04.622 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:04 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:04.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:04.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:04 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:04.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.930+0000 7f59a9ffb700 1 -- 172.21.15.5:0/4003995914 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f59ac107790 con 0x7f59980a1140 2023-07-21T19:01:04.941 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 3 on host 'smithi005' 2023-07-21T19:01:04.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.932+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 msgr2=0x7f59980a35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.932+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 0x7f59980a35f0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f59ac100bf0 tx=0x7f599c009260 comp rx=0 tx=0).stop 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 msgr2=0x7f59ac197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f5994000c00 tx=0x7f5994005720 comp rx=0 tx=0).stop 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 shutdown_connections 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f59980a1140 0x7f59980a35f0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:04.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59ac101720 0x7f59ac19efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59ac0ffb90 0x7f59ac197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 --2- 172.21.15.5:0/4003995914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59ac100d90 0x7f59ac197f60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 >> 172.21.15.5:0/4003995914 conn(0x7f59ac0fb1a0 msgr2=0x7f59ac0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:04.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 shutdown_connections 2023-07-21T19:01:04.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:01:04.933+0000 7f59b22d4700 1 -- 172.21.15.5:0/4003995914 wait complete. 2023-07-21T19:01:05.321 DEBUG:teuthology.orchestra.run.smithi005:osd.3> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.3.service 2023-07-21T19:01:05.326 INFO:tasks.cephadm:Deploying osd.4 on smithi101 with /dev/vg_nvme/lv_3... 2023-07-21T19:01:05.326 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T19:01:05.417 INFO:journalctl@ceph.osd.3.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:01:06.071 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:01:06.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:05 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.205 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:05 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.206 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:05 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:06.206 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:05 smithi005 ceph-mon[126460]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T19:01:06.206 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:05 smithi005 ceph-mon[126460]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:06.329 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:05 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.329 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:05 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.329 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:05 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:06.329 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:05 smithi101 ceph-mon[128136]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T19:01:06.330 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:05 smithi101 ceph-mon[128136]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:06.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:05 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:05 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:06.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:05 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:06.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:05 smithi156 ceph-mon[128316]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-07-21T19:01:06.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:05 smithi156 ceph-mon[128316]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:07.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:06 smithi101 ceph-mon[128136]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T19:01:07.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:06 smithi101 ceph-mon[128136]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T19:01:07.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:07.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:06 smithi101 ceph-mon[128136]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:01:07.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-mon[126460]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T19:01:07.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-mon[126460]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T19:01:07.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:07.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-mon[126460]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:01:07.420 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:01:06.935+0000 7fa05d088700 -1 osd.3 0 waiting for initial osdmap 2023-07-21T19:01:07.420 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:01:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:01:06.941+0000 7fa05867c700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:01:07.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:06 smithi156 ceph-mon[128316]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-07-21T19:01:07.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:06 smithi156 ceph-mon[128316]: osdmap e23: 4 total, 3 up, 4 in 2023-07-21T19:01:07.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:07.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:06 smithi156 ceph-mon[128316]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:01:08.033 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T19:01:08.033 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T19:01:08.033 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-21T19:01:08.033 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-21T19:01:08.034 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0313801 s, 334 MB/s 2023-07-21T19:01:08.034 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-21T19:01:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:01:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T19:01:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] boot 2023-07-21T19:01:08.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T19:01:08.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:07 smithi005 ceph-mon[126460]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:08.202 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:01:08.202 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T19:01:08.202 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.203 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.203 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] boot 2023-07-21T19:01:08.203 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T19:01:08.203 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.203 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:07 smithi101 ceph-mon[128136]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:08.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: from='osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:01:08.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: osdmap e24: 4 total, 3 up, 4 in 2023-07-21T19:01:08.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: osd.3 [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] boot 2023-07-21T19:01:08.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: osdmap e25: 4 total, 4 up, 4 in 2023-07-21T19:01:08.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:01:08.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:07 smithi156 ceph-mon[128316]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 80 MiB used, 268 GiB / 268 GiB avail 2023-07-21T19:01:08.546 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_3 2023-07-21T19:01:09.262 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:01:09.492 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi005 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.493 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 8636M 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:09.494 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi005 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 8636M 2023-07-21T19:01:09.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:09.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:09.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:01:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:01:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: osdmap e26: 4 total, 4 up, 4 in 2023-07-21T19:01:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi005 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 8636M 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:09.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:09.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:10.483 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:10 smithi101 ceph-mon[128136]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T19:01:10.484 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:10 smithi101 ceph-mon[128136]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:10 smithi005 ceph-mon[126460]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T19:01:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:10 smithi005 ceph-mon[126460]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:10.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 -- 172.21.15.101:0/3145463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18107a60 msgr2=0x7faf18109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:10.675 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 --2- 172.21.15.101:0/3145463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18107a60 0x7faf18109ea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7faf1000b0a0 tx=0x7faf1000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 -- 172.21.15.101:0/3145463423 shutdown_connections 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 --2- 172.21.15.101:0/3145463423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18107a60 0x7faf18109ea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 --2- 172.21.15.101:0/3145463423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18074540 0x7faf180729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 --2- 172.21.15.101:0/3145463423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18072f70 0x7faf180733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 -- 172.21.15.101:0/3145463423 >> 172.21.15.101:0/3145463423 conn(0x7faf180fb1a0 msgr2=0x7faf180fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 -- 172.21.15.101:0/3145463423 shutdown_connections 2023-07-21T19:01:10.676 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.666+0000 7faf1cb38700 1 -- 172.21.15.101:0/3145463423 wait complete. 2023-07-21T19:01:10.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 Processor -- start 2023-07-21T19:01:10.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 -- start start 2023-07-21T19:01:10.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 0x7faf18197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:10.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:10.678 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18107a60 0x7faf1819f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819f5d0 con 0x7faf18072f70 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819f710 con 0x7faf18074540 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.667+0000 7faf1cb38700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf1819f8a0 con 0x7faf18107a60 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:43200/0 (socket says 172.21.15.101:43200) 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 -- 172.21.15.101:0/1883232664 learned_addr learned my addr 172.21.15.101:0/1883232664 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-21T19:01:10.679 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf16d9d700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18107a60 0x7faf1819f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf1659c700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 0x7faf18197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 -- 172.21.15.101:0/1883232664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18107a60 msgr2=0x7faf1819f000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18107a60 0x7faf1819f000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 -- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 msgr2=0x7faf18197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 0x7faf18197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 -- 172.21.15.101:0/1883232664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf10009d00 con 0x7faf18074540 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf1659c700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 0x7faf18197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:01:10.680 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.668+0000 7faf15d9b700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c9b0 tx=0x7faf0c00cd70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:10.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.669+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c007b00 con 0x7faf18074540 2023-07-21T19:01:10.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.669+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf0c007c60 con 0x7faf18074540 2023-07-21T19:01:10.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.669+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf1819fb80 con 0x7faf18074540 2023-07-21T19:01:10.681 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.669+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0c018660 con 0x7faf18074540 2023-07-21T19:01:10.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:10 smithi156 ceph-mon[128316]: osdmap e27: 4 total, 4 up, 4 in 2023-07-21T19:01:10.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:10 smithi156 ceph-mon[128316]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:10.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.669+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf181a00d0 con 0x7faf18074540 2023-07-21T19:01:10.682 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.670+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf18106820 con 0x7faf18074540 2023-07-21T19:01:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.672+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7faf0c00f450 con 0x7faf18074540 2023-07-21T19:01:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.673+0000 7faf077fe700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 0x7faf000a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.673+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 3669+0+0 (secure 0 0 0) 0x7faf0c0cba40 con 0x7faf18074540 2023-07-21T19:01:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.673+0000 7faf1659c700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 0x7faf000a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:10.683 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.674+0000 7faf1659c700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 0x7faf000a3680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf08009be0 tx=0x7faf08009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:10.685 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.676+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf0c096390 con 0x7faf18074540 2023-07-21T19:01:10.886 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:10.877+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7faf18060950 con 0x7faf000a11d0 2023-07-21T19:01:11.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:11 smithi101 ceph-mon[128136]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T19:01:11.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:11.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:11.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:11 smithi005 ceph-mon[126460]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T19:01:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:11.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:11.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:11 smithi156 ceph-mon[128316]: osdmap e28: 4 total, 4 up, 4 in 2023-07-21T19:01:11.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:11.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:11.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:12.548 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:12 smithi101 ceph-mon[128136]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:12.549 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:12 smithi101 ceph-mon[128136]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T19:01:12.549 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:12 smithi101 ceph-mon[128136]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T19:01:12.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:12 smithi005 ceph-mon[126460]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:12 smithi005 ceph-mon[126460]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T19:01:12.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:12 smithi005 ceph-mon[126460]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T19:01:12.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:12 smithi156 ceph-mon[128316]: from='client.24239 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:12.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:12 smithi156 ceph-mon[128316]: osdmap e29: 4 total, 4 up, 4 in 2023-07-21T19:01:12.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:12 smithi156 ceph-mon[128316]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-07-21T19:01:14.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T19:01:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/3723654541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]': finished 2023-07-21T19:01:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T19:01:14.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:14.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T19:01:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/3723654541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]': finished 2023-07-21T19:01:14.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T19:01:14.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 114 KiB/s, 0 objects/s recovering 2023-07-21T19:01:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/3723654541' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]: dispatch 2023-07-21T19:01:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "157f6702-def5-47b3-a2e1-497954863a97"}]': finished 2023-07-21T19:01:14.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: osdmap e30: 5 total, 4 up, 5 in 2023-07-21T19:01:14.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:15.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:15 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/3320845793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:15.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:15 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/3320845793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:15.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:15 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/3320845793' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:16.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:16 smithi101 ceph-mon[128136]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:01:16.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:16 smithi005 ceph-mon[126460]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:01:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:16 smithi156 ceph-mon[128316]: pgmap v114: 1 pgs: 1 peering; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:01:18.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:18 smithi101 ceph-mon[128136]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-21T19:01:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:18 smithi005 ceph-mon[126460]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-21T19:01:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:18 smithi156 ceph-mon[128316]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-07-21T19:01:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:20 smithi101 ceph-mon[128136]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:20.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:20 smithi005 ceph-mon[126460]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:20 smithi156 ceph-mon[128316]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:22 smithi101 ceph-mon[128136]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T19:01:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:22 smithi101 ceph-mon[128136]: Deploying daemon osd.4 on smithi101 2023-07-21T19:01:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:22 smithi005 ceph-mon[126460]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T19:01:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:22 smithi005 ceph-mon[126460]: Deploying daemon osd.4 on smithi101 2023-07-21T19:01:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:22 smithi156 ceph-mon[128316]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-07-21T19:01:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:22 smithi156 ceph-mon[128316]: Deploying daemon osd.4 on smithi101 2023-07-21T19:01:24.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:24 smithi101 ceph-mon[128136]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:24 smithi005 ceph-mon[126460]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:24 smithi156 ceph-mon[128316]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:26 smithi101 ceph-mon[128136]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:26.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:26 smithi005 ceph-mon[126460]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:26 smithi156 ceph-mon[128316]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:28 smithi101 ceph-mon[128136]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:28.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:28 smithi005 ceph-mon[126460]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:28 smithi156 ceph-mon[128316]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:30.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:30 smithi101 ceph-mon[128136]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:30.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:30 smithi005 ceph-mon[126460]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:30.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:30 smithi156 ceph-mon[128316]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:31.312 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.301+0000 7faf077fe700 1 -- 172.21.15.101:0/1883232664 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7faf18060950 con 0x7faf000a11d0 2023-07-21T19:01:31.312 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 4 on host 'smithi101' 2023-07-21T19:01:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 msgr2=0x7faf000a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:31.314 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 0x7faf000a3680 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7faf08009be0 tx=0x7faf08009260 comp rx=0 tx=0).stop 2023-07-21T19:01:31.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 msgr2=0x7faf18197f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:31.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7faf0c00c9b0 tx=0x7faf0c00cd70 comp rx=0 tx=0).stop 2023-07-21T19:01:31.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 shutdown_connections 2023-07-21T19:01:31.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7faf000a11d0 0x7faf000a3680 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:31.315 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf18107a60 0x7faf1819f000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:31.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf18072f70 0x7faf18197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:31.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 --2- 172.21.15.101:0/1883232664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf18074540 0x7faf18197f80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:31.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 >> 172.21.15.101:0/1883232664 conn(0x7faf180fb1a0 msgr2=0x7faf18103960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:31.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 shutdown_connections 2023-07-21T19:01:31.316 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:01:31.304+0000 7faf1cb38700 1 -- 172.21.15.101:0/1883232664 wait complete. 2023-07-21T19:01:31.879 DEBUG:teuthology.orchestra.run.smithi101:osd.4> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.4.service 2023-07-21T19:01:31.884 INFO:tasks.cephadm:Deploying osd.5 on smithi156 with /dev/vg_nvme/lv_3... 2023-07-21T19:01:31.884 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-07-21T19:01:31.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:31.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:31.944 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.944 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:31.945 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:31.957 INFO:journalctl@ceph.osd.4.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:01:32.635 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:01:32.744 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:32 smithi101 ceph-mon[128136]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:32.744 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:32 smithi101 ceph-mon[128136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:32.899 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:32 smithi156 ceph-mon[128316]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:32.899 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:32 smithi156 ceph-mon[128316]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:32.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:32 smithi005 ceph-mon[126460]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:32.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:32 smithi005 ceph-mon[126460]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-07-21T19:01:33.873 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T19:01:33.873 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T19:01:33.873 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:33.873 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:33.874 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:33.874 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:33 smithi156 ceph-mon[128316]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:33.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T19:01:33.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T19:01:33.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:33.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:33.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:33.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:33 smithi005 ceph-mon[126460]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:34.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-07-21T19:01:34.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: osdmap e31: 5 total, 4 up, 5 in 2023-07-21T19:01:34.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: from='osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:34.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:01:34.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-mon[128136]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 106 MiB used, 358 GiB / 358 GiB avail 2023-07-21T19:01:34.092 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:01:33.648+0000 7fa68cabc700 -1 osd.4 0 waiting for initial osdmap 2023-07-21T19:01:34.092 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:01:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:01:33.658+0000 7fa6880b0700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:01:34.566 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-07-21T19:01:34.567 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-07-21T19:01:34.567 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-21T19:01:34.567 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-21T19:01:34.567 INFO:teuthology.orchestra.run.smithi156.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0302891 s, 346 MB/s 2023-07-21T19:01:34.568 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 2023-07-21T19:01:34.568 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:34.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:01:34.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:34.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:34.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:01:34.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: osdmap e32: 5 total, 4 up, 5 in 2023-07-21T19:01:34.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:34.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:35.014 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_3 2023-07-21T19:01:35.772 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:01:35.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:01:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:01:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi101 2023-07-21T19:01:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-21T19:01:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] boot 2023-07-21T19:01:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T19:01:35.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:35.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:35 smithi005 ceph-mon[126460]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:35.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:01:35.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:01:35.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi101 2023-07-21T19:01:35.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-21T19:01:35.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] boot 2023-07-21T19:01:35.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T19:01:35.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:35.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:35 smithi156 ceph-mon[128316]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:36.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:01:36.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:01:36.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi101 2023-07-21T19:01:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 8633M 2023-07-21T19:01:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: osd.4 [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] boot 2023-07-21T19:01:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: osdmap e33: 5 total, 5 up, 5 in 2023-07-21T19:01:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:01:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:35 smithi101 ceph-mon[128136]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:36.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:36 smithi005 ceph-mon[126460]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T19:01:36.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:36 smithi156 ceph-mon[128316]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T19:01:37.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:36 smithi101 ceph-mon[128136]: osdmap e34: 5 total, 5 up, 5 in 2023-07-21T19:01:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 -- 172.21.15.156:0/2631820321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 msgr2=0x7f14741011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 --2- 172.21.15.156:0/2631820321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f14741011e0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f1464009990 tx=0x7f1464009ca0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 -- 172.21.15.156:0/2631820321 shutdown_connections 2023-07-21T19:01:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 --2- 172.21.15.156:0/2631820321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f1474107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.214 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 --2- 172.21.15.156:0/2631820321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 0x7f14740fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 --2- 172.21.15.156:0/2631820321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f14741011e0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 -- 172.21.15.156:0/2631820321 >> 172.21.15.156:0/2631820321 conn(0x7f14740fb1a0 msgr2=0x7f14740fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:37.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 -- 172.21.15.156:0/2631820321 shutdown_connections 2023-07-21T19:01:37.215 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.203+0000 7f147ba83700 1 -- 172.21.15.156:0/2631820321 wait complete. 2023-07-21T19:01:37.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.204+0000 7f147ba83700 1 Processor -- start 2023-07-21T19:01:37.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.204+0000 7f147ba83700 1 -- start start 2023-07-21T19:01:37.216 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 0x7f14741979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f1474197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419f540 con 0x7f1474101720 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419f680 con 0x7f14740ffb90 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147ba83700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f147419f810 con 0x7f1474100d90 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147a020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:37.217 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147a020700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.156:40588/0 (socket says 172.21.15.156:40588) 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147a020700 1 -- 172.21.15.156:0/209229612 learned_addr learned my addr 172.21.15.156:0/209229612 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147981f700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 0x7f14741979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.205+0000 7f147901e700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f1474197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 -- 172.21.15.156:0/209229612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 msgr2=0x7f1474197ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f1474197ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 -- 172.21.15.156:0/209229612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 msgr2=0x7f14741979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 0x7f14741979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:37.218 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 -- 172.21.15.156:0/209229612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14640095f0 con 0x7f1474101720 2023-07-21T19:01:37.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147901e700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f1474197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:01:37.219 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.206+0000 7f147a020700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f146c007f10 tx=0x7f146c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.207+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c00ddd0 con 0x7f1474101720 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.207+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f146c00f040 con 0x7f1474101720 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.207+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f146c004870 con 0x7f1474101720 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.208+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147419faf0 con 0x7f1474101720 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.208+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f147419ffa0 con 0x7f1474101720 2023-07-21T19:01:37.220 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.209+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1474192110 con 0x7f1474101720 2023-07-21T19:01:37.221 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.211+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f146c003a00 con 0x7f1474101720 2023-07-21T19:01:37.222 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.212+0000 7f146affd700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:01:37.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.212+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 4101+0+0 (secure 0 0 0) 0x7f146c0cb990 con 0x7f1474101720 2023-07-21T19:01:37.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.213+0000 7f147981f700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:01:37.223 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.213+0000 7f147981f700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 0x7f14600a3680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1474100bf0 tx=0x7f147000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:01:37.226 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.215+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f146c0960b0 con 0x7f1474101720 2023-07-21T19:01:37.430 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:37.419+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f1474060950 con 0x7f14600a11d0 2023-07-21T19:01:38.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:38 smithi156 ceph-mon[128316]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:38.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:38 smithi156 ceph-mon[128316]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:38.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:38.499 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:38.500 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:38 smithi101 ceph-mon[128136]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:38 smithi101 ceph-mon[128136]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:38 smithi005 ceph-mon[126460]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:38 smithi005 ceph-mon[126460]: from='client.14361 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:01:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:01:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:01:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:40 smithi101 ceph-mon[128136]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/1135368535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:40.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:40 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:40 smithi005 ceph-mon[126460]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/1135368535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:40 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:40 smithi156 ceph-mon[128316]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/1135368535' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:40 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]: dispatch 2023-07-21T19:01:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:41 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]': finished 2023-07-21T19:01:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:41 smithi101 ceph-mon[128136]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T19:01:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:01:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:41 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/2901403225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:41 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]': finished 2023-07-21T19:01:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:41 smithi005 ceph-mon[126460]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T19:01:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:01:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:41 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/2901403225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:41 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "04a4eedc-bd98-4441-8bdb-443010dfc21e"}]': finished 2023-07-21T19:01:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:41 smithi156 ceph-mon[128316]: osdmap e35: 6 total, 5 up, 6 in 2023-07-21T19:01:41.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:01:41.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:41 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/2901403225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:01:42.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:42 smithi101 ceph-mon[128136]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:42.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:42 smithi005 ceph-mon[126460]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:42 smithi156 ceph-mon[128316]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:44 smithi101 ceph-mon[128136]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:44 smithi005 ceph-mon[126460]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:44.676 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:44 smithi156 ceph-mon[128316]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:46 smithi005 ceph-mon[126460]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:46 smithi156 ceph-mon[128316]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:46.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:46 smithi101 ceph-mon[128136]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:48 smithi101 ceph-mon[128136]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T19:01:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:48.596 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:48 smithi156 ceph-mon[128316]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:48.596 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T19:01:48.596 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:48 smithi005 ceph-mon[126460]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-07-21T19:01:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:49.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:49 smithi101 ceph-mon[128136]: Deploying daemon osd.5 on smithi156 2023-07-21T19:01:49.638 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:49 smithi156 ceph-mon[128316]: Deploying daemon osd.5 on smithi156 2023-07-21T19:01:49.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:49 smithi005 ceph-mon[126460]: Deploying daemon osd.5 on smithi156 2023-07-21T19:01:50.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:50 smithi156 ceph-mon[128316]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:50.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:50 smithi101 ceph-mon[128136]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:50.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:50 smithi005 ceph-mon[126460]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:52.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:52 smithi101 ceph-mon[128136]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:52.665 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:52 smithi156 ceph-mon[128316]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:52.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:52 smithi005 ceph-mon[126460]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:54.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:54 smithi101 ceph-mon[128136]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:54.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:54.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:54 smithi005 ceph-mon[126460]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:54.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:54 smithi156 ceph-mon[128316]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:56.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:56 smithi101 ceph-mon[128136]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:56.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:56 smithi005 ceph-mon[126460]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:56 smithi156 ceph-mon[128316]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:57.561 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.550+0000 7f146affd700 1 -- 172.21.15.156:0/209229612 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1474060950 con 0x7f14600a11d0 2023-07-21T19:01:57.561 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 5 on host 'smithi156' 2023-07-21T19:01:57.563 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 msgr2=0x7f14600a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 0x7f14600a3680 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1474100bf0 tx=0x7f147000a3b0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 msgr2=0x7f147419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f146c007f10 tx=0x7f146c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 shutdown_connections 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f14600a11d0 0x7f14600a3680 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1474101720 0x7f147419ef70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.564 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14740ffb90 0x7f14741979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 --2- 172.21.15.156:0/209229612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1474100d90 0x7f1474197ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:01:57.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 >> 172.21.15.156:0/209229612 conn(0x7f14740fb1a0 msgr2=0x7f14741066b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:01:57.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 shutdown_connections 2023-07-21T19:01:57.565 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:01:57.553+0000 7f147ba83700 1 -- 172.21.15.156:0/209229612 wait complete. 2023-07-21T19:01:58.091 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.091 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.091 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:58.091 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:58.092 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.092 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:58.092 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.092 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.092 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:58.094 DEBUG:teuthology.orchestra.run.smithi156:osd.5> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.5.service 2023-07-21T19:01:58.098 INFO:tasks.cephadm:Deploying osd.6 on smithi005 with /dev/vg_nvme/lv_2... 2023-07-21T19:01:58.099 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T19:01:58.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:58.172 INFO:journalctl@ceph.osd.5.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:01:58.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:01:58.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:01:58.839 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:01:59.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:58 smithi156 ceph-mon[128316]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:01:59.099 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:58 smithi156 ceph-mon[128316]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:01:59.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:58 smithi005 ceph-mon[126460]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:01:59.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:58 smithi005 ceph-mon[126460]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:01:59.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:58 smithi101 ceph-mon[128136]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:01:59.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:58 smithi101 ceph-mon[128136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-07-21T19:02:00.080 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T19:02:00.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T19:02:00.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:00.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.081 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:01:59 smithi005 ceph-mon[126460]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:02:00.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T19:02:00.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T19:02:00.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:00.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-mon[128316]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:02:00.182 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:01:59.849+0000 7f089d5b7700 -1 osd.5 0 waiting for initial osdmap 2023-07-21T19:02:00.182 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:01:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:01:59.858+0000 7f0895ba5700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:02:00.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-07-21T19:02:00.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: osdmap e36: 6 total, 5 up, 6 in 2023-07-21T19:02:00.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:00.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: from='osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:02:00.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:01:59 smithi101 ceph-mon[128136]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 133 MiB used, 447 GiB / 447 GiB avail 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0435107 s, 241 MB/s 2023-07-21T19:02:00.806 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-07-21T19:02:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:00 smithi156 ceph-mon[128316]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:02:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:00 smithi156 ceph-mon[128316]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T19:02:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:01.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:00 smithi005 ceph-mon[126460]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:02:01.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:00 smithi005 ceph-mon[126460]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T19:02:01.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:01.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:01.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi005:vg_nvme/lv_2 2023-07-21T19:02:01.340 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:00 smithi101 ceph-mon[128136]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:02:01.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:00 smithi101 ceph-mon[128136]: osdmap e37: 6 total, 5 up, 6 in 2023-07-21T19:02:01.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:01.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:01.968 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:02:02.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] boot 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi156 2023-07-21T19:02:02.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 8633M 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:02.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:01 smithi005 ceph-mon[126460]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:02.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:02:02.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:02:02.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] boot 2023-07-21T19:02:02.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi156 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 8633M 2023-07-21T19:02:02.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:02.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:02.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:01 smithi156 ceph-mon[128316]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:02:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:02:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: osd.5 [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] boot 2023-07-21T19:02:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: osdmap e38: 6 total, 6 up, 6 in 2023-07-21T19:02:02.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi156 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 8633M 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:02.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:02.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:02.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:01 smithi101 ceph-mon[128136]: pgmap v145: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:03.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:02 smithi005 ceph-mon[126460]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T19:02:03.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:02 smithi005 ceph-mon[126460]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T19:02:03.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:02 smithi156 ceph-mon[128316]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T19:02:03.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:02 smithi156 ceph-mon[128316]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T19:02:03.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:02 smithi101 ceph-mon[128136]: osdmap e39: 6 total, 6 up, 6 in 2023-07-21T19:02:03.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:02 smithi101 ceph-mon[128136]: osdmap e40: 6 total, 6 up, 6 in 2023-07-21T19:02:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.434+0000 7f6c01e04700 1 -- 172.21.15.5:0/3478199417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc0ffb90 msgr2=0x7f6bfc0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.434+0000 7f6c01e04700 1 --2- 172.21.15.5:0/3478199417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc0fffa0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009990 tx=0x7f6bec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 -- 172.21.15.5:0/3478199417 shutdown_connections 2023-07-21T19:02:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 --2- 172.21.15.5:0/3478199417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 0x7f6bfc107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 --2- 172.21.15.5:0/3478199417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc0fffa0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 --2- 172.21.15.5:0/3478199417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc100d90 0x7f6bfc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 -- 172.21.15.5:0/3478199417 >> 172.21.15.5:0/3478199417 conn(0x7f6bfc0fb1a0 msgr2=0x7f6bfc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 -- 172.21.15.5:0/3478199417 shutdown_connections 2023-07-21T19:02:03.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.435+0000 7f6c01e04700 1 -- 172.21.15.5:0/3478199417 wait complete. 2023-07-21T19:02:03.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 Processor -- start 2023-07-21T19:02:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 -- start start 2023-07-21T19:02:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:03.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 0x7f6bfc19cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19d3a0 con 0x7f6bfc100d90 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19d4e0 con 0x7f6bfc0ffb90 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.436+0000 7f6c01e04700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bfc19d670 con 0x7f6bfc101720 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:03.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60542/0 (socket says 172.21.15.5:60542) 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 -- 172.21.15.5:0/2305533868 learned_addr learned my addr 172.21.15.5:0/2305533868 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfb7fe700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfbfff700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 0x7f6bfc19cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 -- 172.21.15.5:0/2305533868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 msgr2=0x7f6bfc19cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 0x7f6bfc19cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 -- 172.21.15.5:0/2305533868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc0ffb90 msgr2=0x7f6bfc195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 -- 172.21.15.5:0/2305533868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0095f0 con 0x7f6bfc100d90 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.437+0000 7f6bfaffd700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000d9b0 tx=0x7f6bf000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.438+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0004220 con 0x7f6bfc100d90 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.438+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bfc19d950 con 0x7f6bfc100d90 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.438+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6bf0007640 con 0x7f6bfc100d90 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.438+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bf0005480 con 0x7f6bfc100d90 2023-07-21T19:02:03.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.438+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bfc19dea0 con 0x7f6bfc100d90 2023-07-21T19:02:03.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.439+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bfc10b250 con 0x7f6bfc100d90 2023-07-21T19:02:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.441+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6bf00055e0 con 0x7f6bfc100d90 2023-07-21T19:02:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.442+0000 7f6bf8ff9700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 0x7f6be40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.442+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4562+0+0 (secure 0 0 0) 0x7f6bf00cb910 con 0x7f6bfc100d90 2023-07-21T19:02:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.442+0000 7f6bfb7fe700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 0x7f6be40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:03.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.443+0000 7f6bfb7fe700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 0x7f6be40a3600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0098e0 tx=0x7f6bec00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:03.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.447+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6bf0095e60 con 0x7f6bfc100d90 2023-07-21T19:02:03.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:03.662+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bfc060950 con 0x7f6be40a1150 2023-07-21T19:02:04.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:03 smithi005 ceph-mon[126460]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:04.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:04.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:04.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:04.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:03 smithi156 ceph-mon[128316]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:04.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:04.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:04.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:04.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:03 smithi101 ceph-mon[128136]: pgmap v148: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:04.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:04.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:04.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:05.045 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:04 smithi005 ceph-mon[126460]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:05.046 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:04 smithi005 ceph-mon[126460]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T19:02:05.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:04 smithi156 ceph-mon[128316]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:05.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:04 smithi156 ceph-mon[128316]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T19:02:05.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:04 smithi101 ceph-mon[128136]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:05.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:04 smithi101 ceph-mon[128136]: osdmap e41: 6 total, 6 up, 6 in 2023-07-21T19:02:06.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:05 smithi005 ceph-mon[126460]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T19:02:06.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:05 smithi005 ceph-mon[126460]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:06.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:05 smithi156 ceph-mon[128316]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T19:02:06.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:05 smithi156 ceph-mon[128316]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:06.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:05 smithi101 ceph-mon[128136]: osdmap e42: 6 total, 6 up, 6 in 2023-07-21T19:02:06.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:05 smithi101 ceph-mon[128136]: pgmap v151: 1 pgs: 1 remapped+peering; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:07.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]: dispatch 2023-07-21T19:02:07.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]': finished 2023-07-21T19:02:07.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:06 smithi005 ceph-mon[126460]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T19:02:07.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:07.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]: dispatch 2023-07-21T19:02:07.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]': finished 2023-07-21T19:02:07.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:06 smithi156 ceph-mon[128316]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T19:02:07.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:07.344 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]: dispatch 2023-07-21T19:02:07.345 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2437281092' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bd7ecefb-ba21-4052-adc1-17da27ca0595"}]': finished 2023-07-21T19:02:07.345 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:06 smithi101 ceph-mon[128136]: osdmap e43: 7 total, 6 up, 7 in 2023-07-21T19:02:07.345 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:08.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:08 smithi101 ceph-mon[128136]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-21T19:02:08.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/389995142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:08 smithi005 ceph-mon[126460]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-21T19:02:08.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/389995142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:08 smithi156 ceph-mon[128316]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 131 KiB/s, 0 objects/s recovering 2023-07-21T19:02:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/389995142' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:10 smithi101 ceph-mon[128136]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:02:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:10 smithi005 ceph-mon[126460]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:02:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:10 smithi156 ceph-mon[128316]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:02:12.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:12 smithi005 ceph-mon[126460]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-21T19:02:12.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:12 smithi101 ceph-mon[128136]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-21T19:02:12.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:12 smithi156 ceph-mon[128316]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 78 KiB/s, 0 objects/s recovering 2023-07-21T19:02:14.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:14 smithi101 ceph-mon[128136]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-21T19:02:14.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T19:02:14.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:14.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:14 smithi005 ceph-mon[126460]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-21T19:02:14.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T19:02:14.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:14 smithi156 ceph-mon[128316]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 69 KiB/s, 0 objects/s recovering 2023-07-21T19:02:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-07-21T19:02:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:15.637 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:15 smithi005 ceph-mon[126460]: Deploying daemon osd.6 on smithi005 2023-07-21T19:02:15.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:15 smithi156 ceph-mon[128316]: Deploying daemon osd.6 on smithi005 2023-07-21T19:02:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:15 smithi101 ceph-mon[128136]: Deploying daemon osd.6 on smithi005 2023-07-21T19:02:16.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:16 smithi005 ceph-mon[126460]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T19:02:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:16 smithi156 ceph-mon[128316]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T19:02:16.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:16 smithi101 ceph-mon[128136]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-07-21T19:02:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:18 smithi101 ceph-mon[128136]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:18.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:18 smithi005 ceph-mon[126460]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:18 smithi156 ceph-mon[128316]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:20 smithi101 ceph-mon[128136]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:20.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:20 smithi005 ceph-mon[126460]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:20.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:20 smithi156 ceph-mon[128316]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:22.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:22 smithi005 ceph-mon[126460]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:22 smithi101 ceph-mon[128136]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:22 smithi156 ceph-mon[128316]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:23.745 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 6 on host 'smithi005' 2023-07-21T19:02:23.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.734+0000 7f6bf8ff9700 1 -- 172.21.15.5:0/2305533868 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6bfc060950 con 0x7f6be40a1150 2023-07-21T19:02:23.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 msgr2=0x7f6be40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 0x7f6be40a3600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6bec0098e0 tx=0x7f6bec00b260 comp rx=0 tx=0).stop 2023-07-21T19:02:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 msgr2=0x7f6bfc195d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:23.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bf000d9b0 tx=0x7f6bf000dcc0 comp rx=0 tx=0).stop 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 shutdown_connections 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6be40a1150 0x7f6be40a3600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6bfc101720 0x7f6bfc19cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6bfc0ffb90 0x7f6bfc195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 --2- 172.21.15.5:0/2305533868 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6bfc100d90 0x7f6bfc195d50 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:23.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.737+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 >> 172.21.15.5:0/2305533868 conn(0x7f6bfc0fb1a0 msgr2=0x7f6bfc0fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:23.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.738+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 shutdown_connections 2023-07-21T19:02:23.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:02:23.738+0000 7f6c01e04700 1 -- 172.21.15.5:0/2305533868 wait complete. 2023-07-21T19:02:24.204 DEBUG:teuthology.orchestra.run.smithi005:osd.6> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.6.service 2023-07-21T19:02:24.208 INFO:tasks.cephadm:Deploying osd.7 on smithi101 with /dev/vg_nvme/lv_2... 2023-07-21T19:02:24.209 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T19:02:24.304 INFO:journalctl@ceph.osd.6.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:02:24.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:24.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:24 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:24 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:24 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:24.937 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:02:25.495 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:25 smithi101 ceph-mon[128136]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T19:02:25.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:25 smithi005 ceph-mon[126460]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T19:02:25.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:25 smithi156 ceph-mon[128316]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-07-21T19:02:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:26 smithi101 ceph-mon[128136]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T19:02:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:26 smithi101 ceph-mon[128136]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T19:02:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:26 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:26 smithi101 ceph-mon[128136]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:02:26.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:26 smithi101 ceph-mon[128136]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:26.626 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:02:26.265+0000 7fa14ef54700 -1 osd.6 0 waiting for initial osdmap 2023-07-21T19:02:26.627 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:02:26.276+0000 7fa14a548700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:02:26.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-mon[126460]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T19:02:26.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-mon[126460]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T19:02:26.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:26.627 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-mon[126460]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:02:26.628 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:26 smithi005 ceph-mon[126460]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:26 smithi156 ceph-mon[128316]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-07-21T19:02:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:26 smithi156 ceph-mon[128316]: osdmap e44: 7 total, 6 up, 7 in 2023-07-21T19:02:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:26 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:26 smithi156 ceph-mon[128316]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:02:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:26 smithi156 ceph-mon[128316]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:26.904 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T19:02:26.904 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T19:02:26.905 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-21T19:02:26.905 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-21T19:02:26.905 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0368661 s, 284 MB/s 2023-07-21T19:02:26.905 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-21T19:02:27.510 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_2 2023-07-21T19:02:27.573 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.574 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:27.575 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:02:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T19:02:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:27.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:02:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: osdmap e45: 7 total, 6 up, 7 in 2023-07-21T19:02:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:27.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:27.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:27.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:28.227 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:02:28.562 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:02:28.562 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:02:28.562 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi005 2023-07-21T19:02:28.562 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 5757M 2023-07-21T19:02:28.562 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] boot 2023-07-21T19:02:28.563 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T19:02:28.563 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:28.563 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:28 smithi101 ceph-mon[128136]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:02:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi005 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 5757M 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] boot 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:28 smithi005 ceph-mon[126460]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi005 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 5757M 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: osd.6 [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] boot 2023-07-21T19:02:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: osdmap e46: 7 total, 7 up, 7 in 2023-07-21T19:02:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:02:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:28 smithi156 ceph-mon[128316]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 160 MiB used, 536 GiB / 536 GiB avail 2023-07-21T19:02:29.583 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:29 smithi101 ceph-mon[128136]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T19:02:29.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:29 smithi005 ceph-mon[126460]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T19:02:29.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:29 smithi156 ceph-mon[128316]: osdmap e47: 7 total, 7 up, 7 in 2023-07-21T19:02:29.829 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 -- 172.21.15.101:0/2395905788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 msgr2=0x7fcb7c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 --2- 172.21.15.101:0/2395905788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c0fffa0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fcb78009990 tx=0x7fcb78009ca0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 -- 172.21.15.101:0/2395905788 shutdown_connections 2023-07-21T19:02:29.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 --2- 172.21.15.101:0/2395905788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c101720 0x7fcb7c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 --2- 172.21.15.101:0/2395905788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c0fffa0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.830 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 --2- 172.21.15.101:0/2395905788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c100d90 0x7fcb7c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 -- 172.21.15.101:0/2395905788 >> 172.21.15.101:0/2395905788 conn(0x7fcb7c0fb1a0 msgr2=0x7fcb7c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:29.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 -- 172.21.15.101:0/2395905788 shutdown_connections 2023-07-21T19:02:29.831 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.819+0000 7fcb83985700 1 -- 172.21.15.101:0/2395905788 wait complete. 2023-07-21T19:02:29.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.820+0000 7fcb83985700 1 Processor -- start 2023-07-21T19:02:29.832 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.820+0000 7fcb83985700 1 -- start start 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c100d90 0x7fcb7c195d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c101720 0x7fcb7c19cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19d3c0 con 0x7fcb7c100d90 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19d500 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb83985700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7c19d690 con 0x7fcb7c101720 2023-07-21T19:02:29.833 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:48960/0 (socket says 172.21.15.101:48960) 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 -- 172.21.15.101:0/2914249223 learned_addr learned my addr 172.21.15.101:0/2914249223 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81f22700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c101720 0x7fcb7c19cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb80f20700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c100d90 0x7fcb7c195d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 -- 172.21.15.101:0/2914249223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c101720 msgr2=0x7fcb7c19cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c101720 0x7fcb7c19cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.834 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 -- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c100d90 msgr2=0x7fcb7c195d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c100d90 0x7fcb7c195d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.821+0000 7fcb81721700 1 -- 172.21.15.101:0/2914249223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb780095f0 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb81721700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005910 tx=0x7fcb78005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb7801d070 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb7800beb0 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7c19d910 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb78022e10 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.835 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.822+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7c19dd80 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.823+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb7c107260 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.836 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.825+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcb780225e0 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.826+0000 7fcb727fc700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 0x7fcb680a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:29.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.826+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 4965+0+0 (secure 0 0 0) 0x7fcb780cd7d0 con 0x7fcb7c0ffb90 2023-07-21T19:02:29.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.827+0000 7fcb80f20700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 0x7fcb680a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:29.837 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.827+0000 7fcb80f20700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 0x7fcb680a3680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005950 tx=0x7fcb6c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:29.838 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:29.828+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcb78097cb0 con 0x7fcb7c0ffb90 2023-07-21T19:02:30.041 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:30.031+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb7c060950 con 0x7fcb680a11d0 2023-07-21T19:02:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:30 smithi101 ceph-mon[128136]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:30 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:30 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:30 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:30.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:30 smithi005 ceph-mon[126460]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:30.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:30 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:30.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:30 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:30.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:30 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:30.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:30 smithi156 ceph-mon[128316]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:30.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:30 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:30.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:30 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:30.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:30 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:31.378 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:31 smithi101 ceph-mon[128136]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:31.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:31 smithi005 ceph-mon[126460]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:31.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:31 smithi156 ceph-mon[128316]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:32.340 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:32 smithi101 ceph-mon[128136]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:32.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:32 smithi005 ceph-mon[126460]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:32.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:32 smithi156 ceph-mon[128316]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:33.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:33 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/2966448388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:33 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:33 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]': finished 2023-07-21T19:02:33.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:33 smithi101 ceph-mon[128136]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T19:02:33.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:33 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:33.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:33 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/2966448388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:33 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:33 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]': finished 2023-07-21T19:02:33.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:33 smithi005 ceph-mon[126460]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T19:02:33.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:33 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:33 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/2966448388' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:33 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]: dispatch 2023-07-21T19:02:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:33 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a839fb0f-1bc6-4f9d-a381-079c5f730f63"}]': finished 2023-07-21T19:02:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:33 smithi156 ceph-mon[128316]: osdmap e48: 8 total, 7 up, 8 in 2023-07-21T19:02:33.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:33 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:34 smithi101 ceph-mon[128136]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:34 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/1081914891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:34 smithi005 ceph-mon[126460]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/1081914891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:34 smithi156 ceph-mon[128316]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:34 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/1081914891' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:02:36.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:36 smithi101 ceph-mon[128136]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:36.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:36 smithi005 ceph-mon[126460]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:36 smithi156 ceph-mon[128316]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:38.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:38 smithi101 ceph-mon[128136]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:38 smithi005 ceph-mon[126460]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:38.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:38 smithi156 ceph-mon[128316]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:40.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:40 smithi101 ceph-mon[128136]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:40 smithi005 ceph-mon[126460]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:40 smithi156 ceph-mon[128316]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T19:02:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:41.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:41 smithi005 ceph-mon[126460]: Deploying daemon osd.7 on smithi101 2023-07-21T19:02:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T19:02:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:41.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:41 smithi156 ceph-mon[128316]: Deploying daemon osd.7 on smithi101 2023-07-21T19:02:41.705 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-07-21T19:02:41.706 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:41.706 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:41 smithi101 ceph-mon[128136]: Deploying daemon osd.7 on smithi101 2023-07-21T19:02:42.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:42 smithi005 ceph-mon[126460]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:42 smithi156 ceph-mon[128316]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:42 smithi101 ceph-mon[128136]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:44 smithi005 ceph-mon[126460]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:44 smithi156 ceph-mon[128316]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:44.779 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:44 smithi101 ceph-mon[128136]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:46.652 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:46 smithi101 ceph-mon[128136]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:46 smithi005 ceph-mon[126460]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:46 smithi156 ceph-mon[128316]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:48.142 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.143 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.143 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:48.143 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:47 smithi101 ceph-mon[128136]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:48.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:48.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:47 smithi005 ceph-mon[126460]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:48.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:48.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:48.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:47 smithi156 ceph-mon[128316]: pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:50.517 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 7 on host 'smithi101' 2023-07-21T19:02:50.517 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.506+0000 7fcb727fc700 1 -- 172.21.15.101:0/2914249223 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb7c060950 con 0x7fcb680a11d0 2023-07-21T19:02:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.508+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 msgr2=0x7fcb680a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.508+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 0x7fcb680a3680 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c005950 tx=0x7fcb6c00b410 comp rx=0 tx=0).stop 2023-07-21T19:02:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.508+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 msgr2=0x7fcb7c195830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.508+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fcb78005910 tx=0x7fcb78005740 comp rx=0 tx=0).stop 2023-07-21T19:02:50.519 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 shutdown_connections 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb680a11d0 0x7fcb680a3680 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb7c101720 0x7fcb7c19cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb7c0ffb90 0x7fcb7c195830 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 --2- 172.21.15.101:0/2914249223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb7c100d90 0x7fcb7c195d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 >> 172.21.15.101:0/2914249223 conn(0x7fcb7c0fb1a0 msgr2=0x7fcb7c1043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:50.520 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 shutdown_connections 2023-07-21T19:02:50.521 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:02:50.509+0000 7fcb83985700 1 -- 172.21.15.101:0/2914249223 wait complete. 2023-07-21T19:02:50.524 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:50.525 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.525 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.525 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:50.525 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:50.525 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:50.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:50.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:50.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:50.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:50.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:50.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.047 DEBUG:teuthology.orchestra.run.smithi101:osd.7> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.7.service 2023-07-21T19:02:51.051 INFO:tasks.cephadm:Deploying osd.8 on smithi156 with /dev/vg_nvme/lv_2... 2023-07-21T19:02:51.051 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-07-21T19:02:51.124 INFO:journalctl@ceph.osd.7.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:02:51.784 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:02:51.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:51 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:51 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.799 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:51 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:51.799 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:51 smithi101 ceph-mon[128136]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:51.799 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:51 smithi101 ceph-mon[128136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:51.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:51 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:51 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:51 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:51.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:51 smithi005 ceph-mon[126460]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:51.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:51 smithi005 ceph-mon[126460]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:51 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:02:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:51 smithi156 ceph-mon[128316]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:51.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:51 smithi156 ceph-mon[128316]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-07-21T19:02:52.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:52.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T19:02:52.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T19:02:52.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:52.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:52.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:52 smithi156 ceph-mon[128316]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T19:02:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T19:02:52.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:52.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:52.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-mon[128136]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:52.842 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:02:52.504+0000 7f60c455f700 -1 osd.7 0 waiting for initial osdmap 2023-07-21T19:02:52.842 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:02:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:02:52.519+0000 7f60bdb4f700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:02:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 187 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-07-21T19:02:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: osdmap e49: 8 total, 7 up, 8 in 2023-07-21T19:02:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: from='osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:52.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:52 smithi005 ceph-mon[126460]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:02:53.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:53 smithi101 ceph-mon[128136]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:02:53.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:53 smithi101 ceph-mon[128136]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T19:02:53.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:53.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:53.775 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-07-21T19:02:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-07-21T19:02:53.776 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-21T19:02:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-21T19:02:53.776 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0391893 s, 268 MB/s 2023-07-21T19:02:53.776 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-21T19:02:53.812 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:53 smithi156 ceph-mon[128316]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:02:53.812 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:53 smithi156 ceph-mon[128316]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T19:02:53.812 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:53.812 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:53 smithi005 ceph-mon[126460]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:02:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:53 smithi005 ceph-mon[126460]: osdmap e50: 8 total, 7 up, 8 in 2023-07-21T19:02:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:54.258 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_2 2023-07-21T19:02:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:02:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:02:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] boot 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:54.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:54.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:54 smithi101 ceph-mon[128136]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T19:02:54.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:02:54.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] boot 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:54 smithi005 ceph-mon[126460]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 188 MiB used, 626 GiB / 626 GiB avail 2023-07-21T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: osd.7 [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] boot 2023-07-21T19:02:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: osdmap e51: 8 total, 8 up, 8 in 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:02:54.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:54.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:02:54.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:02:54.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:54 smithi156 ceph-mon[128316]: osdmap e52: 8 total, 8 up, 8 in 2023-07-21T19:02:54.980 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:02:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:55 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi101 2023-07-21T19:02:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:55 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-21T19:02:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:55 smithi101 ceph-mon[128136]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T19:02:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:55 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi101 2023-07-21T19:02:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:55 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-21T19:02:55.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:55 smithi005 ceph-mon[126460]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T19:02:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:55 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi101 2023-07-21T19:02:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:55 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 5755M 2023-07-21T19:02:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:55 smithi156 ceph-mon[128316]: osdmap e53: 8 total, 8 up, 8 in 2023-07-21T19:02:56.588 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 -- 172.21.15.156:0/2185955591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa880074540 msgr2=0x7fa8800729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 --2- 172.21.15.156:0/2185955591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa880074540 0x7fa8800729a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa87c0099c0 tx=0x7fa87c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 -- 172.21.15.156:0/2185955591 shutdown_connections 2023-07-21T19:02:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 --2- 172.21.15.156:0/2185955591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa88010c170 0x7fa88010e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 --2- 172.21.15.156:0/2185955591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa880074540 0x7fa8800729a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 --2- 172.21.15.156:0/2185955591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880072f70 0x7fa8800733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 -- 172.21.15.156:0/2185955591 >> 172.21.15.156:0/2185955591 conn(0x7fa8800fb1a0 msgr2=0x7fa8800fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:02:56.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 -- 172.21.15.156:0/2185955591 shutdown_connections 2023-07-21T19:02:56.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.578+0000 7fa887f4e700 1 -- 172.21.15.156:0/2185955591 wait complete. 2023-07-21T19:02:56.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.579+0000 7fa887f4e700 1 Processor -- start 2023-07-21T19:02:56.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 -- start start 2023-07-21T19:02:56.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa880072f70 0x7fa88019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:56.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 0x7fa88019c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8801a3930 con 0x7fa880074540 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8801a3a70 con 0x7fa880072f70 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa887f4e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8801a3c00 con 0x7fa88010c170 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.580+0000 7fa8864eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:56.592 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:46354/0 (socket says 172.21.15.156:46354) 2023-07-21T19:02:56.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 -- 172.21.15.156:0/3346289568 learned_addr learned my addr 172.21.15.156:0/3346289568 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T19:02:56.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa885cea700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa880072f70 0x7fa88019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8854e9700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 0x7fa88019c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 -- 172.21.15.156:0/3346289568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa880072f70 msgr2=0x7fa88019bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa880072f70 0x7fa88019bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 -- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 msgr2=0x7fa88019c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 0x7fa88019c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:02:56.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 -- 172.21.15.156:0/3346289568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa87c0095f0 con 0x7fa88010c170 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8854e9700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 0x7fa88019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa8864eb700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa878007d40 tx=0x7fa87800a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa87800ad60 con 0x7fa88010c170 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa878007e90 con 0x7fa88010c170 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8801a3ee0 con 0x7fa88010c170 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.581+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8780115e0 con 0x7fa88010c170 2023-07-21T19:02:56.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.582+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8801a4300 con 0x7fa88010c170 2023-07-21T19:02:56.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.583+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa880101cc0 con 0x7fa88010c170 2023-07-21T19:02:56.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.584+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa878012420 con 0x7fa88010c170 2023-07-21T19:02:56.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.586+0000 7fa876ffd700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 0x7fa86c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:02:56.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.586+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5397+0+0 (secure 0 0 0) 0x7fa8780cc100 con 0x7fa88010c170 2023-07-21T19:02:56.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.586+0000 7fa885cea700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 0x7fa86c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:02:56.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.587+0000 7fa885cea700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 0x7fa86c0a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:02:56.601 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.590+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa878096310 con 0x7fa88010c170 2023-07-21T19:02:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:56 smithi156 ceph-mon[128316]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:56.803 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:02:56.792+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fa880060950 con 0x7fa86c0a1150 2023-07-21T19:02:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:56 smithi101 ceph-mon[128136]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:56.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:56 smithi005 ceph-mon[126460]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:57.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:57 smithi101 ceph-mon[128136]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T19:02:57.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:57.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:57.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:57.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:57 smithi156 ceph-mon[128316]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T19:02:57.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:57.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:57.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:57.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:57 smithi005 ceph-mon[126460]: osdmap e54: 8 total, 8 up, 8 in 2023-07-21T19:02:57.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:02:57.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:02:57.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:02:58.822 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:58 smithi156 ceph-mon[128316]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:58.822 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:58 smithi156 ceph-mon[128316]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:58.822 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:02:58 smithi156 ceph-mon[128316]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T19:02:58.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:58 smithi101 ceph-mon[128136]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:58.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:58 smithi101 ceph-mon[128136]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:58.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:02:58 smithi101 ceph-mon[128136]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T19:02:58.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:58 smithi005 ceph-mon[126460]: from='client.24316 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:02:58.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:58 smithi005 ceph-mon[126460]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:02:58.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:02:58 smithi005 ceph-mon[126460]: osdmap e55: 8 total, 8 up, 8 in 2023-07-21T19:03:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/96590334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]': finished 2023-07-21T19:03:00.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T19:03:00.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:00.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:00.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/96590334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]': finished 2023-07-21T19:03:00.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T19:03:00.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/96590334' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]: dispatch 2023-07-21T19:03:00.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fc535b54-e81e-457d-ac39-57b75be1a082"}]': finished 2023-07-21T19:03:00.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: osdmap e56: 9 total, 8 up, 9 in 2023-07-21T19:03:00.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:01.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/3282043863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:01.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:01 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/3282043863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:01.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:01 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/3282043863' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:02.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:02 smithi101 ceph-mon[128136]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:02.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:02 smithi005 ceph-mon[126460]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:02.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:02 smithi156 ceph-mon[128316]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:04.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:04 smithi101 ceph-mon[128136]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:04.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:04 smithi005 ceph-mon[126460]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:04.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:04 smithi156 ceph-mon[128316]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:06.810 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:06 smithi156 ceph-mon[128316]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:06.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:06 smithi101 ceph-mon[128136]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:06.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:06 smithi005 ceph-mon[126460]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T19:03:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:07.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T19:03:07.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-07-21T19:03:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:08.667 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:08 smithi156 ceph-mon[128316]: Deploying daemon osd.8 on smithi156 2023-07-21T19:03:08.668 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:08 smithi156 ceph-mon[128316]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:08 smithi101 ceph-mon[128136]: Deploying daemon osd.8 on smithi156 2023-07-21T19:03:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:08 smithi101 ceph-mon[128136]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:08.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:08 smithi005 ceph-mon[126460]: Deploying daemon osd.8 on smithi156 2023-07-21T19:03:08.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:08 smithi005 ceph-mon[126460]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:10.662 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:10 smithi156 ceph-mon[128316]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:10.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:10 smithi101 ceph-mon[128136]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:10.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:10 smithi005 ceph-mon[126460]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:12.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:12 smithi101 ceph-mon[128136]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:12.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:12 smithi005 ceph-mon[126460]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:12.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:12 smithi156 ceph-mon[128316]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:14 smithi101 ceph-mon[128136]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:14.865 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:14 smithi156 ceph-mon[128316]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:14.865 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.865 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.865 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:14.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:14 smithi005 ceph-mon[126460]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:14.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:14.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:16.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:16 smithi101 ceph-mon[128136]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:16.861 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:16 smithi156 ceph-mon[128316]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:16.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:16 smithi005 ceph-mon[126460]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:17.230 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.219+0000 7fa876ffd700 1 -- 172.21.15.156:0/3346289568 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa880060950 con 0x7fa86c0a1150 2023-07-21T19:03:17.230 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 8 on host 'smithi156' 2023-07-21T19:03:17.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 msgr2=0x7fa86c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:17.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 0x7fa86c0a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa87c006010 tx=0x7fa87c009f10 comp rx=0 tx=0).stop 2023-07-21T19:03:17.232 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 msgr2=0x7fa8801a3360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa878007d40 tx=0x7fa87800a340 comp rx=0 tx=0).stop 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 shutdown_connections 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa86c0a1150 0x7fa86c0a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa88010c170 0x7fa8801a3360 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa880072f70 0x7fa88019bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 --2- 172.21.15.156:0/3346289568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa880074540 0x7fa88019c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.222+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 >> 172.21.15.156:0/3346289568 conn(0x7fa8800fb1a0 msgr2=0x7fa8800fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:17.233 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.223+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 shutdown_connections 2023-07-21T19:03:17.234 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:03:17.223+0000 7fa887f4e700 1 -- 172.21.15.156:0/3346289568 wait complete. 2023-07-21T19:03:17.705 DEBUG:teuthology.orchestra.run.smithi156:osd.8> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.8.service 2023-07-21T19:03:17.710 INFO:tasks.cephadm:Deploying osd.9 on smithi005 with /dev/vg_nvme/lv_1... 2023-07-21T19:03:17.710 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T19:03:17.785 INFO:journalctl@ceph.osd.8.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:03:17.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:17.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:17 smithi005 ceph-mon[126460]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:17.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:17.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:17.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:17.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:17.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:17.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:17 smithi156 ceph-mon[128316]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:18.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:18.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:18.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:18.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:17 smithi101 ceph-mon[128136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-07-21T19:03:18.450 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:03:18.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:18.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T19:03:18.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T19:03:18.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:18.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:18.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:18 smithi005 ceph-mon[126460]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:18.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:18 smithi156 ceph-mon[128316]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:19.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:19.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-07-21T19:03:19.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: osdmap e57: 9 total, 8 up, 9 in 2023-07-21T19:03:19.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:19.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: from='osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:19.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:18 smithi101 ceph-mon[128136]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:03:19.430 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:03:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:03:19.249+0000 7fe54a5e9700 -1 osd.8 0 waiting for initial osdmap 2023-07-21T19:03:19.430 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:03:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:03:19.263+0000 7fe5433d8700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:03:20.335 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T19:03:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T19:03:20.336 INFO:teuthology.orchestra.run.smithi005.stderr: stderr: 10+0 records in 2023-07-21T19:03:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:10+0 records out 2023-07-21T19:03:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0300669 s, 349 MB/s 2023-07-21T19:03:20.336 INFO:teuthology.orchestra.run.smithi005.stderr:--> Zapping successful for: 2023-07-21T19:03:20.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:20 smithi005 ceph-mon[126460]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:03:20.543 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:20 smithi005 ceph-mon[126460]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T19:03:20.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:20 smithi005 ceph-mon[126460]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:20 smithi101 ceph-mon[128136]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:03:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:20 smithi101 ceph-mon[128136]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T19:03:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:20 smithi101 ceph-mon[128136]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:20.653 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:20 smithi156 ceph-mon[128316]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:03:20.653 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:20 smithi156 ceph-mon[128316]: osdmap e58: 9 total, 8 up, 9 in 2023-07-21T19:03:20.653 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.653 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:20.654 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:20 smithi156 ceph-mon[128316]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 215 MiB used, 715 GiB / 715 GiB avail 2023-07-21T19:03:20.835 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi005:vg_nvme/lv_1 2023-07-21T19:03:21.570 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:03:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:03:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:03:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] boot 2023-07-21T19:03:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T19:03:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:21.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:21.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:21 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:03:21.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] boot 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:21.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:21.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:21 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: osd.8 [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] boot 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: osdmap e59: 9 total, 9 up, 9 in 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:03:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:21.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:21 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:22.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:22 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi156 2023-07-21T19:03:22.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:22 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 5755M 2023-07-21T19:03:22.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:22 smithi005 ceph-mon[126460]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T19:03:22.388 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:22 smithi005 ceph-mon[126460]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:22.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:22 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi156 2023-07-21T19:03:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:22 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 5755M 2023-07-21T19:03:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:22 smithi101 ceph-mon[128136]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T19:03:22.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:22 smithi101 ceph-mon[128136]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:22 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi156 2023-07-21T19:03:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:22 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 5755M 2023-07-21T19:03:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:22 smithi156 ceph-mon[128316]: osdmap e60: 9 total, 9 up, 9 in 2023-07-21T19:03:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:22 smithi156 ceph-mon[128316]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:23.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 -- 172.21.15.5:0/2183918794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 msgr2=0x7f01d01011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:23.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 --2- 172.21.15.5:0/2183918794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d01011e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009990 tx=0x7f01c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 -- 172.21.15.5:0/2183918794 shutdown_connections 2023-07-21T19:03:23.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 --2- 172.21.15.5:0/2183918794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 0x7f01d0107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 --2- 172.21.15.5:0/2183918794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 0x7f01d00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 --2- 172.21.15.5:0/2183918794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d01011e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 -- 172.21.15.5:0/2183918794 >> 172.21.15.5:0/2183918794 conn(0x7f01d00fb1a0 msgr2=0x7f01d00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 -- 172.21.15.5:0/2183918794 shutdown_connections 2023-07-21T19:03:23.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.006+0000 7f01d4c56700 1 -- 172.21.15.5:0/2183918794 wait complete. 2023-07-21T19:03:23.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.007+0000 7f01d4c56700 1 Processor -- start 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.007+0000 7f01d4c56700 1 -- start start 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.007+0000 7f01d4c56700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 0x7f01d01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.007+0000 7f01d4c56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01d4c56700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 0x7f01d019ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01d4c56700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d019f540 con 0x7f01d0100d90 2023-07-21T19:03:23.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01d4c56700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d019f680 con 0x7f01d00ffb90 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01d4c56700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d019f810 con 0x7f01d0101720 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47390/0 (socket says 172.21.15.5:47390) 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 -- 172.21.15.5:0/3648405384 learned_addr learned my addr 172.21.15.5:0/3648405384 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01ce59c700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 0x7f01d01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01ced9d700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 0x7f01d019ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 -- 172.21.15.5:0/3648405384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 msgr2=0x7f01d019ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:23.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 0x7f01d019ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 -- 172.21.15.5:0/3648405384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 msgr2=0x7f01d01979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 0x7f01d01979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.008+0000 7f01cdd9b700 1 -- 172.21.15.5:0/3648405384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40095f0 con 0x7f01d0100d90 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01cdd9b700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01c40059d0 tx=0x7f01c4003a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c401d070 con 0x7f01d0100d90 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01c400bd70 con 0x7f01d0100d90 2023-07-21T19:03:23.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d019fa90 con 0x7f01d0100d90 2023-07-21T19:03:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c4022bf0 con 0x7f01d0100d90 2023-07-21T19:03:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.009+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d019ff00 con 0x7f01d0100d90 2023-07-21T19:03:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.010+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d010b200 con 0x7f01d0100d90 2023-07-21T19:03:23.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.012+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f01c4022d50 con 0x7f01d0100d90 2023-07-21T19:03:23.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.015+0000 7f01bf7fe700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 0x7f01b80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:23.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.015+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5858+0+0 (secure 0 0 0) 0x7f01c40cd330 con 0x7f01d0100d90 2023-07-21T19:03:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.015+0000 7f01ce59c700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 0x7f01b80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:23.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.015+0000 7f01ce59c700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 0x7f01b80a3680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01d0100bf0 tx=0x7f01c000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:23.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.019+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f01c4097400 con 0x7f01d0100d90 2023-07-21T19:03:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:23.235+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f01d0107750 con 0x7f01b80a11d0 2023-07-21T19:03:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:23 smithi101 ceph-mon[128136]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T19:03:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:23 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:23 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:23.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:23 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:23 smithi005 ceph-mon[126460]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T19:03:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:23 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:23 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:23.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:23 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:23 smithi156 ceph-mon[128316]: osdmap e61: 9 total, 9 up, 9 in 2023-07-21T19:03:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:23 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:23.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:23 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:23.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:23 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:24 smithi101 ceph-mon[128136]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:24.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:24 smithi101 ceph-mon[128136]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T19:03:24.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:24 smithi101 ceph-mon[128136]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:24.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:24 smithi005 ceph-mon[126460]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:24.661 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:24 smithi005 ceph-mon[126460]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T19:03:24.661 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:24 smithi005 ceph-mon[126460]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:24 smithi156 ceph-mon[128316]: from='client.14460 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi005:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:24 smithi156 ceph-mon[128316]: osdmap e62: 9 total, 9 up, 9 in 2023-07-21T19:03:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:24 smithi156 ceph-mon[128316]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:25.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:25 smithi101 ceph-mon[128136]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T19:03:25.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:25 smithi005 ceph-mon[126460]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T19:03:25.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:25 smithi156 ceph-mon[128316]: osdmap e63: 9 total, 9 up, 9 in 2023-07-21T19:03:26.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:26 smithi101 ceph-mon[128136]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-07-21T19:03:26.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:26 smithi005 ceph-mon[126460]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-07-21T19:03:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:26 smithi156 ceph-mon[128316]: pgmap v212: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 143 KiB/s, 0 objects/s recovering 2023-07-21T19:03:27.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]: dispatch 2023-07-21T19:03:27.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]': finished 2023-07-21T19:03:27.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:27 smithi101 ceph-mon[128136]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T19:03:27.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:27 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:27.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3224665860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:27.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]: dispatch 2023-07-21T19:03:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]': finished 2023-07-21T19:03:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:27 smithi005 ceph-mon[126460]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T19:03:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:27 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:27.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3224665860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:27.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]: dispatch 2023-07-21T19:03:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1201606840' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "41283e76-6d87-47be-b423-725502de65f4"}]': finished 2023-07-21T19:03:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:27 smithi156 ceph-mon[128316]: osdmap e64: 10 total, 9 up, 10 in 2023-07-21T19:03:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:27 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3224665860' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:28.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:28 smithi101 ceph-mon[128136]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-21T19:03:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:28 smithi005 ceph-mon[126460]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-21T19:03:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:28 smithi156 ceph-mon[128316]: pgmap v214: 1 pgs: 1 peering; 577 KiB data, 242 MiB used, 804 GiB / 805 GiB avail; 115 KiB/s, 0 objects/s recovering 2023-07-21T19:03:30.556 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:30 smithi005 ceph-mon[126460]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:03:30.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:30 smithi101 ceph-mon[128136]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:03:30.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:30 smithi156 ceph-mon[128316]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-07-21T19:03:32.584 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:32 smithi005 ceph-mon[126460]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:03:32.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:32 smithi101 ceph-mon[128136]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:03:32.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:32 smithi156 ceph-mon[128316]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:03:34.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:34 smithi101 ceph-mon[128136]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T19:03:34.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:34 smithi005 ceph-mon[126460]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T19:03:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:34 smithi156 ceph-mon[128316]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-07-21T19:03:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:35.523 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:35 smithi005 ceph-mon[126460]: Deploying daemon osd.9 on smithi005 2023-07-21T19:03:35.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:35 smithi101 ceph-mon[128136]: Deploying daemon osd.9 on smithi005 2023-07-21T19:03:35.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:35 smithi156 ceph-mon[128316]: Deploying daemon osd.9 on smithi005 2023-07-21T19:03:36.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:36 smithi101 ceph-mon[128136]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:36.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:36 smithi005 ceph-mon[126460]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:36 smithi156 ceph-mon[128316]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:38.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:38 smithi005 ceph-mon[126460]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:38.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:38 smithi101 ceph-mon[128136]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:38.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:38 smithi156 ceph-mon[128316]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:40 smithi005 ceph-mon[126460]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:40 smithi156 ceph-mon[128316]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:40.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:40 smithi101 ceph-mon[128136]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:40.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:40.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:42.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:42 smithi005 ceph-mon[126460]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:42.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:42 smithi156 ceph-mon[128316]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:42 smithi101 ceph-mon[128136]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.095+0000 7f01bf7fe700 1 -- 172.21.15.5:0/3648405384 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f01d0107750 con 0x7f01b80a11d0 2023-07-21T19:03:44.107 INFO:teuthology.orchestra.run.smithi005.stdout:Created osd(s) 9 on host 'smithi005' 2023-07-21T19:03:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 msgr2=0x7f01b80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 0x7f01b80a3680 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f01d0100bf0 tx=0x7f01c000b410 comp rx=0 tx=0).stop 2023-07-21T19:03:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 msgr2=0x7f01d0197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f01c40059d0 tx=0x7f01c4003a50 comp rx=0 tx=0).stop 2023-07-21T19:03:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 shutdown_connections 2023-07-21T19:03:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f01b80a11d0 0x7f01b80a3680 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01d0101720 0x7f01d019ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01d00ffb90 0x7f01d01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 --2- 172.21.15.5:0/3648405384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01d0100d90 0x7f01d0197ef0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:44.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 >> 172.21.15.5:0/3648405384 conn(0x7f01d00fb1a0 msgr2=0x7f01d00fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.098+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 shutdown_connections 2023-07-21T19:03:44.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:03:44.099+0000 7f01d4c56700 1 -- 172.21.15.5:0/3648405384 wait complete. 2023-07-21T19:03:44.601 DEBUG:teuthology.orchestra.run.smithi005:osd.9> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.9.service 2023-07-21T19:03:44.606 INFO:tasks.cephadm:Deploying osd.10 on smithi101 with /dev/vg_nvme/lv_1... 2023-07-21T19:03:44.607 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T19:03:44.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:44.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:44.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:44.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T19:03:44.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:44.677 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:44.677 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.677 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.678 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:44.678 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:44.678 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.679 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T19:03:44.679 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.679 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.680 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:44.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:44.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-07-21T19:03:44.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:44.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:03:44.684 INFO:journalctl@ceph.osd.9.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:03:45.334 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:03:45.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-mon[126460]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T19:03:45.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-mon[126460]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T19:03:45.462 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:45.463 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-mon[126460]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:03:45.463 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:03:45.435+0000 7f66547a8700 -1 osd.9 0 waiting for initial osdmap 2023-07-21T19:03:45.463 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:03:45 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:03:45.451+0000 7f664c595700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:03:45.742 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:45 smithi101 ceph-mon[128136]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T19:03:45.742 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:45 smithi101 ceph-mon[128136]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T19:03:45.742 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:45.743 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:45 smithi101 ceph-mon[128136]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:03:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:45 smithi156 ceph-mon[128316]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-07-21T19:03:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:45 smithi156 ceph-mon[128316]: osdmap e65: 10 total, 9 up, 10 in 2023-07-21T19:03:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:45 smithi156 ceph-mon[128316]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]: dispatch 2023-07-21T19:03:46.544 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:46 smithi101 ceph-mon[128136]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:46.545 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:46 smithi101 ceph-mon[128136]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:03:46.545 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:46 smithi101 ceph-mon[128136]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T19:03:46.545 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:46 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:46.546 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:46 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:46.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:46 smithi005 ceph-mon[126460]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:46.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:46 smithi005 ceph-mon[126460]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:03:46.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:46 smithi005 ceph-mon[126460]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T19:03:46.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:46.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:46 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:46 smithi156 ceph-mon[128316]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 243 MiB used, 804 GiB / 805 GiB avail 2023-07-21T19:03:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:46 smithi156 ceph-mon[128316]: from='osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi005", "root=default"]}]': finished 2023-07-21T19:03:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:46 smithi156 ceph-mon[128316]: osdmap e66: 10 total, 9 up, 10 in 2023-07-21T19:03:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:46 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:46.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:46 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:47.251 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T19:03:47.251 INFO:teuthology.orchestra.run.smithi101.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T19:03:47.251 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 10+0 records in 2023-07-21T19:03:47.251 INFO:teuthology.orchestra.run.smithi101.stderr:10+0 records out 2023-07-21T19:03:47.251 INFO:teuthology.orchestra.run.smithi101.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0357807 s, 293 MB/s 2023-07-21T19:03:47.252 INFO:teuthology.orchestra.run.smithi101.stderr: stderr: 2023-07-21T19:03:47.252 INFO:teuthology.orchestra.run.smithi101.stderr:--> Zapping successful for: 2023-07-21T19:03:47.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:03:47.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:03:47.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] boot 2023-07-21T19:03:47.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: osdmap e67: 10 total, 10 up, 10 in 2023-07-21T19:03:47.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:47 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:03:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] boot 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: osdmap e67: 10 total, 10 up, 10 in 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:47 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.706 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi101:vg_nvme/lv_1 2023-07-21T19:03:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:03:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:03:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: osd.9 [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] boot 2023-07-21T19:03:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: osdmap e67: 10 total, 10 up, 10 in 2023-07-21T19:03:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:47.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:47 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:03:48.426 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:03:48.729 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:48.729 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi005 2023-07-21T19:03:48.729 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 4318M 2023-07-21T19:03:48.729 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:48.730 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:48.730 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:48.730 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T19:03:48.730 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:48 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:48.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi005 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 4318M 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T19:03:48.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:48 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi005 2023-07-21T19:03:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 4318M 2023-07-21T19:03:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:48.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:03:48.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: osdmap e68: 10 total, 10 up, 10 in 2023-07-21T19:03:48.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:48 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:03:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 -- 172.21.15.101:0/1904302158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984101720 msgr2=0x7fb984107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 --2- 172.21.15.101:0/1904302158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984101720 0x7fb984107fe0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00b0a0 tx=0x7fb97c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 -- 172.21.15.101:0/1904302158 shutdown_connections 2023-07-21T19:03:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 --2- 172.21.15.101:0/1904302158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984101720 0x7fb984107fe0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.754 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 --2- 172.21.15.101:0/1904302158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb9840fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 --2- 172.21.15.101:0/1904302158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984100d90 0x7fb9841011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 -- 172.21.15.101:0/1904302158 >> 172.21.15.101:0/1904302158 conn(0x7fb9840fb1a0 msgr2=0x7fb9840fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:03:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 -- 172.21.15.101:0/1904302158 shutdown_connections 2023-07-21T19:03:49.755 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.743+0000 7fb98ace0700 1 -- 172.21.15.101:0/1904302158 wait complete. 2023-07-21T19:03:49.756 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.744+0000 7fb98ace0700 1 Processor -- start 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 -- start start 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb984197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984101720 0x7fb98419f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419f670 con 0x7fb984101720 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419f7b0 con 0x7fb984100d90 2023-07-21T19:03:49.757 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98ace0700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419f940 con 0x7fb9840ffb90 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb983fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb983fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.101:59572/0 (socket says 172.21.15.101:59572) 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb983fff700 1 -- 172.21.15.101:0/2450442970 learned_addr learned my addr 172.21.15.101:0/2450442970 (peer_addr_for_me v2:172.21.15.101:0/0) 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb98927d700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984101720 0x7fb98419f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.745+0000 7fb988a7c700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb984197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 -- 172.21.15.101:0/2450442970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 msgr2=0x7fb984197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb984197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.758 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 -- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984101720 msgr2=0x7fb98419f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984101720 0x7fb98419f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 -- 172.21.15.101:0/2450442970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009d00 con 0x7fb984100d90 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb988a7c700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb984197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb983fff700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9a0 tx=0x7fb97000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970009790 con 0x7fb984100d90 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98419fc20 con 0x7fb984100d90 2023-07-21T19:03:49.759 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb97000cbd0 con 0x7fb984100d90 2023-07-21T19:03:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb970018680 con 0x7fb984100d90 2023-07-21T19:03:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.746+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9841a0120 con 0x7fb984100d90 2023-07-21T19:03:49.760 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.747+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb984192080 con 0x7fb984100d90 2023-07-21T19:03:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.749+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb97000cd40 con 0x7fb984100d90 2023-07-21T19:03:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.751+0000 7fb981ffb700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 0x7fb9740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:03:49.761 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.751+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6261+0+0 (secure 0 0 0) 0x7fb970014070 con 0x7fb984100d90 2023-07-21T19:03:49.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.751+0000 7fb988a7c700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 0x7fb9740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:03:49.762 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.752+0000 7fb988a7c700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 0x7fb9740a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb984100bf0 tx=0x7fb97800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:03:49.763 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.753+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb970096620 con 0x7fb984100d90 2023-07-21T19:03:49.968 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:03:49.958+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fb984060950 con 0x7fb9740a1150 2023-07-21T19:03:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:50 smithi101 ceph-mon[128136]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:50 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:50 smithi005 ceph-mon[126460]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:50.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:50 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:50 smithi156 ceph-mon[128316]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:03:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:03:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:50 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:03:51.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:51 smithi101 ceph-mon[128136]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:51.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:51 smithi005 ceph-mon[126460]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:51 smithi156 ceph-mon[128316]: from='client.24383 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi101:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:03:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:52 smithi101 ceph-mon[128136]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:52 smithi005 ceph-mon[126460]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:52.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:52 smithi156 ceph-mon[128316]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:53.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:53 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/2806030831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:53 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:53 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]': finished 2023-07-21T19:03:53.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:53 smithi101 ceph-mon[128136]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T19:03:53.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:03:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:53 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/2806030831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:53 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:53 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]': finished 2023-07-21T19:03:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:53 smithi005 ceph-mon[126460]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T19:03:53.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:03:53.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:53 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/2806030831' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:53 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]: dispatch 2023-07-21T19:03:53.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:53 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ba0f83a2-7456-498d-bf66-564b17ccbce2"}]': finished 2023-07-21T19:03:53.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:53 smithi156 ceph-mon[128316]: osdmap e69: 11 total, 10 up, 11 in 2023-07-21T19:03:53.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:03:54.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:54 smithi101 ceph-mon[128136]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:54 smithi101 ceph-mon[128136]: from='client.? 172.21.15.101:0/1810433371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:54.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:54 smithi005 ceph-mon[126460]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:54.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:54 smithi005 ceph-mon[126460]: from='client.? 172.21.15.101:0/1810433371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:54.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:54 smithi156 ceph-mon[128316]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:54.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:54 smithi156 ceph-mon[128316]: from='client.? 172.21.15.101:0/1810433371' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:03:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:56 smithi101 ceph-mon[128136]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:56.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:56 smithi005 ceph-mon[126460]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:56.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:56 smithi156 ceph-mon[128316]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:58.564 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:03:58 smithi101 ceph-mon[128136]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:03:58 smithi005 ceph-mon[126460]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:03:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:03:58 smithi156 ceph-mon[128316]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:00.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:00 smithi101 ceph-mon[128136]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:00.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:00 smithi005 ceph-mon[126460]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:00 smithi156 ceph-mon[128316]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:01.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T19:04:01.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:01 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:01.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:01 smithi005 ceph-mon[126460]: Deploying daemon osd.10 on smithi101 2023-07-21T19:04:01.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T19:04:01.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:01 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:01.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:01 smithi156 ceph-mon[128316]: Deploying daemon osd.10 on smithi101 2023-07-21T19:04:01.690 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-07-21T19:04:01.691 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:01 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:01.691 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:01 smithi101 ceph-mon[128136]: Deploying daemon osd.10 on smithi101 2023-07-21T19:04:02.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:02 smithi005 ceph-mon[126460]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:02 smithi156 ceph-mon[128316]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:02.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:02 smithi101 ceph-mon[128136]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:04.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:04 smithi005 ceph-mon[126460]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:04 smithi156 ceph-mon[128316]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:04.734 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:04 smithi101 ceph-mon[128136]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:06.646 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:06 smithi101 ceph-mon[128136]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:06.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:06 smithi005 ceph-mon[126460]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:06.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:06 smithi156 ceph-mon[128316]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:08.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:08.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:07 smithi101 ceph-mon[128136]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:08.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:07 smithi005 ceph-mon[126460]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:08.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:08.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:08.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:07 smithi156 ceph-mon[128316]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:10.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:10 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:10.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:10.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:10 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:10.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:10 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:10.696 INFO:teuthology.orchestra.run.smithi101.stdout:Created osd(s) 10 on host 'smithi101' 2023-07-21T19:04:10.697 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.686+0000 7fb981ffb700 1 -- 172.21.15.101:0/2450442970 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fb984060950 con 0x7fb9740a1150 2023-07-21T19:04:10.698 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.689+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 msgr2=0x7fb9740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:10.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.689+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 0x7fb9740a3600 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fb984100bf0 tx=0x7fb97800b670 comp rx=0 tx=0).stop 2023-07-21T19:04:10.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 msgr2=0x7fb984198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:10.699 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb97000e9a0 tx=0x7fb97000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:04:10.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 shutdown_connections 2023-07-21T19:04:10.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9740a1150 0x7fb9740a3600 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:10.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb984101720 0x7fb98419f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:10.701 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9840ffb90 0x7fb984197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:10.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 --2- 172.21.15.101:0/2450442970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb984100d90 0x7fb984198020 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:10.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 >> 172.21.15.101:0/2450442970 conn(0x7fb9840fb1a0 msgr2=0x7fb984106730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:10.702 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 shutdown_connections 2023-07-21T19:04:10.703 INFO:teuthology.orchestra.run.smithi101.stderr:2023-07-21T19:04:10.690+0000 7fb98ace0700 1 -- 172.21.15.101:0/2450442970 wait complete. 2023-07-21T19:04:11.147 DEBUG:teuthology.orchestra.run.smithi101:osd.10> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.10.service 2023-07-21T19:04:11.152 INFO:tasks.cephadm:Deploying osd.11 on smithi156 with /dev/vg_nvme/lv_1... 2023-07-21T19:04:11.152 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-07-21T19:04:11.229 INFO:journalctl@ceph.osd.10.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:04:11.887 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:04:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:11.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:11.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:11 smithi156 ceph-mon[128316]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:11.954 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:11.954 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:11.955 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:11.955 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:11.955 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:11.955 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:11 smithi101 ceph-mon[128136]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:12.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:12.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:12.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:12.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:12.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-07-21T19:04:12.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:11 smithi005 ceph-mon[126460]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:12.795 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:12 smithi156 ceph-mon[128316]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T19:04:12.795 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:12 smithi156 ceph-mon[128316]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T19:04:12.795 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:12.796 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:12 smithi156 ceph-mon[128316]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:12.796 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:12 smithi156 ceph-mon[128316]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-mon[128136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T19:04:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-mon[128136]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T19:04:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:13.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-mon[128136]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:13.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-mon[128136]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:13.093 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:04:12.685+0000 7f922fbc3700 -1 osd.10 0 waiting for initial osdmap 2023-07-21T19:04:13.093 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:04:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:04:12.700+0000 7f922b9b8700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:04:13.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:12 smithi005 ceph-mon[126460]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-07-21T19:04:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:12 smithi005 ceph-mon[126460]: osdmap e70: 11 total, 10 up, 11 in 2023-07-21T19:04:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:12 smithi005 ceph-mon[126460]: from='osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:12 smithi005 ceph-mon[126460]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]: dispatch 2023-07-21T19:04:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:13 smithi101 ceph-mon[128136]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:04:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:13 smithi101 ceph-mon[128136]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T19:04:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:13.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:13 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:13.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:13 smithi101 ceph-mon[128136]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:13.895 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-07-21T19:04:13.896 INFO:teuthology.orchestra.run.smithi156.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-07-21T19:04:13.896 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10+0 records in 2023-07-21T19:04:13.896 INFO:teuthology.orchestra.run.smithi156.stderr:10+0 records out 2023-07-21T19:04:13.896 INFO:teuthology.orchestra.run.smithi156.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0304529 s, 344 MB/s 2023-07-21T19:04:13.896 INFO:teuthology.orchestra.run.smithi156.stderr:--> Zapping successful for: 2023-07-21T19:04:14.060 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:13 smithi156 ceph-mon[128316]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:04:14.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:13 smithi156 ceph-mon[128316]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T19:04:14.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:14.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:13 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:14.061 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:13 smithi156 ceph-mon[128316]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:14.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:13 smithi005 ceph-mon[126460]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi101", "root=default"]}]': finished 2023-07-21T19:04:14.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:13 smithi005 ceph-mon[126460]: osdmap e71: 11 total, 10 up, 11 in 2023-07-21T19:04:14.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:14.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:13 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:14.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:13 smithi005 ceph-mon[126460]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 894 GiB / 894 GiB avail 2023-07-21T19:04:14.365 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch daemon add osd smithi156:vg_nvme/lv_1 2023-07-21T19:04:15.089 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] boot 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T19:04:15.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi101 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-21T19:04:15.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:15.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:15.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:14 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:04:15.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] boot 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi101 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:15.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:15.171 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:14 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: osd.10 [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] boot 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: osdmap e72: 11 total, 11 up, 11 in 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi101 2023-07-21T19:04:15.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:15.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 4316M 2023-07-21T19:04:15.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:15.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:15.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:14 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:15.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:15 smithi156 ceph-mon[128316]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T19:04:15.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:15 smithi156 ceph-mon[128316]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:16.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:15 smithi101 ceph-mon[128136]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T19:04:16.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:15 smithi101 ceph-mon[128136]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:16.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:15 smithi005 ceph-mon[126460]: osdmap e73: 11 total, 11 up, 11 in 2023-07-21T19:04:16.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:15 smithi005 ceph-mon[126460]: pgmap v247: 1 pgs: 1 peering; 577 KiB data, 297 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:16.589 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/3150124428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 msgr2=0x7f3f94072940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:16.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/3150124428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94072940 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f3f90009990 tx=0x7f3f90009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/3150124428 shutdown_connections 2023-07-21T19:04:16.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/3150124428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 0x7f3f94109eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.590 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/3150124428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94072940 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/3150124428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 0x7f3f94073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/3150124428 >> 172.21.15.156:0/3150124428 conn(0x7f3f940fb1a0 msgr2=0x7f3f940fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:16.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/3150124428 shutdown_connections 2023-07-21T19:04:16.591 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.579+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/3150124428 wait complete. 2023-07-21T19:04:16.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 Processor -- start 2023-07-21T19:04:16.593 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 -- start start 2023-07-21T19:04:16.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 0x7f3f941979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:16.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:16.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 0x7f3f9419efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:16.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9419f570 con 0x7f3f94072f10 2023-07-21T19:04:16.594 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9419f6b0 con 0x7f3f94107a30 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.580+0000 7f3f9b6b3700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9419f840 con 0x7f3f940744e0 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:38248/0 (socket says 172.21.15.156:38248) 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 -- 172.21.15.156:0/321500123 learned_addr learned my addr 172.21.15.156:0/321500123 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f9944f700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 0x7f3f941979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:16.595 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f99c50700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 0x7f3f9419efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 -- 172.21.15.156:0/321500123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 msgr2=0x7f3f9419efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 0x7f3f9419efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 -- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 msgr2=0x7f3f941979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 0x7f3f941979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 -- 172.21.15.156:0/321500123 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f900095f0 con 0x7f3f940744e0 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.581+0000 7f3f98c4e700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400e9d0 tx=0x7f3f8400ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:16.596 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.582+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84009790 con 0x7f3f940744e0 2023-07-21T19:04:16.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.582+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f8400ee70 con 0x7f3f940744e0 2023-07-21T19:04:16.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.582+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9419fb20 con 0x7f3f940744e0 2023-07-21T19:04:16.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.582+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f84018420 con 0x7f3f940744e0 2023-07-21T19:04:16.597 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.582+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f941a0020 con 0x7f3f940744e0 2023-07-21T19:04:16.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.583+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f94106170 con 0x7f3f940744e0 2023-07-21T19:04:16.598 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.585+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3f8400cbd0 con 0x7f3f940744e0 2023-07-21T19:04:16.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.587+0000 7f3f8a7fc700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 0x7f3f800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:16.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.587+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f3f84014070 con 0x7f3f940744e0 2023-07-21T19:04:16.599 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.588+0000 7f3f9944f700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 0x7f3f800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:16.600 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.589+0000 7f3f9944f700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 0x7f3f800a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f90006010 tx=0x7f3f9000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:16.602 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.592+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3f84051790 con 0x7f3f940744e0 2023-07-21T19:04:16.807 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:16.797+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f94060950 con 0x7f3f800a1150 2023-07-21T19:04:17.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:16 smithi101 ceph-mon[128136]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T19:04:17.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:16 smithi005 ceph-mon[126460]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T19:04:17.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:16 smithi156 ceph-mon[128316]: osdmap e74: 11 total, 11 up, 11 in 2023-07-21T19:04:17.987 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T19:04:17.987 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:17.988 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:04:17.988 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:04:17.988 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:17.988 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:17 smithi156 ceph-mon[128316]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T19:04:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:04:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:04:18.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:18.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:17 smithi101 ceph-mon[128136]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:18.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: osdmap e75: 11 total, 11 up, 11 in 2023-07-21T19:04:18.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: from='client.24382 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi156:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:18.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-07-21T19:04:18.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-07-21T19:04:18.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:18.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:17 smithi005 ceph-mon[126460]: pgmap v250: 1 pgs: 1 peering; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:18.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:18 smithi156 ceph-mon[128316]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T19:04:19.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:18 smithi101 ceph-mon[128136]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T19:04:19.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:18 smithi005 ceph-mon[126460]: osdmap e76: 11 total, 11 up, 11 in 2023-07-21T19:04:20.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:19 smithi101 ceph-mon[128136]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T19:04:20.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:19 smithi005 ceph-mon[126460]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T19:04:20.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:19 smithi156 ceph-mon[128316]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 125 KiB/s, 0 objects/s recovering 2023-07-21T19:04:21.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/1231140689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]': finished 2023-07-21T19:04:21.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T19:04:21.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:21.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:20 smithi101 ceph-mon[128136]: from='client.? 172.21.15.156:0/2126158128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:04:21.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/1231140689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]': finished 2023-07-21T19:04:21.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T19:04:21.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:21.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.156:0/2126158128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:04:21.179 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/1231140689' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]: dispatch 2023-07-21T19:04:21.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2"}]': finished 2023-07-21T19:04:21.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: osdmap e77: 12 total, 11 up, 12 in 2023-07-21T19:04:21.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:21.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:20 smithi156 ceph-mon[128316]: from='client.? 172.21.15.156:0/2126158128' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-07-21T19:04:22.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:21 smithi101 ceph-mon[128136]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-21T19:04:22.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:21 smithi005 ceph-mon[126460]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-21T19:04:22.179 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:21 smithi156 ceph-mon[128316]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 103 KiB/s, 0 objects/s recovering 2023-07-21T19:04:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:24 smithi005 ceph-mon[126460]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-21T19:04:24.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:24 smithi156 ceph-mon[128316]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-21T19:04:24.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:24 smithi101 ceph-mon[128136]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 88 KiB/s, 0 objects/s recovering 2023-07-21T19:04:26.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:26 smithi005 ceph-mon[126460]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:04:26.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:26 smithi156 ceph-mon[128316]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:04:26.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:26 smithi101 ceph-mon[128136]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-07-21T19:04:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:28 smithi101 ceph-mon[128136]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-07-21T19:04:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T19:04:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:28.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:28 smithi101 ceph-mon[128136]: Deploying daemon osd.11 on smithi156 2023-07-21T19:04:28.599 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:28 smithi156 ceph-mon[128316]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-07-21T19:04:28.599 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T19:04:28.600 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:28.600 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:28 smithi156 ceph-mon[128316]: Deploying daemon osd.11 on smithi156 2023-07-21T19:04:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:28 smithi005 ceph-mon[126460]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail; 60 KiB/s, 0 objects/s recovering 2023-07-21T19:04:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-07-21T19:04:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:28 smithi005 ceph-mon[126460]: Deploying daemon osd.11 on smithi156 2023-07-21T19:04:30.597 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:30 smithi156 ceph-mon[128316]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:30.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:30 smithi005 ceph-mon[126460]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:30.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:30 smithi101 ceph-mon[128136]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:32.616 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:32 smithi156 ceph-mon[128316]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:32.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:32 smithi005 ceph-mon[126460]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:32.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:32 smithi101 ceph-mon[128136]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:34 smithi005 ceph-mon[126460]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:34 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:34 smithi156 ceph-mon[128316]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:34 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:34.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:34 smithi101 ceph-mon[128136]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:34 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:36.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:36 smithi005 ceph-mon[126460]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:36 smithi156 ceph-mon[128316]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:36.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:36 smithi101 ceph-mon[128136]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:37.434 INFO:teuthology.orchestra.run.smithi156.stdout:Created osd(s) 11 on host 'smithi156' 2023-07-21T19:04:37.435 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.424+0000 7f3f8a7fc700 1 -- 172.21.15.156:0/321500123 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f3f94060950 con 0x7f3f800a1150 2023-07-21T19:04:37.436 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 msgr2=0x7f3f800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:37.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 0x7f3f800a3600 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f90006010 tx=0x7f3f9000b260 comp rx=0 tx=0).stop 2023-07-21T19:04:37.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 msgr2=0x7f3f94197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:37.437 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3f8400e9d0 tx=0x7f3f8400ece0 comp rx=0 tx=0).stop 2023-07-21T19:04:37.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 shutdown_connections 2023-07-21T19:04:37.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3f800a1150 0x7f3f800a3600 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:37.438 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f94107a30 0x7f3f9419efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:37.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f94072f10 0x7f3f941979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:37.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 --2- 172.21.15.156:0/321500123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f940744e0 0x7f3f94197f20 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:37.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 >> 172.21.15.156:0/321500123 conn(0x7f3f940fb1a0 msgr2=0x7f3f940fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:37.439 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 shutdown_connections 2023-07-21T19:04:37.440 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:37.427+0000 7f3f9b6b3700 1 -- 172.21.15.156:0/321500123 wait complete. 2023-07-21T19:04:37.912 DEBUG:teuthology.orchestra.run.smithi156:osd.11> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.11.service 2023-07-21T19:04:37.917 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-07-21T19:04:37.918 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T19:04:37.991 INFO:journalctl@ceph.osd.11.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:04:38.564 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:04:38.645 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:38.645 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.646 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.646 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.646 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.646 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:38.646 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:38 smithi156 ceph-mon[128316]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:38 smithi005 ceph-mon[126460]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:38.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:38 smithi101 ceph-mon[128136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-07-21T19:04:39.630 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-mon[128316]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T19:04:39.630 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-mon[128316]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T19:04:39.630 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:39.630 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-mon[128316]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:39.630 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-mon[128316]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:39.631 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:04:39.489+0000 7f3bdec9c700 -1 osd.11 0 waiting for initial osdmap 2023-07-21T19:04:39.631 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:04:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:04:39.506+0000 7f3bd728a700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-07-21T19:04:39.834 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:39 smithi005 ceph-mon[126460]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T19:04:39.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:39 smithi005 ceph-mon[126460]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T19:04:39.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:39.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:39 smithi005 ceph-mon[126460]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:39.835 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:39 smithi005 ceph-mon[126460]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:39.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:39 smithi101 ceph-mon[128136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-07-21T19:04:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:39 smithi101 ceph-mon[128136]: osdmap e78: 12 total, 11 up, 12 in 2023-07-21T19:04:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:39 smithi101 ceph-mon[128136]: from='osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:39 smithi101 ceph-mon[128136]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]: dispatch 2023-07-21T19:04:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 -- 172.21.15.5:0/2430945815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc101720 msgr2=0x7fcedc107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 --2- 172.21.15.5:0/2430945815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc101720 0x7fcedc107fe0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fced400b0a0 tx=0x7fced400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 -- 172.21.15.5:0/2430945815 shutdown_connections 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 --2- 172.21.15.5:0/2430945815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc101720 0x7fcedc107fe0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 --2- 172.21.15.5:0/2430945815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc0ffb90 0x7fcedc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 --2- 172.21.15.5:0/2430945815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc100d90 0x7fcedc1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 -- 172.21.15.5:0/2430945815 >> 172.21.15.5:0/2430945815 conn(0x7fcedc0fb1a0 msgr2=0x7fcedc0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 -- 172.21.15.5:0/2430945815 shutdown_connections 2023-07-21T19:04:40.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.068+0000 7fcee2672700 1 -- 172.21.15.5:0/2430945815 wait complete. 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.069+0000 7fcee2672700 1 Processor -- start 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.069+0000 7fcee2672700 1 -- start start 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc0ffb90 0x7fcedc197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc101720 0x7fcedc19f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:40.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc19f5e0 con 0x7fcedc100d90 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc19f720 con 0x7fcedc101720 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee2672700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcedc19f8b0 con 0x7fcedc0ffb90 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc0ffb90 0x7fcedc197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53204/0 (socket says 172.21.15.5:53204) 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 -- 172.21.15.5:0/580963036 learned_addr learned my addr 172.21.15.5:0/580963036 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcee0c0f700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc101720 0x7fcedc19f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:40.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 -- 172.21.15.5:0/580963036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc0ffb90 msgr2=0x7fcedc197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc0ffb90 0x7fcedc197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.070+0000 7fcedb7fe700 1 -- 172.21.15.5:0/580963036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc101720 msgr2=0x7fcedc19f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc101720 0x7fcedc19f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fcedb7fe700 1 -- 172.21.15.5:0/580963036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced4009d00 con 0x7fcedc100d90 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fcedb7fe700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fced000d9b0 tx=0x7fced000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:40.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0004a30 con 0x7fcedc100d90 2023-07-21T19:04:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fced00054b0 con 0x7fcedc100d90 2023-07-21T19:04:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcedc19fb90 con 0x7fcedc100d90 2023-07-21T19:04:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fced0009bc0 con 0x7fcedc100d90 2023-07-21T19:04:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.071+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcedc1a0090 con 0x7fcedc100d90 2023-07-21T19:04:40.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.072+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcedc1920f0 con 0x7fcedc100d90 2023-07-21T19:04:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.074+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fced0005620 con 0x7fcedc100d90 2023-07-21T19:04:40.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.076+0000 7fced97fa700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 0x7fcec40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.076+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 6847+0+0 (secure 0 0 0) 0x7fced00cc250 con 0x7fcedc100d90 2023-07-21T19:04:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.076+0000 7fcedbfff700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 0x7fcec40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:40.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.076+0000 7fcedbfff700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 0x7fcec40a3680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcedc100bf0 tx=0x7fcecc00a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:40.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.080+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fced0095ec0 con 0x7fcedc100d90 2023-07-21T19:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.287+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcedc02c850 con 0x7fcedc100d90 2023-07-21T19:04:40.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.287+0000 7fced97fa700 1 -- 172.21.15.5:0/580963036 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v79) v1 ==== 74+0+133 (secure 0 0 0) 0x7fced00994e0 con 0x7fcedc100d90 2023-07-21T19:04:40.298 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 msgr2=0x7fcec40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:40.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 0x7fcec40a3680 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fcedc100bf0 tx=0x7fcecc00a3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 msgr2=0x7fcedc197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fced000d9b0 tx=0x7fced000dcc0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 shutdown_connections 2023-07-21T19:04:40.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcec40a11d0 0x7fcec40a3680 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcedc101720 0x7fcedc19f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcedc0ffb90 0x7fcedc197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 --2- 172.21.15.5:0/580963036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcedc100d90 0x7fcedc197f90 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.289+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 >> 172.21.15.5:0/580963036 conn(0x7fcedc0fb1a0 msgr2=0x7fcedc106670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.290+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 shutdown_connections 2023-07-21T19:04:40.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:40.290+0000 7fcee2672700 1 -- 172.21.15.5:0/580963036 wait complete. 2023-07-21T19:04:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:04:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T19:04:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/580963036' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:04:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T19:04:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/580963036' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:40.750 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":79,"num_osds":12,"num_up_osds":11,"osd_up_since":1689966253,"num_in_osds":12,"osd_in_since":1689966259,"num_remapped_pgs":0} 2023-07-21T19:04:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi156", "root=default"]}]': finished 2023-07-21T19:04:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: osdmap e79: 12 total, 11 up, 12 in 2023-07-21T19:04:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:40.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/580963036' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:41.752 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd stat -f json 2023-07-21T19:04:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: purged_snaps scrub starts 2023-07-21T19:04:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: purged_snaps scrub ok 2023-07-21T19:04:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] boot 2023-07-21T19:04:41.820 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.821 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:41.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:41.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.822 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:41 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: purged_snaps scrub starts 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: purged_snaps scrub ok 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] boot 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:41 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: purged_snaps scrub starts 2023-07-21T19:04:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: purged_snaps scrub ok 2023-07-21T19:04:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: osd.11 [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] boot 2023-07-21T19:04:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: osdmap e80: 12 total, 12 up, 12 in 2023-07-21T19:04:41.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:04:41.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:41.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:41.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:41.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:41 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:04:42.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: Detected new or changed devices on smithi156 2023-07-21T19:04:42.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 4316M 2023-07-21T19:04:42.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:42.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T19:04:42.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:42.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:42.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:42 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:42.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: Detected new or changed devices on smithi156 2023-07-21T19:04:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 4316M 2023-07-21T19:04:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T19:04:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:42 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: Detected new or changed devices on smithi156 2023-07-21T19:04:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 4316M 2023-07-21T19:04:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 298 MiB used, 983 GiB / 983 GiB avail 2023-07-21T19:04:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: osdmap e81: 12 total, 12 up, 12 in 2023-07-21T19:04:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:42 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/2782863908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28100d90 msgr2=0x7fcf281011e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/2782863908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28100d90 0x7fcf281011e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009990 tx=0x7fcf1c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/2782863908 shutdown_connections 2023-07-21T19:04:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/2782863908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf28101720 0x7fcf28107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/2782863908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf280ffb90 0x7fcf280fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/2782863908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28100d90 0x7fcf281011e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/2782863908 >> 172.21.15.5:0/2782863908 conn(0x7fcf280fb1a0 msgr2=0x7fcf280fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:43.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/2782863908 shutdown_connections 2023-07-21T19:04:43.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.976+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/2782863908 wait complete. 2023-07-21T19:04:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.977+0000 7fcf2d3a1700 1 Processor -- start 2023-07-21T19:04:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.977+0000 7fcf2d3a1700 1 -- start start 2023-07-21T19:04:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.977+0000 7fcf2d3a1700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf280ffb90 0x7fcf280742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:43.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.977+0000 7fcf2d3a1700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf28100d90 0x7fcf28072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.977+0000 7fcf2d3a1700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf2d3a1700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf280735c0 con 0x7fcf28101720 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf2d3a1700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf28073730 con 0x7fcf28100d90 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf2d3a1700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2819f850 con 0x7fcf280ffb90 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf26ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf280ffb90 0x7fcf280742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:43.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53224/0 (socket says 172.21.15.5:53224) 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 -- 172.21.15.5:0/1111560655 learned_addr learned my addr 172.21.15.5:0/1111560655 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf267fc700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf28100d90 0x7fcf28072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 -- 172.21.15.5:0/1111560655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf280ffb90 msgr2=0x7fcf280742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf280ffb90 0x7fcf280742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 -- 172.21.15.5:0/1111560655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf28100d90 msgr2=0x7fcf28072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf28100d90 0x7fcf28072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:43.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.978+0000 7fcf277fe700 1 -- 172.21.15.5:0/1111560655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1c0095f0 con 0x7fcf28101720 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf277fe700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf20007ee0 tx=0x7fcf2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20017070 con 0x7fcf28101720 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf2000f040 con 0x7fcf28101720 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2819fb30 con 0x7fcf28101720 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf20005570 con 0x7fcf28101720 2023-07-21T19:04:43.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.979+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf2819ffe0 con 0x7fcf28101720 2023-07-21T19:04:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.980+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf28107180 con 0x7fcf28101720 2023-07-21T19:04:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.982+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcf2000db20 con 0x7fcf28101720 2023-07-21T19:04:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.983+0000 7fcf0ffff700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 0x7fcf100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:43.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.984+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcf200cbc10 con 0x7fcf28101720 2023-07-21T19:04:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.984+0000 7fcf26ffd700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 0x7fcf100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:43.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.984+0000 7fcf26ffd700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 0x7fcf100a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf28100bf0 tx=0x7fcf1800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:44.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:43.988+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcf20095760 con 0x7fcf28101720 2023-07-21T19:04:44.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.195+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fcf2802cc70 con 0x7fcf28101720 2023-07-21T19:04:44.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.195+0000 7fcf0ffff700 1 -- 172.21.15.5:0/1111560655 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v81) v1 ==== 74+0+133 (secure 0 0 0) 0x7fcf20098d80 con 0x7fcf28101720 2023-07-21T19:04:44.206 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 msgr2=0x7fcf100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 0x7fcf100a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcf28100bf0 tx=0x7fcf1800a3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 msgr2=0x7fcf28072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fcf20007ee0 tx=0x7fcf2000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 shutdown_connections 2023-07-21T19:04:44.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcf100a1150 0x7fcf100a3600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf28101720 0x7fcf28072e40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf280ffb90 0x7fcf280742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 --2- 172.21.15.5:0/1111560655 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf28100d90 0x7fcf28072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 >> 172.21.15.5:0/1111560655 conn(0x7fcf280fb1a0 msgr2=0x7fcf281042c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 shutdown_connections 2023-07-21T19:04:44.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:44.198+0000 7fcf2d3a1700 1 -- 172.21.15.5:0/1111560655 wait complete. 2023-07-21T19:04:44.678 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"num_osds":12,"num_up_osds":12,"osd_up_since":1689966280,"num_in_osds":12,"osd_in_since":1689966259,"num_remapped_pgs":0} 2023-07-21T19:04:44.679 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T19:04:44.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:44 smithi101 ceph-mon[128136]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:44 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1111560655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:44.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:44 smithi005 ceph-mon[126460]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:44.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:44 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1111560655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:44 smithi156 ceph-mon[128316]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:44 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1111560655' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-07-21T19:04:45.410 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:04:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 -- 172.21.15.5:0/16698808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea00ffe00 msgr2=0x7f3ea01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 --2- 172.21.15.5:0/16698808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea00ffe00 0x7f3ea01001d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c0099c0 tx=0x7f3e9c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 -- 172.21.15.5:0/16698808 shutdown_connections 2023-07-21T19:04:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 --2- 172.21.15.5:0/16698808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea0104ea0 0x7f3ea010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 --2- 172.21.15.5:0/16698808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea00ffe00 0x7f3ea01001d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 --2- 172.21.15.5:0/16698808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 0x7f3ea0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 -- 172.21.15.5:0/16698808 >> 172.21.15.5:0/16698808 conn(0x7f3ea00fb660 msgr2=0x7f3ea00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:46.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 -- 172.21.15.5:0/16698808 shutdown_connections 2023-07-21T19:04:46.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.804+0000 7f3ea8082700 1 -- 172.21.15.5:0/16698808 wait complete. 2023-07-21T19:04:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.805+0000 7f3ea8082700 1 Processor -- start 2023-07-21T19:04:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.805+0000 7f3ea8082700 1 -- start start 2023-07-21T19:04:46.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 0x7f3ea0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 0x7f3ea0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a3fb0 con 0x7f3ea0104ea0 2023-07-21T19:04:46.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a4120 con 0x7f3ea00ffe00 2023-07-21T19:04:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea8082700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea01a42e0 con 0x7f3ea01007a0 2023-07-21T19:04:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea661f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea5e1e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 0x7f3ea0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea661f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44818/0 (socket says 172.21.15.5:44818) 2023-07-21T19:04:46.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea661f700 1 -- 172.21.15.5:0/2453285420 learned_addr learned my addr 172.21.15.5:0/2453285420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:46.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.806+0000 7f3ea561d700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 0x7f3ea0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:46.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 -- 172.21.15.5:0/2453285420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 msgr2=0x7f3ea0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 0x7f3ea0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 -- 172.21.15.5:0/2453285420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 msgr2=0x7f3ea0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 0x7f3ea0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 -- 172.21.15.5:0/2453285420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e9c0095f0 con 0x7f3ea0104ea0 2023-07-21T19:04:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea5e1e700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 0x7f3ea0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:46.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea661f700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007ee0 tx=0x7f3e9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:46.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9800ddd0 con 0x7f3ea0104ea0 2023-07-21T19:04:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e9800f040 con 0x7f3ea0104ea0 2023-07-21T19:04:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea01a45c0 con 0x7f3ea0104ea0 2023-07-21T19:04:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980047d0 con 0x7f3ea0104ea0 2023-07-21T19:04:46.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.807+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea01a4b10 con 0x7f3ea0104ea0 2023-07-21T19:04:46.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.808+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea004e210 con 0x7f3ea0104ea0 2023-07-21T19:04:46.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.810+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f3e98003a00 con 0x7f3ea0104ea0 2023-07-21T19:04:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.813+0000 7f3e96ffd700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 0x7f3e8c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.813+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3e980cbd10 con 0x7f3ea0104ea0 2023-07-21T19:04:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.813+0000 7f3ea5e1e700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 0x7f3e8c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:46.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.813+0000 7f3ea5e1e700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 0x7f3e8c0a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c006010 tx=0x7f3e9c003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:46.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:46.816+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3e9804e790 con 0x7f3ea0104ea0 2023-07-21T19:04:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:46 smithi101 ceph-mon[128136]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:46.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:46 smithi005 ceph-mon[126460]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:46.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:46 smithi156 ceph-mon[128316]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:47.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.023+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ea0073d40 con 0x7f3ea0104ea0 2023-07-21T19:04:47.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.025+0000 7f3e96ffd700 1 -- 172.21.15.5:0/2453285420 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18399 (secure 0 0 0) 0x7f3e980d1020 con 0x7f3ea0104ea0 2023-07-21T19:04:47.034 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:47.035 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:04:41.495890+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:04:47.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 msgr2=0x7f3e8c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 0x7f3e8c0a3680 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3e9c006010 tx=0x7f3e9c003680 comp rx=0 tx=0).stop 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 msgr2=0x7f3ea00732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e98007ee0 tx=0x7f3e9800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 shutdown_connections 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3e8c0a11d0 0x7f3e8c0a3680 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ea0104ea0 0x7f3ea00732d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:47.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ea00ffe00 0x7f3ea0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 --2- 172.21.15.5:0/2453285420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ea01007a0 0x7f3ea0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 >> 172.21.15.5:0/2453285420 conn(0x7f3ea00fb660 msgr2=0x7f3ea00fc1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 shutdown_connections 2023-07-21T19:04:47.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:47.027+0000 7f3ea8082700 1 -- 172.21.15.5:0/2453285420 wait complete. 2023-07-21T19:04:47.598 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:04:47.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd pool get .mgr pg_num 2023-07-21T19:04:47.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:47 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2453285420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:47 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2453285420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:47.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:47 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2453285420' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:48.330 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:04:48.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:48 smithi101 ceph-mon[128136]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:48.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:48 smithi005 ceph-mon[126460]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:48.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:48 smithi156 ceph-mon[128316]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:49.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 -- 172.21.15.5:0/3745198390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540728b0 msgr2=0x7f5154072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 --2- 172.21.15.5:0/3745198390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540728b0 0x7f5154072c80 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5144009990 tx=0x7f5144009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 -- 172.21.15.5:0/3745198390 shutdown_connections 2023-07-21T19:04:49.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 --2- 172.21.15.5:0/3745198390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 0x7f5154112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 --2- 172.21.15.5:0/3745198390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540728b0 0x7f5154072c80 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 --2- 172.21.15.5:0/3745198390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540731c0 0x7f5154110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 -- 172.21.15.5:0/3745198390 >> 172.21.15.5:0/3745198390 conn(0x7f51540fb660 msgr2=0x7f51540fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:49.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 -- 172.21.15.5:0/3745198390 shutdown_connections 2023-07-21T19:04:49.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.830+0000 7f515912d700 1 -- 172.21.15.5:0/3745198390 wait complete. 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.831+0000 7f515912d700 1 Processor -- start 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 -- start start 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540728b0 0x7f515419c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 0x7f51541a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51541a11b0 con 0x7f51540731c0 2023-07-21T19:04:49.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51541a1320 con 0x7f5154110bb0 2023-07-21T19:04:49.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.832+0000 7f515912d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51541a14e0 con 0x7f51540728b0 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f5152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540728b0 0x7f515419c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44838/0 (socket says 172.21.15.5:44838) 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 -- 172.21.15.5:0/1393843728 learned_addr learned my addr 172.21.15.5:0/1393843728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515359e700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 0x7f51541a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 -- 172.21.15.5:0/1393843728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540728b0 msgr2=0x7f515419c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540728b0 0x7f515419c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 -- 172.21.15.5:0/1393843728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 msgr2=0x7f51541a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 0x7f51541a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 -- 172.21.15.5:0/1393843728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440095f0 con 0x7f51540731c0 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.833+0000 7f515259c700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f514800d950 tx=0x7f514800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.834+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800f890 con 0x7f51540731c0 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.834+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f514800fe80 con 0x7f51540731c0 2023-07-21T19:04:49.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.834+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f514800e3b0 con 0x7f51540731c0 2023-07-21T19:04:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.834+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51541a17c0 con 0x7f51540731c0 2023-07-21T19:04:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.834+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51541a9020 con 0x7f51540731c0 2023-07-21T19:04:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.835+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5154103090 con 0x7f51540731c0 2023-07-21T19:04:49.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.837+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5148010460 con 0x7f51540731c0 2023-07-21T19:04:49.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.838+0000 7f513bfff700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 0x7f513c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:49.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.838+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f51480cc270 con 0x7f51540731c0 2023-07-21T19:04:49.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.838+0000 7f5152d9d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 0x7f513c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:49.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.839+0000 7f5152d9d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 0x7f513c0a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f514400b2e0 tx=0x7f5144005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:49.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:49.842+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5148095dc0 con 0x7f51540731c0 2023-07-21T19:04:50.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.047+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f515402c850 con 0x7f51540731c0 2023-07-21T19:04:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.047+0000 7f513bfff700 1 -- 172.21.15.5:0/1393843728 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f51480993e0 con 0x7f51540731c0 2023-07-21T19:04:50.057 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:04:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.049+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 msgr2=0x7f513c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 0x7f513c0a3680 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f514400b2e0 tx=0x7f5144005fb0 comp rx=0 tx=0).stop 2023-07-21T19:04:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 msgr2=0x7f515419c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f514800d950 tx=0x7f514800dc60 comp rx=0 tx=0).stop 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 shutdown_connections 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f513c0a11d0 0x7f513c0a3680 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5154110bb0 0x7f51541a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51540728b0 0x7f515419c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 --2- 172.21.15.5:0/1393843728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51540731c0 0x7f515419c730 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 >> 172.21.15.5:0/1393843728 conn(0x7f51540fb660 msgr2=0x7f51540fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 shutdown_connections 2023-07-21T19:04:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:50.050+0000 7f515912d700 1 -- 172.21.15.5:0/1393843728 wait complete. 2023-07-21T19:04:50.510 INFO:tasks.cephadm:Adding mds.a on smithi005 2023-07-21T19:04:50.510 INFO:tasks.cephadm:Adding mds.d on smithi005 2023-07-21T19:04:50.510 INFO:tasks.cephadm:Adding mds.g on smithi005 2023-07-21T19:04:50.510 INFO:tasks.cephadm:Adding mds.j on smithi005 2023-07-21T19:04:50.511 INFO:tasks.cephadm:Adding mds.b on smithi101 2023-07-21T19:04:50.511 INFO:tasks.cephadm:Adding mds.e on smithi101 2023-07-21T19:04:50.511 INFO:tasks.cephadm:Adding mds.h on smithi101 2023-07-21T19:04:50.511 INFO:tasks.cephadm:Adding mds.k on smithi101 2023-07-21T19:04:50.511 INFO:tasks.cephadm:Adding mds.c on smithi156 2023-07-21T19:04:50.512 INFO:tasks.cephadm:Adding mds.f on smithi156 2023-07-21T19:04:50.512 INFO:tasks.cephadm:Adding mds.i on smithi156 2023-07-21T19:04:50.512 INFO:tasks.cephadm:Adding mds.l on smithi156 2023-07-21T19:04:50.512 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph orch apply mds all '12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l' 2023-07-21T19:04:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:50 smithi101 ceph-mon[128136]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:50 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1393843728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:50 smithi005 ceph-mon[126460]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:50.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:50 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1393843728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:50.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:50 smithi156 ceph-mon[128316]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:50.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:50 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1393843728' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:51.241 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:04:52.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 -- 172.21.15.156:0/1485685787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 msgr2=0x7f5bc4107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:52.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/1485685787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4107fe0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00b0a0 tx=0x7f5bbc00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.767 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 -- 172.21.15.156:0/1485685787 shutdown_connections 2023-07-21T19:04:52.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/1485685787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4107fe0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/1485685787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc40ffb90 0x7f5bc40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/1485685787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc4100d90 0x7f5bc41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 -- 172.21.15.156:0/1485685787 >> 172.21.15.156:0/1485685787 conn(0x7f5bc40fb1a0 msgr2=0x7f5bc40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:52.768 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 -- 172.21.15.156:0/1485685787 shutdown_connections 2023-07-21T19:04:52.769 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.756+0000 7f5bcaa40700 1 -- 172.21.15.156:0/1485685787 wait complete. 2023-07-21T19:04:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.757+0000 7f5bcaa40700 1 Processor -- start 2023-07-21T19:04:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.757+0000 7f5bcaa40700 1 -- start start 2023-07-21T19:04:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc40ffb90 0x7f5bc40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc4100d90 0x7f5bc4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:52.770 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc40735c0 con 0x7f5bc40ffb90 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc4073730 con 0x7f5bc4100d90 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bcaa40700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc419f990 con 0x7f5bc4101720 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.156:35228/0 (socket says 172.21.15.156:35228) 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 -- 172.21.15.156:0/2665736387 learned_addr learned my addr 172.21.15.156:0/2665736387 (peer_addr_for_me v2:172.21.15.156:0/0) 2023-07-21T19:04:52.771 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc37fe700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc4100d90 0x7f5bc4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc3fff700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc40ffb90 0x7f5bc40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 -- 172.21.15.156:0/2665736387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc4100d90 msgr2=0x7f5bc4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc4100d90 0x7f5bc4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 -- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc40ffb90 msgr2=0x7f5bc40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc40ffb90 0x7f5bc40742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.758+0000 7f5bc8fdd700 1 -- 172.21.15.156:0/2665736387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc009d00 con 0x7f5bc4101720 2023-07-21T19:04:52.772 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bc8fdd700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00b070 tx=0x7f5bbc004c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc00e040 con 0x7f5bc4101720 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc419fc10 con 0x7f5bc4101720 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bbc003e90 con 0x7f5bc4101720 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bbc004020 con 0x7f5bc4101720 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.759+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc41a00d0 con 0x7f5bc4101720 2023-07-21T19:04:52.773 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.760+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc41071e0 con 0x7f5bc4101720 2023-07-21T19:04:52.774 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.764+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5bbc003a00 con 0x7f5bc4101720 2023-07-21T19:04:52.775 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.765+0000 7f5bc17fa700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 0x7f5bac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:52.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.765+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f5bbc0ce0f0 con 0x7f5bc4101720 2023-07-21T19:04:52.776 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.766+0000 7f5bc3fff700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 0x7f5bac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:52.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.767+0000 7f5bc3fff700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 0x7f5bac0a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4100bf0 tx=0x7f5bb400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:52.777 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.767+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5bbc097cd0 con 0x7f5bc4101720 2023-07-21T19:04:52.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:52 smithi101 ceph-mon[128136]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:52.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:52 smithi005 ceph-mon[126460]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:52.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:52 smithi156 ceph-mon[128316]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:52.981 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.971+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f5bc4060b50 con 0x7f5bac0a11d0 2023-07-21T19:04:52.991 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.980+0000 7f5bc17fa700 1 -- 172.21.15.156:0/2665736387 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f5bc4060b50 con 0x7f5bac0a11d0 2023-07-21T19:04:52.991 INFO:teuthology.orchestra.run.smithi156.stdout:Scheduled mds.all update... 2023-07-21T19:04:52.993 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 msgr2=0x7f5bac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 0x7f5bac0a3680 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4100bf0 tx=0x7f5bb400b410 comp rx=0 tx=0).stop 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 msgr2=0x7f5bc4072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5bbc00b070 tx=0x7f5bbc004c80 comp rx=0 tx=0).stop 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 shutdown_connections 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bac0a11d0 0x7f5bac0a3680 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.994 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc4101720 0x7f5bc4072e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc40ffb90 0x7f5bc40742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 --2- 172.21.15.156:0/2665736387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc4100d90 0x7f5bc4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:52.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 >> 172.21.15.156:0/2665736387 conn(0x7f5bc40fb1a0 msgr2=0x7f5bc4104320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:52.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 shutdown_connections 2023-07-21T19:04:52.995 INFO:teuthology.orchestra.run.smithi156.stderr:2023-07-21T19:04:52.983+0000 7f5bcaa40700 1 -- 172.21.15.156:0/2665736387 wait complete. 2023-07-21T19:04:53.523 DEBUG:teuthology.orchestra.run.smithi005:mds.a> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.a.service 2023-07-21T19:04:53.529 DEBUG:teuthology.orchestra.run.smithi005:mds.d> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.d.service 2023-07-21T19:04:53.575 DEBUG:teuthology.orchestra.run.smithi005:mds.g> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.g.service 2023-07-21T19:04:53.613 INFO:journalctl@ceph.mds.a.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:04:53.616 DEBUG:teuthology.orchestra.run.smithi005:mds.j> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.j.service 2023-07-21T19:04:53.653 INFO:journalctl@ceph.mds.d.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:04:53.656 DEBUG:teuthology.orchestra.run.smithi101:mds.b> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.b.service 2023-07-21T19:04:53.661 DEBUG:teuthology.orchestra.run.smithi101:mds.e> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service 2023-07-21T19:04:53.692 INFO:journalctl@ceph.mds.g.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:04:53.707 DEBUG:teuthology.orchestra.run.smithi101:mds.h> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.h.service 2023-07-21T19:04:53.726 INFO:journalctl@ceph.mds.j.smithi005.stdout:-- Logs begin at Fri 2023-07-21 18:25:48 UTC. -- 2023-07-21T19:04:53.741 INFO:journalctl@ceph.mds.b.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:04:53.744 DEBUG:teuthology.orchestra.run.smithi101:mds.k> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.k.service 2023-07-21T19:04:53.779 INFO:journalctl@ceph.mds.e.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:04:53.782 DEBUG:teuthology.orchestra.run.smithi156:mds.c> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.c.service 2023-07-21T19:04:53.788 DEBUG:teuthology.orchestra.run.smithi156:mds.f> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.f.service 2023-07-21T19:04:53.814 INFO:journalctl@ceph.mds.h.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:04:53.833 DEBUG:teuthology.orchestra.run.smithi156:mds.i> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.i.service 2023-07-21T19:04:53.848 INFO:journalctl@ceph.mds.k.smithi101.stdout:-- Logs begin at Fri 2023-07-21 18:26:18 UTC. -- 2023-07-21T19:04:53.863 INFO:journalctl@ceph.mds.c.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:04:53.866 DEBUG:teuthology.orchestra.run.smithi156:mds.l> sudo journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.l.service 2023-07-21T19:04:53.904 INFO:journalctl@ceph.mds.f.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:04:53.908 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-07-21T19:04:53.908 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:04:53.942 INFO:journalctl@ceph.mds.i.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:04:53.977 INFO:journalctl@ceph.mds.l.smithi156.stdout:-- Logs begin at Fri 2023-07-21 18:25:04 UTC. -- 2023-07-21T19:04:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-21T19:04:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:54.106 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:54.107 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:54.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: Deploying daemon mds.l on smithi156 2023-07-21T19:04:54.108 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:53 smithi156 ceph-mon[128316]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:54.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.119+0000 7f83f2dff700 1 -- 172.21.15.5:0/2465642271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 msgr2=0x7f83ec0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.119+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2465642271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec0fff10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009990 tx=0x7f83e0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 -- 172.21.15.5:0/2465642271 shutdown_connections 2023-07-21T19:04:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2465642271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec101680 0x7f83ec107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2465642271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec0fff10 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2465642271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec100cf0 0x7f83ec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 -- 172.21.15.5:0/2465642271 >> 172.21.15.5:0/2465642271 conn(0x7f83ec0fb0e0 msgr2=0x7f83ec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 -- 172.21.15.5:0/2465642271 shutdown_connections 2023-07-21T19:04:54.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.120+0000 7f83f2dff700 1 -- 172.21.15.5:0/2465642271 wait complete. 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.121+0000 7f83f2dff700 1 Processor -- start 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.121+0000 7f83f2dff700 1 -- start start 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec100cf0 0x7f83ec197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec101680 0x7f83ec19ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec19f500 con 0x7f83ec0ffb00 2023-07-21T19:04:54.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec19f640 con 0x7f83ec100cf0 2023-07-21T19:04:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f2dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec19f7d0 con 0x7f83ec101680 2023-07-21T19:04:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44854/0 (socket says 172.21.15.5:44854) 2023-07-21T19:04:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 -- 172.21.15.5:0/2234661210 learned_addr learned my addr 172.21.15.5:0/2234661210 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:54.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f139c700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec101680 0x7f83ec19ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83ebfff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec100cf0 0x7f83ec197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 -- 172.21.15.5:0/2234661210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec101680 msgr2=0x7f83ec19ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec101680 0x7f83ec19ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 -- 172.21.15.5:0/2234661210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec100cf0 msgr2=0x7f83ec197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec100cf0 0x7f83ec197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.122+0000 7f83f0b9b700 1 -- 172.21.15.5:0/2234661210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00095f0 con 0x7f83ec0ffb00 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83f0b9b700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83e0005910 tx=0x7f83e0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e001d070 con 0x7f83ec0ffb00 2023-07-21T19:04:54.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f83e000bdd0 con 0x7f83ec0ffb00 2023-07-21T19:04:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0022d60 con 0x7f83ec0ffb00 2023-07-21T19:04:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec19fa50 con 0x7f83ec0ffb00 2023-07-21T19:04:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.123+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec19fe20 con 0x7f83ec0ffb00 2023-07-21T19:04:54.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.124+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec192090 con 0x7f83ec0ffb00 2023-07-21T19:04:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.126+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f83e0022500 con 0x7f83ec0ffb00 2023-07-21T19:04:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.127+0000 7f83e9ffb700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 0x7f83dc0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.128+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f83e00cd9c0 con 0x7f83ec0ffb00 2023-07-21T19:04:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.128+0000 7f83ebfff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 0x7f83dc0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.128+0000 7f83ebfff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 0x7f83dc0a36d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009bf0 tx=0x7f83d8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:54.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.132+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f83e00975a0 con 0x7f83ec0ffb00 2023-07-21T19:04:54.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:54.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-21T19:04:54.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:54.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.342 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: Deploying daemon mds.l on smithi156 2023-07-21T19:04:54.343 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:53 smithi101 ceph-mon[128136]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:54.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.343+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f83ec02c850 con 0x7f83ec0ffb00 2023-07-21T19:04:54.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.344+0000 7f83e9ffb700 1 -- 172.21.15.5:0/2234661210 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18399 (secure 0 0 0) 0x7f83e009abc0 con 0x7f83ec0ffb00 2023-07-21T19:04:54.354 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:54.354 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:04:41.495890+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:04:54.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 msgr2=0x7f83dc0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 0x7f83dc0a36d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009bf0 tx=0x7f83d8009260 comp rx=0 tx=0).stop 2023-07-21T19:04:54.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 msgr2=0x7f83ec197980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83e0005910 tx=0x7f83e0005740 comp rx=0 tx=0).stop 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 shutdown_connections 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f83dc0a1220 0x7f83dc0a36d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f83ec101680 0x7f83ec19ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f83ec0ffb00 0x7f83ec197980 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 --2- 172.21.15.5:0/2234661210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f83ec100cf0 0x7f83ec197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 >> 172.21.15.5:0/2234661210 conn(0x7f83ec0fb0e0 msgr2=0x7f83ec106630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:54.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 shutdown_connections 2023-07-21T19:04:54.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.346+0000 7f83f2dff700 1 -- 172.21.15.5:0/2234661210 wait complete. 2023-07-21T19:04:54.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='client.24430 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:04:54.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: Saving service mds.all spec with placement smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-21T19:04:54.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:04:54.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: Deploying daemon mds.l on smithi156 2023-07-21T19:04:54.375 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:53 smithi005 ceph-mon[126460]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:54.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:04:54.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:04:54.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 -- 172.21.15.5:0/832685487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14300ffae0 msgr2=0x7f14300ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 --2- 172.21.15.5:0/832685487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14300ffae0 0x7f14300ffef0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f142c009990 tx=0x7f142c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 -- 172.21.15.5:0/832685487 shutdown_connections 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 --2- 172.21.15.5:0/832685487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1430101660 0x7f1430107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 --2- 172.21.15.5:0/832685487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14300ffae0 0x7f14300ffef0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 --2- 172.21.15.5:0/832685487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430100cd0 0x7f1430101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 -- 172.21.15.5:0/832685487 >> 172.21.15.5:0/832685487 conn(0x7f14300fb0e0 msgr2=0x7f14300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:54.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 -- 172.21.15.5:0/832685487 shutdown_connections 2023-07-21T19:04:54.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.575+0000 7f14379ff700 1 -- 172.21.15.5:0/832685487 wait complete. 2023-07-21T19:04:54.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 Processor -- start 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 -- start start 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 0x7f14300742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430101660 0x7f1430072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14300735e0 con 0x7f1430100cd0 2023-07-21T19:04:54.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1430073750 con 0x7f1430101660 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f14379ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143019f7b0 con 0x7f14300ffae0 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.576+0000 7f1434f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44872/0 (socket says 172.21.15.5:44872) 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 -- 172.21.15.5:0/243285513 learned_addr learned my addr 172.21.15.5:0/243285513 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1435f9c700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430101660 0x7f1430072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f143579b700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 0x7f14300742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 -- 172.21.15.5:0/243285513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 msgr2=0x7f14300742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 0x7f14300742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 -- 172.21.15.5:0/243285513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430101660 msgr2=0x7f1430072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430101660 0x7f1430072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 -- 172.21.15.5:0/243285513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0095f0 con 0x7f1430100cd0 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f143579b700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 0x7f14300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f1434f9a700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f142000e9a0 tx=0x7f142000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420009790 con 0x7f1430100cd0 2023-07-21T19:04:54.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f142000cbd0 con 0x7f1430100cd0 2023-07-21T19:04:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143019fa90 con 0x7f1430100cd0 2023-07-21T19:04:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.577+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1420018680 con 0x7f1430100cd0 2023-07-21T19:04:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.578+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143019ffa0 con 0x7f1430100cd0 2023-07-21T19:04:54.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.579+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143010b160 con 0x7f1430100cd0 2023-07-21T19:04:54.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.581+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f142000cd40 con 0x7f1430100cd0 2023-07-21T19:04:54.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.582+0000 7f14267fc700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 0x7f141c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:54.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.582+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f1420014070 con 0x7f1430100cd0 2023-07-21T19:04:54.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.582+0000 7f143579b700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 0x7f141c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:54.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.583+0000 7f143579b700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 0x7f141c0a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f142c00b2e0 tx=0x7f142c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:54.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.586+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1420051790 con 0x7f1430100cd0 2023-07-21T19:04:54.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.794+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1430106a90 con 0x7f1430100cd0 2023-07-21T19:04:54.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.795+0000 7f14267fc700 1 -- 172.21.15.5:0/243285513 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1420096300 con 0x7f1430100cd0 2023-07-21T19:04:54.804 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:04:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 msgr2=0x7f141c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 0x7f141c0a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f142c00b2e0 tx=0x7f142c005960 comp rx=0 tx=0).stop 2023-07-21T19:04:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 msgr2=0x7f1430072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f142000e9a0 tx=0x7f142000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 shutdown_connections 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f141c0a1150 0x7f141c0a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1430101660 0x7f1430072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14300ffae0 0x7f14300742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 --2- 172.21.15.5:0/243285513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1430100cd0 0x7f1430072900 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 >> 172.21.15.5:0/243285513 conn(0x7f14300fb0e0 msgr2=0x7f14300fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:54.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 shutdown_connections 2023-07-21T19:04:54.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:54.797+0000 7f14379ff700 1 -- 172.21.15.5:0/243285513 wait complete. 2023-07-21T19:04:54.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:04:55.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 -- 172.21.15.5:0/4027724311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec101680 msgr2=0x7f71ec107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 --2- 172.21.15.5:0/4027724311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec101680 0x7f71ec107f40 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f71e400b0a0 tx=0x7f71e400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 -- 172.21.15.5:0/4027724311 shutdown_connections 2023-07-21T19:04:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 --2- 172.21.15.5:0/4027724311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec101680 0x7f71ec107f40 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 --2- 172.21.15.5:0/4027724311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 --2- 172.21.15.5:0/4027724311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec100cf0 0x7f71ec101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 -- 172.21.15.5:0/4027724311 >> 172.21.15.5:0/4027724311 conn(0x7f71ec0fb0e0 msgr2=0x7f71ec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 -- 172.21.15.5:0/4027724311 shutdown_connections 2023-07-21T19:04:55.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.020+0000 7f71f33ff700 1 -- 172.21.15.5:0/4027724311 wait complete. 2023-07-21T19:04:55.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.021+0000 7f71f33ff700 1 Processor -- start 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.021+0000 7f71f33ff700 1 -- start start 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.021+0000 7f71f33ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.021+0000 7f71f33ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f33ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 0x7f71ec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f33ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec0735e0 con 0x7f71ec100cf0 2023-07-21T19:04:55.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f33ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec073750 con 0x7f71ec101680 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f33ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec19f8e0 con 0x7f71ec0ffb00 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f119b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44886/0 (socket says 172.21.15.5:44886) 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 -- 172.21.15.5:0/1414460190 learned_addr learned my addr 172.21.15.5:0/1414460190 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:55.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 -- 172.21.15.5:0/1414460190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 msgr2=0x7f71ec0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f199c700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 0x7f71ec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:55.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 -- 172.21.15.5:0/1414460190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 msgr2=0x7f71ec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:55.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 0x7f71ec072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 -- 172.21.15.5:0/1414460190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e4009d00 con 0x7f71ec100cf0 2023-07-21T19:04:55.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f119b700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f199c700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 0x7f71ec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.022+0000 7f71f099a700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d980 tx=0x7f71dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.023+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc004220 con 0x7f71ec100cf0 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.023+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71dc007640 con 0x7f71ec100cf0 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.023+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec19fbc0 con 0x7f71ec100cf0 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.023+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71dc0055e0 con 0x7f71ec100cf0 2023-07-21T19:04:55.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.023+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1a00d0 con 0x7f71ec100cf0 2023-07-21T19:04:55.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.024+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71ec10b1e0 con 0x7f71ec100cf0 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.026+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f71dc004750 con 0x7f71ec100cf0 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.027+0000 7f71e27fc700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 0x7f71d80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.027+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f71dc0cc310 con 0x7f71ec100cf0 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.028+0000 7f71f119b700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 0x7f71d80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.028+0000 7f71f119b700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 0x7f71d80a3650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71ec100b50 tx=0x7f71e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:55.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.031+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f71dc095e60 con 0x7f71ec100cf0 2023-07-21T19:04:55.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:55 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2234661210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:55.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:55 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/243285513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:55.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.284+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71ec10b400 con 0x7f71ec100cf0 2023-07-21T19:04:55.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.285+0000 7f71e27fc700 1 -- 172.21.15.5:0/1414460190 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7f71dc005740 con 0x7f71ec100cf0 2023-07-21T19:04:55.295 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:55.295 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-07-21T19:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 msgr2=0x7f71d80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 0x7f71d80a3650 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f71ec100b50 tx=0x7f71e8009260 comp rx=0 tx=0).stop 2023-07-21T19:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 msgr2=0x7f71ec072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:55.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f71dc00d980 tx=0x7f71dc00dc90 comp rx=0 tx=0).stop 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 shutdown_connections 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f71d80a11a0 0x7f71d80a3650 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec101680 0x7f71ec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0ffb00 0x7f71ec0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 --2- 172.21.15.5:0/1414460190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec100cf0 0x7f71ec072900 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 >> 172.21.15.5:0/1414460190 conn(0x7f71ec0fb0e0 msgr2=0x7f71ec0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 shutdown_connections 2023-07-21T19:04:55.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:55.287+0000 7f71f33ff700 1 -- 172.21.15.5:0/1414460190 wait complete. 2023-07-21T19:04:55.299 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 1 2023-07-21T19:04:55.340 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:55 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2234661210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:55.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:55 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/243285513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:55.389 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:55 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2234661210' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:04:55.389 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:55 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/243285513' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:04:55.680 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:55 smithi156 systemd[1]: Starting Ceph mds.l for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:04:56.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:56 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1414460190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:56.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:56 smithi156 ceph-mon[128316]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:56.101 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:55 smithi156 podman[153729]: 2023-07-21T19:04:56.340 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:56 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1414460190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:56.341 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:56 smithi101 ceph-mon[128136]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:56.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:56 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1414460190' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:56.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:56 smithi005 ceph-mon[126460]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:56.430 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:56 smithi156 bash[153729]: 1fe14c3cc3c943ac591d7fbe59b6b12ba59378783bf2dc595dbc086447da2186 2023-07-21T19:04:56.430 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:56 smithi156 ceph-mds[153770]: -- 172.21.15.156:0/3361570471 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ed2554b1e0 con 0x55ed25520c00 2023-07-21T19:04:56.430 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-l[153746]: starting mds.l at 2023-07-21T19:04:56.430 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:04:56 smithi156 systemd[1]: Started Ceph mds.l for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:04:57.316 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:04:57.384 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.156:6832/2408636934,v1:172.21.15.156:6833/2408636934] up:boot 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: fsmap 1 up:standby 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:57.385 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:57.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:57.386 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:57 smithi005 ceph-mon[126460]: Deploying daemon mds.k on smithi101 2023-07-21T19:04:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.156:6832/2408636934,v1:172.21.15.156:6833/2408636934] up:boot 2023-07-21T19:04:57.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: fsmap 1 up:standby 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:57.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:57.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:57 smithi156 ceph-mon[128316]: Deploying daemon mds.k on smithi101 2023-07-21T19:04:57.509 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.156:6832/2408636934,v1:172.21.15.156:6833/2408636934] up:boot 2023-07-21T19:04:57.509 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: fsmap 1 up:standby 2023-07-21T19:04:57.509 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:04:57.509 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:04:57.510 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:57 smithi101 ceph-mon[128136]: Deploying daemon mds.k on smithi101 2023-07-21T19:04:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 -- 172.21.15.5:0/3881265421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 msgr2=0x7efbfc101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 --2- 172.21.15.5:0/3881265421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc101140 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7efbe4009990 tx=0x7efbe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 -- 172.21.15.5:0/3881265421 shutdown_connections 2023-07-21T19:04:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 --2- 172.21.15.5:0/3881265421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc101680 0x7efbfc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 --2- 172.21.15.5:0/3881265421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc0ffb00 0x7efbfc0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 --2- 172.21.15.5:0/3881265421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc101140 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 -- 172.21.15.5:0/3881265421 >> 172.21.15.5:0/3881265421 conn(0x7efbfc0fb0e0 msgr2=0x7efbfc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 -- 172.21.15.5:0/3881265421 shutdown_connections 2023-07-21T19:04:57.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.521+0000 7efc017ff700 1 -- 172.21.15.5:0/3881265421 wait complete. 2023-07-21T19:04:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.522+0000 7efc017ff700 1 Processor -- start 2023-07-21T19:04:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.522+0000 7efc017ff700 1 -- start start 2023-07-21T19:04:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0ffb00 0x7efbfc1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:57.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc101680 0x7efbfc19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19f520 con 0x7efbfc100cf0 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19f660 con 0x7efbfc101680 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efc017ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc19f7f0 con 0x7efbfc0ffb00 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39614/0 (socket says 172.21.15.5:39614) 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 -- 172.21.15.5:0/1486768034 learned_addr learned my addr 172.21.15.5:0/1486768034 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:04:57.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbfaffd700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0ffb00 0x7efbfc1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbfb7fe700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc101680 0x7efbfc19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 -- 172.21.15.5:0/1486768034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0ffb00 msgr2=0x7efbfc1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0ffb00 0x7efbfc1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 -- 172.21.15.5:0/1486768034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc101680 msgr2=0x7efbfc19ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc101680 0x7efbfc19ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.523+0000 7efbf3fff700 1 -- 172.21.15.5:0/1486768034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbe40095f0 con 0x7efbfc100cf0 2023-07-21T19:04:57.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efbf3fff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efbe40098e0 tx=0x7efbe4004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe401d070 con 0x7efbfc100cf0 2023-07-21T19:04:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efbe400bdd0 con 0x7efbfc100cf0 2023-07-21T19:04:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbe4022d50 con 0x7efbfc100cf0 2023-07-21T19:04:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc19fa70 con 0x7efbfc100cf0 2023-07-21T19:04:57.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.524+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc19fe40 con 0x7efbfc100cf0 2023-07-21T19:04:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.525+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc192090 con 0x7efbfc100cf0 2023-07-21T19:04:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.527+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efbe400b940 con 0x7efbfc100cf0 2023-07-21T19:04:57.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.529+0000 7efbf8ff9700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 0x7efbe80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:04:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.529+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efbe40cdba0 con 0x7efbfc100cf0 2023-07-21T19:04:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.529+0000 7efbfaffd700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 0x7efbe80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:04:57.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.530+0000 7efbfaffd700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 0x7efbe80a3650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efbfc100b50 tx=0x7efbec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:04:57.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.533+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efbe4097780 con 0x7efbfc100cf0 2023-07-21T19:04:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.795+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7efbfc02c850 con 0x7efbfc100cf0 2023-07-21T19:04:57.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.796+0000 7efbf8ff9700 1 -- 172.21.15.5:0/1486768034 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7efbe409ada0 con 0x7efbfc100cf0 2023-07-21T19:04:57.806 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:04:57.806 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-07-21T19:04:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.798+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 msgr2=0x7efbe80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.798+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 0x7efbe80a3650 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efbfc100b50 tx=0x7efbec009260 comp rx=0 tx=0).stop 2023-07-21T19:04:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.798+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 msgr2=0x7efbfc197ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:04:57.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.798+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7efbe40098e0 tx=0x7efbe4004730 comp rx=0 tx=0).stop 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 shutdown_connections 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efbe80a11a0 0x7efbe80a3650 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efbfc101680 0x7efbfc19ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efbfc0ffb00 0x7efbfc1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 --2- 172.21.15.5:0/1486768034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efbfc100cf0 0x7efbfc197ee0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 >> 172.21.15.5:0/1486768034 conn(0x7efbfc0fb0e0 msgr2=0x7efbfc106650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 shutdown_connections 2023-07-21T19:04:57.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:04:57.799+0000 7efc017ff700 1 -- 172.21.15.5:0/1486768034 wait complete. 2023-07-21T19:04:57.810 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 2 2023-07-21T19:04:58.495 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:58 smithi101 ceph-mon[128136]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:58.496 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:04:58 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1486768034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:58.496 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:58 smithi101 systemd[1]: Starting Ceph mds.k for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:04:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:58 smithi005 ceph-mon[126460]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:04:58 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1486768034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:58.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:58 smithi156 ceph-mon[128316]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:04:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:04:58 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1486768034' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:04:59.223 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:58 smithi101 podman[152690]: 2023-07-21T19:04:59.591 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:59 smithi101 bash[152690]: 58d65fd645c7540cc134776db5b2b15fbb5068415168fc0cf4c3884265c62c1f 2023-07-21T19:04:59.591 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:59 smithi101 ceph-mds[152730]: -- 172.21.15.101:0/3429268698 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x557b35b511e0 con 0x557b35b27000 2023-07-21T19:04:59.591 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:59 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: starting mds.k at 2023-07-21T19:04:59.591 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:04:59 smithi101 systemd[1]: Started Ceph mds.k for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:04:59.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:00.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 -- 172.21.15.5:0/531175086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c074550 msgr2=0x7fa17c0729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 --2- 172.21.15.5:0/531175086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c074550 0x7fa17c0729a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fa16c009990 tx=0x7fa16c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 -- 172.21.15.5:0/531175086 shutdown_connections 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 --2- 172.21.15.5:0/531175086 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c10c0d0 0x7fa17c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 --2- 172.21.15.5:0/531175086 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c074550 0x7fa17c0729a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 --2- 172.21.15.5:0/531175086 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 0x7fa17c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.059+0000 7fa1813ff700 1 -- 172.21.15.5:0/531175086 >> 172.21.15.5:0/531175086 conn(0x7fa17c0fb0e0 msgr2=0x7fa17c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:00.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.060+0000 7fa1813ff700 1 -- 172.21.15.5:0/531175086 shutdown_connections 2023-07-21T19:05:00.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.060+0000 7fa1813ff700 1 -- 172.21.15.5:0/531175086 wait complete. 2023-07-21T19:05:00.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 Processor -- start 2023-07-21T19:05:00.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 -- start start 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 0x7fa17c19bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 0x7fa17c1a3320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a38f0 con 0x7fa17c10c0d0 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a3a60 con 0x7fa17c074550 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa1813ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa17c1a3c20 con 0x7fa17c072f70 2023-07-21T19:05:00.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa17b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 0x7fa17c1a3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.061+0000 7fa17a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51184/0 (socket says 172.21.15.5:51184) 2023-07-21T19:05:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 -- 172.21.15.5:0/426397029 learned_addr learned my addr 172.21.15.5:0/426397029 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:00.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17affd700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 0x7fa17c19bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 -- 172.21.15.5:0/426397029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 msgr2=0x7fa17c19bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 0x7fa17c19bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 -- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 msgr2=0x7fa17c1a3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 0x7fa17c1a3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 -- 172.21.15.5:0/426397029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa16c0095f0 con 0x7fa17c074550 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17b7fe700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 0x7fa17c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:00.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.062+0000 7fa17a7fc700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d980 tx=0x7fa17000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.063+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa170004020 con 0x7fa17c074550 2023-07-21T19:05:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.063+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa17000f690 con 0x7fa17c074550 2023-07-21T19:05:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.063+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa17c1a3f00 con 0x7fa17c074550 2023-07-21T19:05:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.063+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1700039e0 con 0x7fa17c074550 2023-07-21T19:05:00.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.063+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa17c1a4320 con 0x7fa17c074550 2023-07-21T19:05:00.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.064+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa17c101c20 con 0x7fa17c074550 2023-07-21T19:05:00.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.066+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa17000f800 con 0x7fa17c074550 2023-07-21T19:05:00.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.067+0000 7fa163fff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 0x7fa1640a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:00.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.068+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa1700cc080 con 0x7fa17c074550 2023-07-21T19:05:00.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.068+0000 7fa17affd700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 0x7fa1640a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:00.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.068+0000 7fa17affd700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 0x7fa1640a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c010 tx=0x7fa16c009ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:00.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.070+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa170095bd0 con 0x7fa17c074550 2023-07-21T19:05:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.351+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa17c02c850 con 0x7fa17c074550 2023-07-21T19:05:00.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.352+0000 7fa163fff700 1 -- 172.21.15.5:0/426397029 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7fa1700991f0 con 0x7fa17c074550 2023-07-21T19:05:00.363 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:00.363 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T19:05:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 msgr2=0x7fa1640a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 0x7fa1640a3650 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa16c00c010 tx=0x7fa16c009ee0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 msgr2=0x7fa17c19c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa17000d980 tx=0x7fa17000dc90 comp rx=0 tx=0).stop 2023-07-21T19:05:00.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 shutdown_connections 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa1640a11a0 0x7fa1640a3650 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa17c10c0d0 0x7fa17c1a3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa17c072f70 0x7fa17c19bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 --2- 172.21.15.5:0/426397029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa17c074550 0x7fa17c19c2f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 >> 172.21.15.5:0/426397029 conn(0x7fa17c0fb0e0 msgr2=0x7fa17c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:00.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 shutdown_connections 2023-07-21T19:05:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:00.354+0000 7fa1813ff700 1 -- 172.21.15.5:0/426397029 wait complete. 2023-07-21T19:05:00.367 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-07-21T19:05:00.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6832/2387040321,v1:172.21.15.101:6833/2387040321] up:boot 2023-07-21T19:05:00.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: fsmap 2 up:standby 2023-07-21T19:05:00.563 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:05:00.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:00.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.564 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:00.565 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:00.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:00.569 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:00 smithi005 ceph-mon[126460]: Deploying daemon mds.j on smithi005 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6832/2387040321,v1:172.21.15.101:6833/2387040321] up:boot 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: fsmap 2 up:standby 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:00.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:00.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:00.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:00 smithi101 ceph-mon[128136]: Deploying daemon mds.j on smithi005 2023-07-21T19:05:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6832/2387040321,v1:172.21.15.101:6833/2387040321] up:boot 2023-07-21T19:05:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: fsmap 2 up:standby 2023-07-21T19:05:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:05:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:00.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:00 smithi156 ceph-mon[128316]: Deploying daemon mds.j on smithi005 2023-07-21T19:05:01.529 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:01 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/426397029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:01.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:01 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/426397029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:01.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:01 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/426397029' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:01.918 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:01 smithi005 systemd[1]: Starting Ceph mds.j for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:02.264 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:01 smithi005 podman[159330]: 2023-07-21T19:05:02.386 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.609+0000 7f6c39dff700 1 -- 172.21.15.5:0/1514749551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 msgr2=0x7f6c3410e4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.609+0000 7f6c39dff700 1 --2- 172.21.15.5:0/1514749551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c3410e4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00b0a0 tx=0x7f6c2c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 -- 172.21.15.5:0/1514749551 shutdown_connections 2023-07-21T19:05:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 --2- 172.21.15.5:0/1514749551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c3410e4b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 --2- 172.21.15.5:0/1514749551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 0x7f6c340729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 --2- 172.21.15.5:0/1514749551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 0x7f6c340733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 -- 172.21.15.5:0/1514749551 >> 172.21.15.5:0/1514749551 conn(0x7f6c340fb0e0 msgr2=0x7f6c340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 -- 172.21.15.5:0/1514749551 shutdown_connections 2023-07-21T19:05:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.610+0000 7f6c39dff700 1 -- 172.21.15.5:0/1514749551 wait complete. 2023-07-21T19:05:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 Processor -- start 2023-07-21T19:05:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 -- start start 2023-07-21T19:05:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 0x7f6c3419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 0x7f6c3419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a3990 con 0x7f6c3410c0d0 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a3ad0 con 0x7f6c34074550 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.611+0000 7f6c39dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c341a3c60 con 0x7f6c34072f70 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c32ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 0x7f6c3419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39654/0 (socket says 172.21.15.5:39654) 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 -- 172.21.15.5:0/36348896 learned_addr learned my addr 172.21.15.5:0/36348896 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c337fe700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 0x7f6c3419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 -- 172.21.15.5:0/36348896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 msgr2=0x7f6c3419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 0x7f6c3419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 -- 172.21.15.5:0/36348896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 msgr2=0x7f6c3419c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 0x7f6c3419c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 -- 172.21.15.5:0/36348896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c28009520 con 0x7f6c3410c0d0 2023-07-21T19:05:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.612+0000 7f6c33fff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00b780 tx=0x7f6c2c00bd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.613+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c00e040 con 0x7f6c3410c0d0 2023-07-21T19:05:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.613+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c2c004e60 con 0x7f6c3410c0d0 2023-07-21T19:05:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.613+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c2c009d00 con 0x7f6c3410c0d0 2023-07-21T19:05:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.613+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0052a0 con 0x7f6c3410c0d0 2023-07-21T19:05:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.613+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c341a41d0 con 0x7f6c3410c0d0 2023-07-21T19:05:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.614+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c34101c20 con 0x7f6c3410c0d0 2023-07-21T19:05:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.616+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6c2c01d470 con 0x7f6c3410c0d0 2023-07-21T19:05:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.617+0000 7f6c30ff9700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 0x7f6c1c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.618+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f6c2c0cdc20 con 0x7f6c3410c0d0 2023-07-21T19:05:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.618+0000 7f6c337fe700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 0x7f6c1c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.618+0000 7f6c337fe700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 0x7f6c1c0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005950 tx=0x7f6c240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.621+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6c2c097800 con 0x7f6c3410c0d0 2023-07-21T19:05:02.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:02 smithi005 ceph-mon[126460]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:02.668 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:02 smithi005 bash[159330]: 8974ab257df2cbe7ed34fca0051834c1cacf2e4a8f66c10ccee2115f707ceac8 2023-07-21T19:05:02.669 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:02 smithi005 ceph-mds[159370]: -- 172.21.15.5:0/2473262492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564368a931e0 con 0x564368a69000 2023-07-21T19:05:02.669 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-j[159346]: starting mds.j at 2023-07-21T19:05:02.669 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:05:02 smithi005 systemd[1]: Started Ceph mds.j for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:02 smithi156 ceph-mon[128316]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:02.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:02 smithi101 ceph-mon[128136]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:02.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.881+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c3402c850 con 0x7f6c3410c0d0 2023-07-21T19:05:02.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.881+0000 7f6c30ff9700 1 -- 172.21.15.5:0/36348896 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+2020 (secure 0 0 0) 0x7f6c2c02e350 con 0x7f6c3410c0d0 2023-07-21T19:05:02.892 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:02.892 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T19:05:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 msgr2=0x7f6c1c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 0x7f6c1c0a36d0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6c24005950 tx=0x7f6c240058e0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 msgr2=0x7f6c341a33c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c00b780 tx=0x7f6c2c00bd40 comp rx=0 tx=0).stop 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 shutdown_connections 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6c1c0a1220 0x7f6c1c0a36d0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c3410c0d0 0x7f6c341a33c0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c34072f70 0x7f6c3419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 --2- 172.21.15.5:0/36348896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c34074550 0x7f6c3419c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:02.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 >> 172.21.15.5:0/36348896 conn(0x7f6c340fb0e0 msgr2=0x7f6c340fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 shutdown_connections 2023-07-21T19:05:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:02.884+0000 7f6c39dff700 1 -- 172.21.15.5:0/36348896 wait complete. 2023-07-21T19:05:02.896 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 3 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:03.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: Deploying daemon mds.d on smithi005 2023-07-21T19:05:03.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/36348896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:03.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.5:6834/3927516205,v1:172.21.15.5:6835/3927516205] up:boot 2023-07-21T19:05:03.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: fsmap 3 up:standby 2023-07-21T19:05:03.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:03 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:03.879 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:03.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: Deploying daemon mds.d on smithi005 2023-07-21T19:05:03.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/36348896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:03.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.5:6834/3927516205,v1:172.21.15.5:6835/3927516205] up:boot 2023-07-21T19:05:03.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: fsmap 3 up:standby 2023-07-21T19:05:03.880 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:03 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:05:03.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: Deploying daemon mds.d on smithi005 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/36348896' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:03.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.5:6834/3927516205,v1:172.21.15.5:6835/3927516205] up:boot 2023-07-21T19:05:03.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: fsmap 3 up:standby 2023-07-21T19:05:03.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:03 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:05:04.823 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:04 smithi005 ceph-mon[126460]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:04.915 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:04.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:04 smithi156 ceph-mon[128316]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:05.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:04 smithi101 ceph-mon[128136]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 -- 172.21.15.5:0/2843127817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234101680 msgr2=0x7ff234107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/2843127817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234101680 0x7ff234107f40 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff22c00b0a0 tx=0x7ff22c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 -- 172.21.15.5:0/2843127817 shutdown_connections 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/2843127817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234101680 0x7ff234107f40 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/2843127817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 0x7ff2340fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/2843127817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234100cf0 0x7ff234101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.138+0000 7ff23a5ff700 1 -- 172.21.15.5:0/2843127817 >> 172.21.15.5:0/2843127817 conn(0x7ff2340fb0e0 msgr2=0x7ff2340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.139+0000 7ff23a5ff700 1 -- 172.21.15.5:0/2843127817 shutdown_connections 2023-07-21T19:05:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.139+0000 7ff23a5ff700 1 -- 172.21.15.5:0/2843127817 wait complete. 2023-07-21T19:05:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 Processor -- start 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 -- start start 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 0x7ff2340742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 0x7ff234072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2341996b0 con 0x7ff234100cf0 2023-07-21T19:05:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234073380 con 0x7ff234101680 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.140+0000 7ff23a5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff234073540 con 0x7ff2340ffb00 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff2337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff233fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 0x7ff2340742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 0x7ff234072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff2337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39678/0 (socket says 172.21.15.5:39678) 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff2337fe700 1 -- 172.21.15.5:0/4164251055 learned_addr learned my addr 172.21.15.5:0/4164251055 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 -- 172.21.15.5:0/4164251055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 msgr2=0x7ff2340742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 0x7ff2340742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 -- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 msgr2=0x7ff234072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 -- 172.21.15.5:0/4164251055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff22c009d00 con 0x7ff234101680 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff2337fe700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.141+0000 7ff238b9c700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 0x7ff234072e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009fd0 tx=0x7ff22c004c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.142+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c00e040 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.142+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2341a4060 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.142+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff22c0056e0 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.142+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff22c01da70 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.142+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2341a43d0 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.143+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff234107040 con 0x7ff234101680 2023-07-21T19:05:05.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.145+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff22c005850 con 0x7ff234101680 2023-07-21T19:05:05.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.146+0000 7ff2317fa700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 0x7ff21c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:05.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.147+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff22c0ce320 con 0x7ff234101680 2023-07-21T19:05:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.147+0000 7ff233fff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 0x7ff21c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.147+0000 7ff233fff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 0x7ff21c0a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff234100b50 tx=0x7ff22400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.148+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff22c097f00 con 0x7ff234101680 2023-07-21T19:05:05.168 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:04 smithi005 systemd[1]: Starting Ceph mds.d for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:05.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.416+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff23402c850 con 0x7ff234101680 2023-07-21T19:05:05.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.417+0000 7ff2317fa700 1 -- 172.21.15.5:0/4164251055 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2774 (secure 0 0 0) 0x7ff22c09b520 con 0x7ff234101680 2023-07-21T19:05:05.428 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:05.428 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T19:05:05.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 msgr2=0x7ff21c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:05.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 0x7ff21c0a3650 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7ff234100b50 tx=0x7ff22400b410 comp rx=0 tx=0).stop 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 msgr2=0x7ff234072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 0x7ff234072e40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff22c009fd0 tx=0x7ff22c004c80 comp rx=0 tx=0).stop 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 shutdown_connections 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff21c0a11a0 0x7ff21c0a3650 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff234101680 0x7ff234072e40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2340ffb00 0x7ff2340742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 --2- 172.21.15.5:0/4164251055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff234100cf0 0x7ff234072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:05.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 >> 172.21.15.5:0/4164251055 conn(0x7ff2340fb0e0 msgr2=0x7ff234104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:05.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 shutdown_connections 2023-07-21T19:05:05.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:05.420+0000 7ff23a5ff700 1 -- 172.21.15.5:0/4164251055 wait complete. 2023-07-21T19:05:05.434 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 4 2023-07-21T19:05:05.608 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:05 smithi005 podman[159749]: 2023-07-21T19:05:05.608 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:05 smithi005 bash[159749]: 5c88db6a67203df0ff137b4159fe50d644ee993c9e875349f559750f611ba091 2023-07-21T19:05:05.609 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:05 smithi005 ceph-mds[159808]: -- 172.21.15.5:0/2347931081 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5558fcc051e0 con 0x5558fcbdac00 2023-07-21T19:05:05.609 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-d[159782]: starting mds.d at 2023-07-21T19:05:05.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4164251055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:05.919 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:05:05 smithi005 systemd[1]: Started Ceph mds.d for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:05.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4164251055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:06.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4164251055' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:06.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.5:6836/3579918721,v1:172.21.15.5:6837/3579918721] up:boot 2023-07-21T19:05:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: fsmap 4 up:standby 2023-07-21T19:05:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:05:06.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:06.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:06 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.5:6836/3579918721,v1:172.21.15.5:6837/3579918721] up:boot 2023-07-21T19:05:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: fsmap 4 up:standby 2023-07-21T19:05:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:05:06.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:06.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:06 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:06.991 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:06.991 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.5:6836/3579918721,v1:172.21.15.5:6837/3579918721] up:boot 2023-07-21T19:05:06.991 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: fsmap 4 up:standby 2023-07-21T19:05:06.991 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:05:06.992 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.992 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.992 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:06.992 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:06.993 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:06.993 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:06 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:07.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 -- 172.21.15.5:0/3254808696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684072f70 msgr2=0x7ff6840733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 --2- 172.21.15.5:0/3254808696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684072f70 0x7ff6840733e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff674009990 tx=0x7ff674009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 -- 172.21.15.5:0/3254808696 shutdown_connections 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 --2- 172.21.15.5:0/3254808696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6841079c0 0x7ff684109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 --2- 172.21.15.5:0/3254808696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff684074550 0x7ff6840729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 --2- 172.21.15.5:0/3254808696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684072f70 0x7ff6840733e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 -- 172.21.15.5:0/3254808696 >> 172.21.15.5:0/3254808696 conn(0x7ff6840fb0e0 msgr2=0x7ff6840fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 -- 172.21.15.5:0/3254808696 shutdown_connections 2023-07-21T19:05:07.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.658+0000 7ff68adff700 1 -- 172.21.15.5:0/3254808696 wait complete. 2023-07-21T19:05:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.659+0000 7ff68adff700 1 Processor -- start 2023-07-21T19:05:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.659+0000 7ff68adff700 1 -- start start 2023-07-21T19:05:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff684072f70 0x7ff684197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:07.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6841079c0 0x7ff68419eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419f490 con 0x7ff684074550 2023-07-21T19:05:07.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419f5d0 con 0x7ff684072f70 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68419f760 con 0x7ff6841079c0 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48396/0 (socket says 172.21.15.5:48396) 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 -- 172.21.15.5:0/4060127136 learned_addr learned my addr 172.21.15.5:0/4060127136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff688b9b700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff684072f70 0x7ff684197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff68939c700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6841079c0 0x7ff68419eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 -- 172.21.15.5:0/4060127136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6841079c0 msgr2=0x7ff68419eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:07.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6841079c0 0x7ff68419eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 -- 172.21.15.5:0/4060127136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff684072f70 msgr2=0x7ff684197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff684072f70 0x7ff684197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.660+0000 7ff67bfff700 1 -- 172.21.15.5:0/4060127136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6740095f0 con 0x7ff684074550 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff67bfff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff6740095c0 tx=0x7ff674005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67401d070 con 0x7ff684074550 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff674003f80 con 0x7ff684074550 2023-07-21T19:05:07.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff68419f9e0 con 0x7ff684074550 2023-07-21T19:05:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff674022bb0 con 0x7ff684074550 2023-07-21T19:05:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.661+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68419fe00 con 0x7ff684074550 2023-07-21T19:05:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.662+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff684106070 con 0x7ff684074550 2023-07-21T19:05:07.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.664+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff674003af0 con 0x7ff684074550 2023-07-21T19:05:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.666+0000 7ff679ffb700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 0x7ff66c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.666+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff6740cdba0 con 0x7ff684074550 2023-07-21T19:05:07.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.667+0000 7ff688b9b700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 0x7ff66c0a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:07.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.667+0000 7ff688b9b700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 0x7ff66c0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff684074920 tx=0x7ff68000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:07.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.671+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff674097780 con 0x7ff684074550 2023-07-21T19:05:07.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:07 smithi005 ceph-mon[126460]: Deploying daemon mds.e on smithi101 2023-07-21T19:05:07.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:07 smithi156 ceph-mon[128316]: Deploying daemon mds.e on smithi101 2023-07-21T19:05:07.934 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:07 smithi101 ceph-mon[128136]: Deploying daemon mds.e on smithi101 2023-07-21T19:05:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.928+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff68410d400 con 0x7ff684074550 2023-07-21T19:05:07.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.929+0000 7ff679ffb700 1 -- 172.21.15.5:0/4060127136 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3528 (secure 0 0 0) 0x7ff67409ada0 con 0x7ff684074550 2023-07-21T19:05:07.940 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:07.941 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-07-21T19:05:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.931+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 msgr2=0x7ff66c0a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.931+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 0x7ff66c0a3650 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7ff684074920 tx=0x7ff68000a380 comp rx=0 tx=0).stop 2023-07-21T19:05:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.931+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 msgr2=0x7ff684197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:07.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.931+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff6740095c0 tx=0x7ff674005740 comp rx=0 tx=0).stop 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 shutdown_connections 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff66c0a11a0 0x7ff66c0a3650 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6841079c0 0x7ff68419eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff684072f70 0x7ff684197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 --2- 172.21.15.5:0/4060127136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff684074550 0x7ff684197e50 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 >> 172.21.15.5:0/4060127136 conn(0x7ff6840fb0e0 msgr2=0x7ff6840fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:07.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 shutdown_connections 2023-07-21T19:05:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:07.932+0000 7ff68adff700 1 -- 172.21.15.5:0/4060127136 wait complete. 2023-07-21T19:05:07.944 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 5 2023-07-21T19:05:08.341 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:07 smithi101 systemd[1]: Starting Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:08.663 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:08 smithi101 ceph-mon[128136]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:08.663 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4060127136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:08.664 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:08 smithi101 podman[153032]: 2023-07-21T19:05:08.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:08 smithi005 ceph-mon[126460]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:08.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4060127136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:08.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:08 smithi156 ceph-mon[128316]: pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:08.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4060127136' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:09.090 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:08 smithi101 bash[153032]: a56c2826a016be88930e15c775a56fd47c4a6f908be035494c26adfdc10aa661 2023-07-21T19:05:09.091 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:08 smithi101 ceph-mds[153072]: -- 172.21.15.101:0/3858977881 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555c0174f1e0 con 0x555c01725000 2023-07-21T19:05:09.091 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: starting mds.e at 2023-07-21T19:05:09.092 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:05:08 smithi101 systemd[1]: Started Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:09.961 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:10.029 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:10.030 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: Deploying daemon mds.f on smithi156 2023-07-21T19:05:10.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:10.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6834/153908041,v1:172.21.15.101:6835/153908041] up:boot 2023-07-21T19:05:10.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: fsmap 5 up:standby 2023-07-21T19:05:10.031 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:09 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:05:10.084 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:10.085 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: Deploying daemon mds.f on smithi156 2023-07-21T19:05:10.086 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:10.086 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6834/153908041,v1:172.21.15.101:6835/153908041] up:boot 2023-07-21T19:05:10.086 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: fsmap 5 up:standby 2023-07-21T19:05:10.087 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:09 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:05:10.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:10.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:10.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:10.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:10.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: Deploying daemon mds.f on smithi156 2023-07-21T19:05:10.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:10.094 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6834/153908041,v1:172.21.15.101:6835/153908041] up:boot 2023-07-21T19:05:10.094 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: fsmap 5 up:standby 2023-07-21T19:05:10.094 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:09 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:05:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 -- 172.21.15.5:0/2002114028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c100cf0 msgr2=0x7f4d6c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 --2- 172.21.15.5:0/2002114028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c101140 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4d5c009990 tx=0x7f4d5c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 -- 172.21.15.5:0/2002114028 shutdown_connections 2023-07-21T19:05:10.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 --2- 172.21.15.5:0/2002114028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c101680 0x7f4d6c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 --2- 172.21.15.5:0/2002114028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c0ffb00 0x7f4d6c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 --2- 172.21.15.5:0/2002114028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c101140 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 -- 172.21.15.5:0/2002114028 >> 172.21.15.5:0/2002114028 conn(0x7f4d6c0fb0e0 msgr2=0x7f4d6c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:10.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 -- 172.21.15.5:0/2002114028 shutdown_connections 2023-07-21T19:05:10.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.165+0000 7f4d739ff700 1 -- 172.21.15.5:0/2002114028 wait complete. 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.166+0000 7f4d739ff700 1 Processor -- start 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.166+0000 7f4d739ff700 1 -- start start 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c0ffb00 0x7f4d6c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:10.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c0735e0 con 0x7f4d6c101680 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c073750 con 0x7f4d6c0ffb00 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d739ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c19f780 con 0x7f4d6c100cf0 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48416/0 (socket says 172.21.15.5:48416) 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 -- 172.21.15.5:0/1999994067 learned_addr learned my addr 172.21.15.5:0/1999994067 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d7179b700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c0ffb00 0x7f4d6c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:10.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d70f9a700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 -- 172.21.15.5:0/1999994067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c100cf0 msgr2=0x7f4d6c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 -- 172.21.15.5:0/1999994067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c0ffb00 msgr2=0x7f4d6c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c0ffb00 0x7f4d6c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.167+0000 7f4d71f9c700 1 -- 172.21.15.5:0/1999994067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d5c0095f0 con 0x7f4d6c101680 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d71f9c700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d64007ee0 tx=0x7f4d6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:10.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d6400ddd0 con 0x7f4d6c101680 2023-07-21T19:05:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4d6400f040 con 0x7f4d6c101680 2023-07-21T19:05:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6c19fa60 con 0x7f4d6c101680 2023-07-21T19:05:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d640047d0 con 0x7f4d6c101680 2023-07-21T19:05:10.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.168+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d6c19ff70 con 0x7f4d6c101680 2023-07-21T19:05:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.169+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6c10b180 con 0x7f4d6c101680 2023-07-21T19:05:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.171+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4d64003a00 con 0x7f4d6c101680 2023-07-21T19:05:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.172+0000 7f4d627fc700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 0x7f4d580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:10.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.173+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4d640cbd60 con 0x7f4d6c101680 2023-07-21T19:05:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.173+0000 7f4d7179b700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 0x7f4d580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:10.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.173+0000 7f4d7179b700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 0x7f4d580a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c100b50 tx=0x7f4d68007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:10.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.176+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4d6404e790 con 0x7f4d6c101680 2023-07-21T19:05:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.434+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4d6c10b340 con 0x7f4d6c101680 2023-07-21T19:05:10.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.435+0000 7f4d627fc700 1 -- 172.21.15.5:0/1999994067 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4285 (secure 0 0 0) 0x7f4d640958b0 con 0x7f4d6c101680 2023-07-21T19:05:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:10.446 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 msgr2=0x7f4d580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 0x7f4d580a36d0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f4d6c100b50 tx=0x7f4d68007400 comp rx=0 tx=0).stop 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 msgr2=0x7f4d6c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f4d64007ee0 tx=0x7f4d6400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 shutdown_connections 2023-07-21T19:05:10.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4d580a1220 0x7f4d580a36d0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4d6c101680 0x7f4d6c072e40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4d6c0ffb00 0x7f4d6c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 --2- 172.21.15.5:0/1999994067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4d6c100cf0 0x7f4d6c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 >> 172.21.15.5:0/1999994067 conn(0x7f4d6c0fb0e0 msgr2=0x7f4d6c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:10.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 shutdown_connections 2023-07-21T19:05:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:10.438+0000 7f4d739ff700 1 -- 172.21.15.5:0/1999994067 wait complete. 2023-07-21T19:05:10.451 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 6 2023-07-21T19:05:11.051 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:10 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1999994067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:11.051 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 systemd[1]: Starting Ceph mds.f for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:11.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:10 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1999994067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:11.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:10 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1999994067' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:11.790 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 podman[154075]: 2023-07-21T19:05:12.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:11 smithi101 ceph-mon[128136]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:12.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:11 smithi005 ceph-mon[126460]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:12.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:11 smithi156 ceph-mon[128316]: pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:12.180 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 bash[154075]: 40c926eb52922f14f8b2a86d643d2249194a86862d463cf0c77dbf67b405028e 2023-07-21T19:05:12.180 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 ceph-mds[154115]: -- 172.21.15.156:0/3361383706 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5643919531e0 con 0x564391929000 2023-07-21T19:05:12.181 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-f[154091]: starting mds.f at 2023-07-21T19:05:12.181 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:05:11 smithi156 systemd[1]: Started Ceph mds.f for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:12.468 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/1635155089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 msgr2=0x7fc3f4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/1635155089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4101140 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009990 tx=0x7fc3dc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/1635155089 shutdown_connections 2023-07-21T19:05:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/1635155089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 0x7fc3f4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/1635155089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/1635155089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4101140 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/1635155089 >> 172.21.15.5:0/1635155089 conn(0x7fc3f40fb0e0 msgr2=0x7fc3f40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/1635155089 shutdown_connections 2023-07-21T19:05:12.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.673+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/1635155089 wait complete. 2023-07-21T19:05:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 Processor -- start 2023-07-21T19:05:12.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 -- start start 2023-07-21T19:05:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 0x7fc3f419cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:12.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419d2d0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419d410 con 0x7fc3f4101680 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.674+0000 7fc3fa7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419d5a0 con 0x7fc3f40ffb00 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3f3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58512/0 (socket says 172.21.15.5:58512) 2023-07-21T19:05:12.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3f3fff700 1 -- 172.21.15.5:0/3427120346 learned_addr learned my addr 172.21.15.5:0/3427120346 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48440/0 (socket says 172.21.15.5:48440) 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3f8d9c700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 0x7fc3f419cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 -- 172.21.15.5:0/3427120346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 msgr2=0x7fc3f4195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 -- 172.21.15.5:0/3427120346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 msgr2=0x7fc3f419cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 0x7fc3f419cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 -- 172.21.15.5:0/3427120346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0095f0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3f3fff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.675+0000 7fc3ebfff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.676+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc01d070 con 0x7fc3f4100cf0 2023-07-21T19:05:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.676+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3dc00bd50 con 0x7fc3f4100cf0 2023-07-21T19:05:12.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.676+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f419d820 con 0x7fc3f4100cf0 2023-07-21T19:05:12.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.676+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3dc022d30 con 0x7fc3f4100cf0 2023-07-21T19:05:12.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.676+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f419dca0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.677+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4107090 con 0x7fc3f4100cf0 2023-07-21T19:05:12.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.679+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc3dc022470 con 0x7fc3f4100cf0 2023-07-21T19:05:12.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.680+0000 7fc3f1ffb700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 0x7fc3e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:12.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.680+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fc3dc0cd9c0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.680+0000 7fc3f3fff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 0x7fc3e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:12.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.681+0000 7fc3f3fff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 0x7fc3e00a36d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4005950 tx=0x7fc3e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:12.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.685+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc3dc0975a0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.940+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3f402c850 con 0x7fc3f4100cf0 2023-07-21T19:05:12.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.940+0000 7fc3f1ffb700 1 -- 172.21.15.5:0/3427120346 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+5045 (secure 0 0 0) 0x7fc3dc09abc0 con 0x7fc3f4100cf0 2023-07-21T19:05:12.951 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:12.951 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T19:05:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 msgr2=0x7fc3e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 0x7fc3e00a36d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc3e4005950 tx=0x7fc3e4009260 comp rx=0 tx=0).stop 2023-07-21T19:05:12.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 msgr2=0x7fc3f4195c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:12.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc000c00 tx=0x7fc3dc004730 comp rx=0 tx=0).stop 2023-07-21T19:05:12.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 shutdown_connections 2023-07-21T19:05:12.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3e00a1220 0x7fc3e00a36d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc3f4101680 0x7fc3f419cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3f40ffb00 0x7fc3f4195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 --2- 172.21.15.5:0/3427120346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc3f4100cf0 0x7fc3f4195c90 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:12.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 >> 172.21.15.5:0/3427120346 conn(0x7fc3f40fb0e0 msgr2=0x7fc3f41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:12.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 shutdown_connections 2023-07-21T19:05:12.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:12.943+0000 7fc3fa7ff700 1 -- 172.21.15.5:0/3427120346 wait complete. 2023-07-21T19:05:12.956 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 7 2023-07-21T19:05:13.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.156:6834/2855288918,v1:172.21.15.156:6835/2855288918] up:boot 2023-07-21T19:05:13.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: fsmap 6 up:standby 2023-07-21T19:05:13.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:05:13.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:13.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:13.093 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:12 smithi101 ceph-mon[128136]: Deploying daemon mds.c on smithi156 2023-07-21T19:05:13.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.156:6834/2855288918,v1:172.21.15.156:6835/2855288918] up:boot 2023-07-21T19:05:13.100 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: fsmap 6 up:standby 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:13.101 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:13.102 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:12 smithi156 ceph-mon[128316]: Deploying daemon mds.c on smithi156 2023-07-21T19:05:13.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.156:6834/2855288918,v1:172.21.15.156:6835/2855288918] up:boot 2023-07-21T19:05:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: fsmap 6 up:standby 2023-07-21T19:05:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:05:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.169 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:13.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:13.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:13.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:13.170 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:12 smithi005 ceph-mon[126460]: Deploying daemon mds.c on smithi156 2023-07-21T19:05:14.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:13 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3427120346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:14.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:13 smithi101 ceph-mon[128136]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:14.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:13 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3427120346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:14.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:13 smithi156 ceph-mon[128316]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:14.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:13 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3427120346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:14.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:13 smithi005 ceph-mon[126460]: pgmap v284: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:14.415 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:14 smithi156 systemd[1]: Starting Ceph mds.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:14.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:15.157 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:14 smithi156 podman[154426]: 2023-07-21T19:05:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 -- 172.21.15.5:0/293396090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c0ff9f0 msgr2=0x7f0d5c0ffe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:15.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/293396090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0ffe00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0d50009990 tx=0x7f0d50009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 -- 172.21.15.5:0/293396090 shutdown_connections 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/293396090 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c101570 0x7f0d5c107e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/293396090 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0ffe00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/293396090 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c100be0 0x7f0d5c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 -- 172.21.15.5:0/293396090 >> 172.21.15.5:0/293396090 conn(0x7f0d5c0fb010 msgr2=0x7f0d5c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 -- 172.21.15.5:0/293396090 shutdown_connections 2023-07-21T19:05:15.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.175+0000 7f0d61f9d700 1 -- 172.21.15.5:0/293396090 wait complete. 2023-07-21T19:05:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.176+0000 7f0d61f9d700 1 Processor -- start 2023-07-21T19:05:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 -- start start 2023-07-21T19:05:15.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c100be0 0x7f0d5c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c0735e0 con 0x7f0d5c101570 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c073750 con 0x7f0d5c0ff9f0 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d61f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c19f510 con 0x7f0d5c100be0 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d6179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:15.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d60f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d6179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48464/0 (socket says 172.21.15.5:48464) 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d5bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c100be0 0x7f0d5c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.177+0000 7f0d6179c700 1 -- 172.21.15.5:0/772251548 learned_addr learned my addr 172.21.15.5:0/772251548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 -- 172.21.15.5:0/772251548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c100be0 msgr2=0x7f0d5c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c100be0 0x7f0d5c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 -- 172.21.15.5:0/772251548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c0ff9f0 msgr2=0x7f0d5c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 -- 172.21.15.5:0/772251548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d500095f0 con 0x7f0d5c101570 2023-07-21T19:05:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d6179c700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0d54007ee0 tx=0x7f0d5400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54017070 con 0x7f0d5c101570 2023-07-21T19:05:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0d5400f040 con 0x7f0d5c101570 2023-07-21T19:05:15.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c19f7f0 con 0x7f0d5c101570 2023-07-21T19:05:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d54005570 con 0x7f0d5c101570 2023-07-21T19:05:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.178+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c19fd00 con 0x7f0d5c101570 2023-07-21T19:05:15.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.179+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c10b0c0 con 0x7f0d5c101570 2023-07-21T19:05:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.181+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f0d5400db20 con 0x7f0d5c101570 2023-07-21T19:05:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.183+0000 7f0d59ffb700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 0x7f0d4c0a36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.183+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f0d540cbc40 con 0x7f0d5c101570 2023-07-21T19:05:15.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.183+0000 7f0d60f9b700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 0x7f0d4c0a36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:15.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.184+0000 7f0d60f9b700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 0x7f0d4c0a36f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005910 tx=0x7f0d5000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:15.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.187+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0d54095790 con 0x7f0d5c101570 2023-07-21T19:05:15.430 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:15 smithi156 bash[154426]: adcf8fc2b4eb18725ed6ca82c1fea681e61d0079310bfdd956501328fb231568 2023-07-21T19:05:15.431 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:15 smithi156 ceph-mds[154466]: -- 172.21.15.156:0/1896562239 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5616e4b711e0 con 0x5616e4b46c00 2023-07-21T19:05:15.431 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-c[154442]: starting mds.c at 2023-07-21T19:05:15.431 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:05:15 smithi156 systemd[1]: Started Ceph mds.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.447+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d5c10b250 con 0x7f0d5c101570 2023-07-21T19:05:15.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.448+0000 7f0d59ffb700 1 -- 172.21.15.5:0/772251548 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5805 (secure 0 0 0) 0x7f0d54098db0 con 0x7f0d5c101570 2023-07-21T19:05:15.459 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:15.459 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T19:05:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 msgr2=0x7f0d4c0a36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 0x7f0d4c0a36f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0d50005910 tx=0x7f0d5000b260 comp rx=0 tx=0).stop 2023-07-21T19:05:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 msgr2=0x7f0d5c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0d54007ee0 tx=0x7f0d5400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 shutdown_connections 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0d4c0a1240 0x7f0d4c0a36f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0d5c101570 0x7f0d5c072e40 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0d5c0ff9f0 0x7f0d5c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 --2- 172.21.15.5:0/772251548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0d5c100be0 0x7f0d5c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 >> 172.21.15.5:0/772251548 conn(0x7f0d5c0fb010 msgr2=0x7f0d5c0fd1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 shutdown_connections 2023-07-21T19:05:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:15.450+0000 7f0d61f9d700 1 -- 172.21.15.5:0/772251548 wait complete. 2023-07-21T19:05:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-07-21T19:05:16.497 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.156:6836/1947602937,v1:172.21.15.156:6837/1947602937] up:boot 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: fsmap 7 up:standby 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.498 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:16.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:16.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:16.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: Deploying daemon mds.h on smithi101 2023-07-21T19:05:16.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:16.499 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:16 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/772251548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:16.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.156:6836/1947602937,v1:172.21.15.156:6837/1947602937] up:boot 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: fsmap 7 up:standby 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:16.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: Deploying daemon mds.h on smithi101 2023-07-21T19:05:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:16.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:16 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/772251548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.156:6836/1947602937,v1:172.21.15.156:6837/1947602937] up:boot 2023-07-21T19:05:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: fsmap 7 up:standby 2023-07-21T19:05:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:05:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: Deploying daemon mds.h on smithi101 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:16.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:16 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/772251548' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:17.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:17.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 -- 172.21.15.5:0/4277288654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb70100cf0 msgr2=0x7fcb70101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 --2- 172.21.15.5:0/4277288654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb70100cf0 0x7fcb70101140 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcb64009990 tx=0x7fcb64009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 -- 172.21.15.5:0/4277288654 shutdown_connections 2023-07-21T19:05:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 --2- 172.21.15.5:0/4277288654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70101680 0x7fcb70107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 --2- 172.21.15.5:0/4277288654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb700ffb00 0x7fcb700fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 --2- 172.21.15.5:0/4277288654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb70100cf0 0x7fcb70101140 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 -- 172.21.15.5:0/4277288654 >> 172.21.15.5:0/4277288654 conn(0x7fcb700fb0e0 msgr2=0x7fcb700fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 -- 172.21.15.5:0/4277288654 shutdown_connections 2023-07-21T19:05:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.686+0000 7fcb75dff700 1 -- 172.21.15.5:0/4277288654 wait complete. 2023-07-21T19:05:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.687+0000 7fcb75dff700 1 Processor -- start 2023-07-21T19:05:17.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.687+0000 7fcb75dff700 1 -- start start 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 0x7fcb70072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70101680 0x7fcb70072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb700735e0 con 0x7fcb700ffb00 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb70073750 con 0x7fcb70100cf0 2023-07-21T19:05:17.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb75dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb7019f7b0 con 0x7fcb70101680 2023-07-21T19:05:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45816/0 (socket says 172.21.15.5:45816) 2023-07-21T19:05:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6f7fe700 1 -- 172.21.15.5:0/2657710186 learned_addr learned my addr 172.21.15.5:0/2657710186 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6effd700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 0x7fcb70072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6ffff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70101680 0x7fcb70072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6effd700 1 -- 172.21.15.5:0/2657710186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70101680 msgr2=0x7fcb70072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6effd700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70101680 0x7fcb70072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.688+0000 7fcb6effd700 1 -- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 msgr2=0x7fcb700742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6effd700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6effd700 1 -- 172.21.15.5:0/2657710186 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb640095f0 con 0x7fcb70100cf0 2023-07-21T19:05:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6f7fe700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6effd700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 0x7fcb70072900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098e0 tx=0x7fcb64005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb6401d070 con 0x7fcb70100cf0 2023-07-21T19:05:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb6400bd50 con 0x7fcb70100cf0 2023-07-21T19:05:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb64022cd0 con 0x7fcb70100cf0 2023-07-21T19:05:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb7019fa30 con 0x7fcb70100cf0 2023-07-21T19:05:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.689+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb7019fe00 con 0x7fcb70100cf0 2023-07-21T19:05:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.690+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb70107090 con 0x7fcb70100cf0 2023-07-21T19:05:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.692+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcb64022470 con 0x7fcb70100cf0 2023-07-21T19:05:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.694+0000 7fcb6cff9700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 0x7fcb580a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.694+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcb640cdba0 con 0x7fcb70100cf0 2023-07-21T19:05:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.694+0000 7fcb6f7fe700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 0x7fcb580a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.695+0000 7fcb6f7fe700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 0x7fcb580a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb60005950 tx=0x7fcb60009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.696+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcb64097780 con 0x7fcb70100cf0 2023-07-21T19:05:17.841 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:17 smithi101 systemd[1]: Starting Ceph mds.h for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.948+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb7002c850 con 0x7fcb70100cf0 2023-07-21T19:05:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.949+0000 7fcb6cff9700 1 -- 172.21.15.5:0/2657710186 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5805 (secure 0 0 0) 0x7fcb6409ada0 con 0x7fcb70100cf0 2023-07-21T19:05:17.960 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:17.960 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-07-21T19:05:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 msgr2=0x7fcb580a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 0x7fcb580a36d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fcb60005950 tx=0x7fcb60009260 comp rx=0 tx=0).stop 2023-07-21T19:05:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 msgr2=0x7fcb70072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 0x7fcb70072900 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcb640098e0 tx=0x7fcb64005740 comp rx=0 tx=0).stop 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 shutdown_connections 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb580a1220 0x7fcb580a36d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb70101680 0x7fcb70072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb700ffb00 0x7fcb700742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 --2- 172.21.15.5:0/2657710186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb70100cf0 0x7fcb70072900 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.951+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 >> 172.21.15.5:0/2657710186 conn(0x7fcb700fb0e0 msgr2=0x7fcb701041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.952+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 shutdown_connections 2023-07-21T19:05:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:17.952+0000 7fcb75dff700 1 -- 172.21.15.5:0/2657710186 wait complete. 2023-07-21T19:05:17.964 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 8 2023-07-21T19:05:18.213 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:17 smithi101 podman[153371]: 2023-07-21T19:05:18.213 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:18 smithi101 bash[153371]: 713946f821d879343647a95b57526e06fa9dc1e089fb225c9cbbb84f97288fb8 2023-07-21T19:05:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:18 smithi101 ceph-mon[128136]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:18 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2657710186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:18.591 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:18 smithi101 ceph-mds[153411]: -- 172.21.15.101:0/737021946 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556d952111e0 con 0x556d951e6c00 2023-07-21T19:05:18.591 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-h[153387]: starting mds.h at 2023-07-21T19:05:18.592 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:05:18 smithi101 systemd[1]: Started Ceph mds.h for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:18 smithi005 ceph-mon[126460]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:18.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:18 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2657710186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:18 smithi156 ceph-mon[128316]: pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:18 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2657710186' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:19.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:boot 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: fsmap 8 up:standby 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:19.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:19.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:19.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:19 smithi101 ceph-mon[128136]: Deploying daemon mds.i on smithi156 2023-07-21T19:05:19.632 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:boot 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: fsmap 8 up:standby 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.633 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:19.634 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:19.634 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:19.634 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:19 smithi156 ceph-mon[128316]: Deploying daemon mds.i on smithi156 2023-07-21T19:05:19.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:boot 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: fsmap 8 up:standby 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:19.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:19.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:19 smithi005 ceph-mon[126460]: Deploying daemon mds.i on smithi156 2023-07-21T19:05:19.982 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:20.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 -- 172.21.15.5:0/2411735685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 msgr2=0x7f49f80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/2411735685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f80fff10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009990 tx=0x7f49e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 -- 172.21.15.5:0/2411735685 shutdown_connections 2023-07-21T19:05:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/2411735685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8101680 0x7f49f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/2411735685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f80fff10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/2411735685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8100cf0 0x7f49f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 -- 172.21.15.5:0/2411735685 >> 172.21.15.5:0/2411735685 conn(0x7f49f80fb0e0 msgr2=0x7f49f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 -- 172.21.15.5:0/2411735685 shutdown_connections 2023-07-21T19:05:20.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.186+0000 7f49fe7ff700 1 -- 172.21.15.5:0/2411735685 wait complete. 2023-07-21T19:05:20.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.187+0000 7f49fe7ff700 1 Processor -- start 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.187+0000 7f49fe7ff700 1 -- start start 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8100cf0 0x7f49f8197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8101680 0x7f49f819ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f819f4e0 con 0x7f49f80ffb00 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f819f620 con 0x7f49f8101680 2023-07-21T19:05:20.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fe7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f819f7b0 con 0x7f49f8100cf0 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45828/0 (socket says 172.21.15.5:45828) 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 -- 172.21.15.5:0/3726334368 learned_addr learned my addr 172.21.15.5:0/3726334368 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f77fe700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8100cf0 0x7f49f8197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49fcd9c700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8101680 0x7f49f819ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 -- 172.21.15.5:0/3726334368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8100cf0 msgr2=0x7f49f8197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:20.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8100cf0 0x7f49f8197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 -- 172.21.15.5:0/3726334368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8101680 msgr2=0x7f49f819ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8101680 0x7f49f819ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.188+0000 7f49f7fff700 1 -- 172.21.15.5:0/3726334368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80095f0 con 0x7f49f80ffb00 2023-07-21T19:05:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49f7fff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b5c0 tx=0x7f49e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:20.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e801d070 con 0x7f49f80ffb00 2023-07-21T19:05:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f819fa30 con 0x7f49f80ffb00 2023-07-21T19:05:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49e8003ec0 con 0x7f49f80ffb00 2023-07-21T19:05:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49e8022be0 con 0x7f49f80ffb00 2023-07-21T19:05:20.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.189+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f819fe60 con 0x7f49f80ffb00 2023-07-21T19:05:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.190+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8192090 con 0x7f49f80ffb00 2023-07-21T19:05:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.192+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f49e8004030 con 0x7f49f80ffb00 2023-07-21T19:05:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.193+0000 7f49f57fa700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 0x7f49e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:20.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.193+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f49e80cdb80 con 0x7f49f80ffb00 2023-07-21T19:05:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.194+0000 7f49f77fe700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 0x7f49e00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:20.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.194+0000 7f49f77fe700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 0x7f49e00a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005950 tx=0x7f49ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:20.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.197+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f49e8097760 con 0x7f49f80ffb00 2023-07-21T19:05:20.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:20 smithi005 ceph-mon[126460]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:20.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.451+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49f802c850 con 0x7f49f80ffb00 2023-07-21T19:05:20.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.452+0000 7f49f57fa700 1 -- 172.21.15.5:0/3726334368 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6565 (secure 0 0 0) 0x7f49e809ad80 con 0x7f49f80ffb00 2023-07-21T19:05:20.463 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:20.463 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T19:05:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.454+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 msgr2=0x7f49e00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:20.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.454+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 0x7f49e00a3650 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49ec005950 tx=0x7f49ec00b410 comp rx=0 tx=0).stop 2023-07-21T19:05:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.454+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 msgr2=0x7f49f8197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:20.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.454+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f49e800b5c0 tx=0x7f49e8005740 comp rx=0 tx=0).stop 2023-07-21T19:05:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 shutdown_connections 2023-07-21T19:05:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49e00a11a0 0x7f49e00a3650 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49f8101680 0x7f49f819ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49f80ffb00 0x7f49f8197960 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 --2- 172.21.15.5:0/3726334368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49f8100cf0 0x7f49f8197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:20.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 >> 172.21.15.5:0/3726334368 conn(0x7f49f80fb0e0 msgr2=0x7f49f81065c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 shutdown_connections 2023-07-21T19:05:20.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:20.455+0000 7f49fe7ff700 1 -- 172.21.15.5:0/3726334368 wait complete. 2023-07-21T19:05:20.470 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 9 2023-07-21T19:05:20.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:20 smithi101 ceph-mon[128136]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:20.591 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:20 smithi156 ceph-mon[128316]: pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:20.930 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:20 smithi156 systemd[1]: Starting Ceph mds.i for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:21.346 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:21 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3726334368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:21.347 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:21 smithi156 podman[154769]: 2023-07-21T19:05:21.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:21 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3726334368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:21.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:21 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3726334368' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:21.680 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:21 smithi156 bash[154769]: 0bc95f98f26523503fb05a7cc3bd89536f8f79ffbccb3b08a5406a1896e3efbf 2023-07-21T19:05:21.680 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:21 smithi156 ceph-mds[154809]: -- 172.21.15.156:0/233146740 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55621764d1e0 con 0x556217622c00 2023-07-21T19:05:21.680 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-i[154785]: starting mds.i at 2023-07-21T19:05:21.680 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:05:21 smithi156 systemd[1]: Started Ceph mds.i for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:22.485 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:22.702 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:22.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.703 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: Deploying daemon mds.g on smithi005 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:boot 2023-07-21T19:05:22.704 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: fsmap 9 up:standby 2023-07-21T19:05:22.705 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:22 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:05:22.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 -- 172.21.15.5:0/2269565670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0ffbc0 msgr2=0x7fbf3c100010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:22.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 --2- 172.21.15.5:0/2269565670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100010 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009990 tx=0x7fbf30009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 -- 172.21.15.5:0/2269565670 shutdown_connections 2023-07-21T19:05:22.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 --2- 172.21.15.5:0/2269565670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c100550 0x7fbf3c106e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 --2- 172.21.15.5:0/2269565670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c0fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 --2- 172.21.15.5:0/2269565670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100010 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 -- 172.21.15.5:0/2269565670 >> 172.21.15.5:0/2269565670 conn(0x7fbf3c0f9fd0 msgr2=0x7fbf3c0fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:22.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 -- 172.21.15.5:0/2269565670 shutdown_connections 2023-07-21T19:05:22.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.706+0000 7fbf413ff700 1 -- 172.21.15.5:0/2269565670 wait complete. 2023-07-21T19:05:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.707+0000 7fbf413ff700 1 Processor -- start 2023-07-21T19:05:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.707+0000 7fbf413ff700 1 -- start start 2023-07-21T19:05:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:22.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c100550 0x7fbf3c1014e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c101c80 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c101df0 con 0x7fbf3c100550 2023-07-21T19:05:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf413ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf3c19f7a0 con 0x7fbf3c0ffbc0 2023-07-21T19:05:22.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45846/0 (socket says 172.21.15.5:45846) 2023-07-21T19:05:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 -- 172.21.15.5:0/1647298710 learned_addr learned my addr 172.21.15.5:0/1647298710 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3a7fc700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3b7fe700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c100550 0x7fbf3c1014e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:22.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 -- 172.21.15.5:0/1647298710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c0ffbc0 msgr2=0x7fbf3c100fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 -- 172.21.15.5:0/1647298710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c100550 msgr2=0x7fbf3c1014e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c100550 0x7fbf3c1014e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.708+0000 7fbf3affd700 1 -- 172.21.15.5:0/1647298710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf300095f0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf3affd700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00e970 tx=0x7fbf2c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:22.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c009790 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf2c0044b0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c003dd0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf3c19fa80 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.709+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf3c19ff40 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.711+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3c10a0a0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.712+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fbf2c003f30 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.714+0000 7fbf23fff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 0x7fbf240a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.714+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fbf2c014070 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.717+0000 7fbf3a7fc700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 0x7fbf240a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:22.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.717+0000 7fbf3a7fc700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 0x7fbf240a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006010 tx=0x7fbf3000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:22.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.719+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf2c0963a0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:22.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: Deploying daemon mds.g on smithi005 2023-07-21T19:05:22.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:boot 2023-07-21T19:05:22.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: fsmap 9 up:standby 2023-07-21T19:05:22.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:22 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:05:22.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:22.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: Deploying daemon mds.g on smithi005 2023-07-21T19:05:22.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:boot 2023-07-21T19:05:22.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: fsmap 9 up:standby 2023-07-21T19:05:22.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:22 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:05:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.977+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf3c10a260 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.978+0000 7fbf23fff700 1 -- 172.21.15.5:0/1647298710 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7327 (secure 0 0 0) 0x7fbf2c0999c0 con 0x7fbf3c0fe9d0 2023-07-21T19:05:22.989 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:22.989 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T19:05:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 msgr2=0x7fbf240a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 0x7fbf240a3650 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fbf30006010 tx=0x7fbf3000b260 comp rx=0 tx=0).stop 2023-07-21T19:05:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 msgr2=0x7fbf3c102960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:22.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c00e970 tx=0x7fbf2c00ed30 comp rx=0 tx=0).stop 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 shutdown_connections 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fbf240a11a0 0x7fbf240a3650 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf3c100550 0x7fbf3c1014e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf3c0fe9d0 0x7fbf3c102960 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.980+0000 7fbf413ff700 1 --2- 172.21.15.5:0/1647298710 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf3c0ffbc0 0x7fbf3c100fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:22.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.981+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 >> 172.21.15.5:0/1647298710 conn(0x7fbf3c0f9fd0 msgr2=0x7fbf3c067670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.981+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 shutdown_connections 2023-07-21T19:05:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:22.981+0000 7fbf413ff700 1 -- 172.21.15.5:0/1647298710 wait complete. 2023-07-21T19:05:22.993 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 10 2023-07-21T19:05:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:23 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1647298710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:23.669 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:23 smithi005 systemd[1]: Starting Ceph mds.g for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:23.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:23 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1647298710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:23.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:23 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1647298710' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:24.382 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:24 smithi005 podman[160424]: 2023-07-21T19:05:24.383 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:24 smithi005 bash[160424]: 63251037eedba835024ce4cee6061f1de59dd3e91d9f0c816584fb01a362b4b8 2023-07-21T19:05:24.383 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:24 smithi005 ceph-mds[160464]: -- 172.21.15.5:0/4235903178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56364c5b51e0 con 0x56364c58a800 2023-07-21T19:05:24.383 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-g[160440]: starting mds.g at 2023-07-21T19:05:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:24 smithi005 ceph-mon[126460]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:24.669 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:05:24 smithi005 systemd[1]: Started Ceph mds.g for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:24.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:24 smithi101 ceph-mon[128136]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:24.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:24 smithi156 ceph-mon[128316]: pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:25.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.224+0000 7ff1529ff700 1 -- 172.21.15.5:0/456330978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c072f70 msgr2=0x7ff14c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:25.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.224+0000 7ff1529ff700 1 --2- 172.21.15.5:0/456330978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c072f70 0x7ff14c0733e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff140009990 tx=0x7ff140009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 -- 172.21.15.5:0/456330978 shutdown_connections 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 --2- 172.21.15.5:0/456330978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c10c0d0 0x7ff14c10e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 --2- 172.21.15.5:0/456330978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c074550 0x7ff14c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 --2- 172.21.15.5:0/456330978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c072f70 0x7ff14c0733e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 -- 172.21.15.5:0/456330978 >> 172.21.15.5:0/456330978 conn(0x7ff14c0fb0e0 msgr2=0x7ff14c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 -- 172.21.15.5:0/456330978 shutdown_connections 2023-07-21T19:05:25.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.225+0000 7ff1529ff700 1 -- 172.21.15.5:0/456330978 wait complete. 2023-07-21T19:05:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 Processor -- start 2023-07-21T19:05:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 -- start start 2023-07-21T19:05:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c072f70 0x7ff14c19bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:25.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c074550 0x7ff14c19c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a3940 con 0x7ff14c10c0d0 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a3a80 con 0x7ff14c072f70 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff1529ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14c1a3c10 con 0x7ff14c074550 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.226+0000 7ff150f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45876/0 (socket says 172.21.15.5:45876) 2023-07-21T19:05:25.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 -- 172.21.15.5:0/269794140 learned_addr learned my addr 172.21.15.5:0/269794140 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff14b7fe700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c074550 0x7ff14c19c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff14bfff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c072f70 0x7ff14c19bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 -- 172.21.15.5:0/269794140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c074550 msgr2=0x7ff14c19c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c074550 0x7ff14c19c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 -- 172.21.15.5:0/269794140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c072f70 msgr2=0x7ff14c19bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c072f70 0x7ff14c19bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 -- 172.21.15.5:0/269794140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1400095f0 con 0x7ff14c10c0d0 2023-07-21T19:05:25.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff150f9c700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff144007ee0 tx=0x7ff14400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144017070 con 0x7ff14c10c0d0 2023-07-21T19:05:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff14400f040 con 0x7ff14c10c0d0 2023-07-21T19:05:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14c1a3ef0 con 0x7ff14c10c0d0 2023-07-21T19:05:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.227+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff144005570 con 0x7ff14c10c0d0 2023-07-21T19:05:25.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.228+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14c1a4370 con 0x7ff14c10c0d0 2023-07-21T19:05:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.229+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14c101c20 con 0x7ff14c10c0d0 2023-07-21T19:05:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.231+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff14400db20 con 0x7ff14c10c0d0 2023-07-21T19:05:25.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.232+0000 7ff1497fa700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 0x7ff1340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.232+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7ff1440cbc20 con 0x7ff14c10c0d0 2023-07-21T19:05:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.232+0000 7ff14bfff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 0x7ff1340a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:25.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.233+0000 7ff14bfff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 0x7ff1340a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff14c074920 tx=0x7ff13c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:25.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.236+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff144095770 con 0x7ff14c10c0d0 2023-07-21T19:05:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.499+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff14c02c850 con 0x7ff14c10c0d0 2023-07-21T19:05:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.500+0000 7ff1497fa700 1 -- 172.21.15.5:0/269794140 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7ff144098d90 con 0x7ff14c10c0d0 2023-07-21T19:05:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:25.511 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 msgr2=0x7ff1340a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 0x7ff1340a3650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff14c074920 tx=0x7ff13c009260 comp rx=0 tx=0).stop 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 msgr2=0x7ff14c1a3370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff144007ee0 tx=0x7ff14400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 shutdown_connections 2023-07-21T19:05:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff1340a11a0 0x7ff1340a3650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff14c10c0d0 0x7ff14c1a3370 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff14c072f70 0x7ff14c19bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 --2- 172.21.15.5:0/269794140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff14c074550 0x7ff14c19c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 >> 172.21.15.5:0/269794140 conn(0x7ff14c0fb0e0 msgr2=0x7ff14c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 shutdown_connections 2023-07-21T19:05:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:25.502+0000 7ff1529ff700 1 -- 172.21.15.5:0/269794140 wait complete. 2023-07-21T19:05:25.515 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2023-07-21T19:05:25.709 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.5:6838/666617424,v1:172.21.15.5:6839/666617424] up:boot 2023-07-21T19:05:25.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: fsmap 10 up:standby 2023-07-21T19:05:25.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:05:25.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.710 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:25.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:25.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:25.711 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:25 smithi005 ceph-mon[126460]: Deploying daemon mds.a on smithi005 2023-07-21T19:05:25.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.5:6838/666617424,v1:172.21.15.5:6839/666617424] up:boot 2023-07-21T19:05:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: fsmap 10 up:standby 2023-07-21T19:05:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:05:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:25.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:25.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:25.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:25 smithi101 ceph-mon[128136]: Deploying daemon mds.a on smithi005 2023-07-21T19:05:25.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.5:6838/666617424,v1:172.21.15.5:6839/666617424] up:boot 2023-07-21T19:05:25.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: fsmap 10 up:standby 2023-07-21T19:05:25.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:05:25.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:25.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:25 smithi156 ceph-mon[128316]: Deploying daemon mds.a on smithi005 2023-07-21T19:05:26.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:26 smithi005 ceph-mon[126460]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:26.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:26 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/269794140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:26.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:26 smithi101 ceph-mon[128136]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:26.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:26 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/269794140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:26.918 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:26 smithi005 systemd[1]: Starting Ceph mds.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:26.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:26 smithi156 ceph-mon[128316]: pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:26.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:26 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/269794140' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:27.417 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:27 smithi005 podman[160811]: 2023-07-21T19:05:27.417 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:27 smithi005 bash[160811]: 60f978f4fac58602e3cd65e2414fde11f6154edd0269776d26578e6079008057 2023-07-21T19:05:27.417 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:27 smithi005 ceph-mds[160851]: -- 172.21.15.5:0/1114749448 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55b55371d1e0 con 0x55b5536f2800 2023-07-21T19:05:27.417 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-a[160827]: starting mds.a at 2023-07-21T19:05:27.535 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:27.668 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:05:27 smithi005 systemd[1]: Started Ceph mds.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:27.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 -- 172.21.15.5:0/523194438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23380ffb00 msgr2=0x7f23380fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:27.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 --2- 172.21.15.5:0/523194438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23380ffb00 0x7f23380fff10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f232c00b0a0 tx=0x7f232c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 -- 172.21.15.5:0/523194438 shutdown_connections 2023-07-21T19:05:27.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 --2- 172.21.15.5:0/523194438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 0x7f2338107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 --2- 172.21.15.5:0/523194438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23380ffb00 0x7f23380fff10 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 --2- 172.21.15.5:0/523194438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2338100cf0 0x7f2338101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 -- 172.21.15.5:0/523194438 >> 172.21.15.5:0/523194438 conn(0x7f23380fb0e0 msgr2=0x7f23380fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:27.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 -- 172.21.15.5:0/523194438 shutdown_connections 2023-07-21T19:05:27.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.740+0000 7f233f1ff700 1 -- 172.21.15.5:0/523194438 wait complete. 2023-07-21T19:05:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 Processor -- start 2023-07-21T19:05:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 -- start start 2023-07-21T19:05:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23380ffb00 0x7f2338195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:27.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 0x7f233819cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819d4c0 con 0x7f2338100cf0 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819d600 con 0x7f2338101680 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.741+0000 7f233f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f233819d790 con 0x7f23380ffb00 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49556/0 (socket says 172.21.15.5:49556) 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 -- 172.21.15.5:0/2351905706 learned_addr learned my addr 172.21.15.5:0/2351905706 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:27.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f233cf9b700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23380ffb00 0x7f2338195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f233d79c700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 0x7f233819cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 -- 172.21.15.5:0/2351905706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23380ffb00 msgr2=0x7f2338195940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23380ffb00 0x7f2338195940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 -- 172.21.15.5:0/2351905706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 msgr2=0x7f233819cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 0x7f233819cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:27.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 -- 172.21.15.5:0/2351905706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f232c009d00 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.742+0000 7f2337fff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232400d950 tx=0x7f232400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.743+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400f890 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.743+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f232400fe80 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.743+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232400e3b0 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.743+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233819da70 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.743+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233819df80 con 0x7f2338100cf0 2023-07-21T19:05:27.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.744+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23381070e0 con 0x7f2338100cf0 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.746+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2324010460 con 0x7f2338100cf0 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.747+0000 7f2335ffb700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 0x7f23280a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.748+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f23240cc210 con 0x7f2338100cf0 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.748+0000 7f233cf9b700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 0x7f23280a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.748+0000 7f233cf9b700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 0x7f23280a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f232c0062a0 tx=0x7f232c006210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:27.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:27.750+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2324095d60 con 0x7f2338100cf0 2023-07-21T19:05:28.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.005+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f233802c850 con 0x7f2338100cf0 2023-07-21T19:05:28.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.005+0000 7f2335ffb700 1 -- 172.21.15.5:0/2351905706 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) v1 ==== 94+0+8079 (secure 0 0 0) 0x7f2324099380 con 0x7f2338100cf0 2023-07-21T19:05:28.016 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:28.016 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T19:05:28.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 msgr2=0x7f23280a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 0x7f23280a36d0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f232c0062a0 tx=0x7f232c006210 comp rx=0 tx=0).stop 2023-07-21T19:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 msgr2=0x7f2338195e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f232400d950 tx=0x7f232400dc60 comp rx=0 tx=0).stop 2023-07-21T19:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 shutdown_connections 2023-07-21T19:05:28.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23280a1220 0x7f23280a36d0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2338101680 0x7f233819cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23380ffb00 0x7f2338195940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 --2- 172.21.15.5:0/2351905706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2338100cf0 0x7f2338195e80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 >> 172.21.15.5:0/2351905706 conn(0x7f23380fb0e0 msgr2=0x7f2338104220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 shutdown_connections 2023-07-21T19:05:28.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:28.008+0000 7f233f1ff700 1 -- 172.21.15.5:0/2351905706 wait complete. 2023-07-21T19:05:28.021 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 11 2023-07-21T19:05:28.796 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:28.796 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.796 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.796 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.797 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:28.797 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:28.797 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:28.797 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: Deploying daemon mds.b on smithi101 2023-07-21T19:05:28.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2351905706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:28.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.5:6840/2768414598,v1:172.21.15.5:6841/2768414598] up:boot 2023-07-21T19:05:28.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: fsmap 11 up:standby 2023-07-21T19:05:28.798 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:28 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:05:28.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:28.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:28.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:28.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:28.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: Deploying daemon mds.b on smithi101 2023-07-21T19:05:28.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2351905706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:28.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.5:6840/2768414598,v1:172.21.15.5:6841/2768414598] up:boot 2023-07-21T19:05:28.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: fsmap 11 up:standby 2023-07-21T19:05:28.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:28 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:05:28.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:28.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: Deploying daemon mds.b on smithi101 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2351905706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.5:6840/2768414598,v1:172.21.15.5:6841/2768414598] up:boot 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: fsmap 11 up:standby 2023-07-21T19:05:28.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:28 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:05:30.039 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:30.091 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:29 smithi101 systemd[1]: Starting Ceph mds.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:05:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 -- 172.21.15.5:0/2191453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050074550 msgr2=0x7fa0500729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:30.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 --2- 172.21.15.5:0/2191453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050074550 0x7fa0500729a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa044009990 tx=0x7fa044009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 -- 172.21.15.5:0/2191453920 shutdown_connections 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 --2- 172.21.15.5:0/2191453920 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa0501079c0 0x7fa050109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 --2- 172.21.15.5:0/2191453920 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa050074550 0x7fa0500729a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 --2- 172.21.15.5:0/2191453920 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050072f70 0x7fa0500733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 -- 172.21.15.5:0/2191453920 >> 172.21.15.5:0/2191453920 conn(0x7fa0500fb0e0 msgr2=0x7fa0500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 -- 172.21.15.5:0/2191453920 shutdown_connections 2023-07-21T19:05:30.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.243+0000 7fa056dff700 1 -- 172.21.15.5:0/2191453920 wait complete. 2023-07-21T19:05:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.244+0000 7fa056dff700 1 Processor -- start 2023-07-21T19:05:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.244+0000 7fa056dff700 1 -- start start 2023-07-21T19:05:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 0x7fa050197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:30.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:30.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05019f4e0 con 0x7fa0501079c0 2023-07-21T19:05:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05019f620 con 0x7fa050074550 2023-07-21T19:05:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa056dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa05019f7b0 con 0x7fa050072f70 2023-07-21T19:05:30.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa05539c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa054b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 0x7fa050197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa05539c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49568/0 (socket says 172.21.15.5:49568) 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35914/0 (socket says 172.21.15.5:35914) 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa05539c700 1 -- 172.21.15.5:0/4215700965 learned_addr learned my addr 172.21.15.5:0/4215700965 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa054b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 0x7fa050197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49082/0 (socket says 172.21.15.5:49082) 2023-07-21T19:05:30.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 -- 172.21.15.5:0/4215700965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 msgr2=0x7fa050197960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 0x7fa050197960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 -- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 msgr2=0x7fa05019ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa04ffff700 1 -- 172.21.15.5:0/4215700965 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0440095f0 con 0x7fa050074550 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.245+0000 7fa05539c700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa04ffff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9c0 tx=0x7fa03c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c004220 con 0x7fa050074550 2023-07-21T19:05:30.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa05019fa90 con 0x7fa050074550 2023-07-21T19:05:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa03c004380 con 0x7fa050074550 2023-07-21T19:05:30.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa03c003b40 con 0x7fa050074550 2023-07-21T19:05:30.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.246+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa05019fff0 con 0x7fa050074550 2023-07-21T19:05:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.247+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa050106070 con 0x7fa050074550 2023-07-21T19:05:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.249+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa03c003ca0 con 0x7fa050074550 2023-07-21T19:05:30.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.250+0000 7fa04dffb700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 0x7fa0400a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.251+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa03c014070 con 0x7fa050074550 2023-07-21T19:05:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.251+0000 7fa054b9b700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 0x7fa0400a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.251+0000 7fa054b9b700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 0x7fa0400a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa044006010 tx=0x7fa044005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:30.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.255+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa03c0962b0 con 0x7fa050074550 2023-07-21T19:05:30.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.509+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa05002c850 con 0x7fa050074550 2023-07-21T19:05:30.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.510+0000 7fa04dffb700 1 -- 172.21.15.5:0/4215700965 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8834 (secure 0 0 0) 0x7fa03c016330 con 0x7fa050074550 2023-07-21T19:05:30.520 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:30.521 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-07-21T19:05:30.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 msgr2=0x7fa0400a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 0x7fa0400a3650 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fa044006010 tx=0x7fa044005960 comp rx=0 tx=0).stop 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 msgr2=0x7fa050197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa03c00e9c0 tx=0x7fa03c00ecd0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 shutdown_connections 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa0400a11a0 0x7fa0400a3650 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa0501079c0 0x7fa05019ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa050072f70 0x7fa050197960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 --2- 172.21.15.5:0/4215700965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa050074550 0x7fa050197ea0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 >> 172.21.15.5:0/4215700965 conn(0x7fa0500fb0e0 msgr2=0x7fa0500fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 shutdown_connections 2023-07-21T19:05:30.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:30.512+0000 7fa056dff700 1 -- 172.21.15.5:0/4215700965 wait complete. 2023-07-21T19:05:30.525 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 12 2023-07-21T19:05:30.561 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:30 smithi101 ceph-mon[128136]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:30.562 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:30 smithi101 podman[153725]: 2023-07-21T19:05:30.841 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:30 smithi101 ceph-mds[153766]: -- 172.21.15.101:0/761243992 <== mon.2 v2:172.21.15.101:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558a8b5051e0 con 0x558a8b4db000 2023-07-21T19:05:30.842 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: starting mds.b at 2023-07-21T19:05:30.842 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:30 smithi101 bash[153725]: d2429d563f0f00a07d65714231c12008eda8c78c40598c748828678f48dfd6b8 2023-07-21T19:05:30.842 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:30 smithi101 systemd[1]: Started Ceph mds.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:05:30.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:30 smithi005 ceph-mon[126460]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:30.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:30 smithi156 ceph-mon[128316]: pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:31.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4215700965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:31 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:31.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4215700965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:31 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4215700965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:31.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:31 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:32.544 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:32.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 -- 172.21.15.5:0/2189174425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3241079c0 msgr2=0x7fe324109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/2189174425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3241079c0 0x7fe324109e40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fe31c00b0a0 tx=0x7fe31c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 -- 172.21.15.5:0/2189174425 shutdown_connections 2023-07-21T19:05:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/2189174425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3241079c0 0x7fe324109e40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/2189174425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe324074550 0x7fe3240729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/2189174425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe3240733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 -- 172.21.15.5:0/2189174425 >> 172.21.15.5:0/2189174425 conn(0x7fe3240fb0e0 msgr2=0x7fe3240fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.756+0000 7fe32a9ff700 1 -- 172.21.15.5:0/2189174425 shutdown_connections 2023-07-21T19:05:32.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.757+0000 7fe32a9ff700 1 -- 172.21.15.5:0/2189174425 wait complete. 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.758+0000 7fe32a9ff700 1 Processor -- start 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.758+0000 7fe32a9ff700 1 -- start start 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe324197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 0x7fe324197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 0x7fe32419f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419f630 con 0x7fe324074550 2023-07-21T19:05:32.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419f770 con 0x7fe3241079c0 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe32a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe32419f900 con 0x7fe324072f70 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe323fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe324197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 0x7fe324197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe323fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe324197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49100/0 (socket says 172.21.15.5:49100) 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe323fff700 1 -- 172.21.15.5:0/450154816 learned_addr learned my addr 172.21.15.5:0/450154816 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe328f9c700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 0x7fe32419f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 -- 172.21.15.5:0/450154816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 msgr2=0x7fe324197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe324197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 -- 172.21.15.5:0/450154816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 msgr2=0x7fe32419f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 0x7fe32419f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.759+0000 7fe3237fe700 1 -- 172.21.15.5:0/450154816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe31c009d00 con 0x7fe324074550 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe328f9c700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 0x7fe32419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe3237fe700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 0x7fe324197ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9c0 tx=0x7fe31800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe31800cbd0 con 0x7fe324074550 2023-07-21T19:05:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe31800eeb0 con 0x7fe324074550 2023-07-21T19:05:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe318003c40 con 0x7fe324074550 2023-07-21T19:05:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe32419fbe0 con 0x7fe324074550 2023-07-21T19:05:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.760+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3241a00a0 con 0x7fe324074550 2023-07-21T19:05:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.761+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe324106070 con 0x7fe324074550 2023-07-21T19:05:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.764+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fe318003da0 con 0x7fe324074550 2023-07-21T19:05:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.765+0000 7fe3217fa700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 0x7fe30c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.765+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fe318014070 con 0x7fe324074550 2023-07-21T19:05:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.765+0000 7fe323fff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 0x7fe30c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.766+0000 7fe323fff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 0x7fe30c0a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe314005950 tx=0x7fe314009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:32.769+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe318096270 con 0x7fe324074550 2023-07-21T19:05:32.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:32 smithi101 ceph-mon[128136]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:32.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:32 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:boot 2023-07-21T19:05:32.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:32 smithi101 ceph-mon[128136]: fsmap 12 up:standby 2023-07-21T19:05:32.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:32 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:05:32.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:32 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:32.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:32 smithi005 ceph-mon[126460]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:32.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:32 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:boot 2023-07-21T19:05:32.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:32 smithi005 ceph-mon[126460]: fsmap 12 up:standby 2023-07-21T19:05:32.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:32 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:05:32.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:32 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:32.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:32 smithi156 ceph-mon[128316]: pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:32.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:32 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:boot 2023-07-21T19:05:32.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:32 smithi156 ceph-mon[128316]: fsmap 12 up:standby 2023-07-21T19:05:32.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:32 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:05:32.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:32 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.028+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe32402c850 con 0x7fe324074550 2023-07-21T19:05:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.029+0000 7fe3217fa700 1 -- 172.21.15.5:0/450154816 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9592 (secure 0 0 0) 0x7fe318099890 con 0x7fe324074550 2023-07-21T19:05:33.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:33.040 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24511,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24523,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24524,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24553,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24566,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24614,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-07-21T19:05:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 msgr2=0x7fe30c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 0x7fe30c0a36d0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe314005950 tx=0x7fe314009260 comp rx=0 tx=0).stop 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 msgr2=0x7fe324197ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 0x7fe324197ff0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe31800e9c0 tx=0x7fe31800ecd0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 shutdown_connections 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe30c0a1220 0x7fe30c0a36d0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3241079c0 0x7fe32419f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe324072f70 0x7fe324197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 --2- 172.21.15.5:0/450154816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe324074550 0x7fe324197ff0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 >> 172.21.15.5:0/450154816 conn(0x7fe3240fb0e0 msgr2=0x7fe3240fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 shutdown_connections 2023-07-21T19:05:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.031+0000 7fe32a9ff700 1 -- 172.21.15.5:0/450154816 wait complete. 2023-07-21T19:05:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 13 2023-07-21T19:05:33.061 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:05:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 -- 172.21.15.5:0/3513741936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 msgr2=0x7fdafc107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 --2- 172.21.15.5:0/3513741936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc107c00 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b0a0 tx=0x7fdaf400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 -- 172.21.15.5:0/3513741936 shutdown_connections 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 --2- 172.21.15.5:0/3513741936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc107c00 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 --2- 172.21.15.5:0/3513741936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 0x7fdafc102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 --2- 172.21.15.5:0/3513741936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 0x7fdafc105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 -- 172.21.15.5:0/3513741936 >> 172.21.15.5:0/3513741936 conn(0x7fdafc0fa000 msgr2=0x7fdafc0fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.261+0000 7fdb009ff700 1 -- 172.21.15.5:0/3513741936 shutdown_connections 2023-07-21T19:05:33.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.262+0000 7fdb009ff700 1 -- 172.21.15.5:0/3513741936 wait complete. 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.262+0000 7fdb009ff700 1 Processor -- start 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 -- start start 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 0x7fdafc197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 0x7fdafc197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc19f5a0 con 0x7fdafc1057a0 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc19f6e0 con 0x7fdafc100560 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdb009ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdafc19f870 con 0x7fdafc102e80 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdafad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdafad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49600/0 (socket says 172.21.15.5:49600) 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdafad9d700 1 -- 172.21.15.5:0/4208787077 learned_addr learned my addr 172.21.15.5:0/4208787077 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:33.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdaf3fff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 0x7fdafc197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.263+0000 7fdafa59c700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 0x7fdafc197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 -- 172.21.15.5:0/4208787077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 msgr2=0x7fdafc197f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 0x7fdafc197f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 -- 172.21.15.5:0/4208787077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 msgr2=0x7fdafc197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 0x7fdafc197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 -- 172.21.15.5:0/4208787077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf4009d00 con 0x7fdafc1057a0 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdafad9d700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b070 tx=0x7fdaf4004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:33.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf400e040 con 0x7fdafc1057a0 2023-07-21T19:05:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdaf4003e90 con 0x7fdafc1057a0 2023-07-21T19:05:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdafc19faf0 con 0x7fdafc1057a0 2023-07-21T19:05:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4004020 con 0x7fdafc1057a0 2023-07-21T19:05:33.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.264+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdafc19ff20 con 0x7fdafc1057a0 2023-07-21T19:05:33.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.265+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdafc107310 con 0x7fdafc1057a0 2023-07-21T19:05:33.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.267+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdaf4003a00 con 0x7fdafc1057a0 2023-07-21T19:05:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.270+0000 7fdaf37fe700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 0x7fdae80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.270+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdaf40ce5b0 con 0x7fdafc1057a0 2023-07-21T19:05:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.270+0000 7fdafa59c700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 0x7fdae80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.271+0000 7fdafa59c700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 0x7fdae80a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005950 tx=0x7fdaec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.272+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdaf4098190 con 0x7fdafc1057a0 2023-07-21T19:05:33.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.499+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdafc02caa0 con 0x7fdafc1057a0 2023-07-21T19:05:33.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.500+0000 7fdaf37fe700 1 -- 172.21.15.5:0/4208787077 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+18399 (secure 0 0 0) 0x7fdaf40243b0 con 0x7fdafc1057a0 2023-07-21T19:05:33.511 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:33.511 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":81,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:04:41.495890+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 msgr2=0x7fdae80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 0x7fdae80a3650 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdaec005950 tx=0x7fdaec00b410 comp rx=0 tx=0).stop 2023-07-21T19:05:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 msgr2=0x7fdafc19efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400b070 tx=0x7fdaf4004c80 comp rx=0 tx=0).stop 2023-07-21T19:05:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 shutdown_connections 2023-07-21T19:05:33.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdae80a11a0 0x7fdae80a3650 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdafc1057a0 0x7fdafc19efd0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdafc100560 0x7fdafc197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 --2- 172.21.15.5:0/4208787077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdafc102e80 0x7fdafc197f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 >> 172.21.15.5:0/4208787077 conn(0x7fdafc0fa000 msgr2=0x7fdafc1011e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 shutdown_connections 2023-07-21T19:05:33.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.502+0000 7fdb009ff700 1 -- 172.21.15.5:0/4208787077 wait complete. 2023-07-21T19:05:33.533 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:05:33.534 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:05:33.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 -- 172.21.15.5:0/496994239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc780ffae0 msgr2=0x7efc780ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/496994239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc780ffae0 0x7efc780ffef0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7efc68009990 tx=0x7efc68009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 -- 172.21.15.5:0/496994239 shutdown_connections 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/496994239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc78101660 0x7efc78107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/496994239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc780ffae0 0x7efc780ffef0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/496994239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78100cd0 0x7efc78101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 -- 172.21.15.5:0/496994239 >> 172.21.15.5:0/496994239 conn(0x7efc780fb0e0 msgr2=0x7efc780fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 -- 172.21.15.5:0/496994239 shutdown_connections 2023-07-21T19:05:33.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.737+0000 7efc7d9ff700 1 -- 172.21.15.5:0/496994239 wait complete. 2023-07-21T19:05:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.738+0000 7efc7d9ff700 1 Processor -- start 2023-07-21T19:05:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 -- start start 2023-07-21T19:05:33.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 0x7efc78072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 0x7efc78072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc780735e0 con 0x7efc78100cd0 2023-07-21T19:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc78073750 con 0x7efc78101660 2023-07-21T19:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc7d9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc7819f800 con 0x7efc780ffae0 2023-07-21T19:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc76ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc767fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 0x7efc78072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc76ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49140/0 (socket says 172.21.15.5:49140) 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc76ffd700 1 -- 172.21.15.5:0/2862735358 learned_addr learned my addr 172.21.15.5:0/2862735358 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.739+0000 7efc777fe700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 0x7efc78072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 -- 172.21.15.5:0/2862735358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 msgr2=0x7efc780742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 -- 172.21.15.5:0/2862735358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 msgr2=0x7efc78072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 0x7efc78072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 -- 172.21.15.5:0/2862735358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc6c009520 con 0x7efc78100cd0 2023-07-21T19:05:33.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc76ffd700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc777fe700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 0x7efc78072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc767fc700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 0x7efc78072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00e9f0 tx=0x7efc6c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:33.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c00cb70 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efc6c00ee90 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc680095f0 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.740+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6c003bc0 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.741+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc7819fe60 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.742+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc7810b1b0 con 0x7efc78100cd0 2023-07-21T19:05:33.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.744+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7efc6c003d40 con 0x7efc78100cd0 2023-07-21T19:05:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.745+0000 7efc7c9fd700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 0x7efc600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.745+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7efc6c014070 con 0x7efc78100cd0 2023-07-21T19:05:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.745+0000 7efc76ffd700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 0x7efc600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:33.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.746+0000 7efc76ffd700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 0x7efc600a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc68006010 tx=0x7efc6800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:33.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.749+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7efc6c096440 con 0x7efc78100cd0 2023-07-21T19:05:33.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:33 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/450154816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:33.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:33 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/450154816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:33.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:33 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/450154816' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:33.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.968+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efc78106a90 con 0x7efc78100cd0 2023-07-21T19:05:33.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.968+0000 7efc7c9fd700 1 -- 172.21.15.5:0/2862735358 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) v1 ==== 93+0+10 (secure 0 0 0) 0x7efc6c099a60 con 0x7efc78100cd0 2023-07-21T19:05:33.980 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:05:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 msgr2=0x7efc600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 0x7efc600a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7efc68006010 tx=0x7efc6800b260 comp rx=0 tx=0).stop 2023-07-21T19:05:33.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 msgr2=0x7efc78072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 0x7efc78072900 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7efc6c00e9f0 tx=0x7efc6c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:05:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 shutdown_connections 2023-07-21T19:05:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7efc600a1150 0x7efc600a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efc78101660 0x7efc78072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efc780ffae0 0x7efc780742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 --2- 172.21.15.5:0/2862735358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efc78100cd0 0x7efc78072900 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:33.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 >> 172.21.15.5:0/2862735358 conn(0x7efc780fb0e0 msgr2=0x7efc780fd400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:33.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 shutdown_connections 2023-07-21T19:05:33.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:33.971+0000 7efc7d9ff700 1 -- 172.21.15.5:0/2862735358 wait complete. 2023-07-21T19:05:34.009 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-07-21T19:05:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 -- 172.21.15.5:0/1670758915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 msgr2=0x7f98c8109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/1670758915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c8109e20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f98c000b0a0 tx=0x7f98c000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 -- 172.21.15.5:0/1670758915 shutdown_connections 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/1670758915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c8109e20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/1670758915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 0x7f98c80729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/1670758915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c80733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 -- 172.21.15.5:0/1670758915 >> 172.21.15.5:0/1670758915 conn(0x7f98c80fb100 msgr2=0x7f98c80fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 -- 172.21.15.5:0/1670758915 shutdown_connections 2023-07-21T19:05:34.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.210+0000 7f98cd7ff700 1 -- 172.21.15.5:0/1670758915 wait complete. 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.211+0000 7f98cd7ff700 1 Processor -- start 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.211+0000 7f98cd7ff700 1 -- start start 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c8197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 0x7f98c8197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819f580 con 0x7f98c81079a0 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819f6c0 con 0x7f98c8074550 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98cd7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c819f850 con 0x7f98c8072f70 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98c67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 0x7f98c8197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98c77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49648/0 (socket says 172.21.15.5:49648) 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c8197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.212+0000 7f98c77fe700 1 -- 172.21.15.5:0/3235650908 learned_addr learned my addr 172.21.15.5:0/3235650908 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 -- 172.21.15.5:0/3235650908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 msgr2=0x7f98c8197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c8197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 -- 172.21.15.5:0/3235650908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 msgr2=0x7f98c8197f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 0x7f98c8197f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 -- 172.21.15.5:0/3235650908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009d00 con 0x7f98c81079a0 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c6ffd700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:34.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98c77fe700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98c000d010 tx=0x7f98c000bcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c000e040 con 0x7f98c81079a0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98c0003e10 con 0x7f98c81079a0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98c819fad0 con 0x7f98c81079a0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.213+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00052a0 con 0x7f98c81079a0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.214+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c819fea0 con 0x7f98c81079a0 2023-07-21T19:05:34.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.215+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c8106050 con 0x7f98c81079a0 2023-07-21T19:05:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.217+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f98c001d9a0 con 0x7f98c81079a0 2023-07-21T19:05:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.218+0000 7f98affff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 0x7f98b00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.218+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f98c00cd7e0 con 0x7f98c81079a0 2023-07-21T19:05:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.218+0000 7f98c6ffd700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 0x7f98b00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.219+0000 7f98c6ffd700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 0x7f98b00a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98c8074920 tx=0x7f98b80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:34.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.224+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f98c0097460 con 0x7f98c81079a0 2023-07-21T19:05:34.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.465+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f98c810d390 con 0x7f98c81079a0 2023-07-21T19:05:34.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.465+0000 7f98affff700 1 -- 172.21.15.5:0/3235650908 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) v1 ==== 84+0+65 (secure 0 0 0) 0x7f98c009aa80 con 0x7f98c81079a0 2023-07-21T19:05:34.477 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:34.477 INFO:teuthology.orchestra.run.smithi005.stdout:[ 2023-07-21T19:05:34.477 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:05:34.478 INFO:teuthology.orchestra.run.smithi005.stdout: "poolnum": 1, 2023-07-21T19:05:34.478 INFO:teuthology.orchestra.run.smithi005.stdout: "poolname": ".mgr" 2023-07-21T19:05:34.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:05:34.478 INFO:teuthology.orchestra.run.smithi005.stdout:] 2023-07-21T19:05:34.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 msgr2=0x7f98b00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 0x7f98b00a3650 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f98c8074920 tx=0x7f98b80073d0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 msgr2=0x7f98c819efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f98c000d010 tx=0x7f98c000bcc0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 shutdown_connections 2023-07-21T19:05:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98b00a11a0 0x7f98b00a3650 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98c81079a0 0x7f98c819efb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98c8072f70 0x7f98c8197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 --2- 172.21.15.5:0/3235650908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98c8074550 0x7f98c8197f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 >> 172.21.15.5:0/3235650908 conn(0x7f98c80fb100 msgr2=0x7f98c80fd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 shutdown_connections 2023-07-21T19:05:34.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.469+0000 7f98cd7ff700 1 -- 172.21.15.5:0/3235650908 wait complete. 2023-07-21T19:05:34.502 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-07-21T19:05:34.502 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-07-21T19:05:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.719+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1647188340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00feca0 msgr2=0x7fd2c00ff110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.719+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1647188340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00feca0 0x7fd2c00ff110 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd2a80098f0 tx=0x7fd2a8009c00 comp rx=0 tx=0).stop 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1647188340 shutdown_connections 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1647188340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00ff650 0x7fd2c00ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1647188340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00fe350 0x7fd2c00fe760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1647188340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00feca0 0x7fd2c00ff110 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1647188340 >> 172.21.15.5:0/1647188340 conn(0x7fd2c00fa000 msgr2=0x7fd2c00fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1647188340 shutdown_connections 2023-07-21T19:05:34.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.720+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1647188340 wait complete. 2023-07-21T19:05:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.721+0000 7fd2c4bff700 1 Processor -- start 2023-07-21T19:05:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.721+0000 7fd2c4bff700 1 -- start start 2023-07-21T19:05:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00fe350 0x7fd2c01979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00ff650 0x7fd2c019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019f530 con 0x7fd2c00feca0 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019f670 con 0x7fd2c00fe350 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2c4bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c019f800 con 0x7fd2c00ff650 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49664/0 (socket says 172.21.15.5:49664) 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 -- 172.21.15.5:0/1681753357 learned_addr learned my addr 172.21.15.5:0/1681753357 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00fe350 0x7fd2c01979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2bed9d700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00ff650 0x7fd2c019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 -- 172.21.15.5:0/1681753357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00ff650 msgr2=0x7fd2c019ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00ff650 0x7fd2c019ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 -- 172.21.15.5:0/1681753357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00fe350 msgr2=0x7fd2c01979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00fe350 0x7fd2c01979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.722+0000 7fd2b7fff700 1 -- 172.21.15.5:0/1681753357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8009520 con 0x7fd2c00feca0 2023-07-21T19:05:34.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2b7fff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009fd0 tx=0x7fd2a800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a801c070 con 0x7fd2c00feca0 2023-07-21T19:05:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2a800f7b0 con 0x7fd2c00feca0 2023-07-21T19:05:34.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8004910 con 0x7fd2c00feca0 2023-07-21T19:05:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c019fa80 con 0x7fd2c00feca0 2023-07-21T19:05:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.723+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c019fea0 con 0x7fd2c00feca0 2023-07-21T19:05:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.724+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c0109130 con 0x7fd2c00feca0 2023-07-21T19:05:34.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.726+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd2a8020070 con 0x7fd2c00feca0 2023-07-21T19:05:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.728+0000 7fd2b77fe700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 0x7fd2ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:34.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.728+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd2a80cd220 con 0x7fd2c00feca0 2023-07-21T19:05:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.728+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 0x7fd2ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:34.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.729+0000 7fd2be59c700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 0x7fd2ac0a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:34.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.730+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd2a8096e00 con 0x7fd2c00feca0 2023-07-21T19:05:34.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:34 smithi101 ceph-mon[128136]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:34 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4208787077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:34 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2862735358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:05:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:34 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3235650908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T19:05:34.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:34 smithi005 ceph-mon[126460]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:34.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4208787077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:34.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2862735358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:05:34.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3235650908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T19:05:34.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:34 smithi156 ceph-mon[128316]: pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:34.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:34 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4208787077' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:34.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:34 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2862735358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:05:34.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:34 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3235650908' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-07-21T19:05:34.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:34.938+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd2c002cc70 con 0x7fd2c00feca0 2023-07-21T19:05:35.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.669+0000 7fd2b77fe700 1 -- 172.21.15.5:0/1681753357 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd2a809a420 con 0x7fd2c00feca0 2023-07-21T19:05:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.673+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 msgr2=0x7fd2ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:35.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.673+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 0x7fd2ac0a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b0009310 comp rx=0 tx=0).stop 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.673+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 msgr2=0x7fd2c0197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.673+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8009fd0 tx=0x7fd2a800b790 comp rx=0 tx=0).stop 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 shutdown_connections 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd2ac0a11d0 0x7fd2ac0a3680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2c00ff650 0x7fd2c019ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2c00fe350 0x7fd2c01979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 --2- 172.21.15.5:0/1681753357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd2c00feca0 0x7fd2c0197ef0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.674+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 >> 172.21.15.5:0/1681753357 conn(0x7fd2c00fa000 msgr2=0x7fd2c00fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:35.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.675+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 shutdown_connections 2023-07-21T19:05:35.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.675+0000 7fd2c4bff700 1 -- 172.21.15.5:0/1681753357 wait complete. 2023-07-21T19:05:35.686 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_metadata' created 2023-07-21T19:05:35.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-07-21T19:05:35.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:35.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:35.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T19:05:35.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:35 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:35.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:35.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:35.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T19:05:35.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:35 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:35.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 -- 172.21.15.5:0/3638002923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c70100ce0 msgr2=0x7f8c70101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:35.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3638002923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c70100ce0 0x7f8c70101130 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8c600098f0 tx=0x7f8c60009c00 comp rx=0 tx=0).stop 2023-07-21T19:05:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 -- 172.21.15.5:0/3638002923 shutdown_connections 2023-07-21T19:05:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3638002923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 0x7f8c70107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3638002923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c700ffaf0 0x7f8c700fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3638002923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c70100ce0 0x7f8c70101130 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.915+0000 7f8c775ff700 1 -- 172.21.15.5:0/3638002923 >> 172.21.15.5:0/3638002923 conn(0x7f8c700fb130 msgr2=0x7f8c700fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.916+0000 7f8c775ff700 1 -- 172.21.15.5:0/3638002923 shutdown_connections 2023-07-21T19:05:35.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.916+0000 7f8c775ff700 1 -- 172.21.15.5:0/3638002923 wait complete. 2023-07-21T19:05:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.917+0000 7f8c775ff700 1 Processor -- start 2023-07-21T19:05:35.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.917+0000 7f8c775ff700 1 -- start start 2023-07-21T19:05:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c70100ce0 0x7f8c70195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 0x7f8c7019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7019d330 con 0x7f8c700ffaf0 2023-07-21T19:05:35.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7019d470 con 0x7f8c70100ce0 2023-07-21T19:05:35.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c775ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c7019d600 con 0x7f8c70101670 2023-07-21T19:05:35.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c74b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c70100ce0 0x7f8c70195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:35.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49690/0 (socket says 172.21.15.5:49690) 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 -- 172.21.15.5:0/3152998020 learned_addr learned my addr 172.21.15.5:0/3152998020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c75b9c700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 0x7f8c7019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 -- 172.21.15.5:0/3152998020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 msgr2=0x7f8c7019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 0x7f8c7019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 -- 172.21.15.5:0/3152998020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c70100ce0 msgr2=0x7f8c70195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.918+0000 7f8c7539b700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c70100ce0 0x7f8c70195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:35.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c7539b700 1 -- 172.21.15.5:0/3152998020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c60009520 con 0x7f8c700ffaf0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c7539b700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e9f0 tx=0x7f8c6c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c004220 con 0x7f8c700ffaf0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c6c004380 con 0x7f8c700ffaf0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6c003a80 con 0x7f8c700ffaf0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c7019d8e0 con 0x7f8c700ffaf0 2023-07-21T19:05:35.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.919+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c7019dda0 con 0x7f8c700ffaf0 2023-07-21T19:05:35.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:35.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:35.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-07-21T19:05:35.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:35 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.921+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c70106ad0 con 0x7f8c700ffaf0 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.923+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8c6c003c20 con 0x7f8c700ffaf0 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.924+0000 7f8c667fc700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 0x7f8c5c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.924+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f8c6c014070 con 0x7f8c700ffaf0 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.924+0000 7f8c74b9a700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 0x7f8c5c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:35.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.925+0000 7f8c74b9a700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 0x7f8c5c0a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005e90 tx=0x7f8c60019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:35.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:35.927+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8c6c096780 con 0x7f8c700ffaf0 2023-07-21T19:05:36.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.144+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f8c7002c850 con 0x7f8c700ffaf0 2023-07-21T19:05:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.680+0000 7f8c667fc700 1 -- 172.21.15.5:0/3152998020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v83) v1 ==== 170+0+0 (secure 0 0 0) 0x7f8c6c099da0 con 0x7f8c700ffaf0 2023-07-21T19:05:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.697+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 msgr2=0x7f8c5c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.697+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 0x7f8c5c0a3600 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8c60005e90 tx=0x7f8c60019040 comp rx=0 tx=0).stop 2023-07-21T19:05:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.697+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 msgr2=0x7f8c701957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.697+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8c6c00e9f0 tx=0x7f8c6c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 shutdown_connections 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f8c5c0a1150 0x7f8c5c0a3600 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c70101670 0x7f8c7019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c700ffaf0 0x7f8c701957b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 --2- 172.21.15.5:0/3152998020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c70100ce0 0x7f8c70195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.704+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 >> 172.21.15.5:0/3152998020 conn(0x7f8c700fb130 msgr2=0x7f8c700fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.705+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 shutdown_connections 2023-07-21T19:05:36.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.705+0000 7f8c775ff700 1 -- 172.21.15.5:0/3152998020 wait complete. 2023-07-21T19:05:36.716 INFO:teuthology.orchestra.run.smithi005.stderr:pool 'cephfs_data' created 2023-07-21T19:05:36.740 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-07-21T19:05:36.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:36 smithi156 ceph-mon[128316]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:36.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:36 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T19:05:36.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:36 smithi156 ceph-mon[128316]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T19:05:36.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.945+0000 7f1c405ff700 1 -- 172.21.15.5:0/1748767212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38074550 msgr2=0x7f1c380729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.945+0000 7f1c405ff700 1 --2- 172.21.15.5:0/1748767212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38074550 0x7f1c380729a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1c340098f0 tx=0x7f1c34009c00 comp rx=0 tx=0).stop 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 -- 172.21.15.5:0/1748767212 shutdown_connections 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 --2- 172.21.15.5:0/1748767212 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3810c0c0 0x7f1c3810e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 --2- 172.21.15.5:0/1748767212 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c38074550 0x7f1c380729a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 --2- 172.21.15.5:0/1748767212 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38072f70 0x7f1c380733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 -- 172.21.15.5:0/1748767212 >> 172.21.15.5:0/1748767212 conn(0x7f1c380fb130 msgr2=0x7f1c380fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 -- 172.21.15.5:0/1748767212 shutdown_connections 2023-07-21T19:05:36.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.946+0000 7f1c405ff700 1 -- 172.21.15.5:0/1748767212 wait complete. 2023-07-21T19:05:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 Processor -- start 2023-07-21T19:05:36.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 -- start start 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38074550 0x7f1c3819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3810c0c0 0x7f1c381a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381a3990 con 0x7f1c38074550 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381a3ad0 con 0x7f1c38072f70 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c405ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c381a3c60 con 0x7f1c3810c0c0 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c3db9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38074550 0x7f1c3819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:36.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c3e39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c3e39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59898/0 (socket says 172.21.15.5:59898) 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c3e39b700 1 -- 172.21.15.5:0/190901574 learned_addr learned my addr 172.21.15.5:0/190901574 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.947+0000 7f1c3eb9c700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3810c0c0 0x7f1c381a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 -- 172.21.15.5:0/190901574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3810c0c0 msgr2=0x7f1c381a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3810c0c0 0x7f1c381a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 -- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38074550 msgr2=0x7f1c3819c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:36.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38074550 0x7f1c3819c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 -- 172.21.15.5:0/190901574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009520 con 0x7f1c38072f70 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c3e39b700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009fd0 tx=0x7f1c3400b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3401c070 con 0x7f1c38072f70 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c381a3ee0 con 0x7f1c38072f70 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.948+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c3400f7b0 con 0x7f1c38072f70 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.949+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340038e0 con 0x7f1c38072f70 2023-07-21T19:05:36.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.949+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c381a42a0 con 0x7f1c38072f70 2023-07-21T19:05:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.950+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c38101c10 con 0x7f1c38072f70 2023-07-21T19:05:36.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.952+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1c34020030 con 0x7f1c38072f70 2023-07-21T19:05:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.953+0000 7f1c2f7fe700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 0x7f1c240a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.953+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 7879+0+0 (secure 0 0 0) 0x7f1c340cd970 con 0x7f1c38072f70 2023-07-21T19:05:36.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.953+0000 7f1c3db9a700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 0x7f1c240a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:36.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.954+0000 7f1c3db9a700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 0x7f1c240a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c38074270 tx=0x7f1c28009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:36.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:36.959+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1c34097260 con 0x7f1c38072f70 2023-07-21T19:05:37.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:36 smithi101 ceph-mon[128136]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:37.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:36 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T19:05:37.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:36 smithi101 ceph-mon[128136]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T19:05:37.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:36 smithi005 ceph-mon[126460]: pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:37.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:36 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1681753357' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-07-21T19:05:37.168 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:36 smithi005 ceph-mon[126460]: osdmap e82: 12 total, 12 up, 12 in 2023-07-21T19:05:37.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.252+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f1c3802c850 con 0x7f1c38072f70 2023-07-21T19:05:37.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.681+0000 7f1c2f7fe700 1 -- 172.21.15.5:0/190901574 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7f1c3409a880 con 0x7f1c38072f70 2023-07-21T19:05:37.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 msgr2=0x7f1c240a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 0x7f1c240a3680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1c38074270 tx=0x7f1c28009310 comp rx=0 tx=0).stop 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 msgr2=0x7f1c3819be10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009fd0 tx=0x7f1c3400b790 comp rx=0 tx=0).stop 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 shutdown_connections 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c240a11d0 0x7f1c240a3680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:37.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3810c0c0 0x7f1c381a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c38072f70 0x7f1c3819be10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 --2- 172.21.15.5:0/190901574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c38074550 0x7f1c3819c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 >> 172.21.15.5:0/190901574 conn(0x7f1c380fb130 msgr2=0x7f1c380fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 shutdown_connections 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.683+0000 7f1c405ff700 1 -- 172.21.15.5:0/190901574 wait complete. 2023-07-21T19:05:37.694 INFO:teuthology.orchestra.run.smithi005.stderr:new fs with metadata pool 2 and data pool 3 2023-07-21T19:05:37.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:05:37.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 -- 172.21.15.5:0/2738694560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 msgr2=0x7f24f80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/2738694560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f80fff10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f24e8009990 tx=0x7f24e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.370 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/190901574' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T19:05:38.371 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:37 smithi156 ceph-mon[128316]: pgmap v299: 97 pgs: 12 creating+peering, 1 active+clean, 84 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/190901574' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.372 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T19:05:38.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T19:05:38.373 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:37 smithi101 ceph-mon[128136]: pgmap v299: 97 pgs: 12 creating+peering, 1 active+clean, 84 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 -- 172.21.15.5:0/2738694560 shutdown_connections 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/2738694560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 0x7f24f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/2738694560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f80fff10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/2738694560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 0x7f24f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.928+0000 7f24fd5ff700 1 -- 172.21.15.5:0/2738694560 >> 172.21.15.5:0/2738694560 conn(0x7f24f80fb0e0 msgr2=0x7f24f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:38.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.929+0000 7f24fd5ff700 1 -- 172.21.15.5:0/2738694560 shutdown_connections 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.929+0000 7f24fd5ff700 1 -- 172.21.15.5:0/2738694560 wait complete. 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.930+0000 7f24fd5ff700 1 Processor -- start 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.930+0000 7f24fd5ff700 1 -- start start 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 0x7f24f8195d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 0x7f24f819cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f819d350 con 0x7f24f80ffb00 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f819d490 con 0x7f24f8101680 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24fd5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f819d620 con 0x7f24f8100cf0 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60544/0 (socket says 172.21.15.5:60544) 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 -- 172.21.15.5:0/671946786 learned_addr learned my addr 172.21.15.5:0/671946786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f67fc700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 0x7f24f8195d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f77fe700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 0x7f24f819cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 -- 172.21.15.5:0/671946786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 msgr2=0x7f24f8195d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 0x7f24f8195d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 -- 172.21.15.5:0/671946786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 msgr2=0x7f24f819cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 0x7f24f819cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.931+0000 7f24f6ffd700 1 -- 172.21.15.5:0/671946786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e80095f0 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24f6ffd700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f24e8003ec0 tx=0x7f24e8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e801d070 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f24e800bd50 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8022d00 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f819d8a0 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.932+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f819dcc0 con 0x7f24f80ffb00 2023-07-21T19:05:38.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.933+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f8107160 con 0x7f24f80ffb00 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.935+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f24e8022500 con 0x7f24f80ffb00 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.936+0000 7f24dffff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 0x7f24e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.937+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f24e80cdc80 con 0x7f24f80ffb00 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.937+0000 7f24f67fc700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 0x7f24e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.937+0000 7f24f67fc700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 0x7f24e00a36d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009c60 tx=0x7f24ec009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:38.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:37.940+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f24e8097510 con 0x7f24f80ffb00 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3152998020' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: osdmap e83: 12 total, 12 up, 12 in 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/190901574' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-07-21T19:05:38.381 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-07-21T19:05:38.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: osdmap e84: 12 total, 12 up, 12 in 2023-07-21T19:05:38.382 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:37 smithi005 ceph-mon[126460]: pgmap v299: 97 pgs: 12 creating+peering, 1 active+clean, 84 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:05:38.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.162+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24f802c850 con 0x7f24f80ffb00 2023-07-21T19:05:38.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.163+0000 7f24dffff700 1 -- 172.21.15.5:0/671946786 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+22109 (secure 0 0 0) 0x7f24e8022e60 con 0x7f24f80ffb00 2023-07-21T19:05:38.383 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:38.383 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":84,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:05:37.253657+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:38.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 msgr2=0x7f24e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 0x7f24e00a36d0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f24ec009c60 tx=0x7f24ec009260 comp rx=0 tx=0).stop 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 msgr2=0x7f24f81957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f24e8003ec0 tx=0x7f24e8005740 comp rx=0 tx=0).stop 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 shutdown_connections 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f24e00a1220 0x7f24e00a36d0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f24f8101680 0x7f24f819cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f24f80ffb00 0x7f24f81957d0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 --2- 172.21.15.5:0/671946786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f24f8100cf0 0x7f24f8195d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 >> 172.21.15.5:0/671946786 conn(0x7f24f80fb0e0 msgr2=0x7f24f81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:38.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 shutdown_connections 2023-07-21T19:05:38.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.166+0000 7f24fd5ff700 1 -- 172.21.15.5:0/671946786 wait complete. 2023-07-21T19:05:38.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:05:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.593+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/1863840447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58102e70 msgr2=0x7f4a58105250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.593+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/1863840447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58102e70 0x7f4a58105250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4a48009990 tx=0x7f4a48009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/1863840447 shutdown_connections 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/1863840447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58105790 0x7f4a58107bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/1863840447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58100550 0x7f4a58102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/1863840447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58102e70 0x7f4a58105250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/1863840447 >> 172.21.15.5:0/1863840447 conn(0x7f4a580f9fd0 msgr2=0x7f4a580fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/1863840447 shutdown_connections 2023-07-21T19:05:38.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.594+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/1863840447 wait complete. 2023-07-21T19:05:38.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.595+0000 7f4a5f9ff700 1 Processor -- start 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.595+0000 7f4a5f9ff700 1 -- start start 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 0x7f4a58195cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58105790 0x7f4a5819cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819d2f0 con 0x7f4a58100550 2023-07-21T19:05:38.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819d430 con 0x7f4a58102e70 2023-07-21T19:05:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5f9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a5819d5c0 con 0x7f4a58105790 2023-07-21T19:05:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5d79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5d79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60566/0 (socket says 172.21.15.5:60566) 2023-07-21T19:05:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5d79b700 1 -- 172.21.15.5:0/303847292 learned_addr learned my addr 172.21.15.5:0/303847292 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:38.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.596+0000 7f4a5df9c700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58105790 0x7f4a5819cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5cf9a700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 0x7f4a58195cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 -- 172.21.15.5:0/303847292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58105790 msgr2=0x7f4a5819cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58105790 0x7f4a5819cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 -- 172.21.15.5:0/303847292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 msgr2=0x7f4a58195cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 0x7f4a58195cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 -- 172.21.15.5:0/303847292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a480095f0 con 0x7f4a58100550 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5cf9a700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 0x7f4a58195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5d79b700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54009790 con 0x7f4a58100550 2023-07-21T19:05:38.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4a5400cbd0 con 0x7f4a58100550 2023-07-21T19:05:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a5819d8a0 con 0x7f4a58100550 2023-07-21T19:05:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a54018680 con 0x7f4a58100550 2023-07-21T19:05:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.597+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a5819ddb0 con 0x7f4a58100550 2023-07-21T19:05:38.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.598+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a580fbb40 con 0x7f4a58100550 2023-07-21T19:05:38.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.602+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4a5400cd40 con 0x7f4a58100550 2023-07-21T19:05:38.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.604+0000 7f4a4e7fc700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 0x7f4a440a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.604+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f4a54014070 con 0x7f4a58100550 2023-07-21T19:05:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.604+0000 7f4a5cf9a700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 0x7f4a440a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.604+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4a5409a150 con 0x7f4a58100550 2023-07-21T19:05:38.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.605+0000 7f4a5cf9a700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 0x7f4a440a3650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4a480098e0 tx=0x7f4a48004740 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:38.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.829+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a5802c850 con 0x7f4a58100550 2023-07-21T19:05:38.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.831+0000 7f4a4e7fc700 1 -- 172.21.15.5:0/303847292 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22096 (secure 0 0 0) 0x7f4a54099ce0 con 0x7f4a58100550 2023-07-21T19:05:38.841 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:38.841 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:05:38.273223+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:38.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 msgr2=0x7f4a440a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 0x7f4a440a3650 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4a480098e0 tx=0x7f4a48004740 comp rx=0 tx=0).stop 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 msgr2=0x7f4a58195770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4a5400e9a0 tx=0x7f4a5400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 shutdown_connections 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4a440a11a0 0x7f4a440a3650 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4a58105790 0x7f4a5819cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4a58100550 0x7f4a58195770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 --2- 172.21.15.5:0/303847292 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4a58102e70 0x7f4a58195cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 >> 172.21.15.5:0/303847292 conn(0x7f4a580f9fd0 msgr2=0x7f4a581011b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 shutdown_connections 2023-07-21T19:05:38.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:38.834+0000 7f4a5f9ff700 1 -- 172.21.15.5:0/303847292 wait complete. 2023-07-21T19:05:38.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:38 smithi005 ceph-mon[126460]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T19:05:38.862 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:38 smithi005 ceph-mon[126460]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T19:05:38.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:38 smithi005 ceph-mon[126460]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T19:05:38.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:38 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/671946786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:38.863 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:38 smithi005 ceph-mon[126460]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T19:05:38.878 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-07-21T19:05:39.090 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:38 smithi101 ceph-mon[128136]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T19:05:39.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:38 smithi101 ceph-mon[128136]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T19:05:39.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:38 smithi101 ceph-mon[128136]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T19:05:39.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:38 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/671946786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:39.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:38 smithi101 ceph-mon[128136]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T19:05:39.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3160693317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 msgr2=0x7f6410102940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:39.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3160693317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f6410102940 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f640c009990 tx=0x7f640c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 -- 172.21.15.5:0/3160693317 shutdown_connections 2023-07-21T19:05:39.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 --2- 172.21.15.5:0/3160693317 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 0x7f6410107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 --2- 172.21.15.5:0/3160693317 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f6410102940 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 --2- 172.21.15.5:0/3160693317 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 0x7f6410105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 -- 172.21.15.5:0/3160693317 >> 172.21.15.5:0/3160693317 conn(0x7f64100fa000 msgr2=0x7f64100fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:39.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 -- 172.21.15.5:0/3160693317 shutdown_connections 2023-07-21T19:05:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.092+0000 7f64179ff700 1 -- 172.21.15.5:0/3160693317 wait complete. 2023-07-21T19:05:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 Processor -- start 2023-07-21T19:05:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 -- start start 2023-07-21T19:05:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:39.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 0x7f6410195cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 0x7f641019cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641019d330 con 0x7f6410100560 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641019d470 con 0x7f64101057a0 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f64179ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f641019d600 con 0x7f6410102e80 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.093+0000 7f641579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:39.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60586/0 (socket says 172.21.15.5:60586) 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 -- 172.21.15.5:0/3902822745 learned_addr learned my addr 172.21.15.5:0/3902822745 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f6414f9a700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 0x7f6410195cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f6415f9c700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 0x7f641019cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 -- 172.21.15.5:0/3902822745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 msgr2=0x7f6410195cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 0x7f6410195cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 -- 172.21.15.5:0/3902822745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 msgr2=0x7f641019cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 0x7f641019cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 -- 172.21.15.5:0/3902822745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f640c0095f0 con 0x7f6410100560 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.094+0000 7f641579b700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f640c0098e0 tx=0x7f640c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.095+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c01d070 con 0x7f6410100560 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.095+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f640c00bdb0 con 0x7f6410100560 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.095+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f641019d880 con 0x7f6410100560 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.095+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f640c022d20 con 0x7f6410100560 2023-07-21T19:05:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.095+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f641019dd00 con 0x7f6410100560 2023-07-21T19:05:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.096+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64100fbb70 con 0x7f6410100560 2023-07-21T19:05:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.098+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f640c00b920 con 0x7f6410100560 2023-07-21T19:05:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.099+0000 7f64067fc700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 0x7f63fc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.100+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f640c0cde00 con 0x7f6410100560 2023-07-21T19:05:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.100+0000 7f6414f9a700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 0x7f63fc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.100+0000 7f6414f9a700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 0x7f63fc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6400009c60 tx=0x7f6400009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.103+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f640c097690 con 0x7f6410100560 2023-07-21T19:05:39.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:38 smithi156 ceph-mon[128316]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-07-21T19:05:39.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:38 smithi156 ceph-mon[128316]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-07-21T19:05:39.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:38 smithi156 ceph-mon[128316]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-07-21T19:05:39.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:38 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/671946786' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:39.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:38 smithi156 ceph-mon[128316]: osdmap e85: 12 total, 12 up, 12 in 2023-07-21T19:05:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:39.358+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f641002c850 con 0x7f6410100560 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:active 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/303847292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: pgmap v301: 97 pgs: 12 creating+peering, 31 active+clean, 54 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-21T19:05:40.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:39 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T19:05:40.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.088+0000 7f64067fc700 1 -- 172.21.15.5:0/3902822745 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) v1 ==== 118+0+0 (secure 0 0 0) 0x7f640c09acb0 con 0x7f6410100560 2023-07-21T19:05:40.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.090+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 msgr2=0x7f63fc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.090+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 0x7f63fc0a3680 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f6400009c60 tx=0x7f6400009260 comp rx=0 tx=0).stop 2023-07-21T19:05:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 msgr2=0x7f64101957b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f640c0098e0 tx=0x7f640c004730 comp rx=0 tx=0).stop 2023-07-21T19:05:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 shutdown_connections 2023-07-21T19:05:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f63fc0a11d0 0x7f63fc0a3680 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f64101057a0 0x7f641019cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6410100560 0x7f64101957b0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 --2- 172.21.15.5:0/3902822745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6410102e80 0x7f6410195cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 >> 172.21.15.5:0/3902822745 conn(0x7f64100fa000 msgr2=0x7f64101011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 shutdown_connections 2023-07-21T19:05:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.091+0000 7f64179ff700 1 -- 172.21.15.5:0/3902822745 wait complete. 2023-07-21T19:05:40.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:active 2023-07-21T19:05:40.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.117 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/303847292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.118 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: pgmap v301: 97 pgs: 12 creating+peering, 31 active+clean, 54 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-21T19:05:40.119 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:39 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T19:05:40.120 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2023-07-21T19:05:40.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-07-21T19:05:40.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] up:active 2023-07-21T19:05:40.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:40.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/303847292' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:40.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: pgmap v301: 97 pgs: 12 creating+peering, 31 active+clean, 54 unknown; 577 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s wr, 0 op/s 2023-07-21T19:05:40.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:39 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-07-21T19:05:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 -- 172.21.15.5:0/2922659777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c072f70 msgr2=0x7f874c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 --2- 172.21.15.5:0/2922659777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c072f70 0x7f874c0733e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8740009990 tx=0x7f8740009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 -- 172.21.15.5:0/2922659777 shutdown_connections 2023-07-21T19:05:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 --2- 172.21.15.5:0/2922659777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c1079a0 0x7f874c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 --2- 172.21.15.5:0/2922659777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c074550 0x7f874c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 --2- 172.21.15.5:0/2922659777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c072f70 0x7f874c0733e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 -- 172.21.15.5:0/2922659777 >> 172.21.15.5:0/2922659777 conn(0x7f874c0fb0e0 msgr2=0x7f874c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 -- 172.21.15.5:0/2922659777 shutdown_connections 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.319+0000 7f87517ff700 1 -- 172.21.15.5:0/2922659777 wait complete. 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.320+0000 7f87517ff700 1 Processor -- start 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 -- start start 2023-07-21T19:05:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c072f70 0x7f874c197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c074550 0x7f874c197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c19f590 con 0x7f874c1079a0 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c19f6d0 con 0x7f874c072f70 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f87517ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874c19f860 con 0x7f874c074550 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60612/0 (socket says 172.21.15.5:60612) 2023-07-21T19:05:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874b7fe700 1 -- 172.21.15.5:0/380774001 learned_addr learned my addr 172.21.15.5:0/380774001 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874affd700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c072f70 0x7f874c197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874a7fc700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c074550 0x7f874c197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874b7fe700 1 -- 172.21.15.5:0/380774001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c074550 msgr2=0x7f874c197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.321+0000 7f874b7fe700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c074550 0x7f874c197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f874b7fe700 1 -- 172.21.15.5:0/380774001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c072f70 msgr2=0x7f874c197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f874b7fe700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c072f70 0x7f874c197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f874b7fe700 1 -- 172.21.15.5:0/380774001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400095f0 con 0x7f874c1079a0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f874b7fe700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ee0 tx=0x7f874400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f874400ddd0 con 0x7f874c1079a0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f874400f040 con 0x7f874c1079a0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87440047d0 con 0x7f874c1079a0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f874c19fb40 con 0x7f874c1079a0 2023-07-21T19:05:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.322+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f874c1a0000 con 0x7f874c1079a0 2023-07-21T19:05:40.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.323+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f874c106050 con 0x7f874c1079a0 2023-07-21T19:05:40.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.325+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f8744003a00 con 0x7f874c1079a0 2023-07-21T19:05:40.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.327+0000 7f8733fff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 0x7f87340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.328+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f87440cc0e0 con 0x7f874c1079a0 2023-07-21T19:05:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.328+0000 7f874affd700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 0x7f87340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.328+0000 7f874affd700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 0x7f87340a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f874c074920 tx=0x7f873c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.331+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f874404e790 con 0x7f874c1079a0 2023-07-21T19:05:40.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:40.590+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f874c0611c0 con 0x7f874c1079a0 2023-07-21T19:05:41.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:41.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T19:05:41.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:41.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T19:05:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.091+0000 7f8733fff700 1 -- 172.21.15.5:0/380774001 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v18) v1 ==== 105+0+0 (secure 0 0 0) 0x7f87440958e0 con 0x7f874c1079a0 2023-07-21T19:05:41.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.094+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 msgr2=0x7f87340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.094+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 0x7f87340a3680 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f874c074920 tx=0x7f873c009260 comp rx=0 tx=0).stop 2023-07-21T19:05:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.094+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 msgr2=0x7f874c19efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.094+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f8744007ee0 tx=0x7f874400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 shutdown_connections 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f87340a11d0 0x7f87340a3680 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f874c1079a0 0x7f874c19efc0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f874c072f70 0x7f874c197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 --2- 172.21.15.5:0/380774001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f874c074550 0x7f874c197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 >> 172.21.15.5:0/380774001 conn(0x7f874c0fb0e0 msgr2=0x7f874c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:41.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 shutdown_connections 2023-07-21T19:05:41.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.095+0000 7f87517ff700 1 -- 172.21.15.5:0/380774001 wait complete. 2023-07-21T19:05:41.126 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.126 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:41.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T19:05:41.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:41.127 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T19:05:41.128 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-07-21T19:05:41.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:05:41.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:05:41.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:05:41.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3902822745' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-07-21T19:05:41.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:41.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-07-21T19:05:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 -- 172.21.15.5:0/3250898702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 msgr2=0x7f082c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:41.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 --2- 172.21.15.5:0/3250898702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c0733e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f081c009990 tx=0x7f081c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 -- 172.21.15.5:0/3250898702 shutdown_connections 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 --2- 172.21.15.5:0/3250898702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c1079a0 0x7f082c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 --2- 172.21.15.5:0/3250898702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c074550 0x7f082c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 --2- 172.21.15.5:0/3250898702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c0733e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 -- 172.21.15.5:0/3250898702 >> 172.21.15.5:0/3250898702 conn(0x7f082c0fb0e0 msgr2=0x7f082c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 -- 172.21.15.5:0/3250898702 shutdown_connections 2023-07-21T19:05:41.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.328+0000 7f08343ff700 1 -- 172.21.15.5:0/3250898702 wait complete. 2023-07-21T19:05:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.329+0000 7f08343ff700 1 Processor -- start 2023-07-21T19:05:41.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.329+0000 7f08343ff700 1 -- start start 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 0x7f082c103340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c1079a0 0x7f082c0ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c100130 con 0x7f082c072f70 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c1002a0 con 0x7f082c074550 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f08343ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f082c100460 con 0x7f082c1079a0 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:41.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60620/0 (socket says 172.21.15.5:60620) 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 -- 172.21.15.5:0/1844448061 learned_addr learned my addr 172.21.15.5:0/1844448061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083299c700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c1079a0 0x7f082c0ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083199a700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 0x7f082c103340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 -- 172.21.15.5:0/1844448061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c1079a0 msgr2=0x7f082c0ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c1079a0 0x7f082c0ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 -- 172.21.15.5:0/1844448061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 msgr2=0x7f082c103340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:41.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 0x7f082c103340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083219b700 1 -- 172.21.15.5:0/1844448061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081c0095f0 con 0x7f082c072f70 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.330+0000 7f083199a700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 0x7f082c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f083219b700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f082800c8f0 tx=0x7f082800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828007b00 con 0x7f082c072f70 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0828007c60 con 0x7f082c072f70 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0828018660 con 0x7f082c072f70 2023-07-21T19:05:41.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f082c1a3ef0 con 0x7f082c072f70 2023-07-21T19:05:41.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.331+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f082c1a42c0 con 0x7f082c072f70 2023-07-21T19:05:41.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.332+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f082c106050 con 0x7f082c072f70 2023-07-21T19:05:41.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.334+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f082800f450 con 0x7f082c072f70 2023-07-21T19:05:41.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.336+0000 7f08237fe700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 0x7f08180a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:41.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.337+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f08280cc7f0 con 0x7f082c072f70 2023-07-21T19:05:41.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.337+0000 7f083199a700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 0x7f08180a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:41.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.337+0000 7f083199a700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 0x7f08180a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b5c0 tx=0x7f081c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:41.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.340+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0828095ff0 con 0x7f082c072f70 2023-07-21T19:05:41.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:41.598+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f082c02c850 con 0x7f082c072f70 2023-07-21T19:05:41.841 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:05:41.491+0000 7fccf126a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-07-21T19:05:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T19:05:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T19:05:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T19:05:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T19:05:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T19:05:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: pgmap v302: 97 pgs: 3 creating+peering, 85 active+clean, 9 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2023-07-21T19:05:41.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:41 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T19:05:42.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.102+0000 7f08237fe700 1 -- 172.21.15.5:0/1844448061 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v20) v1 ==== 121+0+0 (secure 0 0 0) 0x7f0828099610 con 0x7f082c072f70 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 msgr2=0x7f08180a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 0x7f08180a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f081c00b5c0 tx=0x7f081c000bc0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 msgr2=0x7f082c102e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f082800c8f0 tx=0x7f082800ccb0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 shutdown_connections 2023-07-21T19:05:42.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f08180a1150 0x7f08180a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f082c1079a0 0x7f082c0ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f082c072f70 0x7f082c102e00 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 --2- 172.21.15.5:0/1844448061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f082c074550 0x7f082c103340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 >> 172.21.15.5:0/1844448061 conn(0x7f082c0fb0e0 msgr2=0x7f082c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 shutdown_connections 2023-07-21T19:05:42.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.105+0000 7f08343ff700 1 -- 172.21.15.5:0/1844448061 wait complete. 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T19:05:42.139 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T19:05:42.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: pgmap v302: 97 pgs: 3 creating+peering, 85 active+clean, 9 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2023-07-21T19:05:42.140 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:41 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T19:05:42.141 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-07-21T19:05:42.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-07-21T19:05:42.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/380774001' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-07-21T19:05:42.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-07-21T19:05:42.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-07-21T19:05:42.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2023-07-21T19:05:42.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: daemon mds.h is now active in filesystem cephfs as rank 1 2023-07-21T19:05:42.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: pgmap v302: 97 pgs: 3 creating+peering, 85 active+clean, 9 unknown; 578 KiB data, 326 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s wr, 8 op/s 2023-07-21T19:05:42.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:41 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-07-21T19:05:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 -- 172.21.15.5:0/2419135229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 msgr2=0x7f1acc101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/2419135229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc101130 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1abc009990 tx=0x7f1abc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 -- 172.21.15.5:0/2419135229 shutdown_connections 2023-07-21T19:05:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/2419135229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc101670 0x7f1acc107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/2419135229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc0ffaf0 0x7f1acc0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/2419135229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc101130 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 -- 172.21.15.5:0/2419135229 >> 172.21.15.5:0/2419135229 conn(0x7f1acc0fb130 msgr2=0x7f1acc0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 -- 172.21.15.5:0/2419135229 shutdown_connections 2023-07-21T19:05:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.344+0000 7f1ad3dff700 1 -- 172.21.15.5:0/2419135229 wait complete. 2023-07-21T19:05:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 Processor -- start 2023-07-21T19:05:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 -- start start 2023-07-21T19:05:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc0ffaf0 0x7f1acc0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 0x7f1acc072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc0735e0 con 0x7f1acc100ce0 2023-07-21T19:05:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc073750 con 0x7f1acc101670 2023-07-21T19:05:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.345+0000 7f1ad3dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1acc19f750 con 0x7f1acc0ffaf0 2023-07-21T19:05:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60646/0 (socket says 172.21.15.5:60646) 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 -- 172.21.15.5:0/3978564008 learned_addr learned my addr 172.21.15.5:0/3978564008 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad239c700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 0x7f1acc072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad1b9b700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc0ffaf0 0x7f1acc0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 -- 172.21.15.5:0/3978564008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc0ffaf0 msgr2=0x7f1acc0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc0ffaf0 0x7f1acc0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 -- 172.21.15.5:0/3978564008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 msgr2=0x7f1acc072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 0x7f1acc072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 -- 172.21.15.5:0/3978564008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1abc0095f0 con 0x7f1acc100ce0 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad239c700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 0x7f1acc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.346+0000 7f1ad139a700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006010 tx=0x7f1abc005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.347+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc01d070 con 0x7f1acc100ce0 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.347+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1abc004100 con 0x7f1acc100ce0 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.347+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1abc005290 con 0x7f1acc100ce0 2023-07-21T19:05:42.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.347+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1acc19f9d0 con 0x7f1acc100ce0 2023-07-21T19:05:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.347+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1acc19fda0 con 0x7f1acc100ce0 2023-07-21T19:05:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.348+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1acc10b1c0 con 0x7f1acc100ce0 2023-07-21T19:05:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.350+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f1abc0047e0 con 0x7f1acc100ce0 2023-07-21T19:05:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.351+0000 7f1ac2ffd700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 0x7f1ab80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.351+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f1abc0cd870 con 0x7f1acc100ce0 2023-07-21T19:05:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.351+0000 7f1ad1b9b700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 0x7f1ab80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.352+0000 7f1ad1b9b700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 0x7f1ab80a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005fd0 tx=0x7f1ac8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.355+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1abc0971a0 con 0x7f1acc100ce0 2023-07-21T19:05:42.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:42.609+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7f1acc1069f0 con 0x7f1acc100ce0 2023-07-21T19:05:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.284+0000 7f1ac2ffd700 1 -- 172.21.15.5:0/3978564008 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1abc09a7c0 con 0x7f1acc100ce0 2023-07-21T19:05:43.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 msgr2=0x7f1ab80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 0x7f1ab80a3680 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1ac8005fd0 tx=0x7f1ac8005e50 comp rx=0 tx=0).stop 2023-07-21T19:05:43.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 msgr2=0x7f1acc072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f1abc006010 tx=0x7f1abc005ef0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 shutdown_connections 2023-07-21T19:05:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ab80a11d0 0x7f1ab80a3680 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1acc101670 0x7f1acc072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1acc0ffaf0 0x7f1acc0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 --2- 172.21.15.5:0/3978564008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1acc100ce0 0x7f1acc072900 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 >> 172.21.15.5:0/3978564008 conn(0x7f1acc0fb130 msgr2=0x7f1acc0fd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 shutdown_connections 2023-07-21T19:05:43.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.287+0000 7f1ad3dff700 1 -- 172.21.15.5:0/3978564008 wait complete. 2023-07-21T19:05:43.322 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2023-07-21T19:05:43.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:active 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: Cluster is now healthy 2023-07-21T19:05:43.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T19:05:43.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T19:05:43.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T19:05:43.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:active 2023-07-21T19:05:43.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:43.390 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T19:05:43.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:active 2023-07-21T19:05:43.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T19:05:43.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T19:05:43.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T19:05:43.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: Cluster is now healthy 2023-07-21T19:05:43.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T19:05:43.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T19:05:43.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T19:05:43.432 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:active 2023-07-21T19:05:43.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:43.433 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:43 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T19:05:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 -- 172.21.15.5:0/1838590526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4880ffae0 msgr2=0x7fc4880ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 --2- 172.21.15.5:0/1838590526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4880ffae0 0x7fc4880ffef0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fc484009990 tx=0x7fc484009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 -- 172.21.15.5:0/1838590526 shutdown_connections 2023-07-21T19:05:43.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 --2- 172.21.15.5:0/1838590526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc488101660 0x7fc488107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 --2- 172.21.15.5:0/1838590526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4880ffae0 0x7fc4880ffef0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 --2- 172.21.15.5:0/1838590526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 -- 172.21.15.5:0/1838590526 >> 172.21.15.5:0/1838590526 conn(0x7fc4880fb0e0 msgr2=0x7fc4880fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 -- 172.21.15.5:0/1838590526 shutdown_connections 2023-07-21T19:05:43.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.526+0000 7fc4903ff700 1 -- 172.21.15.5:0/1838590526 wait complete. 2023-07-21T19:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.527+0000 7fc4903ff700 1 Processor -- start 2023-07-21T19:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 -- start start 2023-07-21T19:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 0x7fc4880742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4880735e0 con 0x7fc488101660 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc488073750 con 0x7fc4880ffae0 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc4903ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc48819f850 con 0x7fc488100cd0 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc48e99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc48e99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60668/0 (socket says 172.21.15.5:60668) 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc48e99c700 1 -- 172.21.15.5:0/2731073011 learned_addr learned my addr 172.21.15.5:0/2731073011 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:43.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc48e19b700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 0x7fc4880742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.528+0000 7fc48d99a700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 -- 172.21.15.5:0/2731073011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 msgr2=0x7fc488072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 -- 172.21.15.5:0/2731073011 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 msgr2=0x7fc4880742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 0x7fc4880742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 -- 172.21.15.5:0/2731073011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4840095f0 con 0x7fc488101660 2023-07-21T19:05:43.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48d99a700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e19b700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 0x7fc4880742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc48e99c700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc480007ee0 tx=0x7fc48000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480017070 con 0x7fc488101660 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc48819fb30 con 0x7fc488101660 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc48000f040 con 0x7fc488101660 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc480005710 con 0x7fc488101660 2023-07-21T19:05:43.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.529+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4881a0090 con 0x7fc488101660 2023-07-21T19:05:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.530+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc48810b1b0 con 0x7fc488101660 2023-07-21T19:05:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.532+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fc480005870 con 0x7fc488101660 2023-07-21T19:05:43.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.534+0000 7fc47f7fe700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 0x7fc4740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.534+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc4800cbff0 con 0x7fc488101660 2023-07-21T19:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.534+0000 7fc48e19b700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 0x7fc4740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:43.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.535+0000 7fc48e19b700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 0x7fc4740a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc48400b5c0 tx=0x7fc484005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:43.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.538+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc4800fe790 con 0x7fc488101660 2023-07-21T19:05:43.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] up:active 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1844448061' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: Cluster is now healthy 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2023-07-21T19:05:43.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: daemon mds.i is now active in filesystem cephfs as rank 2 2023-07-21T19:05:43.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] up:active 2023-07-21T19:05:43.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:43.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:43 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-07-21T19:05:43.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.778+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7fc488060950 con 0x7fc4740a1150 2023-07-21T19:05:43.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.827+0000 7fc47f7fe700 1 -- 172.21.15.5:0/2731073011 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc488060950 con 0x7fc4740a1150 2023-07-21T19:05:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 msgr2=0x7fc4740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 0x7fc4740a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc48400b5c0 tx=0x7fc484005fb0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 msgr2=0x7fc488072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:43.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fc480007ee0 tx=0x7fc48000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 shutdown_connections 2023-07-21T19:05:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc4740a1150 0x7fc4740a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc488101660 0x7fc488072e40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4880ffae0 0x7fc4880742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 --2- 172.21.15.5:0/2731073011 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc488100cd0 0x7fc488072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 >> 172.21.15.5:0/2731073011 conn(0x7fc4880fb0e0 msgr2=0x7fc4880fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 shutdown_connections 2023-07-21T19:05:43.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:43.829+0000 7fc4903ff700 1 -- 172.21.15.5:0/2731073011 wait complete. 2023-07-21T19:05:43.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-07-21T19:05:44.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 -- 172.21.15.5:0/1750707243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458100ce0 msgr2=0x7fd458101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 --2- 172.21.15.5:0/1750707243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458100ce0 0x7fd458101130 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd448009990 tx=0x7fd448009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 -- 172.21.15.5:0/1750707243 shutdown_connections 2023-07-21T19:05:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 --2- 172.21.15.5:0/1750707243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd458101670 0x7fd458107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 --2- 172.21.15.5:0/1750707243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4580ffaf0 0x7fd4580fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 --2- 172.21.15.5:0/1750707243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458100ce0 0x7fd458101130 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 -- 172.21.15.5:0/1750707243 >> 172.21.15.5:0/1750707243 conn(0x7fd4580fb130 msgr2=0x7fd4580fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 -- 172.21.15.5:0/1750707243 shutdown_connections 2023-07-21T19:05:44.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.058+0000 7fd4605ff700 1 -- 172.21.15.5:0/1750707243 wait complete. 2023-07-21T19:05:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 Processor -- start 2023-07-21T19:05:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 -- start start 2023-07-21T19:05:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4580ffaf0 0x7fd458197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458100ce0 0x7fd458197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f4f0 con 0x7fd458101670 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f630 con 0x7fd4580ffaf0 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.059+0000 7fd4605ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45819f7c0 con 0x7fd458100ce0 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60720/0 (socket says 172.21.15.5:60720) 2023-07-21T19:05:44.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 -- 172.21.15.5:0/373623085 learned_addr learned my addr 172.21.15.5:0/373623085 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45e39b700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4580ffaf0 0x7fd458197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45db9a700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458100ce0 0x7fd458197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 -- 172.21.15.5:0/373623085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458100ce0 msgr2=0x7fd458197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458100ce0 0x7fd458197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 -- 172.21.15.5:0/373623085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4580ffaf0 msgr2=0x7fd458197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4580ffaf0 0x7fd458197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 -- 172.21.15.5:0/373623085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4480095f0 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.060+0000 7fd45eb9c700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd450007d70 tx=0x7fd45000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.061+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd45000ac90 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.061+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd45819faa0 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.061+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd45000adf0 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.061+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4500115e0 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.061+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4581a0000 con 0x7fd458101670 2023-07-21T19:05:44.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.062+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4581920d0 con 0x7fd458101670 2023-07-21T19:05:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.064+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd450004ad0 con 0x7fd458101670 2023-07-21T19:05:44.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.066+0000 7fd44f7fe700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 0x7fd4440a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.066+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd4500ccdd0 con 0x7fd458101670 2023-07-21T19:05:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.066+0000 7fd45e39b700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 0x7fd4440a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.067+0000 7fd45e39b700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 0x7fd4440a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd458100b40 tx=0x7fd454009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:44.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.069+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd4500965d0 con 0x7fd458101670 2023-07-21T19:05:44.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:05:43.785+0000 7f006cc55700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:05:44.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:05:43.785+0000 7f006cc55700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:05:44.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:05:43.785+0000 7f006cc55700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:05:44.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:05:43.785+0000 7f006cc55700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:05:44.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:05:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:05:43.785+0000 7f006cc55700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:05:44.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.317+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7fd458060950 con 0x7fd4440a11d0 2023-07-21T19:05:44.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.322+0000 7fd44f7fe700 1 -- 172.21.15.5:0/373623085 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fd458060950 con 0x7fd4440a11d0 2023-07-21T19:05:44.332 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 2023-07-21T19:05:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 msgr2=0x7fd4440a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 0x7fd4440a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fd458100b40 tx=0x7fd454009190 comp rx=0 tx=0).stop 2023-07-21T19:05:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 msgr2=0x7fd45819ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd450007d70 tx=0x7fd45000a340 comp rx=0 tx=0).stop 2023-07-21T19:05:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 shutdown_connections 2023-07-21T19:05:44.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4440a11d0 0x7fd4440a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd458101670 0x7fd45819ef20 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4580ffaf0 0x7fd458197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 --2- 172.21.15.5:0/373623085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd458100ce0 0x7fd458197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 >> 172.21.15.5:0/373623085 conn(0x7fd4580fb130 msgr2=0x7fd458106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 shutdown_connections 2023-07-21T19:05:44.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.324+0000 7fd4605ff700 1 -- 172.21.15.5:0/373623085 wait complete. 2023-07-21T19:05:44.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-07-21T19:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 -- 172.21.15.5:0/4263325440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0334102e80 msgr2=0x7f0334105260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 --2- 172.21.15.5:0/4263325440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0334102e80 0x7f0334105260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0328009990 tx=0x7f0328009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 -- 172.21.15.5:0/4263325440 shutdown_connections 2023-07-21T19:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 --2- 172.21.15.5:0/4263325440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03341057a0 0x7f0334107c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 --2- 172.21.15.5:0/4263325440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 0x7f0334102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 --2- 172.21.15.5:0/4263325440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0334102e80 0x7f0334105260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 -- 172.21.15.5:0/4263325440 >> 172.21.15.5:0/4263325440 conn(0x7f03340fa000 msgr2=0x7f03340fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 -- 172.21.15.5:0/4263325440 shutdown_connections 2023-07-21T19:05:44.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.550+0000 7f033a9ff700 1 -- 172.21.15.5:0/4263325440 wait complete. 2023-07-21T19:05:44.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 Processor -- start 2023-07-21T19:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 -- start start 2023-07-21T19:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 0x7f03341979c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0334102e80 0x7f0334197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033419f540 con 0x7f03341057a0 2023-07-21T19:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033419f680 con 0x7f0334102e80 2023-07-21T19:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.551+0000 7f033a9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f033419f810 con 0x7f0334100560 2023-07-21T19:05:44.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60738/0 (socket says 172.21.15.5:60738) 2023-07-21T19:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 -- 172.21.15.5:0/1484041154 learned_addr learned my addr 172.21.15.5:0/1484041154 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0333fff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 0x7f03341979c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f03337fe700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0334102e80 0x7f0334197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 -- 172.21.15.5:0/1484041154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 msgr2=0x7f03341979c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 0x7f03341979c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 -- 172.21.15.5:0/1484041154 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0334102e80 msgr2=0x7f0334197f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0334102e80 0x7f0334197f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 -- 172.21.15.5:0/1484041154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03280095f0 con 0x7f03341057a0 2023-07-21T19:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f0338f9c700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f032c007ee0 tx=0x7f032c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:44.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.552+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c00ddd0 con 0x7f03341057a0 2023-07-21T19:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.553+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f033419faf0 con 0x7f03341057a0 2023-07-21T19:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.553+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f032c00f040 con 0x7f03341057a0 2023-07-21T19:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.553+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f032c004870 con 0x7f03341057a0 2023-07-21T19:05:44.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.553+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03341a0050 con 0x7f03341057a0 2023-07-21T19:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.554+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0334107360 con 0x7f03341057a0 2023-07-21T19:05:44.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.556+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f032c003a00 con 0x7f03341057a0 2023-07-21T19:05:44.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.557+0000 7f03317fa700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 0x7f031c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.557+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f032c0cc950 con 0x7f03341057a0 2023-07-21T19:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.557+0000 7f0333fff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 0x7f031c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.558+0000 7f0333fff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 0x7f031c0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0324005950 tx=0x7f03240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:44.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.559+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f032c0961f0 con 0x7f03341057a0 2023-07-21T19:05:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:44 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T19:05:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:44 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:44.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:44 smithi101 ceph-mon[128136]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 2.4 KiB/s wr, 9 op/s 2023-07-21T19:05:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:44 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T19:05:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:44 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:44.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:44 smithi005 ceph-mon[126460]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 2.4 KiB/s wr, 9 op/s 2023-07-21T19:05:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:44 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3978564008' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-07-21T19:05:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:44 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:05:44.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:44 smithi156 ceph-mon[128316]: pgmap v303: 97 pgs: 97 active+clean; 579 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 154 B/s rd, 2.4 KiB/s wr, 9 op/s 2023-07-21T19:05:44.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.804+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f0334060950 con 0x7f031c0a1150 2023-07-21T19:05:44.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.830+0000 7f03317fa700 1 -- 172.21.15.5:0/1484041154 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f0334060950 con 0x7f031c0a1150 2023-07-21T19:05:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.832+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 msgr2=0x7f031c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.832+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 0x7f031c0a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0324005950 tx=0x7f03240058e0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.832+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 msgr2=0x7f033419ef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:44.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.832+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f032c007ee0 tx=0x7f032c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 shutdown_connections 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f031c0a1150 0x7f031c0a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03341057a0 0x7f033419ef70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0334100560 0x7f03341979c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 --2- 172.21.15.5:0/1484041154 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0334102e80 0x7f0334197f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 >> 172.21.15.5:0/1484041154 conn(0x7f03340fa000 msgr2=0x7f03340febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 shutdown_connections 2023-07-21T19:05:44.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:44.833+0000 7f033a9ff700 1 -- 172.21.15.5:0/1484041154 wait complete. 2023-07-21T19:05:44.861 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-07-21T19:05:45.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.057+0000 7f9e91fff700 1 -- 172.21.15.5:0/2588301706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c100cd0 msgr2=0x7f9e8c101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.057+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2588301706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c101120 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9e80009990 tx=0x7f9e80009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 -- 172.21.15.5:0/2588301706 shutdown_connections 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2588301706 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c101660 0x7f9e8c107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2588301706 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2588301706 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c101120 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 -- 172.21.15.5:0/2588301706 >> 172.21.15.5:0/2588301706 conn(0x7f9e8c0fb0e0 msgr2=0x7f9e8c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 -- 172.21.15.5:0/2588301706 shutdown_connections 2023-07-21T19:05:45.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.058+0000 7f9e91fff700 1 -- 172.21.15.5:0/2588301706 wait complete. 2023-07-21T19:05:45.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 Processor -- start 2023-07-21T19:05:45.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 -- start start 2023-07-21T19:05:45.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c0735e0 con 0x7f9e8c101660 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c073750 con 0x7f9e8c100cd0 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e91fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e8c19f800 con 0x7f9e8c0ffae0 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60762/0 (socket says 172.21.15.5:60762) 2023-07-21T19:05:45.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e8bfff700 1 -- 172.21.15.5:0/2123589728 learned_addr learned my addr 172.21.15.5:0/2123589728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.059+0000 7f9e8affd700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8b7fe700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 -- 172.21.15.5:0/2123589728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 msgr2=0x7f9e8c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 -- 172.21.15.5:0/2123589728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c100cd0 msgr2=0x7f9e8c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 -- 172.21.15.5:0/2123589728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e800095f0 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8b7fe700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e8bfff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e84007ee0 tx=0x7f9e8400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84017070 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e8400f040 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e84005710 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e8c19fae0 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.060+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e8c19fff0 con 0x7f9e8c101660 2023-07-21T19:05:45.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.061+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e8c106ac0 con 0x7f9e8c101660 2023-07-21T19:05:45.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.063+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9e84005870 con 0x7f9e8c101660 2023-07-21T19:05:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.065+0000 7f9e88ff9700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 0x7f9e740a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.065+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f9e840cbfe0 con 0x7f9e8c101660 2023-07-21T19:05:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.065+0000 7f9e8b7fe700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 0x7f9e740a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.065+0000 7f9e8b7fe700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 0x7f9e740a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c005950 tx=0x7f9e7c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:45.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.069+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9e840957e0 con 0x7f9e8c101660 2023-07-21T19:05:45.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.318+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9e8c060950 con 0x7f9e740a11d0 2023-07-21T19:05:45.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.322+0000 7f9e88ff9700 1 -- 172.21.15.5:0/2123589728 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f9e8c060950 con 0x7f9e740a11d0 2023-07-21T19:05:45.331 INFO:teuthology.orchestra.run.smithi005.stdout:/volumes/_nogroup/sv_1/45ec36f2-c9c7-4a9f-8eb0-88519258f52c 2023-07-21T19:05:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 msgr2=0x7f9e740a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 0x7f9e740a3680 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e7c005950 tx=0x7f9e7c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 msgr2=0x7f9e8c072e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f9e84007ee0 tx=0x7f9e8400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 shutdown_connections 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9e740a11d0 0x7f9e740a3680 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e8c101660 0x7f9e8c072e40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e8c0ffae0 0x7f9e8c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.324+0000 7f9e91fff700 1 --2- 172.21.15.5:0/2123589728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e8c100cd0 0x7f9e8c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.325+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 >> 172.21.15.5:0/2123589728 conn(0x7f9e8c0fb0e0 msgr2=0x7f9e8c0fd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.325+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 shutdown_connections 2023-07-21T19:05:45.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.325+0000 7f9e91fff700 1 -- 172.21.15.5:0/2123589728 wait complete. 2023-07-21T19:05:45.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:05:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 -- 172.21.15.5:0/4067831533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 msgr2=0x7ffa1c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 --2- 172.21.15.5:0/4067831533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c101140 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7ffa04009990 tx=0x7ffa04009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 -- 172.21.15.5:0/4067831533 shutdown_connections 2023-07-21T19:05:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 --2- 172.21.15.5:0/4067831533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 --2- 172.21.15.5:0/4067831533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 0x7ffa1c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 --2- 172.21.15.5:0/4067831533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c101140 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 -- 172.21.15.5:0/4067831533 >> 172.21.15.5:0/4067831533 conn(0x7ffa1c0fb0e0 msgr2=0x7ffa1c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 -- 172.21.15.5:0/4067831533 shutdown_connections 2023-07-21T19:05:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.547+0000 7ffa219ff700 1 -- 172.21.15.5:0/4067831533 wait complete. 2023-07-21T19:05:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 Processor -- start 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 -- start start 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 0x7ffa1c197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c19eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19f490 con 0x7ffa1c100cf0 2023-07-21T19:05:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19f5d0 con 0x7ffa1c0ffb00 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.548+0000 7ffa219ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c19f760 con 0x7ffa1c101680 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60790/0 (socket says 172.21.15.5:60790) 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 -- 172.21.15.5:0/225828627 learned_addr learned my addr 172.21.15.5:0/225828627 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa1affd700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 0x7ffa1c197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa1b7fe700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c19eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 -- 172.21.15.5:0/225828627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 msgr2=0x7ffa1c19eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c19eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 -- 172.21.15.5:0/225828627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 msgr2=0x7ffa1c197910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 0x7ffa1c197910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 -- 172.21.15.5:0/225828627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa040095f0 con 0x7ffa1c100cf0 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa1b7fe700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.549+0000 7ffa13fff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa04005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.550+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0401d070 con 0x7ffa1c100cf0 2023-07-21T19:05:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.550+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c19f9e0 con 0x7ffa1c100cf0 2023-07-21T19:05:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.550+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffa04003f40 con 0x7ffa1c100cf0 2023-07-21T19:05:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.550+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa04022c70 con 0x7ffa1c100cf0 2023-07-21T19:05:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.550+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c19fe60 con 0x7ffa1c100cf0 2023-07-21T19:05:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.551+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c10b1d0 con 0x7ffa1c100cf0 2023-07-21T19:05:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.553+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ffa04003ab0 con 0x7ffa1c100cf0 2023-07-21T19:05:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.555+0000 7ffa209fd700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 0x7ffa080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.555+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ffa040cdd60 con 0x7ffa1c100cf0 2023-07-21T19:05:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.555+0000 7ffa1affd700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 0x7ffa080a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.555+0000 7ffa1affd700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 0x7ffa080a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c100b50 tx=0x7ffa0c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.558+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffa040975f0 con 0x7ffa1c100cf0 2023-07-21T19:05:45.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:45 smithi101 ceph-mon[128136]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:45 smithi101 ceph-mon[128136]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:45 smithi005 ceph-mon[126460]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:45 smithi005 ceph-mon[126460]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:45 smithi156 ceph-mon[128316]: from='client.14808 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:45 smithi156 ceph-mon[128316]: from='client.14820 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.810+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa1c1a0470 con 0x7ffa1c100cf0 2023-07-21T19:05:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.811+0000 7ffa209fd700 1 -- 172.21.15.5:0/225828627 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7ffa0409ac10 con 0x7ffa1c100cf0 2023-07-21T19:05:45.821 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:45.821 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:05:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.813+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 msgr2=0x7ffa080a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 0x7ffa080a36d0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c100b50 tx=0x7ffa0c007400 comp rx=0 tx=0).stop 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 msgr2=0x7ffa1c197e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffa04000c00 tx=0x7ffa04005740 comp rx=0 tx=0).stop 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 shutdown_connections 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffa080a1220 0x7ffa080a36d0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffa1c101680 0x7ffa1c19eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffa1c0ffb00 0x7ffa1c197910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 --2- 172.21.15.5:0/225828627 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffa1c100cf0 0x7ffa1c197e50 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 >> 172.21.15.5:0/225828627 conn(0x7ffa1c0fb0e0 msgr2=0x7ffa1c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 shutdown_connections 2023-07-21T19:05:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:45.814+0000 7ffa219ff700 1 -- 172.21.15.5:0/225828627 wait complete. 2023-07-21T19:05:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:05:45.842 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:05:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 -- 172.21.15.5:0/2422262670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60100cf0 msgr2=0x7f5b60101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 --2- 172.21.15.5:0/2422262670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60100cf0 0x7f5b60101140 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5b54009990 tx=0x7f5b54009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 -- 172.21.15.5:0/2422262670 shutdown_connections 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 --2- 172.21.15.5:0/2422262670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60101680 0x7f5b60107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 --2- 172.21.15.5:0/2422262670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 0x7f5b600fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 --2- 172.21.15.5:0/2422262670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60100cf0 0x7f5b60101140 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 -- 172.21.15.5:0/2422262670 >> 172.21.15.5:0/2422262670 conn(0x7f5b600fb0e0 msgr2=0x7f5b600fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 -- 172.21.15.5:0/2422262670 shutdown_connections 2023-07-21T19:05:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.036+0000 7f5b65fff700 1 -- 172.21.15.5:0/2422262670 wait complete. 2023-07-21T19:05:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.037+0000 7f5b65fff700 1 Processor -- start 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.037+0000 7f5b65fff700 1 -- start start 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 0x7f5b60197950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60100cf0 0x7f5b60197e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019f4d0 con 0x7f5b60101680 2023-07-21T19:05:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019f610 con 0x7f5b600ffb00 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b65fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6019f7a0 con 0x7f5b60100cf0 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60806/0 (socket says 172.21.15.5:60806) 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 -- 172.21.15.5:0/4165120728 learned_addr learned my addr 172.21.15.5:0/4165120728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5effd700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60100cf0 0x7f5b60197e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5f7fe700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 0x7f5b60197950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 -- 172.21.15.5:0/4165120728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60100cf0 msgr2=0x7f5b60197e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60100cf0 0x7f5b60197e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 -- 172.21.15.5:0/4165120728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 msgr2=0x7f5b60197950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 0x7f5b60197950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.038+0000 7f5b5ffff700 1 -- 172.21.15.5:0/4165120728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b540095f0 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b5ffff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800ef30 tx=0x7f5b5800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b58018070 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b5800f040 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6019fa20 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b5800cbc0 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.039+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6019ff80 con 0x7f5b60101680 2023-07-21T19:05:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.040+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b6010b1d0 con 0x7f5b60101680 2023-07-21T19:05:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.042+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 16) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5b5800cd80 con 0x7f5b60101680 2023-07-21T19:05:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.043+0000 7f5b5cff9700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 0x7f5b480a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.044+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f5b580cc280 con 0x7f5b60101680 2023-07-21T19:05:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.044+0000 7f5b5f7fe700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 0x7f5b480a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.044+0000 7f5b5f7fe700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 0x7f5b480a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b60100b50 tx=0x7f5b50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.047+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5b58095b10 con 0x7f5b60101680 2023-07-21T19:05:46.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.258+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b6010b390 con 0x7f5b60101680 2023-07-21T19:05:46.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.260+0000 7f5b5cff9700 1 -- 172.21.15.5:0/4165120728 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22096 (secure 0 0 0) 0x7f5b58099130 con 0x7f5b60101680 2023-07-21T19:05:46.270 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:46.270 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:05:38.273223+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 msgr2=0x7f5b480a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 0x7f5b480a36d0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f5b60100b50 tx=0x7f5b50009260 comp rx=0 tx=0).stop 2023-07-21T19:05:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 msgr2=0x7f5b6019ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5b5800ef30 tx=0x7f5b5800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 shutdown_connections 2023-07-21T19:05:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5b480a1220 0x7f5b480a36d0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b60101680 0x7f5b6019ef00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b600ffb00 0x7f5b60197950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 --2- 172.21.15.5:0/4165120728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b60100cf0 0x7f5b60197e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.262+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 >> 172.21.15.5:0/4165120728 conn(0x7f5b600fb0e0 msgr2=0x7f5b600fd310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.263+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 shutdown_connections 2023-07-21T19:05:46.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.263+0000 7f5b65fff700 1 -- 172.21.15.5:0/4165120728 wait complete. 2023-07-21T19:05:46.291 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-07-21T19:05:46.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T19:05:46.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 -- 172.21.15.5:0/2659134188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 msgr2=0x7f52f8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/2659134188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8101140 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f52ec009990 tx=0x7f52ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 -- 172.21.15.5:0/2659134188 shutdown_connections 2023-07-21T19:05:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/2659134188 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f8101680 0x7f52f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/2659134188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f80ffb00 0x7f52f80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/2659134188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8101140 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 -- 172.21.15.5:0/2659134188 >> 172.21.15.5:0/2659134188 conn(0x7f52f80fb0e0 msgr2=0x7f52f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 -- 172.21.15.5:0/2659134188 shutdown_connections 2023-07-21T19:05:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.493+0000 7f52fe5ff700 1 -- 172.21.15.5:0/2659134188 wait complete. 2023-07-21T19:05:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.494+0000 7f52fe5ff700 1 Processor -- start 2023-07-21T19:05:46.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.494+0000 7f52fe5ff700 1 -- start start 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f80ffb00 0x7f52f8195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 0x7f52f819cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d2e0 con 0x7f52f8100cf0 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d420 con 0x7f52f80ffb00 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fe5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f819d5b0 con 0x7f52f8101680 2023-07-21T19:05:46.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38292/0 (socket says 172.21.15.5:38292) 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 -- 172.21.15.5:0/1801149269 learned_addr learned my addr 172.21.15.5:0/1801149269 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f7fff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f80ffb00 0x7f52f8195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fcb9c700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 0x7f52f819cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 -- 172.21.15.5:0/1801149269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 msgr2=0x7f52f819cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 0x7f52f819cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 -- 172.21.15.5:0/1801149269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f80ffb00 msgr2=0x7f52f8195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f80ffb00 0x7f52f8195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52f77fe700 1 -- 172.21.15.5:0/1801149269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ec0095f0 con 0x7f52f8100cf0 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.495+0000 7f52fcb9c700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 0x7f52f819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52f77fe700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005910 tx=0x7f52ec005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec01d070 con 0x7f52f8100cf0 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f52ec00be90 con 0x7f52f8100cf0 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52f819d830 con 0x7f52f8100cf0 2023-07-21T19:05:46.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec022e60 con 0x7f52f8100cf0 2023-07-21T19:05:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.496+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f819dc50 con 0x7f52f8100cf0 2023-07-21T19:05:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.497+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f8107090 con 0x7f52f8100cf0 2023-07-21T19:05:46.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.500+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f52ec022600 con 0x7f52f8100cf0 2023-07-21T19:05:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.502+0000 7f52f57fa700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 0x7f52e00a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.502+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f52ec0ce2f0 con 0x7f52f8100cf0 2023-07-21T19:05:46.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.502+0000 7f52f7fff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 0x7f52e00a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:46.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.503+0000 7f52f7fff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 0x7f52e00a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52f8100b50 tx=0x7f52e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:46.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.504+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f52ec097a20 con 0x7f52f8100cf0 2023-07-21T19:05:46.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:46 smithi101 ceph-mon[128136]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:46 smithi101 ceph-mon[128136]: pgmap v304: 97 pgs: 97 active+clean; 597 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-07-21T19:05:46.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:46 smithi101 ceph-mon[128136]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:46 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/225828627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:46 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4165120728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:46 smithi005 ceph-mon[126460]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:46 smithi005 ceph-mon[126460]: pgmap v304: 97 pgs: 97 active+clean; 597 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-07-21T19:05:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:46 smithi005 ceph-mon[126460]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:46 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/225828627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:46 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4165120728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:46 smithi156 ceph-mon[128316]: from='client.14826 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:46 smithi156 ceph-mon[128316]: pgmap v304: 97 pgs: 97 active+clean; 597 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 6.6 KiB/s wr, 16 op/s 2023-07-21T19:05:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:46 smithi156 ceph-mon[128316]: from='client.14832 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:05:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:46 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/225828627' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:46 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4165120728' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:46.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.708+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52f8060950 con 0x7f52e00a1220 2023-07-21T19:05:46.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.721+0000 7f52f57fa700 1 -- 172.21.15.5:0/1801149269 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339092 (secure 0 0 0) 0x7f52f8060950 con 0x7f52e00a1220 2023-07-21T19:05:46.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 msgr2=0x7f52e00a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 0x7f52e00a36d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f52f8100b50 tx=0x7f52e800b410 comp rx=0 tx=0).stop 2023-07-21T19:05:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 msgr2=0x7f52f8195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f52ec005910 tx=0x7f52ec005720 comp rx=0 tx=0).stop 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 shutdown_connections 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f52e00a1220 0x7f52e00a36d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f52f8101680 0x7f52f819cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f52f80ffb00 0x7f52f8195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 --2- 172.21.15.5:0/1801149269 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f52f8100cf0 0x7f52f8195ca0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 >> 172.21.15.5:0/1801149269 conn(0x7f52f80fb0e0 msgr2=0x7f52f81041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 shutdown_connections 2023-07-21T19:05:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.726+0000 7f52fe5ff700 1 -- 172.21.15.5:0/1801149269 wait complete. 2023-07-21T19:05:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:05:46.754 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-07-21T19:05:45.313566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611196,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":163,"num_read_kb":156,"num_write":289,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334488,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607336,"statfs":{"total":1151940427776,"available":1151597912064,"internally_reserved":0,"allocated":11329536,"data_stored":7060404,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20828,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":27,"num_read_kb":37,"num_write":108,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040186"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712948+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:05:39.712948+0000","last_peered":"2023-07-21T19:05:39.712948+0000","last_clean":"2023-07-21T19:05:39.712948+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:05:39.712948+0000","last_undegraded":"2023-07-21T19:05:39.712948+0000","last_fullsized":"2023-07-21T19:05:39.712948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:20:58.783182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604930+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:05:39.604930+0000","last_peered":"2023-07-21T19:05:39.604930+0000","last_clean":"2023-07-21T19:05:39.604930+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:05:39.604930+0000","last_undegraded":"2023-07-21T19:05:39.604930+0000","last_fullsized":"2023-07-21T19:05:39.604930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:56:51.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604793+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:05:39.604793+0000","last_peered":"2023-07-21T19:05:39.604793+0000","last_clean":"2023-07-21T19:05:39.604793+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:05:39.604793+0000","last_undegraded":"2023-07-21T19:05:39.604793+0000","last_fullsized":"2023-07-21T19:05:39.604793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:04:50.978719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605202+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:05:39.605202+0000","last_peered":"2023-07-21T19:05:39.605202+0000","last_clean":"2023-07-21T19:05:39.605202+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:05:39.605202+0000","last_undegraded":"2023-07-21T19:05:39.605202+0000","last_fullsized":"2023-07-21T19:05:39.605202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:59.286011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689477+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:05:38.689477+0000","last_peered":"2023-07-21T19:05:38.689477+0000","last_clean":"2023-07-21T19:05:38.689477+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:05:38.689477+0000","last_undegraded":"2023-07-21T19:05:38.689477+0000","last_fullsized":"2023-07-21T19:05:38.689477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:18.537651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280299+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:05:38.280299+0000","last_peered":"2023-07-21T19:05:38.280299+0000","last_clean":"2023-07-21T19:05:38.280299+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:05:38.280299+0000","last_undegraded":"2023-07-21T19:05:38.280299+0000","last_fullsized":"2023-07-21T19:05:38.280299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:54.681325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605701+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:05:39.605701+0000","last_peered":"2023-07-21T19:05:39.605701+0000","last_clean":"2023-07-21T19:05:39.605701+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:05:39.605701+0000","last_undegraded":"2023-07-21T19:05:39.605701+0000","last_fullsized":"2023-07-21T19:05:39.605701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:02.351004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605628+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:05:39.605628+0000","last_peered":"2023-07-21T19:05:39.605628+0000","last_clean":"2023-07-21T19:05:39.605628+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:05:39.605628+0000","last_undegraded":"2023-07-21T19:05:39.605628+0000","last_fullsized":"2023-07-21T19:05:39.605628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:33.230338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606202+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:05:39.606202+0000","last_peered":"2023-07-21T19:05:39.606202+0000","last_clean":"2023-07-21T19:05:39.606202+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:05:39.606202+0000","last_undegraded":"2023-07-21T19:05:39.606202+0000","last_fullsized":"2023-07-21T19:05:39.606202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:48.724651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280630+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:05:38.280630+0000","last_peered":"2023-07-21T19:05:38.280630+0000","last_clean":"2023-07-21T19:05:38.280630+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:05:38.280630+0000","last_undegraded":"2023-07-21T19:05:38.280630+0000","last_fullsized":"2023-07-21T19:05:38.280630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:56.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606006+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:05:39.606006+0000","last_peered":"2023-07-21T19:05:39.606006+0000","last_clean":"2023-07-21T19:05:39.606006+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:05:39.606006+0000","last_undegraded":"2023-07-21T19:05:39.606006+0000","last_fullsized":"2023-07-21T19:05:39.606006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:15:44.329484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605051+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:05:39.605051+0000","last_peered":"2023-07-21T19:05:39.605051+0000","last_clean":"2023-07-21T19:05:39.605051+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:05:39.605051+0000","last_undegraded":"2023-07-21T19:05:39.605051+0000","last_fullsized":"2023-07-21T19:05:39.605051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:50:46.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280172+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:05:38.280172+0000","last_peered":"2023-07-21T19:05:38.280172+0000","last_clean":"2023-07-21T19:05:38.280172+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:05:38.280172+0000","last_undegraded":"2023-07-21T19:05:38.280172+0000","last_fullsized":"2023-07-21T19:05:38.280172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:03:52.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604436+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:05:39.604436+0000","last_peered":"2023-07-21T19:05:39.604436+0000","last_clean":"2023-07-21T19:05:39.604436+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:05:39.604436+0000","last_undegraded":"2023-07-21T19:05:39.604436+0000","last_fullsized":"2023-07-21T19:05:39.604436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:38.736877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604319+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:05:39.604319+0000","last_peered":"2023-07-21T19:05:39.604319+0000","last_clean":"2023-07-21T19:05:39.604319+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:05:39.604319+0000","last_undegraded":"2023-07-21T19:05:39.604319+0000","last_fullsized":"2023-07-21T19:05:39.604319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:23.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605042+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:05:39.605042+0000","last_peered":"2023-07-21T19:05:39.605042+0000","last_clean":"2023-07-21T19:05:39.605042+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:05:39.605042+0000","last_undegraded":"2023-07-21T19:05:39.605042+0000","last_fullsized":"2023-07-21T19:05:39.605042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:33.404439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.713006+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:05:39.713006+0000","last_peered":"2023-07-21T19:05:39.713006+0000","last_clean":"2023-07-21T19:05:39.713006+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:05:39.713006+0000","last_undegraded":"2023-07-21T19:05:39.713006+0000","last_fullsized":"2023-07-21T19:05:39.713006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:02:06.348958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605178+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:05:39.605178+0000","last_peered":"2023-07-21T19:05:39.605178+0000","last_clean":"2023-07-21T19:05:39.605178+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:05:39.605178+0000","last_undegraded":"2023-07-21T19:05:39.605178+0000","last_fullsized":"2023-07-21T19:05:39.605178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:11.975218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982978+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:05:38.982978+0000","last_peered":"2023-07-21T19:05:38.982978+0000","last_clean":"2023-07-21T19:05:38.982978+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:05:38.982978+0000","last_undegraded":"2023-07-21T19:05:38.982978+0000","last_fullsized":"2023-07-21T19:05:38.982978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:27:41.530811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605306+0000","last_change":"2023-07-21T19:05:36.693433+0000","last_active":"2023-07-21T19:05:39.605306+0000","last_peered":"2023-07-21T19:05:39.605306+0000","last_clean":"2023-07-21T19:05:39.605306+0000","last_became_active":"2023-07-21T19:05:36.693131+0000","last_became_peered":"2023-07-21T19:05:36.693131+0000","last_unstale":"2023-07-21T19:05:39.605306+0000","last_undegraded":"2023-07-21T19:05:39.605306+0000","last_fullsized":"2023-07-21T19:05:39.605306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:30.985725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605632+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:05:39.605632+0000","last_peered":"2023-07-21T19:05:39.605632+0000","last_clean":"2023-07-21T19:05:39.605632+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:05:39.605632+0000","last_undegraded":"2023-07-21T19:05:39.605632+0000","last_fullsized":"2023-07-21T19:05:39.605632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:14.093922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605883+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:05:39.605883+0000","last_peered":"2023-07-21T19:05:39.605883+0000","last_clean":"2023-07-21T19:05:39.605883+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:05:39.605883+0000","last_undegraded":"2023-07-21T19:05:39.605883+0000","last_fullsized":"2023-07-21T19:05:39.605883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:05.016166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604693+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:05:39.604693+0000","last_peered":"2023-07-21T19:05:39.604693+0000","last_clean":"2023-07-21T19:05:39.604693+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:05:39.604693+0000","last_undegraded":"2023-07-21T19:05:39.604693+0000","last_fullsized":"2023-07-21T19:05:39.604693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:23:18.293241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.117934+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:05:42.117934+0000","last_peered":"2023-07-21T19:05:42.117934+0000","last_clean":"2023-07-21T19:05:42.117934+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:05:42.117934+0000","last_undegraded":"2023-07-21T19:05:42.117934+0000","last_fullsized":"2023-07-21T19:05:42.117934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:39.807659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605492+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:05:39.605492+0000","last_peered":"2023-07-21T19:05:39.605492+0000","last_clean":"2023-07-21T19:05:39.605492+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:05:39.605492+0000","last_undegraded":"2023-07-21T19:05:39.605492+0000","last_fullsized":"2023-07-21T19:05:39.605492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:55:31.296842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689751+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:05:38.689751+0000","last_peered":"2023-07-21T19:05:38.689751+0000","last_clean":"2023-07-21T19:05:38.689751+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:05:38.689751+0000","last_undegraded":"2023-07-21T19:05:38.689751+0000","last_fullsized":"2023-07-21T19:05:38.689751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:10.911304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280818+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:05:38.280818+0000","last_peered":"2023-07-21T19:05:38.280818+0000","last_clean":"2023-07-21T19:05:38.280818+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:05:38.280818+0000","last_undegraded":"2023-07-21T19:05:38.280818+0000","last_fullsized":"2023-07-21T19:05:38.280818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:34.631965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280499+0000","last_change":"2023-07-21T19:05:36.700941+0000","last_active":"2023-07-21T19:05:38.280499+0000","last_peered":"2023-07-21T19:05:38.280499+0000","last_clean":"2023-07-21T19:05:38.280499+0000","last_became_active":"2023-07-21T19:05:36.700646+0000","last_became_peered":"2023-07-21T19:05:36.700646+0000","last_unstale":"2023-07-21T19:05:38.280499+0000","last_undegraded":"2023-07-21T19:05:38.280499+0000","last_fullsized":"2023-07-21T19:05:38.280499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:09:33.485772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689650+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:05:38.689650+0000","last_peered":"2023-07-21T19:05:38.689650+0000","last_clean":"2023-07-21T19:05:38.689650+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:05:38.689650+0000","last_undegraded":"2023-07-21T19:05:38.689650+0000","last_fullsized":"2023-07-21T19:05:38.689650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:19:36.124026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.116831+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:05:42.116831+0000","last_peered":"2023-07-21T19:05:42.116831+0000","last_clean":"2023-07-21T19:05:42.116831+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:05:42.116831+0000","last_undegraded":"2023-07-21T19:05:42.116831+0000","last_fullsized":"2023-07-21T19:05:42.116831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:01:03.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280612+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:05:38.280612+0000","last_peered":"2023-07-21T19:05:38.280612+0000","last_clean":"2023-07-21T19:05:38.280612+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:05:38.280612+0000","last_undegraded":"2023-07-21T19:05:38.280612+0000","last_fullsized":"2023-07-21T19:05:38.280612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:03:58.158529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.119227+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:05:42.119227+0000","last_peered":"2023-07-21T19:05:42.119227+0000","last_clean":"2023-07-21T19:05:42.119227+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:05:42.119227+0000","last_undegraded":"2023-07-21T19:05:42.119227+0000","last_fullsized":"2023-07-21T19:05:42.119227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:54:26.215439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604821+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:05:39.604821+0000","last_peered":"2023-07-21T19:05:39.604821+0000","last_clean":"2023-07-21T19:05:39.604821+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:05:39.604821+0000","last_undegraded":"2023-07-21T19:05:39.604821+0000","last_fullsized":"2023-07-21T19:05:39.604821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:47.855231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280263+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:05:38.280263+0000","last_peered":"2023-07-21T19:05:38.280263+0000","last_clean":"2023-07-21T19:05:38.280263+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:05:38.280263+0000","last_undegraded":"2023-07-21T19:05:38.280263+0000","last_fullsized":"2023-07-21T19:05:38.280263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:39:06.512567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605247+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:05:39.605247+0000","last_peered":"2023-07-21T19:05:39.605247+0000","last_clean":"2023-07-21T19:05:39.605247+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:05:39.605247+0000","last_undegraded":"2023-07-21T19:05:39.605247+0000","last_fullsized":"2023-07-21T19:05:39.605247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:22.034088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:43.287499+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:05:43.287499+0000","last_peered":"2023-07-21T19:05:43.287499+0000","last_clean":"2023-07-21T19:05:43.287499+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:05:43.287499+0000","last_undegraded":"2023-07-21T19:05:43.287499+0000","last_fullsized":"2023-07-21T19:05:43.287499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:18.540576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605933+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:05:39.605933+0000","last_peered":"2023-07-21T19:05:39.605933+0000","last_clean":"2023-07-21T19:05:39.605933+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:05:39.605933+0000","last_undegraded":"2023-07-21T19:05:39.605933+0000","last_fullsized":"2023-07-21T19:05:39.605933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:53.349885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109359+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:05:41.109359+0000","last_peered":"2023-07-21T19:05:41.109359+0000","last_clean":"2023-07-21T19:05:41.109359+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:05:41.109359+0000","last_undegraded":"2023-07-21T19:05:41.109359+0000","last_fullsized":"2023-07-21T19:05:41.109359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:51:26.052671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604833+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:05:39.604833+0000","last_peered":"2023-07-21T19:05:39.604833+0000","last_clean":"2023-07-21T19:05:39.604833+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:05:39.604833+0000","last_undegraded":"2023-07-21T19:05:39.604833+0000","last_fullsized":"2023-07-21T19:05:39.604833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:51:33.432465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280877+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:05:38.280877+0000","last_peered":"2023-07-21T19:05:38.280877+0000","last_clean":"2023-07-21T19:05:38.280877+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:05:38.280877+0000","last_undegraded":"2023-07-21T19:05:38.280877+0000","last_fullsized":"2023-07-21T19:05:38.280877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:47:35.842718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605701+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:05:39.605701+0000","last_peered":"2023-07-21T19:05:39.605701+0000","last_clean":"2023-07-21T19:05:39.605701+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:05:39.605701+0000","last_undegraded":"2023-07-21T19:05:39.605701+0000","last_fullsized":"2023-07-21T19:05:39.605701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:19:42.213098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'14","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:44.290668+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:05:44.290668+0000","last_peered":"2023-07-21T19:05:44.290668+0000","last_clean":"2023-07-21T19:05:44.290668+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:05:44.290668+0000","last_undegraded":"2023-07-21T19:05:44.290668+0000","last_fullsized":"2023-07-21T19:05:44.290668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:49:58.331772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689721+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:05:38.689721+0000","last_peered":"2023-07-21T19:05:38.689721+0000","last_clean":"2023-07-21T19:05:38.689721+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:05:38.689721+0000","last_undegraded":"2023-07-21T19:05:38.689721+0000","last_fullsized":"2023-07-21T19:05:38.689721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:19:16.744848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.110784+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:05:41.110784+0000","last_peered":"2023-07-21T19:05:41.110784+0000","last_clean":"2023-07-21T19:05:41.110784+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:05:41.110784+0000","last_undegraded":"2023-07-21T19:05:41.110784+0000","last_fullsized":"2023-07-21T19:05:41.110784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:35:37.953554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279935+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:05:38.279935+0000","last_peered":"2023-07-21T19:05:38.279935+0000","last_clean":"2023-07-21T19:05:38.279935+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:05:38.279935+0000","last_undegraded":"2023-07-21T19:05:38.279935+0000","last_fullsized":"2023-07-21T19:05:38.279935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:50.852919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280617+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:05:38.280617+0000","last_peered":"2023-07-21T19:05:38.280617+0000","last_clean":"2023-07-21T19:05:38.280617+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:05:38.280617+0000","last_undegraded":"2023-07-21T19:05:38.280617+0000","last_fullsized":"2023-07-21T19:05:38.280617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:28.567411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605040+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:05:39.605040+0000","last_peered":"2023-07-21T19:05:39.605040+0000","last_clean":"2023-07-21T19:05:39.605040+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:05:39.605040+0000","last_undegraded":"2023-07-21T19:05:39.605040+0000","last_fullsized":"2023-07-21T19:05:39.605040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:06.205378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:43.289064+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:05:43.289064+0000","last_peered":"2023-07-21T19:05:43.289064+0000","last_clean":"2023-07-21T19:05:43.289064+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:05:43.289064+0000","last_undegraded":"2023-07-21T19:05:43.289064+0000","last_fullsized":"2023-07-21T19:05:43.289064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:14:12.044002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604751+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:05:39.604751+0000","last_peered":"2023-07-21T19:05:39.604751+0000","last_clean":"2023-07-21T19:05:39.604751+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:05:39.604751+0000","last_undegraded":"2023-07-21T19:05:39.604751+0000","last_fullsized":"2023-07-21T19:05:39.604751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:53:18.697494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.119906+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:05:42.119906+0000","last_peered":"2023-07-21T19:05:42.119906+0000","last_clean":"2023-07-21T19:05:42.119906+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:05:42.119906+0000","last_undegraded":"2023-07-21T19:05:42.119906+0000","last_fullsized":"2023-07-21T19:05:42.119906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:43.481871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280462+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:05:38.280462+0000","last_peered":"2023-07-21T19:05:38.280462+0000","last_clean":"2023-07-21T19:05:38.280462+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:05:38.280462+0000","last_undegraded":"2023-07-21T19:05:38.280462+0000","last_fullsized":"2023-07-21T19:05:38.280462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:28:02.437526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:40.273949+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:05:40.273949+0000","last_peered":"2023-07-21T19:05:40.273949+0000","last_clean":"2023-07-21T19:05:40.273949+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:05:40.273949+0000","last_undegraded":"2023-07-21T19:05:40.273949+0000","last_fullsized":"2023-07-21T19:05:40.273949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:04:15.919453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712781+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:05:39.712781+0000","last_peered":"2023-07-21T19:05:39.712781+0000","last_clean":"2023-07-21T19:05:39.712781+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:05:39.712781+0000","last_undegraded":"2023-07-21T19:05:39.712781+0000","last_fullsized":"2023-07-21T19:05:39.712781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:24.995798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605657+0000","last_change":"2023-07-21T19:05:36.700605+0000","last_active":"2023-07-21T19:05:39.605657+0000","last_peered":"2023-07-21T19:05:39.605657+0000","last_clean":"2023-07-21T19:05:39.605657+0000","last_became_active":"2023-07-21T19:05:36.700117+0000","last_became_peered":"2023-07-21T19:05:36.700117+0000","last_unstale":"2023-07-21T19:05:39.605657+0000","last_undegraded":"2023-07-21T19:05:39.605657+0000","last_fullsized":"2023-07-21T19:05:39.605657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:30.993635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.695096+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:05:39.695096+0000","last_peered":"2023-07-21T19:05:39.695096+0000","last_clean":"2023-07-21T19:05:39.695096+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:05:39.695096+0000","last_undegraded":"2023-07-21T19:05:39.695096+0000","last_fullsized":"2023-07-21T19:05:39.695096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:56:02.714363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605465+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:05:39.605465+0000","last_peered":"2023-07-21T19:05:39.605465+0000","last_clean":"2023-07-21T19:05:39.605465+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:05:39.605465+0000","last_undegraded":"2023-07-21T19:05:39.605465+0000","last_fullsized":"2023-07-21T19:05:39.605465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:02.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280255+0000","last_change":"2023-07-21T19:05:37.301107+0000","last_active":"2023-07-21T19:05:38.280255+0000","last_peered":"2023-07-21T19:05:38.280255+0000","last_clean":"2023-07-21T19:05:38.280255+0000","last_became_active":"2023-07-21T19:05:37.300547+0000","last_became_peered":"2023-07-21T19:05:37.300547+0000","last_unstale":"2023-07-21T19:05:38.280255+0000","last_undegraded":"2023-07-21T19:05:38.280255+0000","last_fullsized":"2023-07-21T19:05:38.280255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:15:08.496017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280532+0000","last_change":"2023-07-21T19:05:37.295173+0000","last_active":"2023-07-21T19:05:38.280532+0000","last_peered":"2023-07-21T19:05:38.280532+0000","last_clean":"2023-07-21T19:05:38.280532+0000","last_became_active":"2023-07-21T19:05:37.294425+0000","last_became_peered":"2023-07-21T19:05:37.294425+0000","last_unstale":"2023-07-21T19:05:38.280532+0000","last_undegraded":"2023-07-21T19:05:38.280532+0000","last_fullsized":"2023-07-21T19:05:38.280532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:47:32.039905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.120153+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:05:42.120153+0000","last_peered":"2023-07-21T19:05:42.120153+0000","last_clean":"2023-07-21T19:05:42.120153+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:05:42.120153+0000","last_undegraded":"2023-07-21T19:05:42.120153+0000","last_fullsized":"2023-07-21T19:05:42.120153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:44.502330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605429+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:05:39.605429+0000","last_peered":"2023-07-21T19:05:39.605429+0000","last_clean":"2023-07-21T19:05:39.605429+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:05:39.605429+0000","last_undegraded":"2023-07-21T19:05:39.605429+0000","last_fullsized":"2023-07-21T19:05:39.605429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:41.882514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.281744+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:05:42.281744+0000","last_peered":"2023-07-21T19:05:42.281744+0000","last_clean":"2023-07-21T19:05:42.281744+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:05:42.281744+0000","last_undegraded":"2023-07-21T19:05:42.281744+0000","last_fullsized":"2023-07-21T19:05:42.281744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:12.412105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604883+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:05:39.604883+0000","last_peered":"2023-07-21T19:05:39.604883+0000","last_clean":"2023-07-21T19:05:39.604883+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:05:39.604883+0000","last_undegraded":"2023-07-21T19:05:39.604883+0000","last_fullsized":"2023-07-21T19:05:39.604883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:02.966802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279621+0000","last_change":"2023-07-21T19:05:37.288734+0000","last_active":"2023-07-21T19:05:38.279621+0000","last_peered":"2023-07-21T19:05:38.279621+0000","last_clean":"2023-07-21T19:05:38.279621+0000","last_became_active":"2023-07-21T19:05:37.288382+0000","last_became_peered":"2023-07-21T19:05:37.288382+0000","last_unstale":"2023-07-21T19:05:38.279621+0000","last_undegraded":"2023-07-21T19:05:38.279621+0000","last_fullsized":"2023-07-21T19:05:38.279621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:52:31.362689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.117964+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:05:42.117964+0000","last_peered":"2023-07-21T19:05:42.117964+0000","last_clean":"2023-07-21T19:05:42.117964+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:05:42.117964+0000","last_undegraded":"2023-07-21T19:05:42.117964+0000","last_fullsized":"2023-07-21T19:05:42.117964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:47:59.441291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.107930+0000","last_change":"2023-07-21T19:05:36.704381+0000","last_active":"2023-07-21T19:05:41.107930+0000","last_peered":"2023-07-21T19:05:41.107930+0000","last_clean":"2023-07-21T19:05:41.107930+0000","last_became_active":"2023-07-21T19:05:36.703664+0000","last_became_peered":"2023-07-21T19:05:36.703664+0000","last_unstale":"2023-07-21T19:05:41.107930+0000","last_undegraded":"2023-07-21T19:05:41.107930+0000","last_fullsized":"2023-07-21T19:05:41.107930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:50:43.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279992+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:05:38.279992+0000","last_peered":"2023-07-21T19:05:38.279992+0000","last_clean":"2023-07-21T19:05:38.279992+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:05:38.279992+0000","last_undegraded":"2023-07-21T19:05:38.279992+0000","last_fullsized":"2023-07-21T19:05:38.279992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:59.069236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605034+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:05:39.605034+0000","last_peered":"2023-07-21T19:05:39.605034+0000","last_clean":"2023-07-21T19:05:39.605034+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:05:39.605034+0000","last_undegraded":"2023-07-21T19:05:39.605034+0000","last_fullsized":"2023-07-21T19:05:39.605034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:55:58.518652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605239+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:05:39.605239+0000","last_peered":"2023-07-21T19:05:39.605239+0000","last_clean":"2023-07-21T19:05:39.605239+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:05:39.605239+0000","last_undegraded":"2023-07-21T19:05:39.605239+0000","last_fullsized":"2023-07-21T19:05:39.605239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:06.818995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109869+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:05:41.109869+0000","last_peered":"2023-07-21T19:05:41.109869+0000","last_clean":"2023-07-21T19:05:41.109869+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:05:41.109869+0000","last_undegraded":"2023-07-21T19:05:41.109869+0000","last_fullsized":"2023-07-21T19:05:41.109869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:39.637910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605058+0000","last_change":"2023-07-21T19:05:37.298964+0000","last_active":"2023-07-21T19:05:39.605058+0000","last_peered":"2023-07-21T19:05:39.605058+0000","last_clean":"2023-07-21T19:05:39.605058+0000","last_became_active":"2023-07-21T19:05:37.298675+0000","last_became_peered":"2023-07-21T19:05:37.298675+0000","last_unstale":"2023-07-21T19:05:39.605058+0000","last_undegraded":"2023-07-21T19:05:39.605058+0000","last_fullsized":"2023-07-21T19:05:39.605058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:30.050668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.108896+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:05:41.108896+0000","last_peered":"2023-07-21T19:05:41.108896+0000","last_clean":"2023-07-21T19:05:41.108896+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:05:41.108896+0000","last_undegraded":"2023-07-21T19:05:41.108896+0000","last_fullsized":"2023-07-21T19:05:41.108896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:20:09.881230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689437+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:05:38.689437+0000","last_peered":"2023-07-21T19:05:38.689437+0000","last_clean":"2023-07-21T19:05:38.689437+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:05:38.689437+0000","last_undegraded":"2023-07-21T19:05:38.689437+0000","last_fullsized":"2023-07-21T19:05:38.689437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:45.813738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109600+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:05:41.109600+0000","last_peered":"2023-07-21T19:05:41.109600+0000","last_clean":"2023-07-21T19:05:41.109600+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:05:41.109600+0000","last_undegraded":"2023-07-21T19:05:41.109600+0000","last_fullsized":"2023-07-21T19:05:41.109600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:30.177296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712624+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:05:39.712624+0000","last_peered":"2023-07-21T19:05:39.712624+0000","last_clean":"2023-07-21T19:05:39.712624+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:05:39.712624+0000","last_undegraded":"2023-07-21T19:05:39.712624+0000","last_fullsized":"2023-07-21T19:05:39.712624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:49:34.364930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982671+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:05:38.982671+0000","last_peered":"2023-07-21T19:05:38.982671+0000","last_clean":"2023-07-21T19:05:38.982671+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:05:38.982671+0000","last_undegraded":"2023-07-21T19:05:38.982671+0000","last_fullsized":"2023-07-21T19:05:38.982671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:41:00.688642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604734+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:05:39.604734+0000","last_peered":"2023-07-21T19:05:39.604734+0000","last_clean":"2023-07-21T19:05:39.604734+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:05:39.604734+0000","last_undegraded":"2023-07-21T19:05:39.604734+0000","last_fullsized":"2023-07-21T19:05:39.604734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:47.291733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.110565+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:05:41.110565+0000","last_peered":"2023-07-21T19:05:41.110565+0000","last_clean":"2023-07-21T19:05:41.110565+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:05:41.110565+0000","last_undegraded":"2023-07-21T19:05:41.110565+0000","last_fullsized":"2023-07-21T19:05:41.110565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:15:25.607217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605626+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:05:39.605626+0000","last_peered":"2023-07-21T19:05:39.605626+0000","last_clean":"2023-07-21T19:05:39.605626+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:05:39.605626+0000","last_undegraded":"2023-07-21T19:05:39.605626+0000","last_fullsized":"2023-07-21T19:05:39.605626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:45.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.983010+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:05:38.983010+0000","last_peered":"2023-07-21T19:05:38.983010+0000","last_clean":"2023-07-21T19:05:38.983010+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:05:38.983010+0000","last_undegraded":"2023-07-21T19:05:38.983010+0000","last_fullsized":"2023-07-21T19:05:38.983010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:20.807340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606031+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:05:39.606031+0000","last_peered":"2023-07-21T19:05:39.606031+0000","last_clean":"2023-07-21T19:05:39.606031+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:05:39.606031+0000","last_undegraded":"2023-07-21T19:05:39.606031+0000","last_fullsized":"2023-07-21T19:05:39.606031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:23.107616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280752+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:05:38.280752+0000","last_peered":"2023-07-21T19:05:38.280752+0000","last_clean":"2023-07-21T19:05:38.280752+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:05:38.280752+0000","last_undegraded":"2023-07-21T19:05:38.280752+0000","last_fullsized":"2023-07-21T19:05:38.280752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:19.029881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605134+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:05:39.605134+0000","last_peered":"2023-07-21T19:05:39.605134+0000","last_clean":"2023-07-21T19:05:39.605134+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:05:39.605134+0000","last_undegraded":"2023-07-21T19:05:39.605134+0000","last_fullsized":"2023-07-21T19:05:39.605134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.288298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280107+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:05:38.280107+0000","last_peered":"2023-07-21T19:05:38.280107+0000","last_clean":"2023-07-21T19:05:38.280107+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:05:38.280107+0000","last_undegraded":"2023-07-21T19:05:38.280107+0000","last_fullsized":"2023-07-21T19:05:38.280107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:36:39.312004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605487+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:05:39.605487+0000","last_peered":"2023-07-21T19:05:39.605487+0000","last_clean":"2023-07-21T19:05:39.605487+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:05:39.605487+0000","last_undegraded":"2023-07-21T19:05:39.605487+0000","last_fullsized":"2023-07-21T19:05:39.605487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:38.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712706+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:05:39.712706+0000","last_peered":"2023-07-21T19:05:39.712706+0000","last_clean":"2023-07-21T19:05:39.712706+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:05:39.712706+0000","last_undegraded":"2023-07-21T19:05:39.712706+0000","last_fullsized":"2023-07-21T19:05:39.712706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:10:37.860659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982908+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:05:38.982908+0000","last_peered":"2023-07-21T19:05:38.982908+0000","last_clean":"2023-07-21T19:05:38.982908+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:05:38.982908+0000","last_undegraded":"2023-07-21T19:05:38.982908+0000","last_fullsized":"2023-07-21T19:05:38.982908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:56:14.208338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982743+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:05:38.982743+0000","last_peered":"2023-07-21T19:05:38.982743+0000","last_clean":"2023-07-21T19:05:38.982743+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:05:38.982743+0000","last_undegraded":"2023-07-21T19:05:38.982743+0000","last_fullsized":"2023-07-21T19:05:38.982743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:42:39.432370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604712+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:05:39.604712+0000","last_peered":"2023-07-21T19:05:39.604712+0000","last_clean":"2023-07-21T19:05:39.604712+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:05:39.604712+0000","last_undegraded":"2023-07-21T19:05:39.604712+0000","last_fullsized":"2023-07-21T19:05:39.604712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:01:46.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280015+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:05:38.280015+0000","last_peered":"2023-07-21T19:05:38.280015+0000","last_clean":"2023-07-21T19:05:38.280015+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:05:38.280015+0000","last_undegraded":"2023-07-21T19:05:38.280015+0000","last_fullsized":"2023-07-21T19:05:38.280015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:00:13.542991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712912+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:05:39.712912+0000","last_peered":"2023-07-21T19:05:39.712912+0000","last_clean":"2023-07-21T19:05:39.712912+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:05:39.712912+0000","last_undegraded":"2023-07-21T19:05:39.712912+0000","last_fullsized":"2023-07-21T19:05:39.712912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:27:30.166821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.983143+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:05:38.983143+0000","last_peered":"2023-07-21T19:05:38.983143+0000","last_clean":"2023-07-21T19:05:38.983143+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:05:38.983143+0000","last_undegraded":"2023-07-21T19:05:38.983143+0000","last_fullsized":"2023-07-21T19:05:38.983143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:34:45.714573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605089+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:05:39.605089+0000","last_peered":"2023-07-21T19:05:39.605089+0000","last_clean":"2023-07-21T19:05:39.605089+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:05:39.605089+0000","last_undegraded":"2023-07-21T19:05:39.605089+0000","last_fullsized":"2023-07-21T19:05:39.605089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:32:37.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280427+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:05:38.280427+0000","last_peered":"2023-07-21T19:05:38.280427+0000","last_clean":"2023-07-21T19:05:38.280427+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:05:38.280427+0000","last_undegraded":"2023-07-21T19:05:38.280427+0000","last_fullsized":"2023-07-21T19:05:38.280427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:31:36.193709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605141+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:05:39.605141+0000","last_peered":"2023-07-21T19:05:39.605141+0000","last_clean":"2023-07-21T19:05:39.605141+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:05:39.605141+0000","last_undegraded":"2023-07-21T19:05:39.605141+0000","last_fullsized":"2023-07-21T19:05:39.605141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:24:57.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605726+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:05:39.605726+0000","last_peered":"2023-07-21T19:05:39.605726+0000","last_clean":"2023-07-21T19:05:39.605726+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:05:39.605726+0000","last_undegraded":"2023-07-21T19:05:39.605726+0000","last_fullsized":"2023-07-21T19:05:39.605726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:47:50.017188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279990+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:05:38.279990+0000","last_peered":"2023-07-21T19:05:38.279990+0000","last_clean":"2023-07-21T19:05:38.279990+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:05:38.279990+0000","last_undegraded":"2023-07-21T19:05:38.279990+0000","last_fullsized":"2023-07-21T19:05:38.279990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:13.559642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712741+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:05:39.712741+0000","last_peered":"2023-07-21T19:05:39.712741+0000","last_clean":"2023-07-21T19:05:39.712741+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:05:39.712741+0000","last_undegraded":"2023-07-21T19:05:39.712741+0000","last_fullsized":"2023-07-21T19:05:39.712741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:36.652413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":20705,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":27,"num_read_kb":37,"num_write":107,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":30043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":438309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.131}]},{"osd":1,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.395},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.1859999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.208},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.24},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.595},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":9,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.522}]},{"osd":10,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.615},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.681}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1049259,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.437}]},{"osd":2,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.624},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.85199999999999998}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82799999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.307},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.436}]},{"osd":1,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.63},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.501},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.83399999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.69899999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.751}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":439047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.746},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.169,"15min":1.169},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.109,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.032},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.2150000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.86699999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.359,"15min":1.359},"last":1.117}]},{"osd":7,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84999999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0680000000000001,"5min":1.403,"15min":1.403},"last":1.157},{"interface":"front","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.65900000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.3160000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.246},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.238}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.27},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":1.2829999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":1030373,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.50700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.05,"15min":1.05},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.54900000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.877}]},{"osd":4,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.62}]},{"osd":5,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.002,"15min":1.044},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.133,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.67100000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91000000000000003,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.67200000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.05,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.79300000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":9,"last update":"Fri Jul 21 19:04:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.99299999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":438399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95199999999999996,"5min":1.087,"15min":1.087},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.018,"5min":1.149,"15min":1.149},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.04,"15min":1.04},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0669999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:05:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.83999999999999997,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.85799999999999998,"15min":0.83999999999999997},"min":{"1min":0.71099999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.79200000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.623}]},{"osd":10,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.88},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":439873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.61899999999999999},"min":{"1min":0.47999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97899999999999998,"5min":1.054,"15min":1.054},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.52900000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:04:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86599999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.048,"15min":1.048},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95799999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.496,"15min":0.496},"max":{"1min":0.98899999999999999,"5min":1.502,"15min":1.502},"last":0.75}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":439787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0369999999999999,"5min":1.177,"15min":1.177},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96599999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0169999999999999,"5min":1.139,"15min":1.139},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93300000000000005,"5min":1.113,"15min":1.113},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.503}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84399999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91400000000000003,"5min":1.042,"15min":1.042},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93999999999999995,"5min":1.163,"15min":1.163},"last":0.55200000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.032,"5min":1.304,"15min":1.304},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.626}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.75900000000000001}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.85099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38900000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.59799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.081,"15min":1.081},"last":0.502}]},{"osd":3,"last update":"Fri Jul 21 19:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.04,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.92500000000000004,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Jul 21 19:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.745},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78500000000000003,"5min":1.167,"15min":1.167},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.58199999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0609999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.95799999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.98099999999999998}]},{"osd":9,"last update":"Fri Jul 21 19:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.873},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.008}]},{"osd":10,"last update":"Fri Jul 21 19:05:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":438593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.441,"15min":0.41299999999999998},"min":{"1min":0.32000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.63200000000000001},"min":{"1min":0.53900000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.55000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98199999999999998,"5min":1.048,"15min":1.048},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.54200000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71099999999999997,"15min":0.68300000000000005},"min":{"1min":0.57699999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1160000000000001,"5min":1.175,"15min":1.175},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.044,"5min":1.264,"15min":1.264},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":1.141,"15min":1.141},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:04:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79100000000000004,"15min":0.78600000000000003},"min":{"1min":0.56999999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.83699999999999997,"15min":0.82099999999999995},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2490000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.80200000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.626}]},{"osd":10,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.748},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":438905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.47299999999999998,"15min":0.439},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":1.167,"15min":1.167},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96199999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68400000000000005,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97799999999999998,"5min":1.083,"15min":1.083},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91000000000000003,"5min":1.079,"15min":1.079},"last":0.498}]},{"osd":5,"last update":"Fri Jul 21 19:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.752,"15min":0.745},"min":{"1min":0.57599999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":1.044,"15min":1.044},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.006,"5min":1.119,"15min":1.119},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.71299999999999997,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0309999999999999}]},{"osd":8,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69699999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0920000000000001,"5min":1.155,"15min":1.155},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1220000000000001,"5min":1.298,"15min":1.298},"last":1.004}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.95199999999999996}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":42.396000000000001,"5min":42.396000000000001,"15min":42.396000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":42.792000000000002,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":1028859,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.69799999999999995,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.02,"5min":1.028,"15min":1.028},"last":1.0189999999999999}]},{"osd":2,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.32300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63}]},{"osd":4,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94999999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88800000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:04:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.58499999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56399999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93000000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.96099999999999997}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":1.002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.089},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.0660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:05:46.764 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T19:05:46.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 -- 172.21.15.5:0/2107833071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 msgr2=0x7f7ae8101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 --2- 172.21.15.5:0/2107833071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8101140 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f7adc009990 tx=0x7f7adc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 -- 172.21.15.5:0/2107833071 shutdown_connections 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 --2- 172.21.15.5:0/2107833071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 0x7f7ae8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 --2- 172.21.15.5:0/2107833071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 --2- 172.21.15.5:0/2107833071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8101140 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:46.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 -- 172.21.15.5:0/2107833071 >> 172.21.15.5:0/2107833071 conn(0x7f7ae80fb0e0 msgr2=0x7f7ae80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 -- 172.21.15.5:0/2107833071 shutdown_connections 2023-07-21T19:05:47.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.989+0000 7f7aecfff700 1 -- 172.21.15.5:0/2107833071 wait complete. 2023-07-21T19:05:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 Processor -- start 2023-07-21T19:05:47.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 -- start start 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 0x7f7ae8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae80735e0 con 0x7f7ae8100cf0 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae8073750 con 0x7f7ae8101680 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.990+0000 7f7aecfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ae819f7e0 con 0x7f7ae80ffb00 2023-07-21T19:05:47.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38312/0 (socket says 172.21.15.5:38312) 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 -- 172.21.15.5:0/885238969 learned_addr learned my addr 172.21.15.5:0/885238969 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae6d9d700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 0x7f7ae8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae659c700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 -- 172.21.15.5:0/885238969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 msgr2=0x7f7ae80742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 -- 172.21.15.5:0/885238969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 msgr2=0x7f7ae8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 0x7f7ae8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 -- 172.21.15.5:0/885238969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7adc0095f0 con 0x7f7ae8100cf0 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae659c700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.991+0000 7f7ae5d9b700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7adc000c00 tx=0x7f7adc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.992+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc01d070 con 0x7f7ae8100cf0 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.992+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7adc003f40 con 0x7f7ae8100cf0 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.992+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7adc022c70 con 0x7f7ae8100cf0 2023-07-21T19:05:47.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.992+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ae819fa60 con 0x7f7ae8100cf0 2023-07-21T19:05:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.992+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae819fe30 con 0x7f7ae8100cf0 2023-07-21T19:05:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.993+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ae810b1d0 con 0x7f7ae8100cf0 2023-07-21T19:05:47.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.995+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7adc003ab0 con 0x7f7ae8100cf0 2023-07-21T19:05:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.996+0000 7f7ad77fe700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 0x7f7ad00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.996+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7adc0cde20 con 0x7f7ae8100cf0 2023-07-21T19:05:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.996+0000 7f7ae659c700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 0x7f7ad00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:47.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:46.997+0000 7f7ae659c700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 0x7f7ad00a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005950 tx=0x7f7ad80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:47.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.000+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7adc097650 con 0x7f7ae8100cf0 2023-07-21T19:05:47.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.205+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7ae8060950 con 0x7f7ad00a12a0 2023-07-21T19:05:47.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.216+0000 7f7ad77fe700 1 -- 172.21.15.5:0/885238969 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339092 (secure 0 0 0) 0x7f7ae8060950 con 0x7f7ad00a12a0 2023-07-21T19:05:47.229 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:47.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 msgr2=0x7f7ad00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:47.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 0x7f7ad00a3750 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f7ad8005950 tx=0x7f7ad80058e0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 msgr2=0x7f7ae8072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f7adc000c00 tx=0x7f7adc005740 comp rx=0 tx=0).stop 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 shutdown_connections 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7ad00a12a0 0x7f7ad00a3750 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7ae8101680 0x7f7ae8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7ae80ffb00 0x7f7ae80742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 --2- 172.21.15.5:0/885238969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7ae8100cf0 0x7f7ae8072900 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 >> 172.21.15.5:0/885238969 conn(0x7f7ae80fb0e0 msgr2=0x7f7ae80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 shutdown_connections 2023-07-21T19:05:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:47.221+0000 7f7aecfff700 1 -- 172.21.15.5:0/885238969 wait complete. 2023-07-21T19:05:47.233 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:05:47.249 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":304,"stamp":"2023-07-21T19:05:45.313566+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":611196,"num_objects":51,"num_object_clones":0,"num_object_copies":153,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":0,"num_read":163,"num_read_kb":156,"num_write":289,"num_write_kb":2991,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":230,"ondisk_log_size":230,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334488,"kb_used_data":11064,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124607336,"statfs":{"total":1151940427776,"available":1151597912064,"internally_reserved":0,"allocated":11329536,"data_stored":7060404,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":20828,"num_objects":49,"num_object_clones":0,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":0,"num_read":27,"num_read_kb":37,"num_write":108,"num_write_kb":53,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.040186"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712948+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:05:39.712948+0000","last_peered":"2023-07-21T19:05:39.712948+0000","last_clean":"2023-07-21T19:05:39.712948+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:05:39.712948+0000","last_undegraded":"2023-07-21T19:05:39.712948+0000","last_fullsized":"2023-07-21T19:05:39.712948+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:20:58.783182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604930+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:05:39.604930+0000","last_peered":"2023-07-21T19:05:39.604930+0000","last_clean":"2023-07-21T19:05:39.604930+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:05:39.604930+0000","last_undegraded":"2023-07-21T19:05:39.604930+0000","last_fullsized":"2023-07-21T19:05:39.604930+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:56:51.769921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604793+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:05:39.604793+0000","last_peered":"2023-07-21T19:05:39.604793+0000","last_clean":"2023-07-21T19:05:39.604793+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:05:39.604793+0000","last_undegraded":"2023-07-21T19:05:39.604793+0000","last_fullsized":"2023-07-21T19:05:39.604793+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:04:50.978719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605202+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:05:39.605202+0000","last_peered":"2023-07-21T19:05:39.605202+0000","last_clean":"2023-07-21T19:05:39.605202+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:05:39.605202+0000","last_undegraded":"2023-07-21T19:05:39.605202+0000","last_fullsized":"2023-07-21T19:05:39.605202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:59.286011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689477+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:05:38.689477+0000","last_peered":"2023-07-21T19:05:38.689477+0000","last_clean":"2023-07-21T19:05:38.689477+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:05:38.689477+0000","last_undegraded":"2023-07-21T19:05:38.689477+0000","last_fullsized":"2023-07-21T19:05:38.689477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:18.537651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280299+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:05:38.280299+0000","last_peered":"2023-07-21T19:05:38.280299+0000","last_clean":"2023-07-21T19:05:38.280299+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:05:38.280299+0000","last_undegraded":"2023-07-21T19:05:38.280299+0000","last_fullsized":"2023-07-21T19:05:38.280299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:16:54.681325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605701+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:05:39.605701+0000","last_peered":"2023-07-21T19:05:39.605701+0000","last_clean":"2023-07-21T19:05:39.605701+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:05:39.605701+0000","last_undegraded":"2023-07-21T19:05:39.605701+0000","last_fullsized":"2023-07-21T19:05:39.605701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:03:02.351004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605628+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:05:39.605628+0000","last_peered":"2023-07-21T19:05:39.605628+0000","last_clean":"2023-07-21T19:05:39.605628+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:05:39.605628+0000","last_undegraded":"2023-07-21T19:05:39.605628+0000","last_fullsized":"2023-07-21T19:05:39.605628+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:33.230338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606202+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:05:39.606202+0000","last_peered":"2023-07-21T19:05:39.606202+0000","last_clean":"2023-07-21T19:05:39.606202+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:05:39.606202+0000","last_undegraded":"2023-07-21T19:05:39.606202+0000","last_fullsized":"2023-07-21T19:05:39.606202+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:48.724651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280630+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:05:38.280630+0000","last_peered":"2023-07-21T19:05:38.280630+0000","last_clean":"2023-07-21T19:05:38.280630+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:05:38.280630+0000","last_undegraded":"2023-07-21T19:05:38.280630+0000","last_fullsized":"2023-07-21T19:05:38.280630+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:26:56.658392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606006+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:05:39.606006+0000","last_peered":"2023-07-21T19:05:39.606006+0000","last_clean":"2023-07-21T19:05:39.606006+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:05:39.606006+0000","last_undegraded":"2023-07-21T19:05:39.606006+0000","last_fullsized":"2023-07-21T19:05:39.606006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:15:44.329484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605051+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:05:39.605051+0000","last_peered":"2023-07-21T19:05:39.605051+0000","last_clean":"2023-07-21T19:05:39.605051+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:05:39.605051+0000","last_undegraded":"2023-07-21T19:05:39.605051+0000","last_fullsized":"2023-07-21T19:05:39.605051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:50:46.711361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280172+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:05:38.280172+0000","last_peered":"2023-07-21T19:05:38.280172+0000","last_clean":"2023-07-21T19:05:38.280172+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:05:38.280172+0000","last_undegraded":"2023-07-21T19:05:38.280172+0000","last_fullsized":"2023-07-21T19:05:38.280172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:03:52.324459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604436+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:05:39.604436+0000","last_peered":"2023-07-21T19:05:39.604436+0000","last_clean":"2023-07-21T19:05:39.604436+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:05:39.604436+0000","last_undegraded":"2023-07-21T19:05:39.604436+0000","last_fullsized":"2023-07-21T19:05:39.604436+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:38.736877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604319+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:05:39.604319+0000","last_peered":"2023-07-21T19:05:39.604319+0000","last_clean":"2023-07-21T19:05:39.604319+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:05:39.604319+0000","last_undegraded":"2023-07-21T19:05:39.604319+0000","last_fullsized":"2023-07-21T19:05:39.604319+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:11:23.843726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605042+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:05:39.605042+0000","last_peered":"2023-07-21T19:05:39.605042+0000","last_clean":"2023-07-21T19:05:39.605042+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:05:39.605042+0000","last_undegraded":"2023-07-21T19:05:39.605042+0000","last_fullsized":"2023-07-21T19:05:39.605042+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:33.404439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.713006+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:05:39.713006+0000","last_peered":"2023-07-21T19:05:39.713006+0000","last_clean":"2023-07-21T19:05:39.713006+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:05:39.713006+0000","last_undegraded":"2023-07-21T19:05:39.713006+0000","last_fullsized":"2023-07-21T19:05:39.713006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:02:06.348958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605178+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:05:39.605178+0000","last_peered":"2023-07-21T19:05:39.605178+0000","last_clean":"2023-07-21T19:05:39.605178+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:05:39.605178+0000","last_undegraded":"2023-07-21T19:05:39.605178+0000","last_fullsized":"2023-07-21T19:05:39.605178+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:11.975218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982978+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:05:38.982978+0000","last_peered":"2023-07-21T19:05:38.982978+0000","last_clean":"2023-07-21T19:05:38.982978+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:05:38.982978+0000","last_undegraded":"2023-07-21T19:05:38.982978+0000","last_fullsized":"2023-07-21T19:05:38.982978+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:27:41.530811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605306+0000","last_change":"2023-07-21T19:05:36.693433+0000","last_active":"2023-07-21T19:05:39.605306+0000","last_peered":"2023-07-21T19:05:39.605306+0000","last_clean":"2023-07-21T19:05:39.605306+0000","last_became_active":"2023-07-21T19:05:36.693131+0000","last_became_peered":"2023-07-21T19:05:36.693131+0000","last_unstale":"2023-07-21T19:05:39.605306+0000","last_undegraded":"2023-07-21T19:05:39.605306+0000","last_fullsized":"2023-07-21T19:05:39.605306+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:30.985725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605632+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:05:39.605632+0000","last_peered":"2023-07-21T19:05:39.605632+0000","last_clean":"2023-07-21T19:05:39.605632+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:05:39.605632+0000","last_undegraded":"2023-07-21T19:05:39.605632+0000","last_fullsized":"2023-07-21T19:05:39.605632+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:14.093922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605883+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:05:39.605883+0000","last_peered":"2023-07-21T19:05:39.605883+0000","last_clean":"2023-07-21T19:05:39.605883+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:05:39.605883+0000","last_undegraded":"2023-07-21T19:05:39.605883+0000","last_fullsized":"2023-07-21T19:05:39.605883+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:12:05.016166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604693+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:05:39.604693+0000","last_peered":"2023-07-21T19:05:39.604693+0000","last_clean":"2023-07-21T19:05:39.604693+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:05:39.604693+0000","last_undegraded":"2023-07-21T19:05:39.604693+0000","last_fullsized":"2023-07-21T19:05:39.604693+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:23:18.293241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.117934+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:05:42.117934+0000","last_peered":"2023-07-21T19:05:42.117934+0000","last_clean":"2023-07-21T19:05:42.117934+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:05:42.117934+0000","last_undegraded":"2023-07-21T19:05:42.117934+0000","last_fullsized":"2023-07-21T19:05:42.117934+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:44:39.807659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605492+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:05:39.605492+0000","last_peered":"2023-07-21T19:05:39.605492+0000","last_clean":"2023-07-21T19:05:39.605492+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:05:39.605492+0000","last_undegraded":"2023-07-21T19:05:39.605492+0000","last_fullsized":"2023-07-21T19:05:39.605492+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:55:31.296842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689751+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:05:38.689751+0000","last_peered":"2023-07-21T19:05:38.689751+0000","last_clean":"2023-07-21T19:05:38.689751+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:05:38.689751+0000","last_undegraded":"2023-07-21T19:05:38.689751+0000","last_fullsized":"2023-07-21T19:05:38.689751+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:57:10.911304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280818+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:05:38.280818+0000","last_peered":"2023-07-21T19:05:38.280818+0000","last_clean":"2023-07-21T19:05:38.280818+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:05:38.280818+0000","last_undegraded":"2023-07-21T19:05:38.280818+0000","last_fullsized":"2023-07-21T19:05:38.280818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:38:34.631965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280499+0000","last_change":"2023-07-21T19:05:36.700941+0000","last_active":"2023-07-21T19:05:38.280499+0000","last_peered":"2023-07-21T19:05:38.280499+0000","last_clean":"2023-07-21T19:05:38.280499+0000","last_became_active":"2023-07-21T19:05:36.700646+0000","last_became_peered":"2023-07-21T19:05:36.700646+0000","last_unstale":"2023-07-21T19:05:38.280499+0000","last_undegraded":"2023-07-21T19:05:38.280499+0000","last_fullsized":"2023-07-21T19:05:38.280499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:09:33.485772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689650+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:05:38.689650+0000","last_peered":"2023-07-21T19:05:38.689650+0000","last_clean":"2023-07-21T19:05:38.689650+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:05:38.689650+0000","last_undegraded":"2023-07-21T19:05:38.689650+0000","last_fullsized":"2023-07-21T19:05:38.689650+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:19:36.124026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.116831+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:05:42.116831+0000","last_peered":"2023-07-21T19:05:42.116831+0000","last_clean":"2023-07-21T19:05:42.116831+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:05:42.116831+0000","last_undegraded":"2023-07-21T19:05:42.116831+0000","last_fullsized":"2023-07-21T19:05:42.116831+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:01:03.552766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280612+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:05:38.280612+0000","last_peered":"2023-07-21T19:05:38.280612+0000","last_clean":"2023-07-21T19:05:38.280612+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:05:38.280612+0000","last_undegraded":"2023-07-21T19:05:38.280612+0000","last_fullsized":"2023-07-21T19:05:38.280612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:03:58.158529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.119227+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:05:42.119227+0000","last_peered":"2023-07-21T19:05:42.119227+0000","last_clean":"2023-07-21T19:05:42.119227+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:05:42.119227+0000","last_undegraded":"2023-07-21T19:05:42.119227+0000","last_fullsized":"2023-07-21T19:05:42.119227+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:54:26.215439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604821+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:05:39.604821+0000","last_peered":"2023-07-21T19:05:39.604821+0000","last_clean":"2023-07-21T19:05:39.604821+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:05:39.604821+0000","last_undegraded":"2023-07-21T19:05:39.604821+0000","last_fullsized":"2023-07-21T19:05:39.604821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:42:47.855231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280263+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:05:38.280263+0000","last_peered":"2023-07-21T19:05:38.280263+0000","last_clean":"2023-07-21T19:05:38.280263+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:05:38.280263+0000","last_undegraded":"2023-07-21T19:05:38.280263+0000","last_fullsized":"2023-07-21T19:05:38.280263+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:39:06.512567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605247+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:05:39.605247+0000","last_peered":"2023-07-21T19:05:39.605247+0000","last_clean":"2023-07-21T19:05:39.605247+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:05:39.605247+0000","last_undegraded":"2023-07-21T19:05:39.605247+0000","last_fullsized":"2023-07-21T19:05:39.605247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:22.034088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":20,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:43.287499+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:05:43.287499+0000","last_peered":"2023-07-21T19:05:43.287499+0000","last_clean":"2023-07-21T19:05:43.287499+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:05:43.287499+0000","last_undegraded":"2023-07-21T19:05:43.287499+0000","last_fullsized":"2023-07-21T19:05:43.287499+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:31:18.540576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605933+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:05:39.605933+0000","last_peered":"2023-07-21T19:05:39.605933+0000","last_clean":"2023-07-21T19:05:39.605933+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:05:39.605933+0000","last_undegraded":"2023-07-21T19:05:39.605933+0000","last_fullsized":"2023-07-21T19:05:39.605933+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:10:53.349885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109359+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:05:41.109359+0000","last_peered":"2023-07-21T19:05:41.109359+0000","last_clean":"2023-07-21T19:05:41.109359+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:05:41.109359+0000","last_undegraded":"2023-07-21T19:05:41.109359+0000","last_fullsized":"2023-07-21T19:05:41.109359+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:51:26.052671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604833+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:05:39.604833+0000","last_peered":"2023-07-21T19:05:39.604833+0000","last_clean":"2023-07-21T19:05:39.604833+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:05:39.604833+0000","last_undegraded":"2023-07-21T19:05:39.604833+0000","last_fullsized":"2023-07-21T19:05:39.604833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:51:33.432465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280877+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:05:38.280877+0000","last_peered":"2023-07-21T19:05:38.280877+0000","last_clean":"2023-07-21T19:05:38.280877+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:05:38.280877+0000","last_undegraded":"2023-07-21T19:05:38.280877+0000","last_fullsized":"2023-07-21T19:05:38.280877+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:47:35.842718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605701+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:05:39.605701+0000","last_peered":"2023-07-21T19:05:39.605701+0000","last_clean":"2023-07-21T19:05:39.605701+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:05:39.605701+0000","last_undegraded":"2023-07-21T19:05:39.605701+0000","last_fullsized":"2023-07-21T19:05:39.605701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:19:42.213098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"85'14","reported_seq":37,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:44.290668+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:05:44.290668+0000","last_peered":"2023-07-21T19:05:44.290668+0000","last_clean":"2023-07-21T19:05:44.290668+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:05:44.290668+0000","last_undegraded":"2023-07-21T19:05:44.290668+0000","last_fullsized":"2023-07-21T19:05:44.290668+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:49:58.331772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18929,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":9,"num_read_kb":22,"num_write":15,"num_write_kb":21,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689721+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:05:38.689721+0000","last_peered":"2023-07-21T19:05:38.689721+0000","last_clean":"2023-07-21T19:05:38.689721+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:05:38.689721+0000","last_undegraded":"2023-07-21T19:05:38.689721+0000","last_fullsized":"2023-07-21T19:05:38.689721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:19:16.744848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.110784+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:05:41.110784+0000","last_peered":"2023-07-21T19:05:41.110784+0000","last_clean":"2023-07-21T19:05:41.110784+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:05:41.110784+0000","last_undegraded":"2023-07-21T19:05:41.110784+0000","last_fullsized":"2023-07-21T19:05:41.110784+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:35:37.953554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279935+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:05:38.279935+0000","last_peered":"2023-07-21T19:05:38.279935+0000","last_clean":"2023-07-21T19:05:38.279935+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:05:38.279935+0000","last_undegraded":"2023-07-21T19:05:38.279935+0000","last_fullsized":"2023-07-21T19:05:38.279935+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:50.852919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280617+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:05:38.280617+0000","last_peered":"2023-07-21T19:05:38.280617+0000","last_clean":"2023-07-21T19:05:38.280617+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:05:38.280617+0000","last_undegraded":"2023-07-21T19:05:38.280617+0000","last_fullsized":"2023-07-21T19:05:38.280617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:28.567411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605040+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:05:39.605040+0000","last_peered":"2023-07-21T19:05:39.605040+0000","last_clean":"2023-07-21T19:05:39.605040+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:05:39.605040+0000","last_undegraded":"2023-07-21T19:05:39.605040+0000","last_fullsized":"2023-07-21T19:05:39.605040+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:07:06.205378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:43.289064+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:05:43.289064+0000","last_peered":"2023-07-21T19:05:43.289064+0000","last_clean":"2023-07-21T19:05:43.289064+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:05:43.289064+0000","last_undegraded":"2023-07-21T19:05:43.289064+0000","last_fullsized":"2023-07-21T19:05:43.289064+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:14:12.044002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604751+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:05:39.604751+0000","last_peered":"2023-07-21T19:05:39.604751+0000","last_clean":"2023-07-21T19:05:39.604751+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:05:39.604751+0000","last_undegraded":"2023-07-21T19:05:39.604751+0000","last_fullsized":"2023-07-21T19:05:39.604751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:53:18.697494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":30,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.119906+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:05:42.119906+0000","last_peered":"2023-07-21T19:05:42.119906+0000","last_clean":"2023-07-21T19:05:42.119906+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:05:42.119906+0000","last_undegraded":"2023-07-21T19:05:42.119906+0000","last_fullsized":"2023-07-21T19:05:42.119906+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:43.481871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280462+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:05:38.280462+0000","last_peered":"2023-07-21T19:05:38.280462+0000","last_clean":"2023-07-21T19:05:38.280462+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:05:38.280462+0000","last_undegraded":"2023-07-21T19:05:38.280462+0000","last_fullsized":"2023-07-21T19:05:38.280462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:28:02.437526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:40.273949+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:05:40.273949+0000","last_peered":"2023-07-21T19:05:40.273949+0000","last_clean":"2023-07-21T19:05:40.273949+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:05:40.273949+0000","last_undegraded":"2023-07-21T19:05:40.273949+0000","last_fullsized":"2023-07-21T19:05:40.273949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:04:15.919453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712781+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:05:39.712781+0000","last_peered":"2023-07-21T19:05:39.712781+0000","last_clean":"2023-07-21T19:05:39.712781+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:05:39.712781+0000","last_undegraded":"2023-07-21T19:05:39.712781+0000","last_fullsized":"2023-07-21T19:05:39.712781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:13:24.995798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605657+0000","last_change":"2023-07-21T19:05:36.700605+0000","last_active":"2023-07-21T19:05:39.605657+0000","last_peered":"2023-07-21T19:05:39.605657+0000","last_clean":"2023-07-21T19:05:39.605657+0000","last_became_active":"2023-07-21T19:05:36.700117+0000","last_became_peered":"2023-07-21T19:05:36.700117+0000","last_unstale":"2023-07-21T19:05:39.605657+0000","last_undegraded":"2023-07-21T19:05:39.605657+0000","last_fullsized":"2023-07-21T19:05:39.605657+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:14:30.993635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":13,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.695096+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:05:39.695096+0000","last_peered":"2023-07-21T19:05:39.695096+0000","last_clean":"2023-07-21T19:05:39.695096+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:05:39.695096+0000","last_undegraded":"2023-07-21T19:05:39.695096+0000","last_fullsized":"2023-07-21T19:05:39.695096+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:56:02.714363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605465+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:05:39.605465+0000","last_peered":"2023-07-21T19:05:39.605465+0000","last_clean":"2023-07-21T19:05:39.605465+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:05:39.605465+0000","last_undegraded":"2023-07-21T19:05:39.605465+0000","last_fullsized":"2023-07-21T19:05:39.605465+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:02.691852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280255+0000","last_change":"2023-07-21T19:05:37.301107+0000","last_active":"2023-07-21T19:05:38.280255+0000","last_peered":"2023-07-21T19:05:38.280255+0000","last_clean":"2023-07-21T19:05:38.280255+0000","last_became_active":"2023-07-21T19:05:37.300547+0000","last_became_peered":"2023-07-21T19:05:37.300547+0000","last_unstale":"2023-07-21T19:05:38.280255+0000","last_undegraded":"2023-07-21T19:05:38.280255+0000","last_fullsized":"2023-07-21T19:05:38.280255+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:15:08.496017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280532+0000","last_change":"2023-07-21T19:05:37.295173+0000","last_active":"2023-07-21T19:05:38.280532+0000","last_peered":"2023-07-21T19:05:38.280532+0000","last_clean":"2023-07-21T19:05:38.280532+0000","last_became_active":"2023-07-21T19:05:37.294425+0000","last_became_peered":"2023-07-21T19:05:37.294425+0000","last_unstale":"2023-07-21T19:05:38.280532+0000","last_undegraded":"2023-07-21T19:05:38.280532+0000","last_fullsized":"2023-07-21T19:05:38.280532+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:47:32.039905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.120153+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:05:42.120153+0000","last_peered":"2023-07-21T19:05:42.120153+0000","last_clean":"2023-07-21T19:05:42.120153+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:05:42.120153+0000","last_undegraded":"2023-07-21T19:05:42.120153+0000","last_fullsized":"2023-07-21T19:05:42.120153+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:09:44.502330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605429+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:05:39.605429+0000","last_peered":"2023-07-21T19:05:39.605429+0000","last_clean":"2023-07-21T19:05:39.605429+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:05:39.605429+0000","last_undegraded":"2023-07-21T19:05:39.605429+0000","last_fullsized":"2023-07-21T19:05:39.605429+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:21:41.882514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.281744+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:05:42.281744+0000","last_peered":"2023-07-21T19:05:42.281744+0000","last_clean":"2023-07-21T19:05:42.281744+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:05:42.281744+0000","last_undegraded":"2023-07-21T19:05:42.281744+0000","last_fullsized":"2023-07-21T19:05:42.281744+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:34:12.412105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":344,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604883+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:05:39.604883+0000","last_peered":"2023-07-21T19:05:39.604883+0000","last_clean":"2023-07-21T19:05:39.604883+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:05:39.604883+0000","last_undegraded":"2023-07-21T19:05:39.604883+0000","last_fullsized":"2023-07-21T19:05:39.604883+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:02.966802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279621+0000","last_change":"2023-07-21T19:05:37.288734+0000","last_active":"2023-07-21T19:05:38.279621+0000","last_peered":"2023-07-21T19:05:38.279621+0000","last_clean":"2023-07-21T19:05:38.279621+0000","last_became_active":"2023-07-21T19:05:37.288382+0000","last_became_peered":"2023-07-21T19:05:37.288382+0000","last_unstale":"2023-07-21T19:05:38.279621+0000","last_undegraded":"2023-07-21T19:05:38.279621+0000","last_fullsized":"2023-07-21T19:05:38.279621+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:52:31.362689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:42.117964+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:05:42.117964+0000","last_peered":"2023-07-21T19:05:42.117964+0000","last_clean":"2023-07-21T19:05:42.117964+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:05:42.117964+0000","last_undegraded":"2023-07-21T19:05:42.117964+0000","last_fullsized":"2023-07-21T19:05:42.117964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:47:59.441291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.107930+0000","last_change":"2023-07-21T19:05:36.704381+0000","last_active":"2023-07-21T19:05:41.107930+0000","last_peered":"2023-07-21T19:05:41.107930+0000","last_clean":"2023-07-21T19:05:41.107930+0000","last_became_active":"2023-07-21T19:05:36.703664+0000","last_became_peered":"2023-07-21T19:05:36.703664+0000","last_unstale":"2023-07-21T19:05:41.107930+0000","last_undegraded":"2023-07-21T19:05:41.107930+0000","last_fullsized":"2023-07-21T19:05:41.107930+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:50:43.215721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279992+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:05:38.279992+0000","last_peered":"2023-07-21T19:05:38.279992+0000","last_clean":"2023-07-21T19:05:38.279992+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:05:38.279992+0000","last_undegraded":"2023-07-21T19:05:38.279992+0000","last_fullsized":"2023-07-21T19:05:38.279992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:26:59.069236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605034+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:05:39.605034+0000","last_peered":"2023-07-21T19:05:39.605034+0000","last_clean":"2023-07-21T19:05:39.605034+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:05:39.605034+0000","last_undegraded":"2023-07-21T19:05:39.605034+0000","last_fullsized":"2023-07-21T19:05:39.605034+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:55:58.518652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605239+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:05:39.605239+0000","last_peered":"2023-07-21T19:05:39.605239+0000","last_clean":"2023-07-21T19:05:39.605239+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:05:39.605239+0000","last_undegraded":"2023-07-21T19:05:39.605239+0000","last_fullsized":"2023-07-21T19:05:39.605239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:06.818995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":18,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109869+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:05:41.109869+0000","last_peered":"2023-07-21T19:05:41.109869+0000","last_clean":"2023-07-21T19:05:41.109869+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:05:41.109869+0000","last_undegraded":"2023-07-21T19:05:41.109869+0000","last_fullsized":"2023-07-21T19:05:41.109869+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:57:39.637910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605058+0000","last_change":"2023-07-21T19:05:37.298964+0000","last_active":"2023-07-21T19:05:39.605058+0000","last_peered":"2023-07-21T19:05:39.605058+0000","last_clean":"2023-07-21T19:05:39.605058+0000","last_became_active":"2023-07-21T19:05:37.298675+0000","last_became_peered":"2023-07-21T19:05:37.298675+0000","last_unstale":"2023-07-21T19:05:39.605058+0000","last_undegraded":"2023-07-21T19:05:39.605058+0000","last_fullsized":"2023-07-21T19:05:39.605058+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T07:02:30.050668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.108896+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:05:41.108896+0000","last_peered":"2023-07-21T19:05:41.108896+0000","last_clean":"2023-07-21T19:05:41.108896+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:05:41.108896+0000","last_undegraded":"2023-07-21T19:05:41.108896+0000","last_fullsized":"2023-07-21T19:05:41.108896+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:20:09.881230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.689437+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:05:38.689437+0000","last_peered":"2023-07-21T19:05:38.689437+0000","last_clean":"2023-07-21T19:05:38.689437+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:05:38.689437+0000","last_undegraded":"2023-07-21T19:05:38.689437+0000","last_fullsized":"2023-07-21T19:05:38.689437+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:45.813738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":17,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.109600+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:05:41.109600+0000","last_peered":"2023-07-21T19:05:41.109600+0000","last_clean":"2023-07-21T19:05:41.109600+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:05:41.109600+0000","last_undegraded":"2023-07-21T19:05:41.109600+0000","last_fullsized":"2023-07-21T19:05:41.109600+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:50:30.177296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712624+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:05:39.712624+0000","last_peered":"2023-07-21T19:05:39.712624+0000","last_clean":"2023-07-21T19:05:39.712624+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:05:39.712624+0000","last_undegraded":"2023-07-21T19:05:39.712624+0000","last_fullsized":"2023-07-21T19:05:39.712624+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:49:34.364930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982671+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:05:38.982671+0000","last_peered":"2023-07-21T19:05:38.982671+0000","last_clean":"2023-07-21T19:05:38.982671+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:05:38.982671+0000","last_undegraded":"2023-07-21T19:05:38.982671+0000","last_fullsized":"2023-07-21T19:05:38.982671+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:41:00.688642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604734+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:05:39.604734+0000","last_peered":"2023-07-21T19:05:39.604734+0000","last_clean":"2023-07-21T19:05:39.604734+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:05:39.604734+0000","last_undegraded":"2023-07-21T19:05:39.604734+0000","last_fullsized":"2023-07-21T19:05:39.604734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:47.291733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":19,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:41.110565+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:05:41.110565+0000","last_peered":"2023-07-21T19:05:41.110565+0000","last_clean":"2023-07-21T19:05:41.110565+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:05:41.110565+0000","last_undegraded":"2023-07-21T19:05:41.110565+0000","last_fullsized":"2023-07-21T19:05:41.110565+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:15:25.607217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605626+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:05:39.605626+0000","last_peered":"2023-07-21T19:05:39.605626+0000","last_clean":"2023-07-21T19:05:39.605626+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:05:39.605626+0000","last_undegraded":"2023-07-21T19:05:39.605626+0000","last_fullsized":"2023-07-21T19:05:39.605626+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:26:45.220492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.983010+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:05:38.983010+0000","last_peered":"2023-07-21T19:05:38.983010+0000","last_clean":"2023-07-21T19:05:38.983010+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:05:38.983010+0000","last_undegraded":"2023-07-21T19:05:38.983010+0000","last_fullsized":"2023-07-21T19:05:38.983010+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:17:20.807340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.606031+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:05:39.606031+0000","last_peered":"2023-07-21T19:05:39.606031+0000","last_clean":"2023-07-21T19:05:39.606031+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:05:39.606031+0000","last_undegraded":"2023-07-21T19:05:39.606031+0000","last_fullsized":"2023-07-21T19:05:39.606031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:23.107616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280752+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:05:38.280752+0000","last_peered":"2023-07-21T19:05:38.280752+0000","last_clean":"2023-07-21T19:05:38.280752+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:05:38.280752+0000","last_undegraded":"2023-07-21T19:05:38.280752+0000","last_fullsized":"2023-07-21T19:05:38.280752+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:19.029881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605134+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:05:39.605134+0000","last_peered":"2023-07-21T19:05:39.605134+0000","last_clean":"2023-07-21T19:05:39.605134+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:05:39.605134+0000","last_undegraded":"2023-07-21T19:05:39.605134+0000","last_fullsized":"2023-07-21T19:05:39.605134+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:36.288298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":16,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280107+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:05:38.280107+0000","last_peered":"2023-07-21T19:05:38.280107+0000","last_clean":"2023-07-21T19:05:38.280107+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:05:38.280107+0000","last_undegraded":"2023-07-21T19:05:38.280107+0000","last_fullsized":"2023-07-21T19:05:38.280107+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:36:39.312004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605487+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:05:39.605487+0000","last_peered":"2023-07-21T19:05:39.605487+0000","last_clean":"2023-07-21T19:05:39.605487+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:05:39.605487+0000","last_undegraded":"2023-07-21T19:05:39.605487+0000","last_fullsized":"2023-07-21T19:05:39.605487+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:12:38.196292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712706+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:05:39.712706+0000","last_peered":"2023-07-21T19:05:39.712706+0000","last_clean":"2023-07-21T19:05:39.712706+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:05:39.712706+0000","last_undegraded":"2023-07-21T19:05:39.712706+0000","last_fullsized":"2023-07-21T19:05:39.712706+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:10:37.860659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982908+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:05:38.982908+0000","last_peered":"2023-07-21T19:05:38.982908+0000","last_clean":"2023-07-21T19:05:38.982908+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:05:38.982908+0000","last_undegraded":"2023-07-21T19:05:38.982908+0000","last_fullsized":"2023-07-21T19:05:38.982908+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:56:14.208338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.982743+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:05:38.982743+0000","last_peered":"2023-07-21T19:05:38.982743+0000","last_clean":"2023-07-21T19:05:38.982743+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:05:38.982743+0000","last_undegraded":"2023-07-21T19:05:38.982743+0000","last_fullsized":"2023-07-21T19:05:38.982743+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:42:39.432370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.604712+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:05:39.604712+0000","last_peered":"2023-07-21T19:05:39.604712+0000","last_clean":"2023-07-21T19:05:39.604712+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:05:39.604712+0000","last_undegraded":"2023-07-21T19:05:39.604712+0000","last_fullsized":"2023-07-21T19:05:39.604712+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:01:46.864454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280015+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:05:38.280015+0000","last_peered":"2023-07-21T19:05:38.280015+0000","last_clean":"2023-07-21T19:05:38.280015+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:05:38.280015+0000","last_undegraded":"2023-07-21T19:05:38.280015+0000","last_fullsized":"2023-07-21T19:05:38.280015+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:00:13.542991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712912+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:05:39.712912+0000","last_peered":"2023-07-21T19:05:39.712912+0000","last_clean":"2023-07-21T19:05:39.712912+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:05:39.712912+0000","last_undegraded":"2023-07-21T19:05:39.712912+0000","last_fullsized":"2023-07-21T19:05:39.712912+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:27:30.166821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.983143+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:05:38.983143+0000","last_peered":"2023-07-21T19:05:38.983143+0000","last_clean":"2023-07-21T19:05:38.983143+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:05:38.983143+0000","last_undegraded":"2023-07-21T19:05:38.983143+0000","last_fullsized":"2023-07-21T19:05:38.983143+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:34:45.714573+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605089+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:05:39.605089+0000","last_peered":"2023-07-21T19:05:39.605089+0000","last_clean":"2023-07-21T19:05:39.605089+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:05:39.605089+0000","last_undegraded":"2023-07-21T19:05:39.605089+0000","last_fullsized":"2023-07-21T19:05:39.605089+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:32:37.506788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.280427+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:05:38.280427+0000","last_peered":"2023-07-21T19:05:38.280427+0000","last_clean":"2023-07-21T19:05:38.280427+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:05:38.280427+0000","last_undegraded":"2023-07-21T19:05:38.280427+0000","last_fullsized":"2023-07-21T19:05:38.280427+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:31:36.193709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605141+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:05:39.605141+0000","last_peered":"2023-07-21T19:05:39.605141+0000","last_clean":"2023-07-21T19:05:39.605141+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:05:39.605141+0000","last_undegraded":"2023-07-21T19:05:39.605141+0000","last_fullsized":"2023-07-21T19:05:39.605141+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:24:57.943527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.605726+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:05:39.605726+0000","last_peered":"2023-07-21T19:05:39.605726+0000","last_clean":"2023-07-21T19:05:39.605726+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:05:39.605726+0000","last_undegraded":"2023-07-21T19:05:39.605726+0000","last_fullsized":"2023-07-21T19:05:39.605726+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:47:50.017188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:38.279990+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:05:38.279990+0000","last_peered":"2023-07-21T19:05:38.279990+0000","last_clean":"2023-07-21T19:05:38.279990+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:05:38.279990+0000","last_undegraded":"2023-07-21T19:05:38.279990+0000","last_fullsized":"2023-07-21T19:05:38.279990+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:14:13.559642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":12,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-07-21T19:05:39.712741+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:05:39.712741+0000","last_peered":"2023-07-21T19:05:39.712741+0000","last_clean":"2023-07-21T19:05:39.712741+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:05:39.712741+0000","last_undegraded":"2023-07-21T19:05:39.712741+0000","last_fullsized":"2023-07-21T19:05:39.712741+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:28:36.652413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":20705,"num_objects":48,"num_object_clones":0,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":0,"num_read":27,"num_read_kb":37,"num_write":107,"num_write_kb":52,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":32,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":225280,"data_stored":30043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383694,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27844,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717308,"statfs":{"total":95995035648,"available":95966523392,"internally_reserved":0,"allocated":913408,"data_stored":438309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":1.131}]},{"osd":1,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":1.4279999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":1.0660000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.395},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":1.1859999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":1.208},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":1.4570000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":1.24},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.3100000000000001}]},{"osd":7,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":1.595},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":1.5660000000000001}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.262}]},{"osd":9,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":1.3779999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.522}]},{"osd":10,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":1.615},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":1.681}]}]},{"osd":10,"up_from":72,"seq":309237645332,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":1004,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":1028096,"data_stored":1049259,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":1,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.437}]},{"osd":2,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.028,"5min":1.028,"15min":1.028},"last":0.624},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.206,"5min":1.206,"15min":1.206},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2569999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.75800000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.222,"5min":1.222,"15min":1.222},"last":0.76800000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.171,"5min":1.171,"15min":1.171},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.86599999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.6679999999999999,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.88500000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.625,"5min":0.625,"15min":0.625},"max":{"1min":2.0169999999999999,"5min":2.0169999999999999,"15min":2.0169999999999999},"last":0.85199999999999998}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.82799999999999996}]}]},{"osd":9,"up_from":67,"seq":287762808857,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439093,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.307},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.436}]},{"osd":1,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.81799999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.63},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.66100000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.501},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.52200000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1970000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.64900000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.93100000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.508,"5min":1.508,"15min":1.508},"last":0.67700000000000005}]},{"osd":6,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.60599999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"min":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"max":{"1min":1.3120000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":1.04,"5min":1.04,"15min":1.04},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.5800000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.83399999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"min":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"max":{"1min":1.4079999999999999,"5min":1.4079999999999999,"15min":1.4079999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"min":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"max":{"1min":1.446,"5min":1.446,"15min":1.446},"last":0.69899999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.86499999999999999},{"interface":"front","average":{"1min":1.054,"5min":1.054,"15min":1.054},"min":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"max":{"1min":1.391,"5min":1.391,"15min":1.391},"last":0.751}]}]},{"osd":8,"up_from":59,"seq":253403070494,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":933888,"data_stored":439047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.95699999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":1.1379999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.90600000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.746},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.78200000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":1.1870000000000001},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":1.1719999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.169,"15min":1.169},"last":1.0920000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.109,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.032},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":1.2150000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":1.2050000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.86699999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.359,"15min":1.359},"last":1.117}]},{"osd":7,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84999999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0680000000000001,"5min":1.403,"15min":1.403},"last":1.157},{"interface":"front","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.65900000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":1.3160000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":1.246},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":1.238}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.27},{"interface":"front","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":1.2829999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.292}]}]},{"osd":7,"up_from":51,"seq":219043332131,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27908,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717244,"statfs":{"total":95995035648,"available":95966457856,"internally_reserved":0,"allocated":978944,"data_stored":1030373,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.36799999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.64800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.69299999999999995,"15min":0.69699999999999995},"min":{"1min":0.50700000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.753},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.72199999999999998,"15min":0.71999999999999997},"min":{"1min":0.54400000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.0369999999999999,"5min":1.05,"15min":1.05},"last":0.76800000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.71199999999999997,"15min":0.69999999999999996},"min":{"1min":0.54900000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.96099999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.68000000000000005,"15min":0.67600000000000005},"min":{"1min":0.44900000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.85999999999999999,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.877}]},{"osd":4,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.628,"15min":0.626},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.627,"15min":0.63200000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.62}]},{"osd":5,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":1.002,"15min":1.044},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.133,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77000000000000002,"15min":0.76400000000000001},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1850000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.67100000000000004,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":0.91000000000000003,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.86399999999999999,"15min":0.86699999999999999},"min":{"1min":0.67200000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.05,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.79300000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.80300000000000005,"15min":0.78900000000000003},"min":{"1min":0.58499999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.68600000000000005,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.90400000000000003}]},{"osd":9,"last update":"Fri Jul 21 19:04:50 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.0800000000000001,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.99299999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:05:14 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.93999999999999995},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.113,"5min":1.113,"15min":1.113},"last":0.96099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.008}]}]},{"osd":6,"up_from":46,"seq":197568495656,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":917504,"data_stored":438399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.42999999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95199999999999996,"5min":1.087,"15min":1.087},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.71499999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.018,"5min":1.149,"15min":1.149},"last":0.59999999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.04,"15min":1.04},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0669999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.84099999999999997}]},{"osd":5,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.67600000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:05:01 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.83999999999999997,"15min":0.82099999999999995},"min":{"1min":0.66100000000000003,"5min":0.629,"15min":0.629},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.96699999999999997,"5min":0.85799999999999998,"15min":0.83999999999999997},"min":{"1min":0.71099999999999997,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.79200000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.82499999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.984,"5min":1.984,"15min":1.984},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.754,"5min":1.754,"15min":1.754},"last":0.623}]},{"osd":10,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"max":{"1min":1.498,"5min":1.498,"15min":1.498},"last":0.88},{"interface":"front","average":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.923,"5min":1.923,"15min":1.923},"last":0.80600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574915,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":439873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:04:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61599999999999999},"min":{"1min":0.42299999999999999,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79800000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.63800000000000001,"15min":0.61899999999999999},"min":{"1min":0.47999999999999998,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.97899999999999998,"5min":1.054,"15min":1.054},"last":0.67000000000000004}]},{"osd":3,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.67800000000000005,"15min":0.65600000000000003},"min":{"1min":0.47699999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.52900000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.70699999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:04:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.51800000000000002,"15min":0.50900000000000001},"min":{"1min":0.35199999999999998,"5min":0.254,"15min":0.254},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.51800000000000002,"15min":0.496},"min":{"1min":0.32400000000000001,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.48999999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65200000000000002,"15min":0.63600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.69299999999999995,"15min":0.68000000000000005},"min":{"1min":0.53000000000000003,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.86599999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.048,"15min":1.048},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.81499999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:04:54 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.71799999999999997,"15min":0.71599999999999997},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.95799999999999996,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68400000000000005,"15min":0.67800000000000005},"min":{"1min":0.45900000000000002,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.92500000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:22 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82899999999999996,"15min":0.81999999999999995},"min":{"1min":0.67500000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.82099999999999995,"15min":0.82299999999999995},"min":{"1min":0.60999999999999999,"5min":0.496,"15min":0.496},"max":{"1min":0.98899999999999999,"5min":1.502,"15min":1.502},"last":0.75}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.149,"5min":1.149,"15min":1.149},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.90100000000000002}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.1839999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":0.88200000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82899999999999996}]}]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":942080,"data_stored":439787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0369999999999999,"5min":1.177,"15min":1.177},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96599999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69299999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.499,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.0169999999999999,"5min":1.139,"15min":1.139},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.66900000000000004,"15min":0.66400000000000003},"min":{"1min":0.503,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.93300000000000005,"5min":1.113,"15min":1.113},"last":0.58599999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.372},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.503}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84399999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61099999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.73699999999999999,"15min":0.72699999999999998},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.51200000000000001,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.91400000000000003,"5min":1.042,"15min":1.042},"last":0.77300000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93999999999999995,"5min":1.163,"15min":1.163},"last":0.55200000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.032,"5min":1.304,"15min":1.304},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.748,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.83299999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":0.626}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.75900000000000001}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.88},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.85099999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411389,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":439907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59299999999999997,"15min":0.59099999999999997},"min":{"1min":0.38900000000000001,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.91400000000000003,"5min":0.91400000000000003,"15min":0.91400000000000003},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.58499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39300000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.88500000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:04:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.628,"15min":0.59799999999999998},"min":{"1min":0.48399999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.59899999999999998},"min":{"1min":0.45000000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.081,"15min":1.081},"last":0.502}]},{"osd":3,"last update":"Fri Jul 21 19:05:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68600000000000005,"15min":0.66400000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.04,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.67300000000000004,"15min":0.65100000000000002},"min":{"1min":0.48299999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.92500000000000004,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.92900000000000005}]},{"osd":4,"last update":"Fri Jul 21 19:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.76500000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:08 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.51900000000000002},"min":{"1min":0.42499999999999999,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.93200000000000005,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.745},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.55500000000000005,"15min":0.52700000000000002},"min":{"1min":0.38200000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.78500000000000003,"5min":1.167,"15min":1.167},"last":0.67300000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.58199999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0609999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.95799999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.98099999999999998}]},{"osd":9,"last update":"Fri Jul 21 19:04:47 2023","interfaces":[{"interface":"back","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.333,"5min":1.333,"15min":1.333},"last":0.873},{"interface":"front","average":{"1min":0.873,"5min":0.873,"15min":0.873},"min":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":1.008}]},{"osd":10,"last update":"Fri Jul 21 19:05:17 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80100000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.03}]}]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27860,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717292,"statfs":{"total":95995035648,"available":95966507008,"internally_reserved":0,"allocated":929792,"data_stored":438593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.441,"15min":0.41299999999999998},"min":{"1min":0.32000000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.63100000000000001,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.46700000000000003,"15min":0.441},"min":{"1min":0.32900000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.67100000000000004,"15min":0.65700000000000003},"min":{"1min":0.46800000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.66300000000000003,"15min":0.64900000000000002},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.67700000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:05:11 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.67900000000000005,"15min":0.63200000000000001},"min":{"1min":0.53900000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.66700000000000004,"15min":0.628},"min":{"1min":0.55000000000000004,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.98199999999999998,"5min":1.048,"15min":1.048},"last":0.73499999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.60099999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.69799999999999995,"15min":0.67700000000000005},"min":{"1min":0.54200000000000004,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.71099999999999997,"15min":0.68300000000000005},"min":{"1min":0.57699999999999996,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.1160000000000001,"5min":1.175,"15min":1.175},"last":0.68700000000000006}]},{"osd":6,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.044,"5min":1.264,"15min":1.264},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":1.141,"15min":1.141},"last":0.56100000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:04:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.79100000000000004,"15min":0.78600000000000003},"min":{"1min":0.56999999999999995,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.93300000000000005,"5min":0.83699999999999997,"15min":0.82099999999999995},"min":{"1min":0.50900000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2490000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.80200000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:04:50 2023","interfaces":[{"interface":"back","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.626}]},{"osd":10,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"min":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.748},{"interface":"front","average":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.1719999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.70899999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920819,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27856,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717296,"statfs":{"total":95995035648,"available":95966511104,"internally_reserved":0,"allocated":925696,"data_stored":438905,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.47299999999999998,"15min":0.439},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.51800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":1.167,"15min":1.167},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96199999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.59699999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68400000000000005,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97799999999999998,"5min":1.083,"15min":1.083},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91000000000000003,"5min":1.079,"15min":1.079},"last":0.498}]},{"osd":5,"last update":"Fri Jul 21 19:05:04 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.752,"15min":0.745},"min":{"1min":0.57599999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.98599999999999999,"5min":1.044,"15min":1.044},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.753,"15min":0.73499999999999999},"min":{"1min":0.59199999999999997,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.91900000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.006,"5min":1.119,"15min":1.119},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.81999999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83099999999999996,"5min":0.68700000000000006,"15min":0.66300000000000003},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.871,"5min":0.71299999999999997,"15min":0.68700000000000006},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.2669999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":1.0309999999999999}]},{"osd":8,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69699999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0920000000000001,"5min":1.155,"15min":1.155},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1220000000000001,"5min":1.298,"15min":1.298},"last":1.004}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"min":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"max":{"1min":1.244,"5min":1.244,"15min":1.244},"last":0.95199999999999996}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"min":{"1min":0.66900000000000004,"5min":0.66900000000000004,"15min":0.66900000000000004},"max":{"1min":42.396000000000001,"5min":42.396000000000001,"15min":42.396000000000001},"last":1.0840000000000001},{"interface":"front","average":{"1min":3.0219999999999998,"5min":3.0219999999999998,"15min":3.0219999999999998},"min":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"max":{"1min":42.792000000000002,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.84599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.02}]}]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27904,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717248,"statfs":{"total":95995035648,"available":95966461952,"internally_reserved":0,"allocated":974848,"data_stored":1028859,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.50700000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.61799999999999999,"15min":0.59199999999999997},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.72399999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69599999999999995,"15min":0.69099999999999995},"min":{"1min":0.49299999999999999,"5min":0.439,"15min":0.439},"max":{"1min":1.004,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.753,"5min":0.69799999999999995,"15min":0.68300000000000005},"min":{"1min":0.55800000000000005,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.02,"5min":1.028,"15min":1.028},"last":1.0189999999999999}]},{"osd":2,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.505,"15min":0.49299999999999999},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.52100000000000002,"15min":0.51900000000000002},"min":{"1min":0.32300000000000001,"5min":0.313,"15min":0.313},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.53800000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.71299999999999997,"15min":0.69699999999999995},"min":{"1min":0.52100000000000002,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90300000000000002,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63,"5min":0.66500000000000004,"15min":0.67100000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.63}]},{"osd":4,"last update":"Fri Jul 21 19:05:05 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.72999999999999998,"15min":0.71899999999999997},"min":{"1min":0.46200000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73199999999999998,"15min":0.73499999999999999},"min":{"1min":0.53900000000000003,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.94999999999999996,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.94499999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88800000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82599999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:04:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.80200000000000005,"15min":0.80000000000000004},"min":{"1min":0.58499999999999996,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.0569999999999999,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.89900000000000002},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.79700000000000004},"min":{"1min":0.56399999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.93000000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.96099999999999997}]},{"osd":9,"last update":"Fri Jul 21 19:04:48 2023","interfaces":[{"interface":"back","average":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":1.002},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.97799999999999998}]},{"osd":10,"last update":"Fri Jul 21 19:05:15 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.4970000000000001,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":1.089},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":1.0660000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":20459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:05:47.258 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-07-21T19:05:47.259 INFO:tasks.cephadm:Setting up client nodes... 2023-07-21T19:05:47.260 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-07-21T19:05:47.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:47 smithi101 ceph-mon[128136]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T19:05:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:47 smithi005 ceph-mon[126460]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T19:05:47.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:47 smithi156 ceph-mon[128316]: mgrmap e17: x(active, since 7m), standbys: z, y 2023-07-21T19:05:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:05:48.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:48 smithi101 ceph-mon[128136]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:48 smithi101 ceph-mon[128136]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:48 smithi101 ceph-mon[128136]: pgmap v305: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T19:05:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:48 smithi005 ceph-mon[126460]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:48 smithi005 ceph-mon[126460]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:48 smithi005 ceph-mon[126460]: pgmap v305: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T19:05:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:48 smithi156 ceph-mon[128316]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:48 smithi156 ceph-mon[128316]: from='client.14856 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:05:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:48 smithi156 ceph-mon[128316]: pgmap v305: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.9 KiB/s wr, 17 op/s 2023-07-21T19:05:49.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 -- 172.21.15.5:0/3666462967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 msgr2=0x7fe01c0fffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 --2- 172.21.15.5:0/3666462967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c0fffc0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe018009990 tx=0x7fe018009ca0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 -- 172.21.15.5:0/3666462967 shutdown_connections 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 --2- 172.21.15.5:0/3666462967 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 0x7fe01c108000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 --2- 172.21.15.5:0/3666462967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c0fffc0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 --2- 172.21.15.5:0/3666462967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 0x7fe01c101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 -- 172.21.15.5:0/3666462967 >> 172.21.15.5:0/3666462967 conn(0x7fe01c0fb200 msgr2=0x7fe01c0fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.509+0000 7fe0243c4700 1 -- 172.21.15.5:0/3666462967 shutdown_connections 2023-07-21T19:05:49.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.510+0000 7fe0243c4700 1 -- 172.21.15.5:0/3666462967 wait complete. 2023-07-21T19:05:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.511+0000 7fe0243c4700 1 Processor -- start 2023-07-21T19:05:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.511+0000 7fe0243c4700 1 -- start start 2023-07-21T19:05:49.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 0x7fe01c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 0x7fe01c19ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19f4f0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19f630 con 0x7fe01c101740 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe0243c4700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe01c19f7c0 con 0x7fe01c100db0 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe022160700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:49.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe022160700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38326/0 (socket says 172.21.15.5:38326) 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe022160700 1 -- 172.21.15.5:0/444432971 learned_addr learned my addr 172.21.15.5:0/444432971 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe02195f700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 0x7fe01c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.512+0000 7fe022961700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 0x7fe01c19ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 -- 172.21.15.5:0/444432971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 msgr2=0x7fe01c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 0x7fe01c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 -- 172.21.15.5:0/444432971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 msgr2=0x7fe01c19ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:49.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 0x7fe01c19ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 -- 172.21.15.5:0/444432971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0180095f0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe022160700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe018005740 tx=0x7fe018005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe01801d070 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe01c19fa40 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe018003f80 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe018022af0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.513+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe01c19feb0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.515+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe01c10b270 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.516+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe018003af0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.518+0000 7fe0137fe700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 0x7fe0080a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.518+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fe0180ce0f0 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.518+0000 7fe02195f700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 0x7fe0080a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:49.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.519+0000 7fe02195f700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 0x7fe0080a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00ac50 tx=0x7fe00c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:49.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.522+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe018097820 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.807+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fe01c1a0500 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.812+0000 7fe0137fe700 1 -- 172.21.15.5:0/444432971 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) v1 ==== 170+0+59 (secure 0 0 0) 0x7fe01809ae40 con 0x7fe01c0ffbb0 2023-07-21T19:05:49.823 INFO:teuthology.orchestra.run.smithi005.stdout:[client.0] 2023-07-21T19:05:49.823 INFO:teuthology.orchestra.run.smithi005.stdout: key = AQAN17pkbL87MBAAKDhLVto0RVm73N3b1i9eVA== 2023-07-21T19:05:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 msgr2=0x7fe0080a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 0x7fe0080a3690 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe00c00ac50 tx=0x7fe00c00a380 comp rx=0 tx=0).stop 2023-07-21T19:05:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 msgr2=0x7fe01c197960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:49.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe018005740 tx=0x7fe018005770 comp rx=0 tx=0).stop 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 shutdown_connections 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0080a11e0 0x7fe0080a3690 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe01c101740 0x7fe01c19ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe01c0ffbb0 0x7fe01c197960 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 --2- 172.21.15.5:0/444432971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe01c100db0 0x7fe01c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:49.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 >> 172.21.15.5:0/444432971 conn(0x7fe01c0fb200 msgr2=0x7fe01c106680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 shutdown_connections 2023-07-21T19:05:49.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:49.815+0000 7fe0243c4700 1 -- 172.21.15.5:0/444432971 wait complete. 2023-07-21T19:05:50.404 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T19:05:50.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-07-21T19:05:50.404 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-07-21T19:05:50.505 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-07-21T19:05:50.505 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-07-21T19:05:50.506 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph mgr dump --format=json 2023-07-21T19:05:50.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:50 smithi005 ceph-mon[126460]: pgmap v306: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.4 KiB/s wr, 16 op/s 2023-07-21T19:05:50.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:50 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T19:05:50.606 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:50 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T19:05:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:50 smithi156 ceph-mon[128316]: pgmap v306: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.4 KiB/s wr, 16 op/s 2023-07-21T19:05:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:50 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T19:05:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:50 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T19:05:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:50 smithi101 ceph-mon[128136]: pgmap v306: 97 pgs: 97 active+clean; 599 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 5.4 KiB/s wr, 16 op/s 2023-07-21T19:05:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:50 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-07-21T19:05:50.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:50 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/444432971' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-07-21T19:05:51.275 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:05:52.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 -- 172.21.15.5:0/3003769474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c1007a0 msgr2=0x7f325c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 --2- 172.21.15.5:0/3003769474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c1007a0 0x7f325c104770 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f32500099c0 tx=0x7f3250009cd0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 -- 172.21.15.5:0/3003769474 shutdown_connections 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 --2- 172.21.15.5:0/3003769474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c104ea0 0x7f325c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 --2- 172.21.15.5:0/3003769474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c0ffe00 0x7f325c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 --2- 172.21.15.5:0/3003769474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c1007a0 0x7f325c104770 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 -- 172.21.15.5:0/3003769474 >> 172.21.15.5:0/3003769474 conn(0x7f325c0fb660 msgr2=0x7f325c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 -- 172.21.15.5:0/3003769474 shutdown_connections 2023-07-21T19:05:52.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.648+0000 7f32612d5700 1 -- 172.21.15.5:0/3003769474 wait complete. 2023-07-21T19:05:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.650+0000 7f32612d5700 1 Processor -- start 2023-07-21T19:05:52.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.650+0000 7f32612d5700 1 -- start start 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.650+0000 7f32612d5700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c0ffe00 0x7f325c19b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.650+0000 7f32612d5700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c1007a0 0x7f325c19be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f32612d5700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f32612d5700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c195ff0 con 0x7f325c104ea0 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f32612d5700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c196160 con 0x7f325c1007a0 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f32612d5700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f325c196320 con 0x7f325c0ffe00 2023-07-21T19:05:52.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38338/0 (socket says 172.21.15.5:38338) 2023-07-21T19:05:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 -- 172.21.15.5:0/4106285580 learned_addr learned my addr 172.21.15.5:0/4106285580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:52.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325a7fc700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c1007a0 0x7f325c19be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325affd700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c0ffe00 0x7f325c19b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 -- 172.21.15.5:0/4106285580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c0ffe00 msgr2=0x7f325c19b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c0ffe00 0x7f325c19b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 -- 172.21.15.5:0/4106285580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c1007a0 msgr2=0x7f325c19be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c1007a0 0x7f325c19be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.651+0000 7f325b7fe700 1 -- 172.21.15.5:0/4106285580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32500095f0 con 0x7f325c104ea0 2023-07-21T19:05:52.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f325b7fe700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3254007d70 tx=0x7f325400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f325400ac90 con 0x7f325c104ea0 2023-07-21T19:05:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f325400adf0 con 0x7f325c104ea0 2023-07-21T19:05:52.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f325c196600 con 0x7f325c104ea0 2023-07-21T19:05:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32540115e0 con 0x7f325c104ea0 2023-07-21T19:05:52.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.652+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f325c1a28f0 con 0x7f325c104ea0 2023-07-21T19:05:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.653+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f325c04e210 con 0x7f325c104ea0 2023-07-21T19:05:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.655+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3254012420 con 0x7f325c104ea0 2023-07-21T19:05:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.657+0000 7f3243fff700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 0x7f32440a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:52.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.657+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32540cca70 con 0x7f325c104ea0 2023-07-21T19:05:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.657+0000 7f325affd700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 0x7f32440a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:52.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:52 smithi005 ceph-mon[126460]: pgmap v307: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 17 op/s 2023-07-21T19:05:52.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.658+0000 7f325affd700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 0x7f32440a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f324c007be0 tx=0x7f324c0073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:52.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.661+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3254096220 con 0x7f325c104ea0 2023-07-21T19:05:52.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:52 smithi156 ceph-mon[128316]: pgmap v307: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 17 op/s 2023-07-21T19:05:52.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:52 smithi101 ceph-mon[128136]: pgmap v307: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.9 KiB/s rd, 6.9 KiB/s wr, 17 op/s 2023-07-21T19:05:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.929+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f325c1a2b80 con 0x7f325c104ea0 2023-07-21T19:05:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.935+0000 7f3243fff700 1 -- 172.21.15.5:0/4106285580 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260117 (secure 0 0 0) 0x7f3254099840 con 0x7f325c104ea0 2023-07-21T19:05:52.951 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:52.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 msgr2=0x7f32440a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 0x7f32440a3690 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f324c007be0 tx=0x7f324c0073d0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 msgr2=0x7f325c195980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3254007d70 tx=0x7f325400a340 comp rx=0 tx=0).stop 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 shutdown_connections 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32440a11e0 0x7f32440a3690 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f325c104ea0 0x7f325c195980 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f325c0ffe00 0x7f325c19b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 --2- 172.21.15.5:0/4106285580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f325c1007a0 0x7f325c19be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 >> 172.21.15.5:0/4106285580 conn(0x7f325c0fb660 msgr2=0x7f325c0fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 shutdown_connections 2023-07-21T19:05:52.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:52.940+0000 7f32612d5700 1 -- 172.21.15.5:0/4106285580 wait complete. 2023-07-21T19:05:53.457 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6800","nonce":1627438252},{"type":"v1","addr":"172.21.15.5:6801","nonce":1627438252}]},"active_addr":"172.21.15.5:6801/1627438252","active_change":"2023-07-21T18:58:07.219253+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14217,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.5:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2496466053}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":1985166852}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":4257470084}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":3558708925}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.5:0","nonce":2222480520}]}]} 2023-07-21T19:05:53.465 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-07-21T19:05:53.465 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-07-21T19:05:53.466 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T19:05:53.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:53 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4106285580' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T19:05:53.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:53 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4106285580' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T19:05:53.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:53 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4106285580' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-07-21T19:05:54.203 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:05:54.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:54 smithi005 ceph-mon[126460]: pgmap v308: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-07-21T19:05:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:54 smithi156 ceph-mon[128316]: pgmap v308: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-07-21T19:05:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:54 smithi101 ceph-mon[128136]: pgmap v308: 97 pgs: 97 active+clean; 621 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-07-21T19:05:55.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 -- 172.21.15.5:0/3231139773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 msgr2=0x7ff664104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 --2- 172.21.15.5:0/3231139773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664104770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7ff6580099c0 tx=0x7ff658009cd0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 -- 172.21.15.5:0/3231139773 shutdown_connections 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 --2- 172.21.15.5:0/3231139773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 0x7ff66410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 --2- 172.21.15.5:0/3231139773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 0x7ff6641001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 --2- 172.21.15.5:0/3231139773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664104770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 -- 172.21.15.5:0/3231139773 >> 172.21.15.5:0/3231139773 conn(0x7ff6640fb660 msgr2=0x7ff6640fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 -- 172.21.15.5:0/3231139773 shutdown_connections 2023-07-21T19:05:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.715+0000 7ff669132700 1 -- 172.21.15.5:0/3231139773 wait complete. 2023-07-21T19:05:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 Processor -- start 2023-07-21T19:05:55.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 -- start start 2023-07-21T19:05:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 0x7ff664197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:55.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 0x7ff66419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66419cdd0 con 0x7ff6641007a0 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66419cf40 con 0x7ff6640ffe00 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.716+0000 7ff669132700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff66419d100 con 0x7ff664104ea0 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38354/0 (socket says 172.21.15.5:38354) 2023-07-21T19:05:55.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 -- 172.21.15.5:0/2459226055 learned_addr learned my addr 172.21.15.5:0/2459226055 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff662d9d700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 0x7ff664197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66359e700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 0x7ff66419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 -- 172.21.15.5:0/2459226055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 msgr2=0x7ff66419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 0x7ff66419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 -- 172.21.15.5:0/2459226055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 msgr2=0x7ff664197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 0x7ff664197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 -- 172.21.15.5:0/2459226055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6580095f0 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.717+0000 7ff66259c700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff658009990 tx=0x7ff658009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.718+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff65801d070 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.718+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff658003ec0 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.718+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff658022bf0 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.718+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff66419d380 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.718+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6641a4c40 con 0x7ff6641007a0 2023-07-21T19:05:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.719+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff66404e210 con 0x7ff6641007a0 2023-07-21T19:05:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.721+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff658022d50 con 0x7ff6641007a0 2023-07-21T19:05:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.722+0000 7ff64bfff700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 0x7ff64c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.722+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7ff6580cdb70 con 0x7ff6641007a0 2023-07-21T19:05:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.722+0000 7ff662d9d700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 0x7ff64c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:55.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.723+0000 7ff662d9d700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 0x7ff64c0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff654005950 tx=0x7ff65400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.726+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff6580972a0 con 0x7ff6641007a0 2023-07-21T19:05:55.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.934+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff664199170 con 0x7ff6641007a0 2023-07-21T19:05:55.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.935+0000 7ff64bfff700 1 -- 172.21.15.5:0/2459226055 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22096 (secure 0 0 0) 0x7ff65809a8c0 con 0x7ff6641007a0 2023-07-21T19:05:55.946 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:55.946 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:05:38.273223+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 msgr2=0x7ff64c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 0x7ff64c0a3710 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff654005950 tx=0x7ff65400b410 comp rx=0 tx=0).stop 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 msgr2=0x7ff664198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7ff658009990 tx=0x7ff658009580 comp rx=0 tx=0).stop 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 shutdown_connections 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff64c0a1260 0x7ff64c0a3710 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff664104ea0 0x7ff66419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6640ffe00 0x7ff664197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 --2- 172.21.15.5:0/2459226055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6641007a0 0x7ff664198350 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 >> 172.21.15.5:0/2459226055 conn(0x7ff6640fb660 msgr2=0x7ff6640fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 shutdown_connections 2023-07-21T19:05:55.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:55.938+0000 7ff669132700 1 -- 172.21.15.5:0/2459226055 wait complete. 2023-07-21T19:05:56.484 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-07-21T19:05:56.484 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd dump --format=json 2023-07-21T19:05:56.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:56 smithi005 ceph-mon[126460]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2023-07-21T19:05:56.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:56 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2459226055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:56 smithi156 ceph-mon[128316]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2023-07-21T19:05:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:56 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2459226055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:56 smithi101 ceph-mon[128136]: pgmap v309: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 6.6 KiB/s wr, 15 op/s 2023-07-21T19:05:56.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:56 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2459226055' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:57.218 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:05:58.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:58 smithi101 ceph-mon[128136]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-07-21T19:05:58.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 -- 172.21.15.5:0/3972504518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 msgr2=0x7f330410ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 --2- 172.21.15.5:0/3972504518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330410ca20 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f32fc00b0d0 tx=0x7f32fc00b3e0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 -- 172.21.15.5:0/3972504518 shutdown_connections 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 --2- 172.21.15.5:0/3972504518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330410ca20 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 --2- 172.21.15.5:0/3972504518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 0x7f33041001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 --2- 172.21.15.5:0/3972504518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 0x7f3304104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 -- 172.21.15.5:0/3972504518 >> 172.21.15.5:0/3972504518 conn(0x7f33040fb660 msgr2=0x7f33040fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.624+0000 7f330a849700 1 -- 172.21.15.5:0/3972504518 shutdown_connections 2023-07-21T19:05:58.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.625+0000 7f330a849700 1 -- 172.21.15.5:0/3972504518 wait complete. 2023-07-21T19:05:58.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 Processor -- start 2023-07-21T19:05:58.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 -- start start 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 0x7f3304197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 0x7f3304198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330419cd70 con 0x7f3304104ea0 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330419cee0 con 0x7f33041007a0 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f330a849700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f330419d0a0 con 0x7f33040ffe00 2023-07-21T19:05:58.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f3308de6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.626+0000 7f3308de6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58864/0 (socket says 172.21.15.5:58864) 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 -- 172.21.15.5:0/4069667919 learned_addr learned my addr 172.21.15.5:0/4069667919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f33037fe700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 0x7f3304198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3303fff700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 0x7f3304197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 -- 172.21.15.5:0/4069667919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 msgr2=0x7f3304197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:58.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 0x7f3304197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 -- 172.21.15.5:0/4069667919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 msgr2=0x7f3304198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 0x7f3304198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 -- 172.21.15.5:0/4069667919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc009d00 con 0x7f3304104ea0 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f3308de6700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009300 tx=0x7f32fc009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00e040 con 0x7f3304104ea0 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32fc003e10 con 0x7f3304104ea0 2023-07-21T19:05:58.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f330419d320 con 0x7f3304104ea0 2023-07-21T19:05:58.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.627+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc01b930 con 0x7f3304104ea0 2023-07-21T19:05:58.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.628+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33041a4be0 con 0x7f3304104ea0 2023-07-21T19:05:58.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.629+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f330404e210 con 0x7f3304104ea0 2023-07-21T19:05:58.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.631+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f32fc024410 con 0x7f3304104ea0 2023-07-21T19:05:58.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.632+0000 7f33017fa700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 0x7f32ec0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:05:58.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.632+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f32fc0ce120 con 0x7f3304104ea0 2023-07-21T19:05:58.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.632+0000 7f3303fff700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 0x7f32ec0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:05:58.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.633+0000 7f3303fff700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 0x7f32ec0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005ea0 tx=0x7f32f4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:05:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.636+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f32fc097850 con 0x7f3304104ea0 2023-07-21T19:05:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:58 smithi005 ceph-mon[126460]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-07-21T19:05:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:58 smithi156 ceph-mon[128316]: pgmap v310: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.5 KiB/s wr, 9 op/s 2023-07-21T19:05:58.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.841+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f33041a4e30 con 0x7f3304104ea0 2023-07-21T19:05:58.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.843+0000 7f33017fa700 1 -- 172.21.15.5:0/4069667919 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+22096 (secure 0 0 0) 0x7f32fc09ae70 con 0x7f3304104ea0 2023-07-21T19:05:58.854 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:05:58.855 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":85,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:05:38.273223+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:05:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.845+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 msgr2=0x7f32ec0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.845+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 0x7f32ec0a3690 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f32f4005ea0 tx=0x7f32f4005e30 comp rx=0 tx=0).stop 2023-07-21T19:05:58.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.845+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 msgr2=0x7f330419c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.845+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009300 tx=0x7f32fc009330 comp rx=0 tx=0).stop 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 shutdown_connections 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f32ec0a11e0 0x7f32ec0a3690 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304104ea0 0x7f330419c760 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ffe00 0x7f3304197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 --2- 172.21.15.5:0/4069667919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041007a0 0x7f3304198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:05:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 >> 172.21.15.5:0/4069667919 conn(0x7f33040fb660 msgr2=0x7f33040fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:05:58.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 shutdown_connections 2023-07-21T19:05:58.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:05:58.846+0000 7f330a849700 1 -- 172.21.15.5:0/4069667919 wait complete. 2023-07-21T19:05:59.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.0 flush_pg_stats 2023-07-21T19:05:59.400 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.1 flush_pg_stats 2023-07-21T19:05:59.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.2 flush_pg_stats 2023-07-21T19:05:59.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.3 flush_pg_stats 2023-07-21T19:05:59.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.4 flush_pg_stats 2023-07-21T19:05:59.402 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.5 flush_pg_stats 2023-07-21T19:05:59.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.6 flush_pg_stats 2023-07-21T19:05:59.403 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.7 flush_pg_stats 2023-07-21T19:05:59.408 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.8 flush_pg_stats 2023-07-21T19:05:59.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.9 flush_pg_stats 2023-07-21T19:05:59.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.10 flush_pg_stats 2023-07-21T19:05:59.410 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph tell osd.11 flush_pg_stats 2023-07-21T19:05:59.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:05:59 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4069667919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:59.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:05:59 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4069667919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:05:59.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:05:59 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4069667919' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:00.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:00 smithi101 ceph-mon[128136]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-07-21T19:06:00.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:00 smithi005 ceph-mon[126460]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-07-21T19:06:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:00 smithi156 ceph-mon[128316]: pgmap v311: 97 pgs: 97 active+clean; 629 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 3.0 KiB/s wr, 6 op/s 2023-07-21T19:06:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.272 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.497 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.593 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.594 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.794 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:01.794 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:02.323 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:02.546 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:02 smithi005 ceph-mon[126460]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-07-21T19:06:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:02 smithi156 ceph-mon[128316]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-07-21T19:06:02.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:02 smithi101 ceph-mon[128136]: pgmap v312: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.4 KiB/s wr, 8 op/s 2023-07-21T19:06:03.796 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:04.232 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:04.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:04 smithi005 ceph-mon[126460]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-07-21T19:06:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:04 smithi156 ceph-mon[128316]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-07-21T19:06:04.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:04 smithi101 ceph-mon[128136]: pgmap v313: 97 pgs: 97 active+clean; 633 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.5 KiB/s wr, 6 op/s 2023-07-21T19:06:05.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 -- 172.21.15.5:0/2600959054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d741007a0 msgr2=0x7f7d74104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 --2- 172.21.15.5:0/2600959054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d741007a0 0x7f7d74104770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7d680099c0 tx=0x7f7d68009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 -- 172.21.15.5:0/2600959054 shutdown_connections 2023-07-21T19:06:05.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 --2- 172.21.15.5:0/2600959054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d74104ea0 0x7f7d7410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 --2- 172.21.15.5:0/2600959054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d740ffe00 0x7f7d741001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 --2- 172.21.15.5:0/2600959054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d741007a0 0x7f7d74104770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 -- 172.21.15.5:0/2600959054 >> 172.21.15.5:0/2600959054 conn(0x7f7d740fb660 msgr2=0x7f7d740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:05.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 -- 172.21.15.5:0/2600959054 shutdown_connections 2023-07-21T19:06:05.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.547+0000 7f7d79a18700 1 -- 172.21.15.5:0/2600959054 wait complete. 2023-07-21T19:06:05.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 Processor -- start 2023-07-21T19:06:05.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 -- start start 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d740ffe00 0x7f7d74197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d741007a0 0x7f7d741982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 0x7f7d7419c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7419cd30 con 0x7f7d74104ea0 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7419cea0 con 0x7f7d740ffe00 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.548+0000 7f7d79a18700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7419d060 con 0x7f7d741007a0 2023-07-21T19:06:05.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 0x7f7d7419c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 0x7f7d7419c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58912/0 (socket says 172.21.15.5:58912) 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 -- 172.21.15.5:0/982915313 learned_addr learned my addr 172.21.15.5:0/982915313 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d72ffd700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d740ffe00 0x7f7d74197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d727fc700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d741007a0 0x7f7d741982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 -- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d741007a0 msgr2=0x7f7d741982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d741007a0 0x7f7d741982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 -- 172.21.15.5:0/982915313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d740ffe00 msgr2=0x7f7d74197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d740ffe00 0x7f7d74197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.549+0000 7f7d737fe700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d680095f0 con 0x7f7d74104ea0 2023-07-21T19:06:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d737fe700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 0x7f7d7419c720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c007ee0 tx=0x7f7d6c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:05.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c00dd20 con 0x7f7d74104ea0 2023-07-21T19:06:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d6c00f040 con 0x7f7d74104ea0 2023-07-21T19:06:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7419d340 con 0x7f7d74104ea0 2023-07-21T19:06:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6c004750 con 0x7f7d74104ea0 2023-07-21T19:06:05.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.550+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d741a4ba0 con 0x7f7d74104ea0 2023-07-21T19:06:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.551+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7d7404e210 con 0x7f7d74104ea0 2023-07-21T19:06:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.553+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f7d6c003a00 con 0x7f7d74104ea0 2023-07-21T19:06:05.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.554+0000 7f7d78a16700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7d5c0a1260 0x7f7d5c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.554+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f7d6c0ccc90 con 0x7f7d74104ea0 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.554+0000 7f7d72ffd700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7d5c0a1260 0x7f7d5c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.555+0000 7f7d78a16700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] conn(0x7f7d5c0aaa30 0x7f7d5c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.555+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7d5c0ad4f0 con 0x7f7d5c0aaa30 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.555+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d6c0cd000 con 0x7f7d74104ea0 2023-07-21T19:06:05.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.555+0000 7f7d72ffd700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7d5c0a1260 0x7f7d5c0a3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009c60 tx=0x7f7d64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.555+0000 7f7d727fc700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] conn(0x7f7d5c0aaa30 0x7f7d5c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.556+0000 7f7d727fc700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] conn(0x7f7d5c0aaa30 0x7f7d5c0ace40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:05.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.557+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== osd.11 v2:172.21.15.156:6824/2474999994 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7d5c0ad4f0 con 0x7f7d5c0aaa30 2023-07-21T19:06:05.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.575+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 --> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7d7410fc10 con 0x7f7d5c0aaa30 2023-07-21T19:06:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d78a16700 1 -- 172.21.15.5:0/982915313 <== osd.11 v2:172.21.15.156:6824/2474999994 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f7d6801e040 con 0x7f7d5c0aaa30 2023-07-21T19:06:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] conn(0x7f7d5c0aaa30 msgr2=0x7f7d5c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] conn(0x7f7d5c0aaa30 0x7f7d5c0ace40 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7d5c0a1260 msgr2=0x7f7d5c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f7d5c0a1260 0x7f7d5c0a3710 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f7d64009c60 tx=0x7f7d64009260 comp rx=0 tx=0).stop 2023-07-21T19:06:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 msgr2=0x7f7d7419c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 --2- 172.21.15.5:0/982915313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d74104ea0 0x7f7d7419c720 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c007ee0 tx=0x7f7d6c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d737fe700 1 -- 172.21.15.5:0/982915313 reap_dead start 2023-07-21T19:06:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 shutdown_connections 2023-07-21T19:06:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 >> 172.21.15.5:0/982915313 conn(0x7f7d740fb660 msgr2=0x7f7d740fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.576+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 shutdown_connections 2023-07-21T19:06:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.577+0000 7f7d79a18700 1 -- 172.21.15.5:0/982915313 wait complete. 2023-07-21T19:06:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 -- 172.21.15.5:0/1243501351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea00ffe00 msgr2=0x7faea01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 --2- 172.21.15.5:0/1243501351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea00ffe00 0x7faea01001d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fae900099c0 tx=0x7fae90009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 -- 172.21.15.5:0/1243501351 shutdown_connections 2023-07-21T19:06:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 --2- 172.21.15.5:0/1243501351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faea0104ea0 0x7faea010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 --2- 172.21.15.5:0/1243501351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea00ffe00 0x7faea01001d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 --2- 172.21.15.5:0/1243501351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea01007a0 0x7faea0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 -- 172.21.15.5:0/1243501351 >> 172.21.15.5:0/1243501351 conn(0x7faea00fb660 msgr2=0x7faea00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 -- 172.21.15.5:0/1243501351 shutdown_connections 2023-07-21T19:06:06.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.157+0000 7faea5916700 1 -- 172.21.15.5:0/1243501351 wait complete. 2023-07-21T19:06:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.158+0000 7faea5916700 1 Processor -- start 2023-07-21T19:06:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 -- start start 2023-07-21T19:06:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faea00ffe00 0x7faea0197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:06.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea01007a0 0x7faea0198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 0x7faea019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019cdc0 con 0x7faea0104ea0 2023-07-21T19:06:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019cf30 con 0x7faea00ffe00 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7faea5916700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faea019d0f0 con 0x7faea01007a0 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7fae9f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 0x7faea019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7fae9f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 0x7faea019c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.159+0000 7fae9f7fe700 1 -- 172.21.15.5:0/3518081309 learned_addr learned my addr 172.21.15.5:0/3518081309 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9effd700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faea00ffe00 0x7faea0197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9e7fc700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea01007a0 0x7faea0198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 -- 172.21.15.5:0/3518081309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea01007a0 msgr2=0x7faea0198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faea01007a0 0x7faea0198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 -- 172.21.15.5:0/3518081309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faea00ffe00 msgr2=0x7faea0197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faea00ffe00 0x7faea0197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae900095f0 con 0x7faea0104ea0 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7fae9f7fe700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 0x7faea019c7b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae9800ef30 tx=0x7fae9800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:06.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae98018070 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fae9800f040 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faea019d370 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae9800cc70 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.160+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faea01a4c30 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.161+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faea010fc60 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.163+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fae98007500 con 0x7faea0104ea0 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7faea4914700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fae880a11e0 0x7fae880a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:06.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fae980cceb0 con 0x7faea0104ea0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7fae9effd700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fae880a11e0 0x7fae880a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7faea4914700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] conn(0x7fae880aa9b0 0x7fae880acdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fae880ad470 con 0x7fae880aa9b0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.165+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fae980cd220 con 0x7faea0104ea0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.166+0000 7fae9e7fc700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] conn(0x7fae880aa9b0 0x7fae880acdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.166+0000 7fae9effd700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fae880a11e0 0x7fae880a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae90009990 tx=0x7fae90005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:06.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.166+0000 7fae9e7fc700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] conn(0x7fae880aa9b0 0x7fae880acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.167+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== osd.0 v2:172.21.15.5:6802/2154692488 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fae880ad470 con 0x7fae880aa9b0 2023-07-21T19:06:06.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.184+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 --> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7faea004e210 con 0x7fae880aa9b0 2023-07-21T19:06:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.184+0000 7faea4914700 1 -- 172.21.15.5:0/3518081309 <== osd.0 v2:172.21.15.5:6802/2154692488 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7faea004e210 con 0x7fae880aa9b0 2023-07-21T19:06:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] conn(0x7fae880aa9b0 msgr2=0x7fae880acdc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] conn(0x7fae880aa9b0 0x7fae880acdc0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fae880a11e0 msgr2=0x7fae880a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fae880a11e0 0x7fae880a3690 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fae90009990 tx=0x7fae90005c00 comp rx=0 tx=0).stop 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 msgr2=0x7faea019c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 --2- 172.21.15.5:0/3518081309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faea0104ea0 0x7faea019c7b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fae9800ef30 tx=0x7fae9800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7fae9f7fe700 1 -- 172.21.15.5:0/3518081309 reap_dead start 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 shutdown_connections 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 >> 172.21.15.5:0/3518081309 conn(0x7faea00fb660 msgr2=0x7faea00fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 shutdown_connections 2023-07-21T19:06:06.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:05.185+0000 7faea5916700 1 -- 172.21.15.5:0/3518081309 wait complete. 2023-07-21T19:06:06.506 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:06 smithi005 ceph-mon[126460]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-21T19:06:06.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:06 smithi156 ceph-mon[128316]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-21T19:06:06.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:06 smithi101 ceph-mon[128136]: pgmap v314: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-07-21T19:06:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 -- 172.21.15.5:0/3733645985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 msgr2=0x7fd240072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 --2- 172.21.15.5:0/3733645985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240072c00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fd23c009990 tx=0x7fd23c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 -- 172.21.15.5:0/3733645985 shutdown_connections 2023-07-21T19:06:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 --2- 172.21.15.5:0/3733645985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd24010c460 0x7fd24010e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 --2- 172.21.15.5:0/3733645985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240072c00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 --2- 172.21.15.5:0/3733645985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd240073140 0x7fd24010bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 -- 172.21.15.5:0/3733645985 >> 172.21.15.5:0/3733645985 conn(0x7fd2400fb660 msgr2=0x7fd2400fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:07.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 -- 172.21.15.5:0/3733645985 shutdown_connections 2023-07-21T19:06:07.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.058+0000 7fd247a12700 1 -- 172.21.15.5:0/3733645985 wait complete. 2023-07-21T19:06:07.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 Processor -- start 2023-07-21T19:06:07.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 -- start start 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd240073140 0x7fd240198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24010c460 0x7fd24019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019cd00 con 0x7fd240072830 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019ce70 con 0x7fd240073140 2023-07-21T19:06:07.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.059+0000 7fd247a12700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd24019d030 con 0x7fd24010c460 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39332/0 (socket says 172.21.15.5:39332) 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 -- 172.21.15.5:0/4036501940 learned_addr learned my addr 172.21.15.5:0/4036501940 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd244fad700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd240073140 0x7fd240198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd245faf700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24010c460 0x7fd24019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 -- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24010c460 msgr2=0x7fd24019c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24010c460 0x7fd24019c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 -- 172.21.15.5:0/4036501940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd240073140 msgr2=0x7fd240198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd240073140 0x7fd240198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0095f0 con 0x7fd240072830 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd245faf700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd24010c460 0x7fd24019c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.060+0000 7fd2457ae700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240197d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005770 tx=0x7fd23c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.061+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c01d070 con 0x7fd240072830 2023-07-21T19:06:07.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.061+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd24019d2b0 con 0x7fd240072830 2023-07-21T19:06:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.061+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd23c00fbb0 con 0x7fd240072830 2023-07-21T19:06:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.061+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23c0173f0 con 0x7fd240072830 2023-07-21T19:06:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.061+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2401a4b70 con 0x7fd240072830 2023-07-21T19:06:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.062+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd240111dd0 con 0x7fd240072830 2023-07-21T19:06:07.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.064+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd23c021410 con 0x7fd240072830 2023-07-21T19:06:07.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.065+0000 7fd2367fc700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd22c0a1260 0x7fd22c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fd23c0ce760 con 0x7fd240072830 2023-07-21T19:06:07.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd244fad700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd22c0a1260 0x7fd22c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd2367fc700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] conn(0x7fd22c0aaa30 0x7fd22c0ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd22c0ad4f0 con 0x7fd22c0aaa30 2023-07-21T19:06:07.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd23c0cead0 con 0x7fd240072830 2023-07-21T19:06:07.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd244fad700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd22c0a1260 0x7fd22c0a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd240199360 tx=0x7fd230009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.066+0000 7fd245faf700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] conn(0x7fd22c0aaa30 0x7fd22c0ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.067+0000 7fd245faf700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] conn(0x7fd22c0aaa30 0x7fd22c0ace40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.068+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== osd.5 v2:172.21.15.156:6808/1994365713 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd22c0ad4f0 con 0x7fd22c0aaa30 2023-07-21T19:06:07.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.085+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 --> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd24004e210 con 0x7fd22c0aaa30 2023-07-21T19:06:07.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.086+0000 7fd2367fc700 1 -- 172.21.15.5:0/4036501940 <== osd.5 v2:172.21.15.156:6808/1994365713 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd24004e210 con 0x7fd22c0aaa30 2023-07-21T19:06:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.086+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] conn(0x7fd22c0aaa30 msgr2=0x7fd22c0ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.086+0000 7fd247a12700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] conn(0x7fd22c0aaa30 0x7fd22c0ace40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd22c0a1260 msgr2=0x7fd22c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd22c0a1260 0x7fd22c0a3710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fd240199360 tx=0x7fd230009260 comp rx=0 tx=0).stop 2023-07-21T19:06:07.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 msgr2=0x7fd240197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 --2- 172.21.15.5:0/4036501940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd240072830 0x7fd240197d40 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005770 tx=0x7fd23c0057a0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd245faf700 1 -- 172.21.15.5:0/4036501940 reap_dead start 2023-07-21T19:06:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 shutdown_connections 2023-07-21T19:06:07.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 >> 172.21.15.5:0/4036501940 conn(0x7fd2400fb660 msgr2=0x7fd24010e6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 shutdown_connections 2023-07-21T19:06:07.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.087+0000 7fd247a12700 1 -- 172.21.15.5:0/4036501940 wait complete. 2023-07-21T19:06:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 -- 172.21.15.5:0/3911557490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880febe0 msgr2=0x7fc0880fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 --2- 172.21.15.5:0/3911557490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880febe0 0x7fc0880fefb0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc084009990 tx=0x7fc084009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 -- 172.21.15.5:0/3911557490 shutdown_connections 2023-07-21T19:06:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 --2- 172.21.15.5:0/3911557490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc088103f10 0x7fc088104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 --2- 172.21.15.5:0/3911557490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880febe0 0x7fc0880fefb0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 --2- 172.21.15.5:0/3911557490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc0880ff4f0 0x7fc0881039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 -- 172.21.15.5:0/3911557490 >> 172.21.15.5:0/3911557490 conn(0x7fc088075b70 msgr2=0x7fc088075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 -- 172.21.15.5:0/3911557490 shutdown_connections 2023-07-21T19:06:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.250+0000 7fc08fd92700 1 -- 172.21.15.5:0/3911557490 wait complete. 2023-07-21T19:06:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.252+0000 7fc08fd92700 1 Processor -- start 2023-07-21T19:06:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.252+0000 7fc08fd92700 1 -- start start 2023-07-21T19:06:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 0x7fc08819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880ff4f0 0x7fc08819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc088103f10 0x7fc0881a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a1160 con 0x7fc0880ff4f0 2023-07-21T19:06:07.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a12d0 con 0x7fc088103f10 2023-07-21T19:06:07.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08fd92700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0881a1490 con 0x7fc0880febe0 2023-07-21T19:06:07.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08db2e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 0x7fc08819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08db2e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 0x7fc08819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56518/0 (socket says 172.21.15.5:56518) 2023-07-21T19:06:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.253+0000 7fc08db2e700 1 -- 172.21.15.5:0/2157436841 learned_addr learned my addr 172.21.15.5:0/2157436841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08d32d700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880ff4f0 0x7fc08819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08e32f700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc088103f10 0x7fc0881a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 -- 172.21.15.5:0/2157436841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc088103f10 msgr2=0x7fc0881a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc088103f10 0x7fc0881a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 -- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880ff4f0 msgr2=0x7fc08819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc0880ff4f0 0x7fc08819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0840095f0 con 0x7fc0880febe0 2023-07-21T19:06:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.254+0000 7fc08db2e700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 0x7fc08819c1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc084005910 tx=0x7fc084009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.255+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08401d070 con 0x7fc0880febe0 2023-07-21T19:06:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.255+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0881a1710 con 0x7fc0880febe0 2023-07-21T19:06:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.255+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0881a8fd0 con 0x7fc0880febe0 2023-07-21T19:06:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.256+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc084004c80 con 0x7fc0880febe0 2023-07-21T19:06:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.256+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084003a30 con 0x7fc0880febe0 2023-07-21T19:06:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.256+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc06c000ff0 con 0x7fc0880febe0 2023-07-21T19:06:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.259+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc0840047f0 con 0x7fc0880febe0 2023-07-21T19:06:07.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.260+0000 7fc07effd700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc0740a1260 0x7fc0740a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.260+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 7979+0+0 (secure 0 0 0) 0x7fc0840ce080 con 0x7fc0880febe0 2023-07-21T19:06:07.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc07effd700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] conn(0x7fc0740aaa30 0x7fc0740ace40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:07.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc0740ad4f0 con 0x7fc0740aaa30 2023-07-21T19:06:07.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_get_version_reply(handle=1 version=85) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc0840ce3f0 con 0x7fc0880febe0 2023-07-21T19:06:07.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc08d32d700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc0740a1260 0x7fc0740a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc08e32f700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] conn(0x7fc0740aaa30 0x7fc0740ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:07.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.261+0000 7fc08d32d700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc0740a1260 0x7fc0740a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc08819d7c0 tx=0x7fc07800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.262+0000 7fc08e32f700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] conn(0x7fc0740aaa30 0x7fc0740ace40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:07.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.263+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== osd.2 v2:172.21.15.156:6800/2113778318 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc0740ad4f0 con 0x7fc0740aaa30 2023-07-21T19:06:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.295+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 --> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc06c002d70 con 0x7fc0740aaa30 2023-07-21T19:06:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.296+0000 7fc07effd700 1 -- 172.21.15.5:0/2157436841 <== osd.2 v2:172.21.15.156:6800/2113778318 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc06c002d70 con 0x7fc0740aaa30 2023-07-21T19:06:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.296+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] conn(0x7fc0740aaa30 msgr2=0x7fc0740ace40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.296+0000 7fc08fd92700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] conn(0x7fc0740aaa30 0x7fc0740ace40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:07.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc0740a1260 msgr2=0x7fc0740a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc0740a1260 0x7fc0740a3710 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc08819d7c0 tx=0x7fc07800b410 comp rx=0 tx=0).stop 2023-07-21T19:06:07.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 msgr2=0x7fc08819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:07.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 --2- 172.21.15.5:0/2157436841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc0880febe0 0x7fc08819c1a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc084005910 tx=0x7fc084009580 comp rx=0 tx=0).stop 2023-07-21T19:06:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08e32f700 1 -- 172.21.15.5:0/2157436841 reap_dead start 2023-07-21T19:06:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 shutdown_connections 2023-07-21T19:06:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 >> 172.21.15.5:0/2157436841 conn(0x7fc088075b70 msgr2=0x7fc088101ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 shutdown_connections 2023-07-21T19:06:07.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:07.297+0000 7fc08fd92700 1 -- 172.21.15.5:0/2157436841 wait complete. 2023-07-21T19:06:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T19:06:07.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-21T19:06:07.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-21T19:06:07.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-21T19:06:07.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T19:06:07.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:07 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2023-07-21T19:06:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T19:06:07.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-21T19:06:07.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-21T19:06:07.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-21T19:06:07.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T19:06:07.682 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.683 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:07 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2023-07-21T19:06:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]: dispatch 2023-07-21T19:06:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2023-07-21T19:06:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-07-21T19:06:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]: dispatch 2023-07-21T19:06:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]: dispatch 2023-07-21T19:06:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2023-07-21T19:06:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:07 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]: dispatch 2023-07-21T19:06:08.150 INFO:teuthology.orchestra.run.smithi005.stdout:343597383699 2023-07-21T19:06:08.150 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.11 2023-07-21T19:06:08.589 INFO:teuthology.orchestra.run.smithi005.stdout:34359738446 2023-07-21T19:06:08.590 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.0 2023-07-21T19:06:08.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-21T19:06:08.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T19:06:08.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-21T19:06:08.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-21T19:06:08.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-21T19:06:08.612 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T19:06:08.613 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-21T19:06:08.613 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-21T19:06:08.613 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2023-07-21T19:06:08.614 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:08 smithi005 ceph-mon[126460]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T19:06:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-21T19:06:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T19:06:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-21T19:06:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-21T19:06:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-21T19:06:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T19:06:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-21T19:06:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-21T19:06:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2023-07-21T19:06:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:08 smithi156 ceph-mon[128316]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T19:06:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 -- 172.21.15.5:0/3110473998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c10a160 msgr2=0x7f848c10c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:08.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 --2- 172.21.15.5:0/3110473998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c10a160 0x7f848c10c5a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8484009f90 tx=0x7f848400b330 comp rx=0 tx=0).stop 2023-07-21T19:06:08.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 -- 172.21.15.5:0/3110473998 shutdown_connections 2023-07-21T19:06:08.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 --2- 172.21.15.5:0/3110473998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c10a160 0x7f848c10c5a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 --2- 172.21.15.5:0/3110473998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c100c40 0x7f848c101010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.497+0000 7f849095b700 1 --2- 172.21.15.5:0/3110473998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f848c101550 0x7f848c109c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.498+0000 7f849095b700 1 -- 172.21.15.5:0/3110473998 >> 172.21.15.5:0/3110473998 conn(0x7f848c075b70 msgr2=0x7f848c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.498+0000 7f849095b700 1 -- 172.21.15.5:0/3110473998 shutdown_connections 2023-07-21T19:06:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.498+0000 7f849095b700 1 -- 172.21.15.5:0/3110473998 wait complete. 2023-07-21T19:06:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.499+0000 7f849095b700 1 Processor -- start 2023-07-21T19:06:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 -- start start 2023-07-21T19:06:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f848c100c40 0x7f848c0fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c101550 0x7f848c0feee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 0x7f848c0ff460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:08.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c070c70 con 0x7f848c101550 2023-07-21T19:06:08.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c070de0 con 0x7f848c10a160 2023-07-21T19:06:08.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.500+0000 7f849095b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f848c070fa0 con 0x7f848c100c40 2023-07-21T19:06:08.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.501+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 0x7f848c0ff460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:08.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.501+0000 7f848ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 0x7f848c0ff460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34678/0 (socket says 172.21.15.5:34678) 2023-07-21T19:06:08.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.501+0000 7f848ad9d700 1 -- 172.21.15.5:0/2370786105 learned_addr learned my addr 172.21.15.5:0/2370786105 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:08.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f8489d9b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c101550 0x7f848c0feee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:08.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f848ad9d700 1 -- 172.21.15.5:0/2370786105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f848c100c40 msgr2=0x7f848c0fe9a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:06:08.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f848ad9d700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f848c100c40 0x7f848c0fe9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f848ad9d700 1 -- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c101550 msgr2=0x7f848c0feee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:08.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f848ad9d700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f848c101550 0x7f848c0feee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.502+0000 7f848ad9d700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8484009c40 con 0x7f848c10a160 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.503+0000 7f848ad9d700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 0x7f848c0ff460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8484009f60 tx=0x7f8484008a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.504+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f848400e040 con 0x7f848c10a160 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.504+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f848400bbf0 con 0x7f848c10a160 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.504+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848c071220 con 0x7f848c10a160 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.504+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84840052a0 con 0x7f848c10a160 2023-07-21T19:06:08.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.504+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f848c071770 con 0x7f848c10a160 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.505+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f848c10fc00 con 0x7f848c10a160 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.508+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f848401d470 con 0x7f848c10a160 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f84837fe700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f84700a1260 0x7f84700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f84840ce110 con 0x7f848c10a160 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f848a59c700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f84700a1260 0x7f84700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f84837fe700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] conn(0x7f84700aad90 0x7f84700ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:08.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f84700ad8f0 con 0x7f84700aad90 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.509+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84840ce4c0 con 0x7f848c10a160 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.510+0000 7f8489d9b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] conn(0x7f84700aad90 0x7f84700ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.510+0000 7f848a59c700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f84700a1260 0x7f84700a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f847c005fd0 tx=0x7f847c009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.510+0000 7f8489d9b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] conn(0x7f84700aad90 0x7f84700ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.512+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== osd.3 v2:172.21.15.5:6810/4237119296 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f84700ad8f0 con 0x7f84700aad90 2023-07-21T19:06:08.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.539+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 --> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f848c04e210 con 0x7f84700aad90 2023-07-21T19:06:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.539+0000 7f84837fe700 1 -- 172.21.15.5:0/2370786105 <== osd.3 v2:172.21.15.5:6810/4237119296 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f848c04e210 con 0x7f84700aad90 2023-07-21T19:06:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] conn(0x7f84700aad90 msgr2=0x7f84700ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] conn(0x7f84700aad90 0x7f84700ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:08.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f84700a1260 msgr2=0x7f84700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f84700a1260 0x7f84700a3710 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f847c005fd0 tx=0x7f847c009310 comp rx=0 tx=0).stop 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 msgr2=0x7f848c0ff460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 --2- 172.21.15.5:0/2370786105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f848c10a160 0x7f848c0ff460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8484009f60 tx=0x7f8484008a70 comp rx=0 tx=0).stop 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f848ad9d700 1 -- 172.21.15.5:0/2370786105 reap_dead start 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 shutdown_connections 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 >> 172.21.15.5:0/2370786105 conn(0x7f848c075b70 msgr2=0x7f848c101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:08.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.540+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 shutdown_connections 2023-07-21T19:06:08.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:08.541+0000 7f849095b700 1 -- 172.21.15.5:0/2370786105 wait complete. 2023-07-21T19:06:08.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: pgmap v315: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 853 B/s wr, 6 op/s 2023-07-21T19:06:08.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.4", "id": [2, 11]}]': finished 2023-07-21T19:06:08.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2023-07-21T19:06:08.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-07-21T19:06:08.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [8, 1]}]': finished 2023-07-21T19:06:08.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2023-07-21T19:06:08.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1", "id": [4, 9]}]': finished 2023-07-21T19:06:08.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2023-07-21T19:06:08.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2023-07-21T19:06:08.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:08 smithi101 ceph-mon[128136]: osdmap e86: 12 total, 12 up, 12 in 2023-07-21T19:06:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 -- 172.21.15.5:0/1321519569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c680ffe00 msgr2=0x7f0c681001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 --2- 172.21.15.5:0/1321519569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c680ffe00 0x7f0c681001d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0c580099c0 tx=0x7f0c58009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 -- 172.21.15.5:0/1321519569 shutdown_connections 2023-07-21T19:06:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 --2- 172.21.15.5:0/1321519569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c68104ea0 0x7f0c6810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 --2- 172.21.15.5:0/1321519569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c680ffe00 0x7f0c681001d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 --2- 172.21.15.5:0/1321519569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c681007a0 0x7f0c68104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 -- 172.21.15.5:0/1321519569 >> 172.21.15.5:0/1321519569 conn(0x7f0c680fb660 msgr2=0x7f0c680fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 -- 172.21.15.5:0/1321519569 shutdown_connections 2023-07-21T19:06:09.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.244+0000 7f0c6de54700 1 -- 172.21.15.5:0/1321519569 wait complete. 2023-07-21T19:06:09.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.245+0000 7f0c6de54700 1 Processor -- start 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.245+0000 7f0c6de54700 1 -- start start 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 0x7f0c68072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c681007a0 0x7f0c68072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c68104ea0 0x7f0c680732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681a4040 con 0x7f0c680ffe00 2023-07-21T19:06:09.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681a41b0 con 0x7f0c68104ea0 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c6de54700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c681a4370 con 0x7f0c681007a0 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 0x7f0c68072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 0x7f0c68072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39400/0 (socket says 172.21.15.5:39400) 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.246+0000 7f0c677fe700 1 -- 172.21.15.5:0/2550468644 learned_addr learned my addr 172.21.15.5:0/2550468644 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c67fff700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c68104ea0 0x7f0c680732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c66ffd700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c681007a0 0x7f0c68072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 -- 172.21.15.5:0/2550468644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c681007a0 msgr2=0x7f0c68072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c681007a0 0x7f0c68072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 -- 172.21.15.5:0/2550468644 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c68104ea0 msgr2=0x7f0c680732d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c68104ea0 0x7f0c680732d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c009520 con 0x7f0c680ffe00 2023-07-21T19:06:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c677fe700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 0x7f0c68072810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c58006010 tx=0x7f0c5800f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.247+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5801d070 con 0x7f0c680ffe00 2023-07-21T19:06:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.248+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c58005030 con 0x7f0c680ffe00 2023-07-21T19:06:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.248+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c580049c0 con 0x7f0c680ffe00 2023-07-21T19:06:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.248+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c580095f0 con 0x7f0c680ffe00 2023-07-21T19:06:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.248+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c681a4820 con 0x7f0c680ffe00 2023-07-21T19:06:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.248+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0c6810fc60 con 0x7f0c680ffe00 2023-07-21T19:06:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.251+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0c580051a0 con 0x7f0c680ffe00 2023-07-21T19:06:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c64ff9700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0c500a1260 0x7f0c500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0c580cdde0 con 0x7f0c680ffe00 2023-07-21T19:06:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c66ffd700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0c500a1260 0x7f0c500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c64ff9700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] conn(0x7f0c500aad90 0x7f0c500ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0c500ad8f0 con 0x7f0c500aad90 2023-07-21T19:06:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.252+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c580ce150 con 0x7f0c680ffe00 2023-07-21T19:06:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.253+0000 7f0c67fff700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] conn(0x7f0c500aad90 0x7f0c500ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.253+0000 7f0c66ffd700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0c500a1260 0x7f0c500a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c68074000 tx=0x7f0c5c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.253+0000 7f0c67fff700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] conn(0x7f0c500aad90 0x7f0c500ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.254+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== osd.6 v2:172.21.15.5:6818/3271423477 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0c500ad8f0 con 0x7f0c500aad90 2023-07-21T19:06:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.270+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 --> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0c6804e210 con 0x7f0c500aad90 2023-07-21T19:06:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c64ff9700 1 -- 172.21.15.5:0/2550468644 <== osd.6 v2:172.21.15.5:6818/3271423477 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0c6804e210 con 0x7f0c500aad90 2023-07-21T19:06:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] conn(0x7f0c500aad90 msgr2=0x7f0c500ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] conn(0x7f0c500aad90 0x7f0c500ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0c500a1260 msgr2=0x7f0c500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0c500a1260 0x7f0c500a3710 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0c68074000 tx=0x7f0c5c009260 comp rx=0 tx=0).stop 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 msgr2=0x7f0c68072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.271+0000 7f0c6de54700 1 --2- 172.21.15.5:0/2550468644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c680ffe00 0x7f0c68072810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f0c58006010 tx=0x7f0c5800f670 comp rx=0 tx=0).stop 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.272+0000 7f0c67fff700 1 -- 172.21.15.5:0/2550468644 reap_dead start 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.272+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 shutdown_connections 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.272+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 >> 172.21.15.5:0/2550468644 conn(0x7f0c680fb660 msgr2=0x7f0c681037f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.272+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 shutdown_connections 2023-07-21T19:06:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:09.272+0000 7f0c6de54700 1 -- 172.21.15.5:0/2550468644 wait complete. 2023-07-21T19:06:09.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:09 smithi005 ceph-mon[126460]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T19:06:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:09 smithi156 ceph-mon[128316]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T19:06:09.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:09 smithi101 ceph-mon[128136]: osdmap e87: 12 total, 12 up, 12 in 2023-07-21T19:06:10.172 INFO:teuthology.orchestra.run.smithi005.stdout:77309411395 2023-07-21T19:06:10.172 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.2 2023-07-21T19:06:10.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:10 smithi005 ceph-mon[126460]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 4 op/s 2023-07-21T19:06:10.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:10 smithi156 ceph-mon[128316]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 4 op/s 2023-07-21T19:06:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 -- 172.21.15.5:0/1387722909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 msgr2=0x7f1c3c10e7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 --2- 172.21.15.5:0/1387722909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c10e7c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c3400b0a0 tx=0x7f1c3400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 -- 172.21.15.5:0/1387722909 shutdown_connections 2023-07-21T19:06:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 --2- 172.21.15.5:0/1387722909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c10e7c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 --2- 172.21.15.5:0/1387722909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c0728b0 0x7f1c3c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 --2- 172.21.15.5:0/1387722909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c0731c0 0x7f1c3c10bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 -- 172.21.15.5:0/1387722909 >> 172.21.15.5:0/1387722909 conn(0x7f1c3c0fb660 msgr2=0x7f1c3c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 -- 172.21.15.5:0/1387722909 shutdown_connections 2023-07-21T19:06:10.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.814+0000 7f1c43a56700 1 -- 172.21.15.5:0/1387722909 wait complete. 2023-07-21T19:06:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.815+0000 7f1c43a56700 1 Processor -- start 2023-07-21T19:06:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 -- start start 2023-07-21T19:06:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c0728b0 0x7f1c3c197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c0731c0 0x7f1c3c198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c19c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c19ce80 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c19cff0 con 0x7f1c3c0731c0 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c43a56700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c19d1b0 con 0x7f1c3c0728b0 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c41ff3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c19c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c41ff3700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c19c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39438/0 (socket says 172.21.15.5:39438) 2023-07-21T19:06:10.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c41ff3700 1 -- 172.21.15.5:0/2282276440 learned_addr learned my addr 172.21.15.5:0/2282276440 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c417f2700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c0728b0 0x7f1c3c197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.816+0000 7f1c40ff1700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c0731c0 0x7f1c3c198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 -- 172.21.15.5:0/2282276440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c0728b0 msgr2=0x7f1c3c197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c0728b0 0x7f1c3c197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 -- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c0731c0 msgr2=0x7f1c3c198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c0731c0 0x7f1c3c198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c34009d00 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c41ff3700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c19c7e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009fd0 tx=0x7f1c34004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400e040 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c3400ceb0 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c3c19d430 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.817+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c340055d0 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.818+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c1a4c60 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.819+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1c3c04e210 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.821+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1c34005730 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.822+0000 7f1c327fc700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c280a1260 0x7f1c280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1c340ce220 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c417f2700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c280a1260 0x7f1c280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c327fc700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] conn(0x7f1c280aad90 0x7f1c280ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1c280ad8f0 con 0x7f1c280aad90 2023-07-21T19:06:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c340d0800 con 0x7f1c3c10c3e0 2023-07-21T19:06:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c417f2700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c280a1260 0x7f1c280a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005950 tx=0x7f1c3800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.823+0000 7f1c40ff1700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] conn(0x7f1c280aad90 0x7f1c280ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.824+0000 7f1c40ff1700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] conn(0x7f1c280aad90 0x7f1c280ad220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.825+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== osd.10 v2:172.21.15.101:6824/3222350780 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1c280ad8f0 con 0x7f1c280aad90 2023-07-21T19:06:10.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:10 smithi101 ceph-mon[128136]: pgmap v318: 97 pgs: 97 active+clean; 636 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 511 B/s wr, 4 op/s 2023-07-21T19:06:10.846 INFO:teuthology.orchestra.run.smithi005.stdout:107374182463 2023-07-21T19:06:10.846 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.3 2023-07-21T19:06:10.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.847+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 --> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1c3c111e20 con 0x7f1c280aad90 2023-07-21T19:06:10.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.848+0000 7f1c327fc700 1 -- 172.21.15.5:0/2282276440 <== osd.10 v2:172.21.15.101:6824/3222350780 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f1c2c007bf0 con 0x7f1c280aad90 2023-07-21T19:06:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.848+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] conn(0x7f1c280aad90 msgr2=0x7f1c280ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] conn(0x7f1c280aad90 0x7f1c280ad220 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c280a1260 msgr2=0x7f1c280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1c280a1260 0x7f1c280a3710 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005950 tx=0x7f1c3800b410 comp rx=0 tx=0).stop 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 msgr2=0x7f1c3c19c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 --2- 172.21.15.5:0/2282276440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c10c3e0 0x7f1c3c19c7e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009fd0 tx=0x7f1c34004c80 comp rx=0 tx=0).stop 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c41ff3700 1 -- 172.21.15.5:0/2282276440 reap_dead start 2023-07-21T19:06:10.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 shutdown_connections 2023-07-21T19:06:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 >> 172.21.15.5:0/2282276440 conn(0x7f1c3c0fb660 msgr2=0x7f1c3c10e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 shutdown_connections 2023-07-21T19:06:10.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.849+0000 7f1c43a56700 1 -- 172.21.15.5:0/2282276440 wait complete. 2023-07-21T19:06:10.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 -- 172.21.15.5:0/2292147316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9494101670 msgr2=0x7f9494109b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 --2- 172.21.15.5:0/2292147316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9494101670 0x7f9494109b70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f9484009990 tx=0x7f9484009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 -- 172.21.15.5:0/2292147316 shutdown_connections 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 --2- 172.21.15.5:0/2292147316 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f949410a140 0x7f949410c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 --2- 172.21.15.5:0/2292147316 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9494100d60 0x7f9494101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 --2- 172.21.15.5:0/2292147316 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9494101670 0x7f9494109b70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 -- 172.21.15.5:0/2292147316 >> 172.21.15.5:0/2292147316 conn(0x7f94940fa5a0 msgr2=0x7f94940fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 -- 172.21.15.5:0/2292147316 shutdown_connections 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.558+0000 7f949b34c700 1 -- 172.21.15.5:0/2292147316 wait complete. 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.560+0000 7f949b34c700 1 Processor -- start 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.560+0000 7f949b34c700 1 -- start start 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.560+0000 7f949b34c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9494100d60 0x7f949419b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.560+0000 7f949b34c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9494101670 0x7f949419bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f949b34c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 0x7f9494195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f949b34c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9494196070 con 0x7f949410a140 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f949b34c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94941961e0 con 0x7f9494101670 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f949b34c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94941963a0 con 0x7f9494100d60 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 0x7f9494195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 0x7f9494195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39420/0 (socket says 172.21.15.5:39420) 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 -- 172.21.15.5:0/1778754580 learned_addr learned my addr 172.21.15.5:0/1778754580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94990e8700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9494100d60 0x7f949419b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94988e7700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9494101670 0x7f949419bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 -- 172.21.15.5:0/1778754580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9494100d60 msgr2=0x7f949419b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9494100d60 0x7f949419b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 -- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9494101670 msgr2=0x7f949419bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9494101670 0x7f949419bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.561+0000 7f94998e9700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94840095f0 con 0x7f949410a140 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f94998e9700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 0x7f9494195a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f948c007ee0 tx=0x7f948c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c00dd20 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f948c00f040 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494196680 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f948c004750 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.562+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94941a2970 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.563+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f94940681a0 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.565+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f948c003a00 con 0x7f949410a140 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.566+0000 7f948a7fc700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f94800a1260 0x7f94800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.566+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f948c0ccd90 con 0x7f949410a140 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.566+0000 7f94990e8700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f94800a1260 0x7f94800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.567+0000 7f948a7fc700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] conn(0x7f94800aad90 0x7f94800ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.567+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f94800ad8f0 con 0x7f94800aad90 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.567+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f948c0cd100 con 0x7f949410a140 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.567+0000 7f94990e8700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f94800a1260 0x7f94800a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9490005950 tx=0x7f949000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.567+0000 7f94988e7700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] conn(0x7f94800aad90 0x7f94800ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:10.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.568+0000 7f94988e7700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] conn(0x7f94800aad90 0x7f94800ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.569+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== osd.7 v2:172.21.15.101:6816/1417474005 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f94800ad8f0 con 0x7f94800aad90 2023-07-21T19:06:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.585+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 --> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9494060b90 con 0x7f94800aad90 2023-07-21T19:06:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f948a7fc700 1 -- 172.21.15.5:0/1778754580 <== osd.7 v2:172.21.15.101:6816/1417474005 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f9494060b90 con 0x7f94800aad90 2023-07-21T19:06:10.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] conn(0x7f94800aad90 msgr2=0x7f94800ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] conn(0x7f94800aad90 0x7f94800ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f94800a1260 msgr2=0x7f94800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f94800a1260 0x7f94800a3710 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9490005950 tx=0x7f949000b410 comp rx=0 tx=0).stop 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 msgr2=0x7f9494195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f949b34c700 1 --2- 172.21.15.5:0/1778754580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f949410a140 0x7f9494195a00 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f948c007ee0 tx=0x7f948c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.586+0000 7f94998e9700 1 -- 172.21.15.5:0/1778754580 reap_dead start 2023-07-21T19:06:10.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.587+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 shutdown_connections 2023-07-21T19:06:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.587+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 >> 172.21.15.5:0/1778754580 conn(0x7f94940fa5a0 msgr2=0x7f94940fb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.587+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 shutdown_connections 2023-07-21T19:06:10.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:10.587+0000 7f949b34c700 1 -- 172.21.15.5:0/1778754580 wait complete. 2023-07-21T19:06:11.114 INFO:teuthology.orchestra.run.smithi005.stdout:197568495662 2023-07-21T19:06:11.115 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.6 2023-07-21T19:06:11.307 INFO:teuthology.orchestra.run.smithi005.stdout:163208757299 2023-07-21T19:06:11.307 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.5 2023-07-21T19:06:11.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 -- 172.21.15.5:0/937608523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958101700 msgr2=0x7fd958109c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/937608523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958101700 0x7fd958109c00 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fd944009990 tx=0x7fd944009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 -- 172.21.15.5:0/937608523 shutdown_connections 2023-07-21T19:06:11.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/937608523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd95810a140 0x7fd95810c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/937608523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd958100df0 0x7fd9581011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/937608523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958101700 0x7fd958109c00 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 -- 172.21.15.5:0/937608523 >> 172.21.15.5:0/937608523 conn(0x7fd958075b70 msgr2=0x7fd958075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 -- 172.21.15.5:0/937608523 shutdown_connections 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.477+0000 7fd95ecf9700 1 -- 172.21.15.5:0/937608523 wait complete. 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.479+0000 7fd95ecf9700 1 Processor -- start 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.479+0000 7fd95ecf9700 1 -- start start 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 0x7fd95819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd958101700 0x7fd95819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:11.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd95810a140 0x7fd9581959d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:11.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd958196040 con 0x7fd958100df0 2023-07-21T19:06:11.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9581961b0 con 0x7fd95810a140 2023-07-21T19:06:11.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ecf9700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd958196370 con 0x7fd958101700 2023-07-21T19:06:11.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ca95700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 0x7fd95819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:11.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ca95700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 0x7fd95819b950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39456/0 (socket says 172.21.15.5:39456) 2023-07-21T19:06:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95ca95700 1 -- 172.21.15.5:0/1248344631 learned_addr learned my addr 172.21.15.5:0/1248344631 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd957fff700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd958101700 0x7fd95819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.480+0000 7fd95d296700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd95810a140 0x7fd9581959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 -- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd958101700 msgr2=0x7fd95819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd958101700 0x7fd95819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 -- 172.21.15.5:0/1248344631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd95810a140 msgr2=0x7fd9581959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd95810a140 0x7fd9581959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd94c009520 con 0x7fd958100df0 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ca95700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 0x7fd95819b950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00eb30 tx=0x7fd94c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c009880 con 0x7fd958100df0 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd94c0044b0 con 0x7fd958100df0 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9440095f0 con 0x7fd958100df0 2023-07-21T19:06:11.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd94c003e40 con 0x7fd958100df0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.481+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd958196840 con 0x7fd958100df0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.482+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd95810fbf0 con 0x7fd958100df0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.486+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd94c003fa0 con 0x7fd958100df0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd955ffb700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9480a1260 0x7fd9480a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd94c014070 con 0x7fd958100df0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd957fff700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9480a1260 0x7fd9480a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:11.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd955ffb700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] conn(0x7fd9480aad90 0x7fd9480ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd9480ad8f0 con 0x7fd9480aad90 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.487+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd94c0cd190 con 0x7fd958100df0 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.488+0000 7fd95d296700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] conn(0x7fd9480aad90 0x7fd9480ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.488+0000 7fd957fff700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9480a1260 0x7fd9480a3710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd95810c890 tx=0x7fd94400b2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.488+0000 7fd95d296700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] conn(0x7fd9480aad90 0x7fd9480ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.489+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== osd.8 v2:172.21.15.156:6816/1480568600 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fd9480ad8f0 con 0x7fd9480aad90 2023-07-21T19:06:11.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.507+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 --> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd958060b90 con 0x7fd9480aad90 2023-07-21T19:06:11.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.507+0000 7fd955ffb700 1 -- 172.21.15.5:0/1248344631 <== osd.8 v2:172.21.15.156:6816/1480568600 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fd958060b90 con 0x7fd9480aad90 2023-07-21T19:06:11.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] conn(0x7fd9480aad90 msgr2=0x7fd9480ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] conn(0x7fd9480aad90 0x7fd9480ad220 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9480a1260 msgr2=0x7fd9480a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9480a1260 0x7fd9480a3710 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fd95810c890 tx=0x7fd94400b2a0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 msgr2=0x7fd95819b950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 --2- 172.21.15.5:0/1248344631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd958100df0 0x7fd95819b950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fd94c00eb30 tx=0x7fd94c00eef0 comp rx=0 tx=0).stop 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95d296700 1 -- 172.21.15.5:0/1248344631 reap_dead start 2023-07-21T19:06:11.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 shutdown_connections 2023-07-21T19:06:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 >> 172.21.15.5:0/1248344631 conn(0x7fd958075b70 msgr2=0x7fd958101b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 shutdown_connections 2023-07-21T19:06:11.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:11.508+0000 7fd95ecf9700 1 -- 172.21.15.5:0/1248344631 wait complete. 2023-07-21T19:06:11.821 INFO:teuthology.orchestra.run.smithi005.stdout:219043332137 2023-07-21T19:06:11.822 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.7 2023-07-21T19:06:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.059+0000 7f4287298700 1 -- 172.21.15.5:0/1305857883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42800ffe00 msgr2=0x7f42801001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.059+0000 7f4287298700 1 --2- 172.21.15.5:0/1305857883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42800ffe00 0x7f42801001d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f427c0099c0 tx=0x7f427c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 -- 172.21.15.5:0/1305857883 shutdown_connections 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 --2- 172.21.15.5:0/1305857883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f428010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 --2- 172.21.15.5:0/1305857883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42800ffe00 0x7f42801001d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 --2- 172.21.15.5:0/1305857883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42801007a0 0x7f4280104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 -- 172.21.15.5:0/1305857883 >> 172.21.15.5:0/1305857883 conn(0x7f42800fb660 msgr2=0x7f42800fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 -- 172.21.15.5:0/1305857883 shutdown_connections 2023-07-21T19:06:12.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.060+0000 7f4287298700 1 -- 172.21.15.5:0/1305857883 wait complete. 2023-07-21T19:06:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.061+0000 7f4287298700 1 Processor -- start 2023-07-21T19:06:12.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 -- start start 2023-07-21T19:06:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42800ffe00 0x7f4280072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42801007a0 0x7f4280072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f42800732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42801a3fe0 con 0x7f42801007a0 2023-07-21T19:06:12.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42801a4150 con 0x7f4280104ea0 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.062+0000 7f4287298700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42801a4310 con 0x7f42800ffe00 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4284833700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42801007a0 0x7f4280072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f42800732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f42800732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34764/0 (socket says 172.21.15.5:34764) 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 -- 172.21.15.5:0/1866347517 learned_addr learned my addr 172.21.15.5:0/1866347517 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:12.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285034700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42800ffe00 0x7f4280072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 -- 172.21.15.5:0/1866347517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42800ffe00 msgr2=0x7f4280072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42800ffe00 0x7f4280072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 -- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42801007a0 msgr2=0x7f4280072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42801007a0 0x7f4280072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f427c0095f0 con 0x7f4280104ea0 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4284833700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f42801007a0 0x7f4280072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.063+0000 7f4285835700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f42800732d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4278007ee0 tx=0x7f427800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.064+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278017070 con 0x7f4280104ea0 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.064+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42801a45f0 con 0x7f4280104ea0 2023-07-21T19:06:12.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.064+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f427800f040 con 0x7f4280104ea0 2023-07-21T19:06:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.064+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278005570 con 0x7f4280104ea0 2023-07-21T19:06:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.064+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42801a4af0 con 0x7f4280104ea0 2023-07-21T19:06:12.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.065+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f428010fc00 con 0x7f4280104ea0 2023-07-21T19:06:12.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.069+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f427800db20 con 0x7f4280104ea0 2023-07-21T19:06:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.070+0000 7f42767fc700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f426c0a1260 0x7f426c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.071+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f42780cc1c0 con 0x7f4280104ea0 2023-07-21T19:06:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.071+0000 7f4285034700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f426c0a1260 0x7f426c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.071+0000 7f42767fc700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] conn(0x7f426c0aad90 0x7f426c0ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.071+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f426c0ad8f0 con 0x7f426c0aad90 2023-07-21T19:06:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.071+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42780ce7a0 con 0x7f4280104ea0 2023-07-21T19:06:12.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.072+0000 7f4284833700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] conn(0x7f426c0aad90 0x7f426c0ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.072+0000 7f4285034700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f426c0a1260 0x7f426c0a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b580 tx=0x7f427c005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.072+0000 7f4284833700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] conn(0x7f426c0aad90 0x7f426c0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.073+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== osd.1 v2:172.21.15.101:6800/3147533687 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f426c0ad8f0 con 0x7f426c0aad90 2023-07-21T19:06:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.092+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 --> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f428004e210 con 0x7f426c0aad90 2023-07-21T19:06:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f42767fc700 1 -- 172.21.15.5:0/1866347517 <== osd.1 v2:172.21.15.101:6800/3147533687 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f428004e210 con 0x7f426c0aad90 2023-07-21T19:06:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] conn(0x7f426c0aad90 msgr2=0x7f426c0ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] conn(0x7f426c0aad90 0x7f426c0ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f426c0a1260 msgr2=0x7f426c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f426c0a1260 0x7f426c0a3710 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f427c00b580 tx=0x7f427c005c00 comp rx=0 tx=0).stop 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 msgr2=0x7f42800732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4287298700 1 --2- 172.21.15.5:0/1866347517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4280104ea0 0x7f42800732d0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f4278007ee0 tx=0x7f427800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.093+0000 7f4285835700 1 -- 172.21.15.5:0/1866347517 reap_dead start 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.094+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 shutdown_connections 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.094+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 >> 172.21.15.5:0/1866347517 conn(0x7f42800fb660 msgr2=0x7f42801037b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.094+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 shutdown_connections 2023-07-21T19:06:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.094+0000 7f4287298700 1 -- 172.21.15.5:0/1866347517 wait complete. 2023-07-21T19:06:12.315 INFO:teuthology.orchestra.run.smithi005.stdout:253403070500 2023-07-21T19:06:12.315 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.8 2023-07-21T19:06:12.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.479+0000 7f25e1957700 1 -- 172.21.15.5:0/183881273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc10a270 msgr2=0x7f25dc10c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.479+0000 7f25e1957700 1 --2- 172.21.15.5:0/183881273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc10a270 0x7f25dc10c6c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f25d400b0a0 tx=0x7f25d400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 -- 172.21.15.5:0/183881273 shutdown_connections 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 --2- 172.21.15.5:0/183881273 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc10a270 0x7f25dc10c6c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 --2- 172.21.15.5:0/183881273 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25dc100df0 0x7f25dc1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 --2- 172.21.15.5:0/183881273 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc101790 0x7f25dc109c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 -- 172.21.15.5:0/183881273 >> 172.21.15.5:0/183881273 conn(0x7f25dc075b60 msgr2=0x7f25dc075f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 -- 172.21.15.5:0/183881273 shutdown_connections 2023-07-21T19:06:12.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.480+0000 7f25e1957700 1 -- 172.21.15.5:0/183881273 wait complete. 2023-07-21T19:06:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.481+0000 7f25e1957700 1 Processor -- start 2023-07-21T19:06:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.481+0000 7f25e1957700 1 -- start start 2023-07-21T19:06:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 0x7f25dc06a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc101790 0x7f25dc06af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25dc10a270 0x7f25dc1aa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1aa760 con 0x7f25dc100df0 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1aa8d0 con 0x7f25dc10a270 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25e1957700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25dc1aaa90 con 0x7f25dc101790 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 0x7f25dc06a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25daffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 0x7f25dc06a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39488/0 (socket says 172.21.15.5:39488) 2023-07-21T19:06:12.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.482+0000 7f25daffd700 1 -- 172.21.15.5:0/4288881192 learned_addr learned my addr 172.21.15.5:0/4288881192 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:12.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 -- 172.21.15.5:0/4288881192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc101790 msgr2=0x7f25dc06af20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:06:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25dc101790 0x7f25dc06af20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 -- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25dc10a270 msgr2=0x7f25dc1aa1f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:06:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25dc10a270 0x7f25dc1aa1f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d4009d00 con 0x7f25dc100df0 2023-07-21T19:06:12.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25daffd700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 0x7f25dc06a9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00e9a0 tx=0x7f25cc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.483+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc009790 con 0x7f25dc100df0 2023-07-21T19:06:12.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.484+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25cc00cbd0 con 0x7f25dc100df0 2023-07-21T19:06:12.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.484+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25dc0feaa0 con 0x7f25dc100df0 2023-07-21T19:06:12.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.484+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc018680 con 0x7f25dc100df0 2023-07-21T19:06:12.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.484+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25dc0feff0 con 0x7f25dc100df0 2023-07-21T19:06:12.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.485+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f25dc10fd60 con 0x7f25dc100df0 2023-07-21T19:06:12.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.488+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f25cc00cd40 con 0x7f25dc100df0 2023-07-21T19:06:12.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25e0955700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f25c40a1260 0x7f25c40a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f25cc014070 con 0x7f25dc100df0 2023-07-21T19:06:12.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25da7fc700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f25c40a1260 0x7f25c40a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25e0955700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] conn(0x7f25c40aad90 0x7f25c40ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:12.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f25c40ad8f0 con 0x7f25c40aad90 2023-07-21T19:06:12.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.490+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25cc0cd930 con 0x7f25dc100df0 2023-07-21T19:06:12.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.491+0000 7f25da7fc700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f25c40a1260 0x7f25c40a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0fce90 tx=0x7f25d0006c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.500+0000 7f25db7fe700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] conn(0x7f25c40aad90 0x7f25c40ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.508+0000 7f25db7fe700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] conn(0x7f25c40aad90 0x7f25c40ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.509+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== osd.4 v2:172.21.15.101:6808/1960590612 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f25c40ad8f0 con 0x7f25c40aad90 2023-07-21T19:06:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.526+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 --> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f25dc04e210 con 0x7f25c40aad90 2023-07-21T19:06:12.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.527+0000 7f25e0955700 1 -- 172.21.15.5:0/4288881192 <== osd.4 v2:172.21.15.101:6808/1960590612 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f25dc04e210 con 0x7f25c40aad90 2023-07-21T19:06:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.527+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] conn(0x7f25c40aad90 msgr2=0x7f25c40ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.527+0000 7f25e1957700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] conn(0x7f25c40aad90 0x7f25c40ad220 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f25c40a1260 msgr2=0x7f25c40a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f25c40a1260 0x7f25c40a3710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f25dc0fce90 tx=0x7f25d0006c60 comp rx=0 tx=0).stop 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 msgr2=0x7f25dc06a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 --2- 172.21.15.5:0/4288881192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25dc100df0 0x7f25dc06a9e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f25cc00e9a0 tx=0x7f25cc00ed60 comp rx=0 tx=0).stop 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25db7fe700 1 -- 172.21.15.5:0/4288881192 reap_dead start 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 shutdown_connections 2023-07-21T19:06:12.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 >> 172.21.15.5:0/4288881192 conn(0x7f25dc075b60 msgr2=0x7f25dc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 shutdown_connections 2023-07-21T19:06:12.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:12.528+0000 7f25e1957700 1 -- 172.21.15.5:0/4288881192 wait complete. 2023-07-21T19:06:12.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:12 smithi005 ceph-mon[126460]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 7 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T19:06:12.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:12 smithi156 ceph-mon[128316]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 7 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T19:06:12.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:12 smithi101 ceph-mon[128136]: pgmap v319: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 895 B/s wr, 7 op/s; 120 B/s, 0 objects/s recovering 2023-07-21T19:06:13.132 INFO:teuthology.orchestra.run.smithi005.stdout:55834574922 2023-07-21T19:06:13.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.1 2023-07-21T19:06:13.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 -- 172.21.15.5:0/139880153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c101570 msgr2=0x7fc64c109c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 --2- 172.21.15.5:0/139880153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c101570 0x7fc64c109c40 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc63c009990 tx=0x7fc63c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 -- 172.21.15.5:0/139880153 shutdown_connections 2023-07-21T19:06:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 --2- 172.21.15.5:0/139880153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc64c10a180 0x7fc64c10c5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 --2- 172.21.15.5:0/139880153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc64c100c60 0x7fc64c101030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 --2- 172.21.15.5:0/139880153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c101570 0x7fc64c109c40 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 -- 172.21.15.5:0/139880153 >> 172.21.15.5:0/139880153 conn(0x7fc64c075b70 msgr2=0x7fc64c075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 -- 172.21.15.5:0/139880153 shutdown_connections 2023-07-21T19:06:13.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.211+0000 7fc65332c700 1 -- 172.21.15.5:0/139880153 wait complete. 2023-07-21T19:06:13.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.212+0000 7fc65332c700 1 Processor -- start 2023-07-21T19:06:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 -- start start 2023-07-21T19:06:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc64c100c60 0x7fc64c0fea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc64c101570 0x7fc64c0fefd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:13.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 0x7fc64c0ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:13.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c1a3f40 con 0x7fc64c10a180 2023-07-21T19:06:13.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c1a40b0 con 0x7fc64c101570 2023-07-21T19:06:13.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.213+0000 7fc65332c700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc64c1a4270 con 0x7fc64c100c60 2023-07-21T19:06:13.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 0x7fc64c0ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:13.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6510c8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc64c100c60 0x7fc64c0fea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:13.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 0x7fc64c0ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39518/0 (socket says 172.21.15.5:39518) 2023-07-21T19:06:13.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 -- 172.21.15.5:0/640441766 learned_addr learned my addr 172.21.15.5:0/640441766 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:13.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6508c7700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc64c101570 0x7fc64c0fefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 -- 172.21.15.5:0/640441766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc64c100c60 msgr2=0x7fc64c0fea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc64c100c60 0x7fc64c0fea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 -- 172.21.15.5:0/640441766 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc64c101570 msgr2=0x7fc64c0fefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc64c101570 0x7fc64c0fefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc63c0095f0 con 0x7fc64c10a180 2023-07-21T19:06:13.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.214+0000 7fc6518c9700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 0x7fc64c0ff550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc64400cc30 tx=0x7fc64400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:13.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.215+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64400aac0 con 0x7fc64c10a180 2023-07-21T19:06:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.215+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc64400ac20 con 0x7fc64c10a180 2023-07-21T19:06:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.215+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc64c1a4550 con 0x7fc64c10a180 2023-07-21T19:06:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.215+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440115e0 con 0x7fc64c10a180 2023-07-21T19:06:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.215+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc64c1a4a20 con 0x7fc64c10a180 2023-07-21T19:06:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.216+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc64c10fc70 con 0x7fc64c10a180 2023-07-21T19:06:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.218+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc644012420 con 0x7fc64c10a180 2023-07-21T19:06:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.219+0000 7fc6427fc700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc6380a1260 0x7fc6380a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.219+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc6440ccef0 con 0x7fc64c10a180 2023-07-21T19:06:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6510c8700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc6380a1260 0x7fc6380a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6427fc700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] conn(0x7fc6380aad90 0x7fc6380ad220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc6380ad8f0 con 0x7fc6380aad90 2023-07-21T19:06:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6440cd260 con 0x7fc64c10a180 2023-07-21T19:06:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6508c7700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] conn(0x7fc6380aad90 0x7fc6380ad220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:13.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.220+0000 7fc6510c8700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc6380a1260 0x7fc6380a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc64800ac50 tx=0x7fc64800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:13.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.221+0000 7fc6508c7700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] conn(0x7fc6380aad90 0x7fc6380ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:13.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.222+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== osd.9 v2:172.21.15.5:6826/3583557754 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc6380ad8f0 con 0x7fc6380aad90 2023-07-21T19:06:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.239+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 --> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc64c04e210 con 0x7fc6380aad90 2023-07-21T19:06:13.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc6427fc700 1 -- 172.21.15.5:0/640441766 <== osd.9 v2:172.21.15.5:6826/3583557754 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fc64c04e210 con 0x7fc6380aad90 2023-07-21T19:06:13.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] conn(0x7fc6380aad90 msgr2=0x7fc6380ad220 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] conn(0x7fc6380aad90 0x7fc6380ad220 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc6380a1260 msgr2=0x7fc6380a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc6380a1260 0x7fc6380a3710 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fc64800ac50 tx=0x7fc64800a380 comp rx=0 tx=0).stop 2023-07-21T19:06:13.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 msgr2=0x7fc64c0ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 --2- 172.21.15.5:0/640441766 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc64c10a180 0x7fc64c0ff550 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc64400cc30 tx=0x7fc64400cf40 comp rx=0 tx=0).stop 2023-07-21T19:06:13.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc6518c9700 1 -- 172.21.15.5:0/640441766 reap_dead start 2023-07-21T19:06:13.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.240+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 shutdown_connections 2023-07-21T19:06:13.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.241+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 >> 172.21.15.5:0/640441766 conn(0x7fc64c075b70 msgr2=0x7fc64c103c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.241+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 shutdown_connections 2023-07-21T19:06:13.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:13.241+0000 7fc65332c700 1 -- 172.21.15.5:0/640441766 wait complete. 2023-07-21T19:06:13.488 INFO:teuthology.orchestra.run.smithi005.stdout:309237645338 2023-07-21T19:06:13.489 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.10 2023-07-21T19:06:13.490 INFO:teuthology.orchestra.run.smithi005.stdout:141733920825 2023-07-21T19:06:13.490 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.4 2023-07-21T19:06:13.758 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:14.400 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:14.411 INFO:teuthology.orchestra.run.smithi005.stdout:287762808864 2023-07-21T19:06:14.411 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph osd last-stat-seq osd.9 2023-07-21T19:06:14.483 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:14.602 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:14 smithi005 ceph-mon[126460]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 4 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:14.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:14 smithi156 ceph-mon[128316]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 4 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:14.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:14 smithi101 ceph-mon[128136]: pgmap v320: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 383 B/s wr, 4 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:15.117 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:15.935 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:16.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:16 smithi156 ceph-mon[128316]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:16.703 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:16.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:16 smithi101 ceph-mon[128136]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:16.841 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:16 smithi005 ceph-mon[126460]: pgmap v321: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 383 B/s wr, 6 op/s; 195 B/s, 0 objects/s recovering 2023-07-21T19:06:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:17.550 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.576+0000 7f4c662ca700 1 -- 172.21.15.5:0/4087990289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c600febe0 msgr2=0x7f4c600fefb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.576+0000 7f4c662ca700 1 --2- 172.21.15.5:0/4087990289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c600febe0 0x7f4c600fefb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4c50009990 tx=0x7f4c50009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.577+0000 7f4c662ca700 1 -- 172.21.15.5:0/4087990289 shutdown_connections 2023-07-21T19:06:17.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.577+0000 7f4c662ca700 1 --2- 172.21.15.5:0/4087990289 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c60103f10 0x7f4c60104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.577+0000 7f4c662ca700 1 --2- 172.21.15.5:0/4087990289 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c600febe0 0x7f4c600fefb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.577+0000 7f4c662ca700 1 --2- 172.21.15.5:0/4087990289 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600ff4f0 0x7f4c601039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.577+0000 7f4c662ca700 1 -- 172.21.15.5:0/4087990289 >> 172.21.15.5:0/4087990289 conn(0x7f4c60075b70 msgr2=0x7f4c60075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:17.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.580+0000 7f4c662ca700 1 -- 172.21.15.5:0/4087990289 shutdown_connections 2023-07-21T19:06:17.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.580+0000 7f4c662ca700 1 -- 172.21.15.5:0/4087990289 wait complete. 2023-07-21T19:06:17.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.581+0000 7f4c662ca700 1 Processor -- start 2023-07-21T19:06:17.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.581+0000 7f4c662ca700 1 -- start start 2023-07-21T19:06:17.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600ff4f0 0x7f4c6019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c60103f10 0x7f4c601a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601a11b0 con 0x7f4c600ff4f0 2023-07-21T19:06:17.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601a1320 con 0x7f4c60103f10 2023-07-21T19:06:17.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c662ca700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c601a14e0 con 0x7f4c600febe0 2023-07-21T19:06:17.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c5ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36072/0 (socket says 172.21.15.5:36072) 2023-07-21T19:06:17.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c5ffff700 1 -- 172.21.15.5:0/1096575782 learned_addr learned my addr 172.21.15.5:0/1096575782 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:17.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c5f7fe700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600ff4f0 0x7f4c6019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.582+0000 7f4c64867700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c60103f10 0x7f4c601a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 -- 172.21.15.5:0/1096575782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c60103f10 msgr2=0x7f4c601a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c60103f10 0x7f4c601a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 -- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600ff4f0 msgr2=0x7f4c6019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600ff4f0 0x7f4c6019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 -- 172.21.15.5:0/1096575782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c500095f0 con 0x7f4c600febe0 2023-07-21T19:06:17.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5ffff700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005770 tx=0x7f4c500057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:17.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5001d070 con 0x7f4c600febe0 2023-07-21T19:06:17.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c5000be30 con 0x7f4c600febe0 2023-07-21T19:06:17.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.583+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c601a1760 con 0x7f4c600febe0 2023-07-21T19:06:17.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.584+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5000fd70 con 0x7f4c600febe0 2023-07-21T19:06:17.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.584+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c601a9020 con 0x7f4c600febe0 2023-07-21T19:06:17.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.585+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6004e210 con 0x7f4c600febe0 2023-07-21T19:06:17.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.588+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4c5000f460 con 0x7f4c600febe0 2023-07-21T19:06:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.590+0000 7f4c5d7fa700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 0x7f4c480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.591+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4c500cdc40 con 0x7f4c600febe0 2023-07-21T19:06:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.591+0000 7f4c5f7fe700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 0x7f4c480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.591+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4c500972d0 con 0x7f4c600febe0 2023-07-21T19:06:17.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.595+0000 7f4c5f7fe700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 0x7f4c480a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c6019d810 tx=0x7f4c54009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:17.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.841+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7f4c601a91b0 con 0x7f4c600febe0 2023-07-21T19:06:17.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.842+0000 7f4c5d7fa700 1 -- 172.21.15.5:0/1096575782 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7f4c5009a8f0 con 0x7f4c600febe0 2023-07-21T19:06:17.854 INFO:teuthology.orchestra.run.smithi005.stdout:343597383701 2023-07-21T19:06:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 msgr2=0x7f4c480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 0x7f4c480a3690 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4c6019d810 tx=0x7f4c54009260 comp rx=0 tx=0).stop 2023-07-21T19:06:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 msgr2=0x7f4c6019c1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f4c50005770 tx=0x7f4c500057a0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 shutdown_connections 2023-07-21T19:06:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4c480a11e0 0x7f4c480a3690 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c60103f10 0x7f4c601a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c600febe0 0x7f4c6019c1f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 --2- 172.21.15.5:0/1096575782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c600ff4f0 0x7f4c6019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 >> 172.21.15.5:0/1096575782 conn(0x7f4c60075b70 msgr2=0x7f4c60102a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 shutdown_connections 2023-07-21T19:06:17.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.845+0000 7f4c662ca700 1 -- 172.21.15.5:0/1096575782 wait complete. 2023-07-21T19:06:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 -- 172.21.15.5:0/3473501749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 msgr2=0x7fb16c10ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 --2- 172.21.15.5:0/3473501749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c10ca20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb16400b0d0 tx=0x7fb16400b3e0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 -- 172.21.15.5:0/3473501749 shutdown_connections 2023-07-21T19:06:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 --2- 172.21.15.5:0/3473501749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c10ca20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 --2- 172.21.15.5:0/3473501749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c0ffe00 0x7fb16c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 --2- 172.21.15.5:0/3473501749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c1007a0 0x7fb16c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 -- 172.21.15.5:0/3473501749 >> 172.21.15.5:0/3473501749 conn(0x7fb16c0fb660 msgr2=0x7fb16c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 -- 172.21.15.5:0/3473501749 shutdown_connections 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.893+0000 7fb170832700 1 -- 172.21.15.5:0/3473501749 wait complete. 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.894+0000 7fb170832700 1 Processor -- start 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.894+0000 7fb170832700 1 -- start start 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 0x7fb16c197fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 0x7fb16c198510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c19cef0 con 0x7fb16c104ea0 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c19d060 con 0x7fb16c1007a0 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb170832700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb16c19d220 con 0x7fb16c0ffe00 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb16ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb16ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38546/0 (socket says 172.21.15.5:38546) 2023-07-21T19:06:17.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb16ad9d700 1 -- 172.21.15.5:0/377388353 learned_addr learned my addr 172.21.15.5:0/377388353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb169d9b700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 0x7fb16c198510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb16a59c700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 0x7fb16c197fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.895+0000 7fb16ad9d700 1 -- 172.21.15.5:0/377388353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 msgr2=0x7fb16c197fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16ad9d700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 0x7fb16c197fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16ad9d700 1 -- 172.21.15.5:0/377388353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 msgr2=0x7fb16c198510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16ad9d700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 0x7fb16c198510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16ad9d700 1 -- 172.21.15.5:0/377388353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb164009d00 con 0x7fb16c104ea0 2023-07-21T19:06:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb169d9b700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 0x7fb16c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16a59c700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 0x7fb16c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb16ad9d700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb164008ec0 tx=0x7fb164008fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16400e040 con 0x7fb16c104ea0 2023-07-21T19:06:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb16c19d4a0 con 0x7fb16c104ea0 2023-07-21T19:06:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb164003c00 con 0x7fb16c104ea0 2023-07-21T19:06:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16401b730 con 0x7fb16c104ea0 2023-07-21T19:06:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.896+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb16c1a4d60 con 0x7fb16c104ea0 2023-07-21T19:06:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.898+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb16c04e210 con 0x7fb16c104ea0 2023-07-21T19:06:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.901+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb164005090 con 0x7fb16c104ea0 2023-07-21T19:06:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.902+0000 7fb15b7fe700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 0x7fb1540a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.903+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb1640ce1b0 con 0x7fb16c104ea0 2023-07-21T19:06:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.903+0000 7fb16a59c700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 0x7fb1540a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.903+0000 7fb16a59c700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 0x7fb1540a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb15c005fd0 tx=0x7fb15c005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:17.905+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb1640977e0 con 0x7fb16c104ea0 2023-07-21T19:06:18.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.009+0000 7fb9bee69700 1 -- 172.21.15.5:0/55964494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b8103f10 msgr2=0x7fb9b8104340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.009+0000 7fb9bee69700 1 --2- 172.21.15.5:0/55964494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b8103f10 0x7fb9b8104340 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb9b0009f90 tx=0x7fb9b000b330 comp rx=0 tx=0).stop 2023-07-21T19:06:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 -- 172.21.15.5:0/55964494 shutdown_connections 2023-07-21T19:06:18.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 --2- 172.21.15.5:0/55964494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b8103f10 0x7fb9b8104340 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 --2- 172.21.15.5:0/55964494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9b80febe0 0x7fb9b80fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 --2- 172.21.15.5:0/55964494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb9b80ff4f0 0x7fb9b81039d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 -- 172.21.15.5:0/55964494 >> 172.21.15.5:0/55964494 conn(0x7fb9b8075b70 msgr2=0x7fb9b8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:18.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.010+0000 7fb9bee69700 1 -- 172.21.15.5:0/55964494 shutdown_connections 2023-07-21T19:06:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.012+0000 7fb9bee69700 1 -- 172.21.15.5:0/55964494 wait complete. 2023-07-21T19:06:18.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.014+0000 7fb9bee69700 1 Processor -- start 2023-07-21T19:06:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.014+0000 7fb9bee69700 1 -- start start 2023-07-21T19:06:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:18.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb9b80ff4f0 0x7fb9b819c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9b8103f10 0x7fb9b81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b81a12a0 con 0x7fb9b80febe0 2023-07-21T19:06:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b81a1410 con 0x7fb9b80ff4f0 2023-07-21T19:06:18.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.015+0000 7fb9bee69700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9b81a15d0 con 0x7fb9b8103f10 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38562/0 (socket says 172.21.15.5:38562) 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 -- 172.21.15.5:0/3693465944 learned_addr learned my addr 172.21.15.5:0/3693465944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 -- 172.21.15.5:0/3693465944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9b8103f10 msgr2=0x7fb9b81a0c90 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9b8103f10 0x7fb9b81a0c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 -- 172.21.15.5:0/3693465944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb9b80ff4f0 msgr2=0x7fb9b819c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb9b80ff4f0 0x7fb9b819c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.016+0000 7fb9bcc05700 1 -- 172.21.15.5:0/3693465944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9b0009c40 con 0x7fb9b80febe0 2023-07-21T19:06:18.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.017+0000 7fb9bcc05700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00b550 tx=0x7fb9ac00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.017+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac011830 con 0x7fb9b80febe0 2023-07-21T19:06:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.017+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb9ac011e20 con 0x7fb9b80febe0 2023-07-21T19:06:18.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.017+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9ac010e10 con 0x7fb9b80febe0 2023-07-21T19:06:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.020+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9b81a18b0 con 0x7fb9b80febe0 2023-07-21T19:06:18.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.020+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9b8105170 con 0x7fb9b80febe0 2023-07-21T19:06:18.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.021+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9b81a1aa0 con 0x7fb9b80febe0 2023-07-21T19:06:18.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.028+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb9ac011990 con 0x7fb9b80febe0 2023-07-21T19:06:18.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.030+0000 7fb9b5ffb700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 0x7fb9a80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:18.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.031+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb9ac0cc9f0 con 0x7fb9b80febe0 2023-07-21T19:06:18.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.032+0000 7fb9b7fff700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 0x7fb9a80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:18.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.033+0000 7fb9b7fff700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 0x7fb9a80a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005950 tx=0x7fb9a40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:18.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.034+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb9ac0960a0 con 0x7fb9b80febe0 2023-07-21T19:06:18.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.146+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fb16c1a4ef0 con 0x7fb16c104ea0 2023-07-21T19:06:18.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.147+0000 7fb15b7fe700 1 -- 172.21.15.5:0/377388353 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb164011390 con 0x7fb16c104ea0 2023-07-21T19:06:18.158 INFO:teuthology.orchestra.run.smithi005.stdout:34359738448 2023-07-21T19:06:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 msgr2=0x7fb1540a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 0x7fb1540a3710 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb15c005fd0 tx=0x7fb15c005ee0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 msgr2=0x7fb16c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb164008ec0 tx=0x7fb164008fa0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 shutdown_connections 2023-07-21T19:06:18.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb1540a1260 0x7fb1540a3710 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb16c104ea0 0x7fb16c19c8e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb16c0ffe00 0x7fb16c197fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 --2- 172.21.15.5:0/377388353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb16c1007a0 0x7fb16c198510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.150+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 >> 172.21.15.5:0/377388353 conn(0x7fb16c0fb660 msgr2=0x7fb16c0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:18.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.151+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 shutdown_connections 2023-07-21T19:06:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.154+0000 7fb170832700 1 -- 172.21.15.5:0/377388353 wait complete. 2023-07-21T19:06:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.308+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb9b804e210 con 0x7fb9b80febe0 2023-07-21T19:06:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.309+0000 7fb9b5ffb700 1 -- 172.21.15.5:0/3693465944 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb9ac0996c0 con 0x7fb9b80febe0 2023-07-21T19:06:18.320 INFO:teuthology.orchestra.run.smithi005.stdout:107374182464 2023-07-21T19:06:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 msgr2=0x7fb9a80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 0x7fb9a80a3710 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fb9a4005950 tx=0x7fb9a40058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 msgr2=0x7fb9b819c2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb9ac00b550 tx=0x7fb9ac00b910 comp rx=0 tx=0).stop 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 shutdown_connections 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb9a80a1260 0x7fb9a80a3710 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb9b8103f10 0x7fb9b81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb9b80febe0 0x7fb9b819c2e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.311+0000 7fb9bee69700 1 --2- 172.21.15.5:0/3693465944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb9b80ff4f0 0x7fb9b819c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.312+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 >> 172.21.15.5:0/3693465944 conn(0x7fb9b8075b70 msgr2=0x7fb9b8101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:18.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.312+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 shutdown_connections 2023-07-21T19:06:18.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:18.312+0000 7fb9bee69700 1 -- 172.21.15.5:0/3693465944 wait complete. 2023-07-21T19:06:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:18 smithi101 ceph-mon[128136]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 157 B/s, 0 objects/s recovering 2023-07-21T19:06:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:18 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1096575782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T19:06:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:18 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/377388353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T19:06:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:18 smithi005 ceph-mon[126460]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 157 B/s, 0 objects/s recovering 2023-07-21T19:06:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:18 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1096575782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T19:06:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:18 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/377388353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T19:06:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:18 smithi156 ceph-mon[128316]: pgmap v322: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 309 B/s wr, 5 op/s; 157 B/s, 0 objects/s recovering 2023-07-21T19:06:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:18 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1096575782' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-07-21T19:06:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:18 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/377388353' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-07-21T19:06:19.070 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738446 got 34359738448 for osd.0 2023-07-21T19:06:19.071 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:19.279 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383699 got 343597383701 for osd.11 2023-07-21T19:06:19.280 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:19.312 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:19 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3693465944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T19:06:19.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:19 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3693465944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T19:06:19.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:19 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3693465944' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-07-21T19:06:20.375 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182463 got 107374182464 for osd.3 2023-07-21T19:06:20.376 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:20.551 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:20.552 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:20.583 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:20 smithi005 ceph-mon[126460]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 4 op/s; 143 B/s, 0 objects/s recovering 2023-07-21T19:06:20.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:20 smithi101 ceph-mon[128136]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 4 op/s; 143 B/s, 0 objects/s recovering 2023-07-21T19:06:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:20 smithi156 ceph-mon[128316]: pgmap v323: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 281 B/s wr, 4 op/s; 143 B/s, 0 objects/s recovering 2023-07-21T19:06:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.717+0000 7fe2c9418700 1 -- 172.21.15.5:0/4181342707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c40ff4f0 msgr2=0x7fe2c41039d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.717+0000 7fe2c9418700 1 --2- 172.21.15.5:0/4181342707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c41039d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8009990 tx=0x7fe2b8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.717+0000 7fe2c9418700 1 -- 172.21.15.5:0/4181342707 shutdown_connections 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 --2- 172.21.15.5:0/4181342707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c4103f10 0x7fe2c4104340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 --2- 172.21.15.5:0/4181342707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40febe0 0x7fe2c40fefb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 --2- 172.21.15.5:0/4181342707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c41039d0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 -- 172.21.15.5:0/4181342707 >> 172.21.15.5:0/4181342707 conn(0x7fe2c4075b70 msgr2=0x7fe2c4075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 -- 172.21.15.5:0/4181342707 shutdown_connections 2023-07-21T19:06:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.718+0000 7fe2c9418700 1 -- 172.21.15.5:0/4181342707 wait complete. 2023-07-21T19:06:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.719+0000 7fe2c9418700 1 Processor -- start 2023-07-21T19:06:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.719+0000 7fe2c9418700 1 -- start start 2023-07-21T19:06:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 0x7fe2c41a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41a1160 con 0x7fe2c4103f10 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41a12d0 con 0x7fe2c40ff4f0 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c9418700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c41a1490 con 0x7fe2c40febe0 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36138/0 (socket says 172.21.15.5:36138) 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c2ffd700 1 -- 172.21.15.5:0/2188133092 learned_addr learned my addr 172.21.15.5:0/2188133092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c27fc700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.720+0000 7fe2c37fe700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 0x7fe2c41a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 -- 172.21.15.5:0/2188133092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40ff4f0 msgr2=0x7fe2c419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 -- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 msgr2=0x7fe2c41a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 0x7fe2c41a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 -- 172.21.15.5:0/2188133092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b80095f0 con 0x7fe2c40febe0 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c37fe700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 0x7fe2c41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c2ffd700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9a0 tx=0x7fe2b400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4009790 con 0x7fe2c40febe0 2023-07-21T19:06:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c41a1770 con 0x7fe2c40febe0 2023-07-21T19:06:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b400cbd0 con 0x7fe2c40febe0 2023-07-21T19:06:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.721+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b4018680 con 0x7fe2c40febe0 2023-07-21T19:06:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.722+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c41a8fd0 con 0x7fe2c40febe0 2023-07-21T19:06:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.723+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c404e210 con 0x7fe2c40febe0 2023-07-21T19:06:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.726+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe2b400cd40 con 0x7fe2c40febe0 2023-07-21T19:06:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.727+0000 7fe2abfff700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 0x7fe2ac0a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.727+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe2b4014070 con 0x7fe2c40febe0 2023-07-21T19:06:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.727+0000 7fe2c27fc700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 0x7fe2ac0a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.728+0000 7fe2c27fc700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 0x7fe2ac0a3790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2c419d7c0 tx=0x7fe2b800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:21.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.729+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe2b40963d0 con 0x7fe2c40febe0 2023-07-21T19:06:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 -- 172.21.15.5:0/1190043206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 msgr2=0x7fe694102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 --2- 172.21.15.5:0/1190043206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694102410 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6900099c0 tx=0x7fe690009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 -- 172.21.15.5:0/1190043206 shutdown_connections 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 --2- 172.21.15.5:0/1190043206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 0x7fe69410ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 --2- 172.21.15.5:0/1190043206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694102410 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 --2- 172.21.15.5:0/1190043206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe6941069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 -- 172.21.15.5:0/1190043206 >> 172.21.15.5:0/1190043206 conn(0x7fe6940fd900 msgr2=0x7fe6940ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 -- 172.21.15.5:0/1190043206 shutdown_connections 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.683+0000 7fe69bf8b700 1 -- 172.21.15.5:0/1190043206 wait complete. 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.684+0000 7fe69bf8b700 1 Processor -- start 2023-07-21T19:06:21.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 -- start start 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe694198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 0x7fe69419c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69419cdb0 con 0x7fe694102040 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69419cf20 con 0x7fe6941070e0 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.685+0000 7fe69bf8b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69419d0e0 con 0x7fe6941029e0 2023-07-21T19:06:21.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699526700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe694198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699526700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe694198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36118/0 (socket says 172.21.15.5:36118) 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699526700 1 -- 172.21.15.5:0/3927370359 learned_addr learned my addr 172.21.15.5:0/3927370359 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe69a528700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 0x7fe69419c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 -- 172.21.15.5:0/3927370359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 msgr2=0x7fe694198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe694198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 -- 172.21.15.5:0/3927370359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 msgr2=0x7fe69419c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 0x7fe69419c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.686+0000 7fe699d27700 1 -- 172.21.15.5:0/3927370359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6900095f0 con 0x7fe694102040 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe699d27700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe690009990 tx=0x7fe690009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69001d070 con 0x7fe694102040 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe69bf8b700 1 -- 172.21.15.5:0/3927370359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe69419d360 con 0x7fe694102040 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe690003ec0 con 0x7fe694102040 2023-07-21T19:06:21.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe690022bc0 con 0x7fe694102040 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.687+0000 7fe69bf8b700 1 -- 172.21.15.5:0/3927370359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6941a4c20 con 0x7fe694102040 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.688+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69404e210 con 0x7fe694102040 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.692+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe690003a30 con 0x7fe694102040 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.695+0000 7fe68affd700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 0x7fe6800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.695+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe6900ce4c0 con 0x7fe694102040 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.695+0000 7fe699526700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 0x7fe6800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:21.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.696+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe69009b5e0 con 0x7fe694102040 2023-07-21T19:06:21.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.696+0000 7fe699526700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 0x7fe6800a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe694199410 tx=0x7fe684007480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.956+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fe694199150 con 0x7fe694102040 2023-07-21T19:06:21.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.957+0000 7fe68affd700 1 -- 172.21.15.5:0/3927370359 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe69009b170 con 0x7fe694102040 2023-07-21T19:06:21.968 INFO:teuthology.orchestra.run.smithi005.stdout:163208757302 2023-07-21T19:06:21.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 msgr2=0x7fe6800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 0x7fe6800a3710 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe694199410 tx=0x7fe684007480 comp rx=0 tx=0).stop 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 msgr2=0x7fe694197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694197df0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fe690009990 tx=0x7fe690009580 comp rx=0 tx=0).stop 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 shutdown_connections 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe6800a1260 0x7fe6800a3710 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe6941070e0 0x7fe69419c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe694102040 0x7fe694197df0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 --2- 172.21.15.5:0/3927370359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe6941029e0 0x7fe694198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 >> 172.21.15.5:0/3927370359 conn(0x7fe6940fd900 msgr2=0x7fe6940fe410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:21.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 shutdown_connections 2023-07-21T19:06:21.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.960+0000 7fe688ff9700 1 -- 172.21.15.5:0/3927370359 wait complete. 2023-07-21T19:06:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:21.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.982+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fe2c419d500 con 0x7fe2c40febe0 2023-07-21T19:06:21.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.983+0000 7fe2abfff700 1 -- 172.21.15.5:0/2188133092 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe2b40999f0 con 0x7fe2c40febe0 2023-07-21T19:06:21.994 INFO:teuthology.orchestra.run.smithi005.stdout:77309411398 2023-07-21T19:06:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 msgr2=0x7fe2ac0a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 0x7fe2ac0a3790 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fe2c419d7c0 tx=0x7fe2b800b260 comp rx=0 tx=0).stop 2023-07-21T19:06:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 msgr2=0x7fe2c419c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe2b400e9a0 tx=0x7fe2b400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 shutdown_connections 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe2ac0a12e0 0x7fe2ac0a3790 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2c4103f10 0x7fe2c41a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2c40febe0 0x7fe2c419c1a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 --2- 172.21.15.5:0/2188133092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2c40ff4f0 0x7fe2c419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.986+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 >> 172.21.15.5:0/2188133092 conn(0x7fe2c4075b70 msgr2=0x7fe2c4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.987+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 shutdown_connections 2023-07-21T19:06:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:21.987+0000 7fe2c9418700 1 -- 172.21.15.5:0/2188133092 wait complete. 2023-07-21T19:06:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 -- 172.21.15.5:0/3729950931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280728b0 msgr2=0x7fdf28072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/3729950931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280728b0 0x7fdf28072c80 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fdf18009990 tx=0x7fdf18009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 -- 172.21.15.5:0/3729950931 shutdown_connections 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/3729950931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf28112f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/3729950931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280728b0 0x7fdf28072c80 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/3729950931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280731c0 0x7fdf28110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 -- 172.21.15.5:0/3729950931 >> 172.21.15.5:0/3729950931 conn(0x7fdf280fb660 msgr2=0x7fdf280fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 -- 172.21.15.5:0/3729950931 shutdown_connections 2023-07-21T19:06:22.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.236+0000 7fdf2d11d700 1 -- 172.21.15.5:0/3729950931 wait complete. 2023-07-21T19:06:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.238+0000 7fdf2d11d700 1 Processor -- start 2023-07-21T19:06:22.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.238+0000 7fdf2d11d700 1 -- start start 2023-07-21T19:06:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 0x7fdf2819c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf281a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a11b0 con 0x7fdf280731c0 2023-07-21T19:06:22.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a1320 con 0x7fdf28110bb0 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2d11d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf281a14e0 con 0x7fdf280728b0 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf26d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 0x7fdf2819c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38618/0 (socket says 172.21.15.5:38618) 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2659c700 1 -- 172.21.15.5:0/931719995 learned_addr learned my addr 172.21.15.5:0/931719995 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:22.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2759e700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf281a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.239+0000 7fdf2659c700 1 -- 172.21.15.5:0/931719995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 msgr2=0x7fdf2819c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2659c700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 0x7fdf2819c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2659c700 1 -- 172.21.15.5:0/931719995 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 msgr2=0x7fdf281a0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2659c700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf281a0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2659c700 1 -- 172.21.15.5:0/931719995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf180095f0 con 0x7fdf280731c0 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf26d9d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 0x7fdf2819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:22.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2759e700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf281a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2659c700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9a0 tx=0x7fdf1c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c009790 con 0x7fdf280731c0 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf1c00cbd0 con 0x7fdf280731c0 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c018680 con 0x7fdf280731c0 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf281a17c0 con 0x7fdf280731c0 2023-07-21T19:06:22.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.240+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf281a9020 con 0x7fdf280731c0 2023-07-21T19:06:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.242+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf28103090 con 0x7fdf280731c0 2023-07-21T19:06:22.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.243+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fdf1c00cd40 con 0x7fdf280731c0 2023-07-21T19:06:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.245+0000 7fdf0ffff700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 0x7fdf100a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.245+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdf1c014070 con 0x7fdf280731c0 2023-07-21T19:06:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.245+0000 7fdf26d9d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 0x7fdf100a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.246+0000 7fdf26d9d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 0x7fdf100a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf18006010 tx=0x7fdf180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:22.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.249+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdf1c051790 con 0x7fdf280731c0 2023-07-21T19:06:22.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 -- 172.21.15.5:0/1475907149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901070e0 msgr2=0x7f939010ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 --2- 172.21.15.5:0/1475907149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901070e0 0x7f939010ec60 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f938800b0d0 tx=0x7f938800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 -- 172.21.15.5:0/1475907149 shutdown_connections 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 --2- 172.21.15.5:0/1475907149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901070e0 0x7f939010ec60 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 --2- 172.21.15.5:0/1475907149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390102410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 --2- 172.21.15.5:0/1475907149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901029e0 0x7f93901069b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 -- 172.21.15.5:0/1475907149 >> 172.21.15.5:0/1475907149 conn(0x7f93900fd900 msgr2=0x7f93900ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 -- 172.21.15.5:0/1475907149 shutdown_connections 2023-07-21T19:06:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.289+0000 7f9395e11700 1 -- 172.21.15.5:0/1475907149 wait complete. 2023-07-21T19:06:22.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.290+0000 7f9395e11700 1 Processor -- start 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.290+0000 7f9395e11700 1 -- start start 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390197f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 0x7f9390198440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901070e0 0x7f939019c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019cec0 con 0x7f93901029e0 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019d030 con 0x7f9390102040 2023-07-21T19:06:22.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f9395e11700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939019d1f0 con 0x7f93901070e0 2023-07-21T19:06:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 0x7f9390198440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390197f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 0x7f9390198440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38640/0 (socket says 172.21.15.5:38640) 2023-07-21T19:06:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938effd700 1 -- 172.21.15.5:0/2730790474 learned_addr learned my addr 172.21.15.5:0/2730790474 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:22.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938ffff700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901070e0 0x7f939019c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938f7fe700 1 -- 172.21.15.5:0/2730790474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901070e0 msgr2=0x7f939019c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938f7fe700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901070e0 0x7f939019c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.291+0000 7f938f7fe700 1 -- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 msgr2=0x7f9390198440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f938f7fe700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 0x7f9390198440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f938f7fe700 1 -- 172.21.15.5:0/2730790474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9388009d00 con 0x7f9390102040 2023-07-21T19:06:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f938f7fe700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390197f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f938000e9f0 tx=0x7f938000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f938000c9c0 con 0x7f9390102040 2023-07-21T19:06:22.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9380010430 con 0x7f9390102040 2023-07-21T19:06:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.292+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939019d4d0 con 0x7f9390102040 2023-07-21T19:06:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.293+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9380018680 con 0x7f9390102040 2023-07-21T19:06:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.293+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93901a4d30 con 0x7f9390102040 2023-07-21T19:06:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.294+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f939004e210 con 0x7f9390102040 2023-07-21T19:06:22.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.296+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f938000cb20 con 0x7f9390102040 2023-07-21T19:06:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.298+0000 7f938cff9700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 0x7f93780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.298+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9380014070 con 0x7f9390102040 2023-07-21T19:06:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.298+0000 7f938effd700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 0x7f93780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.299+0000 7f938effd700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 0x7f93780a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9390199520 tx=0x7f9384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.302+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9380051790 con 0x7f9390102040 2023-07-21T19:06:22.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.503+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fdf2804e210 con 0x7fdf280731c0 2023-07-21T19:06:22.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.503+0000 7fdf0ffff700 1 -- 172.21.15.5:0/931719995 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fdf1c0962b0 con 0x7fdf280731c0 2023-07-21T19:06:22.514 INFO:teuthology.orchestra.run.smithi005.stdout:55834574924 2023-07-21T19:06:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 msgr2=0x7fdf100a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 0x7fdf100a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdf18006010 tx=0x7fdf180058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 msgr2=0x7fdf2819c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e9a0 tx=0x7fdf1c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 shutdown_connections 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdf100a1260 0x7fdf100a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf28110bb0 0x7fdf281a0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf280728b0 0x7fdf2819c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 --2- 172.21.15.5:0/931719995 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf280731c0 0x7fdf2819c730 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 >> 172.21.15.5:0/931719995 conn(0x7fdf280fb660 msgr2=0x7fdf280fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 shutdown_connections 2023-07-21T19:06:22.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.506+0000 7fdf2d11d700 1 -- 172.21.15.5:0/931719995 wait complete. 2023-07-21T19:06:22.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.540+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f93901a4ec0 con 0x7f9390102040 2023-07-21T19:06:22.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.541+0000 7f938cff9700 1 -- 172.21.15.5:0/2730790474 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9380096220 con 0x7f9390102040 2023-07-21T19:06:22.552 INFO:teuthology.orchestra.run.smithi005.stdout:197568495664 2023-07-21T19:06:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 msgr2=0x7f93780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 0x7f93780a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f9390199520 tx=0x7f9384009260 comp rx=0 tx=0).stop 2023-07-21T19:06:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 msgr2=0x7f9390197f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390197f00 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f938000e9f0 tx=0x7f938000ed00 comp rx=0 tx=0).stop 2023-07-21T19:06:22.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 shutdown_connections 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93780a11e0 0x7f93780a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93901070e0 0x7f939019c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9390102040 0x7f9390197f00 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 --2- 172.21.15.5:0/2730790474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f93901029e0 0x7f9390198440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.543+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 >> 172.21.15.5:0/2730790474 conn(0x7f93900fd900 msgr2=0x7f93900fe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.544+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 shutdown_connections 2023-07-21T19:06:22.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:22.544+0000 7f9395e11700 1 -- 172.21.15.5:0/2730790474 wait complete. 2023-07-21T19:06:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:22 smithi005 ceph-mon[126460]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 130 B/s, 0 objects/s recovering 2023-07-21T19:06:22.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:22 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3927370359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T19:06:22.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:22 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2188133092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T19:06:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:22 smithi156 ceph-mon[128316]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 130 B/s, 0 objects/s recovering 2023-07-21T19:06:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:22 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3927370359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T19:06:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:22 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2188133092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T19:06:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:22 smithi101 ceph-mon[128136]: pgmap v324: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 255 B/s wr, 6 op/s; 130 B/s, 0 objects/s recovering 2023-07-21T19:06:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:22 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3927370359' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-07-21T19:06:22.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:22 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2188133092' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-07-21T19:06:22.928 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757302 for osd.5 2023-07-21T19:06:22.929 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:23.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 -- 172.21.15.5:0/3711577898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c00ffe00 msgr2=0x7fb0c01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/3711577898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c00ffe00 0x7fb0c01001d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb0b00099c0 tx=0x7fb0b0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 -- 172.21.15.5:0/3711577898 shutdown_connections 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/3711577898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c0104ea0 0x7fb0c010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/3711577898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c00ffe00 0x7fb0c01001d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/3711577898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c01007a0 0x7fb0c0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 -- 172.21.15.5:0/3711577898 >> 172.21.15.5:0/3711577898 conn(0x7fb0c00fb660 msgr2=0x7fb0c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.154+0000 7fb0c62d8700 1 -- 172.21.15.5:0/3711577898 shutdown_connections 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.155+0000 7fb0c62d8700 1 -- 172.21.15.5:0/3711577898 wait complete. 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.155+0000 7fb0c62d8700 1 Processor -- start 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 -- start start 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c00ffe00 0x7fb0c0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c01007a0 0x7fb0c0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c01a3f80 con 0x7fb0c0104ea0 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c01a40f0 con 0x7fb0c01007a0 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c62d8700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c01a42b0 con 0x7fb0c00ffe00 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c4875700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c4875700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38650/0 (socket says 172.21.15.5:38650) 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0c4875700 1 -- 172.21.15.5:0/1641362879 learned_addr learned my addr 172.21.15.5:0/1641362879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.156+0000 7fb0bffff700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c00ffe00 0x7fb0c0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0bf7fe700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c01007a0 0x7fb0c0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 -- 172.21.15.5:0/1641362879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c00ffe00 msgr2=0x7fb0c0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c00ffe00 0x7fb0c0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 -- 172.21.15.5:0/1641362879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c01007a0 msgr2=0x7fb0c0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c01007a0 0x7fb0c0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 -- 172.21.15.5:0/1641362879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00095f0 con 0x7fb0c0104ea0 2023-07-21T19:06:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c4875700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800ef30 tx=0x7fb0b800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b8018070 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0b800f040 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c01a4530 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800cbc0 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.157+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c01a49e0 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.159+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c004e210 con 0x7fb0c0104ea0 2023-07-21T19:06:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.160+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fb0b8007500 con 0x7fb0c0104ea0 2023-07-21T19:06:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.162+0000 7fb0bd7fa700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 0x7fb0a80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.162+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb0b80cc550 con 0x7fb0c0104ea0 2023-07-21T19:06:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.162+0000 7fb0bffff700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 0x7fb0a80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.162+0000 7fb0bffff700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 0x7fb0a80a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b580 tx=0x7fb0b0005c00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:23.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.166+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb0b8095b80 con 0x7fb0c0104ea0 2023-07-21T19:06:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.387+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fb0c01a4c00 con 0x7fb0c0104ea0 2023-07-21T19:06:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.387+0000 7fb0bd7fa700 1 -- 172.21.15.5:0/1641362879 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb0b80991a0 con 0x7fb0c0104ea0 2023-07-21T19:06:23.398 INFO:teuthology.orchestra.run.smithi005.stdout:253403070502 2023-07-21T19:06:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.389+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 msgr2=0x7fb0a80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.389+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 0x7fb0a80a3690 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fb0b000b580 tx=0x7fb0b0005c00 comp rx=0 tx=0).stop 2023-07-21T19:06:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.389+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 msgr2=0x7fb0c00732d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.389+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb0b800ef30 tx=0x7fb0b800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 shutdown_connections 2023-07-21T19:06:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb0a80a11e0 0x7fb0a80a3690 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0c0104ea0 0x7fb0c00732d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0c00ffe00 0x7fb0c0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 --2- 172.21.15.5:0/1641362879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0c01007a0 0x7fb0c0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 >> 172.21.15.5:0/1641362879 conn(0x7fb0c00fb660 msgr2=0x7fb0c0103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 shutdown_connections 2023-07-21T19:06:23.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:23.390+0000 7fb0c62d8700 1 -- 172.21.15.5:0/1641362879 wait complete. 2023-07-21T19:06:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:23 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/931719995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T19:06:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:23 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2730790474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T19:06:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:23 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/931719995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T19:06:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:23 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2730790474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T19:06:23.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:23 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/931719995' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-07-21T19:06:23.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:23 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2730790474' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-07-21T19:06:24.531 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411395 got 77309411398 for osd.2 2023-07-21T19:06:24.532 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:24.532 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574924 for osd.1 2023-07-21T19:06:24.532 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:24 smithi005 ceph-mon[126460]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 objects/s recovering 2023-07-21T19:06:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:24 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1641362879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T19:06:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:24 smithi156 ceph-mon[128316]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 objects/s recovering 2023-07-21T19:06:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:24 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1641362879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T19:06:24.823 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070500 got 253403070502 for osd.8 2023-07-21T19:06:24.823 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/25260310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 msgr2=0x7fe0c0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/25260310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c0104770 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe0b00099c0 tx=0x7fe0b0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/25260310 shutdown_connections 2023-07-21T19:06:24.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/25260310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/25260310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 0x7fe0c01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/25260310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c0104770 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/25260310 >> 172.21.15.5:0/25260310 conn(0x7fe0c00fb660 msgr2=0x7fe0c00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/25260310 shutdown_connections 2023-07-21T19:06:24.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.816+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/25260310 wait complete. 2023-07-21T19:06:24.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.817+0000 7fe0c7c6d700 1 Processor -- start 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.817+0000 7fe0c7c6d700 1 -- start start 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 0x7fe0c0197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c019c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c019ce40 con 0x7fe0c01007a0 2023-07-21T19:06:24.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c019cfb0 con 0x7fe0c0104ea0 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c7c6d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0c019d170 con 0x7fe0c00ffe00 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c5208700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c5208700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38670/0 (socket says 172.21.15.5:38670) 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c5208700 1 -- 172.21.15.5:0/258220307 learned_addr learned my addr 172.21.15.5:0/258220307 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c5a09700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 0x7fe0c0197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.818+0000 7fe0c620a700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c019c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 -- 172.21.15.5:0/258220307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 msgr2=0x7fe0c0197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 0x7fe0c0197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 -- 172.21.15.5:0/258220307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 msgr2=0x7fe0c019c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c019c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 -- 172.21.15.5:0/258220307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b00095f0 con 0x7fe0c01007a0 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c620a700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c5208700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009990 tx=0x7fe0b0009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b001d070 con 0x7fe0c01007a0 2023-07-21T19:06:24.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0c019d3f0 con 0x7fe0c01007a0 2023-07-21T19:06:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe0b0003fc0 con 0x7fe0c01007a0 2023-07-21T19:06:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0b000fd50 con 0x7fe0c01007a0 2023-07-21T19:06:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.819+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0c01a4cb0 con 0x7fe0c01007a0 2023-07-21T19:06:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.820+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0c004e210 con 0x7fe0c01007a0 2023-07-21T19:06:24.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.822+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe0b000f4f0 con 0x7fe0c01007a0 2023-07-21T19:06:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.824+0000 7fe0b6ffd700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 0x7fe0ac0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:24.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.824+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe0b00cdda0 con 0x7fe0c01007a0 2023-07-21T19:06:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.824+0000 7fe0c5a09700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 0x7fe0ac0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:24.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.825+0000 7fe0c5a09700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 0x7fe0ac0a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005950 tx=0x7fe0bc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:24.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:24.826+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe0b00973d0 con 0x7fe0c01007a0 2023-07-21T19:06:24.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:24 smithi101 ceph-mon[128136]: pgmap v325: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s; 50 B/s, 0 objects/s recovering 2023-07-21T19:06:24.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:24 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1641362879' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-07-21T19:06:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.049+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fe0c01a4e40 con 0x7fe0c01007a0 2023-07-21T19:06:25.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.049+0000 7fe0b6ffd700 1 -- 172.21.15.5:0/258220307 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fe0b009a9f0 con 0x7fe0c01007a0 2023-07-21T19:06:25.059 INFO:teuthology.orchestra.run.smithi005.stdout:219043332139 2023-07-21T19:06:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 msgr2=0x7fe0ac0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 0x7fe0ac0a3690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe0bc005950 tx=0x7fe0bc0058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 msgr2=0x7fe0c01983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fe0b0009990 tx=0x7fe0b0009580 comp rx=0 tx=0).stop 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 shutdown_connections 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe0ac0a11e0 0x7fe0ac0a3690 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe0c0104ea0 0x7fe0c019c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe0c00ffe00 0x7fe0c0197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 --2- 172.21.15.5:0/258220307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe0c01007a0 0x7fe0c01983c0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 >> 172.21.15.5:0/258220307 conn(0x7fe0c00fb660 msgr2=0x7fe0c00fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 shutdown_connections 2023-07-21T19:06:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.052+0000 7fe0c7c6d700 1 -- 172.21.15.5:0/258220307 wait complete. 2023-07-21T19:06:25.179 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495662 got 197568495664 for osd.6 2023-07-21T19:06:25.180 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 -- 172.21.15.5:0/3175259599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c1007a0 msgr2=0x7f930c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 --2- 172.21.15.5:0/3175259599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c1007a0 0x7f930c104770 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f92f80099c0 tx=0x7f92f8009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 -- 172.21.15.5:0/3175259599 shutdown_connections 2023-07-21T19:06:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 --2- 172.21.15.5:0/3175259599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c104ea0 0x7f930c10ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 --2- 172.21.15.5:0/3175259599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c0ffe00 0x7f930c1001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 --2- 172.21.15.5:0/3175259599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c1007a0 0x7f930c104770 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 -- 172.21.15.5:0/3175259599 >> 172.21.15.5:0/3175259599 conn(0x7f930c0fb660 msgr2=0x7f930c0fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 -- 172.21.15.5:0/3175259599 shutdown_connections 2023-07-21T19:06:25.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.359+0000 7f93131bd700 1 -- 172.21.15.5:0/3175259599 wait complete. 2023-07-21T19:06:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.360+0000 7f93131bd700 1 Processor -- start 2023-07-21T19:06:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 -- start start 2023-07-21T19:06:25.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c0ffe00 0x7f930c197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 0x7f930c198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19cd90 con 0x7f930c104ea0 2023-07-21T19:06:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19cf00 con 0x7f930c1007a0 2023-07-21T19:06:25.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f93131bd700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930c19d0c0 con 0x7f930c0ffe00 2023-07-21T19:06:25.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.361+0000 7f931175a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:25.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f931175a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38674/0 (socket says 172.21.15.5:38674) 2023-07-21T19:06:25.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f931175a700 1 -- 172.21.15.5:0/2192982767 learned_addr learned my addr 172.21.15.5:0/2192982767 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:25.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 0x7f930c198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:25.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 -- 172.21.15.5:0/2192982767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c0ffe00 msgr2=0x7f930c197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c0ffe00 0x7f930c197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 -- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 msgr2=0x7f930c19c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 -- 172.21.15.5:0/2192982767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f80095f0 con 0x7f930c1007a0 2023-07-21T19:06:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f931175a700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:25.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.362+0000 7f930bfff700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 0x7f930c198310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009fd0 tx=0x7f92f800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.363+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f801d070 con 0x7f930c1007a0 2023-07-21T19:06:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.363+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f930c19d340 con 0x7f930c1007a0 2023-07-21T19:06:25.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.363+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92f8005050 con 0x7f930c1007a0 2023-07-21T19:06:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.363+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f8004a90 con 0x7f930c1007a0 2023-07-21T19:06:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.363+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f930c19d5e0 con 0x7f930c1007a0 2023-07-21T19:06:25.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.364+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f930c04e210 con 0x7f930c1007a0 2023-07-21T19:06:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.367+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f92f80053a0 con 0x7f930c1007a0 2023-07-21T19:06:25.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.368+0000 7f9309ffb700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 0x7f92fc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.368+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f92f80cd770 con 0x7f930c1007a0 2023-07-21T19:06:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.369+0000 7f9310f59700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 0x7f92fc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:25.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.369+0000 7f9310f59700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 0x7f92fc0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9300005950 tx=0x7f93000058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:25.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.373+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f92f805b790 con 0x7f930c1007a0 2023-07-21T19:06:25.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.589+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f930c1a4d00 con 0x7f930c1007a0 2023-07-21T19:06:25.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.590+0000 7f9309ffb700 1 -- 172.21.15.5:0/2192982767 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f92f8096da0 con 0x7f930c1007a0 2023-07-21T19:06:25.600 INFO:teuthology.orchestra.run.smithi005.stdout:141733920828 2023-07-21T19:06:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 msgr2=0x7f92fc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 0x7f92fc0a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9300005950 tx=0x7f93000058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 msgr2=0x7f930c198310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 0x7f930c198310 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009fd0 tx=0x7f92f800f690 comp rx=0 tx=0).stop 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 shutdown_connections 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f92fc0a11e0 0x7f92fc0a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f930c104ea0 0x7f930c19c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f930c0ffe00 0x7f930c197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 --2- 172.21.15.5:0/2192982767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f930c1007a0 0x7f930c198310 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 >> 172.21.15.5:0/2192982767 conn(0x7f930c0fb660 msgr2=0x7f930c0fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 shutdown_connections 2023-07-21T19:06:25.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:25.593+0000 7f93131bd700 1 -- 172.21.15.5:0/2192982767 wait complete. 2023-07-21T19:06:25.625 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:25 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/258220307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T19:06:25.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:25 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/258220307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T19:06:25.817 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332137 got 219043332139 for osd.7 2023-07-21T19:06:25.817 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:25.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:25 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/258220307' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-07-21T19:06:26.394 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920825 got 141733920828 for osd.4 2023-07-21T19:06:26.394 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:26.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 -- 172.21.15.5:0/1567922436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9830110bb0 msgr2=0x7f9830112f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 --2- 172.21.15.5:0/1567922436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9830110bb0 0x7f9830112f90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f982800b0a0 tx=0x7f982800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 -- 172.21.15.5:0/1567922436 shutdown_connections 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 --2- 172.21.15.5:0/1567922436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9830110bb0 0x7f9830112f90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 --2- 172.21.15.5:0/1567922436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 0x7f9830072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 --2- 172.21.15.5:0/1567922436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98300731c0 0x7f9830110670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 -- 172.21.15.5:0/1567922436 >> 172.21.15.5:0/1567922436 conn(0x7f98300fb660 msgr2=0x7f98300fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 -- 172.21.15.5:0/1567922436 shutdown_connections 2023-07-21T19:06:26.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.576+0000 7f983705e700 1 -- 172.21.15.5:0/1567922436 wait complete. 2023-07-21T19:06:26.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 Processor -- start 2023-07-21T19:06:26.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 -- start start 2023-07-21T19:06:26.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 0x7f983019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9830110bb0 0x7f98301a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.578+0000 7f983705e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301a1260 con 0x7f98300731c0 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f983705e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301a13d0 con 0x7f9830110bb0 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f983705e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98301a1590 con 0x7f98300728b0 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42904/0 (socket says 172.21.15.5:42904) 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 -- 172.21.15.5:0/4073710841 learned_addr learned my addr 172.21.15.5:0/4073710841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f9834dfa700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 0x7f983019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 -- 172.21.15.5:0/4073710841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 msgr2=0x7f983019c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 0x7f983019c2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 -- 172.21.15.5:0/4073710841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9830110bb0 msgr2=0x7f98301a0c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9830110bb0 0x7f98301a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 -- 172.21.15.5:0/4073710841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9828009d00 con 0x7f98300731c0 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.579+0000 7f982ffff700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e9a0 tx=0x7f981c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.580+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c009790 con 0x7f98300731c0 2023-07-21T19:06:26.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.580+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98301a1870 con 0x7f98300731c0 2023-07-21T19:06:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.580+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f981c00cbd0 con 0x7f98300731c0 2023-07-21T19:06:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.580+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981c018680 con 0x7f98300731c0 2023-07-21T19:06:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.580+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98301a90d0 con 0x7f98300731c0 2023-07-21T19:06:26.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.581+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9830103090 con 0x7f98300731c0 2023-07-21T19:06:26.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:26 smithi005 ceph-mon[126460]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:26.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:26 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2192982767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T19:06:26.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.583+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f981c00cd40 con 0x7f98300731c0 2023-07-21T19:06:26.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.584+0000 7f982dffb700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 0x7f98200a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.585+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f981c014070 con 0x7f98300731c0 2023-07-21T19:06:26.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.585+0000 7f9834dfa700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 0x7f98200a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.586+0000 7f9834dfa700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 0x7f98200a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9824007be0 tx=0x7f98240073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:26.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.588+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f981c051790 con 0x7f98300731c0 2023-07-21T19:06:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:26 smithi156 ceph-mon[128316]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:26 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2192982767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T19:06:26.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.817+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f983004e210 con 0x7f98300731c0 2023-07-21T19:06:26.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.817+0000 7f982dffb700 1 -- 172.21.15.5:0/4073710841 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f981c096210 con 0x7f98300731c0 2023-07-21T19:06:26.827 INFO:teuthology.orchestra.run.smithi005.stdout:287762808866 2023-07-21T19:06:26.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 msgr2=0x7f98200a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 0x7f98200a3710 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f9824007be0 tx=0x7f98240073d0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 msgr2=0x7f983019c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f981c00e9a0 tx=0x7f981c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 shutdown_connections 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f98200a1260 0x7f98200a3710 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9830110bb0 0x7f98301a0c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98300728b0 0x7f983019c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 --2- 172.21.15.5:0/4073710841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98300731c0 0x7f983019c7e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 >> 172.21.15.5:0/4073710841 conn(0x7f98300fb660 msgr2=0x7f98300fc2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 shutdown_connections 2023-07-21T19:06:26.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.821+0000 7f983705e700 1 -- 172.21.15.5:0/4073710841 wait complete. 2023-07-21T19:06:26.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:26 smithi101 ceph-mon[128136]: pgmap v326: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:26.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:26 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2192982767' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-07-21T19:06:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 -- 172.21.15.5:0/2171756808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec41007a0 msgr2=0x7fcec4104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 --2- 172.21.15.5:0/2171756808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec41007a0 0x7fcec4104770 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fceac0099c0 tx=0x7fceac009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 -- 172.21.15.5:0/2171756808 shutdown_connections 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 --2- 172.21.15.5:0/2171756808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec4104ea0 0x7fcec410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 --2- 172.21.15.5:0/2171756808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec40ffe00 0x7fcec41001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 --2- 172.21.15.5:0/2171756808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec41007a0 0x7fcec4104770 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 -- 172.21.15.5:0/2171756808 >> 172.21.15.5:0/2171756808 conn(0x7fcec40fb660 msgr2=0x7fcec40fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 -- 172.21.15.5:0/2171756808 shutdown_connections 2023-07-21T19:06:26.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.840+0000 7fcec9b41700 1 -- 172.21.15.5:0/2171756808 wait complete. 2023-07-21T19:06:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 Processor -- start 2023-07-21T19:06:26.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 -- start start 2023-07-21T19:06:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec40ffe00 0x7fcec4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 0x7fcec4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419cde0 con 0x7fcec4104ea0 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419cf50 con 0x7fcec40ffe00 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.842+0000 7fcec9b41700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcec419d110 con 0x7fcec41007a0 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcec3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 0x7fcec4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcec3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42936/0 (socket says 172.21.15.5:42936) 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcec3fff700 1 -- 172.21.15.5:0/749170415 learned_addr learned my addr 172.21.15.5:0/749170415 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:26.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcec37fe700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec40ffe00 0x7fcec4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 -- 172.21.15.5:0/749170415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec40ffe00 msgr2=0x7fcec4197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec40ffe00 0x7fcec4197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 -- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 msgr2=0x7fcec419c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcebbfff700 1 -- 172.21.15.5:0/749170415 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceac0095f0 con 0x7fcec41007a0 2023-07-21T19:06:26.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.843+0000 7fcec3fff700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.844+0000 7fcebbfff700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 0x7fcec4198360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fceac0098e0 tx=0x7fceac005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.844+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac01d070 con 0x7fcec41007a0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.844+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fceac00fe70 con 0x7fcec41007a0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.844+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcec419d390 con 0x7fcec41007a0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.844+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceac022df0 con 0x7fcec41007a0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.845+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcec41a4c50 con 0x7fcec41007a0 2023-07-21T19:06:26.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.845+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcec404e210 con 0x7fcec41007a0 2023-07-21T19:06:26.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.848+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fceac00f9e0 con 0x7fcec41007a0 2023-07-21T19:06:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.849+0000 7fcec17fa700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 0x7fceb00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.849+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fceac0cdad0 con 0x7fcec41007a0 2023-07-21T19:06:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.850+0000 7fcec37fe700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 0x7fceb00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:26.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.850+0000 7fcec37fe700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 0x7fceb00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009c60 tx=0x7fceb4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:26.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:26.851+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fceac097100 con 0x7fcec41007a0 2023-07-21T19:06:27.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.063+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7fcec41a4de0 con 0x7fcec41007a0 2023-07-21T19:06:27.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.063+0000 7fcec17fa700 1 -- 172.21.15.5:0/749170415 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fceac09a720 con 0x7fcec41007a0 2023-07-21T19:06:27.073 INFO:teuthology.orchestra.run.smithi005.stdout:309237645341 2023-07-21T19:06:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 msgr2=0x7fceb00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 0x7fceb00a3710 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fceb4009c60 tx=0x7fceb4009260 comp rx=0 tx=0).stop 2023-07-21T19:06:27.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 msgr2=0x7fcec4198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 0x7fcec4198360 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fceac0098e0 tx=0x7fceac005f10 comp rx=0 tx=0).stop 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 shutdown_connections 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fceb00a1260 0x7fceb00a3710 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcec4104ea0 0x7fcec419c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcec40ffe00 0x7fcec4197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 --2- 172.21.15.5:0/749170415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcec41007a0 0x7fcec4198360 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:27.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 >> 172.21.15.5:0/749170415 conn(0x7fcec40fb660 msgr2=0x7fcec40fc370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 shutdown_connections 2023-07-21T19:06:27.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:27.066+0000 7fcec9b41700 1 -- 172.21.15.5:0/749170415 wait complete. 2023-07-21T19:06:27.434 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808864 got 287762808866 for osd.9 2023-07-21T19:06:27.435 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:27.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4073710841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T19:06:27.611 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/749170415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T19:06:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4073710841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T19:06:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/749170415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T19:06:27.681 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645338 got 309237645341 for osd.10 2023-07-21T19:06:27.681 DEBUG:teuthology.parallel:result is None 2023-07-21T19:06:27.682 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-07-21T19:06:27.682 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T19:06:27.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4073710841' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-07-21T19:06:27.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/749170415' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-07-21T19:06:28.416 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:28 smithi005 ceph-mon[126460]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:28 smithi156 ceph-mon[128316]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:28.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:28 smithi101 ceph-mon[128136]: pgmap v327: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 -- 172.21.15.5:0/2376557006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 msgr2=0x7f0aa010ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 --2- 172.21.15.5:0/2376557006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa010ca20 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b0d0 tx=0x7f0a9800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 -- 172.21.15.5:0/2376557006 shutdown_connections 2023-07-21T19:06:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 --2- 172.21.15.5:0/2376557006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa010ca20 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 --2- 172.21.15.5:0/2376557006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa01001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 --2- 172.21.15.5:0/2376557006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 0x7f0aa0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 -- 172.21.15.5:0/2376557006 >> 172.21.15.5:0/2376557006 conn(0x7f0aa00fb660 msgr2=0x7f0aa00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 -- 172.21.15.5:0/2376557006 shutdown_connections 2023-07-21T19:06:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.887+0000 7f0aa7670700 1 -- 172.21.15.5:0/2376557006 wait complete. 2023-07-21T19:06:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.888+0000 7f0aa7670700 1 Processor -- start 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.888+0000 7f0aa7670700 1 -- start start 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa0197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 0x7f0aa0198420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019cea0 con 0x7f0aa0104ea0 2023-07-21T19:06:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019d010 con 0x7f0aa00ffe00 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa7670700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0aa019d1d0 con 0x7f0aa01007a0 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa5c0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa5c0d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42952/0 (socket says 172.21.15.5:42952) 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa5c0d700 1 -- 172.21.15.5:0/4175258530 learned_addr learned my addr 172.21.15.5:0/4175258530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa540c700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa0197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.889+0000 7f0aa4c0b700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 0x7f0aa0198420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 -- 172.21.15.5:0/4175258530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 msgr2=0x7f0aa0198420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 0x7f0aa0198420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 -- 172.21.15.5:0/4175258530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 msgr2=0x7f0aa0197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa0197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 -- 172.21.15.5:0/4175258530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a98009d00 con 0x7f0aa0104ea0 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa540c700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa5c0d700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009300 tx=0x7f0a980093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9800e040 con 0x7f0aa0104ea0 2023-07-21T19:06:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a98003e10 con 0x7f0aa0104ea0 2023-07-21T19:06:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a9801b8e0 con 0x7f0aa0104ea0 2023-07-21T19:06:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0aa019d450 con 0x7f0aa0104ea0 2023-07-21T19:06:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.890+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0aa01a4b40 con 0x7f0aa0104ea0 2023-07-21T19:06:29.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.891+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0aa004e210 con 0x7f0aa0104ea0 2023-07-21T19:06:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.893+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f0a98012470 con 0x7f0aa0104ea0 2023-07-21T19:06:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.895+0000 7f0a967fc700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 0x7f0a8c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.895+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f0a980ce220 con 0x7f0aa0104ea0 2023-07-21T19:06:29.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.895+0000 7f0aa540c700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 0x7f0a8c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:29.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.896+0000 7f0aa540c700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 0x7f0a8c0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005950 tx=0x7f0a9c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:29.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:29.899+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0a98097850 con 0x7f0aa0104ea0 2023-07-21T19:06:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.100+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0aa00fc810 con 0x7f0a8c0a1260 2023-07-21T19:06:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.111+0000 7f0a967fc700 1 -- 172.21.15.5:0/4175258530 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339486 (secure 0 0 0) 0x7f0aa00fc810 con 0x7f0a8c0a1260 2023-07-21T19:06:30.124 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 msgr2=0x7f0a8c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 0x7f0a8c0a3710 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c005950 tx=0x7f0a9c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 msgr2=0x7f0aa019c890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0a98009300 tx=0x7f0a980093e0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 shutdown_connections 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f0a8c0a1260 0x7f0a8c0a3710 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0aa0104ea0 0x7f0aa019c890 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0aa00ffe00 0x7f0aa0197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 --2- 172.21.15.5:0/4175258530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0aa01007a0 0x7f0aa0198420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 >> 172.21.15.5:0/4175258530 conn(0x7f0aa00fb660 msgr2=0x7f0aa00fc100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 shutdown_connections 2023-07-21T19:06:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:30.116+0000 7f0aa7670700 1 -- 172.21.15.5:0/4175258530 wait complete. 2023-07-21T19:06:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:06:30.595 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":328,"stamp":"2023-07-21T19:06:29.325124+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652601,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":418,"num_read_kb":330,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182304,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334992,"kb_used_data":11568,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606832,"statfs":{"total":1151940427776,"available":1151597395968,"internally_reserved":0,"allocated":11845632,"data_stored":7425519,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":65,"num_read_kb":35,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002958"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400686+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:06:08.400686+0000","last_peered":"2023-07-21T19:06:08.400686+0000","last_clean":"2023-07-21T19:06:08.400686+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:06:08.400686+0000","last_undegraded":"2023-07-21T19:06:08.400686+0000","last_fullsized":"2023-07-21T19:06:08.400686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:46.910120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397988+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:06:08.397988+0000","last_peered":"2023-07-21T19:06:08.397988+0000","last_clean":"2023-07-21T19:06:08.397988+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:06:08.397988+0000","last_undegraded":"2023-07-21T19:06:08.397988+0000","last_fullsized":"2023-07-21T19:06:08.397988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:32.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397678+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:06:08.397678+0000","last_peered":"2023-07-21T19:06:08.397678+0000","last_clean":"2023-07-21T19:06:08.397678+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:06:08.397678+0000","last_undegraded":"2023-07-21T19:06:08.397678+0000","last_fullsized":"2023-07-21T19:06:08.397678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:43.019102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399400+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:06:08.399400+0000","last_peered":"2023-07-21T19:06:08.399400+0000","last_clean":"2023-07-21T19:06:08.399400+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:06:08.399400+0000","last_undegraded":"2023-07-21T19:06:08.399400+0000","last_fullsized":"2023-07-21T19:06:08.399400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:20.204377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398192+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:06:08.398192+0000","last_peered":"2023-07-21T19:06:08.398192+0000","last_clean":"2023-07-21T19:06:08.398192+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:06:08.398192+0000","last_undegraded":"2023-07-21T19:06:08.398192+0000","last_fullsized":"2023-07-21T19:06:08.398192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:04.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400393+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:06:08.400393+0000","last_peered":"2023-07-21T19:06:08.400393+0000","last_clean":"2023-07-21T19:06:08.400393+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:06:08.400393+0000","last_undegraded":"2023-07-21T19:06:08.400393+0000","last_fullsized":"2023-07-21T19:06:08.400393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:17.572777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404051+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:06:08.404051+0000","last_peered":"2023-07-21T19:06:08.404051+0000","last_clean":"2023-07-21T19:06:08.404051+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:06:08.404051+0000","last_undegraded":"2023-07-21T19:06:08.404051+0000","last_fullsized":"2023-07-21T19:06:08.404051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:50:34.608525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841189+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:06:08.841189+0000","last_peered":"2023-07-21T19:06:08.841189+0000","last_clean":"2023-07-21T19:06:08.841189+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:06:08.841189+0000","last_undegraded":"2023-07-21T19:06:08.841189+0000","last_fullsized":"2023-07-21T19:06:08.841189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:29.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.843235+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:06:08.843235+0000","last_peered":"2023-07-21T19:06:08.843235+0000","last_clean":"2023-07-21T19:06:08.843235+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:06:08.843235+0000","last_undegraded":"2023-07-21T19:06:08.843235+0000","last_fullsized":"2023-07-21T19:06:08.843235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:08:37.146951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400414+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:06:08.400414+0000","last_peered":"2023-07-21T19:06:08.400414+0000","last_clean":"2023-07-21T19:06:08.400414+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:06:08.400414+0000","last_undegraded":"2023-07-21T19:06:08.400414+0000","last_fullsized":"2023-07-21T19:06:08.400414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:32.152857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404505+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:06:08.404505+0000","last_peered":"2023-07-21T19:06:08.404505+0000","last_clean":"2023-07-21T19:06:08.404505+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:06:08.404505+0000","last_undegraded":"2023-07-21T19:06:08.404505+0000","last_fullsized":"2023-07-21T19:06:08.404505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:23.961311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399377+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:06:08.399377+0000","last_peered":"2023-07-21T19:06:08.399377+0000","last_clean":"2023-07-21T19:06:08.399377+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:06:08.399377+0000","last_undegraded":"2023-07-21T19:06:08.399377+0000","last_fullsized":"2023-07-21T19:06:08.399377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:15.583393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402871+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:06:08.402871+0000","last_peered":"2023-07-21T19:06:08.402871+0000","last_clean":"2023-07-21T19:06:08.402871+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:06:08.402871+0000","last_undegraded":"2023-07-21T19:06:08.402871+0000","last_fullsized":"2023-07-21T19:06:08.402871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.309276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397190+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:06:08.397190+0000","last_peered":"2023-07-21T19:06:08.397190+0000","last_clean":"2023-07-21T19:06:08.397190+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:06:08.397190+0000","last_undegraded":"2023-07-21T19:06:08.397190+0000","last_fullsized":"2023-07-21T19:06:08.397190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:09:18.215822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397323+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:06:08.397323+0000","last_peered":"2023-07-21T19:06:08.397323+0000","last_clean":"2023-07-21T19:06:08.397323+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:06:08.397323+0000","last_undegraded":"2023-07-21T19:06:08.397323+0000","last_fullsized":"2023-07-21T19:06:08.397323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:59.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842577+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:06:08.842577+0000","last_peered":"2023-07-21T19:06:08.842577+0000","last_clean":"2023-07-21T19:06:08.842577+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:06:08.842577+0000","last_undegraded":"2023-07-21T19:06:08.842577+0000","last_fullsized":"2023-07-21T19:06:08.842577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:13.624531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400685+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:06:08.400685+0000","last_peered":"2023-07-21T19:06:08.400685+0000","last_clean":"2023-07-21T19:06:08.400685+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:06:08.400685+0000","last_undegraded":"2023-07-21T19:06:08.400685+0000","last_fullsized":"2023-07-21T19:06:08.400685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:17.986171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401123+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:06:08.401123+0000","last_peered":"2023-07-21T19:06:08.401123+0000","last_clean":"2023-07-21T19:06:08.401123+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:06:08.401123+0000","last_undegraded":"2023-07-21T19:06:08.401123+0000","last_fullsized":"2023-07-21T19:06:08.401123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:48.916207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405245+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:06:08.405245+0000","last_peered":"2023-07-21T19:06:08.405245+0000","last_clean":"2023-07-21T19:06:08.405245+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:06:08.405245+0000","last_undegraded":"2023-07-21T19:06:08.405245+0000","last_fullsized":"2023-07-21T19:06:08.405245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:08.380972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.851006+0000","last_change":"2023-07-21T19:06:08.851006+0000","last_active":"2023-07-21T19:06:08.851006+0000","last_peered":"2023-07-21T19:06:08.851006+0000","last_clean":"2023-07-21T19:06:08.851006+0000","last_became_active":"2023-07-21T19:06:08.843812+0000","last_became_peered":"2023-07-21T19:06:08.843812+0000","last_unstale":"2023-07-21T19:06:08.851006+0000","last_undegraded":"2023-07-21T19:06:08.851006+0000","last_fullsized":"2023-07-21T19:06:08.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:03.813996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841892+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:06:08.841892+0000","last_peered":"2023-07-21T19:06:08.841892+0000","last_clean":"2023-07-21T19:06:08.841892+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:06:08.841892+0000","last_undegraded":"2023-07-21T19:06:08.841892+0000","last_fullsized":"2023-07-21T19:06:08.841892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:59.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840757+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:06:08.840757+0000","last_peered":"2023-07-21T19:06:08.840757+0000","last_clean":"2023-07-21T19:06:08.840757+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:06:08.840757+0000","last_undegraded":"2023-07-21T19:06:08.840757+0000","last_fullsized":"2023-07-21T19:06:08.840757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:19.820365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397764+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:06:08.397764+0000","last_peered":"2023-07-21T19:06:08.397764+0000","last_clean":"2023-07-21T19:06:08.397764+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:06:08.397764+0000","last_undegraded":"2023-07-21T19:06:08.397764+0000","last_fullsized":"2023-07-21T19:06:08.397764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:48.371444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398380+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:06:08.398380+0000","last_peered":"2023-07-21T19:06:08.398380+0000","last_clean":"2023-07-21T19:06:08.398380+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:06:08.398380+0000","last_undegraded":"2023-07-21T19:06:08.398380+0000","last_fullsized":"2023-07-21T19:06:08.398380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:48.967070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842807+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:06:08.842807+0000","last_peered":"2023-07-21T19:06:08.842807+0000","last_clean":"2023-07-21T19:06:08.842807+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:06:08.842807+0000","last_undegraded":"2023-07-21T19:06:08.842807+0000","last_fullsized":"2023-07-21T19:06:08.842807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:03:26.702743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398317+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:06:08.398317+0000","last_peered":"2023-07-21T19:06:08.398317+0000","last_clean":"2023-07-21T19:06:08.398317+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:06:08.398317+0000","last_undegraded":"2023-07-21T19:06:08.398317+0000","last_fullsized":"2023-07-21T19:06:08.398317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:56.271039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402196+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:06:08.402196+0000","last_peered":"2023-07-21T19:06:08.402196+0000","last_clean":"2023-07-21T19:06:08.402196+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:06:08.402196+0000","last_undegraded":"2023-07-21T19:06:08.402196+0000","last_fullsized":"2023-07-21T19:06:08.402196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:12.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":83,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:24.431383+0000","last_change":"2023-07-21T19:06:08.410381+0000","last_active":"2023-07-21T19:06:24.431383+0000","last_peered":"2023-07-21T19:06:24.431383+0000","last_clean":"2023-07-21T19:06:24.431383+0000","last_became_active":"2023-07-21T19:06:08.405822+0000","last_became_peered":"2023-07-21T19:06:08.405822+0000","last_unstale":"2023-07-21T19:06:24.431383+0000","last_undegraded":"2023-07-21T19:06:24.431383+0000","last_fullsized":"2023-07-21T19:06:24.431383+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:14:23.390484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398080+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:06:08.398080+0000","last_peered":"2023-07-21T19:06:08.398080+0000","last_clean":"2023-07-21T19:06:08.398080+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:06:08.398080+0000","last_undegraded":"2023-07-21T19:06:08.398080+0000","last_fullsized":"2023-07-21T19:06:08.398080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:21.320856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400567+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:06:08.400567+0000","last_peered":"2023-07-21T19:06:08.400567+0000","last_clean":"2023-07-21T19:06:08.400567+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:06:08.400567+0000","last_undegraded":"2023-07-21T19:06:08.400567+0000","last_fullsized":"2023-07-21T19:06:08.400567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:41.115053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402164+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:06:08.402164+0000","last_peered":"2023-07-21T19:06:08.402164+0000","last_clean":"2023-07-21T19:06:08.402164+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:06:08.402164+0000","last_undegraded":"2023-07-21T19:06:08.402164+0000","last_fullsized":"2023-07-21T19:06:08.402164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:14:01.147711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404354+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:06:08.404354+0000","last_peered":"2023-07-21T19:06:08.404354+0000","last_clean":"2023-07-21T19:06:08.404354+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:06:08.404354+0000","last_undegraded":"2023-07-21T19:06:08.404354+0000","last_fullsized":"2023-07-21T19:06:08.404354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:28.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841118+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:06:08.841118+0000","last_peered":"2023-07-21T19:06:08.841118+0000","last_clean":"2023-07-21T19:06:08.841118+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:06:08.841118+0000","last_undegraded":"2023-07-21T19:06:08.841118+0000","last_fullsized":"2023-07-21T19:06:08.841118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:34.147350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401383+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:06:08.401383+0000","last_peered":"2023-07-21T19:06:08.401383+0000","last_clean":"2023-07-21T19:06:08.401383+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:06:08.401383+0000","last_undegraded":"2023-07-21T19:06:08.401383+0000","last_fullsized":"2023-07-21T19:06:08.401383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:39:13.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401273+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:06:08.401273+0000","last_peered":"2023-07-21T19:06:08.401273+0000","last_clean":"2023-07-21T19:06:08.401273+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:06:08.401273+0000","last_undegraded":"2023-07-21T19:06:08.401273+0000","last_fullsized":"2023-07-21T19:06:08.401273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:46.382674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":64,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:23.429160+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:06:23.429160+0000","last_peered":"2023-07-21T19:06:23.429160+0000","last_clean":"2023-07-21T19:06:23.429160+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:06:23.429160+0000","last_undegraded":"2023-07-21T19:06:23.429160+0000","last_fullsized":"2023-07-21T19:06:23.429160+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:59.692187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":43,"num_read_kb":43,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.843091+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:06:08.843091+0000","last_peered":"2023-07-21T19:06:08.843091+0000","last_clean":"2023-07-21T19:06:08.843091+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:06:08.843091+0000","last_undegraded":"2023-07-21T19:06:08.843091+0000","last_fullsized":"2023-07-21T19:06:08.843091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:32.432702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403970+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:06:08.403970+0000","last_peered":"2023-07-21T19:06:08.403970+0000","last_clean":"2023-07-21T19:06:08.403970+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:06:08.403970+0000","last_undegraded":"2023-07-21T19:06:08.403970+0000","last_fullsized":"2023-07-21T19:06:08.403970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:09.662672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397886+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:06:08.397886+0000","last_peered":"2023-07-21T19:06:08.397886+0000","last_clean":"2023-07-21T19:06:08.397886+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:06:08.397886+0000","last_undegraded":"2023-07-21T19:06:08.397886+0000","last_fullsized":"2023-07-21T19:06:08.397886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.677074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402384+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:06:08.402384+0000","last_peered":"2023-07-21T19:06:08.402384+0000","last_clean":"2023-07-21T19:06:08.402384+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:06:08.402384+0000","last_undegraded":"2023-07-21T19:06:08.402384+0000","last_fullsized":"2023-07-21T19:06:08.402384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:16:07.189240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842838+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:06:08.842838+0000","last_peered":"2023-07-21T19:06:08.842838+0000","last_clean":"2023-07-21T19:06:08.842838+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:06:08.842838+0000","last_undegraded":"2023-07-21T19:06:08.842838+0000","last_fullsized":"2023-07-21T19:06:08.842838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:01.676198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:24.430816+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:06:24.430816+0000","last_peered":"2023-07-21T19:06:24.430816+0000","last_clean":"2023-07-21T19:06:24.430816+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:06:24.430816+0000","last_undegraded":"2023-07-21T19:06:24.430816+0000","last_fullsized":"2023-07-21T19:06:24.430816+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.353689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58083,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":95,"num_read_kb":104,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398056+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:06:08.398056+0000","last_peered":"2023-07-21T19:06:08.398056+0000","last_clean":"2023-07-21T19:06:08.398056+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:06:08.398056+0000","last_undegraded":"2023-07-21T19:06:08.398056+0000","last_fullsized":"2023-07-21T19:06:08.398056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:31.374240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405148+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:06:08.405148+0000","last_peered":"2023-07-21T19:06:08.405148+0000","last_clean":"2023-07-21T19:06:08.405148+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:06:08.405148+0000","last_undegraded":"2023-07-21T19:06:08.405148+0000","last_fullsized":"2023-07-21T19:06:08.405148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:20.602887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402354+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:06:08.402354+0000","last_peered":"2023-07-21T19:06:08.402354+0000","last_clean":"2023-07-21T19:06:08.402354+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:06:08.402354+0000","last_undegraded":"2023-07-21T19:06:08.402354+0000","last_fullsized":"2023-07-21T19:06:08.402354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:59:21.737635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401848+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:06:08.401848+0000","last_peered":"2023-07-21T19:06:08.401848+0000","last_clean":"2023-07-21T19:06:08.401848+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:06:08.401848+0000","last_undegraded":"2023-07-21T19:06:08.401848+0000","last_fullsized":"2023-07-21T19:06:08.401848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:48.688808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404277+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:06:08.404277+0000","last_peered":"2023-07-21T19:06:08.404277+0000","last_clean":"2023-07-21T19:06:08.404277+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:06:08.404277+0000","last_undegraded":"2023-07-21T19:06:08.404277+0000","last_fullsized":"2023-07-21T19:06:08.404277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:55.454258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":66,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:23.360616+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:06:23.360616+0000","last_peered":"2023-07-21T19:06:23.360616+0000","last_clean":"2023-07-21T19:06:23.360616+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:06:23.360616+0000","last_undegraded":"2023-07-21T19:06:23.360616+0000","last_fullsized":"2023-07-21T19:06:23.360616+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:26.036729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403761+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:06:08.403761+0000","last_peered":"2023-07-21T19:06:08.403761+0000","last_clean":"2023-07-21T19:06:08.403761+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:06:08.403761+0000","last_undegraded":"2023-07-21T19:06:08.403761+0000","last_fullsized":"2023-07-21T19:06:08.403761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:24.967595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:31:53.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401833+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:06:08.401833+0000","last_peered":"2023-07-21T19:06:08.401833+0000","last_clean":"2023-07-21T19:06:08.401833+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:06:08.401833+0000","last_undegraded":"2023-07-21T19:06:08.401833+0000","last_fullsized":"2023-07-21T19:06:08.401833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:37.822944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398316+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:06:08.398316+0000","last_peered":"2023-07-21T19:06:08.398316+0000","last_clean":"2023-07-21T19:06:08.398316+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:06:08.398316+0000","last_undegraded":"2023-07-21T19:06:08.398316+0000","last_fullsized":"2023-07-21T19:06:08.398316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:37.555620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400438+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:06:08.400438+0000","last_peered":"2023-07-21T19:06:08.400438+0000","last_clean":"2023-07-21T19:06:08.400438+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:06:08.400438+0000","last_undegraded":"2023-07-21T19:06:08.400438+0000","last_fullsized":"2023-07-21T19:06:08.400438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:42.470455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.406908+0000","last_change":"2023-07-21T19:06:08.406908+0000","last_active":"2023-07-21T19:06:08.406908+0000","last_peered":"2023-07-21T19:06:08.406908+0000","last_clean":"2023-07-21T19:06:08.406908+0000","last_became_active":"2023-07-21T19:06:08.406531+0000","last_became_peered":"2023-07-21T19:06:08.406531+0000","last_unstale":"2023-07-21T19:06:08.406908+0000","last_undegraded":"2023-07-21T19:06:08.406908+0000","last_fullsized":"2023-07-21T19:06:08.406908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:04.808701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400382+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:06:08.400382+0000","last_peered":"2023-07-21T19:06:08.400382+0000","last_clean":"2023-07-21T19:06:08.400382+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:06:08.400382+0000","last_undegraded":"2023-07-21T19:06:08.400382+0000","last_fullsized":"2023-07-21T19:06:08.400382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:01.821241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404216+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:06:08.404216+0000","last_peered":"2023-07-21T19:06:08.404216+0000","last_clean":"2023-07-21T19:06:08.404216+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:06:08.404216+0000","last_undegraded":"2023-07-21T19:06:08.404216+0000","last_fullsized":"2023-07-21T19:06:08.404216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:18.679466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.408400+0000","last_change":"2023-07-21T19:06:08.408400+0000","last_active":"2023-07-21T19:06:08.408400+0000","last_peered":"2023-07-21T19:06:08.408400+0000","last_clean":"2023-07-21T19:06:08.408400+0000","last_became_active":"2023-07-21T19:06:08.408005+0000","last_became_peered":"2023-07-21T19:06:08.408005+0000","last_unstale":"2023-07-21T19:06:08.408400+0000","last_undegraded":"2023-07-21T19:06:08.408400+0000","last_fullsized":"2023-07-21T19:06:08.408400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:20.230383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.407629+0000","last_change":"2023-07-21T19:06:08.407039+0000","last_active":"2023-07-21T19:06:08.407629+0000","last_peered":"2023-07-21T19:06:08.407629+0000","last_clean":"2023-07-21T19:06:08.407629+0000","last_became_active":"2023-07-21T19:06:08.406635+0000","last_became_peered":"2023-07-21T19:06:08.406635+0000","last_unstale":"2023-07-21T19:06:08.407629+0000","last_undegraded":"2023-07-21T19:06:08.407629+0000","last_fullsized":"2023-07-21T19:06:08.407629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:18.504252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:26.364659+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:06:26.364659+0000","last_peered":"2023-07-21T19:06:26.364659+0000","last_clean":"2023-07-21T19:06:26.364659+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:06:26.364659+0000","last_undegraded":"2023-07-21T19:06:26.364659+0000","last_fullsized":"2023-07-21T19:06:26.364659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:21.302828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841283+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:06:08.841283+0000","last_peered":"2023-07-21T19:06:08.841283+0000","last_clean":"2023-07-21T19:06:08.841283+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:06:08.841283+0000","last_undegraded":"2023-07-21T19:06:08.841283+0000","last_fullsized":"2023-07-21T19:06:08.841283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:42.921526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398968+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:06:08.398968+0000","last_peered":"2023-07-21T19:06:08.398968+0000","last_clean":"2023-07-21T19:06:08.398968+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:06:08.398968+0000","last_undegraded":"2023-07-21T19:06:08.398968+0000","last_fullsized":"2023-07-21T19:06:08.398968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:50.624939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404235+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:06:08.404235+0000","last_peered":"2023-07-21T19:06:08.404235+0000","last_clean":"2023-07-21T19:06:08.404235+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:06:08.404235+0000","last_undegraded":"2023-07-21T19:06:08.404235+0000","last_fullsized":"2023-07-21T19:06:08.404235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:45.400614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.409780+0000","last_change":"2023-07-21T19:06:08.409780+0000","last_active":"2023-07-21T19:06:08.409780+0000","last_peered":"2023-07-21T19:06:08.409780+0000","last_clean":"2023-07-21T19:06:08.409780+0000","last_became_active":"2023-07-21T19:06:08.409394+0000","last_became_peered":"2023-07-21T19:06:08.409394+0000","last_unstale":"2023-07-21T19:06:08.409780+0000","last_undegraded":"2023-07-21T19:06:08.409780+0000","last_fullsized":"2023-07-21T19:06:08.409780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:36:45.599367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404518+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:06:08.404518+0000","last_peered":"2023-07-21T19:06:08.404518+0000","last_clean":"2023-07-21T19:06:08.404518+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:06:08.404518+0000","last_undegraded":"2023-07-21T19:06:08.404518+0000","last_fullsized":"2023-07-21T19:06:08.404518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:04.683020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.413663+0000","last_change":"2023-07-21T19:06:08.413252+0000","last_active":"2023-07-21T19:06:08.413663+0000","last_peered":"2023-07-21T19:06:08.413663+0000","last_clean":"2023-07-21T19:06:08.413663+0000","last_became_active":"2023-07-21T19:06:08.407914+0000","last_became_peered":"2023-07-21T19:06:08.407914+0000","last_unstale":"2023-07-21T19:06:08.413663+0000","last_undegraded":"2023-07-21T19:06:08.413663+0000","last_fullsized":"2023-07-21T19:06:08.413663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:08.813981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403061+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:06:08.403061+0000","last_peered":"2023-07-21T19:06:08.403061+0000","last_clean":"2023-07-21T19:06:08.403061+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:06:08.403061+0000","last_undegraded":"2023-07-21T19:06:08.403061+0000","last_fullsized":"2023-07-21T19:06:08.403061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:17.603158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400979+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:06:08.400979+0000","last_peered":"2023-07-21T19:06:08.400979+0000","last_clean":"2023-07-21T19:06:08.400979+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:06:08.400979+0000","last_undegraded":"2023-07-21T19:06:08.400979+0000","last_fullsized":"2023-07-21T19:06:08.400979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:46.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841994+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:06:08.841994+0000","last_peered":"2023-07-21T19:06:08.841994+0000","last_clean":"2023-07-21T19:06:08.841994+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:06:08.841994+0000","last_undegraded":"2023-07-21T19:06:08.841994+0000","last_fullsized":"2023-07-21T19:06:08.841994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:34:12.413946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399221+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:06:08.399221+0000","last_peered":"2023-07-21T19:06:08.399221+0000","last_clean":"2023-07-21T19:06:08.399221+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:06:08.399221+0000","last_undegraded":"2023-07-21T19:06:08.399221+0000","last_fullsized":"2023-07-21T19:06:08.399221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.493447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842743+0000","last_change":"2023-07-21T19:06:08.842743+0000","last_active":"2023-07-21T19:06:08.842743+0000","last_peered":"2023-07-21T19:06:08.842743+0000","last_clean":"2023-07-21T19:06:08.842743+0000","last_became_active":"2023-07-21T19:06:08.842349+0000","last_became_peered":"2023-07-21T19:06:08.842349+0000","last_unstale":"2023-07-21T19:06:08.842743+0000","last_undegraded":"2023-07-21T19:06:08.842743+0000","last_fullsized":"2023-07-21T19:06:08.842743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:05:30.960744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841889+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:06:08.841889+0000","last_peered":"2023-07-21T19:06:08.841889+0000","last_clean":"2023-07-21T19:06:08.841889+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:06:08.841889+0000","last_undegraded":"2023-07-21T19:06:08.841889+0000","last_fullsized":"2023-07-21T19:06:08.841889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:57.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:17.675575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840681+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:06:08.840681+0000","last_peered":"2023-07-21T19:06:08.840681+0000","last_clean":"2023-07-21T19:06:08.840681+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:06:08.840681+0000","last_undegraded":"2023-07-21T19:06:08.840681+0000","last_fullsized":"2023-07-21T19:06:08.840681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:11.395258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400162+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:06:08.400162+0000","last_peered":"2023-07-21T19:06:08.400162+0000","last_clean":"2023-07-21T19:06:08.400162+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:06:08.400162+0000","last_undegraded":"2023-07-21T19:06:08.400162+0000","last_fullsized":"2023-07-21T19:06:08.400162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:28:35.734063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404763+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:06:08.404763+0000","last_peered":"2023-07-21T19:06:08.404763+0000","last_clean":"2023-07-21T19:06:08.404763+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:06:08.404763+0000","last_undegraded":"2023-07-21T19:06:08.404763+0000","last_fullsized":"2023-07-21T19:06:08.404763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:53.659216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841762+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:06:08.841762+0000","last_peered":"2023-07-21T19:06:08.841762+0000","last_clean":"2023-07-21T19:06:08.841762+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:06:08.841762+0000","last_undegraded":"2023-07-21T19:06:08.841762+0000","last_fullsized":"2023-07-21T19:06:08.841762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:19:55.786796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398085+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:06:08.398085+0000","last_peered":"2023-07-21T19:06:08.398085+0000","last_clean":"2023-07-21T19:06:08.398085+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:06:08.398085+0000","last_undegraded":"2023-07-21T19:06:08.398085+0000","last_fullsized":"2023-07-21T19:06:08.398085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:02.430774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404717+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:06:08.404717+0000","last_peered":"2023-07-21T19:06:08.404717+0000","last_clean":"2023-07-21T19:06:08.404717+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:06:08.404717+0000","last_undegraded":"2023-07-21T19:06:08.404717+0000","last_fullsized":"2023-07-21T19:06:08.404717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:08:39.735825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405117+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:06:08.405117+0000","last_peered":"2023-07-21T19:06:08.405117+0000","last_clean":"2023-07-21T19:06:08.405117+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:06:08.405117+0000","last_undegraded":"2023-07-21T19:06:08.405117+0000","last_fullsized":"2023-07-21T19:06:08.405117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:44.759183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404544+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:06:08.404544+0000","last_peered":"2023-07-21T19:06:08.404544+0000","last_clean":"2023-07-21T19:06:08.404544+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:06:08.404544+0000","last_undegraded":"2023-07-21T19:06:08.404544+0000","last_fullsized":"2023-07-21T19:06:08.404544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:16.325005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402542+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:06:08.402542+0000","last_peered":"2023-07-21T19:06:08.402542+0000","last_clean":"2023-07-21T19:06:08.402542+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:06:08.402542+0000","last_undegraded":"2023-07-21T19:06:08.402542+0000","last_fullsized":"2023-07-21T19:06:08.402542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:41.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404357+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:06:08.404357+0000","last_peered":"2023-07-21T19:06:08.404357+0000","last_clean":"2023-07-21T19:06:08.404357+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:06:08.404357+0000","last_undegraded":"2023-07-21T19:06:08.404357+0000","last_fullsized":"2023-07-21T19:06:08.404357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:08:59.431298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401998+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:06:08.401998+0000","last_peered":"2023-07-21T19:06:08.401998+0000","last_clean":"2023-07-21T19:06:08.401998+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:06:08.401998+0000","last_undegraded":"2023-07-21T19:06:08.401998+0000","last_fullsized":"2023-07-21T19:06:08.401998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:17.091156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404223+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:06:08.404223+0000","last_peered":"2023-07-21T19:06:08.404223+0000","last_clean":"2023-07-21T19:06:08.404223+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:06:08.404223+0000","last_undegraded":"2023-07-21T19:06:08.404223+0000","last_fullsized":"2023-07-21T19:06:08.404223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:09.484987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400233+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:06:08.400233+0000","last_peered":"2023-07-21T19:06:08.400233+0000","last_clean":"2023-07-21T19:06:08.400233+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:06:08.400233+0000","last_undegraded":"2023-07-21T19:06:08.400233+0000","last_fullsized":"2023-07-21T19:06:08.400233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:56.095641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404919+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:06:08.404919+0000","last_peered":"2023-07-21T19:06:08.404919+0000","last_clean":"2023-07-21T19:06:08.404919+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:06:08.404919+0000","last_undegraded":"2023-07-21T19:06:08.404919+0000","last_fullsized":"2023-07-21T19:06:08.404919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:24.580856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404758+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:06:08.404758+0000","last_peered":"2023-07-21T19:06:08.404758+0000","last_clean":"2023-07-21T19:06:08.404758+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:06:08.404758+0000","last_undegraded":"2023-07-21T19:06:08.404758+0000","last_fullsized":"2023-07-21T19:06:08.404758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:17.790433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397718+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:06:08.397718+0000","last_peered":"2023-07-21T19:06:08.397718+0000","last_clean":"2023-07-21T19:06:08.397718+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:06:08.397718+0000","last_undegraded":"2023-07-21T19:06:08.397718+0000","last_fullsized":"2023-07-21T19:06:08.397718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:29.261412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403190+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:06:08.403190+0000","last_peered":"2023-07-21T19:06:08.403190+0000","last_clean":"2023-07-21T19:06:08.403190+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:06:08.403190+0000","last_undegraded":"2023-07-21T19:06:08.403190+0000","last_fullsized":"2023-07-21T19:06:08.403190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:47:56.761424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400984+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:06:08.400984+0000","last_peered":"2023-07-21T19:06:08.400984+0000","last_clean":"2023-07-21T19:06:08.400984+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:06:08.400984+0000","last_undegraded":"2023-07-21T19:06:08.400984+0000","last_fullsized":"2023-07-21T19:06:08.400984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:45:42.804371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405668+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:06:08.405668+0000","last_peered":"2023-07-21T19:06:08.405668+0000","last_clean":"2023-07-21T19:06:08.405668+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:06:08.405668+0000","last_undegraded":"2023-07-21T19:06:08.405668+0000","last_fullsized":"2023-07-21T19:06:08.405668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:42.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841925+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:06:08.841925+0000","last_peered":"2023-07-21T19:06:08.841925+0000","last_clean":"2023-07-21T19:06:08.841925+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:06:08.841925+0000","last_undegraded":"2023-07-21T19:06:08.841925+0000","last_fullsized":"2023-07-21T19:06:08.841925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:40:21.846553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401440+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:06:08.401440+0000","last_peered":"2023-07-21T19:06:08.401440+0000","last_clean":"2023-07-21T19:06:08.401440+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:06:08.401440+0000","last_undegraded":"2023-07-21T19:06:08.401440+0000","last_fullsized":"2023-07-21T19:06:08.401440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:19.869711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841159+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:06:08.841159+0000","last_peered":"2023-07-21T19:06:08.841159+0000","last_clean":"2023-07-21T19:06:08.841159+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:06:08.841159+0000","last_undegraded":"2023-07-21T19:06:08.841159+0000","last_fullsized":"2023-07-21T19:06:08.841159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:33.674612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842744+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:06:08.842744+0000","last_peered":"2023-07-21T19:06:08.842744+0000","last_clean":"2023-07-21T19:06:08.842744+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:06:08.842744+0000","last_undegraded":"2023-07-21T19:06:08.842744+0000","last_fullsized":"2023-07-21T19:06:08.842744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:23:36.091795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402746+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:06:08.402746+0000","last_peered":"2023-07-21T19:06:08.402746+0000","last_clean":"2023-07-21T19:06:08.402746+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:06:08.402746+0000","last_undegraded":"2023-07-21T19:06:08.402746+0000","last_fullsized":"2023-07-21T19:06:08.402746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:45:18.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400237+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:06:08.400237+0000","last_peered":"2023-07-21T19:06:08.400237+0000","last_clean":"2023-07-21T19:06:08.400237+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:06:08.400237+0000","last_undegraded":"2023-07-21T19:06:08.400237+0000","last_fullsized":"2023-07-21T19:06:08.400237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:02.887348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61987,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":282,"num_read_kb":211,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1568,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":185961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":456247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.752},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.88900000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645341,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1105747,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.627,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.754,"15min":0.746},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.71399999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81999999999999995,"15min":0.80800000000000005},"min":{"1min":0.54700000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.89700000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.71699999999999997},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96099999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.78300000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83199999999999996,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90000000000000002,"5min":1.238,"15min":1.238},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.90700000000000003,"15min":0.88200000000000001},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.82299999999999995},{"interface":"front","average":{"1min":1.252,"5min":0.98599999999999999,"15min":0.94199999999999995},"min":{"1min":0.63300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.7720000000000002,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.98599999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93700000000000006,"15min":0.93400000000000005},"min":{"1min":0.68300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":1.016},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":0.98499999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.034,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.83599999999999997}]},{"osd":9,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.0529999999999999,"15min":1.0129999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":6.4249999999999998,"5min":6.4249999999999998,"15min":6.4249999999999998},"last":1.0429999999999999},{"interface":"front","average":{"1min":1.004,"5min":1.0249999999999999,"15min":1.0289999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.94899999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.97399999999999998}]}]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":456461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.76600000000000001},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":1.204,"15min":1.204},"last":0.496},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88200000000000001,"5min":1.087,"15min":1.087},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98299999999999998,"5min":1.383,"15min":1.383},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.998,"15min":0.998},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84399999999999997,"15min":0.86299999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.448,"15min":1.448},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.751},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1240000000000001,"5min":1.508,"15min":1.508},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1040000000000001,"5min":1.345,"15min":1.345},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.028},"min":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1910000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.89300000000000002}]},{"osd":8,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97199999999999998,"15min":0.98099999999999998},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0049999999999999,"15min":1.014},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.0389999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":9.3439999999999994,"5min":9.3439999999999994,"15min":9.3439999999999994},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.014,"15min":1.04},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070503,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":455777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.65700000000000003,"15min":0.64900000000000002},"min":{"1min":0.503,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.70799999999999996,"15min":0.70499999999999996},"min":{"1min":0.57099999999999995,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.85999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.84899999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57499999999999996},"min":{"1min":0.39000000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.40899999999999997,"5min":0.38,"15min":0.38},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.63800000000000001}]},{"osd":3,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.96999999999999997,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.71199999999999997,"15min":0.71899999999999997},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.753}]},{"osd":4,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.73899999999999999,"15min":0.73199999999999998},"min":{"1min":0.505,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75800000000000001,"15min":0.754},"min":{"1min":0.60599999999999998,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.0940000000000001,"5min":1.169,"15min":1.169},"last":0.92200000000000004}]},{"osd":5,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.748,"15min":0.752},"min":{"1min":0.51600000000000001,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.109,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.81999999999999995,"15min":0.82499999999999996},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.80000000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80200000000000005,"15min":0.80600000000000005},"min":{"1min":0.55200000000000005,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.85699999999999998,"15min":0.86699999999999999},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.079,"5min":1.359,"15min":1.359},"last":0.78300000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.84799999999999998,"15min":0.84999999999999998},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0680000000000001,"5min":1.403,"15min":1.403},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.873,"5min":0.88900000000000001,"15min":0.89200000000000002},"min":{"1min":0.65900000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.167,"5min":1.167,"15min":1.167},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.86299999999999999,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.82799999999999996,"15min":0.81299999999999994},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.98599999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94299999999999995,"15min":0.93200000000000005},"min":{"1min":0.69799999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.871},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.89400000000000002,"15min":0.878},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.0129999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93200000000000005}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":1048220,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.05},"last":0.629}]},{"osd":3,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.51200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.435,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.016},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.996,"15min":0.996},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.93700000000000006,"15min":1.0229999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0449999999999999,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0740000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.008,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95699999999999996,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.82799999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.234,"15min":1.234},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81699999999999995,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.208,"15min":1.208},"last":0.69299999999999995}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82099999999999995,"15min":0.81799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0109999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83999999999999997,"15min":0.83799999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.087,"5min":1.095,"15min":1.095},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.93700000000000006,"15min":0.93899999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":455856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95199999999999996,"5min":1.087,"15min":1.087},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.875},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.018,"5min":1.149,"15min":1.149},"last":0.84899999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.04,"15min":1.04},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0669999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89700000000000002,"15min":0.85299999999999998},"min":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.22,"5min":1.984,"15min":1.984},"last":0.754},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1870000000000001,"5min":1.754,"15min":1.754},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94899999999999995,"15min":0.94299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.9969999999999999,"5min":3.9969999999999999,"15min":3.9969999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.3049999999999999,"5min":1.923,"15min":1.923},"last":0.91700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574925,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":514308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.60899999999999999,"15min":0.60399999999999998},"min":{"1min":0.38,"5min":0.31,"15min":0.31},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.63300000000000001,"15min":0.628},"min":{"1min":0.41699999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":1.194,"5min":1.194,"15min":1.194},"last":0.60499999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79500000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.66200000000000003,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":1.054,"15min":1.054},"last":0.996}]},{"osd":3,"last update":"Fri Jul 21 19:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69499999999999995,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Jul 21 19:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.626},{"interface":"front","average":{"1min":0.624,"5min":0.54600000000000004,"15min":0.505},"min":{"1min":0.39700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.90500000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.68600000000000005,"15min":0.64800000000000002},"min":{"1min":0.56100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.72899999999999998,"15min":0.69199999999999995},"min":{"1min":0.59099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":1.1160000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.048,"15min":1.048},"last":1.044},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.79500000000000004}]},{"osd":7,"last update":"Fri Jul 21 19:05:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.44600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.087},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97699999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.95599999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.069},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.61499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.0269999999999999,"5min":1.502,"15min":1.502},"last":1.02}]},{"osd":9,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88500000000000001,"15min":0.88700000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1359999999999999,"5min":1.149,"15min":1.149},"last":1.1439999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90300000000000002,"15min":0.90400000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1459999999999999,"5min":1.147,"15min":1.147},"last":1.0349999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.1679999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1479999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.105}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738450,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":457121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65300000000000002,"15min":0.625},"min":{"1min":0.46600000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.0369999999999999,"5min":1.177,"15min":1.177},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.65500000000000003,"15min":0.622},"min":{"1min":0.48799999999999999,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.96599999999999997,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.57299999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91200000000000003,"5min":1.139,"15min":1.139},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":1.113,"15min":1.113},"last":0.79500000000000004}]},{"osd":3,"last update":"Fri Jul 21 19:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.60599999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84399999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.51900000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.63800000000000001}]},{"osd":6,"last update":"Fri Jul 21 19:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93999999999999995,"5min":1.163,"15min":1.163},"last":0.69999999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:05:27 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.89300000000000002,"15min":0.92000000000000004},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.032,"5min":1.304,"15min":1.304},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.83599999999999997,"15min":0.84999999999999998},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.94899999999999995,"5min":1.2470000000000001,"15min":1.2470000000000001},"last":0.87}]},{"osd":9,"last update":"Fri Jul 21 19:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.622},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.94199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.274,"15min":1.274},"last":0.93999999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.84899999999999998,"15min":0.85499999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.034,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.72099999999999997}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.77800000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411399,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":514454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.746}]},{"osd":1,"last update":"Fri Jul 21 19:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.99399999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":1.081,"15min":1.081},"last":0.79400000000000004}]},{"osd":3,"last update":"Fri Jul 21 19:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.92900000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.57699999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":1.002}]},{"osd":5,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58799999999999997,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94599999999999995,"5min":1.167,"15min":1.167},"last":0.54900000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.75800000000000001,"15min":0.751},"min":{"1min":0.58199999999999996,"5min":0.499,"15min":0.499},"max":{"1min":1.0609999999999999,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.71899999999999997,"15min":0.71499999999999997},"min":{"1min":0.48399999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.86099999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:05:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":1.0309999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.88300000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.72499999999999998,"15min":0.72299999999999998},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.0129999999999999},"last":0.89900000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.85499999999999998,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.153,"5min":1.333,"15min":1.333},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.88,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.089,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.95799999999999996}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.84099999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":456173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.90900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.44400000000000001},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76400000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.53700000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.048,"15min":1.048},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.64100000000000001},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.052,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.63700000000000001},"min":{"1min":0.56499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.048,"15min":1.048},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jul 21 19:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.006,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73099999999999998,"15min":0.68999999999999995},"min":{"1min":0.57299999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.175,"15min":1.175},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.044,"5min":1.264,"15min":1.264},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":1.141,"15min":1.141},"last":0.36599999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.61699999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.167,"15min":1.167},"last":0.624},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84599999999999997,"15min":0.82399999999999995},"min":{"1min":0.70999999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.034,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.133,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.196,"15min":1.196},"last":0.54100000000000004}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.091,"5min":1.153,"15min":1.153},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920828,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":457203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.47299999999999998,"15min":0.439},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.47599999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":1.167,"15min":1.167},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96199999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.66800000000000004}]},{"osd":3,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68400000000000005,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97799999999999998,"5min":1.083,"15min":1.083},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91000000000000003,"5min":1.079,"15min":1.079},"last":0.60599999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.60999999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93600000000000005,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.92100000000000004}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.006,"5min":1.119,"15min":1.119},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.64400000000000002}]},{"osd":7,"last update":"Fri Jul 21 19:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.105,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.87}]},{"osd":8,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69699999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0920000000000001,"5min":1.155,"15min":1.155},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1220000000000001,"5min":1.298,"15min":1.298},"last":0.94899999999999995}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.746},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.244,"15min":1.244},"last":0.88100000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":2.6499999999999999,"15min":2.9460000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1559999999999999,"5min":42.396000000000001,"15min":42.396000000000001},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":2.5619999999999998,"15min":2.8679999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.86699999999999999,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.79300000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83799999999999997}]}]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1047952,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":1.042,"15min":1.042},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.877}]},{"osd":1,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70099999999999996,"15min":0.68400000000000005},"min":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0189999999999999,"5min":1.028,"15min":1.028},"last":0.81999999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.499},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91100000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.84699999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88700000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.93100000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98999999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.77000000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88800000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.80200000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.90300000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":1.093}]},{"osd":8,"last update":"Fri Jul 21 19:05:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.73699999999999999,"15min":0.73299999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.746,"15min":0.74399999999999999},"min":{"1min":0.38500000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":1.02}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0309999999999999,"5min":1.05,"15min":1.05},"last":1.139},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97799999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.84399999999999997}]},{"osd":10,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.69799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.141,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.71699999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.1819999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":59613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:06:30.609 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph pg dump --format=json 2023-07-21T19:06:30.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:30 smithi005 ceph-mon[126460]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:30.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:30 smithi156 ceph-mon[128316]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:30.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:30 smithi101 ceph-mon[128136]: pgmap v328: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:31.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:31 smithi005 ceph-mon[126460]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:31.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:31 smithi156 ceph-mon[128316]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:31.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:31 smithi101 ceph-mon[128136]: from='client.15030 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:32.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:32 smithi156 ceph-mon[128316]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:32.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 -- 172.21.15.5:0/3138074206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740ffe00 msgr2=0x7fd9741001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 --2- 172.21.15.5:0/3138074206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740ffe00 0x7fd9741001d0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fd9700099c0 tx=0x7fd970009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 -- 172.21.15.5:0/3138074206 shutdown_connections 2023-07-21T19:06:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 --2- 172.21.15.5:0/3138074206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd974104ea0 0x7fd97410ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 --2- 172.21.15.5:0/3138074206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd9740ffe00 0x7fd9741001d0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 --2- 172.21.15.5:0/3138074206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 -- 172.21.15.5:0/3138074206 >> 172.21.15.5:0/3138074206 conn(0x7fd9740fb660 msgr2=0x7fd9740fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 -- 172.21.15.5:0/3138074206 shutdown_connections 2023-07-21T19:06:32.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.763+0000 7fd97c56f700 1 -- 172.21.15.5:0/3138074206 wait complete. 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.764+0000 7fd97c56f700 1 Processor -- start 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 -- start start 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 0x7fd974197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 0x7fd97419c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:32.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419cda0 con 0x7fd974104ea0 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419cf10 con 0x7fd9740ffe00 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.765+0000 7fd97c56f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97419d0d0 con 0x7fd9741007a0 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97a30b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 0x7fd974197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 0x7fd97419c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97a30b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 0x7fd974197de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:39406/0 (socket says 172.21.15.5:39406) 2023-07-21T19:06:32.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97a30b700 1 -- 172.21.15.5:0/1245023066 learned_addr learned my addr 172.21.15.5:0/1245023066 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd979b0a700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:32.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 -- 172.21.15.5:0/1245023066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 msgr2=0x7fd974198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 -- 172.21.15.5:0/1245023066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 msgr2=0x7fd974197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 0x7fd974197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 -- 172.21.15.5:0/1245023066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9700095f0 con 0x7fd974104ea0 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd979b0a700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd97ab0c700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 0x7fd97419c790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007f10 tx=0x7fd96c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:32.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.766+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c009ea0 con 0x7fd974104ea0 2023-07-21T19:06:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.767+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97419d3b0 con 0x7fd974104ea0 2023-07-21T19:06:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.767+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd96c00f040 con 0x7fd974104ea0 2023-07-21T19:06:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.767+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004260 con 0x7fd974104ea0 2023-07-21T19:06:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.767+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9741a4c10 con 0x7fd974104ea0 2023-07-21T19:06:32.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.768+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97404e210 con 0x7fd974104ea0 2023-07-21T19:06:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.770+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fd96c003a00 con 0x7fd974104ea0 2023-07-21T19:06:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.771+0000 7fd96b7fe700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 0x7fd9600a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:32.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.771+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd96c0cc440 con 0x7fd974104ea0 2023-07-21T19:06:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.771+0000 7fd97a30b700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 0x7fd9600a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:32.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.772+0000 7fd97a30b700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 0x7fd9600a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd970005b00 tx=0x7fd970005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:32.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.775+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd96c0959e0 con 0x7fd974104ea0 2023-07-21T19:06:32.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:32 smithi101 ceph-mon[128136]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:32.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:32 smithi005 ceph-mon[126460]: pgmap v329: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.975+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd97410bd90 con 0x7fd9600a11e0 2023-07-21T19:06:32.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.986+0000 7fd96b7fe700 1 -- 172.21.15.5:0/1245023066 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+339682 (secure 0 0 0) 0x7fd97410bd90 con 0x7fd9600a11e0 2023-07-21T19:06:33.001 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 msgr2=0x7fd9600a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 0x7fd9600a3690 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd970005b00 tx=0x7fd970005a90 comp rx=0 tx=0).stop 2023-07-21T19:06:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 msgr2=0x7fd97419c790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 0x7fd97419c790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fd96c007f10 tx=0x7fd96c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 shutdown_connections 2023-07-21T19:06:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd9600a11e0 0x7fd9600a3690 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd974104ea0 0x7fd97419c790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd9740ffe00 0x7fd974197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 --2- 172.21.15.5:0/1245023066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd9741007a0 0x7fd974198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.991+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 >> 172.21.15.5:0/1245023066 conn(0x7fd9740fb660 msgr2=0x7fd9740fc380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.992+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 shutdown_connections 2023-07-21T19:06:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:32.992+0000 7fd97c56f700 1 -- 172.21.15.5:0/1245023066 wait complete. 2023-07-21T19:06:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:06:33.448 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":329,"stamp":"2023-07-21T19:06:31.325589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":652601,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":443,"num_read_kb":340,"num_write":337,"num_write_kb":3046,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182304,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":273,"ondisk_log_size":273,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334992,"kb_used_data":11568,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606832,"statfs":{"total":1151940427776,"available":1151597395968,"internally_reserved":0,"allocated":11845632,"data_stored":7425519,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":85,"num_read_kb":40,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002933"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400686+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:06:08.400686+0000","last_peered":"2023-07-21T19:06:08.400686+0000","last_clean":"2023-07-21T19:06:08.400686+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:06:08.400686+0000","last_undegraded":"2023-07-21T19:06:08.400686+0000","last_fullsized":"2023-07-21T19:06:08.400686+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:46.910120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397988+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:06:08.397988+0000","last_peered":"2023-07-21T19:06:08.397988+0000","last_clean":"2023-07-21T19:06:08.397988+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:06:08.397988+0000","last_undegraded":"2023-07-21T19:06:08.397988+0000","last_fullsized":"2023-07-21T19:06:08.397988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:32.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397678+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:06:08.397678+0000","last_peered":"2023-07-21T19:06:08.397678+0000","last_clean":"2023-07-21T19:06:08.397678+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:06:08.397678+0000","last_undegraded":"2023-07-21T19:06:08.397678+0000","last_fullsized":"2023-07-21T19:06:08.397678+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:43.019102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399400+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:06:08.399400+0000","last_peered":"2023-07-21T19:06:08.399400+0000","last_clean":"2023-07-21T19:06:08.399400+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:06:08.399400+0000","last_undegraded":"2023-07-21T19:06:08.399400+0000","last_fullsized":"2023-07-21T19:06:08.399400+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:20.204377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398192+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:06:08.398192+0000","last_peered":"2023-07-21T19:06:08.398192+0000","last_clean":"2023-07-21T19:06:08.398192+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:06:08.398192+0000","last_undegraded":"2023-07-21T19:06:08.398192+0000","last_fullsized":"2023-07-21T19:06:08.398192+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:04.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400393+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:06:08.400393+0000","last_peered":"2023-07-21T19:06:08.400393+0000","last_clean":"2023-07-21T19:06:08.400393+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:06:08.400393+0000","last_undegraded":"2023-07-21T19:06:08.400393+0000","last_fullsized":"2023-07-21T19:06:08.400393+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:17.572777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404051+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:06:08.404051+0000","last_peered":"2023-07-21T19:06:08.404051+0000","last_clean":"2023-07-21T19:06:08.404051+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:06:08.404051+0000","last_undegraded":"2023-07-21T19:06:08.404051+0000","last_fullsized":"2023-07-21T19:06:08.404051+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:50:34.608525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841189+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:06:08.841189+0000","last_peered":"2023-07-21T19:06:08.841189+0000","last_clean":"2023-07-21T19:06:08.841189+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:06:08.841189+0000","last_undegraded":"2023-07-21T19:06:08.841189+0000","last_fullsized":"2023-07-21T19:06:08.841189+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:29.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.843235+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:06:08.843235+0000","last_peered":"2023-07-21T19:06:08.843235+0000","last_clean":"2023-07-21T19:06:08.843235+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:06:08.843235+0000","last_undegraded":"2023-07-21T19:06:08.843235+0000","last_fullsized":"2023-07-21T19:06:08.843235+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:08:37.146951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400414+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:06:08.400414+0000","last_peered":"2023-07-21T19:06:08.400414+0000","last_clean":"2023-07-21T19:06:08.400414+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:06:08.400414+0000","last_undegraded":"2023-07-21T19:06:08.400414+0000","last_fullsized":"2023-07-21T19:06:08.400414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:32.152857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404505+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:06:08.404505+0000","last_peered":"2023-07-21T19:06:08.404505+0000","last_clean":"2023-07-21T19:06:08.404505+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:06:08.404505+0000","last_undegraded":"2023-07-21T19:06:08.404505+0000","last_fullsized":"2023-07-21T19:06:08.404505+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:23.961311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399377+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:06:08.399377+0000","last_peered":"2023-07-21T19:06:08.399377+0000","last_clean":"2023-07-21T19:06:08.399377+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:06:08.399377+0000","last_undegraded":"2023-07-21T19:06:08.399377+0000","last_fullsized":"2023-07-21T19:06:08.399377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:15.583393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402871+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:06:08.402871+0000","last_peered":"2023-07-21T19:06:08.402871+0000","last_clean":"2023-07-21T19:06:08.402871+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:06:08.402871+0000","last_undegraded":"2023-07-21T19:06:08.402871+0000","last_fullsized":"2023-07-21T19:06:08.402871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.309276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397190+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:06:08.397190+0000","last_peered":"2023-07-21T19:06:08.397190+0000","last_clean":"2023-07-21T19:06:08.397190+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:06:08.397190+0000","last_undegraded":"2023-07-21T19:06:08.397190+0000","last_fullsized":"2023-07-21T19:06:08.397190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:09:18.215822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397323+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:06:08.397323+0000","last_peered":"2023-07-21T19:06:08.397323+0000","last_clean":"2023-07-21T19:06:08.397323+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:06:08.397323+0000","last_undegraded":"2023-07-21T19:06:08.397323+0000","last_fullsized":"2023-07-21T19:06:08.397323+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:59.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842577+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:06:08.842577+0000","last_peered":"2023-07-21T19:06:08.842577+0000","last_clean":"2023-07-21T19:06:08.842577+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:06:08.842577+0000","last_undegraded":"2023-07-21T19:06:08.842577+0000","last_fullsized":"2023-07-21T19:06:08.842577+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:13.624531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400685+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:06:08.400685+0000","last_peered":"2023-07-21T19:06:08.400685+0000","last_clean":"2023-07-21T19:06:08.400685+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:06:08.400685+0000","last_undegraded":"2023-07-21T19:06:08.400685+0000","last_fullsized":"2023-07-21T19:06:08.400685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:17.986171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401123+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:06:08.401123+0000","last_peered":"2023-07-21T19:06:08.401123+0000","last_clean":"2023-07-21T19:06:08.401123+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:06:08.401123+0000","last_undegraded":"2023-07-21T19:06:08.401123+0000","last_fullsized":"2023-07-21T19:06:08.401123+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:48.916207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405245+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:06:08.405245+0000","last_peered":"2023-07-21T19:06:08.405245+0000","last_clean":"2023-07-21T19:06:08.405245+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:06:08.405245+0000","last_undegraded":"2023-07-21T19:06:08.405245+0000","last_fullsized":"2023-07-21T19:06:08.405245+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:08.380972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.851006+0000","last_change":"2023-07-21T19:06:08.851006+0000","last_active":"2023-07-21T19:06:08.851006+0000","last_peered":"2023-07-21T19:06:08.851006+0000","last_clean":"2023-07-21T19:06:08.851006+0000","last_became_active":"2023-07-21T19:06:08.843812+0000","last_became_peered":"2023-07-21T19:06:08.843812+0000","last_unstale":"2023-07-21T19:06:08.851006+0000","last_undegraded":"2023-07-21T19:06:08.851006+0000","last_fullsized":"2023-07-21T19:06:08.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:03.813996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841892+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:06:08.841892+0000","last_peered":"2023-07-21T19:06:08.841892+0000","last_clean":"2023-07-21T19:06:08.841892+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:06:08.841892+0000","last_undegraded":"2023-07-21T19:06:08.841892+0000","last_fullsized":"2023-07-21T19:06:08.841892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:59.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840757+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:06:08.840757+0000","last_peered":"2023-07-21T19:06:08.840757+0000","last_clean":"2023-07-21T19:06:08.840757+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:06:08.840757+0000","last_undegraded":"2023-07-21T19:06:08.840757+0000","last_fullsized":"2023-07-21T19:06:08.840757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:19.820365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397764+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:06:08.397764+0000","last_peered":"2023-07-21T19:06:08.397764+0000","last_clean":"2023-07-21T19:06:08.397764+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:06:08.397764+0000","last_undegraded":"2023-07-21T19:06:08.397764+0000","last_fullsized":"2023-07-21T19:06:08.397764+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:48.371444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398380+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:06:08.398380+0000","last_peered":"2023-07-21T19:06:08.398380+0000","last_clean":"2023-07-21T19:06:08.398380+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:06:08.398380+0000","last_undegraded":"2023-07-21T19:06:08.398380+0000","last_fullsized":"2023-07-21T19:06:08.398380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:48.967070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842807+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:06:08.842807+0000","last_peered":"2023-07-21T19:06:08.842807+0000","last_clean":"2023-07-21T19:06:08.842807+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:06:08.842807+0000","last_undegraded":"2023-07-21T19:06:08.842807+0000","last_fullsized":"2023-07-21T19:06:08.842807+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:03:26.702743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398317+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:06:08.398317+0000","last_peered":"2023-07-21T19:06:08.398317+0000","last_clean":"2023-07-21T19:06:08.398317+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:06:08.398317+0000","last_undegraded":"2023-07-21T19:06:08.398317+0000","last_fullsized":"2023-07-21T19:06:08.398317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:56.271039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402196+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:06:08.402196+0000","last_peered":"2023-07-21T19:06:08.402196+0000","last_clean":"2023-07-21T19:06:08.402196+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:06:08.402196+0000","last_undegraded":"2023-07-21T19:06:08.402196+0000","last_fullsized":"2023-07-21T19:06:08.402196+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:12.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"85'6","reported_seq":88,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:29.438902+0000","last_change":"2023-07-21T19:06:08.410381+0000","last_active":"2023-07-21T19:06:29.438902+0000","last_peered":"2023-07-21T19:06:29.438902+0000","last_clean":"2023-07-21T19:06:29.438902+0000","last_became_active":"2023-07-21T19:06:08.405822+0000","last_became_peered":"2023-07-21T19:06:08.405822+0000","last_unstale":"2023-07-21T19:06:29.438902+0000","last_undegraded":"2023-07-21T19:06:29.438902+0000","last_fullsized":"2023-07-21T19:06:29.438902+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:14:23.390484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398080+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:06:08.398080+0000","last_peered":"2023-07-21T19:06:08.398080+0000","last_clean":"2023-07-21T19:06:08.398080+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:06:08.398080+0000","last_undegraded":"2023-07-21T19:06:08.398080+0000","last_fullsized":"2023-07-21T19:06:08.398080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:21.320856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400567+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:06:08.400567+0000","last_peered":"2023-07-21T19:06:08.400567+0000","last_clean":"2023-07-21T19:06:08.400567+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:06:08.400567+0000","last_undegraded":"2023-07-21T19:06:08.400567+0000","last_fullsized":"2023-07-21T19:06:08.400567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:41.115053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402164+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:06:08.402164+0000","last_peered":"2023-07-21T19:06:08.402164+0000","last_clean":"2023-07-21T19:06:08.402164+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:06:08.402164+0000","last_undegraded":"2023-07-21T19:06:08.402164+0000","last_fullsized":"2023-07-21T19:06:08.402164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:14:01.147711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404354+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:06:08.404354+0000","last_peered":"2023-07-21T19:06:08.404354+0000","last_clean":"2023-07-21T19:06:08.404354+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:06:08.404354+0000","last_undegraded":"2023-07-21T19:06:08.404354+0000","last_fullsized":"2023-07-21T19:06:08.404354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:28.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841118+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:06:08.841118+0000","last_peered":"2023-07-21T19:06:08.841118+0000","last_clean":"2023-07-21T19:06:08.841118+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:06:08.841118+0000","last_undegraded":"2023-07-21T19:06:08.841118+0000","last_fullsized":"2023-07-21T19:06:08.841118+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:34.147350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401383+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:06:08.401383+0000","last_peered":"2023-07-21T19:06:08.401383+0000","last_clean":"2023-07-21T19:06:08.401383+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:06:08.401383+0000","last_undegraded":"2023-07-21T19:06:08.401383+0000","last_fullsized":"2023-07-21T19:06:08.401383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:39:13.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401273+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:06:08.401273+0000","last_peered":"2023-07-21T19:06:08.401273+0000","last_clean":"2023-07-21T19:06:08.401273+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:06:08.401273+0000","last_undegraded":"2023-07-21T19:06:08.401273+0000","last_fullsized":"2023-07-21T19:06:08.401273+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:46.382674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"85'3","reported_seq":69,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:28.436932+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:06:28.436932+0000","last_peered":"2023-07-21T19:06:28.436932+0000","last_clean":"2023-07-21T19:06:28.436932+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:06:28.436932+0000","last_undegraded":"2023-07-21T19:06:28.436932+0000","last_fullsized":"2023-07-21T19:06:28.436932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:59.692187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":48,"num_read_kb":48,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.843091+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:06:08.843091+0000","last_peered":"2023-07-21T19:06:08.843091+0000","last_clean":"2023-07-21T19:06:08.843091+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:06:08.843091+0000","last_undegraded":"2023-07-21T19:06:08.843091+0000","last_fullsized":"2023-07-21T19:06:08.843091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:32.432702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403970+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:06:08.403970+0000","last_peered":"2023-07-21T19:06:08.403970+0000","last_clean":"2023-07-21T19:06:08.403970+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:06:08.403970+0000","last_undegraded":"2023-07-21T19:06:08.403970+0000","last_fullsized":"2023-07-21T19:06:08.403970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:09.662672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397886+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:06:08.397886+0000","last_peered":"2023-07-21T19:06:08.397886+0000","last_clean":"2023-07-21T19:06:08.397886+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:06:08.397886+0000","last_undegraded":"2023-07-21T19:06:08.397886+0000","last_fullsized":"2023-07-21T19:06:08.397886+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.677074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402384+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:06:08.402384+0000","last_peered":"2023-07-21T19:06:08.402384+0000","last_clean":"2023-07-21T19:06:08.402384+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:06:08.402384+0000","last_undegraded":"2023-07-21T19:06:08.402384+0000","last_fullsized":"2023-07-21T19:06:08.402384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:16:07.189240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842838+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:06:08.842838+0000","last_peered":"2023-07-21T19:06:08.842838+0000","last_clean":"2023-07-21T19:06:08.842838+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:06:08.842838+0000","last_undegraded":"2023-07-21T19:06:08.842838+0000","last_fullsized":"2023-07-21T19:06:08.842838+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:01.676198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'29","reported_seq":152,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:29.438281+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:06:29.438281+0000","last_peered":"2023-07-21T19:06:29.438281+0000","last_clean":"2023-07-21T19:06:29.438281+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:06:29.438281+0000","last_undegraded":"2023-07-21T19:06:29.438281+0000","last_fullsized":"2023-07-21T19:06:29.438281+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.353689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58083,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":105,"num_read_kb":109,"num_write":30,"num_write_kb":68,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398056+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:06:08.398056+0000","last_peered":"2023-07-21T19:06:08.398056+0000","last_clean":"2023-07-21T19:06:08.398056+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:06:08.398056+0000","last_undegraded":"2023-07-21T19:06:08.398056+0000","last_fullsized":"2023-07-21T19:06:08.398056+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:31.374240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405148+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:06:08.405148+0000","last_peered":"2023-07-21T19:06:08.405148+0000","last_clean":"2023-07-21T19:06:08.405148+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:06:08.405148+0000","last_undegraded":"2023-07-21T19:06:08.405148+0000","last_fullsized":"2023-07-21T19:06:08.405148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:20.602887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402354+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:06:08.402354+0000","last_peered":"2023-07-21T19:06:08.402354+0000","last_clean":"2023-07-21T19:06:08.402354+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:06:08.402354+0000","last_undegraded":"2023-07-21T19:06:08.402354+0000","last_fullsized":"2023-07-21T19:06:08.402354+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:59:21.737635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401848+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:06:08.401848+0000","last_peered":"2023-07-21T19:06:08.401848+0000","last_clean":"2023-07-21T19:06:08.401848+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:06:08.401848+0000","last_undegraded":"2023-07-21T19:06:08.401848+0000","last_fullsized":"2023-07-21T19:06:08.401848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:48.688808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404277+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:06:08.404277+0000","last_peered":"2023-07-21T19:06:08.404277+0000","last_clean":"2023-07-21T19:06:08.404277+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:06:08.404277+0000","last_undegraded":"2023-07-21T19:06:08.404277+0000","last_fullsized":"2023-07-21T19:06:08.404277+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:55.454258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"85'3","reported_seq":71,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:28.369292+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:06:28.369292+0000","last_peered":"2023-07-21T19:06:28.369292+0000","last_clean":"2023-07-21T19:06:28.369292+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:06:28.369292+0000","last_undegraded":"2023-07-21T19:06:28.369292+0000","last_fullsized":"2023-07-21T19:06:28.369292+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:26.036729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":51,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403761+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:06:08.403761+0000","last_peered":"2023-07-21T19:06:08.403761+0000","last_clean":"2023-07-21T19:06:08.403761+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:06:08.403761+0000","last_undegraded":"2023-07-21T19:06:08.403761+0000","last_fullsized":"2023-07-21T19:06:08.403761+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:24.967595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:31:53.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401833+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:06:08.401833+0000","last_peered":"2023-07-21T19:06:08.401833+0000","last_clean":"2023-07-21T19:06:08.401833+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:06:08.401833+0000","last_undegraded":"2023-07-21T19:06:08.401833+0000","last_fullsized":"2023-07-21T19:06:08.401833+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:37.822944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398316+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:06:08.398316+0000","last_peered":"2023-07-21T19:06:08.398316+0000","last_clean":"2023-07-21T19:06:08.398316+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:06:08.398316+0000","last_undegraded":"2023-07-21T19:06:08.398316+0000","last_fullsized":"2023-07-21T19:06:08.398316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:37.555620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400438+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:06:08.400438+0000","last_peered":"2023-07-21T19:06:08.400438+0000","last_clean":"2023-07-21T19:06:08.400438+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:06:08.400438+0000","last_undegraded":"2023-07-21T19:06:08.400438+0000","last_fullsized":"2023-07-21T19:06:08.400438+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:42.470455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.406908+0000","last_change":"2023-07-21T19:06:08.406908+0000","last_active":"2023-07-21T19:06:08.406908+0000","last_peered":"2023-07-21T19:06:08.406908+0000","last_clean":"2023-07-21T19:06:08.406908+0000","last_became_active":"2023-07-21T19:06:08.406531+0000","last_became_peered":"2023-07-21T19:06:08.406531+0000","last_unstale":"2023-07-21T19:06:08.406908+0000","last_undegraded":"2023-07-21T19:06:08.406908+0000","last_fullsized":"2023-07-21T19:06:08.406908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:04.808701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"85'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400382+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:06:08.400382+0000","last_peered":"2023-07-21T19:06:08.400382+0000","last_clean":"2023-07-21T19:06:08.400382+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:06:08.400382+0000","last_undegraded":"2023-07-21T19:06:08.400382+0000","last_fullsized":"2023-07-21T19:06:08.400382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:01.821241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404216+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:06:08.404216+0000","last_peered":"2023-07-21T19:06:08.404216+0000","last_clean":"2023-07-21T19:06:08.404216+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:06:08.404216+0000","last_undegraded":"2023-07-21T19:06:08.404216+0000","last_fullsized":"2023-07-21T19:06:08.404216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:18.679466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.408400+0000","last_change":"2023-07-21T19:06:08.408400+0000","last_active":"2023-07-21T19:06:08.408400+0000","last_peered":"2023-07-21T19:06:08.408400+0000","last_clean":"2023-07-21T19:06:08.408400+0000","last_became_active":"2023-07-21T19:06:08.408005+0000","last_became_peered":"2023-07-21T19:06:08.408005+0000","last_unstale":"2023-07-21T19:06:08.408400+0000","last_undegraded":"2023-07-21T19:06:08.408400+0000","last_fullsized":"2023-07-21T19:06:08.408400+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:20.230383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.407629+0000","last_change":"2023-07-21T19:06:08.407039+0000","last_active":"2023-07-21T19:06:08.407629+0000","last_peered":"2023-07-21T19:06:08.407629+0000","last_clean":"2023-07-21T19:06:08.407629+0000","last_became_active":"2023-07-21T19:06:08.406635+0000","last_became_peered":"2023-07-21T19:06:08.406635+0000","last_unstale":"2023-07-21T19:06:08.407629+0000","last_undegraded":"2023-07-21T19:06:08.407629+0000","last_fullsized":"2023-07-21T19:06:08.407629+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:18.504252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"85'4","reported_seq":68,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:26.364659+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:06:26.364659+0000","last_peered":"2023-07-21T19:06:26.364659+0000","last_clean":"2023-07-21T19:06:26.364659+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:06:26.364659+0000","last_undegraded":"2023-07-21T19:06:26.364659+0000","last_fullsized":"2023-07-21T19:06:26.364659+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:21.302828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841283+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:06:08.841283+0000","last_peered":"2023-07-21T19:06:08.841283+0000","last_clean":"2023-07-21T19:06:08.841283+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:06:08.841283+0000","last_undegraded":"2023-07-21T19:06:08.841283+0000","last_fullsized":"2023-07-21T19:06:08.841283+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:42.921526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398968+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:06:08.398968+0000","last_peered":"2023-07-21T19:06:08.398968+0000","last_clean":"2023-07-21T19:06:08.398968+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:06:08.398968+0000","last_undegraded":"2023-07-21T19:06:08.398968+0000","last_fullsized":"2023-07-21T19:06:08.398968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:50.624939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":348,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404235+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:06:08.404235+0000","last_peered":"2023-07-21T19:06:08.404235+0000","last_clean":"2023-07-21T19:06:08.404235+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:06:08.404235+0000","last_undegraded":"2023-07-21T19:06:08.404235+0000","last_fullsized":"2023-07-21T19:06:08.404235+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:45.400614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.409780+0000","last_change":"2023-07-21T19:06:08.409780+0000","last_active":"2023-07-21T19:06:08.409780+0000","last_peered":"2023-07-21T19:06:08.409780+0000","last_clean":"2023-07-21T19:06:08.409780+0000","last_became_active":"2023-07-21T19:06:08.409394+0000","last_became_peered":"2023-07-21T19:06:08.409394+0000","last_unstale":"2023-07-21T19:06:08.409780+0000","last_undegraded":"2023-07-21T19:06:08.409780+0000","last_fullsized":"2023-07-21T19:06:08.409780+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:36:45.599367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404518+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:06:08.404518+0000","last_peered":"2023-07-21T19:06:08.404518+0000","last_clean":"2023-07-21T19:06:08.404518+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:06:08.404518+0000","last_undegraded":"2023-07-21T19:06:08.404518+0000","last_fullsized":"2023-07-21T19:06:08.404518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:04.683020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.413663+0000","last_change":"2023-07-21T19:06:08.413252+0000","last_active":"2023-07-21T19:06:08.413663+0000","last_peered":"2023-07-21T19:06:08.413663+0000","last_clean":"2023-07-21T19:06:08.413663+0000","last_became_active":"2023-07-21T19:06:08.407914+0000","last_became_peered":"2023-07-21T19:06:08.407914+0000","last_unstale":"2023-07-21T19:06:08.413663+0000","last_undegraded":"2023-07-21T19:06:08.413663+0000","last_fullsized":"2023-07-21T19:06:08.413663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:08.813981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403061+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:06:08.403061+0000","last_peered":"2023-07-21T19:06:08.403061+0000","last_clean":"2023-07-21T19:06:08.403061+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:06:08.403061+0000","last_undegraded":"2023-07-21T19:06:08.403061+0000","last_fullsized":"2023-07-21T19:06:08.403061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:17.603158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"85'6","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400979+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:06:08.400979+0000","last_peered":"2023-07-21T19:06:08.400979+0000","last_clean":"2023-07-21T19:06:08.400979+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:06:08.400979+0000","last_undegraded":"2023-07-21T19:06:08.400979+0000","last_fullsized":"2023-07-21T19:06:08.400979+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:46.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":11,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841994+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:06:08.841994+0000","last_peered":"2023-07-21T19:06:08.841994+0000","last_clean":"2023-07-21T19:06:08.841994+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:06:08.841994+0000","last_undegraded":"2023-07-21T19:06:08.841994+0000","last_fullsized":"2023-07-21T19:06:08.841994+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:34:12.413946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399221+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:06:08.399221+0000","last_peered":"2023-07-21T19:06:08.399221+0000","last_clean":"2023-07-21T19:06:08.399221+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:06:08.399221+0000","last_undegraded":"2023-07-21T19:06:08.399221+0000","last_fullsized":"2023-07-21T19:06:08.399221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.493447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842743+0000","last_change":"2023-07-21T19:06:08.842743+0000","last_active":"2023-07-21T19:06:08.842743+0000","last_peered":"2023-07-21T19:06:08.842743+0000","last_clean":"2023-07-21T19:06:08.842743+0000","last_became_active":"2023-07-21T19:06:08.842349+0000","last_became_peered":"2023-07-21T19:06:08.842349+0000","last_unstale":"2023-07-21T19:06:08.842743+0000","last_undegraded":"2023-07-21T19:06:08.842743+0000","last_fullsized":"2023-07-21T19:06:08.842743+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:05:30.960744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841889+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:06:08.841889+0000","last_peered":"2023-07-21T19:06:08.841889+0000","last_clean":"2023-07-21T19:06:08.841889+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:06:08.841889+0000","last_undegraded":"2023-07-21T19:06:08.841889+0000","last_fullsized":"2023-07-21T19:06:08.841889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:57.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:17.675575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840681+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:06:08.840681+0000","last_peered":"2023-07-21T19:06:08.840681+0000","last_clean":"2023-07-21T19:06:08.840681+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:06:08.840681+0000","last_undegraded":"2023-07-21T19:06:08.840681+0000","last_fullsized":"2023-07-21T19:06:08.840681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:11.395258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400162+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:06:08.400162+0000","last_peered":"2023-07-21T19:06:08.400162+0000","last_clean":"2023-07-21T19:06:08.400162+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:06:08.400162+0000","last_undegraded":"2023-07-21T19:06:08.400162+0000","last_fullsized":"2023-07-21T19:06:08.400162+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:28:35.734063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404763+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:06:08.404763+0000","last_peered":"2023-07-21T19:06:08.404763+0000","last_clean":"2023-07-21T19:06:08.404763+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:06:08.404763+0000","last_undegraded":"2023-07-21T19:06:08.404763+0000","last_fullsized":"2023-07-21T19:06:08.404763+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:53.659216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841762+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:06:08.841762+0000","last_peered":"2023-07-21T19:06:08.841762+0000","last_clean":"2023-07-21T19:06:08.841762+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:06:08.841762+0000","last_undegraded":"2023-07-21T19:06:08.841762+0000","last_fullsized":"2023-07-21T19:06:08.841762+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:19:55.786796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398085+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:06:08.398085+0000","last_peered":"2023-07-21T19:06:08.398085+0000","last_clean":"2023-07-21T19:06:08.398085+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:06:08.398085+0000","last_undegraded":"2023-07-21T19:06:08.398085+0000","last_fullsized":"2023-07-21T19:06:08.398085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:02.430774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404717+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:06:08.404717+0000","last_peered":"2023-07-21T19:06:08.404717+0000","last_clean":"2023-07-21T19:06:08.404717+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:06:08.404717+0000","last_undegraded":"2023-07-21T19:06:08.404717+0000","last_fullsized":"2023-07-21T19:06:08.404717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:08:39.735825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405117+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:06:08.405117+0000","last_peered":"2023-07-21T19:06:08.405117+0000","last_clean":"2023-07-21T19:06:08.405117+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:06:08.405117+0000","last_undegraded":"2023-07-21T19:06:08.405117+0000","last_fullsized":"2023-07-21T19:06:08.405117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:44.759183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404544+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:06:08.404544+0000","last_peered":"2023-07-21T19:06:08.404544+0000","last_clean":"2023-07-21T19:06:08.404544+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:06:08.404544+0000","last_undegraded":"2023-07-21T19:06:08.404544+0000","last_fullsized":"2023-07-21T19:06:08.404544+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:16.325005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402542+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:06:08.402542+0000","last_peered":"2023-07-21T19:06:08.402542+0000","last_clean":"2023-07-21T19:06:08.402542+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:06:08.402542+0000","last_undegraded":"2023-07-21T19:06:08.402542+0000","last_fullsized":"2023-07-21T19:06:08.402542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:41.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"85'1","reported_seq":17,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404357+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:06:08.404357+0000","last_peered":"2023-07-21T19:06:08.404357+0000","last_clean":"2023-07-21T19:06:08.404357+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:06:08.404357+0000","last_undegraded":"2023-07-21T19:06:08.404357+0000","last_fullsized":"2023-07-21T19:06:08.404357+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:08:59.431298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401998+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:06:08.401998+0000","last_peered":"2023-07-21T19:06:08.401998+0000","last_clean":"2023-07-21T19:06:08.401998+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:06:08.401998+0000","last_undegraded":"2023-07-21T19:06:08.401998+0000","last_fullsized":"2023-07-21T19:06:08.401998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:17.091156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404223+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:06:08.404223+0000","last_peered":"2023-07-21T19:06:08.404223+0000","last_clean":"2023-07-21T19:06:08.404223+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:06:08.404223+0000","last_undegraded":"2023-07-21T19:06:08.404223+0000","last_fullsized":"2023-07-21T19:06:08.404223+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:09.484987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400233+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:06:08.400233+0000","last_peered":"2023-07-21T19:06:08.400233+0000","last_clean":"2023-07-21T19:06:08.400233+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:06:08.400233+0000","last_undegraded":"2023-07-21T19:06:08.400233+0000","last_fullsized":"2023-07-21T19:06:08.400233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:56.095641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404919+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:06:08.404919+0000","last_peered":"2023-07-21T19:06:08.404919+0000","last_clean":"2023-07-21T19:06:08.404919+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:06:08.404919+0000","last_undegraded":"2023-07-21T19:06:08.404919+0000","last_fullsized":"2023-07-21T19:06:08.404919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:24.580856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404758+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:06:08.404758+0000","last_peered":"2023-07-21T19:06:08.404758+0000","last_clean":"2023-07-21T19:06:08.404758+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:06:08.404758+0000","last_undegraded":"2023-07-21T19:06:08.404758+0000","last_fullsized":"2023-07-21T19:06:08.404758+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:17.790433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397718+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:06:08.397718+0000","last_peered":"2023-07-21T19:06:08.397718+0000","last_clean":"2023-07-21T19:06:08.397718+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:06:08.397718+0000","last_undegraded":"2023-07-21T19:06:08.397718+0000","last_fullsized":"2023-07-21T19:06:08.397718+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:29.261412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403190+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:06:08.403190+0000","last_peered":"2023-07-21T19:06:08.403190+0000","last_clean":"2023-07-21T19:06:08.403190+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:06:08.403190+0000","last_undegraded":"2023-07-21T19:06:08.403190+0000","last_fullsized":"2023-07-21T19:06:08.403190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:47:56.761424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400984+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:06:08.400984+0000","last_peered":"2023-07-21T19:06:08.400984+0000","last_clean":"2023-07-21T19:06:08.400984+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:06:08.400984+0000","last_undegraded":"2023-07-21T19:06:08.400984+0000","last_fullsized":"2023-07-21T19:06:08.400984+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:45:42.804371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405668+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:06:08.405668+0000","last_peered":"2023-07-21T19:06:08.405668+0000","last_clean":"2023-07-21T19:06:08.405668+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:06:08.405668+0000","last_undegraded":"2023-07-21T19:06:08.405668+0000","last_fullsized":"2023-07-21T19:06:08.405668+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:42.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841925+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:06:08.841925+0000","last_peered":"2023-07-21T19:06:08.841925+0000","last_clean":"2023-07-21T19:06:08.841925+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:06:08.841925+0000","last_undegraded":"2023-07-21T19:06:08.841925+0000","last_fullsized":"2023-07-21T19:06:08.841925+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:40:21.846553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401440+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:06:08.401440+0000","last_peered":"2023-07-21T19:06:08.401440+0000","last_clean":"2023-07-21T19:06:08.401440+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:06:08.401440+0000","last_undegraded":"2023-07-21T19:06:08.401440+0000","last_fullsized":"2023-07-21T19:06:08.401440+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:19.869711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841159+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:06:08.841159+0000","last_peered":"2023-07-21T19:06:08.841159+0000","last_clean":"2023-07-21T19:06:08.841159+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:06:08.841159+0000","last_undegraded":"2023-07-21T19:06:08.841159+0000","last_fullsized":"2023-07-21T19:06:08.841159+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:33.674612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.842744+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:06:08.842744+0000","last_peered":"2023-07-21T19:06:08.842744+0000","last_clean":"2023-07-21T19:06:08.842744+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:06:08.842744+0000","last_undegraded":"2023-07-21T19:06:08.842744+0000","last_fullsized":"2023-07-21T19:06:08.842744+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:23:36.091795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402746+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:06:08.402746+0000","last_peered":"2023-07-21T19:06:08.402746+0000","last_clean":"2023-07-21T19:06:08.402746+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:06:08.402746+0000","last_undegraded":"2023-07-21T19:06:08.402746+0000","last_fullsized":"2023-07-21T19:06:08.402746+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:45:18.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":16,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400237+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:06:08.400237+0000","last_peered":"2023-07-21T19:06:08.400237+0000","last_clean":"2023-07-21T19:06:08.400237+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:06:08.400237+0000","last_undegraded":"2023-07-21T19:06:08.400237+0000","last_fullsized":"2023-07-21T19:06:08.400237+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:02.887348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":61987,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":307,"num_read_kb":221,"num_write":154,"num_write_kb":106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1568,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":185961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":134,"ondisk_log_size":134,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383704,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":456247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.41899999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.748,"15min":0.748},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.4279999999999999,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.872}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.752},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.1859999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.84399999999999997}]},{"osd":5,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.208,"5min":1.208,"15min":1.208},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.4570000000000001,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81299999999999994}]},{"osd":6,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.72499999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.595,"5min":1.595,"15min":1.595},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.5660000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.77100000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83099999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"max":{"1min":1.3779999999999999,"5min":1.3779999999999999,"15min":1.3779999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.93700000000000006}]},{"osd":10,"last update":"Fri Jul 21 19:05:41 2023","interfaces":[{"interface":"back","average":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.615,"5min":1.615,"15min":1.615},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":0.98399999999999999},"min":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.681,"5min":1.681,"15min":1.681},"last":0.88900000000000001}]}]},{"osd":10,"up_from":72,"seq":309237645342,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1105747,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.58399999999999996,"15min":0.57399999999999995},"min":{"1min":0.42299999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":2.3809999999999998,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.627,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.36499999999999999,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":2.323,"5min":2.323,"15min":2.323},"last":0.38500000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72899999999999998,"15min":0.73099999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.878,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.754,"15min":0.746},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":2.3460000000000001,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.74099999999999999}]},{"osd":3,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.82099999999999995,"15min":0.79200000000000004},"min":{"1min":0.49199999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":3.399,"5min":3.399,"15min":3.399},"last":0.502},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.81999999999999995,"15min":0.80800000000000005},"min":{"1min":0.54700000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":2.2490000000000001,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.66000000000000003}]},{"osd":4,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.73699999999999999,"15min":0.73099999999999998},"min":{"1min":0.443,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":2.4649999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.628,"5min":0.70399999999999996,"15min":0.71699999999999997},"min":{"1min":0.42499999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.96099999999999997,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.83199999999999996,"15min":0.84599999999999997},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.90000000000000002,"5min":1.238,"15min":1.238},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"min":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":2.4929999999999999,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.78500000000000003}]},{"osd":6,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":0.90700000000000003,"15min":0.88200000000000001},"min":{"1min":0.55200000000000005,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":3.331,"5min":3.331,"15min":3.331},"last":0.69599999999999995},{"interface":"front","average":{"1min":1.252,"5min":0.98599999999999999,"15min":0.94199999999999995},"min":{"1min":0.63300000000000001,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":4.7720000000000002,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.72799999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.93700000000000006,"15min":0.93400000000000005},"min":{"1min":0.68300000000000005,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":2.661,"5min":2.661,"15min":2.661},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.95699999999999996,"15min":0.98499999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.034,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.75700000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":1.2889999999999999,"5min":1.0529999999999999,"15min":1.0129999999999999},"min":{"1min":0.63,"5min":0.63,"15min":0.63},"max":{"1min":6.4249999999999998,"5min":6.4249999999999998,"15min":6.4249999999999998},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.0249999999999999,"15min":1.0289999999999999},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":3.2749999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.64200000000000002}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.6339999999999999,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.875,"5min":0.875,"15min":0.875},"min":{"1min":0.624,"5min":0.624,"15min":0.624},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.80300000000000005}]}]},{"osd":9,"up_from":67,"seq":287762808867,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":456461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.55600000000000005,"15min":0.56399999999999995},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.59299999999999997,"15min":0.60399999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.76700000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.54800000000000004}]},{"osd":1,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.752,"15min":0.76600000000000001},"min":{"1min":0.49199999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.81799999999999995,"5min":1.204,"15min":1.204},"last":0.496},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.76400000000000001,"15min":0.77400000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88200000000000001,"5min":1.087,"15min":1.087},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.80200000000000005,"15min":0.81799999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.98299999999999998,"5min":1.383,"15min":1.383},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79700000000000004,"15min":0.79400000000000004},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.73799999999999999}]},{"osd":3,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65500000000000003,"15min":0.67100000000000004},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.998,"15min":0.998},"last":0.65300000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.84399999999999997,"15min":0.86299999999999999},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":0.97999999999999998,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.86499999999999999,"15min":0.88200000000000001},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.075,"5min":1.448,"15min":1.448},"last":0.76300000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.637,"5min":1.637,"15min":1.637},"last":0.751},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.90000000000000002,"15min":0.92000000000000004},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":1.1240000000000001,"5min":1.508,"15min":1.508},"last":0.80100000000000005}]},{"osd":6,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1040000000000001,"5min":1.345,"15min":1.345},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.95799999999999996,"15min":0.97399999999999998},"min":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"max":{"1min":1.2110000000000001,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":1.004,"15min":1.028},"min":{"1min":0.65500000000000003,"5min":0.63700000000000001,"15min":0.63700000000000001},"max":{"1min":1.1910000000000001,"5min":1.5800000000000001,"15min":1.5800000000000001},"last":0.89300000000000002}]},{"osd":8,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.91800000000000004,"5min":0.97199999999999998,"15min":0.98099999999999998},"min":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"max":{"1min":1.7010000000000001,"5min":1.7010000000000001,"15min":1.7010000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.94899999999999995,"5min":1.0049999999999999,"15min":1.014},"min":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"max":{"1min":1.6859999999999999,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":0.91400000000000003}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.0389999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":9.3439999999999994,"5min":9.3439999999999994,"15min":9.3439999999999994},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.014,"15min":1.04},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.877}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96399999999999997}]}]},{"osd":8,"up_from":59,"seq":253403070504,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":455777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.66500000000000004},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.91000000000000003}]},{"osd":1,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.72299999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.874,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.85699999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.56100000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.93400000000000005}]},{"osd":4,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.169,"15min":1.169},"last":0.74199999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0669999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.98599999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":0.79400000000000004},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.88,"15min":0.874},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.1310000000000002,"5min":4.1310000000000002,"15min":4.1310000000000002},"last":0.89000000000000001}]},{"osd":7,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.157,"5min":1.403,"15min":1.403},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90200000000000002,"15min":0.89600000000000002},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":0.77100000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.93100000000000005,"5min":0.86299999999999999,"15min":0.85099999999999998},"min":{"1min":0.68400000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.95799999999999996},{"interface":"front","average":{"1min":0.91600000000000004,"5min":0.82799999999999996,"15min":0.81299999999999994},"min":{"1min":0.66200000000000003,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.238,"5min":1.238,"15min":1.238},"last":0.86699999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":0.94299999999999995,"15min":0.93200000000000005},"min":{"1min":0.69799999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.04},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.89400000000000002,"15min":0.878},"min":{"1min":0.71299999999999997,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":1.0049999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.018}]}]},{"osd":7,"up_from":51,"seq":219043332141,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":1048220,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"min":{"1min":0.39100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.67200000000000004,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.433},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.53000000000000003,"15min":0.53200000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.73199999999999998,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.45800000000000002}]},{"osd":2,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.69699999999999995,"15min":0.69899999999999995},"min":{"1min":0.46999999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.93799999999999994,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.72699999999999998,"15min":0.72199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.0249999999999999,"5min":1.05,"15min":1.05},"last":0.629}]},{"osd":3,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72099999999999997,"15min":0.70299999999999996},"min":{"1min":0.51200000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.98099999999999998,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.69199999999999995,"15min":0.68000000000000005},"min":{"1min":0.48999999999999999,"5min":0.44,"15min":0.44},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.435,"5min":0.379,"15min":0.379},"max":{"1min":0.91300000000000003,"5min":1.016,"15min":1.016},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.60999999999999999,"15min":0.626},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.996,"15min":0.996},"last":0.52000000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.93700000000000006,"15min":1.0229999999999999},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0449999999999999,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.54800000000000004,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0740000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.80100000000000005,"15min":0.80500000000000005},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.008,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.84199999999999997,"15min":0.85999999999999999},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":0.95699999999999996,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.82799999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.234,"15min":1.234},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81699999999999995,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.208,"15min":1.208},"last":0.69299999999999995}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.82099999999999995,"15min":0.81799999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.0109999999999999,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.90000000000000002,"15min":0.89800000000000002},"min":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.86199999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.83999999999999997,"15min":0.83799999999999997},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.087,"5min":1.095,"15min":1.095},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.93700000000000006,"15min":0.93899999999999995},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.83899999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]}]},{"osd":6,"up_from":46,"seq":197568495666,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":455856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.47499999999999998,"15min":0.46600000000000003},"min":{"1min":0.39200000000000002,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.497},{"interface":"front","average":{"1min":0.502,"5min":0.5,"15min":0.49199999999999999},"min":{"1min":0.39400000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.747,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.65500000000000003,"15min":0.64300000000000002},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95199999999999996,"5min":1.087,"15min":1.087},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.70899999999999996,"15min":0.69199999999999995},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":1.1020000000000001,"5min":1.1020000000000001,"15min":1.1020000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.71599999999999997,"15min":0.70599999999999996},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.875},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.70499999999999996},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":1.018,"5min":1.149,"15min":1.149},"last":0.84899999999999998}]},{"osd":3,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.58999999999999997,"15min":0.58099999999999996},"min":{"1min":0.443,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.97899999999999998,"5min":1.04,"15min":1.04},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.67200000000000004,"15min":0.66300000000000003},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.64400000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.76100000000000001,"15min":0.748},"min":{"1min":0.57699999999999996,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.78700000000000003,"15min":0.79500000000000004},"min":{"1min":0.61399999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.0669999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.68600000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.872,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.68300000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.883,"5min":1.883,"15min":1.883},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.89700000000000002,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"max":{"1min":1.8129999999999999,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.94599999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:06:01 2023","interfaces":[{"interface":"back","average":{"1min":0.91600000000000004,"5min":0.86099999999999999,"15min":0.82799999999999996},"min":{"1min":0.63200000000000001,"5min":0.629,"15min":0.629},"max":{"1min":1.9570000000000001,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":1.0249999999999999,"5min":0.89700000000000002,"15min":0.85299999999999998},"min":{"1min":0.63100000000000001,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":3.5680000000000001,"5min":3.5680000000000001,"15min":3.5680000000000001},"last":0.80000000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:24 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.879,"15min":0.86399999999999999},"min":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":2.0070000000000001,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.90800000000000003,"15min":0.90000000000000002},"min":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"max":{"1min":1.944,"5min":1.944,"15min":1.944},"last":0.90300000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85199999999999998,"5min":0.93300000000000005,"15min":0.94699999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.22,"5min":1.984,"15min":1.984},"last":0.754},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.93200000000000005,"15min":0.94899999999999995},"min":{"1min":0.58799999999999997,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":1.1870000000000001,"5min":1.754,"15min":1.754},"last":0.81699999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.98099999999999998,"5min":0.94899999999999995,"15min":0.94299999999999995},"min":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"max":{"1min":3.9969999999999999,"5min":3.9969999999999999,"15min":3.9969999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.92200000000000004,"5min":1.0229999999999999,"15min":1.04},"min":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"max":{"1min":1.3049999999999999,"5min":1.923,"15min":1.923},"last":0.91700000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574926,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27940,"kb_used_data":988,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717212,"statfs":{"total":95995035648,"available":95966425088,"internally_reserved":0,"allocated":1011712,"data_stored":514308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.98099999999999998,"5min":1.073,"15min":1.073},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.76200000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.40999999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.79500000000000004,"5min":0.93999999999999995,"15min":0.93999999999999995},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.72999999999999998,"5min":0.66200000000000003,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":1.054,"15min":1.054},"last":0.73999999999999999}]},{"osd":3,"last update":"Fri Jul 21 19:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.69499999999999995,"15min":0.66100000000000003},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.0509999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66000000000000003,"15min":0.66200000000000003},"min":{"1min":0.39600000000000002,"5min":0.379,"15min":0.379},"max":{"1min":0.93500000000000005,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.81000000000000005}]},{"osd":4,"last update":"Fri Jul 21 19:05:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.52400000000000002,"15min":0.51100000000000001},"min":{"1min":0.34499999999999997,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.54600000000000004,"15min":0.505},"min":{"1min":0.39700000000000002,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.68600000000000005,"15min":0.64800000000000002},"min":{"1min":0.56100000000000005,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.72899999999999998,"15min":0.69199999999999995},"min":{"1min":0.59099999999999997,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.98699999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:05:32 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.75800000000000001,"15min":0.74199999999999999},"min":{"1min":0.57099999999999995,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.032,"5min":1.048,"15min":1.048},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.72199999999999998,"15min":0.70799999999999996},"min":{"1min":0.51700000000000002,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.629}]},{"osd":7,"last update":"Fri Jul 21 19:05:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.72999999999999998,"15min":0.71999999999999997},"min":{"1min":0.44600000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0840000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.70299999999999996,"15min":0.68500000000000005},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.97699999999999998,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.83199999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.61499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.0269999999999999,"5min":1.502,"15min":1.502},"last":0.89900000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.871,"5min":0.88500000000000001,"15min":0.88700000000000001},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.1359999999999999,"5min":1.149,"15min":1.149},"last":1.0349999999999999},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.90300000000000002,"15min":0.90400000000000003},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.1459999999999999,"5min":1.147,"15min":1.147},"last":1.204}]},{"osd":10,"last update":"Fri Jul 21 19:06:18 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.93100000000000005,"15min":0.93100000000000005},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94099999999999995,"5min":0.95799999999999996,"15min":0.96099999999999997},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.1479999999999999,"5min":1.1839999999999999,"15min":1.1839999999999999},"last":1.169}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96599999999999997}]}]},{"osd":0,"up_from":8,"seq":34359738451,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":457121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68300000000000005,"15min":0.63500000000000001},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1539999999999999,"5min":1.177,"15min":1.177},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.39300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.70599999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.64000000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.91200000000000003,"5min":1.139,"15min":1.139},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.66300000000000003,"15min":0.66200000000000003},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78900000000000003,"5min":1.113,"15min":1.113},"last":0.66400000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:04:10 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55000000000000004,"15min":0.53100000000000003},"min":{"1min":0.34200000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.83099999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64500000000000002,"15min":0.625},"min":{"1min":0.38700000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.65600000000000003,"15min":0.64000000000000001},"min":{"1min":0.438,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.84399999999999997,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.61799999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73899999999999999,"15min":0.72799999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.278,"5min":1.278,"15min":1.278},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70699999999999996,"15min":0.69499999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":1.4610000000000001,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.78200000000000003}]},{"osd":6,"last update":"Fri Jul 21 19:05:33 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64100000000000001,"15min":0.64200000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76200000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.69199999999999995,"15min":0.69799999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.93999999999999995,"5min":1.163,"15min":1.163},"last":0.57099999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.91500000000000004},"min":{"1min":0.68799999999999994,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.244,"5min":1.304,"15min":1.304},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82399999999999995,"15min":0.84599999999999997},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.71999999999999997}]},{"osd":9,"last update":"Fri Jul 21 19:05:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.77400000000000002,"15min":0.79400000000000004},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.88900000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.91000000000000003,"15min":0.94199999999999995},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.0229999999999999,"5min":1.274,"15min":1.274},"last":0.86099999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.83699999999999997,"15min":0.84599999999999997},"min":{"1min":0.49099999999999999,"5min":0.435,"15min":0.435},"max":{"1min":1.429,"5min":1.429,"15min":1.429},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.84899999999999998,"15min":0.85499999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.034,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.95999999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":0.97599999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"min":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.93799999999999994}]}]},{"osd":2,"up_from":18,"seq":77309411400,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":514454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.60399999999999998,"15min":0.59399999999999997},"min":{"1min":0.374,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.60499999999999998,"15min":0.58099999999999996},"min":{"1min":0.39800000000000002,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89700000000000002}]},{"osd":1,"last update":"Fri Jul 21 19:05:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.65700000000000003,"15min":0.60699999999999998},"min":{"1min":0.41399999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.99399999999999999,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.64700000000000002,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.96699999999999997,"5min":1.081,"15min":1.081},"last":0.78600000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:06:12 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.69699999999999995,"15min":0.66800000000000004},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.91400000000000003,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.69099999999999995,"15min":0.65700000000000003},"min":{"1min":0.47199999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.92900000000000005,"5min":1.1890000000000001,"15min":1.1890000000000001},"last":0.65100000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:04:39 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69399999999999995,"15min":0.68100000000000005},"min":{"1min":0.52300000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.72199999999999998,"15min":0.70199999999999996},"min":{"1min":0.64300000000000002,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.83799999999999997}]},{"osd":5,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.57099999999999995,"15min":0.52700000000000002},"min":{"1min":0.39100000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.58799999999999997,"15min":0.53800000000000003},"min":{"1min":0.379,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.94599999999999995,"5min":1.167,"15min":1.167},"last":0.45600000000000002}]},{"osd":6,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.70399999999999996}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.94399999999999995}]},{"osd":9,"last update":"Fri Jul 21 19:05:48 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.85499999999999998,"15min":0.85399999999999998},"min":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"max":{"1min":1.153,"5min":1.333,"15min":1.333},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.91200000000000003,"5min":0.88,"15min":0.875},"min":{"1min":0.71999999999999997,"5min":0.67500000000000004,"15min":0.67500000000000004},"max":{"1min":1.089,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.749}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.80900000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98399999999999999}]}]},{"osd":3,"up_from":25,"seq":107374182467,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":456173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.45800000000000002,"15min":0.41799999999999998},"min":{"1min":0.34799999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.90900000000000003,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47599999999999998,"15min":0.44400000000000001},"min":{"1min":0.372,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.76400000000000001,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.46700000000000003}]},{"osd":1,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.68500000000000005,"15min":0.66100000000000003},"min":{"1min":0.45800000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.99099999999999999,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67000000000000004,"15min":0.65100000000000002},"min":{"1min":0.53700000000000003,"5min":0.437,"15min":0.437},"max":{"1min":0.88800000000000001,"5min":1.048,"15min":1.048},"last":0.44900000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:06:13 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.64100000000000001},"min":{"1min":0.48199999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.052,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.745,"5min":0.69499999999999995,"15min":0.63700000000000001},"min":{"1min":0.56499999999999995,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.026,"5min":1.048,"15min":1.048},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:05:42 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.69599999999999995,"15min":0.65800000000000003},"min":{"1min":0.495,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.88800000000000001,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.67400000000000004},"min":{"1min":0.51900000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.998,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.56200000000000006}]},{"osd":5,"last update":"Fri Jul 21 19:06:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":0.68000000000000005},"min":{"1min":0.497,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.006,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73099999999999998,"15min":0.68999999999999995},"min":{"1min":0.57299999999999995,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":0.96999999999999997,"5min":1.175,"15min":1.175},"last":0.71599999999999997}]},{"osd":6,"last update":"Fri Jul 21 19:05:35 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.69299999999999995,"15min":0.66300000000000003},"min":{"1min":0.41899999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.044,"5min":1.264,"15min":1.264},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.375,"15min":0.375},"max":{"1min":0.871,"5min":1.141,"15min":1.141},"last":0.36599999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:06:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.78100000000000003,"15min":0.78300000000000003},"min":{"1min":0.61699999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.92100000000000004,"5min":1.167,"15min":1.167},"last":0.624},{"interface":"front","average":{"1min":0.86199999999999999,"5min":0.84599999999999997,"15min":0.82399999999999995},"min":{"1min":0.70999999999999996,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.034,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.75800000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.85499999999999998,"5min":0.93700000000000006,"15min":0.95099999999999996},"min":{"1min":0.50900000000000001,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.133,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.89300000000000002,"15min":0.91300000000000003},"min":{"1min":0.60199999999999998,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.026,"5min":1.196,"15min":1.196},"last":0.54100000000000004}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.091,"5min":1.153,"15min":1.153},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.73599999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":4,"up_from":33,"seq":141733920829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":457203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.47299999999999998,"15min":0.439},"min":{"1min":0.34000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.441},"min":{"1min":0.31900000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.66000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63700000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.71099999999999997,"15min":0.66300000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":1.167,"15min":1.167},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.69599999999999995,"15min":0.67800000000000005},"min":{"1min":0.45100000000000001,"5min":0.436,"15min":0.436},"max":{"1min":0.96199999999999997,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":1.0009999999999999}]},{"osd":3,"last update":"Fri Jul 21 19:05:43 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.68400000000000005,"15min":0.64100000000000001},"min":{"1min":0.499,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.97799999999999998,"5min":1.083,"15min":1.083},"last":1.03},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.66000000000000003,"15min":0.63500000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.91000000000000003,"5min":1.079,"15min":1.079},"last":0.82799999999999996}]},{"osd":5,"last update":"Fri Jul 21 19:06:05 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.75700000000000001,"15min":0.747},"min":{"1min":0.60999999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.95599999999999996,"5min":1.044,"15min":1.044},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.76000000000000001,"15min":0.73699999999999999},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.93600000000000005,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.97799999999999998}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78300000000000003,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.501,"15min":0.501},"max":{"1min":1.006,"5min":1.119,"15min":1.119},"last":1.091},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.78900000000000003,"15min":0.77900000000000003},"min":{"1min":0.58099999999999996,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.018,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.94199999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:05:57 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.70599999999999996,"15min":0.66900000000000004},"min":{"1min":0.44400000000000001,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.105,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.749},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73199999999999998,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.0309999999999999,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.79500000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:05:28 2023","interfaces":[{"interface":"back","average":{"1min":0.91900000000000004,"5min":0.88400000000000001,"15min":0.878},"min":{"1min":0.69699999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0920000000000001,"5min":1.155,"15min":1.155},"last":1.1599999999999999},{"interface":"front","average":{"1min":0.94799999999999995,"5min":0.89600000000000002,"15min":0.88700000000000001},"min":{"1min":0.70599999999999996,"5min":0.69099999999999995,"15min":0.69099999999999995},"max":{"1min":1.1220000000000001,"5min":1.298,"15min":1.298},"last":1.173}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.82299999999999995,"15min":0.82899999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.0669999999999999,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":1.1080000000000001},{"interface":"front","average":{"1min":0.89000000000000001,"5min":0.94099999999999995,"15min":0.94899999999999995},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1279999999999999,"5min":1.244,"15min":1.244},"last":1.1259999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":2.6499999999999999,"15min":2.9460000000000002},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.1559999999999999,"5min":42.396000000000001,"15min":42.396000000000001},"last":1.145},{"interface":"front","average":{"1min":0.72599999999999998,"5min":2.5619999999999998,"15min":2.8679999999999999},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.86699999999999999,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.91800000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1879999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757304,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1047952,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.63300000000000001,"15min":0.61299999999999999},"min":{"1min":0.42399999999999999,"5min":0.379,"15min":0.379},"max":{"1min":0.84799999999999998,"5min":1.042,"15min":1.042},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.627,"15min":0.59499999999999997},"min":{"1min":0.41399999999999998,"5min":0.38,"15min":0.38},"max":{"1min":0.88,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.55200000000000005}]},{"osd":1,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.68300000000000005,"15min":0.68700000000000006},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.86699999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.70099999999999996,"15min":0.68400000000000005},"min":{"1min":0.41999999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.0189999999999999,"5min":1.028,"15min":1.028},"last":0.71299999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.52200000000000002,"15min":0.499},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.91100000000000003,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.33600000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.73999999999999999,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.76700000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.71399999999999997,"15min":0.69699999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.84699999999999998,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.65000000000000002,"15min":0.66600000000000004},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":0.88700000000000001,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.59999999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:06:10 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73699999999999999,"15min":0.72099999999999997},"min":{"1min":0.50600000000000001,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92700000000000005,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.72799999999999998,"15min":0.73399999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.98999999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.69699999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:05:30 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.76700000000000002,"15min":0.752},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.62},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.752,"15min":0.751},"min":{"1min":0.55300000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.88800000000000001,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.74299999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:06:00 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.80100000000000005,"15min":0.79900000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.53000000000000003,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.83399999999999996}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.505,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1080000000000001,"5min":1.111,"15min":1.111},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.091,"5min":1.103,"15min":1.103},"last":0.86499999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:05:50 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.877,"15min":0.88300000000000001},"min":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0309999999999999,"5min":1.05,"15min":1.05},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.82299999999999995},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.97799999999999998,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.81599999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:16 2023","interfaces":[{"interface":"back","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.69799999999999995,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.141,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.878,"15min":0.86699999999999999},"min":{"1min":0.71699999999999997,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.92000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":58665,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":58811,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":59613,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:06:33.456 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-07-21T19:06:33.456 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-07-21T19:06:33.456 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-07-21T19:06:33.457 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph health --format=json 2023-07-21T19:06:34.190 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:34.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:34 smithi156 ceph-mon[128316]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:34 smithi156 ceph-mon[128316]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:34.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:34 smithi005 ceph-mon[126460]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:34.749 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:34 smithi005 ceph-mon[126460]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:34.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:34 smithi101 ceph-mon[128136]: from='client.15036 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:06:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:34 smithi101 ceph-mon[128136]: pgmap v330: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:35.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.677+0000 7fafdd824700 1 -- 172.21.15.5:0/4022091408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd81007a0 msgr2=0x7fafd8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:35.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.677+0000 7fafdd824700 1 --2- 172.21.15.5:0/4022091408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd81007a0 0x7fafd8104770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fafcc0099c0 tx=0x7fafcc009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 -- 172.21.15.5:0/4022091408 shutdown_connections 2023-07-21T19:06:35.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 --2- 172.21.15.5:0/4022091408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd8104ea0 0x7fafd810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 --2- 172.21.15.5:0/4022091408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 0x7fafd81001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 --2- 172.21.15.5:0/4022091408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd81007a0 0x7fafd8104770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 -- 172.21.15.5:0/4022091408 >> 172.21.15.5:0/4022091408 conn(0x7fafd80fb660 msgr2=0x7fafd80fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:35.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 -- 172.21.15.5:0/4022091408 shutdown_connections 2023-07-21T19:06:35.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.678+0000 7fafdd824700 1 -- 172.21.15.5:0/4022091408 wait complete. 2023-07-21T19:06:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 Processor -- start 2023-07-21T19:06:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 -- start start 2023-07-21T19:06:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 0x7fafd8197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 0x7fafd81983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:35.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819ce40 con 0x7fafd8104ea0 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819cfb0 con 0x7fafd80ffe00 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.679+0000 7fafdd824700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafd819d170 con 0x7fafd81007a0 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42980/0 (socket says 172.21.15.5:42980) 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 -- 172.21.15.5:0/1944836390 learned_addr learned my addr 172.21.15.5:0/1944836390 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:35.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd6ffd700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 0x7fafd8197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd67fc700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 0x7fafd81983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 -- 172.21.15.5:0/1944836390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 msgr2=0x7fafd81983c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 0x7fafd81983c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 -- 172.21.15.5:0/1944836390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 msgr2=0x7fafd8197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 0x7fafd8197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd77fe700 1 -- 172.21.15.5:0/1944836390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0095f0 con 0x7fafd8104ea0 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.680+0000 7fafd67fc700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 0x7fafd81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:35.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafd77fe700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007ee0 tx=0x7fafd000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd000ddd0 con 0x7fafd8104ea0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafd000f040 con 0x7fafd8104ea0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd00047d0 con 0x7fafd8104ea0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafd819d450 con 0x7fafd8104ea0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.681+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafd81a4cb0 con 0x7fafd8104ea0 2023-07-21T19:06:35.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.682+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafd804e210 con 0x7fafd8104ea0 2023-07-21T19:06:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.684+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fafd0003f20 con 0x7fafd8104ea0 2023-07-21T19:06:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.686+0000 7fafdc822700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 0x7fafc00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:35.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.687+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafd00ccb50 con 0x7fafd8104ea0 2023-07-21T19:06:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.687+0000 7fafd6ffd700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 0x7fafc00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:35.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.687+0000 7fafd6ffd700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 0x7fafc00a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fafc8005950 tx=0x7fafc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:35.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.688+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fafd00960f0 con 0x7fafd8104ea0 2023-07-21T19:06:35.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.940+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fafd81991e0 con 0x7fafd8104ea0 2023-07-21T19:06:35.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.940+0000 7fafdc822700 1 -- 172.21.15.5:0/1944836390 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fafd0023080 con 0x7fafd8104ea0 2023-07-21T19:06:35.950 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:35.950 INFO:teuthology.orchestra.run.smithi005.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-07-21T19:06:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 msgr2=0x7fafc00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 0x7fafc00a3710 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fafc8005950 tx=0x7fafc800b410 comp rx=0 tx=0).stop 2023-07-21T19:06:35.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 msgr2=0x7fafd819c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fafd0007ee0 tx=0x7fafd000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 shutdown_connections 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafc00a1260 0x7fafc00a3710 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafd8104ea0 0x7fafd819c830 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafd80ffe00 0x7fafd8197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 --2- 172.21.15.5:0/1944836390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafd81007a0 0x7fafd81983c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:35.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.942+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 >> 172.21.15.5:0/1944836390 conn(0x7fafd80fb660 msgr2=0x7fafd80fc390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.943+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 shutdown_connections 2023-07-21T19:06:35.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:35.943+0000 7fafdd824700 1 -- 172.21.15.5:0/1944836390 wait complete. 2023-07-21T19:06:36.474 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-07-21T19:06:36.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:06:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:36 smithi156 ceph-mon[128316]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:36.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:36 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1944836390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T19:06:36.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 -- 172.21.15.5:0/1237905640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a8101680 msgr2=0x7fc1a8107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:36.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1237905640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a8101680 0x7fc1a8107f40 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc1a000b0a0 tx=0x7fc1a000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 -- 172.21.15.5:0/1237905640 shutdown_connections 2023-07-21T19:06:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1237905640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a8101680 0x7fc1a8107f40 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1237905640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a80ffb00 0x7fc1a80fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1237905640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1a8100cf0 0x7fc1a8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 -- 172.21.15.5:0/1237905640 >> 172.21.15.5:0/1237905640 conn(0x7fc1a80fb0e0 msgr2=0x7fc1a80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 -- 172.21.15.5:0/1237905640 shutdown_connections 2023-07-21T19:06:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.674+0000 7fc1adbff700 1 -- 172.21.15.5:0/1237905640 wait complete. 2023-07-21T19:06:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 Processor -- start 2023-07-21T19:06:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 -- start start 2023-07-21T19:06:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1a8100cf0 0x7fc1a8195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a8101680 0x7fc1a819ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a819d440 con 0x7fc1a80ffb00 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a819d580 con 0x7fc1a8100cf0 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1adbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1a819d710 con 0x7fc1a8101680 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.676+0000 7fc1a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49374/0 (socket says 172.21.15.5:49374) 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 -- 172.21.15.5:0/1390689524 learned_addr learned my addr 172.21.15.5:0/1390689524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a7fff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a8101680 0x7fc1a819ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 -- 172.21.15.5:0/1390689524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a8101680 msgr2=0x7fc1a819ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a8101680 0x7fc1a819ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 -- 172.21.15.5:0/1390689524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1a8100cf0 msgr2=0x7fc1a8195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1a8100cf0 0x7fc1a8195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 -- 172.21.15.5:0/1390689524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1a0009d00 con 0x7fc1a80ffb00 2023-07-21T19:06:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a77fe700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e970 tx=0x7fc19800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198009790 con 0x7fc1a80ffb00 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1a819d9f0 con 0x7fc1a80ffb00 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1980044b0 con 0x7fc1a80ffb00 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc198003dd0 con 0x7fc1a80ffb00 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.677+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1a819df00 con 0x7fc1a80ffb00 2023-07-21T19:06:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.679+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1a8107040 con 0x7fc1a80ffb00 2023-07-21T19:06:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.680+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc198003f30 con 0x7fc1a80ffb00 2023-07-21T19:06:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.682+0000 7fc1a57fa700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 0x7fc1940a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.682+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc198014070 con 0x7fc1a80ffb00 2023-07-21T19:06:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.682+0000 7fc19ffff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 0x7fc1940a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.683+0000 7fc19ffff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 0x7fc1940a36d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc190009b60 tx=0x7fc190009620 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:36.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.686+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc1980963f0 con 0x7fc1a80ffb00 2023-07-21T19:06:36.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:36 smithi101 ceph-mon[128136]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:36.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:36 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1944836390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T19:06:36.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.894+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc1a802c850 con 0x7fc1a80ffb00 2023-07-21T19:06:36.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.896+0000 7fc1a57fa700 1 -- 172.21.15.5:0/1390689524 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7fc198099a10 con 0x7fc1a80ffb00 2023-07-21T19:06:36.906 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:36.906 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:06:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 msgr2=0x7fc1940a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 0x7fc1940a36d0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc190009b60 tx=0x7fc190009620 comp rx=0 tx=0).stop 2023-07-21T19:06:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 msgr2=0x7fc1a81958c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc19800e970 tx=0x7fc19800ed30 comp rx=0 tx=0).stop 2023-07-21T19:06:36.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 shutdown_connections 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc1940a1220 0x7fc1940a36d0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1a8101680 0x7fc1a819ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1a80ffb00 0x7fc1a81958c0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 --2- 172.21.15.5:0/1390689524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1a8100cf0 0x7fc1a8195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 >> 172.21.15.5:0/1390689524 conn(0x7fc1a80fb0e0 msgr2=0x7fc1a8104180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.898+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 shutdown_connections 2023-07-21T19:06:36.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:36.899+0000 7fc1adbff700 1 -- 172.21.15.5:0/1390689524 wait complete. 2023-07-21T19:06:36.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:36 smithi005 ceph-mon[126460]: pgmap v331: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:36.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:36 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1944836390' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-07-21T19:06:36.927 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:06:36.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:06:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.125+0000 7f20d8dff700 1 -- 172.21.15.5:0/301724033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 msgr2=0x7f20d40fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.125+0000 7f20d8dff700 1 --2- 172.21.15.5:0/301724033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d40fff00 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009990 tx=0x7f20c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 -- 172.21.15.5:0/301724033 shutdown_connections 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 --2- 172.21.15.5:0/301724033 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4101670 0x7f20d4107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 --2- 172.21.15.5:0/301724033 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d40fff00 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 --2- 172.21.15.5:0/301724033 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4100ce0 0x7f20d4101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 -- 172.21.15.5:0/301724033 >> 172.21.15.5:0/301724033 conn(0x7f20d40fb130 msgr2=0x7f20d40fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 -- 172.21.15.5:0/301724033 shutdown_connections 2023-07-21T19:06:37.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.126+0000 7f20d8dff700 1 -- 172.21.15.5:0/301724033 wait complete. 2023-07-21T19:06:37.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 Processor -- start 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 -- start start 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4100ce0 0x7f20d4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4101670 0x7f20d419efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419f5b0 con 0x7f20d40ffaf0 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419f6f0 con 0x7f20d4101670 2023-07-21T19:06:37.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d8dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419f880 con 0x7f20d4100ce0 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.127+0000 7f20d259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49386/0 (socket says 172.21.15.5:49386) 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 -- 172.21.15.5:0/3511818469 learned_addr learned my addr 172.21.15.5:0/3511818469 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d2d9d700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4101670 0x7f20d419efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d1d9b700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4100ce0 0x7f20d4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 -- 172.21.15.5:0/3511818469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4100ce0 msgr2=0x7f20d4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4100ce0 0x7f20d4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 -- 172.21.15.5:0/3511818469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4101670 msgr2=0x7f20d419efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4101670 0x7f20d419efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 -- 172.21.15.5:0/3511818469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40095f0 con 0x7f20d40ffaf0 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d259c700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005910 tx=0x7f20c4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c401d070 con 0x7f20d40ffaf0 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d419fb00 con 0x7f20d40ffaf0 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20c400bdd0 con 0x7f20d40ffaf0 2023-07-21T19:06:37.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.128+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4022d60 con 0x7f20d40ffaf0 2023-07-21T19:06:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.129+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d419ff30 con 0x7f20d40ffaf0 2023-07-21T19:06:37.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.130+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d410b1c0 con 0x7f20d40ffaf0 2023-07-21T19:06:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.131+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f20c4022500 con 0x7f20d40ffaf0 2023-07-21T19:06:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.133+0000 7f20c37fe700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 0x7f20bc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.133+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f20c40cdf70 con 0x7f20d40ffaf0 2023-07-21T19:06:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.133+0000 7f20d1d9b700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 0x7f20bc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.134+0000 7f20d1d9b700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 0x7f20bc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009bf0 tx=0x7f20c8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.136+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f20c4097600 con 0x7f20d40ffaf0 2023-07-21T19:06:37.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.344+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20d41a04e0 con 0x7f20d40ffaf0 2023-07-21T19:06:37.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.344+0000 7f20c37fe700 1 -- 172.21.15.5:0/3511818469 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20c409ac20 con 0x7f20d40ffaf0 2023-07-21T19:06:37.354 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:06:37.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.346+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 msgr2=0x7f20bc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.346+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 0x7f20bc0a3690 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f20c8009bf0 tx=0x7f20c8009260 comp rx=0 tx=0).stop 2023-07-21T19:06:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.346+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 msgr2=0x7f20d4197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.346+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005910 tx=0x7f20c4005740 comp rx=0 tx=0).stop 2023-07-21T19:06:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 shutdown_connections 2023-07-21T19:06:37.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f20bc0a11e0 0x7f20bc0a3690 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d4101670 0x7f20d419efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40ffaf0 0x7f20d4197a30 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 --2- 172.21.15.5:0/3511818469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d4100ce0 0x7f20d4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 >> 172.21.15.5:0/3511818469 conn(0x7f20d40fb130 msgr2=0x7f20d40fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 shutdown_connections 2023-07-21T19:06:37.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.347+0000 7f20d8dff700 1 -- 172.21.15.5:0/3511818469 wait complete. 2023-07-21T19:06:37.376 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:06:37.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 -- 172.21.15.5:0/3027433731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe0072f70 msgr2=0x7fafe00733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3027433731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe0072f70 0x7fafe00733e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fafcc009990 tx=0x7fafcc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 -- 172.21.15.5:0/3027433731 shutdown_connections 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3027433731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe01079a0 0x7fafe0109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3027433731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0074550 0x7fafe00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3027433731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe0072f70 0x7fafe00733e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 -- 172.21.15.5:0/3027433731 >> 172.21.15.5:0/3027433731 conn(0x7fafe00fb0e0 msgr2=0x7fafe00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 -- 172.21.15.5:0/3027433731 shutdown_connections 2023-07-21T19:06:37.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.573+0000 7fafe6fff700 1 -- 172.21.15.5:0/3027433731 wait complete. 2023-07-21T19:06:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.574+0000 7fafe6fff700 1 Processor -- start 2023-07-21T19:06:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.574+0000 7fafe6fff700 1 -- start start 2023-07-21T19:06:37.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.574+0000 7fafe6fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0072f70 0x7fafe0197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.574+0000 7fafe6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe0074550 0x7fafe0197fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe6fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe6fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019f600 con 0x7fafe01079a0 2023-07-21T19:06:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe6fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019f740 con 0x7fafe0072f70 2023-07-21T19:06:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafe019f8d0 con 0x7fafe0074550 2023-07-21T19:06:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49406/0 (socket says 172.21.15.5:49406) 2023-07-21T19:06:37.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 -- 172.21.15.5:0/3536221379 learned_addr learned my addr 172.21.15.5:0/3536221379 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe4d9b700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0072f70 0x7fafe0197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafdffff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe0074550 0x7fafe0197fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 -- 172.21.15.5:0/3536221379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe0074550 msgr2=0x7fafe0197fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe0074550 0x7fafe0197fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 -- 172.21.15.5:0/3536221379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0072f70 msgr2=0x7fafe0197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0072f70 0x7fafe0197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 -- 172.21.15.5:0/3536221379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafcc0095f0 con 0x7fafe01079a0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.575+0000 7fafe559c700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007ee0 tx=0x7fafd800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.576+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8017070 con 0x7fafe01079a0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.576+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fafd800f040 con 0x7fafe01079a0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.576+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafe019fbb0 con 0x7fafe01079a0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.576+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd8005570 con 0x7fafe01079a0 2023-07-21T19:06:37.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.576+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafe01a00c0 con 0x7fafe01079a0 2023-07-21T19:06:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.577+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafe0106050 con 0x7fafe01079a0 2023-07-21T19:06:37.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.579+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fafd800db20 con 0x7fafe01079a0 2023-07-21T19:06:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.580+0000 7fafddffb700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 0x7fafd00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.580+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fafd80cc180 con 0x7fafe01079a0 2023-07-21T19:06:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.580+0000 7fafe4d9b700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 0x7fafd00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.581+0000 7fafe4d9b700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 0x7fafd00a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fafd4005950 tx=0x7fafd400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:37.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.584+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fafd8095720 con 0x7fafe01079a0 2023-07-21T19:06:37.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:37 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1390689524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:37.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:37 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3511818469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:37.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:37 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1390689524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:37.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:37 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3511818469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.793+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fafe002caa0 con 0x7fafe01079a0 2023-07-21T19:06:37.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.793+0000 7fafddffb700 1 -- 172.21.15.5:0/3536221379 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fafd8098d40 con 0x7fafe01079a0 2023-07-21T19:06:37.803 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:06:37.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 msgr2=0x7fafd00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 0x7fafd00a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fafd4005950 tx=0x7fafd400b410 comp rx=0 tx=0).stop 2023-07-21T19:06:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 msgr2=0x7fafe019f030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fafd8007ee0 tx=0x7fafd800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 shutdown_connections 2023-07-21T19:06:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fafd00a11e0 0x7fafd00a3690 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fafe01079a0 0x7fafe019f030 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fafe0072f70 0x7fafe0197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 --2- 172.21.15.5:0/3536221379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fafe0074550 0x7fafe0197fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:37.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 >> 172.21.15.5:0/3536221379 conn(0x7fafe00fb0e0 msgr2=0x7fafe00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 shutdown_connections 2023-07-21T19:06:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:37.796+0000 7fafe6fff700 1 -- 172.21.15.5:0/3536221379 wait complete. 2023-07-21T19:06:37.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:06:37.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:37 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1390689524' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:37.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:37 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3511818469' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 -- 172.21.15.5:0/3635543286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30100cd0 msgr2=0x7fca30101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 --2- 172.21.15.5:0/3635543286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30100cd0 0x7fca30101120 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fca24009990 tx=0x7fca24009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 -- 172.21.15.5:0/3635543286 shutdown_connections 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 --2- 172.21.15.5:0/3635543286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30101660 0x7fca30107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 --2- 172.21.15.5:0/3635543286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca300ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 --2- 172.21.15.5:0/3635543286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30100cd0 0x7fca30101120 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 -- 172.21.15.5:0/3635543286 >> 172.21.15.5:0/3635543286 conn(0x7fca300fb0e0 msgr2=0x7fca300fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 -- 172.21.15.5:0/3635543286 shutdown_connections 2023-07-21T19:06:38.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.023+0000 7fca34fff700 1 -- 172.21.15.5:0/3635543286 wait complete. 2023-07-21T19:06:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.024+0000 7fca34fff700 1 Processor -- start 2023-07-21T19:06:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.024+0000 7fca34fff700 1 -- start start 2023-07-21T19:06:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca30195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30100cd0 0x7fca30195c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019d2d0 con 0x7fca30101660 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019d410 con 0x7fca300ffae0 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca34fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca3019d5a0 con 0x7fca30100cd0 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca2ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca2ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49424/0 (socket says 172.21.15.5:49424) 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca2ed9d700 1 -- 172.21.15.5:0/1248488960 learned_addr learned my addr 172.21.15.5:0/1248488960 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:38.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca30195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.025+0000 7fca2dd9b700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30100cd0 0x7fca30195c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 -- 172.21.15.5:0/1248488960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30100cd0 msgr2=0x7fca30195c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30100cd0 0x7fca30195c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 -- 172.21.15.5:0/1248488960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 msgr2=0x7fca30195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca30195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 -- 172.21.15.5:0/1248488960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240095f0 con 0x7fca30101660 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca30195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:38.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca2ed9d700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca28007d70 tx=0x7fca2800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca2800ad40 con 0x7fca30101660 2023-07-21T19:06:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca3019d880 con 0x7fca30101660 2023-07-21T19:06:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fca280128b0 con 0x7fca30101660 2023-07-21T19:06:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca280116b0 con 0x7fca30101660 2023-07-21T19:06:38.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.026+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca3019dde0 con 0x7fca30101660 2023-07-21T19:06:38.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.027+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca301070c0 con 0x7fca30101660 2023-07-21T19:06:38.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.029+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fca28012a20 con 0x7fca30101660 2023-07-21T19:06:38.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.031+0000 7fca1f7fe700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 0x7fca180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.032+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fca280ccd40 con 0x7fca30101660 2023-07-21T19:06:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.032+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 0x7fca180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.032+0000 7fca2e59c700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 0x7fca180a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca30100b30 tx=0x7fca20009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.036+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fca280963f0 con 0x7fca30101660 2023-07-21T19:06:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.243+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fca3002caa0 con 0x7fca30101660 2023-07-21T19:06:38.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.244+0000 7fca1f7fe700 1 -- 172.21.15.5:0/1248488960 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fca28099a10 con 0x7fca30101660 2023-07-21T19:06:38.254 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:06:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 msgr2=0x7fca180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 0x7fca180a3690 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fca30100b30 tx=0x7fca20009260 comp rx=0 tx=0).stop 2023-07-21T19:06:38.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 msgr2=0x7fca3019cd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fca28007d70 tx=0x7fca2800a340 comp rx=0 tx=0).stop 2023-07-21T19:06:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 shutdown_connections 2023-07-21T19:06:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fca180a11e0 0x7fca180a3690 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fca30101660 0x7fca3019cd00 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fca300ffae0 0x7fca30195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 --2- 172.21.15.5:0/1248488960 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fca30100cd0 0x7fca30195c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 >> 172.21.15.5:0/1248488960 conn(0x7fca300fb0e0 msgr2=0x7fca30104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.246+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 shutdown_connections 2023-07-21T19:06:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.247+0000 7fca34fff700 1 -- 172.21.15.5:0/1248488960 wait complete. 2023-07-21T19:06:38.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:06:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 -- 172.21.15.5:0/3194659688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c100cf0 msgr2=0x7ff63c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 --2- 172.21.15.5:0/3194659688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c100cf0 0x7ff63c101140 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7ff630009990 tx=0x7ff630009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 -- 172.21.15.5:0/3194659688 shutdown_connections 2023-07-21T19:06:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 --2- 172.21.15.5:0/3194659688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 --2- 172.21.15.5:0/3194659688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 --2- 172.21.15.5:0/3194659688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c100cf0 0x7ff63c101140 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 -- 172.21.15.5:0/3194659688 >> 172.21.15.5:0/3194659688 conn(0x7ff63c0fb0e0 msgr2=0x7ff63c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 -- 172.21.15.5:0/3194659688 shutdown_connections 2023-07-21T19:06:38.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.472+0000 7ff6409ff700 1 -- 172.21.15.5:0/3194659688 wait complete. 2023-07-21T19:06:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.473+0000 7ff6409ff700 1 Processor -- start 2023-07-21T19:06:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.473+0000 7ff6409ff700 1 -- start start 2023-07-21T19:06:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 0x7ff63c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c0735e0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c073750 con 0x7ff63c100cf0 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff6409ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63c19f7e0 con 0x7ff63c101680 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49450/0 (socket says 172.21.15.5:49450) 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 -- 172.21.15.5:0/689540675 learned_addr learned my addr 172.21.15.5:0/689540675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:38.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff639d9b700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 0x7ff63c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63ad9d700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 -- 172.21.15.5:0/689540675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 msgr2=0x7ff63c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 -- 172.21.15.5:0/689540675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 msgr2=0x7ff63c072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 0x7ff63c072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63a59c700 1 -- 172.21.15.5:0/689540675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6300095f0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff63ad9d700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:38.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.474+0000 7ff639d9b700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 0x7ff63c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff63a59c700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d980 tx=0x7ff62c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00bb60 con 0x7ff63c0ffb00 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff62c007620 con 0x7ff63c0ffb00 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63c19fac0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff62c00f3c0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.475+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63c19ffd0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.476+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63c10b1d0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.478+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff62c010460 con 0x7ff63c0ffb00 2023-07-21T19:06:38.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.480+0000 7ff62b7fe700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 0x7ff6240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.480+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff62c0cd2d0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.480+0000 7ff639d9b700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 0x7ff6240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.480+0000 7ff639d9b700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 0x7ff6240a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b5c0 tx=0x7ff630005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.482+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff62c0968d0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:38 smithi005 ceph-mon[126460]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:38.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:38 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3536221379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:38 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1248488960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.738+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff63c106a30 con 0x7ff63c0ffb00 2023-07-21T19:06:38.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.739+0000 7ff62b7fe700 1 -- 172.21.15.5:0/689540675 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7ff62c099ef0 con 0x7ff63c0ffb00 2023-07-21T19:06:38.749 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:38.749 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 msgr2=0x7ff6240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 0x7ff6240a36d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ff63000b5c0 tx=0x7ff630005fd0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 msgr2=0x7ff63c0742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff62c00d980 tx=0x7ff62c00dd40 comp rx=0 tx=0).stop 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 shutdown_connections 2023-07-21T19:06:38.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff6240a1220 0x7ff6240a36d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff63c101680 0x7ff63c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff63c0ffb00 0x7ff63c0742c0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 --2- 172.21.15.5:0/689540675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff63c100cf0 0x7ff63c072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.741+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 >> 172.21.15.5:0/689540675 conn(0x7ff63c0fb0e0 msgr2=0x7ff63c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.742+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 shutdown_connections 2023-07-21T19:06:38.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.742+0000 7ff6409ff700 1 -- 172.21.15.5:0/689540675 wait complete. 2023-07-21T19:06:38.753 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:06:38.770 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:06:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:38 smithi101 ceph-mon[128136]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:38 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3536221379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:38 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1248488960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:38 smithi156 ceph-mon[128316]: pgmap v332: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:38.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:38 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3536221379' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:38 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1248488960' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:38.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.964+0000 7fc33afff700 1 -- 172.21.15.5:0/3994557251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 msgr2=0x7fc334101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.964+0000 7fc33afff700 1 --2- 172.21.15.5:0/3994557251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334101140 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc320009990 tx=0x7fc320009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 -- 172.21.15.5:0/3994557251 shutdown_connections 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 --2- 172.21.15.5:0/3994557251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 0x7fc334107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 --2- 172.21.15.5:0/3994557251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 --2- 172.21.15.5:0/3994557251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334101140 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 -- 172.21.15.5:0/3994557251 >> 172.21.15.5:0/3994557251 conn(0x7fc3340fb0e0 msgr2=0x7fc3340fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 -- 172.21.15.5:0/3994557251 shutdown_connections 2023-07-21T19:06:38.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.965+0000 7fc33afff700 1 -- 172.21.15.5:0/3994557251 wait complete. 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 Processor -- start 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 -- start start 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 0x7fc334072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3340735e0 con 0x7fc334100cf0 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc334073750 con 0x7fc334101680 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc33afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc33419f710 con 0x7fc3340ffb00 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc333fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc333fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49468/0 (socket says 172.21.15.5:49468) 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.966+0000 7fc333fff700 1 -- 172.21.15.5:0/1387145439 learned_addr learned my addr 172.21.15.5:0/1387145439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:38.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc33959c700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 0x7fc334072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc338d9b700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 -- 172.21.15.5:0/1387145439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 msgr2=0x7fc3340742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 -- 172.21.15.5:0/1387145439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 msgr2=0x7fc334072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 0x7fc334072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 -- 172.21.15.5:0/1387145439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3200095f0 con 0x7fc334100cf0 2023-07-21T19:06:38.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc338d9b700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc333fff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3200098e0 tx=0x7fc320004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32001d070 con 0x7fc334100cf0 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc32000bdd0 con 0x7fc334100cf0 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc320022da0 con 0x7fc334100cf0 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.967+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc33419f990 con 0x7fc334100cf0 2023-07-21T19:06:38.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.968+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc33419fd60 con 0x7fc334100cf0 2023-07-21T19:06:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.969+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc334106ff0 con 0x7fc334100cf0 2023-07-21T19:06:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.971+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fc32000b940 con 0x7fc334100cf0 2023-07-21T19:06:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.972+0000 7fc331ffb700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 0x7fc3240a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:38.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.972+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fc3200ce350 con 0x7fc334100cf0 2023-07-21T19:06:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.972+0000 7fc338d9b700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 0x7fc3240a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.973+0000 7fc338d9b700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 0x7fc3240a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc334100b50 tx=0x7fc328009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:38.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:38.974+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc320097980 con 0x7fc334100cf0 2023-07-21T19:06:39.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.183+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc33402ca20 con 0x7fc334100cf0 2023-07-21T19:06:39.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.185+0000 7fc331ffb700 1 -- 172.21.15.5:0/1387145439 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7fc32009afa0 con 0x7fc334100cf0 2023-07-21T19:06:39.196 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:39.196 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:06:39.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 msgr2=0x7fc3240a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 0x7fc3240a3750 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc334100b50 tx=0x7fc328009260 comp rx=0 tx=0).stop 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 msgr2=0x7fc334072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fc3200098e0 tx=0x7fc320004730 comp rx=0 tx=0).stop 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 shutdown_connections 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fc3240a12a0 0x7fc3240a3750 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc334101680 0x7fc334072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc3340ffb00 0x7fc3340742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 --2- 172.21.15.5:0/1387145439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc334100cf0 0x7fc334072900 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 >> 172.21.15.5:0/1387145439 conn(0x7fc3340fb0e0 msgr2=0x7fc334104130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 shutdown_connections 2023-07-21T19:06:39.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.187+0000 7fc33afff700 1 -- 172.21.15.5:0/1387145439 wait complete. 2023-07-21T19:06:39.218 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:06:39.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:06:39.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 -- 172.21.15.5:0/4270286486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18941057a0 msgr2=0x7f1894107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 --2- 172.21.15.5:0/4270286486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18941057a0 0x7f1894107c00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f188c00b0a0 tx=0x7f188c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 -- 172.21.15.5:0/4270286486 shutdown_connections 2023-07-21T19:06:39.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 --2- 172.21.15.5:0/4270286486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18941057a0 0x7f1894107c00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 --2- 172.21.15.5:0/4270286486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894100560 0x7f1894102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 --2- 172.21.15.5:0/4270286486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1894102e80 0x7f1894105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 -- 172.21.15.5:0/4270286486 >> 172.21.15.5:0/4270286486 conn(0x7f18940fa000 msgr2=0x7f18940fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:39.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 -- 172.21.15.5:0/4270286486 shutdown_connections 2023-07-21T19:06:39.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.417+0000 7f189b5ff700 1 -- 172.21.15.5:0/4270286486 wait complete. 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.418+0000 7f189b5ff700 1 Processor -- start 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.418+0000 7f189b5ff700 1 -- start start 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894102e80 0x7f1894197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18941057a0 0x7f189419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419f590 con 0x7f1894100560 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419f6d0 con 0x7f18941057a0 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189b5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f189419f860 con 0x7f1894102e80 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49486/0 (socket says 172.21.15.5:49486) 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 -- 172.21.15.5:0/1528005466 learned_addr learned my addr 172.21.15.5:0/1528005466 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f1898b9a700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894102e80 0x7f1894197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f1899b9c700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18941057a0 0x7f189419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 -- 172.21.15.5:0/1528005466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894102e80 msgr2=0x7f1894197f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894102e80 0x7f1894197f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 -- 172.21.15.5:0/1528005466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18941057a0 msgr2=0x7f189419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18941057a0 0x7f189419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.419+0000 7f189939b700 1 -- 172.21.15.5:0/1528005466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c009d00 con 0x7f1894100560 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f189939b700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9a0 tx=0x7f189000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890009790 con 0x7f1894100560 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f189000cbd0 con 0x7f1894100560 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f189419fb40 con 0x7f1894100560 2023-07-21T19:06:39.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890018680 con 0x7f1894100560 2023-07-21T19:06:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.420+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18941a00a0 con 0x7f1894100560 2023-07-21T19:06:39.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.421+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1894107310 con 0x7f1894100560 2023-07-21T19:06:39.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.424+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f189000cd40 con 0x7f1894100560 2023-07-21T19:06:39.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.426+0000 7f188a7fc700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 0x7f18800a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.426+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1890014070 con 0x7f1894100560 2023-07-21T19:06:39.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.426+0000 7f1898b9a700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 0x7f18800a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.427+0000 7f1898b9a700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 0x7f18800a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1884009c60 tx=0x7f1884009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:39.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.430+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1890051790 con 0x7f1894100560 2023-07-21T19:06:39.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.636+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f189402cad0 con 0x7f1894100560 2023-07-21T19:06:39.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.636+0000 7f188a7fc700 1 -- 172.21.15.5:0/1528005466 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f18900962b0 con 0x7f1894100560 2023-07-21T19:06:39.648 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:06:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.638+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 msgr2=0x7f18800a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.638+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 0x7f18800a3710 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1884009c60 tx=0x7f1884009260 comp rx=0 tx=0).stop 2023-07-21T19:06:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.638+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 msgr2=0x7f1894197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.638+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f189000e9a0 tx=0x7f189000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 shutdown_connections 2023-07-21T19:06:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f18800a1260 0x7f18800a3710 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18941057a0 0x7f189419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1894100560 0x7f1894197a10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 --2- 172.21.15.5:0/1528005466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1894102e80 0x7f1894197f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 >> 172.21.15.5:0/1528005466 conn(0x7f18940fa000 msgr2=0x7f18940fec20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:39.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 shutdown_connections 2023-07-21T19:06:39.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.639+0000 7f189b5ff700 1 -- 172.21.15.5:0/1528005466 wait complete. 2023-07-21T19:06:39.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:39 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/689540675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:39.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:39 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1387145439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:39.670 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:06:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:39 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/689540675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:39 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1387145439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 -- 172.21.15.5:0/1782187680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 msgr2=0x7f5c10107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1782187680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c10107f30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b0a0 tx=0x7f5c0800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 -- 172.21.15.5:0/1782187680 shutdown_connections 2023-07-21T19:06:39.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1782187680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c10107f30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1782187680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c100ffaf0 0x7f5c100fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1782187680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 -- 172.21.15.5:0/1782187680 >> 172.21.15.5:0/1782187680 conn(0x7f5c100fb130 msgr2=0x7f5c100fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 -- 172.21.15.5:0/1782187680 shutdown_connections 2023-07-21T19:06:39.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.866+0000 7f5c153ff700 1 -- 172.21.15.5:0/1782187680 wait complete. 2023-07-21T19:06:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.867+0000 7f5c153ff700 1 Processor -- start 2023-07-21T19:06:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.867+0000 7f5c153ff700 1 -- start start 2023-07-21T19:06:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c100ffaf0 0x7f5c10197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10198070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1019f6b0 con 0x7f5c10101670 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1019f7f0 con 0x7f5c100ffaf0 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c153ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c1019f980 con 0x7f5c10100ce0 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49504/0 (socket says 172.21.15.5:49504) 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 -- 172.21.15.5:0/1389868890 learned_addr learned my addr 172.21.15.5:0/1389868890 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:39.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c07fff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10198070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 -- 172.21.15.5:0/1389868890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 msgr2=0x7f5c10198070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10198070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 -- 172.21.15.5:0/1389868890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c100ffaf0 msgr2=0x7f5c10197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c100ffaf0 0x7f5c10197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:39.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c0f7fe700 1 -- 172.21.15.5:0/1389868890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c08009d00 con 0x7f5c10101670 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.868+0000 7f5c07fff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c0f7fe700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009fd0 tx=0x7f5c0800bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800e040 con 0x7f5c10101670 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c1019fc00 con 0x7f5c10101670 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5c08003c00 con 0x7f5c10101670 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0801b6c0 con 0x7f5c10101670 2023-07-21T19:06:39.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.869+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c101a0020 con 0x7f5c10101670 2023-07-21T19:06:39.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.871+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c10192090 con 0x7f5c10101670 2023-07-21T19:06:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.872+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5c0801d470 con 0x7f5c10101670 2023-07-21T19:06:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.873+0000 7f5c0cff9700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 0x7f5bfc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.874+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5c080cdfe0 con 0x7f5c10101670 2023-07-21T19:06:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.874+0000 7f5c0effd700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 0x7f5bfc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:39.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.874+0000 7f5c0effd700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 0x7f5bfc0a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c10100b40 tx=0x7f5c000096c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:39.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:39.877+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5c08100790 con 0x7f5c10101670 2023-07-21T19:06:39.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:39 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/689540675' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:39.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:39 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1387145439' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:40.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.087+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5c1002c850 con 0x7f5c10101670 2023-07-21T19:06:40.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.088+0000 7f5c0cff9700 1 -- 172.21.15.5:0/1389868890 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5c08097670 con 0x7f5c10101670 2023-07-21T19:06:40.099 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:06:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.090+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 msgr2=0x7f5bfc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.090+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 0x7f5bfc0a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5c10100b40 tx=0x7f5c000096c0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.090+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 msgr2=0x7f5c1019f0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.090+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5c08009fd0 tx=0x7f5c0800bb00 comp rx=0 tx=0).stop 2023-07-21T19:06:40.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 shutdown_connections 2023-07-21T19:06:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5bfc0a11e0 0x7f5bfc0a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5c10101670 0x7f5c1019f0e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5c100ffaf0 0x7f5c10197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 --2- 172.21.15.5:0/1389868890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5c10100ce0 0x7f5c10198070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 >> 172.21.15.5:0/1389868890 conn(0x7f5c100fb130 msgr2=0x7f5c10106640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 shutdown_connections 2023-07-21T19:06:40.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.091+0000 7f5c153ff700 1 -- 172.21.15.5:0/1389868890 wait complete. 2023-07-21T19:06:40.121 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:06:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.327+0000 7f88417ff700 1 -- 172.21.15.5:0/2286561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c101d10 msgr2=0x7f883c102120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.327+0000 7f88417ff700 1 --2- 172.21.15.5:0/2286561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c101d10 0x7f883c102120 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f882c009990 tx=0x7f882c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 -- 172.21.15.5:0/2286561741 shutdown_connections 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 --2- 172.21.15.5:0/2286561741 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c103890 0x7f883c10a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 --2- 172.21.15.5:0/2286561741 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c101d10 0x7f883c102120 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 --2- 172.21.15.5:0/2286561741 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c102f00 0x7f883c103350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 -- 172.21.15.5:0/2286561741 >> 172.21.15.5:0/2286561741 conn(0x7f883c0fd350 msgr2=0x7f883c0ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 -- 172.21.15.5:0/2286561741 shutdown_connections 2023-07-21T19:06:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.328+0000 7f88417ff700 1 -- 172.21.15.5:0/2286561741 wait complete. 2023-07-21T19:06:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 Processor -- start 2023-07-21T19:06:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 -- start start 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 0x7f883c197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 0x7f883c19ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c19f550 con 0x7f883c103890 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c19f690 con 0x7f883c101d10 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f88417ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f883c19f820 con 0x7f883c102f00 2023-07-21T19:06:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.329+0000 7f883affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50946/0 (socket says 172.21.15.5:50946) 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 -- 172.21.15.5:0/2285975682 learned_addr learned my addr 172.21.15.5:0/2285975682 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f8833fff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 0x7f883c197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883b7fe700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 0x7f883c19ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 -- 172.21.15.5:0/2285975682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 msgr2=0x7f883c197f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 0x7f883c197f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 -- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 msgr2=0x7f883c19ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 0x7f883c19ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 -- 172.21.15.5:0/2285975682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f882c0095f0 con 0x7f883c101d10 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f8833fff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 0x7f883c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883b7fe700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 0x7f883c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.330+0000 7f883affd700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f882c005910 tx=0x7f882c0057f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.331+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c01d070 con 0x7f883c101d10 2023-07-21T19:06:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.331+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f882c00bed0 con 0x7f883c101d10 2023-07-21T19:06:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.331+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f883c19faa0 con 0x7f883c101d10 2023-07-21T19:06:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.331+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f882c022e60 con 0x7f883c101d10 2023-07-21T19:06:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.331+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f883c19ff20 con 0x7f883c101d10 2023-07-21T19:06:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.332+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f883c10d3e0 con 0x7f883c101d10 2023-07-21T19:06:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.334+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f882c022600 con 0x7f883c101d10 2023-07-21T19:06:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.336+0000 7f8838ff9700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 0x7f88280a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.336+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f882c0ce350 con 0x7f883c101d10 2023-07-21T19:06:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.336+0000 7f8833fff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 0x7f88280a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.337+0000 7f8833fff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 0x7f88280a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8824009cb0 tx=0x7f8824009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.338+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f882c097980 con 0x7f883c101d10 2023-07-21T19:06:40.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.550+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f883c108cc0 con 0x7f883c101d10 2023-07-21T19:06:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.550+0000 7f8838ff9700 1 -- 172.21.15.5:0/2285975682 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f882c09afa0 con 0x7f883c101d10 2023-07-21T19:06:40.562 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:06:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 msgr2=0x7f88280a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 0x7f88280a3710 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f8824009cb0 tx=0x7f8824009260 comp rx=0 tx=0).stop 2023-07-21T19:06:40.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 msgr2=0x7f883c1979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f882c005910 tx=0x7f882c0057f0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 shutdown_connections 2023-07-21T19:06:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f88280a1260 0x7f88280a3710 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f883c103890 0x7f883c19ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f883c101d10 0x7f883c1979d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 --2- 172.21.15.5:0/2285975682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f883c102f00 0x7f883c197f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 >> 172.21.15.5:0/2285975682 conn(0x7f883c0fd350 msgr2=0x7f883c0ff620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:40.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 shutdown_connections 2023-07-21T19:06:40.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.553+0000 7f88417ff700 1 -- 172.21.15.5:0/2285975682 wait complete. 2023-07-21T19:06:40.589 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:06:40.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.795+0000 7f3ef3bff700 1 -- 172.21.15.5:0/1002924201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec072f70 msgr2=0x7f3eec0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.795+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/1002924201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec072f70 0x7f3eec0733e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009990 tx=0x7f3edc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 -- 172.21.15.5:0/1002924201 shutdown_connections 2023-07-21T19:06:40.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/1002924201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec1079c0 0x7f3eec109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/1002924201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 0x7f3eec0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/1002924201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec072f70 0x7f3eec0733e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 -- 172.21.15.5:0/1002924201 >> 172.21.15.5:0/1002924201 conn(0x7f3eec0fb0e0 msgr2=0x7f3eec0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 -- 172.21.15.5:0/1002924201 shutdown_connections 2023-07-21T19:06:40.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.796+0000 7f3ef3bff700 1 -- 172.21.15.5:0/1002924201 wait complete. 2023-07-21T19:06:40.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 Processor -- start 2023-07-21T19:06:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 -- start start 2023-07-21T19:06:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec072f70 0x7f3eec1979a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 0x7f3eec197ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec19f520 con 0x7f3eec1079c0 2023-07-21T19:06:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec19f660 con 0x7f3eec074550 2023-07-21T19:06:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec19f7f0 con 0x7f3eec072f70 2023-07-21T19:06:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.797+0000 7f3ef219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49540/0 (socket says 172.21.15.5:49540) 2023-07-21T19:06:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 -- 172.21.15.5:0/2600456840 learned_addr learned my addr 172.21.15.5:0/2600456840 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef199b700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec072f70 0x7f3eec1979a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef119a700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 0x7f3eec197ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 -- 172.21.15.5:0/2600456840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec072f70 msgr2=0x7f3eec1979a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec072f70 0x7f3eec1979a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 -- 172.21.15.5:0/2600456840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 msgr2=0x7f3eec197ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 0x7f3eec197ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 -- 172.21.15.5:0/2600456840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ee8009520 con 0x7f3eec1079c0 2023-07-21T19:06:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef219c700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4007ee0 tx=0x7f3ee400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:40.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4017070 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3edc0095f0 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ee400f040 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.798+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ee4005570 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.799+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec19fea0 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.800+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec106070 con 0x7f3eec1079c0 2023-07-21T19:06:40.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.801+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f3ee400db20 con 0x7f3eec1079c0 2023-07-21T19:06:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.803+0000 7f3ee2ffd700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 0x7f3ed80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.803+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f3ee40cc220 con 0x7f3eec1079c0 2023-07-21T19:06:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.803+0000 7f3ef199b700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 0x7f3ed80a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:40.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.804+0000 7f3ef199b700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 0x7f3ed80a36d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eec074920 tx=0x7f3ee8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:40.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:40.807+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ee40957c0 con 0x7f3eec1079c0 2023-07-21T19:06:40.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:40 smithi101 ceph-mon[128136]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1528005466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1389868890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:40 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:40.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:40 smithi005 ceph-mon[126460]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1528005466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1389868890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:40 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:40.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:40 smithi156 ceph-mon[128316]: pgmap v333: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:40.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1528005466' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1389868890' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:06:40.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:40 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:06:41.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.070+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3eec02ca50 con 0x7f3eec1079c0 2023-07-21T19:06:41.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.071+0000 7f3ee2ffd700 1 -- 172.21.15.5:0/2600456840 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f3ee4098de0 con 0x7f3eec1079c0 2023-07-21T19:06:41.081 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:41.082 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:06:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.073+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 msgr2=0x7f3ed80a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.073+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 0x7f3ed80a36d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f3eec074920 tx=0x7f3ee8009260 comp rx=0 tx=0).stop 2023-07-21T19:06:41.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 msgr2=0x7f3eec19ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f3ee4007ee0 tx=0x7f3ee400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 shutdown_connections 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f3ed80a1220 0x7f3ed80a36d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3eec1079c0 0x7f3eec19ef50 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3eec072f70 0x7f3eec1979a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 --2- 172.21.15.5:0/2600456840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3eec074550 0x7f3eec197ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 >> 172.21.15.5:0/2600456840 conn(0x7f3eec0fb0e0 msgr2=0x7f3eec0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 shutdown_connections 2023-07-21T19:06:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.074+0000 7f3ef3bff700 1 -- 172.21.15.5:0/2600456840 wait complete. 2023-07-21T19:06:41.086 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:06:41.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:06:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 -- 172.21.15.5:0/589600026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640072f70 msgr2=0x7f56400733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 --2- 172.21.15.5:0/589600026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640072f70 0x7f56400733e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5630009990 tx=0x7f5630009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 -- 172.21.15.5:0/589600026 shutdown_connections 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 --2- 172.21.15.5:0/589600026 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f564010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 --2- 172.21.15.5:0/589600026 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640074550 0x7f56400729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 --2- 172.21.15.5:0/589600026 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640072f70 0x7f56400733e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 -- 172.21.15.5:0/589600026 >> 172.21.15.5:0/589600026 conn(0x7f56400fb0e0 msgr2=0x7f56400fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 -- 172.21.15.5:0/589600026 shutdown_connections 2023-07-21T19:06:41.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.315+0000 7f5647fff700 1 -- 172.21.15.5:0/589600026 wait complete. 2023-07-21T19:06:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.317+0000 7f5647fff700 1 Processor -- start 2023-07-21T19:06:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.317+0000 7f5647fff700 1 -- start start 2023-07-21T19:06:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.317+0000 7f5647fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 0x7f564019bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5647fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640074550 0x7f564019c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5647fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5647fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56401a3960 con 0x7f564010c0d0 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5647fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56401a3aa0 con 0x7f5640072f70 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5647fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56401a3c30 con 0x7f5640074550 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49560/0 (socket says 172.21.15.5:49560) 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 -- 172.21.15.5:0/596678630 learned_addr learned my addr 172.21.15.5:0/596678630 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:41.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5645d9b700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 0x7f564019bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564559a700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640074550 0x7f564019c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 -- 172.21.15.5:0/596678630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640074550 msgr2=0x7f564019c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640074550 0x7f564019c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 -- 172.21.15.5:0/596678630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 msgr2=0x7f564019bde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 0x7f564019bde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f564659c700 1 -- 172.21.15.5:0/596678630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56300095f0 con 0x7f564010c0d0 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.318+0000 7f5645d9b700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 0x7f564019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:41.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f564659c700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f563800ef30 tx=0x7f563800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5638018070 con 0x7f564010c0d0 2023-07-21T19:06:41.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f563800f040 con 0x7f564010c0d0 2023-07-21T19:06:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f563800cbc0 con 0x7f564010c0d0 2023-07-21T19:06:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56401a3eb0 con 0x7f564010c0d0 2023-07-21T19:06:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.319+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56401a42e0 con 0x7f564010c0d0 2023-07-21T19:06:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.320+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5640101c20 con 0x7f564010c0d0 2023-07-21T19:06:41.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.323+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f563800cd80 con 0x7f564010c0d0 2023-07-21T19:06:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.324+0000 7f5636ffd700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 0x7f562c0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.325+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f56380cc4e0 con 0x7f564010c0d0 2023-07-21T19:06:41.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.325+0000 7f5645d9b700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 0x7f562c0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.326+0000 7f5645d9b700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 0x7f562c0a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5640074920 tx=0x7f563c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:41.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.329+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5638095b10 con 0x7f564010c0d0 2023-07-21T19:06:41.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.559+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f564002caa0 con 0x7f564010c0d0 2023-07-21T19:06:41.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.560+0000 7f5636ffd700 1 -- 172.21.15.5:0/596678630 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f5638099130 con 0x7f564010c0d0 2023-07-21T19:06:41.571 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:41.571 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:06:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 msgr2=0x7f562c0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 0x7f562c0a3750 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5640074920 tx=0x7f563c009260 comp rx=0 tx=0).stop 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 msgr2=0x7f56401a3390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f563800ef30 tx=0x7f563800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 shutdown_connections 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f562c0a12a0 0x7f562c0a3750 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f564010c0d0 0x7f56401a3390 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5640072f70 0x7f564019bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 --2- 172.21.15.5:0/596678630 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5640074550 0x7f564019c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 >> 172.21.15.5:0/596678630 conn(0x7f56400fb0e0 msgr2=0x7f56400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 shutdown_connections 2023-07-21T19:06:41.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.563+0000 7f5647fff700 1 -- 172.21.15.5:0/596678630 wait complete. 2023-07-21T19:06:41.594 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 -- 172.21.15.5:0/3655371650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 msgr2=0x7f2bd4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/3655371650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4101140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 -- 172.21.15.5:0/3655371650 shutdown_connections 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/3655371650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd4101680 0x7f2bd4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/3655371650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/3655371650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4101140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 -- 172.21.15.5:0/3655371650 >> 172.21.15.5:0/3655371650 conn(0x7f2bd40fb0e0 msgr2=0x7f2bd40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 -- 172.21.15.5:0/3655371650 shutdown_connections 2023-07-21T19:06:41.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.798+0000 7f2bd8bff700 1 -- 172.21.15.5:0/3655371650 wait complete. 2023-07-21T19:06:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.800+0000 7f2bd8bff700 1 Processor -- start 2023-07-21T19:06:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.800+0000 7f2bd8bff700 1 -- start start 2023-07-21T19:06:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.800+0000 7f2bd8bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.800+0000 7f2bd8bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.800+0000 7f2bd8bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd4101680 0x7f2bd4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd8bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd40735e0 con 0x7f2bd4100cf0 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd8bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd4073750 con 0x7f2bd4101680 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd8bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd419f7c0 con 0x7f2bd40ffb00 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41850/0 (socket says 172.21.15.5:41850) 2023-07-21T19:06:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 -- 172.21.15.5:0/2633934678 learned_addr learned my addr 172.21.15.5:0/2633934678 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:41.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd2d9d700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd4101680 0x7f2bd4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 -- 172.21.15.5:0/2633934678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd4101680 msgr2=0x7f2bd4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd4101680 0x7f2bd4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 -- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 msgr2=0x7f2bd4072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.801+0000 7f2bd259c700 1 -- 172.21.15.5:0/2633934678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80095f0 con 0x7f2bd40ffb00 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bd259c700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400d980 tx=0x7f2bc400dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:41.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4004020 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bc400f690 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd419faa0 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc40039e0 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.802+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd41a0000 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.803+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd4107090 con 0x7f2bd40ffb00 2023-07-21T19:06:41.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.805+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2bc400f800 con 0x7f2bd40ffb00 2023-07-21T19:06:41.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.806+0000 7f2bc37fe700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 0x7f2bbc0a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.807+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2bc40cc670 con 0x7f2bd40ffb00 2023-07-21T19:06:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.807+0000 7f2bd1d9b700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 0x7f2bbc0a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:41.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.807+0000 7f2bd1d9b700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 0x7f2bbc0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8006010 tx=0x7f2bc800b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:41.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:41.811+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2bc4095c10 con 0x7f2bd40ffb00 2023-07-21T19:06:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:41 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2285975682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:41 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2600456840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:41 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2285975682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:41.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:41 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2600456840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:41.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:41 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2285975682' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:06:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:41 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2600456840' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.090+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2bd402c850 con 0x7f2bd40ffb00 2023-07-21T19:06:42.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.090+0000 7f2bc37fe700 1 -- 172.21.15.5:0/2633934678 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f2bc4099230 con 0x7f2bd40ffb00 2023-07-21T19:06:42.101 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:06:42.101 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:06:42.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 msgr2=0x7f2bbc0a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 0x7f2bbc0a3750 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8006010 tx=0x7f2bc800b260 comp rx=0 tx=0).stop 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 msgr2=0x7f2bd40742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f2bc400d980 tx=0x7f2bc400dc90 comp rx=0 tx=0).stop 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 shutdown_connections 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2bbc0a12a0 0x7f2bbc0a3750 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bd4101680 0x7f2bd4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bd40ffb00 0x7f2bd40742c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 --2- 172.21.15.5:0/2633934678 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bd4100cf0 0x7f2bd4072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 >> 172.21.15.5:0/2633934678 conn(0x7f2bd40fb0e0 msgr2=0x7f2bd41041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 shutdown_connections 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.093+0000 7f2bd8bff700 1 -- 172.21.15.5:0/2633934678 wait complete. 2023-07-21T19:06:42.105 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:06:42.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T19:05:37.254064+0000', 'modified': '2023-07-21T19:05:43.279232+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24614, 'mds_1': 24566, 'mds_2': 24553}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2855288918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2855288918}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2855288918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/1947602937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 1947602937}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 1947602937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.101:6835/153908041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6834', 'nonce': 153908041}, {'type': 'v1', 'addr': '172.21.15.101:6835', 'nonce': 153908041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6839/2252036160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 2252036160}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 2252036160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.101:6837/3565857598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6836', 'nonce': 3565857598}, {'type': 'v1', 'addr': '172.21.15.101:6837', 'nonce': 3565857598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24614': {'gid': 24614, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.101:6839/715074805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6838', 'nonce': 715074805}, {'type': 'v1', 'addr': '172.21.15.101:6839', 'nonce': 715074805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T19:06:42.123 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-21T19:06:42.123 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T19:06:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 -- 172.21.15.5:0/3661341740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 msgr2=0x7f9378107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 --2- 172.21.15.5:0/3661341740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f9378107c00 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f937000b0a0 tx=0x7f937000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 -- 172.21.15.5:0/3661341740 shutdown_connections 2023-07-21T19:06:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 --2- 172.21.15.5:0/3661341740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f9378107c00 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 --2- 172.21.15.5:0/3661341740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f93781029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 --2- 172.21.15.5:0/3661341740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f93781052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 -- 172.21.15.5:0/3661341740 >> 172.21.15.5:0/3661341740 conn(0x7f93780fa000 msgr2=0x7f93780fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 -- 172.21.15.5:0/3661341740 shutdown_connections 2023-07-21T19:06:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.335+0000 7f937efff700 1 -- 172.21.15.5:0/3661341740 wait complete. 2023-07-21T19:06:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.336+0000 7f937efff700 1 Processor -- start 2023-07-21T19:06:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 -- start start 2023-07-21T19:06:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f9378197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f937819efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937819f5b0 con 0x7f9378102ee0 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937819f6f0 con 0x7f9378105800 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937efff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937819f880 con 0x7f93781005c0 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f9377fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f9378197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41860/0 (socket says 172.21.15.5:41860) 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937d59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f937819efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.337+0000 7f937cd9b700 1 -- 172.21.15.5:0/2306356637 learned_addr learned my addr 172.21.15.5:0/2306356637 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 -- 172.21.15.5:0/2306356637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 msgr2=0x7f937819efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f937819efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 -- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 msgr2=0x7f9378197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f9378197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 -- 172.21.15.5:0/2306356637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009d00 con 0x7f93781005c0 2023-07-21T19:06:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937d59c700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f937819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f9377fff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f9378197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937cd9b700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f936c00e9a0 tx=0x7f936c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f9375ffb700 1 -- 172.21.15.5:0/2306356637 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c009790 con 0x7f93781005c0 2023-07-21T19:06:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f9375ffb700 1 -- 172.21.15.5:0/2306356637 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f936c010430 con 0x7f93781005c0 2023-07-21T19:06:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937819fb60 con 0x7f93781005c0 2023-07-21T19:06:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f9375ffb700 1 -- 172.21.15.5:0/2306356637 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c00cea0 con 0x7f93781005c0 2023-07-21T19:06:42.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.338+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781a0070 con 0x7f93781005c0 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.341+0000 7f9375ffb700 1 -- 172.21.15.5:0/2306356637 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f936c0105a0 con 0x7f93781005c0 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 Processor -- start 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9375ffb700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 0x7f93680a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 -- start start 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9375ffb700 1 -- 172.21.15.5:0/2306356637 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f936c014070 con 0x7f93781005c0 2023-07-21T19:06:42.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd397d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd397e690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd397f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f937d59c700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 0x7f93680a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd3882c60 con 0x555dd397ec20 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd38d59f0 con 0x555dd397ddd0 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.343+0000 7f9392970b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd38cee80 con 0x555dd397d480 2023-07-21T19:06:42.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd397f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd397d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd397f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49626/0 (socket says 172.21.15.5:49626) 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937d59c700 1 -- 172.21.15.5:0/1701344481 learned_addr learned my addr 172.21.15.5:0/1701344481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f9377fff700 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd397e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 -- 172.21.15.5:0/1701344481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 msgr2=0x555dd397e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd397e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 -- 172.21.15.5:0/1701344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 msgr2=0x555dd397f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd397f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 -- 172.21.15.5:0/1701344481 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd36fcd90 con 0x555dd397d480 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f9377fff700 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd397e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937d59c700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 0x7f93680a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9370009610 tx=0x7f93700093e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd397d890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f936c0ca6a0 tx=0x7f936c0999f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4521365f7a6de39d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.344+0000 7f9362ffd700 1 -- 172.21.15.5:0/1701344481 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c09dfd0 con 0x555dd397d480 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9362ffd700 1 -- 172.21.15.5:0/1701344481 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f936c0a23f0 con 0x555dd397d480 2023-07-21T19:06:42.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9362ffd700 1 -- 172.21.15.5:0/1701344481 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0a1f60 con 0x555dd397d480 2023-07-21T19:06:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 -- 172.21.15.5:0/1701344481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 msgr2=0x555dd397d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd397d890 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f936c0ca6a0 tx=0x7f936c0999f0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 -- 172.21.15.5:0/1701344481 shutdown_connections 2023-07-21T19:06:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd397f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd397d890 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 --2- 172.21.15.5:0/1701344481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd397e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 -- 172.21.15.5:0/1701344481 >> 172.21.15.5:0/1701344481 conn(0x555dd3972d50 msgr2=0x555dd3978180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 -- 172.21.15.5:0/1701344481 shutdown_connections 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.345+0000 7f9392970b80 1 -- 172.21.15.5:0/1701344481 wait complete. 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.346+0000 7f9392970b80 1 -- start start 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.346+0000 7f9392970b80 1 Processor -- start 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.346+0000 7f9392970b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.346+0000 7f9392970b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd39875e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9392970b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd398e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9392970b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd386ea70 con 0x555dd397d480 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9392970b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd386e770 con 0x555dd397ec20 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9392970b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd37528d0 con 0x555dd397ddd0 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9392970b80 10 client.0 _ll_register_callbacks cb 0x555dd386c710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9377fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd39875e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49634/0 (socket says 172.21.15.5:49634) 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9377fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd39875e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41886/0 (socket says 172.21.15.5:41886) 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937d59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd398e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 -- 172.21.15.5:0/471449848 learned_addr learned my addr 172.21.15.5:0/471449848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 -- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 msgr2=0x555dd39875e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd39875e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 -- 172.21.15.5:0/471449848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 msgr2=0x555dd398e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd398e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 -- 172.21.15.5:0/471449848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93781a0070 con 0x555dd397d480 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f937cd9b700 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f936c0ca6d0 tx=0x7f936c00c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ad7ac811fa2164 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.347+0000 7f9357fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0b6070 con 0x555dd397d480 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f936c098340 con 0x555dd397d480 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555dd38284a0 con 0x555dd397d480 2023-07-21T19:06:42.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f936c0b82b0 con 0x555dd397d480 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 10 client.15126 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555dd36e5c10 con 0x555dd397d480 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f936c063080 con 0x555dd397d480 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f936c095d90 con 0x555dd397d480 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 10 client.15126 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 10 client.15126 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 20 client.15126 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39925f0 0x555dd3991550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 4 client.15126 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.348+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555dd38c7e60 con 0x555dd39925f0 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.349+0000 7f9377fff700 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39925f0 0x555dd3991550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.349+0000 7f9377fff700 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39925f0 0x555dd3991550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4e16b0882ee5f602 server_cookie=24981fbc6c37092d in_seq=0 out_seq=0 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.349+0000 7f9357fff700 10 client.15126 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9357fff700 1 -- 172.21.15.5:0/471449848 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555dd38c7e60 con 0x555dd39925f0 2023-07-21T19:06:42.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9357fff700 10 client.15126 handle_command_reply: tid=0 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9392970b80 1 client.15126 shutdown 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 msgr2=0x555dd39870a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f936c0ca6d0 tx=0x7f936c00c480 comp rx=0 tx=0).stop 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.350+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 shutdown_connections 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39925f0 0x555dd3991550 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd398e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9357fff700 0 client.15126 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:06:42.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd39870a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 --2- 172.21.15.5:0/471449848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd39875e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 >> 172.21.15.5:0/471449848 conn(0x555dd3972d50 msgr2=0x555dd3977860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 shutdown_connections 2023-07-21T19:06:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 1 -- 172.21.15.5:0/471449848 wait complete. 2023-07-21T19:06:42.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.351+0000 7f9392970b80 20 client.15126 trim_cache size 0 max 16384 2023-07-21T19:06:42.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 Processor -- start 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 -- start start 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd398e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd3984ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd3988290 con 0x555dd397d480 2023-07-21T19:06:42.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd3878470 con 0x555dd397ec20 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f9392970b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd389d790 con 0x555dd397ddd0 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f937cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49648/0 (socket says 172.21.15.5:49648) 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f937cd9b700 1 -- 172.21.15.5:0/3834608652 learned_addr learned my addr 172.21.15.5:0/3834608652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.370+0000 7f937d59c700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd3984ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f9377fff700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd398e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 -- 172.21.15.5:0/3834608652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 msgr2=0x555dd398e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd398e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 -- 172.21.15.5:0/3834608652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 msgr2=0x555dd3984ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd3984ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 -- 172.21.15.5:0/3834608652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555dd3723200 con 0x555dd397d480 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f9377fff700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd398e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f937cd9b700 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f936c099ca0 tx=0x7f936c0ca750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4416bf0b5257269e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f93637fe700 1 -- 172.21.15.5:0/3834608652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c09e050 con 0x555dd397d480 2023-07-21T19:06:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f93637fe700 1 -- 172.21.15.5:0/3834608652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555dd38284a0 con 0x555dd397d480 2023-07-21T19:06:42.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.371+0000 7f93637fe700 1 -- 172.21.15.5:0/3834608652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936c0ca3d0 con 0x555dd397d480 2023-07-21T19:06:42.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 -- 172.21.15.5:0/3834608652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 msgr2=0x555dd3977860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f936c099ca0 tx=0x7f936c0ca750 comp rx=0 tx=0).stop 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 -- 172.21.15.5:0/3834608652 shutdown_connections 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ec20 0x555dd3984ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397d480 0x555dd3977860 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 --2- 172.21.15.5:0/3834608652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397ddd0 0x555dd398e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 -- 172.21.15.5:0/3834608652 >> 172.21.15.5:0/3834608652 conn(0x555dd3972d50 msgr2=0x555dd399ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 -- 172.21.15.5:0/3834608652 shutdown_connections 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.372+0000 7f9392970b80 1 -- 172.21.15.5:0/3834608652 wait complete. 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 -- start start 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 Processor -- start 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd3989930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd3989e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd398d3d0 con 0x555dd397ec20 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd398d540 con 0x555dd397ddd0 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555dd398d700 con 0x555dd397d480 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.373+0000 7f9392970b80 10 client.0 _ll_register_callbacks cb 0x7f9364005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f9377fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd3989e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49652/0 (socket says 172.21.15.5:49652) 2023-07-21T19:06:42.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 -- 172.21.15.5:0/1448828836 learned_addr learned my addr 172.21.15.5:0/1448828836 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd3989930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 -- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 msgr2=0x555dd3989930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd3989930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 -- 172.21.15.5:0/1448828836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 msgr2=0x555dd3989e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd3989e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 -- 172.21.15.5:0/1448828836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555dd36e5c10 con 0x555dd397ec20 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f9377fff700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd3989e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f937d59c700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93700079c0 tx=0x7f937000b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d07d67bc7394e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.374+0000 7f9357fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9370025070 con 0x555dd397ec20 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f936c0a23f0 con 0x555dd397ec20 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f937000bcc0 con 0x555dd397ec20 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f937000e040 con 0x555dd397ec20 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 10 client.15132 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x555dd36fb260 con 0x555dd397ec20 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f936c0a23f0 con 0x555dd397ec20 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f937001d840 con 0x555dd397ec20 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 10 client.15132 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 10 client.15132 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 20 client.15132 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39eb7e0 0x555dd39e8170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 4 client.15132 mds_command: new command op to 24553 tid=0[{"prefix": "status"}] 2023-07-21T19:06:42.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.375+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555dd388a000 con 0x555dd39eb7e0 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.376+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39eb7e0 0x555dd39e8170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.376+0000 7f937cd9b700 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39eb7e0 0x555dd39e8170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8e23224110b18721 server_cookie=26077de9d3998c4d in_seq=0 out_seq=0 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.376+0000 7f9357fff700 10 client.15132 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.376+0000 7f9357fff700 1 -- 172.21.15.5:0/1448828836 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x555dd388a000 con 0x555dd39eb7e0 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.376+0000 7f9357fff700 10 client.15132 handle_command_reply: tid=0 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 client.15132 shutdown 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 msgr2=0x555dd398ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f93700079c0 tx=0x7f937000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 shutdown_connections 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555dd397ec20 0x555dd398ce90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555dd397d480 0x555dd3989930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x555dd39eb7e0 0x555dd39e8170 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 --2- 172.21.15.5:0/1448828836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555dd397ddd0 0x555dd3989e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9357fff700 0 client.15132 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:06:42.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 >> 172.21.15.5:0/1448828836 conn(0x555dd3972d50 msgr2=0x555dd3977860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 shutdown_connections 2023-07-21T19:06:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 1 -- 172.21.15.5:0/1448828836 wait complete. 2023-07-21T19:06:42.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.377+0000 7f9392970b80 20 client.15132 trim_cache size 0 max 16384 2023-07-21T19:06:42.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:06:42.396 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24553, 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 60.271307096999998, 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:06:42.397 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:06:42.398 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:06:42.398 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 81.031701566999999 2023-07-21T19:06:42.398 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 msgr2=0x7f93680a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 0x7f93680a3690 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9370009610 tx=0x7f93700093e0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 msgr2=0x7f9378197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f936c00e9a0 tx=0x7f936c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 shutdown_connections 2023-07-21T19:06:42.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f93680a11e0 0x7f93680a3690 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9378105800 0x7f937819efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93781005c0 0x7f9378197a30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 --2- 172.21.15.5:0/2306356637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9378102ee0 0x7f9378197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 >> 172.21.15.5:0/2306356637 conn(0x7f93780fa000 msgr2=0x7f9378101220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 shutdown_connections 2023-07-21T19:06:42.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.378+0000 7f937efff700 1 -- 172.21.15.5:0/2306356637 wait complete. 2023-07-21T19:06:42.407 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T19:06:42.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.617+0000 7f99f49ff700 1 -- 172.21.15.5:0/3890988185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0074550 msgr2=0x7f99f00729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.617+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3890988185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0074550 0x7f99f00729a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f99e0009990 tx=0x7f99e0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 -- 172.21.15.5:0/3890988185 shutdown_connections 2023-07-21T19:06:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3890988185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f010c0b0 0x7f99f010e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3890988185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0074550 0x7f99f00729a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3890988185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f0072f70 0x7f99f00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 -- 172.21.15.5:0/3890988185 >> 172.21.15.5:0/3890988185 conn(0x7f99f00fb0e0 msgr2=0x7f99f00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 -- 172.21.15.5:0/3890988185 shutdown_connections 2023-07-21T19:06:42.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.618+0000 7f99f49ff700 1 -- 172.21.15.5:0/3890988185 wait complete. 2023-07-21T19:06:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 Processor -- start 2023-07-21T19:06:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 -- start start 2023-07-21T19:06:42.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f0074550 0x7f99f019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f010c0b0 0x7f99f01a33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a3970 con 0x7f99f0072f70 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a3ab0 con 0x7f99f010c0b0 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.619+0000 7f99f49ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99f01a3c40 con 0x7f99f0074550 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49678/0 (socket says 172.21.15.5:49678) 2023-07-21T19:06:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 -- 172.21.15.5:0/3972657326 learned_addr learned my addr 172.21.15.5:0/3972657326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99edd9b700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f0074550 0x7f99f019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99eed9d700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f010c0b0 0x7f99f01a33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 -- 172.21.15.5:0/3972657326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f0074550 msgr2=0x7f99f019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f0074550 0x7f99f019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 -- 172.21.15.5:0/3972657326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f010c0b0 msgr2=0x7f99f01a33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f010c0b0 0x7f99f01a33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 -- 172.21.15.5:0/3972657326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e00095f0 con 0x7f99f0072f70 2023-07-21T19:06:42.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99ee59c700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098e0 tx=0x7f99e0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.620+0000 7f99df7fe700 1 -- 172.21.15.5:0/3972657326 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e001d070 con 0x7f99f0072f70 2023-07-21T19:06:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.621+0000 7f99df7fe700 1 -- 172.21.15.5:0/3972657326 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99e000be30 con 0x7f99f0072f70 2023-07-21T19:06:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.621+0000 7f99df7fe700 1 -- 172.21.15.5:0/3972657326 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e0022d80 con 0x7f99f0072f70 2023-07-21T19:06:42.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.621+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99f01a3ec0 con 0x7f99f0072f70 2023-07-21T19:06:42.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.621+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99f01a4230 con 0x7f99f0072f70 2023-07-21T19:06:42.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.624+0000 7f99df7fe700 1 -- 172.21.15.5:0/3972657326 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f99e0022470 con 0x7f99f0072f70 2023-07-21T19:06:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.626+0000 7f9a0841ab80 1 Processor -- start 2023-07-21T19:06:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.626+0000 7f9a0841ab80 1 -- start start 2023-07-21T19:06:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dface690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99df7fe700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 0x7f99d80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfacf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5df9d2c60 con 0x55d5dfacd480 2023-07-21T19:06:42.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfa259f0 con 0x55d5dfacddd0 2023-07-21T19:06:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfa1ee80 con 0x55d5dfacec20 2023-07-21T19:06:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99df7fe700 1 -- 172.21.15.5:0/3972657326 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99e00cdf40 con 0x7f99f0072f70 2023-07-21T19:06:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49684/0 (socket says 172.21.15.5:49684) 2023-07-21T19:06:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99edd9b700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dface690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99eed9d700 1 -- 172.21.15.5:0/286219384 learned_addr learned my addr 172.21.15.5:0/286219384 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.627+0000 7f99ee59c700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 0x7f99d80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99eed9d700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfacf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 -- 172.21.15.5:0/286219384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 msgr2=0x55d5dfacf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfacf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 -- 172.21.15.5:0/286219384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 msgr2=0x55d5dfacd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 -- 172.21.15.5:0/286219384 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5df84cd90 con 0x55d5dfacddd0 2023-07-21T19:06:42.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99eed9d700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99ee59c700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 0x7f99d80a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006010 tx=0x7f99e0009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.628+0000 7f99edd9b700 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dface690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007c30 tx=0x7f99e4007f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8f44c6979bbe47f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.629+0000 7f99dcff9700 1 -- 172.21.15.5:0/286219384 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4010ca0 con 0x55d5dfacddd0 2023-07-21T19:06:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.629+0000 7f99dcff9700 1 -- 172.21.15.5:0/286219384 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99e40044e0 con 0x55d5dfacddd0 2023-07-21T19:06:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.629+0000 7f99dcff9700 1 -- 172.21.15.5:0/286219384 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e4005020 con 0x55d5dfacddd0 2023-07-21T19:06:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.629+0000 7f9a0841ab80 1 -- 172.21.15.5:0/286219384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 msgr2=0x55d5dface690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.629+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dface690 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f99e4007c30 tx=0x7f99e4007f40 comp rx=0 tx=0).stop 2023-07-21T19:06:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 -- 172.21.15.5:0/286219384 shutdown_connections 2023-07-21T19:06:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfacf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacd480 0x55d5dfacd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/286219384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dface690 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 -- 172.21.15.5:0/286219384 >> 172.21.15.5:0/286219384 conn(0x55d5dfac2d50 msgr2=0x55d5dfac8180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 -- 172.21.15.5:0/286219384 shutdown_connections 2023-07-21T19:06:42.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 -- 172.21.15.5:0/286219384 wait complete. 2023-07-21T19:06:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 -- start start 2023-07-21T19:06:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.630+0000 7f9a0841ab80 1 Processor -- start 2023-07-21T19:06:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfade6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5df9bea70 con 0x55d5dfacddd0 2023-07-21T19:06:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5df9be770 con 0x55d5dfacd480 2023-07-21T19:06:42.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5df8a28d0 con 0x55d5dfacec20 2023-07-21T19:06:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f9a0841ab80 10 client.0 _ll_register_callbacks cb 0x55d5df9bc710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f99edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f99edd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49694/0 (socket says 172.21.15.5:49694) 2023-07-21T19:06:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f99edd9b700 1 -- 172.21.15.5:0/713291012 learned_addr learned my addr 172.21.15.5:0/713291012 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.631+0000 7f99eed9d700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfade6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 -- 172.21.15.5:0/713291012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 msgr2=0x55d5dfade6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfade6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 -- 172.21.15.5:0/713291012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 msgr2=0x55d5dfad7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 -- 172.21.15.5:0/713291012 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e8013820 con 0x55d5dfacd480 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99edd9b700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99eed9d700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad7120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009ec0 tx=0x7f99e800b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95b723d1fe4735d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800e040 con 0x55d5dfacd480 2023-07-21T19:06:42.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99e8003e10 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d5df9784a0 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e8004bb0 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f9a0841ab80 10 client.24989 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.632+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f99e4007890 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d5df9784a0 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99e801b470 con 0x55d5dfacd480 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 10 client.24989 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 10 client.24989 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 20 client.24989 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfae2400 0x55d5dfae1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 4 client.24989 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d5dfa17e60 con 0x55d5dfae2400 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.633+0000 7f99edd9b700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfae2400 0x55d5dfae1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.634+0000 7f99d5ffb700 10 client.24989 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.634+0000 7f99edd9b700 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfae2400 0x55d5dfae1800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ed0f026234929ba0 server_cookie=3258df0b07d3d11f in_seq=0 out_seq=0 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f99d5ffb700 1 -- 172.21.15.5:0/713291012 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d5dfa17e60 con 0x55d5dfae2400 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f99d5ffb700 10 client.24989 handle_command_reply: tid=0 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 client.24989 shutdown 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 msgr2=0x55d5dfad7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad7120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009ec0 tx=0x7f99e800b040 comp rx=0 tx=0).stop 2023-07-21T19:06:42.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 shutdown_connections 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacec20 0x55d5dfade6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfae2400 0x55d5dfae1800 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad7120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/713291012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacddd0 0x55d5dfad7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f99d5ffb700 0 client.24989 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 >> 172.21.15.5:0/713291012 conn(0x55d5dfac2d50 msgr2=0x55d5dfac7860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 shutdown_connections 2023-07-21T19:06:42.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.635+0000 7f9a0841ab80 1 -- 172.21.15.5:0/713291012 wait complete. 2023-07-21T19:06:42.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.636+0000 7f9a0841ab80 20 client.24989 trim_cache size 0 max 16384 2023-07-21T19:06:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.655+0000 7f9a0841ab80 1 Processor -- start 2023-07-21T19:06:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.655+0000 7f9a0841ab80 1 -- start start 2023-07-21T19:06:42.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.655+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacd480 0x55d5dfad5030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dfade660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfad6cf0 con 0x55d5dfacec20 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfad8310 con 0x55d5dfacddd0 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5df9ed790 con 0x55d5dfacd480 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49702/0 (socket says 172.21.15.5:49702) 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 -- 172.21.15.5:0/253858612 learned_addr learned my addr 172.21.15.5:0/253858612 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99edd9b700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dfade660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacd480 0x55d5dfad5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 -- 172.21.15.5:0/253858612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacd480 msgr2=0x55d5dfad5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacd480 0x55d5dfad5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 -- 172.21.15.5:0/253858612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 msgr2=0x55d5dfade660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dfade660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99eed9d700 1 -- 172.21.15.5:0/253858612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d5df835c10 con 0x55d5dfacec20 2023-07-21T19:06:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.656+0000 7f99edd9b700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dfade660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.657+0000 7f99eed9d700 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99e8014010 tx=0x7f99e8013f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d60e21ad1b6f5a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.657+0000 7f99dd7fa700 1 -- 172.21.15.5:0/253858612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800e040 con 0x55d5dfacec20 2023-07-21T19:06:42.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.657+0000 7f99dd7fa700 1 -- 172.21.15.5:0/253858612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99e800f040 con 0x55d5dfacec20 2023-07-21T19:06:42.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.657+0000 7f99dd7fa700 1 -- 172.21.15.5:0/253858612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e801a9d0 con 0x55d5dfacec20 2023-07-21T19:06:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:42 smithi005 ceph-mon[126460]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:42 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/596678630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:42 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2633934678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 -- 172.21.15.5:0/253858612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 msgr2=0x55d5dfae1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f99e8014010 tx=0x7f99e8013f60 comp rx=0 tx=0).stop 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 -- 172.21.15.5:0/253858612 shutdown_connections 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfae1800 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacd480 0x55d5dfad5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/253858612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacddd0 0x55d5dfade660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 -- 172.21.15.5:0/253858612 >> 172.21.15.5:0/253858612 conn(0x55d5dfac2d50 msgr2=0x55d5dfad5e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 -- 172.21.15.5:0/253858612 shutdown_connections 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.658+0000 7f9a0841ab80 1 -- 172.21.15.5:0/253858612 wait complete. 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.659+0000 7f9a0841ab80 1 -- start start 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.659+0000 7f9a0841ab80 1 Processor -- start 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 0x55d5dfad9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfb36260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfada430 con 0x55d5dfacec20 2023-07-21T19:06:42.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfb367a0 con 0x55d5dfacd480 2023-07-21T19:06:42.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5dfb478f0 con 0x55d5dfacddd0 2023-07-21T19:06:42.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f9a0841ab80 10 client.0 _ll_register_callbacks cb 0x7f99e8013a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f99eed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51112/0 (socket says 172.21.15.5:51112) 2023-07-21T19:06:42.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f99eed9d700 1 -- 172.21.15.5:0/3771390826 learned_addr learned my addr 172.21.15.5:0/3771390826 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.660+0000 7f99edd9b700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 0x55d5dfad9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 -- 172.21.15.5:0/3771390826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 msgr2=0x55d5dfad9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 0x55d5dfad9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 -- 172.21.15.5:0/3771390826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 msgr2=0x55d5dfb36260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfb36260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 -- 172.21.15.5:0/3771390826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99e4007890 con 0x55d5dfacd480 2023-07-21T19:06:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99edd9b700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 0x55d5dfad9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99eed9d700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098f0 tx=0x7f99e80137a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5662ae5777be8b00 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99d5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:06:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e800e070 con 0x55d5dfacd480 2023-07-21T19:06:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f99e801b9f0 con 0x55d5dfacd480 2023-07-21T19:06:42.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.661+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f99e40044e0 con 0x55d5dfacd480 2023-07-21T19:06:42.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99e803a070 con 0x55d5dfacd480 2023-07-21T19:06:42.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 10 client.24995 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d5df873200 con 0x55d5dfacd480 2023-07-21T19:06:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f99e8043030 con 0x55d5dfacd480 2023-07-21T19:06:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f99e8029590 con 0x55d5dfacd480 2023-07-21T19:06:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 10 client.24995 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 10 client.24995 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T19:06:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 20 client.24995 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.662+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfb4c260 0x55d5dfb48bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.663+0000 7f9a0841ab80 4 client.24995 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-07-21T19:06:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.663+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d5df9da000 con 0x55d5dfb4c260 2023-07-21T19:06:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.663+0000 7f99edd9b700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfb4c260 0x55d5dfb48bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.663+0000 7f99edd9b700 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfb4c260 0x55d5dfb48bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2728117cd2baf3c server_cookie=3ad0077f73f928fb in_seq=0 out_seq=0 2023-07-21T19:06:42.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.663+0000 7f99d5ffb700 10 client.24995 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:06:42.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f99d5ffb700 1 -- 172.21.15.5:0/3771390826 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d5df9da000 con 0x55d5dfb4c260 2023-07-21T19:06:42.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f99d5ffb700 10 client.24995 handle_command_reply: tid=0 2023-07-21T19:06:42.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 client.24995 shutdown 2023-07-21T19:06:42.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 msgr2=0x55d5dfad98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f99e80098f0 tx=0x7f99e80137a0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 shutdown_connections 2023-07-21T19:06:42.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5dfacec20 0x55d5dfb36260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55d5dfb4c260 0x55d5dfb48bb0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5dfacd480 0x55d5dfad98a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 --2- 172.21.15.5:0/3771390826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5dfacddd0 0x55d5dfad9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f99d5ffb700 0 client.24995 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:06:42.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 >> 172.21.15.5:0/3771390826 conn(0x55d5dfac2d50 msgr2=0x55d5dfad5030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 shutdown_connections 2023-07-21T19:06:42.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 1 -- 172.21.15.5:0/3771390826 wait complete. 2023-07-21T19:06:42.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.664+0000 7f9a0841ab80 20 client.24995 trim_cache size 0 max 16384 2023-07-21T19:06:42.691 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:06:42.691 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:06:42.691 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2023-07-21T19:06:42.691 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24566, 2023-07-21T19:06:42.691 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 61.567735349000003, 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:06:42.692 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 84.456290241000005 2023-07-21T19:06:42.693 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:06:42.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 msgr2=0x7f99d80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 0x7f99d80a3710 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f99e0006010 tx=0x7f99e0009310 comp rx=0 tx=0).stop 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 msgr2=0x7f99f019bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f99e00098e0 tx=0x7f99e0004730 comp rx=0 tx=0).stop 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 shutdown_connections 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f99d80a1260 0x7f99d80a3710 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f99f010c0b0 0x7f99f01a33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f99f0072f70 0x7f99f019bdf0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 --2- 172.21.15.5:0/3972657326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f99f0074550 0x7f99f019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 >> 172.21.15.5:0/3972657326 conn(0x7f99f00fb0e0 msgr2=0x7f99f00fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 shutdown_connections 2023-07-21T19:06:42.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.665+0000 7f99f49ff700 1 -- 172.21.15.5:0/3972657326 wait complete. 2023-07-21T19:06:42.696 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T19:06:42.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:42 smithi101 ceph-mon[128136]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:42 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/596678630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:42 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2633934678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 -- 172.21.15.5:0/907746213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954105800 msgr2=0x7f2954107c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 --2- 172.21.15.5:0/907746213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954105800 0x7f2954107c60 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b0a0 tx=0x7f294c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 -- 172.21.15.5:0/907746213 shutdown_connections 2023-07-21T19:06:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 --2- 172.21.15.5:0/907746213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954105800 0x7f2954107c60 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 --2- 172.21.15.5:0/907746213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 0x7f29541029a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 --2- 172.21.15.5:0/907746213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954102ee0 0x7f29541052c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 -- 172.21.15.5:0/907746213 >> 172.21.15.5:0/907746213 conn(0x7f29540fa060 msgr2=0x7f29540fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 -- 172.21.15.5:0/907746213 shutdown_connections 2023-07-21T19:06:42.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.912+0000 7f29597ff700 1 -- 172.21.15.5:0/907746213 wait complete. 2023-07-21T19:06:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.913+0000 7f29597ff700 1 Processor -- start 2023-07-21T19:06:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.913+0000 7f29597ff700 1 -- start start 2023-07-21T19:06:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 0x7f2954197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105800 0x7f295419efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419f590 con 0x7f2954102ee0 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419f6d0 con 0x7f29541005c0 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29597ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295419f860 con 0x7f2954105800 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49716/0 (socket says 172.21.15.5:49716) 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 -- 172.21.15.5:0/1027292396 learned_addr learned my addr 172.21.15.5:0/1027292396 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 0x7f2954197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29537fe700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105800 0x7f295419efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 -- 172.21.15.5:0/1027292396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105800 msgr2=0x7f295419efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105800 0x7f295419efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 -- 172.21.15.5:0/1027292396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 msgr2=0x7f2954197a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 0x7f2954197a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.914+0000 7f29527fc700 1 -- 172.21.15.5:0/1027292396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f294c009d00 con 0x7f2954102ee0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f29527fc700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f294800e9a0 tx=0x7f294800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f293bfff700 1 -- 172.21.15.5:0/1027292396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2948009790 con 0x7f2954102ee0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295419fb40 con 0x7f2954102ee0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f293bfff700 1 -- 172.21.15.5:0/1027292396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2948010430 con 0x7f2954102ee0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f293bfff700 1 -- 172.21.15.5:0/1027292396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800cea0 con 0x7f2954102ee0 2023-07-21T19:06:42.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.915+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29541a0050 con 0x7f2954102ee0 2023-07-21T19:06:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:42 smithi156 ceph-mon[128316]: pgmap v334: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:42 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/596678630' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:42 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2633934678' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:06:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.918+0000 7f293bfff700 1 -- 172.21.15.5:0/1027292396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f29480105a0 con 0x7f2954102ee0 2023-07-21T19:06:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.919+0000 7f296d05db80 1 Processor -- start 2023-07-21T19:06:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 -- start start 2023-07-21T19:06:42.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f293bfff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 0x7f293c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f293bfff700 1 -- 172.21.15.5:0/1027292396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2948014070 con 0x7f2954102ee0 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df7e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df7f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df80090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de83c60 con 0x55c15df7e480 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15ded69f0 con 0x55c15df7edd0 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f296d05db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15decfe80 con 0x55c15df7fc20 2023-07-21T19:06:42.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f29537fe700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 0x7f293c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f29527fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df7f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f2952ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f29527fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df7f690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51130/0 (socket says 172.21.15.5:51130) 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f29527fc700 1 -- 172.21.15.5:0/3214381938 learned_addr learned my addr 172.21.15.5:0/3214381938 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.920+0000 7f29537fe700 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df80090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 -- 172.21.15.5:0/3214381938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 msgr2=0x55c15df80090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df80090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 -- 172.21.15.5:0/3214381938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 msgr2=0x55c15df7f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df7f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 -- 172.21.15.5:0/3214381938 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15dcfdd90 con 0x55c15df7e480 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f29537fe700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 0x7f293c0a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b780 tx=0x7f294c007700 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f2952ffd700 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df7e890 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2944009cb0 tx=0x7f2944011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8704235c2578ffd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f29397fa700 1 -- 172.21.15.5:0/3214381938 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944011d20 con 0x55c15df7e480 2023-07-21T19:06:42.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f29397fa700 1 -- 172.21.15.5:0/3214381938 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2944011e80 con 0x55c15df7e480 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.921+0000 7f29397fa700 1 -- 172.21.15.5:0/3214381938 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29440193e0 con 0x55c15df7e480 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 -- 172.21.15.5:0/3214381938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 msgr2=0x55c15df7e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df7e890 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2944009cb0 tx=0x7f2944011440 comp rx=0 tx=0).stop 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 -- 172.21.15.5:0/3214381938 shutdown_connections 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df80090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df7e890 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 --2- 172.21.15.5:0/3214381938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df7f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 -- 172.21.15.5:0/3214381938 >> 172.21.15.5:0/3214381938 conn(0x55c15df73d50 msgr2=0x55c15df79180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 -- 172.21.15.5:0/3214381938 shutdown_connections 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.922+0000 7f296d05db80 1 -- 172.21.15.5:0/3214381938 wait complete. 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 -- start start 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 Processor -- start 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de6fa70 con 0x55c15df7fc20 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de6f770 con 0x55c15df7e480 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd538d0 con 0x55c15df7edd0 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.923+0000 7f296d05db80 10 client.0 _ll_register_callbacks cb 0x55c15de6d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f2952ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29527fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29527fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41980/0 (socket says 172.21.15.5:41980) 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f2952ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51140/0 (socket says 172.21.15.5:51140) 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29527fc700 1 -- 172.21.15.5:0/2652990558 learned_addr learned my addr 172.21.15.5:0/2652990558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49728/0 (socket says 172.21.15.5:49728) 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 -- 172.21.15.5:0/2652990558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 msgr2=0x55c15df88620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 -- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 msgr2=0x55c15df880e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 -- 172.21.15.5:0/2652990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29440098e0 con 0x55c15df7fc20 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29527fc700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f2952ffd700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f29537fe700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b070 tx=0x7f294c00b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca3a434ec54c001d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.924+0000 7f2936ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c00e040 con 0x55c15df7fc20 2023-07-21T19:06:42.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c15dec8e60 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f294c007e30 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c003e10 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 10 client.15159 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c15dce6c10 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c15dec8e60 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f294c005040 con 0x55c15df7fc20 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 10 client.15159 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 10 client.15159 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T19:06:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 20 client.15159 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.925+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15df933b0 0x55c15df92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.926+0000 7f296d05db80 4 client.15159 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:06:42.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.926+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c15de8b000 con 0x55c15df933b0 2023-07-21T19:06:42.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.926+0000 7f2952ffd700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15df933b0 0x55c15df92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.926+0000 7f2952ffd700 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15df933b0 0x55c15df92760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd79049af286350 server_cookie=134c972b457f4a86 in_seq=0 out_seq=0 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.926+0000 7f2936ff5700 10 client.15159 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f2936ff5700 1 -- 172.21.15.5:0/2652990558 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c15de8b000 con 0x55c15df933b0 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f2936ff5700 10 client.15159 handle_command_reply: tid=0 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 client.15159 shutdown 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 msgr2=0x55c15df8f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b070 tx=0x7f294c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 shutdown_connections 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15df933b0 0x55c15df92760 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7fc20 0x55c15df8f660 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df880e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 --2- 172.21.15.5:0/2652990558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7edd0 0x55c15df88620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f2936ff5700 0 client.15159 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 >> 172.21.15.5:0/2652990558 conn(0x55c15df73d50 msgr2=0x55c15df78860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 shutdown_connections 2023-07-21T19:06:42.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 1 -- 172.21.15.5:0/2652990558 wait complete. 2023-07-21T19:06:42.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.928+0000 7f296d05db80 20 client.15159 trim_cache size 0 max 16384 2023-07-21T19:06:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 Processor -- start 2023-07-21T19:06:42.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 -- start start 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df92ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de79470 con 0x55c15df7e480 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15de9e790 con 0x55c15df7edd0 2023-07-21T19:06:42.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f296d05db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15dd5fde0 con 0x55c15df7fc20 2023-07-21T19:06:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.947+0000 7f2952ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49742/0 (socket says 172.21.15.5:49742) 2023-07-21T19:06:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 -- 172.21.15.5:0/1673955940 learned_addr learned my addr 172.21.15.5:0/1673955940 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f29527fc700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df92ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f29537fe700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 -- 172.21.15.5:0/1673955940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 msgr2=0x55c15df8f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 -- 172.21.15.5:0/1673955940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 msgr2=0x55c15df92ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df92ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 -- 172.21.15.5:0/1673955940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c15dd24200 con 0x55c15df7e480 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f29527fc700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df92ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2944009cb0 tx=0x7f2944003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9a72cbf9676629 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2939ffb700 1 -- 172.21.15.5:0/1673955940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944003b60 con 0x55c15df7e480 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2939ffb700 1 -- 172.21.15.5:0/1673955940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2944005550 con 0x55c15df7e480 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.948+0000 7f2939ffb700 1 -- 172.21.15.5:0/1673955940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294401a930 con 0x55c15df7e480 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- 172.21.15.5:0/1673955940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 msgr2=0x55c15df92760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2944009cb0 tx=0x7f2944003950 comp rx=0 tx=0).stop 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- 172.21.15.5:0/1673955940 shutdown_connections 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7e480 0x55c15df92760 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 --2- 172.21.15.5:0/1673955940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7edd0 0x55c15df92ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- 172.21.15.5:0/1673955940 >> 172.21.15.5:0/1673955940 conn(0x55c15df73d50 msgr2=0x55c15df78860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- 172.21.15.5:0/1673955940 shutdown_connections 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- 172.21.15.5:0/1673955940 wait complete. 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.949+0000 7f296d05db80 1 -- start start 2023-07-21T19:06:42.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 Processor -- start 2023-07-21T19:06:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df8d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15df8e9c0 con 0x55c15df7edd0 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15df8eb00 con 0x55c15df7e480 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c15df8ecc0 con 0x55c15df7fc20 2023-07-21T19:06:42.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f296d05db80 10 client.0 _ll_register_callbacks cb 0x55c15de6d710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f29527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f29527fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49752/0 (socket says 172.21.15.5:49752) 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f29537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f29527fc700 1 -- 172.21.15.5:0/1088532556 learned_addr learned my addr 172.21.15.5:0/1088532556 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.950+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df8d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 -- 172.21.15.5:0/1088532556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 msgr2=0x55c15df8e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 -- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 msgr2=0x55c15df8d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df8d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 -- 172.21.15.5:0/1088532556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294401e040 con 0x55c15df7edd0 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f29527fc700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2948096960 tx=0x7f2948099710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f61d2354105fa0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f2936ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294800cc10 con 0x55c15df7edd0 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f294809df70 con 0x55c15df7edd0 2023-07-21T19:06:42.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c15de294a0 con 0x55c15df7edd0 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29480a2390 con 0x55c15df7edd0 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f296d05db80 10 client.15165 fetch_fsmap learned FSMap version 23 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.951+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55c15dcfc260 con 0x55c15df7edd0 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c15de294a0 con 0x55c15df7edd0 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f2948095f30 con 0x55c15df7edd0 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 10 client.15165 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 10 client.15165 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T19:06:42.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 20 client.15165 populate_metadata read hostname 'smithi005' 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15dfed240 0x55c15dfe9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 4 client.15165 mds_command: new command op to 24614 tid=0[{"prefix": "status"}] 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c15dd1ef20 con 0x55c15dfed240 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.952+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15dfed240 0x55c15dfe9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f2952ffd700 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15dfed240 0x55c15dfe9b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=60384f5b20e42c3f server_cookie=f086d735871e5ed2 in_seq=0 out_seq=0 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f2936ff5700 10 client.15165 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:06:42.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f2936ff5700 1 -- 172.21.15.5:0/1088532556 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55c15dd1ef20 con 0x55c15dfed240 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f2936ff5700 10 client.15165 handle_command_reply: tid=0 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 client.15165 shutdown 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 msgr2=0x55c15df8dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f2948096960 tx=0x7f2948099710 comp rx=0 tx=0).stop 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 shutdown_connections 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c15df7fc20 0x55c15df8e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c15df7e480 0x55c15df8d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c15dfed240 0x55c15dfe9b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 --2- 172.21.15.5:0/1088532556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c15df7edd0 0x55c15df8dda0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f2936ff5700 0 client.15165 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.953+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 >> 172.21.15.5:0/1088532556 conn(0x55c15df73d50 msgr2=0x55c15df92760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 shutdown_connections 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f296d05db80 1 -- 172.21.15.5:0/1088532556 wait complete. 2023-07-21T19:06:42.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f296d05db80 20 client.15165 trim_cache size 0 max 16384 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24614, 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:06:42.973 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 65.274935718999998, 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 72.396545821000004 2023-07-21T19:06:42.974 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:06:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 msgr2=0x7f293c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 0x7f293c0a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f294c00b780 tx=0x7f294c007700 comp rx=0 tx=0).stop 2023-07-21T19:06:42.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 msgr2=0x7f2954197f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f294800e9a0 tx=0x7f294800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 shutdown_connections 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f293c0a11e0 0x7f293c0a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2954105800 0x7f295419efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f29541005c0 0x7f2954197a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 --2- 172.21.15.5:0/1027292396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2954102ee0 0x7f2954197f50 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:42.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.954+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 >> 172.21.15.5:0/1027292396 conn(0x7f29540fa060 msgr2=0x7f29540feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:42.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.955+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 shutdown_connections 2023-07-21T19:06:42.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:42.955+0000 7f29597ff700 1 -- 172.21.15.5:0/1027292396 wait complete. 2023-07-21T19:06:42.983 INFO:tasks.cephadm:Setup complete, yielding 2023-07-21T19:06:42.983 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T19:06:42.993 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:06:42.994 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph orch status' 2023-07-21T19:06:43.733 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.585 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.586 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:44 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:44 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: pgmap v335: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:44.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:44 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:45.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 -- 172.21.15.5:0/4285669506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 msgr2=0x7f1ae40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/4285669506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae40fffa0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f1ae0009990 tx=0x7f1ae0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 -- 172.21.15.5:0/4285669506 shutdown_connections 2023-07-21T19:06:45.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/4285669506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4101720 0x7f1ae4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/4285669506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae40fffa0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/4285669506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4100d90 0x7f1ae41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 -- 172.21.15.5:0/4285669506 >> 172.21.15.5:0/4285669506 conn(0x7f1ae40fb1a0 msgr2=0x7f1ae40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 -- 172.21.15.5:0/4285669506 shutdown_connections 2023-07-21T19:06:45.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.226+0000 7f1aeb63a700 1 -- 172.21.15.5:0/4285669506 wait complete. 2023-07-21T19:06:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.227+0000 7f1aeb63a700 1 Processor -- start 2023-07-21T19:06:45.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 -- start start 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4100d90 0x7f1ae4195d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 0x7f1ae419cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae419d390 con 0x7f1ae40ffb90 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae419d4d0 con 0x7f1ae4100d90 2023-07-21T19:06:45.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1aeb63a700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae419d660 con 0x7f1ae4101720 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1ae93d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1ae93d6700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49770/0 (socket says 172.21.15.5:49770) 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1ae93d6700 1 -- 172.21.15.5:0/3541029139 learned_addr learned my addr 172.21.15.5:0/3541029139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.228+0000 7f1ae8bd5700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4100d90 0x7f1ae4195d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae9bd7700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 0x7f1ae419cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 -- 172.21.15.5:0/3541029139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 msgr2=0x7f1ae419cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:45.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 0x7f1ae419cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 -- 172.21.15.5:0/3541029139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4100d90 msgr2=0x7f1ae4195d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4100d90 0x7f1ae4195d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 -- 172.21.15.5:0/3541029139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae00095f0 con 0x7f1ae40ffb90 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae9bd7700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 0x7f1ae419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ae93d6700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098e0 tx=0x7f1ae0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae001d070 con 0x7f1ae40ffb90 2023-07-21T19:06:45.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ae000beb0 con 0x7f1ae40ffb90 2023-07-21T19:06:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae419d8e0 con 0x7f1ae40ffb90 2023-07-21T19:06:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0022e50 con 0x7f1ae40ffb90 2023-07-21T19:06:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.229+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae419dcf0 con 0x7f1ae40ffb90 2023-07-21T19:06:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.231+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae410b250 con 0x7f1ae40ffb90 2023-07-21T19:06:45.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.233+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1ae0022650 con 0x7f1ae40ffb90 2023-07-21T19:06:45.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.234+0000 7f1ada7fc700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 0x7f1ad00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.234+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f1ae00cdf20 con 0x7f1ae40ffb90 2023-07-21T19:06:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.234+0000 7f1ae8bd5700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 0x7f1ad00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.235+0000 7f1ae8bd5700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 0x7f1ad00a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009c60 tx=0x7f1ad4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:45.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.238+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ae005c790 con 0x7f1ae40ffb90 2023-07-21T19:06:45.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.462+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ae4060950 con 0x7f1ad00a11e0 2023-07-21T19:06:45.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.463+0000 7f1ada7fc700 1 -- 172.21.15.5:0/3541029139 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1ae4060950 con 0x7f1ad00a11e0 2023-07-21T19:06:45.473 INFO:teuthology.orchestra.run.smithi005.stdout:Backend: cephadm 2023-07-21T19:06:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:Available: Yes 2023-07-21T19:06:45.474 INFO:teuthology.orchestra.run.smithi005.stdout:Paused: No 2023-07-21T19:06:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 msgr2=0x7f1ad00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:45.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 0x7f1ad00a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009c60 tx=0x7f1ad4009260 comp rx=0 tx=0).stop 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 msgr2=0x7f1ae4195800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f1ae00098e0 tx=0x7f1ae0005740 comp rx=0 tx=0).stop 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 shutdown_connections 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f1ad00a11e0 0x7f1ad00a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae4101720 0x7f1ae419cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae40ffb90 0x7f1ae4195800 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 --2- 172.21.15.5:0/3541029139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae4100d90 0x7f1ae4195d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.465+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 >> 172.21.15.5:0/3541029139 conn(0x7f1ae40fb1a0 msgr2=0x7f1ae40fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.466+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 shutdown_connections 2023-07-21T19:06:45.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:45.466+0000 7f1aeb63a700 1 -- 172.21.15.5:0/3541029139 wait complete. 2023-07-21T19:06:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:45.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:45.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:45 smithi101 ceph-mon[128136]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:45.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:45.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:45.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:45 smithi005 ceph-mon[126460]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:45.928 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ps' 2023-07-21T19:06:45.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:06:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:06:45.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:45 smithi156 ceph-mon[128316]: from='mgr.14152 172.21.15.5:0/1786384506' entity='mgr.x' 2023-07-21T19:06:46.662 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:46.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:46 smithi101 ceph-mon[128136]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:46 smithi101 ceph-mon[128136]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:46.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:46 smithi005 ceph-mon[126460]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:46.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:46 smithi005 ceph-mon[126460]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:46 smithi156 ceph-mon[128316]: pgmap v336: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:46.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:46 smithi156 ceph-mon[128316]: from='client.15171 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:48.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 -- 172.21.15.5:0/3456650469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f40ffb90 msgr2=0x7f67f40fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:48.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 --2- 172.21.15.5:0/3456650469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f40ffb90 0x7f67f40fffa0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f67e8009990 tx=0x7f67e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 -- 172.21.15.5:0/3456650469 shutdown_connections 2023-07-21T19:06:48.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 --2- 172.21.15.5:0/3456650469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f4101720 0x7f67f4107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 --2- 172.21.15.5:0/3456650469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f40ffb90 0x7f67f40fffa0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 --2- 172.21.15.5:0/3456650469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4100d90 0x7f67f41011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 -- 172.21.15.5:0/3456650469 >> 172.21.15.5:0/3456650469 conn(0x7f67f40fb1a0 msgr2=0x7f67f40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 -- 172.21.15.5:0/3456650469 shutdown_connections 2023-07-21T19:06:48.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.042+0000 7f67fb050700 1 -- 172.21.15.5:0/3456650469 wait complete. 2023-07-21T19:06:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 Processor -- start 2023-07-21T19:06:48.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 -- start start 2023-07-21T19:06:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f40ffb90 0x7f67f40742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:48.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4101720 0x7f67f4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:48.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f40735c0 con 0x7f67f4100d90 2023-07-21T19:06:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f4073730 con 0x7f67f40ffb90 2023-07-21T19:06:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.043+0000 7f67fb050700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67f419f810 con 0x7f67f4101720 2023-07-21T19:06:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57432/0 (socket says 172.21.15.5:57432) 2023-07-21T19:06:48.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 -- 172.21.15.5:0/2742538169 learned_addr learned my addr 172.21.15.5:0/2742538169 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f8dec700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f40ffb90 0x7f67f40742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f95ed700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4101720 0x7f67f4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:48.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 -- 172.21.15.5:0/2742538169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4101720 msgr2=0x7f67f4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4101720 0x7f67f4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 -- 172.21.15.5:0/2742538169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f40ffb90 msgr2=0x7f67f40742b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f40ffb90 0x7f67f40742b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 -- 172.21.15.5:0/2742538169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67e80095f0 con 0x7f67f4100d90 2023-07-21T19:06:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f3fff700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67e000d950 tx=0x7f67e000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:48.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.044+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e0009940 con 0x7f67f4100d90 2023-07-21T19:06:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.045+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67e0010460 con 0x7f67f4100d90 2023-07-21T19:06:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.045+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67f419faf0 con 0x7f67f4100d90 2023-07-21T19:06:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.045+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67e000f620 con 0x7f67f4100d90 2023-07-21T19:06:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.045+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67f419fff0 con 0x7f67f4100d90 2023-07-21T19:06:48.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.046+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67f4107170 con 0x7f67f4100d90 2023-07-21T19:06:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.048+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f67e00105d0 con 0x7f67f4100d90 2023-07-21T19:06:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.049+0000 7f67f1ffb700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 0x7f67e40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.049+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f67e00cc960 con 0x7f67f4100d90 2023-07-21T19:06:48.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.049+0000 7f67f8dec700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 0x7f67e40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.050+0000 7f67f8dec700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 0x7f67e40a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e800b560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:48.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.053+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f67e0095f00 con 0x7f67f4100d90 2023-07-21T19:06:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.271+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f67f4060950 con 0x7f67e40a11e0 2023-07-21T19:06:48.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.284+0000 7f67f1ffb700 1 -- 172.21.15.5:0/2742538169 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7f67f4060950 con 0x7f67e40a11e0 2023-07-21T19:06:48.295 INFO:teuthology.orchestra.run.smithi005.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-07-21T19:06:48.295 INFO:teuthology.orchestra.run.smithi005.stdout:mds.a smithi005 running (81s) 4s ago 80s 13.6M - 18.0.0-4746-g81cf785a 66a4f7b015b5 60f978f4fac5 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.b smithi101 running (78s) 4s ago 77s 28.9M - 18.0.0-4746-g81cf785a 66a4f7b015b5 d2429d563f0f 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.c smithi156 running (93s) 4s ago 93s 16.9M - 18.0.0-4746-g81cf785a 66a4f7b015b5 adcf8fc2b4eb 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.d smithi005 running (103s) 4s ago 102s 13.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 5c88db6a6720 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.e smithi101 running (99s) 4s ago 99s 18.2M - 18.0.0-4746-g81cf785a 66a4f7b015b5 a56c2826a016 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.f smithi156 running (96s) 4s ago 96s 16.9M - 18.0.0-4746-g81cf785a 66a4f7b015b5 40c926eb5292 2023-07-21T19:06:48.296 INFO:teuthology.orchestra.run.smithi005.stdout:mds.g smithi005 running (84s) 4s ago 83s 13.7M - 18.0.0-4746-g81cf785a 66a4f7b015b5 63251037eedb 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mds.h smithi101 running (90s) 4s ago 90s 18.3M - 18.0.0-4746-g81cf785a 66a4f7b015b5 713946f821d8 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mds.i smithi156 running (87s) 4s ago 86s 17.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 0bc95f98f265 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mds.j smithi005 running (106s) 4s ago 105s 13.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 8974ab257df2 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mds.k smithi101 running (109s) 4s ago 109s 13.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 58d65fd645c7 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mds.l smithi156 running (112s) 4s ago 112s 13.8M - 18.0.0-4746-g81cf785a 66a4f7b015b5 1fe14c3cc3c9 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.x smithi005 *:9283,8765 running (9m) 4s ago 9m 523M - 18.0.0-4746-g81cf785a 66a4f7b015b5 8d3ba823c238 2023-07-21T19:06:48.297 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.y smithi101 *:8443,8765 running (7m) 4s ago 7m 449M - 18.0.0-4746-g81cf785a 66a4f7b015b5 f53b2c591fe5 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:mgr.z smithi156 *:8443,8765 running (7m) 4s ago 7m 448M - 18.0.0-4746-g81cf785a 66a4f7b015b5 84611237cacf 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:mon.a smithi005 running (9m) 4s ago 9m 97.3M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 39dc741d8592 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:mon.b smithi101 running (7m) 4s ago 7m 64.6M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 8ea5ffd7397f 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:mon.c smithi156 running (7m) 4s ago 7m 61.9M 2048M 18.0.0-4746-g81cf785a 66a4f7b015b5 44be25cf07bf 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 smithi005 running (7m) 4s ago 7m 69.6M 4318M 18.0.0-4746-g81cf785a 66a4f7b015b5 38f6d04e92b7 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 smithi101 running (6m) 4s ago 6m 76.3M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 0f9c01c8eb5b 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 smithi156 running (6m) 4s ago 6m 73.0M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 9b86bf0a724c 2023-07-21T19:06:48.298 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 smithi005 running (5m) 4s ago 5m 63.5M 4318M 18.0.0-4746-g81cf785a 66a4f7b015b5 3ecb3d063dab 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 smithi101 running (5m) 4s ago 5m 61.1M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 d6ef63033499 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 smithi156 running (4m) 4s ago 4m 65.8M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 375d6188323e 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 smithi005 running (4m) 4s ago 4m 59.3M 4318M 18.0.0-4746-g81cf785a 66a4f7b015b5 27271941a30d 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 smithi101 running (4m) 4s ago 4m 63.8M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 a4f9863d9fd3 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 smithi156 running (3m) 4s ago 3m 61.5M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 108b4c8b2761 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 smithi005 running (3m) 4s ago 3m 56.5M 4318M 18.0.0-4746-g81cf785a 66a4f7b015b5 c1e44899093c 2023-07-21T19:06:48.299 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 smithi101 running (2m) 4s ago 2m 62.0M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 5727fd6a01f4 2023-07-21T19:06:48.300 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 smithi156 running (2m) 4s ago 2m 51.7M 4316M 18.0.0-4746-g81cf785a 66a4f7b015b5 3791a5305476 2023-07-21T19:06:48.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 msgr2=0x7f67e40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 0x7f67e40a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f67e8000c00 tx=0x7f67e800b560 comp rx=0 tx=0).stop 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 msgr2=0x7f67f4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f67e000d950 tx=0x7f67e000dc60 comp rx=0 tx=0).stop 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 shutdown_connections 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f67e40a11e0 0x7f67e40a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f67f4101720 0x7f67f4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f67f40ffb90 0x7f67f40742b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 --2- 172.21.15.5:0/2742538169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f67f4100d90 0x7f67f4072900 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:48.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 >> 172.21.15.5:0/2742538169 conn(0x7f67f40fb1a0 msgr2=0x7f67f41042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:48.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 shutdown_connections 2023-07-21T19:06:48.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:48.287+0000 7f67fb050700 1 -- 172.21.15.5:0/2742538169 wait complete. 2023-07-21T19:06:48.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:48 smithi101 ceph-mon[128136]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:48 smithi005 ceph-mon[126460]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:48 smithi156 ceph-mon[128316]: pgmap v337: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:48.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph orch ls' 2023-07-21T19:06:49.483 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:49.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:49 smithi101 ceph-mon[128136]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:49.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:49 smithi005 ceph-mon[126460]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:49.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:49 smithi156 ceph-mon[128316]: from='client.15177 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:50.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:50 smithi101 ceph-mon[128136]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:50.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:50 smithi005 ceph-mon[126460]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:50.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:50 smithi156 ceph-mon[128316]: pgmap v338: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:51.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/758701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0107a60 msgr2=0x7ffbb0109ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/758701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0107a60 0x7ffbb0109ea0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffba800b0a0 tx=0x7ffba800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/758701634 shutdown_connections 2023-07-21T19:06:51.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/758701634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0107a60 0x7ffbb0109ea0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/758701634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0074540 0x7ffbb00729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/758701634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0072f70 0x7ffbb00733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.033+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/758701634 >> 172.21.15.5:0/758701634 conn(0x7ffbb00fb1a0 msgr2=0x7ffbb00fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:51.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.034+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/758701634 shutdown_connections 2023-07-21T19:06:51.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.034+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/758701634 wait complete. 2023-07-21T19:06:51.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.035+0000 7ffbb6e8f700 1 Processor -- start 2023-07-21T19:06:51.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.035+0000 7ffbb6e8f700 1 -- start start 2023-07-21T19:06:51.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0074540 0x7ffbb01980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0107a60 0x7ffbb019f130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f700 con 0x7ffbb0072f70 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f840 con 0x7ffbb0074540 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb6e8f700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbb019f9d0 con 0x7ffbb0107a60 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbaffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0074540 0x7ffbb01980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:51.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb4c2b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb4c2b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57440/0 (socket says 172.21.15.5:57440) 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb4c2b700 1 -- 172.21.15.5:0/1687060988 learned_addr learned my addr 172.21.15.5:0/1687060988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.036+0000 7ffbb542c700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0107a60 0x7ffbb019f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 -- 172.21.15.5:0/1687060988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0107a60 msgr2=0x7ffbb019f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0107a60 0x7ffbb019f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 -- 172.21.15.5:0/1687060988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0074540 msgr2=0x7ffbb01980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0074540 0x7ffbb01980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 -- 172.21.15.5:0/1687060988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffba8009d00 con 0x7ffbb0072f70 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb4c2b700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d980 tx=0x7ffba400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba40098d0 con 0x7ffbb0072f70 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbb019fcb0 con 0x7ffbb0072f70 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffba4010460 con 0x7ffbb0072f70 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400f620 con 0x7ffbb0072f70 2023-07-21T19:06:51.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.037+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbb01a01b0 con 0x7ffbb0072f70 2023-07-21T19:06:51.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.038+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbb0106110 con 0x7ffbb0072f70 2023-07-21T19:06:51.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.040+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ffba4009a30 con 0x7ffbb0072f70 2023-07-21T19:06:51.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.042+0000 7ffbadffb700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 0x7ffba00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:51.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.042+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ffba40cc860 con 0x7ffbb0072f70 2023-07-21T19:06:51.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.042+0000 7ffbaffff700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 0x7ffba00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:51.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.043+0000 7ffbaffff700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 0x7ffba00a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009a10 tx=0x7ffb9c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:51.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.046+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ffba4095e60 con 0x7ffbb0072f70 2023-07-21T19:06:51.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.264+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbb0060950 con 0x7ffba00a11e0 2023-07-21T19:06:51.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.269+0000 7ffbadffb700 1 -- 172.21.15.5:0/1687060988 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7ffbb0060950 con 0x7ffba00a11e0 2023-07-21T19:06:51.280 INFO:teuthology.orchestra.run.smithi005.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-07-21T19:06:51.280 INFO:teuthology.orchestra.run.smithi005.stdout:mds.all 12/12 7s ago 118s smithi005=a;smithi005=d;smithi005=g;smithi005=j;smithi101=b;smithi101=e;smithi101=h;smithi101=k;smithi156=c;smithi156=f;smithi156=i;smithi156=l;count:12 2023-07-21T19:06:51.280 INFO:teuthology.orchestra.run.smithi005.stdout:mgr 3/3 7s ago 7m smithi005=x;smithi101=y;smithi156=z;count:3 2023-07-21T19:06:51.281 INFO:teuthology.orchestra.run.smithi005.stdout:mon 3/3 7s ago 8m smithi005:172.21.15.5=a;smithi101:172.21.15.101=b;smithi156:172.21.15.156=c;count:3 2023-07-21T19:06:51.281 INFO:teuthology.orchestra.run.smithi005.stdout:osd 12 7s ago - 2023-07-21T19:06:51.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 msgr2=0x7ffba00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 0x7ffba00a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffb9c009a10 tx=0x7ffb9c009190 comp rx=0 tx=0).stop 2023-07-21T19:06:51.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 msgr2=0x7ffbb0197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:51.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7ffba400d980 tx=0x7ffba400dc90 comp rx=0 tx=0).stop 2023-07-21T19:06:51.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 shutdown_connections 2023-07-21T19:06:51.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ffba00a11e0 0x7ffba00a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffbb0107a60 0x7ffbb019f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffbb0072f70 0x7ffbb0197b70 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 --2- 172.21.15.5:0/1687060988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffbb0074540 0x7ffbb01980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 >> 172.21.15.5:0/1687060988 conn(0x7ffbb00fb1a0 msgr2=0x7ffbb00fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 shutdown_connections 2023-07-21T19:06:51.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:51.272+0000 7ffbb6e8f700 1 -- 172.21.15.5:0/1687060988 wait complete. 2023-07-21T19:06:51.765 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph orch host ls' 2023-07-21T19:06:52.496 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:52.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:52 smithi005 ceph-mon[126460]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:52.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:52 smithi005 ceph-mon[126460]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:52.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:52 smithi156 ceph-mon[128316]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:52.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:52 smithi156 ceph-mon[128316]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:52.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:52 smithi101 ceph-mon[128136]: from='client.15183 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:52.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:52 smithi101 ceph-mon[128136]: pgmap v339: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:06:54.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2837978980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd4072f70 msgr2=0x7f5fd40733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2837978980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd4072f70 0x7f5fd40733e0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5fc40099c0 tx=0x7f5fc4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2837978980 shutdown_connections 2023-07-21T19:06:54.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2837978980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd410c170 0x7f5fd410e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2837978980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4074540 0x7f5fd40729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2837978980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd4072f70 0x7f5fd40733e0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2837978980 >> 172.21.15.5:0/2837978980 conn(0x7f5fd40fb1a0 msgr2=0x7f5fd40fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2837978980 shutdown_connections 2023-07-21T19:06:54.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.060+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2837978980 wait complete. 2023-07-21T19:06:54.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.062+0000 7f5fdc5ee700 1 Processor -- start 2023-07-21T19:06:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.062+0000 7f5fdc5ee700 1 -- start start 2023-07-21T19:06:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 0x7f5fd419bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4074540 0x7f5fd419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a3900 con 0x7f5fd410c170 2023-07-21T19:06:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a3a40 con 0x7f5fd4072f70 2023-07-21T19:06:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdc5ee700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd41a3c00 con 0x7f5fd4074540 2023-07-21T19:06:54.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fda38a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 0x7f5fd419bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57464/0 (socket says 172.21.15.5:57464) 2023-07-21T19:06:54.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 -- 172.21.15.5:0/2024800862 learned_addr learned my addr 172.21.15.5:0/2024800862 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fd9b89700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4074540 0x7f5fd419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 -- 172.21.15.5:0/2024800862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4074540 msgr2=0x7f5fd419c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4074540 0x7f5fd419c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 -- 172.21.15.5:0/2024800862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 msgr2=0x7f5fd419bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.063+0000 7f5fdab8b700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 0x7f5fd419bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fdab8b700 1 -- 172.21.15.5:0/2024800862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40095f0 con 0x7f5fd410c170 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fda38a700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 0x7f5fd419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fdab8b700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc007d70 tx=0x7f5fcc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc00ad40 con 0x7f5fd410c170 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fcc0128b0 con 0x7f5fd410c170 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd41a3ee0 con 0x7f5fd410c170 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0115a0 con 0x7f5fd410c170 2023-07-21T19:06:54.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.064+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd41a42f0 con 0x7f5fd410c170 2023-07-21T19:06:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.065+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fd4101cc0 con 0x7f5fd410c170 2023-07-21T19:06:54.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.067+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5fcc012a20 con 0x7f5fd410c170 2023-07-21T19:06:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.069+0000 7f5fcb7fe700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 0x7f5fc00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.069+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fcc0ccff0 con 0x7f5fd410c170 2023-07-21T19:06:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.069+0000 7f5fda38a700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 0x7f5fc00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.070+0000 7f5fda38a700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 0x7f5fc00a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5fd419d290 tx=0x7f5fd000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:54.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.071+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5fcc0966c0 con 0x7f5fd410c170 2023-07-21T19:06:54.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.284+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f5fd4060950 con 0x7f5fc00a11e0 2023-07-21T19:06:54.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.286+0000 7f5fcb7fe700 1 -- 172.21.15.5:0/2024800862 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7f5fd4060950 con 0x7f5fc00a11e0 2023-07-21T19:06:54.296 INFO:teuthology.orchestra.run.smithi005.stdout:HOST ADDR LABELS STATUS 2023-07-21T19:06:54.297 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 172.21.15.5 2023-07-21T19:06:54.297 INFO:teuthology.orchestra.run.smithi005.stdout:smithi101 172.21.15.101 2023-07-21T19:06:54.297 INFO:teuthology.orchestra.run.smithi005.stdout:smithi156 172.21.15.156 2023-07-21T19:06:54.297 INFO:teuthology.orchestra.run.smithi005.stdout:3 hosts in cluster 2023-07-21T19:06:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 msgr2=0x7f5fc00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 0x7f5fc00a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5fd419d290 tx=0x7f5fd000b410 comp rx=0 tx=0).stop 2023-07-21T19:06:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 msgr2=0x7f5fd41a3330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:54.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc007d70 tx=0x7f5fcc00a340 comp rx=0 tx=0).stop 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 shutdown_connections 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fc00a11e0 0x7f5fc00a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd410c170 0x7f5fd41a3330 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.288+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd4072f70 0x7f5fd419bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.289+0000 7f5fdc5ee700 1 --2- 172.21.15.5:0/2024800862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd4074540 0x7f5fd419c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.289+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 >> 172.21.15.5:0/2024800862 conn(0x7f5fd40fb1a0 msgr2=0x7f5fd40fd580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:54.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.289+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 shutdown_connections 2023-07-21T19:06:54.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:54.289+0000 7f5fdc5ee700 1 -- 172.21.15.5:0/2024800862 wait complete. 2023-07-21T19:06:54.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:54 smithi005 ceph-mon[126460]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:54.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:54 smithi156 ceph-mon[128316]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:54.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph orch device ls' 2023-07-21T19:06:54.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:54 smithi101 ceph-mon[128136]: pgmap v340: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:06:55.548 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:55.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:55 smithi005 ceph-mon[126460]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:55.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:55 smithi156 ceph-mon[128316]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:55 smithi101 ceph-mon[128136]: from='client.15189 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:56.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:56 smithi005 ceph-mon[126460]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:56.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:56 smithi156 ceph-mon[128316]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:56.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:56 smithi101 ceph-mon[128136]: pgmap v341: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:06:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 -- 172.21.15.5:0/2429452070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6238101740 msgr2=0x7f6238108000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 --2- 172.21.15.5:0/2429452070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6238101740 0x7f6238108000 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f623000b0a0 tx=0x7f623000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 -- 172.21.15.5:0/2429452070 shutdown_connections 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 --2- 172.21.15.5:0/2429452070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6238101740 0x7f6238108000 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 --2- 172.21.15.5:0/2429452070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f62380ffbb0 0x7f62380fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 --2- 172.21.15.5:0/2429452070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238100db0 0x7f6238101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 -- 172.21.15.5:0/2429452070 >> 172.21.15.5:0/2429452070 conn(0x7f62380fb1c0 msgr2=0x7f62380fd630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 -- 172.21.15.5:0/2429452070 shutdown_connections 2023-07-21T19:06:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.025+0000 7f6240481700 1 -- 172.21.15.5:0/2429452070 wait complete. 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.026+0000 7f6240481700 1 Processor -- start 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 -- start start 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6238100db0 0x7f6238072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238101740 0x7f6238072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238199770 con 0x7f62380ffbb0 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238073380 con 0x7f6238100db0 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f6240481700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6238073540 con 0x7f6238101740 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f623e21d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f623e21d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56682/0 (socket says 172.21.15.5:56682) 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f623e21d700 1 -- 172.21.15.5:0/3861351435 learned_addr learned my addr 172.21.15.5:0/3861351435 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.027+0000 7f623da1c700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6238100db0 0x7f6238072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623ea1e700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238101740 0x7f6238072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 -- 172.21.15.5:0/3861351435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238101740 msgr2=0x7f6238072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238101740 0x7f6238072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 -- 172.21.15.5:0/3861351435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6238100db0 msgr2=0x7f6238072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6238100db0 0x7f6238072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 -- 172.21.15.5:0/3861351435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230009d00 con 0x7f62380ffbb0 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f623e21d700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f623400e970 tx=0x7f623400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234009790 con 0x7f62380ffbb0 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62381a4140 con 0x7f62380ffbb0 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f623400eec0 con 0x7f62380ffbb0 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6234003c10 con 0x7f62380ffbb0 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.028+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62381a4550 con 0x7f62380ffbb0 2023-07-21T19:06:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.030+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62381071d0 con 0x7f62380ffbb0 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.031+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f6234003db0 con 0x7f62380ffbb0 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.033+0000 7f622f7fe700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 0x7f62240a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.033+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6234014070 con 0x7f62380ffbb0 2023-07-21T19:06:57.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.033+0000 7f623da1c700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 0x7f62240a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.034+0000 7f623da1c700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 0x7f62240a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c60 tx=0x7f6228009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.036+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f62340962e0 con 0x7f62380ffbb0 2023-07-21T19:06:57.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.250+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6238060950 con 0x7f62240a11e0 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.251+0000 7f622f7fe700 1 -- 172.21.15.5:0/3861351435 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7f6238060950 con 0x7f62240a11e0 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi005.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi005.stdout:smithi005 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB751300EW480DGN 447G No 3m ago LVM detected, locked 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi005.stdout:smithi101 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204017B400BGN 372G No 2m ago LVM detected, locked 2023-07-21T19:06:57.262 INFO:teuthology.orchestra.run.smithi005.stdout:smithi156 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB74760040480DGN 447G No 2m ago LVM detected, locked 2023-07-21T19:06:57.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 msgr2=0x7f62240a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 0x7f62240a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6228009c60 tx=0x7f6228009260 comp rx=0 tx=0).stop 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 msgr2=0x7f62380742b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f623400e970 tx=0x7f623400ec80 comp rx=0 tx=0).stop 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 shutdown_connections 2023-07-21T19:06:57.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f62240a11e0 0x7f62240a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6238101740 0x7f6238072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f62380ffbb0 0x7f62380742b0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 --2- 172.21.15.5:0/3861351435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6238100db0 0x7f6238072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 >> 172.21.15.5:0/3861351435 conn(0x7f62380fb1c0 msgr2=0x7f6238104310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 shutdown_connections 2023-07-21T19:06:57.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:57.254+0000 7f6240481700 1 -- 172.21.15.5:0/3861351435 wait complete. 2023-07-21T19:06:57.824 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-07-21T19:06:57.833 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:06:57.834 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph fs dump' 2023-07-21T19:06:58.566 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:06:58.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:58 smithi101 ceph-mon[128136]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:58.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:06:58 smithi101 ceph-mon[128136]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:58 smithi005 ceph-mon[126460]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:58.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:06:58 smithi005 ceph-mon[126460]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:58 smithi156 ceph-mon[128316]: from='client.15195 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:06:58.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:06:58 smithi156 ceph-mon[128316]: pgmap v342: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:06:59.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 -- 172.21.15.5:0/2197673221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 msgr2=0x7f1260107fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:59.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 --2- 172.21.15.5:0/2197673221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f1260107fe0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f125800b0a0 tx=0x7f125800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 -- 172.21.15.5:0/2197673221 shutdown_connections 2023-07-21T19:06:59.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 --2- 172.21.15.5:0/2197673221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f1260107fe0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 --2- 172.21.15.5:0/2197673221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 0x7f12600fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 --2- 172.21.15.5:0/2197673221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 0x7f12601011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 -- 172.21.15.5:0/2197673221 >> 172.21.15.5:0/2197673221 conn(0x7f12600fb1a0 msgr2=0x7f12600fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:06:59.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 -- 172.21.15.5:0/2197673221 shutdown_connections 2023-07-21T19:06:59.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.908+0000 7f126680b700 1 -- 172.21.15.5:0/2197673221 wait complete. 2023-07-21T19:06:59.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.909+0000 7f126680b700 1 Processor -- start 2023-07-21T19:06:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.909+0000 7f126680b700 1 -- start start 2023-07-21T19:06:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 0x7f1260195860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 0x7f1260195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:59.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126019d3f0 con 0x7f1260101720 2023-07-21T19:06:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126019d530 con 0x7f12600ffb90 2023-07-21T19:06:59.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f126680b700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126019d6c0 con 0x7f1260100d90 2023-07-21T19:06:59.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:59.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56700/0 (socket says 172.21.15.5:56700) 2023-07-21T19:06:59.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 -- 172.21.15.5:0/1930634263 learned_addr learned my addr 172.21.15.5:0/1930634263 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:06:59.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f125ffff700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 0x7f1260195860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:59.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f125f7fe700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 0x7f1260195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:59.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 -- 172.21.15.5:0/1930634263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 msgr2=0x7f1260195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:59.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 0x7f1260195da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 -- 172.21.15.5:0/1930634263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 msgr2=0x7f1260195860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:06:59.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 0x7f1260195860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:06:59.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.910+0000 7f1264da8700 1 -- 172.21.15.5:0/1930634263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1258009d00 con 0x7f1260101720 2023-07-21T19:06:59.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f1264da8700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1258006070 tx=0x7f1258004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:59.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125800e040 con 0x7f1260101720 2023-07-21T19:06:59.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126019d940 con 0x7f1260101720 2023-07-21T19:06:59.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1258003e90 con 0x7f1260101720 2023-07-21T19:06:59.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258004020 con 0x7f1260101720 2023-07-21T19:06:59.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.911+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126019de00 con 0x7f1260101720 2023-07-21T19:06:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.912+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12601070f0 con 0x7f1260101720 2023-07-21T19:06:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.914+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f1258003a00 con 0x7f1260101720 2023-07-21T19:06:59.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.915+0000 7f125d7fa700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:06:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.916+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f12580ce020 con 0x7f1260101720 2023-07-21T19:06:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.916+0000 7f125ffff700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:06:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.916+0000 7f125ffff700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 0x7f12480a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1260100bf0 tx=0x7f125000a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:06:59.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:06:59.920+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f12580976f0 con 0x7f1260101720 2023-07-21T19:07:00.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.171+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f126002c9e0 con 0x7f1260101720 2023-07-21T19:07:00.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.171+0000 7f125d7fa700 1 -- 172.21.15.5:0/1930634263 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 23 v23) v1 ==== 76+0+2891 (secure 0 0 0) 0x7f1258011390 con 0x7f1260101720 2023-07-21T19:07:00.182 INFO:teuthology.orchestra.run.smithi005.stdout:e23 2023-07-21T19:07:00.182 INFO:teuthology.orchestra.run.smithi005.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:legacy client fscid: 1 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem 'cephfs' (1) 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:fs_name cephfs 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 23 2023-07-21T19:07:00.183 INFO:teuthology.orchestra.run.smithi005.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-07-21T19:05:37.254064+0000 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2023-07-21T19:05:43.279232+0000 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:tableserver 0 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:root 0 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:session_timeout 300 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:session_autoclose 300 2023-07-21T19:07:00.184 INFO:teuthology.orchestra.run.smithi005.stdout:max_file_size 1099511627776 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:max_xattr_size 65536 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:required_client_features {} 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure 0 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:last_failure_osd_epoch 0 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:max_mds 3 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:in 0,1,2 2023-07-21T19:07:00.185 INFO:teuthology.orchestra.run.smithi005.stdout:up {0=24614,1=24566,2=24553} 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:failed 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:damaged 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:stopped 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:data_pools [3] 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:metadata_pool 2 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:inline_data disabled 2023-07-21T19:07:00.186 INFO:teuthology.orchestra.run.smithi005.stdout:balancer 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:bal_rank_mask -1 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:standby_count_wanted 0 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.b{0:24614} state up:active seq 3 addr [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.e{0:24524} state up:standby-replay seq 1 addr [v2:172.21.15.101:6834/153908041,v1:172.21.15.101:6835/153908041] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.h{1:24566} state up:active seq 7 addr [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.c{1:24523} state up:standby-replay seq 1 addr [v2:172.21.15.156:6836/1947602937,v1:172.21.15.156:6837/1947602937] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.i{2:24553} state up:active seq 7 addr [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.187 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.f{2:24511} state up:standby-replay seq 1 addr [v2:172.21.15.156:6834/2855288918,v1:172.21.15.156:6835/2855288918] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:Standby daemons: 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.j{-1:14604} state up:standby seq 1 addr [v2:172.21.15.5:6834/3927516205,v1:172.21.15.5:6835/3927516205] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.d{-1:14622} state up:standby seq 1 addr [v2:172.21.15.5:6836/3579918721,v1:172.21.15.5:6837/3579918721] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.g{-1:14694} state up:standby seq 1 addr [v2:172.21.15.5:6838/666617424,v1:172.21.15.5:6839/666617424] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.188 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.a{-1:14706} state up:standby seq 1 addr [v2:172.21.15.5:6840/2768414598,v1:172.21.15.5:6841/2768414598] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.189 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.l{-1:24451} state up:standby seq 1 addr [v2:172.21.15.156:6832/2408636934,v1:172.21.15.156:6833/2408636934] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.189 INFO:teuthology.orchestra.run.smithi005.stdout:[mds.k{-1:24482} state up:standby seq 1 addr [v2:172.21.15.101:6832/2387040321,v1:172.21.15.101:6833/2387040321] compat {c=[1],r=[1],i=[7ff]}] 2023-07-21T19:07:00.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 msgr2=0x7f12480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 0x7f12480a3690 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f1260100bf0 tx=0x7f125000a3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 msgr2=0x7f126019ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f1258006070 tx=0x7f1258004c80 comp rx=0 tx=0).stop 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 shutdown_connections 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f12480a11e0 0x7f12480a3690 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:00.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1260101720 0x7f126019ce20 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12600ffb90 0x7f1260195860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 --2- 172.21.15.5:0/1930634263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1260100d90 0x7f1260195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 >> 172.21.15.5:0/1930634263 conn(0x7f12600fb1a0 msgr2=0x7f1260104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 shutdown_connections 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:00.174+0000 7f126680b700 1 -- 172.21.15.5:0/1930634263 wait complete. 2023-07-21T19:07:00.191 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:00.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:00 smithi005 ceph-mon[126460]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:00.610 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:00 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1930634263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T19:07:00.646 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- bash -c 'ceph osd dump' 2023-07-21T19:07:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:00 smithi156 ceph-mon[128316]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:00 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1930634263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T19:07:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:00 smithi101 ceph-mon[128136]: pgmap v343: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:00.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:00 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1930634263' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-07-21T19:07:01.378 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:07:02.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:02 smithi005 ceph-mon[126460]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:07:02.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:02 smithi156 ceph-mon[128316]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:07:02.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:02 smithi101 ceph-mon[128136]: pgmap v344: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 7 op/s 2023-07-21T19:07:02.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 -- 172.21.15.5:0/2654771498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c0ffb90 msgr2=0x7fe13c0fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:02.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 --2- 172.21.15.5:0/2654771498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c0ffb90 0x7fe13c0fffa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fe138009990 tx=0x7fe138009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 -- 172.21.15.5:0/2654771498 shutdown_connections 2023-07-21T19:07:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 --2- 172.21.15.5:0/2654771498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c101720 0x7fe13c107fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 --2- 172.21.15.5:0/2654771498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c0ffb90 0x7fe13c0fffa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 --2- 172.21.15.5:0/2654771498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c100d90 0x7fe13c1011e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 -- 172.21.15.5:0/2654771498 >> 172.21.15.5:0/2654771498 conn(0x7fe13c0fb1a0 msgr2=0x7fe13c0fd610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:02.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 -- 172.21.15.5:0/2654771498 shutdown_connections 2023-07-21T19:07:02.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.908+0000 7fe144557700 1 -- 172.21.15.5:0/2654771498 wait complete. 2023-07-21T19:07:02.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.909+0000 7fe144557700 1 Processor -- start 2023-07-21T19:07:02.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.909+0000 7fe144557700 1 -- start start 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c0ffb90 0x7fe13c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c101720 0x7fe13c19efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19f590 con 0x7fe13c100d90 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19f6d0 con 0x7fe13c101720 2023-07-21T19:07:02.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe144557700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13c19f860 con 0x7fe13c0ffb90 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe141af2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe141af2700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56714/0 (socket says 172.21.15.5:56714) 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe141af2700 1 -- 172.21.15.5:0/119482631 learned_addr learned my addr 172.21.15.5:0/119482631 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe1422f3700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c0ffb90 0x7fe13c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.910+0000 7fe142af4700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c101720 0x7fe13c19efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 -- 172.21.15.5:0/119482631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c0ffb90 msgr2=0x7fe13c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:02.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c0ffb90 0x7fe13c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 -- 172.21.15.5:0/119482631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c101720 msgr2=0x7fe13c19efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c101720 0x7fe13c19efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 -- 172.21.15.5:0/119482631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380095f0 con 0x7fe13c100d90 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe141af2700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00d980 tx=0x7fe12c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c0098d0 con 0x7fe13c100d90 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe13c19fb40 con 0x7fe13c100d90 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe12c010460 con 0x7fe13c100d90 2023-07-21T19:07:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe12c00f620 con 0x7fe13c100d90 2023-07-21T19:07:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.911+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13c1a0090 con 0x7fe13c100d90 2023-07-21T19:07:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.912+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe13c1920c0 con 0x7fe13c100d90 2023-07-21T19:07:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.914+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7fe12c009a30 con 0x7fe13c100d90 2023-07-21T19:07:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.917+0000 7fe1337fe700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 0x7fe1280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.917+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fe12c0cc840 con 0x7fe13c100d90 2023-07-21T19:07:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.917+0000 7fe1422f3700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 0x7fe1280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.918+0000 7fe1422f3700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 0x7fe1280a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b5c0 tx=0x7fe138005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:02.921+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe12c095de0 con 0x7fe13c100d90 2023-07-21T19:07:03.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.129+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fe13c02ca50 con 0x7fe13c100d90 2023-07-21T19:07:03.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.130+0000 7fe1337fe700 1 -- 172.21.15.5:0/119482631 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v87) v1 ==== 56+0+5264 (secure 0 0 0) 0x7fe12c099400 con 0x7fe13c100d90 2023-07-21T19:07:03.140 INFO:teuthology.orchestra.run.smithi005.stdout:epoch 87 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:created 2023-07-21T18:57:04.739728+0000 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:modified 2023-07-21T19:06:08.388082+0000 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:crush_version 26 2023-07-21T19:07:03.141 INFO:teuthology.orchestra.run.smithi005.stdout:full_ratio 0.95 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:backfillfull_ratio 0.9 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:nearfull_ratio 0.85 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:require_min_compat_client luminous 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:min_compat_client luminous 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:require_osd_release reef 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:stretch_mode_enabled false 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-07-21T19:07:03.142 INFO:teuthology.orchestra.run.smithi005.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:max_osd 12 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.0 up in weight 1 up_from 8 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6802/2154692488,v1:172.21.15.5:6803/2154692488] [v2:172.21.15.5:6804/2154692488,v1:172.21.15.5:6805/2154692488] exists,up d6825a11-5019-461a-a2af-a0337a93556d 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.1 up in weight 1 up_from 13 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6800/3147533687,v1:172.21.15.101:6801/3147533687] [v2:172.21.15.101:6802/3147533687,v1:172.21.15.101:6803/3147533687] exists,up 22a957d4-0669-466d-88ee-0f05ba0b0909 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.2 up in weight 1 up_from 18 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6800/2113778318,v1:172.21.15.156:6801/2113778318] [v2:172.21.15.156:6802/2113778318,v1:172.21.15.156:6803/2113778318] exists,up d6ba0786-1e76-44da-b171-81015c102bc4 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.3 up in weight 1 up_from 25 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6810/4237119296,v1:172.21.15.5:6811/4237119296] [v2:172.21.15.5:6812/4237119296,v1:172.21.15.5:6813/4237119296] exists,up 01f22961-e7d2-423a-8511-6b3ed03de822 2023-07-21T19:07:03.143 INFO:teuthology.orchestra.run.smithi005.stdout:osd.4 up in weight 1 up_from 33 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6808/1960590612,v1:172.21.15.101:6809/1960590612] [v2:172.21.15.101:6810/1960590612,v1:172.21.15.101:6811/1960590612] exists,up 157f6702-def5-47b3-a2e1-497954863a97 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.5 up in weight 1 up_from 38 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6808/1994365713,v1:172.21.15.156:6809/1994365713] [v2:172.21.15.156:6810/1994365713,v1:172.21.15.156:6811/1994365713] exists,up 04a4eedc-bd98-4441-8bdb-443010dfc21e 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.6 up in weight 1 up_from 46 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6818/3271423477,v1:172.21.15.5:6819/3271423477] [v2:172.21.15.5:6820/3271423477,v1:172.21.15.5:6821/3271423477] exists,up bd7ecefb-ba21-4052-adc1-17da27ca0595 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.7 up in weight 1 up_from 51 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6816/1417474005,v1:172.21.15.101:6817/1417474005] [v2:172.21.15.101:6818/1417474005,v1:172.21.15.101:6819/1417474005] exists,up a839fb0f-1bc6-4f9d-a381-079c5f730f63 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.8 up in weight 1 up_from 59 up_thru 83 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6816/1480568600,v1:172.21.15.156:6817/1480568600] [v2:172.21.15.156:6818/1480568600,v1:172.21.15.156:6819/1480568600] exists,up fc535b54-e81e-457d-ac39-57b75be1a082 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.9 up in weight 1 up_from 67 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.5:6826/3583557754,v1:172.21.15.5:6827/3583557754] [v2:172.21.15.5:6828/3583557754,v1:172.21.15.5:6829/3583557754] exists,up 41283e76-6d87-47be-b423-725502de65f4 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.10 up in weight 1 up_from 72 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.101:6824/3222350780,v1:172.21.15.101:6825/3222350780] [v2:172.21.15.101:6826/3222350780,v1:172.21.15.101:6827/3222350780] exists,up ba0f83a2-7456-498d-bf66-564b17ccbce2 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:osd.11 up in weight 1 up_from 80 up_thru 86 down_at 0 last_clean_interval [0,0) [v2:172.21.15.156:6824/2474999994,v1:172.21.15.156:6825/2474999994] [v2:172.21.15.156:6826/2474999994,v1:172.21.15.156:6827/2474999994] exists,up 1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2 2023-07-21T19:07:03.144 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.4 [2,11] 2023-07-21T19:07:03.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.5 [8,11] 2023-07-21T19:07:03.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1b [2,4] 2023-07-21T19:07:03.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 2.1f [8,1] 2023-07-21T19:07:03.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.0 [5,9] 2023-07-21T19:07:03.145 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.1 [4,9] 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.3 [4,9] 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:pg_upmap_items 3.6 [4,6] 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4175825231 expires 2023-07-22T18:58:07.219133+0000 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/4029410450 expires 2023-07-22T18:58:07.219133+0000 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/1672036156 expires 2023-07-22T18:58:07.219133+0000 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6801/474183959 expires 2023-07-22T18:57:41.027247+0000 2023-07-21T19:07:03.146 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/697337206 expires 2023-07-22T18:58:07.219133+0000 2023-07-21T19:07:03.147 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/2485032310 expires 2023-07-22T18:57:41.027247+0000 2023-07-21T19:07:03.147 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/474183959 expires 2023-07-22T18:57:41.027247+0000 2023-07-21T19:07:03.147 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/1317509932 expires 2023-07-22T18:57:41.027247+0000 2023-07-21T19:07:03.147 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:0/3754075095 expires 2023-07-22T18:57:41.027247+0000 2023-07-21T19:07:03.147 INFO:teuthology.orchestra.run.smithi005.stdout:blocklist 172.21.15.5:6800/1672036156 expires 2023-07-22T18:58:07.219133+0000 2023-07-21T19:07:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 msgr2=0x7fe1280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 0x7fe1280a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fe13800b5c0 tx=0x7fe138005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 msgr2=0x7fe13c197f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:03.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fe12c00d980 tx=0x7fe12c00dc90 comp rx=0 tx=0).stop 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 shutdown_connections 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fe1280a11e0 0x7fe1280a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe13c101720 0x7fe13c19efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe13c0ffb90 0x7fe13c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 --2- 172.21.15.5:0/119482631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe13c100d90 0x7fe13c197f40 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 >> 172.21.15.5:0/119482631 conn(0x7fe13c0fb1a0 msgr2=0x7fe13c1067a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:03.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 shutdown_connections 2023-07-21T19:07:03.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.132+0000 7fe144557700 1 -- 172.21.15.5:0/119482631 wait complete. 2023-07-21T19:07:03.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:03 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/119482631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T19:07:03.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:03 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/119482631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T19:07:03.711 INFO:teuthology.run_tasks:Running task fs.ready... 2023-07-21T19:07:03.723 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:03.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:03 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/119482631' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-07-21T19:07:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 -- 172.21.15.5:0/889584331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 msgr2=0x7f13b4101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/889584331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4101140 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f13a4009990 tx=0x7f13a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 -- 172.21.15.5:0/889584331 shutdown_connections 2023-07-21T19:07:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/889584331 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b4101680 0x7f13b4107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/889584331 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b40ffb00 0x7f13b40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/889584331 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4101140 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 -- 172.21.15.5:0/889584331 >> 172.21.15.5:0/889584331 conn(0x7f13b40fb0e0 msgr2=0x7f13b40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 -- 172.21.15.5:0/889584331 shutdown_connections 2023-07-21T19:07:03.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.920+0000 7f13bc5ff700 1 -- 172.21.15.5:0/889584331 wait complete. 2023-07-21T19:07:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.921+0000 7f13bc5ff700 1 Processor -- start 2023-07-21T19:07:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.921+0000 7f13bc5ff700 1 -- start start 2023-07-21T19:07:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b40ffb00 0x7f13b40742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:03.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b4101680 0x7f13b4072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b40735e0 con 0x7f13b4100cf0 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b4073750 con 0x7f13b40ffb00 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b419f7a0 con 0x7f13b4101680 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13b9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13b9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56724/0 (socket says 172.21.15.5:56724) 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13b9b9a700 1 -- 172.21.15.5:0/872926881 learned_addr learned my addr 172.21.15.5:0/872926881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:03.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13ba39b700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b40ffb00 0x7f13b40742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13bab9c700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b4101680 0x7f13b4072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13b9b9a700 1 -- 172.21.15.5:0/872926881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b4101680 msgr2=0x7f13b4072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.922+0000 7f13b9b9a700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b4101680 0x7f13b4072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13b9b9a700 1 -- 172.21.15.5:0/872926881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b40ffb00 msgr2=0x7f13b40742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13b9b9a700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b40ffb00 0x7f13b40742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13b9b9a700 1 -- 172.21.15.5:0/872926881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a40095f0 con 0x7f13b4100cf0 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13b9b9a700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13a4000c00 tx=0x7f13a4005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:03.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a401d070 con 0x7f13b4100cf0 2023-07-21T19:07:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f13a4003f40 con 0x7f13b4100cf0 2023-07-21T19:07:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b419fa20 con 0x7f13b4100cf0 2023-07-21T19:07:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a4022c70 con 0x7f13b4100cf0 2023-07-21T19:07:03.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.923+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b419fea0 con 0x7f13b4100cf0 2023-07-21T19:07:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.924+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b41070f0 con 0x7f13b4100cf0 2023-07-21T19:07:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.926+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f13a4003ab0 con 0x7f13b4100cf0 2023-07-21T19:07:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.928+0000 7f13ab7fe700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 0x7f13a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:03.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.928+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f13a40cdfc0 con 0x7f13b4100cf0 2023-07-21T19:07:03.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.928+0000 7f13ba39b700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 0x7f13a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:03.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.928+0000 7f13ba39b700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 0x7f13a00a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13b4100b50 tx=0x7f13b000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:03.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:03.932+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f13a40975f0 con 0x7f13b4100cf0 2023-07-21T19:07:04.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.140+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f13b402ca50 con 0x7f13b4100cf0 2023-07-21T19:07:04.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.141+0000 7f13ab7fe700 1 -- 172.21.15.5:0/872926881 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f13a409ac10 con 0x7f13b4100cf0 2023-07-21T19:07:04.153 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:04.154 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 msgr2=0x7f13a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 0x7f13a00a3750 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f13b4100b50 tx=0x7f13b000b410 comp rx=0 tx=0).stop 2023-07-21T19:07:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 msgr2=0x7f13b4072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f13a4000c00 tx=0x7f13a4005740 comp rx=0 tx=0).stop 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 shutdown_connections 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f13a00a12a0 0x7f13a00a3750 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13b4101680 0x7f13b4072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13b40ffb00 0x7f13b40742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 --2- 172.21.15.5:0/872926881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13b4100cf0 0x7f13b4072900 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 >> 172.21.15.5:0/872926881 conn(0x7f13b40fb0e0 msgr2=0x7f13b4104230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 shutdown_connections 2023-07-21T19:07:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.144+0000 7f13bc5ff700 1 -- 172.21.15.5:0/872926881 wait complete. 2023-07-21T19:07:04.174 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:04.175 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 -- 172.21.15.5:0/2231271925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 msgr2=0x7ff4080ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/2231271925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff4080ffef0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009990 tx=0x7ff3f8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 -- 172.21.15.5:0/2231271925 shutdown_connections 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/2231271925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff408107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/2231271925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff4080ffef0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/2231271925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 0x7ff408101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 -- 172.21.15.5:0/2231271925 >> 172.21.15.5:0/2231271925 conn(0x7ff4080fb0e0 msgr2=0x7ff4080fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.369+0000 7ff40d7ff700 1 -- 172.21.15.5:0/2231271925 shutdown_connections 2023-07-21T19:07:04.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.370+0000 7ff40d7ff700 1 -- 172.21.15.5:0/2231271925 wait complete. 2023-07-21T19:07:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 Processor -- start 2023-07-21T19:07:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 -- start start 2023-07-21T19:07:04.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 0x7ff408195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff40819cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40819d310 con 0x7ff4080ffae0 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40819d450 con 0x7ff408101660 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff40d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40819d5e0 con 0x7ff408100cd0 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff406ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff406ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56744/0 (socket says 172.21.15.5:56744) 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff406ffd700 1 -- 172.21.15.5:0/3232220073 learned_addr learned my addr 172.21.15.5:0/3232220073 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff4077fe700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff40819cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.371+0000 7ff4067fc700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 0x7ff408195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 -- 172.21.15.5:0/3232220073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 msgr2=0x7ff408195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 0x7ff408195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 -- 172.21.15.5:0/3232220073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 msgr2=0x7ff40819cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff40819cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 -- 172.21.15.5:0/3232220073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80095f0 con 0x7ff4080ffae0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff4077fe700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff40819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff406ffd700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8003ec0 tx=0x7ff3f8005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f801d070 con 0x7ff4080ffae0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3f8022470 con 0x7ff4080ffae0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40819d860 con 0x7ff4080ffae0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800f6c0 con 0x7ff4080ffae0 2023-07-21T19:07:04.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.372+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40819dce0 con 0x7ff4080ffae0 2023-07-21T19:07:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.373+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff40810b160 con 0x7ff4080ffae0 2023-07-21T19:07:04.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.375+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff3f800ba80 con 0x7ff4080ffae0 2023-07-21T19:07:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.376+0000 7ff3effff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 0x7ff3f00a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.377+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3f80ce7d0 con 0x7ff4080ffae0 2023-07-21T19:07:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.377+0000 7ff4067fc700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 0x7ff3f00a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.377+0000 7ff4067fc700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 0x7ff3f00a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009bf0 tx=0x7ff3fc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.379+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff3f8097f20 con 0x7ff4080ffae0 2023-07-21T19:07:04.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.588+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff408106ae0 con 0x7ff4080ffae0 2023-07-21T19:07:04.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.588+0000 7ff3effff700 1 -- 172.21.15.5:0/3232220073 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3f8027020 con 0x7ff4080ffae0 2023-07-21T19:07:04.599 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:07:04.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 msgr2=0x7ff3f00a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 0x7ff3f00a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7ff3fc009bf0 tx=0x7ff3fc009260 comp rx=0 tx=0).stop 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 msgr2=0x7ff408195790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8003ec0 tx=0x7ff3f8005ef0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 shutdown_connections 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff3f00a1260 0x7ff3f00a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff408101660 0x7ff40819cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4080ffae0 0x7ff408195790 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 --2- 172.21.15.5:0/3232220073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff408100cd0 0x7ff408195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 >> 172.21.15.5:0/3232220073 conn(0x7ff4080fb0e0 msgr2=0x7ff4080fd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 shutdown_connections 2023-07-21T19:07:04.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.591+0000 7ff40d7ff700 1 -- 172.21.15.5:0/3232220073 wait complete. 2023-07-21T19:07:04.618 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:04 smithi005 ceph-mon[126460]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:04.619 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:04 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/872926881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:04.621 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:04 smithi156 ceph-mon[128316]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:04.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:04 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/872926881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:04.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 -- 172.21.15.5:0/1080401365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 msgr2=0x7f75d0101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 --2- 172.21.15.5:0/1080401365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0101120 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009990 tx=0x7f75c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 -- 172.21.15.5:0/1080401365 shutdown_connections 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 --2- 172.21.15.5:0/1080401365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d0101660 0x7f75d0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 --2- 172.21.15.5:0/1080401365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d00ffae0 0x7f75d00ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 --2- 172.21.15.5:0/1080401365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0101120 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 -- 172.21.15.5:0/1080401365 >> 172.21.15.5:0/1080401365 conn(0x7f75d00fb0e0 msgr2=0x7f75d00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.817+0000 7f75d59ff700 1 -- 172.21.15.5:0/1080401365 shutdown_connections 2023-07-21T19:07:04.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.818+0000 7f75d59ff700 1 -- 172.21.15.5:0/1080401365 wait complete. 2023-07-21T19:07:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 Processor -- start 2023-07-21T19:07:04.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 -- start start 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d00ffae0 0x7f75d00742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 0x7f75d0072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d00735e0 con 0x7f75d0100cd0 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d0073750 con 0x7f75d00ffae0 2023-07-21T19:07:04.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75d59ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d019f810 con 0x7f75d0101660 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56764/0 (socket says 172.21.15.5:56764) 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75ce7fc700 1 -- 172.21.15.5:0/406132433 learned_addr learned my addr 172.21.15.5:0/406132433 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.819+0000 7f75ceffd700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d00ffae0 0x7f75d00742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75cf7fe700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 0x7f75d0072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 -- 172.21.15.5:0/406132433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 msgr2=0x7f75d0072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 0x7f75d0072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 -- 172.21.15.5:0/406132433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d00ffae0 msgr2=0x7f75d00742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d00ffae0 0x7f75d00742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 -- 172.21.15.5:0/406132433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c40095f0 con 0x7f75d0100cd0 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75cf7fe700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 0x7f75d0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75ce7fc700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005910 tx=0x7f75c4005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c401d070 con 0x7f75d0100cd0 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75c400be90 con 0x7f75d0100cd0 2023-07-21T19:07:04.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4022e60 con 0x7f75d0100cd0 2023-07-21T19:07:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.820+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75d019fa90 con 0x7f75d0100cd0 2023-07-21T19:07:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.821+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75d019fe60 con 0x7f75d0100cd0 2023-07-21T19:07:04.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.822+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75d010b1b0 con 0x7f75d0100cd0 2023-07-21T19:07:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.824+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f75c4022600 con 0x7f75d0100cd0 2023-07-21T19:07:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.825+0000 7f75d49fd700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 0x7f75b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.825+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f75c40ce3c0 con 0x7f75d0100cd0 2023-07-21T19:07:04.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.825+0000 7f75ceffd700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 0x7f75b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.826+0000 7f75ceffd700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 0x7f75b80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75d0100b30 tx=0x7f75c0007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:04.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:04.827+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f75c4097b10 con 0x7f75d0100cd0 2023-07-21T19:07:04.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:04 smithi101 ceph-mon[128136]: pgmap v345: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:04.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:04 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/872926881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:05.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.038+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f75d0106a80 con 0x7f75d0100cd0 2023-07-21T19:07:05.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.038+0000 7f75d49fd700 1 -- 172.21.15.5:0/406132433 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f75c409b130 con 0x7f75d0100cd0 2023-07-21T19:07:05.049 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:07:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 msgr2=0x7f75b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 0x7f75b80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f75d0100b30 tx=0x7f75c0007400 comp rx=0 tx=0).stop 2023-07-21T19:07:05.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 msgr2=0x7f75d0072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f75c4005910 tx=0x7f75c4005720 comp rx=0 tx=0).stop 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 shutdown_connections 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f75b80a11e0 0x7f75b80a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75d0101660 0x7f75d0072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75d00ffae0 0x7f75d00742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 --2- 172.21.15.5:0/406132433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75d0100cd0 0x7f75d0072900 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 >> 172.21.15.5:0/406132433 conn(0x7f75d00fb0e0 msgr2=0x7f75d00fd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 shutdown_connections 2023-07-21T19:07:05.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.041+0000 7f75d59ff700 1 -- 172.21.15.5:0/406132433 wait complete. 2023-07-21T19:07:05.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:05.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 -- 172.21.15.5:0/3063336235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 msgr2=0x7f2f400729a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3063336235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f400729a0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f2f30009990 tx=0x7f2f30009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 -- 172.21.15.5:0/3063336235 shutdown_connections 2023-07-21T19:07:05.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3063336235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f40109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3063336235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f400729a0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3063336235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f400733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 -- 172.21.15.5:0/3063336235 >> 172.21.15.5:0/3063336235 conn(0x7f2f400fb0e0 msgr2=0x7f2f400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:05.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 -- 172.21.15.5:0/3063336235 shutdown_connections 2023-07-21T19:07:05.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.267+0000 7f2f461ff700 1 -- 172.21.15.5:0/3063336235 wait complete. 2023-07-21T19:07:05.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 Processor -- start 2023-07-21T19:07:05.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 -- start start 2023-07-21T19:07:05.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f40197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f4019ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4019f510 con 0x7f2f40074550 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4019f650 con 0x7f2f40072f70 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.269+0000 7f2f461ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4019f7e0 con 0x7f2f401079a0 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56788/0 (socket says 172.21.15.5:56788) 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 -- 172.21.15.5:0/3231566017 learned_addr learned my addr 172.21.15.5:0/3231566017 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3ffff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f4019ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3f7fe700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f40197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 -- 172.21.15.5:0/3231566017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 msgr2=0x7f2f4019ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f4019ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 -- 172.21.15.5:0/3231566017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 msgr2=0x7f2f40197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f40197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 -- 172.21.15.5:0/3231566017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f300095f0 con 0x7f2f40074550 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3ffff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f4019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3f7fe700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f40197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.270+0000 7f2f3effd700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d980 tx=0x7f2f3400dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.271+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3400de60 con 0x7f2f40074550 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.271+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f4019fac0 con 0x7f2f40074550 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.271+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2f340044b0 con 0x7f2f40074550 2023-07-21T19:07:05.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.271+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f34003b40 con 0x7f2f40074550 2023-07-21T19:07:05.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.271+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f401a0020 con 0x7f2f40074550 2023-07-21T19:07:05.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.272+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f40106050 con 0x7f2f40074550 2023-07-21T19:07:05.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.274+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f2f34003ca0 con 0x7f2f40074550 2023-07-21T19:07:05.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.275+0000 7f2f3cff9700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 0x7f2f280a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.275+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2f340cc820 con 0x7f2f40074550 2023-07-21T19:07:05.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.275+0000 7f2f3f7fe700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 0x7f2f280a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.276+0000 7f2f3f7fe700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 0x7f2f280a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000b5c0 tx=0x7f2f30005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:05.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.279+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2f34095dc0 con 0x7f2f40074550 2023-07-21T19:07:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.495+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2f4002ca70 con 0x7f2f40074550 2023-07-21T19:07:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.495+0000 7f2f3cff9700 1 -- 172.21.15.5:0/3231566017 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2f34009940 con 0x7f2f40074550 2023-07-21T19:07:05.506 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:07:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 msgr2=0x7f2f280a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 0x7f2f280a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f3000b5c0 tx=0x7f2f30005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 msgr2=0x7f2f40197ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f2f3400d980 tx=0x7f2f3400dd40 comp rx=0 tx=0).stop 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 shutdown_connections 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.497+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2f280a11e0 0x7f2f280a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2f401079a0 0x7f2f4019ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2f40072f70 0x7f2f40197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 --2- 172.21.15.5:0/3231566017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2f40074550 0x7f2f40197ed0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 >> 172.21.15.5:0/3231566017 conn(0x7f2f400fb0e0 msgr2=0x7f2f400fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:05.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 shutdown_connections 2023-07-21T19:07:05.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.498+0000 7f2f461ff700 1 -- 172.21.15.5:0/3231566017 wait complete. 2023-07-21T19:07:05.527 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:05.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3232220073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/406132433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3232220073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/406132433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 -- 172.21.15.5:0/774388540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 msgr2=0x7f4b900ffe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 --2- 172.21.15.5:0/774388540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b900ffe90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c009990 tx=0x7f4b8c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 -- 172.21.15.5:0/774388540 shutdown_connections 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 --2- 172.21.15.5:0/774388540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 0x7f4b90107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 --2- 172.21.15.5:0/774388540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b900ffe90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 --2- 172.21.15.5:0/774388540 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 0x7f4b901010c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 -- 172.21.15.5:0/774388540 >> 172.21.15.5:0/774388540 conn(0x7f4b900fb060 msgr2=0x7f4b900fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:05.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 -- 172.21.15.5:0/774388540 shutdown_connections 2023-07-21T19:07:05.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.720+0000 7f4b985ff700 1 -- 172.21.15.5:0/774388540 wait complete. 2023-07-21T19:07:05.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 Processor -- start 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 -- start start 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 0x7f4b90195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 0x7f4b9019cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019d2e0 con 0x7f4b900ffa80 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019d420 con 0x7f4b90100c70 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.721+0000 7f4b985ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9019d5b0 con 0x7f4b90101600 2023-07-21T19:07:05.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56802/0 (socket says 172.21.15.5:56802) 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 -- 172.21.15.5:0/1534995874 learned_addr learned my addr 172.21.15.5:0/1534995874 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b95b9a700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 0x7f4b90195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b96b9c700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 0x7f4b9019cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 -- 172.21.15.5:0/1534995874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 msgr2=0x7f4b9019cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 0x7f4b9019cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 -- 172.21.15.5:0/1534995874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 msgr2=0x7f4b90195ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 0x7f4b90195ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 -- 172.21.15.5:0/1534995874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b8c0095f0 con 0x7f4b900ffa80 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.722+0000 7f4b9639b700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098e0 tx=0x7f4b8c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.723+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c01d070 con 0x7f4b900ffa80 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.723+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b8c00bdd0 con 0x7f4b900ffa80 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.723+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9019d830 con 0x7f4b900ffa80 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.723+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b8c022d20 con 0x7f4b900ffa80 2023-07-21T19:07:05.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.723+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9019dcd0 con 0x7f4b900ffa80 2023-07-21T19:07:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.724+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9010b1d0 con 0x7f4b900ffa80 2023-07-21T19:07:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.726+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f4b8c00b940 con 0x7f4b900ffa80 2023-07-21T19:07:05.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.727+0000 7f4b877fe700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 0x7f4b7c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.727+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4b8c0ce150 con 0x7f4b900ffa80 2023-07-21T19:07:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.727+0000 7f4b95b9a700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 0x7f4b7c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:05.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.728+0000 7f4b95b9a700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 0x7f4b7c0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009c60 tx=0x7f4b80009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:05.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.731+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4b8c097780 con 0x7f4b900ffa80 2023-07-21T19:07:05.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3232220073' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/406132433' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:05.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.989+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b9010b3c0 con 0x7f4b900ffa80 2023-07-21T19:07:06.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.990+0000 7f4b877fe700 1 -- 172.21.15.5:0/1534995874 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f4b8c09ada0 con 0x7f4b900ffa80 2023-07-21T19:07:06.000 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:06.000 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:06.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 msgr2=0x7f4b7c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 0x7f4b7c0a36d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f4b80009c60 tx=0x7f4b80009260 comp rx=0 tx=0).stop 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 msgr2=0x7f4b90195760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c0098e0 tx=0x7f4b8c004730 comp rx=0 tx=0).stop 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 shutdown_connections 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4b7c0a1220 0x7f4b7c0a36d0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b90101600 0x7f4b9019cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b900ffa80 0x7f4b90195760 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 --2- 172.21.15.5:0/1534995874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b90100c70 0x7f4b90195ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 >> 172.21.15.5:0/1534995874 conn(0x7f4b900fb060 msgr2=0x7f4b900fd3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 shutdown_connections 2023-07-21T19:07:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:05.992+0000 7f4b985ff700 1 -- 172.21.15.5:0/1534995874 wait complete. 2023-07-21T19:07:06.004 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:06.022 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:06.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 -- 172.21.15.5:0/3100844299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c100cf0 msgr2=0x7f9d6c101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3100844299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c101140 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f9d58009990 tx=0x7f9d58009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 -- 172.21.15.5:0/3100844299 shutdown_connections 2023-07-21T19:07:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3100844299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c101680 0x7f9d6c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3100844299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3100844299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c101140 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 -- 172.21.15.5:0/3100844299 >> 172.21.15.5:0/3100844299 conn(0x7f9d6c0fb0e0 msgr2=0x7f9d6c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.215+0000 7f9d731ff700 1 -- 172.21.15.5:0/3100844299 shutdown_connections 2023-07-21T19:07:06.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.216+0000 7f9d731ff700 1 -- 172.21.15.5:0/3100844299 wait complete. 2023-07-21T19:07:06.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.217+0000 7f9d731ff700 1 Processor -- start 2023-07-21T19:07:06.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.217+0000 7f9d731ff700 1 -- start start 2023-07-21T19:07:06.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c101680 0x7f9d6c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19d290 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19d3d0 con 0x7f9d6c101680 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d731ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d6c19d560 con 0x7f9d6c100cf0 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56820/0 (socket says 172.21.15.5:56820) 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 -- 172.21.15.5:0/3071540692 learned_addr learned my addr 172.21.15.5:0/3071540692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:06.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d6bfff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d7179c700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c101680 0x7f9d6c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 -- 172.21.15.5:0/3071540692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100cf0 msgr2=0x7f9d6c195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 -- 172.21.15.5:0/3071540692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c101680 msgr2=0x7f9d6c19ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c101680 0x7f9d6c19ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.218+0000 7f9d70f9b700 1 -- 172.21.15.5:0/3071540692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d580095f0 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d70f9b700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e970 tx=0x7f9d6000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60009790 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9d6000eec0 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d6c19d840 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d60003c10 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.219+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d6c19dd50 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.220+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6c107090 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.222+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f9d60003db0 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.223+0000 7f9d69ffb700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 0x7f9d5c0a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.223+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9d60014070 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.223+0000 7f9d6bfff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 0x7f9d5c0a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.224+0000 7f9d6bfff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 0x7f9d5c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b2e0 tx=0x7f9d58004000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:06.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.227+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f9d600962e0 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.439+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d6c02c850 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.441+0000 7f9d69ffb700 1 -- 172.21.15.5:0/3071540692 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f9d60099900 con 0x7f9d6c0ffb00 2023-07-21T19:07:06.452 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:06.452 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 msgr2=0x7f9d5c0a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 0x7f9d5c0a36d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9d5800b2e0 tx=0x7f9d58004000 comp rx=0 tx=0).stop 2023-07-21T19:07:06.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 msgr2=0x7f9d6c195710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e970 tx=0x7f9d6000ec80 comp rx=0 tx=0).stop 2023-07-21T19:07:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 shutdown_connections 2023-07-21T19:07:06.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9d5c0a1220 0x7f9d5c0a36d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9d6c101680 0x7f9d6c19ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9d6c0ffb00 0x7f9d6c195710 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 --2- 172.21.15.5:0/3071540692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9d6c100cf0 0x7f9d6c195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 >> 172.21.15.5:0/3071540692 conn(0x7f9d6c0fb0e0 msgr2=0x7f9d6c1041d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 shutdown_connections 2023-07-21T19:07:06.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.443+0000 7f9d731ff700 1 -- 172.21.15.5:0/3071540692 wait complete. 2023-07-21T19:07:06.475 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:06.476 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:06.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:06 smithi005 ceph-mon[126460]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:07:06.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3231566017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:06.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1534995874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:06.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:06 smithi156 ceph-mon[128316]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:07:06.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3231566017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:06.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1534995874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 -- 172.21.15.5:0/1585790250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340101660 msgr2=0x7ff340107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1585790250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340101660 0x7ff340107f20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7ff33800b0a0 tx=0x7ff33800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 -- 172.21.15.5:0/1585790250 shutdown_connections 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1585790250 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340101660 0x7ff340107f20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1585790250 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff3400ffae0 0x7ff3400ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1585790250 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff340100cd0 0x7ff340101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 -- 172.21.15.5:0/1585790250 >> 172.21.15.5:0/1585790250 conn(0x7ff3400fb0e0 msgr2=0x7ff3400fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 -- 172.21.15.5:0/1585790250 shutdown_connections 2023-07-21T19:07:06.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.672+0000 7ff3481ff700 1 -- 172.21.15.5:0/1585790250 wait complete. 2023-07-21T19:07:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 Processor -- start 2023-07-21T19:07:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 -- start start 2023-07-21T19:07:06.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff3400ffae0 0x7ff3400742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 0x7ff340072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3400735e0 con 0x7ff340100cd0 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff340073750 con 0x7ff3400ffae0 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.673+0000 7ff3481ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff34019f940 con 0x7ff340101660 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36050/0 (socket says 172.21.15.5:36050) 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 -- 172.21.15.5:0/1016476489 learned_addr learned my addr 172.21.15.5:0/1016476489 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff345f9b700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff3400ffae0 0x7ff3400742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34679c700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 0x7ff340072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 -- 172.21.15.5:0/1016476489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 msgr2=0x7ff340072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 0x7ff340072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 -- 172.21.15.5:0/1016476489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff3400ffae0 msgr2=0x7ff3400742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff3400ffae0 0x7ff3400742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 -- 172.21.15.5:0/1016476489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff338009d00 con 0x7ff340100cd0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34679c700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 0x7ff340072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff34579a700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d950 tx=0x7ff33000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.674+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff330009940 con 0x7ff340100cd0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.675+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff34019fc20 con 0x7ff340100cd0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.675+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff330010460 con 0x7ff340100cd0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.675+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff33000f620 con 0x7ff340100cd0 2023-07-21T19:07:06.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.675+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3401a0180 con 0x7ff340100cd0 2023-07-21T19:07:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.676+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff340107070 con 0x7ff340100cd0 2023-07-21T19:07:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.678+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff3300105d0 con 0x7ff340100cd0 2023-07-21T19:07:06.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.679+0000 7ff336ffd700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 0x7ff32c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.679+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff3300cc7c0 con 0x7ff340100cd0 2023-07-21T19:07:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.679+0000 7ff345f9b700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 0x7ff32c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:06.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.680+0000 7ff345f9b700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 0x7ff32c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff340100b30 tx=0x7ff33c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:06.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.683+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff330095dc0 con 0x7ff340100cd0 2023-07-21T19:07:06.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:06 smithi101 ceph-mon[128136]: pgmap v346: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 6 op/s 2023-07-21T19:07:06.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3231566017' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:06.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1534995874' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.891+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff34002caa0 con 0x7ff340100cd0 2023-07-21T19:07:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.892+0000 7ff336ffd700 1 -- 172.21.15.5:0/1016476489 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff3300993e0 con 0x7ff340100cd0 2023-07-21T19:07:06.902 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:07:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 msgr2=0x7ff32c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 0x7ff32c0a3710 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ff340100b30 tx=0x7ff33c00b410 comp rx=0 tx=0).stop 2023-07-21T19:07:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 msgr2=0x7ff340072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7ff33000d950 tx=0x7ff33000dc60 comp rx=0 tx=0).stop 2023-07-21T19:07:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 shutdown_connections 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff32c0a1260 0x7ff32c0a3710 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff340101660 0x7ff340072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff3400ffae0 0x7ff3400742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 --2- 172.21.15.5:0/1016476489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff340100cd0 0x7ff340072900 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 >> 172.21.15.5:0/1016476489 conn(0x7ff3400fb0e0 msgr2=0x7ff3401041b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 shutdown_connections 2023-07-21T19:07:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:06.894+0000 7ff3481ff700 1 -- 172.21.15.5:0/1016476489 wait complete. 2023-07-21T19:07:06.923 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 -- 172.21.15.5:0/626302919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc1079a0 msgr2=0x7f5fcc109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/626302919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc109e20 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f5fc400b0a0 tx=0x7f5fc400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 -- 172.21.15.5:0/626302919 shutdown_connections 2023-07-21T19:07:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/626302919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc109e20 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/626302919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc074550 0x7f5fcc0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/626302919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc072f70 0x7f5fcc0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 -- 172.21.15.5:0/626302919 >> 172.21.15.5:0/626302919 conn(0x7f5fcc0fb0e0 msgr2=0x7f5fcc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 -- 172.21.15.5:0/626302919 shutdown_connections 2023-07-21T19:07:07.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.118+0000 7f5fd39ff700 1 -- 172.21.15.5:0/626302919 wait complete. 2023-07-21T19:07:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 Processor -- start 2023-07-21T19:07:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 -- start start 2023-07-21T19:07:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 0x7f5fcc197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc19f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc19f640 con 0x7f5fcc074550 2023-07-21T19:07:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc19f780 con 0x7f5fcc072f70 2023-07-21T19:07:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.119+0000 7f5fd39ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc19f910 con 0x7f5fcc1079a0 2023-07-21T19:07:07.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36072/0 (socket says 172.21.15.5:36072) 2023-07-21T19:07:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 -- 172.21.15.5:0/2463334430 learned_addr learned my addr 172.21.15.5:0/2463334430 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:07.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd179b700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 0x7f5fcc197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd1f9c700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc19f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 -- 172.21.15.5:0/2463334430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc1079a0 msgr2=0x7f5fcc19f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc19f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 -- 172.21.15.5:0/2463334430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 msgr2=0x7f5fcc197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 0x7f5fcc197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 -- 172.21.15.5:0/2463334430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc4009d00 con 0x7f5fcc074550 2023-07-21T19:07:07.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd179b700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 0x7f5fcc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:07.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.120+0000 7f5fd0f9a700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00d980 tx=0x7f5fbc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:07.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.121+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc004020 con 0x7f5fcc074550 2023-07-21T19:07:07.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.121+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc19fbf0 con 0x7f5fcc074550 2023-07-21T19:07:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.121+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fbc00f690 con 0x7f5fcc074550 2023-07-21T19:07:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.121+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0039e0 con 0x7f5fcc074550 2023-07-21T19:07:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.121+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc1a0100 con 0x7f5fcc074550 2023-07-21T19:07:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.122+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc106050 con 0x7f5fcc074550 2023-07-21T19:07:07.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.124+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f5fbc00f800 con 0x7f5fcc074550 2023-07-21T19:07:07.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.126+0000 7f5fc27fc700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 0x7f5fb80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.126+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5fbc0cc6e0 con 0x7f5fcc074550 2023-07-21T19:07:07.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.126+0000 7f5fd179b700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 0x7f5fb80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.127+0000 7f5fd179b700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 0x7f5fb80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005950 tx=0x7f5fc800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:07.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.130+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5fbc095c80 con 0x7f5fcc074550 2023-07-21T19:07:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.369+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5fcc02ca70 con 0x7f5fcc074550 2023-07-21T19:07:07.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.369+0000 7f5fc27fc700 1 -- 172.21.15.5:0/2463334430 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5fbc0992a0 con 0x7f5fcc074550 2023-07-21T19:07:07.380 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:07:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 msgr2=0x7f5fb80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 0x7f5fb80a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8005950 tx=0x7f5fc800b410 comp rx=0 tx=0).stop 2023-07-21T19:07:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 msgr2=0x7f5fcc198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00d980 tx=0x7f5fbc00dc90 comp rx=0 tx=0).stop 2023-07-21T19:07:07.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 shutdown_connections 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f5fb80a11e0 0x7f5fb80a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fcc1079a0 0x7f5fcc19f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fcc072f70 0x7f5fcc197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 --2- 172.21.15.5:0/2463334430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fcc074550 0x7f5fcc198050 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.371+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 >> 172.21.15.5:0/2463334430 conn(0x7f5fcc0fb0e0 msgr2=0x7f5fcc0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.372+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 shutdown_connections 2023-07-21T19:07:07.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.372+0000 7f5fd39ff700 1 -- 172.21.15.5:0/2463334430 wait complete. 2023-07-21T19:07:07.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:07.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 -- 172.21.15.5:0/710599652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 msgr2=0x7ff84c101130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 --2- 172.21.15.5:0/710599652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c101130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff840009990 tx=0x7ff840009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 -- 172.21.15.5:0/710599652 shutdown_connections 2023-07-21T19:07:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 --2- 172.21.15.5:0/710599652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 0x7ff84c107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 --2- 172.21.15.5:0/710599652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 --2- 172.21.15.5:0/710599652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c101130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 -- 172.21.15.5:0/710599652 >> 172.21.15.5:0/710599652 conn(0x7ff84c0fb130 msgr2=0x7ff84c0fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:07.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 -- 172.21.15.5:0/710599652 shutdown_connections 2023-07-21T19:07:07.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.597+0000 7ff850fff700 1 -- 172.21.15.5:0/710599652 wait complete. 2023-07-21T19:07:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.598+0000 7ff850fff700 1 Processor -- start 2023-07-21T19:07:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.598+0000 7ff850fff700 1 -- start start 2023-07-21T19:07:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 0x7ff84c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c0735e0 con 0x7ff84c100ce0 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c073750 con 0x7ff84c0ffaf0 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff850fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff84c19f7e0 con 0x7ff84c101670 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36082/0 (socket says 172.21.15.5:36082) 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 -- 172.21.15.5:0/430979761 learned_addr learned my addr 172.21.15.5:0/430979761 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:07.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff84ad9d700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 0x7ff84c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff84a59c700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 -- 172.21.15.5:0/430979761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 msgr2=0x7ff84c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 0x7ff84c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 -- 172.21.15.5:0/430979761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 msgr2=0x7ff84c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 -- 172.21.15.5:0/430979761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8400095f0 con 0x7ff84c100ce0 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff84a59c700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:07.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.599+0000 7ff849d9b700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff8400095c0 tx=0x7ff840005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.600+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff84001d070 con 0x7ff84c100ce0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.600+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff840003fc0 con 0x7ff84c100ce0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.600+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff840022d00 con 0x7ff84c100ce0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.600+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff84c19fa60 con 0x7ff84c100ce0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.600+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff84c19ff00 con 0x7ff84c100ce0 2023-07-21T19:07:07.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.601+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff84c10b260 con 0x7ff84c100ce0 2023-07-21T19:07:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.603+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7ff840003b30 con 0x7ff84c100ce0 2023-07-21T19:07:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.604+0000 7ff83b7fe700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 0x7ff8340a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:07.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.604+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff8400cdee0 con 0x7ff84c100ce0 2023-07-21T19:07:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.605+0000 7ff84a59c700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 0x7ff8340a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:07.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.605+0000 7ff84a59c700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 0x7ff8340a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff83c005fd0 tx=0x7ff83c005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:07.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.608+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff840097510 con 0x7ff84c100ce0 2023-07-21T19:07:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3071540692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1016476489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2463334430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3071540692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1016476489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2463334430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.818+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff84c106a40 con 0x7ff84c100ce0 2023-07-21T19:07:07.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.819+0000 7ff83b7fe700 1 -- 172.21.15.5:0/430979761 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff840022e60 con 0x7ff84c100ce0 2023-07-21T19:07:07.830 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:07:07.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 msgr2=0x7ff8340a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 0x7ff8340a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff83c005fd0 tx=0x7ff83c005f00 comp rx=0 tx=0).stop 2023-07-21T19:07:07.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 msgr2=0x7ff84c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:07.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff8400095c0 tx=0x7ff840005ef0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 shutdown_connections 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff8340a1260 0x7ff8340a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff84c101670 0x7ff84c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff84c0ffaf0 0x7ff84c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 --2- 172.21.15.5:0/430979761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff84c100ce0 0x7ff84c072900 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 >> 172.21.15.5:0/430979761 conn(0x7ff84c0fb130 msgr2=0x7ff84c0fd3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:07.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 shutdown_connections 2023-07-21T19:07:07.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:07.821+0000 7ff850fff700 1 -- 172.21.15.5:0/430979761 wait complete. 2023-07-21T19:07:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3071540692' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1016476489' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2463334430' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:07.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:08.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 -- 172.21.15.5:0/2811837129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 msgr2=0x7f50b4107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/2811837129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b4107f40 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f50ac00b0a0 tx=0x7f50ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 -- 172.21.15.5:0/2811837129 shutdown_connections 2023-07-21T19:07:08.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/2811837129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b4107f40 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/2811837129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b40ffb00 0x7f50b40fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/2811837129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b4100cf0 0x7f50b4101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 -- 172.21.15.5:0/2811837129 >> 172.21.15.5:0/2811837129 conn(0x7f50b40fb0e0 msgr2=0x7f50b40fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 -- 172.21.15.5:0/2811837129 shutdown_connections 2023-07-21T19:07:08.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.042+0000 7f50bb3ff700 1 -- 172.21.15.5:0/2811837129 wait complete. 2023-07-21T19:07:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 Processor -- start 2023-07-21T19:07:08.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 -- start start 2023-07-21T19:07:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b40ffb00 0x7f50b4197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b4100cf0 0x7f50b4198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b419f660 con 0x7f50b4101680 2023-07-21T19:07:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b419f7a0 con 0x7f50b40ffb00 2023-07-21T19:07:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.043+0000 7f50bb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50b419f930 con 0x7f50b4100cf0 2023-07-21T19:07:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36112/0 (socket says 172.21.15.5:36112) 2023-07-21T19:07:08.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 -- 172.21.15.5:0/562531165 learned_addr learned my addr 172.21.15.5:0/562531165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b899a700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b4100cf0 0x7f50b4198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b919b700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b40ffb00 0x7f50b4197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 -- 172.21.15.5:0/562531165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b4100cf0 msgr2=0x7f50b4198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b4100cf0 0x7f50b4198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 -- 172.21.15.5:0/562531165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b40ffb00 msgr2=0x7f50b4197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b40ffb00 0x7f50b4197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 -- 172.21.15.5:0/562531165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50ac009d00 con 0x7f50b4101680 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50b999c700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009fd0 tx=0x7f50ac00bb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.044+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac00e040 con 0x7f50b4101680 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.045+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50ac003c00 con 0x7f50b4101680 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.045+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50b419fbb0 con 0x7f50b4101680 2023-07-21T19:07:08.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.045+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ac01b6c0 con 0x7f50b4101680 2023-07-21T19:07:08.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.045+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50b419ffd0 con 0x7f50b4101680 2023-07-21T19:07:08.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.046+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50b410b250 con 0x7f50b4101680 2023-07-21T19:07:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.048+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 17) v1 ==== 135765+0+0 (secure 0 0 0) 0x7f50ac01d470 con 0x7f50b4101680 2023-07-21T19:07:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.049+0000 7f50aa7fc700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 0x7f50a00a3750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.049+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f50ac0cdf30 con 0x7f50b4101680 2023-07-21T19:07:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.049+0000 7f50b919b700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 0x7f50a00a3750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.050+0000 7f50b919b700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 0x7f50a00a3750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f50b4100b50 tx=0x7f50b0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:08.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.053+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f50ac0975c0 con 0x7f50b4101680 2023-07-21T19:07:08.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.307+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f50b41a0580 con 0x7f50b4101680 2023-07-21T19:07:08.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.307+0000 7f50aa7fc700 1 -- 172.21.15.5:0/562531165 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f50ac09abe0 con 0x7f50b4101680 2023-07-21T19:07:08.318 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:08.319 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:08.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 msgr2=0x7f50a00a3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 0x7f50a00a3750 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f50b4100b50 tx=0x7f50b0009260 comp rx=0 tx=0).stop 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 msgr2=0x7f50b419f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f50ac009fd0 tx=0x7f50ac00bb00 comp rx=0 tx=0).stop 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 shutdown_connections 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f50a00a12a0 0x7f50a00a3750 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50b4101680 0x7f50b419f090 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50b40ffb00 0x7f50b4197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 --2- 172.21.15.5:0/562531165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50b4100cf0 0x7f50b4198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 >> 172.21.15.5:0/562531165 conn(0x7f50b40fb0e0 msgr2=0x7f50b4106580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 shutdown_connections 2023-07-21T19:07:08.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.310+0000 7f50bb3ff700 1 -- 172.21.15.5:0/562531165 wait complete. 2023-07-21T19:07:08.323 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:08.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 -- 172.21.15.5:0/4000839509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac100cf0 msgr2=0x7f22ac101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 --2- 172.21.15.5:0/4000839509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac100cf0 0x7f22ac101140 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f229c009990 tx=0x7f229c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 -- 172.21.15.5:0/4000839509 shutdown_connections 2023-07-21T19:07:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 --2- 172.21.15.5:0/4000839509 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac101680 0x7f22ac107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 --2- 172.21.15.5:0/4000839509 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac0ffb00 0x7f22ac0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 --2- 172.21.15.5:0/4000839509 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac100cf0 0x7f22ac101140 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 -- 172.21.15.5:0/4000839509 >> 172.21.15.5:0/4000839509 conn(0x7f22ac0fb0e0 msgr2=0x7f22ac0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:08.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 -- 172.21.15.5:0/4000839509 shutdown_connections 2023-07-21T19:07:08.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.537+0000 7f22b37ff700 1 -- 172.21.15.5:0/4000839509 wait complete. 2023-07-21T19:07:08.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.538+0000 7f22b37ff700 1 Processor -- start 2023-07-21T19:07:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.538+0000 7f22b37ff700 1 -- start start 2023-07-21T19:07:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac100cf0 0x7f22ac197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac101680 0x7f22ac19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19f4b0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19f5f0 con 0x7f22ac100cf0 2023-07-21T19:07:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22ac19f780 con 0x7f22ac101680 2023-07-21T19:07:08.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36142/0 (socket says 172.21.15.5:36142) 2023-07-21T19:07:08.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 -- 172.21.15.5:0/389338083 learned_addr learned my addr 172.21.15.5:0/389338083 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b0d9a700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac100cf0 0x7f22ac197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b1d9c700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac101680 0x7f22ac19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 -- 172.21.15.5:0/389338083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac101680 msgr2=0x7f22ac19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac101680 0x7f22ac19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 -- 172.21.15.5:0/389338083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac100cf0 msgr2=0x7f22ac197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac100cf0 0x7f22ac197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.539+0000 7f22b159b700 1 -- 172.21.15.5:0/389338083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c0095f0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22b159b700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c990 tx=0x7f22a800cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8007940 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22a800f450 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22ac19fa60 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22a8018660 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.540+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22ac19ffc0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.541+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22ac10b1d0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.543+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f22a800f5c0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.544+0000 7f22a27fc700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 0x7f22980a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.544+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22a80cc800 con 0x7f22ac0ffb00 2023-07-21T19:07:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.544+0000 7f22b0d9a700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 0x7f22980a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.545+0000 7f22b0d9a700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 0x7f22980a36d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:08.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.548+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f22a8095f00 con 0x7f22ac0ffb00 2023-07-21T19:07:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:08 smithi005 ceph-mon[126460]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:07:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/430979761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/562531165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:08.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.758+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22ac10b3c0 con 0x7f22ac0ffb00 2023-07-21T19:07:08.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.760+0000 7f22a27fc700 1 -- 172.21.15.5:0/389338083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f22a8099520 con 0x7f22ac0ffb00 2023-07-21T19:07:08.771 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:08.771 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 msgr2=0x7f22980a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 0x7f22980a36d0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f229c006010 tx=0x7f229c00b260 comp rx=0 tx=0).stop 2023-07-21T19:07:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 msgr2=0x7f22ac197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f22a800c990 tx=0x7f22a800cca0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 shutdown_connections 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22980a1220 0x7f22980a36d0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f22ac101680 0x7f22ac19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f22ac0ffb00 0x7f22ac197930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 --2- 172.21.15.5:0/389338083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f22ac100cf0 0x7f22ac197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.762+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 >> 172.21.15.5:0/389338083 conn(0x7f22ac0fb0e0 msgr2=0x7f22ac106590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.763+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 shutdown_connections 2023-07-21T19:07:08.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.763+0000 7f22b37ff700 1 -- 172.21.15.5:0/389338083 wait complete. 2023-07-21T19:07:08.792 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-07-21T19:05:37.254064+0000', 'modified': '2023-07-21T19:05:43.279232+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24614, 'mds_1': 24566, 'mds_2': 24553}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24511': {'gid': 24511, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6835/2855288918', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6834', 'nonce': 2855288918}, {'type': 'v1', 'addr': '172.21.15.156:6835', 'nonce': 2855288918}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24523': {'gid': 24523, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.156:6837/1947602937', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6836', 'nonce': 1947602937}, {'type': 'v1', 'addr': '172.21.15.156:6837', 'nonce': 1947602937}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24524': {'gid': 24524, 'name': 'e', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.101:6835/153908041', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6834', 'nonce': 153908041}, {'type': 'v1', 'addr': '172.21.15.101:6835', 'nonce': 153908041}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24553': {'gid': 24553, 'name': 'i', 'rank': 2, 'incarnation': 21, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.156:6839/2252036160', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.156:6838', 'nonce': 2252036160}, {'type': 'v1', 'addr': '172.21.15.156:6839', 'nonce': 2252036160}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24566': {'gid': 24566, 'name': 'h', 'rank': 1, 'incarnation': 19, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.101:6837/3565857598', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6836', 'nonce': 3565857598}, {'type': 'v1', 'addr': '172.21.15.101:6837', 'nonce': 3565857598}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24614': {'gid': 24614, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.101:6839/715074805', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.101:6838', 'nonce': 715074805}, {'type': 'v1', 'addr': '172.21.15.101:6839', 'nonce': 715074805}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-07-21T19:07:08.793 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-07-21T19:07:08.793 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-07-21T19:07:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:08 smithi101 ceph-mon[128136]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:07:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/430979761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/562531165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:08.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:08 smithi156 ceph-mon[128316]: pgmap v347: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:07:08.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/430979761' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:08.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/562531165' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:08.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 -- 172.21.15.5:0/3847463516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 msgr2=0x7f9df00ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:08.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 --2- 172.21.15.5:0/3847463516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df00ffef0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9dec009990 tx=0x7f9dec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 -- 172.21.15.5:0/3847463516 shutdown_connections 2023-07-21T19:07:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 --2- 172.21.15.5:0/3847463516 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9df0101660 0x7f9df0107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 --2- 172.21.15.5:0/3847463516 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df00ffef0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 --2- 172.21.15.5:0/3847463516 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9df0100cd0 0x7f9df0101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:08.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 -- 172.21.15.5:0/3847463516 >> 172.21.15.5:0/3847463516 conn(0x7f9df00fb0e0 msgr2=0x7f9df00fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 -- 172.21.15.5:0/3847463516 shutdown_connections 2023-07-21T19:07:09.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.986+0000 7f9df81ff700 1 -- 172.21.15.5:0/3847463516 wait complete. 2023-07-21T19:07:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.987+0000 7f9df81ff700 1 Processor -- start 2023-07-21T19:07:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.987+0000 7f9df81ff700 1 -- start start 2023-07-21T19:07:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9df0100cd0 0x7f9df0195c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9df0101660 0x7f9df019cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019d2b0 con 0x7f9df00ffae0 2023-07-21T19:07:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019d3f0 con 0x7f9df0101660 2023-07-21T19:07:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df81ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df019d580 con 0x7f9df0100cd0 2023-07-21T19:07:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36160/0 (socket says 172.21.15.5:36160) 2023-07-21T19:07:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1327991196 learned_addr learned my addr 172.21.15.5:0/1327991196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1327991196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9df0100cd0 msgr2=0x7f9df0195c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9df0100cd0 0x7f9df0195c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1327991196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9df0101660 msgr2=0x7f9df019cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9df0101660 0x7f9df019cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.988+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1327991196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9dec0095f0 con 0x7f9df00ffae0 2023-07-21T19:07:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0098e0 tx=0x7f9dec005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9de6ffd700 1 -- 172.21.15.5:0/1327991196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec01d070 con 0x7f9df00ffae0 2023-07-21T19:07:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9de6ffd700 1 -- 172.21.15.5:0/1327991196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dec00bed0 con 0x7f9df00ffae0 2023-07-21T19:07:09.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9de6ffd700 1 -- 172.21.15.5:0/1327991196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec00f460 con 0x7f9df00ffae0 2023-07-21T19:07:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9df019d800 con 0x7f9df00ffae0 2023-07-21T19:07:09.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.989+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df019dbd0 con 0x7f9df00ffae0 2023-07-21T19:07:09.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.992+0000 7f9de6ffd700 1 -- 172.21.15.5:0/1327991196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f9dec00ba40 con 0x7f9df00ffae0 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9de6ffd700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 0x7f9ddc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 Processor -- start 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9de6ffd700 1 -- 172.21.15.5:0/1327991196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9dec0cde50 con 0x7f9df00ffae0 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9df579a700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 0x7f9ddc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 -- start start 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a73690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a74090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811977c60 con 0x560811a72480 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608119ca9f0 con 0x560811a73c20 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.994+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608119c3e80 con 0x560811a72dd0 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36172/0 (socket says 172.21.15.5:36172) 2023-07-21T19:07:09.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1262736930 learned_addr learned my addr 172.21.15.5:0/1262736930 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df579a700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 0x7f9ddc0a3600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005950 tx=0x7f9de00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df679c700 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a74090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1262736930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 msgr2=0x560811a73690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a73690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1262736930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 msgr2=0x560811a74090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a74090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 -- 172.21.15.5:0/1262736930 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608117f1d90 con 0x560811a72480 2023-07-21T19:07:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0a3130 tx=0x7f9dec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d497dd792a44822 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9ddbfff700 1 -- 172.21.15.5:0/1262736930 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec09b2a0 con 0x560811a72480 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9ddbfff700 1 -- 172.21.15.5:0/1262736930 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dec09f2d0 con 0x560811a72480 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.995+0000 7f9ddbfff700 1 -- 172.21.15.5:0/1262736930 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec099640 con 0x560811a72480 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/1262736930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 msgr2=0x560811a72890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0a3130 tx=0x7f9dec009310 comp rx=0 tx=0).stop 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/1262736930 shutdown_connections 2023-07-21T19:07:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a74090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a72890 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/1262736930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a73690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/1262736930 >> 172.21.15.5:0/1262736930 conn(0x560811a67d50 msgr2=0x560811a6d180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/1262736930 shutdown_connections 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.996+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/1262736930 wait complete. 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 -- start start 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 Processor -- start 2023-07-21T19:07:09.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a7c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a83310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811963a70 con 0x560811a72480 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811963770 con 0x560811a73c20 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.997+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608118478d0 con 0x560811a72dd0 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9e0bb77b80 10 client.0 _ll_register_callbacks cb 0x560811961710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36186/0 (socket says 172.21.15.5:36186) 2023-07-21T19:07:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df579a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a7c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 -- 172.21.15.5:0/2329186102 learned_addr learned my addr 172.21.15.5:0/2329186102 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df679c700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a83310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 -- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 msgr2=0x560811a7c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a7c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 -- 172.21.15.5:0/2329186102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 msgr2=0x560811a83310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a83310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.998+0000 7f9df5f9b700 1 -- 172.21.15.5:0/2329186102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9df019dbd0 con 0x560811a72480 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005c40 tx=0x7f9dec097710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e3c1a0bfeb6196 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec0a3660 con 0x560811a72480 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9dec09b2a0 con 0x560811a72480 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56081191d4a0 con 0x560811a72480 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec0a1070 con 0x560811a72480 2023-07-21T19:07:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 10 client.15285 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608117dac10 con 0x560811a72480 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56081191d4a0 con 0x560811a72480 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9dec097200 con 0x560811a72480 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 10 client.15285 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 10 client.15285 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 20 client.15285 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811a87190 0x560811a86240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 4 client.15285 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:08.999+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608119bce60 con 0x560811a87190 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.000+0000 7f9df579a700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811a87190 0x560811a86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.000+0000 7f9df579a700 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811a87190 0x560811a86240 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a78a25551f0c029d server_cookie=d450fa404f7ac26b in_seq=0 out_seq=0 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.000+0000 7f9dd97fa700 10 client.15285 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.001+0000 7f9dd97fa700 1 -- 172.21.15.5:0/2329186102 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5608119bce60 con 0x560811a87190 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.001+0000 7f9dd97fa700 10 client.15285 handle_command_reply: tid=0 2023-07-21T19:07:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.001+0000 7f9e0bb77b80 1 client.15285 shutdown 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.001+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 msgr2=0x560811a7bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.001+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f9dec005c40 tx=0x7f9dec097710 comp rx=0 tx=0).stop 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 shutdown_connections 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811a87190 0x560811a86240 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a73c20 0x560811a83310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811a7bd90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9dd97fa700 0 client.15285 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/2329186102 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72dd0 0x560811a7c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 >> 172.21.15.5:0/2329186102 conn(0x560811a67d50 msgr2=0x560811a6c860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 shutdown_connections 2023-07-21T19:07:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/2329186102 wait complete. 2023-07-21T19:07:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.002+0000 7f9e0bb77b80 20 client.15285 trim_cache size 0 max 16384 2023-07-21T19:07:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.020+0000 7f9e0bb77b80 1 Processor -- start 2023-07-21T19:07:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 -- start start 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72480 0x560811a86240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811a832d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56081196d470 con 0x560811a73c20 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811992790 con 0x560811a72dd0 2023-07-21T19:07:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811853de0 con 0x560811a72480 2023-07-21T19:07:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36192/0 (socket says 172.21.15.5:36192) 2023-07-21T19:07:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 -- 172.21.15.5:0/3264758401 learned_addr learned my addr 172.21.15.5:0/3264758401 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72480 0x560811a86240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df579a700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811a832d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 -- 172.21.15.5:0/3264758401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72480 msgr2=0x560811a86240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72480 0x560811a86240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 -- 172.21.15.5:0/3264758401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 msgr2=0x560811a832d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.021+0000 7f9df679c700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811a832d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9df679c700 1 -- 172.21.15.5:0/3264758401 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560811818200 con 0x560811a73c20 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9df579a700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811a832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9df679c700 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9de80123b0 tx=0x7f9de80126c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e69ce3b9f1e5c074 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9de4ff9700 1 -- 172.21.15.5:0/3264758401 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8008db0 con 0x560811a73c20 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9de4ff9700 1 -- 172.21.15.5:0/3264758401 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9de8012e70 con 0x560811a73c20 2023-07-21T19:07:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9de4ff9700 1 -- 172.21.15.5:0/3264758401 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9de8016430 con 0x560811a73c20 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/3264758401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 msgr2=0x560811a83b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.022+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f9de80123b0 tx=0x7f9de80126c0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/3264758401 shutdown_connections 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a73c20 0x560811a83b40 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a72480 0x560811a86240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/3264758401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811a832d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/3264758401 >> 172.21.15.5:0/3264758401 conn(0x560811a67d50 msgr2=0x560811a8f8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/3264758401 shutdown_connections 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/3264758401 wait complete. 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 -- start start 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.023+0000 7f9e0bb77b80 1 Processor -- start 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811adcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a73c20 0x560811adf480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811adfa00 con 0x560811a72480 2023-07-21T19:07:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811adfb70 con 0x560811a72dd0 2023-07-21T19:07:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560811adfd30 con 0x560811a73c20 2023-07-21T19:07:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9e0bb77b80 10 client.0 _ll_register_callbacks cb 0x7f9de80087a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36204/0 (socket says 172.21.15.5:36204) 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 -- 172.21.15.5:0/830778398 learned_addr learned my addr 172.21.15.5:0/830778398 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df679c700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a73c20 0x560811adf480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df579a700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811adcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 -- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a73c20 msgr2=0x560811adf480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a73c20 0x560811adf480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 -- 172.21.15.5:0/830778398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 msgr2=0x560811adcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811adcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df5f9b700 1 -- 172.21.15.5:0/830778398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9de8010040 con 0x560811a72480 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.024+0000 7f9df579a700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811adcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9df5f9b700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0c7040 tx=0x7f9dec022930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0fb72c3746c4bce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:09.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dec09b2a0 con 0x560811a72480 2023-07-21T19:07:09.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f9dec0a3660 con 0x560811a72480 2023-07-21T19:07:09.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9dec09f2d0 con 0x560811a72480 2023-07-21T19:07:09.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dec09a440 con 0x560811a72480 2023-07-21T19:07:09.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 10 client.15291 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5608117f0260 con 0x560811a72480 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dec09f2d0 con 0x560811a72480 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9dec097090 con 0x560811a72480 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 10 client.15291 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 10 client.15291 resolve_mds: resolved name 'i' to daemon mds.i 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.025+0000 7f9e0bb77b80 20 client.15291 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811ae4480 0x560811ae0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9e0bb77b80 4 client.15291 mds_command: new command op to 24553 tid=0[{"prefix": "status"}] 2023-07-21T19:07:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56081197f000 con 0x560811ae4480 2023-07-21T19:07:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9df579a700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811ae4480 0x560811ae0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9df579a700 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811ae4480 0x560811ae0ed0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a6687a2e90bbd4d3 server_cookie=b8e6bd140576aea2 in_seq=0 out_seq=0 2023-07-21T19:07:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.026+0000 7f9dd97fa700 10 client.15291 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9dd97fa700 1 -- 172.21.15.5:0/830778398 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x56081197f000 con 0x560811ae4480 2023-07-21T19:07:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9dd97fa700 10 client.15291 handle_command_reply: tid=0 2023-07-21T19:07:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 client.15291 shutdown 2023-07-21T19:07:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 msgr2=0x560811ada9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0c7040 tx=0x7f9dec022930 comp rx=0 tx=0).stop 2023-07-21T19:07:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 shutdown_connections 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560811a73c20 0x560811adf480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560811a72480 0x560811ada9e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x560811ae4480 0x560811ae0ed0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 --2- 172.21.15.5:0/830778398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560811a72dd0 0x560811adcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9dd97fa700 0 client.15291 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 >> 172.21.15.5:0/830778398 conn(0x560811a67d50 msgr2=0x560811a86240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 shutdown_connections 2023-07-21T19:07:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 1 -- 172.21.15.5:0/830778398 wait complete. 2023-07-21T19:07:09.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.027+0000 7f9e0bb77b80 20 client.15291 trim_cache size 0 max 16384 2023-07-21T19:07:09.050 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:09.050 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:07:09.050 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 2, 2023-07-21T19:07:09.050 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24553, 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 86.921483475000002, 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:07:09.051 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 107.68187007 2023-07-21T19:07:09.052 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 msgr2=0x7f9ddc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 0x7f9ddc0a3600 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9de0005950 tx=0x7f9de00058e0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 msgr2=0x7f9df0195730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9dec0098e0 tx=0x7f9dec005ef0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 shutdown_connections 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f9ddc0a1150 0x7f9ddc0a3600 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9df0101660 0x7f9df019cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9df00ffae0 0x7f9df0195730 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 --2- 172.21.15.5:0/1327991196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9df0100cd0 0x7f9df0195c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 >> 172.21.15.5:0/1327991196 conn(0x7f9df00fb0e0 msgr2=0x7f9df0104200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 shutdown_connections 2023-07-21T19:07:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.028+0000 7f9df81ff700 1 -- 172.21.15.5:0/1327991196 wait complete. 2023-07-21T19:07:09.056 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2023-07-21T19:07:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.248+0000 7f583cfff700 1 -- 172.21.15.5:0/401414455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 msgr2=0x7f58380ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.248+0000 7f583cfff700 1 --2- 172.21.15.5:0/401414455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58380ffef0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f5828009990 tx=0x7f5828009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 -- 172.21.15.5:0/401414455 shutdown_connections 2023-07-21T19:07:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 --2- 172.21.15.5:0/401414455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 0x7f5838107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 --2- 172.21.15.5:0/401414455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58380ffef0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 --2- 172.21.15.5:0/401414455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 0x7f5838101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 -- 172.21.15.5:0/401414455 >> 172.21.15.5:0/401414455 conn(0x7f58380fb0e0 msgr2=0x7f58380fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 -- 172.21.15.5:0/401414455 shutdown_connections 2023-07-21T19:07:09.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.249+0000 7f583cfff700 1 -- 172.21.15.5:0/401414455 wait complete. 2023-07-21T19:07:09.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 Processor -- start 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 -- start start 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 0x7f5838195ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 0x7f583819cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583819d320 con 0x7f58380ffae0 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583819d460 con 0x7f5838101660 2023-07-21T19:07:09.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.250+0000 7f583cfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583819d5f0 con 0x7f5838100cd0 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36210/0 (socket says 172.21.15.5:36210) 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 -- 172.21.15.5:0/3921288082 learned_addr learned my addr 172.21.15.5:0/3921288082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f582ffff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 0x7f5838195ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 0x7f583819cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 -- 172.21.15.5:0/3921288082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 msgr2=0x7f5838195ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 0x7f5838195ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 -- 172.21.15.5:0/3921288082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 msgr2=0x7f583819cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 0x7f583819cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 -- 172.21.15.5:0/3921288082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58280095f0 con 0x7f58380ffae0 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f583659c700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5828000c00 tx=0x7f58280047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.251+0000 7f582f7fe700 1 -- 172.21.15.5:0/3921288082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582801d070 con 0x7f58380ffae0 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.252+0000 7f582f7fe700 1 -- 172.21.15.5:0/3921288082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582800be50 con 0x7f58380ffae0 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.252+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583819d870 con 0x7f58380ffae0 2023-07-21T19:07:09.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.252+0000 7f582f7fe700 1 -- 172.21.15.5:0/3921288082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582800f460 con 0x7f58380ffae0 2023-07-21T19:07:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.252+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583819dc90 con 0x7f58380ffae0 2023-07-21T19:07:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.255+0000 7f582f7fe700 1 -- 172.21.15.5:0/3921288082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f5828022670 con 0x7f58380ffae0 2023-07-21T19:07:09.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f5850988b80 1 Processor -- start 2023-07-21T19:07:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f5850988b80 1 -- start start 2023-07-21T19:07:09.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f582f7fe700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 0x7f58240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f582f7fe700 1 -- 172.21.15.5:0/3921288082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f58280cdf80 con 0x7f58380ffae0 2023-07-21T19:07:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f5850988b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f5850988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f24aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.256+0000 7f5850988b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f25030 0x564402f254a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5850988b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e24c60 con 0x564402f23890 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5850988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e779f0 con 0x564402f241e0 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5850988b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e70e80 con 0x564402f25030 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 0x7f58240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36218/0 (socket says 172.21.15.5:36218) 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 -- 172.21.15.5:0/1725968590 learned_addr learned my addr 172.21.15.5:0/1725968590 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f582ffff700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f24aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5836d9d700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f25030 0x564402f254a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 -- 172.21.15.5:0/1725968590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f25030 msgr2=0x564402f254a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f25030 0x564402f254a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 -- 172.21.15.5:0/1725968590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 msgr2=0x564402f24aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f24aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 0x7f58240a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5830003f20 tx=0x7f5830009ba0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f583659c700 1 -- 172.21.15.5:0/1725968590 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564402c9ed90 con 0x564402f23890 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.257+0000 7f582ffff700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f24aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f583659c700 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58280a3260 tx=0x7f5828004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66c51c1c9290274 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f582cff9700 1 -- 172.21.15.5:0/1725968590 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582809b3d0 con 0x564402f23890 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f582cff9700 1 -- 172.21.15.5:0/1725968590 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582809f400 con 0x564402f23890 2023-07-21T19:07:09.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f582cff9700 1 -- 172.21.15.5:0/1725968590 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582809a8c0 con 0x564402f23890 2023-07-21T19:07:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f5850988b80 1 -- 172.21.15.5:0/1725968590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 msgr2=0x564402f23ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.258+0000 7f5850988b80 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f58280a3260 tx=0x7f5828004320 comp rx=0 tx=0).stop 2023-07-21T19:07:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 -- 172.21.15.5:0/1725968590 shutdown_connections 2023-07-21T19:07:09.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f25030 0x564402f254a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f23890 0x564402f23ca0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 --2- 172.21.15.5:0/1725968590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f24aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 -- 172.21.15.5:0/1725968590 >> 172.21.15.5:0/1725968590 conn(0x564402f14d50 msgr2=0x564402f1a180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 -- 172.21.15.5:0/1725968590 shutdown_connections 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 -- 172.21.15.5:0/1725968590 wait complete. 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 -- start start 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.259+0000 7f5850988b80 1 Processor -- start 2023-07-21T19:07:09.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f23890 0x564402f2d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 0x564402f2d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e10a70 con 0x564402f25030 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e10770 con 0x564402f23890 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402cf48d0 con 0x564402f241e0 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5850988b80 10 client.0 _ll_register_callbacks cb 0x564402e0e710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5836d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5836d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36224/0 (socket says 172.21.15.5:36224) 2023-07-21T19:07:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f5836d9d700 1 -- 172.21.15.5:0/3747378959 learned_addr learned my addr 172.21.15.5:0/3747378959 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f583659c700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f23890 0x564402f2d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.260+0000 7f582ffff700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 0x564402f2d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 -- 172.21.15.5:0/3747378959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 msgr2=0x564402f2d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 0x564402f2d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 -- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f23890 msgr2=0x564402f2d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f23890 0x564402f2d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 -- 172.21.15.5:0/3747378959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583819dc90 con 0x564402f25030 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f582ffff700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 0x564402f2d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5830008890 tx=0x7f5830004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a96b21dacb6eeba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f581dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:09.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830017770 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564402dca4a0 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f58300178d0 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.261+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5830017a90 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 10 client.15303 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564402c87c10 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564402dca4a0 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5830020070 con 0x564402f25030 2023-07-21T19:07:09.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 10 client.15303 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 10 client.15303 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T19:07:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 20 client.15303 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f38500 0x564402f378b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 4 client.15303 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:09.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564402e69e60 con 0x564402f38500 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.262+0000 7f583659c700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f38500 0x564402f378b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.263+0000 7f583659c700 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f38500 0x564402f378b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a9d71aca2ae6de7 server_cookie=325caaf0fa43739f in_seq=0 out_seq=0 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.263+0000 7f581dffb700 10 client.15303 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f581dffb700 1 -- 172.21.15.5:0/3747378959 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564402e69e60 con 0x564402f38500 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f581dffb700 10 client.15303 handle_command_reply: tid=0 2023-07-21T19:07:09.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 client.15303 shutdown 2023-07-21T19:07:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 msgr2=0x564402f34700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f5830008890 tx=0x7f5830004b40 comp rx=0 tx=0).stop 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 shutdown_connections 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f34700 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f38500 0x564402f378b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f23890 0x564402f2d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 --2- 172.21.15.5:0/3747378959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f241e0 0x564402f2d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f581dffb700 0 client.15303 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:09.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 >> 172.21.15.5:0/3747378959 conn(0x564402f14d50 msgr2=0x564402f19860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 shutdown_connections 2023-07-21T19:07:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.264+0000 7f5850988b80 1 -- 172.21.15.5:0/3747378959 wait complete. 2023-07-21T19:07:09.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.265+0000 7f5850988b80 20 client.15303 trim_cache size 0 max 16384 2023-07-21T19:07:09.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 Processor -- start 2023-07-21T19:07:09.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 -- start start 2023-07-21T19:07:09.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f34ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e779f0 con 0x564402f25030 2023-07-21T19:07:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402e69e60 con 0x564402f241e0 2023-07-21T19:07:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.283+0000 7f5850988b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402dca4a0 con 0x564402f23890 2023-07-21T19:07:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f583659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f34ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36230/0 (socket says 172.21.15.5:36230) 2023-07-21T19:07:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 -- 172.21.15.5:0/265447662 learned_addr learned my addr 172.21.15.5:0/265447662 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f582ffff700 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 -- 172.21.15.5:0/265447662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 msgr2=0x564402f34ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f34ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 -- 172.21.15.5:0/265447662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 msgr2=0x564402f34620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 -- 172.21.15.5:0/265447662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564402f2c210 con 0x564402f25030 2023-07-21T19:07:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.284+0000 7f5836d9d700 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f80 tx=0x7f5830004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89a6b09816c2b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f582d7fa700 1 -- 172.21.15.5:0/265447662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583002c620 con 0x564402f25030 2023-07-21T19:07:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f582d7fa700 1 -- 172.21.15.5:0/265447662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f583000f040 con 0x564402f25030 2023-07-21T19:07:09.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f582d7fa700 1 -- 172.21.15.5:0/265447662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5830035440 con 0x564402f25030 2023-07-21T19:07:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 -- 172.21.15.5:0/265447662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 msgr2=0x564402f2b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f5830000f80 tx=0x7f5830004270 comp rx=0 tx=0).stop 2023-07-21T19:07:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 -- 172.21.15.5:0/265447662 shutdown_connections 2023-07-21T19:07:09.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f25030 0x564402f2b380 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f34ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 --2- 172.21.15.5:0/265447662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 -- 172.21.15.5:0/265447662 >> 172.21.15.5:0/265447662 conn(0x564402f14d50 msgr2=0x564402f41230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 -- 172.21.15.5:0/265447662 shutdown_connections 2023-07-21T19:07:09.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.285+0000 7f5850988b80 1 -- 172.21.15.5:0/265447662 wait complete. 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 -- start start 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 Processor -- start 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f2a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f25030 0x564402f8c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402f8cab0 con 0x564402f241e0 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402f8cc20 con 0x564402f25030 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564402f8cde0 con 0x564402f23890 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.286+0000 7f5850988b80 10 client.0 _ll_register_callbacks cb 0x7f5830004750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f582ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f582ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36242/0 (socket says 172.21.15.5:36242) 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f582ffff700 1 -- 172.21.15.5:0/3565290221 learned_addr learned my addr 172.21.15.5:0/3565290221 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f5836d9d700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f25030 0x564402f8c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f2a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 -- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f25030 msgr2=0x564402f8c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f25030 0x564402f8c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 -- 172.21.15.5:0/3565290221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 msgr2=0x564402f34620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 -- 172.21.15.5:0/3565290221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564402c87c10 con 0x564402f23890 2023-07-21T19:07:09.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f582ffff700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f583659c700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f2a820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f582809b0d0 tx=0x7f582809f120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb807672efc1e3c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.287+0000 7f581dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5828099510 con 0x564402f23890 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f582809f400 con 0x564402f23890 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f5828099770 con 0x564402f23890 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58280ca050 con 0x564402f23890 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 10 client.25096 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x564402cc5200 con 0x564402f23890 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f582809f400 con 0x564402f23890 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5828097ae0 con 0x564402f23890 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 10 client.25096 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 10 client.25096 resolve_mds: resolved name 'h' to daemon mds.h 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 20 client.25096 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f91730 0x564402f8e080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 4 client.25096 mds_command: new command op to 24566 tid=0[{"prefix": "status"}] 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.288+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564402e2c000 con 0x564402f91730 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.289+0000 7f582ffff700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f91730 0x564402f8e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.289+0000 7f582ffff700 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f91730 0x564402f8e080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f58558c734fd8e26 server_cookie=ac68975c5bb2d447 in_seq=0 out_seq=0 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.289+0000 7f581dffb700 10 client.25096 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f581dffb700 1 -- 172.21.15.5:0/3565290221 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x564402e2c000 con 0x564402f91730 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f581dffb700 10 client.25096 handle_command_reply: tid=0 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 client.25096 shutdown 2023-07-21T19:07:09.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 msgr2=0x564402f2a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f2a820 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f582809b0d0 tx=0x7f582809f120 comp rx=0 tx=0).stop 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 shutdown_connections 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x564402f91730 0x564402f8e080 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564402f25030 0x564402f8c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564402f23890 0x564402f2a820 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 --2- 172.21.15.5:0/3565290221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564402f241e0 0x564402f34620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f581dffb700 0 client.25096 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 >> 172.21.15.5:0/3565290221 conn(0x564402f14d50 msgr2=0x564402f34ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 shutdown_connections 2023-07-21T19:07:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 1 -- 172.21.15.5:0/3565290221 wait complete. 2023-07-21T19:07:09.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.290+0000 7f5850988b80 20 client.25096 trim_cache size 0 max 16384 2023-07-21T19:07:09.313 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 1, 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24566, 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 88.193815752000006, 2023-07-21T19:07:09.314 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:07:09.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:07:09.315 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:07:09.315 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 111.082371761 2023-07-21T19:07:09.315 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 msgr2=0x7f58240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 0x7f58240a3600 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5830003f20 tx=0x7f5830009ba0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 msgr2=0x7f58381957a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f5828000c00 tx=0x7f58280047a0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 shutdown_connections 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f58240a1150 0x7f58240a3600 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5838101660 0x7f583819cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58380ffae0 0x7f58381957a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 --2- 172.21.15.5:0/3921288082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5838100cd0 0x7f5838195ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 >> 172.21.15.5:0/3921288082 conn(0x7f58380fb0e0 msgr2=0x7f58380fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 shutdown_connections 2023-07-21T19:07:09.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.291+0000 7f583cfff700 1 -- 172.21.15.5:0/3921288082 wait complete. 2023-07-21T19:07:09.319 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-07-21T19:07:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 -- 172.21.15.5:0/703613837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 msgr2=0x7ff0e0107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/703613837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e0107f80 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800b0a0 tx=0x7ff0d800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 -- 172.21.15.5:0/703613837 shutdown_connections 2023-07-21T19:07:09.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/703613837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e0107f80 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/703613837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e00ffeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/703613837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e01011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 -- 172.21.15.5:0/703613837 >> 172.21.15.5:0/703613837 conn(0x7ff0e00fb0c0 msgr2=0x7ff0e00fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 -- 172.21.15.5:0/703613837 shutdown_connections 2023-07-21T19:07:09.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.513+0000 7ff0e67ff700 1 -- 172.21.15.5:0/703613837 wait complete. 2023-07-21T19:07:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.514+0000 7ff0e67ff700 1 Processor -- start 2023-07-21T19:07:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.514+0000 7ff0e67ff700 1 -- start start 2023-07-21T19:07:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e01979f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e0197f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019f570 con 0x7ff0e01016e0 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019f6b0 con 0x7ff0e0100d30 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e019f840 con 0x7ff0e00ffaa0 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36270/0 (socket says 172.21.15.5:36270) 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2687926418 learned_addr learned my addr 172.21.15.5:0/2687926418 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e0197f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e01979f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2687926418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 msgr2=0x7ff0e01979f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e01979f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2687926418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 msgr2=0x7ff0e0197f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e0197f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2687926418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d8009d00 con 0x7ff0e01016e0 2023-07-21T19:07:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.515+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d010 tx=0x7ff0d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0dd7fa700 1 -- 172.21.15.5:0/2687926418 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d800e040 con 0x7ff0e01016e0 2023-07-21T19:07:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e019fac0 con 0x7ff0e01016e0 2023-07-21T19:07:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0dd7fa700 1 -- 172.21.15.5:0/2687926418 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d8003e10 con 0x7ff0e01016e0 2023-07-21T19:07:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0dd7fa700 1 -- 172.21.15.5:0/2687926418 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8005310 con 0x7ff0e01016e0 2023-07-21T19:07:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.516+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e019fe90 con 0x7ff0e01016e0 2023-07-21T19:07:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.519+0000 7ff0dd7fa700 1 -- 172.21.15.5:0/2687926418 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7ff0d801daa0 con 0x7ff0e01016e0 2023-07-21T19:07:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.520+0000 7ff0dd7fa700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 0x7ff0c80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.520+0000 7ff0fa0b3b80 1 Processor -- start 2023-07-21T19:07:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0dd7fa700 1 -- 172.21.15.5:0/2687926418 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff0d80cdda0 con 0x7ff0e01016e0 2023-07-21T19:07:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 0x7ff0c80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 -- start start 2023-07-21T19:07:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdb480 0x55d918cdb890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cdc690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918be0c60 con 0x55d918cdcc20 2023-07-21T19:07:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918c339f0 con 0x55d918cdbdd0 2023-07-21T19:07:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918c2ce80 con 0x55d918cdb480 2023-07-21T19:07:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.521+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 0x7ff0c80a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009be0 tx=0x7ff0d0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36274/0 (socket says 172.21.15.5:36274) 2023-07-21T19:07:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2670048361 learned_addr learned my addr 172.21.15.5:0/2670048361 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cdc690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdb480 0x55d918cdb890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2670048361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdb480 msgr2=0x55d918cdb890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdb480 0x55d918cdb890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2670048361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 msgr2=0x55d918cdc690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cdc690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 -- 172.21.15.5:0/2670048361 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d918a5ad90 con 0x55d918cdcc20 2023-07-21T19:07:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cdc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80a3080 tx=0x7ff0d80975b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6ebe0563401ce8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0c67fc700 1 -- 172.21.15.5:0/2670048361 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d809ab50 con 0x55d918cdcc20 2023-07-21T19:07:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0c67fc700 1 -- 172.21.15.5:0/2670048361 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d809b1f0 con 0x55d918cdcc20 2023-07-21T19:07:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.522+0000 7ff0c67fc700 1 -- 172.21.15.5:0/2670048361 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d8099590 con 0x55d918cdcc20 2023-07-21T19:07:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2670048361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 msgr2=0x55d918cdd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80a3080 tx=0x7ff0d80975b0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2670048361 shutdown_connections 2023-07-21T19:07:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdcc20 0x55d918cdd090 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdb480 0x55d918cdb890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2670048361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cdc690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2670048361 >> 172.21.15.5:0/2670048361 conn(0x55d918cd0d50 msgr2=0x55d918cd6180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2670048361 shutdown_connections 2023-07-21T19:07:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.523+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2670048361 wait complete. 2023-07-21T19:07:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.524+0000 7ff0fa0b3b80 1 -- start start 2023-07-21T19:07:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.524+0000 7ff0fa0b3b80 1 Processor -- start 2023-07-21T19:07:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 0x55d918ce4ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cec470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918bcca70 con 0x55d918cdbdd0 2023-07-21T19:07:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918bcc770 con 0x55d918cdb480 2023-07-21T19:07:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918ab08d0 con 0x55d918cdcc20 2023-07-21T19:07:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0fa0b3b80 10 client.0 _ll_register_callbacks cb 0x55d918bca710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36290/0 (socket says 172.21.15.5:36290) 2023-07-21T19:07:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0df7fe700 1 -- 172.21.15.5:0/2547819048 learned_addr learned my addr 172.21.15.5:0/2547819048 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.525+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cec470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 0x55d918ce4ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 -- 172.21.15.5:0/2547819048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 msgr2=0x55d918cec470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cec470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 -- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 msgr2=0x55d918ce4ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 0x55d918ce4ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 -- 172.21.15.5:0/2547819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0d40098f0 con 0x55d918cdbdd0 2023-07-21T19:07:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 0x55d918ce4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4012a10 tx=0x7ff0d4012d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7feb6fccf44603b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d4017070 con 0x55d918cdbdd0 2023-07-21T19:07:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d918b864a0 con 0x55d918cdbdd0 2023-07-21T19:07:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.526+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff0d4011ac0 con 0x55d918cdbdd0 2023-07-21T19:07:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d4011c80 con 0x55d918cdbdd0 2023-07-21T19:07:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 10 client.15321 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7ff0e019fe90 con 0x55d918cdbdd0 2023-07-21T19:07:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d918b864a0 con 0x55d918cdbdd0 2023-07-21T19:07:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0d401c420 con 0x55d918cdbdd0 2023-07-21T19:07:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 10 client.15321 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 10 client.15321 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T19:07:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 20 client.15321 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918cf0220 0x55d918cef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 4 client.15321 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.527+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d918c25e60 con 0x55d918cf0220 2023-07-21T19:07:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.528+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918cf0220 0x55d918cef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.528+0000 7ff0dffff700 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918cf0220 0x55d918cef5d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac3f17029305ac18 server_cookie=26c487c0c6476a35 in_seq=0 out_seq=0 2023-07-21T19:07:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.528+0000 7ff0bf7fe700 10 client.15321 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/2547819048 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d918c25e60 con 0x55d918cf0220 2023-07-21T19:07:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0bf7fe700 10 client.15321 handle_command_reply: tid=0 2023-07-21T19:07:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0fa0b3b80 1 client.15321 shutdown 2023-07-21T19:07:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 msgr2=0x55d918ce5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4012a10 tx=0x7ff0d4012d20 comp rx=0 tx=0).stop 2023-07-21T19:07:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 shutdown_connections 2023-07-21T19:07:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.529+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cec470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdb480 0x55d918ce4ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918cf0220 0x55d918cef5d0 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/2547819048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdbdd0 0x55d918ce5430 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0bf7fe700 0 client.15321 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 >> 172.21.15.5:0/2547819048 conn(0x55d918cd0d50 msgr2=0x55d918cd5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 shutdown_connections 2023-07-21T19:07:09.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/2547819048 wait complete. 2023-07-21T19:07:09.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.530+0000 7ff0fa0b3b80 20 client.15321 trim_cache size 0 max 16384 2023-07-21T19:07:09.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.548+0000 7ff0fa0b3b80 1 Processor -- start 2023-07-21T19:07:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 -- start start 2023-07-21T19:07:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cec430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cef5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918c25e60 con 0x55d918cdb480 2023-07-21T19:07:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918b864a0 con 0x55d918cdbdd0 2023-07-21T19:07:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918bfb790 con 0x55d918cdcc20 2023-07-21T19:07:09.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36304/0 (socket says 172.21.15.5:36304) 2023-07-21T19:07:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 -- 172.21.15.5:0/316694114 learned_addr learned my addr 172.21.15.5:0/316694114 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cef5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cec430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 -- 172.21.15.5:0/316694114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 msgr2=0x55d918cef5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cef5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 -- 172.21.15.5:0/316694114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 msgr2=0x55d918cec430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cec430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0dffff700 1 -- 172.21.15.5:0/316694114 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d918a43c10 con 0x55d918cdb480 2023-07-21T19:07:09.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.549+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0dffff700 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55d918ce4e90 tx=0x7ff0d0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c38c782969be99 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0c6ffd700 1 -- 172.21.15.5:0/316694114 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0027070 con 0x55d918cdb480 2023-07-21T19:07:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0c6ffd700 1 -- 172.21.15.5:0/316694114 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d0010ea0 con 0x55d918cdb480 2023-07-21T19:07:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0c6ffd700 1 -- 172.21.15.5:0/316694114 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0021440 con 0x55d918cdb480 2023-07-21T19:07:09.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/316694114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 msgr2=0x55d918ce2fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.550+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x55d918ce4e90 tx=0x7ff0d0016000 comp rx=0 tx=0).stop 2023-07-21T19:07:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/316694114 shutdown_connections 2023-07-21T19:07:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918cef5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918ce2fc0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/316694114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918cec430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/316694114 >> 172.21.15.5:0/316694114 conn(0x55d918cd0d50 msgr2=0x55d918cd5860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/316694114 shutdown_connections 2023-07-21T19:07:09.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/316694114 wait complete. 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 -- start start 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.551+0000 7ff0fa0b3b80 1 Processor -- start 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918ceab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918ceb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918ceb730 con 0x55d918cdb480 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918ceb8a0 con 0x55d918cdbdd0 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d918ceba60 con 0x55d918cdcc20 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0fa0b3b80 10 client.0 _ll_register_callbacks cb 0x7ff0d40124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36306/0 (socket says 172.21.15.5:36306) 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.552+0000 7ff0dffff700 1 -- 172.21.15.5:0/479686419 learned_addr learned my addr 172.21.15.5:0/479686419 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:09.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918ceab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918ceb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 -- 172.21.15.5:0/479686419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 msgr2=0x55d918ceb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918ceb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 -- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 msgr2=0x55d918ceab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918ceab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 -- 172.21.15.5:0/479686419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d918a81200 con 0x55d918cdb480 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0e4d9c700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918ceb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0dffff700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00094f0 tx=0x7ff0d00275b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=762d2bcb467a937 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0bf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0d0025070 con 0x55d918cdb480 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0d809b1f0 con 0x55d918cdb480 2023-07-21T19:07:09.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7ff0d0004030 con 0x55d918cdb480 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0d00041f0 con 0x55d918cdb480 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.553+0000 7ff0fa0b3b80 10 client.15327 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55d918a59260 con 0x55d918cdb480 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0d809b1f0 con 0x55d918cdb480 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0d0021460 con 0x55d918cdb480 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 10 client.15327 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:09.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 10 client.15327 resolve_mds: resolved name 'b' to daemon mds.b 2023-07-21T19:07:09.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 20 client.15327 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:09.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918d49c40 0x55d918d465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:09.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 4 client.15327 mds_command: new command op to 24614 tid=0[{"prefix": "status"}] 2023-07-21T19:07:09.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d918be8000 con 0x55d918d49c40 2023-07-21T19:07:09.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.554+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918d49c40 0x55d918d465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:09.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.555+0000 7ff0df7fe700 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918d49c40 0x55d918d465d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3badd41533c70f3b server_cookie=a06a9c25ecbb0513 in_seq=0 out_seq=0 2023-07-21T19:07:09.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.555+0000 7ff0bf7fe700 10 client.15327 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:09.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.555+0000 7ff0bf7fe700 1 -- 172.21.15.5:0/479686419 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55d918be8000 con 0x55d918d49c40 2023-07-21T19:07:09.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.555+0000 7ff0bf7fe700 10 client.15327 handle_command_reply: tid=0 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.555+0000 7ff0fa0b3b80 1 client.15327 shutdown 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 msgr2=0x55d918cea5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7ff0d00094f0 tx=0x7ff0d00275b0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 shutdown_connections 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d918cdcc20 0x55d918ceb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d918d49c40 0x55d918d465d0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d918cdb480 0x55d918cea5d0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 --2- 172.21.15.5:0/479686419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d918cdbdd0 0x55d918ceab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0bf7fe700 0 client.15327 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 >> 172.21.15.5:0/479686419 conn(0x55d918cd0d50 msgr2=0x55d918ce2fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 shutdown_connections 2023-07-21T19:07:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 1 -- 172.21.15.5:0/479686419 wait complete. 2023-07-21T19:07:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0fa0b3b80 20 client.15327 trim_cache size 0 max 16384 2023-07-21T19:07:09.592 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "cluster_fsid": "3dc43aa4-27f8-11ee-9b34-001a4aab830c", 2023-07-21T19:07:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "whoami": 0, 2023-07-21T19:07:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 24614, 2023-07-21T19:07:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "want_state": "up:active", 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "state": "up:active", 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs", 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "rank_uptime": 91.877339671000001, 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "mdsmap_epoch": 23, 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch": 87, 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_epoch_barrier": 85, 2023-07-21T19:07:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "uptime": 98.998948581999997 2023-07-21T19:07:09.594 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:09.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 msgr2=0x7ff0c80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 0x7ff0c80a3600 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff0d0009be0 tx=0x7ff0d0009260 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 msgr2=0x7ff0e019efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7ff0d800d010 tx=0x7ff0d800bd70 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 shutdown_connections 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.556+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7ff0c80a1150 0x7ff0c80a3600 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0e01016e0 0x7ff0e019efa0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0e00ffaa0 0x7ff0e01979f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 --2- 172.21.15.5:0/2687926418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0e0100d30 0x7ff0e0197f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 >> 172.21.15.5:0/2687926418 conn(0x7ff0e00fb0c0 msgr2=0x7ff0e0106620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 shutdown_connections 2023-07-21T19:07:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:09.557+0000 7ff0e67ff700 1 -- 172.21.15.5:0/2687926418 wait complete. 2023-07-21T19:07:09.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:09 smithi005 ceph-mon[126460]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-21T19:07:09.596 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:09 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/389338083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:09.597 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-07-21T19:07:09.630 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-07-21T19:07:09.630 INFO:tasks.ceph_fuse:config is {'client.0': {}} 2023-07-21T19:07:09.631 INFO:tasks.ceph_fuse:client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-07-21T19:07:09.631 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-07-21T19:07:09.632 INFO:tasks.cephfs.mount:using client_config["hostfs_mntpt"] = /tmp/cephfs 2023-07-21T19:07:09.632 INFO:tasks.cephfs.mount:hostfs_mntpt = /tmp/cephfs 2023-07-21T19:07:09.632 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:09.632 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-07-21T19:07:09.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:09.678 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-07-21T19:07:09.791 INFO:teuthology.orchestra.run.smithi005.stderr:Cannot find device "ceph-brx" 2023-07-21T19:07:09.793 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T19:07:09.794 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-07-21T19:07:09.795 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph fs ls 2023-07-21T19:07:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:09 smithi101 ceph-mon[128136]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-21T19:07:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:09 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/389338083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:09 smithi156 ceph-mon[128316]: mgrmap e18: x(active, since 9m), standbys: z, y 2023-07-21T19:07:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:09 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/389338083' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:10.521 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:07:10.788 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:10 smithi005 ceph-mon[126460]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:10.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:10 smithi101 ceph-mon[128136]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:10.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:10 smithi156 ceph-mon[128316]: pgmap v348: 97 pgs: 97 active+clean; 637 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 5 op/s 2023-07-21T19:07:11.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 -- 172.21.15.5:0/1064312988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 msgr2=0x7f4de01001d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1064312988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de01001d0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc009990 tx=0x7f4ddc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 -- 172.21.15.5:0/1064312988 shutdown_connections 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1064312988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de0104ea0 0x7f4de010ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1064312988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de01001d0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1064312988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de01007a0 0x7f4de0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 -- 172.21.15.5:0/1064312988 >> 172.21.15.5:0/1064312988 conn(0x7f4de00fb660 msgr2=0x7f4de00fda70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:11.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.954+0000 7f4de7b84700 1 -- 172.21.15.5:0/1064312988 shutdown_connections 2023-07-21T19:07:11.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.955+0000 7f4de7b84700 1 -- 172.21.15.5:0/1064312988 wait complete. 2023-07-21T19:07:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 Processor -- start 2023-07-21T19:07:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 -- start start 2023-07-21T19:07:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de0197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:11.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de01007a0 0x7f4de0198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019ce80 con 0x7f4de0104ea0 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019cff0 con 0x7f4de00ffe00 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de7b84700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4de019d1b0 con 0x7f4de01007a0 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.956+0000 7f4de6121700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36334/0 (socket says 172.21.15.5:36334) 2023-07-21T19:07:11.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 -- 172.21.15.5:0/1509440248 learned_addr learned my addr 172.21.15.5:0/1509440248 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de511f700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de01007a0 0x7f4de0198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de5920700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de0197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 -- 172.21.15.5:0/1509440248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de01007a0 msgr2=0x7f4de0198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de01007a0 0x7f4de0198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 -- 172.21.15.5:0/1509440248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 msgr2=0x7f4de0197ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de0197ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 -- 172.21.15.5:0/1509440248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ddc0095f0 con 0x7f4de0104ea0 2023-07-21T19:07:11.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4de6121700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ef30 tx=0x7f4dd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.957+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd8018070 con 0x7f4de0104ea0 2023-07-21T19:07:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.958+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4de019d430 con 0x7f4de0104ea0 2023-07-21T19:07:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.958+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dd800f040 con 0x7f4de0104ea0 2023-07-21T19:07:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.958+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dd800cbc0 con 0x7f4de0104ea0 2023-07-21T19:07:11.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.958+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4de01a4cf0 con 0x7f4de0104ea0 2023-07-21T19:07:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.959+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4de004e210 con 0x7f4de0104ea0 2023-07-21T19:07:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.961+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f4dd8007500 con 0x7f4de0104ea0 2023-07-21T19:07:11.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.963+0000 7f4dd6ffd700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 0x7f4dcc0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.963+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f4dd80cc350 con 0x7f4de0104ea0 2023-07-21T19:07:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.963+0000 7f4de5920700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 0x7f4dcc0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:11.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.964+0000 7f4de5920700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 0x7f4dcc0a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b2e0 tx=0x7f4ddc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:11.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:11.967+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4dd8095ae0 con 0x7f4de0104ea0 2023-07-21T19:07:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.219+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4de01a4e80 con 0x7f4de0104ea0 2023-07-21T19:07:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.220+0000 7f4dd6ffd700 1 -- 172.21.15.5:0/1509440248 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v23) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4dd8099100 con 0x7f4de0104ea0 2023-07-21T19:07:12.230 INFO:teuthology.orchestra.run.smithi005.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 msgr2=0x7f4dcc0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 0x7f4dcc0a3680 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b2e0 tx=0x7f4ddc005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 msgr2=0x7f4de019c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f4dd800ef30 tx=0x7f4dd800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 shutdown_connections 2023-07-21T19:07:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f4dcc0a11d0 0x7f4dcc0a3680 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4de0104ea0 0x7f4de019c870 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4de00ffe00 0x7f4de0197ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 --2- 172.21.15.5:0/1509440248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4de01007a0 0x7f4de0198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 >> 172.21.15.5:0/1509440248 conn(0x7f4de00fb660 msgr2=0x7f4de010bcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.222+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 shutdown_connections 2023-07-21T19:07:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:12.223+0000 7f4de7b84700 1 -- 172.21.15.5:0/1509440248 wait complete. 2023-07-21T19:07:12.592 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:12 smithi005 ceph-mon[126460]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:12.593 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:12 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1509440248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T19:07:12.603 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-07-21T19:07:12.604 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-07-21T19:07:12.604 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi005.front.sepia.ceph.com 2023-07-21T19:07:12.605 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-07-21T19:07:12.605 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /tmp/cephfs 2023-07-21T19:07:12.605 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-07-21T19:07:12.605 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-07-21T19:07:12.606 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-07-21T19:07:12.606 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/tmp/cephfs' 2023-07-21T19:07:12.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:12.607 DEBUG:teuthology.orchestra.run.smithi005:> ip addr 2023-07-21T19:07:12.652 INFO:teuthology.orchestra.run.smithi005.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-07-21T19:07:12.652 INFO:teuthology.orchestra.run.smithi005.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-07-21T19:07:12.652 INFO:teuthology.orchestra.run.smithi005.stdout: inet 127.0.0.1/8 scope host lo 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 ::1/128 scope host 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:de brd ff:ff:ff:ff:ff:ff 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f0 2023-07-21T19:07:12.653 INFO:teuthology.orchestra.run.smithi005.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:6b:fd:df brd ff:ff:ff:ff:ff:ff 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp7s0f1 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:42 brd ff:ff:ff:ff:ff:ff 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f0 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: inet 172.21.15.5/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft 40730sec preferred_lft 40730sec 2023-07-21T19:07:12.654 INFO:teuthology.orchestra.run.smithi005.stdout: inet6 fe80::ec4:7aff:febd:1442/64 scope link 2023-07-21T19:07:12.655 INFO:teuthology.orchestra.run.smithi005.stdout: valid_lft forever preferred_lft forever 2023-07-21T19:07:12.655 INFO:teuthology.orchestra.run.smithi005.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-07-21T19:07:12.655 INFO:teuthology.orchestra.run.smithi005.stdout: link/ether 0c:c4:7a:bd:14:43 brd ff:ff:ff:ff:ff:ff 2023-07-21T19:07:12.655 INFO:teuthology.orchestra.run.smithi005.stdout: altname enp1s0f1 2023-07-21T19:07:12.657 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-07-21T19:07:12.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:12.658 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-07-21T19:07:12.658 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-07-21T19:07:12.658 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add name ceph-brx type bridge 2023-07-21T19:07:12.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr flush dev ceph-brx 2023-07-21T19:07:12.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set ceph-brx up 2023-07-21T19:07:12.659 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-07-21T19:07:12.660 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-07-21T19:07:12.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:12 smithi101 ceph-mon[128136]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:12.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:12 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1509440248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T19:07:12.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:12 smithi156 ceph-mon[128316]: pgmap v349: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:12.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:12 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1509440248' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-07-21T19:07:13.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:13.153 DEBUG:teuthology.orchestra.run.smithi005:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-07-21T19:07:13.227 INFO:teuthology.orchestra.run.smithi005.stdout:1 2023-07-21T19:07:13.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:13.230 DEBUG:teuthology.orchestra.run.smithi005:> route 2023-07-21T19:07:13.288 INFO:teuthology.orchestra.run.smithi005.stdout:Kernel IP routing table 2023-07-21T19:07:13.289 INFO:teuthology.orchestra.run.smithi005.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-07-21T19:07:13.289 INFO:teuthology.orchestra.run.smithi005.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-07-21T19:07:13.289 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-07-21T19:07:13.289 INFO:teuthology.orchestra.run.smithi005.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-07-21T19:07:13.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:13.291 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-07-21T19:07:13.291 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-07-21T19:07:13.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-07-21T19:07:13.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-07-21T19:07:13.292 DEBUG:teuthology.orchestra.run.smithi005:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-07-21T19:07:13.292 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-07-21T19:07:14.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:14.038 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-07-21T19:07:14.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:14.083 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list-id 2023-07-21T19:07:14.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:14.167 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-07-21T19:07:14.167 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-07-21T19:07:14.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns add ceph-ns--tmp-cephfs 2023-07-21T19:07:14.168 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns set ceph-ns--tmp-cephfs 0 2023-07-21T19:07:14.168 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-07-21T19:07:14.351 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--tmp-cephfs' with 192.168.0.1/16 2023-07-21T19:07:14.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:14.352 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-07-21T19:07:14.352 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-07-21T19:07:14.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link add veth0 netns ceph-ns--tmp-cephfs type veth peer name brx.0 2023-07-21T19:07:14.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-07-21T19:07:14.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set veth0 up 2023-07-21T19:07:14.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip link set lo up 2023-07-21T19:07:14.353 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns exec ceph-ns--tmp-cephfs ip route add default via 192.168.255.254 2023-07-21T19:07:14.354 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-07-21T19:07:14.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:14 smithi005 ceph-mon[126460]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-21T19:07:14.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:14.651 DEBUG:teuthology.orchestra.run.smithi005:> (cd / && exec bash -c ' 2023-07-21T19:07:14.651 DEBUG:teuthology.orchestra.run.smithi005:> set -e 2023-07-21T19:07:14.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set brx.0 up 2023-07-21T19:07:14.652 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link set dev brx.0 master ceph-brx 2023-07-21T19:07:14.652 DEBUG:teuthology.orchestra.run.smithi005:> ') 2023-07-21T19:07:14.828 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {'mntopts': ['context=system_u:object_r:postgresql_db_t:s0'], 'mount_subvol_num': 0, 'mountpoint': '/tmp/cephfs'} 2023-07-21T19:07:14.828 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T19:07:14.829 DEBUG:teuthology.orchestra.run.smithi005:> mkdir -p -v /tmp/cephfs 2023-07-21T19:07:14.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:14 smithi101 ceph-mon[128136]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-21T19:07:14.872 INFO:teuthology.orchestra.run.smithi005.stdout:mkdir: created directory '/tmp/cephfs' 2023-07-21T19:07:14.873 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-07-21T19:07:14.873 DEBUG:teuthology.orchestra.run.smithi005:> chmod 0000 /tmp/cephfs 2023-07-21T19:07:14.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:14 smithi156 ceph-mon[128316]: pgmap v350: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 426 B/s wr, 6 op/s 2023-07-21T19:07:14.956 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-07-21T19:07:15.067 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T19:07:15.067 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T19:07:15.137 INFO:teuthology.orchestra.run.smithi005.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T19:07:15.139 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T19:07:15.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:15.140 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-07-21T19:07:15.183 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-07-21T19:07:15.184 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--tmp-cephfs sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /tmp/cephfs --id 0 --client_mountpoint=/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 --client_fs=cephfs -o context=system_u:object_r:postgresql_db_t:s0) 2023-07-21T19:07:15.227 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-07-21T19:07:15.299 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T19:07:15.300 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T19:07:15.372 INFO:teuthology.orchestra.run.smithi005.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T19:07:15.375 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T19:07:15.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:15.376 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-07-21T19:07:15.456 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:2023-07-21T19:07:15.446+0000 7f5f80b5c5c0 -1 init, newargv = 0x55ea33485320 newargc=17 2023-07-21T19:07:15.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[174015]: starting ceph client 2023-07-21T19:07:15.504 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[174015]: starting fuse 2023-07-21T19:07:16.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo modprobe fuse 2023-07-21T19:07:16.591 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:16 smithi005 ceph-mon[126460]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:16.602 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-07-21T19:07:16.603 DEBUG:teuthology.orchestra.run.smithi005:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-07-21T19:07:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-07-21T19:07:16.677 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-07-21T19:07:16.678 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:16.678 DEBUG:teuthology.orchestra.run.smithi005:> ls /sys/fs/fuse/connections 2023-07-21T19:07:16.723 INFO:teuthology.orchestra.run.smithi005.stdout:121 2023-07-21T19:07:16.724 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [121] 2023-07-21T19:07:16.724 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-07-21T19:07:16.725 DEBUG:teuthology.orchestra.run.smithi005:> import glob 2023-07-21T19:07:16.725 DEBUG:teuthology.orchestra.run.smithi005:> import re 2023-07-21T19:07:16.725 DEBUG:teuthology.orchestra.run.smithi005:> import os 2023-07-21T19:07:16.725 DEBUG:teuthology.orchestra.run.smithi005:> import subprocess 2023-07-21T19:07:16.727 DEBUG:teuthology.orchestra.run.smithi005:> 2023-07-21T19:07:16.727 DEBUG:teuthology.orchestra.run.smithi005:> def _find_admin_socket(client_name): 2023-07-21T19:07:16.727 DEBUG:teuthology.orchestra.run.smithi005:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-07-21T19:07:16.728 DEBUG:teuthology.orchestra.run.smithi005:> files = glob.glob(asok_path) 2023-07-21T19:07:16.728 DEBUG:teuthology.orchestra.run.smithi005:> mountpoint = "/tmp/cephfs" 2023-07-21T19:07:16.728 DEBUG:teuthology.orchestra.run.smithi005:> 2023-07-21T19:07:16.728 DEBUG:teuthology.orchestra.run.smithi005:> # Given a non-glob path, it better be there 2023-07-21T19:07:16.729 DEBUG:teuthology.orchestra.run.smithi005:> if "*" not in asok_path: 2023-07-21T19:07:16.729 DEBUG:teuthology.orchestra.run.smithi005:> assert(len(files) == 1) 2023-07-21T19:07:16.729 DEBUG:teuthology.orchestra.run.smithi005:> return files[0] 2023-07-21T19:07:16.729 DEBUG:teuthology.orchestra.run.smithi005:> 2023-07-21T19:07:16.729 DEBUG:teuthology.orchestra.run.smithi005:> for f in files: 2023-07-21T19:07:16.730 DEBUG:teuthology.orchestra.run.smithi005:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-07-21T19:07:16.730 DEBUG:teuthology.orchestra.run.smithi005:> if os.path.exists("/proc/{0}".format(pid)): 2023-07-21T19:07:16.730 DEBUG:teuthology.orchestra.run.smithi005:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-07-21T19:07:16.730 DEBUG:teuthology.orchestra.run.smithi005:> contents = proc_f.read() 2023-07-21T19:07:16.731 DEBUG:teuthology.orchestra.run.smithi005:> if mountpoint in contents: 2023-07-21T19:07:16.731 DEBUG:teuthology.orchestra.run.smithi005:> return f 2023-07-21T19:07:16.731 DEBUG:teuthology.orchestra.run.smithi005:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-07-21T19:07:16.732 DEBUG:teuthology.orchestra.run.smithi005:> 2023-07-21T19:07:16.732 DEBUG:teuthology.orchestra.run.smithi005:> print(_find_admin_socket("client.0")) 2023-07-21T19:07:16.732 DEBUG:teuthology.orchestra.run.smithi005:> ' 2023-07-21T19:07:16.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:16 smithi101 ceph-mon[128136]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:16.911 INFO:teuthology.orchestra.run.smithi005.stdout:/var/run/ceph/ceph-client.0.174015.asok 2023-07-21T19:07:16.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:16 smithi156 ceph-mon[128316]: pgmap v351: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 426 B/s wr, 7 op/s 2023-07-21T19:07:17.089 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.174015.asok 2023-07-21T19:07:17.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:17.090 DEBUG:teuthology.orchestra.run.smithi005:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.174015.asok status 2023-07-21T19:07:17.323 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "metadata": { 2023-07-21T19:07:17.323 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_sha1": "81cf785aa7781c4e08381c540f890cb9bc987ef9", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_version": "ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev)", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "entity_id": "0", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": "smithi005", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "mount_point": "/tmp/cephfs", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "pid": "174015", 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: "root": "/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1" 2023-07-21T19:07:17.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_count": 0, 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "dentry_pinned_count": 0, 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "id": 15339, 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "inst": { 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": { 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "client", 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: "num": 15339 2023-07-21T19:07:17.325 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2130170411 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:07:17.326 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": { 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "192.168.0.1:0", 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2130170411 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "inst_str": "client.15339 192.168.0.1:0/2130170411", 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "addr_str": "192.168.0.1:0/2130170411", 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "inode_count": 5, 2023-07-21T19:07:17.327 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_epoch": 23, 2023-07-21T19:07:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch": 87, 2023-07-21T19:07:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_epoch_barrier": 0, 2023-07-21T19:07:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "blocklisted": false, 2023-07-21T19:07:17.328 INFO:teuthology.orchestra.run.smithi005.stdout: "fs_name": "cephfs" 2023-07-21T19:07:17.328 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:17.338 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:17.339 DEBUG:teuthology.orchestra.run.smithi005:> stat --file-system '--printf=%T 2023-07-21T19:07:17.339 DEBUG:teuthology.orchestra.run.smithi005:> ' -- /tmp/cephfs 2023-07-21T19:07:17.384 INFO:teuthology.orchestra.run.smithi005.stdout:fuseblk 2023-07-21T19:07:17.384 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /tmp/cephfs 2023-07-21T19:07:17.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:07:17.385 DEBUG:teuthology.orchestra.run.smithi005:> sudo chmod 1777 /tmp/cephfs 2023-07-21T19:07:17.497 INFO:teuthology.run_tasks:Running task check-counter... 2023-07-21T19:07:17.521 INFO:teuthology.run_tasks:Running task fwd_scrub... 2023-07-21T19:07:17.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 -- 172.21.15.5:0/4213044684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604101680 msgr2=0x7fb604107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 --2- 172.21.15.5:0/4213044684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604101680 0x7fb604107f40 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb5fc00b0a0 tx=0x7fb5fc00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 -- 172.21.15.5:0/4213044684 shutdown_connections 2023-07-21T19:07:17.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 --2- 172.21.15.5:0/4213044684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604101680 0x7fb604107f40 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 --2- 172.21.15.5:0/4213044684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb6040ffb00 0x7fb6040fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 --2- 172.21.15.5:0/4213044684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb604100cf0 0x7fb604101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 -- 172.21.15.5:0/4213044684 >> 172.21.15.5:0/4213044684 conn(0x7fb6040fb0e0 msgr2=0x7fb6040fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 -- 172.21.15.5:0/4213044684 shutdown_connections 2023-07-21T19:07:17.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.730+0000 7fb6089ff700 1 -- 172.21.15.5:0/4213044684 wait complete. 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.731+0000 7fb6089ff700 1 Processor -- start 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 -- start start 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6040ffb00 0x7fb6041979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 0x7fb60419ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:17.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419f560 con 0x7fb604100cf0 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419f6a0 con 0x7fb6040ffb00 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb6089ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60419f830 con 0x7fb604101680 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb5fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47750/0 (socket says 172.21.15.5:47750) 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.732+0000 7fb5fbfff700 1 -- 172.21.15.5:0/1784664941 learned_addr learned my addr 172.21.15.5:0/1784664941 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb60259c700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6040ffb00 0x7fb6041979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:17.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb602d9d700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 0x7fb60419ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 -- 172.21.15.5:0/1784664941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 msgr2=0x7fb60419ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 0x7fb60419ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 -- 172.21.15.5:0/1784664941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6040ffb00 msgr2=0x7fb6041979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6040ffb00 0x7fb6041979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 -- 172.21.15.5:0/1784664941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5fc009d00 con 0x7fb604100cf0 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb602d9d700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 0x7fb60419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fbfff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9a0 tx=0x7fb5ec00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:17.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec009790 con 0x7fb604100cf0 2023-07-21T19:07:17.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5ec00cbd0 con 0x7fb604100cf0 2023-07-21T19:07:17.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60419fb10 con 0x7fb604100cf0 2023-07-21T19:07:17.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec018680 con 0x7fb604100cf0 2023-07-21T19:07:17.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.733+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6041a0020 con 0x7fb604100cf0 2023-07-21T19:07:17.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.734+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60410b190 con 0x7fb604100cf0 2023-07-21T19:07:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.736+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb5ec00cd40 con 0x7fb604100cf0 2023-07-21T19:07:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.738+0000 7fb5fb7fe700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 0x7fb5f00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:17.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.738+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb5ec014070 con 0x7fb604100cf0 2023-07-21T19:07:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.738+0000 7fb60259c700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 0x7fb5f00a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:17.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.739+0000 7fb60259c700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 0x7fb5f00a3650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb604100b50 tx=0x7fb5f400b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:17.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.742+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb5ec051790 con 0x7fb604100cf0 2023-07-21T19:07:17.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.957+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6041a05d0 con 0x7fb604100cf0 2023-07-21T19:07:17.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.959+0000 7fb5fb7fe700 1 -- 172.21.15.5:0/1784664941 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7fb5ec099d90 con 0x7fb604100cf0 2023-07-21T19:07:17.968 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:17.969 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:17.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 msgr2=0x7fb5f00a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:17.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 0x7fb5f00a3650 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fb604100b50 tx=0x7fb5f400b5b0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 msgr2=0x7fb604197f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:17.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9a0 tx=0x7fb5ec00ed60 comp rx=0 tx=0).stop 2023-07-21T19:07:17.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 shutdown_connections 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb5f00a11a0 0x7fb5f00a3650 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb604101680 0x7fb60419ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb6040ffb00 0x7fb6041979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 --2- 172.21.15.5:0/1784664941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb604100cf0 0x7fb604197f20 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 >> 172.21.15.5:0/1784664941 conn(0x7fb6040fb0e0 msgr2=0x7fb604106520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 shutdown_connections 2023-07-21T19:07:17.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:17.961+0000 7fb6089ff700 1 -- 172.21.15.5:0/1784664941 wait complete. 2023-07-21T19:07:17.989 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:17.990 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 -- 172.21.15.5:0/1984081367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 msgr2=0x7fdd1c107f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 --2- 172.21.15.5:0/1984081367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c107f20 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fdd1400b0a0 tx=0x7fdd1400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 -- 172.21.15.5:0/1984081367 shutdown_connections 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 --2- 172.21.15.5:0/1984081367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c107f20 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 --2- 172.21.15.5:0/1984081367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c0ffae0 0x7fdd1c0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 --2- 172.21.15.5:0/1984081367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c100cd0 0x7fdd1c101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 -- 172.21.15.5:0/1984081367 >> 172.21.15.5:0/1984081367 conn(0x7fdd1c0fb0e0 msgr2=0x7fdd1c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 -- 172.21.15.5:0/1984081367 shutdown_connections 2023-07-21T19:07:18.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.186+0000 7fdd229ff700 1 -- 172.21.15.5:0/1984081367 wait complete. 2023-07-21T19:07:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.187+0000 7fdd229ff700 1 Processor -- start 2023-07-21T19:07:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 -- start start 2023-07-21T19:07:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c0ffae0 0x7fdd1c197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c100cd0 0x7fdd1c198080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19f6c0 con 0x7fdd1c101660 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19f800 con 0x7fdd1c100cd0 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd229ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1c19f990 con 0x7fdd1c0ffae0 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd20f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd20f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47760/0 (socket says 172.21.15.5:47760) 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd20f9c700 1 -- 172.21.15.5:0/3504727190 learned_addr learned my addr 172.21.15.5:0/3504727190 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:18.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd1bfff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c0ffae0 0x7fdd1c197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.188+0000 7fdd1b7fe700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c100cd0 0x7fdd1c198080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 -- 172.21.15.5:0/3504727190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c0ffae0 msgr2=0x7fdd1c197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c0ffae0 0x7fdd1c197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 -- 172.21.15.5:0/3504727190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c100cd0 msgr2=0x7fdd1c198080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c100cd0 0x7fdd1c198080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 -- 172.21.15.5:0/3504727190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd14009d00 con 0x7fdd1c101660 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd20f9c700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009fd0 tx=0x7fdd1400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:18.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1400e040 con 0x7fdd1c101660 2023-07-21T19:07:18.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd14003c00 con 0x7fdd1c101660 2023-07-21T19:07:18.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1c19fc10 con 0x7fdd1c101660 2023-07-21T19:07:18.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd1401b6c0 con 0x7fdd1c101660 2023-07-21T19:07:18.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.189+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1c19ffe0 con 0x7fdd1c101660 2023-07-21T19:07:18.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.190+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd1c192080 con 0x7fdd1c101660 2023-07-21T19:07:18.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.192+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fdd1401d470 con 0x7fdd1c101660 2023-07-21T19:07:18.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.194+0000 7fdd197fa700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.194+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fdd140cdbe0 con 0x7fdd1c101660 2023-07-21T19:07:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.194+0000 7fdd1bfff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.194+0000 7fdd1bfff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 0x7fdd040a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c100b30 tx=0x7fdd0c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:18.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.197+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fdd14097370 con 0x7fdd1c101660 2023-07-21T19:07:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.406+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdd1c0611c0 con 0x7fdd1c101660 2023-07-21T19:07:18.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.407+0000 7fdd197fa700 1 -- 172.21.15.5:0/3504727190 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdd1409a990 con 0x7fdd1c101660 2023-07-21T19:07:18.417 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:07:18.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 msgr2=0x7fdd040a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 0x7fdd040a3600 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fdd1c100b30 tx=0x7fdd0c009260 comp rx=0 tx=0).stop 2023-07-21T19:07:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 msgr2=0x7fdd1c19f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fdd14009fd0 tx=0x7fdd1400ba50 comp rx=0 tx=0).stop 2023-07-21T19:07:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 shutdown_connections 2023-07-21T19:07:18.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fdd040a1150 0x7fdd040a3600 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd1c101660 0x7fdd1c19f0f0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd1c0ffae0 0x7fdd1c197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 --2- 172.21.15.5:0/3504727190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd1c100cd0 0x7fdd1c198080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.409+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 >> 172.21.15.5:0/3504727190 conn(0x7fdd1c0fb0e0 msgr2=0x7fdd1c106610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.410+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 shutdown_connections 2023-07-21T19:07:18.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.410+0000 7fdd229ff700 1 -- 172.21.15.5:0/3504727190 wait complete. 2023-07-21T19:07:18.437 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:18 smithi101 ceph-mon[128136]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T19:07:18.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:18 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1784664941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.635+0000 7f042dfff700 1 -- 172.21.15.5:0/603923329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428072f70 msgr2=0x7f04280733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.635+0000 7f042dfff700 1 --2- 172.21.15.5:0/603923329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428072f70 0x7f04280733e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f041c009990 tx=0x7f041c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 -- 172.21.15.5:0/603923329 shutdown_connections 2023-07-21T19:07:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 --2- 172.21.15.5:0/603923329 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f042810c0b0 0x7f042810e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 --2- 172.21.15.5:0/603923329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0428074550 0x7f04280729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 --2- 172.21.15.5:0/603923329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428072f70 0x7f04280733e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 -- 172.21.15.5:0/603923329 >> 172.21.15.5:0/603923329 conn(0x7f04280fb0e0 msgr2=0x7f04280fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:18.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 -- 172.21.15.5:0/603923329 shutdown_connections 2023-07-21T19:07:18.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.636+0000 7f042dfff700 1 -- 172.21.15.5:0/603923329 wait complete. 2023-07-21T19:07:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 Processor -- start 2023-07-21T19:07:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 -- start start 2023-07-21T19:07:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0428072f70 0x7f042819be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 0x7f04281a33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04281a3990 con 0x7f0428074550 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04281a3ad0 con 0x7f0428072f70 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.637+0000 7f042dfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04281a3c60 con 0x7f042810c0b0 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47778/0 (socket says 172.21.15.5:47778) 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 -- 172.21.15.5:0/3939637638 learned_addr learned my addr 172.21.15.5:0/3939637638 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:18.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f04277fe700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0428072f70 0x7f042819be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0427fff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 0x7f04281a33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 -- 172.21.15.5:0/3939637638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 msgr2=0x7f04281a33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 0x7f04281a33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 -- 172.21.15.5:0/3939637638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0428072f70 msgr2=0x7f042819be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0428072f70 0x7f042819be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 -- 172.21.15.5:0/3939637638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f041c0095f0 con 0x7f0428074550 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0427fff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 0x7f04281a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0426ffd700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f041c0098e0 tx=0x7f041c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.638+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c01d070 con 0x7f0428074550 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.639+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f041c00be30 con 0x7f0428074550 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.639+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04281a3ee0 con 0x7f0428074550 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.639+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f041c022dd0 con 0x7f0428074550 2023-07-21T19:07:18.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.639+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04281a42a0 con 0x7f0428074550 2023-07-21T19:07:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.640+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0428101c00 con 0x7f0428074550 2023-07-21T19:07:18.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.642+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f041c022570 con 0x7f0428074550 2023-07-21T19:07:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.643+0000 7f0424ff9700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 0x7f04100a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.643+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f041c0cde60 con 0x7f0428074550 2023-07-21T19:07:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.643+0000 7f04277fe700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 0x7f04100a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.644+0000 7f04277fe700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 0x7f04100a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0428074920 tx=0x7f0418009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:18.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.647+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f041c0975f0 con 0x7f0428074550 2023-07-21T19:07:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:18 smithi005 ceph-mon[126460]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T19:07:18.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:18 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1784664941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:18 smithi156 ceph-mon[128316]: pgmap v352: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 597 B/s wr, 6 op/s 2023-07-21T19:07:18.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:18 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1784664941' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.857+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f04280611c0 con 0x7f0428074550 2023-07-21T19:07:18.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.858+0000 7f0424ff9700 1 -- 172.21.15.5:0/3939637638 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f041c09ac10 con 0x7f0428074550 2023-07-21T19:07:18.867 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:07:18.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 msgr2=0x7f04100a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 0x7f04100a3680 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0428074920 tx=0x7f0418009260 comp rx=0 tx=0).stop 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 msgr2=0x7f042819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f041c0098e0 tx=0x7f041c004730 comp rx=0 tx=0).stop 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 shutdown_connections 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f04100a11d0 0x7f04100a3680 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f042810c0b0 0x7f04281a33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0428072f70 0x7f042819be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 --2- 172.21.15.5:0/3939637638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0428074550 0x7f042819c350 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:18.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 >> 172.21.15.5:0/3939637638 conn(0x7f04280fb0e0 msgr2=0x7f04280fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:18.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 shutdown_connections 2023-07-21T19:07:18.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:18.860+0000 7f042dfff700 1 -- 172.21.15.5:0/3939637638 wait complete. 2023-07-21T19:07:18.888 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:19.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 -- 172.21.15.5:0/1908848243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334101670 msgr2=0x7f4334107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 --2- 172.21.15.5:0/1908848243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334101670 0x7f4334107f30 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f432c00b0a0 tx=0x7f432c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 -- 172.21.15.5:0/1908848243 shutdown_connections 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 --2- 172.21.15.5:0/1908848243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334101670 0x7f4334107f30 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 --2- 172.21.15.5:0/1908848243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43340ffaf0 0x7f43340fff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 --2- 172.21.15.5:0/1908848243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4334100ce0 0x7f4334101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 -- 172.21.15.5:0/1908848243 >> 172.21.15.5:0/1908848243 conn(0x7f43340fb130 msgr2=0x7f43340fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 -- 172.21.15.5:0/1908848243 shutdown_connections 2023-07-21T19:07:19.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.086+0000 7f433a5ff700 1 -- 172.21.15.5:0/1908848243 wait complete. 2023-07-21T19:07:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.087+0000 7f433a5ff700 1 Processor -- start 2023-07-21T19:07:19.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.087+0000 7f433a5ff700 1 -- start start 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.087+0000 7f433a5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340ffaf0 0x7f4334195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.087+0000 7f433a5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.087+0000 7f433a5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 0x7f433419ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f433a5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433419d400 con 0x7f4334100ce0 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f433a5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433419d540 con 0x7f4334101670 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f433a5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433419d6d0 con 0x7f43340ffaf0 2023-07-21T19:07:19.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47802/0 (socket says 172.21.15.5:47802) 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 -- 172.21.15.5:0/2447715074 learned_addr learned my addr 172.21.15.5:0/2447715074 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f4333fff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340ffaf0 0x7f4334195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f4338b9c700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 0x7f433419ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 -- 172.21.15.5:0/2447715074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340ffaf0 msgr2=0x7f4334195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340ffaf0 0x7f4334195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 -- 172.21.15.5:0/2447715074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 msgr2=0x7f433419ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 0x7f433419ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 -- 172.21.15.5:0/2447715074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f432c009d00 con 0x7f4334100ce0 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f4338b9c700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 0x7f433419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.088+0000 7f43337fe700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f432800d950 tx=0x7f432800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.089+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4328009940 con 0x7f4334100ce0 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.089+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4328010460 con 0x7f4334100ce0 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.089+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f433419d9b0 con 0x7f4334100ce0 2023-07-21T19:07:19.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.089+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800f620 con 0x7f4334100ce0 2023-07-21T19:07:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.089+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f433419df10 con 0x7f4334100ce0 2023-07-21T19:07:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.090+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4334107020 con 0x7f4334100ce0 2023-07-21T19:07:19.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.092+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f43280105d0 con 0x7f4334100ce0 2023-07-21T19:07:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.093+0000 7f43317fa700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 0x7f431c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.093+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43280cc730 con 0x7f4334100ce0 2023-07-21T19:07:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.093+0000 7f4333fff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 0x7f431c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.094+0000 7f4333fff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 0x7f431c0a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4334100b40 tx=0x7f432400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:19.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.097+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4328095e30 con 0x7f4334100ce0 2023-07-21T19:07:19.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.308+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f4334061120 con 0x7f4334100ce0 2023-07-21T19:07:19.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.309+0000 7f43317fa700 1 -- 172.21.15.5:0/2447715074 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f4328099450 con 0x7f4334100ce0 2023-07-21T19:07:19.319 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:07:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 msgr2=0x7f431c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 0x7f431c0a3680 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4334100b40 tx=0x7f432400b410 comp rx=0 tx=0).stop 2023-07-21T19:07:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 msgr2=0x7f4334195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f432800d950 tx=0x7f432800dc60 comp rx=0 tx=0).stop 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 shutdown_connections 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f431c0a11d0 0x7f431c0a3680 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4334101670 0x7f433419ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43340ffaf0 0x7f4334195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 --2- 172.21.15.5:0/2447715074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4334100ce0 0x7f4334195dc0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 >> 172.21.15.5:0/2447715074 conn(0x7f43340fb130 msgr2=0x7f4334104160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:19.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 shutdown_connections 2023-07-21T19:07:19.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.311+0000 7f433a5ff700 1 -- 172.21.15.5:0/2447715074 wait complete. 2023-07-21T19:07:19.339 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:19.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 -- 172.21.15.5:0/746886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c101680 msgr2=0x7fcb2c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 --2- 172.21.15.5:0/746886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c101680 0x7fcb2c107f40 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fcb2400b0a0 tx=0x7fcb2400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 -- 172.21.15.5:0/746886961 shutdown_connections 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 --2- 172.21.15.5:0/746886961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c101680 0x7fcb2c107f40 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 --2- 172.21.15.5:0/746886961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c0ffb00 0x7fcb2c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 --2- 172.21.15.5:0/746886961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 -- 172.21.15.5:0/746886961 >> 172.21.15.5:0/746886961 conn(0x7fcb2c0fb0e0 msgr2=0x7fcb2c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 -- 172.21.15.5:0/746886961 shutdown_connections 2023-07-21T19:07:19.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.535+0000 7fcb327ff700 1 -- 172.21.15.5:0/746886961 wait complete. 2023-07-21T19:07:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 Processor -- start 2023-07-21T19:07:19.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 -- start start 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c0ffb00 0x7fcb2c0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 0x7fcb2c072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c0735e0 con 0x7fcb2c100cf0 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c073750 con 0x7fcb2c101680 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.536+0000 7fcb327ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb2c19f8b0 con 0x7fcb2c0ffb00 2023-07-21T19:07:19.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47816/0 (socket says 172.21.15.5:47816) 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 -- 172.21.15.5:0/4266631310 learned_addr learned my addr 172.21.15.5:0/4266631310 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb2bfff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c0ffb00 0x7fcb2c0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb30d9c700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 0x7fcb2c072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 -- 172.21.15.5:0/4266631310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c0ffb00 msgr2=0x7fcb2c0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c0ffb00 0x7fcb2c0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 -- 172.21.15.5:0/4266631310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 msgr2=0x7fcb2c072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 0x7fcb2c072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 -- 172.21.15.5:0/4266631310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009d00 con 0x7fcb2c100cf0 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb30d9c700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 0x7fcb2c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb23fff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400d950 tx=0x7fcb1400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.537+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb14009940 con 0x7fcb2c100cf0 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.538+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb2c19fb90 con 0x7fcb2c100cf0 2023-07-21T19:07:19.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.538+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcb14010460 con 0x7fcb2c100cf0 2023-07-21T19:07:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.538+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1400f620 con 0x7fcb2c100cf0 2023-07-21T19:07:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.538+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb2c1a00a0 con 0x7fcb2c100cf0 2023-07-21T19:07:19.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.539+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb2c107050 con 0x7fcb2c100cf0 2023-07-21T19:07:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.541+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fcb140105d0 con 0x7fcb2c100cf0 2023-07-21T19:07:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.542+0000 7fcb29ffb700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 0x7fcb180a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.543+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fcb140cc730 con 0x7fcb2c100cf0 2023-07-21T19:07:19.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.543+0000 7fcb2bfff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 0x7fcb180a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.543+0000 7fcb2bfff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 0x7fcb180a3650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c100b50 tx=0x7fcb1c007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:19.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.546+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcb14095e30 con 0x7fcb2c100cf0 2023-07-21T19:07:19.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:19 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3504727190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:19 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3939637638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:19 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2447715074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:19 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3504727190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:19 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3939637638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:19 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2447715074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:19 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3504727190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:19 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3939637638' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:19 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2447715074' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.755+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcb2c02c850 con 0x7fcb2c100cf0 2023-07-21T19:07:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.757+0000 7fcb29ffb700 1 -- 172.21.15.5:0/4266631310 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7fcb14099450 con 0x7fcb2c100cf0 2023-07-21T19:07:19.767 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:19.767 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.759+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 msgr2=0x7fcb180a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.759+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 0x7fcb180a3650 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcb2c100b50 tx=0x7fcb1c007400 comp rx=0 tx=0).stop 2023-07-21T19:07:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.759+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 msgr2=0x7fcb2c072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.759+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fcb1400d950 tx=0x7fcb1400dc60 comp rx=0 tx=0).stop 2023-07-21T19:07:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 shutdown_connections 2023-07-21T19:07:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fcb180a11a0 0x7fcb180a3650 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcb2c101680 0x7fcb2c072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcb2c0ffb00 0x7fcb2c0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 --2- 172.21.15.5:0/4266631310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcb2c100cf0 0x7fcb2c072900 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 >> 172.21.15.5:0/4266631310 conn(0x7fcb2c0fb0e0 msgr2=0x7fcb2c104190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 shutdown_connections 2023-07-21T19:07:19.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.760+0000 7fcb327ff700 1 -- 172.21.15.5:0/4266631310 wait complete. 2023-07-21T19:07:19.787 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:19.788 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 -- 172.21.15.5:0/2916145099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a681079a0 msgr2=0x7f6a68109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/2916145099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a681079a0 0x7f6a68109e20 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000b0a0 tx=0x7f6a6000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 -- 172.21.15.5:0/2916145099 shutdown_connections 2023-07-21T19:07:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/2916145099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a681079a0 0x7f6a68109e20 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/2916145099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a68074550 0x7f6a680729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/2916145099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 0x7f6a680733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 -- 172.21.15.5:0/2916145099 >> 172.21.15.5:0/2916145099 conn(0x7f6a680fb0e0 msgr2=0x7f6a680fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.985+0000 7f6a6ffff700 1 -- 172.21.15.5:0/2916145099 shutdown_connections 2023-07-21T19:07:19.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.986+0000 7f6a6ffff700 1 -- 172.21.15.5:0/2916145099 wait complete. 2023-07-21T19:07:19.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.986+0000 7f6a6ffff700 1 Processor -- start 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 -- start start 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 0x7f6a68197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 0x7f6a6819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819f5f0 con 0x7f6a68074550 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819f730 con 0x7f6a68072f70 2023-07-21T19:07:19.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6819f8c0 con 0x7f6a681079a0 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6d59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6d59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47836/0 (socket says 172.21.15.5:47836) 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6d59a700 1 -- 172.21.15.5:0/3309935553 learned_addr learned my addr 172.21.15.5:0/3309935553 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.987+0000 7f6a6dd9b700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 0x7f6a68197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6e59c700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 0x7f6a6819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 -- 172.21.15.5:0/3309935553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 msgr2=0x7f6a6819f020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:19.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 0x7f6a6819f020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 -- 172.21.15.5:0/3309935553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 msgr2=0x7f6a68197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 0x7f6a68197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 -- 172.21.15.5:0/3309935553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a60009d00 con 0x7f6a68074550 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6e59c700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 0x7f6a6819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6d59a700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800d980 tx=0x7f6a5800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800b950 con 0x7f6a68074550 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a58007620 con 0x7f6a68074550 2023-07-21T19:07:20.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5800f3c0 con 0x7f6a68074550 2023-07-21T19:07:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6819fba0 con 0x7f6a68074550 2023-07-21T19:07:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.988+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a681a0060 con 0x7f6a68074550 2023-07-21T19:07:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.989+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a68106050 con 0x7f6a68074550 2023-07-21T19:07:20.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.991+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f6a5800bab0 con 0x7f6a68074550 2023-07-21T19:07:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.994+0000 7f6a5effd700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 0x7f6a540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.994+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6a580ccb40 con 0x7f6a68074550 2023-07-21T19:07:20.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.994+0000 7f6a6dd9b700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 0x7f6a540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.995+0000 7f6a6dd9b700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 0x7f6a540a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a68074920 tx=0x7f6a64009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:19.998+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f6a58096240 con 0x7f6a68074550 2023-07-21T19:07:20.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.206+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6a68061260 con 0x7f6a68074550 2023-07-21T19:07:20.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.206+0000 7f6a5effd700 1 -- 172.21.15.5:0/3309935553 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6a580160a0 con 0x7f6a68074550 2023-07-21T19:07:20.216 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:07:20.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 msgr2=0x7f6a540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 0x7f6a540a3600 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f6a68074920 tx=0x7f6a64009260 comp rx=0 tx=0).stop 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 msgr2=0x7f6a68197fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800d980 tx=0x7f6a5800dc90 comp rx=0 tx=0).stop 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 shutdown_connections 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f6a540a1150 0x7f6a540a3600 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a681079a0 0x7f6a6819f020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68072f70 0x7f6a68197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 --2- 172.21.15.5:0/3309935553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68074550 0x7f6a68197fb0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 >> 172.21.15.5:0/3309935553 conn(0x7f6a680fb0e0 msgr2=0x7f6a680fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 shutdown_connections 2023-07-21T19:07:20.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.209+0000 7f6a6ffff700 1 -- 172.21.15.5:0/3309935553 wait complete. 2023-07-21T19:07:20.237 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 -- 172.21.15.5:0/2075105543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec0ffae0 msgr2=0x7fb8ec0ffef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/2075105543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0ffef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc009990 tx=0x7fb8dc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 -- 172.21.15.5:0/2075105543 shutdown_connections 2023-07-21T19:07:20.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/2075105543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb8ec101660 0x7fb8ec107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/2075105543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0ffef0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/2075105543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec101120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 -- 172.21.15.5:0/2075105543 >> 172.21.15.5:0/2075105543 conn(0x7fb8ec0fb0e0 msgr2=0x7fb8ec0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 -- 172.21.15.5:0/2075105543 shutdown_connections 2023-07-21T19:07:20.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.435+0000 7fb8f09ff700 1 -- 172.21.15.5:0/2075105543 wait complete. 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.436+0000 7fb8f09ff700 1 Processor -- start 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.436+0000 7fb8f09ff700 1 -- start start 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb8ec101660 0x7fb8ec072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec1996c0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec073380 con 0x7fb8ec101660 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8f09ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec073540 con 0x7fb8ec0ffae0 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47852/0 (socket says 172.21.15.5:47852) 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 -- 172.21.15.5:0/4071637022 learned_addr learned my addr 172.21.15.5:0/4071637022 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8ea59c700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 -- 172.21.15.5:0/4071637022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 msgr2=0x7fb8ec0742c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0742c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 -- 172.21.15.5:0/4071637022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb8ec101660 msgr2=0x7fb8ec072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb8ec101660 0x7fb8ec072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 -- 172.21.15.5:0/4071637022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0095f0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8ea59c700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.437+0000 7fb8e3fff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e9a0 tx=0x7fb8d400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.438+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4009790 con 0x7fb8ec100cd0 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.438+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec1a4070 con 0x7fb8ec100cd0 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.438+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb8d400cbd0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.438+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8d4018680 con 0x7fb8ec100cd0 2023-07-21T19:07:20.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.438+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec1a44f0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.439+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec10b1b0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.441+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb8d400cd40 con 0x7fb8ec100cd0 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.442+0000 7fb8e37fe700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.442+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb8d4014070 con 0x7fb8ec100cd0 2023-07-21T19:07:20.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.443+0000 7fb8ea59c700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.443+0000 7fb8ea59c700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 0x7fb8d80a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005910 tx=0x7fb8dc00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.446+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb8d4051790 con 0x7fb8ec100cd0 2023-07-21T19:07:20.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:20 smithi101 ceph-mon[128136]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-21T19:07:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:20 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4266631310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:20.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:20 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3309935553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.652+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8ec106ac0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.653+0000 7fb8e37fe700 1 -- 172.21.15.5:0/4071637022 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb8d4099db0 con 0x7fb8ec100cd0 2023-07-21T19:07:20.663 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 msgr2=0x7fb8d80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 0x7fb8d80a3680 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc005910 tx=0x7fb8dc00b260 comp rx=0 tx=0).stop 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 msgr2=0x7fb8ec072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb8d400e9a0 tx=0x7fb8d400ed60 comp rx=0 tx=0).stop 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 shutdown_connections 2023-07-21T19:07:20.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8d80a11d0 0x7fb8d80a3680 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb8ec101660 0x7fb8ec072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8ec0ffae0 0x7fb8ec0742c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 --2- 172.21.15.5:0/4071637022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb8ec100cd0 0x7fb8ec072900 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 >> 172.21.15.5:0/4071637022 conn(0x7fb8ec0fb0e0 msgr2=0x7fb8ec0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.655+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 shutdown_connections 2023-07-21T19:07:20.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.656+0000 7fb8f09ff700 1 -- 172.21.15.5:0/4071637022 wait complete. 2023-07-21T19:07:20.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:20 smithi005 ceph-mon[126460]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-21T19:07:20.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4266631310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:20.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:20 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3309935553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:20 smithi156 ceph-mon[128316]: pgmap v353: 97 pgs: 97 active+clean; 638 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 682 B/s wr, 6 op/s 2023-07-21T19:07:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:20 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4266631310' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:20.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:20 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3309935553' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:20.683 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:20.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 -- 172.21.15.5:0/2999903465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 msgr2=0x7fb89c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/2999903465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c0733e0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb890009990 tx=0x7fb890009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 -- 172.21.15.5:0/2999903465 shutdown_connections 2023-07-21T19:07:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/2999903465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c1079a0 0x7fb89c109e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/2999903465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c074550 0x7fb89c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/2999903465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c0733e0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 -- 172.21.15.5:0/2999903465 >> 172.21.15.5:0/2999903465 conn(0x7fb89c0fb0e0 msgr2=0x7fb89c0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 -- 172.21.15.5:0/2999903465 shutdown_connections 2023-07-21T19:07:20.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.881+0000 7fb8a1dff700 1 -- 172.21.15.5:0/2999903465 wait complete. 2023-07-21T19:07:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 Processor -- start 2023-07-21T19:07:20.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 -- start start 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c074550 0x7fb89c197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 0x7fb89c19eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19f4b0 con 0x7fb89c072f70 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19f5f0 con 0x7fb89c1079a0 2023-07-21T19:07:20.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb8a1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89c19f780 con 0x7fb89c074550 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.883+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47878/0 (socket says 172.21.15.5:47878) 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 -- 172.21.15.5:0/1913927097 learned_addr learned my addr 172.21.15.5:0/1913927097 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89affd700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c074550 0x7fb89c197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89bfff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 0x7fb89c19eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 -- 172.21.15.5:0/1913927097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c074550 msgr2=0x7fb89c197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c074550 0x7fb89c197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 -- 172.21.15.5:0/1913927097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 msgr2=0x7fb89c19eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 0x7fb89c19eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 -- 172.21.15.5:0/1913927097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8900095f0 con 0x7fb89c072f70 2023-07-21T19:07:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89bfff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 0x7fb89c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:20.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.884+0000 7fb89b7fe700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c9e0 tx=0x7fb88c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.885+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c007aa0 con 0x7fb89c072f70 2023-07-21T19:07:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.885+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb89c19fa60 con 0x7fb89c072f70 2023-07-21T19:07:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.885+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb88c007c00 con 0x7fb89c072f70 2023-07-21T19:07:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.885+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88c018660 con 0x7fb89c072f70 2023-07-21T19:07:20.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.885+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb89c19ff70 con 0x7fb89c072f70 2023-07-21T19:07:20.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.886+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb89c106050 con 0x7fb89c072f70 2023-07-21T19:07:20.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.888+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fb88c00f450 con 0x7fb89c072f70 2023-07-21T19:07:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.889+0000 7fb898ff9700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 0x7fb8840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.889+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fb88c0cc8e0 con 0x7fb89c072f70 2023-07-21T19:07:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.889+0000 7fb89affd700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 0x7fb8840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:20.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.890+0000 7fb89affd700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 0x7fb8840a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb89000b5c0 tx=0x7fb890005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:20.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:20.893+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb88c096040 con 0x7fb89c072f70 2023-07-21T19:07:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.099+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb89c0611c0 con 0x7fb89c072f70 2023-07-21T19:07:21.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.100+0000 7fb898ff9700 1 -- 172.21.15.5:0/1913927097 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb88c00fe60 con 0x7fb89c072f70 2023-07-21T19:07:21.110 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:07:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 msgr2=0x7fb8840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 0x7fb8840a3600 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb89000b5c0 tx=0x7fb890005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 msgr2=0x7fb89c197930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb88c00c9e0 tx=0x7fb88c00ccf0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 shutdown_connections 2023-07-21T19:07:21.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fb8840a1150 0x7fb8840a3600 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb89c1079a0 0x7fb89c19eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb89c072f70 0x7fb89c197930 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 --2- 172.21.15.5:0/1913927097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb89c074550 0x7fb89c197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 >> 172.21.15.5:0/1913927097 conn(0x7fb89c0fb0e0 msgr2=0x7fb89c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 shutdown_connections 2023-07-21T19:07:21.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.102+0000 7fb8a1dff700 1 -- 172.21.15.5:0/1913927097 wait complete. 2023-07-21T19:07:21.131 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2023-07-21T19:07:21.132 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 -- 172.21.15.5:0/3396330563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 msgr2=0x7f55f80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 --2- 172.21.15.5:0/3396330563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80fff10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f55ec009990 tx=0x7f55ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 -- 172.21.15.5:0/3396330563 shutdown_connections 2023-07-21T19:07:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 --2- 172.21.15.5:0/3396330563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 0x7f55f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 --2- 172.21.15.5:0/3396330563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80fff10 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 --2- 172.21.15.5:0/3396330563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 0x7f55f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 -- 172.21.15.5:0/3396330563 >> 172.21.15.5:0/3396330563 conn(0x7f55f80fb0e0 msgr2=0x7f55f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 -- 172.21.15.5:0/3396330563 shutdown_connections 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.330+0000 7f55febff700 1 -- 172.21.15.5:0/3396330563 wait complete. 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.331+0000 7f55febff700 1 Processor -- start 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.331+0000 7f55febff700 1 -- start start 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 0x7f55f8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 0x7f55f8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f80735e0 con 0x7f55f80ffb00 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f8073750 con 0x7f55f8101680 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55febff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55f819f790 con 0x7f55f8100cf0 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47910/0 (socket says 172.21.15.5:47910) 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 -- 172.21.15.5:0/1854892342 learned_addr learned my addr 172.21.15.5:0/1854892342 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fd19c700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 0x7f55f8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55f7fff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 0x7f55f8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 -- 172.21.15.5:0/1854892342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 msgr2=0x7f55f8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 0x7f55f8072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 -- 172.21.15.5:0/1854892342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 msgr2=0x7f55f8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 0x7f55f8072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.332+0000 7f55fc99b700 1 -- 172.21.15.5:0/1854892342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ec0095f0 con 0x7f55f80ffb00 2023-07-21T19:07:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55fc99b700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b5c0 tx=0x7f55ec0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec01d070 con 0x7f55f80ffb00 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55ec003f40 con 0x7f55f80ffb00 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55f819fa10 con 0x7f55f80ffb00 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ec022c50 con 0x7f55f80ffb00 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.333+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55f819fe30 con 0x7f55f80ffb00 2023-07-21T19:07:21.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.334+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55f81070a0 con 0x7f55f80ffb00 2023-07-21T19:07:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.336+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f55ec003ab0 con 0x7f55f80ffb00 2023-07-21T19:07:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.337+0000 7f55f5ffb700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 0x7f55e80a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.338+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f55ec0cdff0 con 0x7f55f80ffb00 2023-07-21T19:07:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.338+0000 7f55f7fff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 0x7f55e80a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.338+0000 7f55f7fff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 0x7f55e80a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55e4005ea0 tx=0x7f55e4005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.341+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f55ec097780 con 0x7f55f80ffb00 2023-07-21T19:07:21.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:21 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4071637022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:21 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1913927097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:21 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4071637022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:21 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1913927097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.600+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f55f802c850 con 0x7f55f80ffb00 2023-07-21T19:07:21.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.600+0000 7f55f5ffb700 1 -- 172.21.15.5:0/1854892342 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f55ec09ada0 con 0x7f55f80ffb00 2023-07-21T19:07:21.610 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:21.610 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:21.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 msgr2=0x7f55e80a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 0x7f55e80a3650 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f55e4005ea0 tx=0x7f55e4005e30 comp rx=0 tx=0).stop 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 msgr2=0x7f55f80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f55ec00b5c0 tx=0x7f55ec0057f0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 shutdown_connections 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f55e80a11a0 0x7f55e80a3650 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55f8101680 0x7f55f8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55f80ffb00 0x7f55f80742c0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 --2- 172.21.15.5:0/1854892342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55f8100cf0 0x7f55f8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 >> 172.21.15.5:0/1854892342 conn(0x7f55f80fb0e0 msgr2=0x7f55f81041e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 shutdown_connections 2023-07-21T19:07:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.603+0000 7f55febff700 1 -- 172.21.15.5:0/1854892342 wait complete. 2023-07-21T19:07:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:21.631 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2023-07-21T19:07:21.632 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2023-07-21T19:07:21.632 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T19:07:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:21 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4071637022' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:21 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1913927097' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 -- 172.21.15.5:0/2918025811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 msgr2=0x7f28f80fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 --2- 172.21.15.5:0/2918025811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80fff10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f28f4009990 tx=0x7f28f4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 -- 172.21.15.5:0/2918025811 shutdown_connections 2023-07-21T19:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 --2- 172.21.15.5:0/2918025811 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8101680 0x7f28f8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 --2- 172.21.15.5:0/2918025811 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80fff10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 --2- 172.21.15.5:0/2918025811 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8100cf0 0x7f28f8101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 -- 172.21.15.5:0/2918025811 >> 172.21.15.5:0/2918025811 conn(0x7f28f80fb0e0 msgr2=0x7f28f80fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 -- 172.21.15.5:0/2918025811 shutdown_connections 2023-07-21T19:07:21.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.826+0000 7f28fffff700 1 -- 172.21.15.5:0/2918025811 wait complete. 2023-07-21T19:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.828+0000 7f28fffff700 1 Processor -- start 2023-07-21T19:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.828+0000 7f28fffff700 1 -- start start 2023-07-21T19:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8100cf0 0x7f28f8072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8101680 0x7f28f8072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f80735e0 con 0x7f28f80ffb00 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f8073750 con 0x7f28f8100cf0 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28f819f830 con 0x7f28f8101680 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47940/0 (socket says 172.21.15.5:47940) 2023-07-21T19:07:21.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 -- 172.21.15.5:0/2205066861 learned_addr learned my addr 172.21.15.5:0/2205066861 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:21.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 -- 172.21.15.5:0/2205066861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8101680 msgr2=0x7f28f8072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8101680 0x7f28f8072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 -- 172.21.15.5:0/2205066861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8100cf0 msgr2=0x7f28f8072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fd59a700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8100cf0 0x7f28f8072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8100cf0 0x7f28f8072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.829+0000 7f28fdd9b700 1 -- 172.21.15.5:0/2205066861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28f40095f0 con 0x7f28f80ffb00 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28fdd9b700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:21.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f401d070 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28f400bdd0 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28f819fab0 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f400f460 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.830+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28f819fed0 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.831+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28f8107160 con 0x7f28f80ffb00 2023-07-21T19:07:21.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.833+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f28f4022650 con 0x7f28f80ffb00 2023-07-21T19:07:21.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.834+0000 7f28eeffd700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 0x7f28e40a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:21.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.834+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f28f40cdf50 con 0x7f28f80ffb00 2023-07-21T19:07:21.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.835+0000 7f28fd59a700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 0x7f28e40a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:21.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.835+0000 7f28fd59a700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 0x7f28e40a3650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009b30 tx=0x7f28e8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:21.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:21.838+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f28f40976e0 con 0x7f28f80ffb00 2023-07-21T19:07:22.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.041+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f28f8060950 con 0x7f28e40a11a0 2023-07-21T19:07:22.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.053+0000 7f28eeffd700 1 -- 172.21.15.5:0/2205066861 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343494 (secure 0 0 0) 0x7f28f8060950 con 0x7f28e40a11a0 2023-07-21T19:07:22.067 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:22.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 msgr2=0x7f28e40a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 0x7f28e40a3650 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f28e8009b30 tx=0x7f28e8009260 comp rx=0 tx=0).stop 2023-07-21T19:07:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 msgr2=0x7f28f80742c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f28f4000c00 tx=0x7f28f4005ef0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 shutdown_connections 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28e40a11a0 0x7f28e40a3650 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28f8101680 0x7f28f8072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28f80ffb00 0x7f28f80742c0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 --2- 172.21.15.5:0/2205066861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28f8100cf0 0x7f28f8072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 >> 172.21.15.5:0/2205066861 conn(0x7f28f80fb0e0 msgr2=0x7f28f81042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 shutdown_connections 2023-07-21T19:07:22.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.058+0000 7f28fffff700 1 -- 172.21.15.5:0/2205066861 wait complete. 2023-07-21T19:07:22.072 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:07:22.086 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-07-21T19:07:21.341914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654440,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":495,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182304,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334996,"kb_used_data":11572,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606828,"statfs":{"total":1151940427776,"available":1151597391872,"internally_reserved":0,"allocated":11849728,"data_stored":7435356,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002992"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360521+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:07:07.360521+0000","last_peered":"2023-07-21T19:07:07.360521+0000","last_clean":"2023-07-21T19:07:07.360521+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:07:07.360521+0000","last_undegraded":"2023-07-21T19:07:07.360521+0000","last_fullsized":"2023-07-21T19:07:07.360521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:46.910120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354073+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:07:07.354073+0000","last_peered":"2023-07-21T19:07:07.354073+0000","last_clean":"2023-07-21T19:07:07.354073+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:07:07.354073+0000","last_undegraded":"2023-07-21T19:07:07.354073+0000","last_fullsized":"2023-07-21T19:07:07.354073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:32.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358692+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:07:07.358692+0000","last_peered":"2023-07-21T19:07:07.358692+0000","last_clean":"2023-07-21T19:07:07.358692+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:07:07.358692+0000","last_undegraded":"2023-07-21T19:07:07.358692+0000","last_fullsized":"2023-07-21T19:07:07.358692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:43.019102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353325+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:07:07.353325+0000","last_peered":"2023-07-21T19:07:07.353325+0000","last_clean":"2023-07-21T19:07:07.353325+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:07:07.353325+0000","last_undegraded":"2023-07-21T19:07:07.353325+0000","last_fullsized":"2023-07-21T19:07:07.353325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:20.204377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354815+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:07:07.354815+0000","last_peered":"2023-07-21T19:07:07.354815+0000","last_clean":"2023-07-21T19:07:07.354815+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:07:07.354815+0000","last_undegraded":"2023-07-21T19:07:07.354815+0000","last_fullsized":"2023-07-21T19:07:07.354815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:04.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354849+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:07:07.354849+0000","last_peered":"2023-07-21T19:07:07.354849+0000","last_clean":"2023-07-21T19:07:07.354849+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:07:07.354849+0000","last_undegraded":"2023-07-21T19:07:07.354849+0000","last_fullsized":"2023-07-21T19:07:07.354849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:17.572777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354524+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:07:07.354524+0000","last_peered":"2023-07-21T19:07:07.354524+0000","last_clean":"2023-07-21T19:07:07.354524+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:07:07.354524+0000","last_undegraded":"2023-07-21T19:07:07.354524+0000","last_fullsized":"2023-07-21T19:07:07.354524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:50:34.608525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357913+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:07:07.357913+0000","last_peered":"2023-07-21T19:07:07.357913+0000","last_clean":"2023-07-21T19:07:07.357913+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:07:07.357913+0000","last_undegraded":"2023-07-21T19:07:07.357913+0000","last_fullsized":"2023-07-21T19:07:07.357913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:29.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363733+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:07:07.363733+0000","last_peered":"2023-07-21T19:07:07.363733+0000","last_clean":"2023-07-21T19:07:07.363733+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:07:07.363733+0000","last_undegraded":"2023-07-21T19:07:07.363733+0000","last_fullsized":"2023-07-21T19:07:07.363733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:08:37.146951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353019+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:07:07.353019+0000","last_peered":"2023-07-21T19:07:07.353019+0000","last_clean":"2023-07-21T19:07:07.353019+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:07:07.353019+0000","last_undegraded":"2023-07-21T19:07:07.353019+0000","last_fullsized":"2023-07-21T19:07:07.353019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:32.152857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362661+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:07:07.362661+0000","last_peered":"2023-07-21T19:07:07.362661+0000","last_clean":"2023-07-21T19:07:07.362661+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:07:07.362661+0000","last_undegraded":"2023-07-21T19:07:07.362661+0000","last_fullsized":"2023-07-21T19:07:07.362661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:23.961311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345988+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:07:07.345988+0000","last_peered":"2023-07-21T19:07:07.345988+0000","last_clean":"2023-07-21T19:07:07.345988+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:07:07.345988+0000","last_undegraded":"2023-07-21T19:07:07.345988+0000","last_fullsized":"2023-07-21T19:07:07.345988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:15.583393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344330+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:07:07.344330+0000","last_peered":"2023-07-21T19:07:07.344330+0000","last_clean":"2023-07-21T19:07:07.344330+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:07:07.344330+0000","last_undegraded":"2023-07-21T19:07:07.344330+0000","last_fullsized":"2023-07-21T19:07:07.344330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.309276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356265+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:07:07.356265+0000","last_peered":"2023-07-21T19:07:07.356265+0000","last_clean":"2023-07-21T19:07:07.356265+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:07:07.356265+0000","last_undegraded":"2023-07-21T19:07:07.356265+0000","last_fullsized":"2023-07-21T19:07:07.356265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:09:18.215822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359113+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:07:07.359113+0000","last_peered":"2023-07-21T19:07:07.359113+0000","last_clean":"2023-07-21T19:07:07.359113+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:07:07.359113+0000","last_undegraded":"2023-07-21T19:07:07.359113+0000","last_fullsized":"2023-07-21T19:07:07.359113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:59.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360472+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:07:07.360472+0000","last_peered":"2023-07-21T19:07:07.360472+0000","last_clean":"2023-07-21T19:07:07.360472+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:07:07.360472+0000","last_undegraded":"2023-07-21T19:07:07.360472+0000","last_fullsized":"2023-07-21T19:07:07.360472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:13.624531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360608+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:07:07.360608+0000","last_peered":"2023-07-21T19:07:07.360608+0000","last_clean":"2023-07-21T19:07:07.360608+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:07:07.360608+0000","last_undegraded":"2023-07-21T19:07:07.360608+0000","last_fullsized":"2023-07-21T19:07:07.360608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:17.986171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358850+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:07:07.358850+0000","last_peered":"2023-07-21T19:07:07.358850+0000","last_clean":"2023-07-21T19:07:07.358850+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:07:07.358850+0000","last_undegraded":"2023-07-21T19:07:07.358850+0000","last_fullsized":"2023-07-21T19:07:07.358850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:48.916207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360880+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:07:07.360880+0000","last_peered":"2023-07-21T19:07:07.360880+0000","last_clean":"2023-07-21T19:07:07.360880+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:07:07.360880+0000","last_undegraded":"2023-07-21T19:07:07.360880+0000","last_fullsized":"2023-07-21T19:07:07.360880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:08.380972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.851006+0000","last_change":"2023-07-21T19:06:08.851006+0000","last_active":"2023-07-21T19:06:08.851006+0000","last_peered":"2023-07-21T19:06:08.851006+0000","last_clean":"2023-07-21T19:06:08.851006+0000","last_became_active":"2023-07-21T19:06:08.843812+0000","last_became_peered":"2023-07-21T19:06:08.843812+0000","last_unstale":"2023-07-21T19:06:08.851006+0000","last_undegraded":"2023-07-21T19:06:08.851006+0000","last_fullsized":"2023-07-21T19:06:08.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:03.813996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359554+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:07:07.359554+0000","last_peered":"2023-07-21T19:07:07.359554+0000","last_clean":"2023-07-21T19:07:07.359554+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:07:07.359554+0000","last_undegraded":"2023-07-21T19:07:07.359554+0000","last_fullsized":"2023-07-21T19:07:07.359554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:59.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840757+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:06:08.840757+0000","last_peered":"2023-07-21T19:06:08.840757+0000","last_clean":"2023-07-21T19:06:08.840757+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:06:08.840757+0000","last_undegraded":"2023-07-21T19:06:08.840757+0000","last_fullsized":"2023-07-21T19:06:08.840757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:19.820365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363262+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:07:07.363262+0000","last_peered":"2023-07-21T19:07:07.363262+0000","last_clean":"2023-07-21T19:07:07.363262+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:07:07.363262+0000","last_undegraded":"2023-07-21T19:07:07.363262+0000","last_fullsized":"2023-07-21T19:07:07.363262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:48.371444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398380+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:06:08.398380+0000","last_peered":"2023-07-21T19:06:08.398380+0000","last_clean":"2023-07-21T19:06:08.398380+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:06:08.398380+0000","last_undegraded":"2023-07-21T19:06:08.398380+0000","last_fullsized":"2023-07-21T19:06:08.398380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:48.967070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362365+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:07:07.362365+0000","last_peered":"2023-07-21T19:07:07.362365+0000","last_clean":"2023-07-21T19:07:07.362365+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:07:07.362365+0000","last_undegraded":"2023-07-21T19:07:07.362365+0000","last_fullsized":"2023-07-21T19:07:07.362365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:03:26.702743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398317+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:06:08.398317+0000","last_peered":"2023-07-21T19:06:08.398317+0000","last_clean":"2023-07-21T19:06:08.398317+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:06:08.398317+0000","last_undegraded":"2023-07-21T19:06:08.398317+0000","last_fullsized":"2023-07-21T19:06:08.398317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:56.271039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361258+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:07:07.361258+0000","last_peered":"2023-07-21T19:07:07.361258+0000","last_clean":"2023-07-21T19:07:07.361258+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:07:07.361258+0000","last_undegraded":"2023-07-21T19:07:07.361258+0000","last_fullsized":"2023-07-21T19:07:07.361258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:12.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.515456+0000","last_change":"2023-07-21T19:06:08.410381+0000","last_active":"2023-07-21T19:07:19.515456+0000","last_peered":"2023-07-21T19:07:19.515456+0000","last_clean":"2023-07-21T19:07:19.515456+0000","last_became_active":"2023-07-21T19:06:08.405822+0000","last_became_peered":"2023-07-21T19:06:08.405822+0000","last_unstale":"2023-07-21T19:07:19.515456+0000","last_undegraded":"2023-07-21T19:07:19.515456+0000","last_fullsized":"2023-07-21T19:07:19.515456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:14:23.390484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344030+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:07:07.344030+0000","last_peered":"2023-07-21T19:07:07.344030+0000","last_clean":"2023-07-21T19:07:07.344030+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:07:07.344030+0000","last_undegraded":"2023-07-21T19:07:07.344030+0000","last_fullsized":"2023-07-21T19:07:07.344030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:21.320856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400567+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:06:08.400567+0000","last_peered":"2023-07-21T19:06:08.400567+0000","last_clean":"2023-07-21T19:06:08.400567+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:06:08.400567+0000","last_undegraded":"2023-07-21T19:06:08.400567+0000","last_fullsized":"2023-07-21T19:06:08.400567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:41.115053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360782+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:07:07.360782+0000","last_peered":"2023-07-21T19:07:07.360782+0000","last_clean":"2023-07-21T19:07:07.360782+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:07:07.360782+0000","last_undegraded":"2023-07-21T19:07:07.360782+0000","last_fullsized":"2023-07-21T19:07:07.360782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:14:01.147711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404354+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:06:08.404354+0000","last_peered":"2023-07-21T19:06:08.404354+0000","last_clean":"2023-07-21T19:06:08.404354+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:06:08.404354+0000","last_undegraded":"2023-07-21T19:06:08.404354+0000","last_fullsized":"2023-07-21T19:06:08.404354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:28.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361025+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:07:07.361025+0000","last_peered":"2023-07-21T19:07:07.361025+0000","last_clean":"2023-07-21T19:07:07.361025+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:07:07.361025+0000","last_undegraded":"2023-07-21T19:07:07.361025+0000","last_fullsized":"2023-07-21T19:07:07.361025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:34.147350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401383+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:06:08.401383+0000","last_peered":"2023-07-21T19:06:08.401383+0000","last_clean":"2023-07-21T19:06:08.401383+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:06:08.401383+0000","last_undegraded":"2023-07-21T19:06:08.401383+0000","last_fullsized":"2023-07-21T19:06:08.401383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:39:13.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360977+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:07:07.360977+0000","last_peered":"2023-07-21T19:07:07.360977+0000","last_clean":"2023-07-21T19:07:07.360977+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:07:07.360977+0000","last_undegraded":"2023-07-21T19:07:07.360977+0000","last_fullsized":"2023-07-21T19:07:07.360977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:46.382674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.513549+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:07:18.513549+0000","last_peered":"2023-07-21T19:07:18.513549+0000","last_clean":"2023-07-21T19:07:18.513549+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:07:18.513549+0000","last_undegraded":"2023-07-21T19:07:18.513549+0000","last_fullsized":"2023-07-21T19:07:18.513549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:59.692187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363144+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:07:07.363144+0000","last_peered":"2023-07-21T19:07:07.363144+0000","last_clean":"2023-07-21T19:07:07.363144+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:07:07.363144+0000","last_undegraded":"2023-07-21T19:07:07.363144+0000","last_fullsized":"2023-07-21T19:07:07.363144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:32.432702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403970+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:06:08.403970+0000","last_peered":"2023-07-21T19:06:08.403970+0000","last_clean":"2023-07-21T19:06:08.403970+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:06:08.403970+0000","last_undegraded":"2023-07-21T19:06:08.403970+0000","last_fullsized":"2023-07-21T19:06:08.403970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:09.662672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357562+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:07:07.357562+0000","last_peered":"2023-07-21T19:07:07.357562+0000","last_clean":"2023-07-21T19:07:07.357562+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:07:07.357562+0000","last_undegraded":"2023-07-21T19:07:07.357562+0000","last_fullsized":"2023-07-21T19:07:07.357562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.677074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402384+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:06:08.402384+0000","last_peered":"2023-07-21T19:06:08.402384+0000","last_clean":"2023-07-21T19:06:08.402384+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:06:08.402384+0000","last_undegraded":"2023-07-21T19:06:08.402384+0000","last_fullsized":"2023-07-21T19:06:08.402384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:16:07.189240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358434+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:07:07.358434+0000","last_peered":"2023-07-21T19:07:07.358434+0000","last_clean":"2023-07-21T19:07:07.358434+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:07:07.358434+0000","last_undegraded":"2023-07-21T19:07:07.358434+0000","last_fullsized":"2023-07-21T19:07:07.358434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:01.676198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.514756+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:07:19.514756+0000","last_peered":"2023-07-21T19:07:19.514756+0000","last_clean":"2023-07-21T19:07:19.514756+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:07:19.514756+0000","last_undegraded":"2023-07-21T19:07:19.514756+0000","last_fullsized":"2023-07-21T19:07:19.514756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.353689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58815,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":161,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.334355+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:07:07.334355+0000","last_peered":"2023-07-21T19:07:07.334355+0000","last_clean":"2023-07-21T19:07:07.334355+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:07:07.334355+0000","last_undegraded":"2023-07-21T19:07:07.334355+0000","last_fullsized":"2023-07-21T19:07:07.334355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:31.374240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405148+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:06:08.405148+0000","last_peered":"2023-07-21T19:06:08.405148+0000","last_clean":"2023-07-21T19:06:08.405148+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:06:08.405148+0000","last_undegraded":"2023-07-21T19:06:08.405148+0000","last_fullsized":"2023-07-21T19:06:08.405148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:20.602887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.351617+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:07:07.351617+0000","last_peered":"2023-07-21T19:07:07.351617+0000","last_clean":"2023-07-21T19:07:07.351617+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:07:07.351617+0000","last_undegraded":"2023-07-21T19:07:07.351617+0000","last_fullsized":"2023-07-21T19:07:07.351617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:59:21.737635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401848+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:06:08.401848+0000","last_peered":"2023-07-21T19:06:08.401848+0000","last_clean":"2023-07-21T19:06:08.401848+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:06:08.401848+0000","last_undegraded":"2023-07-21T19:06:08.401848+0000","last_fullsized":"2023-07-21T19:06:08.401848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:48.688808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361072+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:07:07.361072+0000","last_peered":"2023-07-21T19:07:07.361072+0000","last_clean":"2023-07-21T19:07:07.361072+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:07:07.361072+0000","last_undegraded":"2023-07-21T19:07:07.361072+0000","last_fullsized":"2023-07-21T19:07:07.361072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:55.454258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":123,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.467900+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:07:18.467900+0000","last_peered":"2023-07-21T19:07:18.467900+0000","last_clean":"2023-07-21T19:07:18.467900+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:07:18.467900+0000","last_undegraded":"2023-07-21T19:07:18.467900+0000","last_fullsized":"2023-07-21T19:07:18.467900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:26.036729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358107+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:07:07.358107+0000","last_peered":"2023-07-21T19:07:07.358107+0000","last_clean":"2023-07-21T19:07:07.358107+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:07:07.358107+0000","last_undegraded":"2023-07-21T19:07:07.358107+0000","last_fullsized":"2023-07-21T19:07:07.358107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:24.967595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:31:53.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361317+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:07:07.361317+0000","last_peered":"2023-07-21T19:07:07.361317+0000","last_clean":"2023-07-21T19:07:07.361317+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:07:07.361317+0000","last_undegraded":"2023-07-21T19:07:07.361317+0000","last_fullsized":"2023-07-21T19:07:07.361317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:37.822944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398316+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:06:08.398316+0000","last_peered":"2023-07-21T19:06:08.398316+0000","last_clean":"2023-07-21T19:06:08.398316+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:06:08.398316+0000","last_undegraded":"2023-07-21T19:06:08.398316+0000","last_fullsized":"2023-07-21T19:06:08.398316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:37.555620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.343477+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:07:07.343477+0000","last_peered":"2023-07-21T19:07:07.343477+0000","last_clean":"2023-07-21T19:07:07.343477+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:07:07.343477+0000","last_undegraded":"2023-07-21T19:07:07.343477+0000","last_fullsized":"2023-07-21T19:07:07.343477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:42.470455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.406908+0000","last_change":"2023-07-21T19:06:08.406908+0000","last_active":"2023-07-21T19:06:08.406908+0000","last_peered":"2023-07-21T19:06:08.406908+0000","last_clean":"2023-07-21T19:06:08.406908+0000","last_became_active":"2023-07-21T19:06:08.406531+0000","last_became_peered":"2023-07-21T19:06:08.406531+0000","last_unstale":"2023-07-21T19:06:08.406908+0000","last_undegraded":"2023-07-21T19:06:08.406908+0000","last_fullsized":"2023-07-21T19:06:08.406908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:04.808701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357349+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:07:07.357349+0000","last_peered":"2023-07-21T19:07:07.357349+0000","last_clean":"2023-07-21T19:07:07.357349+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:07:07.357349+0000","last_undegraded":"2023-07-21T19:07:07.357349+0000","last_fullsized":"2023-07-21T19:07:07.357349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:01.821241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404216+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:06:08.404216+0000","last_peered":"2023-07-21T19:06:08.404216+0000","last_clean":"2023-07-21T19:06:08.404216+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:06:08.404216+0000","last_undegraded":"2023-07-21T19:06:08.404216+0000","last_fullsized":"2023-07-21T19:06:08.404216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:18.679466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359555+0000","last_change":"2023-07-21T19:06:08.408400+0000","last_active":"2023-07-21T19:07:07.359555+0000","last_peered":"2023-07-21T19:07:07.359555+0000","last_clean":"2023-07-21T19:07:07.359555+0000","last_became_active":"2023-07-21T19:06:08.408005+0000","last_became_peered":"2023-07-21T19:06:08.408005+0000","last_unstale":"2023-07-21T19:07:07.359555+0000","last_undegraded":"2023-07-21T19:07:07.359555+0000","last_fullsized":"2023-07-21T19:07:07.359555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:20.230383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342774+0000","last_change":"2023-07-21T19:06:08.407039+0000","last_active":"2023-07-21T19:07:07.342774+0000","last_peered":"2023-07-21T19:07:07.342774+0000","last_clean":"2023-07-21T19:07:07.342774+0000","last_became_active":"2023-07-21T19:06:08.406635+0000","last_became_peered":"2023-07-21T19:06:08.406635+0000","last_unstale":"2023-07-21T19:07:07.342774+0000","last_undegraded":"2023-07-21T19:07:07.342774+0000","last_fullsized":"2023-07-21T19:07:07.342774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:18.504252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:16.461334+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:07:16.461334+0000","last_peered":"2023-07-21T19:07:16.461334+0000","last_clean":"2023-07-21T19:07:16.461334+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:07:16.461334+0000","last_undegraded":"2023-07-21T19:07:16.461334+0000","last_fullsized":"2023-07-21T19:07:16.461334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:21.302828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362955+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:07:07.362955+0000","last_peered":"2023-07-21T19:07:07.362955+0000","last_clean":"2023-07-21T19:07:07.362955+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:07:07.362955+0000","last_undegraded":"2023-07-21T19:07:07.362955+0000","last_fullsized":"2023-07-21T19:07:07.362955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:42.921526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398968+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:06:08.398968+0000","last_peered":"2023-07-21T19:06:08.398968+0000","last_clean":"2023-07-21T19:06:08.398968+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:06:08.398968+0000","last_undegraded":"2023-07-21T19:06:08.398968+0000","last_fullsized":"2023-07-21T19:06:08.398968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:50.624939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404291+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:06:08.404291+0000","last_peered":"2023-07-21T19:06:08.404291+0000","last_clean":"2023-07-21T19:06:08.404291+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:06:08.404291+0000","last_undegraded":"2023-07-21T19:06:08.404291+0000","last_fullsized":"2023-07-21T19:06:08.404291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:45.400614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.346572+0000","last_change":"2023-07-21T19:06:08.409780+0000","last_active":"2023-07-21T19:07:07.346572+0000","last_peered":"2023-07-21T19:07:07.346572+0000","last_clean":"2023-07-21T19:07:07.346572+0000","last_became_active":"2023-07-21T19:06:08.409394+0000","last_became_peered":"2023-07-21T19:06:08.409394+0000","last_unstale":"2023-07-21T19:07:07.346572+0000","last_undegraded":"2023-07-21T19:07:07.346572+0000","last_fullsized":"2023-07-21T19:07:07.346572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:36:45.599367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404518+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:06:08.404518+0000","last_peered":"2023-07-21T19:06:08.404518+0000","last_clean":"2023-07-21T19:06:08.404518+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:06:08.404518+0000","last_undegraded":"2023-07-21T19:06:08.404518+0000","last_fullsized":"2023-07-21T19:06:08.404518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:04.683020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.413663+0000","last_change":"2023-07-21T19:06:08.413252+0000","last_active":"2023-07-21T19:06:08.413663+0000","last_peered":"2023-07-21T19:06:08.413663+0000","last_clean":"2023-07-21T19:06:08.413663+0000","last_became_active":"2023-07-21T19:06:08.407914+0000","last_became_peered":"2023-07-21T19:06:08.407914+0000","last_unstale":"2023-07-21T19:06:08.413663+0000","last_undegraded":"2023-07-21T19:06:08.413663+0000","last_fullsized":"2023-07-21T19:06:08.413663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:08.813981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.347131+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:07:07.347131+0000","last_peered":"2023-07-21T19:07:07.347131+0000","last_clean":"2023-07-21T19:07:07.347131+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:07:07.347131+0000","last_undegraded":"2023-07-21T19:07:07.347131+0000","last_fullsized":"2023-07-21T19:07:07.347131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:17.603158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:10.564825+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:07:10.564825+0000","last_peered":"2023-07-21T19:07:10.564825+0000","last_clean":"2023-07-21T19:07:10.564825+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:07:10.564825+0000","last_undegraded":"2023-07-21T19:07:10.564825+0000","last_fullsized":"2023-07-21T19:07:10.564825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:46.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356114+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:07:07.356114+0000","last_peered":"2023-07-21T19:07:07.356114+0000","last_clean":"2023-07-21T19:07:07.356114+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:07:07.356114+0000","last_undegraded":"2023-07-21T19:07:07.356114+0000","last_fullsized":"2023-07-21T19:07:07.356114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:34:12.413946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399221+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:06:08.399221+0000","last_peered":"2023-07-21T19:06:08.399221+0000","last_clean":"2023-07-21T19:06:08.399221+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:06:08.399221+0000","last_undegraded":"2023-07-21T19:06:08.399221+0000","last_fullsized":"2023-07-21T19:06:08.399221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.493447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360651+0000","last_change":"2023-07-21T19:06:08.842743+0000","last_active":"2023-07-21T19:07:07.360651+0000","last_peered":"2023-07-21T19:07:07.360651+0000","last_clean":"2023-07-21T19:07:07.360651+0000","last_became_active":"2023-07-21T19:06:08.842349+0000","last_became_peered":"2023-07-21T19:06:08.842349+0000","last_unstale":"2023-07-21T19:07:07.360651+0000","last_undegraded":"2023-07-21T19:07:07.360651+0000","last_fullsized":"2023-07-21T19:07:07.360651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:05:30.960744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841889+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:06:08.841889+0000","last_peered":"2023-07-21T19:06:08.841889+0000","last_clean":"2023-07-21T19:06:08.841889+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:06:08.841889+0000","last_undegraded":"2023-07-21T19:06:08.841889+0000","last_fullsized":"2023-07-21T19:06:08.841889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:57.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.310459+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:07:07.310459+0000","last_peered":"2023-07-21T19:07:07.310459+0000","last_clean":"2023-07-21T19:07:07.310459+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:07:07.310459+0000","last_undegraded":"2023-07-21T19:07:07.310459+0000","last_fullsized":"2023-07-21T19:07:07.310459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:17.675575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840681+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:06:08.840681+0000","last_peered":"2023-07-21T19:06:08.840681+0000","last_clean":"2023-07-21T19:06:08.840681+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:06:08.840681+0000","last_undegraded":"2023-07-21T19:06:08.840681+0000","last_fullsized":"2023-07-21T19:06:08.840681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:11.395258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354043+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:07:07.354043+0000","last_peered":"2023-07-21T19:07:07.354043+0000","last_clean":"2023-07-21T19:07:07.354043+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:07:07.354043+0000","last_undegraded":"2023-07-21T19:07:07.354043+0000","last_fullsized":"2023-07-21T19:07:07.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:28:35.734063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404850+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:06:08.404850+0000","last_peered":"2023-07-21T19:06:08.404850+0000","last_clean":"2023-07-21T19:06:08.404850+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:06:08.404850+0000","last_undegraded":"2023-07-21T19:06:08.404850+0000","last_fullsized":"2023-07-21T19:06:08.404850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:53.659216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356469+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:07:07.356469+0000","last_peered":"2023-07-21T19:07:07.356469+0000","last_clean":"2023-07-21T19:07:07.356469+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:07:07.356469+0000","last_undegraded":"2023-07-21T19:07:07.356469+0000","last_fullsized":"2023-07-21T19:07:07.356469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:19:55.786796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398085+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:06:08.398085+0000","last_peered":"2023-07-21T19:06:08.398085+0000","last_clean":"2023-07-21T19:06:08.398085+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:06:08.398085+0000","last_undegraded":"2023-07-21T19:06:08.398085+0000","last_fullsized":"2023-07-21T19:06:08.398085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:02.430774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362043+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:07:07.362043+0000","last_peered":"2023-07-21T19:07:07.362043+0000","last_clean":"2023-07-21T19:07:07.362043+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:07:07.362043+0000","last_undegraded":"2023-07-21T19:07:07.362043+0000","last_fullsized":"2023-07-21T19:07:07.362043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:08:39.735825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405117+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:06:08.405117+0000","last_peered":"2023-07-21T19:06:08.405117+0000","last_clean":"2023-07-21T19:06:08.405117+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:06:08.405117+0000","last_undegraded":"2023-07-21T19:06:08.405117+0000","last_fullsized":"2023-07-21T19:06:08.405117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:44.759183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362557+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:07:07.362557+0000","last_peered":"2023-07-21T19:07:07.362557+0000","last_clean":"2023-07-21T19:07:07.362557+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:07:07.362557+0000","last_undegraded":"2023-07-21T19:07:07.362557+0000","last_fullsized":"2023-07-21T19:07:07.362557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:16.325005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402542+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:06:08.402542+0000","last_peered":"2023-07-21T19:06:08.402542+0000","last_clean":"2023-07-21T19:06:08.402542+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:06:08.402542+0000","last_undegraded":"2023-07-21T19:06:08.402542+0000","last_fullsized":"2023-07-21T19:06:08.402542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:41.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.355564+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:07:07.355564+0000","last_peered":"2023-07-21T19:07:07.355564+0000","last_clean":"2023-07-21T19:07:07.355564+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:07:07.355564+0000","last_undegraded":"2023-07-21T19:07:07.355564+0000","last_fullsized":"2023-07-21T19:07:07.355564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:08:59.431298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401998+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:06:08.401998+0000","last_peered":"2023-07-21T19:06:08.401998+0000","last_clean":"2023-07-21T19:06:08.401998+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:06:08.401998+0000","last_undegraded":"2023-07-21T19:06:08.401998+0000","last_fullsized":"2023-07-21T19:06:08.401998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:17.091156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354174+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:07:07.354174+0000","last_peered":"2023-07-21T19:07:07.354174+0000","last_clean":"2023-07-21T19:07:07.354174+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:07:07.354174+0000","last_undegraded":"2023-07-21T19:07:07.354174+0000","last_fullsized":"2023-07-21T19:07:07.354174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:09.484987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345151+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:07:07.345151+0000","last_peered":"2023-07-21T19:07:07.345151+0000","last_clean":"2023-07-21T19:07:07.345151+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:07:07.345151+0000","last_undegraded":"2023-07-21T19:07:07.345151+0000","last_fullsized":"2023-07-21T19:07:07.345151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:56.095641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352815+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:07:07.352815+0000","last_peered":"2023-07-21T19:07:07.352815+0000","last_clean":"2023-07-21T19:07:07.352815+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:07:07.352815+0000","last_undegraded":"2023-07-21T19:07:07.352815+0000","last_fullsized":"2023-07-21T19:07:07.352815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:24.580856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361130+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:07:07.361130+0000","last_peered":"2023-07-21T19:07:07.361130+0000","last_clean":"2023-07-21T19:07:07.361130+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:07:07.361130+0000","last_undegraded":"2023-07-21T19:07:07.361130+0000","last_fullsized":"2023-07-21T19:07:07.361130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:17.790433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359810+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:07:07.359810+0000","last_peered":"2023-07-21T19:07:07.359810+0000","last_clean":"2023-07-21T19:07:07.359810+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:07:07.359810+0000","last_undegraded":"2023-07-21T19:07:07.359810+0000","last_fullsized":"2023-07-21T19:07:07.359810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:29.261412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354985+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:07:07.354985+0000","last_peered":"2023-07-21T19:07:07.354985+0000","last_clean":"2023-07-21T19:07:07.354985+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:07:07.354985+0000","last_undegraded":"2023-07-21T19:07:07.354985+0000","last_fullsized":"2023-07-21T19:07:07.354985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:47:56.761424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342375+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:07:07.342375+0000","last_peered":"2023-07-21T19:07:07.342375+0000","last_clean":"2023-07-21T19:07:07.342375+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:07:07.342375+0000","last_undegraded":"2023-07-21T19:07:07.342375+0000","last_fullsized":"2023-07-21T19:07:07.342375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:45:42.804371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359579+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:07:07.359579+0000","last_peered":"2023-07-21T19:07:07.359579+0000","last_clean":"2023-07-21T19:07:07.359579+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:07:07.359579+0000","last_undegraded":"2023-07-21T19:07:07.359579+0000","last_fullsized":"2023-07-21T19:07:07.359579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:42.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360257+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:07:07.360257+0000","last_peered":"2023-07-21T19:07:07.360257+0000","last_clean":"2023-07-21T19:07:07.360257+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:07:07.360257+0000","last_undegraded":"2023-07-21T19:07:07.360257+0000","last_fullsized":"2023-07-21T19:07:07.360257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:40:21.846553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360450+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:07:07.360450+0000","last_peered":"2023-07-21T19:07:07.360450+0000","last_clean":"2023-07-21T19:07:07.360450+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:07:07.360450+0000","last_undegraded":"2023-07-21T19:07:07.360450+0000","last_fullsized":"2023-07-21T19:07:07.360450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:19.869711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359999+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:07:07.359999+0000","last_peered":"2023-07-21T19:07:07.359999+0000","last_clean":"2023-07-21T19:07:07.359999+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:07:07.359999+0000","last_undegraded":"2023-07-21T19:07:07.359999+0000","last_fullsized":"2023-07-21T19:07:07.359999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:33.674612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362482+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:07:07.362482+0000","last_peered":"2023-07-21T19:07:07.362482+0000","last_clean":"2023-07-21T19:07:07.362482+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:07:07.362482+0000","last_undegraded":"2023-07-21T19:07:07.362482+0000","last_fullsized":"2023-07-21T19:07:07.362482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:23:36.091795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352462+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:07:07.352462+0000","last_peered":"2023-07-21T19:07:07.352462+0000","last_clean":"2023-07-21T19:07:07.352462+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:07:07.352462+0000","last_undegraded":"2023-07-21T19:07:07.352462+0000","last_fullsized":"2023-07-21T19:07:07.352462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:45:18.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345299+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:07:07.345299+0000","last_peered":"2023-07-21T19:07:07.345299+0000","last_clean":"2023-07-21T19:07:07.345299+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:07:07.345299+0000","last_undegraded":"2023-07-21T19:07:07.345299+0000","last_fullsized":"2023-07-21T19:07:07.345299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:02.887348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":376,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1568,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":456247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64900000000000002,"15min":0.65300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":1.131,"15min":1.131},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.75},"min":{"1min":0.56899999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.044,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":1.083,"15min":1.083},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.006,"5min":1.395,"15min":1.395},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89800000000000002,"5min":1.208,"15min":1.208},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.073,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":1.24,"15min":1.24},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93000000000000005,"15min":0.93600000000000005},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.095,"5min":1.595,"15min":1.595},"last":1.026},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"min":{"1min":0.59099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0840000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.877}]},{"osd":8,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.98799999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.97299999999999998,"15min":0.97099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.101},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.0309999999999999,"15min":0.98299999999999998},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.2670000000000003,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.085}]},{"osd":10,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.95399999999999996},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.615,"15min":1.615},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.75900000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.099,"5min":1.681,"15min":1.681},"last":1.1839999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1106962,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.4770000000000003,"5min":6.4770000000000003,"15min":6.4770000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.441},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86099999999999999,"5min":2.323,"15min":2.323},"last":0.65400000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.876,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":3.399,"15min":3.399},"last":0.5},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.626}]},{"osd":4,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0049999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.84399999999999997},"min":{"1min":0.67800000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.0669999999999999,"5min":1.238,"15min":1.238},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.878,"15min":0.872},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":3.331,"15min":3.331},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.96599999999999997,"15min":0.93500000000000005},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98899999999999999,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93600000000000005,"15min":0.93400000000000005},"min":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.171,"5min":2.661,"15min":2.661},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.93500000000000005,"15min":0.97699999999999998},"min":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0900000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.87}]},{"osd":9,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.032,"15min":1.006},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0780000000000001,"5min":6.4249999999999998,"15min":6.4249999999999998},"last":0.755},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":1.016},"min":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0449999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.94299999999999995,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0840000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.878,"15min":0.876},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.93799999999999994}]}]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":456461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.63900000000000001,"15min":0.59199999999999997},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.59799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":9.2650000000000006,"5min":9.2650000000000006,"15min":9.2650000000000006},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.433,"15min":0.433},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.84499999999999997,"15min":0.83299999999999996},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.4019999999999992,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.032}]},{"osd":3,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":9.3130000000000006,"5min":9.3130000000000006,"15min":9.3130000000000006},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6220000000000001,"5min":1.637,"15min":1.637},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.371,"5min":0.98799999999999999,"15min":0.94999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":9.4749999999999996,"5min":9.4749999999999996,"15min":9.4749999999999996},"last":1.002}]},{"osd":6,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1040000000000001,"5min":1.345,"15min":1.345},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.0189999999999999,"15min":1.0329999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":9.3740000000000006,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.84199999999999997}]},{"osd":8,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.651,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.018}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.0389999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":9.3439999999999994,"5min":9.3439999999999994,"15min":9.3439999999999994},"last":0.747},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.014,"15min":1.04},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94999999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":455777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.66500000000000004},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.036}]},{"osd":2,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.874,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.85699999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.012},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.169,"15min":1.169},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0669999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.056}]},{"osd":6,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.88,"15min":0.874},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.1310000000000002,"5min":4.1310000000000002,"15min":4.1310000000000002},"last":1.137}]},{"osd":7,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.157,"5min":1.403,"15min":1.403},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90200000000000002,"15min":0.89600000000000002},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1870000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.91700000000000004,"15min":0.86899999999999999},"min":{"1min":0.64700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.3230000000000004,"5min":4.3230000000000004,"15min":4.3230000000000004},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.86599999999999999,"15min":0.82599999999999996},"min":{"1min":0.73199999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.238,"15min":1.238},"last":1.1559999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.96999999999999997,"15min":0.94099999999999995},"min":{"1min":0.69299999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.329},{"interface":"front","average":{"1min":1.083,"5min":0.93600000000000005,"15min":0.89200000000000002},"min":{"1min":0.90200000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2609999999999999}]},{"osd":11,"last update":"Fri Jul 21 19:06:36 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.288},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.204}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":1049327,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91800000000000004,"5min":1.05,"15min":1.05},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72899999999999998,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.96499999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.94699999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":1.016,"15min":1.016},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.84399999999999997,"5min":0.996,"15min":0.996},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.877,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98199999999999998,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79800000000000004,"15min":0.77300000000000002},"min":{"1min":0.67900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0069999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.81799999999999995,"15min":0.85199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.234,"15min":1.234},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81699999999999995,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.208,"15min":1.208},"last":0.60399999999999998}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":0.89700000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.079,"5min":1.095,"15min":1.095},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.93799999999999994},"min":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0660000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.93700000000000006}]},{"osd":11,"last update":"Fri Jul 21 19:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91300000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":455856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.51400000000000001,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.68899999999999995,"15min":0.65400000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.505},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.52300000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.68600000000000005,"15min":0.61299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6099999999999994,"5min":9.6099999999999994,"15min":9.6099999999999994},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81899999999999995,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.867,"5min":6.867,"15min":6.867},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.163,"5min":0.85999999999999999,"15min":0.81899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.9029999999999996,"5min":6.9029999999999996,"15min":6.9029999999999996},"last":0.75}]},{"osd":5,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.90800000000000003,"15min":0.84199999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Jul 21 19:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86199999999999999,"15min":0.82799999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2589999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.095,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":0.73099999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.88,"15min":0.86399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.26,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.93000000000000005,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.609,"5min":3.609,"15min":3.609},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.94599999999999995,"15min":0.95099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.91000000000000003,"15min":0.94199999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93400000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1539999999999999,"5min":3.9969999999999999,"15min":3.9969999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0469999999999999,"15min":1.048},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":519360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.98099999999999998,"5min":1.073,"15min":1.073},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.93999999999999995},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.64200000000000002},"min":{"1min":0.56499999999999995,"5min":0.38600000000000001,"15min":0.36599999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.014}]},{"osd":3,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.47599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.45600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.52000000000000002},"min":{"1min":0.48199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.72599999999999998,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.78400000000000003,"15min":0.70999999999999996},"min":{"1min":0.68300000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.079}]},{"osd":6,"last update":"Fri Jul 21 19:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.67100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.629,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]},{"osd":7,"last update":"Fri Jul 21 19:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76200000000000001,"15min":0.72999999999999998},"min":{"1min":0.57799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73499999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.91600000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.61499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.0269999999999999,"5min":1.502,"15min":1.502},"last":0.86599999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.91400000000000003,"15min":0.89700000000000002},"min":{"1min":0.71699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.113},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.94399999999999995,"15min":0.91800000000000004},"min":{"1min":0.76300000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.0980000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95599999999999996,"15min":0.93999999999999995},"min":{"1min":0.78500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.169},{"interface":"front","average":{"1min":1.024,"5min":0.96999999999999997,"15min":0.96499999999999997},"min":{"1min":0.76400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.129}]},{"osd":11,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.052},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.1419999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":457121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68300000000000005,"15min":0.63500000000000001},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1539999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.39300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.53200000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.495}]},{"osd":3,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.77300000000000002,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.91500000000000004},"min":{"1min":0.68799999999999994,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.244,"5min":1.304,"15min":1.304},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82399999999999995,"15min":0.84599999999999997},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89000000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.93200000000000005},"min":{"1min":0.55600000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84899999999999998,"15min":0.84999999999999998},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.034,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":5.9950000000000001,"5min":5.9950000000000001,"15min":5.9950000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.68799999999999994,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":515186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95599999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68100000000000005,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":1.081,"15min":1.081},"last":0.754}]},{"osd":3,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.53700000000000003},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94799999999999995,"5min":1.026,"15min":1.026},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90200000000000002,"5min":1.167,"15min":1.167},"last":0.625}]},{"osd":6,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.218,"5min":1.333,"15min":1.333},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.88700000000000001,"15min":0.877},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.82099999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":456173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46800000000000003,"15min":0.42199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70999999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.34200000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.52000000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.873,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.86599999999999999,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.74299999999999999,"15min":0.65300000000000002},"min":{"1min":0.56200000000000006,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71699999999999997,"15min":0.64500000000000002},"min":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.42099999999999999},"max":{"1min":0.98299999999999998,"5min":1.048,"15min":1.048},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1020000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.75700000000000001,"15min":0.69899999999999995},"min":{"1min":0.53300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92500000000000004,"5min":1.264,"15min":1.264},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0649999999999999,"5min":1.141,"15min":1.141},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.61099999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2070000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.876}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89900000000000002,"15min":0.93799999999999994},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2030000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.84699999999999998,"15min":0.89700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.151,"5min":1.196,"15min":1.196},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.091,"5min":1.153,"15min":1.153},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.89800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":458310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.443},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.44600000000000001},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.167,"15min":1.167},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69999999999999996,"15min":0.68000000000000005},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.76900000000000002,"15min":0.66900000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97199999999999998,"5min":1.079,"15min":1.079},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97799999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.82999999999999996,"15min":0.79700000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71399999999999997,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.754,"15min":0.69999999999999996},"min":{"1min":0.51300000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.877},"min":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.173,"5min":1.298,"15min":1.298},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93200000000000005,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":2.2120000000000002,"15min":2.7999999999999998},"min":{"1min":0.69699999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.145,"5min":42.396000000000001,"15min":42.396000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":2.105,"15min":2.7160000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.70799999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1048576,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63600000000000001,"15min":0.61399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90200000000000002,"5min":1.042,"15min":1.042},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71799999999999997,"15min":0.68999999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.40799999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.57499999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.61499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92900000000000005,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.135,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.71399999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1499999999999999,"5min":1.163,"15min":1.163},"last":1.006}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.505,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1080000000000001,"5min":1.111,"15min":1.111},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.091,"5min":1.103,"15min":1.103},"last":0.91400000000000003}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.048},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80400000000000005}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.161,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90100000000000002,"15min":0.875},"min":{"1min":0.74199999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1819999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.095}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:07:22.096 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T19:07:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.323+0000 7f436fdff700 1 -- 172.21.15.5:0/1580852272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43680ffb00 msgr2=0x7f43680fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.323+0000 7f436fdff700 1 --2- 172.21.15.5:0/1580852272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43680ffb00 0x7f43680fff10 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f4364009990 tx=0x7f4364009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 -- 172.21.15.5:0/1580852272 shutdown_connections 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 --2- 172.21.15.5:0/1580852272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4368101680 0x7f4368107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 --2- 172.21.15.5:0/1580852272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43680ffb00 0x7f43680fff10 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 --2- 172.21.15.5:0/1580852272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368100cf0 0x7f4368101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 -- 172.21.15.5:0/1580852272 >> 172.21.15.5:0/1580852272 conn(0x7f43680fb0e0 msgr2=0x7f43680fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 -- 172.21.15.5:0/1580852272 shutdown_connections 2023-07-21T19:07:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.324+0000 7f436fdff700 1 -- 172.21.15.5:0/1580852272 wait complete. 2023-07-21T19:07:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 Processor -- start 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 -- start start 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43680ffb00 0x7f43681957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4368100cf0 0x7f4368195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d340 con 0x7f4368101680 2023-07-21T19:07:22.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d480 con 0x7f4368100cf0 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.325+0000 7f436fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f436819d610 con 0x7f43680ffb00 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47964/0 (socket says 172.21.15.5:47964) 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 -- 172.21.15.5:0/1608112631 learned_addr learned my addr 172.21.15.5:0/1608112631 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436db9b700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43680ffb00 0x7f43681957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436d39a700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4368100cf0 0x7f4368195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 -- 172.21.15.5:0/1608112631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43680ffb00 msgr2=0x7f43681957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43680ffb00 0x7f43681957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 -- 172.21.15.5:0/1608112631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4368100cf0 msgr2=0x7f4368195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4368100cf0 0x7f4368195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 -- 172.21.15.5:0/1608112631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4358009520 con 0x7f4368101680 2023-07-21T19:07:22.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.326+0000 7f436e39c700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f436000ef30 tx=0x7f436000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.327+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4360018070 con 0x7f4368101680 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.327+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43640095f0 con 0x7f4368101680 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.327+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f436000f040 con 0x7f4368101680 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.327+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f436000cb80 con 0x7f4368101680 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.327+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436819dc60 con 0x7f4368101680 2023-07-21T19:07:22.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.328+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4368106ab0 con 0x7f4368101680 2023-07-21T19:07:22.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.330+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f436000cd60 con 0x7f4368101680 2023-07-21T19:07:22.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.331+0000 7f435effd700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 0x7f43540a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.331+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f43600cc390 con 0x7f4368101680 2023-07-21T19:07:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.332+0000 7f436db9b700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 0x7f43540a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.332+0000 7f436db9b700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 0x7f43540a3650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f43640098e0 tx=0x7f436400b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:22.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.335+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4360099610 con 0x7f4368101680 2023-07-21T19:07:22.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:22 smithi005 ceph-mon[126460]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2023-07-21T19:07:22.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:22 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1854892342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:22.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.540+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4368060950 con 0x7f43540a11a0 2023-07-21T19:07:22.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.551+0000 7f435effd700 1 -- 172.21.15.5:0/1608112631 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343494 (secure 0 0 0) 0x7f4368060950 con 0x7f43540a11a0 2023-07-21T19:07:22.564 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.555+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 msgr2=0x7f43540a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.555+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 0x7f43540a3650 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f43640098e0 tx=0x7f436400b260 comp rx=0 tx=0).stop 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.555+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 msgr2=0x7f436819cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.555+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f436000ef30 tx=0x7f436000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 shutdown_connections 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f43540a11a0 0x7f43540a3650 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4368101680 0x7f436819cd70 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43680ffb00 0x7f43681957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 --2- 172.21.15.5:0/1608112631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4368100cf0 0x7f4368195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 >> 172.21.15.5:0/1608112631 conn(0x7f43680fb0e0 msgr2=0x7f43680fd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:22.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 shutdown_connections 2023-07-21T19:07:22.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.556+0000 7f436fdff700 1 -- 172.21.15.5:0/1608112631 wait complete. 2023-07-21T19:07:22.568 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:07:22.583 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-07-21T19:07:21.341914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654440,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":495,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182304,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334996,"kb_used_data":11572,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606828,"statfs":{"total":1151940427776,"available":1151597391872,"internally_reserved":0,"allocated":11849728,"data_stored":7435356,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002992"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360521+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:07:07.360521+0000","last_peered":"2023-07-21T19:07:07.360521+0000","last_clean":"2023-07-21T19:07:07.360521+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:07:07.360521+0000","last_undegraded":"2023-07-21T19:07:07.360521+0000","last_fullsized":"2023-07-21T19:07:07.360521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:46.910120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354073+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:07:07.354073+0000","last_peered":"2023-07-21T19:07:07.354073+0000","last_clean":"2023-07-21T19:07:07.354073+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:07:07.354073+0000","last_undegraded":"2023-07-21T19:07:07.354073+0000","last_fullsized":"2023-07-21T19:07:07.354073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:32.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358692+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:07:07.358692+0000","last_peered":"2023-07-21T19:07:07.358692+0000","last_clean":"2023-07-21T19:07:07.358692+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:07:07.358692+0000","last_undegraded":"2023-07-21T19:07:07.358692+0000","last_fullsized":"2023-07-21T19:07:07.358692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:43.019102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353325+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:07:07.353325+0000","last_peered":"2023-07-21T19:07:07.353325+0000","last_clean":"2023-07-21T19:07:07.353325+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:07:07.353325+0000","last_undegraded":"2023-07-21T19:07:07.353325+0000","last_fullsized":"2023-07-21T19:07:07.353325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:20.204377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354815+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:07:07.354815+0000","last_peered":"2023-07-21T19:07:07.354815+0000","last_clean":"2023-07-21T19:07:07.354815+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:07:07.354815+0000","last_undegraded":"2023-07-21T19:07:07.354815+0000","last_fullsized":"2023-07-21T19:07:07.354815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:04.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354849+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:07:07.354849+0000","last_peered":"2023-07-21T19:07:07.354849+0000","last_clean":"2023-07-21T19:07:07.354849+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:07:07.354849+0000","last_undegraded":"2023-07-21T19:07:07.354849+0000","last_fullsized":"2023-07-21T19:07:07.354849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:17.572777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354524+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:07:07.354524+0000","last_peered":"2023-07-21T19:07:07.354524+0000","last_clean":"2023-07-21T19:07:07.354524+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:07:07.354524+0000","last_undegraded":"2023-07-21T19:07:07.354524+0000","last_fullsized":"2023-07-21T19:07:07.354524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:50:34.608525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357913+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:07:07.357913+0000","last_peered":"2023-07-21T19:07:07.357913+0000","last_clean":"2023-07-21T19:07:07.357913+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:07:07.357913+0000","last_undegraded":"2023-07-21T19:07:07.357913+0000","last_fullsized":"2023-07-21T19:07:07.357913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:29.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363733+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:07:07.363733+0000","last_peered":"2023-07-21T19:07:07.363733+0000","last_clean":"2023-07-21T19:07:07.363733+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:07:07.363733+0000","last_undegraded":"2023-07-21T19:07:07.363733+0000","last_fullsized":"2023-07-21T19:07:07.363733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:08:37.146951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353019+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:07:07.353019+0000","last_peered":"2023-07-21T19:07:07.353019+0000","last_clean":"2023-07-21T19:07:07.353019+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:07:07.353019+0000","last_undegraded":"2023-07-21T19:07:07.353019+0000","last_fullsized":"2023-07-21T19:07:07.353019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:32.152857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362661+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:07:07.362661+0000","last_peered":"2023-07-21T19:07:07.362661+0000","last_clean":"2023-07-21T19:07:07.362661+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:07:07.362661+0000","last_undegraded":"2023-07-21T19:07:07.362661+0000","last_fullsized":"2023-07-21T19:07:07.362661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:23.961311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345988+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:07:07.345988+0000","last_peered":"2023-07-21T19:07:07.345988+0000","last_clean":"2023-07-21T19:07:07.345988+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:07:07.345988+0000","last_undegraded":"2023-07-21T19:07:07.345988+0000","last_fullsized":"2023-07-21T19:07:07.345988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:15.583393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344330+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:07:07.344330+0000","last_peered":"2023-07-21T19:07:07.344330+0000","last_clean":"2023-07-21T19:07:07.344330+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:07:07.344330+0000","last_undegraded":"2023-07-21T19:07:07.344330+0000","last_fullsized":"2023-07-21T19:07:07.344330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.309276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356265+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:07:07.356265+0000","last_peered":"2023-07-21T19:07:07.356265+0000","last_clean":"2023-07-21T19:07:07.356265+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:07:07.356265+0000","last_undegraded":"2023-07-21T19:07:07.356265+0000","last_fullsized":"2023-07-21T19:07:07.356265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:09:18.215822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359113+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:07:07.359113+0000","last_peered":"2023-07-21T19:07:07.359113+0000","last_clean":"2023-07-21T19:07:07.359113+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:07:07.359113+0000","last_undegraded":"2023-07-21T19:07:07.359113+0000","last_fullsized":"2023-07-21T19:07:07.359113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:59.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360472+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:07:07.360472+0000","last_peered":"2023-07-21T19:07:07.360472+0000","last_clean":"2023-07-21T19:07:07.360472+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:07:07.360472+0000","last_undegraded":"2023-07-21T19:07:07.360472+0000","last_fullsized":"2023-07-21T19:07:07.360472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:13.624531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360608+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:07:07.360608+0000","last_peered":"2023-07-21T19:07:07.360608+0000","last_clean":"2023-07-21T19:07:07.360608+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:07:07.360608+0000","last_undegraded":"2023-07-21T19:07:07.360608+0000","last_fullsized":"2023-07-21T19:07:07.360608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:17.986171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358850+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:07:07.358850+0000","last_peered":"2023-07-21T19:07:07.358850+0000","last_clean":"2023-07-21T19:07:07.358850+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:07:07.358850+0000","last_undegraded":"2023-07-21T19:07:07.358850+0000","last_fullsized":"2023-07-21T19:07:07.358850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:48.916207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360880+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:07:07.360880+0000","last_peered":"2023-07-21T19:07:07.360880+0000","last_clean":"2023-07-21T19:07:07.360880+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:07:07.360880+0000","last_undegraded":"2023-07-21T19:07:07.360880+0000","last_fullsized":"2023-07-21T19:07:07.360880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:08.380972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.851006+0000","last_change":"2023-07-21T19:06:08.851006+0000","last_active":"2023-07-21T19:06:08.851006+0000","last_peered":"2023-07-21T19:06:08.851006+0000","last_clean":"2023-07-21T19:06:08.851006+0000","last_became_active":"2023-07-21T19:06:08.843812+0000","last_became_peered":"2023-07-21T19:06:08.843812+0000","last_unstale":"2023-07-21T19:06:08.851006+0000","last_undegraded":"2023-07-21T19:06:08.851006+0000","last_fullsized":"2023-07-21T19:06:08.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:03.813996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359554+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:07:07.359554+0000","last_peered":"2023-07-21T19:07:07.359554+0000","last_clean":"2023-07-21T19:07:07.359554+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:07:07.359554+0000","last_undegraded":"2023-07-21T19:07:07.359554+0000","last_fullsized":"2023-07-21T19:07:07.359554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:59.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840757+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:06:08.840757+0000","last_peered":"2023-07-21T19:06:08.840757+0000","last_clean":"2023-07-21T19:06:08.840757+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:06:08.840757+0000","last_undegraded":"2023-07-21T19:06:08.840757+0000","last_fullsized":"2023-07-21T19:06:08.840757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:19.820365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363262+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:07:07.363262+0000","last_peered":"2023-07-21T19:07:07.363262+0000","last_clean":"2023-07-21T19:07:07.363262+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:07:07.363262+0000","last_undegraded":"2023-07-21T19:07:07.363262+0000","last_fullsized":"2023-07-21T19:07:07.363262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:48.371444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398380+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:06:08.398380+0000","last_peered":"2023-07-21T19:06:08.398380+0000","last_clean":"2023-07-21T19:06:08.398380+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:06:08.398380+0000","last_undegraded":"2023-07-21T19:06:08.398380+0000","last_fullsized":"2023-07-21T19:06:08.398380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:48.967070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362365+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:07:07.362365+0000","last_peered":"2023-07-21T19:07:07.362365+0000","last_clean":"2023-07-21T19:07:07.362365+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:07:07.362365+0000","last_undegraded":"2023-07-21T19:07:07.362365+0000","last_fullsized":"2023-07-21T19:07:07.362365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:03:26.702743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398317+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:06:08.398317+0000","last_peered":"2023-07-21T19:06:08.398317+0000","last_clean":"2023-07-21T19:06:08.398317+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:06:08.398317+0000","last_undegraded":"2023-07-21T19:06:08.398317+0000","last_fullsized":"2023-07-21T19:06:08.398317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:56.271039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361258+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:07:07.361258+0000","last_peered":"2023-07-21T19:07:07.361258+0000","last_clean":"2023-07-21T19:07:07.361258+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:07:07.361258+0000","last_undegraded":"2023-07-21T19:07:07.361258+0000","last_fullsized":"2023-07-21T19:07:07.361258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:12.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.515456+0000","last_change":"2023-07-21T19:06:08.410381+0000","last_active":"2023-07-21T19:07:19.515456+0000","last_peered":"2023-07-21T19:07:19.515456+0000","last_clean":"2023-07-21T19:07:19.515456+0000","last_became_active":"2023-07-21T19:06:08.405822+0000","last_became_peered":"2023-07-21T19:06:08.405822+0000","last_unstale":"2023-07-21T19:07:19.515456+0000","last_undegraded":"2023-07-21T19:07:19.515456+0000","last_fullsized":"2023-07-21T19:07:19.515456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:14:23.390484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344030+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:07:07.344030+0000","last_peered":"2023-07-21T19:07:07.344030+0000","last_clean":"2023-07-21T19:07:07.344030+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:07:07.344030+0000","last_undegraded":"2023-07-21T19:07:07.344030+0000","last_fullsized":"2023-07-21T19:07:07.344030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:21.320856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400567+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:06:08.400567+0000","last_peered":"2023-07-21T19:06:08.400567+0000","last_clean":"2023-07-21T19:06:08.400567+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:06:08.400567+0000","last_undegraded":"2023-07-21T19:06:08.400567+0000","last_fullsized":"2023-07-21T19:06:08.400567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:41.115053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360782+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:07:07.360782+0000","last_peered":"2023-07-21T19:07:07.360782+0000","last_clean":"2023-07-21T19:07:07.360782+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:07:07.360782+0000","last_undegraded":"2023-07-21T19:07:07.360782+0000","last_fullsized":"2023-07-21T19:07:07.360782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:14:01.147711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404354+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:06:08.404354+0000","last_peered":"2023-07-21T19:06:08.404354+0000","last_clean":"2023-07-21T19:06:08.404354+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:06:08.404354+0000","last_undegraded":"2023-07-21T19:06:08.404354+0000","last_fullsized":"2023-07-21T19:06:08.404354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:28.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361025+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:07:07.361025+0000","last_peered":"2023-07-21T19:07:07.361025+0000","last_clean":"2023-07-21T19:07:07.361025+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:07:07.361025+0000","last_undegraded":"2023-07-21T19:07:07.361025+0000","last_fullsized":"2023-07-21T19:07:07.361025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:34.147350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401383+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:06:08.401383+0000","last_peered":"2023-07-21T19:06:08.401383+0000","last_clean":"2023-07-21T19:06:08.401383+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:06:08.401383+0000","last_undegraded":"2023-07-21T19:06:08.401383+0000","last_fullsized":"2023-07-21T19:06:08.401383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:39:13.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360977+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:07:07.360977+0000","last_peered":"2023-07-21T19:07:07.360977+0000","last_clean":"2023-07-21T19:07:07.360977+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:07:07.360977+0000","last_undegraded":"2023-07-21T19:07:07.360977+0000","last_fullsized":"2023-07-21T19:07:07.360977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:46.382674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.513549+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:07:18.513549+0000","last_peered":"2023-07-21T19:07:18.513549+0000","last_clean":"2023-07-21T19:07:18.513549+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:07:18.513549+0000","last_undegraded":"2023-07-21T19:07:18.513549+0000","last_fullsized":"2023-07-21T19:07:18.513549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:59.692187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363144+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:07:07.363144+0000","last_peered":"2023-07-21T19:07:07.363144+0000","last_clean":"2023-07-21T19:07:07.363144+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:07:07.363144+0000","last_undegraded":"2023-07-21T19:07:07.363144+0000","last_fullsized":"2023-07-21T19:07:07.363144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:32.432702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403970+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:06:08.403970+0000","last_peered":"2023-07-21T19:06:08.403970+0000","last_clean":"2023-07-21T19:06:08.403970+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:06:08.403970+0000","last_undegraded":"2023-07-21T19:06:08.403970+0000","last_fullsized":"2023-07-21T19:06:08.403970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:09.662672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357562+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:07:07.357562+0000","last_peered":"2023-07-21T19:07:07.357562+0000","last_clean":"2023-07-21T19:07:07.357562+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:07:07.357562+0000","last_undegraded":"2023-07-21T19:07:07.357562+0000","last_fullsized":"2023-07-21T19:07:07.357562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.677074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402384+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:06:08.402384+0000","last_peered":"2023-07-21T19:06:08.402384+0000","last_clean":"2023-07-21T19:06:08.402384+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:06:08.402384+0000","last_undegraded":"2023-07-21T19:06:08.402384+0000","last_fullsized":"2023-07-21T19:06:08.402384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:16:07.189240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358434+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:07:07.358434+0000","last_peered":"2023-07-21T19:07:07.358434+0000","last_clean":"2023-07-21T19:07:07.358434+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:07:07.358434+0000","last_undegraded":"2023-07-21T19:07:07.358434+0000","last_fullsized":"2023-07-21T19:07:07.358434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:01.676198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.514756+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:07:19.514756+0000","last_peered":"2023-07-21T19:07:19.514756+0000","last_clean":"2023-07-21T19:07:19.514756+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:07:19.514756+0000","last_undegraded":"2023-07-21T19:07:19.514756+0000","last_fullsized":"2023-07-21T19:07:19.514756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.353689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58815,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":161,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.334355+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:07:07.334355+0000","last_peered":"2023-07-21T19:07:07.334355+0000","last_clean":"2023-07-21T19:07:07.334355+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:07:07.334355+0000","last_undegraded":"2023-07-21T19:07:07.334355+0000","last_fullsized":"2023-07-21T19:07:07.334355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:31.374240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405148+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:06:08.405148+0000","last_peered":"2023-07-21T19:06:08.405148+0000","last_clean":"2023-07-21T19:06:08.405148+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:06:08.405148+0000","last_undegraded":"2023-07-21T19:06:08.405148+0000","last_fullsized":"2023-07-21T19:06:08.405148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:20.602887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.351617+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:07:07.351617+0000","last_peered":"2023-07-21T19:07:07.351617+0000","last_clean":"2023-07-21T19:07:07.351617+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:07:07.351617+0000","last_undegraded":"2023-07-21T19:07:07.351617+0000","last_fullsized":"2023-07-21T19:07:07.351617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:59:21.737635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401848+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:06:08.401848+0000","last_peered":"2023-07-21T19:06:08.401848+0000","last_clean":"2023-07-21T19:06:08.401848+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:06:08.401848+0000","last_undegraded":"2023-07-21T19:06:08.401848+0000","last_fullsized":"2023-07-21T19:06:08.401848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:48.688808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361072+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:07:07.361072+0000","last_peered":"2023-07-21T19:07:07.361072+0000","last_clean":"2023-07-21T19:07:07.361072+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:07:07.361072+0000","last_undegraded":"2023-07-21T19:07:07.361072+0000","last_fullsized":"2023-07-21T19:07:07.361072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:55.454258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":123,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.467900+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:07:18.467900+0000","last_peered":"2023-07-21T19:07:18.467900+0000","last_clean":"2023-07-21T19:07:18.467900+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:07:18.467900+0000","last_undegraded":"2023-07-21T19:07:18.467900+0000","last_fullsized":"2023-07-21T19:07:18.467900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:26.036729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358107+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:07:07.358107+0000","last_peered":"2023-07-21T19:07:07.358107+0000","last_clean":"2023-07-21T19:07:07.358107+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:07:07.358107+0000","last_undegraded":"2023-07-21T19:07:07.358107+0000","last_fullsized":"2023-07-21T19:07:07.358107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:24.967595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:31:53.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361317+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:07:07.361317+0000","last_peered":"2023-07-21T19:07:07.361317+0000","last_clean":"2023-07-21T19:07:07.361317+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:07:07.361317+0000","last_undegraded":"2023-07-21T19:07:07.361317+0000","last_fullsized":"2023-07-21T19:07:07.361317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:37.822944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398316+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:06:08.398316+0000","last_peered":"2023-07-21T19:06:08.398316+0000","last_clean":"2023-07-21T19:06:08.398316+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:06:08.398316+0000","last_undegraded":"2023-07-21T19:06:08.398316+0000","last_fullsized":"2023-07-21T19:06:08.398316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:37.555620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.343477+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:07:07.343477+0000","last_peered":"2023-07-21T19:07:07.343477+0000","last_clean":"2023-07-21T19:07:07.343477+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:07:07.343477+0000","last_undegraded":"2023-07-21T19:07:07.343477+0000","last_fullsized":"2023-07-21T19:07:07.343477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:42.470455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.406908+0000","last_change":"2023-07-21T19:06:08.406908+0000","last_active":"2023-07-21T19:06:08.406908+0000","last_peered":"2023-07-21T19:06:08.406908+0000","last_clean":"2023-07-21T19:06:08.406908+0000","last_became_active":"2023-07-21T19:06:08.406531+0000","last_became_peered":"2023-07-21T19:06:08.406531+0000","last_unstale":"2023-07-21T19:06:08.406908+0000","last_undegraded":"2023-07-21T19:06:08.406908+0000","last_fullsized":"2023-07-21T19:06:08.406908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:04.808701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357349+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:07:07.357349+0000","last_peered":"2023-07-21T19:07:07.357349+0000","last_clean":"2023-07-21T19:07:07.357349+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:07:07.357349+0000","last_undegraded":"2023-07-21T19:07:07.357349+0000","last_fullsized":"2023-07-21T19:07:07.357349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:01.821241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404216+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:06:08.404216+0000","last_peered":"2023-07-21T19:06:08.404216+0000","last_clean":"2023-07-21T19:06:08.404216+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:06:08.404216+0000","last_undegraded":"2023-07-21T19:06:08.404216+0000","last_fullsized":"2023-07-21T19:06:08.404216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:18.679466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359555+0000","last_change":"2023-07-21T19:06:08.408400+0000","last_active":"2023-07-21T19:07:07.359555+0000","last_peered":"2023-07-21T19:07:07.359555+0000","last_clean":"2023-07-21T19:07:07.359555+0000","last_became_active":"2023-07-21T19:06:08.408005+0000","last_became_peered":"2023-07-21T19:06:08.408005+0000","last_unstale":"2023-07-21T19:07:07.359555+0000","last_undegraded":"2023-07-21T19:07:07.359555+0000","last_fullsized":"2023-07-21T19:07:07.359555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:20.230383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342774+0000","last_change":"2023-07-21T19:06:08.407039+0000","last_active":"2023-07-21T19:07:07.342774+0000","last_peered":"2023-07-21T19:07:07.342774+0000","last_clean":"2023-07-21T19:07:07.342774+0000","last_became_active":"2023-07-21T19:06:08.406635+0000","last_became_peered":"2023-07-21T19:06:08.406635+0000","last_unstale":"2023-07-21T19:07:07.342774+0000","last_undegraded":"2023-07-21T19:07:07.342774+0000","last_fullsized":"2023-07-21T19:07:07.342774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:18.504252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:16.461334+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:07:16.461334+0000","last_peered":"2023-07-21T19:07:16.461334+0000","last_clean":"2023-07-21T19:07:16.461334+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:07:16.461334+0000","last_undegraded":"2023-07-21T19:07:16.461334+0000","last_fullsized":"2023-07-21T19:07:16.461334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:21.302828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362955+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:07:07.362955+0000","last_peered":"2023-07-21T19:07:07.362955+0000","last_clean":"2023-07-21T19:07:07.362955+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:07:07.362955+0000","last_undegraded":"2023-07-21T19:07:07.362955+0000","last_fullsized":"2023-07-21T19:07:07.362955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:42.921526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398968+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:06:08.398968+0000","last_peered":"2023-07-21T19:06:08.398968+0000","last_clean":"2023-07-21T19:06:08.398968+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:06:08.398968+0000","last_undegraded":"2023-07-21T19:06:08.398968+0000","last_fullsized":"2023-07-21T19:06:08.398968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:50.624939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404291+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:06:08.404291+0000","last_peered":"2023-07-21T19:06:08.404291+0000","last_clean":"2023-07-21T19:06:08.404291+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:06:08.404291+0000","last_undegraded":"2023-07-21T19:06:08.404291+0000","last_fullsized":"2023-07-21T19:06:08.404291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:45.400614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.346572+0000","last_change":"2023-07-21T19:06:08.409780+0000","last_active":"2023-07-21T19:07:07.346572+0000","last_peered":"2023-07-21T19:07:07.346572+0000","last_clean":"2023-07-21T19:07:07.346572+0000","last_became_active":"2023-07-21T19:06:08.409394+0000","last_became_peered":"2023-07-21T19:06:08.409394+0000","last_unstale":"2023-07-21T19:07:07.346572+0000","last_undegraded":"2023-07-21T19:07:07.346572+0000","last_fullsized":"2023-07-21T19:07:07.346572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:36:45.599367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404518+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:06:08.404518+0000","last_peered":"2023-07-21T19:06:08.404518+0000","last_clean":"2023-07-21T19:06:08.404518+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:06:08.404518+0000","last_undegraded":"2023-07-21T19:06:08.404518+0000","last_fullsized":"2023-07-21T19:06:08.404518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:04.683020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.413663+0000","last_change":"2023-07-21T19:06:08.413252+0000","last_active":"2023-07-21T19:06:08.413663+0000","last_peered":"2023-07-21T19:06:08.413663+0000","last_clean":"2023-07-21T19:06:08.413663+0000","last_became_active":"2023-07-21T19:06:08.407914+0000","last_became_peered":"2023-07-21T19:06:08.407914+0000","last_unstale":"2023-07-21T19:06:08.413663+0000","last_undegraded":"2023-07-21T19:06:08.413663+0000","last_fullsized":"2023-07-21T19:06:08.413663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:08.813981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.347131+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:07:07.347131+0000","last_peered":"2023-07-21T19:07:07.347131+0000","last_clean":"2023-07-21T19:07:07.347131+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:07:07.347131+0000","last_undegraded":"2023-07-21T19:07:07.347131+0000","last_fullsized":"2023-07-21T19:07:07.347131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:17.603158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:10.564825+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:07:10.564825+0000","last_peered":"2023-07-21T19:07:10.564825+0000","last_clean":"2023-07-21T19:07:10.564825+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:07:10.564825+0000","last_undegraded":"2023-07-21T19:07:10.564825+0000","last_fullsized":"2023-07-21T19:07:10.564825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:46.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356114+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:07:07.356114+0000","last_peered":"2023-07-21T19:07:07.356114+0000","last_clean":"2023-07-21T19:07:07.356114+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:07:07.356114+0000","last_undegraded":"2023-07-21T19:07:07.356114+0000","last_fullsized":"2023-07-21T19:07:07.356114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:34:12.413946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399221+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:06:08.399221+0000","last_peered":"2023-07-21T19:06:08.399221+0000","last_clean":"2023-07-21T19:06:08.399221+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:06:08.399221+0000","last_undegraded":"2023-07-21T19:06:08.399221+0000","last_fullsized":"2023-07-21T19:06:08.399221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.493447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360651+0000","last_change":"2023-07-21T19:06:08.842743+0000","last_active":"2023-07-21T19:07:07.360651+0000","last_peered":"2023-07-21T19:07:07.360651+0000","last_clean":"2023-07-21T19:07:07.360651+0000","last_became_active":"2023-07-21T19:06:08.842349+0000","last_became_peered":"2023-07-21T19:06:08.842349+0000","last_unstale":"2023-07-21T19:07:07.360651+0000","last_undegraded":"2023-07-21T19:07:07.360651+0000","last_fullsized":"2023-07-21T19:07:07.360651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:05:30.960744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841889+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:06:08.841889+0000","last_peered":"2023-07-21T19:06:08.841889+0000","last_clean":"2023-07-21T19:06:08.841889+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:06:08.841889+0000","last_undegraded":"2023-07-21T19:06:08.841889+0000","last_fullsized":"2023-07-21T19:06:08.841889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:57.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.310459+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:07:07.310459+0000","last_peered":"2023-07-21T19:07:07.310459+0000","last_clean":"2023-07-21T19:07:07.310459+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:07:07.310459+0000","last_undegraded":"2023-07-21T19:07:07.310459+0000","last_fullsized":"2023-07-21T19:07:07.310459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:17.675575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840681+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:06:08.840681+0000","last_peered":"2023-07-21T19:06:08.840681+0000","last_clean":"2023-07-21T19:06:08.840681+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:06:08.840681+0000","last_undegraded":"2023-07-21T19:06:08.840681+0000","last_fullsized":"2023-07-21T19:06:08.840681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:11.395258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354043+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:07:07.354043+0000","last_peered":"2023-07-21T19:07:07.354043+0000","last_clean":"2023-07-21T19:07:07.354043+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:07:07.354043+0000","last_undegraded":"2023-07-21T19:07:07.354043+0000","last_fullsized":"2023-07-21T19:07:07.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:28:35.734063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404850+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:06:08.404850+0000","last_peered":"2023-07-21T19:06:08.404850+0000","last_clean":"2023-07-21T19:06:08.404850+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:06:08.404850+0000","last_undegraded":"2023-07-21T19:06:08.404850+0000","last_fullsized":"2023-07-21T19:06:08.404850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:53.659216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356469+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:07:07.356469+0000","last_peered":"2023-07-21T19:07:07.356469+0000","last_clean":"2023-07-21T19:07:07.356469+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:07:07.356469+0000","last_undegraded":"2023-07-21T19:07:07.356469+0000","last_fullsized":"2023-07-21T19:07:07.356469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:19:55.786796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398085+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:06:08.398085+0000","last_peered":"2023-07-21T19:06:08.398085+0000","last_clean":"2023-07-21T19:06:08.398085+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:06:08.398085+0000","last_undegraded":"2023-07-21T19:06:08.398085+0000","last_fullsized":"2023-07-21T19:06:08.398085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:02.430774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362043+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:07:07.362043+0000","last_peered":"2023-07-21T19:07:07.362043+0000","last_clean":"2023-07-21T19:07:07.362043+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:07:07.362043+0000","last_undegraded":"2023-07-21T19:07:07.362043+0000","last_fullsized":"2023-07-21T19:07:07.362043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:08:39.735825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405117+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:06:08.405117+0000","last_peered":"2023-07-21T19:06:08.405117+0000","last_clean":"2023-07-21T19:06:08.405117+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:06:08.405117+0000","last_undegraded":"2023-07-21T19:06:08.405117+0000","last_fullsized":"2023-07-21T19:06:08.405117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:44.759183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362557+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:07:07.362557+0000","last_peered":"2023-07-21T19:07:07.362557+0000","last_clean":"2023-07-21T19:07:07.362557+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:07:07.362557+0000","last_undegraded":"2023-07-21T19:07:07.362557+0000","last_fullsized":"2023-07-21T19:07:07.362557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:16.325005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402542+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:06:08.402542+0000","last_peered":"2023-07-21T19:06:08.402542+0000","last_clean":"2023-07-21T19:06:08.402542+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:06:08.402542+0000","last_undegraded":"2023-07-21T19:06:08.402542+0000","last_fullsized":"2023-07-21T19:06:08.402542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:41.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.355564+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:07:07.355564+0000","last_peered":"2023-07-21T19:07:07.355564+0000","last_clean":"2023-07-21T19:07:07.355564+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:07:07.355564+0000","last_undegraded":"2023-07-21T19:07:07.355564+0000","last_fullsized":"2023-07-21T19:07:07.355564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:08:59.431298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401998+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:06:08.401998+0000","last_peered":"2023-07-21T19:06:08.401998+0000","last_clean":"2023-07-21T19:06:08.401998+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:06:08.401998+0000","last_undegraded":"2023-07-21T19:06:08.401998+0000","last_fullsized":"2023-07-21T19:06:08.401998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:17.091156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354174+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:07:07.354174+0000","last_peered":"2023-07-21T19:07:07.354174+0000","last_clean":"2023-07-21T19:07:07.354174+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:07:07.354174+0000","last_undegraded":"2023-07-21T19:07:07.354174+0000","last_fullsized":"2023-07-21T19:07:07.354174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:09.484987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345151+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:07:07.345151+0000","last_peered":"2023-07-21T19:07:07.345151+0000","last_clean":"2023-07-21T19:07:07.345151+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:07:07.345151+0000","last_undegraded":"2023-07-21T19:07:07.345151+0000","last_fullsized":"2023-07-21T19:07:07.345151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:56.095641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352815+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:07:07.352815+0000","last_peered":"2023-07-21T19:07:07.352815+0000","last_clean":"2023-07-21T19:07:07.352815+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:07:07.352815+0000","last_undegraded":"2023-07-21T19:07:07.352815+0000","last_fullsized":"2023-07-21T19:07:07.352815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:24.580856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361130+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:07:07.361130+0000","last_peered":"2023-07-21T19:07:07.361130+0000","last_clean":"2023-07-21T19:07:07.361130+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:07:07.361130+0000","last_undegraded":"2023-07-21T19:07:07.361130+0000","last_fullsized":"2023-07-21T19:07:07.361130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:17.790433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359810+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:07:07.359810+0000","last_peered":"2023-07-21T19:07:07.359810+0000","last_clean":"2023-07-21T19:07:07.359810+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:07:07.359810+0000","last_undegraded":"2023-07-21T19:07:07.359810+0000","last_fullsized":"2023-07-21T19:07:07.359810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:29.261412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354985+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:07:07.354985+0000","last_peered":"2023-07-21T19:07:07.354985+0000","last_clean":"2023-07-21T19:07:07.354985+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:07:07.354985+0000","last_undegraded":"2023-07-21T19:07:07.354985+0000","last_fullsized":"2023-07-21T19:07:07.354985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:47:56.761424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342375+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:07:07.342375+0000","last_peered":"2023-07-21T19:07:07.342375+0000","last_clean":"2023-07-21T19:07:07.342375+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:07:07.342375+0000","last_undegraded":"2023-07-21T19:07:07.342375+0000","last_fullsized":"2023-07-21T19:07:07.342375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:45:42.804371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359579+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:07:07.359579+0000","last_peered":"2023-07-21T19:07:07.359579+0000","last_clean":"2023-07-21T19:07:07.359579+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:07:07.359579+0000","last_undegraded":"2023-07-21T19:07:07.359579+0000","last_fullsized":"2023-07-21T19:07:07.359579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:42.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360257+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:07:07.360257+0000","last_peered":"2023-07-21T19:07:07.360257+0000","last_clean":"2023-07-21T19:07:07.360257+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:07:07.360257+0000","last_undegraded":"2023-07-21T19:07:07.360257+0000","last_fullsized":"2023-07-21T19:07:07.360257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:40:21.846553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360450+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:07:07.360450+0000","last_peered":"2023-07-21T19:07:07.360450+0000","last_clean":"2023-07-21T19:07:07.360450+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:07:07.360450+0000","last_undegraded":"2023-07-21T19:07:07.360450+0000","last_fullsized":"2023-07-21T19:07:07.360450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:19.869711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359999+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:07:07.359999+0000","last_peered":"2023-07-21T19:07:07.359999+0000","last_clean":"2023-07-21T19:07:07.359999+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:07:07.359999+0000","last_undegraded":"2023-07-21T19:07:07.359999+0000","last_fullsized":"2023-07-21T19:07:07.359999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:33.674612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362482+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:07:07.362482+0000","last_peered":"2023-07-21T19:07:07.362482+0000","last_clean":"2023-07-21T19:07:07.362482+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:07:07.362482+0000","last_undegraded":"2023-07-21T19:07:07.362482+0000","last_fullsized":"2023-07-21T19:07:07.362482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:23:36.091795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352462+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:07:07.352462+0000","last_peered":"2023-07-21T19:07:07.352462+0000","last_clean":"2023-07-21T19:07:07.352462+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:07:07.352462+0000","last_undegraded":"2023-07-21T19:07:07.352462+0000","last_fullsized":"2023-07-21T19:07:07.352462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:45:18.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345299+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:07:07.345299+0000","last_peered":"2023-07-21T19:07:07.345299+0000","last_clean":"2023-07-21T19:07:07.345299+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:07:07.345299+0000","last_undegraded":"2023-07-21T19:07:07.345299+0000","last_fullsized":"2023-07-21T19:07:07.345299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:02.887348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":376,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1568,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":456247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64900000000000002,"15min":0.65300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":1.131,"15min":1.131},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.75},"min":{"1min":0.56899999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.044,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":1.083,"15min":1.083},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.006,"5min":1.395,"15min":1.395},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89800000000000002,"5min":1.208,"15min":1.208},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.073,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":1.24,"15min":1.24},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93000000000000005,"15min":0.93600000000000005},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.095,"5min":1.595,"15min":1.595},"last":1.026},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"min":{"1min":0.59099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0840000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.877}]},{"osd":8,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.98799999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.97299999999999998,"15min":0.97099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.101},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.0309999999999999,"15min":0.98299999999999998},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.2670000000000003,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.085}]},{"osd":10,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.95399999999999996},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.615,"15min":1.615},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.75900000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.099,"5min":1.681,"15min":1.681},"last":1.1839999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1106962,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.4770000000000003,"5min":6.4770000000000003,"15min":6.4770000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.441},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86099999999999999,"5min":2.323,"15min":2.323},"last":0.65400000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.876,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":3.399,"15min":3.399},"last":0.5},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.626}]},{"osd":4,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0049999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.84399999999999997},"min":{"1min":0.67800000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.0669999999999999,"5min":1.238,"15min":1.238},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.878,"15min":0.872},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":3.331,"15min":3.331},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.96599999999999997,"15min":0.93500000000000005},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98899999999999999,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93600000000000005,"15min":0.93400000000000005},"min":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.171,"5min":2.661,"15min":2.661},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.93500000000000005,"15min":0.97699999999999998},"min":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0900000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.87}]},{"osd":9,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.032,"15min":1.006},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0780000000000001,"5min":6.4249999999999998,"15min":6.4249999999999998},"last":0.755},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":1.016},"min":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0449999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.94299999999999995,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0840000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.878,"15min":0.876},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.93799999999999994}]}]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":456461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.63900000000000001,"15min":0.59199999999999997},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.59799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":9.2650000000000006,"5min":9.2650000000000006,"15min":9.2650000000000006},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.433,"15min":0.433},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.84499999999999997,"15min":0.83299999999999996},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.4019999999999992,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.032}]},{"osd":3,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":9.3130000000000006,"5min":9.3130000000000006,"15min":9.3130000000000006},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6220000000000001,"5min":1.637,"15min":1.637},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.371,"5min":0.98799999999999999,"15min":0.94999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":9.4749999999999996,"5min":9.4749999999999996,"15min":9.4749999999999996},"last":1.002}]},{"osd":6,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1040000000000001,"5min":1.345,"15min":1.345},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.0189999999999999,"15min":1.0329999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":9.3740000000000006,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.84199999999999997}]},{"osd":8,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.651,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.018}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.0389999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":9.3439999999999994,"5min":9.3439999999999994,"15min":9.3439999999999994},"last":0.747},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.014,"15min":1.04},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94999999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":455777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.66500000000000004},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.036}]},{"osd":2,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.874,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.85699999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.012},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.169,"15min":1.169},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0669999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.056}]},{"osd":6,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.88,"15min":0.874},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.1310000000000002,"5min":4.1310000000000002,"15min":4.1310000000000002},"last":1.137}]},{"osd":7,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.157,"5min":1.403,"15min":1.403},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90200000000000002,"15min":0.89600000000000002},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1870000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.91700000000000004,"15min":0.86899999999999999},"min":{"1min":0.64700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.3230000000000004,"5min":4.3230000000000004,"15min":4.3230000000000004},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.86599999999999999,"15min":0.82599999999999996},"min":{"1min":0.73199999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.238,"15min":1.238},"last":1.1559999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.96999999999999997,"15min":0.94099999999999995},"min":{"1min":0.69299999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.329},{"interface":"front","average":{"1min":1.083,"5min":0.93600000000000005,"15min":0.89200000000000002},"min":{"1min":0.90200000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2609999999999999}]},{"osd":11,"last update":"Fri Jul 21 19:06:36 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.288},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.204}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":1049327,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91800000000000004,"5min":1.05,"15min":1.05},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72899999999999998,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.96499999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.94699999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":1.016,"15min":1.016},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.84399999999999997,"5min":0.996,"15min":0.996},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.877,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98199999999999998,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79800000000000004,"15min":0.77300000000000002},"min":{"1min":0.67900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0069999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.81799999999999995,"15min":0.85199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.234,"15min":1.234},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81699999999999995,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.208,"15min":1.208},"last":0.60399999999999998}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":0.89700000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.079,"5min":1.095,"15min":1.095},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.93799999999999994},"min":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0660000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.93700000000000006}]},{"osd":11,"last update":"Fri Jul 21 19:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91300000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":455856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.51400000000000001,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.68899999999999995,"15min":0.65400000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.505},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.52300000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.68600000000000005,"15min":0.61299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6099999999999994,"5min":9.6099999999999994,"15min":9.6099999999999994},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81899999999999995,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.867,"5min":6.867,"15min":6.867},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.163,"5min":0.85999999999999999,"15min":0.81899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.9029999999999996,"5min":6.9029999999999996,"15min":6.9029999999999996},"last":0.75}]},{"osd":5,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.90800000000000003,"15min":0.84199999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Jul 21 19:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86199999999999999,"15min":0.82799999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2589999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.095,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":0.73099999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.88,"15min":0.86399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.26,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.93000000000000005,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.609,"5min":3.609,"15min":3.609},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.94599999999999995,"15min":0.95099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.91000000000000003,"15min":0.94199999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93400000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1539999999999999,"5min":3.9969999999999999,"15min":3.9969999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0469999999999999,"15min":1.048},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":519360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.98099999999999998,"5min":1.073,"15min":1.073},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.93999999999999995},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.64200000000000002},"min":{"1min":0.56499999999999995,"5min":0.38600000000000001,"15min":0.36599999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.014}]},{"osd":3,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.47599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.45600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.52000000000000002},"min":{"1min":0.48199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.72599999999999998,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.78400000000000003,"15min":0.70999999999999996},"min":{"1min":0.68300000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.079}]},{"osd":6,"last update":"Fri Jul 21 19:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.67100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.629,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]},{"osd":7,"last update":"Fri Jul 21 19:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76200000000000001,"15min":0.72999999999999998},"min":{"1min":0.57799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73499999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.91600000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.61499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.0269999999999999,"5min":1.502,"15min":1.502},"last":0.86599999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.91400000000000003,"15min":0.89700000000000002},"min":{"1min":0.71699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.113},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.94399999999999995,"15min":0.91800000000000004},"min":{"1min":0.76300000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.0980000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95599999999999996,"15min":0.93999999999999995},"min":{"1min":0.78500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.169},{"interface":"front","average":{"1min":1.024,"5min":0.96999999999999997,"15min":0.96499999999999997},"min":{"1min":0.76400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.129}]},{"osd":11,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.052},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.1419999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":457121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68300000000000005,"15min":0.63500000000000001},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1539999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.39300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.53200000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.495}]},{"osd":3,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.77300000000000002,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.91500000000000004},"min":{"1min":0.68799999999999994,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.244,"5min":1.304,"15min":1.304},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82399999999999995,"15min":0.84599999999999997},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89000000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.93200000000000005},"min":{"1min":0.55600000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84899999999999998,"15min":0.84999999999999998},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.034,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":5.9950000000000001,"5min":5.9950000000000001,"15min":5.9950000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.68799999999999994,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":515186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95599999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68100000000000005,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":1.081,"15min":1.081},"last":0.754}]},{"osd":3,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.53700000000000003},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94799999999999995,"5min":1.026,"15min":1.026},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90200000000000002,"5min":1.167,"15min":1.167},"last":0.625}]},{"osd":6,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.218,"5min":1.333,"15min":1.333},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.88700000000000001,"15min":0.877},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.82099999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":456173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46800000000000003,"15min":0.42199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70999999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.34200000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.52000000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.873,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.86599999999999999,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.74299999999999999,"15min":0.65300000000000002},"min":{"1min":0.56200000000000006,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71699999999999997,"15min":0.64500000000000002},"min":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.42099999999999999},"max":{"1min":0.98299999999999998,"5min":1.048,"15min":1.048},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1020000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.75700000000000001,"15min":0.69899999999999995},"min":{"1min":0.53300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92500000000000004,"5min":1.264,"15min":1.264},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0649999999999999,"5min":1.141,"15min":1.141},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.61099999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2070000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.876}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89900000000000002,"15min":0.93799999999999994},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2030000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.84699999999999998,"15min":0.89700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.151,"5min":1.196,"15min":1.196},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.091,"5min":1.153,"15min":1.153},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.89800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":458310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.443},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.44600000000000001},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.167,"15min":1.167},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69999999999999996,"15min":0.68000000000000005},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.76900000000000002,"15min":0.66900000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97199999999999998,"5min":1.079,"15min":1.079},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97799999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.82999999999999996,"15min":0.79700000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71399999999999997,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.754,"15min":0.69999999999999996},"min":{"1min":0.51300000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.877},"min":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.173,"5min":1.298,"15min":1.298},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93200000000000005,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":2.2120000000000002,"15min":2.7999999999999998},"min":{"1min":0.69699999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.145,"5min":42.396000000000001,"15min":42.396000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":2.105,"15min":2.7160000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.70799999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1048576,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63600000000000001,"15min":0.61399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90200000000000002,"5min":1.042,"15min":1.042},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71799999999999997,"15min":0.68999999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.40799999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.57499999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.61499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92900000000000005,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.135,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.71399999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1499999999999999,"5min":1.163,"15min":1.163},"last":1.006}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.505,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1080000000000001,"5min":1.111,"15min":1.111},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.091,"5min":1.103,"15min":1.103},"last":0.91400000000000003}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.048},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80400000000000005}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.161,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90100000000000002,"15min":0.875},"min":{"1min":0.74199999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1819999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.095}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:07:22.592 INFO:tasks.fwd_scrub.ceph_manager:clean! 2023-07-21T19:07:22.592 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-07-21T19:07:22.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:22 smithi101 ceph-mon[128136]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2023-07-21T19:07:22.593 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:22 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1854892342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:22 smithi156 ceph-mon[128316]: pgmap v354: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 938 B/s wr, 8 op/s 2023-07-21T19:07:22.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:22 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1854892342' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 -- 172.21.15.5:0/2263914722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c0ffb00 msgr2=0x7f223c0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 --2- 172.21.15.5:0/2263914722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c0ffb00 0x7f223c0fff10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f222c009990 tx=0x7f222c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 -- 172.21.15.5:0/2263914722 shutdown_connections 2023-07-21T19:07:22.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 --2- 172.21.15.5:0/2263914722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c101680 0x7f223c107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 --2- 172.21.15.5:0/2263914722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c0ffb00 0x7f223c0fff10 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 --2- 172.21.15.5:0/2263914722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 -- 172.21.15.5:0/2263914722 >> 172.21.15.5:0/2263914722 conn(0x7f223c0fb0e0 msgr2=0x7f223c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 -- 172.21.15.5:0/2263914722 shutdown_connections 2023-07-21T19:07:22.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.818+0000 7f22415ff700 1 -- 172.21.15.5:0/2263914722 wait complete. 2023-07-21T19:07:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.819+0000 7f22415ff700 1 Processor -- start 2023-07-21T19:07:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 -- start start 2023-07-21T19:07:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c0ffb00 0x7f223c195710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c195c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19d290 con 0x7f223c101680 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19d3d0 con 0x7f223c100cf0 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f22415ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f223c19d560 con 0x7f223c0ffb00 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f223b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f223b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47988/0 (socket says 172.21.15.5:47988) 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.820+0000 7f223b7fe700 1 -- 172.21.15.5:0/3017766750 learned_addr learned my addr 172.21.15.5:0/3017766750 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:22.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223affd700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c0ffb00 0x7f223c195710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223a7fc700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c195c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 -- 172.21.15.5:0/3017766750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c0ffb00 msgr2=0x7f223c195710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c0ffb00 0x7f223c195710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 -- 172.21.15.5:0/3017766750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 msgr2=0x7f223c195c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c195c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 -- 172.21.15.5:0/3017766750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f222c0095f0 con 0x7f223c101680 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223a7fc700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:22.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f223b7fe700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2234007ee0 tx=0x7f223400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234017070 con 0x7f223c101680 2023-07-21T19:07:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f223400f040 con 0x7f223c101680 2023-07-21T19:07:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f223c19d840 con 0x7f223c101680 2023-07-21T19:07:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.821+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2234005710 con 0x7f223c101680 2023-07-21T19:07:22.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.822+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f223c19dd50 con 0x7f223c101680 2023-07-21T19:07:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.823+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f223c106a80 con 0x7f223c101680 2023-07-21T19:07:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.825+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2234005870 con 0x7f223c101680 2023-07-21T19:07:22.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.826+0000 7f2223fff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 0x7f22240a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.826+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f22340cc100 con 0x7f223c101680 2023-07-21T19:07:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.826+0000 7f223affd700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 0x7f22240a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:22.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.827+0000 7f223affd700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 0x7f22240a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b5c0 tx=0x7f222c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:22.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:22.830+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2234095800 con 0x7f223c101680 2023-07-21T19:07:23.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.034+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f223c060950 con 0x7f22240a1220 2023-07-21T19:07:23.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.045+0000 7f2223fff700 1 -- 172.21.15.5:0/3017766750 <== mgr.14152 v2:172.21.15.5:6800/1627438252 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+343494 (secure 0 0 0) 0x7f223c060950 con 0x7f22240a1220 2023-07-21T19:07:23.063 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:23.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 msgr2=0x7f22240a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 0x7f22240a36d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f222c00b5c0 tx=0x7f222c005960 comp rx=0 tx=0).stop 2023-07-21T19:07:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 msgr2=0x7f223c19ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2234007ee0 tx=0x7f223400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 shutdown_connections 2023-07-21T19:07:23.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f22240a1220 0x7f22240a36d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f223c101680 0x7f223c19ccc0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f223c0ffb00 0x7f223c195710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 --2- 172.21.15.5:0/3017766750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f223c100cf0 0x7f223c195c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.049+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 >> 172.21.15.5:0/3017766750 conn(0x7f223c0fb0e0 msgr2=0x7f223c0fd3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:23.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.050+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 shutdown_connections 2023-07-21T19:07:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.050+0000 7f22415ff700 1 -- 172.21.15.5:0/3017766750 wait complete. 2023-07-21T19:07:23.071 INFO:teuthology.orchestra.run.smithi005.stderr:dumped all 2023-07-21T19:07:23.077 INFO:teuthology.orchestra.run.smithi005.stdout:{"pg_ready":true,"pg_map":{"version":354,"stamp":"2023-07-21T19:07:21.341914+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":654440,"num_objects":64,"num_object_clones":0,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":748,"num_read_kb":495,"num_write":352,"num_write_kb":3057,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":11,"num_bytes_recovered":1182304,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":785,"ondisk_log_size":785,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":334996,"kb_used_data":11572,"kb_used_omap":0,"kb_used_meta":323328,"kb_avail":1124606828,"statfs":{"total":1151940427776,"available":1151597391872,"internally_reserved":0,"allocated":11849728,"data_stored":7435356,"data_compressed":108678,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":331087872},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1839,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":90,"num_read_kb":45,"num_write":15,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002992"},"pg_stats":[{"pgid":"3.3f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360521+0000","last_change":"2023-07-21T19:05:37.296226+0000","last_active":"2023-07-21T19:07:07.360521+0000","last_peered":"2023-07-21T19:07:07.360521+0000","last_clean":"2023-07-21T19:07:07.360521+0000","last_became_active":"2023-07-21T19:05:37.295418+0000","last_became_peered":"2023-07-21T19:05:37.295418+0000","last_unstale":"2023-07-21T19:07:07.360521+0000","last_undegraded":"2023-07-21T19:07:07.360521+0000","last_fullsized":"2023-07-21T19:07:07.360521+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:11:46.910120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354073+0000","last_change":"2023-07-21T19:05:37.295083+0000","last_active":"2023-07-21T19:07:07.354073+0000","last_peered":"2023-07-21T19:07:07.354073+0000","last_clean":"2023-07-21T19:07:07.354073+0000","last_became_active":"2023-07-21T19:05:37.294758+0000","last_became_peered":"2023-07-21T19:05:37.294758+0000","last_unstale":"2023-07-21T19:07:07.354073+0000","last_undegraded":"2023-07-21T19:07:07.354073+0000","last_fullsized":"2023-07-21T19:07:07.354073+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:24:32.384094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358692+0000","last_change":"2023-07-21T19:05:37.294376+0000","last_active":"2023-07-21T19:07:07.358692+0000","last_peered":"2023-07-21T19:07:07.358692+0000","last_clean":"2023-07-21T19:07:07.358692+0000","last_became_active":"2023-07-21T19:05:37.294074+0000","last_became_peered":"2023-07-21T19:05:37.294074+0000","last_unstale":"2023-07-21T19:07:07.358692+0000","last_undegraded":"2023-07-21T19:07:07.358692+0000","last_fullsized":"2023-07-21T19:07:07.358692+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:44:43.019102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353325+0000","last_change":"2023-07-21T19:05:37.296362+0000","last_active":"2023-07-21T19:07:07.353325+0000","last_peered":"2023-07-21T19:07:07.353325+0000","last_clean":"2023-07-21T19:07:07.353325+0000","last_became_active":"2023-07-21T19:05:37.295969+0000","last_became_peered":"2023-07-21T19:05:37.295969+0000","last_unstale":"2023-07-21T19:07:07.353325+0000","last_undegraded":"2023-07-21T19:07:07.353325+0000","last_fullsized":"2023-07-21T19:07:07.353325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:58:20.204377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354815+0000","last_change":"2023-07-21T19:05:37.289347+0000","last_active":"2023-07-21T19:07:07.354815+0000","last_peered":"2023-07-21T19:07:07.354815+0000","last_clean":"2023-07-21T19:07:07.354815+0000","last_became_active":"2023-07-21T19:05:37.288956+0000","last_became_peered":"2023-07-21T19:05:37.288956+0000","last_unstale":"2023-07-21T19:07:07.354815+0000","last_undegraded":"2023-07-21T19:07:07.354815+0000","last_fullsized":"2023-07-21T19:07:07.354815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:04.096288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354849+0000","last_change":"2023-07-21T19:05:37.300915+0000","last_active":"2023-07-21T19:07:07.354849+0000","last_peered":"2023-07-21T19:07:07.354849+0000","last_clean":"2023-07-21T19:07:07.354849+0000","last_became_active":"2023-07-21T19:05:37.300352+0000","last_became_peered":"2023-07-21T19:05:37.300352+0000","last_unstale":"2023-07-21T19:07:07.354849+0000","last_undegraded":"2023-07-21T19:07:07.354849+0000","last_fullsized":"2023-07-21T19:07:07.354849+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:26:17.572777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354524+0000","last_change":"2023-07-21T19:05:37.294041+0000","last_active":"2023-07-21T19:07:07.354524+0000","last_peered":"2023-07-21T19:07:07.354524+0000","last_clean":"2023-07-21T19:07:07.354524+0000","last_became_active":"2023-07-21T19:05:37.293758+0000","last_became_peered":"2023-07-21T19:05:37.293758+0000","last_unstale":"2023-07-21T19:07:07.354524+0000","last_undegraded":"2023-07-21T19:07:07.354524+0000","last_fullsized":"2023-07-21T19:07:07.354524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:50:34.608525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357913+0000","last_change":"2023-07-21T19:05:37.290881+0000","last_active":"2023-07-21T19:07:07.357913+0000","last_peered":"2023-07-21T19:07:07.357913+0000","last_clean":"2023-07-21T19:07:07.357913+0000","last_became_active":"2023-07-21T19:05:37.289449+0000","last_became_peered":"2023-07-21T19:05:37.289449+0000","last_unstale":"2023-07-21T19:07:07.357913+0000","last_undegraded":"2023-07-21T19:07:07.357913+0000","last_fullsized":"2023-07-21T19:07:07.357913+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:21:29.555623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363733+0000","last_change":"2023-07-21T19:05:37.291122+0000","last_active":"2023-07-21T19:07:07.363733+0000","last_peered":"2023-07-21T19:07:07.363733+0000","last_clean":"2023-07-21T19:07:07.363733+0000","last_became_active":"2023-07-21T19:05:37.290802+0000","last_became_peered":"2023-07-21T19:05:37.290802+0000","last_unstale":"2023-07-21T19:07:07.363733+0000","last_undegraded":"2023-07-21T19:07:07.363733+0000","last_fullsized":"2023-07-21T19:07:07.363733+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:08:37.146951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"87'16","reported_seq":32,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.353019+0000","last_change":"2023-07-21T19:05:37.301330+0000","last_active":"2023-07-21T19:07:07.353019+0000","last_peered":"2023-07-21T19:07:07.353019+0000","last_clean":"2023-07-21T19:07:07.353019+0000","last_became_active":"2023-07-21T19:05:37.300728+0000","last_became_peered":"2023-07-21T19:05:37.300728+0000","last_unstale":"2023-07-21T19:07:07.353019+0000","last_undegraded":"2023-07-21T19:07:07.353019+0000","last_fullsized":"2023-07-21T19:07:07.353019+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:57:32.152857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362661+0000","last_change":"2023-07-21T19:05:37.297839+0000","last_active":"2023-07-21T19:07:07.362661+0000","last_peered":"2023-07-21T19:07:07.362661+0000","last_clean":"2023-07-21T19:07:07.362661+0000","last_became_active":"2023-07-21T19:05:37.297242+0000","last_became_peered":"2023-07-21T19:05:37.297242+0000","last_unstale":"2023-07-21T19:07:07.362661+0000","last_undegraded":"2023-07-21T19:07:07.362661+0000","last_fullsized":"2023-07-21T19:07:07.362661+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:23.961311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345988+0000","last_change":"2023-07-21T19:05:37.304391+0000","last_active":"2023-07-21T19:07:07.345988+0000","last_peered":"2023-07-21T19:07:07.345988+0000","last_clean":"2023-07-21T19:07:07.345988+0000","last_became_active":"2023-07-21T19:05:37.304008+0000","last_became_peered":"2023-07-21T19:05:37.304008+0000","last_unstale":"2023-07-21T19:07:07.345988+0000","last_undegraded":"2023-07-21T19:07:07.345988+0000","last_fullsized":"2023-07-21T19:07:07.345988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:50:15.583393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344330+0000","last_change":"2023-07-21T19:05:37.291663+0000","last_active":"2023-07-21T19:07:07.344330+0000","last_peered":"2023-07-21T19:07:07.344330+0000","last_clean":"2023-07-21T19:07:07.344330+0000","last_became_active":"2023-07-21T19:05:37.290896+0000","last_became_peered":"2023-07-21T19:05:37.290896+0000","last_unstale":"2023-07-21T19:07:07.344330+0000","last_undegraded":"2023-07-21T19:07:07.344330+0000","last_fullsized":"2023-07-21T19:07:07.344330+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:59:42.309276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356265+0000","last_change":"2023-07-21T19:05:37.292213+0000","last_active":"2023-07-21T19:07:07.356265+0000","last_peered":"2023-07-21T19:07:07.356265+0000","last_clean":"2023-07-21T19:07:07.356265+0000","last_became_active":"2023-07-21T19:05:37.291845+0000","last_became_peered":"2023-07-21T19:05:37.291845+0000","last_unstale":"2023-07-21T19:07:07.356265+0000","last_undegraded":"2023-07-21T19:07:07.356265+0000","last_fullsized":"2023-07-21T19:07:07.356265+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:09:18.215822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359113+0000","last_change":"2023-07-21T19:05:37.298917+0000","last_active":"2023-07-21T19:07:07.359113+0000","last_peered":"2023-07-21T19:07:07.359113+0000","last_clean":"2023-07-21T19:07:07.359113+0000","last_became_active":"2023-07-21T19:05:37.298565+0000","last_became_peered":"2023-07-21T19:05:37.298565+0000","last_unstale":"2023-07-21T19:07:07.359113+0000","last_undegraded":"2023-07-21T19:07:07.359113+0000","last_fullsized":"2023-07-21T19:07:07.359113+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:59.686045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360472+0000","last_change":"2023-07-21T19:05:37.289848+0000","last_active":"2023-07-21T19:07:07.360472+0000","last_peered":"2023-07-21T19:07:07.360472+0000","last_clean":"2023-07-21T19:07:07.360472+0000","last_became_active":"2023-07-21T19:05:37.289481+0000","last_became_peered":"2023-07-21T19:05:37.289481+0000","last_unstale":"2023-07-21T19:07:07.360472+0000","last_undegraded":"2023-07-21T19:07:07.360472+0000","last_fullsized":"2023-07-21T19:07:07.360472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:21:13.624531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360608+0000","last_change":"2023-07-21T19:05:37.296491+0000","last_active":"2023-07-21T19:07:07.360608+0000","last_peered":"2023-07-21T19:07:07.360608+0000","last_clean":"2023-07-21T19:07:07.360608+0000","last_became_active":"2023-07-21T19:05:37.295866+0000","last_became_peered":"2023-07-21T19:05:37.295866+0000","last_unstale":"2023-07-21T19:07:07.360608+0000","last_undegraded":"2023-07-21T19:07:07.360608+0000","last_fullsized":"2023-07-21T19:07:07.360608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:33:17.986171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358850+0000","last_change":"2023-07-21T19:05:37.296379+0000","last_active":"2023-07-21T19:07:07.358850+0000","last_peered":"2023-07-21T19:07:07.358850+0000","last_clean":"2023-07-21T19:07:07.358850+0000","last_became_active":"2023-07-21T19:05:37.296036+0000","last_became_peered":"2023-07-21T19:05:37.296036+0000","last_unstale":"2023-07-21T19:07:07.358850+0000","last_undegraded":"2023-07-21T19:07:07.358850+0000","last_fullsized":"2023-07-21T19:07:07.358850+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:17:48.916207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360880+0000","last_change":"2023-07-21T19:05:37.292752+0000","last_active":"2023-07-21T19:07:07.360880+0000","last_peered":"2023-07-21T19:07:07.360880+0000","last_clean":"2023-07-21T19:07:07.360880+0000","last_became_active":"2023-07-21T19:05:37.292451+0000","last_became_peered":"2023-07-21T19:05:37.292451+0000","last_unstale":"2023-07-21T19:07:07.360880+0000","last_undegraded":"2023-07-21T19:07:07.360880+0000","last_fullsized":"2023-07-21T19:07:07.360880+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:08.380972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"85'9","reported_seq":44,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.851006+0000","last_change":"2023-07-21T19:06:08.851006+0000","last_active":"2023-07-21T19:06:08.851006+0000","last_peered":"2023-07-21T19:06:08.851006+0000","last_clean":"2023-07-21T19:06:08.851006+0000","last_became_active":"2023-07-21T19:06:08.843812+0000","last_became_peered":"2023-07-21T19:06:08.843812+0000","last_unstale":"2023-07-21T19:06:08.851006+0000","last_undegraded":"2023-07-21T19:06:08.851006+0000","last_fullsized":"2023-07-21T19:06:08.851006+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:23:03.813996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":582,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,1],"acting":[9,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359554+0000","last_change":"2023-07-21T19:05:37.300344+0000","last_active":"2023-07-21T19:07:07.359554+0000","last_peered":"2023-07-21T19:07:07.359554+0000","last_clean":"2023-07-21T19:07:07.359554+0000","last_became_active":"2023-07-21T19:05:37.295539+0000","last_became_peered":"2023-07-21T19:05:37.295539+0000","last_unstale":"2023-07-21T19:07:07.359554+0000","last_undegraded":"2023-07-21T19:07:07.359554+0000","last_fullsized":"2023-07-21T19:07:07.359554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:32:59.560470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1e","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840757+0000","last_change":"2023-07-21T19:05:36.703787+0000","last_active":"2023-07-21T19:06:08.840757+0000","last_peered":"2023-07-21T19:06:08.840757+0000","last_clean":"2023-07-21T19:06:08.840757+0000","last_became_active":"2023-07-21T19:05:36.703426+0000","last_became_peered":"2023-07-21T19:05:36.703426+0000","last_unstale":"2023-07-21T19:06:08.840757+0000","last_undegraded":"2023-07-21T19:06:08.840757+0000","last_fullsized":"2023-07-21T19:06:08.840757+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:29:19.820365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363262+0000","last_change":"2023-07-21T19:05:37.294607+0000","last_active":"2023-07-21T19:07:07.363262+0000","last_peered":"2023-07-21T19:07:07.363262+0000","last_clean":"2023-07-21T19:07:07.363262+0000","last_became_active":"2023-07-21T19:05:37.294278+0000","last_became_peered":"2023-07-21T19:05:37.294278+0000","last_unstale":"2023-07-21T19:07:07.363262+0000","last_undegraded":"2023-07-21T19:07:07.363262+0000","last_fullsized":"2023-07-21T19:07:07.363262+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:30:48.371444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1d","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398380+0000","last_change":"2023-07-21T19:05:36.704169+0000","last_active":"2023-07-21T19:06:08.398380+0000","last_peered":"2023-07-21T19:06:08.398380+0000","last_clean":"2023-07-21T19:06:08.398380+0000","last_became_active":"2023-07-21T19:05:36.703176+0000","last_became_peered":"2023-07-21T19:05:36.703176+0000","last_unstale":"2023-07-21T19:06:08.398380+0000","last_undegraded":"2023-07-21T19:06:08.398380+0000","last_fullsized":"2023-07-21T19:06:08.398380+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:43:48.967070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362365+0000","last_change":"2023-07-21T19:05:37.293975+0000","last_active":"2023-07-21T19:07:07.362365+0000","last_peered":"2023-07-21T19:07:07.362365+0000","last_clean":"2023-07-21T19:07:07.362365+0000","last_became_active":"2023-07-21T19:05:37.293699+0000","last_became_peered":"2023-07-21T19:05:37.293699+0000","last_unstale":"2023-07-21T19:07:07.362365+0000","last_undegraded":"2023-07-21T19:07:07.362365+0000","last_fullsized":"2023-07-21T19:07:07.362365+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:03:26.702743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398317+0000","last_change":"2023-07-21T19:05:36.699350+0000","last_active":"2023-07-21T19:06:08.398317+0000","last_peered":"2023-07-21T19:06:08.398317+0000","last_clean":"2023-07-21T19:06:08.398317+0000","last_became_active":"2023-07-21T19:05:36.699007+0000","last_became_peered":"2023-07-21T19:05:36.699007+0000","last_unstale":"2023-07-21T19:06:08.398317+0000","last_undegraded":"2023-07-21T19:06:08.398317+0000","last_fullsized":"2023-07-21T19:06:08.398317+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:40:56.271039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361258+0000","last_change":"2023-07-21T19:05:37.298868+0000","last_active":"2023-07-21T19:07:07.361258+0000","last_peered":"2023-07-21T19:07:07.361258+0000","last_clean":"2023-07-21T19:07:07.361258+0000","last_became_active":"2023-07-21T19:05:37.298524+0000","last_became_peered":"2023-07-21T19:05:37.298524+0000","last_unstale":"2023-07-21T19:07:07.361258+0000","last_undegraded":"2023-07-21T19:07:07.361258+0000","last_fullsized":"2023-07-21T19:07:07.361258+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:15:12.685718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1b","version":"87'8","reported_seq":142,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.515456+0000","last_change":"2023-07-21T19:06:08.410381+0000","last_active":"2023-07-21T19:07:19.515456+0000","last_peered":"2023-07-21T19:07:19.515456+0000","last_clean":"2023-07-21T19:07:19.515456+0000","last_became_active":"2023-07-21T19:06:08.405822+0000","last_became_peered":"2023-07-21T19:06:08.405822+0000","last_unstale":"2023-07-21T19:07:19.515456+0000","last_undegraded":"2023-07-21T19:07:19.515456+0000","last_fullsized":"2023-07-21T19:07:19.515456+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:14:23.390484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":97,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.344030+0000","last_change":"2023-07-21T19:05:37.289984+0000","last_active":"2023-07-21T19:07:07.344030+0000","last_peered":"2023-07-21T19:07:07.344030+0000","last_clean":"2023-07-21T19:07:07.344030+0000","last_became_active":"2023-07-21T19:05:37.289611+0000","last_became_peered":"2023-07-21T19:05:37.289611+0000","last_unstale":"2023-07-21T19:07:07.344030+0000","last_undegraded":"2023-07-21T19:07:07.344030+0000","last_fullsized":"2023-07-21T19:07:07.344030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:27:21.320856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1a","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.400567+0000","last_change":"2023-07-21T19:05:36.703331+0000","last_active":"2023-07-21T19:06:08.400567+0000","last_peered":"2023-07-21T19:06:08.400567+0000","last_clean":"2023-07-21T19:06:08.400567+0000","last_became_active":"2023-07-21T19:05:36.702971+0000","last_became_peered":"2023-07-21T19:05:36.702971+0000","last_unstale":"2023-07-21T19:06:08.400567+0000","last_undegraded":"2023-07-21T19:06:08.400567+0000","last_fullsized":"2023-07-21T19:06:08.400567+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:57:41.115053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360782+0000","last_change":"2023-07-21T19:05:37.295167+0000","last_active":"2023-07-21T19:07:07.360782+0000","last_peered":"2023-07-21T19:07:07.360782+0000","last_clean":"2023-07-21T19:07:07.360782+0000","last_became_active":"2023-07-21T19:05:37.294831+0000","last_became_peered":"2023-07-21T19:05:37.294831+0000","last_unstale":"2023-07-21T19:07:07.360782+0000","last_undegraded":"2023-07-21T19:07:07.360782+0000","last_fullsized":"2023-07-21T19:07:07.360782+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:14:01.147711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.19","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404354+0000","last_change":"2023-07-21T19:05:36.698670+0000","last_active":"2023-07-21T19:06:08.404354+0000","last_peered":"2023-07-21T19:06:08.404354+0000","last_clean":"2023-07-21T19:06:08.404354+0000","last_became_active":"2023-07-21T19:05:36.698218+0000","last_became_peered":"2023-07-21T19:05:36.698218+0000","last_unstale":"2023-07-21T19:06:08.404354+0000","last_undegraded":"2023-07-21T19:06:08.404354+0000","last_fullsized":"2023-07-21T19:06:08.404354+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:37:28.704125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361025+0000","last_change":"2023-07-21T19:05:37.287889+0000","last_active":"2023-07-21T19:07:07.361025+0000","last_peered":"2023-07-21T19:07:07.361025+0000","last_clean":"2023-07-21T19:07:07.361025+0000","last_became_active":"2023-07-21T19:05:37.287559+0000","last_became_peered":"2023-07-21T19:05:37.287559+0000","last_unstale":"2023-07-21T19:07:07.361025+0000","last_undegraded":"2023-07-21T19:07:07.361025+0000","last_fullsized":"2023-07-21T19:07:07.361025+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:11:34.147350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401383+0000","last_change":"2023-07-21T19:05:36.698604+0000","last_active":"2023-07-21T19:06:08.401383+0000","last_peered":"2023-07-21T19:06:08.401383+0000","last_clean":"2023-07-21T19:06:08.401383+0000","last_became_active":"2023-07-21T19:05:36.698297+0000","last_became_peered":"2023-07-21T19:05:36.698297+0000","last_unstale":"2023-07-21T19:06:08.401383+0000","last_undegraded":"2023-07-21T19:06:08.401383+0000","last_fullsized":"2023-07-21T19:06:08.401383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:39:13.512678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360977+0000","last_change":"2023-07-21T19:05:37.293389+0000","last_active":"2023-07-21T19:07:07.360977+0000","last_peered":"2023-07-21T19:07:07.360977+0000","last_clean":"2023-07-21T19:07:07.360977+0000","last_became_active":"2023-07-21T19:05:37.292787+0000","last_became_peered":"2023-07-21T19:05:37.292787+0000","last_unstale":"2023-07-21T19:07:07.360977+0000","last_undegraded":"2023-07-21T19:07:07.360977+0000","last_fullsized":"2023-07-21T19:07:07.360977+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:11:46.382674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"87'4","reported_seq":120,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.513549+0000","last_change":"2023-07-21T19:05:36.694559+0000","last_active":"2023-07-21T19:07:18.513549+0000","last_peered":"2023-07-21T19:07:18.513549+0000","last_clean":"2023-07-21T19:07:18.513549+0000","last_became_active":"2023-07-21T19:05:36.694104+0000","last_became_peered":"2023-07-21T19:05:36.694104+0000","last_unstale":"2023-07-21T19:07:18.513549+0000","last_undegraded":"2023-07-21T19:07:18.513549+0000","last_fullsized":"2023-07-21T19:07:18.513549+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:13:59.692187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":98,"num_read_kb":98,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.363144+0000","last_change":"2023-07-21T19:05:37.291427+0000","last_active":"2023-07-21T19:07:07.363144+0000","last_peered":"2023-07-21T19:07:07.363144+0000","last_clean":"2023-07-21T19:07:07.363144+0000","last_became_active":"2023-07-21T19:05:37.290962+0000","last_became_peered":"2023-07-21T19:05:37.290962+0000","last_unstale":"2023-07-21T19:07:07.363144+0000","last_undegraded":"2023-07-21T19:07:07.363144+0000","last_fullsized":"2023-07-21T19:07:07.363144+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:06:32.432702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.403970+0000","last_change":"2023-07-21T19:05:36.700667+0000","last_active":"2023-07-21T19:06:08.403970+0000","last_peered":"2023-07-21T19:06:08.403970+0000","last_clean":"2023-07-21T19:06:08.403970+0000","last_became_active":"2023-07-21T19:05:36.700240+0000","last_became_peered":"2023-07-21T19:05:36.700240+0000","last_unstale":"2023-07-21T19:06:08.403970+0000","last_undegraded":"2023-07-21T19:06:08.403970+0000","last_fullsized":"2023-07-21T19:06:08.403970+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:02:09.662672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357562+0000","last_change":"2023-07-21T19:05:37.299608+0000","last_active":"2023-07-21T19:07:07.357562+0000","last_peered":"2023-07-21T19:07:07.357562+0000","last_clean":"2023-07-21T19:07:07.357562+0000","last_became_active":"2023-07-21T19:05:37.299287+0000","last_became_peered":"2023-07-21T19:05:37.299287+0000","last_unstale":"2023-07-21T19:07:07.357562+0000","last_undegraded":"2023-07-21T19:07:07.357562+0000","last_fullsized":"2023-07-21T19:07:07.357562+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:43:03.677074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402384+0000","last_change":"2023-07-21T19:05:36.704224+0000","last_active":"2023-07-21T19:06:08.402384+0000","last_peered":"2023-07-21T19:06:08.402384+0000","last_clean":"2023-07-21T19:06:08.402384+0000","last_became_active":"2023-07-21T19:05:36.703896+0000","last_became_peered":"2023-07-21T19:05:36.703896+0000","last_unstale":"2023-07-21T19:06:08.402384+0000","last_undegraded":"2023-07-21T19:06:08.402384+0000","last_fullsized":"2023-07-21T19:06:08.402384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:16:07.189240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358434+0000","last_change":"2023-07-21T19:05:37.292797+0000","last_active":"2023-07-21T19:07:07.358434+0000","last_peered":"2023-07-21T19:07:07.358434+0000","last_clean":"2023-07-21T19:07:07.358434+0000","last_became_active":"2023-07-21T19:05:37.292340+0000","last_became_peered":"2023-07-21T19:05:37.292340+0000","last_unstale":"2023-07-21T19:07:07.358434+0000","last_undegraded":"2023-07-21T19:07:07.358434+0000","last_fullsized":"2023-07-21T19:07:07.358434+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:00:01.676198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"87'33","reported_seq":258,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:19.514756+0000","last_change":"2023-07-21T19:05:36.699787+0000","last_active":"2023-07-21T19:07:19.514756+0000","last_peered":"2023-07-21T19:07:19.514756+0000","last_clean":"2023-07-21T19:07:19.514756+0000","last_became_active":"2023-07-21T19:05:36.699480+0000","last_became_peered":"2023-07-21T19:05:36.699480+0000","last_unstale":"2023-07-21T19:07:19.514756+0000","last_undegraded":"2023-07-21T19:07:19.514756+0000","last_fullsized":"2023-07-21T19:07:19.514756+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:33:31.353689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":58815,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":207,"num_read_kb":161,"num_write":34,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.334355+0000","last_change":"2023-07-21T19:05:37.290876+0000","last_active":"2023-07-21T19:07:07.334355+0000","last_peered":"2023-07-21T19:07:07.334355+0000","last_clean":"2023-07-21T19:07:07.334355+0000","last_became_active":"2023-07-21T19:05:37.290586+0000","last_became_peered":"2023-07-21T19:05:37.290586+0000","last_unstale":"2023-07-21T19:07:07.334355+0000","last_undegraded":"2023-07-21T19:07:07.334355+0000","last_fullsized":"2023-07-21T19:07:07.334355+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:31:31.374240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"85'6","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405148+0000","last_change":"2023-07-21T19:05:36.700968+0000","last_active":"2023-07-21T19:06:08.405148+0000","last_peered":"2023-07-21T19:06:08.405148+0000","last_clean":"2023-07-21T19:06:08.405148+0000","last_became_active":"2023-07-21T19:05:36.700620+0000","last_became_peered":"2023-07-21T19:05:36.700620+0000","last_unstale":"2023-07-21T19:06:08.405148+0000","last_undegraded":"2023-07-21T19:06:08.405148+0000","last_fullsized":"2023-07-21T19:06:08.405148+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:20.602887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.351617+0000","last_change":"2023-07-21T19:05:37.289882+0000","last_active":"2023-07-21T19:07:07.351617+0000","last_peered":"2023-07-21T19:07:07.351617+0000","last_clean":"2023-07-21T19:07:07.351617+0000","last_became_active":"2023-07-21T19:05:37.289432+0000","last_became_peered":"2023-07-21T19:05:37.289432+0000","last_unstale":"2023-07-21T19:07:07.351617+0000","last_undegraded":"2023-07-21T19:07:07.351617+0000","last_fullsized":"2023-07-21T19:07:07.351617+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:59:21.737635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401848+0000","last_change":"2023-07-21T19:05:36.698626+0000","last_active":"2023-07-21T19:06:08.401848+0000","last_peered":"2023-07-21T19:06:08.401848+0000","last_clean":"2023-07-21T19:06:08.401848+0000","last_became_active":"2023-07-21T19:05:36.698191+0000","last_became_peered":"2023-07-21T19:05:36.698191+0000","last_unstale":"2023-07-21T19:06:08.401848+0000","last_undegraded":"2023-07-21T19:06:08.401848+0000","last_fullsized":"2023-07-21T19:06:08.401848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:13:48.688808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361072+0000","last_change":"2023-07-21T19:05:37.299320+0000","last_active":"2023-07-21T19:07:07.361072+0000","last_peered":"2023-07-21T19:07:07.361072+0000","last_clean":"2023-07-21T19:07:07.361072+0000","last_became_active":"2023-07-21T19:05:37.298956+0000","last_became_peered":"2023-07-21T19:05:37.298956+0000","last_unstale":"2023-07-21T19:07:07.361072+0000","last_undegraded":"2023-07-21T19:07:07.361072+0000","last_fullsized":"2023-07-21T19:07:07.361072+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:25:55.454258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"87'4","reported_seq":123,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:18.467900+0000","last_change":"2023-07-21T19:05:36.690550+0000","last_active":"2023-07-21T19:07:18.467900+0000","last_peered":"2023-07-21T19:07:18.467900+0000","last_clean":"2023-07-21T19:07:18.467900+0000","last_became_active":"2023-07-21T19:05:36.690147+0000","last_became_peered":"2023-07-21T19:05:36.690147+0000","last_unstale":"2023-07-21T19:07:18.467900+0000","last_undegraded":"2023-07-21T19:07:18.467900+0000","last_fullsized":"2023-07-21T19:07:18.467900+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:17:26.036729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1023,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.358107+0000","last_change":"2023-07-21T19:05:37.292823+0000","last_active":"2023-07-21T19:07:07.358107+0000","last_peered":"2023-07-21T19:07:07.358107+0000","last_clean":"2023-07-21T19:07:07.358107+0000","last_became_active":"2023-07-21T19:05:37.292498+0000","last_became_peered":"2023-07-21T19:05:37.292498+0000","last_unstale":"2023-07-21T19:07:07.358107+0000","last_undegraded":"2023-07-21T19:07:07.358107+0000","last_fullsized":"2023-07-21T19:07:07.358107+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:06:24.967595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"85'15","reported_seq":34,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.397845+0000","last_change":"2023-07-21T19:05:36.704464+0000","last_active":"2023-07-21T19:06:08.397845+0000","last_peered":"2023-07-21T19:06:08.397845+0000","last_clean":"2023-07-21T19:06:08.397845+0000","last_became_active":"2023-07-21T19:05:36.704109+0000","last_became_peered":"2023-07-21T19:05:36.704109+0000","last_unstale":"2023-07-21T19:06:08.397845+0000","last_undegraded":"2023-07-21T19:06:08.397845+0000","last_fullsized":"2023-07-21T19:06:08.397845+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:31:53.019352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361317+0000","last_change":"2023-07-21T19:05:37.298847+0000","last_active":"2023-07-21T19:07:07.361317+0000","last_peered":"2023-07-21T19:07:07.361317+0000","last_clean":"2023-07-21T19:07:07.361317+0000","last_became_active":"2023-07-21T19:05:37.298517+0000","last_became_peered":"2023-07-21T19:05:37.298517+0000","last_unstale":"2023-07-21T19:07:07.361317+0000","last_undegraded":"2023-07-21T19:07:07.361317+0000","last_fullsized":"2023-07-21T19:07:07.361317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:01:37.822944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"85'2","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398316+0000","last_change":"2023-07-21T19:05:36.695695+0000","last_active":"2023-07-21T19:06:08.398316+0000","last_peered":"2023-07-21T19:06:08.398316+0000","last_clean":"2023-07-21T19:06:08.398316+0000","last_became_active":"2023-07-21T19:05:36.695337+0000","last_became_peered":"2023-07-21T19:05:36.695337+0000","last_unstale":"2023-07-21T19:06:08.398316+0000","last_undegraded":"2023-07-21T19:06:08.398316+0000","last_fullsized":"2023-07-21T19:06:08.398316+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:32:37.555620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.343477+0000","last_change":"2023-07-21T19:05:37.298897+0000","last_active":"2023-07-21T19:07:07.343477+0000","last_peered":"2023-07-21T19:07:07.343477+0000","last_clean":"2023-07-21T19:07:07.343477+0000","last_became_active":"2023-07-21T19:05:37.298567+0000","last_became_peered":"2023-07-21T19:05:37.298567+0000","last_unstale":"2023-07-21T19:07:07.343477+0000","last_undegraded":"2023-07-21T19:07:07.343477+0000","last_fullsized":"2023-07-21T19:07:07.343477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:15:42.470455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.406908+0000","last_change":"2023-07-21T19:06:08.406908+0000","last_active":"2023-07-21T19:06:08.406908+0000","last_peered":"2023-07-21T19:06:08.406908+0000","last_clean":"2023-07-21T19:06:08.406908+0000","last_became_active":"2023-07-21T19:06:08.406531+0000","last_became_peered":"2023-07-21T19:06:08.406531+0000","last_unstale":"2023-07-21T19:06:08.406908+0000","last_undegraded":"2023-07-21T19:06:08.406908+0000","last_fullsized":"2023-07-21T19:06:08.406908+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:35:04.808701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,0],"acting":[9,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.357349+0000","last_change":"2023-07-21T19:05:37.295258+0000","last_active":"2023-07-21T19:07:07.357349+0000","last_peered":"2023-07-21T19:07:07.357349+0000","last_clean":"2023-07-21T19:07:07.357349+0000","last_became_active":"2023-07-21T19:05:37.294920+0000","last_became_peered":"2023-07-21T19:05:37.294920+0000","last_unstale":"2023-07-21T19:07:07.357349+0000","last_undegraded":"2023-07-21T19:07:07.357349+0000","last_fullsized":"2023-07-21T19:07:07.357349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:21:01.821241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.2","version":"85'1","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404216+0000","last_change":"2023-07-21T19:05:36.700054+0000","last_active":"2023-07-21T19:06:08.404216+0000","last_peered":"2023-07-21T19:06:08.404216+0000","last_clean":"2023-07-21T19:06:08.404216+0000","last_became_active":"2023-07-21T19:05:36.699674+0000","last_became_peered":"2023-07-21T19:05:36.699674+0000","last_unstale":"2023-07-21T19:06:08.404216+0000","last_undegraded":"2023-07-21T19:06:08.404216+0000","last_fullsized":"2023-07-21T19:06:08.404216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:35:18.679466+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359555+0000","last_change":"2023-07-21T19:06:08.408400+0000","last_active":"2023-07-21T19:07:07.359555+0000","last_peered":"2023-07-21T19:07:07.359555+0000","last_clean":"2023-07-21T19:07:07.359555+0000","last_became_active":"2023-07-21T19:06:08.408005+0000","last_became_peered":"2023-07-21T19:06:08.408005+0000","last_unstale":"2023-07-21T19:07:07.359555+0000","last_undegraded":"2023-07-21T19:07:07.359555+0000","last_fullsized":"2023-07-21T19:07:07.359555+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:04:20.230383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.0","version":"87'8","reported_seq":19,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342774+0000","last_change":"2023-07-21T19:06:08.407039+0000","last_active":"2023-07-21T19:07:07.342774+0000","last_peered":"2023-07-21T19:07:07.342774+0000","last_clean":"2023-07-21T19:07:07.342774+0000","last_became_active":"2023-07-21T19:06:08.406635+0000","last_became_peered":"2023-07-21T19:06:08.406635+0000","last_unstale":"2023-07-21T19:07:07.342774+0000","last_undegraded":"2023-07-21T19:07:07.342774+0000","last_fullsized":"2023-07-21T19:07:07.342774+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:59:18.504252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1","version":"87'5","reported_seq":119,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:16.461334+0000","last_change":"2023-07-21T19:05:36.702801+0000","last_active":"2023-07-21T19:07:16.461334+0000","last_peered":"2023-07-21T19:07:16.461334+0000","last_clean":"2023-07-21T19:07:16.461334+0000","last_became_active":"2023-07-21T19:05:36.702484+0000","last_became_peered":"2023-07-21T19:05:36.702484+0000","last_unstale":"2023-07-21T19:07:16.461334+0000","last_undegraded":"2023-07-21T19:07:16.461334+0000","last_fullsized":"2023-07-21T19:07:16.461334+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:09:21.302828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":96,"num_read_kb":96,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362955+0000","last_change":"2023-07-21T19:05:37.293303+0000","last_active":"2023-07-21T19:07:07.362955+0000","last_peered":"2023-07-21T19:07:07.362955+0000","last_clean":"2023-07-21T19:07:07.362955+0000","last_became_active":"2023-07-21T19:05:37.292746+0000","last_became_peered":"2023-07-21T19:05:37.292746+0000","last_unstale":"2023-07-21T19:07:07.362955+0000","last_undegraded":"2023-07-21T19:07:07.362955+0000","last_fullsized":"2023-07-21T19:07:07.362955+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:49:42.921526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"85'3","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398968+0000","last_change":"2023-07-21T19:05:36.691556+0000","last_active":"2023-07-21T19:06:08.398968+0000","last_peered":"2023-07-21T19:06:08.398968+0000","last_clean":"2023-07-21T19:06:08.398968+0000","last_became_active":"2023-07-21T19:05:36.691221+0000","last_became_peered":"2023-07-21T19:05:36.691221+0000","last_unstale":"2023-07-21T19:06:08.398968+0000","last_undegraded":"2023-07-21T19:06:08.398968+0000","last_fullsized":"2023-07-21T19:06:08.398968+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:13:50.624939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"22'137","reported_seq":349,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404291+0000","last_change":"2023-07-21T19:04:17.716903+0000","last_active":"2023-07-21T19:06:08.404291+0000","last_peered":"2023-07-21T19:06:08.404291+0000","last_clean":"2023-07-21T19:06:08.404291+0000","last_became_active":"2023-07-21T19:04:17.716216+0000","last_became_peered":"2023-07-21T19:04:17.716216+0000","last_unstale":"2023-07-21T19:06:08.404291+0000","last_undegraded":"2023-07-21T19:06:08.404291+0000","last_fullsized":"2023-07-21T19:06:08.404291+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:00:42.214047+0000","last_clean_scrub_stamp":"2023-07-21T19:00:42.214047+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:37:45.400614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"87'5","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.346572+0000","last_change":"2023-07-21T19:06:08.409780+0000","last_active":"2023-07-21T19:07:07.346572+0000","last_peered":"2023-07-21T19:07:07.346572+0000","last_clean":"2023-07-21T19:07:07.346572+0000","last_became_active":"2023-07-21T19:06:08.409394+0000","last_became_peered":"2023-07-21T19:06:08.409394+0000","last_unstale":"2023-07-21T19:07:07.346572+0000","last_undegraded":"2023-07-21T19:07:07.346572+0000","last_fullsized":"2023-07-21T19:07:07.346572+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T19:36:45.599367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,2],"acting":[6,9,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404518+0000","last_change":"2023-07-21T19:05:36.695563+0000","last_active":"2023-07-21T19:06:08.404518+0000","last_peered":"2023-07-21T19:06:08.404518+0000","last_clean":"2023-07-21T19:06:08.404518+0000","last_became_active":"2023-07-21T19:05:36.695227+0000","last_became_peered":"2023-07-21T19:05:36.695227+0000","last_unstale":"2023-07-21T19:06:08.404518+0000","last_undegraded":"2023-07-21T19:06:08.404518+0000","last_fullsized":"2023-07-21T19:06:08.404518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:04.683020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"85'1","reported_seq":15,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.413663+0000","last_change":"2023-07-21T19:06:08.413252+0000","last_active":"2023-07-21T19:06:08.413663+0000","last_peered":"2023-07-21T19:06:08.413663+0000","last_clean":"2023-07-21T19:06:08.413663+0000","last_became_active":"2023-07-21T19:06:08.407914+0000","last_became_peered":"2023-07-21T19:06:08.407914+0000","last_unstale":"2023-07-21T19:06:08.413663+0000","last_undegraded":"2023-07-21T19:06:08.413663+0000","last_fullsized":"2023-07-21T19:06:08.413663+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:30:08.813981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.347131+0000","last_change":"2023-07-21T19:05:37.288667+0000","last_active":"2023-07-21T19:07:07.347131+0000","last_peered":"2023-07-21T19:07:07.347131+0000","last_clean":"2023-07-21T19:07:07.347131+0000","last_became_active":"2023-07-21T19:05:37.288210+0000","last_became_peered":"2023-07-21T19:05:37.288210+0000","last_unstale":"2023-07-21T19:07:07.347131+0000","last_undegraded":"2023-07-21T19:07:07.347131+0000","last_fullsized":"2023-07-21T19:07:07.347131+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:02:17.603158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"87'8","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:10.564825+0000","last_change":"2023-07-21T19:05:36.693362+0000","last_active":"2023-07-21T19:07:10.564825+0000","last_peered":"2023-07-21T19:07:10.564825+0000","last_clean":"2023-07-21T19:07:10.564825+0000","last_became_active":"2023-07-21T19:05:36.692948+0000","last_became_peered":"2023-07-21T19:05:36.692948+0000","last_unstale":"2023-07-21T19:07:10.564825+0000","last_undegraded":"2023-07-21T19:07:10.564825+0000","last_fullsized":"2023-07-21T19:07:10.564825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:32:46.547354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356114+0000","last_change":"2023-07-21T19:05:37.294954+0000","last_active":"2023-07-21T19:07:07.356114+0000","last_peered":"2023-07-21T19:07:07.356114+0000","last_clean":"2023-07-21T19:07:07.356114+0000","last_became_active":"2023-07-21T19:05:37.294246+0000","last_became_peered":"2023-07-21T19:05:37.294246+0000","last_unstale":"2023-07-21T19:07:07.356114+0000","last_undegraded":"2023-07-21T19:07:07.356114+0000","last_fullsized":"2023-07-21T19:07:07.356114+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:34:12.413946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"85'4","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.399221+0000","last_change":"2023-07-21T19:05:36.699061+0000","last_active":"2023-07-21T19:06:08.399221+0000","last_peered":"2023-07-21T19:06:08.399221+0000","last_clean":"2023-07-21T19:06:08.399221+0000","last_became_active":"2023-07-21T19:05:36.698734+0000","last_became_peered":"2023-07-21T19:05:36.698734+0000","last_unstale":"2023-07-21T19:06:08.399221+0000","last_undegraded":"2023-07-21T19:06:08.399221+0000","last_fullsized":"2023-07-21T19:06:08.399221+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:24:03.493447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"87'7","reported_seq":29,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360651+0000","last_change":"2023-07-21T19:06:08.842743+0000","last_active":"2023-07-21T19:07:07.360651+0000","last_peered":"2023-07-21T19:07:07.360651+0000","last_clean":"2023-07-21T19:07:07.360651+0000","last_became_active":"2023-07-21T19:06:08.842349+0000","last_became_peered":"2023-07-21T19:06:08.842349+0000","last_unstale":"2023-07-21T19:07:07.360651+0000","last_undegraded":"2023-07-21T19:07:07.360651+0000","last_fullsized":"2023-07-21T19:07:07.360651+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:05:30.960744+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"85'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.841889+0000","last_change":"2023-07-21T19:05:36.703195+0000","last_active":"2023-07-21T19:06:08.841889+0000","last_peered":"2023-07-21T19:06:08.841889+0000","last_clean":"2023-07-21T19:06:08.841889+0000","last_became_active":"2023-07-21T19:05:36.702785+0000","last_became_peered":"2023-07-21T19:05:36.702785+0000","last_unstale":"2023-07-21T19:06:08.841889+0000","last_undegraded":"2023-07-21T19:06:08.841889+0000","last_fullsized":"2023-07-21T19:06:08.841889+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:18:57.325736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.310459+0000","last_change":"2023-07-21T19:05:37.290202+0000","last_active":"2023-07-21T19:07:07.310459+0000","last_peered":"2023-07-21T19:07:07.310459+0000","last_clean":"2023-07-21T19:07:07.310459+0000","last_became_active":"2023-07-21T19:05:37.289881+0000","last_became_peered":"2023-07-21T19:05:37.289881+0000","last_unstale":"2023-07-21T19:07:07.310459+0000","last_undegraded":"2023-07-21T19:07:07.310459+0000","last_fullsized":"2023-07-21T19:07:07.310459+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:17.675575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"85'3","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.840681+0000","last_change":"2023-07-21T19:05:36.693859+0000","last_active":"2023-07-21T19:06:08.840681+0000","last_peered":"2023-07-21T19:06:08.840681+0000","last_clean":"2023-07-21T19:06:08.840681+0000","last_became_active":"2023-07-21T19:05:36.693536+0000","last_became_peered":"2023-07-21T19:05:36.693536+0000","last_unstale":"2023-07-21T19:06:08.840681+0000","last_undegraded":"2023-07-21T19:06:08.840681+0000","last_fullsized":"2023-07-21T19:06:08.840681+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:48:11.395258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354043+0000","last_change":"2023-07-21T19:05:37.294609+0000","last_active":"2023-07-21T19:07:07.354043+0000","last_peered":"2023-07-21T19:07:07.354043+0000","last_clean":"2023-07-21T19:07:07.354043+0000","last_became_active":"2023-07-21T19:05:37.294297+0000","last_became_peered":"2023-07-21T19:05:37.294297+0000","last_unstale":"2023-07-21T19:07:07.354043+0000","last_undegraded":"2023-07-21T19:07:07.354043+0000","last_fullsized":"2023-07-21T19:07:07.354043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:28:35.734063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"87'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.404850+0000","last_change":"2023-07-21T19:05:36.699822+0000","last_active":"2023-07-21T19:06:08.404850+0000","last_peered":"2023-07-21T19:06:08.404850+0000","last_clean":"2023-07-21T19:06:08.404850+0000","last_became_active":"2023-07-21T19:05:36.699450+0000","last_became_peered":"2023-07-21T19:05:36.699450+0000","last_unstale":"2023-07-21T19:06:08.404850+0000","last_undegraded":"2023-07-21T19:06:08.404850+0000","last_fullsized":"2023-07-21T19:06:08.404850+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T03:12:53.659216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.356469+0000","last_change":"2023-07-21T19:05:37.301152+0000","last_active":"2023-07-21T19:07:07.356469+0000","last_peered":"2023-07-21T19:07:07.356469+0000","last_clean":"2023-07-21T19:07:07.356469+0000","last_became_active":"2023-07-21T19:05:37.300824+0000","last_became_peered":"2023-07-21T19:05:37.300824+0000","last_unstale":"2023-07-21T19:07:07.356469+0000","last_undegraded":"2023-07-21T19:07:07.356469+0000","last_fullsized":"2023-07-21T19:07:07.356469+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:19:55.786796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"85'4","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.398085+0000","last_change":"2023-07-21T19:05:36.705257+0000","last_active":"2023-07-21T19:06:08.398085+0000","last_peered":"2023-07-21T19:06:08.398085+0000","last_clean":"2023-07-21T19:06:08.398085+0000","last_became_active":"2023-07-21T19:05:36.704782+0000","last_became_peered":"2023-07-21T19:05:36.704782+0000","last_unstale":"2023-07-21T19:06:08.398085+0000","last_undegraded":"2023-07-21T19:06:08.398085+0000","last_fullsized":"2023-07-21T19:06:08.398085+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:49:02.430774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"87'5","reported_seq":21,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362043+0000","last_change":"2023-07-21T19:05:37.296394+0000","last_active":"2023-07-21T19:07:07.362043+0000","last_peered":"2023-07-21T19:07:07.362043+0000","last_clean":"2023-07-21T19:07:07.362043+0000","last_became_active":"2023-07-21T19:05:37.296057+0000","last_became_peered":"2023-07-21T19:05:37.296057+0000","last_unstale":"2023-07-21T19:07:07.362043+0000","last_undegraded":"2023-07-21T19:07:07.362043+0000","last_fullsized":"2023-07-21T19:07:07.362043+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:08:39.735825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"85'3","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.405117+0000","last_change":"2023-07-21T19:05:36.702188+0000","last_active":"2023-07-21T19:06:08.405117+0000","last_peered":"2023-07-21T19:06:08.405117+0000","last_clean":"2023-07-21T19:06:08.405117+0000","last_became_active":"2023-07-21T19:05:36.701683+0000","last_became_peered":"2023-07-21T19:05:36.701683+0000","last_unstale":"2023-07-21T19:06:08.405117+0000","last_undegraded":"2023-07-21T19:06:08.405117+0000","last_fullsized":"2023-07-21T19:06:08.405117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T05:51:44.759183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362557+0000","last_change":"2023-07-21T19:05:37.297579+0000","last_active":"2023-07-21T19:07:07.362557+0000","last_peered":"2023-07-21T19:07:07.362557+0000","last_clean":"2023-07-21T19:07:07.362557+0000","last_became_active":"2023-07-21T19:05:37.296759+0000","last_became_peered":"2023-07-21T19:05:37.296759+0000","last_unstale":"2023-07-21T19:07:07.362557+0000","last_undegraded":"2023-07-21T19:07:07.362557+0000","last_fullsized":"2023-07-21T19:07:07.362557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:17:16.325005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.402542+0000","last_change":"2023-07-21T19:05:36.695342+0000","last_active":"2023-07-21T19:06:08.402542+0000","last_peered":"2023-07-21T19:06:08.402542+0000","last_clean":"2023-07-21T19:06:08.402542+0000","last_became_active":"2023-07-21T19:05:36.695016+0000","last_became_peered":"2023-07-21T19:05:36.695016+0000","last_unstale":"2023-07-21T19:06:08.402542+0000","last_undegraded":"2023-07-21T19:06:08.402542+0000","last_fullsized":"2023-07-21T19:06:08.402542+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:10:41.914439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.355564+0000","last_change":"2023-07-21T19:05:37.294048+0000","last_active":"2023-07-21T19:07:07.355564+0000","last_peered":"2023-07-21T19:07:07.355564+0000","last_clean":"2023-07-21T19:07:07.355564+0000","last_became_active":"2023-07-21T19:05:37.293661+0000","last_became_peered":"2023-07-21T19:05:37.293661+0000","last_unstale":"2023-07-21T19:07:07.355564+0000","last_undegraded":"2023-07-21T19:07:07.355564+0000","last_fullsized":"2023-07-21T19:07:07.355564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T00:08:59.431298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:06:08.401998+0000","last_change":"2023-07-21T19:05:36.697131+0000","last_active":"2023-07-21T19:06:08.401998+0000","last_peered":"2023-07-21T19:06:08.401998+0000","last_clean":"2023-07-21T19:06:08.401998+0000","last_became_active":"2023-07-21T19:05:36.696832+0000","last_became_peered":"2023-07-21T19:05:36.696832+0000","last_unstale":"2023-07-21T19:06:08.401998+0000","last_undegraded":"2023-07-21T19:06:08.401998+0000","last_fullsized":"2023-07-21T19:06:08.401998+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:35.662330+0000","last_clean_scrub_stamp":"2023-07-21T19:05:35.662330+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:43:17.091156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354174+0000","last_change":"2023-07-21T19:05:37.294702+0000","last_active":"2023-07-21T19:07:07.354174+0000","last_peered":"2023-07-21T19:07:07.354174+0000","last_clean":"2023-07-21T19:07:07.354174+0000","last_became_active":"2023-07-21T19:05:37.294419+0000","last_became_peered":"2023-07-21T19:05:37.294419+0000","last_unstale":"2023-07-21T19:07:07.354174+0000","last_undegraded":"2023-07-21T19:07:07.354174+0000","last_fullsized":"2023-07-21T19:07:07.354174+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:52:09.484987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"87'9","reported_seq":25,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345151+0000","last_change":"2023-07-21T19:05:37.290896+0000","last_active":"2023-07-21T19:07:07.345151+0000","last_peered":"2023-07-21T19:07:07.345151+0000","last_clean":"2023-07-21T19:07:07.345151+0000","last_became_active":"2023-07-21T19:05:37.290413+0000","last_became_peered":"2023-07-21T19:05:37.290413+0000","last_unstale":"2023-07-21T19:07:07.345151+0000","last_undegraded":"2023-07-21T19:07:07.345151+0000","last_fullsized":"2023-07-21T19:07:07.345151+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:30:56.095641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"87'12","reported_seq":28,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352815+0000","last_change":"2023-07-21T19:05:37.290985+0000","last_active":"2023-07-21T19:07:07.352815+0000","last_peered":"2023-07-21T19:07:07.352815+0000","last_clean":"2023-07-21T19:07:07.352815+0000","last_became_active":"2023-07-21T19:05:37.290622+0000","last_became_peered":"2023-07-21T19:05:37.290622+0000","last_unstale":"2023-07-21T19:07:07.352815+0000","last_undegraded":"2023-07-21T19:07:07.352815+0000","last_fullsized":"2023-07-21T19:07:07.352815+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:22:24.580856+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.361130+0000","last_change":"2023-07-21T19:05:37.291973+0000","last_active":"2023-07-21T19:07:07.361130+0000","last_peered":"2023-07-21T19:07:07.361130+0000","last_clean":"2023-07-21T19:07:07.361130+0000","last_became_active":"2023-07-21T19:05:37.291674+0000","last_became_peered":"2023-07-21T19:05:37.291674+0000","last_unstale":"2023-07-21T19:07:07.361130+0000","last_undegraded":"2023-07-21T19:07:07.361130+0000","last_fullsized":"2023-07-21T19:07:07.361130+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:10:17.790433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"87'7","reported_seq":23,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359810+0000","last_change":"2023-07-21T19:05:37.293017+0000","last_active":"2023-07-21T19:07:07.359810+0000","last_peered":"2023-07-21T19:07:07.359810+0000","last_clean":"2023-07-21T19:07:07.359810+0000","last_became_active":"2023-07-21T19:05:37.292732+0000","last_became_peered":"2023-07-21T19:05:37.292732+0000","last_unstale":"2023-07-21T19:07:07.359810+0000","last_undegraded":"2023-07-21T19:07:07.359810+0000","last_fullsized":"2023-07-21T19:07:07.359810+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:51:29.261412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"87'4","reported_seq":20,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.354985+0000","last_change":"2023-07-21T19:05:37.291049+0000","last_active":"2023-07-21T19:07:07.354985+0000","last_peered":"2023-07-21T19:07:07.354985+0000","last_clean":"2023-07-21T19:07:07.354985+0000","last_became_active":"2023-07-21T19:05:37.290615+0000","last_became_peered":"2023-07-21T19:05:37.290615+0000","last_unstale":"2023-07-21T19:07:07.354985+0000","last_undegraded":"2023-07-21T19:07:07.354985+0000","last_fullsized":"2023-07-21T19:07:07.354985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T21:47:56.761424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.342375+0000","last_change":"2023-07-21T19:05:37.291287+0000","last_active":"2023-07-21T19:07:07.342375+0000","last_peered":"2023-07-21T19:07:07.342375+0000","last_clean":"2023-07-21T19:07:07.342375+0000","last_became_active":"2023-07-21T19:05:37.290977+0000","last_became_peered":"2023-07-21T19:05:37.290977+0000","last_unstale":"2023-07-21T19:07:07.342375+0000","last_undegraded":"2023-07-21T19:07:07.342375+0000","last_fullsized":"2023-07-21T19:07:07.342375+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T06:45:42.804371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"87'15","reported_seq":31,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359579+0000","last_change":"2023-07-21T19:05:37.294267+0000","last_active":"2023-07-21T19:07:07.359579+0000","last_peered":"2023-07-21T19:07:07.359579+0000","last_clean":"2023-07-21T19:07:07.359579+0000","last_became_active":"2023-07-21T19:05:37.293901+0000","last_became_peered":"2023-07-21T19:05:37.293901+0000","last_unstale":"2023-07-21T19:07:07.359579+0000","last_undegraded":"2023-07-21T19:07:07.359579+0000","last_fullsized":"2023-07-21T19:07:07.359579+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T22:49:42.743095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"87'2","reported_seq":18,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360257+0000","last_change":"2023-07-21T19:05:37.295182+0000","last_active":"2023-07-21T19:07:07.360257+0000","last_peered":"2023-07-21T19:07:07.360257+0000","last_clean":"2023-07-21T19:07:07.360257+0000","last_became_active":"2023-07-21T19:05:37.294666+0000","last_became_peered":"2023-07-21T19:05:37.294666+0000","last_unstale":"2023-07-21T19:07:07.360257+0000","last_undegraded":"2023-07-21T19:07:07.360257+0000","last_fullsized":"2023-07-21T19:07:07.360257+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:40:21.846553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"87'6","reported_seq":22,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.360450+0000","last_change":"2023-07-21T19:05:37.298125+0000","last_active":"2023-07-21T19:07:07.360450+0000","last_peered":"2023-07-21T19:07:07.360450+0000","last_clean":"2023-07-21T19:07:07.360450+0000","last_became_active":"2023-07-21T19:05:37.297794+0000","last_became_peered":"2023-07-21T19:05:37.297794+0000","last_unstale":"2023-07-21T19:07:07.360450+0000","last_undegraded":"2023-07-21T19:07:07.360450+0000","last_fullsized":"2023-07-21T19:07:07.360450+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T01:34:19.869711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.359999+0000","last_change":"2023-07-21T19:05:37.288739+0000","last_active":"2023-07-21T19:07:07.359999+0000","last_peered":"2023-07-21T19:07:07.359999+0000","last_clean":"2023-07-21T19:07:07.359999+0000","last_became_active":"2023-07-21T19:05:37.288457+0000","last_became_peered":"2023-07-21T19:05:37.288457+0000","last_unstale":"2023-07-21T19:07:07.359999+0000","last_undegraded":"2023-07-21T19:07:07.359999+0000","last_fullsized":"2023-07-21T19:07:07.359999+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T23:50:33.674612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"87'10","reported_seq":26,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.362482+0000","last_change":"2023-07-21T19:05:37.290566+0000","last_active":"2023-07-21T19:07:07.362482+0000","last_peered":"2023-07-21T19:07:07.362482+0000","last_clean":"2023-07-21T19:07:07.362482+0000","last_became_active":"2023-07-21T19:05:37.290208+0000","last_became_peered":"2023-07-21T19:05:37.290208+0000","last_unstale":"2023-07-21T19:07:07.362482+0000","last_undegraded":"2023-07-21T19:07:07.362482+0000","last_fullsized":"2023-07-21T19:07:07.362482+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T04:23:36.091795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"87'11","reported_seq":27,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.352462+0000","last_change":"2023-07-21T19:05:37.291108+0000","last_active":"2023-07-21T19:07:07.352462+0000","last_peered":"2023-07-21T19:07:07.352462+0000","last_clean":"2023-07-21T19:07:07.352462+0000","last_became_active":"2023-07-21T19:05:37.289854+0000","last_became_peered":"2023-07-21T19:05:37.289854+0000","last_unstale":"2023-07-21T19:07:07.352462+0000","last_undegraded":"2023-07-21T19:07:07.352462+0000","last_fullsized":"2023-07-21T19:07:07.352462+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-22T20:45:18.507436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"87'8","reported_seq":24,"reported_epoch":87,"state":"active+clean","last_fresh":"2023-07-21T19:07:07.345299+0000","last_change":"2023-07-21T19:05:37.299048+0000","last_active":"2023-07-21T19:07:07.345299+0000","last_peered":"2023-07-21T19:07:07.345299+0000","last_clean":"2023-07-21T19:07:07.345299+0000","last_became_active":"2023-07-21T19:05:37.298711+0000","last_became_peered":"2023-07-21T19:05:37.298711+0000","last_unstale":"2023-07-21T19:07:07.345299+0000","last_undegraded":"2023-07-21T19:07:07.345299+0000","last_fullsized":"2023-07-21T19:07:07.345299+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-07-21T19:05:36.668143+0000","last_clean_scrub_stamp":"2023-07-21T19:05:36.668143+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-07-23T02:07:02.887348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":442368,"data_stored":4722944,"data_compressed":284882,"data_compressed_allocated":360448,"data_compressed_original":4669440,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":63826,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":612,"num_read_kb":376,"num_write":169,"num_write_kb":117,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1568,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":425984,"data_stored":200118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":146,"ondisk_log_size":146,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":456247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":0.77800000000000002,"15min":0.72299999999999998},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":6.1920000000000002,"5min":6.1920000000000002,"15min":6.1920000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.64900000000000002,"15min":0.65300000000000002},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.87,"5min":1.131,"15min":1.131},"last":0.76000000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.754,"15min":0.75},"min":{"1min":0.56899999999999995,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.044,"5min":1.4279999999999999,"15min":1.4279999999999999},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71699999999999997,"15min":0.72099999999999997},"min":{"1min":0.503,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.874,"5min":1.083,"15min":1.083},"last":0.79800000000000004}]},{"osd":2,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57699999999999996,"15min":0.58299999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.81399999999999995,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.628,"15min":0.62},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95099999999999996,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.63600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.84199999999999997,"5min":0.83799999999999997,"15min":0.83699999999999997},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.006,"5min":1.395,"15min":1.395},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.79900000000000004,"15min":0.80500000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.0249999999999999,"5min":1.1859999999999999,"15min":1.1859999999999999},"last":0.92700000000000005}]},{"osd":5,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.76000000000000001,"15min":0.76800000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":0.89800000000000002,"5min":1.208,"15min":1.208},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83699999999999997,"15min":0.83899999999999997},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.073,"5min":1.4570000000000001,"15min":1.4570000000000001},"last":0.81599999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.751,"15min":0.76300000000000001},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.91300000000000003,"5min":1.24,"15min":1.24},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.86399999999999999,"15min":0.876},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.97499999999999998,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":0.89200000000000002}]},{"osd":7,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.89800000000000002,"5min":0.93000000000000005,"15min":0.93600000000000005},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.095,"5min":1.595,"15min":1.595},"last":1.026},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.90400000000000003,"15min":0.91700000000000004},"min":{"1min":0.59099999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.0840000000000001,"5min":1.5660000000000001,"15min":1.5660000000000001},"last":0.877}]},{"osd":8,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.879,"5min":0.879,"15min":0.879},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.262,"5min":1.262,"15min":1.262},"last":1.0049999999999999},{"interface":"front","average":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"min":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.98799999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.98899999999999999,"5min":0.97299999999999998,"15min":0.97099999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":3.044,"5min":3.044,"15min":3.044},"last":1.101},{"interface":"front","average":{"1min":1.3200000000000001,"5min":1.0309999999999999,"15min":0.98299999999999998},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":6.2670000000000003,"5min":6.2670000000000003,"15min":6.2670000000000003},"last":1.085}]},{"osd":10,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.88400000000000001,"5min":0.94399999999999995,"15min":0.95399999999999996},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.127,"5min":1.615,"15min":1.615},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94199999999999995,"5min":0.97499999999999998,"15min":0.98099999999999998},"min":{"1min":0.75900000000000001,"5min":0.66100000000000003,"15min":0.66100000000000003},"max":{"1min":1.099,"5min":1.681,"15min":1.681},"last":1.1839999999999999}]}]},{"osd":10,"up_from":72,"seq":309237645352,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1093632,"data_stored":1106962,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":1.1659999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":6.4770000000000003,"5min":6.4770000000000003,"15min":6.4770000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":3.028,"5min":3.028,"15min":3.028},"last":0.60699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.36799999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73999999999999999,"5min":2.3809999999999998,"15min":2.3809999999999998},"last":0.441},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.38500000000000001,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.86099999999999999,"5min":2.323,"15min":2.323},"last":0.65400000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.627,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.94099999999999995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.745,"5min":0.755,"15min":0.747},"min":{"1min":0.54800000000000004,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":0.876,"5min":2.3460000000000001,"15min":2.3460000000000001},"last":0.64200000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.78600000000000003,"15min":0.78000000000000003},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89400000000000002,"5min":3.399,"15min":3.399},"last":0.5},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.80400000000000005,"15min":0.80300000000000005},"min":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.89700000000000002,"5min":2.2490000000000001,"15min":2.2490000000000001},"last":0.626}]},{"osd":4,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.73399999999999999,"15min":0.72999999999999998},"min":{"1min":0.47699999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0309999999999999,"5min":2.4649999999999999,"15min":2.4649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.68600000000000005,"15min":0.71099999999999997},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.0049999999999999,"5min":1.2569999999999999,"15min":1.2569999999999999},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.82599999999999996,"15min":0.84399999999999997},"min":{"1min":0.67800000000000005,"5min":0.5,"15min":0.5},"max":{"1min":1.0669999999999999,"5min":1.238,"15min":1.238},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.83699999999999997,"15min":0.84699999999999998},"min":{"1min":0.59999999999999998,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.96399999999999997,"5min":2.4929999999999999,"15min":2.4929999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.878,"15min":0.872},"min":{"1min":0.53200000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.97799999999999998,"5min":3.331,"15min":3.331},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.96599999999999997,"15min":0.93500000000000005},"min":{"1min":0.56399999999999995,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":0.98899999999999999,"5min":4.7720000000000002,"15min":4.7720000000000002},"last":0.69099999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:05:16 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.87,"15min":0.87},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.62}]},{"osd":8,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.93000000000000005,"5min":0.93600000000000005,"15min":0.93400000000000005},"min":{"1min":0.71799999999999997,"5min":0.61199999999999999,"15min":0.61199999999999999},"max":{"1min":1.171,"5min":2.661,"15min":2.661},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.88800000000000001,"5min":0.93500000000000005,"15min":0.97699999999999998},"min":{"1min":0.75700000000000001,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.0900000000000001,"5min":1.6679999999999999,"15min":1.6679999999999999},"last":0.87}]},{"osd":9,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.89300000000000002,"5min":1.032,"15min":1.006},"min":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.0780000000000001,"5min":6.4249999999999998,"15min":6.4249999999999998},"last":0.755},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.98599999999999999,"15min":1.016},"min":{"1min":0.55000000000000004,"5min":0.53500000000000003,"15min":0.53500000000000003},"max":{"1min":1.0449999999999999,"5min":3.2749999999999999,"15min":3.2749999999999999},"last":0.90800000000000003}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.93899999999999995,"5min":0.94299999999999995,"15min":0.94399999999999995},"min":{"1min":0.71499999999999997,"5min":0.65800000000000003,"15min":0.65800000000000003},"max":{"1min":1.0840000000000001,"5min":2.6339999999999999,"15min":2.6339999999999999},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.89100000000000001,"5min":0.878,"15min":0.876},"min":{"1min":0.69699999999999995,"5min":0.624,"15min":0.624},"max":{"1min":1.0660000000000001,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.93799999999999994}]}]},{"osd":9,"up_from":67,"seq":287762808877,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":456461,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.98399999999999999,"5min":0.63900000000000001,"15min":0.59199999999999997},"min":{"1min":0.33200000000000002,"5min":0.307,"15min":0.307},"max":{"1min":12.667999999999999,"5min":12.667999999999999,"15min":12.667999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.57599999999999996,"15min":0.59799999999999998},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":1.6779999999999999,"5min":1.6779999999999999,"15min":1.6779999999999999},"last":0.58999999999999997}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":0.79500000000000004,"15min":0.78100000000000003},"min":{"1min":0.47199999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":9.2650000000000006,"5min":9.2650000000000006,"15min":9.2650000000000006},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.752,"15min":0.77000000000000002},"min":{"1min":0.503,"5min":0.433,"15min":0.433},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.70899999999999996}]},{"osd":2,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":1.044,"5min":0.84499999999999997,"15min":0.83299999999999996},"min":{"1min":0.53900000000000003,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":9.4019999999999992,"5min":9.4019999999999992,"15min":9.4019999999999992},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.78900000000000003,"15min":0.79100000000000004},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.96,"5min":1.96,"15min":1.96},"last":1.032}]},{"osd":3,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.68200000000000005,"15min":0.70199999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":2.093,"5min":2.093,"15min":2.093},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.87,"5min":0.69299999999999995,"15min":0.68400000000000005},"min":{"1min":0.378,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":9.3130000000000006,"5min":9.3130000000000006,"15min":9.3130000000000006},"last":0.77500000000000002}]},{"osd":4,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.66900000000000004}]},{"osd":5,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.85399999999999998,"15min":0.876},"min":{"1min":0.58399999999999996,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.6220000000000001,"5min":1.637,"15min":1.637},"last":0.93100000000000005},{"interface":"front","average":{"1min":1.371,"5min":0.98799999999999999,"15min":0.94999999999999996},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":9.4749999999999996,"5min":9.4749999999999996,"15min":9.4749999999999996},"last":1.002}]},{"osd":6,"last update":"Fri Jul 21 19:05:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.84999999999999998,"15min":0.86899999999999999},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94899999999999995,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.86699999999999999,"15min":0.88300000000000001},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.1040000000000001,"5min":1.345,"15min":1.345},"last":0.66000000000000003}]},{"osd":7,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.92200000000000004,"15min":0.96199999999999997},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":1.552,"5min":1.552,"15min":1.552},"last":0.80600000000000005},{"interface":"front","average":{"1min":1.111,"5min":1.0189999999999999,"15min":1.0329999999999999},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":9.3740000000000006,"5min":9.3740000000000006,"15min":9.3740000000000006},"last":0.84199999999999997}]},{"osd":8,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.94899999999999995,"15min":0.97299999999999998},"min":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.97099999999999997,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.651,"5min":1.6859999999999999,"15min":1.6859999999999999},"last":1.018}]},{"osd":10,"last update":"Fri Jul 21 19:06:15 2023","interfaces":[{"interface":"back","average":{"1min":1.2310000000000001,"5min":1.0389999999999999,"15min":1.0069999999999999},"min":{"1min":0.67500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":9.3439999999999994,"5min":9.3439999999999994,"15min":9.3439999999999994},"last":0.747},{"interface":"front","average":{"1min":0.85799999999999998,"5min":1.014,"15min":1.04},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.88400000000000001}]},{"osd":11,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":1.0489999999999999},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"min":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.94999999999999996}]}]},{"osd":8,"up_from":59,"seq":253403070514,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":950272,"data_stored":455777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.61799999999999999,"15min":0.61599999999999999},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.95699999999999996,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.68500000000000005,"15min":0.66500000000000004},"min":{"1min":0.41799999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":4.1429999999999998,"5min":4.1429999999999998,"15min":4.1429999999999998},"last":0.65000000000000002}]},{"osd":1,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.69099999999999995,"15min":0.66000000000000003},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.94499999999999995},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71399999999999997,"15min":0.70699999999999996},"min":{"1min":0.496,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.036}]},{"osd":2,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.58499999999999996,"15min":0.57799999999999996},"min":{"1min":0.46999999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.874,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.58099999999999996,"15min":0.57299999999999995},"min":{"1min":0.44500000000000001,"5min":0.38,"15min":0.38},"max":{"1min":0.85699999999999998,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.89600000000000002}]},{"osd":3,"last update":"Fri Jul 21 19:06:26 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70899999999999996,"15min":0.72299999999999998},"min":{"1min":0.39500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":1.1870000000000001,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.72699999999999998,"15min":0.72399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.86799999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.749,"15min":0.73499999999999999},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":1.012},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.78000000000000003,"15min":0.76100000000000001},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.0920000000000001,"5min":1.169,"15min":1.169},"last":0.99199999999999999}]},{"osd":5,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.75800000000000001,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0669999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.95899999999999996},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82199999999999995,"15min":0.82599999999999996},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":1.056}]},{"osd":6,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":1.032,"5min":0.84699999999999998,"15min":0.82099999999999995},"min":{"1min":0.60399999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.2069999999999999,"5min":4.2069999999999999,"15min":4.2069999999999999},"last":1.1020000000000001},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.88,"15min":0.874},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":4.1310000000000002,"5min":4.1310000000000002,"15min":4.1310000000000002},"last":1.137}]},{"osd":7,"last update":"Fri Jul 21 19:06:27 2023","interfaces":[{"interface":"back","average":{"1min":0.86399999999999999,"5min":0.85099999999999998,"15min":0.85099999999999998},"min":{"1min":0.61099999999999999,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.157,"5min":1.403,"15min":1.403},"last":1.1739999999999999},{"interface":"front","average":{"1min":0.95899999999999996,"5min":0.90200000000000002,"15min":0.89600000000000002},"min":{"1min":0.65500000000000003,"5min":0.624,"15min":0.624},"max":{"1min":1.3160000000000001,"5min":1.3160000000000001,"15min":1.3160000000000001},"last":1.1870000000000001}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.1200000000000001,"5min":0.91700000000000004,"15min":0.86899999999999999},"min":{"1min":0.64700000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":4.3230000000000004,"5min":4.3230000000000004,"15min":4.3230000000000004},"last":1.2410000000000001},{"interface":"front","average":{"1min":0.997,"5min":0.86599999999999999,"15min":0.82599999999999996},"min":{"1min":0.73199999999999998,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.1719999999999999,"5min":1.238,"15min":1.238},"last":1.1559999999999999}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":0.96999999999999997,"15min":0.94099999999999995},"min":{"1min":0.69299999999999995,"5min":0.629,"15min":0.629},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.329},{"interface":"front","average":{"1min":1.083,"5min":0.93600000000000005,"15min":0.89200000000000002},"min":{"1min":0.90200000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.2949999999999999,"5min":1.2949999999999999,"15min":1.2949999999999999},"last":1.2609999999999999}]},{"osd":11,"last update":"Fri Jul 21 19:06:36 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.024,"15min":1.024},"min":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.76400000000000001},"max":{"1min":1.292,"5min":1.292,"15min":1.292},"last":1.288},{"interface":"front","average":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"min":{"1min":0.72999999999999998,"5min":0.72999999999999998,"15min":0.72999999999999998},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":1.204}]}]},{"osd":7,"up_from":51,"seq":219043332151,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27936,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717216,"statfs":{"total":95995035648,"available":95966429184,"internally_reserved":0,"allocated":1007616,"data_stored":1049327,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.66800000000000004,"15min":0.65300000000000002},"min":{"1min":0.42799999999999999,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67100000000000004,"15min":0.66300000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.76400000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.51500000000000001,"15min":0.51800000000000002},"min":{"1min":0.41499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.621,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54100000000000004,"15min":0.53600000000000003},"min":{"1min":0.39900000000000002,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.89400000000000002,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.47999999999999998}]},{"osd":2,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.69099999999999995,"15min":0.69699999999999995},"min":{"1min":0.44900000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.84899999999999998,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.73399999999999999,"15min":0.72399999999999998},"min":{"1min":0.59699999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91800000000000004,"5min":1.05,"15min":1.05},"last":0.80400000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.72899999999999998,"15min":0.70599999999999996},"min":{"1min":0.54800000000000004,"5min":0.432,"15min":0.432},"max":{"1min":0.96499999999999997,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69499999999999995,"15min":0.68100000000000005},"min":{"1min":0.45700000000000002,"5min":0.44,"15min":0.44},"max":{"1min":0.94699999999999995,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.63200000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64200000000000002,"15min":0.63},"min":{"1min":0.40600000000000003,"5min":0.379,"15min":0.379},"max":{"1min":0.83099999999999996,"5min":1.016,"15min":1.016},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60199999999999998,"15min":0.624},"min":{"1min":0.44800000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.84399999999999997,"5min":0.996,"15min":0.996},"last":0.77000000000000002}]},{"osd":5,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.877,"15min":1.0029999999999999},"min":{"1min":0.54200000000000004,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.98199999999999998,"5min":7.3289999999999997,"15min":7.3289999999999997},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.79800000000000004,"15min":0.77300000000000002},"min":{"1min":0.67900000000000005,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.036,"5min":1.1870000000000001,"15min":1.1870000000000001},"last":0.746}]},{"osd":6,"last update":"Fri Jul 21 19:07:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80000000000000004,"15min":0.80400000000000005},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.0069999999999999,"5min":2.1080000000000001,"15min":2.1080000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.749,"5min":0.81799999999999995,"15min":0.85199999999999998},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.002,"5min":2.1269999999999998,"15min":2.1269999999999998},"last":0.69899999999999995}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.82399999999999995,"15min":0.79600000000000004},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.115,"5min":1.234,"15min":1.234},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.81699999999999995,"15min":0.80500000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.1140000000000001,"5min":1.208,"15min":1.208},"last":0.60399999999999998}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82399999999999995,"15min":0.81899999999999995},"min":{"1min":0.58399999999999996,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.1719999999999999,"15min":1.1719999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.89200000000000002,"5min":0.89900000000000002,"15min":0.89700000000000002},"min":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.88500000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.84499999999999997,"15min":0.83899999999999997},"min":{"1min":0.622,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.079,"5min":1.095,"15min":1.095},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.93500000000000005,"15min":0.93799999999999994},"min":{"1min":0.72999999999999998,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.0660000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.93700000000000006}]},{"osd":11,"last update":"Fri Jul 21 19:06:41 2023","interfaces":[{"interface":"back","average":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"min":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"max":{"1min":1.165,"5min":1.165,"15min":1.165},"last":0.90100000000000002},{"interface":"front","average":{"1min":1.004,"5min":1.004,"15min":1.004},"min":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.91300000000000003}]}]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27876,"kb_used_data":924,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717276,"statfs":{"total":95995035648,"available":95966490624,"internally_reserved":0,"allocated":946176,"data_stored":455856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.51400000000000001,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":3.383,"5min":3.383,"15min":3.383},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.51800000000000002,"15min":0.498},"min":{"1min":0.35099999999999998,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":2.6110000000000002,"5min":2.6110000000000002,"15min":2.6110000000000002},"last":0.48499999999999999}]},{"osd":1,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.68899999999999995,"15min":0.65400000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.8650000000000002,"5min":3.8650000000000002,"15min":3.8650000000000002},"last":0.505},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.74199999999999999,"15min":0.70299999999999996},"min":{"1min":0.44900000000000001,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":3.4689999999999999,"5min":3.4689999999999999,"15min":3.4689999999999999},"last":0.67800000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.72799999999999998,"15min":0.70999999999999996},"min":{"1min":0.505,"5min":0.499,"15min":0.499},"max":{"1min":1.179,"5min":1.179,"15min":1.179},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.73099999999999998,"15min":0.71499999999999997},"min":{"1min":0.52300000000000002,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":3.956,"5min":3.956,"15min":3.956},"last":0.80600000000000005}]},{"osd":3,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.68600000000000005,"15min":0.61299999999999999},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":9.6099999999999994,"5min":9.6099999999999994,"15min":9.6099999999999994},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.88200000000000001,"5min":0.71699999999999997,"15min":0.67800000000000005},"min":{"1min":0.41399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":4.6189999999999998,"5min":4.6189999999999998,"15min":4.6189999999999998},"last":0.57399999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.03,"5min":0.81899999999999995,"15min":0.76700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":6.867,"5min":6.867,"15min":6.867},"last":0.61899999999999999},{"interface":"front","average":{"1min":1.163,"5min":0.85999999999999999,"15min":0.81899999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":6.9029999999999996,"5min":6.9029999999999996,"15min":6.9029999999999996},"last":0.75}]},{"osd":5,"last update":"Fri Jul 21 19:06:35 2023","interfaces":[{"interface":"back","average":{"1min":1.21,"5min":0.90800000000000003,"15min":0.84199999999999997},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":6.9589999999999996,"5min":6.9589999999999996,"15min":6.9589999999999996},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.82299999999999995,"15min":0.80900000000000005},"min":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"max":{"1min":1.2,"5min":1.8129999999999999,"15min":1.8129999999999999},"last":0.84499999999999997}]},{"osd":7,"last update":"Fri Jul 21 19:07:05 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.86199999999999999,"15min":0.82799999999999996},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2589999999999999,"5min":1.9570000000000001,"15min":1.9570000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":1.095,"5min":0.94899999999999995,"15min":0.87},"min":{"1min":0.64800000000000002,"5min":0.61799999999999999,"15min":0.61799999999999999},"max":{"1min":6.8840000000000003,"5min":6.8840000000000003,"15min":6.8840000000000003},"last":0.73099999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.88,"15min":0.86399999999999999},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.26,"5min":2.0070000000000001,"15min":2.0070000000000001},"last":0.90600000000000003},{"interface":"front","average":{"1min":1.004,"5min":0.93000000000000005,"15min":0.90800000000000003},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":3.609,"5min":3.609,"15min":3.609},"last":0.82899999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.016,"5min":0.94599999999999995,"15min":0.95099999999999996},"min":{"1min":0.53900000000000003,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":4.1879999999999997,"5min":4.1879999999999997,"15min":4.1879999999999997},"last":0.86099999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.91000000000000003,"15min":0.94199999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":2.2189999999999999,"5min":2.2189999999999999,"15min":2.2189999999999999},"last":0.81999999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.92000000000000004,"15min":0.93400000000000005},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.1539999999999999,"5min":3.9969999999999999,"15min":3.9969999999999999},"last":0.83799999999999997},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.0469999999999999,"15min":1.048},"min":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"max":{"1min":6.806,"5min":6.806,"15min":6.806},"last":0.878}]}]},{"osd":1,"up_from":13,"seq":55834574936,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27944,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717208,"statfs":{"total":95995035648,"available":95966420992,"internally_reserved":0,"allocated":1015808,"data_stored":519360,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.61599999999999999,"15min":0.60599999999999998},"min":{"1min":0.45700000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.98099999999999998,"5min":1.073,"15min":1.073},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.66600000000000004,"15min":0.63900000000000001},"min":{"1min":0.5,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":3.0779999999999998,"5min":3.0779999999999998,"15min":3.0779999999999998},"last":0.61299999999999999}]},{"osd":2,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.64700000000000002,"15min":0.625},"min":{"1min":0.45100000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.93999999999999995},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.70799999999999996,"15min":0.64200000000000002},"min":{"1min":0.56499999999999995,"5min":0.38600000000000001,"15min":0.36599999999999999},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":1.014}]},{"osd":3,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.70799999999999996,"15min":0.66600000000000004},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.40600000000000003},"max":{"1min":0.91400000000000003,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.70699999999999996,"15min":0.67800000000000005},"min":{"1min":0.47599999999999998,"5min":0.379,"15min":0.379},"max":{"1min":4.3150000000000004,"5min":4.3150000000000004,"15min":4.3150000000000004},"last":0.63700000000000001}]},{"osd":4,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.54700000000000004,"15min":0.51900000000000002},"min":{"1min":0.45600000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.58999999999999997,"15min":0.52000000000000002},"min":{"1min":0.48199999999999998,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.038,"5min":1.038,"15min":1.038},"last":0.58199999999999996}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.72599999999999998,"15min":0.66100000000000003},"min":{"1min":0.502,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.997},{"interface":"front","average":{"1min":0.94699999999999995,"5min":0.78400000000000003,"15min":0.70999999999999996},"min":{"1min":0.68300000000000005,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":1.079}]},{"osd":6,"last update":"Fri Jul 21 19:06:32 2023","interfaces":[{"interface":"back","average":{"1min":0.90000000000000002,"5min":0.79100000000000004,"15min":0.753},"min":{"1min":0.67100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.101,"5min":1.101,"15min":1.101},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.746,"15min":0.71599999999999997},"min":{"1min":0.629,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.749}]},{"osd":7,"last update":"Fri Jul 21 19:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.76200000000000001,"15min":0.72999999999999998},"min":{"1min":0.57799999999999996,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.73499999999999999,"15min":0.69499999999999995},"min":{"1min":0.45100000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0229999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.91600000000000004}]},{"osd":8,"last update":"Fri Jul 21 19:06:24 2023","interfaces":[{"interface":"back","average":{"1min":0.91700000000000004,"5min":0.84899999999999998,"15min":0.82699999999999996},"min":{"1min":0.67400000000000004,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.81599999999999995,"15min":0.82199999999999995},"min":{"1min":0.61499999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.0269999999999999,"5min":1.502,"15min":1.502},"last":0.86599999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":1.036,"5min":0.91400000000000003,"15min":0.89700000000000002},"min":{"1min":0.71699999999999997,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.25,"5min":1.25,"15min":1.25},"last":1.113},{"interface":"front","average":{"1min":1.1120000000000001,"5min":0.94399999999999995,"15min":0.91800000000000004},"min":{"1min":0.76300000000000001,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":3.0470000000000002,"5min":3.0470000000000002,"15min":3.0470000000000002},"last":1.0980000000000001}]},{"osd":10,"last update":"Fri Jul 21 19:07:21 2023","interfaces":[{"interface":"back","average":{"1min":1.0589999999999999,"5min":0.95599999999999996,"15min":0.93999999999999995},"min":{"1min":0.78500000000000003,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.2969999999999999,"5min":1.2969999999999999,"15min":1.2969999999999999},"last":1.169},{"interface":"front","average":{"1min":1.024,"5min":0.96999999999999997,"15min":0.96499999999999997},"min":{"1min":0.76400000000000001,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":1.129}]},{"osd":11,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":1.052},{"interface":"front","average":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"min":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"max":{"1min":1.274,"5min":1.274,"15min":1.274},"last":1.1419999999999999}]}]},{"osd":0,"up_from":8,"seq":34359738461,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27896,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717256,"statfs":{"total":95995035648,"available":95966470144,"internally_reserved":0,"allocated":966656,"data_stored":457121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.68300000000000005,"15min":0.63500000000000001},"min":{"1min":0.46300000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1539999999999999,"5min":1.177,"15min":1.177},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.67400000000000004,"15min":0.628},"min":{"1min":0.53800000000000003,"5min":0.44,"15min":0.39300000000000002},"max":{"1min":0.92100000000000004,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.71799999999999997}]},{"osd":2,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.68000000000000005,"15min":0.65000000000000002},"min":{"1min":0.53200000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.204,"5min":1.204,"15min":1.204},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.45000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":1.397,"5min":1.397,"15min":1.397},"last":0.495}]},{"osd":3,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":2.2080000000000002,"5min":2.2080000000000002,"15min":2.2080000000000002},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":2.2469999999999999,"5min":2.2469999999999999,"15min":2.2469999999999999},"last":0.79100000000000004}]},{"osd":4,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65800000000000003,"15min":0.63},"min":{"1min":0.46300000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.3620000000000001,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66200000000000003,"15min":0.64200000000000002},"min":{"1min":0.498,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.85899999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.76000000000000001}]},{"osd":5,"last update":"Fri Jul 21 19:07:12 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.77300000000000002,"15min":0.73899999999999999},"min":{"1min":0.47199999999999998,"5min":0.374,"15min":0.374},"max":{"1min":2.944,"5min":2.944,"15min":2.944},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.69599999999999995},"min":{"1min":0.51300000000000001,"5min":0.435,"15min":0.435},"max":{"1min":0.92900000000000005,"5min":1.4610000000000001,"15min":1.4610000000000001},"last":0.56299999999999994}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.65000000000000002,"15min":0.64500000000000002},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.7530000000000001,"5min":2.7530000000000001,"15min":2.7530000000000001},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.70899999999999996,"15min":0.70299999999999996},"min":{"1min":0.49099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":2.2810000000000001,"5min":2.2810000000000001,"15min":2.2810000000000001},"last":0.73299999999999998}]},{"osd":7,"last update":"Fri Jul 21 19:04:56 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.82299999999999995,"15min":0.81299999999999994},"min":{"1min":0.57499999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79000000000000004,"15min":0.78400000000000003},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.0900000000000001,"5min":1.1970000000000001,"15min":1.1970000000000001},"last":0.73299999999999998}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.878,"15min":0.91500000000000004},"min":{"1min":0.68799999999999994,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.244,"5min":1.304,"15min":1.304},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.82399999999999995,"15min":0.84599999999999997},"min":{"1min":0.45700000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.61699999999999999}]},{"osd":9,"last update":"Fri Jul 21 19:06:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.497,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89000000000000001,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.88100000000000001,"15min":0.93200000000000005},"min":{"1min":0.55600000000000005,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":1.3109999999999999,"5min":1.3109999999999999,"15min":1.3109999999999999},"last":0.68799999999999994}]},{"osd":10,"last update":"Fri Jul 21 19:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.84899999999999998,"15min":0.84999999999999998},"min":{"1min":0.57599999999999996,"5min":0.435,"15min":0.435},"max":{"1min":3.3479999999999999,"5min":3.3479999999999999,"15min":3.3479999999999999},"last":0.88200000000000001},{"interface":"front","average":{"1min":1.034,"5min":0.88500000000000001,"15min":0.86699999999999999},"min":{"1min":0.505,"5min":0.505,"15min":0.505},"max":{"1min":5.9950000000000001,"5min":5.9950000000000001,"15min":5.9950000000000001},"last":0.81499999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.82799999999999996,"15min":0.81999999999999995},"min":{"1min":0.68799999999999994,"5min":0.59799999999999998,"15min":0.59799999999999998},"max":{"1min":1.5229999999999999,"5min":1.5229999999999999,"15min":1.5229999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.85299999999999998,"5min":0.79700000000000004,"15min":0.78700000000000003},"min":{"1min":0.73599999999999999,"5min":0.58799999999999997,"15min":0.58799999999999997},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.84799999999999998}]}]},{"osd":2,"up_from":18,"seq":77309411410,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27952,"kb_used_data":1000,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717200,"statfs":{"total":95995035648,"available":95966412800,"internally_reserved":0,"allocated":1024000,"data_stored":515186,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.61699999999999999,"15min":0.59899999999999998},"min":{"1min":0.39500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.63300000000000001,"15min":0.59099999999999997},"min":{"1min":0.42199999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.69199999999999995,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.95599999999999996,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.91900000000000004},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.68100000000000005,"15min":0.61599999999999999},"min":{"1min":0.52800000000000002,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.92900000000000005,"5min":1.081,"15min":1.081},"last":0.754}]},{"osd":3,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71099999999999997,"15min":0.67300000000000004},"min":{"1min":0.41299999999999998,"5min":0.38,"15min":0.38},"max":{"1min":1.204,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.2430000000000001,"5min":1.2430000000000001,"15min":1.2430000000000001},"last":0.57099999999999995}]},{"osd":4,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.876,"15min":0.876},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.93700000000000006}]},{"osd":5,"last update":"Fri Jul 21 19:07:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.60299999999999998,"15min":0.53700000000000003},"min":{"1min":0.46600000000000003,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.94799999999999995,"5min":1.026,"15min":1.026},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61299999999999999,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.90200000000000002,"5min":1.167,"15min":1.167},"last":0.625}]},{"osd":6,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.77100000000000002,"15min":0.755},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.88},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.73799999999999999,"15min":0.72199999999999998},"min":{"1min":0.61299999999999999,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.73999999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.76300000000000001,"15min":0.76700000000000002},"min":{"1min":0.56999999999999995,"5min":0.502,"15min":0.502},"max":{"1min":1.083,"5min":1.083,"15min":1.083},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.995,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.63100000000000001}]},{"osd":8,"last update":"Fri Jul 21 19:06:29 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78200000000000003,"15min":0.78700000000000003},"min":{"1min":0.60799999999999998,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.98499999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.748,"15min":0.73099999999999998},"min":{"1min":0.61799999999999999,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.67900000000000005}]},{"osd":9,"last update":"Fri Jul 21 19:06:52 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.85299999999999998,"15min":0.85299999999999998},"min":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"max":{"1min":1.218,"5min":1.333,"15min":1.333},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.88700000000000001,"15min":0.877},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.288,"5min":1.288,"15min":1.288},"last":0.82099999999999995}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":1.04,"5min":0.88400000000000001,"15min":0.85799999999999998},"min":{"1min":0.69799999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":3.0939999999999999,"5min":3.0939999999999999,"15min":3.0939999999999999},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.90900000000000003,"5min":0.89500000000000002,"15min":0.89300000000000002},"min":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"max":{"1min":1.2110000000000001,"5min":1.2110000000000001,"15min":1.2110000000000001},"last":0.70699999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"min":{"1min":0.745,"5min":0.745,"15min":0.745},"max":{"1min":1.129,"5min":1.129,"15min":1.129},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.81200000000000006}]}]},{"osd":3,"up_from":25,"seq":107374182477,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":962560,"data_stored":456173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46800000000000003,"15min":0.42199999999999999},"min":{"1min":0.29799999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.70999999999999996,"5min":0.95699999999999996,"15min":0.95699999999999996},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48699999999999999,"15min":0.44800000000000001},"min":{"1min":0.34200000000000003,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.58199999999999996}]},{"osd":1,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.52000000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.873,"5min":1.0720000000000001,"15min":1.0720000000000001},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.66700000000000004,"15min":0.65000000000000002},"min":{"1min":0.44900000000000001,"5min":0.437,"15min":0.437},"max":{"1min":0.86599999999999999,"5min":1.048,"15min":1.048},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Jul 21 19:07:15 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.74299999999999999,"15min":0.65300000000000002},"min":{"1min":0.56200000000000006,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.3440000000000001,"5min":1.3440000000000001,"15min":1.3440000000000001},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.71699999999999997,"15min":0.64500000000000002},"min":{"1min":0.49099999999999999,"5min":0.44600000000000001,"15min":0.42099999999999999},"max":{"1min":0.98299999999999998,"5min":1.048,"15min":1.048},"last":0.86399999999999999}]},{"osd":4,"last update":"Fri Jul 21 19:06:42 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.70199999999999996,"15min":0.66000000000000003},"min":{"1min":0.46999999999999997,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.94699999999999995,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.74399999999999999,"15min":0.68200000000000005},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.73299999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:11 2023","interfaces":[{"interface":"back","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.68400000000000005},"min":{"1min":0.48799999999999999,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.1020000000000001,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.75700000000000001,"15min":0.69899999999999995},"min":{"1min":0.53300000000000003,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.94199999999999995}]},{"osd":6,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.68400000000000005,"15min":0.66000000000000003},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.92500000000000004,"5min":1.264,"15min":1.264},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.0649999999999999,"5min":1.141,"15min":1.141},"last":0.61099999999999999}]},{"osd":7,"last update":"Fri Jul 21 19:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77900000000000003,"15min":0.78200000000000003},"min":{"1min":0.61099999999999999,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":0.82599999999999996},"min":{"1min":0.67500000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.2070000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.876}]},{"osd":8,"last update":"Fri Jul 21 19:05:20 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.878,"15min":0.88800000000000001},"min":{"1min":0.67700000000000005,"5min":0.67300000000000004,"15min":0.67300000000000004},"max":{"1min":1.0800000000000001,"5min":1.224,"15min":1.224},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.85099999999999998,"15min":0.86399999999999999},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":1.0409999999999999,"5min":1.21,"15min":1.21},"last":0.70599999999999996}]},{"osd":9,"last update":"Fri Jul 21 19:06:51 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.89900000000000002,"15min":0.93799999999999994},"min":{"1min":0.505,"5min":0.47999999999999998,"15min":0.47999999999999998},"max":{"1min":1.2030000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.84699999999999998,"15min":0.89700000000000002},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.151,"5min":1.196,"15min":1.196},"last":0.71599999999999997}]},{"osd":10,"last update":"Fri Jul 21 19:06:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.93500000000000005,"15min":0.94299999999999995},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.54400000000000004},"max":{"1min":1.091,"5min":1.153,"15min":1.153},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.91600000000000004},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.2689999999999999,"5min":1.2689999999999999,"15min":1.2689999999999999},"last":0.81599999999999995}]},{"osd":11,"last update":"Fri Jul 21 19:06:37 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.89800000000000002}]}]},{"osd":4,"up_from":33,"seq":141733920839,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":458310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:04:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.61199999999999999,"15min":0.58399999999999996},"min":{"1min":0.435,"5min":0.433,"15min":0.433},"max":{"1min":0.89800000000000002,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.624,"5min":0.59999999999999998,"15min":0.57899999999999996},"min":{"1min":0.434,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.872,"5min":1.1080000000000001,"15min":1.1080000000000001},"last":0.41699999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.443},"min":{"1min":0.311,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.67100000000000004,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.48799999999999999,"15min":0.44600000000000001},"min":{"1min":0.372,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.55400000000000005}]},{"osd":2,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.73799999999999999,"15min":0.67200000000000004},"min":{"1min":0.55100000000000005,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95999999999999996,"5min":1.167,"15min":1.167},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69999999999999996,"15min":0.68000000000000005},"min":{"1min":0.52300000000000002,"5min":0.436,"15min":0.436},"max":{"1min":1.0009999999999999,"5min":1.0009999999999999,"15min":1.0009999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Jul 21 19:06:44 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.76900000000000002,"15min":0.66900000000000004},"min":{"1min":0.48599999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":4.7140000000000004,"5min":4.7140000000000004,"15min":4.7140000000000004},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66700000000000004,"15min":0.63700000000000001},"min":{"1min":0.46200000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.97199999999999998,"5min":1.079,"15min":1.079},"last":0.60699999999999998}]},{"osd":5,"last update":"Fri Jul 21 19:07:10 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76200000000000001,"15min":0.748},"min":{"1min":0.54500000000000004,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0649999999999999,"5min":1.0649999999999999,"15min":1.0649999999999999},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.78200000000000003,"15min":0.74399999999999999},"min":{"1min":0.52100000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.97799999999999998,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.83499999999999996}]},{"osd":6,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":1.0189999999999999,"5min":0.82999999999999996,"15min":0.79700000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":4.6260000000000003,"5min":4.6260000000000003,"15min":4.6260000000000003},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.754,"5min":0.78500000000000003,"15min":0.77700000000000002},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":1.198,"5min":1.198,"15min":1.198},"last":0.69199999999999995}]},{"osd":7,"last update":"Fri Jul 21 19:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.71399999999999997,"15min":0.67200000000000004},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.94099999999999995,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.754,"15min":0.69999999999999996},"min":{"1min":0.51300000000000001,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.008,"5min":1.2669999999999999,"15min":1.2669999999999999},"last":0.81000000000000005}]},{"osd":8,"last update":"Fri Jul 21 19:06:30 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.88,"15min":0.877},"min":{"1min":0.58399999999999996,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.1599999999999999,"5min":1.1599999999999999,"15min":1.1599999999999999},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.90100000000000002,"15min":0.88900000000000001},"min":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"max":{"1min":1.173,"5min":1.298,"15min":1.298},"last":0.89800000000000002}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.81699999999999995,"15min":0.82699999999999996},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2230000000000001,"5min":1.2230000000000001,"15min":1.2230000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.91000000000000003,"5min":0.93200000000000005,"15min":0.94599999999999995},"min":{"1min":0.60099999999999998,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.252,"5min":1.252,"15min":1.252},"last":0.91600000000000004}]},{"osd":10,"last update":"Fri Jul 21 19:07:18 2023","interfaces":[{"interface":"back","average":{"1min":0.90300000000000002,"5min":2.2120000000000002,"15min":2.7999999999999998},"min":{"1min":0.69699999999999995,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.145,"5min":42.396000000000001,"15min":42.396000000000001},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.73299999999999998,"5min":2.105,"15min":2.7160000000000002},"min":{"1min":0.502,"5min":0.502,"15min":0.502},"max":{"1min":0.91800000000000004,"5min":42.792000000000002,"15min":42.792000000000002},"last":0.70799999999999996}]},{"osd":11,"last update":"Fri Jul 21 19:06:40 2023","interfaces":[{"interface":"back","average":{"1min":0.92700000000000005,"5min":0.92700000000000005,"15min":0.92700000000000005},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.97499999999999998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"max":{"1min":1.2130000000000001,"5min":1.2130000000000001,"15min":1.2130000000000001},"last":0.99099999999999999}]}]},{"osd":5,"up_from":38,"seq":163208757314,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27948,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717204,"statfs":{"total":95995035648,"available":95966416896,"internally_reserved":0,"allocated":1019904,"data_stored":1048576,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.63600000000000001,"15min":0.61399999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.90200000000000002,"5min":1.042,"15min":1.042},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.629,"15min":0.59499999999999997},"min":{"1min":0.42399999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.877,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.60299999999999998}]},{"osd":1,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.67400000000000004,"15min":0.68400000000000005},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.84699999999999998,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.71799999999999997,"15min":0.68999999999999995},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.78800000000000003}]},{"osd":2,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54000000000000004,"15min":0.505},"min":{"1min":0.40799999999999997,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.35899999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.71699999999999997}]},{"osd":3,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.72299999999999998,"15min":0.69999999999999996},"min":{"1min":0.57499999999999996,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.66100000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"last":0.57799999999999996}]},{"osd":4,"last update":"Fri Jul 21 19:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.75900000000000001,"15min":0.72899999999999998},"min":{"1min":0.61499999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.72399999999999998,"15min":0.73199999999999998},"min":{"1min":0.51700000000000002,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.91200000000000003,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.74199999999999999}]},{"osd":6,"last update":"Fri Jul 21 19:06:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.76400000000000001,"15min":0.751},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.73399999999999999,"15min":0.745},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.92900000000000005,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Fri Jul 21 19:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.81100000000000005,"15min":0.80300000000000005},"min":{"1min":0.501,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.135,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.81699999999999995,"15min":0.80400000000000005},"min":{"1min":0.71399999999999997,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":1.1499999999999999,"5min":1.163,"15min":1.163},"last":1.006}]},{"osd":8,"last update":"Fri Jul 21 19:06:28 2023","interfaces":[{"interface":"back","average":{"1min":0.80800000000000005,"5min":0.752,"15min":0.73799999999999999},"min":{"1min":0.505,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1080000000000001,"5min":1.111,"15min":1.111},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.76100000000000001,"15min":0.749},"min":{"1min":0.51100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.091,"5min":1.103,"15min":1.103},"last":0.91400000000000003}]},{"osd":9,"last update":"Fri Jul 21 19:06:50 2023","interfaces":[{"interface":"back","average":{"1min":0.90100000000000002,"5min":0.88,"15min":0.88400000000000001},"min":{"1min":0.73399999999999999,"5min":0.64800000000000002,"15min":0.64800000000000002},"max":{"1min":1.139,"5min":1.139,"15min":1.139},"last":1.048},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.80500000000000005,"15min":0.81899999999999995},"min":{"1min":0.59799999999999998,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.995,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.80400000000000005}]},{"osd":10,"last update":"Fri Jul 21 19:07:17 2023","interfaces":[{"interface":"back","average":{"1min":0.94899999999999995,"5min":0.90600000000000003,"15min":0.89600000000000002},"min":{"1min":0.66900000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.161,"5min":1.4970000000000001,"15min":1.4970000000000001},"last":0.84399999999999997},{"interface":"front","average":{"1min":0.97899999999999998,"5min":0.90100000000000002,"15min":0.875},"min":{"1min":0.74199999999999999,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.1819999999999999,"5min":1.3100000000000001,"15min":1.3100000000000001},"last":1.095}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34584,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":36226,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":63717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":63863,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3193,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":60828,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":123,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-07-21T19:07:23.091 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 -- 172.21.15.5:0/460379541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c101680 msgr2=0x7f285c107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 --2- 172.21.15.5:0/460379541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c101680 0x7f285c107f40 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f285400b0a0 tx=0x7f285400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 -- 172.21.15.5:0/460379541 shutdown_connections 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 --2- 172.21.15.5:0/460379541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c101680 0x7f285c107f40 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 --2- 172.21.15.5:0/460379541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 0x7f285c0fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 --2- 172.21.15.5:0/460379541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c100cf0 0x7f285c101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 -- 172.21.15.5:0/460379541 >> 172.21.15.5:0/460379541 conn(0x7f285c0fb0e0 msgr2=0x7f285c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 -- 172.21.15.5:0/460379541 shutdown_connections 2023-07-21T19:07:23.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.315+0000 7f28635ff700 1 -- 172.21.15.5:0/460379541 wait complete. 2023-07-21T19:07:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 Processor -- start 2023-07-21T19:07:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 -- start start 2023-07-21T19:07:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 0x7f285c195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 0x7f285c19ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19d400 con 0x7f285c100cf0 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19d540 con 0x7f285c101680 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.317+0000 7f28635ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f285c19d6d0 con 0x7f285c0ffb00 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48014/0 (socket says 172.21.15.5:48014) 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 -- 172.21.15.5:0/1307513918 learned_addr learned my addr 172.21.15.5:0/1307513918 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f286139b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 0x7f285c195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2861b9c700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 0x7f285c19ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 -- 172.21.15.5:0/1307513918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 msgr2=0x7f285c195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 0x7f285c195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 -- 172.21.15.5:0/1307513918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 msgr2=0x7f285c19ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 0x7f285c19ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 -- 172.21.15.5:0/1307513918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2854009d00 con 0x7f285c100cf0 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2861b9c700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 0x7f285c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.318+0000 7f2860b9a700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9a0 tx=0x7f284c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.319+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c009790 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.319+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f285c19d9b0 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.319+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f284c00cbd0 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.319+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f284c018680 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.319+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f285c19df10 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.320+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f285c10b1e0 con 0x7f285c100cf0 2023-07-21T19:07:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.322+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f284c00cd40 con 0x7f285c100cf0 2023-07-21T19:07:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.323+0000 7f28527fc700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 0x7f28480a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.323+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f284c014070 con 0x7f285c100cf0 2023-07-21T19:07:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.324+0000 7f286139b700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 0x7f28480a3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.324+0000 7f286139b700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 0x7f28480a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f285c100b50 tx=0x7f2858009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.327+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f284c051790 con 0x7f285c100cf0 2023-07-21T19:07:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.540+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f285c1069a0 con 0x7f285c100cf0 2023-07-21T19:07:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.542+0000 7f28527fc700 1 -- 172.21.15.5:0/1307513918 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f284c096300 con 0x7f285c100cf0 2023-07-21T19:07:23.552 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:23.553 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 msgr2=0x7f28480a3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 0x7f28480a3650 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f285c100b50 tx=0x7f2858009260 comp rx=0 tx=0).stop 2023-07-21T19:07:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 msgr2=0x7f285c195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f284c00e9a0 tx=0x7f284c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 shutdown_connections 2023-07-21T19:07:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f28480a11a0 0x7f28480a3650 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f285c101680 0x7f285c19ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f285c0ffb00 0x7f285c195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 --2- 172.21.15.5:0/1307513918 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f285c100cf0 0x7f285c195dc0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.544+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 >> 172.21.15.5:0/1307513918 conn(0x7f285c0fb0e0 msgr2=0x7f285c0fd2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.545+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 shutdown_connections 2023-07-21T19:07:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.545+0000 7f28635ff700 1 -- 172.21.15.5:0/1307513918 wait complete. 2023-07-21T19:07:23.573 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:07:23.573 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:07:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:23 smithi101 ceph-mon[128136]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:23 smithi101 ceph-mon[128136]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.636 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:23 smithi005 ceph-mon[126460]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.636 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:23 smithi005 ceph-mon[126460]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:23 smithi156 ceph-mon[128316]: from='client.15399 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:23 smithi156 ceph-mon[128316]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:23.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 -- 172.21.15.5:0/775730982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c100c60 msgr2=0x7fa43c1010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 --2- 172.21.15.5:0/775730982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c100c60 0x7fa43c1010d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4300098f0 tx=0x7fa430009c00 comp rx=0 tx=0).stop 2023-07-21T19:07:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 -- 172.21.15.5:0/775730982 shutdown_connections 2023-07-21T19:07:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 --2- 172.21.15.5:0/775730982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c101610 0x7fa43c107eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 --2- 172.21.15.5:0/775730982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c0ff9d0 0x7fa43c0ffde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 --2- 172.21.15.5:0/775730982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c100c60 0x7fa43c1010d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 -- 172.21.15.5:0/775730982 >> 172.21.15.5:0/775730982 conn(0x7fa43c0fb010 msgr2=0x7fa43c0fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 -- 172.21.15.5:0/775730982 shutdown_connections 2023-07-21T19:07:23.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.772+0000 7fa44159d700 1 -- 172.21.15.5:0/775730982 wait complete. 2023-07-21T19:07:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.773+0000 7fa44159d700 1 Processor -- start 2023-07-21T19:07:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 -- start start 2023-07-21T19:07:23.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c0ff9d0 0x7fa43c197760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c101610 0x7fa43c19ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19f2e0 con 0x7fa43c100c60 2023-07-21T19:07:23.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19f420 con 0x7fa43c101610 2023-07-21T19:07:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa44159d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa43c19f5b0 con 0x7fa43c0ff9d0 2023-07-21T19:07:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa43b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa43b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48034/0 (socket says 172.21.15.5:48034) 2023-07-21T19:07:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa43b7fe700 1 -- 172.21.15.5:0/1793522939 learned_addr learned my addr 172.21.15.5:0/1793522939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:23.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa43bfff700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c0ff9d0 0x7fa43c197760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.774+0000 7fa440d9c700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c101610 0x7fa43c19ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 -- 172.21.15.5:0/1793522939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c0ff9d0 msgr2=0x7fa43c197760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c0ff9d0 0x7fa43c197760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 -- 172.21.15.5:0/1793522939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c101610 msgr2=0x7fa43c19ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c101610 0x7fa43c19ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:23.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 -- 172.21.15.5:0/1793522939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa434009d00 con 0x7fa43c100c60 2023-07-21T19:07:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa43b7fe700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa430009fd0 tx=0x7fa43000b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa43001c070 con 0x7fa43c100c60 2023-07-21T19:07:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa43000f7b0 con 0x7fa43c100c60 2023-07-21T19:07:23.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430004a50 con 0x7fa43c100c60 2023-07-21T19:07:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa430009520 con 0x7fa43c100c60 2023-07-21T19:07:23.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.775+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43c19fa70 con 0x7fa43c100c60 2023-07-21T19:07:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.776+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa43c10b0a0 con 0x7fa43c100c60 2023-07-21T19:07:23.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.778+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fa430020070 con 0x7fa43c100c60 2023-07-21T19:07:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.780+0000 7fa4397fa700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 0x7fa4240a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.780+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fa4300cd690 con 0x7fa43c100c60 2023-07-21T19:07:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.780+0000 7fa43bfff700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 0x7fa4240a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.781+0000 7fa43bfff700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 0x7fa4240a3720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009c60 tx=0x7fa42c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:23.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.783+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa430096e20 con 0x7fa43c100c60 2023-07-21T19:07:24.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.997+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa43c1a00b0 con 0x7fa43c100c60 2023-07-21T19:07:24.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:23.998+0000 7fa4397fa700 1 -- 172.21.15.5:0/1793522939 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v87) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa43009a440 con 0x7fa43c100c60 2023-07-21T19:07:24.008 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:07:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.000+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 msgr2=0x7fa4240a3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.000+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 0x7fa4240a3720 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fa42c009c60 tx=0x7fa42c009260 comp rx=0 tx=0).stop 2023-07-21T19:07:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.000+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 msgr2=0x7fa43c197ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.000+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fa430009fd0 tx=0x7fa43000b790 comp rx=0 tx=0).stop 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 shutdown_connections 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fa4240a1270 0x7fa4240a3720 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa43c101610 0x7fa43c19ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa43c0ff9d0 0x7fa43c197760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 --2- 172.21.15.5:0/1793522939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa43c100c60 0x7fa43c197ca0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 >> 172.21.15.5:0/1793522939 conn(0x7fa43c0fb010 msgr2=0x7fa43c0fd260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 shutdown_connections 2023-07-21T19:07:24.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.001+0000 7fa44159d700 1 -- 172.21.15.5:0/1793522939 wait complete. 2023-07-21T19:07:24.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:07:24.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 -- 172.21.15.5:0/681349126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 msgr2=0x7f0598109e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 --2- 172.21.15.5:0/681349126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f0598109e20 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f059000b0a0 tx=0x7f059000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 -- 172.21.15.5:0/681349126 shutdown_connections 2023-07-21T19:07:24.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 --2- 172.21.15.5:0/681349126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f0598109e20 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 --2- 172.21.15.5:0/681349126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 0x7f05980729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 --2- 172.21.15.5:0/681349126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f05980733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 -- 172.21.15.5:0/681349126 >> 172.21.15.5:0/681349126 conn(0x7f05980fb0e0 msgr2=0x7f05980fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:24.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 -- 172.21.15.5:0/681349126 shutdown_connections 2023-07-21T19:07:24.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.226+0000 7f059f7ff700 1 -- 172.21.15.5:0/681349126 wait complete. 2023-07-21T19:07:24.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.227+0000 7f059f7ff700 1 Processor -- start 2023-07-21T19:07:24.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 -- start start 2023-07-21T19:07:24.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f0598197a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 0x7f0598197fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059819f5f0 con 0x7f05981079a0 2023-07-21T19:07:24.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059819f730 con 0x7f0598072f70 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059f7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059819f8c0 con 0x7f0598074550 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.228+0000 7f059dd9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48064/0 (socket says 172.21.15.5:48064) 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 -- 172.21.15.5:0/3316690707 learned_addr learned my addr 172.21.15.5:0/3316690707 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059cd9a700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 0x7f0598197fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059d59b700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f0598197a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 -- 172.21.15.5:0/3316690707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 msgr2=0x7f0598197fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 0x7f0598197fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 -- 172.21.15.5:0/3316690707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 msgr2=0x7f0598197a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f0598197a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 -- 172.21.15.5:0/3316690707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009d00 con 0x7f05981079a0 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059d59b700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f0598197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:24.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059dd9c700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f059000d010 tx=0x7f059000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000e040 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059819fb40 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0590003e10 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.229+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05900053c0 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.230+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059819ff10 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.231+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0598106050 con 0x7f05981079a0 2023-07-21T19:07:24.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.232+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f059001db10 con 0x7f05981079a0 2023-07-21T19:07:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.234+0000 7f058e7fc700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 0x7f05840a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.234+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f05900cddc0 con 0x7f05981079a0 2023-07-21T19:07:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.234+0000 7f059d59b700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 0x7f05840a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.235+0000 7f059d59b700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 0x7f05840a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0594005fd0 tx=0x7f0594005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:24.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.238+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f05900975b0 con 0x7f05981079a0 2023-07-21T19:07:24.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.449+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f059810d390 con 0x7f05981079a0 2023-07-21T19:07:24.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.449+0000 7f058e7fc700 1 -- 172.21.15.5:0/3316690707 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v87) v1 ==== 104+0+11 (secure 0 0 0) 0x7f059009abd0 con 0x7f05981079a0 2023-07-21T19:07:24.460 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:07:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 msgr2=0x7f05840a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 0x7f05840a3600 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f0594005fd0 tx=0x7f0594005ee0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 msgr2=0x7f059819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f059000d010 tx=0x7f059000bd70 comp rx=0 tx=0).stop 2023-07-21T19:07:24.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 shutdown_connections 2023-07-21T19:07:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f05840a1150 0x7f05840a3600 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05981079a0 0x7f059819f020 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0598072f70 0x7f0598197a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 --2- 172.21.15.5:0/3316690707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0598074550 0x7f0598197fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 >> 172.21.15.5:0/3316690707 conn(0x7f05980fb0e0 msgr2=0x7f05980fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.451+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 shutdown_connections 2023-07-21T19:07:24.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.452+0000 7f059f7ff700 1 -- 172.21.15.5:0/3316690707 wait complete. 2023-07-21T19:07:24.479 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:07:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:24 smithi005 ceph-mon[126460]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:24 smithi005 ceph-mon[126460]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T19:07:24.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:24 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1307513918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:24.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:24 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1793522939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:24 smithi156 ceph-mon[128316]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:24 smithi156 ceph-mon[128316]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T19:07:24.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:24 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1307513918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:24.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:24 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1793522939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.677+0000 7f23d35ff700 1 -- 172.21.15.5:0/2751409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 msgr2=0x7f23cc101120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.677+0000 7f23d35ff700 1 --2- 172.21.15.5:0/2751409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc101120 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f23bc009990 tx=0x7f23bc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 -- 172.21.15.5:0/2751409856 shutdown_connections 2023-07-21T19:07:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 --2- 172.21.15.5:0/2751409856 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc101660 0x7f23cc107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 --2- 172.21.15.5:0/2751409856 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc0ffae0 0x7f23cc0ffef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 --2- 172.21.15.5:0/2751409856 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc101120 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 -- 172.21.15.5:0/2751409856 >> 172.21.15.5:0/2751409856 conn(0x7f23cc0fb0e0 msgr2=0x7f23cc0fd530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 -- 172.21.15.5:0/2751409856 shutdown_connections 2023-07-21T19:07:24.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.678+0000 7f23d35ff700 1 -- 172.21.15.5:0/2751409856 wait complete. 2023-07-21T19:07:24.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.679+0000 7f23d35ff700 1 Processor -- start 2023-07-21T19:07:24.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.679+0000 7f23d35ff700 1 -- start start 2023-07-21T19:07:24.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc0ffae0 0x7f23cc1979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc101660 0x7f23cc19ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc19f530 con 0x7f23cc100cd0 2023-07-21T19:07:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc19f670 con 0x7f23cc0ffae0 2023-07-21T19:07:24.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d35ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23cc19f800 con 0x7f23cc101660 2023-07-21T19:07:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d0b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d0b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48078/0 (socket says 172.21.15.5:48078) 2023-07-21T19:07:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d0b9a700 1 -- 172.21.15.5:0/477298284 learned_addr learned my addr 172.21.15.5:0/477298284 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:24.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d139b700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc0ffae0 0x7f23cc1979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.680+0000 7f23d1b9c700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc101660 0x7f23cc19ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 -- 172.21.15.5:0/477298284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc101660 msgr2=0x7f23cc19ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc101660 0x7f23cc19ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 -- 172.21.15.5:0/477298284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc0ffae0 msgr2=0x7f23cc1979b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc0ffae0 0x7f23cc1979b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 -- 172.21.15.5:0/477298284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23bc0095f0 con 0x7f23cc100cd0 2023-07-21T19:07:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d0b9a700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005910 tx=0x7f23bc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc01d070 con 0x7f23cc100cd0 2023-07-21T19:07:24.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23cc19fa80 con 0x7f23cc100cd0 2023-07-21T19:07:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f23bc003fc0 con 0x7f23cc100cd0 2023-07-21T19:07:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc022c60 con 0x7f23cc100cd0 2023-07-21T19:07:24.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.681+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23cc19ff20 con 0x7f23cc100cd0 2023-07-21T19:07:24.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.682+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23cc10b2c0 con 0x7f23cc100cd0 2023-07-21T19:07:24.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.684+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f23bc022470 con 0x7f23cc100cd0 2023-07-21T19:07:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.686+0000 7f23c27fc700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 0x7f23b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.686+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f23bc0cdff0 con 0x7f23cc100cd0 2023-07-21T19:07:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.686+0000 7f23d139b700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 0x7f23b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:24.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.687+0000 7f23d139b700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 0x7f23b80a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f23cc100b30 tx=0x7f23c8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:24.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.689+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f23bc0977a0 con 0x7f23cc100cd0 2023-07-21T19:07:24.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:24 smithi101 ceph-mon[128136]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-07-21T19:07:24.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:24 smithi101 ceph-mon[128136]: pgmap v355: 97 pgs: 97 active+clean; 639 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 511 B/s wr, 6 op/s 2023-07-21T19:07:24.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:24 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1307513918' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:24.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:24 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1793522939' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:07:24.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.896+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f23cc1a04d0 con 0x7f23cc100cd0 2023-07-21T19:07:24.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.897+0000 7f23c27fc700 1 -- 172.21.15.5:0/477298284 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v87) v1 ==== 100+0+11 (secure 0 0 0) 0x7f23bc09adc0 con 0x7f23cc100cd0 2023-07-21T19:07:24.907 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:07:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 msgr2=0x7f23b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 0x7f23b80a3600 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f23cc100b30 tx=0x7f23c8009190 comp rx=0 tx=0).stop 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 msgr2=0x7f23cc197ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f23bc005910 tx=0x7f23bc005740 comp rx=0 tx=0).stop 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 shutdown_connections 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f23b80a1150 0x7f23b80a3600 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f23cc101660 0x7f23cc19ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f23cc0ffae0 0x7f23cc1979b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 --2- 172.21.15.5:0/477298284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f23cc100cd0 0x7f23cc197ef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 >> 172.21.15.5:0/477298284 conn(0x7f23cc0fb0e0 msgr2=0x7f23cc0fd370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 shutdown_connections 2023-07-21T19:07:24.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:24.899+0000 7f23d35ff700 1 -- 172.21.15.5:0/477298284 wait complete. 2023-07-21T19:07:24.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/3637569611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 msgr2=0x7f2d180733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/3637569611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d180733e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2d08009990 tx=0x7f2d08009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/3637569611 shutdown_connections 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/3637569611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d181079c0 0x7f2d18109e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/3637569611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d18074550 0x7f2d180729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/3637569611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d180733e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/3637569611 >> 172.21.15.5:0/3637569611 conn(0x7f2d180fb0e0 msgr2=0x7f2d180fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/3637569611 shutdown_connections 2023-07-21T19:07:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.120+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/3637569611 wait complete. 2023-07-21T19:07:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.121+0000 7f2d1f5ff700 1 Processor -- start 2023-07-21T19:07:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.121+0000 7f2d1f5ff700 1 -- start start 2023-07-21T19:07:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d18074550 0x7f2d18197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d181079c0 0x7f2d1819ee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1819f3e0 con 0x7f2d18072f70 2023-07-21T19:07:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1819f520 con 0x7f2d18074550 2023-07-21T19:07:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1819f6b0 con 0x7f2d181079c0 2023-07-21T19:07:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48100/0 (socket says 172.21.15.5:48100) 2023-07-21T19:07:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 -- 172.21.15.5:0/1941530076 learned_addr learned my addr 172.21.15.5:0/1941530076 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1db9c700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d181079c0 0x7f2d1819ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1cb9a700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d18074550 0x7f2d18197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 -- 172.21.15.5:0/1941530076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d181079c0 msgr2=0x7f2d1819ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d181079c0 0x7f2d1819ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 -- 172.21.15.5:0/1941530076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d18074550 msgr2=0x7f2d18197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d18074550 0x7f2d18197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.122+0000 7f2d1d39b700 1 -- 172.21.15.5:0/1941530076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d080095f0 con 0x7f2d18072f70 2023-07-21T19:07:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d1d39b700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c9e0 tx=0x7f2d1400cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14007aa0 con 0x7f2d18072f70 2023-07-21T19:07:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d14007c00 con 0x7f2d18072f70 2023-07-21T19:07:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d1819f990 con 0x7f2d18072f70 2023-07-21T19:07:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d14018660 con 0x7f2d18072f70 2023-07-21T19:07:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.123+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d1819fef0 con 0x7f2d18072f70 2023-07-21T19:07:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.124+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d18106070 con 0x7f2d18072f70 2023-07-21T19:07:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.126+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f2d1400f450 con 0x7f2d18072f70 2023-07-21T19:07:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.127+0000 7f2d0e7fc700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 0x7f2d040a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.128+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f2d140cc8e0 con 0x7f2d18072f70 2023-07-21T19:07:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.128+0000 7f2d1cb9a700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 0x7f2d040a36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.129+0000 7f2d1cb9a700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 0x7f2d040a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800b5c0 tx=0x7f2d08005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.131+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2d14095fe0 con 0x7f2d18072f70 2023-07-21T19:07:25.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.384+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d1802c850 con 0x7f2d18072f70 2023-07-21T19:07:25.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.385+0000 7f2d0e7fc700 1 -- 172.21.15.5:0/1941530076 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f2d14099600 con 0x7f2d18072f70 2023-07-21T19:07:25.395 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:25.396 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 msgr2=0x7f2d040a36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 0x7f2d040a36d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2d0800b5c0 tx=0x7f2d08005fb0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 msgr2=0x7f2d18197860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2d1400c9e0 tx=0x7f2d1400cda0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 shutdown_connections 2023-07-21T19:07:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f2d040a1220 0x7f2d040a36d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d181079c0 0x7f2d1819ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d18072f70 0x7f2d18197860 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 --2- 172.21.15.5:0/1941530076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d18074550 0x7f2d18197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.387+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 >> 172.21.15.5:0/1941530076 conn(0x7f2d180fb0e0 msgr2=0x7f2d18109be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.388+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 shutdown_connections 2023-07-21T19:07:25.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.388+0000 7f2d1f5ff700 1 -- 172.21.15.5:0/1941530076 wait complete. 2023-07-21T19:07:25.401 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:25.415 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:07:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 -- 172.21.15.5:0/3938741456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 msgr2=0x7f0088107e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 --2- 172.21.15.5:0/3938741456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f0088107e20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f008000b0a0 tx=0x7f008000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 -- 172.21.15.5:0/3938741456 shutdown_connections 2023-07-21T19:07:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 --2- 172.21.15.5:0/3938741456 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f0088107e20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 --2- 172.21.15.5:0/3938741456 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f00880ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 --2- 172.21.15.5:0/3938741456 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 0x7f0088101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 -- 172.21.15.5:0/3938741456 >> 172.21.15.5:0/3938741456 conn(0x7f00880fafe0 msgr2=0x7f00880fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 -- 172.21.15.5:0/3938741456 shutdown_connections 2023-07-21T19:07:25.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.609+0000 7f008c99d700 1 -- 172.21.15.5:0/3938741456 wait complete. 2023-07-21T19:07:25.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.610+0000 7f008c99d700 1 Processor -- start 2023-07-21T19:07:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.610+0000 7f008c99d700 1 -- start start 2023-07-21T19:07:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f0088197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 0x7f0088197c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008819f290 con 0x7f0088101560 2023-07-21T19:07:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008819f3d0 con 0x7f00880ff9e0 2023-07-21T19:07:25.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f008c99d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008819f560 con 0x7f0088100bd0 2023-07-21T19:07:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f0087fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f0087fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48124/0 (socket says 172.21.15.5:48124) 2023-07-21T19:07:25.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f0087fff700 1 -- 172.21.15.5:0/423702056 learned_addr learned my addr 172.21.15.5:0/423702056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f0086ffd700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 0x7f0088197c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.611+0000 7f00877fe700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f0088197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 -- 172.21.15.5:0/423702056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 msgr2=0x7f0088197c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 0x7f0088197c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 -- 172.21.15.5:0/423702056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 msgr2=0x7f0088197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f0088197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 -- 172.21.15.5:0/423702056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009d00 con 0x7f0088101560 2023-07-21T19:07:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f00877fe700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f0088197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0087fff700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0080009fd0 tx=0x7f008000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:25.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008000e040 con 0x7f0088101560 2023-07-21T19:07:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0080007760 con 0x7f0088101560 2023-07-21T19:07:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f008001b6c0 con 0x7f0088101560 2023-07-21T19:07:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008819f7e0 con 0x7f0088101560 2023-07-21T19:07:25.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.612+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f008819fbb0 con 0x7f0088101560 2023-07-21T19:07:25.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.613+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008810b0c0 con 0x7f0088101560 2023-07-21T19:07:25.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.615+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f008001d470 con 0x7f0088101560 2023-07-21T19:07:25.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.617+0000 7f0084ff9700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 0x7f00700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:25.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.617+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f00800cdc30 con 0x7f0088101560 2023-07-21T19:07:25.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.617+0000 7f00877fe700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 0x7f00700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:25.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.617+0000 7f00877fe700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 0x7f00700a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0078005950 tx=0x7f00780058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:25.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.620+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f0080097420 con 0x7f0088101560 2023-07-21T19:07:25.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:25 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3316690707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:25 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/477298284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:25 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3316690707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:25 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/477298284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:25 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3316690707' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:25 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/477298284' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:07:25.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.836+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f00881a02b0 con 0x7f0088101560 2023-07-21T19:07:25.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.837+0000 7f0084ff9700 1 -- 172.21.15.5:0/423702056 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v87) v1 ==== 74+0+22526 (secure 0 0 0) 0x7f008009aa40 con 0x7f0088101560 2023-07-21T19:07:25.847 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:25.847 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":87,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:06:08.388082+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:07:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 msgr2=0x7f00700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 0x7f00700a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0078005950 tx=0x7f00780058e0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 msgr2=0x7f008819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:25.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0080009fd0 tx=0x7f008000bc80 comp rx=0 tx=0).stop 2023-07-21T19:07:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 shutdown_connections 2023-07-21T19:07:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f00700a12c0 0x7f00700a3770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0088101560 0x7f008819ecc0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00880ff9e0 0x7f0088197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 --2- 172.21.15.5:0/423702056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0088100bd0 0x7f0088197c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:25.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 >> 172.21.15.5:0/423702056 conn(0x7f00880fafe0 msgr2=0x7f00880fd110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 shutdown_connections 2023-07-21T19:07:25.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:25.840+0000 7f008c99d700 1 -- 172.21.15.5:0/423702056 wait complete. 2023-07-21T19:07:25.868 DEBUG:tasks.fwd_scrub:Yielding 2023-07-21T19:07:25.869 INFO:teuthology.run_tasks:Running task exec... 2023-07-21T19:07:25.879 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T19:07:25.880 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:07:25.880 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-07-21T19:07:25.881 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2023-07-21T19:07:25.881 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:07:25.881 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:07:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.078+0000 7f8154fff700 1 -- 172.21.15.5:0/552772542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150072f70 msgr2=0x7f81500733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.078+0000 7f8154fff700 1 --2- 172.21.15.5:0/552772542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150072f70 0x7f81500733e0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f8144009990 tx=0x7f8144009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 -- 172.21.15.5:0/552772542 shutdown_connections 2023-07-21T19:07:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 --2- 172.21.15.5:0/552772542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f815010c0d0 0x7f815010e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 --2- 172.21.15.5:0/552772542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 0x7f81500729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 --2- 172.21.15.5:0/552772542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8150072f70 0x7f81500733e0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 -- 172.21.15.5:0/552772542 >> 172.21.15.5:0/552772542 conn(0x7f81500fb0e0 msgr2=0x7f81500fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 -- 172.21.15.5:0/552772542 shutdown_connections 2023-07-21T19:07:26.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.079+0000 7f8154fff700 1 -- 172.21.15.5:0/552772542 wait complete. 2023-07-21T19:07:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.080+0000 7f8154fff700 1 Processor -- start 2023-07-21T19:07:26.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.080+0000 7f8154fff700 1 -- start start 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150072f70 0x7f815019bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 0x7f815019c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019d8f0 con 0x7f815010c0d0 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019da60 con 0x7f8150074550 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f8154fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f815019dc20 con 0x7f8150072f70 2023-07-21T19:07:26.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f814ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f814ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48146/0 (socket says 172.21.15.5:48146) 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f814ed9d700 1 -- 172.21.15.5:0/1258157078 learned_addr learned my addr 172.21.15.5:0/1258157078 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f814e59c700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150072f70 0x7f815019bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.081+0000 7f814dd9b700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 0x7f815019c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 -- 172.21.15.5:0/1258157078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150072f70 msgr2=0x7f815019bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150072f70 0x7f815019bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 -- 172.21.15.5:0/1258157078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 msgr2=0x7f815019c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 0x7f815019c410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 -- 172.21.15.5:0/1258157078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81440095f0 con 0x7f815010c0d0 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f814ed9d700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8148007ee0 tx=0x7f814800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148017070 con 0x7f815010c0d0 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f814800f040 con 0x7f815010c0d0 2023-07-21T19:07:26.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f815019df00 con 0x7f815010c0d0 2023-07-21T19:07:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.082+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8148005570 con 0x7f815010c0d0 2023-07-21T19:07:26.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.083+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81501a46c0 con 0x7f815010c0d0 2023-07-21T19:07:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.084+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8150101c20 con 0x7f815010c0d0 2023-07-21T19:07:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.086+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7f814800db20 con 0x7f815010c0d0 2023-07-21T19:07:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.087+0000 7f813f7fe700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 0x7f81380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.087+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f81480cbf90 con 0x7f815010c0d0 2023-07-21T19:07:26.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.087+0000 7f814e59c700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 0x7f81380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.088+0000 7f814e59c700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 0x7f81380a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8150074920 tx=0x7f8140009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.091+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8148095790 con 0x7f815010c0d0 2023-07-21T19:07:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 -- 172.21.15.5:0/1637934705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 msgr2=0x7febe80fff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 --2- 172.21.15.5:0/1637934705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe80fff00 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7febe4009990 tx=0x7febe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 -- 172.21.15.5:0/1637934705 shutdown_connections 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 --2- 172.21.15.5:0/1637934705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 0x7febe8107f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 --2- 172.21.15.5:0/1637934705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe80fff00 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 --2- 172.21.15.5:0/1637934705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 0x7febe8101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 -- 172.21.15.5:0/1637934705 >> 172.21.15.5:0/1637934705 conn(0x7febe80fb130 msgr2=0x7febe80fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 -- 172.21.15.5:0/1637934705 shutdown_connections 2023-07-21T19:07:26.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.094+0000 7febf07ff700 1 -- 172.21.15.5:0/1637934705 wait complete. 2023-07-21T19:07:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.095+0000 7febf07ff700 1 Processor -- start 2023-07-21T19:07:26.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 -- start start 2023-07-21T19:07:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 0x7febe8195d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 0x7febe819cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe819d340 con 0x7febe80ffaf0 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe819d480 con 0x7febe8100ce0 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.096+0000 7febf07ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe819d610 con 0x7febe8101670 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48172/0 (socket says 172.21.15.5:48172) 2023-07-21T19:07:26.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 -- 172.21.15.5:0/3710690878 learned_addr learned my addr 172.21.15.5:0/3710690878 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febedd9a700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 0x7febe8195d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febeed9c700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 0x7febe819cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 -- 172.21.15.5:0/3710690878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 msgr2=0x7febe819cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 0x7febe819cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 -- 172.21.15.5:0/3710690878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 msgr2=0x7febe8195d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 0x7febe8195d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 -- 172.21.15.5:0/3710690878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd8009520 con 0x7febe80ffaf0 2023-07-21T19:07:26.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.097+0000 7febee59b700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7febe4005f50 tx=0x7febe4004000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.098+0000 7febdf7fe700 1 -- 172.21.15.5:0/3710690878 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe401d070 con 0x7febe80ffaf0 2023-07-21T19:07:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.098+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febe40095f0 con 0x7febe80ffaf0 2023-07-21T19:07:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.098+0000 7febdf7fe700 1 -- 172.21.15.5:0/3710690878 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe40041e0 con 0x7febe80ffaf0 2023-07-21T19:07:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.098+0000 7febdf7fe700 1 -- 172.21.15.5:0/3710690878 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe400f460 con 0x7febe80ffaf0 2023-07-21T19:07:26.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.098+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe819db80 con 0x7febe80ffaf0 2023-07-21T19:07:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.102+0000 7febdf7fe700 1 -- 172.21.15.5:0/3710690878 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7febe4022470 con 0x7febe80ffaf0 2023-07-21T19:07:26.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 Processor -- start 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 -- start start 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7febdf7fe700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 0x7febd40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9d3b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9d4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a8d3d30 con 0x55f11a9d2960 2023-07-21T19:07:26.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a926ac0 con 0x55f11a9d4100 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7fec0406db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a91ff50 con 0x55f11a9d32b0 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7febdf7fe700 1 -- 172.21.15.5:0/3710690878 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe40cddb0 con 0x7febe80ffaf0 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.104+0000 7febeed9c700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 0x7febd40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48188/0 (socket says 172.21.15.5:48188) 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 -- 172.21.15.5:0/3829945045 learned_addr learned my addr 172.21.15.5:0/3829945045 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febeed9c700 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9d4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febedd9a700 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9d3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 -- 172.21.15.5:0/3829945045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 msgr2=0x55f11a9d3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9d3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 -- 172.21.15.5:0/3829945045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 msgr2=0x55f11a9d4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9d4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 -- 172.21.15.5:0/3829945045 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11a74de60 con 0x55f11a9d2960 2023-07-21T19:07:26.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febeed9c700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 0x7febd40a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7febe0009550 tx=0x7febe0007a80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.105+0000 7febee59b700 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7febe409af00 tx=0x7febe4005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16050e86f3e5a043 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7febdcff9700 1 -- 172.21.15.5:0/3829945045 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe406f850 con 0x55f11a9d2960 2023-07-21T19:07:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7febdcff9700 1 -- 172.21.15.5:0/3829945045 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe40673b0 con 0x55f11a9d2960 2023-07-21T19:07:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7febdcff9700 1 -- 172.21.15.5:0/3829945045 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe40a35c0 con 0x55f11a9d2960 2023-07-21T19:07:26.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 -- 172.21.15.5:0/3829945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 msgr2=0x55f11a9d2d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7febe409af00 tx=0x7febe4005c70 comp rx=0 tx=0).stop 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 -- 172.21.15.5:0/3829945045 shutdown_connections 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9d4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9d2d70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 --2- 172.21.15.5:0/3829945045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9d3b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.106+0000 7fec0406db80 1 -- 172.21.15.5:0/3829945045 >> 172.21.15.5:0/3829945045 conn(0x55f11a9c3e20 msgr2=0x55f11a9c9250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.107+0000 7fec0406db80 1 -- 172.21.15.5:0/3829945045 shutdown_connections 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.107+0000 7fec0406db80 1 -- 172.21.15.5:0/3829945045 wait complete. 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.107+0000 7fec0406db80 1 -- start start 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.107+0000 7fec0406db80 1 Processor -- start 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9dc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a8bfb40 con 0x55f11a9d2960 2023-07-21T19:07:26.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a8bf840 con 0x55f11a9d4100 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a7a39a0 con 0x55f11a9d32b0 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7fec0406db80 10 client.0 _ll_register_callbacks cb 0x55f11a8bd7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48198/0 (socket says 172.21.15.5:48198) 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 -- 172.21.15.5:0/2968716183 learned_addr learned my addr 172.21.15.5:0/2968716183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febedd9a700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9dc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febeed9c700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 -- 172.21.15.5:0/2968716183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 msgr2=0x55f11a9dc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9dc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 -- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 msgr2=0x55f11a9e37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e37e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.108+0000 7febee59b700 1 -- 172.21.15.5:0/2968716183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe819db80 con 0x55f11a9d2960 2023-07-21T19:07:26.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febedd9a700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febee59b700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febe4003f60 tx=0x7febe409afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b05205e3f0bc21f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe409f030 con 0x55f11a9d2960 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febe40a35c0 con 0x55f11a9d2960 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f11a879570 con 0x55f11a9d2960 2023-07-21T19:07:26.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febe40cb8e0 con 0x55f11a9d2960 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7fec0406db80 10 client.15465 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f11a736ce0 con 0x55f11a9d2960 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.109+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe40b7020 con 0x55f11a9d2960 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febe40cbd20 con 0x55f11a9d2960 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 10 client.15465 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:26.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 20 client.15465 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11a9e7b60 0x55f11a9e9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 4 client.15465 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f11a918f30 con 0x55f11a9e7b60 2023-07-21T19:07:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.110+0000 7febedd9a700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11a9e7b60 0x55f11a9e9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.111+0000 7febedd9a700 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11a9e7b60 0x55f11a9e9f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e1a4f89ea50657c server_cookie=113e1db7c52deb5 in_seq=0 out_seq=0 2023-07-21T19:07:26.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.111+0000 7febd1ffb700 10 client.15465 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7febd1ffb700 1 -- 172.21.15.5:0/2968716183 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f11a918f30 con 0x55f11a9e7b60 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7febd1ffb700 10 client.15465 handle_command_reply: tid=0 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 client.15465 shutdown 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 msgr2=0x55f11a9dc260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7febe4003f60 tx=0x7febe409afd0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 shutdown_connections 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11a9e7b60 0x55f11a9e9f70 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d2960 0x55f11a9dc260 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 --2- 172.21.15.5:0/2968716183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d32b0 0x55f11a9dc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 >> 172.21.15.5:0/2968716183 conn(0x55f11a9c3e20 msgr2=0x55f11a9c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7febd1ffb700 0 client.15465 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 shutdown_connections 2023-07-21T19:07:26.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 1 -- 172.21.15.5:0/2968716183 wait complete. 2023-07-21T19:07:26.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.112+0000 7fec0406db80 20 client.15465 trim_cache size 0 max 16384 2023-07-21T19:07:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.130+0000 7fec0406db80 1 Processor -- start 2023-07-21T19:07:26.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.130+0000 7fec0406db80 1 -- start start 2023-07-21T19:07:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d32b0 0x55f11a9e37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d4100 0x55f11a9dc550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9c39f0 con 0x55f11a9d4100 2023-07-21T19:07:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9c8510 con 0x55f11a9d32b0 2023-07-21T19:07:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7fec0406db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9dad00 con 0x55f11a9d2960 2023-07-21T19:07:26.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febeed9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d4100 0x55f11a9dc550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febee59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febee59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37448/0 (socket says 172.21.15.5:37448) 2023-07-21T19:07:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febedd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d32b0 0x55f11a9e37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febee59b700 1 -- 172.21.15.5:0/885139790 learned_addr learned my addr 172.21.15.5:0/885139790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febee59b700 1 -- 172.21.15.5:0/885139790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d32b0 msgr2=0x55f11a9e37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.131+0000 7febee59b700 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d32b0 0x55f11a9e37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febee59b700 1 -- 172.21.15.5:0/885139790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d4100 msgr2=0x55f11a9dc550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febee59b700 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d4100 0x55f11a9dc550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febee59b700 1 -- 172.21.15.5:0/885139790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f11a7742d0 con 0x55f11a9d2960 2023-07-21T19:07:26.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febee59b700 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febe4097700 tx=0x7febe40092f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3278fc2280018f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febdd7fa700 1 -- 172.21.15.5:0/885139790 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe40a3130 con 0x55f11a9d2960 2023-07-21T19:07:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febdd7fa700 1 -- 172.21.15.5:0/885139790 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe40b7020 con 0x55f11a9d2960 2023-07-21T19:07:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.132+0000 7febdd7fa700 1 -- 172.21.15.5:0/885139790 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febe4067b90 con 0x55f11a9d2960 2023-07-21T19:07:26.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- 172.21.15.5:0/885139790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 msgr2=0x55f11a9dc010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7febe4097700 tx=0x7febe40092f0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- 172.21.15.5:0/885139790 shutdown_connections 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d4100 0x55f11a9dc550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9dc010 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 --2- 172.21.15.5:0/885139790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d32b0 0x55f11a9e37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- 172.21.15.5:0/885139790 >> 172.21.15.5:0/885139790 conn(0x55f11a9c3e20 msgr2=0x55f11a9d6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- 172.21.15.5:0/885139790 shutdown_connections 2023-07-21T19:07:26.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- 172.21.15.5:0/885139790 wait complete. 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.133+0000 7fec0406db80 1 -- start start 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 Processor -- start 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9e0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9e28d0 con 0x55f11a9d32b0 2023-07-21T19:07:26.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9e2a40 con 0x55f11a9d4100 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f11a9e2c00 con 0x55f11a9d2960 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.134+0000 7fec0406db80 10 client.0 _ll_register_callbacks cb 0x7febe409ad40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48210/0 (socket says 172.21.15.5:48210) 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 -- 172.21.15.5:0/1453765082 learned_addr learned my addr 172.21.15.5:0/1453765082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febee59b700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9e0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 -- 172.21.15.5:0/1453765082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 msgr2=0x55f11a9e0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9e0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febeed9c700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 -- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 msgr2=0x55f11a9e2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 -- 172.21.15.5:0/1453765082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f11a736ce0 con 0x55f11a9d32b0 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febeed9c700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febedd9a700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55f11a9ea850 tx=0x7febd801d950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69c9fcb770b6a84d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.135+0000 7febd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febd8018040 con 0x55f11a9d32b0 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7febe40673b0 con 0x55f11a9d32b0 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7febd8004220 con 0x55f11a9d32b0 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7febd802b030 con 0x55f11a9d32b0 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 10 client.15471 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:26.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55f11a74c330 con 0x55f11a9d32b0 2023-07-21T19:07:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7febe40673b0 con 0x55f11a9d32b0 2023-07-21T19:07:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7febd80036f0 con 0x55f11a9d32b0 2023-07-21T19:07:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 10 client.15471 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:26.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 20 client.15471 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11aa3e1d0 0x55f11aa405e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 4 client.15471 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.136+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f11a8db0d0 con 0x55f11aa3e1d0 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.137+0000 7febee59b700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11aa3e1d0 0x55f11aa405e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.137+0000 7febd1ffb700 10 client.15471 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.137+0000 7febee59b700 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11aa3e1d0 0x55f11aa405e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c93c3a33fd44b43 server_cookie=7aaeb8bedf0cea4c in_seq=0 out_seq=0 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7febd1ffb700 1 -- 172.21.15.5:0/1453765082 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f11a8db0d0 con 0x55f11aa3e1d0 2023-07-21T19:07:26.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7febd1ffb700 10 client.15471 handle_command_reply: tid=0 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 client.15471 shutdown 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 msgr2=0x55f11a9e1380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x55f11a9ea850 tx=0x7febd801d950 comp rx=0 tx=0).stop 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 shutdown_connections 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f11a9d4100 0x55f11a9e2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f11a9d2960 0x55f11a9e0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f11aa3e1d0 0x55f11aa405e0 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 --2- 172.21.15.5:0/1453765082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f11a9d32b0 0x55f11a9e1380 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7febd1ffb700 0 client.15471 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.139+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 >> 172.21.15.5:0/1453765082 conn(0x55f11a9c3e20 msgr2=0x55f11a9dc010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.140+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 shutdown_connections 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.140+0000 7fec0406db80 1 -- 172.21.15.5:0/1453765082 wait complete. 2023-07-21T19:07:26.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.140+0000 7fec0406db80 20 client.15471 trim_cache size 0 max 16384 2023-07-21T19:07:26.162 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:07:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "668869f5-a83a-4d36-a914-470c24366c9b", 2023-07-21T19:07:26.162 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:07:26.162 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 msgr2=0x7febd40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 0x7febd40a3600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7febe0009550 tx=0x7febe0007a80 comp rx=0 tx=0).stop 2023-07-21T19:07:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 msgr2=0x7febe81957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7febe4005f50 tx=0x7febe4004000 comp rx=0 tx=0).stop 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 shutdown_connections 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7febd40a1150 0x7febd40a3600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7febe8101670 0x7febe819cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7febe80ffaf0 0x7febe81957c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 --2- 172.21.15.5:0/3710690878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7febe8100ce0 0x7febe8195d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 >> 172.21.15.5:0/3710690878 conn(0x7febe80fb130 msgr2=0x7febe8104210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 shutdown_connections 2023-07-21T19:07:26.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.141+0000 7febf07ff700 1 -- 172.21.15.5:0/3710690878 wait complete. 2023-07-21T19:07:26.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:07:26.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f815002c9e0 con 0x7f815010c0d0 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 -- 172.21.15.5:0/671826729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c41057a0 msgr2=0x7fd4c4107c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/671826729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c41057a0 0x7fd4c4107c00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc00b0a0 tx=0x7fd4bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 -- 172.21.15.5:0/671826729 shutdown_connections 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/671826729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c41057a0 0x7fd4c4107c00 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/671826729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 0x7fd4c4102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/671826729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c4102e80 0x7fd4c4105260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.379+0000 7fd4c89ff700 1 -- 172.21.15.5:0/671826729 >> 172.21.15.5:0/671826729 conn(0x7fd4c40fa000 msgr2=0x7fd4c40fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.380+0000 7fd4c89ff700 1 -- 172.21.15.5:0/671826729 shutdown_connections 2023-07-21T19:07:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.380+0000 7fd4c89ff700 1 -- 172.21.15.5:0/671826729 wait complete. 2023-07-21T19:07:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 Processor -- start 2023-07-21T19:07:26.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 -- start start 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 0x7fd4c4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c41057a0 0x7fd4c419efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c419f5a0 con 0x7fd4c4102e80 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c419f6e0 con 0x7fd4c4100560 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c89ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4c419f870 con 0x7fd4c41057a0 2023-07-21T19:07:26.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57618/0 (socket says 172.21.15.5:57618) 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.381+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1007657432 learned_addr learned my addr 172.21.15.5:0/1007657432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 0x7fd4c4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1007657432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c41057a0 msgr2=0x7fd4c419efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c41057a0 0x7fd4c419efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1007657432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 msgr2=0x7fd4c4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 0x7fd4c4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1007657432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4bc009d00 con 0x7fd4c4102e80 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d950 tx=0x7fd4b800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4b37fe700 1 -- 172.21.15.5:0/1007657432 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b8009b40 con 0x7fd4c4102e80 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4b37fe700 1 -- 172.21.15.5:0/1007657432 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4b8009ca0 con 0x7fd4c4102e80 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4c419fb50 con 0x7fd4c4102e80 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4b37fe700 1 -- 172.21.15.5:0/1007657432 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b800f620 con 0x7fd4c4102e80 2023-07-21T19:07:26.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.382+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c41a0060 con 0x7fd4c4102e80 2023-07-21T19:07:26.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.385+0000 7fd4b37fe700 1 -- 172.21.15.5:0/1007657432 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 18) v1 ==== 135714+0+0 (secure 0 0 0) 0x7fd4b8010460 con 0x7fd4c4102e80 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.386+0000 7fd4dc346b80 1 Processor -- start 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 -- start start 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4b37fe700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 0x7fd4ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 0x55a2e26e46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4b37fe700 1 -- 172.21.15.5:0/1007657432 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd4b80cccc0 con 0x7fd4c4102e80 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 0x55a2e26e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e25e8cb0 con 0x55a2e26e4c70 2023-07-21T19:07:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e263ba40 con 0x55a2e26e3e20 2023-07-21T19:07:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e2634ed0 con 0x55a2e26e34d0 2023-07-21T19:07:26.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 0x7fd4ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4c2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 0x55a2e26e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 0x55a2e26e46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4c2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 0x55a2e26e50e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57630/0 (socket says 172.21.15.5:57630) 2023-07-21T19:07:26.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.387+0000 7fd4c2d9d700 1 -- 172.21.15.5:0/1604712594 learned_addr learned my addr 172.21.15.5:0/1604712594 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1604712594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 msgr2=0x55a2e26e38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26e38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1604712594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 msgr2=0x55a2e26e50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 0x55a2e26e50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1604712594 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e2462de0 con 0x55a2e26e3e20 2023-07-21T19:07:26.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 0x7fd4ac0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009610 tx=0x7fd4bc009460 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 0x55a2e26e46e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4b809a010 tx=0x7fd4b8099f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56a4989bc5ad5091 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4b0ff9700 1 -- 172.21.15.5:0/1604712594 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b809e0b0 con 0x55a2e26e3e20 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4b0ff9700 1 -- 172.21.15.5:0/1604712594 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4b80a24d0 con 0x55a2e26e3e20 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.388+0000 7fd4b0ff9700 1 -- 172.21.15.5:0/1604712594 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b809a500 con 0x55a2e26e3e20 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1604712594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 msgr2=0x55a2e26e46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 0x55a2e26e46e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd4b809a010 tx=0x7fd4b8099f80 comp rx=0 tx=0).stop 2023-07-21T19:07:26.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1604712594 shutdown_connections 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e4c70 0x55a2e26e50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26e38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1604712594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e3e20 0x55a2e26e46e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1604712594 >> 172.21.15.5:0/1604712594 conn(0x55a2e26d8da0 msgr2=0x55a2e26de1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1604712594 shutdown_connections 2023-07-21T19:07:26.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.389+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1604712594 wait complete. 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.390+0000 7fd4dc346b80 1 -- start start 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.390+0000 7fd4dc346b80 1 Processor -- start 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26ed070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e25d4ac0 con 0x55a2e26e3e20 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e25d47c0 con 0x55a2e26e4c70 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e24b8920 con 0x55a2e26e34d0 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4dc346b80 10 client.0 _ll_register_callbacks cb 0x55a2e25d2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26ed070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57644/0 (socket says 172.21.15.5:57644) 2023-07-21T19:07:26.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1855838666 learned_addr learned my addr 172.21.15.5:0/1855838666 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.391+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1855838666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 msgr2=0x55a2e26ed070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26ed070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 msgr2=0x55a2e26f45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1855838666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c41a0060 con 0x55a2e26e3e20 2023-07-21T19:07:26.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:26.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80a1fa0 tx=0x7fd4b809a1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a6ca1107a14a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b84a0 con 0x55a2e26e3e20 2023-07-21T19:07:26.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2e258e4f0 con 0x55a2e26e3e20 2023-07-21T19:07:26.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd4b8099570 con 0x55a2e26e3e20 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b809e130 con 0x55a2e26e3e20 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4dc346b80 10 client.15483 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.392+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a2e244bc60 con 0x55a2e26e3e20 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4b8063080 con 0x55a2e26e3e20 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd4b80ca170 con 0x55a2e26e3e20 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 10 client.15483 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:26.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 20 client.15483 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e26f89b0 0x55a2e26fadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 4 client.15483 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2e262deb0 con 0x55a2e26f89b0 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.393+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e26f89b0 0x55a2e26fadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.394+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e26f89b0 0x55a2e26fadc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b27f17f52202101a server_cookie=e1423406fdc65838 in_seq=0 out_seq=0 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.394+0000 7fd4a9ffb700 10 client.15483 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/1855838666 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2e262deb0 con 0x55a2e26f89b0 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4a9ffb700 10 client.15483 handle_command_reply: tid=0 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 client.15483 shutdown 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 msgr2=0x55a2e26ed5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fd4b80a1fa0 tx=0x7fd4b809a1b0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 shutdown_connections 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e34d0 0x55a2e26ed070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e26f89b0 0x55a2e26fadc0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1855838666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26ed5b0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 >> 172.21.15.5:0/1855838666 conn(0x55a2e26d8da0 msgr2=0x55a2e26dd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4a9ffb700 0 client.15483 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 shutdown_connections 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.395+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1855838666 wait complete. 2023-07-21T19:07:26.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.396+0000 7fd4dc346b80 20 client.15483 trim_cache size 0 max 16384 2023-07-21T19:07:26.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 Processor -- start 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 -- start start 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 0x55a2e26e7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e4c70 0x55a2e26e80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26df060 con 0x55a2e26e3e20 2023-07-21T19:07:26.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26edf10 con 0x55a2e26e34d0 2023-07-21T19:07:26.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.413+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26e41f0 con 0x55a2e26e4c70 2023-07-21T19:07:26.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57652/0 (socket says 172.21.15.5:57652) 2023-07-21T19:07:26.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1403853049 learned_addr learned my addr 172.21.15.5:0/1403853049 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e4c70 0x55a2e26e80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 0x55a2e26e7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1403853049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e4c70 msgr2=0x55a2e26e80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e4c70 0x55a2e26e80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1403853049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 msgr2=0x55a2e26e7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 0x55a2e26e7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 -- 172.21.15.5:0/1403853049 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2e26e5310 con 0x55a2e26e3e20 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c259c700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 0x55a2e26e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009520 tx=0x7fd4b8098630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4ab6afc751426 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.414+0000 7fd4b17fa700 1 -- 172.21.15.5:0/1403853049 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80056c0 con 0x55a2e26e3e20 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4b17fa700 1 -- 172.21.15.5:0/1403853049 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a2e258e4f0 con 0x55a2e26e3e20 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4b17fa700 1 -- 172.21.15.5:0/1403853049 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b80b82a0 con 0x55a2e26e3e20 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1403853049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 msgr2=0x55a2e26f45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009520 tx=0x7fd4b8098630 comp rx=0 tx=0).stop 2023-07-21T19:07:26.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1403853049 shutdown_connections 2023-07-21T19:07:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e4c70 0x55a2e26e80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e34d0 0x55a2e26e7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/1403853049 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e3e20 0x55a2e26f45b0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1403853049 >> 172.21.15.5:0/1403853049 conn(0x55a2e26d8da0 msgr2=0x55a2e26f4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1403853049 shutdown_connections 2023-07-21T19:07:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.415+0000 7fd4dc346b80 1 -- 172.21.15.5:0/1403853049 wait complete. 2023-07-21T19:07:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.416+0000 7fd4dc346b80 1 -- start start 2023-07-21T19:07:26.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.416+0000 7fd4dc346b80 1 Processor -- start 2023-07-21T19:07:26.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e3e20 0x55a2e26f0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26f3dd0 con 0x55a2e26e34d0 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26f3f40 con 0x55a2e26e4c70 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2e26f4100 con 0x55a2e26e3e20 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4dc346b80 10 client.0 _ll_register_callbacks cb 0x7fd4bc009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:26.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57654/0 (socket says 172.21.15.5:57654) 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 -- 172.21.15.5:0/3920554082 learned_addr learned my addr 172.21.15.5:0/3920554082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c2d9d700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e3e20 0x55a2e26f0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 -- 172.21.15.5:0/3920554082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e3e20 msgr2=0x55a2e26f0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e3e20 0x55a2e26f0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 -- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 msgr2=0x55a2e26f3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.417+0000 7fd4c259c700 1 -- 172.21.15.5:0/3920554082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2e244bc60 con 0x55a2e26e34d0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4c259c700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd4b401f9f0 tx=0x7fd4b401fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a020d05f15977b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4a9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4b4018040 con 0x55a2e26e34d0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7fd4b4024030 con 0x55a2e26e34d0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4b80a24d0 con 0x55a2e26e34d0 2023-07-21T19:07:26.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4b401f5c0 con 0x55a2e26e34d0 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4dc346b80 10 client.15489 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55a2e2489250 con 0x55a2e26e34d0 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.418+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4b80a24d0 con 0x55a2e26e34d0 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd4b4013830 con 0x55a2e26e34d0 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 10 client.15489 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:26.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 20 client.15489 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e274d7d0 0x55a2e274fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 4 client.15489 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a2e25f0050 con 0x55a2e274d7d0 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.419+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e274d7d0 0x55a2e274fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4c1d9b700 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e274d7d0 0x55a2e274fbe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8cd5e0d22813538 server_cookie=773a87c66a67956c in_seq=0 out_seq=0 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4a9ffb700 10 client.15489 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4a9ffb700 1 -- 172.21.15.5:0/3920554082 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a2e25f0050 con 0x55a2e274d7d0 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4a9ffb700 10 client.15489 handle_command_reply: tid=0 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 client.15489 shutdown 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 msgr2=0x55a2e26f07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fd4b401f9f0 tx=0x7fd4b401fd00 comp rx=0 tx=0).stop 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 shutdown_connections 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2e26e4c70 0x55a2e26f3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a2e274d7d0 0x55a2e274fbe0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2e26e34d0 0x55a2e26f07e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 --2- 172.21.15.5:0/3920554082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2e26e3e20 0x55a2e26f0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 >> 172.21.15.5:0/3920554082 conn(0x55a2e26d8da0 msgr2=0x55a2e26e7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4a9ffb700 0 client.15489 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:26.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 shutdown_connections 2023-07-21T19:07:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.420+0000 7fd4dc346b80 1 -- 172.21.15.5:0/3920554082 wait complete. 2023-07-21T19:07:26.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4dc346b80 20 client.15489 trim_cache size 0 max 16384 2023-07-21T19:07:26.439 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:26.439 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: "668869f5-a83a-4d36-a914-470c24366c9b": { 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "668869f5-a83a-4d36-a914-470c24366c9b", 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:07:26.440 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:07:26.441 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 msgr2=0x7fd4ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 0x7fd4ac0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd4bc009610 tx=0x7fd4bc009460 comp rx=0 tx=0).stop 2023-07-21T19:07:26.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 msgr2=0x7fd4c4197f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fd4b800d950 tx=0x7fd4b800dd10 comp rx=0 tx=0).stop 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 shutdown_connections 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7fd4ac0a1150 0x7fd4ac0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4c41057a0 0x7fd4c419efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4c4100560 0x7fd4c4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 --2- 172.21.15.5:0/1007657432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4c4102e80 0x7fd4c4197f60 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:26.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 >> 172.21.15.5:0/1007657432 conn(0x7fd4c40fa000 msgr2=0x7fd4c40fec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 shutdown_connections 2023-07-21T19:07:26.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:26.421+0000 7fd4c89ff700 1 -- 172.21.15.5:0/1007657432 wait complete. 2023-07-21T19:07:26.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:26 smithi005 ceph-mon[126460]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T19:07:26.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:26 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1941530076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:26.446 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:26 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/423702056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:26.448 INFO:tasks.cephfs.filesystem:scrub status for tag:668869f5-a83a-4d36-a914-470c24366c9b - {'path': '/', 'tag': '668869f5-a83a-4d36-a914-470c24366c9b', 'options': 'recursive,force'} 2023-07-21T19:07:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:26 smithi156 ceph-mon[128316]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T19:07:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:26 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1941530076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:26.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:26 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/423702056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:26.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:26 smithi101 ceph-mon[128136]: pgmap v356: 97 pgs: 97 active+clean; 643 KiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 938 B/s wr, 7 op/s 2023-07-21T19:07:26.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:26 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1941530076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:26.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:26 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/423702056' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:07:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.369+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f8148095b60 con 0x7f815010c0d0 2023-07-21T19:07:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.371+0000 7f813f7fe700 1 -- 172.21.15.5:0/1258157078 <== mon.0 v2:172.21.15.5:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f81480aaec0 con 0x7f815010c0d0 2023-07-21T19:07:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 msgr2=0x7f81380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 0x7f81380a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f8150074920 tx=0x7f8140009260 comp rx=0 tx=0).stop 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 msgr2=0x7f815019d290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f8148007ee0 tx=0x7f814800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 shutdown_connections 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f81380a1150 0x7f81380a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f815010c0d0 0x7f815019d290 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8150072f70 0x7f815019bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 --2- 172.21.15.5:0/1258157078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8150074550 0x7f815019c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 >> 172.21.15.5:0/1258157078 conn(0x7f81500fb0e0 msgr2=0x7f81500fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 shutdown_connections 2023-07-21T19:07:27.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.374+0000 7f8154fff700 1 -- 172.21.15.5:0/1258157078 wait complete. 2023-07-21T19:07:27.405 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 -- 172.21.15.5:0/3420808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c1079c0 msgr2=0x7f494c109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 --2- 172.21.15.5:0/3420808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c1079c0 0x7f494c109e40 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f494400b0a0 tx=0x7f494400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 -- 172.21.15.5:0/3420808340 shutdown_connections 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 --2- 172.21.15.5:0/3420808340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c1079c0 0x7f494c109e40 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 --2- 172.21.15.5:0/3420808340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c074550 0x7f494c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 --2- 172.21.15.5:0/3420808340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c072f70 0x7f494c0733e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 -- 172.21.15.5:0/3420808340 >> 172.21.15.5:0/3420808340 conn(0x7f494c0fb0e0 msgr2=0x7f494c0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 -- 172.21.15.5:0/3420808340 shutdown_connections 2023-07-21T19:07:27.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.608+0000 7f49525ff700 1 -- 172.21.15.5:0/3420808340 wait complete. 2023-07-21T19:07:27.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.610+0000 7f49525ff700 1 Processor -- start 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.610+0000 7f49525ff700 1 -- start start 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.610+0000 7f49525ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.610+0000 7f49525ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c074550 0x7f494c198100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f49525ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c1079c0 0x7f494c198f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f49525ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c1995e0 con 0x7f494c072f70 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f49525ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c199750 con 0x7f494c074550 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f49525ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c199910 con 0x7f494c1079c0 2023-07-21T19:07:27.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c074550 0x7f494c198100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57688/0 (socket says 172.21.15.5:57688) 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f4950b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c1079c0 0x7f494c198f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 -- 172.21.15.5:0/2962507040 learned_addr learned my addr 172.21.15.5:0/2962507040 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c1079c0 msgr2=0x7f494c198f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c1079c0 0x7f494c198f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c074550 msgr2=0x7f494c198100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c074550 0x7f494c198100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.611+0000 7f494bfff700 1 -- 172.21.15.5:0/2962507040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4944009d00 con 0x7f494c072f70 2023-07-21T19:07:27.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f494bfff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e970 tx=0x7f493c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c009790 con 0x7f494c072f70 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f494c199bf0 con 0x7f494c072f70 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f493c00c7e0 con 0x7f494c072f70 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f493c005440 con 0x7f494c072f70 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.612+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f494c1a03b0 con 0x7f494c072f70 2023-07-21T19:07:27.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.613+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f494c106070 con 0x7f494c072f70 2023-07-21T19:07:27.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.615+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f493c00c950 con 0x7f494c072f70 2023-07-21T19:07:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.617+0000 7f49497fa700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 0x7f49340a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.617+0000 7f494b7fe700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 msgr2=0x7f49340a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.617+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f493c014070 con 0x7f494c072f70 2023-07-21T19:07:27.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.617+0000 7f494b7fe700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 0x7f49340a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T19:07:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.623+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f493c096450 con 0x7f494c072f70 2023-07-21T19:07:27.650 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:27.651 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:27.651 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:07:27.651 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.652 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:07:27.652 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.652 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T19:07:27.653 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:07:27.654 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.654 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:07:27.654 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.654 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T19:07:27.655 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:27.655 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:27.655 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-mgr[130412]: -- 172.21.15.156:0/3211875548 <== mon.0 v2:172.21.15.5:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561906b351e0 con 0x561906b0a800 2023-07-21T19:07:27.656 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:27.643+0000 7f3707b56200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T19:07:27.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:07:27.664 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:07:27.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:07:27.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-07-21T19:07:27.666 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setuser ceph since I am not root 2023-07-21T19:07:27.666 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: ignoring --setgroup ceph since I am not root 2023-07-21T19:07:27.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.818+0000 7f494b7fe700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 msgr2=0x7f49340a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:27.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.818+0000 7f494b7fe700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 0x7f49340a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T19:07:27.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.837+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f494c02ca50 con 0x7f494c072f70 2023-07-21T19:07:27.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.843+0000 7f49497fa700 1 -- 172.21.15.5:0/2962507040 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f493c099a70 con 0x7f494c072f70 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 msgr2=0x7f49340a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 0x7f49340a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 msgr2=0x7f494c197bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f493c00e970 tx=0x7f493c00ec80 comp rx=0 tx=0).stop 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 shutdown_connections 2023-07-21T19:07:27.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f49340a11a0 0x7f49340a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f494c1079c0 0x7f494c198f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f494c072f70 0x7f494c197bc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 --2- 172.21.15.5:0/2962507040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f494c074550 0x7f494c198100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 >> 172.21.15.5:0/2962507040 conn(0x7f494c0fb0e0 msgr2=0x7f494c0fd4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:27.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 shutdown_connections 2023-07-21T19:07:27.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:27.845+0000 7f49525ff700 1 -- 172.21.15.5:0/2962507040 wait complete. 2023-07-21T19:07:27.873 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-07-21T19:07:27.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:27.654+0000 7fca1c36d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T19:07:27.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:27.762+0000 7fca1c36d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:27.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:27.885+0000 7fca1c36d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T19:07:27.929 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:27.748+0000 7f3707b56200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:27.930 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:27.863+0000 7f3707b56200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T19:07:27.983 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:27.639+0000 7fb11fda3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-07-21T19:07:27.984 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:27.742+0000 7fb11fda3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:27.984 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:27.855+0000 7fb11fda3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-07-21T19:07:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 -- 172.21.15.5:0/3954715511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 msgr2=0x7f97cc0fff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 --2- 172.21.15.5:0/3954715511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc0fff10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f97bc0098f0 tx=0x7f97bc009c00 comp rx=0 tx=0).stop 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 -- 172.21.15.5:0/3954715511 shutdown_connections 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 --2- 172.21.15.5:0/3954715511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc101680 0x7f97cc107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 --2- 172.21.15.5:0/3954715511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc0fff10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 --2- 172.21.15.5:0/3954715511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc100cf0 0x7f97cc101140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.066+0000 7f97d0bff700 1 -- 172.21.15.5:0/3954715511 >> 172.21.15.5:0/3954715511 conn(0x7f97cc0fb0e0 msgr2=0x7f97cc0fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.067+0000 7f97d0bff700 1 -- 172.21.15.5:0/3954715511 shutdown_connections 2023-07-21T19:07:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.067+0000 7f97d0bff700 1 -- 172.21.15.5:0/3954715511 wait complete. 2023-07-21T19:07:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 Processor -- start 2023-07-21T19:07:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 -- start start 2023-07-21T19:07:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc100cf0 0x7f97cc195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc101680 0x7f97cc196c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc1972f0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc197460 con 0x7f97cc101680 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97d0bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc197620 con 0x7f97cc100cf0 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57694/0 (socket says 172.21.15.5:57694) 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97ca59c700 1 -- 172.21.15.5:0/1754825835 learned_addr learned my addr 172.21.15.5:0/1754825835 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97c3fff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc100cf0 0x7f97cc195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.068+0000 7f97cad9d700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc101680 0x7f97cc196c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc100cf0 msgr2=0x7f97cc195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc100cf0 0x7f97cc195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc101680 msgr2=0x7f97cc196c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc101680 0x7f97cc196c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 -- 172.21.15.5:0/1754825835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b40095f0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97ca59c700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f97bc000c00 tx=0x7f97bc00bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc01c070 con 0x7f97cc0ffb00 2023-07-21T19:07:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97bc009520 con 0x7f97cc0ffb00 2023-07-21T19:07:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97bc00ba90 con 0x7f97cc0ffb00 2023-07-21T19:07:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc0038e0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.069+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc19dee0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.070+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97cc04e2f0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.072+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f97bc020030 con 0x7f97cc0ffb00 2023-07-21T19:07:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.074+0000 7f97c37fe700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 0x7f97b80a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.074+0000 7f97c3fff700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 msgr2=0x7f97b80a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:28.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.074+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f97bc0cd8d0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.074+0000 7f97c3fff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 0x7f97b80a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T19:07:28.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.077+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f97bc097060 con 0x7f97cc0ffb00 2023-07-21T19:07:28.254 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:28.012+0000 7fca1c36d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T19:07:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.274+0000 7f97c3fff700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 msgr2=0x7f97b80a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:28.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.274+0000 7f97c3fff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 0x7f97b80a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T19:07:28.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.286+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f97cc106af0 con 0x7f97cc0ffb00 2023-07-21T19:07:28.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.290+0000 7f97c37fe700 1 -- 172.21.15.5:0/1754825835 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f97bc09a680 con 0x7f97cc0ffb00 2023-07-21T19:07:28.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 msgr2=0x7f97b80a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 0x7f97b80a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 msgr2=0x7f97cc1958d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f97bc000c00 tx=0x7f97bc00bf60 comp rx=0 tx=0).stop 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 shutdown_connections 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f97b80a1220 0x7f97b80a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc101680 0x7f97cc196c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc0ffb00 0x7f97cc1958d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 --2- 172.21.15.5:0/1754825835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc100cf0 0x7f97cc195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 >> 172.21.15.5:0/1754825835 conn(0x7f97cc0fb0e0 msgr2=0x7f97cc0fd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 shutdown_connections 2023-07-21T19:07:28.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.293+0000 7f97d0bff700 1 -- 172.21.15.5:0/1754825835 wait complete. 2023-07-21T19:07:28.323 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-07-21T19:07:28.340 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:27.972+0000 7fb11fda3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T19:07:28.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:28.190+0000 7fb11fda3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T19:07:28.368 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:27.983+0000 7f3707b56200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-07-21T19:07:28.368 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:28.207+0000 7f3707b56200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T19:07:28.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 -- 172.21.15.5:0/1549522465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 msgr2=0x7f669c0733e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 --2- 172.21.15.5:0/1549522465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c0733e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f668c009990 tx=0x7f668c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 -- 172.21.15.5:0/1549522465 shutdown_connections 2023-07-21T19:07:28.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 --2- 172.21.15.5:0/1549522465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 0x7f669c10e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 --2- 172.21.15.5:0/1549522465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 0x7f669c0729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 --2- 172.21.15.5:0/1549522465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c0733e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 -- 172.21.15.5:0/1549522465 >> 172.21.15.5:0/1549522465 conn(0x7f669c0fb130 msgr2=0x7f669c0fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 -- 172.21.15.5:0/1549522465 shutdown_connections 2023-07-21T19:07:28.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.564+0000 7f66a3bff700 1 -- 172.21.15.5:0/1549522465 wait complete. 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.565+0000 7f66a3bff700 1 Processor -- start 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.565+0000 7f66a3bff700 1 -- start start 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c19bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 0x7f669c19c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 0x7f669c19d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c19d900 con 0x7f669c10c0e0 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c19da70 con 0x7f669c074550 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f669c19dc30 con 0x7f669c072f70 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c19bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c19bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52834/0 (socket says 172.21.15.5:52834) 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 -- 172.21.15.5:0/173274593 learned_addr learned my addr 172.21.15.5:0/173274593 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 0x7f669c19c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a219c700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 0x7f669c19d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:28.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 msgr2=0x7f669c19c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 0x7f669c19c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 msgr2=0x7f669c19d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 0x7f669c19d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a199b700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6698009520 con 0x7f669c072f70 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a219c700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f669c10c0e0 0x7f669c19d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:28.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.566+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f669c074550 0x7f669c19c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f66a199b700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c19bee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f669800ea30 tx=0x7f669800ed40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698009820 con 0x7f669c072f70 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c0095f0 con 0x7f669c072f70 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66980044b0 con 0x7f669c072f70 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6698003ce0 con 0x7f669c072f70 2023-07-21T19:07:28.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.567+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669c1a44f0 con 0x7f669c072f70 2023-07-21T19:07:28.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.568+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669c101c30 con 0x7f669c072f70 2023-07-21T19:07:28.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.572+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 19) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f6698003ea0 con 0x7f669c072f70 2023-07-21T19:07:28.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.573+0000 7f6692ffd700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:28.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.573+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:28.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.573+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 8211+0+0 (secure 0 0 0) 0x7f6698014070 con 0x7f669c072f70 2023-07-21T19:07:28.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.573+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-07-21T19:07:28.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.573+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f669809a480 con 0x7f669c072f70 2023-07-21T19:07:28.668 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:28.243+0000 7fca1c36d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-07-21T19:07:28.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T19:07:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-mon[126460]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-21T19:07:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2962507040' entity='client.admin' 2023-07-21T19:07:28.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1754825835' entity='client.admin' 2023-07-21T19:07:28.679 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:28.602+0000 7f3707b56200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T19:07:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T19:07:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-mon[128316]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-21T19:07:28.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2962507040' entity='client.admin' 2023-07-21T19:07:28.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1754825835' entity='client.admin' 2023-07-21T19:07:28.701 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:28.575+0000 7fb11fda3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T19:07:28.702 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1258157078' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-07-21T19:07:28.702 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-mon[128136]: mgrmap e19: x(active, since 9m), standbys: z, y 2023-07-21T19:07:28.702 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2962507040' entity='client.admin' 2023-07-21T19:07:28.703 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1754825835' entity='client.admin' 2023-07-21T19:07:28.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.774+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:28.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.774+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-07-21T19:07:28.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:28.812+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f669c060950 con 0x7f66880a11a0 2023-07-21T19:07:28.963 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:28.664+0000 7fca1c36d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-07-21T19:07:28.964 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:28.786+0000 7fca1c36d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T19:07:28.984 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:28.690+0000 7fb11fda3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T19:07:28.984 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:28.853+0000 7fb11fda3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T19:07:29.019 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:28.721+0000 7f3707b56200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-07-21T19:07:29.019 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:28.884+0000 7f3707b56200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T19:07:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:29.175+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:29.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:29.175+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-07-21T19:07:29.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:28.974+0000 7fb11fda3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T19:07:29.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:28.953+0000 7fca1c36d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-07-21T19:07:29.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:29.080+0000 7fca1c36d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T19:07:29.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:29.008+0000 7f3707b56200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-07-21T19:07:29.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:29.508+0000 7fb11fda3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T19:07:29.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:29.635+0000 7fca1c36d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T19:07:29.929 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:29.550+0000 7f3707b56200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-07-21T19:07:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:29.976+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:29.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:29.976+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-07-21T19:07:30.590 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:30.241+0000 7fb11fda3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T19:07:30.590 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:30.447+0000 7fb11fda3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T19:07:30.668 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:30.401+0000 7fca1c36d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T19:07:30.668 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:30.613+0000 7fca1c36d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T19:07:30.679 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:30.292+0000 7f3707b56200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-07-21T19:07:30.680 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:30.504+0000 7f3707b56200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-07-21T19:07:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:31.577+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:31.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:31.577+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-07-21T19:07:31.733 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:31.384+0000 7fb11fda3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:31.733 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:31.504+0000 7fb11fda3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T19:07:31.807 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:31.452+0000 7f3707b56200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:31.807 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:31.575+0000 7f3707b56200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T19:07:31.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:31.591+0000 7fca1c36d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:31.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:31.713+0000 7fca1c36d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-07-21T19:07:32.090 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:31.723+0000 7fb11fda3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:32.168 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:31.796+0000 7f3707b56200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:32.325 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:31.942+0000 7fca1c36d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-07-21T19:07:32.365 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:32.081+0000 7fb11fda3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T19:07:32.429 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:32.157+0000 7f3707b56200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T19:07:32.607 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:32.315+0000 7fca1c36d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-07-21T19:07:32.840 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:32.354+0000 7fb11fda3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T19:07:32.841 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:32.544+0000 7fb11fda3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T19:07:32.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:32.596+0000 7fca1c36d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T19:07:32.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:32.794+0000 7fca1c36d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T19:07:32.930 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:32.439+0000 7f3707b56200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-07-21T19:07:32.930 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:32.634+0000 7f3707b56200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-07-21T19:07:33.341 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:32.886+0000 7fb11fda3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T19:07:33.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:33.147+0000 7fca1c36d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T19:07:33.430 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:32.978+0000 7f3707b56200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-07-21T19:07:34.090 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:33.801+0000 7fb11fda3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T19:07:34.091 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:33.969+0000 7fb11fda3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T19:07:34.179 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:33.910+0000 7f3707b56200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T19:07:34.180 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:34.084+0000 7f3707b56200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T19:07:34.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:34.106+0000 7fca1c36d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-07-21T19:07:34.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:34.285+0000 7fca1c36d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-07-21T19:07:34.590 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:34.210+0000 7fb11fda3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T19:07:34.591 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:34.356+0000 7fb11fda3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:34.679 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:34.330+0000 7f3707b56200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T19:07:34.680 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:34.479+0000 7f3707b56200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:34.779+0000 7f66a119a700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.5:6800/1627438252 2023-07-21T19:07:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:34.779+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 6.400000 2023-07-21T19:07:34.918 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:34.541+0000 7fca1c36d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-07-21T19:07:34.919 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:34.693+0000 7fca1c36d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-07-21T19:07:35.090 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:34.628+0000 7fb11fda3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T19:07:35.179 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:34.754+0000 7f3707b56200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T19:07:35.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:34.978+0000 7fca1c36d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-07-21T19:07:35.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:35.746+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mgrmap(e 20) v1 ==== 135731+0+0 (secure 0 0 0) 0x7f6698096dc0 con 0x7f669c072f70 2023-07-21T19:07:36.091 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y[129590]: 2023-07-21T19:07:35.680+0000 7fb11fda3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T19:07:36.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: Standby manager daemon y restarted 2023-07-21T19:07:36.091 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: Standby manager daemon y started 2023-07-21T19:07:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T19:07:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T19:07:36.092 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:35 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:36.092 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: Standby manager daemon y restarted 2023-07-21T19:07:36.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: Standby manager daemon y started 2023-07-21T19:07:36.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T19:07:36.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:36.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T19:07:36.093 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:35 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:36.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:36.099+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 8 ==== mgrmap(e 21) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f66980cae20 con 0x7f669c072f70 2023-07-21T19:07:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:36.099+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 msgr2=0x7f66880a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:07:36.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:36.099+0000 7f6692ffd700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1627438252,v1:172.21.15.5:6801/1627438252] conn(0x7f66880a11a0 0x7f66880a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:36.180 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z[130347]: 2023-07-21T19:07:35.831+0000 7f3707b56200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T19:07:36.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: Standby manager daemon y restarted 2023-07-21T19:07:36.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: Standby manager daemon y started 2023-07-21T19:07:36.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-07-21T19:07:36.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:36.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-07-21T19:07:36.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:35 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.101:0/480956716' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:36.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.081+0000 7fca1c36d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-07-21T19:07:36.418 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.312+0000 7fc97b7cc700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.312+0000 7fc97b7cc700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.312+0000 7fc97b7cc700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.312+0000 7fc97b7cc700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.312+0000 7fc97b7cc700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.314+0000 7fc9787c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.314+0000 7fc9787c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.419 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.314+0000 7fc9787c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.420 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.314+0000 7fc9787c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.420 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:07:36.314+0000 7fc9787c6700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:07:36.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-07-21T19:07:36.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: Standby manager daemon z restarted 2023-07-21T19:07:36.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: Standby manager daemon z started 2023-07-21T19:07:36.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T19:07:36.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:36.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T19:07:36.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:36.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: Active manager daemon x restarted 2023-07-21T19:07:36.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: Activating manager daemon x 2023-07-21T19:07:36.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T19:07:36.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: mgrmap e21: x(active, starting, since 0.0141327s), standbys: y, z 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:36.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:36.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T19:07:36.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:07:36.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:07:36.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:07:36.849 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: Manager daemon x is now available 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:36.850 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:36.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:36.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:36.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:36.851 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:36 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:36.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: Standby manager daemon z restarted 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: Standby manager daemon z started 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T19:07:36.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: Active manager daemon x restarted 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: Activating manager daemon x 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: mgrmap e21: x(active, starting, since 0.0141327s), standbys: y, z 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:07:36.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:36.921 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:36.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:36.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:36.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:36.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:36.922 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:36.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T19:07:36.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T19:07:36.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T19:07:36.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T19:07:36.923 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:07:36.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:07:36.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:07:36.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:07:36.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:07:36.924 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:07:36.925 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:07:36.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T19:07:36.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T19:07:36.926 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: Manager daemon x is now available 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:36.927 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:36 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.107+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mon.1 v2:172.21.15.156:3300/0 9 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f6698004150 con 0x7f669c072f70 2023-07-21T19:07:37.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.108+0000 7f6692ffd700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6688027f50 0x7f6688028320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.108+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f669c060950 con 0x7f6688027f50 2023-07-21T19:07:37.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.110+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6688027f50 0x7f6688028320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.114+0000 7f66a119a700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6688027f50 0x7f6688028320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f668c0098e0 tx=0x7f668c00b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: mgrmap e20: x(active, since 9m), standbys: z, y 2023-07-21T19:07:37.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: Standby manager daemon z restarted 2023-07-21T19:07:37.180 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: Standby manager daemon z started 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.? 172.21.15.156:0/2736898388' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: Active manager daemon x restarted 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: Activating manager daemon x 2023-07-21T19:07:37.181 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: osdmap e88: 12 total, 12 up, 12 in 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: mgrmap e21: x(active, starting, since 0.0141327s), standbys: y, z 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-07-21T19:07:37.182 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-07-21T19:07:37.183 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-07-21T19:07:37.184 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-07-21T19:07:37.185 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: Manager daemon x is now available 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:37.186 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:37.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-07-21T19:07:37.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:37.187 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:36 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-07-21T19:07:37.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.179+0000 7f6692ffd700 1 -- 172.21.15.5:0/173274593 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f669c060950 con 0x7f6688027f50 2023-07-21T19:07:37.191 INFO:teuthology.orchestra.run.smithi005.stdout:Schedule set for path / 2023-07-21T19:07:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6688027f50 msgr2=0x7f6688028320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6688027f50 0x7f6688028320 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f668c0098e0 tx=0x7f668c00b260 comp rx=0 tx=0).stop 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 msgr2=0x7f669c19bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 --2- 172.21.15.5:0/173274593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f669c072f70 0x7f669c19bee0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f669800ea30 tx=0x7f669800ed40 comp rx=0 tx=0).stop 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a219c700 1 -- 172.21.15.5:0/173274593 reap_dead start 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 shutdown_connections 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 >> 172.21.15.5:0/173274593 conn(0x7f669c0fb130 msgr2=0x7f669c0fd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 shutdown_connections 2023-07-21T19:07:37.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.183+0000 7f66a3bff700 1 -- 172.21.15.5:0/173274593 wait complete. 2023-07-21T19:07:37.214 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-07-21T19:07:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 -- 172.21.15.5:0/1753189160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0144100cf0 msgr2=0x7f0144101140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 --2- 172.21.15.5:0/1753189160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0144100cf0 0x7f0144101140 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f0130009990 tx=0x7f0130009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 -- 172.21.15.5:0/1753189160 shutdown_connections 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 --2- 172.21.15.5:0/1753189160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144101680 0x7f0144107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 --2- 172.21.15.5:0/1753189160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01440ffb00 0x7f01440fff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 --2- 172.21.15.5:0/1753189160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0144100cf0 0x7f0144101140 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 -- 172.21.15.5:0/1753189160 >> 172.21.15.5:0/1753189160 conn(0x7f01440fb0e0 msgr2=0x7f01440fd550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 -- 172.21.15.5:0/1753189160 shutdown_connections 2023-07-21T19:07:37.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.424+0000 7f014adff700 1 -- 172.21.15.5:0/1753189160 wait complete. 2023-07-21T19:07:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 Processor -- start 2023-07-21T19:07:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 -- start start 2023-07-21T19:07:37.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144100cf0 0x7f0144072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 0x7f0144073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419f7f0 con 0x7f01440ffb00 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419f930 con 0x7f0144100cf0 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.425+0000 7f014adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f014419fa70 con 0x7f0144101680 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0148b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0143fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144100cf0 0x7f0144072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 0x7f0144073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0148b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44166/0 (socket says 172.21.15.5:44166) 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0148b9b700 1 -- 172.21.15.5:0/3911431889 learned_addr learned my addr 172.21.15.5:0/3911431889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 -- 172.21.15.5:0/3911431889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144100cf0 msgr2=0x7f0144072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144100cf0 0x7f0144072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 -- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 msgr2=0x7f0144072900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 -- 172.21.15.5:0/3911431889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01300095f0 con 0x7f0144101680 2023-07-21T19:07:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0148b9b700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f014939c700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 0x7f0144073380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f013c00e970 tx=0x7f013c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.426+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c004220 con 0x7f0144101680 2023-07-21T19:07:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.427+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f014419fd00 con 0x7f0144101680 2023-07-21T19:07:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.427+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f013c004cc0 con 0x7f0144101680 2023-07-21T19:07:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.427+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f013c005590 con 0x7f0144101680 2023-07-21T19:07:37.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.427+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01441a0190 con 0x7f0144101680 2023-07-21T19:07:37.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.428+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f014404e2f0 con 0x7f0144101680 2023-07-21T19:07:37.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.430+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f013c004750 con 0x7f0144101680 2023-07-21T19:07:37.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.431+0000 7f0141ffb700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 0x7f01340a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.432+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f013c014070 con 0x7f0144101680 2023-07-21T19:07:37.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.432+0000 7f0148b9b700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 0x7f01340a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.432+0000 7f0148b9b700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 0x7f01340a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0144074210 tx=0x7f013800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.433+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f013c095d80 con 0x7f0144101680 2023-07-21T19:07:37.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.685+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7f0144060950 con 0x7f01340a0fd0 2023-07-21T19:07:37.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.747+0000 7f0141ffb700 1 -- 172.21.15.5:0/3911431889 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7f0144060950 con 0x7f01340a0fd0 2023-07-21T19:07:37.758 INFO:teuthology.orchestra.run.smithi005.stdout:Retention added to path / 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 msgr2=0x7f01340a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 0x7f01340a3480 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0144074210 tx=0x7f013800b040 comp rx=0 tx=0).stop 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 msgr2=0x7f0144073380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 0x7f0144073380 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f013c00e970 tx=0x7f013c00ec80 comp rx=0 tx=0).stop 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 shutdown_connections 2023-07-21T19:07:37.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0144101680 0x7f0144073380 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01340a0fd0 0x7f01340a3480 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01440ffb00 0x7f0144072900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 --2- 172.21.15.5:0/3911431889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0144100cf0 0x7f0144072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 >> 172.21.15.5:0/3911431889 conn(0x7f01440fb0e0 msgr2=0x7f01440fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 shutdown_connections 2023-07-21T19:07:37.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.749+0000 7f014adff700 1 -- 172.21.15.5:0/3911431889 wait complete. 2023-07-21T19:07:37.778 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T19:07:37.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 -- 172.21.15.5:0/1646172246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 msgr2=0x7f1c3c107c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 --2- 172.21.15.5:0/1646172246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c107c40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009e00 tx=0x7f1c3400b330 comp rx=0 tx=0).stop 2023-07-21T19:07:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 -- 172.21.15.5:0/1646172246 shutdown_connections 2023-07-21T19:07:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 --2- 172.21.15.5:0/1646172246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c107c40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 --2- 172.21.15.5:0/1646172246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 --2- 172.21.15.5:0/1646172246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 0x7f1c3c105320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 -- 172.21.15.5:0/1646172246 >> 172.21.15.5:0/1646172246 conn(0x7f1c3c0fa000 msgr2=0x7f1c3c0fc450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:37.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 -- 172.21.15.5:0/1646172246 shutdown_connections 2023-07-21T19:07:37.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.976+0000 7f1c429ff700 1 -- 172.21.15.5:0/1646172246 wait complete. 2023-07-21T19:07:37.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 Processor -- start 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 -- start start 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 0x7f1c3c1980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c198f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c199580 con 0x7f1c3c105860 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c1996f0 con 0x7f1c3c102f40 2023-07-21T19:07:37.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.977+0000 7f1c429ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c3c1998b0 con 0x7f1c3c100620 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c40f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c198f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45090/0 (socket says 172.21.15.5:45090) 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 0x7f1c3c1980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 -- 172.21.15.5:0/3327149866 learned_addr learned my addr 172.21.15.5:0/3327149866 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 -- 172.21.15.5:0/3327149866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 msgr2=0x7f1c3c1980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 0x7f1c3c1980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 -- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 msgr2=0x7f1c3c198f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c198f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c3bfff700 1 -- 172.21.15.5:0/3327149866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c2c0095f0 con 0x7f1c3c100620 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.978+0000 7f1c40f9c700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c3bfff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00ec70 tx=0x7f1c2c00ef80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c0099d0 con 0x7f1c3c100620 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c34005e70 con 0x7f1c3c100620 2023-07-21T19:07:37.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c2c0044b0 con 0x7f1c3c100620 2023-07-21T19:07:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c2c003e30 con 0x7f1c3c100620 2023-07-21T19:07:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.979+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3c1a0170 con 0x7f1c3c100620 2023-07-21T19:07:37.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.980+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c3c1073d0 con 0x7f1c3c100620 2023-07-21T19:07:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.982+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f1c2c003f90 con 0x7f1c3c100620 2023-07-21T19:07:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.983+0000 7f1c397fa700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 0x7f1c240a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.984+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1c2c014070 con 0x7f1c3c100620 2023-07-21T19:07:37.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.984+0000 7f1c3b7fe700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 0x7f1c240a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:37.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.984+0000 7f1c3b7fe700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 0x7f1c240a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009c60 tx=0x7f1c30009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:37.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:37.988+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1c2c0963a0 con 0x7f1c3c100620 2023-07-21T19:07:38.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.228+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c3c060950 con 0x7f1c240a0f50 2023-07-21T19:07:38.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.236+0000 7f1c397fa700 1 -- 172.21.15.5:0/3327149866 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f1c3c060950 con 0x7f1c240a0f50 2023-07-21T19:07:38.247 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T19:07:37", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-07-21T19:07:38.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 msgr2=0x7f1c240a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 0x7f1c240a3400 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1c30009c60 tx=0x7f1c30009260 comp rx=0 tx=0).stop 2023-07-21T19:07:38.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 msgr2=0x7f1c3c197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1c2c00ec70 tx=0x7f1c2c00ef80 comp rx=0 tx=0).stop 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 shutdown_connections 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c240a0f50 0x7f1c240a3400 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c3c105860 0x7f1c3c198f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c3c100620 0x7f1c3c197b60 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 --2- 172.21.15.5:0/3327149866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c3c102f40 0x7f1c3c1980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 >> 172.21.15.5:0/3327149866 conn(0x7f1c3c0fa000 msgr2=0x7f1c3c0feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:38.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 shutdown_connections 2023-07-21T19:07:38.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.239+0000 7f1c429ff700 1 -- 172.21.15.5:0/3327149866 wait complete. 2023-07-21T19:07:38.269 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T19:07:38.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: [21/Jul/2023:19:07:36] ENGINE Bus STARTING 2023-07-21T19:07:38.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: [21/Jul/2023:19:07:36] ENGINE Serving on http://172.21.15.5:8765 2023-07-21T19:07:38.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: [21/Jul/2023:19:07:37] ENGINE Serving on https://172.21.15.5:7150 2023-07-21T19:07:38.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: [21/Jul/2023:19:07:37] ENGINE Bus STARTED 2023-07-21T19:07:38.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: mgrmap e22: x(active, since 1.02232s), standbys: y, z 2023-07-21T19:07:38.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:38.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:38 smithi005 ceph-mon[126460]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:38.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: [21/Jul/2023:19:07:36] ENGINE Bus STARTING 2023-07-21T19:07:38.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: [21/Jul/2023:19:07:36] ENGINE Serving on http://172.21.15.5:8765 2023-07-21T19:07:38.430 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: [21/Jul/2023:19:07:37] ENGINE Serving on https://172.21.15.5:7150 2023-07-21T19:07:38.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: [21/Jul/2023:19:07:37] ENGINE Bus STARTED 2023-07-21T19:07:38.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: mgrmap e22: x(active, since 1.02232s), standbys: y, z 2023-07-21T19:07:38.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:38.431 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:38 smithi156 ceph-mon[128316]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 -- 172.21.15.5:0/1331964743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 msgr2=0x7f5b200fff20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 --2- 172.21.15.5:0/1331964743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b200fff20 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c009990 tx=0x7f5b1c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 -- 172.21.15.5:0/1331964743 shutdown_connections 2023-07-21T19:07:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 --2- 172.21.15.5:0/1331964743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20101690 0x7f5b20107f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 --2- 172.21.15.5:0/1331964743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b200fff20 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 --2- 172.21.15.5:0/1331964743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20100d00 0x7f5b20101150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 -- 172.21.15.5:0/1331964743 >> 172.21.15.5:0/1331964743 conn(0x7f5b200fb130 msgr2=0x7f5b200fd560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 -- 172.21.15.5:0/1331964743 shutdown_connections 2023-07-21T19:07:38.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.481+0000 7f5b283ff700 1 -- 172.21.15.5:0/1331964743 wait complete. 2023-07-21T19:07:38.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 Processor -- start 2023-07-21T19:07:38.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 -- start start 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20100d00 0x7f5b20072e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 0x7f5b20073380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019f810 con 0x7f5b200ffb10 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019f950 con 0x7f5b20101690 2023-07-21T19:07:38.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.483+0000 7f5b283ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b2019fa90 con 0x7f5b20100d00 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44200/0 (socket says 172.21.15.5:44200) 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 -- 172.21.15.5:0/447675438 learned_addr learned my addr 172.21.15.5:0/447675438 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2599a700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20100d00 0x7f5b20072e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2699c700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 0x7f5b20073380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 -- 172.21.15.5:0/447675438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20100d00 msgr2=0x7f5b20072e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20100d00 0x7f5b20072e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 -- 172.21.15.5:0/447675438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 msgr2=0x7f5b20073380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 0x7f5b20073380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 -- 172.21.15.5:0/447675438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0095f0 con 0x7f5b200ffb10 2023-07-21T19:07:38.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2699c700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 0x7f5b20073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.484+0000 7f5b2619b700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098e0 tx=0x7f5b1c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.485+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c01d070 con 0x7f5b200ffb10 2023-07-21T19:07:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.485+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b1c00bdd0 con 0x7f5b200ffb10 2023-07-21T19:07:38.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.485+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2019fcc0 con 0x7f5b200ffb10 2023-07-21T19:07:38.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.485+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1c022d80 con 0x7f5b200ffb10 2023-07-21T19:07:38.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.485+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a0140 con 0x7f5b200ffb10 2023-07-21T19:07:38.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.486+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b20107150 con 0x7f5b200ffb10 2023-07-21T19:07:38.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.488+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f5b1c022470 con 0x7f5b200ffb10 2023-07-21T19:07:38.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.489+0000 7f5b177fe700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 0x7f5b0c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:38.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.489+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5b1c0ce310 con 0x7f5b200ffb10 2023-07-21T19:07:38.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.490+0000 7f5b2599a700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 0x7f5b0c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:38.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.490+0000 7f5b2599a700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 0x7f5b0c0a3880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b20073c00 tx=0x7f5b10009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:38.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.494+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f5b1c097610 con 0x7f5b200ffb10 2023-07-21T19:07:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: [21/Jul/2023:19:07:36] ENGINE Bus STARTING 2023-07-21T19:07:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: [21/Jul/2023:19:07:36] ENGINE Serving on http://172.21.15.5:8765 2023-07-21T19:07:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: [21/Jul/2023:19:07:37] ENGINE Serving on https://172.21.15.5:7150 2023-07-21T19:07:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: [21/Jul/2023:19:07:37] ENGINE Bus STARTED 2023-07-21T19:07:38.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: mgrmap e22: x(active, since 1.02232s), standbys: y, z 2023-07-21T19:07:38.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: from='client.25279 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:38.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:38 smithi101 ceph-mon[128136]: pgmap v3: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:38.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.754+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5b20060950 con 0x7f5b0c0a13d0 2023-07-21T19:07:38.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.762+0000 7f5b177fe700 1 -- 172.21.15.5:0/447675438 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f5b20060950 con 0x7f5b0c0a13d0 2023-07-21T19:07:38.772 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1M 6M3h 2023-07-21T19:07:38.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 msgr2=0x7f5b0c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 0x7f5b0c0a3880 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f5b20073c00 tx=0x7f5b10009260 comp rx=0 tx=0).stop 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 msgr2=0x7f5b20072900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0098e0 tx=0x7f5b1c004730 comp rx=0 tx=0).stop 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 shutdown_connections 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b20101690 0x7f5b20073380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b0c0a13d0 0x7f5b0c0a3880 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b200ffb10 0x7f5b20072900 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 --2- 172.21.15.5:0/447675438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b20100d00 0x7f5b20072e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 >> 172.21.15.5:0/447675438 conn(0x7f5b200fb130 msgr2=0x7f5b20104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 shutdown_connections 2023-07-21T19:07:38.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:38.764+0000 7f5b283ff700 1 -- 172.21.15.5:0/447675438 wait complete. 2023-07-21T19:07:38.794 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-07-21T19:07:38.872 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-07-21T19:07:38.883 INFO:teuthology.run_tasks:Running task background_exec... 2023-07-21T19:07:38.905 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:07:38.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-07-21T19:07:38.910 INFO:teuthology.run_tasks:Running task exec... 2023-07-21T19:07:38.920 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T19:07:38.920 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:07:38.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -l /tmp/cephfs/ && sudo df -h /tmp/cephfs/' 2023-07-21T19:07:39.030 INFO:teuthology.orchestra.run.smithi005.stdout:total 0 2023-07-21T19:07:39.077 INFO:teuthology.orchestra.run.smithi005.stdout:Filesystem Size Used Avail Use% Mounted on 2023-07-21T19:07:39.078 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-fuse 340G 0 340G 0% /tmp/cephfs 2023-07-21T19:07:39.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -m 755 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres && sudo chown postgres:postgres /tmp/cephfs/postgres' 2023-07-21T19:07:39.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- mkdir -m 700 --context=system_u:system_r:postgresql_t:s0 /tmp/cephfs/postgres/data' 2023-07-21T19:07:39.336 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo semanage fcontext -a -t postgresql_db_t "/tmp/cephfs/postgres(/.*)?"' 2023-07-21T19:07:39.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:39 smithi101 ceph-mon[128136]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:39 smithi101 ceph-mon[128136]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:39.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:39 smithi101 ceph-mon[128136]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:39 smithi101 ceph-mon[128136]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-21T19:07:39.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:39 smithi005 ceph-mon[126460]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:39 smithi005 ceph-mon[126460]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:39.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:39 smithi005 ceph-mon[126460]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:39 smithi005 ceph-mon[126460]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-21T19:07:39.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:39 smithi156 ceph-mon[128316]: from='client.25315 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:39 smithi156 ceph-mon[128316]: pgmap v4: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:39.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:39 smithi156 ceph-mon[128316]: from='client.25321 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:39.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:39 smithi156 ceph-mon[128316]: mgrmap e23: x(active, since 2s), standbys: y, z 2023-07-21T19:07:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.592 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='client.15585 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:40.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:41.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:41 smithi101 ceph-mon[128136]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:41.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:41 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:07:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:41 smithi005 ceph-mon[126460]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:41 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:07:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:41 smithi156 ceph-mon[128316]: pgmap v5: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail 2023-07-21T19:07:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:41 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:07:42.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.671 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.673 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.674 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:42.675 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:42 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:42.743 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo grep -i postgresql /etc/selinux/targeted/contexts/files/file_contexts.local' 2023-07-21T19:07:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.844 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:42.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:42 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:42.846 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres(/.*)? system_u:object_r:postgresql_db_t:s0 2023-07-21T19:07:42.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo restorecon -R -v /tmp/cephfs/postgres' 2023-07-21T19:07:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:07:42.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:07:42.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.934 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.935 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.936 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:42.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.938 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.939 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:07:42.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:42.940 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:42 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:42.963 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-21T19:07:43.078 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2023-07-21T19:07:43.078 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:07:43.078 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 .. 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 data 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:07:43.079 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 .. 2023-07-21T19:07:43.084 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo mkdir -p /etc/systemd/system/postgresql.service.d/ && printf '"'"'[Service]\nEnvironment=PGDATA=/tmp/cephfs/postgres/data\nEnvironment=PGLOG=/tmp/cephfs/postgres/pgstartup.log\n'"'"' | sudo tee /etc/systemd/system/postgresql.service.d/env.conf' 2023-07-21T19:07:43.218 INFO:teuthology.orchestra.run.smithi005.stdout:[Service] 2023-07-21T19:07:43.218 INFO:teuthology.orchestra.run.smithi005.stdout:Environment=PGDATA=/tmp/cephfs/postgres/data 2023-07-21T19:07:43.218 INFO:teuthology.orchestra.run.smithi005.stdout:Environment=PGLOG=/tmp/cephfs/postgres/pgstartup.log 2023-07-21T19:07:43.223 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- postgresql-setup --initdb' 2023-07-21T19:07:43.435 INFO:teuthology.orchestra.run.smithi005.stderr: * Initializing database in '/tmp/cephfs/postgres/data' 2023-07-21T19:07:43.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T19:07:43.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-21T19:07:43.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.670 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:43 smithi005 ceph-mon[126460]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:07:43.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:07:43.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:07:43.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:07:43.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T19:07:43.846 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T19:07:43.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T19:07:43.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-21T19:07:43.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.847 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.848 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:43 smithi101 ceph-mon[128136]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:07:43.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:07:43.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:07:43.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:07:43.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi005:/etc/ceph/ceph.conf 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi101:/etc/ceph/ceph.conf 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi156:/etc/ceph/ceph.conf 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: pgmap v6: 97 pgs: 97 active+clean; 1.0 MiB data, 327 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 1 op/s 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:43.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:43 smithi156 ceph-mon[128316]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.conf 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:44.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:44.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:44.843 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:44 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.919 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:44.920 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:44 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi101:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi156:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi005:/etc/ceph/ceph.client.admin.keyring 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: Updating smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/config/ceph.client.admin.keyring 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.931 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:44.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:07:44.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:07:44.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:07:44.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:44 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:07:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:45 smithi101 ceph-mon[128136]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 133 KiB/s wr, 50 op/s 2023-07-21T19:07:45.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:45 smithi005 ceph-mon[126460]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 133 KiB/s wr, 50 op/s 2023-07-21T19:07:45.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:45 smithi156 ceph-mon[128316]: pgmap v7: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 133 KiB/s wr, 50 op/s 2023-07-21T19:07:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:47 smithi101 ceph-mon[128136]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T19:07:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:47 smithi005 ceph-mon[126460]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T19:07:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:47 smithi156 ceph-mon[128316]: pgmap v8: 97 pgs: 97 active+clean; 1.3 MiB data, 328 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 39 op/s 2023-07-21T19:07:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:49 smithi101 ceph-mon[128136]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.9 MiB/s wr, 57 op/s 2023-07-21T19:07:49.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:49 smithi005 ceph-mon[126460]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.9 MiB/s wr, 57 op/s 2023-07-21T19:07:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:49 smithi156 ceph-mon[128316]: pgmap v9: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 1.9 MiB/s wr, 57 op/s 2023-07-21T19:07:51.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:51 smithi101 ceph-mon[128136]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.8 MiB/s wr, 52 op/s 2023-07-21T19:07:51.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:51 smithi005 ceph-mon[126460]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.8 MiB/s wr, 52 op/s 2023-07-21T19:07:51.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:51 smithi156 ceph-mon[128316]: pgmap v10: 97 pgs: 97 active+clean; 21 MiB data, 333 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 1.8 MiB/s wr, 52 op/s 2023-07-21T19:07:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 -- 172.21.15.5:0/1589136739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1059a0 msgr2=0x7f221c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 --2- 172.21.15.5:0/1589136739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1059a0 0x7f221c105d70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f220c009990 tx=0x7f220c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 -- 172.21.15.5:0/1589136739 shutdown_connections 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 --2- 172.21.15.5:0/1589136739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 0x7f221c10c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 --2- 172.21.15.5:0/1589136739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1059a0 0x7f221c105d70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 --2- 172.21.15.5:0/1589136739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1062b0 0x7f221c067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 -- 172.21.15.5:0/1589136739 >> 172.21.15.5:0/1589136739 conn(0x7f221c074770 msgr2=0x7f221c074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 -- 172.21.15.5:0/1589136739 shutdown_connections 2023-07-21T19:07:52.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.174+0000 7f22209fe700 1 -- 172.21.15.5:0/1589136739 wait complete. 2023-07-21T19:07:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.175+0000 7f22209fe700 1 Processor -- start 2023-07-21T19:07:52.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 -- start start 2023-07-21T19:07:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 0x7f221c0fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 0x7f221c1a3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c0ff7d0 con 0x7f221c1062b0 2023-07-21T19:07:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c1a4150 con 0x7f221c1059a0 2023-07-21T19:07:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.176+0000 7f22209fe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f221c1a4290 con 0x7f221c068070 2023-07-21T19:07:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36700/0 (socket says 172.21.15.5:36700) 2023-07-21T19:07:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 -- 172.21.15.5:0/2422374881 learned_addr learned my addr 172.21.15.5:0/2422374881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221b59e700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 0x7f221c1a3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221ad9d700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 0x7f221c0fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 -- 172.21.15.5:0/2422374881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 msgr2=0x7f221c0fea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 0x7f221c0fea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 -- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 msgr2=0x7f221c1a3c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 0x7f221c1a3c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 -- 172.21.15.5:0/2422374881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f220c0095f0 con 0x7f221c1059a0 2023-07-21T19:07:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221b59e700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 0x7f221c1a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.177+0000 7f221a59c700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f221000e970 tx=0x7f221000ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.178+0000 7f2203fff700 1 -- 172.21.15.5:0/2422374881 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210009790 con 0x7f221c1059a0 2023-07-21T19:07:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.178+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f221c1a44f0 con 0x7f221c1059a0 2023-07-21T19:07:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.178+0000 7f2203fff700 1 -- 172.21.15.5:0/2422374881 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221000c7e0 con 0x7f221c1059a0 2023-07-21T19:07:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.178+0000 7f2203fff700 1 -- 172.21.15.5:0/2422374881 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2210005440 con 0x7f221c1059a0 2023-07-21T19:07:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.178+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c1a4a20 con 0x7f221c1059a0 2023-07-21T19:07:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.182+0000 7f2203fff700 1 -- 172.21.15.5:0/2422374881 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f221000c950 con 0x7f221c1059a0 2023-07-21T19:07:52.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.183+0000 7f2234b7db80 1 Processor -- start 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.183+0000 7f2234b7db80 1 -- start start 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a276f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2711a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a271700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a176c50 con 0x55913a277470 2023-07-21T19:07:52.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a1c99e0 con 0x55913a2778c0 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2234b7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a1c2e70 con 0x55913a0cd6b0 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2203fff700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 0x7f22040a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2711a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a271700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2711a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58146/0 (socket says 172.21.15.5:58146) 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221ad9d700 1 -- 172.21.15.5:0/815244659 learned_addr learned my addr 172.21.15.5:0/815244659 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:52.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221b59e700 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a276f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f221b59e700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 0x7f22040a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.184+0000 7f2203fff700 1 -- 172.21.15.5:0/2422374881 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2210014070 con 0x7f221c1059a0 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 -- 172.21.15.5:0/815244659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 msgr2=0x55913a276f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a276f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 -- 172.21.15.5:0/815244659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 msgr2=0x55913a2711a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2711a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 -- 172.21.15.5:0/815244659 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559139ff0d80 con 0x55913a2778c0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221a59c700 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a271700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22100ca610 tx=0x7f2210099cb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc43c7031a711fd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f221b59e700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 0x7f22040a3880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2214009fd0 tx=0x7f221400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.185+0000 7f22017fa700 1 -- 172.21.15.5:0/815244659 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100a6040 con 0x55913a2778c0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.186+0000 7f22017fa700 1 -- 172.21.15.5:0/815244659 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221000cc50 con 0x55913a2778c0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.186+0000 7f22017fa700 1 -- 172.21.15.5:0/815244659 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100055a0 con 0x55913a2778c0 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.186+0000 7f2234b7db80 1 -- 172.21.15.5:0/815244659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 msgr2=0x55913a271700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.186+0000 7f2234b7db80 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a271700 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f22100ca610 tx=0x7f2210099cb0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 -- 172.21.15.5:0/815244659 shutdown_connections 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a271700 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a276f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 --2- 172.21.15.5:0/815244659 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2711a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 -- 172.21.15.5:0/815244659 >> 172.21.15.5:0/815244659 conn(0x55913a266d40 msgr2=0x55913a26c340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 -- 172.21.15.5:0/815244659 shutdown_connections 2023-07-21T19:07:52.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 -- 172.21.15.5:0/815244659 wait complete. 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.187+0000 7f2234b7db80 1 -- start start 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 Processor -- start 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a277470 0x55913a27a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a2778c0 0x55913a27af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a162a60 con 0x55913a2778c0 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a162760 con 0x55913a0cd6b0 2023-07-21T19:07:52.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a0468c0 con 0x55913a277470 2023-07-21T19:07:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f2234b7db80 10 client.0 _ll_register_callbacks cb 0x55913a160700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.188+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36724/0 (socket says 172.21.15.5:36724) 2023-07-21T19:07:52.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 -- 172.21.15.5:0/4170923788 learned_addr learned my addr 172.21.15.5:0/4170923788 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221b59e700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a2778c0 0x55913a27af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221a59c700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a277470 0x55913a27a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 -- 172.21.15.5:0/4170923788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a277470 msgr2=0x55913a27a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a277470 0x55913a27a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 -- 172.21.15.5:0/4170923788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a2778c0 msgr2=0x55913a27af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a2778c0 0x55913a27af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 -- 172.21.15.5:0/4170923788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f221c1a4a20 con 0x55913a0cd6b0 2023-07-21T19:07:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f221ad9d700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f220c006010 tx=0x7f220c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66395c37e6c6bfa4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.189+0000 7f21feff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:07:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c01d070 con 0x55913a0cd6b0 2023-07-21T19:07:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55913a11c490 con 0x55913a0cd6b0 2023-07-21T19:07:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f220c004500 con 0x55913a0cd6b0 2023-07-21T19:07:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f220c0046c0 con 0x55913a0cd6b0 2023-07-21T19:07:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f2234b7db80 10 client.25364 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559139fd9c00 con 0x55913a0cd6b0 2023-07-21T19:07:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.190+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55913a11c490 con 0x55913a0cd6b0 2023-07-21T19:07:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f220c00f830 con 0x55913a0cd6b0 2023-07-21T19:07:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 10 client.25364 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 10 client.25364 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 20 client.25364 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a28ac60 0x55913a28d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 4 client.25364 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55913a1bbe50 con 0x55913a28ac60 2023-07-21T19:07:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.191+0000 7f221a59c700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a28ac60 0x55913a28d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.192+0000 7f221a59c700 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a28ac60 0x55913a28d070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31d5c95b4dd6bd5e server_cookie=f25e40bd4a637a06 in_seq=0 out_seq=0 2023-07-21T19:07:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.192+0000 7f21feff5700 10 client.25364 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.192+0000 7f21feff5700 1 -- 172.21.15.5:0/4170923788 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55913a1bbe50 con 0x55913a28ac60 2023-07-21T19:07:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.192+0000 7f21feff5700 10 client.25364 handle_command_reply: tid=0 2023-07-21T19:07:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.192+0000 7f2234b7db80 1 client.25364 shutdown 2023-07-21T19:07:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 msgr2=0x55913a27d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f220c006010 tx=0x7f220c005be0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 shutdown_connections 2023-07-21T19:07:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a28ac60 0x55913a28d070 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a2778c0 0x55913a27af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a0cd6b0 0x55913a27d720 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f21feff5700 0 client.25364 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 --2- 172.21.15.5:0/4170923788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a277470 0x55913a27a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 >> 172.21.15.5:0/4170923788 conn(0x55913a266d40 msgr2=0x55913a26b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 shutdown_connections 2023-07-21T19:07:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 1 -- 172.21.15.5:0/4170923788 wait complete. 2023-07-21T19:07:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.193+0000 7f2234b7db80 20 client.25364 trim_cache size 0 max 16384 2023-07-21T19:07:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 Processor -- start 2023-07-21T19:07:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 -- start start 2023-07-21T19:07:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a2778c0 0x55913a27d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a26da50 con 0x55913a0cd6b0 2023-07-21T19:07:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a287600 con 0x55913a277470 2023-07-21T19:07:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.210+0000 7f2234b7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a278f40 con 0x55913a2778c0 2023-07-21T19:07:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58156/0 (socket says 172.21.15.5:58156) 2023-07-21T19:07:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 -- 172.21.15.5:0/3565353666 learned_addr learned my addr 172.21.15.5:0/3565353666 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221b59e700 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a2778c0 0x55913a27d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36732/0 (socket says 172.21.15.5:36732) 2023-07-21T19:07:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 -- 172.21.15.5:0/3565353666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a2778c0 msgr2=0x55913a27d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a2778c0 0x55913a27d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 -- 172.21.15.5:0/3565353666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 msgr2=0x55913a2783c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 -- 172.21.15.5:0/3565353666 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55913a0171f0 con 0x55913a0cd6b0 2023-07-21T19:07:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221a59c700 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.211+0000 7f221ad9d700 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f220c000c00 tx=0x7f220c005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd1776e04aa05f9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.212+0000 7f2201ffb700 1 -- 172.21.15.5:0/3565353666 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c01d070 con 0x55913a0cd6b0 2023-07-21T19:07:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.212+0000 7f2201ffb700 1 -- 172.21.15.5:0/3565353666 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f220c01e040 con 0x55913a0cd6b0 2023-07-21T19:07:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.212+0000 7f2201ffb700 1 -- 172.21.15.5:0/3565353666 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f220c00ec80 con 0x55913a0cd6b0 2023-07-21T19:07:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- 172.21.15.5:0/3565353666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 msgr2=0x55913a281e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f220c000c00 tx=0x7f220c005a80 comp rx=0 tx=0).stop 2023-07-21T19:07:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- 172.21.15.5:0/3565353666 shutdown_connections 2023-07-21T19:07:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a2778c0 0x55913a27d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a0cd6b0 0x55913a281e00 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 --2- 172.21.15.5:0/3565353666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a277470 0x55913a2783c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- 172.21.15.5:0/3565353666 >> 172.21.15.5:0/3565353666 conn(0x55913a266d40 msgr2=0x55913a277cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- 172.21.15.5:0/3565353666 shutdown_connections 2023-07-21T19:07:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- 172.21.15.5:0/3565353666 wait complete. 2023-07-21T19:07:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.213+0000 7f2234b7db80 1 -- start start 2023-07-21T19:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 Processor -- start 2023-07-21T19:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a273b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a272990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a2822d0 con 0x55913a277470 2023-07-21T19:07:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a282440 con 0x55913a2778c0 2023-07-21T19:07:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55913a272f50 con 0x55913a0cd6b0 2023-07-21T19:07:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.214+0000 7f2234b7db80 10 client.0 _ll_register_callbacks cb 0x7f220c005370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58164/0 (socket says 172.21.15.5:58164) 2023-07-21T19:07:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 -- 172.21.15.5:0/2106938435 learned_addr learned my addr 172.21.15.5:0/2106938435 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221ad9d700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a273b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 -- 172.21.15.5:0/2106938435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 msgr2=0x55913a273b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a273b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 -- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 msgr2=0x55913a272990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a272990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.215+0000 7f221a59c700 1 -- 172.21.15.5:0/2106938435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559139ff0d80 con 0x55913a277470 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.216+0000 7f221a59c700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f22100ca6f0 tx=0x7f22100ca550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61666de79204f5f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.216+0000 7f21feff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.216+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22100a2330 con 0x55913a277470 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.216+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2210098280 con 0x55913a277470 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.216+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f221000cc50 con 0x55913a277470 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22100b81b0 con 0x55913a277470 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 10 client.15603 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x559139fd9c00 con 0x55913a277470 2023-07-21T19:07:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f221000cc50 con 0x55913a277470 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f221009a5e0 con 0x55913a277470 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 10 client.15603 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 10 client.15603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 20 client.15603 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a2df640 0x55913a2e1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 4 client.15603 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55913a17dff0 con 0x55913a2df640 2023-07-21T19:07:52.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.217+0000 7f221ad9d700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a2df640 0x55913a2e1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.218+0000 7f221ad9d700 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a2df640 0x55913a2e1a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7da0288b454a6f9 server_cookie=669b7df93a6a8678 in_seq=0 out_seq=0 2023-07-21T19:07:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.218+0000 7f21feff5700 10 client.15603 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f21feff5700 1 -- 172.21.15.5:0/2106938435 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55913a17dff0 con 0x55913a2df640 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f21feff5700 10 client.15603 handle_command_reply: tid=0 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 client.15603 shutdown 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 msgr2=0x55913a2740a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f22100ca6f0 tx=0x7f22100ca550 comp rx=0 tx=0).stop 2023-07-21T19:07:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 shutdown_connections 2023-07-21T19:07:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55913a2778c0 0x55913a272990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55913a0cd6b0 0x55913a273b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55913a2df640 0x55913a2e1a50 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 --2- 172.21.15.5:0/2106938435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55913a277470 0x55913a2740a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f21feff5700 0 client.15603 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 >> 172.21.15.5:0/2106938435 conn(0x55913a266d40 msgr2=0x55913a278520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 shutdown_connections 2023-07-21T19:07:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.816+0000 7f2234b7db80 1 -- 172.21.15.5:0/2106938435 wait complete. 2023-07-21T19:07:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f2234b7db80 20 client.15603 trim_cache size 0 max 16384 2023-07-21T19:07:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:07:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:07:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:07:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:07:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 msgr2=0x7f22040a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 0x7f22040a3880 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2214009fd0 tx=0x7f221400b040 comp rx=0 tx=0).stop 2023-07-21T19:07:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 msgr2=0x7f221c0fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f221000e970 tx=0x7f221000ec80 comp rx=0 tx=0).stop 2023-07-21T19:07:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 shutdown_connections 2023-07-21T19:07:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22040a13d0 0x7f22040a3880 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f221c1062b0 0x7f221c1a3c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f221c068070 0x7f221c0fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 --2- 172.21.15.5:0/2422374881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f221c1059a0 0x7f221c0fefa0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.817+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 >> 172.21.15.5:0/2422374881 conn(0x7f221c074770 msgr2=0x7f221c103680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.818+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 shutdown_connections 2023-07-21T19:07:52.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:07:52.818+0000 7f22209fe700 1 -- 172.21.15.5:0/2422374881 wait complete. 2023-07-21T19:07:53.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:53 smithi101 ceph-mon[128136]: pgmap v11: 97 pgs: 97 active+clean; 25 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 69 op/s 2023-07-21T19:07:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:53 smithi005 ceph-mon[126460]: pgmap v11: 97 pgs: 97 active+clean; 25 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 69 op/s 2023-07-21T19:07:53.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:53 smithi156 ceph-mon[128316]: pgmap v11: 97 pgs: 97 active+clean; 25 MiB data, 335 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 2.3 MiB/s wr, 69 op/s 2023-07-21T19:07:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:55 smithi101 ceph-mon[128136]: pgmap v12: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.7 MiB/s wr, 119 op/s 2023-07-21T19:07:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:55 smithi005 ceph-mon[126460]: pgmap v12: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.7 MiB/s wr, 119 op/s 2023-07-21T19:07:55.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:55 smithi156 ceph-mon[128316]: pgmap v12: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 3.7 MiB/s wr, 119 op/s 2023-07-21T19:07:56.449 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:07:56.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 -- 172.21.15.5:0/3085790206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 msgr2=0x7fab1010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 --2- 172.21.15.5:0/3085790206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab1010c4b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fab08009f90 tx=0x7fab0800b330 comp rx=0 tx=0).stop 2023-07-21T19:07:56.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 -- 172.21.15.5:0/3085790206 shutdown_connections 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 --2- 172.21.15.5:0/3085790206 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab1010c4b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 --2- 172.21.15.5:0/3085790206 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab10100ca0 0x7fab10101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 --2- 172.21.15.5:0/3085790206 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab101015b0 0x7fab10109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.677+0000 7fab16bff700 1 -- 172.21.15.5:0/3085790206 >> 172.21.15.5:0/3085790206 conn(0x7fab100fa4b0 msgr2=0x7fab100fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.678+0000 7fab16bff700 1 -- 172.21.15.5:0/3085790206 shutdown_connections 2023-07-21T19:07:56.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.678+0000 7fab16bff700 1 -- 172.21.15.5:0/3085790206 wait complete. 2023-07-21T19:07:56.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.679+0000 7fab16bff700 1 Processor -- start 2023-07-21T19:07:56.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.679+0000 7fab16bff700 1 -- start start 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab101015b0 0x7fab100fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab101a3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab100ff660 con 0x7fab1010a070 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab101a4440 con 0x7fab10100ca0 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab16bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab101a4580 con 0x7fab101015b0 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1519c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab101a3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44924/0 (socket says 172.21.15.5:44924) 2023-07-21T19:07:56.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 -- 172.21.15.5:0/1688062549 learned_addr learned my addr 172.21.15.5:0/1688062549 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:56.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab101015b0 0x7fab100fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 -- 172.21.15.5:0/1688062549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab101015b0 msgr2=0x7fab100fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab101015b0 0x7fab100fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 -- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 msgr2=0x7fab101a3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab101a3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1499b700 1 -- 172.21.15.5:0/1688062549 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab040095f0 con 0x7fab10100ca0 2023-07-21T19:07:56.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.680+0000 7fab1519c700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab101a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab1499b700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab0400ec10 tx=0x7fab0400ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab0dffb700 1 -- 172.21.15.5:0/1688062549 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0400cc90 con 0x7fab10100ca0 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab08009c40 con 0x7fab10100ca0 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab0dffb700 1 -- 172.21.15.5:0/1688062549 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0400cdf0 con 0x7fab10100ca0 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab0dffb700 1 -- 172.21.15.5:0/1688062549 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04003e80 con 0x7fab10100ca0 2023-07-21T19:07:56.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.681+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab101a4b80 con 0x7fab10100ca0 2023-07-21T19:07:56.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.685+0000 7fab0dffb700 1 -- 172.21.15.5:0/1688062549 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fab04003fe0 con 0x7fab10100ca0 2023-07-21T19:07:56.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.686+0000 7fab0dffb700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 0x7fab000a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.686+0000 7fab2a562b80 1 Processor -- start 2023-07-21T19:07:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.686+0000 7fab0dffb700 1 -- 172.21.15.5:0/1688062549 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fab04014070 con 0x7fab10100ca0 2023-07-21T19:07:56.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 -- start start 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 0x7fab000a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae94590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae94ad0 0x557beae96eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bead8fcb0 con 0x557beae89be0 2023-07-21T19:07:56.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beade2a40 con 0x557beae892d0 2023-07-21T19:07:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab2a562b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beaddbed0 con 0x557beae94ad0 2023-07-21T19:07:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab1499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab1499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44930/0 (socket says 172.21.15.5:44930) 2023-07-21T19:07:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab1499b700 1 -- 172.21.15.5:0/531859222 learned_addr learned my addr 172.21.15.5:0/531859222 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:56.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.687+0000 7fab0ffff700 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae94590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 -- 172.21.15.5:0/531859222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae94ad0 msgr2=0x557beae96eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae94ad0 0x557beae96eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 -- 172.21.15.5:0/531859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 msgr2=0x557beae94590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae94590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 -- 172.21.15.5:0/531859222 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557beac09de0 con 0x557beae892d0 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab0ffff700 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae94590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 0x7fab000a3880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab101000e0 tx=0x7faafc011440 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7fab1499b700 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab04096710 tx=0x7fab04099dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd3f1ec180feb7be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7faafaffd700 1 -- 172.21.15.5:0/531859222 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04004290 con 0x557beae892d0 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7faafaffd700 1 -- 172.21.15.5:0/531859222 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0409a210 con 0x557beae892d0 2023-07-21T19:07:56.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.688+0000 7faafaffd700 1 -- 172.21.15.5:0/531859222 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040cae50 con 0x557beae892d0 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 -- 172.21.15.5:0/531859222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 msgr2=0x557beae896a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fab04096710 tx=0x7fab04099dd0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 -- 172.21.15.5:0/531859222 shutdown_connections 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae94ad0 0x557beae96eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae892d0 0x557beae896a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 --2- 172.21.15.5:0/531859222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae94590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 -- 172.21.15.5:0/531859222 >> 172.21.15.5:0/531859222 conn(0x557beae7fda0 msgr2=0x557beae856a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 -- 172.21.15.5:0/531859222 shutdown_connections 2023-07-21T19:07:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.689+0000 7fab2a562b80 1 -- 172.21.15.5:0/531859222 wait complete. 2023-07-21T19:07:56.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.690+0000 7fab2a562b80 1 -- start start 2023-07-21T19:07:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.690+0000 7fab2a562b80 1 Processor -- start 2023-07-21T19:07:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae94550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae917d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bead7bac0 con 0x557beae89be0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557bead7b7c0 con 0x557beae94ad0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beac5f920 con 0x557beae892d0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab2a562b80 10 client.0 _ll_register_callbacks cb 0x557bead79760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab1499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae94550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab1519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae917d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab1519c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44946/0 (socket says 172.21.15.5:44946) 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.691+0000 7fab1519c700 1 -- 172.21.15.5:0/50859069 learned_addr learned my addr 172.21.15.5:0/50859069 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 -- 172.21.15.5:0/50859069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 msgr2=0x557beae94550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae94550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 -- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 msgr2=0x557beae91ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 -- 172.21.15.5:0/50859069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab101a4b80 con 0x557beae89be0 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab1519c700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:56.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7fab0ffff700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae917d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faafc011840 tx=0x7faafc0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea807a3407afb1b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.692+0000 7faaeffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0285f0 con 0x557beae89be0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557bead354f0 con 0x557beae89be0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faafc028750 con 0x557beae89be0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7faafc01ee90 con 0x557beae89be0 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 10 client.15615 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:56.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557beabf2c60 con 0x557beae89be0 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faafc035080 con 0x557beae89be0 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faafc031a40 con 0x557beae89be0 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 10 client.15615 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 20 client.15615 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.693+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaea43d0 0x557beaea67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.694+0000 7fab2a562b80 4 client.15615 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:56.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.694+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557beadd4eb0 con 0x557beaea43d0 2023-07-21T19:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.694+0000 7fab1499b700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaea43d0 0x557beaea67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.694+0000 7fab1499b700 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaea43d0 0x557beaea67e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de608538ac162212 server_cookie=dec31f82bd8a99db in_seq=0 out_seq=0 2023-07-21T19:07:56.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.694+0000 7faaeffff700 10 client.15615 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7faaeffff700 1 -- 172.21.15.5:0/50859069 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557beadd4eb0 con 0x557beaea43d0 2023-07-21T19:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7faaeffff700 10 client.15615 handle_command_reply: tid=0 2023-07-21T19:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 client.15615 shutdown 2023-07-21T19:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 msgr2=0x557beae917d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae917d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7faafc011840 tx=0x7faafc0047c0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 shutdown_connections 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beae91ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae94550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaea43d0 0x557beaea67e0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 --2- 172.21.15.5:0/50859069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae89be0 0x557beae917d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 >> 172.21.15.5:0/50859069 conn(0x557beae7fda0 msgr2=0x557beae85230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7faaeffff700 0 client.15615 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 shutdown_connections 2023-07-21T19:07:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 1 -- 172.21.15.5:0/50859069 wait complete. 2023-07-21T19:07:56.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.696+0000 7fab2a562b80 20 client.15615 trim_cache size 0 max 16384 2023-07-21T19:07:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 Processor -- start 2023-07-21T19:07:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 -- start start 2023-07-21T19:07:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae94ad0 0x557beae8d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beae92db0 con 0x557beae94ad0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beae8ad60 con 0x557beae89be0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.717+0000 7fab2a562b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beae7f970 con 0x557beae892d0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab1499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab1519c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae94ad0 0x557beae8d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab1499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33354/0 (socket says 172.21.15.5:33354) 2023-07-21T19:07:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44952/0 (socket says 172.21.15.5:44952) 2023-07-21T19:07:56.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab1499b700 1 -- 172.21.15.5:0/3784963145 learned_addr learned my addr 172.21.15.5:0/3784963145 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 -- 172.21.15.5:0/3784963145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 msgr2=0x557beae940f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 -- 172.21.15.5:0/3784963145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae94ad0 msgr2=0x557beae8d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae94ad0 0x557beae8d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab0ffff700 1 -- 172.21.15.5:0/3784963145 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557beac30250 con 0x557beae89be0 2023-07-21T19:07:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.718+0000 7fab1499b700 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.719+0000 7fab0ffff700 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faafc031ef0 tx=0x7faafc01eac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77261d7c721cfdf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.719+0000 7faafb7fe700 1 -- 172.21.15.5:0/3784963145 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc0303e0 con 0x557beae89be0 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.719+0000 7faafb7fe700 1 -- 172.21.15.5:0/3784963145 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faafc01ee90 con 0x557beae89be0 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.719+0000 7faafb7fe700 1 -- 172.21.15.5:0/3784963145 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faafc030830 con 0x557beae89be0 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- 172.21.15.5:0/3784963145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 msgr2=0x557beae8caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7faafc031ef0 tx=0x7faafc01eac0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- 172.21.15.5:0/3784963145 shutdown_connections 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae94ad0 0x557beae8d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae892d0 0x557beae940f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 --2- 172.21.15.5:0/3784963145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae89be0 0x557beae8caf0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- 172.21.15.5:0/3784963145 >> 172.21.15.5:0/3784963145 conn(0x557beae7fda0 msgr2=0x557beaea0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- 172.21.15.5:0/3784963145 shutdown_connections 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- 172.21.15.5:0/3784963145 wait complete. 2023-07-21T19:07:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.720+0000 7fab2a562b80 1 -- start start 2023-07-21T19:07:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 Processor -- start 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae89be0 0x557beae999d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beaef7290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beae9a310 con 0x557beae892d0 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beaef77d0 con 0x557beae94ad0 2023-07-21T19:07:56.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557beaef7990 con 0x557beae89be0 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab2a562b80 10 client.0 _ll_register_callbacks cb 0x7faafc011e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab1499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab1499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56202/0 (socket says 172.21.15.5:56202) 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.721+0000 7fab1499b700 1 -- 172.21.15.5:0/1399364736 learned_addr learned my addr 172.21.15.5:0/1399364736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1519c700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beaef7290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae89be0 0x557beae999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 -- 172.21.15.5:0/1399364736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae89be0 msgr2=0x557beae999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae89be0 0x557beae999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 -- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 msgr2=0x557beaef7290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beaef7290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 -- 172.21.15.5:0/1399364736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557beac09de0 con 0x557beae892d0 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab1499b700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fab04096c80 tx=0x7fab0409e170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a2981502d514f9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7faaeffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:56.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04099700 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fab04099860 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.722+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0409a210 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab0409a470 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 10 client.15621 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x557beabf2c60 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0409a210 con 0x557beae892d0 2023-07-21T19:07:56.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fab0407d050 con 0x557beae892d0 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 10 client.15621 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 10 client.15621 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 20 client.15621 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaef8e10 0x557beaefb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 4 client.15621 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.723+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557bead97050 con 0x557beaef8e10 2023-07-21T19:07:56.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaef8e10 0x557beaefb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7faaeffff700 10 client.15621 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7fab0ffff700 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaef8e10 0x557beaefb220 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cca6f77018e538c9 server_cookie=9f1f19b16925b092 in_seq=0 out_seq=0 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7faaeffff700 1 -- 172.21.15.5:0/1399364736 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557bead97050 con 0x557beaef8e10 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7faaeffff700 10 client.15621 handle_command_reply: tid=0 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7fab2a562b80 1 client.15621 shutdown 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 msgr2=0x557beae99490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.724+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fab04096c80 tx=0x7fab0409e170 comp rx=0 tx=0).stop 2023-07-21T19:07:56.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 shutdown_connections 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557beae94ad0 0x557beaef7290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557beaef8e10 0x557beaefb220 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557beae892d0 0x557beae99490 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 --2- 172.21.15.5:0/1399364736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557beae89be0 0x557beae999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 >> 172.21.15.5:0/1399364736 conn(0x557beae7fda0 msgr2=0x557beae8caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7faaeffff700 0 client.15621 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 shutdown_connections 2023-07-21T19:07:56.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 1 -- 172.21.15.5:0/1399364736 wait complete. 2023-07-21T19:07:56.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab2a562b80 20 client.15621 trim_cache size 0 max 16384 2023-07-21T19:07:56.743 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:56.743 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:07:56.743 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:07:56.744 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 msgr2=0x7fab000a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 0x7fab000a3880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fab101000e0 tx=0x7faafc011440 comp rx=0 tx=0).stop 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 msgr2=0x7fab100fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.725+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fab0400ec10 tx=0x7fab0400ef20 comp rx=0 tx=0).stop 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 shutdown_connections 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab000a13d0 0x7fab000a3880 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab1010a070 0x7fab101a3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab10100ca0 0x7fab100fe8f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 --2- 172.21.15.5:0/1688062549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab101015b0 0x7fab100fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 >> 172.21.15.5:0/1688062549 conn(0x7fab100fa4b0 msgr2=0x7fab1010c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 shutdown_connections 2023-07-21T19:07:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.726+0000 7fab16bff700 1 -- 172.21.15.5:0/1688062549 wait complete. 2023-07-21T19:07:56.754 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:07:56.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:07:56.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.986+0000 7f379699d700 1 -- 172.21.15.5:0/446119428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 msgr2=0x7f3790067ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:56.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.986+0000 7f379699d700 1 --2- 172.21.15.5:0/446119428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f3790067ac0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f37800098f0 tx=0x7f3780009c00 comp rx=0 tx=0).stop 2023-07-21T19:07:56.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 -- 172.21.15.5:0/446119428 shutdown_connections 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 --2- 172.21.15.5:0/446119428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790068000 0x7f379010c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 --2- 172.21.15.5:0/446119428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790104a30 0x7f3790104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 --2- 172.21.15.5:0/446119428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f3790067ac0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 -- 172.21.15.5:0/446119428 >> 172.21.15.5:0/446119428 conn(0x7f3790074a30 msgr2=0x7f3790074e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 -- 172.21.15.5:0/446119428 shutdown_connections 2023-07-21T19:07:56.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.987+0000 7f379699d700 1 -- 172.21.15.5:0/446119428 wait complete. 2023-07-21T19:07:56.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 Processor -- start 2023-07-21T19:07:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 -- start start 2023-07-21T19:07:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 0x7f379019dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 0x7f37901978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790197f40 con 0x7f3790067670 2023-07-21T19:07:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901980b0 con 0x7f3790068000 2023-07-21T19:07:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.988+0000 7f379699d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3790198270 con 0x7f3790104a30 2023-07-21T19:07:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 0x7f379019dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 0x7f379019dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44982/0 (socket says 172.21.15.5:44982) 2023-07-21T19:07:57.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379519a700 1 -- 172.21.15.5:0/3334890542 learned_addr learned my addr 172.21.15.5:0/3334890542 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56222/0 (socket says 172.21.15.5:56222) 2023-07-21T19:07:57.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379619c700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 0x7f37901978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 -- 172.21.15.5:0/3334890542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 msgr2=0x7f37901978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 0x7f37901978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 -- 172.21.15.5:0/3334890542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 msgr2=0x7f379019dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 0x7f379019dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379599b700 1 -- 172.21.15.5:0/3334890542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f378c0095f0 con 0x7f3790067670 2023-07-21T19:07:57.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.989+0000 7f379619c700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 0x7f37901978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f379599b700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f378c00eb90 tx=0x7f378c00eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c00ccf0 con 0x7f3790067670 2023-07-21T19:07:57.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f378c00ce50 con 0x7f3790067670 2023-07-21T19:07:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3780009520 con 0x7f3790067670 2023-07-21T19:07:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f378c003e30 con 0x7f3790067670 2023-07-21T19:07:57.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.990+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3790198740 con 0x7f3790067670 2023-07-21T19:07:57.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.991+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3790067220 con 0x7f3790067670 2023-07-21T19:07:57.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.993+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f378c004000 con 0x7f3790067670 2023-07-21T19:07:57.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.995+0000 7f3786ffd700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 0x7f377c0a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.995+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f378c014070 con 0x7f3790067670 2023-07-21T19:07:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.995+0000 7f379519a700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 0x7f377c0a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.996+0000 7f379519a700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 0x7f377c0a3970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3790075220 tx=0x7f3780019040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:56.998+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f378c0964b0 con 0x7f3790067670 2023-07-21T19:07:57.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.308+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3790198740 con 0x7f3790067670 2023-07-21T19:07:57.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.309+0000 7f3786ffd700 1 -- 172.21.15.5:0/3334890542 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+10833 (secure 0 0 0) 0x7f3790198740 con 0x7f3790067670 2023-07-21T19:07:57.319 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:07:57.320 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":23,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:05:43.279232+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:07:57.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 msgr2=0x7f377c0a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 0x7f377c0a3970 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3790075220 tx=0x7f3780019040 comp rx=0 tx=0).stop 2023-07-21T19:07:57.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 msgr2=0x7f379019d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f378c00eb90 tx=0x7f378c00eea0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 shutdown_connections 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3790104a30 0x7f37901978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f377c0a14c0 0x7f377c0a3970 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3790067670 0x7f379019d860 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 --2- 172.21.15.5:0/3334890542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3790068000 0x7f379019dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 >> 172.21.15.5:0/3334890542 conn(0x7f3790074a30 msgr2=0x7f37900fe030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 shutdown_connections 2023-07-21T19:07:57.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.312+0000 7f379699d700 1 -- 172.21.15.5:0/3334890542 wait complete. 2023-07-21T19:07:57.324 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 23 2023-07-21T19:07:57.345 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:07:57.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 -- 172.21.15.5:0/241555494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b40731c0 msgr2=0x7f12b410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/241555494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b40731c0 0x7f12b410bdd0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f12a4009990 tx=0x7f12a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 -- 172.21.15.5:0/241555494 shutdown_connections 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/241555494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b410c310 0x7f12b410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/241555494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/241555494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b40731c0 0x7f12b410bdd0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 -- 172.21.15.5:0/241555494 >> 172.21.15.5:0/241555494 conn(0x7f12b40fb5f0 msgr2=0x7f12b40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 -- 172.21.15.5:0/241555494 shutdown_connections 2023-07-21T19:07:57.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.570+0000 7f12bb3ff700 1 -- 172.21.15.5:0/241555494 wait complete. 2023-07-21T19:07:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.571+0000 7f12bb3ff700 1 Processor -- start 2023-07-21T19:07:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.571+0000 7f12bb3ff700 1 -- start start 2023-07-21T19:07:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 0x7f12b41982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b410c310 0x7f12b419c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b419cd70 con 0x7f12b410c310 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b419cee0 con 0x7f12b40728b0 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12bb3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12b419d0a0 con 0x7f12b40731c0 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 0x7f12b41982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45006/0 (socket says 172.21.15.5:45006) 2023-07-21T19:07:57.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b919b700 1 -- 172.21.15.5:0/3342076470 learned_addr learned my addr 172.21.15.5:0/3342076470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b999c700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b410c310 0x7f12b419c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b899a700 1 -- 172.21.15.5:0/3342076470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 msgr2=0x7f12b4197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.572+0000 7f12b899a700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12b899a700 1 -- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b410c310 msgr2=0x7f12b419c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12b899a700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b410c310 0x7f12b419c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12b899a700 1 -- 172.21.15.5:0/3342076470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12a40095f0 con 0x7f12b40731c0 2023-07-21T19:07:57.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12b919b700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12b899a700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 0x7f12b41982d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005910 tx=0x7f12a4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12aa7fc700 1 -- 172.21.15.5:0/3342076470 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a401d070 con 0x7f12b40731c0 2023-07-21T19:07:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12aa7fc700 1 -- 172.21.15.5:0/3342076470 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12a4022470 con 0x7f12b40731c0 2023-07-21T19:07:57.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12b419d320 con 0x7f12b40731c0 2023-07-21T19:07:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12aa7fc700 1 -- 172.21.15.5:0/3342076470 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a4017650 con 0x7f12b40731c0 2023-07-21T19:07:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.573+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b41a4bc0 con 0x7f12b40731c0 2023-07-21T19:07:57.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.577+0000 7f12aa7fc700 1 -- 172.21.15.5:0/3342076470 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f12a40225e0 con 0x7f12b40731c0 2023-07-21T19:07:57.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.578+0000 7f12cedb8b80 1 Processor -- start 2023-07-21T19:07:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.578+0000 7f12aa7fc700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 0x7f12a00a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.578+0000 7f12cedb8b80 1 -- start start 2023-07-21T19:07:57.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.578+0000 7f12aa7fc700 1 -- 172.21.15.5:0/3342076470 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f12a40cdd50 con 0x7f12b40731c0 2023-07-21T19:07:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12b999c700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 0x7f12a00a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b610da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b6136c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b503cb0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b556a40 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b54fed0 con 0x55bc3b6112e0 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56240/0 (socket says 172.21.15.5:56240) 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.579+0000 7f12b919b700 1 -- 172.21.15.5:0/3516766561 learned_addr learned my addr 172.21.15.5:0/3516766561 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b999c700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 0x7f12a00a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00f590 tx=0x7f12ac00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 -- 172.21.15.5:0/3516766561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 msgr2=0x55bc3b6136c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b6136c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 -- 172.21.15.5:0/3516766561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 msgr2=0x55bc3b610da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b610da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 -- 172.21.15.5:0/3516766561 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc3b37dde0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f12b919b700 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12b00117e0 tx=0x7f12b0011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c610285385517cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f129f7fe700 1 -- 172.21.15.5:0/3516766561 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b000f480 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f129f7fe700 1 -- 172.21.15.5:0/3516766561 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12b000fa70 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.580+0000 7f129f7fe700 1 -- 172.21.15.5:0/3516766561 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b00199c0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3516766561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 msgr2=0x55bc3b5fd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f12b00117e0 tx=0x7f12b0011af0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3516766561 shutdown_connections 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b6136c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b5fd6a0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3516766561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b610da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3516766561 >> 172.21.15.5:0/3516766561 conn(0x55bc3b5f3da0 msgr2=0x55bc3b5f96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3516766561 shutdown_connections 2023-07-21T19:07:57.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.581+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3516766561 wait complete. 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.582+0000 7f12cedb8b80 1 -- start start 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.582+0000 7f12cedb8b80 1 Processor -- start 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b60e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 0x55bc3b60ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b4efac0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b4ef7c0 con 0x55bc3b6112e0 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b3d3920 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12cedb8b80 10 client.0 _ll_register_callbacks cb 0x55bc3b4ed760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56252/0 (socket says 172.21.15.5:56252) 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12b919b700 1 -- 172.21.15.5:0/830437702 learned_addr learned my addr 172.21.15.5:0/830437702 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.583+0000 7f12b999c700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 0x55bc3b60ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b899a700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b60e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 -- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 msgr2=0x55bc3b60e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b60e500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 -- 172.21.15.5:0/830437702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 msgr2=0x55bc3b60ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 0x55bc3b60ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 -- 172.21.15.5:0/830437702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b0011440 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b899a700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b60e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:07:57.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b999c700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 0x55bc3b60ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:57.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12b919b700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12b001c040 tx=0x7f12b000f7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3c689888fc7608d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f129cff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0005ce0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc3b4a94f0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.584+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f12b0005250 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12b0005410 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 10 client.15639 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bc3b366c60 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc3b4a94f0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12b00196d0 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 10 client.15639 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 10 client.15639 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 20 client.15639 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:57.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b620b30 0x55bc3b622f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 4 client.15639 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.585+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc3b548eb0 con 0x55bc3b620b30 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.586+0000 7f12b899a700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b620b30 0x55bc3b622f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.586+0000 7f12b899a700 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b620b30 0x55bc3b622f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=472e5b7bcc37f3cf server_cookie=d8ee6483325bc785 in_seq=0 out_seq=0 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.586+0000 7f129cff9700 10 client.15639 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f129cff9700 1 -- 172.21.15.5:0/830437702 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bc3b548eb0 con 0x55bc3b620b30 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f129cff9700 10 client.15639 handle_command_reply: tid=0 2023-07-21T19:07:57.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 client.15639 shutdown 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 msgr2=0x55bc3b60dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f12b001c040 tx=0x7f12b000f7e0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 shutdown_connections 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b620b30 0x55bc3b622f40 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b6112e0 0x55bc3b60ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60dfc0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/830437702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b60e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 >> 172.21.15.5:0/830437702 conn(0x55bc3b5f3da0 msgr2=0x55bc3b5f9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f129cff9700 0 client.15639 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.587+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 shutdown_connections 2023-07-21T19:07:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.588+0000 7f12cedb8b80 1 -- 172.21.15.5:0/830437702 wait complete. 2023-07-21T19:07:57.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.588+0000 7f12cedb8b80 20 client.15639 trim_cache size 0 max 16384 2023-07-21T19:07:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.606+0000 7f12cedb8b80 1 Processor -- start 2023-07-21T19:07:57.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.606+0000 7f12cedb8b80 1 -- start start 2023-07-21T19:07:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b60e870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b5fe540 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b5fa4c0 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b618260 con 0x55bc3b6112e0 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b899a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56266/0 (socket says 172.21.15.5:56266) 2023-07-21T19:07:57.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 -- 172.21.15.5:0/20997796 learned_addr learned my addr 172.21.15.5:0/20997796 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b999c700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b60e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 -- 172.21.15.5:0/20997796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 msgr2=0x55bc3b60e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b60e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 -- 172.21.15.5:0/20997796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 msgr2=0x55bc3b617d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b919b700 1 -- 172.21.15.5:0/20997796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc3b3a4250 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b899a700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.607+0000 7f12b999c700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b60e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.608+0000 7f12b919b700 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12b0011730 tx=0x7f12b0019ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=518bfd8ab53683c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.608+0000 7f129ffff700 1 -- 172.21.15.5:0/20997796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0019d70 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.608+0000 7f129ffff700 1 -- 172.21.15.5:0/20997796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12b0015070 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.608+0000 7f129ffff700 1 -- 172.21.15.5:0/20997796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b0003770 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- 172.21.15.5:0/20997796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 msgr2=0x55bc3b60d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f12b0011730 tx=0x7f12b0019ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- 172.21.15.5:0/20997796 shutdown_connections 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b6112e0 0x55bc3b60e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b60d250 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/20997796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- 172.21.15.5:0/20997796 >> 172.21.15.5:0/20997796 conn(0x55bc3b5f3da0 msgr2=0x55bc3b61d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- 172.21.15.5:0/20997796 shutdown_connections 2023-07-21T19:07:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- 172.21.15.5:0/20997796 wait complete. 2023-07-21T19:07:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 -- start start 2023-07-21T19:07:57.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.609+0000 7f12cedb8b80 1 Processor -- start 2023-07-21T19:07:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b6112e0 0x55bc3b615d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b6163a0 con 0x55bc3b6112e0 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b616510 con 0x55bc3b5fd2d0 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc3b6166d0 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12cedb8b80 10 client.0 _ll_register_callbacks cb 0x7f12b0011660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.610+0000 7f12b919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45044/0 (socket says 172.21.15.5:45044) 2023-07-21T19:07:57.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b919b700 1 -- 172.21.15.5:0/3557006578 learned_addr learned my addr 172.21.15.5:0/3557006578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b999c700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b6112e0 0x55bc3b615d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 -- 172.21.15.5:0/3557006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 msgr2=0x55bc3b6174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 -- 172.21.15.5:0/3557006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b6112e0 msgr2=0x55bc3b615d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b6112e0 0x55bc3b615d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 -- 172.21.15.5:0/3557006578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12b41a4bc0 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b919b700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f12b899a700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12a40cb410 tx=0x7f12a40971d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d568de1459f81ad2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.611+0000 7f129cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12a409eda0 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f12a40a3130 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f12b000fa70 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12a4022900 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 10 client.25378 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55bc3b366c60 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f12b000fa70 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f12a409a260 con 0x55bc3b5fdbe0 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 10 client.25378 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 10 client.25378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:07:57.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 20 client.25378 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b6757d0 0x55bc3b677be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 4 client.25378 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:07:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.612+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bc3b50b050 con 0x55bc3b6757d0 2023-07-21T19:07:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f12b919b700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b6757d0 0x55bc3b677be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f12b919b700 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b6757d0 0x55bc3b677be0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c6db82c3cf60f1e server_cookie=ff1c82d8553673e6 in_seq=0 out_seq=0 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f129cff9700 10 client.25378 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f129cff9700 1 -- 172.21.15.5:0/3557006578 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bc3b50b050 con 0x55bc3b6757d0 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f129cff9700 10 client.25378 handle_command_reply: tid=0 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f12cedb8b80 1 client.25378 shutdown 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 msgr2=0x55bc3b617a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.613+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617a20 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f12a40cb410 tx=0x7f12a40971d0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 shutdown_connections 2023-07-21T19:07:57.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc3b6112e0 0x55bc3b615d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55bc3b6757d0 0x55bc3b677be0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc3b5fd2d0 0x55bc3b6174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 --2- 172.21.15.5:0/3557006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc3b5fdbe0 0x55bc3b617a20 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f129cff9700 0 client.25378 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 >> 172.21.15.5:0/3557006578 conn(0x55bc3b5f3da0 msgr2=0x55bc3b5ff290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 shutdown_connections 2023-07-21T19:07:57.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 1 -- 172.21.15.5:0/3557006578 wait complete. 2023-07-21T19:07:57.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.614+0000 7f12cedb8b80 20 client.25378 trim_cache size 0 max 16384 2023-07-21T19:07:57.633 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 msgr2=0x7f12a00a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 0x7f12a00a3800 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f12ac00f590 tx=0x7f12ac00b040 comp rx=0 tx=0).stop 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 msgr2=0x7f12b41982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 0x7f12b41982d0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f12a4005910 tx=0x7f12a4004730 comp rx=0 tx=0).stop 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 shutdown_connections 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12a00a1350 0x7f12a00a3800 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f12b410c310 0x7f12b419c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f12b40728b0 0x7f12b4197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 --2- 172.21.15.5:0/3342076470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f12b40731c0 0x7f12b41982d0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 >> 172.21.15.5:0/3342076470 conn(0x7f12b40fb5f0 msgr2=0x7f12b410e460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 shutdown_connections 2023-07-21T19:07:57.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.615+0000 7f12bb3ff700 1 -- 172.21.15.5:0/3342076470 wait complete. 2023-07-21T19:07:57.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:57 smithi005 ceph-mon[126460]: pgmap v13: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 3.6 MiB/s wr, 90 op/s 2023-07-21T19:07:57.643 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:57 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3334890542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:57.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:07:57.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:57 smithi101 ceph-mon[128136]: pgmap v13: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 3.6 MiB/s wr, 90 op/s 2023-07-21T19:07:57.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:57 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3334890542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:57.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 -- 172.21.15.5:0/468560747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 msgr2=0x7fd254101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/468560747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254101070 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd244009990 tx=0x7fd244009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 -- 172.21.15.5:0/468560747 shutdown_connections 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/468560747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 0x7fd25410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/468560747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254101070 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/468560747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 -- 172.21.15.5:0/468560747 >> 172.21.15.5:0/468560747 conn(0x7fd2540fa4b0 msgr2=0x7fd2540fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 -- 172.21.15.5:0/468560747 shutdown_connections 2023-07-21T19:07:57.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.867+0000 7fd25a1ff700 1 -- 172.21.15.5:0/468560747 wait complete. 2023-07-21T19:07:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.868+0000 7fd25a1ff700 1 Processor -- start 2023-07-21T19:07:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.868+0000 7fd25a1ff700 1 -- start start 2023-07-21T19:07:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 0x7fd25419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25419cd00 con 0x7fd254100ca0 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25419ce70 con 0x7fd2541015b0 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd25a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd25419d030 con 0x7fd25410a070 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56298/0 (socket says 172.21.15.5:56298) 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 -- 172.21.15.5:0/3540829518 learned_addr learned my addr 172.21.15.5:0/3540829518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd253fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 0x7fd25419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 -- 172.21.15.5:0/3540829518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 msgr2=0x7fd25419c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 0x7fd25419c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 -- 172.21.15.5:0/3540829518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 msgr2=0x7fd254198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.869+0000 7fd2537fe700 1 -- 172.21.15.5:0/3540829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2440095f0 con 0x7fd254100ca0 2023-07-21T19:07:57.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd244000c00 tx=0x7fd244004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd250ff9700 1 -- 172.21.15.5:0/3540829518 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24401d070 con 0x7fd254100ca0 2023-07-21T19:07:57.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd25419d2b0 con 0x7fd254100ca0 2023-07-21T19:07:57.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd250ff9700 1 -- 172.21.15.5:0/3540829518 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd244022470 con 0x7fd254100ca0 2023-07-21T19:07:57.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd250ff9700 1 -- 172.21.15.5:0/3540829518 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd244017650 con 0x7fd254100ca0 2023-07-21T19:07:57.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.870+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2541a4b50 con 0x7fd254100ca0 2023-07-21T19:07:57.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.874+0000 7fd250ff9700 1 -- 172.21.15.5:0/3540829518 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fd2440225e0 con 0x7fd254100ca0 2023-07-21T19:07:57.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.876+0000 7fd250ff9700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 0x7fd23c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.877+0000 7fd250ff9700 1 -- 172.21.15.5:0/3540829518 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd2440ce1b0 con 0x7fd254100ca0 2023-07-21T19:07:57.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.877+0000 7fd26db43b80 1 Processor -- start 2023-07-21T19:07:57.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.877+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 0x7fd23c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.877+0000 7fd26db43b80 1 -- start start 2023-07-21T19:07:57.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.877+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6ca76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 0x7fd23c0a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd254199330 tx=0x7fd248009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cbada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd26db43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6badcb0 con 0x55bfc6ca72d0 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd26db43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6c00a40 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd26db43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6bf9ed0 con 0x55bfc6ca7be0 2023-07-21T19:07:57.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6ca76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cbada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45066/0 (socket says 172.21.15.5:45066) 2023-07-21T19:07:57.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cbada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33462/0 (socket says 172.21.15.5:33462) 2023-07-21T19:07:57.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 -- 172.21.15.5:0/3177980533 learned_addr learned my addr 172.21.15.5:0/3177980533 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 -- 172.21.15.5:0/3177980533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 msgr2=0x55bfc6cbada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cbada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 -- 172.21.15.5:0/3177980533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 msgr2=0x55bfc6ca76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6ca76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd253fff700 1 -- 172.21.15.5:0/3177980533 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc6a27de0 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.878+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6ca76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.879+0000 7fd253fff700 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011680 tx=0x7fd24c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=942bae51aed67b96 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.879+0000 7fd239ffb700 1 -- 172.21.15.5:0/3177980533 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00e040 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.879+0000 7fd239ffb700 1 -- 172.21.15.5:0/3177980533 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd24c00f040 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.879+0000 7fd239ffb700 1 -- 172.21.15.5:0/3177980533 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c004770 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 -- 172.21.15.5:0/3177980533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 msgr2=0x55bfc6cbd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011680 tx=0x7fd24c009920 comp rx=0 tx=0).stop 2023-07-21T19:07:57.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 -- 172.21.15.5:0/3177980533 shutdown_connections 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cbd6c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6ca76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3177980533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cbada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 -- 172.21.15.5:0/3177980533 >> 172.21.15.5:0/3177980533 conn(0x55bfc6c9dda0 msgr2=0x55bfc6ca36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 -- 172.21.15.5:0/3177980533 shutdown_connections 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.880+0000 7fd26db43b80 1 -- 172.21.15.5:0/3177980533 wait complete. 2023-07-21T19:07:57.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 -- start start 2023-07-21T19:07:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 Processor -- start 2023-07-21T19:07:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6b99ac0 con 0x55bfc6ca7be0 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6b997c0 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.881+0000 7fd26db43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6a7d920 con 0x55bfc6ca72d0 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd26db43b80 10 client.0 _ll_register_callbacks cb 0x55bfc6b97760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33468/0 (socket says 172.21.15.5:33468) 2023-07-21T19:07:57.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 -- 172.21.15.5:0/3418337173 learned_addr learned my addr 172.21.15.5:0/3418337173 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd253fff700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 -- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 msgr2=0x55bfc6cb8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 -- 172.21.15.5:0/3418337173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 msgr2=0x55bfc6cb7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.882+0000 7fd2537fe700 1 -- 172.21.15.5:0/3418337173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2541a4b50 con 0x55bfc6ca72d0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd24409aed0 tx=0x7fd2440cbe00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aba1aa0456be300b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24409ab30 con 0x55bfc6ca72d0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd2440aee40 con 0x55bfc6ca72d0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bfc6b534f0 con 0x55bfc6ca72d0 2023-07-21T19:07:57.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd2440228c0 con 0x55bfc6ca72d0 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd26db43b80 10 client.25390 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd24c011330 con 0x55bfc6ca72d0 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd2440b7080 con 0x55bfc6ca72d0 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.883+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd2440a3100 con 0x55bfc6ca72d0 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 10 client.25390 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 10 client.25390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 20 client.25390 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:57.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6cca7e0 0x55bfc6cccbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 4 client.25390 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bfc6bf2eb0 con 0x55bfc6cca7e0 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.884+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6cca7e0 0x55bfc6cccbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.885+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6cca7e0 0x55bfc6cccbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e31735728512c5ac server_cookie=58ac9c6075ebf6f3 in_seq=0 out_seq=0 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.885+0000 7fd232ffd700 10 client.25390 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.885+0000 7fd232ffd700 1 -- 172.21.15.5:0/3418337173 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bfc6bf2eb0 con 0x55bfc6cca7e0 2023-07-21T19:07:57.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.885+0000 7fd232ffd700 10 client.25390 handle_command_reply: tid=0 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.885+0000 7fd26db43b80 1 client.25390 shutdown 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 msgr2=0x55bfc6cbad00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd24409aed0 tx=0x7fd2440cbe00 comp rx=0 tx=0).stop 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 shutdown_connections 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6cca7e0 0x55bfc6cccbf0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cbad00 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd232ffd700 0 client.25390 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3418337173 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 >> 172.21.15.5:0/3418337173 conn(0x55bfc6c9dda0 msgr2=0x55bfc6ca3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 shutdown_connections 2023-07-21T19:07:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 1 -- 172.21.15.5:0/3418337173 wait complete. 2023-07-21T19:07:57.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.886+0000 7fd26db43b80 20 client.25390 trim_cache size 0 max 16384 2023-07-21T19:07:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.903+0000 7fd26db43b80 1 Processor -- start 2023-07-21T19:07:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 -- start start 2023-07-21T19:07:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cba830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6c9d970 con 0x55bfc6ca7be0 2023-07-21T19:07:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6ca8160 con 0x55bfc6ca72d0 2023-07-21T19:07:57.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd26db43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6ca8de0 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56328/0 (socket says 172.21.15.5:56328) 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd252ffd700 1 -- 172.21.15.5:0/3604032851 learned_addr learned my addr 172.21.15.5:0/3604032851 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cba830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.904+0000 7fd253fff700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 -- 172.21.15.5:0/3604032851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6cbb2e0 msgr2=0x55bfc6cb7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 -- 172.21.15.5:0/3604032851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 msgr2=0x55bfc6cba830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cba830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 -- 172.21.15.5:0/3604032851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bfc6a10c60 con 0x55bfc6ca7be0 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd2537fe700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd252ffd700 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bfc6cb95c0 tx=0x7fd248014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06a3638b8754791 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd23a7fc700 1 -- 172.21.15.5:0/3604032851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248024030 con 0x55bfc6ca7be0 2023-07-21T19:07:57.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd23a7fc700 1 -- 172.21.15.5:0/3604032851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd248025040 con 0x55bfc6ca7be0 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.905+0000 7fd23a7fc700 1 -- 172.21.15.5:0/3604032851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd248028780 con 0x55bfc6ca7be0 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 -- 172.21.15.5:0/3604032851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 msgr2=0x55bfc6cb7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x55bfc6cb95c0 tx=0x7fd248014000 comp rx=0 tx=0).stop 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 -- 172.21.15.5:0/3604032851 shutdown_connections 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cb7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cba830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 --2- 172.21.15.5:0/3604032851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cb7800 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 -- 172.21.15.5:0/3604032851 >> 172.21.15.5:0/3604032851 conn(0x55bfc6c9dda0 msgr2=0x55bfc6cb3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 -- 172.21.15.5:0/3604032851 shutdown_connections 2023-07-21T19:07:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.906+0000 7fd26db43b80 1 -- 172.21.15.5:0/3604032851 wait complete. 2023-07-21T19:07:57.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 -- start start 2023-07-21T19:07:57.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 Processor -- start 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cc02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cc0820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6cc2290 con 0x55bfc6ca72d0 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6d1e6b0 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bfc6d1e7f0 con 0x55bfc6ca7be0 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.907+0000 7fd26db43b80 10 client.0 _ll_register_callbacks cb 0x7fd244097530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd252ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cc0820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd2537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cc02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45096/0 (socket says 172.21.15.5:45096) 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 -- 172.21.15.5:0/1242245287 learned_addr learned my addr 172.21.15.5:0/1242245287 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 -- 172.21.15.5:0/1242245287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 msgr2=0x55bfc6cc0820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cc0820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 -- 172.21.15.5:0/1242245287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 msgr2=0x55bfc6cc02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cc02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 -- 172.21.15.5:0/1242245287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd24c011330 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd2537fe700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cc02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd253fff700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011680 tx=0x7fd24c005a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f1177027af2e6b5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.908+0000 7fd232ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:07:57.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24c00e030 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd24c00f040 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd24c003a00 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd24c003c00 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd26db43b80 10 client.25424 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7fd248017070 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.909+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd24c00f040 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fd24c025020 con 0x55bfc6cbb2e0 2023-07-21T19:07:57.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 10 client.25424 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 10 client.25424 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 20 client.25424 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6d1fec0 0x55bfc6d222d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 4 client.25424 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55bfc6bb5050 con 0x55bfc6d1fec0 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.910+0000 7fd2537fe700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6d1fec0 0x55bfc6d222d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:57.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd2537fe700 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6d1fec0 0x55bfc6d222d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e3ff30817d9a05f0 server_cookie=20ccf14ddab90dd8 in_seq=0 out_seq=0 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd232ffd700 10 client.25424 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd232ffd700 1 -- 172.21.15.5:0/1242245287 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55bfc6bb5050 con 0x55bfc6d1fec0 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd232ffd700 10 client.25424 handle_command_reply: tid=0 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 client.25424 shutdown 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 msgr2=0x55bfc6cc0d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd24c011680 tx=0x7fd24c005a30 comp rx=0 tx=0).stop 2023-07-21T19:07:57.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 shutdown_connections 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55bfc6d1fec0 0x55bfc6d222d0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bfc6cbb2e0 0x55bfc6cc0d60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd232ffd700 0 client.25424 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bfc6ca72d0 0x55bfc6cc02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 --2- 172.21.15.5:0/1242245287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bfc6ca7be0 0x55bfc6cc0820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.911+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 >> 172.21.15.5:0/1242245287 conn(0x55bfc6c9dda0 msgr2=0x55bfc6cb3260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 shutdown_connections 2023-07-21T19:07:57.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd26db43b80 1 -- 172.21.15.5:0/1242245287 wait complete. 2023-07-21T19:07:57.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd26db43b80 20 client.25424 trim_cache size 0 max 16384 2023-07-21T19:07:57.930 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:07:57.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 msgr2=0x7fd23c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 0x7fd23c0a3880 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd254199330 tx=0x7fd248009260 comp rx=0 tx=0).stop 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 msgr2=0x7fd254197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.912+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd244000c00 tx=0x7fd244004730 comp rx=0 tx=0).stop 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 shutdown_connections 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd25410a070 0x7fd25419c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd23c0a13d0 0x7fd23c0a3880 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd254100ca0 0x7fd254197d20 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 --2- 172.21.15.5:0/3540829518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2541015b0 0x7fd254198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 >> 172.21.15.5:0/3540829518 conn(0x7fd2540fa4b0 msgr2=0x7fd25410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 shutdown_connections 2023-07-21T19:07:57.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:57.913+0000 7fd25a1ff700 1 -- 172.21.15.5:0/3540829518 wait complete. 2023-07-21T19:07:57.932 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:57 smithi156 ceph-mon[128316]: pgmap v13: 97 pgs: 97 active+clean; 35 MiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 3.6 MiB/s wr, 90 op/s 2023-07-21T19:07:57.933 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:57 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3334890542' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:07:57.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:07:58.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 -- 172.21.15.5:0/310674799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb001049e0 msgr2=0x7fbb00104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 --2- 172.21.15.5:0/310674799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb001049e0 0x7fbb00104db0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fbafc009990 tx=0x7fbafc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 -- 172.21.15.5:0/310674799 shutdown_connections 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 --2- 172.21.15.5:0/310674799 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff380 0x7fbb0010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 --2- 172.21.15.5:0/310674799 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb001049e0 0x7fbb00104db0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 --2- 172.21.15.5:0/310674799 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 0x7fbb000fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 -- 172.21.15.5:0/310674799 >> 172.21.15.5:0/310674799 conn(0x7fbb000fa4b0 msgr2=0x7fbb000fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 -- 172.21.15.5:0/310674799 shutdown_connections 2023-07-21T19:07:58.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.160+0000 7fbb077ff700 1 -- 172.21.15.5:0/310674799 wait complete. 2023-07-21T19:07:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 Processor -- start 2023-07-21T19:07:58.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 -- start start 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 0x7fbb00197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001049e0 0x7fbb0019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0019cd10 con 0x7fbb000ff380 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0019ce80 con 0x7fbb000fe9d0 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.161+0000 7fbb077ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0019d040 con 0x7fbb001049e0 2023-07-21T19:07:58.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb0559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 0x7fbb00197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56350/0 (socket says 172.21.15.5:56350) 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 -- 172.21.15.5:0/1855519759 learned_addr learned my addr 172.21.15.5:0/1855519759 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001049e0 0x7fbb0019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 -- 172.21.15.5:0/1855519759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001049e0 msgr2=0x7fbb0019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001049e0 0x7fbb0019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 -- 172.21.15.5:0/1855519759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 msgr2=0x7fbb00197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 0x7fbb00197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.162+0000 7fbb04d9a700 1 -- 172.21.15.5:0/1855519759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbafc0095f0 con 0x7fbb000ff380 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e9a0 tx=0x7fbaf000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbaf67fc700 1 -- 172.21.15.5:0/1855519759 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0009790 con 0x7fbb000ff380 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbaf67fc700 1 -- 172.21.15.5:0/1855519759 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf0010430 con 0x7fbb000ff380 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0019d320 con 0x7fbb000ff380 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbaf67fc700 1 -- 172.21.15.5:0/1855519759 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf000cea0 con 0x7fbb000ff380 2023-07-21T19:07:58.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.163+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb001a4b60 con 0x7fbb000ff380 2023-07-21T19:07:58.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.167+0000 7fbaf67fc700 1 -- 172.21.15.5:0/1855519759 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7fbaf00105a0 con 0x7fbb000ff380 2023-07-21T19:07:58.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.168+0000 7fbb1b1bfb80 1 Processor -- start 2023-07-21T19:07:58.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 -- start start 2023-07-21T19:07:58.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbaf67fc700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 0x7fbaec0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbaf67fc700 1 -- 172.21.15.5:0/1855519759 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbaf0014070 con 0x7fbb000ff380 2023-07-21T19:07:58.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa23dccb0 con 0x55eaa24d62d0 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa242fa40 con 0x55eaa24d6be0 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2428ed0 con 0x55eaa24e1ad0 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 0x7fbaec0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb0559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb0559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56366/0 (socket says 172.21.15.5:56366) 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.169+0000 7fbb0559b700 1 -- 172.21.15.5:0/4146465028 learned_addr learned my addr 172.21.15.5:0/4146465028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 -- 172.21.15.5:0/4146465028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 msgr2=0x55eaa24e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 -- 172.21.15.5:0/4146465028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 msgr2=0x55eaa24e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24e1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 -- 172.21.15.5:0/4146465028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa2256de0 con 0x55eaa24d62d0 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 0x7fbaec0a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800f540 tx=0x7fbaf8009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.170+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ffb10 tx=0x7fbafc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b32823cf540a33f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.171+0000 7fbaeb7fe700 1 -- 172.21.15.5:0/4146465028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc01d070 con 0x55eaa24d62d0 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.171+0000 7fbaeb7fe700 1 -- 172.21.15.5:0/4146465028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbafc01e040 con 0x55eaa24d62d0 2023-07-21T19:07:58.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.171+0000 7fbaeb7fe700 1 -- 172.21.15.5:0/4146465028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbafc004790 con 0x55eaa24d62d0 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.171+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4146465028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 msgr2=0x55eaa24d66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.171+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbb000ffb10 tx=0x7fbafc005f00 comp rx=0 tx=0).stop 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4146465028 shutdown_connections 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24d62d0 0x55eaa24d66a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4146465028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4146465028 >> 172.21.15.5:0/4146465028 conn(0x55eaa24ccda0 msgr2=0x55eaa24d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4146465028 shutdown_connections 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4146465028 wait complete. 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 -- start start 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.172+0000 7fbb1b1bfb80 1 Processor -- start 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24dea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24def70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa23c8ac0 con 0x55eaa24df4b0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa23c87c0 con 0x55eaa24d6be0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa22ac920 con 0x55eaa24e1ad0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb1b1bfb80 10 client.0 _ll_register_callbacks cb 0x55eaa23c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb05d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb0559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24dea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb05d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56376/0 (socket says 172.21.15.5:56376) 2023-07-21T19:07:58.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb0559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24dea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45148/0 (socket says 172.21.15.5:45148) 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb04d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24def70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.173+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3070807534 learned_addr learned my addr 172.21.15.5:0/3070807534 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3070807534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 msgr2=0x55eaa24def70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24def70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 msgr2=0x55eaa24dea30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24dea30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3070807534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0010c110 con 0x55eaa24df4b0 2023-07-21T19:07:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8019dd0 tx=0x7fbaf8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a1d1e6e7303583 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbae8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:58.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.174+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf800e040 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eaa23824f0 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbaf8017ae0 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf8017ca0 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 10 client.15675 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eaa223fc60 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eaa23824f0 con 0x55eaa24df4b0 2023-07-21T19:07:58.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbaf8004530 con 0x55eaa24df4b0 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 10 client.15675 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 10 client.15675 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 20 client.15675 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa24f1a20 0x55eaa24f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 4 client.15675 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:58.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.175+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eaa2421eb0 con 0x55eaa24f1a20 2023-07-21T19:07:58.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.176+0000 7fbb0559b700 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa24f1a20 0x55eaa24f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.177+0000 7fbae8ff9700 10 client.15675 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.177+0000 7fbb0559b700 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa24f1a20 0x55eaa24f3e30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e2157fde5ab1c0a server_cookie=1d96c499b4300285 in_seq=0 out_seq=0 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.177+0000 7fbae8ff9700 1 -- 172.21.15.5:0/3070807534 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eaa2421eb0 con 0x55eaa24f1a20 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.177+0000 7fbae8ff9700 10 client.15675 handle_command_reply: tid=0 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 client.15675 shutdown 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 msgr2=0x55eaa24ed4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8019dd0 tx=0x7fbaf8004b40 comp rx=0 tx=0).stop 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 shutdown_connections 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa24f1a20 0x55eaa24f3e30 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa24ed4e0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24d6be0 0x55eaa24dea30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3070807534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24e1ad0 0x55eaa24def70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbae8ff9700 0 client.15675 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:58.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 >> 172.21.15.5:0/3070807534 conn(0x55eaa24ccda0 msgr2=0x55eaa24d2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 shutdown_connections 2023-07-21T19:07:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3070807534 wait complete. 2023-07-21T19:07:58.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.178+0000 7fbb1b1bfb80 20 client.15675 trim_cache size 0 max 16384 2023-07-21T19:07:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 Processor -- start 2023-07-21T19:07:58.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 -- start start 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa24ed4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24df4b0 0x55eaa24e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa24d7e70 con 0x55eaa24e1ad0 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa24cc970 con 0x55eaa24df4b0 2023-07-21T19:07:58.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa24d81a0 con 0x55eaa24d6be0 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb05d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.196+0000 7fbb05d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56384/0 (socket says 172.21.15.5:56384) 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3592190721 learned_addr learned my addr 172.21.15.5:0/3592190721 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24df4b0 0x55eaa24e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb0559b700 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa24ed4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3592190721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 msgr2=0x55eaa24ed4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa24ed4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3592190721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24df4b0 msgr2=0x55eaa24e84b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24df4b0 0x55eaa24e84b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 -- 172.21.15.5:0/3592190721 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eaa227d250 con 0x55eaa24e1ad0 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80074a0 tx=0x7fbaf8006000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3fd2d1435439d4e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.197+0000 7fbaebfff700 1 -- 172.21.15.5:0/3592190721 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf800e030 con 0x55eaa24e1ad0 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbaebfff700 1 -- 172.21.15.5:0/3592190721 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf800f040 con 0x55eaa24e1ad0 2023-07-21T19:07:58.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbaebfff700 1 -- 172.21.15.5:0/3592190721 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf802a3f0 con 0x55eaa24e1ad0 2023-07-21T19:07:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3592190721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 msgr2=0x55eaa24e0a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbaf80074a0 tx=0x7fbaf8006000 comp rx=0 tx=0).stop 2023-07-21T19:07:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3592190721 shutdown_connections 2023-07-21T19:07:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24e1ad0 0x55eaa24e0a50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa24ed4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/3592190721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24df4b0 0x55eaa24e84b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3592190721 >> 172.21.15.5:0/3592190721 conn(0x55eaa24ccda0 msgr2=0x55eaa24e0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3592190721 shutdown_connections 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.198+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/3592190721 wait complete. 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.199+0000 7fbb1b1bfb80 1 -- start start 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.199+0000 7fbb1b1bfb80 1 Processor -- start 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa253ef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 0x55eaa2539870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa2539f80 con 0x55eaa24df4b0 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa253a0f0 con 0x55eaa24e1ad0 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaa253a2b0 con 0x55eaa24d6be0 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb1b1bfb80 10 client.0 _ll_register_callbacks cb 0x7fbaf8019770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56398/0 (socket says 172.21.15.5:56398) 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 -- 172.21.15.5:0/4199248940 learned_addr learned my addr 172.21.15.5:0/4199248940 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa253ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 0x55eaa2539870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 -- 172.21.15.5:0/4199248940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 msgr2=0x55eaa253ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa253ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 -- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 msgr2=0x55eaa2539870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 0x55eaa2539870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb04d9a700 1 -- 172.21.15.5:0/4199248940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eaa223fc60 con 0x55eaa24df4b0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.200+0000 7fbb05d9c700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 0x55eaa2539870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbb04d9a700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0096480 tx=0x7fbaf00cac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92681b9e983c7909 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbae8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf009dfb0 con 0x55eaa24df4b0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fbaf00a23d0 con 0x55eaa24df4b0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbafc01e040 con 0x55eaa24df4b0 2023-07-21T19:07:58.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf00b82c0 con 0x55eaa24df4b0 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbb1b1bfb80 10 client.15681 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x55eaa22552b0 con 0x55eaa24df4b0 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.201+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf007c070 con 0x55eaa24df4b0 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbaf0094b90 con 0x55eaa24df4b0 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 10 client.15681 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 10 client.15681 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 20 client.15681 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:58.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa2546de0 0x55eaa25491f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 4 client.15681 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:07:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55eaa23e4050 con 0x55eaa2546de0 2023-07-21T19:07:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.202+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa2546de0 0x55eaa25491f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbae8ff9700 10 client.15681 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:58.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb0559b700 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa2546de0 0x55eaa25491f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed0fa8d989f1e0ac server_cookie=31b17deb30be5d33 in_seq=0 out_seq=0 2023-07-21T19:07:58.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbae8ff9700 1 -- 172.21.15.5:0/4199248940 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55eaa23e4050 con 0x55eaa2546de0 2023-07-21T19:07:58.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbae8ff9700 10 client.15681 handle_command_reply: tid=0 2023-07-21T19:07:58.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 client.15681 shutdown 2023-07-21T19:07:58.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 msgr2=0x55eaa253f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fbaf0096480 tx=0x7fbaf00cac40 comp rx=0 tx=0).stop 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 shutdown_connections 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaa24e1ad0 0x55eaa2539870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaa24d6be0 0x55eaa253ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaa2546de0 0x55eaa25491f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 --2- 172.21.15.5:0/4199248940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaa24df4b0 0x55eaa253f490 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 >> 172.21.15.5:0/4199248940 conn(0x55eaa24ccda0 msgr2=0x55eaa24ed350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbae8ff9700 0 client.15681 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 shutdown_connections 2023-07-21T19:07:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.203+0000 7fbb1b1bfb80 1 -- 172.21.15.5:0/4199248940 wait complete. 2023-07-21T19:07:58.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb1b1bfb80 20 client.15681 trim_cache size 0 max 16384 2023-07-21T19:07:58.222 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:07:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 msgr2=0x7fbaec0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 0x7fbaec0a3880 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800f540 tx=0x7fbaf8009920 comp rx=0 tx=0).stop 2023-07-21T19:07:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 msgr2=0x7fbb00198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e9a0 tx=0x7fbaf000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 shutdown_connections 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001049e0 0x7fbb0019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb000fe9d0 0x7fbb00197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbaec0a13d0 0x7fbaec0a3880 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 --2- 172.21.15.5:0/1855519759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff380 0x7fbb00198270 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.204+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 >> 172.21.15.5:0/1855519759 conn(0x7fbb000fa4b0 msgr2=0x7fbb00067630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.205+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 shutdown_connections 2023-07-21T19:07:58.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:58.205+0000 7fbb077ff700 1 -- 172.21.15.5:0/1855519759 wait complete. 2023-07-21T19:07:59.185 INFO:teuthology.orchestra.run.smithi005.stderr: * Initialized, logs are in /var/lib/pgsql/initdb_postgresql.log 2023-07-21T19:07:59.190 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-21T19:07:59.235 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:07:59.236 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:07:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.455+0000 7f330a39d700 1 -- 172.21.15.5:0/531105505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041059e0 msgr2=0x7f3304105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.455+0000 7f330a39d700 1 --2- 172.21.15.5:0/531105505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041059e0 0x7f3304105db0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f33000098f0 tx=0x7f3300009c00 comp rx=0 tx=0).stop 2023-07-21T19:07:59.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.455+0000 7f330a39d700 1 -- 172.21.15.5:0/531105505 shutdown_connections 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 --2- 172.21.15.5:0/531105505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 --2- 172.21.15.5:0/531105505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33041059e0 0x7f3304105db0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 --2- 172.21.15.5:0/531105505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33040ff9d0 0x7f33040ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 -- 172.21.15.5:0/531105505 >> 172.21.15.5:0/531105505 conn(0x7f33040fb4b0 msgr2=0x7f33040fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 -- 172.21.15.5:0/531105505 shutdown_connections 2023-07-21T19:07:59.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.456+0000 7f330a39d700 1 -- 172.21.15.5:0/531105505 wait complete. 2023-07-21T19:07:59.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 Processor -- start 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 -- start start 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330419ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33041059e0 0x7f3304197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3304197f70 con 0x7f3304100380 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041980e0 con 0x7f33040ff9d0 2023-07-21T19:07:59.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.457+0000 7f330a39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33041982a0 con 0x7f33041059e0 2023-07-21T19:07:59.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330419ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45184/0 (socket says 172.21.15.5:45184) 2023-07-21T19:07:59.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 -- 172.21.15.5:0/1533024553 learned_addr learned my addr 172.21.15.5:0/1533024553 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f3309b9c700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33041059e0 0x7f3304197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 -- 172.21.15.5:0/1533024553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33041059e0 msgr2=0x7f3304197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33041059e0 0x7f3304197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 -- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 msgr2=0x7f330419ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330419ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 -- 172.21.15.5:0/1533024553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32fc009d00 con 0x7f33040ff9d0 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f3308b9a700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:59.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.458+0000 7f330939b700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3300009fd0 tx=0x7f330000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.459+0000 7f32fa7fc700 1 -- 172.21.15.5:0/1533024553 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330001c070 con 0x7f33040ff9d0 2023-07-21T19:07:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.459+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3300009520 con 0x7f33040ff9d0 2023-07-21T19:07:59.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.459+0000 7f32fa7fc700 1 -- 172.21.15.5:0/1533024553 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f330000ba70 con 0x7f33040ff9d0 2023-07-21T19:07:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.459+0000 7f32fa7fc700 1 -- 172.21.15.5:0/1533024553 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3300004930 con 0x7f33040ff9d0 2023-07-21T19:07:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.459+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33041a46a0 con 0x7f33040ff9d0 2023-07-21T19:07:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.462+0000 7f32fa7fc700 1 -- 172.21.15.5:0/1533024553 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f3300020070 con 0x7f33040ff9d0 2023-07-21T19:07:59.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.463+0000 7f32fa7fc700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 0x7f32f00a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.464+0000 7f32fa7fc700 1 -- 172.21.15.5:0/1533024553 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f33000cda90 con 0x7f33040ff9d0 2023-07-21T19:07:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.464+0000 7f3308b9a700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 0x7f32f00a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.464+0000 7f331ef13b80 1 Processor -- start 2023-07-21T19:07:59.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.464+0000 7f331ef13b80 1 -- start start 2023-07-21T19:07:59.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f3308b9a700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 0x7f32f00a38a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33040fee60 tx=0x7f32f4009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f506ee940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f505f3d30 con 0x562f506f8db0 2023-07-21T19:07:59.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f50646ac0 con 0x562f506eee80 2023-07-21T19:07:59.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f331ef13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5063ff50 con 0x562f506ee570 2023-07-21T19:07:59.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f3309b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f330939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f506ee940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f3309b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56418/0 (socket says 172.21.15.5:56418) 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.465+0000 7f3309b9c700 1 -- 172.21.15.5:0/2554748616 learned_addr learned my addr 172.21.15.5:0/2554748616 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 -- 172.21.15.5:0/2554748616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 msgr2=0x562f506ee940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f506ee940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 -- 172.21.15.5:0/2554748616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 msgr2=0x562f506f8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 -- 172.21.15.5:0/2554748616 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5046de60 con 0x562f506f8db0 2023-07-21T19:07:59.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f330939b700 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f506ee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f3309b9c700 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0094f0 tx=0x7f32fc0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f382a48a3e0aa50 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f32ef7fe700 1 -- 172.21.15.5:0/2554748616 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00e030 con 0x562f506f8db0 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f32ef7fe700 1 -- 172.21.15.5:0/2554748616 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32fc00f040 con 0x562f506f8db0 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.466+0000 7f32ef7fe700 1 -- 172.21.15.5:0/2554748616 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc015960 con 0x562f506f8db0 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.467+0000 7f331ef13b80 1 -- 172.21.15.5:0/2554748616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 msgr2=0x562f506fb190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.467+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f32fc0094f0 tx=0x7f32fc0099e0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 -- 172.21.15.5:0/2554748616 shutdown_connections 2023-07-21T19:07:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506fb190 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f506ee940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2554748616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 -- 172.21.15.5:0/2554748616 >> 172.21.15.5:0/2554748616 conn(0x562f506e4070 msgr2=0x562f506e99b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.481 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:07:59 smithi005 ceph-mon[126460]: pgmap v14: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 4.9 MiB/s wr, 273 op/s 2023-07-21T19:07:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 -- 172.21.15.5:0/2554748616 shutdown_connections 2023-07-21T19:07:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 -- 172.21.15.5:0/2554748616 wait complete. 2023-07-21T19:07:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.469+0000 7f331ef13b80 1 -- start start 2023-07-21T19:07:59.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.470+0000 7f331ef13b80 1 Processor -- start 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.470+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.470+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.470+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506f8db0 0x562f506f5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f331ef13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f505dfb40 con 0x562f506ee570 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f331ef13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f505df840 con 0x562f506eee80 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f331ef13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f504c39a0 con 0x562f506f8db0 2023-07-21T19:07:59.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f331ef13b80 10 client.0 _ll_register_callbacks cb 0x562f505dd7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56430/0 (socket says 172.21.15.5:56430) 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45208/0 (socket says 172.21.15.5:45208) 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 -- 172.21.15.5:0/3931522800 learned_addr learned my addr 172.21.15.5:0/3931522800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f3309b9c700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506f8db0 0x562f506f5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 -- 172.21.15.5:0/3931522800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506f8db0 msgr2=0x562f506f5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506f8db0 0x562f506f5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 -- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 msgr2=0x562f506f59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.471+0000 7f330939b700 1 -- 172.21.15.5:0/3931522800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33041a46a0 con 0x562f506ee570 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f330939b700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3300096f50 tx=0x7f330009a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8c2606a1faae1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f330009aeb0 con 0x562f506ee570 2023-07-21T19:07:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3300005250 con 0x562f506ee570 2023-07-21T19:07:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f50599570 con 0x562f506ee570 2023-07-21T19:07:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33000a2e10 con 0x562f506ee570 2023-07-21T19:07:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f331ef13b80 10 client.15693 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f32fc0091a0 con 0x562f506ee570 2023-07-21T19:07:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562f50599570 con 0x562f506ee570 2023-07-21T19:07:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f33000a2980 con 0x562f506ee570 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f331ef13b80 10 client.15693 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.472+0000 7f331ef13b80 10 client.15693 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.473+0000 7f331ef13b80 20 client.15693 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.473+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50710b20 0x562f50712f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.473+0000 7f331ef13b80 4 client.15693 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.473+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f50638f30 con 0x562f50710b20 2023-07-21T19:07:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.473+0000 7f3308b9a700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50710b20 0x562f50712f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.474+0000 7f32ecff9700 10 client.15693 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.474+0000 7f3308b9a700 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50710b20 0x562f50712f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=351748af42ed8afd server_cookie=94eba597a1df592 in_seq=0 out_seq=0 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.474+0000 7f32ecff9700 1 -- 172.21.15.5:0/3931522800 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f50638f30 con 0x562f50710b20 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.474+0000 7f32ecff9700 10 client.15693 handle_command_reply: tid=0 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 client.15693 shutdown 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 msgr2=0x562f506f8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f3300096f50 tx=0x7f330009a3b0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 shutdown_connections 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506f8db0 0x562f506f5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50710b20 0x562f50712f30 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506ee570 0x562f506f8760 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 --2- 172.21.15.5:0/3931522800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f506f59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f32ecff9700 0 client.15693 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 >> 172.21.15.5:0/3931522800 conn(0x562f506e4070 msgr2=0x562f506e9540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 shutdown_connections 2023-07-21T19:07:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 1 -- 172.21.15.5:0/3931522800 wait complete. 2023-07-21T19:07:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.475+0000 7f331ef13b80 20 client.15693 trim_cache size 0 max 16384 2023-07-21T19:07:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 Processor -- start 2023-07-21T19:07:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 -- start start 2023-07-21T19:07:59.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 0x562f50708120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506eee80 0x562f506edbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f506efa30 con 0x562f506f8db0 2023-07-21T19:07:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f506f69a0 con 0x562f506ee570 2023-07-21T19:07:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f331ef13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f506f0890 con 0x562f506eee80 2023-07-21T19:07:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f3309b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f3309b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56436/0 (socket says 172.21.15.5:56436) 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f3309b9c700 1 -- 172.21.15.5:0/1502080290 learned_addr learned my addr 172.21.15.5:0/1502080290 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f330939b700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 0x562f50708120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.496+0000 7f3308b9a700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506eee80 0x562f506edbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 -- 172.21.15.5:0/1502080290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506eee80 msgr2=0x562f506edbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506eee80 0x562f506edbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 -- 172.21.15.5:0/1502080290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 msgr2=0x562f50708120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 0x562f50708120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 -- 172.21.15.5:0/1502080290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5070d4e0 con 0x562f506f8db0 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f330939b700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 0x562f50708120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f3309b9c700 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009170 tx=0x7f32fc004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfc0276761f1efa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f32effff700 1 -- 172.21.15.5:0/1502080290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc00e040 con 0x562f506f8db0 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f32effff700 1 -- 172.21.15.5:0/1502080290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32fc004d80 con 0x562f506f8db0 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.497+0000 7f32effff700 1 -- 172.21.15.5:0/1502080290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32fc005280 con 0x562f506f8db0 2023-07-21T19:07:59.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 -- 172.21.15.5:0/1502080290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 msgr2=0x562f506f8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f32fc009170 tx=0x7f32fc004a10 comp rx=0 tx=0).stop 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 -- 172.21.15.5:0/1502080290 shutdown_connections 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f506f8290 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506ee570 0x562f50708120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 --2- 172.21.15.5:0/1502080290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506eee80 0x562f506edbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 -- 172.21.15.5:0/1502080290 >> 172.21.15.5:0/1502080290 conn(0x562f506e4070 msgr2=0x562f506ed4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 -- 172.21.15.5:0/1502080290 shutdown_connections 2023-07-21T19:07:59.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.498+0000 7f331ef13b80 1 -- 172.21.15.5:0/1502080290 wait complete. 2023-07-21T19:07:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.499+0000 7f331ef13b80 1 -- start start 2023-07-21T19:07:59.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.499+0000 7f331ef13b80 1 Processor -- start 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.499+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f50707400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.499+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f331ef13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f5075db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f331ef13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5075e0e0 con 0x562f506f8db0 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f331ef13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5075e250 con 0x562f506eee80 2023-07-21T19:07:59.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f331ef13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5075e410 con 0x562f506ee570 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f331ef13b80 10 client.0 _ll_register_callbacks cb 0x7f330009a8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f330939b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f50707400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45228/0 (socket says 172.21.15.5:45228) 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 -- 172.21.15.5:0/2181475469 learned_addr learned my addr 172.21.15.5:0/2181475469 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3309b9c700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f5075db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 -- 172.21.15.5:0/2181475469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 msgr2=0x562f50707400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f50707400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 -- 172.21.15.5:0/2181475469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 msgr2=0x562f5075db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f5075db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.500+0000 7f3308b9a700 1 -- 172.21.15.5:0/2181475469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32fc0091a0 con 0x562f506eee80 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f3308b9a700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32f4013010 tx=0x7f32f40047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b5033de57b9f1bb4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f32ecff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:07:59.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32f4024040 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f32f4003680 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f32fc00f040 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f32f4028780 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f331ef13b80 10 client.25460 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.501+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562f5046de60 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f32fc00f040 con 0x562f506eee80 2023-07-21T19:07:59.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f32f403a7f0 con 0x562f506eee80 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 10 client.25460 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 10 client.25460 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 20 client.25460 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50766df0 0x562f50769200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 4 client.25460 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x562f505fb0d0 con 0x562f50766df0 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.502+0000 7f330939b700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50766df0 0x562f50769200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.503+0000 7f330939b700 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50766df0 0x562f50769200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9e4ee07c1faf076b server_cookie=64ed52d29dfee21 in_seq=0 out_seq=0 2023-07-21T19:07:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.503+0000 7f32ecff9700 10 client.25460 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f32ecff9700 1 -- 172.21.15.5:0/2181475469 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x562f505fb0d0 con 0x562f50766df0 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f32ecff9700 10 client.25460 handle_command_reply: tid=0 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 client.25460 shutdown 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 msgr2=0x562f50707940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f32f4013010 tx=0x7f32f40047c0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 shutdown_connections 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f50766df0 0x562f50769200 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f506f8db0 0x562f5075db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f506ee570 0x562f50707400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f32ecff9700 0 client.25460 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 --2- 172.21.15.5:0/2181475469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f506eee80 0x562f50707940 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 >> 172.21.15.5:0/2181475469 conn(0x562f506e4070 msgr2=0x562f506edd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 shutdown_connections 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.504+0000 7f331ef13b80 1 -- 172.21.15.5:0/2181475469 wait complete. 2023-07-21T19:07:59.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f331ef13b80 20 client.25460 trim_cache size 0 max 16384 2023-07-21T19:07:59.522 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:59.522 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:07:59.522 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9b709a1a-1cac-404a-85f3-c38af8dc50ea", 2023-07-21T19:07:59.522 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:07:59.522 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 msgr2=0x7f32f00a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 0x7f32f00a38a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33040fee60 tx=0x7f32f4009260 comp rx=0 tx=0).stop 2023-07-21T19:07:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 msgr2=0x7f330419d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3300009fd0 tx=0x7f330000fa00 comp rx=0 tx=0).stop 2023-07-21T19:07:59.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 shutdown_connections 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f33041059e0 0x7f3304197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f33040ff9d0 0x7f330419d890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f32f00a13f0 0x7f32f00a38a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 --2- 172.21.15.5:0/1533024553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3304100380 0x7f330419ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 >> 172.21.15.5:0/1533024553 conn(0x7f33040fb4b0 msgr2=0x7f3304072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 shutdown_connections 2023-07-21T19:07:59.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.505+0000 7f330a39d700 1 -- 172.21.15.5:0/1533024553 wait complete. 2023-07-21T19:07:59.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:07:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 -- 172.21.15.5:0/1576142354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77641015b0 msgr2=0x7f7764109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 --2- 172.21.15.5:0/1576142354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77641015b0 0x7f7764109aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f7758009990 tx=0x7f7758009ca0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 -- 172.21.15.5:0/1576142354 shutdown_connections 2023-07-21T19:07:59.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 --2- 172.21.15.5:0/1576142354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f776410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 --2- 172.21.15.5:0/1576142354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7764100ca0 0x7f7764101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 --2- 172.21.15.5:0/1576142354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77641015b0 0x7f7764109aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.785+0000 7f7769fff700 1 -- 172.21.15.5:0/1576142354 >> 172.21.15.5:0/1576142354 conn(0x7f77640fa4b0 msgr2=0x7f77640fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.786+0000 7f7769fff700 1 -- 172.21.15.5:0/1576142354 shutdown_connections 2023-07-21T19:07:59.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.786+0000 7f7769fff700 1 -- 172.21.15.5:0/1576142354 wait complete. 2023-07-21T19:07:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 Processor -- start 2023-07-21T19:07:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 -- start start 2023-07-21T19:07:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 0x7f77640fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77641015b0 0x7f77640fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f77641a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77640ff660 con 0x7f776410a070 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77641a4240 con 0x7f7764100ca0 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.787+0000 7f7769fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77641a4380 con 0x7f77641015b0 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f7763fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f77641a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 0x7f77640fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f7763fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f77641a3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56458/0 (socket says 172.21.15.5:56458) 2023-07-21T19:07:59.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f7763fff700 1 -- 172.21.15.5:0/3143492568 learned_addr learned my addr 172.21.15.5:0/3143492568 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f7762ffd700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77641015b0 0x7f77640fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 -- 172.21.15.5:0/3143492568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77641015b0 msgr2=0x7f77640fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77641015b0 0x7f77640fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 -- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 msgr2=0x7f77641a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f77641a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.788+0000 7f77637fe700 1 -- 172.21.15.5:0/3143492568 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77580095f0 con 0x7f7764100ca0 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f77637fe700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 0x7f77640fe8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9f0 tx=0x7f775400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f7760ff9700 1 -- 172.21.15.5:0/3143492568 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775400c9c0 con 0x7f7764100ca0 2023-07-21T19:07:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f7760ff9700 1 -- 172.21.15.5:0/3143492568 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7754010430 con 0x7f7764100ca0 2023-07-21T19:07:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77641a45e0 con 0x7f7764100ca0 2023-07-21T19:07:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f7760ff9700 1 -- 172.21.15.5:0/3143492568 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754018680 con 0x7f7764100ca0 2023-07-21T19:07:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.789+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77641a4b10 con 0x7f7764100ca0 2023-07-21T19:07:59.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.792+0000 7f7760ff9700 1 -- 172.21.15.5:0/3143492568 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 23) v1 ==== 135890+0+0 (secure 0 0 0) 0x7f77540105a0 con 0x7f7764100ca0 2023-07-21T19:07:59.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.794+0000 7f777d88cb80 1 Processor -- start 2023-07-21T19:07:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.794+0000 7f7760ff9700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 0x7f774c0a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.794+0000 7f777d88cb80 1 -- start start 2023-07-21T19:07:59.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f7760ff9700 1 -- 172.21.15.5:0/3143492568 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7754014070 con 0x7f7764100ca0 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f7763fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 0x7f774c0a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854bab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854badeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854aa6cb0 con 0x562854babad0 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854af9a40 con 0x562854ba02d0 2023-07-21T19:07:59.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f777d88cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854af2ed0 con 0x562854ba0be0 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f77637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f77637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45256/0 (socket says 172.21.15.5:45256) 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f77637fe700 1 -- 172.21.15.5:0/1718626511 learned_addr learned my addr 172.21.15.5:0/1718626511 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f7763fff700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854badeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.795+0000 7f7762ffd700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854bab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 0x7f774c0a3880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7764100a00 tx=0x7f775c009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 -- 172.21.15.5:0/1718626511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 msgr2=0x562854bab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854bab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 -- 172.21.15.5:0/1718626511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 msgr2=0x562854ba06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 -- 172.21.15.5:0/1718626511 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562854920de0 con 0x562854babad0 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7762ffd700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854bab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f77637fe700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:07:59.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7763fff700 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854badeb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f775c009c10 tx=0x7f775c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b63791ef1d77ce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.796+0000 7f7749ffb700 1 -- 172.21.15.5:0/1718626511 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c00e040 con 0x562854babad0 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f7749ffb700 1 -- 172.21.15.5:0/1718626511 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f775c00f040 con 0x562854babad0 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f7749ffb700 1 -- 172.21.15.5:0/1718626511 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c021690 con 0x562854babad0 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 -- 172.21.15.5:0/1718626511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 msgr2=0x562854badeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854badeb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f775c009c10 tx=0x7f775c01c000 comp rx=0 tx=0).stop 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 -- 172.21.15.5:0/1718626511 shutdown_connections 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854badeb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854ba06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1718626511 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854bab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 -- 172.21.15.5:0/1718626511 >> 172.21.15.5:0/1718626511 conn(0x562854b96da0 msgr2=0x562854b9c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 -- 172.21.15.5:0/1718626511 shutdown_connections 2023-07-21T19:07:59.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.797+0000 7f777d88cb80 1 -- 172.21.15.5:0/1718626511 wait complete. 2023-07-21T19:07:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.798+0000 7f777d88cb80 1 -- start start 2023-07-21T19:07:59.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.798+0000 7f777d88cb80 1 Processor -- start 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854bab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854ba87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854a92ac0 con 0x562854babad0 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854a927c0 con 0x562854ba02d0 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854976920 con 0x562854ba0be0 2023-07-21T19:07:59.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f777d88cb80 10 client.0 _ll_register_callbacks cb 0x562854a90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f7763fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f7763fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56478/0 (socket says 172.21.15.5:56478) 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f7763fff700 1 -- 172.21.15.5:0/4291974362 learned_addr learned my addr 172.21.15.5:0/4291974362 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f7762ffd700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854ba87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.799+0000 7f77637fe700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854bab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 -- 172.21.15.5:0/4291974362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 msgr2=0x562854ba87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854ba87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 -- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 msgr2=0x562854bab4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854bab4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 -- 172.21.15.5:0/4291974362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77641a4b10 con 0x562854babad0 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f77637fe700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854bab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7763fff700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f775c004cc0 tx=0x7f775c004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7150152fbbff6c5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.800+0000 7f7742ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775c00e050 con 0x562854babad0 2023-07-21T19:07:59.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f775c02a850 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562854a4c4f0 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f775c017e50 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f777d88cb80 10 client.15711 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f776410d170 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.801+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562854a4c4f0 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f775c005510 con 0x562854babad0 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 10 client.15711 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:59.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 10 client.15711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 20 client.15711 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854bbb420 0x562854bbd830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 4 client.15711 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562854aebeb0 con 0x562854bbb420 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.802+0000 7f77637fe700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854bbb420 0x562854bbd830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.803+0000 7f77637fe700 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854bbb420 0x562854bbd830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867cd85fc2d1bf99 server_cookie=4857f2aa6a9ab597 in_seq=0 out_seq=0 2023-07-21T19:07:59.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.803+0000 7f7742ffd700 10 client.15711 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f7742ffd700 1 -- 172.21.15.5:0/4291974362 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562854aebeb0 con 0x562854bbb420 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f7742ffd700 10 client.15711 handle_command_reply: tid=0 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 client.15711 shutdown 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 msgr2=0x562854ba8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f775c004cc0 tx=0x7f775c004cf0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 shutdown_connections 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854bbb420 0x562854bbd830 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854babad0 0x562854ba8e00 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba02d0 0x562854bab4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 --2- 172.21.15.5:0/4291974362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba0be0 0x562854ba87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 >> 172.21.15.5:0/4291974362 conn(0x562854b96da0 msgr2=0x562854b9c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f7742ffd700 0 client.15711 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 shutdown_connections 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.804+0000 7f777d88cb80 1 -- 172.21.15.5:0/4291974362 wait complete. 2023-07-21T19:07:59.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.805+0000 7f777d88cb80 20 client.15711 trim_cache size 0 max 16384 2023-07-21T19:07:59.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.824+0000 7f777d88cb80 1 Processor -- start 2023-07-21T19:07:59.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.824+0000 7f777d88cb80 1 -- start start 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba0be0 0x562854bab050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854babad0 0x562854ba5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854a4c4f0 con 0x562854ba02d0 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854bb8ae0 con 0x562854ba0be0 2023-07-21T19:07:59.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f777d88cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854ba1590 con 0x562854babad0 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56484/0 (socket says 172.21.15.5:56484) 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 -- 172.21.15.5:0/826816022 learned_addr learned my addr 172.21.15.5:0/826816022 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f7763fff700 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854babad0 0x562854ba5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f7762ffd700 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba0be0 0x562854bab050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 -- 172.21.15.5:0/826816022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854babad0 msgr2=0x562854ba5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854babad0 0x562854ba5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 -- 172.21.15.5:0/826816022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba0be0 msgr2=0x562854bab050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba0be0 0x562854bab050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.825+0000 7f77637fe700 1 -- 172.21.15.5:0/826816022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f775c019460 con 0x562854ba02d0 2023-07-21T19:07:59.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.826+0000 7f77637fe700 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9c0 tx=0x7f7754009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2591de6580f8e3ad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.826+0000 7f774a7fc700 1 -- 172.21.15.5:0/826816022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7754010880 con 0x562854ba02d0 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.826+0000 7f774a7fc700 1 -- 172.21.15.5:0/826816022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7754099ec0 con 0x562854ba02d0 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.826+0000 7f774a7fc700 1 -- 172.21.15.5:0/826816022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775409a380 con 0x562854ba02d0 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- 172.21.15.5:0/826816022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 msgr2=0x562854bb24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9c0 tx=0x7f7754009f70 comp rx=0 tx=0).stop 2023-07-21T19:07:59.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- 172.21.15.5:0/826816022 shutdown_connections 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854babad0 0x562854ba5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba02d0 0x562854bb24b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 --2- 172.21.15.5:0/826816022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854ba0be0 0x562854bab050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- 172.21.15.5:0/826816022 >> 172.21.15.5:0/826816022 conn(0x562854b96da0 msgr2=0x562854ba3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- 172.21.15.5:0/826816022 shutdown_connections 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- 172.21.15.5:0/826816022 wait complete. 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.827+0000 7f777d88cb80 1 -- start start 2023-07-21T19:07:59.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.828+0000 7f777d88cb80 1 Processor -- start 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.828+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba02d0 0x562854c03420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.828+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.828+0000 7f777d88cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854babad0 0x562854c08280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.828+0000 7f777d88cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854c087c0 con 0x562854ba0be0 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f777d88cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854c08900 con 0x562854babad0 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f777d88cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562854c08ac0 con 0x562854ba02d0 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f777d88cb80 10 client.0 _ll_register_callbacks cb 0x7f77540ca480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:07:59.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56488/0 (socket says 172.21.15.5:56488) 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 -- 172.21.15.5:0/1970289071 learned_addr learned my addr 172.21.15.5:0/1970289071 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f77637fe700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba02d0 0x562854c03420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7763fff700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854babad0 0x562854c08280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 -- 172.21.15.5:0/1970289071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba02d0 msgr2=0x562854c03420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba02d0 0x562854c03420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 -- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854babad0 msgr2=0x562854c08280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:07:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854babad0 0x562854c08280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.829+0000 7f7762ffd700 1 -- 172.21.15.5:0/1970289071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77641a45e0 con 0x562854ba0be0 2023-07-21T19:07:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f7762ffd700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7758009990 tx=0x7f775800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a55f18cb78b81d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:07:59.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f7742ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:07:59.845 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:07:59 smithi101 ceph-mon[128136]: pgmap v14: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 4.9 MiB/s wr, 273 op/s 2023-07-21T19:07:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f775801d030 con 0x562854ba0be0 2023-07-21T19:07:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f775c00f040 con 0x562854ba0be0 2023-07-21T19:07:59.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7758004500 con 0x562854ba0be0 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77580046c0 con 0x562854ba0be0 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f777d88cb80 10 client.15717 fetch_fsmap learned FSMap version 23 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.830+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x562854909c60 con 0x562854ba0be0 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f775c00f040 con 0x562854ba0be0 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f775802b020 con 0x562854ba0be0 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 10 client.15717 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:07:59.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 20 client.15717 populate_metadata read hostname 'smithi005' 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854c101c0 0x562854c125d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 4 client.15717 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562854aae050 con 0x562854c101c0 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.831+0000 7f77637fe700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854c101c0 0x562854c125d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f77637fe700 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854c101c0 0x562854c125d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1bdff817aadd308 server_cookie=6b1dd6673b5bba6 in_seq=0 out_seq=0 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f7742ffd700 10 client.15717 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f7742ffd700 1 -- 172.21.15.5:0/1970289071 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562854aae050 con 0x562854c101c0 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f7742ffd700 10 client.15717 handle_command_reply: tid=0 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 client.15717 shutdown 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 msgr2=0x562854c03960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f7758009990 tx=0x7f775800b410 comp rx=0 tx=0).stop 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 shutdown_connections 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562854babad0 0x562854c08280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562854ba02d0 0x562854c03420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562854c101c0 0x562854c125d0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 --2- 172.21.15.5:0/1970289071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562854ba0be0 0x562854c03960 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f7742ffd700 0 client.15717 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.832+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 >> 172.21.15.5:0/1970289071 conn(0x562854b96da0 msgr2=0x562854ba23e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 shutdown_connections 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f777d88cb80 1 -- 172.21.15.5:0/1970289071 wait complete. 2023-07-21T19:07:59.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f777d88cb80 20 client.15717 trim_cache size 0 max 16384 2023-07-21T19:07:59.851 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:07:59.851 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:07:59.851 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:07:59.851 INFO:teuthology.orchestra.run.smithi005.stdout: "9b709a1a-1cac-404a-85f3-c38af8dc50ea": { 2023-07-21T19:07:59.851 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:07:59.852 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9b709a1a-1cac-404a-85f3-c38af8dc50ea", 2023-07-21T19:07:59.852 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:07:59.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:07:59.852 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:07:59.852 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 msgr2=0x7f774c0a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 0x7f774c0a3880 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7764100a00 tx=0x7f775c009920 comp rx=0 tx=0).stop 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 msgr2=0x7f77640fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 0x7f77640fe8f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f775400e9f0 tx=0x7f775400ed00 comp rx=0 tx=0).stop 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 shutdown_connections 2023-07-21T19:07:59.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f774c0a13d0 0x7f774c0a3880 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f776410a070 0x7f77641a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7764100ca0 0x7f77640fe8f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 --2- 172.21.15.5:0/3143492568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f77641015b0 0x7f77640fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.833+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 >> 172.21.15.5:0/3143492568 conn(0x7f77640fa4b0 msgr2=0x7f776410c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.834+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 shutdown_connections 2023-07-21T19:07:59.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:07:59.834+0000 7f7769fff700 1 -- 172.21.15.5:0/3143492568 wait complete. 2023-07-21T19:07:59.862 INFO:tasks.cephfs.filesystem:scrub status for tag:9b709a1a-1cac-404a-85f3-c38af8dc50ea - {'path': '/', 'tag': '9b709a1a-1cac-404a-85f3-c38af8dc50ea', 'options': 'recursive,force'} 2023-07-21T19:07:59.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:07:59 smithi156 ceph-mon[128316]: pgmap v14: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 508 KiB/s rd, 4.9 MiB/s wr, 273 op/s 2023-07-21T19:08:00.033 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 . 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 data 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2023-07-21T19:08:00.034 INFO:teuthology.orchestra.run.smithi005.stdout:total 40 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 . 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22070793 Jul 21 19:07 base 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Jul 21 19:07 global 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 log 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_commit_ts 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_dynshmem 2023-07-21T19:08:00.035 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 21 19:07 pg_hba.conf 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 21 19:07 pg_ident.conf 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:07 pg_logical 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 pg_multixact 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 pg_notify 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_replslot 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_serial 2023-07-21T19:08:00.036 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_snapshots 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_stat 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_stat_tmp 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 pg_subtrans 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_tblspc 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_twophase 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:07 pg_wal 2023-07-21T19:08:00.037 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 pg_xact 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 21 19:07 postgresql.auto.conf 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 21 19:07 postgresql.conf 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base: 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:total 3 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22070793 Jul 21 19:07 . 2023-07-21T19:08:00.038 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 1 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 12346 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 12347 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:total 7186 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 . 2023-07-21T19:08:00.039 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22070793 Jul 21 19:07 .. 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:08:00.040 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:08:00.041 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:08:00.042 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:08:00.043 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:07 1247 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1247_fsm 2023-07-21T19:08:00.044 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1247_vm 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:07 1249 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1249_fsm 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1249_vm 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:07 1259 2023-07-21T19:08:00.045 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1259_vm 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:08:00.046 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:08:00.047 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:08:00.048 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:08:00.049 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606_vm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:08:00.050 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 21 19:07 2608 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2608_fsm 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2608_vm 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610_fsm 2023-07-21T19:08:00.051 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2610_vm 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:08:00.052 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:08:00.053 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 21 19:07 2619 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2619_fsm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2619_vm 2023-07-21T19:08:00.054 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:08:00.055 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 21 19:07 2658 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2659 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2662 2023-07-21T19:08:00.056 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2663 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2664 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2665 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2666 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2667 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:08:00.057 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:07 2673 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:07 2674 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2678 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2679 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:08:00.058 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:08:00.059 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2696 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:08:00.060 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2703 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2704 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:08:00.061 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:08:00.062 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:08:00.063 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:08:00.064 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:08:00.065 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:08:00.066 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:08:00.067 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 3455 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:08:00.068 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:08:00.069 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:08:00.070 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:08:00.071 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:08:00.072 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:08:00.625 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:08:00.626 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:08:00.627 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:08:00.628 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:08:00.629 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:00.630 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:total 7186 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 . 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22070793 Jul 21 19:07 .. 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:08:00.631 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:08:00.632 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:08:00.633 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:08:00.634 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:08:00.635 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:07 1247 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1247_fsm 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1247_vm 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:07 1249 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1249_fsm 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1249_vm 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:08:00.636 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:07 1259 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1259_vm 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:08:00.637 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:08:00.638 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:08:00.639 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:08:00.640 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606_vm 2023-07-21T19:08:00.641 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 21 19:07 2608 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2608_fsm 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2608_vm 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:08:00.642 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610_fsm 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2610_vm 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:08:00.643 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:08:00.644 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:08:00.645 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 21 19:07 2619 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2619_fsm 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2619_vm 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:08:00.646 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 21 19:07 2658 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2659 2023-07-21T19:08:00.647 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2662 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2663 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2664 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2665 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2666 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2667 2023-07-21T19:08:00.648 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:07 2673 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:07 2674 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2678 2023-07-21T19:08:00.649 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2679 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:08:00.650 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2696 2023-07-21T19:08:00.651 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2703 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2704 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:08:00.652 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:08:00.653 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:08:00.654 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:08:00.654 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:08:00.654 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:08:00.654 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:08:00.654 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:08:00.655 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:08:00.656 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:08:00.657 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:08:00.658 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:08:00.659 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 3455 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:08:00.660 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:08:00.661 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:08:00.662 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:08:00.662 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:08:00.662 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:08:00.663 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:08:00.664 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:08:00.665 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:08:00.665 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:08:00.665 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:08:00.665 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:08:00.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:08:00.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:00.771 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:00 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:08:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:00 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:00.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:00 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:00 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:08:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:00 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:00.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:00 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:01.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:08:00.759+0000 7fc959448700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:08:01.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:08:00.759+0000 7fc959448700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:08:01.168 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:08:00.759+0000 7fc959448700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:08:01.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:08:00.759+0000 7fc959448700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:08:01.169 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:08:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:08:00.759+0000 7fc959448700 -1 client.0 error registering admin socket command: (17) File exists 2023-07-21T19:08:01.242 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:08:01.243 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:08:01.244 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:08:01.245 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:08:01.246 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:08:01.247 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:total 7186 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 . 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 5 postgres postgres system_u:object_r:postgresql_db_t:s0 22070793 Jul 21 19:07 .. 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:08:01.248 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:08:01.249 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:08:01.250 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:08:01.251 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:08:01.252 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:07 1247 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1247_fsm 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1247_vm 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:07 1249 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1249_fsm 2023-07-21T19:08:01.253 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1249_vm 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:07 1259 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1259_vm 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:08:01.254 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:08:01.255 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:08:01.256 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:08:01.257 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606 2023-07-21T19:08:01.258 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606_vm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 21 19:07 2608 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2608_fsm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2608_vm 2023-07-21T19:08:01.259 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610_fsm 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2610_vm 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:08:01.260 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:08:01.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:08:01.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 21 19:07 2619 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2619_fsm 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2619_vm 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:08:01.263 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:08:01.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 21 19:07 2658 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2659 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2662 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2663 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2664 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2665 2023-07-21T19:08:01.265 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2666 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2667 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:07 2673 2023-07-21T19:08:01.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:07 2674 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2678 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2679 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:08:01.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:08:01.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2696 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:08:01.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2703 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2704 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:08:01.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:08:01.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:08:01.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:08:01.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:08:01.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:08:01.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:08:01.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 3455 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:08:01.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:08:01.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:08:01.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:08:01.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:08:01.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:08:01.345 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:08:01.346 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:08:01.347 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:08:01.348 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:08:01.349 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:08:01.350 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/global: 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:total 554 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 565760 Jul 21 19:07 . 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1136 2023-07-21T19:08:01.351 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1136_fsm 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1136_vm 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1137 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1213 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1213_fsm 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1213_vm 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1214 2023-07-21T19:08:01.352 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1214_fsm 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1214_vm 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1232 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1233 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1260 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1260_fsm 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1260_vm 2023-07-21T19:08:01.353 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1261 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1261_fsm 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1261_vm 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1262 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1262_fsm 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1262_vm 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2396 2023-07-21T19:08:01.354 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2396_fsm 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2396_vm 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2397 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2671 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2672 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2676 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2677 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2694 2023-07-21T19:08:01.355 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2695 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2697 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2698 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2846 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2846_vm 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2847 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2964 2023-07-21T19:08:01.356 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2964_vm 2023-07-21T19:08:01.357 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2965 2023-07-21T19:08:01.357 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2966 2023-07-21T19:08:01.357 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2966_vm 2023-07-21T19:08:01.357 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2967 2023-07-21T19:08:01.357 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3592 2023-07-21T19:08:01.409 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3592_vm 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3593 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 4060 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 4060_vm 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 4061 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6000 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6000_vm 2023-07-21T19:08:01.410 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6001 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6002 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6100 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6100_vm 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6114 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6115 2023-07-21T19:08:01.411 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 pg_control 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/log: 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-21T19:08:01.413 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:total 3 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:07 . 2023-07-21T19:08:01.414 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 mappings 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:07 replorigin_checkpoint 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 snapshots 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.415 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:07 .. 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:07 .. 2023-07-21T19:08:01.416 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 . 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 members 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 offsets 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.417 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 .. 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.418 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-21T19:08:01.419 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:08:01.419 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:08:01.485 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 .. 2023-07-21T19:08:01.486 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:08:01.486 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.486 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-21T19:08:01.487 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:08:01.487 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:08:01.487 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.488 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:08:01.488 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.488 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-21T19:08:01.489 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.489 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.489 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.489 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.490 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-21T19:08:01.490 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.490 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.491 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.491 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.491 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-21T19:08:01.491 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.492 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.492 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.492 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.492 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-21T19:08:01.492 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.493 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:08:01.494 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.495 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout:total 16386 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:07 . 2023-07-21T19:08:01.496 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.497 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:07 000000010000000000000001 2023-07-21T19:08:01.497 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 archive_status 2023-07-21T19:08:01.497 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.497 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:07 .. 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:08:01.498 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:08:01.499 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 39483722 Jul 21 19:07 .. 2023-07-21T19:08:01.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:08:01.500 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl start postgresql' 2023-07-21T19:08:01.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:01 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:01.785 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:01 smithi005 ceph-mon[126460]: pgmap v15: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.2 MiB/s wr, 251 op/s 2023-07-21T19:08:01.786 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:01 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:08:01.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:01 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:01.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:01 smithi101 ceph-mon[128136]: pgmap v15: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.2 MiB/s wr, 251 op/s 2023-07-21T19:08:01.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:01 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:08:01.858 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -s 500 -i' 2023-07-21T19:08:01.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:01 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:01.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:01 smithi156 ceph-mon[128316]: pgmap v15: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 461 KiB/s rd, 3.2 MiB/s wr, 251 op/s 2023-07-21T19:08:01.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:01 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:08:02.032 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_history" does not exist, skipping 2023-07-21T19:08:02.086 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_tellers" does not exist, skipping 2023-07-21T19:08:02.092 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_accounts" does not exist, skipping 2023-07-21T19:08:02.099 INFO:teuthology.orchestra.run.smithi005.stderr:NOTICE: table "pgbench_branches" does not exist, skipping 2023-07-21T19:08:02.440 INFO:teuthology.orchestra.run.smithi005.stderr:creating tables... 2023-07-21T19:08:03.211 INFO:teuthology.orchestra.run.smithi005.stderr:100000 of 50000000 tuples (0%) done (elapsed 0.77 s, remaining 382.96 s) 2023-07-21T19:08:03.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:03 smithi101 ceph-mon[128136]: pgmap v16: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 3.3 MiB/s wr, 278 op/s 2023-07-21T19:08:03.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:03 smithi101 ceph-mon[128136]: mgrmap e24: x(active, since 26s), standbys: y, z 2023-07-21T19:08:03.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:03 smithi005 ceph-mon[126460]: pgmap v16: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 3.3 MiB/s wr, 278 op/s 2023-07-21T19:08:03.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:03 smithi005 ceph-mon[126460]: mgrmap e24: x(active, since 26s), standbys: y, z 2023-07-21T19:08:03.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:03 smithi156 ceph-mon[128316]: pgmap v16: 97 pgs: 97 active+clean; 45 MiB data, 366 MiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 3.3 MiB/s wr, 278 op/s 2023-07-21T19:08:03.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:03 smithi156 ceph-mon[128316]: mgrmap e24: x(active, since 26s), standbys: y, z 2023-07-21T19:08:05.140 INFO:teuthology.orchestra.run.smithi005.stderr:200000 of 50000000 tuples (0%) done (elapsed 2.70 s, remaining 671.39 s) 2023-07-21T19:08:05.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:05 smithi101 ceph-mon[128136]: pgmap v17: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2023-07-21T19:08:05.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:05 smithi005 ceph-mon[126460]: pgmap v17: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2023-07-21T19:08:05.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:05 smithi156 ceph-mon[128316]: pgmap v17: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 2.9 MiB/s wr, 342 op/s 2023-07-21T19:08:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.982+0000 7f48951ff700 1 -- 172.21.15.5:0/2883512309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 msgr2=0x7f4890106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:05.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.982+0000 7f48951ff700 1 --2- 172.21.15.5:0/2883512309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890106930 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f48840099c0 tx=0x7f4884009cd0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 -- 172.21.15.5:0/2883512309 shutdown_connections 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 --2- 172.21.15.5:0/2883512309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890107010 0x7f489010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 --2- 172.21.15.5:0/2883512309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890101fd0 0x7f48901023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 --2- 172.21.15.5:0/2883512309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890106930 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 -- 172.21.15.5:0/2883512309 >> 172.21.15.5:0/2883512309 conn(0x7f48900fd840 msgr2=0x7f48900ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 -- 172.21.15.5:0/2883512309 shutdown_connections 2023-07-21T19:08:05.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.983+0000 7f48951ff700 1 -- 172.21.15.5:0/2883512309 wait complete. 2023-07-21T19:08:05.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.984+0000 7f48951ff700 1 Processor -- start 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.984+0000 7f48951ff700 1 -- start start 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890101fd0 0x7f4890197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 0x7f489019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019cfb0 con 0x7f4890102970 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019d120 con 0x7f4890107010 2023-07-21T19:08:05.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f48951ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489019d2e0 con 0x7f4890101fd0 2023-07-21T19:08:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56536/0 (socket says 172.21.15.5:56536) 2023-07-21T19:08:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f488e59c700 1 -- 172.21.15.5:0/1635646555 learned_addr learned my addr 172.21.15.5:0/1635646555 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f488ed9d700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890101fd0 0x7f4890197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:05.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.985+0000 7f488f59e700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 0x7f489019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 -- 172.21.15.5:0/1635646555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890101fd0 msgr2=0x7f4890197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890101fd0 0x7f4890197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 -- 172.21.15.5:0/1635646555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 msgr2=0x7f489019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 0x7f489019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 -- 172.21.15.5:0/1635646555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840095f0 con 0x7f4890102970 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488f59e700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 0x7f489019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f488e59c700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4884009fd0 tx=0x7f488400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f4877fff700 1 -- 172.21.15.5:0/1635646555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488401d070 con 0x7f4890102970 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f4877fff700 1 -- 172.21.15.5:0/1635646555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48840059a0 con 0x7f4890102970 2023-07-21T19:08:05.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f489019d560 con 0x7f4890102970 2023-07-21T19:08:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f4877fff700 1 -- 172.21.15.5:0/1635646555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884022e90 con 0x7f4890102970 2023-07-21T19:08:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.986+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48901a4e00 con 0x7f4890102970 2023-07-21T19:08:05.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.990+0000 7f4877fff700 1 -- 172.21.15.5:0/1635646555 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f4884022470 con 0x7f4890102970 2023-07-21T19:08:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.991+0000 7f48a8c0db80 1 Processor -- start 2023-07-21T19:08:06.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.991+0000 7f48a8c0db80 1 -- start start 2023-07-21T19:08:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 0x5634bdbcdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbce2d0 0x5634bdbc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 0x5634bdbc89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdacdc50 con 0x5634bdbc8580 2023-07-21T19:08:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdb209e0 con 0x5634bda246b0 2023-07-21T19:08:06.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdb19e70 con 0x5634bdbce2d0 2023-07-21T19:08:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 0x5634bdbc89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 0x5634bdbc89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56552/0 (socket says 172.21.15.5:56552) 2023-07-21T19:08:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.992+0000 7f488e59c700 1 -- 172.21.15.5:0/3007464910 learned_addr learned my addr 172.21.15.5:0/3007464910 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 0x5634bdbcdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f4877fff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 0x7f48780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f4877fff700 1 -- 172.21.15.5:0/1635646555 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f48840ce0b0 con 0x7f4890102970 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 0x7f48780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 -- 172.21.15.5:0/3007464910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbce2d0 msgr2=0x5634bdbc8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbce2d0 0x5634bdbc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 -- 172.21.15.5:0/3007464910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 msgr2=0x5634bdbc89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 0x5634bdbc89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.993+0000 7f488f59e700 1 -- 172.21.15.5:0/3007464910 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bd947d80 con 0x5634bda246b0 2023-07-21T19:08:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.994+0000 7f488f59e700 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 0x5634bdbcdd90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4888007990 tx=0x7f4888007ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eb0c90535fa085a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:06.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.994+0000 7f488f59e700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 0x7f48780a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f488801a000 tx=0x7f488801c000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.994+0000 7f48757fa700 1 -- 172.21.15.5:0/3007464910 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888021620 con 0x5634bda246b0 2023-07-21T19:08:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.994+0000 7f48757fa700 1 -- 172.21.15.5:0/3007464910 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488800f030 con 0x5634bda246b0 2023-07-21T19:08:06.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.994+0000 7f48757fa700 1 -- 172.21.15.5:0/3007464910 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888019600 con 0x5634bda246b0 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3007464910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 msgr2=0x5634bdbcdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 0x5634bdbcdd90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4888007990 tx=0x7f4888007ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3007464910 shutdown_connections 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbc8580 0x5634bdbc89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bda246b0 0x5634bdbcdd90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3007464910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbce2d0 0x5634bdbc8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3007464910 >> 172.21.15.5:0/3007464910 conn(0x5634bdbbdbe0 msgr2=0x5634bdbc2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:06.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3007464910 shutdown_connections 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.995+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3007464910 wait complete. 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.996+0000 7f48a8c0db80 1 -- start start 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.996+0000 7f48a8c0db80 1 Processor -- start 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.996+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbd19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.996+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd1f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.996+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdab9a60 con 0x5634bdbce2d0 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdab9760 con 0x5634bdbc8580 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bd99d8c0 con 0x5634bda246b0 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f48a8c0db80 10 client.0 _ll_register_callbacks cb 0x5634bdab7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbd19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbd19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33738/0 (socket says 172.21.15.5:33738) 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488ed9d700 1 -- 172.21.15.5:0/2097504785 learned_addr learned my addr 172.21.15.5:0/2097504785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:06.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56566/0 (socket says 172.21.15.5:56566) 2023-07-21T19:08:06.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488e59c700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 -- 172.21.15.5:0/2097504785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 msgr2=0x5634bdbd19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbd19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 -- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 msgr2=0x5634bdbd1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.997+0000 7f488f59e700 1 -- 172.21.15.5:0/2097504785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4888007540 con 0x5634bdbce2d0 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f488f59e700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4888007990 tx=0x7f4888021a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88bf9bc7141c3456 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:06.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f4872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4888004c90 con 0x5634bdbce2d0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634bda73490 con 0x5634bdbce2d0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4888004df0 con 0x5634bdbce2d0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f48880197e0 con 0x5634bdbce2d0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f48a8c0db80 10 client.15735 fetch_fsmap learned FSMap version 23 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.998+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x5634bd930c00 con 0x5634bdbce2d0 2023-07-21T19:08:06.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5634bda73490 con 0x5634bdbce2d0 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f488802a510 con 0x5634bdbce2d0 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 10 client.15735 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 20 client.15735 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdbe1ba0 0x5634bdbe3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 4 client.15735 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:05.999+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5634bdb12e50 con 0x5634bdbe1ba0 2023-07-21T19:08:06.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.000+0000 7f488ed9d700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdbe1ba0 0x5634bdbe3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.000+0000 7f4872ff5700 10 client.15735 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.000+0000 7f488ed9d700 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdbe1ba0 0x5634bdbe3fb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=543628880e66ac4a server_cookie=22c6f445dc83989 in_seq=0 out_seq=0 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f4872ff5700 1 -- 172.21.15.5:0/2097504785 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5634bdb12e50 con 0x5634bdbe1ba0 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f4872ff5700 10 client.15735 handle_command_reply: tid=0 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 client.15735 shutdown 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 msgr2=0x5634bdbd2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f4888007990 tx=0x7f4888021a50 comp rx=0 tx=0).stop 2023-07-21T19:08:06.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 shutdown_connections 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdbe1ba0 0x5634bdbe3fb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbd2460 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f4872ff5700 0 client.15735 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbd19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2097504785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.001+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 >> 172.21.15.5:0/2097504785 conn(0x5634bdbbdbe0 msgr2=0x5634bdbc1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:06.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.002+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 shutdown_connections 2023-07-21T19:08:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.002+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2097504785 wait complete. 2023-07-21T19:08:06.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.002+0000 7f48a8c0db80 20 client.15735 trim_cache size 0 max 16384 2023-07-21T19:08:06.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 Processor -- start 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 -- start start 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbccb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbbac00 con 0x5634bdbce2d0 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbc4b70 con 0x5634bdbc8580 2023-07-21T19:08:06.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.020+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbcb7a0 con 0x5634bda246b0 2023-07-21T19:08:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbccb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33752/0 (socket says 172.21.15.5:33752) 2023-07-21T19:08:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 -- 172.21.15.5:0/2833595391 learned_addr learned my addr 172.21.15.5:0/2833595391 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:06.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488e59c700 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 -- 172.21.15.5:0/2833595391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 msgr2=0x5634bdbd8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 -- 172.21.15.5:0/2833595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 msgr2=0x5634bdbccb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbccb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 -- 172.21.15.5:0/2833595391 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634bd96e1f0 con 0x5634bda246b0 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488f59e700 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.021+0000 7f488ed9d700 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5634bdbe48c0 tx=0x7f48800107c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46818b25aba7b842 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:06.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.022+0000 7f4875ffb700 1 -- 172.21.15.5:0/2833595391 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4880017040 con 0x5634bda246b0 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.022+0000 7f4875ffb700 1 -- 172.21.15.5:0/2833595391 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488001bd70 con 0x5634bda246b0 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.022+0000 7f4875ffb700 1 -- 172.21.15.5:0/2833595391 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488001b780 con 0x5634bda246b0 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.022+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2833595391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 msgr2=0x5634bdbc8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x5634bdbe48c0 tx=0x7f48800107c0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2833595391 shutdown_connections 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdbccb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bda246b0 0x5634bdbc8040 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/2833595391 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdbd8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2833595391 >> 172.21.15.5:0/2833595391 conn(0x5634bdbbdbe0 msgr2=0x5634bdbddf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2833595391 shutdown_connections 2023-07-21T19:08:06.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 -- 172.21.15.5:0/2833595391 wait complete. 2023-07-21T19:08:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 -- start start 2023-07-21T19:08:06.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.023+0000 7f48a8c0db80 1 Processor -- start 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdc2ed60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbd6a30 0x5634bdbd6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbd73e0 con 0x5634bdbce2d0 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbd7520 con 0x5634bdbc8580 2023-07-21T19:08:06.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634bdbd76b0 con 0x5634bdbd6a30 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f48a8c0db80 10 client.0 _ll_register_callbacks cb 0x7f4884097220 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f488e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56582/0 (socket says 172.21.15.5:56582) 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.024+0000 7f488e59c700 1 -- 172.21.15.5:0/3979897565 learned_addr learned my addr 172.21.15.5:0/3979897565 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488ed9d700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdc2ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 -- 172.21.15.5:0/3979897565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbd6a30 msgr2=0x5634bdbd6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbd6a30 0x5634bdbd6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 -- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 msgr2=0x5634bdc2ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdc2ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 -- 172.21.15.5:0/3979897565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5634bd930c00 con 0x5634bdbce2d0 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f488e59c700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4884097b20 tx=0x7f48840972d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a0da2d2e261da2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f4872ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488409aa10 con 0x5634bdbce2d0 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f488400fb80 con 0x5634bdbce2d0 2023-07-21T19:08:06.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.025+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f488800f030 con 0x5634bdbce2d0 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488409b0b0 con 0x5634bdbce2d0 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 10 client.15741 fetch_fsmap learned FSMap version 23 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=23,monmap=4+}) v3 -- 0x7f4880009e20 con 0x5634bdbce2d0 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f48840c9020 con 0x5634bdbce2d0 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 23) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f48840aed20 con 0x5634bdbce2d0 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 10 client.15741 fetch_fsmap finished waiting for FSMap version 23 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:06.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 20 client.15741 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdc36c00 0x5634bdc38fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 4 client.15741 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.026+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5634bdad4ff0 con 0x5634bdc36c00 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.027+0000 7f488ed9d700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdc36c00 0x5634bdc38fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.027+0000 7f4872ff5700 10 client.15741 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:06.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.027+0000 7f488ed9d700 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdc36c00 0x5634bdc38fe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca1176db58e20db6 server_cookie=ee0848f30bbd358a in_seq=0 out_seq=0 2023-07-21T19:08:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f4872ff5700 1 -- 172.21.15.5:0/3979897565 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5634bdad4ff0 con 0x5634bdc36c00 2023-07-21T19:08:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f4872ff5700 10 client.15741 handle_command_reply: tid=0 2023-07-21T19:08:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 client.15741 shutdown 2023-07-21T19:08:06.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 msgr2=0x5634bdc2f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f4884097b20 tx=0x7f48840972d0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 shutdown_connections 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5634bdbd6a30 0x5634bdbd6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5634bdbc8580 0x5634bdc2ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5634bdc36c00 0x5634bdc38fe0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 --2- 172.21.15.5:0/3979897565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5634bdbce2d0 0x5634bdc2f2a0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f4872ff5700 0 client.15741 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 >> 172.21.15.5:0/3979897565 conn(0x5634bdbbdbe0 msgr2=0x5634bdbd01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 shutdown_connections 2023-07-21T19:08:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.116+0000 7f48a8c0db80 1 -- 172.21.15.5:0/3979897565 wait complete. 2023-07-21T19:08:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.117+0000 7f48a8c0db80 20 client.15741 trim_cache size 0 max 16384 2023-07-21T19:08:06.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:08:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:08:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:08:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:08:06.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.117+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 msgr2=0x7f48780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.117+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 0x7f48780a3890 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f488801a000 tx=0x7f488801c000 comp rx=0 tx=0).stop 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.117+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 msgr2=0x7f4890198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.117+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f4884009fd0 tx=0x7f488400f690 comp rx=0 tx=0).stop 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 shutdown_connections 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4890107010 0x7f489019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4890101fd0 0x7f4890197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48780a13e0 0x7f48780a3890 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 --2- 172.21.15.5:0/1635646555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4890102970 0x7f4890198300 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 >> 172.21.15.5:0/1635646555 conn(0x7f48900fd840 msgr2=0x7f48900fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 shutdown_connections 2023-07-21T19:08:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:06.118+0000 7f48951ff700 1 -- 172.21.15.5:0/1635646555 wait complete. 2023-07-21T19:08:07.069 INFO:teuthology.orchestra.run.smithi005.stderr:300000 of 50000000 tuples (0%) done (elapsed 4.63 s, remaining 766.30 s) 2023-07-21T19:08:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:07 smithi101 ceph-mon[128136]: pgmap v18: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-07-21T19:08:07.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:07 smithi005 ceph-mon[126460]: pgmap v18: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-07-21T19:08:07.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:07 smithi156 ceph-mon[128316]: pgmap v18: 97 pgs: 97 active+clean; 47 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 1.5 MiB/s wr, 291 op/s 2023-07-21T19:08:08.409 INFO:teuthology.orchestra.run.smithi005.stderr:400000 of 50000000 tuples (0%) done (elapsed 5.97 s, remaining 739.71 s) 2023-07-21T19:08:09.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:09 smithi101 ceph-mon[128136]: pgmap v19: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 12 MiB/s wr, 357 op/s 2023-07-21T19:08:09.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:09 smithi005 ceph-mon[126460]: pgmap v19: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 12 MiB/s wr, 357 op/s 2023-07-21T19:08:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:09 smithi156 ceph-mon[128316]: pgmap v19: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 633 KiB/s rd, 12 MiB/s wr, 357 op/s 2023-07-21T19:08:10.320 INFO:teuthology.orchestra.run.smithi005.stderr:500000 of 50000000 tuples (1%) done (elapsed 7.88 s, remaining 779.71 s) 2023-07-21T19:08:11.664 INFO:teuthology.orchestra.run.smithi005.stderr:600000 of 50000000 tuples (1%) done (elapsed 9.22 s, remaining 759.11 s) 2023-07-21T19:08:11.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:11 smithi101 ceph-mon[128136]: pgmap v20: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 10 MiB/s wr, 174 op/s 2023-07-21T19:08:11.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:11 smithi005 ceph-mon[126460]: pgmap v20: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 10 MiB/s wr, 174 op/s 2023-07-21T19:08:11.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:11 smithi156 ceph-mon[128316]: pgmap v20: 97 pgs: 97 active+clean; 120 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 10 MiB/s wr, 174 op/s 2023-07-21T19:08:12.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:12.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:12 smithi101 ceph-mon[128136]: pgmap v21: 97 pgs: 97 active+clean; 153 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 176 op/s 2023-07-21T19:08:12.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:12.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:12 smithi005 ceph-mon[126460]: pgmap v21: 97 pgs: 97 active+clean; 153 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 176 op/s 2023-07-21T19:08:12.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:12.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:12 smithi156 ceph-mon[128316]: pgmap v21: 97 pgs: 97 active+clean; 153 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 314 KiB/s rd, 16 MiB/s wr, 176 op/s 2023-07-21T19:08:13.476 INFO:teuthology.orchestra.run.smithi005.stderr:700000 of 50000000 tuples (1%) done (elapsed 11.03 s, remaining 776.97 s) 2023-07-21T19:08:15.362 INFO:teuthology.orchestra.run.smithi005.stderr:800000 of 50000000 tuples (1%) done (elapsed 12.92 s, remaining 794.45 s) 2023-07-21T19:08:15.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:15 smithi101 ceph-mon[128136]: pgmap v22: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 21 MiB/s wr, 157 op/s 2023-07-21T19:08:15.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:15 smithi005 ceph-mon[126460]: pgmap v22: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 21 MiB/s wr, 157 op/s 2023-07-21T19:08:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:15 smithi156 ceph-mon[128316]: pgmap v22: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 323 KiB/s rd, 21 MiB/s wr, 157 op/s 2023-07-21T19:08:16.706 INFO:teuthology.orchestra.run.smithi005.stderr:900000 of 50000000 tuples (1%) done (elapsed 14.26 s, remaining 778.07 s) 2023-07-21T19:08:17.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:17 smithi101 ceph-mon[128136]: pgmap v23: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 76 op/s 2023-07-21T19:08:17.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:17 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:17.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:17 smithi005 ceph-mon[126460]: pgmap v23: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 76 op/s 2023-07-21T19:08:17.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:17 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:17.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:17 smithi156 ceph-mon[128316]: pgmap v23: 97 pgs: 97 active+clean; 184 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 21 MiB/s wr, 76 op/s 2023-07-21T19:08:17.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:17 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:08:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:1000000 of 50000000 tuples (2%) done (elapsed 16.13 s, remaining 790.51 s) 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 -- 172.21.15.5:0/2955517644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec54105940 msgr2=0x7fec54105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 --2- 172.21.15.5:0/2955517644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec54105940 0x7fec54105d10 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fec44009990 tx=0x7fec44009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 -- 172.21.15.5:0/2955517644 shutdown_connections 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 --2- 172.21.15.5:0/2955517644 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec541002e0 0x7fec5410c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 --2- 172.21.15.5:0/2955517644 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec54105940 0x7fec54105d10 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 --2- 172.21.15.5:0/2955517644 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec540ff930 0x7fec540ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 -- 172.21.15.5:0/2955517644 >> 172.21.15.5:0/2955517644 conn(0x7fec540fb450 msgr2=0x7fec540fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.295+0000 7fec5979d700 1 -- 172.21.15.5:0/2955517644 shutdown_connections 2023-07-21T19:08:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.296+0000 7fec5979d700 1 -- 172.21.15.5:0/2955517644 wait complete. 2023-07-21T19:08:19.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.297+0000 7fec5979d700 1 Processor -- start 2023-07-21T19:08:19.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.297+0000 7fec5979d700 1 -- start start 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.297+0000 7fec5979d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.297+0000 7fec5979d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 0x7fec5419dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec5979d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54105940 0x7fec541978d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec5979d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec54197f40 con 0x7fec540ff930 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec5979d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec541980b0 con 0x7fec541002e0 2023-07-21T19:08:19.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec5979d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec54198270 con 0x7fec54105940 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39536/0 (socket says 172.21.15.5:39536) 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 -- 172.21.15.5:0/3990157136 learned_addr learned my addr 172.21.15.5:0/3990157136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec58f9c700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54105940 0x7fec541978d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec537fe700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 0x7fec5419dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 -- 172.21.15.5:0/3990157136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54105940 msgr2=0x7fec541978d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54105940 0x7fec541978d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 -- 172.21.15.5:0/3990157136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 msgr2=0x7fec5419dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 0x7fec5419dda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec53fff700 1 -- 172.21.15.5:0/3990157136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440095f0 con 0x7fec540ff930 2023-07-21T19:08:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.298+0000 7fec537fe700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 0x7fec5419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:08:19.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec53fff700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fec440098e0 tx=0x7fec44004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec517fa700 1 -- 172.21.15.5:0/3990157136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4401d070 con 0x7fec540ff930 2023-07-21T19:08:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec517fa700 1 -- 172.21.15.5:0/3990157136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec44022470 con 0x7fec540ff930 2023-07-21T19:08:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec541984f0 con 0x7fec540ff930 2023-07-21T19:08:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec517fa700 1 -- 172.21.15.5:0/3990157136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec44017650 con 0x7fec540ff930 2023-07-21T19:08:19.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.299+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec541a4850 con 0x7fec540ff930 2023-07-21T19:08:19.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.303+0000 7fec517fa700 1 -- 172.21.15.5:0/3990157136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fec440225e0 con 0x7fec540ff930 2023-07-21T19:08:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.305+0000 7fec517fa700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 0x7fec3c0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.305+0000 7fec6e3fcb80 1 Processor -- start 2023-07-21T19:08:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.305+0000 7fec517fa700 1 -- 172.21.15.5:0/3990157136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fec440ce410 con 0x7fec540ff930 2023-07-21T19:08:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.305+0000 7fec6e3fcb80 1 -- start start 2023-07-21T19:08:19.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.305+0000 7fec537fe700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 0x7fec3c0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f34e6b0 0x561d2f500740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f500c80 0x561d2f4fa9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f3f7c50 con 0x561d2f5010d0 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f44a9e0 con 0x561d2f500c80 2023-07-21T19:08:19.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f443e70 con 0x561d2f34e6b0 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec58f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec58f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39546/0 (socket says 172.21.15.5:39546) 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec58f9c700 1 -- 172.21.15.5:0/4153398444 learned_addr learned my addr 172.21.15.5:0/4153398444 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.306+0000 7fec53fff700 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f34e6b0 0x561d2f500740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec537fe700 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f500c80 0x561d2f4fa9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec537fe700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 0x7fec3c0a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec540fed20 tx=0x7fec48009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 -- 172.21.15.5:0/4153398444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f34e6b0 msgr2=0x561d2f500740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f34e6b0 0x561d2f500740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 -- 172.21.15.5:0/4153398444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f500c80 msgr2=0x561d2f4fa9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f500c80 0x561d2f4fa9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 -- 172.21.15.5:0/4153398444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d2f271d80 con 0x561d2f5010d0 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec58f9c700 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00f570 tx=0x7fec4c00f880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b346543ad09a45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.307+0000 7fec3a7fc700 1 -- 172.21.15.5:0/4153398444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c014470 con 0x561d2f5010d0 2023-07-21T19:08:19.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec3a7fc700 1 -- 172.21.15.5:0/4153398444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c014a60 con 0x561d2f5010d0 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec3a7fc700 1 -- 172.21.15.5:0/4153398444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c013a60 con 0x561d2f5010d0 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/4153398444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 msgr2=0x561d2f4faf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fec4c00f570 tx=0x7fec4c00f880 comp rx=0 tx=0).stop 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/4153398444 shutdown_connections 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f4faf10 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.308+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f34e6b0 0x561d2f500740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.309+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/4153398444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f500c80 0x561d2f4fa9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.309+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/4153398444 >> 172.21.15.5:0/4153398444 conn(0x561d2f4e7d40 msgr2=0x561d2f4ed340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.309+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/4153398444 shutdown_connections 2023-07-21T19:08:19.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.309+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/4153398444 wait complete. 2023-07-21T19:08:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.309+0000 7fec6e3fcb80 1 -- start start 2023-07-21T19:08:19.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 Processor -- start 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f34e6b0 0x561d2f50a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f5010d0 0x561d2f504290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f3e3a60 con 0x561d2f34e6b0 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f3e3760 con 0x561d2f5010d0 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.310+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f2c78c0 con 0x561d2f500c80 2023-07-21T19:08:19.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec6e3fcb80 10 client.0 _ll_register_callbacks cb 0x561d2f3e1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec53fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f34e6b0 0x561d2f50a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52104/0 (socket says 172.21.15.5:52104) 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 -- 172.21.15.5:0/319738501 learned_addr learned my addr 172.21.15.5:0/319738501 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec58f9c700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f5010d0 0x561d2f504290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 -- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f5010d0 msgr2=0x561d2f504290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f5010d0 0x561d2f504290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 -- 172.21.15.5:0/319738501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f34e6b0 msgr2=0x561d2f50a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f34e6b0 0x561d2f50a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.311+0000 7fec537fe700 1 -- 172.21.15.5:0/319738501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec4c00f1d0 con 0x561d2f500c80 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec537fe700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec48010970 tx=0x7fec48004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb5edf0eb593f4b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec337fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48003d30 con 0x561d2f500c80 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561d2f39d490 con 0x561d2f500c80 2023-07-21T19:08:19.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fec48012450 con 0x561d2f500c80 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.312+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec48012630 con 0x561d2f500c80 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 10 client.25474 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561d2f25ac00 con 0x561d2f500c80 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561d2f39d490 con 0x561d2f500c80 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec48026020 con 0x561d2f500c80 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 10 client.25474 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 10 client.25474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 20 client.25474 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f514550 0x561d2f516960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 4 client.25474 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.313+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561d2f43ce50 con 0x561d2f514550 2023-07-21T19:08:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.314+0000 7fec53fff700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f514550 0x561d2f516960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.314+0000 7fec53fff700 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f514550 0x561d2f516960 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f3dd9d8ee911ed server_cookie=a38c18a1f71cb9c7 in_seq=0 out_seq=0 2023-07-21T19:08:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.314+0000 7fec337fe700 10 client.25474 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec337fe700 1 -- 172.21.15.5:0/319738501 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561d2f43ce50 con 0x561d2f514550 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec337fe700 10 client.25474 handle_command_reply: tid=0 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 client.25474 shutdown 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 msgr2=0x561d2f50a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fec48010970 tx=0x7fec48004750 comp rx=0 tx=0).stop 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 shutdown_connections 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f5010d0 0x561d2f504290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f514550 0x561d2f516960 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f34e6b0 0x561d2f50a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/319738501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f50a540 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec337fe700 0 client.25474 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.315+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 >> 172.21.15.5:0/319738501 conn(0x561d2f4e7d40 msgr2=0x561d2f4f1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.316+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 shutdown_connections 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.316+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/319738501 wait complete. 2023-07-21T19:08:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.316+0000 7fec6e3fcb80 20 client.25474 trim_cache size 0 max 16384 2023-07-21T19:08:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.340+0000 7fec6e3fcb80 1 Processor -- start 2023-07-21T19:08:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.340+0000 7fec6e3fcb80 1 -- start start 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f504240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f5010d0 0x561d2f4ed300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f39d490 con 0x561d2f500c80 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f4edf10 con 0x561d2f34e6b0 2023-07-21T19:08:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f4e4d80 con 0x561d2f5010d0 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39572/0 (socket says 172.21.15.5:39572) 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 -- 172.21.15.5:0/3243148569 learned_addr learned my addr 172.21.15.5:0/3243148569 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec58f9c700 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f5010d0 0x561d2f4ed300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 -- 172.21.15.5:0/3243148569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f5010d0 msgr2=0x561d2f4ed300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f5010d0 0x561d2f4ed300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 -- 172.21.15.5:0/3243148569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 msgr2=0x561d2f504240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.341+0000 7fec537fe700 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f504240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.342+0000 7fec537fe700 1 -- 172.21.15.5:0/3243148569 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561d2f2981f0 con 0x561d2f500c80 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.342+0000 7fec537fe700 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fec48020700 tx=0x7fec48020820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a38d3e540dbbb53c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.342+0000 7fec3affd700 1 -- 172.21.15.5:0/3243148569 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480098a0 con 0x561d2f500c80 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.342+0000 7fec3affd700 1 -- 172.21.15.5:0/3243148569 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec48020970 con 0x561d2f500c80 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.342+0000 7fec3affd700 1 -- 172.21.15.5:0/3243148569 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec480126b0 con 0x561d2f500c80 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/3243148569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 msgr2=0x561d2f4ecdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fec48020700 tx=0x7fec48020820 comp rx=0 tx=0).stop 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/3243148569 shutdown_connections 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f5010d0 0x561d2f4ed300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f504240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/3243148569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f500c80 0x561d2f4ecdc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.343+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/3243148569 >> 172.21.15.5:0/3243148569 conn(0x561d2f4e7d40 msgr2=0x561d2f503b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.344+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/3243148569 shutdown_connections 2023-07-21T19:08:19.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.344+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/3243148569 wait complete. 2023-07-21T19:08:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.344+0000 7fec6e3fcb80 1 -- start start 2023-07-21T19:08:19.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.344+0000 7fec6e3fcb80 1 Processor -- start 2023-07-21T19:08:19.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f5081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f5086f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f567950 con 0x561d2f5010d0 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f567ac0 con 0x561d2f34e6b0 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561d2f567c80 con 0x561d2f500c80 2023-07-21T19:08:19.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec6e3fcb80 10 client.0 _ll_register_callbacks cb 0x7fec480042c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec58f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec58f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39578/0 (socket says 172.21.15.5:39578) 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.345+0000 7fec58f9c700 1 -- 172.21.15.5:0/1776251254 learned_addr learned my addr 172.21.15.5:0/1776251254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec537fe700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f5086f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec53fff700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f5081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 -- 172.21.15.5:0/1776251254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 msgr2=0x561d2f5086f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f5086f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 -- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 msgr2=0x561d2f5081b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f5081b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 -- 172.21.15.5:0/1776251254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561d2f25ac00 con 0x561d2f5010d0 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec53fff700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f5081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec58f9c700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fec4c007520 tx=0x7fec4c0146a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ecb0d1dae693a72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.346+0000 7fec337fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:19.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec4c022070 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec4c014a60 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fec4c003a00 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec4c003bc0 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec6e3fcb80 10 client.15759 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561d2f270250 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec4c014a60 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.347+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec4c012070 con 0x561d2f5010d0 2023-07-21T19:08:19.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 10 client.15759 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 10 client.15759 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 20 client.15759 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f569410 0x561d2f56b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 4 client.15759 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561d2f3feff0 con 0x561d2f569410 2023-07-21T19:08:19.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.348+0000 7fec53fff700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f569410 0x561d2f56b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.349+0000 7fec337fe700 10 client.15759 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:19.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.349+0000 7fec53fff700 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f569410 0x561d2f56b820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=800397aed7ad86a server_cookie=1c8e2e5782d08abc in_seq=0 out_seq=0 2023-07-21T19:08:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec337fe700 1 -- 172.21.15.5:0/1776251254 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561d2f3feff0 con 0x561d2f569410 2023-07-21T19:08:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec337fe700 10 client.15759 handle_command_reply: tid=0 2023-07-21T19:08:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 client.15759 shutdown 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 msgr2=0x561d2f567410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fec4c007520 tx=0x7fec4c0146a0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 shutdown_connections 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561d2f569410 0x561d2f56b820 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561d2f5010d0 0x561d2f567410 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561d2f34e6b0 0x561d2f5081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 --2- 172.21.15.5:0/1776251254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561d2f500c80 0x561d2f5086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 >> 172.21.15.5:0/1776251254 conn(0x561d2f4e7d40 msgr2=0x561d2f501510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec337fe700 0 client.15759 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 shutdown_connections 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.365+0000 7fec6e3fcb80 1 -- 172.21.15.5:0/1776251254 wait complete. 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec6e3fcb80 20 client.15759 trim_cache size 0 max 16384 2023-07-21T19:08:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:08:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:08:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:08:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 msgr2=0x7fec3c0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 0x7fec3c0a3910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec540fed20 tx=0x7fec48009260 comp rx=0 tx=0).stop 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 msgr2=0x7fec5419d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fec440098e0 tx=0x7fec44004730 comp rx=0 tx=0).stop 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 shutdown_connections 2023-07-21T19:08:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec54105940 0x7fec541978d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.366+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec3c0a1460 0x7fec3c0a3910 secure :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fec540fed20 tx=0x7fec48009260 comp rx=0 tx=0).stop 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.367+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec540ff930 0x7fec5419d860 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.367+0000 7fec5979d700 1 --2- 172.21.15.5:0/3990157136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec541002e0 0x7fec5419dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.367+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 >> 172.21.15.5:0/3990157136 conn(0x7fec540fb450 msgr2=0x7fec54072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.367+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 shutdown_connections 2023-07-21T19:08:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:19.367+0000 7fec5979d700 1 -- 172.21.15.5:0/3990157136 wait complete. 2023-07-21T19:08:19.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:19 smithi101 ceph-mon[128136]: pgmap v24: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 33 MiB/s wr, 88 op/s 2023-07-21T19:08:19.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:19 smithi005 ceph-mon[126460]: pgmap v24: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 33 MiB/s wr, 88 op/s 2023-07-21T19:08:19.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:19 smithi156 ceph-mon[128316]: pgmap v24: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 33 MiB/s wr, 88 op/s 2023-07-21T19:08:20.342 INFO:teuthology.orchestra.run.smithi005.stderr:1100000 of 50000000 tuples (2%) done (elapsed 17.90 s, remaining 795.64 s) 2023-07-21T19:08:21.590 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:21 smithi101 ceph-mon[128136]: pgmap v25: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:21.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:21 smithi005 ceph-mon[126460]: pgmap v25: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:21.676 INFO:teuthology.orchestra.run.smithi005.stderr:1200000 of 50000000 tuples (2%) done (elapsed 19.23 s, remaining 782.11 s) 2023-07-21T19:08:21.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:21 smithi156 ceph-mon[128316]: pgmap v25: 97 pgs: 97 active+clean; 251 MiB data, 460 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:1300000 of 50000000 tuples (2%) done (elapsed 21.02 s, remaining 787.40 s) 2023-07-21T19:08:23.591 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:23 smithi101 ceph-mon[128136]: pgmap v26: 97 pgs: 97 active+clean; 279 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 27 MiB/s wr, 26 op/s 2023-07-21T19:08:23.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:23 smithi005 ceph-mon[126460]: pgmap v26: 97 pgs: 97 active+clean; 279 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 27 MiB/s wr, 26 op/s 2023-07-21T19:08:23.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:23 smithi156 ceph-mon[128316]: pgmap v26: 97 pgs: 97 active+clean; 279 MiB data, 469 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 27 MiB/s wr, 26 op/s 2023-07-21T19:08:25.357 INFO:teuthology.orchestra.run.smithi005.stderr:1400000 of 50000000 tuples (2%) done (elapsed 22.91 s, remaining 795.43 s) 2023-07-21T19:08:25.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:25 smithi005 ceph-mon[126460]: pgmap v27: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 28 MiB/s wr, 36 op/s 2023-07-21T19:08:25.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:25 smithi156 ceph-mon[128316]: pgmap v27: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 28 MiB/s wr, 36 op/s 2023-07-21T19:08:25.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:25 smithi101 ceph-mon[128136]: pgmap v27: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 28 MiB/s wr, 36 op/s 2023-07-21T19:08:26.680 INFO:teuthology.orchestra.run.smithi005.stderr:1500000 of 50000000 tuples (3%) done (elapsed 24.24 s, remaining 783.64 s) 2023-07-21T19:08:27.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:27 smithi005 ceph-mon[126460]: pgmap v28: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 28 op/s 2023-07-21T19:08:27.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:27 smithi156 ceph-mon[128316]: pgmap v28: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 28 op/s 2023-07-21T19:08:27.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:27 smithi101 ceph-mon[128136]: pgmap v28: 97 pgs: 97 active+clean; 319 MiB data, 488 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 28 op/s 2023-07-21T19:08:28.543 INFO:teuthology.orchestra.run.smithi005.stderr:1600000 of 50000000 tuples (3%) done (elapsed 26.10 s, remaining 789.50 s) 2023-07-21T19:08:29.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:29 smithi005 ceph-mon[126460]: pgmap v29: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 39 op/s 2023-07-21T19:08:29.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:29 smithi156 ceph-mon[128316]: pgmap v29: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 39 op/s 2023-07-21T19:08:29.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:29 smithi101 ceph-mon[128136]: pgmap v29: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 35 MiB/s wr, 39 op/s 2023-07-21T19:08:29.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:08:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:1700000 of 50000000 tuples (3%) done (elapsed 27.46 s, remaining 780.07 s) 2023-07-21T19:08:30.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.092+0000 7f06c61ff700 1 -- 172.21.15.5:0/3458218791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 msgr2=0x7f06c0109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.092+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3458218791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0109b00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009990 tx=0x7f06b4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 -- 172.21.15.5:0/3458218791 shutdown_connections 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3458218791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c010a040 0x7f06c010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3458218791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c0100d00 0x7f06c01010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3458218791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0109b00 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 -- 172.21.15.5:0/3458218791 >> 172.21.15.5:0/3458218791 conn(0x7f06c00fa4b0 msgr2=0x7f06c00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 -- 172.21.15.5:0/3458218791 shutdown_connections 2023-07-21T19:08:30.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.093+0000 7f06c61ff700 1 -- 172.21.15.5:0/3458218791 wait complete. 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.094+0000 7f06c61ff700 1 Processor -- start 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.094+0000 7f06c61ff700 1 -- start start 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c010a040 0x7f06c0193790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0193e00 con 0x7f06c0101610 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0193f70 con 0x7f06c010a040 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06c61ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0194130 con 0x7f06c0100d00 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52970/0 (socket says 172.21.15.5:52970) 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 -- 172.21.15.5:0/3661004621 learned_addr learned my addr 172.21.15.5:0/3661004621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60520/0 (socket says 172.21.15.5:60520) 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bffff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c010a040 0x7f06c0193790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 -- 172.21.15.5:0/3661004621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c010a040 msgr2=0x7f06c0193790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c010a040 0x7f06c0193790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.095+0000 7f06bf7fe700 1 -- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 msgr2=0x7f06c0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bf7fe700 1 -- 172.21.15.5:0/3661004621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06b40095f0 con 0x7f06c0100d00 2023-07-21T19:08:30.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06beffd700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06b000e9a0 tx=0x7f06b000ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bcff9700 1 -- 172.21.15.5:0/3661004621 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0009790 con 0x7f06c0100d00 2023-07-21T19:08:30.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bcff9700 1 -- 172.21.15.5:0/3661004621 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b000cbd0 con 0x7f06c0100d00 2023-07-21T19:08:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0194410 con 0x7f06c0100d00 2023-07-21T19:08:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.096+0000 7f06bcff9700 1 -- 172.21.15.5:0/3661004621 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b0018680 con 0x7f06c0100d00 2023-07-21T19:08:30.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.097+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c01a0710 con 0x7f06c0100d00 2023-07-21T19:08:30.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.101+0000 7f06bcff9700 1 -- 172.21.15.5:0/3661004621 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f06b000cd40 con 0x7f06c0100d00 2023-07-21T19:08:30.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.102+0000 7f06bcff9700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 0x7f06a80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.102+0000 7f06bcff9700 1 -- 172.21.15.5:0/3661004621 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f06b0014070 con 0x7f06c0100d00 2023-07-21T19:08:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.102+0000 7f06d9b94b80 1 Processor -- start 2023-07-21T19:08:30.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.102+0000 7f06beffd700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 0x7f06a80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 -- start start 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06beffd700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 0x7f06a80a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06c0194c00 tx=0x7f06b400b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 0x5620243a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 0x5620243b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620242adcb0 con 0x5620243b2ad0 2023-07-21T19:08:30.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562024300a40 con 0x5620243a72d0 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.103+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620242f9ed0 con 0x5620243a7be0 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 0x5620243b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52982/0 (socket says 172.21.15.5:52982) 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 -- 172.21.15.5:0/995944080 learned_addr learned my addr 172.21.15.5:0/995944080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 0x5620243b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60528/0 (socket says 172.21.15.5:60528) 2023-07-21T19:08:30.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 0x5620243a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 -- 172.21.15.5:0/995944080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 msgr2=0x5620243a76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 0x5620243a76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 -- 172.21.15.5:0/995944080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 msgr2=0x5620243b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 0x5620243b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06beffd700 1 -- 172.21.15.5:0/995944080 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562024127de0 con 0x5620243a7be0 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.104+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 0x5620243a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:30.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.105+0000 7f06beffd700 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009990 tx=0x7f06b4020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3f072700e1430bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.105+0000 7f06a5ffb700 1 -- 172.21.15.5:0/995944080 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b402b070 con 0x5620243a7be0 2023-07-21T19:08:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.105+0000 7f06a5ffb700 1 -- 172.21.15.5:0/995944080 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b401e040 con 0x5620243a7be0 2023-07-21T19:08:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.105+0000 7f06a5ffb700 1 -- 172.21.15.5:0/995944080 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4024d60 con 0x5620243a7be0 2023-07-21T19:08:30.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- 172.21.15.5:0/995944080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 msgr2=0x5620243b2590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f06b4009990 tx=0x7f06b4020000 comp rx=0 tx=0).stop 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- 172.21.15.5:0/995944080 shutdown_connections 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243b2ad0 0x5620243b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a72d0 0x5620243a76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/995944080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a7be0 0x5620243b2590 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- 172.21.15.5:0/995944080 >> 172.21.15.5:0/995944080 conn(0x56202439dda0 msgr2=0x5620243a36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- 172.21.15.5:0/995944080 shutdown_connections 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- 172.21.15.5:0/995944080 wait complete. 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.106+0000 7f06d9b94b80 1 -- start start 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 Processor -- start 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 0x5620243af790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 0x5620243afed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562024299ac0 con 0x5620243a72d0 2023-07-21T19:08:30.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620242997c0 con 0x5620243a7be0 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56202417d920 con 0x5620243b2ad0 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.107+0000 7f06d9b94b80 10 client.0 _ll_register_callbacks cb 0x562024297760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60538/0 (socket says 172.21.15.5:60538) 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bf7fe700 1 -- 172.21.15.5:0/359485969 learned_addr learned my addr 172.21.15.5:0/359485969 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 0x5620243afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06beffd700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 0x5620243af790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 -- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 msgr2=0x5620243af790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 0x5620243af790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 -- 172.21.15.5:0/359485969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 msgr2=0x5620243b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bffff700 1 -- 172.21.15.5:0/359485969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c01a0710 con 0x5620243b2ad0 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06beffd700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 0x5620243af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.108+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:30.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f06bffff700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 0x5620243afed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06c019a420 tx=0x7f06b800bf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=528d5f7e34bc6fb2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f069effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8004db0 con 0x5620243b2ad0 2023-07-21T19:08:30.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5620242534f0 con 0x5620243b2ad0 2023-07-21T19:08:30.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f06b8016a60 con 0x5620243b2ad0 2023-07-21T19:08:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b8016c20 con 0x5620243b2ad0 2023-07-21T19:08:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f06d9b94b80 10 client.25492 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.109+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f06c010d170 con 0x5620243b2ad0 2023-07-21T19:08:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5620242534f0 con 0x5620243b2ad0 2023-07-21T19:08:30.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06b8018040 con 0x5620243b2ad0 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 10 client.25492 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 10 client.25492 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 20 client.25492 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5620243c24b0 0x5620243c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 4 client.25492 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.110+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5620242f2eb0 con 0x5620243c24b0 2023-07-21T19:08:30.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.111+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5620243c24b0 0x5620243c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.111+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5620243c24b0 0x5620243c48c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97c5eee0a8769744 server_cookie=b224052a6d1101b4 in_seq=0 out_seq=0 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.111+0000 7f069effd700 10 client.25492 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f069effd700 1 -- 172.21.15.5:0/359485969 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5620242f2eb0 con 0x5620243c24b0 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f069effd700 10 client.25492 handle_command_reply: tid=0 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 client.25492 shutdown 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 msgr2=0x5620243afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 0x5620243afed0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06c019a420 tx=0x7f06b800bf70 comp rx=0 tx=0).stop 2023-07-21T19:08:30.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 shutdown_connections 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243b2ad0 0x5620243afed0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5620243c24b0 0x5620243c48c0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a72d0 0x5620243b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/359485969 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243a7be0 0x5620243af790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f069effd700 0 client.25492 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.112+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 >> 172.21.15.5:0/359485969 conn(0x56202439dda0 msgr2=0x5620243a3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.113+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 shutdown_connections 2023-07-21T19:08:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.113+0000 7f06d9b94b80 1 -- 172.21.15.5:0/359485969 wait complete. 2023-07-21T19:08:30.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.113+0000 7f06d9b94b80 20 client.25492 trim_cache size 0 max 16384 2023-07-21T19:08:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.138+0000 7f06d9b94b80 1 Processor -- start 2023-07-21T19:08:30.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.138+0000 7f06d9b94b80 1 -- start start 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.138+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243be6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.138+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.138+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x5620243aef50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620243b0ae0 con 0x5620243a7be0 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620243ab2e0 con 0x5620243b2ad0 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620243a78f0 con 0x5620243a72d0 2023-07-21T19:08:30.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60540/0 (socket says 172.21.15.5:60540) 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 -- 172.21.15.5:0/3977921866 learned_addr learned my addr 172.21.15.5:0/3977921866 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243be6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06bffff700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x5620243aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 -- 172.21.15.5:0/3977921866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 msgr2=0x5620243be6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243be6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 -- 172.21.15.5:0/3977921866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 msgr2=0x5620243aef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x5620243aef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06beffd700 1 -- 172.21.15.5:0/3977921866 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562024110c60 con 0x5620243a7be0 2023-07-21T19:08:30.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.139+0000 7f06bffff700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x5620243aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.140+0000 7f06beffd700 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005d10 tx=0x7f06b400bf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55eeda1610a5ab1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.140+0000 7f06a67fc700 1 -- 172.21.15.5:0/3977921866 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b402b040 con 0x5620243a7be0 2023-07-21T19:08:30.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.140+0000 7f06a67fc700 1 -- 172.21.15.5:0/3977921866 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b400fea0 con 0x5620243a7be0 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.140+0000 7f06a67fc700 1 -- 172.21.15.5:0/3977921866 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b4004df0 con 0x5620243a7be0 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 -- 172.21.15.5:0/3977921866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 msgr2=0x5620243b20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f06b4005d10 tx=0x7f06b400bf20 comp rx=0 tx=0).stop 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 -- 172.21.15.5:0/3977921866 shutdown_connections 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x5620243aef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243be6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/3977921866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b20b0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 -- 172.21.15.5:0/3977921866 >> 172.21.15.5:0/3977921866 conn(0x56202439dda0 msgr2=0x5620243c91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 -- 172.21.15.5:0/3977921866 shutdown_connections 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.141+0000 7f06d9b94b80 1 -- 172.21.15.5:0/3977921866 wait complete. 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.142+0000 7f06d9b94b80 1 -- start start 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.142+0000 7f06d9b94b80 1 Processor -- start 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243b72b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x562024415330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562024415870 con 0x5620243a7be0 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5620244159b0 con 0x5620243b2ad0 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562024415af0 con 0x5620243a72d0 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06d9b94b80 10 client.0 _ll_register_callbacks cb 0x7f06b800be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:30.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06beffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243b72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06bf7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243b72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53002/0 (socket says 172.21.15.5:53002) 2023-07-21T19:08:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06bf7fe700 1 -- 172.21.15.5:0/436318217 learned_addr learned my addr 172.21.15.5:0/436318217 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.143+0000 7f06bffff700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x562024415330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 -- 172.21.15.5:0/436318217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 msgr2=0x5620243b72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243b72b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 -- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 msgr2=0x562024415330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x562024415330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 -- 172.21.15.5:0/436318217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562024127de0 con 0x5620243a7be0 2023-07-21T19:08:30.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06beffd700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b77f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56202439ff30 tx=0x7f06b4024f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b3ac81cb39d153 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f069effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b402b030 con 0x5620243a7be0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06b401e040 con 0x5620243a7be0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f06b4038890 con 0x5620243a7be0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b4038a50 con 0x5620243a7be0 2023-07-21T19:08:30.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.144+0000 7f06d9b94b80 10 client.15777 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x56202414e250 con 0x5620243a7be0 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06b401e040 con 0x5620243a7be0 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f06b400f740 con 0x5620243a7be0 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 10 client.15777 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 20 client.15777 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562024416e20 0x562024419230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 4 client.15777 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5620242b5050 con 0x562024416e20 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.145+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562024416e20 0x562024419230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06bf7fe700 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562024416e20 0x562024419230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58a7a474e99eca5b server_cookie=dedbc65562088a4d in_seq=0 out_seq=0 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f069effd700 10 client.15777 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f069effd700 1 -- 172.21.15.5:0/436318217 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5620242b5050 con 0x562024416e20 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f069effd700 10 client.15777 handle_command_reply: tid=0 2023-07-21T19:08:30.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 client.15777 shutdown 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 msgr2=0x5620243b77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b77f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x56202439ff30 tx=0x7f06b4024f30 comp rx=0 tx=0).stop 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 shutdown_connections 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5620243b2ad0 0x562024415330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5620243a72d0 0x5620243b72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562024416e20 0x562024419230 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 --2- 172.21.15.5:0/436318217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5620243a7be0 0x5620243b77f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 >> 172.21.15.5:0/436318217 conn(0x56202439dda0 msgr2=0x5620243aaa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.146+0000 7f069effd700 0 client.15777 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 shutdown_connections 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06d9b94b80 1 -- 172.21.15.5:0/436318217 wait complete. 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06d9b94b80 20 client.15777 trim_cache size 0 max 16384 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:08:30.164 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:08:30.165 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:08:30.165 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:08:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 msgr2=0x7f06a80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 0x7f06a80a3890 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06c0194c00 tx=0x7f06b400b260 comp rx=0 tx=0).stop 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 msgr2=0x7f06c0199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.147+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f06b000e9a0 tx=0x7f06b000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 shutdown_connections 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06c010a040 0x7f06c0193790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06c0100d00 0x7f06c0199720 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06a80a13e0 0x7f06a80a3890 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 --2- 172.21.15.5:0/3661004621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06c0101610 0x7f06c0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 >> 172.21.15.5:0/3661004621 conn(0x7f06c00fa4b0 msgr2=0x7f06c010c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 shutdown_connections 2023-07-21T19:08:30.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.148+0000 7f06c61ff700 1 -- 172.21.15.5:0/3661004621 wait complete. 2023-07-21T19:08:30.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:08:30.177 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:08:30.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 -- 172.21.15.5:0/1772999120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a00072830 msgr2=0x7f8a00072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 --2- 172.21.15.5:0/1772999120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a00072830 0x7f8a00072c00 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009990 tx=0x7f89fc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 -- 172.21.15.5:0/1772999120 shutdown_connections 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 --2- 172.21.15.5:0/1772999120 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a0010c3b0 0x7f8a0010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 --2- 172.21.15.5:0/1772999120 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a00072830 0x7f8a00072c00 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 --2- 172.21.15.5:0/1772999120 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a00073140 0x7f8a0010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 -- 172.21.15.5:0/1772999120 >> 172.21.15.5:0/1772999120 conn(0x7f8a000fb630 msgr2=0x7f8a000fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 -- 172.21.15.5:0/1772999120 shutdown_connections 2023-07-21T19:08:30.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.440+0000 7f8a085ff700 1 -- 172.21.15.5:0/1772999120 wait complete. 2023-07-21T19:08:30.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.442+0000 7f8a085ff700 1 Processor -- start 2023-07-21T19:08:30.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.442+0000 7f8a085ff700 1 -- start start 2023-07-21T19:08:30.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a00072830 0x7f8a00197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a00073140 0x7f8a00198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019cd30 con 0x7f8a0010c3b0 2023-07-21T19:08:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019cea0 con 0x7f8a00073140 2023-07-21T19:08:30.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.443+0000 7f8a085ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0019d060 con 0x7f8a00072830 2023-07-21T19:08:30.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60574/0 (socket says 172.21.15.5:60574) 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 -- 172.21.15.5:0/3737987072 learned_addr learned my addr 172.21.15.5:0/3737987072 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a0639b700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a00072830 0x7f8a00197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 -- 172.21.15.5:0/3737987072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a00072830 msgr2=0x7f8a00197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a00072830 0x7f8a00197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 -- 172.21.15.5:0/3737987072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a00073140 msgr2=0x7f8a00198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a00073140 0x7f8a00198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.444+0000 7f8a06b9c700 1 -- 172.21.15.5:0/3737987072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc0095f0 con 0x7f8a0010c3b0 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f8a06b9c700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f89f8007ee0 tx=0x7f89f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8017070 con 0x7f8a0010c3b0 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89f800f040 con 0x7f8a0010c3b0 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89f8005710 con 0x7f8a0010c3b0 2023-07-21T19:08:30.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0019d340 con 0x7f8a0010c3b0 2023-07-21T19:08:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.445+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a001a4b80 con 0x7f8a0010c3b0 2023-07-21T19:08:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.447+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a00067300 con 0x7f8a0010c3b0 2023-07-21T19:08:30.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.449+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f89f8005870 con 0x7f8a0010c3b0 2023-07-21T19:08:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.451+0000 7f89f77fe700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 0x7f89ec0a3960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.452+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f89f80cc790 con 0x7f8a0010c3b0 2023-07-21T19:08:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.452+0000 7f8a0639b700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 0x7f89ec0a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:30.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.452+0000 7f8a0639b700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 0x7f89ec0a3960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b5c0 tx=0x7f89fc005fb0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:30.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.456+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f89f8095920 con 0x7f8a0010c3b0 2023-07-21T19:08:30.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.777+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a001990a0 con 0x7f8a0010c3b0 2023-07-21T19:08:30.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.778+0000 7f89f77fe700 1 -- 172.21.15.5:0/3737987072 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f89f8098f40 con 0x7f8a0010c3b0 2023-07-21T19:08:30.789 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:08:30.789 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":25,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:08:16.170576+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:08:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 msgr2=0x7f89ec0a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 0x7f89ec0a3960 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f89fc00b5c0 tx=0x7f89fc005fb0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 msgr2=0x7f8a0019c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f89f8007ee0 tx=0x7f89f800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 shutdown_connections 2023-07-21T19:08:30.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89ec0a14b0 0x7f89ec0a3960 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8a0010c3b0 0x7f8a0019c6f0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8a00072830 0x7f8a00197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 --2- 172.21.15.5:0/3737987072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8a00073140 0x7f8a00198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 >> 172.21.15.5:0/3737987072 conn(0x7f8a000fb630 msgr2=0x7f8a0010e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 shutdown_connections 2023-07-21T19:08:30.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:30.781+0000 7f8a085ff700 1 -- 172.21.15.5:0/3737987072 wait complete. 2023-07-21T19:08:30.794 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 25 2023-07-21T19:08:30.812 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:08:31.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 -- 172.21.15.5:0/349751419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 msgr2=0x7fa8e0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/349751419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0105eb0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009990 tx=0x7fa8d0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 -- 172.21.15.5:0/349751419 shutdown_connections 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/349751419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e0100480 0x7fa8e010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/349751419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0105eb0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/349751419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e00ffad0 0x7fa8e00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 -- 172.21.15.5:0/349751419 >> 172.21.15.5:0/349751419 conn(0x7fa8e00fb5f0 msgr2=0x7fa8e00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 -- 172.21.15.5:0/349751419 shutdown_connections 2023-07-21T19:08:31.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.033+0000 7fa8e61ff700 1 -- 172.21.15.5:0/349751419 wait complete. 2023-07-21T19:08:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.034+0000 7fa8e61ff700 1 Processor -- start 2023-07-21T19:08:31.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.034+0000 7fa8e61ff700 1 -- start start 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e00ffad0 0x7fa8e019b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e0100480 0x7fa8e019bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0195f90 con 0x7fa8e0105ae0 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e0196100 con 0x7fa8e00ffad0 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8e61ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e01962c0 con 0x7fa8e0100480 2023-07-21T19:08:31.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60592/0 (socket says 172.21.15.5:60592) 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8deffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e0100480 0x7fa8e019bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 -- 172.21.15.5:0/763735365 learned_addr learned my addr 172.21.15.5:0/763735365 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e00ffad0 0x7fa8e019b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 -- 172.21.15.5:0/763735365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e0100480 msgr2=0x7fa8e019bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e0100480 0x7fa8e019bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 -- 172.21.15.5:0/763735365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e00ffad0 msgr2=0x7fa8e019b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e00ffad0 0x7fa8e019b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.035+0000 7fa8dffff700 1 -- 172.21.15.5:0/763735365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d00095f0 con 0x7fa8e0105ae0 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8dffff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007ee0 tx=0x7fa8d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8dcff9700 1 -- 172.21.15.5:0/763735365 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8017070 con 0x7fa8e0105ae0 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8dcff9700 1 -- 172.21.15.5:0/763735365 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8d800f040 con 0x7fa8e0105ae0 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e01965a0 con 0x7fa8e0105ae0 2023-07-21T19:08:31.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8dcff9700 1 -- 172.21.15.5:0/763735365 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8005570 con 0x7fa8e0105ae0 2023-07-21T19:08:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.036+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e01a28a0 con 0x7fa8e0105ae0 2023-07-21T19:08:31.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.040+0000 7fa8dcff9700 1 -- 172.21.15.5:0/763735365 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7fa8d800db20 con 0x7fa8e0105ae0 2023-07-21T19:08:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.041+0000 7fa8dcff9700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 0x7fa8c80a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8f9ad9b80 1 Processor -- start 2023-07-21T19:08:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8dcff9700 1 -- 172.21.15.5:0/763735365 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa8d80cc680 con 0x7fa8e0105ae0 2023-07-21T19:08:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 0x7fa8c80a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8f9ad9b80 1 -- start start 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d66e0 0x55a2db9d6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.042+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9e42c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db8d8cb0 con 0x55a2db9e1ee0 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db92ba40 con 0x55a2db9d6ff0 2023-07-21T19:08:31.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db924ed0 con 0x55a2db9d66e0 2023-07-21T19:08:31.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 0x7fa8c80a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000b5c0 tx=0x7fa8d0005fb0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50324/0 (socket says 172.21.15.5:50324) 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 -- 172.21.15.5:0/1543355148 learned_addr learned my addr 172.21.15.5:0/1543355148 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8dffff700 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9e42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d66e0 0x55a2db9d6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 -- 172.21.15.5:0/1543355148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d66e0 msgr2=0x55a2db9d6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d66e0 0x55a2db9d6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 -- 172.21.15.5:0/1543355148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 msgr2=0x55a2db9e42c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9e42c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.043+0000 7fa8deffd700 1 -- 172.21.15.5:0/1543355148 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2db752de0 con 0x55a2db9d6ff0 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.044+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400bd30 tx=0x7fa8d40106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee710c1a45e2943b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.044+0000 7fa8c5ffb700 1 -- 172.21.15.5:0/1543355148 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4010e10 con 0x55a2db9d6ff0 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.044+0000 7fa8c5ffb700 1 -- 172.21.15.5:0/1543355148 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8d4015070 con 0x55a2db9d6ff0 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.044+0000 7fa8c5ffb700 1 -- 172.21.15.5:0/1543355148 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d4003de0 con 0x55a2db9d6ff0 2023-07-21T19:08:31.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1543355148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 msgr2=0x55a2db9e19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8d400bd30 tx=0x7fa8d40106b0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1543355148 shutdown_connections 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9e42c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d66e0 0x55a2db9d6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1543355148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e19a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1543355148 >> 172.21.15.5:0/1543355148 conn(0x55a2db9c8da0 msgr2=0x55a2db9ce6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1543355148 shutdown_connections 2023-07-21T19:08:31.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1543355148 wait complete. 2023-07-21T19:08:31.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.045+0000 7fa8f9ad9b80 1 -- start start 2023-07-21T19:08:31.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 Processor -- start 2023-07-21T19:08:31.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9dec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 0x55a2db9df340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db8c4ac0 con 0x55a2db9d66e0 2023-07-21T19:08:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db8c47c0 con 0x55a2db9d6ff0 2023-07-21T19:08:31.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db7a8920 con 0x55a2db9e1ee0 2023-07-21T19:08:31.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8f9ad9b80 10 client.0 _ll_register_callbacks cb 0x55a2db8c2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.046+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60606/0 (socket says 172.21.15.5:60606) 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 -- 172.21.15.5:0/761696453 learned_addr learned my addr 172.21.15.5:0/761696453 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8deffd700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9dec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 -- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 msgr2=0x55a2db9df340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8dffff700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 0x55a2db9df340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 0x55a2db9df340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 -- 172.21.15.5:0/761696453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 msgr2=0x55a2db9dec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9dec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 -- 172.21.15.5:0/761696453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8d400b960 con 0x55a2db9d66e0 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8deffd700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:31.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8dffff700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 0x55a2db9df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000ba50 tx=0x7fa8d0009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6cf5aab135912d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.047+0000 7fa8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0004d20 con 0x55a2db9d66e0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2db87e4f0 con 0x55a2db9d66e0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa8d0003820 con 0x55a2db9d66e0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d000bec0 con 0x55a2db9d66e0 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8f9ad9b80 10 client.15795 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2db73bc60 con 0x55a2db9d66e0 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a2db87e4f0 con 0x55a2db9d66e0 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.048+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8d000f580 con 0x55a2db9d66e0 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 10 client.15795 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 10 client.15795 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 20 client.15795 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2db9f18b0 0x55a2db9f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 4 client.15795 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2db91deb0 con 0x55a2db9f18b0 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.049+0000 7fa8deffd700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2db9f18b0 0x55a2db9f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.050+0000 7fa8deffd700 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2db9f18b0 0x55a2db9f3cc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40a7e16598df60c2 server_cookie=348d8b969f591635 in_seq=0 out_seq=0 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.050+0000 7fa8beffd700 10 client.15795 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8beffd700 1 -- 172.21.15.5:0/761696453 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a2db91deb0 con 0x55a2db9f18b0 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8beffd700 10 client.15795 handle_command_reply: tid=0 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 client.15795 shutdown 2023-07-21T19:08:31.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 msgr2=0x55a2db9e1910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000ba50 tx=0x7fa8d0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 shutdown_connections 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9e1ee0 0x55a2db9df340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e1910 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2db9f18b0 0x55a2db9f3cc0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/761696453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d6ff0 0x55a2db9dec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8beffd700 0 client.15795 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:08:31.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 >> 172.21.15.5:0/761696453 conn(0x55a2db9c8da0 msgr2=0x55a2db9ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 shutdown_connections 2023-07-21T19:08:31.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.051+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/761696453 wait complete. 2023-07-21T19:08:31.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.052+0000 7fa8f9ad9b80 20 client.15795 trim_cache size 0 max 16384 2023-07-21T19:08:31.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 Processor -- start 2023-07-21T19:08:31.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 -- start start 2023-07-21T19:08:31.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9d8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9d9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db9cfd20 con 0x55a2db9e1ee0 2023-07-21T19:08:31.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db9d7950 con 0x55a2db9d66e0 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.067+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db9e0170 con 0x55a2db9d6ff0 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9d9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50350/0 (socket says 172.21.15.5:50350) 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 -- 172.21.15.5:0/2111902180 learned_addr learned my addr 172.21.15.5:0/2111902180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8deffd700 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9d8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 -- 172.21.15.5:0/2111902180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 msgr2=0x55a2db9d8cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9d8cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 -- 172.21.15.5:0/2111902180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 msgr2=0x55a2db9d9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9d9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 -- 172.21.15.5:0/2111902180 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2db779250 con 0x55a2db9d66e0 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.068+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00036c0 tx=0x7fa8d00050f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2533a05e2c33ff7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8c67fc700 1 -- 172.21.15.5:0/2111902180 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d000fbd0 con 0x55a2db9d66e0 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8c67fc700 1 -- 172.21.15.5:0/2111902180 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8d001e040 con 0x55a2db9d66e0 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8c67fc700 1 -- 172.21.15.5:0/2111902180 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d0003c60 con 0x55a2db9d66e0 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/2111902180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 msgr2=0x55a2db9e88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8d00036c0 tx=0x7fa8d00050f0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/2111902180 shutdown_connections 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9e1ee0 0x55a2db9d9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9d66e0 0x55a2db9e88c0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/2111902180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9d8cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.069+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/2111902180 >> 172.21.15.5:0/2111902180 conn(0x55a2db9c8da0 msgr2=0x55a2db9d85b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.070+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/2111902180 shutdown_connections 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.070+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/2111902180 wait complete. 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.070+0000 7fa8f9ad9b80 1 -- start start 2023-07-21T19:08:31.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.070+0000 7fa8f9ad9b80 1 Processor -- start 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 0x55a2dba44770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2db9e7620 con 0x55a2db9d66e0 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dba44cb0 con 0x55a2db9e1ee0 2023-07-21T19:08:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2dba44df0 con 0x55a2db9d6ff0 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8f9ad9b80 10 client.0 _ll_register_callbacks cb 0x7fa8d0005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60630/0 (socket says 172.21.15.5:60630) 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 -- 172.21.15.5:0/1304037619 learned_addr learned my addr 172.21.15.5:0/1304037619 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8dffff700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 0x55a2dba44770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 -- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 msgr2=0x55a2db9e6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 -- 172.21.15.5:0/1304037619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 msgr2=0x55a2dba44770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 0x55a2dba44770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8df7fe700 1 -- 172.21.15.5:0/1304037619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2db73bc60 con 0x55a2db9d66e0 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.071+0000 7fa8dffff700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 0x55a2dba44770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8df7fe700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009ce0 tx=0x7fa8d000bf80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec22eae62057d34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8beffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:31.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d00038c0 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa8d4015070 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa8d0003a20 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa8d0003be0 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8f9ad9b80 10 client.15801 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.072+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55a2db7512b0 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa8d4015070 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa8d0039960 con 0x55a2db9d66e0 2023-07-21T19:08:31.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 10 client.15801 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 10 client.15801 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 20 client.15801 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2dba45fa0 0x55a2dba48380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 4 client.15801 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.073+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a2db8e0050 con 0x55a2dba45fa0 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2dba45fa0 0x55a2dba48380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8deffd700 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2dba45fa0 0x55a2dba48380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a47c209890922b7e server_cookie=b0a94b19b2608445 in_seq=0 out_seq=0 2023-07-21T19:08:31.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8beffd700 10 client.15801 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8beffd700 1 -- 172.21.15.5:0/1304037619 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a2db8e0050 con 0x55a2dba45fa0 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8beffd700 10 client.15801 handle_command_reply: tid=0 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.074+0000 7fa8f9ad9b80 1 client.15801 shutdown 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 msgr2=0x55a2db9e6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fa8d0009ce0 tx=0x7fa8d000bf80 comp rx=0 tx=0).stop 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 shutdown_connections 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a2dba45fa0 0x55a2dba48380 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a2db9e1ee0 0x55a2dba44770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a2db9d66e0 0x55a2db9e6a90 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 --2- 172.21.15.5:0/1304037619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a2db9d6ff0 0x55a2db9e6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8beffd700 0 client.15801 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 >> 172.21.15.5:0/1304037619 conn(0x55a2db9c8da0 msgr2=0x55a2db9d8e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 shutdown_connections 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 1 -- 172.21.15.5:0/1304037619 wait complete. 2023-07-21T19:08:31.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.075+0000 7fa8f9ad9b80 20 client.15801 trim_cache size 0 max 16384 2023-07-21T19:08:31.093 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:08:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 msgr2=0x7fa8c80a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 0x7fa8c80a3910 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fa8d000b5c0 tx=0x7fa8d0005fb0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 msgr2=0x7fa8e0195920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8007ee0 tx=0x7fa8d800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 shutdown_connections 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa8c80a1460 0x7fa8c80a3910 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa8e0105ae0 0x7fa8e0195920 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa8e00ffad0 0x7fa8e019b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 --2- 172.21.15.5:0/763735365 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa8e0100480 0x7fa8e019bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 >> 172.21.15.5:0/763735365 conn(0x7fa8e00fb5f0 msgr2=0x7fa8e01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 shutdown_connections 2023-07-21T19:08:31.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.076+0000 7fa8e61ff700 1 -- 172.21.15.5:0/763735365 wait complete. 2023-07-21T19:08:31.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:08:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/3082508898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01013b0 msgr2=0x7f5fd0109a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/3082508898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01013b0 0x7f5fd0109a80 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5fc0009990 tx=0x7f5fc0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/3082508898 shutdown_connections 2023-07-21T19:08:31.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/3082508898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd0109fc0 0x7f5fd010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/3082508898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 0x7f5fd0100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/3082508898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd01013b0 0x7f5fd0109a80 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.337+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/3082508898 >> 172.21.15.5:0/3082508898 conn(0x7f5fd00745d0 msgr2=0x7f5fd00749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.338+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/3082508898 shutdown_connections 2023-07-21T19:08:31.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.338+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/3082508898 wait complete. 2023-07-21T19:08:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 Processor -- start 2023-07-21T19:08:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 -- start start 2023-07-21T19:08:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 0x7f5fd00fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd01013b0 0x7f5fd00fee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd00ff640 con 0x7f5fd0109fc0 2023-07-21T19:08:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a3ff0 con 0x7f5fd01013b0 2023-07-21T19:08:31.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.339+0000 7f5fd6f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fd01a4130 con 0x7f5fd0100aa0 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60650/0 (socket says 172.21.15.5:60650) 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 -- 172.21.15.5:0/1520991767 learned_addr learned my addr 172.21.15.5:0/1520991767 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 0x7f5fd00fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 -- 172.21.15.5:0/1520991767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 msgr2=0x7f5fd00fe8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 0x7f5fd00fe8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 -- 172.21.15.5:0/1520991767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd01013b0 msgr2=0x7f5fd00fee10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:08:31.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd01013b0 0x7f5fd00fee10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.340+0000 7f5fd679c700 1 -- 172.21.15.5:0/1520991767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc00095f0 con 0x7f5fd0109fc0 2023-07-21T19:08:31.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.341+0000 7f5fd679c700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8007f10 tx=0x7f5fc800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.343+0000 7f5fc6ffd700 1 -- 172.21.15.5:0/1520991767 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8017070 con 0x7f5fd0109fc0 2023-07-21T19:08:31.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.343+0000 7f5fc6ffd700 1 -- 172.21.15.5:0/1520991767 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fc800f040 con 0x7f5fd0109fc0 2023-07-21T19:08:31.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.343+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fd0195690 con 0x7f5fd0109fc0 2023-07-21T19:08:31.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.343+0000 7f5fc6ffd700 1 -- 172.21.15.5:0/1520991767 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc8004020 con 0x7f5fd0109fc0 2023-07-21T19:08:31.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.343+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fd0195b70 con 0x7f5fd0109fc0 2023-07-21T19:08:31.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.347+0000 7f5fc6ffd700 1 -- 172.21.15.5:0/1520991767 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f5fc8003a00 con 0x7f5fd0109fc0 2023-07-21T19:08:31.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.349+0000 7f5febb80b80 1 Processor -- start 2023-07-21T19:08:31.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.349+0000 7f5febb80b80 1 -- start start 2023-07-21T19:08:31.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.349+0000 7f5fc6ffd700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 0x7f5fbc0a3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fc6ffd700 1 -- 172.21.15.5:0/1520991767 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5fc80cc560 con 0x7f5fd0109fc0 2023-07-21T19:08:31.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd579a700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 0x7f5fbc0a3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d662897f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d6628c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d66184cb0 con 0x563d66289d30 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d661d7a40 con 0x563d6627f4f0 2023-07-21T19:08:31.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5febb80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d661d0ed0 con 0x563d6627fe00 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50386/0 (socket says 172.21.15.5:50386) 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/3925203472 learned_addr learned my addr 172.21.15.5:0/3925203472 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd579a700 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d662897f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.350+0000 7f5fd679c700 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d6628c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/3925203472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 msgr2=0x563d662897f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d662897f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/3925203472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 msgr2=0x563d6628c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd579a700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 0x7f5fbc0a3930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00098e0 tx=0x7f5fc0003680 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d6628c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/3925203472 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d65ffede0 con 0x563d6627f4f0 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b870 tx=0x7f5fcc00b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce2ddbd710c3f520 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fbbfff700 1 -- 172.21.15.5:0/3925203472 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01a040 con 0x563d6627f4f0 2023-07-21T19:08:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fbbfff700 1 -- 172.21.15.5:0/3925203472 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fcc014ce0 con 0x563d6627f4f0 2023-07-21T19:08:31.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.351+0000 7f5fbbfff700 1 -- 172.21.15.5:0/3925203472 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc0097a0 con 0x563d6627f4f0 2023-07-21T19:08:31.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 -- 172.21.15.5:0/3925203472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 msgr2=0x563d6627f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc00b870 tx=0x7f5fcc00b410 comp rx=0 tx=0).stop 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 -- 172.21.15.5:0/3925203472 shutdown_connections 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d6628c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6627f8c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 --2- 172.21.15.5:0/3925203472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d662897f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.352+0000 7f5febb80b80 1 -- 172.21.15.5:0/3925203472 >> 172.21.15.5:0/3925203472 conn(0x563d66274ff0 msgr2=0x563d6627a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.353+0000 7f5febb80b80 1 -- 172.21.15.5:0/3925203472 shutdown_connections 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.353+0000 7f5febb80b80 1 -- 172.21.15.5:0/3925203472 wait complete. 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.353+0000 7f5febb80b80 1 -- start start 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.353+0000 7f5febb80b80 1 Processor -- start 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627fe00 0x563d66286960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66286f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d66170ac0 con 0x563d66289d30 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d661707c0 con 0x563d6627fe00 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d66054920 con 0x563d6627f4f0 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5febb80b80 10 client.0 _ll_register_callbacks cb 0x563d6616e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53110/0 (socket says 172.21.15.5:53110) 2023-07-21T19:08:31.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/1734185933 learned_addr learned my addr 172.21.15.5:0/1734185933 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5fd679c700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66286f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.354+0000 7f5fd579a700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627fe00 0x563d66286960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627fe00 msgr2=0x563d66286960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627fe00 0x563d66286960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/1734185933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 msgr2=0x563d66286f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66286f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/1734185933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d65fe7c60 con 0x563d6627f4f0 2023-07-21T19:08:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd679c700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66286f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc0154b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c55a2e80b1e08d91 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:31.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01a040 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5fcc005240 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.355+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d6612a4f0 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fcc015950 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 10 client.25531 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d66025250 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d6612a4f0 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5fcc004750 con 0x563d6627f4f0 2023-07-21T19:08:31.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 10 client.25531 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 10 client.25531 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 20 client.25531 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d66299690 0x563d6629baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 4 client.25531 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.356+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d661c9eb0 con 0x563d66299690 2023-07-21T19:08:31.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.357+0000 7f5fd579a700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d66299690 0x563d6629baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.358+0000 7f5fb97fa700 10 client.25531 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:08:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.358+0000 7f5fd579a700 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d66299690 0x563d6629baa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=674655a1d828175c server_cookie=2a18d17ede613a4 in_seq=0 out_seq=0 2023-07-21T19:08:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.358+0000 7f5fb97fa700 1 -- 172.21.15.5:0/1734185933 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563d661c9eb0 con 0x563d66299690 2023-07-21T19:08:31.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.358+0000 7f5fb97fa700 10 client.25531 handle_command_reply: tid=0 2023-07-21T19:08:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.358+0000 7f5febb80b80 1 client.25531 shutdown 2023-07-21T19:08:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 msgr2=0x563d662896e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc000c00 tx=0x7f5fcc0154b0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 shutdown_connections 2023-07-21T19:08:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66286f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627f4f0 0x563d662896e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d66299690 0x563d6629baa0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 --2- 172.21.15.5:0/1734185933 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627fe00 0x563d66286960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5fb97fa700 0 client.25531 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:08:31.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 >> 172.21.15.5:0/1734185933 conn(0x563d66274ff0 msgr2=0x563d6627a520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 shutdown_connections 2023-07-21T19:08:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 1 -- 172.21.15.5:0/1734185933 wait complete. 2023-07-21T19:08:31.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.359+0000 7f5febb80b80 20 client.25531 trim_cache size 0 max 16384 2023-07-21T19:08:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.379+0000 7f5febb80b80 1 Processor -- start 2023-07-21T19:08:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 -- start start 2023-07-21T19:08:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d66289280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66281c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d66280980 con 0x563d66289d30 2023-07-21T19:08:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d662817e0 con 0x563d6627f4f0 2023-07-21T19:08:31.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5febb80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6627ae70 con 0x563d6627fe00 2023-07-21T19:08:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5fd679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66281c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50402/0 (socket says 172.21.15.5:50402) 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.380+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2398583268 learned_addr learned my addr 172.21.15.5:0/2398583268 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd579a700 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d66289280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2398583268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 msgr2=0x563d66289280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d66289280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2398583268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 msgr2=0x563d66281c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66281c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2398583268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d65ffd2b0 con 0x563d6627f4f0 2023-07-21T19:08:31.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.381+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004b00 tx=0x7f5fcc015c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b78213f19bd121bb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.382+0000 7f5fc4ff9700 1 -- 172.21.15.5:0/2398583268 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01a070 con 0x563d6627f4f0 2023-07-21T19:08:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.382+0000 7f5fc4ff9700 1 -- 172.21.15.5:0/2398583268 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fcc0048c0 con 0x563d6627f4f0 2023-07-21T19:08:31.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.382+0000 7f5fc4ff9700 1 -- 172.21.15.5:0/2398583268 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc02a640 con 0x563d6627f4f0 2023-07-21T19:08:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.383+0000 7f5febb80b80 1 -- 172.21.15.5:0/2398583268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 msgr2=0x563d6628c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.383+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004b00 tx=0x7f5fcc015c50 comp rx=0 tx=0).stop 2023-07-21T19:08:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 -- 172.21.15.5:0/2398583268 shutdown_connections 2023-07-21T19:08:31.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d66289d30 0x563d66281c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d6628c110 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2398583268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6627fe00 0x563d66289280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 -- 172.21.15.5:0/2398583268 >> 172.21.15.5:0/2398583268 conn(0x563d66274ff0 msgr2=0x563d66279060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 -- 172.21.15.5:0/2398583268 shutdown_connections 2023-07-21T19:08:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 -- 172.21.15.5:0/2398583268 wait complete. 2023-07-21T19:08:31.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.384+0000 7f5febb80b80 1 -- start start 2023-07-21T19:08:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.385+0000 7f5febb80b80 1 Processor -- start 2023-07-21T19:08:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.385+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 0x563d662e8a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d66289d30 0x563d662e8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6628e7c0 con 0x563d6627fe00 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6628e900 con 0x563d6627f4f0 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d6628eac0 con 0x563d66289d30 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5febb80b80 10 client.0 _ll_register_callbacks cb 0x7f5fc8009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50412/0 (socket says 172.21.15.5:50412) 2023-07-21T19:08:31.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2855891774 learned_addr learned my addr 172.21.15.5:0/2855891774 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd579a700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 0x563d662e8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd679c700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d66289d30 0x563d662e8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2855891774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d66289d30 msgr2=0x563d662e8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d66289d30 0x563d662e8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2855891774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 msgr2=0x563d662e8a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.386+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 0x563d662e8a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fd5f9b700 1 -- 172.21.15.5:0/2855891774 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563d66025250 con 0x563d6627f4f0 2023-07-21T19:08:31.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fd579a700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 0x563d662e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fd5f9b700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004b80 tx=0x7f5fcc015960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c8deee862496e3e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fb97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fcc01a070 con 0x563d6627f4f0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5fcc000e40 con 0x563d6627f4f0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.387+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5fcc014ce0 con 0x563d6627f4f0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5fcc005450 con 0x563d6627f4f0 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 10 client.25562 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x563d660cba80 con 0x563d6627f4f0 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5fcc04a080 con 0x563d6627f4f0 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5fcc03c480 con 0x563d6627f4f0 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 10 client.25562 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 10 client.25562 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 20 client.25562 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d662fe990 0x563d66300da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 4 client.25562 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:08:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.388+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563d6618c050 con 0x563d662fe990 2023-07-21T19:08:31.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5fd579a700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d662fe990 0x563d66300da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5fd579a700 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d662fe990 0x563d66300da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9c557aa86e7c1a1c server_cookie=23e1a9ae590fbe66 in_seq=0 out_seq=0 2023-07-21T19:08:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5fb97fa700 10 client.25562 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:08:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5fb97fa700 1 -- 172.21.15.5:0/2855891774 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563d6618c050 con 0x563d662fe990 2023-07-21T19:08:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5fb97fa700 10 client.25562 handle_command_reply: tid=0 2023-07-21T19:08:31.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.389+0000 7f5febb80b80 1 client.25562 shutdown 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 msgr2=0x563d662e84f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc004b80 tx=0x7f5fcc015960 comp rx=0 tx=0).stop 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 shutdown_connections 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d66289d30 0x563d662e8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563d662fe990 0x563d66300da0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d6627f4f0 0x563d662e84f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 --2- 172.21.15.5:0/2855891774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6627fe00 0x563d662e8a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5fb97fa700 0 client.25562 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:08:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 >> 172.21.15.5:0/2855891774 conn(0x563d66274ff0 msgr2=0x563d66281c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 shutdown_connections 2023-07-21T19:08:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 1 -- 172.21.15.5:0/2855891774 wait complete. 2023-07-21T19:08:31.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.390+0000 7f5febb80b80 20 client.25562 trim_cache size 0 max 16384 2023-07-21T19:08:31.408 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:08:31.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 msgr2=0x7f5fbc0a3930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 0x7f5fbc0a3930 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00098e0 tx=0x7f5fc0003680 comp rx=0 tx=0).stop 2023-07-21T19:08:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 msgr2=0x7f5fd01a3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f5fc8007f10 tx=0x7f5fc800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 shutdown_connections 2023-07-21T19:08:31.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5fbc0a1480 0x7f5fbc0a3930 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5fd0109fc0 0x7f5fd01a3ab0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5fd0100aa0 0x7f5fd00fe8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 --2- 172.21.15.5:0/1520991767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5fd01013b0 0x7f5fd00fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 >> 172.21.15.5:0/1520991767 conn(0x7f5fd00745d0 msgr2=0x7f5fd010a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 shutdown_connections 2023-07-21T19:08:31.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.391+0000 7f5fd6f9d700 1 -- 172.21.15.5:0/1520991767 wait complete. 2023-07-21T19:08:31.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:31 smithi005 ceph-mon[126460]: pgmap v30: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 27 op/s 2023-07-21T19:08:31.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:31 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3737987072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:08:31.422 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:08:31.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 -- 172.21.15.5:0/2510864698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13680ffb00 msgr2=0x7f13680fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 --2- 172.21.15.5:0/2510864698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13680ffb00 0x7f13680fff70 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f135c009990 tx=0x7f135c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 -- 172.21.15.5:0/2510864698 shutdown_connections 2023-07-21T19:08:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 --2- 172.21.15.5:0/2510864698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 0x7f136810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 --2- 172.21.15.5:0/2510864698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1368105b10 0x7f1368105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 --2- 172.21.15.5:0/2510864698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13680ffb00 0x7f13680fff70 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 -- 172.21.15.5:0/2510864698 >> 172.21.15.5:0/2510864698 conn(0x7f13680fb600 msgr2=0x7f13680fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 -- 172.21.15.5:0/2510864698 shutdown_connections 2023-07-21T19:08:31.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.636+0000 7f136d3ff700 1 -- 172.21.15.5:0/2510864698 wait complete. 2023-07-21T19:08:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.637+0000 7f136d3ff700 1 Processor -- start 2023-07-21T19:08:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 -- start start 2023-07-21T19:08:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13680ffb00 0x7f1368197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 0x7f13681982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819cd60 con 0x7f1368105b10 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819ced0 con 0x7f13680ffb00 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f136d3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136819d090 con 0x7f13681004b0 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f13677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f13677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60708/0 (socket says 172.21.15.5:60708) 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13680ffb00 0x7f1368197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.638+0000 7f13677fe700 1 -- 172.21.15.5:0/1833111023 learned_addr learned my addr 172.21.15.5:0/1833111023 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13667fc700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 0x7f13681982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 -- 172.21.15.5:0/1833111023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 msgr2=0x7f13681982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 0x7f13681982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 -- 172.21.15.5:0/1833111023 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13680ffb00 msgr2=0x7f1368197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13680ffb00 0x7f1368197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 -- 172.21.15.5:0/1833111023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f135c0095f0 con 0x7f1368105b10 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f13677fe700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f136000ef30 tx=0x7f136000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f134ffff700 1 -- 172.21.15.5:0/1833111023 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1360018070 con 0x7f1368105b10 2023-07-21T19:08:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f134ffff700 1 -- 172.21.15.5:0/1833111023 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f136000f040 con 0x7f1368105b10 2023-07-21T19:08:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.639+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136819d310 con 0x7f1368105b10 2023-07-21T19:08:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.640+0000 7f134ffff700 1 -- 172.21.15.5:0/1833111023 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136000cbc0 con 0x7f1368105b10 2023-07-21T19:08:31.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.640+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13681a4bb0 con 0x7f1368105b10 2023-07-21T19:08:31.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.644+0000 7f134ffff700 1 -- 172.21.15.5:0/1833111023 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f1360003a00 con 0x7f1368105b10 2023-07-21T19:08:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.645+0000 7f1380cd4b80 1 Processor -- start 2023-07-21T19:08:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.645+0000 7f1380cd4b80 1 -- start start 2023-07-21T19:08:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f134ffff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 0x7f13500a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0ddebe0 0x561df0de9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0de9ad0 0x561df0debeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0ce4cb0 con 0x561df0dde2d0 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0d37a40 con 0x561df0ddebe0 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0d30ed0 con 0x561df0de9ad0 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f134ffff700 1 -- 172.21.15.5:0/1833111023 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f13600cc7c0 con 0x7f1368105b10 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60710/0 (socket says 172.21.15.5:60710) 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.646+0000 7f1366ffd700 1 -- 172.21.15.5:0/3414329333 learned_addr learned my addr 172.21.15.5:0/3414329333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f13667fc700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 0x7f13500a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 -- 172.21.15.5:0/3414329333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0de9ad0 msgr2=0x561df0debeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0de9ad0 0x561df0debeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 -- 172.21.15.5:0/3414329333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0ddebe0 msgr2=0x561df0de9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0ddebe0 0x561df0de9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 -- 172.21.15.5:0/3414329333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df0b5ede0 con 0x561df0dde2d0 2023-07-21T19:08:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f1366ffd700 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1358009d00 tx=0x7f1358011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26b31374d2e12b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.647+0000 7f13667fc700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 0x7f13500a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1368199390 tx=0x7f135c005c60 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.648+0000 7f134d7fa700 1 -- 172.21.15.5:0/3414329333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358004230 con 0x561df0dde2d0 2023-07-21T19:08:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.648+0000 7f134d7fa700 1 -- 172.21.15.5:0/3414329333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1358004390 con 0x561df0dde2d0 2023-07-21T19:08:31.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.648+0000 7f134d7fa700 1 -- 172.21.15.5:0/3414329333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358019750 con 0x561df0dde2d0 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.648+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3414329333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 msgr2=0x561df0dde6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.648+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f1358009d00 tx=0x7f1358011720 comp rx=0 tx=0).stop 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3414329333 shutdown_connections 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0de9ad0 0x561df0debeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0dde6a0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3414329333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0ddebe0 0x561df0de9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3414329333 >> 172.21.15.5:0/3414329333 conn(0x561df0dd4da0 msgr2=0x561df0dda6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3414329333 shutdown_connections 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3414329333 wait complete. 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.649+0000 7f1380cd4b80 1 -- start start 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 Processor -- start 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0de9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0de67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0cd0ac0 con 0x561df0ddebe0 2023-07-21T19:08:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0cd07c0 con 0x561df0de9ad0 2023-07-21T19:08:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0bb4920 con 0x561df0dde2d0 2023-07-21T19:08:31.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.650+0000 7f1380cd4b80 10 client.0 _ll_register_callbacks cb 0x561df0cce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13667fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0de67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50446/0 (socket says 172.21.15.5:50446) 2023-07-21T19:08:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 -- 172.21.15.5:0/848638638 learned_addr learned my addr 172.21.15.5:0/848638638 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f1366ffd700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0de9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 -- 172.21.15.5:0/848638638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 msgr2=0x561df0de9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0de9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 -- 172.21.15.5:0/848638638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 msgr2=0x561df0de67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0de67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.651+0000 7f13677fe700 1 -- 172.21.15.5:0/848638638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1358009960 con 0x561df0de9ad0 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f13677fe700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13600994b0 tx=0x7f1360003de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4d5833c67e36034 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f134aff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136009d730 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561df0c8a4f0 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f13600a1ac0 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1360098ca0 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f1380cd4b80 10 client.25571 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.652+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561df0b47c60 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561df0c8a4f0 con 0x561df0de9ad0 2023-07-21T19:08:31.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f13600ca0a0 con 0x561df0de9ad0 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 10 client.25571 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 10 client.25571 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 20 client.25571 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0df9490 0x561df0dfb8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 4 client.25571 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561df0d29eb0 con 0x561df0df9490 2023-07-21T19:08:31.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.653+0000 7f1366ffd700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0df9490 0x561df0dfb8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.654+0000 7f1366ffd700 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0df9490 0x561df0dfb8a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdd6d47e4ed8f7fc server_cookie=f4a6109ab283eae4 in_seq=0 out_seq=0 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.654+0000 7f134aff5700 10 client.25571 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f134aff5700 1 -- 172.21.15.5:0/848638638 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561df0d29eb0 con 0x561df0df9490 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f134aff5700 10 client.25571 handle_command_reply: tid=0 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 client.25571 shutdown 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 msgr2=0x561df0de6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f13600994b0 tx=0x7f1360003de0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 shutdown_connections 2023-07-21T19:08:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de6f20 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0de9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0df9490 0x561df0dfb8a0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/848638638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0de67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 >> 172.21.15.5:0/848638638 conn(0x561df0dd4da0 msgr2=0x561df0dda290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f134aff5700 0 client.25571 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.655+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 shutdown_connections 2023-07-21T19:08:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.656+0000 7f1380cd4b80 1 -- 172.21.15.5:0/848638638 wait complete. 2023-07-21T19:08:31.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.656+0000 7f1380cd4b80 20 client.25571 trim_cache size 0 max 16384 2023-07-21T19:08:31.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:31 smithi156 ceph-mon[128316]: pgmap v30: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 27 op/s 2023-07-21T19:08:31.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:31 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3737987072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:08:31.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 Processor -- start 2023-07-21T19:08:31.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 -- start start 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0ddebe0 0x561df0de2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0dd49c0 con 0x561df0dde2d0 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0ddf540 con 0x561df0de9ad0 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0de7df0 con 0x561df0ddebe0 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.670+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60716/0 (socket says 172.21.15.5:60716) 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 -- 172.21.15.5:0/2012387498 learned_addr learned my addr 172.21.15.5:0/2012387498 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f13667fc700 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0ddebe0 0x561df0de2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f13677fe700 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 -- 172.21.15.5:0/2012387498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0ddebe0 msgr2=0x561df0de2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0ddebe0 0x561df0de2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 -- 172.21.15.5:0/2012387498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 msgr2=0x561df0de5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 -- 172.21.15.5:0/2012387498 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561df0b85250 con 0x561df0dde2d0 2023-07-21T19:08:31.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.671+0000 7f1366ffd700 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f1358004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593bf7eafb344d0f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.672+0000 7f134dffb700 1 -- 172.21.15.5:0/2012387498 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1358028b50 con 0x561df0dde2d0 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.672+0000 7f134dffb700 1 -- 172.21.15.5:0/2012387498 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1358028cb0 con 0x561df0dde2d0 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.672+0000 7f134dffb700 1 -- 172.21.15.5:0/2012387498 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13580056b0 con 0x561df0dde2d0 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.672+0000 7f1380cd4b80 1 -- 172.21.15.5:0/2012387498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 msgr2=0x561df0de1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.672+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f1358000c00 tx=0x7f1358004500 comp rx=0 tx=0).stop 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 -- 172.21.15.5:0/2012387498 shutdown_connections 2023-07-21T19:08:31.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0de5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0dde2d0 0x561df0de1ad0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/2012387498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0ddebe0 0x561df0de2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 -- 172.21.15.5:0/2012387498 >> 172.21.15.5:0/2012387498 conn(0x561df0dd4da0 msgr2=0x561df0df5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 -- 172.21.15.5:0/2012387498 shutdown_connections 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 -- 172.21.15.5:0/2012387498 wait complete. 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 -- start start 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.673+0000 7f1380cd4b80 1 Processor -- start 2023-07-21T19:08:31.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0df0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0deeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0deeff0 con 0x561df0ddebe0 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0def160 con 0x561df0de9ad0 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561df0def320 con 0x561df0dde2d0 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1380cd4b80 10 client.0 _ll_register_callbacks cb 0x7f1360099350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:31.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f13667fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0df0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1366ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0df0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53194/0 (socket says 172.21.15.5:53194) 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f1366ffd700 1 -- 172.21.15.5:0/3164246185 learned_addr learned my addr 172.21.15.5:0/3164246185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.674+0000 7f13667fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60730/0 (socket says 172.21.15.5:60730) 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13677fe700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0deeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 -- 172.21.15.5:0/3164246185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 msgr2=0x561df0df0280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0df0280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 -- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 msgr2=0x561df0deeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0deeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 -- 172.21.15.5:0/3164246185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561df0b5ede0 con 0x561df0ddebe0 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f13667fc700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098e0 tx=0x7f135c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cb328d079c84d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f134aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.675+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c004d50 con 0x561df0ddebe0 2023-07-21T19:08:31.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f135c00f460 con 0x561df0ddebe0 2023-07-21T19:08:31.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1358004390 con 0x561df0ddebe0 2023-07-21T19:08:31.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c004eb0 con 0x561df0ddebe0 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 10 client.15837 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x561df0b5d2b0 con 0x561df0ddebe0 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1358004390 con 0x561df0ddebe0 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f135c02f070 con 0x561df0ddebe0 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 10 client.15837 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 10 client.15837 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:31.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 20 client.15837 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0e56f60 0x561df0e59370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 4 client.15837 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.676+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561df0cec050 con 0x561df0e56f60 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.677+0000 7f1366ffd700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0e56f60 0x561df0e59370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.677+0000 7f134aff5700 10 client.15837 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.677+0000 7f1366ffd700 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0e56f60 0x561df0e59370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59255e9a6437ca33 server_cookie=1e0db246322ffc5c in_seq=0 out_seq=0 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f134aff5700 1 -- 172.21.15.5:0/3164246185 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561df0cec050 con 0x561df0e56f60 2023-07-21T19:08:31.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f134aff5700 10 client.15837 handle_command_reply: tid=0 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 client.15837 shutdown 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 msgr2=0x561df0dee570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f135c0098e0 tx=0x7f135c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 shutdown_connections 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561df0de9ad0 0x561df0deeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561df0dde2d0 0x561df0df0280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561df0e56f60 0x561df0e59370 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 --2- 172.21.15.5:0/3164246185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561df0ddebe0 0x561df0dee570 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 >> 172.21.15.5:0/3164246185 conn(0x561df0dd4da0 msgr2=0x561df0df5660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f134aff5700 0 client.15837 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 shutdown_connections 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 1 -- 172.21.15.5:0/3164246185 wait complete. 2023-07-21T19:08:31.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.678+0000 7f1380cd4b80 20 client.15837 trim_cache size 0 max 16384 2023-07-21T19:08:31.697 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:08:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 msgr2=0x7f13500a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 0x7f13500a3910 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f1368199390 tx=0x7f135c005c60 comp rx=0 tx=0).stop 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 msgr2=0x7f136819c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f136000ef30 tx=0x7f136000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 shutdown_connections 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13500a1460 0x7f13500a3910 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1368105b10 0x7f136819c720 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13680ffb00 0x7f1368197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 --2- 172.21.15.5:0/1833111023 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13681004b0 0x7f13681982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 >> 172.21.15.5:0/1833111023 conn(0x7f13680fb600 msgr2=0x7f136810aee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.679+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 shutdown_connections 2023-07-21T19:08:31.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:31.680+0000 7f136d3ff700 1 -- 172.21.15.5:0/1833111023 wait complete. 2023-07-21T19:08:31.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:31 smithi101 ceph-mon[128136]: pgmap v30: 97 pgs: 97 active+clean; 387 MiB data, 513 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 23 MiB/s wr, 27 op/s 2023-07-21T19:08:31.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:31 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3737987072' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:08:31.874 INFO:teuthology.orchestra.run.smithi005.stderr:1800000 of 50000000 tuples (3%) done (elapsed 29.43 s, remaining 788.06 s) 2023-07-21T19:08:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 -- 172.21.15.5:0/889830223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c0ff930 msgr2=0x7f672c0ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 --2- 172.21.15.5:0/889830223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c0ff930 0x7f672c0ffda0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f6720009990 tx=0x7f6720009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 -- 172.21.15.5:0/889830223 shutdown_connections 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 --2- 172.21.15.5:0/889830223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c1002e0 0x7f672c10c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 --2- 172.21.15.5:0/889830223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 0x7f672c105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 --2- 172.21.15.5:0/889830223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c0ff930 0x7f672c0ffda0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.545+0000 7f672b59e700 1 -- 172.21.15.5:0/889830223 >> 172.21.15.5:0/889830223 conn(0x7f672c0fb450 msgr2=0x7f672c0fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.546+0000 7f672b59e700 1 -- 172.21.15.5:0/889830223 shutdown_connections 2023-07-21T19:08:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.546+0000 7f672b59e700 1 -- 172.21.15.5:0/889830223 wait complete. 2023-07-21T19:08:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.547+0000 7f672b59e700 1 Processor -- start 2023-07-21T19:08:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.547+0000 7f672b59e700 1 -- start start 2023-07-21T19:08:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c0ff930 0x7f672c19b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 0x7f672c1956c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c195d30 con 0x7f672c1002e0 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c195ea0 con 0x7f672c0ff930 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672b59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672c196060 con 0x7f672c105940 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f6729d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f6729d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60748/0 (socket says 172.21.15.5:60748) 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f6729d9b700 1 -- 172.21.15.5:0/3849879931 learned_addr learned my addr 172.21.15.5:0/3849879931 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672a59c700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c0ff930 0x7f672c19b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.548+0000 7f672ad9d700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 0x7f672c1956c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 -- 172.21.15.5:0/3849879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 msgr2=0x7f672c1956c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 0x7f672c1956c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 -- 172.21.15.5:0/3849879931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c0ff930 msgr2=0x7f672c19b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c0ff930 0x7f672c19b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 -- 172.21.15.5:0/3849879931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67200095f0 con 0x7f672c1002e0 2023-07-21T19:08:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f6729d9b700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6720005910 tx=0x7f67200047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f671b7fe700 1 -- 172.21.15.5:0/3849879931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672001d070 con 0x7f672c1002e0 2023-07-21T19:08:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.549+0000 7f671b7fe700 1 -- 172.21.15.5:0/3849879931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6720021410 con 0x7f672c1002e0 2023-07-21T19:08:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.550+0000 7f671b7fe700 1 -- 172.21.15.5:0/3849879931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6720017650 con 0x7f672c1002e0 2023-07-21T19:08:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.550+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f672c1962e0 con 0x7f672c1002e0 2023-07-21T19:08:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.550+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672c1a2640 con 0x7f672c1002e0 2023-07-21T19:08:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.553+0000 7f671b7fe700 1 -- 172.21.15.5:0/3849879931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f672000fb00 con 0x7f672c1002e0 2023-07-21T19:08:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.555+0000 7f674451bb80 1 Processor -- start 2023-07-21T19:08:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.555+0000 7f674451bb80 1 -- start start 2023-07-21T19:08:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f671b7fe700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 0x7f67140a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e3b470 0x559ec2e351a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b8c0 0x559ec2e35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d3ac50 con 0x559ec2e3b8c0 2023-07-21T19:08:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d8d9e0 con 0x559ec2c916b0 2023-07-21T19:08:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f674451bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d86e70 con 0x559ec2e3b470 2023-07-21T19:08:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f671b7fe700 1 -- 172.21.15.5:0/3849879931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f67200ce400 con 0x7f672c1002e0 2023-07-21T19:08:32.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f672ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f672ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50500/0 (socket says 172.21.15.5:50500) 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f672ad9d700 1 -- 172.21.15.5:0/2932890832 learned_addr learned my addr 172.21.15.5:0/2932890832 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e3b470 0x559ec2e351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.556+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b8c0 0x559ec2e35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672a59c700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 0x7f67140a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 -- 172.21.15.5:0/2932890832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e3b470 msgr2=0x559ec2e351a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e3b470 0x559ec2e351a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 -- 172.21.15.5:0/2932890832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b8c0 msgr2=0x559ec2e35700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b8c0 0x559ec2e35700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 -- 172.21.15.5:0/2932890832 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ec2bb4d80 con 0x559ec2c916b0 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6724009ed0 tx=0x7f6724009420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d15b6a584bdb6e52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f6718ff9700 1 -- 172.21.15.5:0/2932890832 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672400e070 con 0x559ec2c916b0 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f6718ff9700 1 -- 172.21.15.5:0/2932890832 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6724004bb0 con 0x559ec2c916b0 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.557+0000 7f6718ff9700 1 -- 172.21.15.5:0/2932890832 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6724016e00 con 0x559ec2c916b0 2023-07-21T19:08:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 -- 172.21.15.5:0/2932890832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 msgr2=0x559ec2e3af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6724009ed0 tx=0x7f6724009420 comp rx=0 tx=0).stop 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f672a59c700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 0x7f67140a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f672c100a70 tx=0x7f671c009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 -- 172.21.15.5:0/2932890832 shutdown_connections 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b8c0 0x559ec2e35700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e3af30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 --2- 172.21.15.5:0/2932890832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e3b470 0x559ec2e351a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.558+0000 7f674451bb80 1 -- 172.21.15.5:0/2932890832 >> 172.21.15.5:0/2932890832 conn(0x559ec2e2ad40 msgr2=0x559ec2e30340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.559+0000 7f674451bb80 1 -- 172.21.15.5:0/2932890832 shutdown_connections 2023-07-21T19:08:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.559+0000 7f674451bb80 1 -- 172.21.15.5:0/2932890832 wait complete. 2023-07-21T19:08:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.559+0000 7f674451bb80 1 -- start start 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.559+0000 7f674451bb80 1 Processor -- start 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e42840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e3b8c0 0x559ec2e3f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d26a60 con 0x559ec2e3b470 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d26760 con 0x559ec2e3b8c0 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2c0a8c0 con 0x559ec2c916b0 2023-07-21T19:08:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f674451bb80 10 client.0 _ll_register_callbacks cb 0x559ec2d24700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f6729d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f6729d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60760/0 (socket says 172.21.15.5:60760) 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f6729d9b700 1 -- 172.21.15.5:0/115529860 learned_addr learned my addr 172.21.15.5:0/115529860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.560+0000 7f672a59c700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e42840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 -- 172.21.15.5:0/115529860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 msgr2=0x559ec2e42840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e42840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 -- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e3b8c0 msgr2=0x559ec2e3f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e3b8c0 0x559ec2e3f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 -- 172.21.15.5:0/115529860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6724007500 con 0x559ec2e3b470 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6729d9b700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f672009afe0 tx=0x7f672009ad70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a14ad4fbeaa0f5d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.561+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672009f150 con 0x559ec2e3b470 2023-07-21T19:08:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f672009b2e0 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559ec2ce0490 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f672009f310 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 10 client.15849 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559ec2b9dc00 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559ec2ce0490 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f672009a7d0 con 0x559ec2e3b470 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 10 client.15849 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 10 client.15849 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 20 client.15849 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.562+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e46a30 0x559ec2e48e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.563+0000 7f674451bb80 4 client.15849 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.563+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559ec2d7fe50 con 0x559ec2e46a30 2023-07-21T19:08:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.563+0000 7f672a59c700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e46a30 0x559ec2e48e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.564+0000 7f672a59c700 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e46a30 0x559ec2e48e40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d454e88a538d81d3 server_cookie=94da3809b1c3b51f in_seq=0 out_seq=0 2023-07-21T19:08:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.564+0000 7f6711ffb700 10 client.15849 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.564+0000 7f6711ffb700 1 -- 172.21.15.5:0/115529860 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559ec2d7fe50 con 0x559ec2e46a30 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.564+0000 7f6711ffb700 10 client.15849 handle_command_reply: tid=0 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.564+0000 7f674451bb80 1 client.15849 shutdown 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 msgr2=0x559ec2e3ea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f672009afe0 tx=0x7f672009ad70 comp rx=0 tx=0).stop 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 shutdown_connections 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e3b8c0 0x559ec2e3f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e42840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e46a30 0x559ec2e48e40 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 --2- 172.21.15.5:0/115529860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3ea40 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 >> 172.21.15.5:0/115529860 conn(0x559ec2e2ad40 msgr2=0x559ec2e2f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f6711ffb700 0 client.15849 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 shutdown_connections 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 1 -- 172.21.15.5:0/115529860 wait complete. 2023-07-21T19:08:32.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.565+0000 7f674451bb80 20 client.15849 trim_cache size 0 max 16384 2023-07-21T19:08:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.583+0000 7f674451bb80 1 Processor -- start 2023-07-21T19:08:32.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.583+0000 7f674451bb80 1 -- start start 2023-07-21T19:08:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e4d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2d7fe50 con 0x559ec2e3b470 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2ce0490 con 0x559ec2c916b0 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f674451bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2e311e0 con 0x559ec2e2ad40 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f672a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f672a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53254/0 (socket says 172.21.15.5:53254) 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f672a59c700 1 -- 172.21.15.5:0/2765957182 learned_addr learned my addr 172.21.15.5:0/2765957182 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.584+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e4d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 -- 172.21.15.5:0/2765957182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 msgr2=0x559ec2e4d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e4d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 -- 172.21.15.5:0/2765957182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 msgr2=0x559ec2e3a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 -- 172.21.15.5:0/2765957182 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559ec2bdb1f0 con 0x559ec2e2ad40 2023-07-21T19:08:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f672a59c700 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x559ec2e49720 tx=0x7f671c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=443ad4e0d3f1b6fa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f67197fa700 1 -- 172.21.15.5:0/2765957182 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c025590 con 0x559ec2e2ad40 2023-07-21T19:08:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f67197fa700 1 -- 172.21.15.5:0/2765957182 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f671c023040 con 0x559ec2e2ad40 2023-07-21T19:08:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.585+0000 7f67197fa700 1 -- 172.21.15.5:0/2765957182 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c025a50 con 0x559ec2e2ad40 2023-07-21T19:08:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 -- 172.21.15.5:0/2765957182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 msgr2=0x559ec2e52f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x559ec2e49720 tx=0x7f671c014000 comp rx=0 tx=0).stop 2023-07-21T19:08:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 -- 172.21.15.5:0/2765957182 shutdown_connections 2023-07-21T19:08:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e3a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2e2ad40 0x559ec2e52f60 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 --2- 172.21.15.5:0/2765957182 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2c916b0 0x559ec2e4d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 -- 172.21.15.5:0/2765957182 >> 172.21.15.5:0/2765957182 conn(0x559ec2e43200 msgr2=0x559ec2e39b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 -- 172.21.15.5:0/2765957182 shutdown_connections 2023-07-21T19:08:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.586+0000 7f674451bb80 1 -- 172.21.15.5:0/2765957182 wait complete. 2023-07-21T19:08:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.587+0000 7f674451bb80 1 -- start start 2023-07-21T19:08:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.587+0000 7f674451bb80 1 Processor -- start 2023-07-21T19:08:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 0x559ec2e36de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e37360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2e37a70 con 0x559ec2e3b470 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2e37be0 con 0x559ec2e2ad40 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559ec2eb37e0 con 0x559ec2c916b0 2023-07-21T19:08:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f674451bb80 10 client.0 _ll_register_callbacks cb 0x7f672009b0b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f672a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f672a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53258/0 (socket says 172.21.15.5:53258) 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f672a59c700 1 -- 172.21.15.5:0/2144911652 learned_addr learned my addr 172.21.15.5:0/2144911652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e37360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.588+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 0x559ec2e36de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 -- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 msgr2=0x559ec2e36de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 0x559ec2e36de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 -- 172.21.15.5:0/2144911652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 msgr2=0x559ec2e37360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e37360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 -- 172.21.15.5:0/2144911652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f671c017070 con 0x559ec2c916b0 2023-07-21T19:08:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 0x559ec2e36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672ad9d700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e37360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f672a59c700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f671c01dc80 tx=0x7f671c01dcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7c94cd02282b613 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.589+0000 7f6711ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c005220 con 0x559ec2c916b0 2023-07-21T19:08:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6724004bb0 con 0x559ec2c916b0 2023-07-21T19:08:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f671c005380 con 0x559ec2c916b0 2023-07-21T19:08:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f671c005540 con 0x559ec2c916b0 2023-07-21T19:08:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f674451bb80 10 client.25567 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559ec2bb3250 con 0x559ec2c916b0 2023-07-21T19:08:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.590+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6724004bb0 con 0x559ec2c916b0 2023-07-21T19:08:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f671c02b020 con 0x559ec2c916b0 2023-07-21T19:08:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 10 client.25567 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 10 client.25567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 20 client.25567 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e3b8c0 0x559ec2eb6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 4 client.25567 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:08:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.591+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559ec2d41ff0 con 0x559ec2e3b8c0 2023-07-21T19:08:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.592+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e3b8c0 0x559ec2eb6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.592+0000 7f6729d9b700 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e3b8c0 0x559ec2eb6e00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ea7e5888707b53 server_cookie=5cc2587743741eca in_seq=0 out_seq=0 2023-07-21T19:08:32.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.592+0000 7f6711ffb700 10 client.25567 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.608+0000 7f6711ffb700 1 -- 172.21.15.5:0/2144911652 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559ec2d41ff0 con 0x559ec2e3b8c0 2023-07-21T19:08:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.608+0000 7f6711ffb700 10 client.25567 handle_command_reply: tid=0 2023-07-21T19:08:32.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.608+0000 7f674451bb80 1 client.25567 shutdown 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.608+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 msgr2=0x559ec2e4d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f671c01dc80 tx=0x7f671c01dcb0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 shutdown_connections 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559ec2e3b8c0 0x559ec2eb6e00 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559ec2e3b470 0x559ec2e37360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559ec2c916b0 0x559ec2e4d970 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 --2- 172.21.15.5:0/2144911652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559ec2e2ad40 0x559ec2e36de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f6711ffb700 0 client.25567 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 >> 172.21.15.5:0/2144911652 conn(0x559ec2e43200 msgr2=0x559ec2e3a410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 shutdown_connections 2023-07-21T19:08:32.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 1 -- 172.21.15.5:0/2144911652 wait complete. 2023-07-21T19:08:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.609+0000 7f674451bb80 20 client.25567 trim_cache size 0 max 16384 2023-07-21T19:08:32.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:08:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:08:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:08:32.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:08:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 msgr2=0x7f67140a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 0x7f67140a3910 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f672c100a70 tx=0x7f671c009260 comp rx=0 tx=0).stop 2023-07-21T19:08:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 msgr2=0x7f672c19bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f6720005910 tx=0x7f67200047b0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 shutdown_connections 2023-07-21T19:08:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f672c105940 0x7f672c1956c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f672c0ff930 0x7f672c19b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67140a1460 0x7f67140a3910 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 --2- 172.21.15.5:0/3849879931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f672c1002e0 0x7f672c19bb90 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 >> 172.21.15.5:0/3849879931 conn(0x7f672c0fb450 msgr2=0x7f672c0fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 shutdown_connections 2023-07-21T19:08:32.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:32.610+0000 7f672b59e700 1 -- 172.21.15.5:0/3849879931 wait complete. 2023-07-21T19:08:32.710 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:08:32.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:08:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 -- 172.21.15.5:0/768285004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 msgr2=0x7f39b410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 --2- 172.21.15.5:0/768285004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b410c910 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f39ac00b0a0 tx=0x7f39ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 -- 172.21.15.5:0/768285004 shutdown_connections 2023-07-21T19:08:32.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 --2- 172.21.15.5:0/768285004 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b410c910 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 --2- 172.21.15.5:0/768285004 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 --2- 172.21.15.5:0/768285004 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 0x7f39b40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 -- 172.21.15.5:0/768285004 >> 172.21.15.5:0/768285004 conn(0x7f39b40fb5f0 msgr2=0x7f39b40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 -- 172.21.15.5:0/768285004 shutdown_connections 2023-07-21T19:08:32.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.944+0000 7f39badff700 1 -- 172.21.15.5:0/768285004 wait complete. 2023-07-21T19:08:32.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.945+0000 7f39badff700 1 Processor -- start 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 -- start start 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 0x7f39b4072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b41a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b4073580 con 0x7f39b4100480 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41a42f0 con 0x7f39b4105ae0 2023-07-21T19:08:32.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39badff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39b41a4430 con 0x7f39b40ffad0 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.946+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 0x7f39b4072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60804/0 (socket says 172.21.15.5:60804) 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 -- 172.21.15.5:0/396392191 learned_addr learned my addr 172.21.15.5:0/396392191 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b939c700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b41a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 -- 172.21.15.5:0/396392191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 msgr2=0x7f39b4072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 0x7f39b4072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 -- 172.21.15.5:0/396392191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 msgr2=0x7f39b41a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b41a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 -- 172.21.15.5:0/396392191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39ac009d00 con 0x7f39b4100480 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b939c700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b41a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.947+0000 7f39b3fff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f39a000d980 tx=0x7f39a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.948+0000 7f39b1ffb700 1 -- 172.21.15.5:0/396392191 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0004a30 con 0x7f39b4100480 2023-07-21T19:08:32.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.948+0000 7f39b1ffb700 1 -- 172.21.15.5:0/396392191 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39a00054b0 con 0x7f39b4100480 2023-07-21T19:08:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.948+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39b41a4690 con 0x7f39b4100480 2023-07-21T19:08:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.948+0000 7f39b1ffb700 1 -- 172.21.15.5:0/396392191 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0009bf0 con 0x7f39b4100480 2023-07-21T19:08:32.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.948+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39b41a4b70 con 0x7f39b4100480 2023-07-21T19:08:32.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.952+0000 7f39b1ffb700 1 -- 172.21.15.5:0/396392191 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f39a0009d50 con 0x7f39b4100480 2023-07-21T19:08:32.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.953+0000 7f39ce796b80 1 Processor -- start 2023-07-21T19:08:32.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.953+0000 7f39ce796b80 1 -- start start 2023-07-21T19:08:32.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.953+0000 7f39b1ffb700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 0x7f39a40a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b1ffb700 1 -- 172.21.15.5:0/396392191 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f39a00cce60 con 0x7f39b4100480 2023-07-21T19:08:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 0x5566cfb39e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb3a360 0x5566cfb3c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfa2cd30 con 0x5566cfb3a360 2023-07-21T19:08:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfa7fac0 con 0x5566cfb26c60 2023-07-21T19:08:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39ce796b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfa78f50 con 0x5566cfb26350 2023-07-21T19:08:32.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b939c700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 0x7f39a40a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb3a360 0x5566cfb3c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53280/0 (socket says 172.21.15.5:53280) 2023-07-21T19:08:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.954+0000 7f39b8b9b700 1 -- 172.21.15.5:0/3252289211 learned_addr learned my addr 172.21.15.5:0/3252289211 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b3fff700 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 0x5566cfb39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 -- 172.21.15.5:0/3252289211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 msgr2=0x5566cfb39e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 0x5566cfb39e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 -- 172.21.15.5:0/3252289211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb3a360 msgr2=0x5566cfb3c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb3a360 0x5566cfb3c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 -- 172.21.15.5:0/3252289211 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566cf8a6e60 con 0x5566cfb26350 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b3fff700 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 0x5566cfb39e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b939c700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 0x7f39a40a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009cd0 tx=0x7f39ac009b80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.955+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007c70 tx=0x7f39a8007f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d27ccda735534fab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.956+0000 7f399effd700 1 -- 172.21.15.5:0/3252289211 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a8010b80 con 0x5566cfb26350 2023-07-21T19:08:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.956+0000 7f399effd700 1 -- 172.21.15.5:0/3252289211 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39a8015070 con 0x5566cfb26350 2023-07-21T19:08:32.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.956+0000 7f399effd700 1 -- 172.21.15.5:0/3252289211 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a80193d0 con 0x5566cfb26350 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.956+0000 7f39ce796b80 1 -- 172.21.15.5:0/3252289211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 msgr2=0x5566cfb26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.956+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39a8007c70 tx=0x7f39a8007f80 comp rx=0 tx=0).stop 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 -- 172.21.15.5:0/3252289211 shutdown_connections 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb3a360 0x5566cfb3c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb26720 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3252289211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26c60 0x5566cfb39e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 -- 172.21.15.5:0/3252289211 >> 172.21.15.5:0/3252289211 conn(0x5566cfb1ce20 msgr2=0x5566cfb22720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 -- 172.21.15.5:0/3252289211 shutdown_connections 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 -- 172.21.15.5:0/3252289211 wait complete. 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.957+0000 7f39ce796b80 1 -- start start 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 Processor -- start 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb33670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb39390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfa18b40 con 0x5566cfb26c60 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfa18840 con 0x5566cfb26350 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cf8fc9a0 con 0x5566cfb3a360 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39ce796b80 10 client.0 _ll_register_callbacks cb 0x5566cfa167e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.958+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b8b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb33670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60830/0 (socket says 172.21.15.5:60830) 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 -- 172.21.15.5:0/2419492763 learned_addr learned my addr 172.21.15.5:0/2419492763 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b939c700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb39390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 -- 172.21.15.5:0/2419492763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 msgr2=0x5566cfb39390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb39390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 -- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 msgr2=0x5566cfb33670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:32.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb33670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 -- 172.21.15.5:0/2419492763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39a80078a0 con 0x5566cfb26c60 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb33670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f39a0099c40 tx=0x7f39a0004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e202267d6e54d118 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.959+0000 7f3993fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00b1070 con 0x5566cfb26c60 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f39a00a2100 con 0x5566cfb26c60 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5566cf9d2570 con 0x5566cfb26c60 2023-07-21T19:08:32.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a00b2260 con 0x5566cfb26c60 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f39ce796b80 10 client.15867 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5566cf88fce0 con 0x5566cfb26c60 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39a00c8080 con 0x5566cfb26c60 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.960+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39a00c9f80 con 0x5566cfb26c60 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 10 client.15867 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 10 client.15867 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:32.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 20 client.15867 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfb522d0 0x5566cfb546e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 4 client.15867 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5566cfa71f30 con 0x5566cfb522d0 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.961+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfb522d0 0x5566cfb546e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.962+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfb522d0 0x5566cfb546e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cccf9a887019e788 server_cookie=6e9c8e929637e78d in_seq=0 out_seq=0 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.962+0000 7f3993fff700 10 client.15867 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.962+0000 7f3993fff700 1 -- 172.21.15.5:0/2419492763 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5566cfa71f30 con 0x5566cfb522d0 2023-07-21T19:08:32.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f3993fff700 10 client.15867 handle_command_reply: tid=0 2023-07-21T19:08:32.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 client.15867 shutdown 2023-07-21T19:08:32.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 msgr2=0x5566cfb33bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:32.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f39a0099c40 tx=0x7f39a0004d60 comp rx=0 tx=0).stop 2023-07-21T19:08:32.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 shutdown_connections 2023-07-21T19:08:32.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb39390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb33670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfb522d0 0x5566cfb546e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2419492763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb33bb0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:32.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f3993fff700 0 client.15867 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:32.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 >> 172.21.15.5:0/2419492763 conn(0x5566cfb1ce20 msgr2=0x5566cfb222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 shutdown_connections 2023-07-21T19:08:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 1 -- 172.21.15.5:0/2419492763 wait complete. 2023-07-21T19:08:32.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.963+0000 7f39ce796b80 20 client.15867 trim_cache size 0 max 16384 2023-07-21T19:08:32.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.984+0000 7f39ce796b80 1 Processor -- start 2023-07-21T19:08:32.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.984+0000 7f39ce796b80 1 -- start start 2023-07-21T19:08:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb39300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 0x5566cfb33220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfb1ef40 con 0x5566cfb26c60 2023-07-21T19:08:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfb29470 con 0x5566cfb3a360 2023-07-21T19:08:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39ce796b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfb2ffd0 con 0x5566cfb26350 2023-07-21T19:08:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b939c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 0x5566cfb33220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60836/0 (socket says 172.21.15.5:60836) 2023-07-21T19:08:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b939c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 0x5566cfb33220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50548/0 (socket says 172.21.15.5:50548) 2023-07-21T19:08:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b3fff700 1 -- 172.21.15.5:0/3153258047 learned_addr learned my addr 172.21.15.5:0/3153258047 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.985+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb39300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 -- 172.21.15.5:0/3153258047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 msgr2=0x5566cfb39300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb39300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 -- 172.21.15.5:0/3153258047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 msgr2=0x5566cfb33220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 0x5566cfb33220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 -- 172.21.15.5:0/3153258047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5566cfb2e460 con 0x5566cfb26c60 2023-07-21T19:08:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f39b3fff700 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f39a00a1bd0 tx=0x7f39a0004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b99396782ab3e70 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f399f7fe700 1 -- 172.21.15.5:0/3153258047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00b1070 con 0x5566cfb26c60 2023-07-21T19:08:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f399f7fe700 1 -- 172.21.15.5:0/3153258047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5566cf9d2570 con 0x5566cfb26c60 2023-07-21T19:08:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.986+0000 7f399f7fe700 1 -- 172.21.15.5:0/3153258047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a00a1e70 con 0x5566cfb26c60 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 -- 172.21.15.5:0/3153258047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 msgr2=0x5566cfb2d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f39a00a1bd0 tx=0x7f39a0004e10 comp rx=0 tx=0).stop 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 -- 172.21.15.5:0/3153258047 shutdown_connections 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb3a360 0x5566cfb33220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb26350 0x5566cfb39300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 --2- 172.21.15.5:0/3153258047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb2d970 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 -- 172.21.15.5:0/3153258047 >> 172.21.15.5:0/3153258047 conn(0x5566cfb1ce20 msgr2=0x5566cfb59040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 -- 172.21.15.5:0/3153258047 shutdown_connections 2023-07-21T19:08:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.987+0000 7f39ce796b80 1 -- 172.21.15.5:0/3153258047 wait complete. 2023-07-21T19:08:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.988+0000 7f39ce796b80 1 -- start start 2023-07-21T19:08:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.988+0000 7f39ce796b80 1 Processor -- start 2023-07-21T19:08:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb36c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfba6310 con 0x5566cfb26c60 2023-07-21T19:08:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfba6480 con 0x5566cfb26350 2023-07-21T19:08:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5566cfba6640 con 0x5566cfb3a360 2023-07-21T19:08:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39ce796b80 10 client.0 _ll_register_callbacks cb 0x7f39a0004c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60848/0 (socket says 172.21.15.5:60848) 2023-07-21T19:08:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.989+0000 7f39b3fff700 1 -- 172.21.15.5:0/2268398971 learned_addr learned my addr 172.21.15.5:0/2268398971 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 -- 172.21.15.5:0/2268398971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 msgr2=0x5566cfb36c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb36c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 -- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 msgr2=0x5566cfb361a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb361a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 -- 172.21.15.5:0/2268398971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5566cf88fce0 con 0x5566cfb26c60 2023-07-21T19:08:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:08:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39b3fff700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009520 tx=0x7f39a00b81f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a96b0e5cc153169 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f3993fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:08:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39a0014020 con 0x5566cfb26c60 2023-07-21T19:08:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f39a00a2100 con 0x5566cfb26c60 2023-07-21T19:08:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.990+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39a8015070 con 0x5566cfb26c60 2023-07-21T19:08:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39a00b1070 con 0x5566cfb26c60 2023-07-21T19:08:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 10 client.15873 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5566cfb1e660 con 0x5566cfb26c60 2023-07-21T19:08:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39a8015070 con 0x5566cfb26c60 2023-07-21T19:08:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39a0095aa0 con 0x5566cfb26c60 2023-07-21T19:08:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 10 client.15873 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 10 client.15873 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 20 client.15873 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfba7940 0x5566cfba9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 4 client.15873 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:08:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.991+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5566cfa340d0 con 0x5566cfba7940 2023-07-21T19:08:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.992+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfba7940 0x5566cfba9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.992+0000 7f39b8b9b700 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfba7940 0x5566cfba9d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5707c921a42de9b3 server_cookie=d2981c6836851837 in_seq=0 out_seq=0 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.992+0000 7f3993fff700 10 client.15873 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.993+0000 7f3993fff700 1 -- 172.21.15.5:0/2268398971 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5566cfa340d0 con 0x5566cfba7940 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.993+0000 7f3993fff700 10 client.15873 handle_command_reply: tid=0 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.993+0000 7f39ce796b80 1 client.15873 shutdown 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.993+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 msgr2=0x5566cfb366e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.993+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f39a0009520 tx=0x7f39a00b81f0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 shutdown_connections 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5566cfb3a360 0x5566cfb36c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5566cfb26350 0x5566cfb361a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5566cfba7940 0x5566cfba9d20 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 --2- 172.21.15.5:0/2268398971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5566cfb26c60 0x5566cfb366e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 >> 172.21.15.5:0/2268398971 conn(0x5566cfb1ce20 msgr2=0x5566cfb28cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f3993fff700 0 client.15873 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 shutdown_connections 2023-07-21T19:08:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 1 -- 172.21.15.5:0/2268398971 wait complete. 2023-07-21T19:08:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.994+0000 7f39ce796b80 20 client.15873 trim_cache size 0 max 16384 2023-07-21T19:08:33.023 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:08:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:08:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "8886242e-eb32-49fb-a7fd-08f28461dd49", 2023-07-21T19:08:33.023 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:08:33.023 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:08:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 msgr2=0x7f39a40a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 0x7f39a40a3910 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f39ac009cd0 tx=0x7f39ac009b80 comp rx=0 tx=0).stop 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 msgr2=0x7f39b4072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f39a000d980 tx=0x7f39a000dc90 comp rx=0 tx=0).stop 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 shutdown_connections 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39b4105ae0 0x7f39b41a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39b40ffad0 0x7f39b4072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39a40a1460 0x7f39a40a3910 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 --2- 172.21.15.5:0/396392191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39b4100480 0x7f39b4072d50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.995+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 >> 172.21.15.5:0/396392191 conn(0x7f39b40fb5f0 msgr2=0x7f39b41035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.996+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 shutdown_connections 2023-07-21T19:08:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:32.996+0000 7f39badff700 1 -- 172.21.15.5:0/396392191 wait complete. 2023-07-21T19:08:33.031 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:08:33.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 -- 172.21.15.5:0/2398662585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf0105b10 msgr2=0x7faaf0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2398662585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf0105b10 0x7faaf0105ee0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7faaec009990 tx=0x7faaec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 -- 172.21.15.5:0/2398662585 shutdown_connections 2023-07-21T19:08:33.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2398662585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf01004b0 0x7faaf010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2398662585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf0105b10 0x7faaf0105ee0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2398662585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf00ffb00 0x7faaf00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 -- 172.21.15.5:0/2398662585 >> 172.21.15.5:0/2398662585 conn(0x7faaf00fb600 msgr2=0x7faaf00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 -- 172.21.15.5:0/2398662585 shutdown_connections 2023-07-21T19:08:33.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.268+0000 7faaf75ff700 1 -- 172.21.15.5:0/2398662585 wait complete. 2023-07-21T19:08:33.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.269+0000 7faaf75ff700 1 Processor -- start 2023-07-21T19:08:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.269+0000 7faaf75ff700 1 -- start start 2023-07-21T19:08:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf01004b0 0x7faaf019be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf0105b10 0x7faaf0195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0195fa0 con 0x7faaf00ffb00 2023-07-21T19:08:33.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf0196110 con 0x7faaf01004b0 2023-07-21T19:08:33.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf75ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaf01962d0 con 0x7faaf0105b10 2023-07-21T19:08:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60874/0 (socket says 172.21.15.5:60874) 2023-07-21T19:08:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf539b700 1 -- 172.21.15.5:0/2382161330 learned_addr learned my addr 172.21.15.5:0/2382161330 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf01004b0 0x7faaf019be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.270+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf0105b10 0x7faaf0195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 -- 172.21.15.5:0/2382161330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf0105b10 msgr2=0x7faaf0195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf0105b10 0x7faaf0195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 -- 172.21.15.5:0/2382161330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf01004b0 msgr2=0x7faaf019be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf01004b0 0x7faaf019be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 -- 172.21.15.5:0/2382161330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faae0009520 con 0x7faaf00ffb00 2023-07-21T19:08:33.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf539b700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faaec000c00 tx=0x7faaec005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faae67fc700 1 -- 172.21.15.5:0/2382161330 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec01d070 con 0x7faaf00ffb00 2023-07-21T19:08:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faae67fc700 1 -- 172.21.15.5:0/2382161330 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaec022470 con 0x7faaf00ffb00 2023-07-21T19:08:33.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaec0095f0 con 0x7faaf00ffb00 2023-07-21T19:08:33.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faae67fc700 1 -- 172.21.15.5:0/2382161330 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec022ea0 con 0x7faaf00ffb00 2023-07-21T19:08:33.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.271+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaf01a26d0 con 0x7faaf00ffb00 2023-07-21T19:08:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.276+0000 7faae67fc700 1 -- 172.21.15.5:0/2382161330 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7faaec00fb00 con 0x7faaf00ffb00 2023-07-21T19:08:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.277+0000 7fab0aff0b80 1 Processor -- start 2023-07-21T19:08:33.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.277+0000 7fab0aff0b80 1 -- start start 2023-07-21T19:08:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x55979205b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x55979205deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791f56cb0 con 0x55979205bad0 2023-07-21T19:08:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791fa9a40 con 0x559792050be0 2023-07-21T19:08:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791fa2ed0 con 0x5597920502d0 2023-07-21T19:08:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7faaf5b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7faaf5b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53332/0 (socket says 172.21.15.5:53332) 2023-07-21T19:08:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7faaf5b9c700 1 -- 172.21.15.5:0/1180072489 learned_addr learned my addr 172.21.15.5:0/1180072489 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7faaf539b700 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x55979205b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.278+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x55979205deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faae67fc700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 0x7faadc0a3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faae67fc700 1 -- 172.21.15.5:0/2382161330 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faaec0ce130 con 0x7faaf00ffb00 2023-07-21T19:08:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 -- 172.21.15.5:0/1180072489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 msgr2=0x55979205b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x55979205b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 -- 172.21.15.5:0/1180072489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 msgr2=0x55979205deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x55979205deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 -- 172.21.15.5:0/1180072489 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559791dd0de0 con 0x5597920502d0 2023-07-21T19:08:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 0x7faadc0a3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.279+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faae8011710 tx=0x7faae8011a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0440d4046779ebc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7faadb7fe700 1 -- 172.21.15.5:0/1180072489 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae800e040 con 0x5597920502d0 2023-07-21T19:08:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7faadb7fe700 1 -- 172.21.15.5:0/1180072489 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faae800f040 con 0x5597920502d0 2023-07-21T19:08:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7faadb7fe700 1 -- 172.21.15.5:0/1180072489 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae8020690 con 0x5597920502d0 2023-07-21T19:08:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 0x7faadc0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faae8022000 tx=0x7faae8023000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7fab0aff0b80 1 -- 172.21.15.5:0/1180072489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 msgr2=0x5597920506a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.280+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7faae8011710 tx=0x7faae8011a20 comp rx=0 tx=0).stop 2023-07-21T19:08:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 -- 172.21.15.5:0/1180072489 shutdown_connections 2023-07-21T19:08:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x55979205deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x5597920506a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/1180072489 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x55979205b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 -- 172.21.15.5:0/1180072489 >> 172.21.15.5:0/1180072489 conn(0x559792046da0 msgr2=0x55979204c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 -- 172.21.15.5:0/1180072489 shutdown_connections 2023-07-21T19:08:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 -- 172.21.15.5:0/1180072489 wait complete. 2023-07-21T19:08:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.281+0000 7fab0aff0b80 1 -- start start 2023-07-21T19:08:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 Processor -- start 2023-07-21T19:08:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559792050be0 0x5597920587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55979205bad0 0x559792058e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791f42ac0 con 0x559792050be0 2023-07-21T19:08:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791f427c0 con 0x55979205bad0 2023-07-21T19:08:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559791e26920 con 0x5597920502d0 2023-07-21T19:08:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7fab0aff0b80 10 client.0 _ll_register_callbacks cb 0x559791f40760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7faaf539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7faaf539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53336/0 (socket says 172.21.15.5:53336) 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7faaf539b700 1 -- 172.21.15.5:0/2114537750 learned_addr learned my addr 172.21.15.5:0/2114537750 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.282+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559792050be0 0x5597920587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55979205bad0 0x559792058e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 -- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55979205bad0 msgr2=0x559792058e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55979205bad0 0x559792058e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 -- 172.21.15.5:0/2114537750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559792050be0 msgr2=0x5597920587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559792050be0 0x5597920587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 -- 172.21.15.5:0/2114537750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae8011300 con 0x5597920502d0 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faaf539b700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faaec0975e0 tx=0x7faaec09b620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec65a7f65a8dc5b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faad8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec0d16b0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.283+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faaec09b0e0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559791efc4f0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaec09aeb0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 10 client.25594 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559791db9c60 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559791efc4f0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faaec0b92e0 con 0x5597920502d0 2023-07-21T19:08:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 10 client.25594 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 10 client.25594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.284+0000 7fab0aff0b80 20 client.25594 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.285+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55979206b0b0 0x55979206d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.285+0000 7fab0aff0b80 4 client.25594 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.285+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559791f9beb0 con 0x55979206b0b0 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.285+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55979206b0b0 0x55979206d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.286+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55979206b0b0 0x55979206d4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b78cab088a31936c server_cookie=f77071f790d667fd in_seq=0 out_seq=0 2023-07-21T19:08:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.286+0000 7faad8ff9700 10 client.25594 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.286+0000 7faad8ff9700 1 -- 172.21.15.5:0/2114537750 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559791f9beb0 con 0x55979206b0b0 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.286+0000 7faad8ff9700 10 client.25594 handle_command_reply: tid=0 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.286+0000 7fab0aff0b80 1 client.25594 shutdown 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 msgr2=0x55979205b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7faaec0975e0 tx=0x7faaec09b620 comp rx=0 tx=0).stop 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 shutdown_connections 2023-07-21T19:08:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55979205bad0 0x559792058e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b4e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55979206b0b0 0x55979206d4c0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2114537750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559792050be0 0x5597920587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7faad8ff9700 0 client.25594 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 >> 172.21.15.5:0/2114537750 conn(0x559792046da0 msgr2=0x55979204c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 shutdown_connections 2023-07-21T19:08:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2114537750 wait complete. 2023-07-21T19:08:33.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.287+0000 7fab0aff0b80 20 client.25594 trim_cache size 0 max 16384 2023-07-21T19:08:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.311+0000 7fab0aff0b80 1 Processor -- start 2023-07-21T19:08:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 -- start start 2023-07-21T19:08:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x559792057f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559792046970 con 0x55979205bad0 2023-07-21T19:08:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55979206db20 con 0x559792050be0 2023-07-21T19:08:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559792056d30 con 0x5597920502d0 2023-07-21T19:08:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7faaf5b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7faaf4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x559792057f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7faaf5b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60902/0 (socket says 172.21.15.5:60902) 2023-07-21T19:08:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7faaf5b9c700 1 -- 172.21.15.5:0/652483084 learned_addr learned my addr 172.21.15.5:0/652483084 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.312+0000 7faaf539b700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 -- 172.21.15.5:0/652483084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 msgr2=0x55979205b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 -- 172.21.15.5:0/652483084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 msgr2=0x5597920584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 -- 172.21.15.5:0/652483084 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559791df7250 con 0x559792050be0 2023-07-21T19:08:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf539b700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x559792057f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55979206dd30 tx=0x7faae0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=415e99d04e0f2fc6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faadbfff700 1 -- 172.21.15.5:0/652483084 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0016030 con 0x559792050be0 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faadbfff700 1 -- 172.21.15.5:0/652483084 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faae0017040 con 0x559792050be0 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.313+0000 7faadbfff700 1 -- 172.21.15.5:0/652483084 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0003e20 con 0x559792050be0 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 -- 172.21.15.5:0/652483084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 msgr2=0x559792057f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x559792057f90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55979206dd30 tx=0x7faae0009260 comp rx=0 tx=0).stop 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 -- 172.21.15.5:0/652483084 shutdown_connections 2023-07-21T19:08:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x5597920584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597920502d0 0x55979205b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/652483084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559792050be0 0x559792057f90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 -- 172.21.15.5:0/652483084 >> 172.21.15.5:0/652483084 conn(0x559792046da0 msgr2=0x559792053ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 -- 172.21.15.5:0/652483084 shutdown_connections 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.314+0000 7fab0aff0b80 1 -- 172.21.15.5:0/652483084 wait complete. 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.315+0000 7fab0aff0b80 1 -- start start 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.315+0000 7fab0aff0b80 1 Processor -- start 2023-07-21T19:08:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.315+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559792050be0 0x559792060670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x559792060bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597920610f0 con 0x55979205bad0 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559792061260 con 0x5597920502d0 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559792061420 con 0x559792050be0 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7fab0aff0b80 10 client.0 _ll_register_callbacks cb 0x559791f40760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf4b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559792050be0 0x559792060670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50608/0 (socket says 172.21.15.5:50608) 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 -- 172.21.15.5:0/2436539940 learned_addr learned my addr 172.21.15.5:0/2436539940 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf5b9c700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x559792060bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 -- 172.21.15.5:0/2436539940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559792050be0 msgr2=0x559792060670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559792050be0 0x559792060670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 -- 172.21.15.5:0/2436539940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 msgr2=0x559792060bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x559792060bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.316+0000 7faaf539b700 1 -- 172.21.15.5:0/2436539940 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559791db9c60 con 0x5597920502d0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faaf539b700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaec09ade0 tx=0x7faaec09a940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff1790cacbeb937a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaec09ac20 con 0x5597920502d0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faaec0d16b0 con 0x5597920502d0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faae800f040 con 0x5597920502d0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaec09b0e0 con 0x5597920502d0 2023-07-21T19:08:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7fab0aff0b80 10 client.25625 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x559791dd0de0 con 0x5597920502d0 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faae800f040 con 0x5597920502d0 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.317+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faaec0cb370 con 0x5597920502d0 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 10 client.25625 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 10 client.25625 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 20 client.25625 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5597920bfef0 0x5597920c22d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 4 client.25625 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559791f5e050 con 0x5597920bfef0 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.318+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5597920bfef0 0x5597920c22d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7faad8ff9700 10 client.25625 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7faaf4b9a700 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5597920bfef0 0x5597920c22d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407f880ce77d01de server_cookie=a1b901d4eb6cf800 in_seq=0 out_seq=0 2023-07-21T19:08:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7faad8ff9700 1 -- 172.21.15.5:0/2436539940 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x559791f5e050 con 0x5597920bfef0 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7faad8ff9700 10 client.25625 handle_command_reply: tid=0 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 client.25625 shutdown 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 msgr2=0x559792062340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7faaec09ade0 tx=0x7faaec09a940 comp rx=0 tx=0).stop 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 shutdown_connections 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5597920bfef0 0x5597920c22d0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55979205bad0 0x559792060bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597920502d0 0x559792062340 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7faad8ff9700 0 client.25625 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 --2- 172.21.15.5:0/2436539940 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559792050be0 0x559792060670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 >> 172.21.15.5:0/2436539940 conn(0x559792046da0 msgr2=0x5597920521a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 shutdown_connections 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.319+0000 7fab0aff0b80 1 -- 172.21.15.5:0/2436539940 wait complete. 2023-07-21T19:08:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7fab0aff0b80 20 client.25625 trim_cache size 0 max 16384 2023-07-21T19:08:33.340 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:08:33.340 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-21T19:08:33.341 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:08:33.341 INFO:teuthology.orchestra.run.smithi005.stdout: "8886242e-eb32-49fb-a7fd-08f28461dd49": { 2023-07-21T19:08:33.341 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:08:33.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8886242e-eb32-49fb-a7fd-08f28461dd49", 2023-07-21T19:08:33.342 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:08:33.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:08:33.342 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:08:33.343 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:08:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 msgr2=0x7faadc0a3910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 0x7faadc0a3910 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faae8022000 tx=0x7faae8023000 comp rx=0 tx=0).stop 2023-07-21T19:08:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 msgr2=0x7faaf019b8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7faaec000c00 tx=0x7faaec005740 comp rx=0 tx=0).stop 2023-07-21T19:08:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 shutdown_connections 2023-07-21T19:08:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaf0105b10 0x7faaf0195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faadc0a1460 0x7faadc0a3910 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaf00ffb00 0x7faaf019b8c0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 --2- 172.21.15.5:0/2382161330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaf01004b0 0x7faaf019be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 >> 172.21.15.5:0/2382161330 conn(0x7faaf00fb600 msgr2=0x7faaf00fbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.320+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 shutdown_connections 2023-07-21T19:08:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:08:33.321+0000 7faaf75ff700 1 -- 172.21.15.5:0/2382161330 wait complete. 2023-07-21T19:08:33.349 INFO:tasks.cephfs.filesystem:scrub status for tag:8886242e-eb32-49fb-a7fd-08f28461dd49 - {'path': '/', 'tag': '8886242e-eb32-49fb-a7fd-08f28461dd49', 'options': 'recursive,force'} 2023-07-21T19:08:33.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: pgmap v31: 97 pgs: 97 active+clean; 415 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:08:33.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:33.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.418 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:33.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:33.419 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:33 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: pgmap v31: 97 pgs: 97 active+clean; 415 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:08:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:33.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:33.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:33 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.833 INFO:teuthology.orchestra.run.smithi005.stderr:1900000 of 50000000 tuples (3%) done (elapsed 31.39 s, remaining 794.64 s) 2023-07-21T19:08:33.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: pgmap v31: 97 pgs: 97 active+clean; 415 MiB data, 523 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:33.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:33 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:34 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:08:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:34 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:34 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:34 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:08:34.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:34 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:34 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:08:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:34 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:34 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:34 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:08:34.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:34 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:34 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:08:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:34 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:34 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:34 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:08:34.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:34 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:08:35.065 INFO:teuthology.orchestra.run.smithi005.stderr:2000000 of 50000000 tuples (4%) done (elapsed 32.62 s, remaining 782.92 s) 2023-07-21T19:08:35.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:35 smithi005 ceph-mon[126460]: pgmap v32: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 34 op/s 2023-07-21T19:08:35.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:35 smithi156 ceph-mon[128316]: pgmap v32: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 34 op/s 2023-07-21T19:08:35.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:35 smithi101 ceph-mon[128136]: pgmap v32: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 30 MiB/s wr, 34 op/s 2023-07-21T19:08:36.905 INFO:teuthology.orchestra.run.smithi005.stderr:2100000 of 50000000 tuples (4%) done (elapsed 34.46 s, remaining 786.04 s) 2023-07-21T19:08:37.672 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:37 smithi005 ceph-mon[126460]: pgmap v33: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 20 op/s 2023-07-21T19:08:37.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:37 smithi156 ceph-mon[128316]: pgmap v33: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 20 op/s 2023-07-21T19:08:37.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:37 smithi101 ceph-mon[128136]: pgmap v33: 97 pgs: 97 active+clean; 456 MiB data, 544 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 20 op/s 2023-07-21T19:08:38.784 INFO:teuthology.orchestra.run.smithi005.stderr:2200000 of 50000000 tuples (4%) done (elapsed 36.34 s, remaining 789.57 s) 2023-07-21T19:08:39.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:39 smithi005 ceph-mon[126460]: pgmap v34: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 34 MiB/s wr, 99 op/s 2023-07-21T19:08:39.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:39 smithi156 ceph-mon[128316]: pgmap v34: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 34 MiB/s wr, 99 op/s 2023-07-21T19:08:39.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:39 smithi101 ceph-mon[128136]: pgmap v34: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 34 MiB/s wr, 99 op/s 2023-07-21T19:08:40.105 INFO:teuthology.orchestra.run.smithi005.stderr:2300000 of 50000000 tuples (4%) done (elapsed 37.66 s, remaining 781.05 s) 2023-07-21T19:08:41.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:41 smithi005 ceph-mon[126460]: pgmap v35: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:41.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:41 smithi156 ceph-mon[128316]: pgmap v35: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:41 smithi101 ceph-mon[128136]: pgmap v35: 97 pgs: 97 active+clean; 524 MiB data, 570 MiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2400000 of 50000000 tuples (4%) done (elapsed 39.56 s, remaining 784.52 s) 2023-07-21T19:08:43.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:43 smithi005 ceph-mon[126460]: pgmap v36: 97 pgs: 97 active+clean; 568 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 30 MiB/s wr, 92 op/s 2023-07-21T19:08:43.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:43 smithi156 ceph-mon[128316]: pgmap v36: 97 pgs: 97 active+clean; 568 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 30 MiB/s wr, 92 op/s 2023-07-21T19:08:43.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:43 smithi101 ceph-mon[128136]: pgmap v36: 97 pgs: 97 active+clean; 568 MiB data, 581 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 30 MiB/s wr, 92 op/s 2023-07-21T19:08:43.889 INFO:teuthology.orchestra.run.smithi005.stderr:2500000 of 50000000 tuples (5%) done (elapsed 41.45 s, remaining 787.46 s) 2023-07-21T19:08:45.234 INFO:teuthology.orchestra.run.smithi005.stderr:2600000 of 50000000 tuples (5%) done (elapsed 42.79 s, remaining 780.10 s) 2023-07-21T19:08:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:45 smithi005 ceph-mon[126460]: pgmap v37: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 29 MiB/s wr, 96 op/s 2023-07-21T19:08:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:45 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:08:45.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:45 smithi156 ceph-mon[128316]: pgmap v37: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 29 MiB/s wr, 96 op/s 2023-07-21T19:08:45.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:45 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:08:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.821+0000 7f2d8359e700 1 -- 172.21.15.5:0/2738699320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d840ffc20 msgr2=0x7f2d8410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.821+0000 7f2d8359e700 1 --2- 172.21.15.5:0/2738699320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d840ffc20 0x7f2d8410c000 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f2d780098f0 tx=0x7f2d78009c00 comp rx=0 tx=0).stop 2023-07-21T19:08:45.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 -- 172.21.15.5:0/2738699320 shutdown_connections 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 --2- 172.21.15.5:0/2738699320 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d8410c540 0x7f2d8410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 --2- 172.21.15.5:0/2738699320 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d840ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 --2- 172.21.15.5:0/2738699320 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d840ffc20 0x7f2d8410c000 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 -- 172.21.15.5:0/2738699320 >> 172.21.15.5:0/2738699320 conn(0x7f2d84074770 msgr2=0x7f2d84074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 -- 172.21.15.5:0/2738699320 shutdown_connections 2023-07-21T19:08:45.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.822+0000 7f2d8359e700 1 -- 172.21.15.5:0/2738699320 wait complete. 2023-07-21T19:08:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.823+0000 7f2d8359e700 1 Processor -- start 2023-07-21T19:08:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.823+0000 7f2d8359e700 1 -- start start 2023-07-21T19:08:45.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 0x7f2d8419df00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 0x7f2d84197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841980a0 con 0x7f2d8410c540 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d84198210 con 0x7f2d840ffc20 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d841983d0 con 0x7f2d840ff310 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52858/0 (socket says 172.21.15.5:52858) 2023-07-21T19:08:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d8259c700 1 -- 172.21.15.5:0/1610266081 learned_addr learned my addr 172.21.15.5:0/1610266081 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 0x7f2d8419df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.824+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 0x7f2d84197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 -- 172.21.15.5:0/1610266081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 msgr2=0x7f2d8419d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 -- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 msgr2=0x7f2d84197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 0x7f2d84197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 -- 172.21.15.5:0/1610266081 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009520 con 0x7f2d840ffc20 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d8259c700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 0x7f2d84197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 0x7f2d8419df00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009810 tx=0x7f2d7800fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d737fe700 1 -- 172.21.15.5:0/1610266081 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7801c070 con 0x7f2d840ffc20 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d737fe700 1 -- 172.21.15.5:0/1610266081 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d7800bb70 con 0x7f2d840ffc20 2023-07-21T19:08:45.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d84198650 con 0x7f2d840ffc20 2023-07-21T19:08:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.825+0000 7f2d737fe700 1 -- 172.21.15.5:0/1610266081 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780038c0 con 0x7f2d840ffc20 2023-07-21T19:08:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.826+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d841a49b0 con 0x7f2d840ffc20 2023-07-21T19:08:45.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.829+0000 7f2d737fe700 1 -- 172.21.15.5:0/1610266081 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f2d78020030 con 0x7f2d840ffc20 2023-07-21T19:08:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:45 smithi101 ceph-mon[128136]: pgmap v37: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 29 MiB/s wr, 96 op/s 2023-07-21T19:08:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:45 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:08:45.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 Processor -- start 2023-07-21T19:08:45.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d737fe700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 0x7f2d6c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 -- start start 2023-07-21T19:08:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d737fe700 1 -- 172.21.15.5:0/1610266081 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2d780cde00 con 0x7f2d840ffc20 2023-07-21T19:08:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e02f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.832+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16d02c50 con 0x55ea16e03470 2023-07-21T19:08:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16d559e0 con 0x55ea16c596b0 2023-07-21T19:08:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16d4ee70 con 0x55ea16e038c0 2023-07-21T19:08:45.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d82d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39116/0 (socket says 172.21.15.5:39116) 2023-07-21T19:08:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 -- 172.21.15.5:0/3172270327 learned_addr learned my addr 172.21.15.5:0/3172270327 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 0x7f2d6c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e02f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 -- 172.21.15.5:0/3172270327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 msgr2=0x55ea16dfd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfd700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 -- 172.21.15.5:0/3172270327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 msgr2=0x55ea16e02f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e02f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d81d9b700 1 -- 172.21.15.5:0/3172270327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea16b7cd80 con 0x55ea16e03470 2023-07-21T19:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.833+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.834+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2d780971e0 tx=0x7f2d78000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9da5f6dd7fbb5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.834+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 0x7f2d6c0a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ea16e03cd0 tx=0x7f2d7c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.834+0000 7f2d70ff9700 1 -- 172.21.15.5:0/3172270327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7809ace0 con 0x55ea16e03470 2023-07-21T19:08:45.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.834+0000 7f2d70ff9700 1 -- 172.21.15.5:0/3172270327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d7809d070 con 0x55ea16e03470 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.834+0000 7f2d70ff9700 1 -- 172.21.15.5:0/3172270327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78097550 con 0x55ea16e03470 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3172270327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 msgr2=0x55ea16dfd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f2d780971e0 tx=0x7f2d78000bc0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3172270327 shutdown_connections 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfd700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e02f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3172270327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfd1a0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3172270327 >> 172.21.15.5:0/3172270327 conn(0x55ea16df2d40 msgr2=0x55ea16df8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3172270327 shutdown_connections 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.835+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3172270327 wait complete. 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.836+0000 7f2d9c819b80 1 -- start start 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.836+0000 7f2d9c819b80 1 Processor -- start 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e067a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e06ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16ceea60 con 0x55ea16e03470 2023-07-21T19:08:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16cee760 con 0x55ea16c596b0 2023-07-21T19:08:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16bd28c0 con 0x55ea16e038c0 2023-07-21T19:08:45.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.837+0000 7f2d9c819b80 10 client.0 _ll_register_callbacks cb 0x55ea16cec700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44594/0 (socket says 172.21.15.5:44594) 2023-07-21T19:08:45.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 -- 172.21.15.5:0/3077114843 learned_addr learned my addr 172.21.15.5:0/3077114843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e067a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e06ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 -- 172.21.15.5:0/3077114843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 msgr2=0x55ea16e06ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e06ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 -- 172.21.15.5:0/3077114843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 msgr2=0x55ea16e067a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e067a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 -- 172.21.15.5:0/3077114843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d74009d90 con 0x55ea16c596b0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d74012e80 tx=0x7f2d7400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41241fcad0b71973 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.838+0000 7f2d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74017070 con 0x55ea16c596b0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2d740120d0 con 0x55ea16c596b0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea16ca8490 con 0x55ea16c596b0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d74012290 con 0x55ea16c596b0 2023-07-21T19:08:45.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d9c819b80 10 client.25637 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f2d841a49b0 con 0x55ea16c596b0 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.839+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ea16ca8490 con 0x55ea16c596b0 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d74003b00 con 0x55ea16c596b0 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 10 client.25637 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 10 client.25637 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 20 client.25637 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e30290 0x55ea16e326a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 4 client.25637 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea16d47e50 con 0x55ea16e30290 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.840+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e30290 0x55ea16e326a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.841+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e30290 0x55ea16e326a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33295d7674c95f88 server_cookie=5f0658c83745ab87 in_seq=0 out_seq=0 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.841+0000 7f2d69ffb700 10 client.25637 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3077114843 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea16d47e50 con 0x55ea16e30290 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d69ffb700 10 client.25637 handle_command_reply: tid=0 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 client.25637 shutdown 2023-07-21T19:08:45.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 msgr2=0x55ea16e09590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d74012e80 tx=0x7f2d7400f6a0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 shutdown_connections 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e06ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e09590 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e30290 0x55ea16e326a0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3077114843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e067a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d69ffb700 0 client.25637 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.842+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 >> 172.21.15.5:0/3077114843 conn(0x55ea16df2d40 msgr2=0x55ea16df7640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.843+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 shutdown_connections 2023-07-21T19:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.843+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3077114843 wait complete. 2023-07-21T19:08:45.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.843+0000 7f2d9c819b80 20 client.25637 trim_cache size 0 max 16384 2023-07-21T19:08:45.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 Processor -- start 2023-07-21T19:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 -- start start 2023-07-21T19:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e06de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e1ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16e078b0 con 0x55ea16e03470 2023-07-21T19:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16e013e0 con 0x55ea16c596b0 2023-07-21T19:08:45.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.871+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16df9c30 con 0x55ea16e038c0 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.872+0000 7f2d82d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.872+0000 7f2d82d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52900/0 (socket says 172.21.15.5:52900) 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.872+0000 7f2d82d9d700 1 -- 172.21.15.5:0/234758876 learned_addr learned my addr 172.21.15.5:0/234758876 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.872+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e1ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d8259c700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e06de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 -- 172.21.15.5:0/234758876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 msgr2=0x55ea16e06de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e06de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 -- 172.21.15.5:0/234758876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 msgr2=0x55ea16e1ee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e1ee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 -- 172.21.15.5:0/234758876 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea16b65c00 con 0x55ea16e038c0 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e1ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009340 tx=0x7f2d7c00c000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99b8ab67ef1a2d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.873+0000 7f2d717fa700 1 -- 172.21.15.5:0/234758876 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c00e040 con 0x55ea16e038c0 2023-07-21T19:08:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d717fa700 1 -- 172.21.15.5:0/234758876 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d7c021040 con 0x55ea16e038c0 2023-07-21T19:08:45.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d717fa700 1 -- 172.21.15.5:0/234758876 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c004280 con 0x55ea16e038c0 2023-07-21T19:08:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 -- 172.21.15.5:0/234758876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 msgr2=0x55ea16e23cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f2d7c009340 tx=0x7f2d7c00c000 comp rx=0 tx=0).stop 2023-07-21T19:08:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 -- 172.21.15.5:0/234758876 shutdown_connections 2023-07-21T19:08:45.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16e23cd0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e06de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/234758876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16e1ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.874+0000 7f2d9c819b80 1 -- 172.21.15.5:0/234758876 >> 172.21.15.5:0/234758876 conn(0x55ea16df2d40 msgr2=0x55ea16e235c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.875+0000 7f2d9c819b80 1 -- 172.21.15.5:0/234758876 shutdown_connections 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.875+0000 7f2d9c819b80 1 -- 172.21.15.5:0/234758876 wait complete. 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.875+0000 7f2d9c819b80 1 -- start start 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.875+0000 7f2d9c819b80 1 Processor -- start 2023-07-21T19:08:45.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfc6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16dfcc10 con 0x55ea16e03470 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16dfcd50 con 0x55ea16c596b0 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea16dfcf10 con 0x55ea16e038c0 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d9c819b80 10 client.0 _ll_register_callbacks cb 0x55ea16cec700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:45.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d8259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44622/0 (socket says 172.21.15.5:44622) 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d8259c700 1 -- 172.21.15.5:0/3445620171 learned_addr learned my addr 172.21.15.5:0/3445620171 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.876+0000 7f2d82d9d700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfc6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 -- 172.21.15.5:0/3445620171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 msgr2=0x55ea16dfc6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfc6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 -- 172.21.15.5:0/3445620171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 msgr2=0x55ea16dfc190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:45.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfc190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 -- 172.21.15.5:0/3445620171 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c0090b0 con 0x55ea16c596b0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d8259c700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000c00 tx=0x7f2d74009fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d25b60bb10ed0f51 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.877+0000 7f2d69ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d74017070 con 0x55ea16c596b0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2d74003880 con 0x55ea16c596b0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d7809d070 con 0x55ea16c596b0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d7401c560 con 0x55ea16c596b0 2023-07-21T19:08:45.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d9c819b80 10 client.25643 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ea16ba31f0 con 0x55ea16c596b0 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.878+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d7809d070 con 0x55ea16c596b0 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d7401c9a0 con 0x55ea16c596b0 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 10 client.25643 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 10 client.25643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 20 client.25643 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:45.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e85400 0x55ea16e87810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 4 client.25643 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:08:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea16d09ff0 con 0x55ea16e85400 2023-07-21T19:08:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.879+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e85400 0x55ea16e87810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:45.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.880+0000 7f2d81d9b700 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e85400 0x55ea16e87810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fe9922f7f535516 server_cookie=4097efa16b8bf064 in_seq=0 out_seq=0 2023-07-21T19:08:45.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.880+0000 7f2d69ffb700 10 client.25643 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d69ffb700 1 -- 172.21.15.5:0/3445620171 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea16d09ff0 con 0x55ea16e85400 2023-07-21T19:08:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d69ffb700 10 client.25643 handle_command_reply: tid=0 2023-07-21T19:08:45.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 client.25643 shutdown 2023-07-21T19:08:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 msgr2=0x55ea16e1ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f2d74000c00 tx=0x7f2d74009fb0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 shutdown_connections 2023-07-21T19:08:45.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ea16e038c0 0x55ea16dfc6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ea16c596b0 0x55ea16e1ee20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ea16e85400 0x55ea16e87810 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 --2- 172.21.15.5:0/3445620171 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ea16e03470 0x55ea16dfc190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 >> 172.21.15.5:0/3445620171 conn(0x55ea16df2d40 msgr2=0x55ea16e05800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d69ffb700 0 client.25643 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 shutdown_connections 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.894+0000 7f2d9c819b80 1 -- 172.21.15.5:0/3445620171 wait complete. 2023-07-21T19:08:45.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.895+0000 7f2d9c819b80 20 client.25643 trim_cache size 0 max 16384 2023-07-21T19:08:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:08:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:08:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:08:45.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:08:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.895+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 msgr2=0x7f2d6c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.895+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 0x7f2d6c0a3890 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55ea16e03cd0 tx=0x7f2d7c00b040 comp rx=0 tx=0).stop 2023-07-21T19:08:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.895+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 msgr2=0x7f2d8419df00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.895+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 0x7f2d8419df00 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f2d78009810 tx=0x7f2d7800fa00 comp rx=0 tx=0).stop 2023-07-21T19:08:45.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 shutdown_connections 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d6c0a13e0 0x7f2d6c0a3890 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d8410c540 0x7f2d84197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d840ff310 0x7f2d8419d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 --2- 172.21.15.5:0/1610266081 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d840ffc20 0x7f2d8419df00 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 >> 172.21.15.5:0/1610266081 conn(0x7f2d84074770 msgr2=0x7f2d840fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 shutdown_connections 2023-07-21T19:08:45.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:45.896+0000 7f2d8359e700 1 -- 172.21.15.5:0/1610266081 wait complete. 2023-07-21T19:08:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:46 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:46 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:46.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:46 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:47.110 INFO:teuthology.orchestra.run.smithi005.stderr:2700000 of 50000000 tuples (5%) done (elapsed 44.67 s, remaining 782.49 s) 2023-07-21T19:08:47.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:47 smithi101 ceph-mon[128136]: pgmap v38: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:08:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:08:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:47 smithi005 ceph-mon[126460]: pgmap v38: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:08:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:08:47.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:47.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:47 smithi156 ceph-mon[128316]: pgmap v38: 97 pgs: 97 active+clean; 596 MiB data, 596 MiB used, 1.0 TiB / 1.0 TiB avail; 89 KiB/s rd, 23 MiB/s wr, 89 op/s 2023-07-21T19:08:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:08:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:08:47.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:08:48.899 INFO:teuthology.orchestra.run.smithi005.stderr:2800000 of 50000000 tuples (5%) done (elapsed 46.46 s, remaining 783.10 s) 2023-07-21T19:08:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:49 smithi101 ceph-mon[128136]: pgmap v39: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-07-21T19:08:49.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:49 smithi005 ceph-mon[126460]: pgmap v39: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-07-21T19:08:49.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:49 smithi156 ceph-mon[128316]: pgmap v39: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 34 MiB/s wr, 102 op/s 2023-07-21T19:08:50.199 INFO:teuthology.orchestra.run.smithi005.stderr:2900000 of 50000000 tuples (5%) done (elapsed 47.75 s, remaining 775.60 s) 2023-07-21T19:08:51.812 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:51 smithi101 ceph-mon[128136]: pgmap v40: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:51.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:51 smithi005 ceph-mon[126460]: pgmap v40: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:51.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:51 smithi156 ceph-mon[128316]: pgmap v40: 97 pgs: 97 active+clean; 664 MiB data, 621 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:08:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:3000000 of 50000000 tuples (6%) done (elapsed 49.64 s, remaining 777.64 s) 2023-07-21T19:08:53.408 INFO:teuthology.orchestra.run.smithi005.stderr:3100000 of 50000000 tuples (6%) done (elapsed 50.96 s, remaining 771.03 s) 2023-07-21T19:08:53.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:53 smithi101 ceph-mon[128136]: pgmap v41: 97 pgs: 97 active+clean; 692 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 25 op/s 2023-07-21T19:08:53.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:53 smithi005 ceph-mon[126460]: pgmap v41: 97 pgs: 97 active+clean; 692 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 25 op/s 2023-07-21T19:08:53.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:53 smithi156 ceph-mon[128316]: pgmap v41: 97 pgs: 97 active+clean; 692 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 28 MiB/s wr, 25 op/s 2023-07-21T19:08:55.304 INFO:teuthology.orchestra.run.smithi005.stderr:3200000 of 50000000 tuples (6%) done (elapsed 52.86 s, remaining 773.09 s) 2023-07-21T19:08:55.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:55 smithi101 ceph-mon[128136]: pgmap v42: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2023-07-21T19:08:55.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:55 smithi005 ceph-mon[126460]: pgmap v42: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2023-07-21T19:08:55.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:55 smithi156 ceph-mon[128316]: pgmap v42: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 27 MiB/s wr, 29 op/s 2023-07-21T19:08:57.182 INFO:teuthology.orchestra.run.smithi005.stderr:3300000 of 50000000 tuples (6%) done (elapsed 54.74 s, remaining 774.63 s) 2023-07-21T19:08:57.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:57 smithi101 ceph-mon[128136]: pgmap v43: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:08:57.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:57 smithi005 ceph-mon[126460]: pgmap v43: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:08:57.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:57 smithi156 ceph-mon[128316]: pgmap v43: 97 pgs: 97 active+clean; 736 MiB data, 652 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:08:58.401 INFO:teuthology.orchestra.run.smithi005.stderr:3400000 of 50000000 tuples (6%) done (elapsed 55.96 s, remaining 766.95 s) 2023-07-21T19:08:59.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.069+0000 7f84c47ff700 1 -- 172.21.15.5:0/3543598134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc067c10 msgr2=0x7f84bc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.069+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3543598134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc067c10 0x7f84bc067fe0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f84b80098f0 tx=0x7f84b8009c00 comp rx=0 tx=0).stop 2023-07-21T19:08:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 -- 172.21.15.5:0/3543598134 shutdown_connections 2023-07-21T19:08:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3543598134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc104e50 0x7f84bc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3543598134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc067c10 0x7f84bc067fe0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3543598134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc0685b0 0x7f84bc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 -- 172.21.15.5:0/3543598134 >> 172.21.15.5:0/3543598134 conn(0x7f84bc074760 msgr2=0x7f84bc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 -- 172.21.15.5:0/3543598134 shutdown_connections 2023-07-21T19:08:59.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.070+0000 7f84c47ff700 1 -- 172.21.15.5:0/3543598134 wait complete. 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.071+0000 7f84c47ff700 1 Processor -- start 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.071+0000 7f84c47ff700 1 -- start start 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 0x7f84bc19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc104e50 0x7f84bc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc195fc0 con 0x7f84bc0685b0 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc196130 con 0x7f84bc067c10 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c47ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84bc1962f0 con 0x7f84bc104e50 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 0x7f84bc19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46162/0 (socket says 172.21.15.5:46162) 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c259b700 1 -- 172.21.15.5:0/3706440716 learned_addr learned my addr 172.21.15.5:0/3706440716 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:59.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.072+0000 7f84c2d9c700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc104e50 0x7f84bc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 -- 172.21.15.5:0/3706440716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc104e50 msgr2=0x7f84bc195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc104e50 0x7f84bc195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 -- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 msgr2=0x7f84bc19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 0x7f84bc19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 -- 172.21.15.5:0/3706440716 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b4009d00 con 0x7f84bc067c10 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 0x7f84bc19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:59.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c259b700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009810 tx=0x7f84b800b560 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84b37fe700 1 -- 172.21.15.5:0/3706440716 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b800b960 con 0x7f84bc067c10 2023-07-21T19:08:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84b37fe700 1 -- 172.21.15.5:0/3706440716 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b800bac0 con 0x7f84bc067c10 2023-07-21T19:08:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84b8009520 con 0x7f84bc067c10 2023-07-21T19:08:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84b37fe700 1 -- 172.21.15.5:0/3706440716 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b8004d70 con 0x7f84bc067c10 2023-07-21T19:08:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.073+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1a2900 con 0x7f84bc067c10 2023-07-21T19:08:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.077+0000 7f84b37fe700 1 -- 172.21.15.5:0/3706440716 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7f84b8005040 con 0x7f84bc067c10 2023-07-21T19:08:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.079+0000 7f84b37fe700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 0x7f84a80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.079+0000 7f84d8096b80 1 Processor -- start 2023-07-21T19:08:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.079+0000 7f84b37fe700 1 -- 172.21.15.5:0/3706440716 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84b80cddf0 con 0x7f84bc067c10 2023-07-21T19:08:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 0x7f84a80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 -- start start 2023-07-21T19:08:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x5565347db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x5565347db9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565346e0c50 con 0x5565346376b0 2023-07-21T19:08:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565347339e0 con 0x5565347db580 2023-07-21T19:08:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84d8096b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55653472ce70 con 0x5565347e12d0 2023-07-21T19:08:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84c259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84c259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54460/0 (socket says 172.21.15.5:54460) 2023-07-21T19:08:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.080+0000 7f84c259b700 1 -- 172.21.15.5:0/2065397779 learned_addr learned my addr 172.21.15.5:0/2065397779 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c2d9c700 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x5565347db9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x5565347db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 -- 172.21.15.5:0/2065397779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 msgr2=0x5565347db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x5565347db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 0x7f84a80a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84bc1064b0 tx=0x7f84ac009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 -- 172.21.15.5:0/2065397779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 msgr2=0x5565347db9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x5565347db9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 -- 172.21.15.5:0/2065397779 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55653455ad80 con 0x5565346376b0 2023-07-21T19:08:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84c259b700 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84b809a9d0 tx=0x7f84b809abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5bec18b0631d0fa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84b0ff9700 1 -- 172.21.15.5:0/2065397779 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b809d030 con 0x5565346376b0 2023-07-21T19:08:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84b0ff9700 1 -- 172.21.15.5:0/2065397779 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b809ed00 con 0x5565346376b0 2023-07-21T19:08:59.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.081+0000 7f84b0ff9700 1 -- 172.21.15.5:0/2065397779 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b809a1c0 con 0x5565346376b0 2023-07-21T19:08:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 -- 172.21.15.5:0/2065397779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 msgr2=0x5565347e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f84b809a9d0 tx=0x7f84b809abd0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 -- 172.21.15.5:0/2065397779 shutdown_connections 2023-07-21T19:08:59.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x5565347db9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565346376b0 0x5565347e0d90 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2065397779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x5565347db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 -- 172.21.15.5:0/2065397779 >> 172.21.15.5:0/2065397779 conn(0x5565347d0be0 msgr2=0x5565347d5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 -- 172.21.15.5:0/2065397779 shutdown_connections 2023-07-21T19:08:59.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.082+0000 7f84d8096b80 1 -- 172.21.15.5:0/2065397779 wait complete. 2023-07-21T19:08:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.083+0000 7f84d8096b80 1 -- start start 2023-07-21T19:08:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.083+0000 7f84d8096b80 1 Processor -- start 2023-07-21T19:08:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 0x5565347ea300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e4600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565346cca60 con 0x5565347db580 2023-07-21T19:08:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565346cc760 con 0x5565347e12d0 2023-07-21T19:08:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565345b08c0 con 0x5565346376b0 2023-07-21T19:08:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84d8096b80 10 client.0 _ll_register_callbacks cb 0x5565346ca700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:59.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54464/0 (socket says 172.21.15.5:54464) 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84c1d9a700 1 -- 172.21.15.5:0/1738442862 learned_addr learned my addr 172.21.15.5:0/1738442862 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.084+0000 7f84c259b700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 0x5565347ea300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c2d9c700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 -- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 msgr2=0x5565347e4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 -- 172.21.15.5:0/1738442862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 msgr2=0x5565347ea840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 -- 172.21.15.5:0/1738442862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84bc1a2900 con 0x5565346376b0 2023-07-21T19:08:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c2d9c700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:08:59.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84c259b700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 0x5565347ea300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5565347dc020 tx=0x7f84b809a3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b54be51d3e311a2f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.085+0000 7f84a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b809d050 con 0x5565346376b0 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84b809e070 con 0x5565346376b0 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556534686490 con 0x5565346376b0 2023-07-21T19:08:59.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b80a3090 con 0x5565346376b0 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84d8096b80 10 client.25630 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556534543c00 con 0x5565346376b0 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b8064080 con 0x5565346376b0 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84b8060b40 con 0x5565346376b0 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84d8096b80 10 client.25630 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.086+0000 7f84d8096b80 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:59.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.087+0000 7f84d8096b80 20 client.25630 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.087+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565347f4400 0x5565347f6810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.087+0000 7f84d8096b80 4 client.25630 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.087+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556534725e50 con 0x5565347f4400 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.087+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565347f4400 0x5565347f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.088+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565347f4400 0x5565347f6810 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82a97368fb0c68ec server_cookie=341f495693932f44 in_seq=0 out_seq=0 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.088+0000 7f84a5ffb700 10 client.25630 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.089+0000 7f84a5ffb700 1 -- 172.21.15.5:0/1738442862 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556534725e50 con 0x5565347f4400 2023-07-21T19:08:59.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.089+0000 7f84a5ffb700 10 client.25630 handle_command_reply: tid=0 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 client.25630 shutdown 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 msgr2=0x5565347ea300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 0x5565347ea300 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x5565347dc020 tx=0x7f84b809a3f0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 shutdown_connections 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565346376b0 0x5565347ea300 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565347f4400 0x5565347f6810 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 --2- 172.21.15.5:0/1738442862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347ea840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84a5ffb700 0 client.25630 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 >> 172.21.15.5:0/1738442862 conn(0x5565347d0be0 msgr2=0x5565347d54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 shutdown_connections 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.090+0000 7f84d8096b80 1 -- 172.21.15.5:0/1738442862 wait complete. 2023-07-21T19:08:59.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.091+0000 7f84d8096b80 20 client.25630 trim_cache size 0 max 16384 2023-07-21T19:08:59.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.109+0000 7f84d8096b80 1 Processor -- start 2023-07-21T19:08:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.109+0000 7f84d8096b80 1 -- start start 2023-07-21T19:08:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e4760 0x5565347d5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565347d7800 con 0x5565347db580 2023-07-21T19:08:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565347df380 con 0x5565347e12d0 2023-07-21T19:08:59.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84d8096b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565347e5550 con 0x5565347e4760 2023-07-21T19:08:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46192/0 (socket says 172.21.15.5:46192) 2023-07-21T19:08:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54474/0 (socket says 172.21.15.5:54474) 2023-07-21T19:08:59.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 -- 172.21.15.5:0/222394193 learned_addr learned my addr 172.21.15.5:0/222394193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c2d9c700 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e4760 0x5565347d5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 -- 172.21.15.5:0/222394193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e4760 msgr2=0x5565347d5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e4760 0x5565347d5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 -- 172.21.15.5:0/222394193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 msgr2=0x5565347db040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.110+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84c1d9a700 1 -- 172.21.15.5:0/222394193 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565347dbc20 con 0x5565347e12d0 2023-07-21T19:08:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84c259b700 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:08:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5565347f7120 tx=0x7f84ac014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64f17d0d40cd7c8b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84b17fa700 1 -- 172.21.15.5:0/222394193 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac024040 con 0x5565347e12d0 2023-07-21T19:08:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84b17fa700 1 -- 172.21.15.5:0/222394193 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84ac02a610 con 0x5565347e12d0 2023-07-21T19:08:59.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.111+0000 7f84b17fa700 1 -- 172.21.15.5:0/222394193 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ac01fc10 con 0x5565347e12d0 2023-07-21T19:08:59.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 -- 172.21.15.5:0/222394193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 msgr2=0x5565347e2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x5565347f7120 tx=0x7f84ac014000 comp rx=0 tx=0).stop 2023-07-21T19:08:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 -- 172.21.15.5:0/222394193 shutdown_connections 2023-07-21T19:08:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e4760 0x5565347d5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347db580 0x5565347db040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 --2- 172.21.15.5:0/222394193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347e12d0 0x5565347e2010 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 -- 172.21.15.5:0/222394193 >> 172.21.15.5:0/222394193 conn(0x5565347d0be0 msgr2=0x5565347e1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 -- 172.21.15.5:0/222394193 shutdown_connections 2023-07-21T19:08:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.112+0000 7f84d8096b80 1 -- 172.21.15.5:0/222394193 wait complete. 2023-07-21T19:08:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 -- start start 2023-07-21T19:08:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 Processor -- start 2023-07-21T19:08:59.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x556534844eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347e4760 0x556534845720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556534845cf0 con 0x5565347e4760 2023-07-21T19:08:59.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556534845e60 con 0x5565347db580 2023-07-21T19:08:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556534846020 con 0x5565347e12d0 2023-07-21T19:08:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.113+0000 7f84d8096b80 10 client.0 _ll_register_callbacks cb 0x7f84b809a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:08:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c1d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x556534844eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c2d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347e4760 0x556534845720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46194/0 (socket says 172.21.15.5:46194) 2023-07-21T19:08:59.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 -- 172.21.15.5:0/2839912852 learned_addr learned my addr 172.21.15.5:0/2839912852 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:08:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 -- 172.21.15.5:0/2839912852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 msgr2=0x556534844eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x556534844eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 -- 172.21.15.5:0/2839912852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347e4760 msgr2=0x556534845720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:08:59.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347e4760 0x556534845720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 -- 172.21.15.5:0/2839912852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556534543c00 con 0x5565347db580 2023-07-21T19:08:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84c259b700 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84b80971d0 tx=0x7f84b8099280 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dfa4b5e16eb26d93 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:08:59.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.114+0000 7f84a5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:08:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b809d070 con 0x5565347db580 2023-07-21T19:08:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84b80a2e50 con 0x5565347db580 2023-07-21T19:08:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84b809ed00 con 0x5565347db580 2023-07-21T19:08:59.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b80052f0 con 0x5565347db580 2023-07-21T19:08:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 10 client.25661 fetch_fsmap learned FSMap version 25 2023-07-21T19:08:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f84ac017070 con 0x5565347db580 2023-07-21T19:08:59.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b809ed00 con 0x5565347db580 2023-07-21T19:08:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f84b8096af0 con 0x5565347db580 2023-07-21T19:08:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 10 client.25661 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:08:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 10 client.25661 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:08:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 20 client.25661 populate_metadata read hostname 'smithi005' 2023-07-21T19:08:59.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55653484d410 0x55653484f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:08:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 4 client.25661 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:08:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.115+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565346e7ff0 con 0x55653484d410 2023-07-21T19:08:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.116+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55653484d410 0x55653484f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:08:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.116+0000 7f84a5ffb700 10 client.25661 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:59.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.116+0000 7f84c1d9a700 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55653484d410 0x55653484f820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7a0224b05da1ad1 server_cookie=38d5289801644188 in_seq=0 out_seq=0 2023-07-21T19:08:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84a5ffb700 1 -- 172.21.15.5:0/2839912852 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565346e7ff0 con 0x55653484d410 2023-07-21T19:08:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84a5ffb700 10 client.25661 handle_command_reply: tid=0 2023-07-21T19:08:59.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 client.25661 shutdown 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 msgr2=0x556534844970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f84b80971d0 tx=0x7f84b8099280 comp rx=0 tx=0).stop 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 shutdown_connections 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55653484d410 0x55653484f820 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565347e4760 0x556534845720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565347db580 0x556534844970 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84a5ffb700 0 client.25661 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 --2- 172.21.15.5:0/2839912852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565347e12d0 0x556534844eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 >> 172.21.15.5:0/2839912852 conn(0x5565347d0be0 msgr2=0x5565347e2db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 shutdown_connections 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 1 -- 172.21.15.5:0/2839912852 wait complete. 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.130+0000 7f84d8096b80 20 client.25661 trim_cache size 0 max 16384 2023-07-21T19:08:59.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:08:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:08:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:08:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:08:59.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 msgr2=0x7f84a80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 0x7f84a80a3890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f84bc1064b0 tx=0x7f84ac009260 comp rx=0 tx=0).stop 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 msgr2=0x7f84bc19baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84b8009810 tx=0x7f84b800b560 comp rx=0 tx=0).stop 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 shutdown_connections 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84bc104e50 0x7f84bc195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84bc067c10 0x7f84bc19baf0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84a80a13e0 0x7f84a80a3890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 --2- 172.21.15.5:0/3706440716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84bc0685b0 0x7f84bc19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:08:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 >> 172.21.15.5:0/3706440716 conn(0x7f84bc074760 msgr2=0x7f84bc0fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:08:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.131+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 shutdown_connections 2023-07-21T19:08:59.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:08:59.132+0000 7f84c47ff700 1 -- 172.21.15.5:0/3706440716 wait complete. 2023-07-21T19:08:59.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:08:59 smithi101 ceph-mon[128136]: pgmap v44: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-07-21T19:08:59.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:08:59 smithi005 ceph-mon[126460]: pgmap v44: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-07-21T19:08:59.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:08:59 smithi156 ceph-mon[128316]: pgmap v44: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 33 op/s 2023-07-21T19:09:00.323 INFO:teuthology.orchestra.run.smithi005.stderr:3500000 of 50000000 tuples (7%) done (elapsed 57.88 s, remaining 768.97 s) 2023-07-21T19:09:01.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:01 smithi101 ceph-mon[128136]: pgmap v45: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:09:01.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:01 smithi005 ceph-mon[126460]: pgmap v45: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:09:01.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:01 smithi156 ceph-mon[128316]: pgmap v45: 97 pgs: 97 active+clean; 808 MiB data, 678 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:09:02.220 INFO:teuthology.orchestra.run.smithi005.stderr:3600000 of 50000000 tuples (7%) done (elapsed 59.78 s, remaining 770.46 s) 2023-07-21T19:09:03.350 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:09:03.572 INFO:teuthology.orchestra.run.smithi005.stderr:3700000 of 50000000 tuples (7%) done (elapsed 61.13 s, remaining 764.93 s) 2023-07-21T19:09:03.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 -- 172.21.15.5:0/2824512142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad00100380 msgr2=0x7fad0010c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 --2- 172.21.15.5:0/2824512142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad00100380 0x7fad0010c810 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7facf800b0a0 tx=0x7facf800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 -- 172.21.15.5:0/2824512142 shutdown_connections 2023-07-21T19:09:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 --2- 172.21.15.5:0/2824512142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad00100380 0x7fad0010c810 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 --2- 172.21.15.5:0/2824512142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 0x7fad00105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 --2- 172.21.15.5:0/2824512142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad000ff9d0 0x7fad000ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 -- 172.21.15.5:0/2824512142 >> 172.21.15.5:0/2824512142 conn(0x7fad000fb4b0 msgr2=0x7fad000fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 -- 172.21.15.5:0/2824512142 shutdown_connections 2023-07-21T19:09:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.578+0000 7fad04f9d700 1 -- 172.21.15.5:0/2824512142 wait complete. 2023-07-21T19:09:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.580+0000 7fad04f9d700 1 Processor -- start 2023-07-21T19:09:03.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.580+0000 7fad04f9d700 1 -- start start 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad00100380 0x7fad00072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 0x7fad001a3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad00073580 con 0x7fad000ff9d0 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001a4080 con 0x7fad00100380 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7fad04f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad001a41c0 con 0x7fad001059e0 2023-07-21T19:09:03.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54498/0 (socket says 172.21.15.5:54498) 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 -- 172.21.15.5:0/3791626357 learned_addr learned my addr 172.21.15.5:0/3791626357 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facfeffd700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad00100380 0x7fad00072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facfffff700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 0x7fad001a3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 -- 172.21.15.5:0/3791626357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 msgr2=0x7fad001a3b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 0x7fad001a3b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 -- 172.21.15.5:0/3791626357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad00100380 msgr2=0x7fad00072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad00100380 0x7fad00072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.581+0000 7facff7fe700 1 -- 172.21.15.5:0/3791626357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facf8009d00 con 0x7fad000ff9d0 2023-07-21T19:09:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7facff7fe700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7facf000e970 tx=0x7facf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7facfcff9700 1 -- 172.21.15.5:0/3791626357 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0009790 con 0x7fad000ff9d0 2023-07-21T19:09:03.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad001a4420 con 0x7fad000ff9d0 2023-07-21T19:09:03.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7facfcff9700 1 -- 172.21.15.5:0/3791626357 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf000c7e0 con 0x7fad000ff9d0 2023-07-21T19:09:03.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7facfcff9700 1 -- 172.21.15.5:0/3791626357 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0003bd0 con 0x7fad000ff9d0 2023-07-21T19:09:03.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.582+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001a4950 con 0x7fad000ff9d0 2023-07-21T19:09:03.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.586+0000 7facfcff9700 1 -- 172.21.15.5:0/3791626357 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 24) v1 ==== 135947+0+0 (secure 0 0 0) 0x7facf000c950 con 0x7fad000ff9d0 2023-07-21T19:09:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 Processor -- start 2023-07-21T19:09:03.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 -- start start 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ad78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ae17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ae4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c39dccb0 con 0x5640c3ad7e00 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3a2fa40 con 0x5640c3ae1d30 2023-07-21T19:09:03.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.588+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3a28ed0 con 0x5640c3ad74f0 2023-07-21T19:09:03.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfcff9700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 0x7face80a39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ae17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ad78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ae4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ad78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44404/0 (socket says 172.21.15.5:44404) 2023-07-21T19:09:03.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfffff700 1 -- 172.21.15.5:0/3909444620 learned_addr learned my addr 172.21.15.5:0/3909444620 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfffff700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 0x7face80a39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfcff9700 1 -- 172.21.15.5:0/3791626357 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7facf0014070 con 0x7fad000ff9d0 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 -- 172.21.15.5:0/3909444620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 msgr2=0x5640c3ad78c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ad78c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 -- 172.21.15.5:0/3909444620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 msgr2=0x5640c3ae17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ae17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facfeffd700 1 -- 172.21.15.5:0/3909444620 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c3856de0 con 0x5640c3ae1d30 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.589+0000 7facff7fe700 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ae17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.590+0000 7facfeffd700 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ae4110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facf4009c80 tx=0x7facf4009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d16f6b0c0dff7cc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.590+0000 7facfffff700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 0x7face80a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facf800d010 tx=0x7facf8006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.590+0000 7face5ffb700 1 -- 172.21.15.5:0/3909444620 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4011cb0 con 0x5640c3ae1d30 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.590+0000 7face5ffb700 1 -- 172.21.15.5:0/3909444620 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf4011e10 con 0x5640c3ae1d30 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.590+0000 7face5ffb700 1 -- 172.21.15.5:0/3909444620 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf4005020 con 0x5640c3ae1d30 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3909444620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 msgr2=0x5640c3ae4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ae4110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7facf4009c80 tx=0x7facf4009f90 comp rx=0 tx=0).stop 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3909444620 shutdown_connections 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ae4110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ad78c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3909444620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ae17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3909444620 >> 172.21.15.5:0/3909444620 conn(0x5640c3accff0 msgr2=0x5640c3ad2930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3909444620 shutdown_connections 2023-07-21T19:09:03.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.591+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3909444620 wait complete. 2023-07-21T19:09:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 -- start start 2023-07-21T19:09:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 Processor -- start 2023-07-21T19:09:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ade8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ae1d30 0x5640c3adeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c39c8ac0 con 0x5640c3ad7e00 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c39c87c0 con 0x5640c3ad74f0 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.592+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c38ac920 con 0x5640c3ae1d30 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7fad19b9cb80 10 client.0 _ll_register_callbacks cb 0x5640c39c6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facfeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ade8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46230/0 (socket says 172.21.15.5:46230) 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 -- 172.21.15.5:0/3075031822 learned_addr learned my addr 172.21.15.5:0/3075031822 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facfffff700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ae1d30 0x5640c3adeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 -- 172.21.15.5:0/3075031822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ae1d30 msgr2=0x5640c3adeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ae1d30 0x5640c3adeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 -- 172.21.15.5:0/3075031822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 msgr2=0x5640c3ade8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ade8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.593+0000 7facff7fe700 1 -- 172.21.15.5:0/3075031822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad001a4950 con 0x5640c3ad74f0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facff7fe700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facf0099ea0 tx=0x7facf0065360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1330194628d67591 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf001b360 con 0x5640c3ad74f0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7facf00691b0 con 0x5640c3ad74f0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640c39824f0 con 0x5640c3ad74f0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf0099f10 con 0x5640c3ad74f0 2023-07-21T19:09:03.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7fad19b9cb80 10 client.25673 fetch_fsmap learned FSMap version 25 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7facf40098e0 con 0x5640c3ad74f0 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640c39824f0 con 0x5640c3ad74f0 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.594+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facf00654d0 con 0x5640c3ad74f0 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 10 client.25673 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 20 client.25673 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3af15d0 0x5640c3af39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 4 client.25673 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5640c3a21eb0 con 0x5640c3af15d0 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.595+0000 7facfeffd700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3af15d0 0x5640c3af39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.596+0000 7facfeffd700 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3af15d0 0x5640c3af39e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc79b5df63435356 server_cookie=7446a19f4121161f in_seq=0 out_seq=0 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.596+0000 7facdeffd700 10 client.25673 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7facdeffd700 1 -- 172.21.15.5:0/3075031822 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5640c3a21eb0 con 0x5640c3af15d0 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7facdeffd700 10 client.25673 handle_command_reply: tid=0 2023-07-21T19:09:03.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 client.25673 shutdown 2023-07-21T19:09:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 msgr2=0x5640c3ae1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7facf0099ea0 tx=0x7facf0065360 comp rx=0 tx=0).stop 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 shutdown_connections 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ae1d30 0x5640c3adeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1670 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3af15d0 0x5640c3af39e0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/3075031822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad7e00 0x5640c3ade8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7facdeffd700 0 client.25673 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 >> 172.21.15.5:0/3075031822 conn(0x5640c3accff0 msgr2=0x5640c3ad2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 shutdown_connections 2023-07-21T19:09:03.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 1 -- 172.21.15.5:0/3075031822 wait complete. 2023-07-21T19:09:03.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.597+0000 7fad19b9cb80 20 client.25673 trim_cache size 0 max 16384 2023-07-21T19:09:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 Processor -- start 2023-07-21T19:09:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 -- start start 2023-07-21T19:09:03.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad7e00 0x5640c3ade0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ade5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ad8770 con 0x5640c3ad74f0 2023-07-21T19:09:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ad8980 con 0x5640c3ae1d30 2023-07-21T19:09:03.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3adff00 con 0x5640c3ad7e00 2023-07-21T19:09:03.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7facff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7facff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54522/0 (socket says 172.21.15.5:54522) 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7facff7fe700 1 -- 172.21.15.5:0/68888325 learned_addr learned my addr 172.21.15.5:0/68888325 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7facfeffd700 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad7e00 0x5640c3ade0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.617+0000 7facfffff700 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ade5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 -- 172.21.15.5:0/68888325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad7e00 msgr2=0x5640c3ade0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad7e00 0x5640c3ade0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 -- 172.21.15.5:0/68888325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 msgr2=0x5640c3ade5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ade5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 -- 172.21.15.5:0/68888325 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640c383fc60 con 0x5640c3ad74f0 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7facff7fe700 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7facf000efd0 tx=0x7facf0099b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12b3b4fa5d8ed4bf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7face67fc700 1 -- 172.21.15.5:0/68888325 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0099870 con 0x5640c3ad74f0 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7face67fc700 1 -- 172.21.15.5:0/68888325 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf00ca5a0 con 0x5640c3ad74f0 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.618+0000 7face67fc700 1 -- 172.21.15.5:0/68888325 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf00a2360 con 0x5640c3ad74f0 2023-07-21T19:09:03.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 -- 172.21.15.5:0/68888325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 msgr2=0x5640c3ae1210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7facf000efd0 tx=0x7facf0099b20 comp rx=0 tx=0).stop 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 -- 172.21.15.5:0/68888325 shutdown_connections 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ae1d30 0x5640c3ade5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ad74f0 0x5640c3ae1210 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/68888325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad7e00 0x5640c3ade0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 -- 172.21.15.5:0/68888325 >> 172.21.15.5:0/68888325 conn(0x5640c3accff0 msgr2=0x5640c3ad9c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 -- 172.21.15.5:0/68888325 shutdown_connections 2023-07-21T19:09:03.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.619+0000 7fad19b9cb80 1 -- 172.21.15.5:0/68888325 wait complete. 2023-07-21T19:09:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.620+0000 7fad19b9cb80 1 -- start start 2023-07-21T19:09:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.620+0000 7fad19b9cb80 1 Processor -- start 2023-07-21T19:09:03.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 0x5640c3ae6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 0x5640c3ae6f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ae7500 con 0x5640c3ae1d30 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ae7640 con 0x5640c3ad7e00 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640c3ae7800 con 0x5640c3ad74f0 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7fad19b9cb80 10 client.0 _ll_register_callbacks cb 0x7facf0099dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:03.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facfeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 0x5640c3ae6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facfffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 0x5640c3ae6f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44438/0 (socket says 172.21.15.5:44438) 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 -- 172.21.15.5:0/2879426889 learned_addr learned my addr 172.21.15.5:0/2879426889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facfeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 0x5640c3ae6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46246/0 (socket says 172.21.15.5:46246) 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 -- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 msgr2=0x5640c3ae6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 0x5640c3ae6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 -- 172.21.15.5:0/2879426889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 msgr2=0x5640c3ae6f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 0x5640c3ae6f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facff7fe700 1 -- 172.21.15.5:0/2879426889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7facf40098e0 con 0x5640c3ad74f0 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.621+0000 7facfffff700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 0x5640c3ae6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facff7fe700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7facf0099c10 tx=0x7facf009a4a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a7d720fd9a28c0b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facdeffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facf0099870 con 0x5640c3ad74f0 2023-07-21T19:09:03.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7facf009a2b0 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7facf4011e10 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7facf00691b0 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7fad19b9cb80 10 client.25654 fetch_fsmap learned FSMap version 25 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x5640c387d250 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.622+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7facf007c020 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7facf00b8170 con 0x5640c3ad74f0 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 10 client.25654 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:09:03.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 10 client.25654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 20 client.25654 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3b464b0 0x5640c3b488c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 4 client.25654 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5640c39e4050 con 0x5640c3b464b0 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.623+0000 7facfeffd700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3b464b0 0x5640c3b488c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7facfeffd700 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3b464b0 0x5640c3b488c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d20b4f0c80bba70e server_cookie=2721ad8a0b8a7354 in_seq=0 out_seq=0 2023-07-21T19:09:03.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7facdeffd700 10 client.25654 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7facdeffd700 1 -- 172.21.15.5:0/2879426889 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5640c39e4050 con 0x5640c3b464b0 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7facdeffd700 10 client.25654 handle_command_reply: tid=0 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 client.25654 shutdown 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 msgr2=0x5640c3ae8710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7facf0099c10 tx=0x7facf009a4a0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 shutdown_connections 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640c3b464b0 0x5640c3b488c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5640c3ae1d30 0x5640c3ae6f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5640c3ad74f0 0x5640c3ae8710 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7facdeffd700 0 client.25654 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 --2- 172.21.15.5:0/2879426889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5640c3ad7e00 0x5640c3ae6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 >> 172.21.15.5:0/2879426889 conn(0x5640c3accff0 msgr2=0x5640c3ad6440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 shutdown_connections 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.624+0000 7fad19b9cb80 1 -- 172.21.15.5:0/2879426889 wait complete. 2023-07-21T19:09:03.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad19b9cb80 20 client.25654 trim_cache size 0 max 16384 2023-07-21T19:09:03.642 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:09:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-21T19:09:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:09:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "8886242e-eb32-49fb-a7fd-08f28461dd49": { 2023-07-21T19:09:03.642 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:09:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8886242e-eb32-49fb-a7fd-08f28461dd49", 2023-07-21T19:09:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:09:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:09:03.643 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:09:03.643 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 msgr2=0x7face80a39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 0x7face80a39b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7facf800d010 tx=0x7facf8006210 comp rx=0 tx=0).stop 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 msgr2=0x7fad00072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7facf000e970 tx=0x7facf000ec80 comp rx=0 tx=0).stop 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 shutdown_connections 2023-07-21T19:09:03.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fad001059e0 0x7fad001a3b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7face80a1500 0x7face80a39b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fad000ff9d0 0x7fad00072810 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 --2- 172.21.15.5:0/3791626357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fad00100380 0x7fad00072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 >> 172.21.15.5:0/3791626357 conn(0x7fad000fb4b0 msgr2=0x7fad000fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.625+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 shutdown_connections 2023-07-21T19:09:03.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:03.626+0000 7fad04f9d700 1 -- 172.21.15.5:0/3791626357 wait complete. 2023-07-21T19:09:03.654 INFO:tasks.cephfs.filesystem:scrub status for tag:8886242e-eb32-49fb-a7fd-08f28461dd49 - {'path': '/', 'tag': '8886242e-eb32-49fb-a7fd-08f28461dd49', 'options': 'recursive,force'} 2023-07-21T19:09:03.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:03 smithi101 ceph-mon[128136]: pgmap v46: 97 pgs: 97 active+clean; 850 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 28 op/s 2023-07-21T19:09:03.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:03 smithi005 ceph-mon[126460]: pgmap v46: 97 pgs: 97 active+clean; 850 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 28 op/s 2023-07-21T19:09:03.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:03 smithi156 ceph-mon[128316]: pgmap v46: 97 pgs: 97 active+clean; 850 MiB data, 690 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 31 MiB/s wr, 28 op/s 2023-07-21T19:09:05.418 INFO:teuthology.orchestra.run.smithi005.stderr:3800000 of 50000000 tuples (7%) done (elapsed 62.97 s, remaining 765.63 s) 2023-07-21T19:09:05.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:05 smithi101 ceph-mon[128136]: pgmap v47: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 MiB/s wr, 46 op/s 2023-07-21T19:09:05.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:05 smithi005 ceph-mon[126460]: pgmap v47: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 MiB/s wr, 46 op/s 2023-07-21T19:09:05.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:05 smithi156 ceph-mon[128316]: pgmap v47: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 30 MiB/s wr, 46 op/s 2023-07-21T19:09:07.302 INFO:teuthology.orchestra.run.smithi005.stderr:3900000 of 50000000 tuples (7%) done (elapsed 64.86 s, remaining 766.66 s) 2023-07-21T19:09:07.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:07 smithi101 ceph-mon[128136]: pgmap v48: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 38 op/s 2023-07-21T19:09:07.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:07 smithi005 ceph-mon[126460]: pgmap v48: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 38 op/s 2023-07-21T19:09:07.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:07 smithi156 ceph-mon[128316]: pgmap v48: 97 pgs: 97 active+clean; 883 MiB data, 705 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 38 op/s 2023-07-21T19:09:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:4000000 of 50000000 tuples (8%) done (elapsed 66.17 s, remaining 760.94 s) 2023-07-21T19:09:09.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:09 smithi101 ceph-mon[128136]: pgmap v49: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 53 op/s 2023-07-21T19:09:09.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:09 smithi101 ceph-mon[128136]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-21T19:09:09.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:09 smithi005 ceph-mon[126460]: pgmap v49: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 53 op/s 2023-07-21T19:09:09.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:09 smithi005 ceph-mon[126460]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-21T19:09:09.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:09 smithi156 ceph-mon[128316]: pgmap v49: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 35 MiB/s wr, 53 op/s 2023-07-21T19:09:09.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:09 smithi156 ceph-mon[128316]: mgrmap e25: x(active, since 92s), standbys: y, z 2023-07-21T19:09:10.385 INFO:teuthology.orchestra.run.smithi005.stderr:4100000 of 50000000 tuples (8%) done (elapsed 67.94 s, remaining 760.62 s) 2023-07-21T19:09:11.720 INFO:teuthology.orchestra.run.smithi005.stderr:4200000 of 50000000 tuples (8%) done (elapsed 69.28 s, remaining 755.45 s) 2023-07-21T19:09:11.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:11 smithi101 ceph-mon[128136]: pgmap v50: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:09:11.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:11 smithi005 ceph-mon[126460]: pgmap v50: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:09:11.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:11 smithi156 ceph-mon[128316]: pgmap v50: 97 pgs: 97 active+clean; 966 MiB data, 733 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:09:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.297+0000 7f1373fff700 1 -- 172.21.15.5:0/3566447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c105b10 msgr2=0x7f136c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.297+0000 7f1373fff700 1 --2- 172.21.15.5:0/3566447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c105b10 0x7f136c105ee0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1368009990 tx=0x7f1368009ca0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 -- 172.21.15.5:0/3566447 shutdown_connections 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 --2- 172.21.15.5:0/3566447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c1004b0 0x7f136c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 --2- 172.21.15.5:0/3566447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c105b10 0x7f136c105ee0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 --2- 172.21.15.5:0/3566447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c0ffb00 0x7f136c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 -- 172.21.15.5:0/3566447 >> 172.21.15.5:0/3566447 conn(0x7f136c0fb600 msgr2=0x7f136c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 -- 172.21.15.5:0/3566447 shutdown_connections 2023-07-21T19:09:12.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.298+0000 7f1373fff700 1 -- 172.21.15.5:0/3566447 wait complete. 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 Processor -- start 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 -- start start 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 0x7f136c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c105b10 0x7f136c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19cf70 con 0x7f136c1004b0 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19d0e0 con 0x7f136c0ffb00 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.299+0000 7f1373fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f136c19d2a0 con 0x7f136c105b10 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 0x7f136c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42910/0 (socket says 172.21.15.5:42910) 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f1371d9b700 1 -- 172.21.15.5:0/3839474301 learned_addr learned my addr 172.21.15.5:0/3839474301 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:12.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137259c700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c105b10 0x7f136c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 -- 172.21.15.5:0/3839474301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c105b10 msgr2=0x7f136c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c105b10 0x7f136c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 -- 172.21.15.5:0/3839474301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 msgr2=0x7f136c197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 -- 172.21.15.5:0/3839474301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13680095f0 con 0x7f136c1004b0 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f1371d9b700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:12.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.300+0000 7f137159a700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 0x7f136c1982c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9f0 tx=0x7f135c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.301+0000 7f1362ffd700 1 -- 172.21.15.5:0/3839474301 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c00c9c0 con 0x7f136c1004b0 2023-07-21T19:09:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.301+0000 7f1362ffd700 1 -- 172.21.15.5:0/3839474301 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f135c010430 con 0x7f136c1004b0 2023-07-21T19:09:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.301+0000 7f1362ffd700 1 -- 172.21.15.5:0/3839474301 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c018680 con 0x7f136c1004b0 2023-07-21T19:09:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.301+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f136c19d580 con 0x7f136c1004b0 2023-07-21T19:09:12.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.301+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c1a4dc0 con 0x7f136c1004b0 2023-07-21T19:09:12.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.305+0000 7f1362ffd700 1 -- 172.21.15.5:0/3839474301 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f135c0105a0 con 0x7f136c1004b0 2023-07-21T19:09:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.306+0000 7f138791db80 1 Processor -- start 2023-07-21T19:09:12.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 -- start start 2023-07-21T19:09:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6065d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6060040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad60609d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5f65c50 con 0x556ad6060580 2023-07-21T19:09:12.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5fb89e0 con 0x556ad5ebc6b0 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f138791db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5fb1e70 con 0x556ad60662d0 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f137259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6065d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6060040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f137259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6065d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42926/0 (socket says 172.21.15.5:42926) 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.307+0000 7f137259c700 1 -- 172.21.15.5:0/150379145 learned_addr learned my addr 172.21.15.5:0/150379145 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:12.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f137159a700 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad60609d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 -- 172.21.15.5:0/150379145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 msgr2=0x556ad6065d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6065d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 -- 172.21.15.5:0/150379145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 msgr2=0x556ad60609d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad60609d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1362ffd700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 0x7f13580a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 -- 172.21.15.5:0/150379145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ad5ddfd80 con 0x556ad60662d0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1362ffd700 1 -- 172.21.15.5:0/3839474301 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f135c014070 con 0x7f136c1004b0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f1371d9b700 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6060040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1368000c00 tx=0x7f1368005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf4b0d74b397943b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.308+0000 7f137259c700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 0x7f13580a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f1357fff700 1 -- 172.21.15.5:0/150379145 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136801d050 con 0x556ad60662d0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f1357fff700 1 -- 172.21.15.5:0/150379145 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f136801e040 con 0x556ad60662d0 2023-07-21T19:09:12.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f1357fff700 1 -- 172.21.15.5:0/150379145 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680053b0 con 0x556ad60662d0 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f137259c700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 0x7f13580a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13640075c0 tx=0x7f136400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f138791db80 1 -- 172.21.15.5:0/150379145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 msgr2=0x556ad6060040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.309+0000 7f138791db80 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6060040 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1368000c00 tx=0x7f1368005f00 comp rx=0 tx=0).stop 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 -- 172.21.15.5:0/150379145 shutdown_connections 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad60609d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6065d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 --2- 172.21.15.5:0/150379145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6060040 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 -- 172.21.15.5:0/150379145 >> 172.21.15.5:0/150379145 conn(0x556ad6055be0 msgr2=0x556ad605af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 -- 172.21.15.5:0/150379145 shutdown_connections 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 -- 172.21.15.5:0/150379145 wait complete. 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 -- start start 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.310+0000 7f138791db80 1 Processor -- start 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad6069840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad60662d0 0x556ad6069ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5f51a60 con 0x556ad60662d0 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5f51760 con 0x556ad5ebc6b0 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad5e358c0 con 0x556ad6060580 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f138791db80 10 client.0 _ll_register_callbacks cb 0x556ad5f4f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f137259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad60662d0 0x556ad6069ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f137159a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad6069840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42936/0 (socket says 172.21.15.5:42936) 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.311+0000 7f1371d9b700 1 -- 172.21.15.5:0/3716572889 learned_addr learned my addr 172.21.15.5:0/3716572889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 -- 172.21.15.5:0/3716572889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 msgr2=0x556ad6069840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad6069840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 -- 172.21.15.5:0/3716572889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad60662d0 msgr2=0x556ad6069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad60662d0 0x556ad6069ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 -- 172.21.15.5:0/3716572889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f136c10d240 con 0x556ad5ebc6b0 2023-07-21T19:09:12.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f137159a700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad6069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f1371d9b700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1368006010 tx=0x7f1368004e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d2d439ef8b01b67a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f13557fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13680053e0 con 0x556ad5ebc6b0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ad5f0b490 con 0x556ad5ebc6b0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.312+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1368005540 con 0x556ad5ebc6b0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1368005700 con 0x556ad5ebc6b0 2023-07-21T19:09:12.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f138791db80 10 client.25691 fetch_fsmap learned FSMap version 25 2023-07-21T19:09:12.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x556ad5dc8c00 con 0x556ad5ebc6b0 2023-07-21T19:09:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ad5f0b490 con 0x556ad5ebc6b0 2023-07-21T19:09:12.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1368023460 con 0x556ad5ebc6b0 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f138791db80 10 client.25691 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f138791db80 10 client.25691 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.313+0000 7f138791db80 20 client.25691 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad6079650 0x556ad607ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f138791db80 4 client.25691 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ad5faae50 con 0x556ad6079650 2023-07-21T19:09:12.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f137159a700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad6079650 0x556ad607ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f137159a700 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad6079650 0x556ad607ba60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84487805525f93ad server_cookie=2a9c94c4f1647ef4 in_seq=0 out_seq=0 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.314+0000 7f13557fa700 10 client.25691 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f13557fa700 1 -- 172.21.15.5:0/3716572889 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ad5faae50 con 0x556ad6079650 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f13557fa700 10 client.25691 handle_command_reply: tid=0 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 client.25691 shutdown 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 msgr2=0x556ad606c5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1368006010 tx=0x7f1368004e90 comp rx=0 tx=0).stop 2023-07-21T19:09:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 shutdown_connections 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad6079650 0x556ad607ba60 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad60662d0 0x556ad6069ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f13557fa700 0 client.25691 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad606c5c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 --2- 172.21.15.5:0/3716572889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad6069840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 >> 172.21.15.5:0/3716572889 conn(0x556ad6055be0 msgr2=0x556ad6059c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 shutdown_connections 2023-07-21T19:09:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.316+0000 7f138791db80 1 -- 172.21.15.5:0/3716572889 wait complete. 2023-07-21T19:09:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.317+0000 7f138791db80 20 client.25691 trim_cache size 0 max 16384 2023-07-21T19:09:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.335+0000 7f138791db80 1 Processor -- start 2023-07-21T19:09:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.335+0000 7f138791db80 1 -- start start 2023-07-21T19:09:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 0x556ad6070bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad606c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad606ae90 con 0x556ad5ebc6b0 2023-07-21T19:09:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad6061000 con 0x556ad60662d0 2023-07-21T19:09:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f138791db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad6067df0 con 0x556ad6060580 2023-07-21T19:09:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 0x556ad6070bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f137259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f137259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42944/0 (socket says 172.21.15.5:42944) 2023-07-21T19:09:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.336+0000 7f137259c700 1 -- 172.21.15.5:0/353229517 learned_addr learned my addr 172.21.15.5:0/353229517 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137159a700 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad606c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 -- 172.21.15.5:0/353229517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 msgr2=0x556ad606c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad606c160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 -- 172.21.15.5:0/353229517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 msgr2=0x556ad6070bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 0x556ad6070bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 -- 172.21.15.5:0/353229517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ad5e061f0 con 0x556ad60662d0 2023-07-21T19:09:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f1371d9b700 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 0x556ad6070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.337+0000 7f137259c700 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13640167a0 tx=0x7f1364009050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fcebbfd1c02134 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.338+0000 7f1360ff9700 1 -- 172.21.15.5:0/353229517 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f136400e040 con 0x556ad60662d0 2023-07-21T19:09:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.338+0000 7f1360ff9700 1 -- 172.21.15.5:0/353229517 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f136400f040 con 0x556ad60662d0 2023-07-21T19:09:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.338+0000 7f1360ff9700 1 -- 172.21.15.5:0/353229517 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13640053a0 con 0x556ad60662d0 2023-07-21T19:09:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- 172.21.15.5:0/353229517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 msgr2=0x556ad605ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13640167a0 tx=0x7f1364009050 comp rx=0 tx=0).stop 2023-07-21T19:09:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- 172.21.15.5:0/353229517 shutdown_connections 2023-07-21T19:09:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad60662d0 0x556ad605ac60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad5ebc6b0 0x556ad6070bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 --2- 172.21.15.5:0/353229517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad6060580 0x556ad606c160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- 172.21.15.5:0/353229517 >> 172.21.15.5:0/353229517 conn(0x556ad6055be0 msgr2=0x556ad6063a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- 172.21.15.5:0/353229517 shutdown_connections 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- 172.21.15.5:0/353229517 wait complete. 2023-07-21T19:09:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.339+0000 7f138791db80 1 -- start start 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 Processor -- start 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6062960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad6062ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6061790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad60634c0 con 0x556ad6060580 2023-07-21T19:09:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad6063630 con 0x556ad5ebc6b0 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ad6061d50 con 0x556ad60662d0 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.340+0000 7f138791db80 10 client.0 _ll_register_callbacks cb 0x7f1364009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6062960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad6062ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f1371d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6062960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42952/0 (socket says 172.21.15.5:42952) 2023-07-21T19:09:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f1371d9b700 1 -- 172.21.15.5:0/2579413621 learned_addr learned my addr 172.21.15.5:0/2579413621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137259c700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6061790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 -- 172.21.15.5:0/2579413621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 msgr2=0x556ad6061790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6061790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 -- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 msgr2=0x556ad6062960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6062960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 -- 172.21.15.5:0/2579413621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ad5dc8c00 con 0x556ad6060580 2023-07-21T19:09:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f137159a700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad6062ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ca510 tx=0x7f135c0caa70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a087498aa1bfd363 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.341+0000 7f13557fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:09:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f135c0999d0 con 0x556ad6060580 2023-07-21T19:09:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f135c010880 con 0x556ad6060580 2023-07-21T19:09:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f136801e040 con 0x556ad6060580 2023-07-21T19:09:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f135c0997f0 con 0x556ad6060580 2023-07-21T19:09:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f138791db80 10 client.15963 fetch_fsmap learned FSMap version 25 2023-07-21T19:09:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x7f1364016480 con 0x556ad6060580 2023-07-21T19:09:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.342+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f136801e040 con 0x556ad6060580 2023-07-21T19:09:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 25) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f135c099380 con 0x556ad6060580 2023-07-21T19:09:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 10 client.15963 fetch_fsmap finished waiting for FSMap version 25 2023-07-21T19:09:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 10 client.15963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 20 client.15963 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad60ce230 0x556ad60d0640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:12.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 4 client.15963 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:09:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ad5f6cff0 con 0x556ad60ce230 2023-07-21T19:09:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.343+0000 7f1371d9b700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad60ce230 0x556ad60d0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.344+0000 7f1371d9b700 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad60ce230 0x556ad60d0640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c90b2bead50962 server_cookie=8c6bb8cdb6b6ef7b in_seq=0 out_seq=0 2023-07-21T19:09:12.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.344+0000 7f13557fa700 10 client.15963 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.358+0000 7f13557fa700 1 -- 172.21.15.5:0/2579413621 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ad5f6cff0 con 0x556ad60ce230 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.358+0000 7f13557fa700 10 client.15963 handle_command_reply: tid=0 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.358+0000 7f138791db80 1 client.15963 shutdown 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.358+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 msgr2=0x556ad6062ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.358+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad6062ea0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f135c0ca510 tx=0x7f135c0caa70 comp rx=0 tx=0).stop 2023-07-21T19:09:12.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 shutdown_connections 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ad60662d0 0x556ad6061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ad5ebc6b0 0x556ad6062960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ad60ce230 0x556ad60d0640 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 --2- 172.21.15.5:0/2579413621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ad6060580 0x556ad6062ea0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 >> 172.21.15.5:0/2579413621 conn(0x556ad6055be0 msgr2=0x556ad60649f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f13557fa700 0 client.15963 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:12.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 shutdown_connections 2023-07-21T19:09:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 1 -- 172.21.15.5:0/2579413621 wait complete. 2023-07-21T19:09:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.359+0000 7f138791db80 20 client.15963 trim_cache size 0 max 16384 2023-07-21T19:09:12.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:09:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:09:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:09:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:09:12.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 msgr2=0x7f13580a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 0x7f13580a37f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f13640075c0 tx=0x7f136400b040 comp rx=0 tx=0).stop 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 msgr2=0x7f136c1982c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 0x7f136c1982c0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f135c00e9f0 tx=0x7f135c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 shutdown_connections 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f136c105b10 0x7f136c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f136c0ffb00 0x7f136c197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13580a1340 0x7f13580a37f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 --2- 172.21.15.5:0/3839474301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f136c1004b0 0x7f136c1982c0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 >> 172.21.15.5:0/3839474301 conn(0x7f136c0fb600 msgr2=0x7f136c10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 shutdown_connections 2023-07-21T19:09:12.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:12.360+0000 7f1373fff700 1 -- 172.21.15.5:0/3839474301 wait complete. 2023-07-21T19:09:13.611 INFO:teuthology.orchestra.run.smithi005.stderr:4300000 of 50000000 tuples (8%) done (elapsed 71.17 s, remaining 756.36 s) 2023-07-21T19:09:13.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:13 smithi101 ceph-mon[128136]: pgmap v51: 97 pgs: 97 active+clean; 1001 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 29 MiB/s wr, 47 op/s 2023-07-21T19:09:13.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:13 smithi005 ceph-mon[126460]: pgmap v51: 97 pgs: 97 active+clean; 1001 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 29 MiB/s wr, 47 op/s 2023-07-21T19:09:13.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:13 smithi156 ceph-mon[128316]: pgmap v51: 97 pgs: 97 active+clean; 1001 MiB data, 753 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 29 MiB/s wr, 47 op/s 2023-07-21T19:09:15.488 INFO:teuthology.orchestra.run.smithi005.stderr:4400000 of 50000000 tuples (8%) done (elapsed 73.04 s, remaining 757.01 s) 2023-07-21T19:09:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:15 smithi101 ceph-mon[128136]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 44 op/s 2023-07-21T19:09:15.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:15 smithi005 ceph-mon[126460]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 44 op/s 2023-07-21T19:09:15.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:15 smithi156 ceph-mon[128316]: pgmap v52: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 44 op/s 2023-07-21T19:09:16.806 INFO:teuthology.orchestra.run.smithi005.stderr:4500000 of 50000000 tuples (9%) done (elapsed 74.36 s, remaining 751.89 s) 2023-07-21T19:09:17.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:17 smithi101 ceph-mon[128136]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:09:17.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:17 smithi005 ceph-mon[126460]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:09:17.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:17 smithi156 ceph-mon[128316]: pgmap v53: 97 pgs: 97 active+clean; 1.0 GiB data, 765 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:09:18.720 INFO:teuthology.orchestra.run.smithi005.stderr:4600000 of 50000000 tuples (9%) done (elapsed 76.28 s, remaining 752.81 s) 2023-07-21T19:09:19.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:19 smithi101 ceph-mon[128136]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-07-21T19:09:19.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:19 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:09:19.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:19 smithi005 ceph-mon[126460]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-07-21T19:09:19.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:19 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:09:19.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:19 smithi156 ceph-mon[128316]: pgmap v54: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 34 MiB/s wr, 38 op/s 2023-07-21T19:09:19.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:19 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:09:20.784 INFO:teuthology.orchestra.run.smithi005.stderr:4700000 of 50000000 tuples (9%) done (elapsed 78.34 s, remaining 755.07 s) 2023-07-21T19:09:21.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:21 smithi101 ceph-mon[128136]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:09:21.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:21 smithi005 ceph-mon[126460]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:09:21.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:21 smithi156 ceph-mon[128316]: pgmap v55: 97 pgs: 97 active+clean; 1.1 GiB data, 796 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:09:22.025 INFO:teuthology.orchestra.run.smithi005.stderr:4800000 of 50000000 tuples (9%) done (elapsed 79.58 s, remaining 749.39 s) 2023-07-21T19:09:23.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:23 smithi101 ceph-mon[128136]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 29 op/s 2023-07-21T19:09:23.898 INFO:teuthology.orchestra.run.smithi005.stderr:4900000 of 50000000 tuples (9%) done (elapsed 81.45 s, remaining 749.71 s) 2023-07-21T19:09:23.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:23 smithi005 ceph-mon[126460]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 29 op/s 2023-07-21T19:09:23.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:23 smithi156 ceph-mon[128316]: pgmap v56: 97 pgs: 97 active+clean; 1.1 GiB data, 813 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 30 MiB/s wr, 29 op/s 2023-07-21T19:09:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 -- 172.21.15.5:0/306940271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 msgr2=0x7f28cc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 --2- 172.21.15.5:0/306940271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc067fe0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009990 tx=0x7f28c8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 -- 172.21.15.5:0/306940271 shutdown_connections 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 --2- 172.21.15.5:0/306940271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc104e50 0x7f28cc10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 --2- 172.21.15.5:0/306940271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc067fe0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 --2- 172.21.15.5:0/306940271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc0685b0 0x7f28cc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 -- 172.21.15.5:0/306940271 >> 172.21.15.5:0/306940271 conn(0x7f28cc074760 msgr2=0x7f28cc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 -- 172.21.15.5:0/306940271 shutdown_connections 2023-07-21T19:09:25.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.523+0000 7f28d37ff700 1 -- 172.21.15.5:0/306940271 wait complete. 2023-07-21T19:09:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.524+0000 7f28d37ff700 1 Processor -- start 2023-07-21T19:09:25.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.524+0000 7f28d37ff700 1 -- start start 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc19baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 0x7f28cc195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc195fc0 con 0x7f28cc104e50 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc196130 con 0x7f28cc0685b0 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc1962f0 con 0x7f28cc067c10 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc19baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d0d9a700 1 -- 172.21.15.5:0/3929796040 learned_addr learned my addr 172.21.15.5:0/3929796040 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.525+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 0x7f28cc195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 -- 172.21.15.5:0/3929796040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 msgr2=0x7f28cc19baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc19baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 -- 172.21.15.5:0/3929796040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 msgr2=0x7f28cc19c030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 -- 172.21.15.5:0/3929796040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c80095f0 con 0x7f28cc104e50 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 0x7f28cc195950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28c400efc0 tx=0x7f28c400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28c27fc700 1 -- 172.21.15.5:0/3929796040 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c4018070 con 0x7f28cc104e50 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc196570 con 0x7f28cc104e50 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28c27fc700 1 -- 172.21.15.5:0/3929796040 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c400f040 con 0x7f28cc104e50 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28c27fc700 1 -- 172.21.15.5:0/3929796040 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400cba0 con 0x7f28cc104e50 2023-07-21T19:09:25.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.526+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc1a2ae0 con 0x7f28cc104e50 2023-07-21T19:09:25.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.530+0000 7f28c27fc700 1 -- 172.21.15.5:0/3929796040 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f28c4003a00 con 0x7f28cc104e50 2023-07-21T19:09:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.531+0000 7f28e71c6b80 1 Processor -- start 2023-07-21T19:09:25.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.531+0000 7f28c27fc700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 0x7f28b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.531+0000 7f28e71c6b80 1 -- start start 2023-07-21T19:09:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28c27fc700 1 -- 172.21.15.5:0/3929796040 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f28c40cc7d0 con 0x7f28cc104e50 2023-07-21T19:09:25.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 0x7f28b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a324b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a46cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492937c50 con 0x563492a329f0 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56349298a9e0 con 0x56349288e6b0 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492983e70 con 0x563492a32040 2023-07-21T19:09:25.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d159b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d159b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35706/0 (socket says 172.21.15.5:35706) 2023-07-21T19:09:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d159b700 1 -- 172.21.15.5:0/2670731165 learned_addr learned my addr 172.21.15.5:0/2670731165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a46cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.532+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a324b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 -- 172.21.15.5:0/2670731165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 msgr2=0x563492a324b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a324b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 -- 172.21.15.5:0/2670731165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 msgr2=0x563492a46cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a46cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 -- 172.21.15.5:0/2670731165 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634927b1d80 con 0x56349288e6b0 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 0x7f28b80a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f28cc1063c0 tx=0x7f28bc013cd0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28d159b700 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c8015000 tx=0x7f28c8005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67097a7ea26dd600 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28b77fe700 1 -- 172.21.15.5:0/2670731165 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801d070 con 0x56349288e6b0 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28b77fe700 1 -- 172.21.15.5:0/2670731165 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c801e040 con 0x56349288e6b0 2023-07-21T19:09:25.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.533+0000 7f28b77fe700 1 -- 172.21.15.5:0/2670731165 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8004760 con 0x56349288e6b0 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 -- 172.21.15.5:0/2670731165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 msgr2=0x563492a35de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f28c8015000 tx=0x7f28c8005e80 comp rx=0 tx=0).stop 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 -- 172.21.15.5:0/2670731165 shutdown_connections 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a46cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56349288e6b0 0x563492a35de0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/2670731165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a324b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 -- 172.21.15.5:0/2670731165 >> 172.21.15.5:0/2670731165 conn(0x563492a27be0 msgr2=0x563492a2cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 -- 172.21.15.5:0/2670731165 shutdown_connections 2023-07-21T19:09:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.534+0000 7f28e71c6b80 1 -- 172.21.15.5:0/2670731165 wait complete. 2023-07-21T19:09:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.535+0000 7f28e71c6b80 1 -- start start 2023-07-21T19:09:25.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.535+0000 7f28e71c6b80 1 Processor -- start 2023-07-21T19:09:25.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349288e6b0 0x563492a46b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a44360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492923a60 con 0x56349288e6b0 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492923760 con 0x563492a329f0 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634928078c0 con 0x563492a32040 2023-07-21T19:09:25.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28e71c6b80 10 client.0 _ll_register_callbacks cb 0x563492921700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57438/0 (socket says 172.21.15.5:57438) 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 -- 172.21.15.5:0/1820350896 learned_addr learned my addr 172.21.15.5:0/1820350896 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a44360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d159b700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349288e6b0 0x563492a46b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 -- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 msgr2=0x563492a44360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a44360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 -- 172.21.15.5:0/1820350896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349288e6b0 msgr2=0x563492a46b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349288e6b0 0x563492a46b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.536+0000 7f28d0d9a700 1 -- 172.21.15.5:0/1820350896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc10d210 con 0x563492a32040 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00fce0 tx=0x7f28bc0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=162e1e4302d160c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28bc010dc0 con 0x563492a32040 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5634928dd490 con 0x563492a32040 2023-07-21T19:09:25.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f28bc020430 con 0x563492a32040 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28bc0205f0 con 0x563492a32040 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28e71c6b80 10 client.25684 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.537+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x56349279ac00 con 0x563492a32040 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5634928dd490 con 0x563492a32040 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28bc00e5e0 con 0x563492a32040 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 10 client.25684 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 10 client.25684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:25.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 20 client.25684 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492a54300 0x563492a56710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 4 client.25684 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.538+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56349297ce50 con 0x563492a54300 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.539+0000 7f28d159b700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492a54300 0x563492a56710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.539+0000 7f28d159b700 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492a54300 0x563492a56710 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f56979e369e07421 server_cookie=bea205d983d6ce1a in_seq=0 out_seq=0 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.539+0000 7f28b4ff9700 10 client.25684 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1820350896 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56349297ce50 con 0x563492a54300 2023-07-21T19:09:25.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28b4ff9700 10 client.25684 handle_command_reply: tid=0 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 client.25684 shutdown 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 msgr2=0x563492a43da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f28bc00fce0 tx=0x7f28bc0047c0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 shutdown_connections 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a44360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492a54300 0x563492a56710 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56349288e6b0 0x563492a46b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1820350896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563492a32040 0x563492a43da0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28b4ff9700 0 client.25684 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.540+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 >> 172.21.15.5:0/1820350896 conn(0x563492a27be0 msgr2=0x563492a2c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.541+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 shutdown_connections 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.541+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1820350896 wait complete. 2023-07-21T19:09:25.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.541+0000 7f28e71c6b80 20 client.25684 trim_cache size 0 max 16384 2023-07-21T19:09:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.557+0000 7f28e71c6b80 1 Processor -- start 2023-07-21T19:09:25.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.557+0000 7f28e71c6b80 1 -- start start 2023-07-21T19:09:25.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a43640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5634929f8060 con 0x563492a32040 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a331e0 con 0x563492a329f0 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492a29d00 con 0x56349288e6b0 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d0d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41830/0 (socket says 172.21.15.5:41830) 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57444/0 (socket says 172.21.15.5:57444) 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d0d9a700 1 -- 172.21.15.5:0/3702356736 learned_addr learned my addr 172.21.15.5:0/3702356736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 -- 172.21.15.5:0/3702356736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 msgr2=0x563492a43640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a43640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 -- 172.21.15.5:0/3702356736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 msgr2=0x563492a466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d159b700 1 -- 172.21.15.5:0/3702356736 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5634927d81f0 con 0x56349288e6b0 2023-07-21T19:09:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.558+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:25.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.559+0000 7f28d159b700 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f28c80098e0 tx=0x7f28c801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1969df1b85c5ec2b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.559+0000 7f28b7fff700 1 -- 172.21.15.5:0/3702356736 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801d040 con 0x56349288e6b0 2023-07-21T19:09:25.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.559+0000 7f28b7fff700 1 -- 172.21.15.5:0/3702356736 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c800b2d0 con 0x56349288e6b0 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.559+0000 7f28b7fff700 1 -- 172.21.15.5:0/3702356736 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c8033b50 con 0x56349288e6b0 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- 172.21.15.5:0/3702356736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 msgr2=0x563492a4b3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f28c80098e0 tx=0x7f28c801a040 comp rx=0 tx=0).stop 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- 172.21.15.5:0/3702356736 shutdown_connections 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a329f0 0x563492a43640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a4b3d0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/3702356736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a32040 0x563492a466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- 172.21.15.5:0/3702356736 >> 172.21.15.5:0/3702356736 conn(0x563492a27be0 msgr2=0x563492a3f0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- 172.21.15.5:0/3702356736 shutdown_connections 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- 172.21.15.5:0/3702356736 wait complete. 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 -- start start 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.560+0000 7f28e71c6b80 1 Processor -- start 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a32040 0x563492a498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a49e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492aa72d0 con 0x563492a329f0 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492aa7440 con 0x563492a32040 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563492aa7600 con 0x56349288e6b0 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28e71c6b80 10 client.0 _ll_register_callbacks cb 0x7f28bc00f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28d1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a49e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28d159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28d159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57458/0 (socket says 172.21.15.5:57458) 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28d159b700 1 -- 172.21.15.5:0/1782794451 learned_addr learned my addr 172.21.15.5:0/1782794451 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.561+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a32040 0x563492a498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 -- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a32040 msgr2=0x563492a498f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a32040 0x563492a498f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 -- 172.21.15.5:0/1782794451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 msgr2=0x563492a49e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a49e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 -- 172.21.15.5:0/1782794451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56349279ac00 con 0x56349288e6b0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d1d9c700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28d159b700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009350 tx=0x7f28c800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12ecab1413f504d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28b4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c801d040 con 0x56349288e6b0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f28c8005560 con 0x56349288e6b0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.562+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f28c801e040 con 0x56349288e6b0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f28c800bb00 con 0x56349288e6b0 2023-07-21T19:09:25.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 10 client.25690 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5634927b1d80 con 0x56349288e6b0 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f28c801e040 con 0x56349288e6b0 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f28c800f4f0 con 0x56349288e6b0 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 10 client.25690 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 10 client.25690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 20 client.25690 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:25.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492aa8990 0x563492aaada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 4 client.25690 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:09:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.563+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56349293eff0 con 0x563492aa8990 2023-07-21T19:09:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.564+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492aa8990 0x563492aaada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.564+0000 7f28d0d9a700 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492aa8990 0x563492aaada0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b42f1eea3273b54d server_cookie=4a95c3983265b8e in_seq=0 out_seq=0 2023-07-21T19:09:25.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.564+0000 7f28b4ff9700 10 client.25690 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28b4ff9700 1 -- 172.21.15.5:0/1782794451 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56349293eff0 con 0x563492aa8990 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28b4ff9700 10 client.25690 handle_command_reply: tid=0 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 client.25690 shutdown 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 msgr2=0x563492a493b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f28c8009350 tx=0x7f28c800b8a0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 shutdown_connections 2023-07-21T19:09:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563492aa8990 0x563492aaada0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563492a329f0 0x563492a49e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28b4ff9700 0 client.25690 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56349288e6b0 0x563492a493b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 --2- 172.21.15.5:0/1782794451 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563492a32040 0x563492a498f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.583+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 >> 172.21.15.5:0/1782794451 conn(0x563492a27be0 msgr2=0x563492a43640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.584+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 shutdown_connections 2023-07-21T19:09:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.584+0000 7f28e71c6b80 1 -- 172.21.15.5:0/1782794451 wait complete. 2023-07-21T19:09:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.584+0000 7f28e71c6b80 20 client.25690 trim_cache size 0 max 16384 2023-07-21T19:09:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:09:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:09:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:09:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:09:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.584+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 msgr2=0x7f28b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.584+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 0x7f28b80a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f28cc1063c0 tx=0x7f28bc013cd0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 msgr2=0x7f28cc195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 0x7f28cc195950 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f28c400efc0 tx=0x7f28c400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 shutdown_connections 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f28b80a12c0 0x7f28b80a3770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f28cc104e50 0x7f28cc195950 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f28cc067c10 0x7f28cc19baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 --2- 172.21.15.5:0/3929796040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f28cc0685b0 0x7f28cc19c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 >> 172.21.15.5:0/3929796040 conn(0x7f28cc074760 msgr2=0x7f28cc0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 shutdown_connections 2023-07-21T19:09:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:25.585+0000 7f28d37ff700 1 -- 172.21.15.5:0/3929796040 wait complete. 2023-07-21T19:09:25.801 INFO:teuthology.orchestra.run.smithi005.stderr:5000000 of 50000000 tuples (10%) done (elapsed 83.36 s, remaining 750.22 s) 2023-07-21T19:09:25.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:25 smithi101 ceph-mon[128136]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 30 op/s 2023-07-21T19:09:25.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:25 smithi005 ceph-mon[126460]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 30 op/s 2023-07-21T19:09:25.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:25 smithi156 ceph-mon[128316]: pgmap v57: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 27 MiB/s wr, 30 op/s 2023-07-21T19:09:27.121 INFO:teuthology.orchestra.run.smithi005.stderr:5100000 of 50000000 tuples (10%) done (elapsed 84.68 s, remaining 745.49 s) 2023-07-21T19:09:27.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:27 smithi101 ceph-mon[128136]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:27.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:27 smithi005 ceph-mon[126460]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:27.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:27 smithi156 ceph-mon[128316]: pgmap v58: 97 pgs: 97 active+clean; 1.1 GiB data, 820 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:5200000 of 50000000 tuples (10%) done (elapsed 86.57 s, remaining 745.79 s) 2023-07-21T19:09:29.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:29 smithi101 ceph-mon[128136]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-07-21T19:09:29.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:29 smithi005 ceph-mon[126460]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-07-21T19:09:29.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:29 smithi156 ceph-mon[128316]: pgmap v59: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 40 op/s 2023-07-21T19:09:31.007 INFO:teuthology.orchestra.run.smithi005.stderr:5300000 of 50000000 tuples (10%) done (elapsed 88.56 s, remaining 746.94 s) 2023-07-21T19:09:31.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:31 smithi101 ceph-mon[128136]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:31.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:31 smithi005 ceph-mon[126460]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:31.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:31 smithi156 ceph-mon[128316]: pgmap v60: 97 pgs: 97 active+clean; 1.2 GiB data, 851 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:5400000 of 50000000 tuples (10%) done (elapsed 89.87 s, remaining 742.25 s) 2023-07-21T19:09:33.655 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:09:33.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:33 smithi101 ceph-mon[128136]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 -- 172.21.15.5:0/1829196900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd898105b10 msgr2=0x7fd898105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/1829196900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd898105b10 0x7fd898105ee0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fd888009990 tx=0x7fd888009ca0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 -- 172.21.15.5:0/1829196900 shutdown_connections 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/1829196900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8981004b0 0x7fd89810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/1829196900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd898105b10 0x7fd898105ee0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/1829196900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd8980ffb00 0x7fd8980fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 -- 172.21.15.5:0/1829196900 >> 172.21.15.5:0/1829196900 conn(0x7fd8980fb600 msgr2=0x7fd8980fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 -- 172.21.15.5:0/1829196900 shutdown_connections 2023-07-21T19:09:33.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.880+0000 7fd89e5ff700 1 -- 172.21.15.5:0/1829196900 wait complete. 2023-07-21T19:09:33.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.881+0000 7fd89e5ff700 1 Processor -- start 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.881+0000 7fd89e5ff700 1 -- start start 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981004b0 0x7fd89819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd898105b10 0x7fd898197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981981b0 con 0x7fd8981004b0 2023-07-21T19:09:33.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd898198320 con 0x7fd8980ffb00 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8981984e0 con 0x7fd898105b10 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36420/0 (socket says 172.21.15.5:36420) 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd897fff700 1 -- 172.21.15.5:0/3636589018 learned_addr learned my addr 172.21.15.5:0/3636589018 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd8977fe700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981004b0 0x7fd89819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.882+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd898105b10 0x7fd898197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 -- 172.21.15.5:0/3636589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd898105b10 msgr2=0x7fd898197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd898105b10 0x7fd898197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 -- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981004b0 msgr2=0x7fd89819e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981004b0 0x7fd89819e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 -- 172.21.15.5:0/3636589018 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8880095f0 con 0x7fd8980ffb00 2023-07-21T19:09:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd897fff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd888005910 tx=0x7fd8880047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd8957fa700 1 -- 172.21.15.5:0/3636589018 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88801d070 con 0x7fd8980ffb00 2023-07-21T19:09:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd8957fa700 1 -- 172.21.15.5:0/3636589018 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd888021410 con 0x7fd8980ffb00 2023-07-21T19:09:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd8957fa700 1 -- 172.21.15.5:0/3636589018 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd888017650 con 0x7fd8980ffb00 2023-07-21T19:09:33.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd898198760 con 0x7fd8980ffb00 2023-07-21T19:09:33.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.883+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981a4ac0 con 0x7fd8980ffb00 2023-07-21T19:09:33.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.887+0000 7fd8957fa700 1 -- 172.21.15.5:0/3636589018 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd888021580 con 0x7fd8980ffb00 2023-07-21T19:09:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.889+0000 7fd8b1ed3b80 1 Processor -- start 2023-07-21T19:09:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.889+0000 7fd8b1ed3b80 1 -- start start 2023-07-21T19:09:33.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.889+0000 7fd8957fa700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 0x7fd8800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.889+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f90ead0 0x555e4f910eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8957fa700 1 -- 172.21.15.5:0/3636589018 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd8880cde00 con 0x7fd8980ffb00 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f809cb0 con 0x555e4f90ead0 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f85ca40 con 0x555e4f9032d0 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f855ed0 con 0x555e4f903be0 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36426/0 (socket says 172.21.15.5:36426) 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd8977fe700 1 -- 172.21.15.5:0/4091952012 learned_addr learned my addr 172.21.15.5:0/4091952012 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 0x7fd8800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f90ead0 0x555e4f910eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 -- 172.21.15.5:0/4091952012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 msgr2=0x555e4f90e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 -- 172.21.15.5:0/4091952012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f90ead0 msgr2=0x555e4f910eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f90ead0 0x555e4f910eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.890+0000 7fd897fff700 1 -- 172.21.15.5:0/4091952012 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4f683de0 con 0x555e4f9032d0 2023-07-21T19:09:33.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.891+0000 7fd897fff700 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cb580 tx=0x7fd888096cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a4e9d33552c2e9e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.891+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 0x7fd8800a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd890009ec0 tx=0x7fd890009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.891+0000 7fd87e7fc700 1 -- 172.21.15.5:0/4091952012 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88809af90 con 0x555e4f9032d0 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.891+0000 7fd87e7fc700 1 -- 172.21.15.5:0/4091952012 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd88809efc0 con 0x555e4f9032d0 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.891+0000 7fd87e7fc700 1 -- 172.21.15.5:0/4091952012 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88809a480 con 0x555e4f9032d0 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/4091952012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 msgr2=0x555e4f9036a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cb580 tx=0x7fd888096cc0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/4091952012 shutdown_connections 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f90ead0 0x555e4f910eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f9032d0 0x555e4f9036a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/4091952012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/4091952012 >> 172.21.15.5:0/4091952012 conn(0x555e4f8f9da0 msgr2=0x555e4f8ff6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/4091952012 shutdown_connections 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.892+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/4091952012 wait complete. 2023-07-21T19:09:33.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 -- start start 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 Processor -- start 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f90dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f90ead0 0x555e4f912680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f7f5ac0 con 0x555e4f9032d0 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f7f57c0 con 0x555e4f90ead0 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.893+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f6d9920 con 0x555e4f903be0 2023-07-21T19:09:33.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8b1ed3b80 10 client.0 _ll_register_callbacks cb 0x555e4f7f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd897fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f90dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd89cb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f90ead0 0x555e4f912680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59624/0 (socket says 172.21.15.5:59624) 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 -- 172.21.15.5:0/2934773930 learned_addr learned my addr 172.21.15.5:0/2934773930 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 -- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f90ead0 msgr2=0x555e4f912680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f90ead0 0x555e4f912680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 -- 172.21.15.5:0/2934773930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 msgr2=0x555e4f90dbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f90dbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.894+0000 7fd8977fe700 1 -- 172.21.15.5:0/2934773930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8981a4ac0 con 0x555e4f903be0 2023-07-21T19:09:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8977fe700 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd88c012740 tx=0x7fd88c012a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d92873f6791ed06 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8777fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:09:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c017070 con 0x555e4f903be0 2023-07-21T19:09:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e4f7af4f0 con 0x555e4f903be0 2023-07-21T19:09:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd88c010430 con 0x555e4f903be0 2023-07-21T19:09:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88c0122e0 con 0x555e4f903be0 2023-07-21T19:09:33.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8b1ed3b80 10 client.25702 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.895+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555e4f66cc60 con 0x555e4f903be0 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555e4f7af4f0 con 0x555e4f903be0 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd88c01c720 con 0x555e4f903be0 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 10 client.25702 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 10 client.25702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 20 client.25702 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:33.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f91dfb0 0x555e4f9203c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 4 client.25702 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e4f84eeb0 con 0x555e4f91dfb0 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.896+0000 7fd897fff700 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f91dfb0 0x555e4f9203c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.897+0000 7fd897fff700 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f91dfb0 0x555e4f9203c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c516939586135d2 server_cookie=408413f4bc652af3 in_seq=0 out_seq=0 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.897+0000 7fd8777fe700 10 client.25702 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8777fe700 1 -- 172.21.15.5:0/2934773930 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555e4f84eeb0 con 0x555e4f91dfb0 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8777fe700 10 client.25702 handle_command_reply: tid=0 2023-07-21T19:09:33.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 client.25702 shutdown 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 msgr2=0x555e4f90e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fd88c012740 tx=0x7fd88c012a50 comp rx=0 tx=0).stop 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 shutdown_connections 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f90ead0 0x555e4f912680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f91dfb0 0x555e4f9203c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f90dbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/2934773930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e110 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8777fe700 0 client.25702 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 >> 172.21.15.5:0/2934773930 conn(0x555e4f8f9da0 msgr2=0x555e4f8ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 shutdown_connections 2023-07-21T19:09:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.898+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/2934773930 wait complete. 2023-07-21T19:09:33.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.899+0000 7fd8b1ed3b80 20 client.25702 trim_cache size 0 max 16384 2023-07-21T19:09:33.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:33 smithi005 ceph-mon[126460]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:33.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:33 smithi156 ceph-mon[128316]: pgmap v61: 97 pgs: 97 active+clean; 1.3 GiB data, 879 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:33.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.919+0000 7fd8b1ed3b80 1 Processor -- start 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 -- start start 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f912670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f90f320 con 0x555e4f9032d0 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f90e910 con 0x555e4f8f9da0 2023-07-21T19:09:33.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f8f9970 con 0x555e4f903be0 2023-07-21T19:09:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.920+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36444/0 (socket says 172.21.15.5:36444) 2023-07-21T19:09:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 -- 172.21.15.5:0/3035687094 learned_addr learned my addr 172.21.15.5:0/3035687094 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:33.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd8977fe700 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f912670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 -- 172.21.15.5:0/3035687094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 msgr2=0x555e4f90e060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 -- 172.21.15.5:0/3035687094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 msgr2=0x555e4f912670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f912670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 -- 172.21.15.5:0/3035687094 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e4f8fb5e0 con 0x555e4f8f9da0 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.921+0000 7fd897fff700 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cb5e0 tx=0x7fd88800f8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43c187861e4c0ad5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.922+0000 7fd87effd700 1 -- 172.21.15.5:0/3035687094 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88809b010 con 0x555e4f8f9da0 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.922+0000 7fd87effd700 1 -- 172.21.15.5:0/3035687094 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd888021860 con 0x555e4f8f9da0 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.922+0000 7fd87effd700 1 -- 172.21.15.5:0/3035687094 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88809b450 con 0x555e4f8f9da0 2023-07-21T19:09:33.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.922+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3035687094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 msgr2=0x555e4f8ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.922+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd8880cb5e0 tx=0x7fd88800f8b0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3035687094 shutdown_connections 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f903be0 0x555e4f90e060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f8f9da0 0x555e4f8ff850 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3035687094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f912670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3035687094 >> 172.21.15.5:0/3035687094 conn(0x555e4f8fe170 msgr2=0x555e4f8ff140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3035687094 shutdown_connections 2023-07-21T19:09:33.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.923+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3035687094 wait complete. 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.924+0000 7fd8b1ed3b80 1 -- start start 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.924+0000 7fd8b1ed3b80 1 Processor -- start 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f8f9da0 0x555e4f91a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f903be0 0x555e4f916cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f9173e0 con 0x555e4f9032d0 2023-07-21T19:09:33.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f917550 con 0x555e4f903be0 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e4f97a6a0 con 0x555e4f8f9da0 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8b1ed3b80 10 client.0 _ll_register_callbacks cb 0x555e4f7f3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8977fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41792/0 (socket says 172.21.15.5:41792) 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.925+0000 7fd8977fe700 1 -- 172.21.15.5:0/3126600607 learned_addr learned my addr 172.21.15.5:0/3126600607 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd89cb9c700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f903be0 0x555e4f916cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd8977fe700 1 -- 172.21.15.5:0/3126600607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f8f9da0 msgr2=0x555e4f91a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd897fff700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f8f9da0 0x555e4f91a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd8977fe700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f8f9da0 0x555e4f91a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd8977fe700 1 -- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f903be0 msgr2=0x555e4f916cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd8977fe700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f903be0 0x555e4f916cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.926+0000 7fd8977fe700 1 -- 172.21.15.5:0/3126600607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555e4f683de0 con 0x555e4f9032d0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.927+0000 7fd8977fe700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005a80 tx=0x7fd88c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0cc0534c1ba845 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.927+0000 7fd8777fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.927+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd88c017040 con 0x555e4f9032d0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.927+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd88809efc0 con 0x555e4f9032d0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.927+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fd88c005310 con 0x555e4f9032d0 2023-07-21T19:09:33.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd88c0054f0 con 0x555e4f9032d0 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 10 client.15999 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555e4f6aa250 con 0x555e4f9032d0 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd88809efc0 con 0x555e4f9032d0 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd88c025450 con 0x555e4f9032d0 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 10 client.15999 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 10 client.15999 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 20 client.15999 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:33.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f90ead0 0x555e4f97dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 4 client.15999 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.928+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e4f811050 con 0x555e4f90ead0 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.929+0000 7fd897fff700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f90ead0 0x555e4f97dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.929+0000 7fd8777fe700 10 client.15999 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.929+0000 7fd897fff700 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f90ead0 0x555e4f97dda0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f54a2c425c1aa077 server_cookie=1353acde67a6294b in_seq=0 out_seq=0 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8777fe700 1 -- 172.21.15.5:0/3126600607 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x555e4f811050 con 0x555e4f90ead0 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8777fe700 10 client.15999 handle_command_reply: tid=0 2023-07-21T19:09:33.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 client.15999 shutdown 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 msgr2=0x555e4f916790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd88c005a80 tx=0x7fd88c0058b0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 shutdown_connections 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555e4f903be0 0x555e4f916cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555e4f8f9da0 0x555e4f91a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555e4f90ead0 0x555e4f97dda0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 --2- 172.21.15.5:0/3126600607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555e4f9032d0 0x555e4f916790 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8777fe700 0 client.15999 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.930+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 >> 172.21.15.5:0/3126600607 conn(0x555e4f8fe170 msgr2=0x555e4f911660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.931+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 shutdown_connections 2023-07-21T19:09:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.931+0000 7fd8b1ed3b80 1 -- 172.21.15.5:0/3126600607 wait complete. 2023-07-21T19:09:33.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.931+0000 7fd8b1ed3b80 20 client.15999 trim_cache size 0 max 16384 2023-07-21T19:09:33.946 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:09:33.946 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (2 inodes in the stack)", 2023-07-21T19:09:33.946 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:09:33.946 INFO:teuthology.orchestra.run.smithi005.stdout: "8886242e-eb32-49fb-a7fd-08f28461dd49": { 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "8886242e-eb32-49fb-a7fd-08f28461dd49", 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:09:33.947 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:09:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 msgr2=0x7fd8800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 0x7fd8800a37f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd890009ec0 tx=0x7fd890009920 comp rx=0 tx=0).stop 2023-07-21T19:09:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 msgr2=0x7fd89819dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd888005910 tx=0x7fd8880047b0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 shutdown_connections 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd898105b10 0x7fd898197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd8980ffb00 0x7fd89819dad0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd8800a1340 0x7fd8800a37f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 --2- 172.21.15.5:0/3636589018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd8981004b0 0x7fd89819e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 >> 172.21.15.5:0/3636589018 conn(0x7fd8980fb600 msgr2=0x7fd8980fbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 shutdown_connections 2023-07-21T19:09:33.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:09:33.933+0000 7fd89e5ff700 1 -- 172.21.15.5:0/3636589018 wait complete. 2023-07-21T19:09:33.965 INFO:tasks.cephfs.filesystem:scrub status for tag:8886242e-eb32-49fb-a7fd-08f28461dd49 - {'path': '/', 'tag': '8886242e-eb32-49fb-a7fd-08f28461dd49', 'options': 'recursive,force'} 2023-07-21T19:09:34.116 INFO:teuthology.orchestra.run.smithi005.stderr:5500000 of 50000000 tuples (11%) done (elapsed 91.67 s, remaining 741.71 s) 2023-07-21T19:09:35.428 INFO:teuthology.orchestra.run.smithi005.stderr:5600000 of 50000000 tuples (11%) done (elapsed 92.98 s, remaining 737.23 s) 2023-07-21T19:09:35.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:35 smithi101 ceph-mon[128136]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 30 op/s 2023-07-21T19:09:35.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:35 smithi005 ceph-mon[126460]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 30 op/s 2023-07-21T19:09:35.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:35 smithi156 ceph-mon[128316]: pgmap v62: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 30 op/s 2023-07-21T19:09:36.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:36 smithi101 ceph-mon[128136]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:36.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:36 smithi005 ceph-mon[126460]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:36.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:36 smithi156 ceph-mon[128316]: pgmap v63: 97 pgs: 97 active+clean; 1.3 GiB data, 882 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:09:37.316 INFO:teuthology.orchestra.run.smithi005.stderr:5700000 of 50000000 tuples (11%) done (elapsed 94.87 s, remaining 737.35 s) 2023-07-21T19:09:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.760+0000 7f8ff67ff700 1 -- 172.21.15.5:0/4084025140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00fea30 msgr2=0x7f8ff00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.760+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/4084025140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00fea30 0x7f8ff00feea0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f8fe4009990 tx=0x7f8fe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 -- 172.21.15.5:0/4084025140 shutdown_connections 2023-07-21T19:09:38.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/4084025140 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/4084025140 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 0x7f8ff0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/4084025140 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00fea30 0x7f8ff00feea0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 -- 172.21.15.5:0/4084025140 >> 172.21.15.5:0/4084025140 conn(0x7f8ff0074760 msgr2=0x7f8ff0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 -- 172.21.15.5:0/4084025140 shutdown_connections 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.761+0000 7f8ff67ff700 1 -- 172.21.15.5:0/4084025140 wait complete. 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 Processor -- start 2023-07-21T19:09:38.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 -- start start 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 0x7f8ff0197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 0x7f8ff019c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019cf70 con 0x7f8ff00ff3e0 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019d0e0 con 0x7f8ff0104a30 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.762+0000 7f8ff67ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff019d2a0 con 0x7f8ff00fea30 2023-07-21T19:09:38.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 0x7f8ff0197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47708/0 (socket says 172.21.15.5:47708) 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8fef7fe700 1 -- 172.21.15.5:0/3067113139 learned_addr learned my addr 172.21.15.5:0/3067113139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 0x7f8ff019c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 -- 172.21.15.5:0/3067113139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 msgr2=0x7f8ff019c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 0x7f8ff019c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 -- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 msgr2=0x7f8ff01982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 -- 172.21.15.5:0/3067113139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fe40095f0 con 0x7f8ff00fea30 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.763+0000 7f8feffff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 0x7f8ff0197d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9c0 tx=0x7f8fe000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.764+0000 7f8fed7fa700 1 -- 172.21.15.5:0/3067113139 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0004220 con 0x7f8ff00fea30 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.764+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ff019d580 con 0x7f8ff00fea30 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.764+0000 7f8fed7fa700 1 -- 172.21.15.5:0/3067113139 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe0004380 con 0x7f8ff00fea30 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.764+0000 7f8fed7fa700 1 -- 172.21.15.5:0/3067113139 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0003aa0 con 0x7f8ff00fea30 2023-07-21T19:09:38.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.764+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01a4dc0 con 0x7f8ff00fea30 2023-07-21T19:09:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.768+0000 7f8fed7fa700 1 -- 172.21.15.5:0/3067113139 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8fe0003c00 con 0x7f8ff00fea30 2023-07-21T19:09:38.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.769+0000 7f900a037b80 1 Processor -- start 2023-07-21T19:09:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 -- start start 2023-07-21T19:09:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f8fed7fa700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 0x7f8fd80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f8fed7fa700 1 -- 172.21.15.5:0/3067113139 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8fe0014070 con 0x7f8ff00fea30 2023-07-21T19:09:38.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 0x7f8fd80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc6d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffb72c50 con 0x5605ffc6d580 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffbc59e0 con 0x5605ffac96b0 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f900a037b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffbbee70 con 0x5605ffc732d0 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.770+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54278/0 (socket says 172.21.15.5:54278) 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 -- 172.21.15.5:0/2896965582 learned_addr learned my addr 172.21.15.5:0/2896965582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc6d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 -- 172.21.15.5:0/2896965582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 msgr2=0x5605ffc6d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc6d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 -- 172.21.15.5:0/2896965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 msgr2=0x5605ffc6d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc6d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 -- 172.21.15.5:0/2896965582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ff9ecd80 con 0x5605ffac96b0 2023-07-21T19:09:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 0x7f8fd80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800f870 tx=0x7f8fe800fc30 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.771+0000 7f8feffff700 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009230 tx=0x7f8fe0099950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad44a566367dc067 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.772+0000 7f8fd67fc700 1 -- 172.21.15.5:0/2896965582 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe009df50 con 0x5605ffac96b0 2023-07-21T19:09:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.772+0000 7f8fd67fc700 1 -- 172.21.15.5:0/2896965582 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe00a2370 con 0x5605ffac96b0 2023-07-21T19:09:38.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.772+0000 7f8fd67fc700 1 -- 172.21.15.5:0/2896965582 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0099410 con 0x5605ffac96b0 2023-07-21T19:09:38.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.772+0000 7f900a037b80 1 -- 172.21.15.5:0/2896965582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 msgr2=0x5605ffc72d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.772+0000 7f900a037b80 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0009230 tx=0x7f8fe0099950 comp rx=0 tx=0).stop 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 -- 172.21.15.5:0/2896965582 shutdown_connections 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc6d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc72d90 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 --2- 172.21.15.5:0/2896965582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc6d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 -- 172.21.15.5:0/2896965582 >> 172.21.15.5:0/2896965582 conn(0x5605ffc62be0 msgr2=0x5605ffc67f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 -- 172.21.15.5:0/2896965582 shutdown_connections 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 -- 172.21.15.5:0/2896965582 wait complete. 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.773+0000 7f900a037b80 1 -- start start 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 Processor -- start 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 0x5605ffc79600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 0x5605ffc76fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffb5ea60 con 0x5605ffac96b0 2023-07-21T19:09:38.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffb5e760 con 0x5605ffc732d0 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffa428c0 con 0x5605ffc6d580 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.774+0000 7f900a037b80 10 client.0 _ll_register_callbacks cb 0x5605ffb5c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 0x5605ffc79600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8ff4d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 0x5605ffc76fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33798/0 (socket says 172.21.15.5:33798) 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8fef7fe700 1 -- 172.21.15.5:0/581766895 learned_addr learned my addr 172.21.15.5:0/581766895 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:38.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 -- 172.21.15.5:0/581766895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 msgr2=0x5605ffc76880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 -- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 msgr2=0x5605ffc76fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 0x5605ffc76fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 -- 172.21.15.5:0/581766895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ff01a4dc0 con 0x5605ffac96b0 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 0x5605ffc76fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:38.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8feffff700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 0x5605ffc79600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00cab20 tx=0x7f8fe0096390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d5f02a59bd215c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.775+0000 7f8fcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ca070 con 0x5605ffac96b0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8fe00ca1d0 con 0x5605ffac96b0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605ffb18490 con 0x5605ffac96b0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe009a1b0 con 0x5605ffac96b0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 10 client.16011 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5605ff9d5c00 con 0x5605ffac96b0 2023-07-21T19:09:38.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605ffb18490 con 0x5605ffac96b0 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8fe00a1ee0 con 0x5605ffac96b0 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 10 client.16011 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 10 client.16011 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 20 client.16011 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffc86790 0x5605ffc88ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 4 client.16011 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:38.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.776+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5605ffbb7e50 con 0x5605ffc86790 2023-07-21T19:09:38.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.777+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffc86790 0x5605ffc88ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.777+0000 7f8fcf7fe700 10 client.16011 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.777+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffc86790 0x5605ffc88ba0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=581df77fce516bfc server_cookie=9c16da63d05eb88f in_seq=0 out_seq=0 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.778+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/581766895 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5605ffbb7e50 con 0x5605ffc86790 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.778+0000 7f8fcf7fe700 10 client.16011 handle_command_reply: tid=0 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.778+0000 7f900a037b80 1 client.16011 shutdown 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.778+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 msgr2=0x5605ffc79600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.778+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 0x5605ffc79600 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f8fe00cab20 tx=0x7f8fe0096390 comp rx=0 tx=0).stop 2023-07-21T19:09:38.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 shutdown_connections 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffc732d0 0x5605ffc76fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffc86790 0x5605ffc88ba0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffac96b0 0x5605ffc79600 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 --2- 172.21.15.5:0/581766895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc76880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 >> 172.21.15.5:0/581766895 conn(0x5605ffc62be0 msgr2=0x5605ffc674e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f8fcf7fe700 0 client.16011 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:38.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 shutdown_connections 2023-07-21T19:09:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 1 -- 172.21.15.5:0/581766895 wait complete. 2023-07-21T19:09:38.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.779+0000 7f900a037b80 20 client.16011 trim_cache size 0 max 16384 2023-07-21T19:09:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 Processor -- start 2023-07-21T19:09:38.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 -- start start 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc7dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc732d0 0x5605ffc72130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc69a70 con 0x5605ffc732d0 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc77830 con 0x5605ffac96b0 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.802+0000 7f900a037b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc72a10 con 0x5605ffc6d580 2023-07-21T19:09:38.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc7dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8ff4d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc732d0 0x5605ffc72130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54306/0 (socket says 172.21.15.5:54306) 2023-07-21T19:09:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 -- 172.21.15.5:0/1824591895 learned_addr learned my addr 172.21.15.5:0/1824591895 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:38.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc7dc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33800/0 (socket says 172.21.15.5:33800) 2023-07-21T19:09:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 -- 172.21.15.5:0/1824591895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 msgr2=0x5605ffc7dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc7dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 -- 172.21.15.5:0/1824591895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc732d0 msgr2=0x5605ffc72130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc732d0 0x5605ffc72130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 -- 172.21.15.5:0/1824591895 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5605ffc831d0 con 0x5605ffac96b0 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.803+0000 7f8feffff700 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0099eb0 tx=0x7f8fe00ca650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6d1364b4744cafb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f8fd6ffd700 1 -- 172.21.15.5:0/1824591895 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ca040 con 0x5605ffac96b0 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f8fd6ffd700 1 -- 172.21.15.5:0/1824591895 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8fe00044f0 con 0x5605ffac96b0 2023-07-21T19:09:38.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f8fd6ffd700 1 -- 172.21.15.5:0/1824591895 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe00ca1a0 con 0x5605ffac96b0 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 -- 172.21.15.5:0/1824591895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 msgr2=0x5605ffc6d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0099eb0 tx=0x7f8fe00ca650 comp rx=0 tx=0).stop 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 -- 172.21.15.5:0/1824591895 shutdown_connections 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc732d0 0x5605ffc72130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc6d040 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 --2- 172.21.15.5:0/1824591895 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc6d580 0x5605ffc7dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.804+0000 7f900a037b80 1 -- 172.21.15.5:0/1824591895 >> 172.21.15.5:0/1824591895 conn(0x5605ffc62be0 msgr2=0x5605ffc71a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.805+0000 7f900a037b80 1 -- 172.21.15.5:0/1824591895 shutdown_connections 2023-07-21T19:09:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.805+0000 7f900a037b80 1 -- 172.21.15.5:0/1824591895 wait complete. 2023-07-21T19:09:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.805+0000 7f900a037b80 1 -- start start 2023-07-21T19:09:38.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.805+0000 7f900a037b80 1 Processor -- start 2023-07-21T19:09:38.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc75e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc76370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc7e2a0 con 0x5605ffc6d580 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc768e0 con 0x5605ffac96b0 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5605ffc76aa0 con 0x5605ffc732d0 2023-07-21T19:09:38.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f900a037b80 10 client.0 _ll_register_callbacks cb 0x7f8fe0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f8fef7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc75e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f8feffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54314/0 (socket says 172.21.15.5:54314) 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f8feffff700 1 -- 172.21.15.5:0/921496554 learned_addr learned my addr 172.21.15.5:0/921496554 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.806+0000 7f8ff4d9c700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc76370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 -- 172.21.15.5:0/921496554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 msgr2=0x5605ffc76370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc76370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 -- 172.21.15.5:0/921496554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 msgr2=0x5605ffc75e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc75e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 -- 172.21.15.5:0/921496554 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5605ff9d5c00 con 0x5605ffac96b0 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc75e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8feffff700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0096420 tx=0x7f8fe0099df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=914ed3e5573a116f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8fcf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fe0099410 con 0x5605ffac96b0 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5605ffb79ff0 con 0x5605ffac96b0 2023-07-21T19:09:38.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.807+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8fe0004870 con 0x5605ffac96b0 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8fe00044f0 con 0x5605ffac96b0 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 10 client.25751 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5605ffa131f0 con 0x5605ffac96b0 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5605ffb79ff0 con 0x5605ffac96b0 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8fe00ca040 con 0x5605ffac96b0 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 10 client.25751 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:38.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 10 client.25751 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 20 client.25751 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffcda650 0x5605ffcdca30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 4 client.25751 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5605ffa0df10 con 0x5605ffcda650 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.808+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffcda650 0x5605ffcdca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.809+0000 7f8fef7fe700 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffcda650 0x5605ffcdca30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=288a30bd6ee3bc29 server_cookie=78b3299a12159891 in_seq=0 out_seq=0 2023-07-21T19:09:38.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.809+0000 7f8fcf7fe700 10 client.25751 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f8fcf7fe700 1 -- 172.21.15.5:0/921496554 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5605ffa0df10 con 0x5605ffcda650 2023-07-21T19:09:38.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f8fcf7fe700 10 client.25751 handle_command_reply: tid=0 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 client.25751 shutdown 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 msgr2=0x5605ffc70040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f8fe0096420 tx=0x7f8fe0099df0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 shutdown_connections 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5605ffc732d0 0x5605ffc76370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5605ffac96b0 0x5605ffc70040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5605ffcda650 0x5605ffcdca30 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 --2- 172.21.15.5:0/921496554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5605ffc6d580 0x5605ffc75e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 >> 172.21.15.5:0/921496554 conn(0x5605ffc62be0 msgr2=0x5605ffc722b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f8fcf7fe700 0 client.25751 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 shutdown_connections 2023-07-21T19:09:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 1 -- 172.21.15.5:0/921496554 wait complete. 2023-07-21T19:09:38.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.824+0000 7f900a037b80 20 client.25751 trim_cache size 0 max 16384 2023-07-21T19:09:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:09:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:09:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:09:38.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 msgr2=0x7f8fd80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 0x7f8fd80a37f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8fe800f870 tx=0x7f8fe800fc30 comp rx=0 tx=0).stop 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 msgr2=0x7f8ff0197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 0x7f8ff0197d80 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8fe000e9c0 tx=0x7f8fe000ecd0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 shutdown_connections 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ff0104a30 0x7f8ff019c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ff00fea30 0x7f8ff0197d80 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8fd80a1340 0x7f8fd80a37f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 --2- 172.21.15.5:0/3067113139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ff00ff3e0 0x7f8ff01982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 >> 172.21.15.5:0/3067113139 conn(0x7f8ff0074760 msgr2=0x7f8ff00fd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.825+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 shutdown_connections 2023-07-21T19:09:38.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:38.826+0000 7f8ff67ff700 1 -- 172.21.15.5:0/3067113139 wait complete. 2023-07-21T19:09:39.213 INFO:teuthology.orchestra.run.smithi005.stderr:5800000 of 50000000 tuples (11%) done (elapsed 96.77 s, remaining 737.45 s) 2023-07-21T19:09:39.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:39 smithi005 ceph-mon[126460]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:09:39.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:39 smithi156 ceph-mon[128316]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:09:39.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:39 smithi101 ceph-mon[128136]: pgmap v64: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:09:40.538 INFO:teuthology.orchestra.run.smithi005.stderr:5900000 of 50000000 tuples (11%) done (elapsed 98.09 s, remaining 733.21 s) 2023-07-21T19:09:41.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:41 smithi005 ceph-mon[126460]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 21 op/s 2023-07-21T19:09:41.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:41 smithi156 ceph-mon[128316]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 21 op/s 2023-07-21T19:09:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:41 smithi101 ceph-mon[128136]: pgmap v65: 97 pgs: 97 active+clean; 1.3 GiB data, 909 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 21 op/s 2023-07-21T19:09:42.392 INFO:teuthology.orchestra.run.smithi005.stderr:6000000 of 50000000 tuples (12%) done (elapsed 99.95 s, remaining 732.96 s) 2023-07-21T19:09:43.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:43 smithi005 ceph-mon[126460]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:43.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:43 smithi156 ceph-mon[128316]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:43.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:43 smithi101 ceph-mon[128136]: pgmap v66: 97 pgs: 97 active+clean; 1.4 GiB data, 932 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 33 MiB/s wr, 36 op/s 2023-07-21T19:09:44.282 INFO:teuthology.orchestra.run.smithi005.stderr:6100000 of 50000000 tuples (12%) done (elapsed 101.84 s, remaining 732.90 s) 2023-07-21T19:09:45.498 INFO:teuthology.orchestra.run.smithi005.stderr:6200000 of 50000000 tuples (12%) done (elapsed 103.05 s, remaining 728.03 s) 2023-07-21T19:09:45.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:45 smithi005 ceph-mon[126460]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:09:45.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:45 smithi156 ceph-mon[128316]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:09:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:45 smithi101 ceph-mon[128136]: pgmap v67: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:09:47.442 INFO:teuthology.orchestra.run.smithi005.stderr:6300000 of 50000000 tuples (12%) done (elapsed 105.00 s, remaining 728.32 s) 2023-07-21T19:09:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:47 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:09:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:47 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:47 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:47 smithi005 ceph-mon[126460]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:47.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:09:47.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:47 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:09:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:47 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:47 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:47 smithi156 ceph-mon[128316]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:47.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:09:47.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:47 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:09:47.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:47 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:47 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:09:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:47 smithi101 ceph-mon[128136]: pgmap v68: 97 pgs: 97 active+clean; 1.4 GiB data, 934 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:09:47.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:09:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:48 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:09:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:48 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:09:48.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:48 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:09:48.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:48 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:09:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:48 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:09:48.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:48 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:09:48.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:48 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:09:48.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:48 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:09:48.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:48 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:09:49.285 INFO:teuthology.orchestra.run.smithi005.stderr:6400000 of 50000000 tuples (12%) done (elapsed 106.84 s, remaining 727.86 s) 2023-07-21T19:09:49.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:49 smithi005 ceph-mon[126460]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-07-21T19:09:49.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:49 smithi156 ceph-mon[128316]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-07-21T19:09:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:49 smithi101 ceph-mon[128136]: pgmap v69: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 35 MiB/s wr, 56 op/s 2023-07-21T19:09:50.600 INFO:teuthology.orchestra.run.smithi005.stderr:6500000 of 50000000 tuples (13%) done (elapsed 108.16 s, remaining 723.82 s) 2023-07-21T19:09:51.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:51 smithi005 ceph-mon[126460]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:09:51.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:51 smithi156 ceph-mon[128316]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:09:51.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:51 smithi101 ceph-mon[128136]: pgmap v70: 97 pgs: 97 active+clean; 1.5 GiB data, 961 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:09:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 -- 172.21.15.5:0/3399747520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c104a30 msgr2=0x7f203c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 --2- 172.21.15.5:0/3399747520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c104a30 0x7f203c104e00 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f2038009990 tx=0x7f2038009ca0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 -- 172.21.15.5:0/3399747520 shutdown_connections 2023-07-21T19:09:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 --2- 172.21.15.5:0/3399747520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0ff3e0 0x7f203c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 --2- 172.21.15.5:0/3399747520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c104a30 0x7f203c104e00 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 --2- 172.21.15.5:0/3399747520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c0fea30 0x7f203c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 -- 172.21.15.5:0/3399747520 >> 172.21.15.5:0/3399747520 conn(0x7f203c074760 msgr2=0x7f203c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 -- 172.21.15.5:0/3399747520 shutdown_connections 2023-07-21T19:09:52.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.992+0000 7f20441ff700 1 -- 172.21.15.5:0/3399747520 wait complete. 2023-07-21T19:09:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.993+0000 7f20441ff700 1 Processor -- start 2023-07-21T19:09:52.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.994+0000 7f20441ff700 1 -- start start 2023-07-21T19:09:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.994+0000 7f20441ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.994+0000 7f20441ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 0x7f203c197020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.994+0000 7f20441ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c104a30 0x7f203c1a1980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f20441ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1a1f80 con 0x7f203c0ff3e0 2023-07-21T19:09:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f20441ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1a20f0 con 0x7f203c0fea30 2023-07-21T19:09:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f20441ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f203c1a22b0 con 0x7f203c104a30 2023-07-21T19:09:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f204179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 0x7f203c197020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43006/0 (socket says 172.21.15.5:43006) 2023-07-21T19:09:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 -- 172.21.15.5:0/1926905595 learned_addr learned my addr 172.21.15.5:0/1926905595 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:52.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f204279c700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c104a30 0x7f203c1a1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 -- 172.21.15.5:0/1926905595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c104a30 msgr2=0x7f203c1a1980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c104a30 0x7f203c1a1980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 -- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 msgr2=0x7f203c197020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 0x7f203c197020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f2041f9b700 1 -- 172.21.15.5:0/1926905595 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20380095f0 con 0x7f203c0fea30 2023-07-21T19:09:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.995+0000 7f204179a700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 0x7f203c197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:52.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f2041f9b700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2038000c00 tx=0x7f20380057c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f2032ffd700 1 -- 172.21.15.5:0/1926905595 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203801d070 con 0x7f203c0fea30 2023-07-21T19:09:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f203c1a2530 con 0x7f203c0fea30 2023-07-21T19:09:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f2032ffd700 1 -- 172.21.15.5:0/1926905595 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f203800fbc0 con 0x7f203c0fea30 2023-07-21T19:09:52.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f2032ffd700 1 -- 172.21.15.5:0/1926905595 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20380173f0 con 0x7f203c0fea30 2023-07-21T19:09:52.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:51.996+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f203c1a29e0 con 0x7f203c0fea30 2023-07-21T19:09:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.001+0000 7f2032ffd700 1 -- 172.21.15.5:0/1926905595 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f203800fd30 con 0x7f203c0fea30 2023-07-21T19:09:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 Processor -- start 2023-07-21T19:09:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 -- start start 2023-07-21T19:09:52.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c87b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c875450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c776c50 con 0x559f2c875990 2023-07-21T19:09:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c7c99e0 con 0x559f2c87b6e0 2023-07-21T19:09:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.002+0000 7f2057c03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c7c2e70 con 0x559f2c6cd6b0 2023-07-21T19:09:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204279c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c87b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37252/0 (socket says 172.21.15.5:37252) 2023-07-21T19:09:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 -- 172.21.15.5:0/412857717 learned_addr learned my addr 172.21.15.5:0/412857717 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:52.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f2041f9b700 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c875450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 -- 172.21.15.5:0/412857717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 msgr2=0x559f2c87b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c87b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 -- 172.21.15.5:0/412857717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 msgr2=0x559f2c875450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c875450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f204179a700 1 -- 172.21.15.5:0/412857717 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f2c5f0d80 con 0x559f2c875990 2023-07-21T19:09:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.003+0000 7f2032ffd700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 0x7f20280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f2032ffd700 1 -- 172.21.15.5:0/1926905595 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f20380cdff0 con 0x7f203c0fea30 2023-07-21T19:09:52.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f204179a700 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f202c011540 tx=0x7f202c011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34e544119c1b5b1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f204279c700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 0x7f20280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f2027fff700 1 -- 172.21.15.5:0/412857717 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c004a30 con 0x559f2c875990 2023-07-21T19:09:52.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f2027fff700 1 -- 172.21.15.5:0/412857717 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f202c0054b0 con 0x559f2c875990 2023-07-21T19:09:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.004+0000 7f2027fff700 1 -- 172.21.15.5:0/412857717 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c00e7e0 con 0x559f2c875990 2023-07-21T19:09:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 -- 172.21.15.5:0/412857717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 msgr2=0x559f2c875de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f202c011540 tx=0x7f202c011850 comp rx=0 tx=0).stop 2023-07-21T19:09:52.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f204279c700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 0x7f20280a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2034009e10 tx=0x7f2034009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 -- 172.21.15.5:0/412857717 shutdown_connections 2023-07-21T19:09:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c875de0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c87b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 --2- 172.21.15.5:0/412857717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c875450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 -- 172.21.15.5:0/412857717 >> 172.21.15.5:0/412857717 conn(0x559f2c866be0 msgr2=0x559f2c86bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 -- 172.21.15.5:0/412857717 shutdown_connections 2023-07-21T19:09:52.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.005+0000 7f2057c03b80 1 -- 172.21.15.5:0/412857717 wait complete. 2023-07-21T19:09:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 -- start start 2023-07-21T19:09:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 Processor -- start 2023-07-21T19:09:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87eb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c87f140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c762a60 con 0x559f2c87b6e0 2023-07-21T19:09:52.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c762760 con 0x559f2c6cd6b0 2023-07-21T19:09:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c6468c0 con 0x559f2c875990 2023-07-21T19:09:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.006+0000 7f2057c03b80 10 client.0 _ll_register_callbacks cb 0x559f2c760700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f204279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c87f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43020/0 (socket says 172.21.15.5:43020) 2023-07-21T19:09:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 -- 172.21.15.5:0/3786615412 learned_addr learned my addr 172.21.15.5:0/3786615412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f204179a700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 -- 172.21.15.5:0/3786615412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 msgr2=0x559f2c87eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 -- 172.21.15.5:0/3786615412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 msgr2=0x559f2c87f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c87f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f2041f9b700 1 -- 172.21.15.5:0/3786615412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f202c009d10 con 0x559f2c6cd6b0 2023-07-21T19:09:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.007+0000 7f204179a700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:09:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f2041f9b700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2038005f50 tx=0x7f2038005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd7bf81835cd1023 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f20257fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:09:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f203809af50 con 0x559f2c6cd6b0 2023-07-21T19:09:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f203809b180 con 0x559f2c6cd6b0 2023-07-21T19:09:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f2c71c490 con 0x559f2c6cd6b0 2023-07-21T19:09:52.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f203809acc0 con 0x559f2c6cd6b0 2023-07-21T19:09:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f2057c03b80 10 client.25763 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.008+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x559f2c5d9c00 con 0x559f2c6cd6b0 2023-07-21T19:09:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559f2c71c490 con 0x559f2c6cd6b0 2023-07-21T19:09:52.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f203809a670 con 0x559f2c6cd6b0 2023-07-21T19:09:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 10 client.25763 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 10 client.25763 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:52.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 20 client.25763 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c890f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 4 client.25763 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:09:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f2c7bbe50 con 0x559f2c88eb60 2023-07-21T19:09:52.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.009+0000 7f204179a700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c890f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.010+0000 7f204179a700 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c890f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=61447beea9388a75 server_cookie=79e67dfdaf1b2ab5 in_seq=0 out_seq=0 2023-07-21T19:09:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.010+0000 7f20257fa700 10 client.25763 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:52.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f20257fa700 1 -- 172.21.15.5:0/3786615412 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f2c7bbe50 con 0x559f2c88eb60 2023-07-21T19:09:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f20257fa700 10 client.25763 handle_command_reply: tid=0 2023-07-21T19:09:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 client.25763 shutdown 2023-07-21T19:09:52.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 msgr2=0x559f2c881900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f2038005f50 tx=0x7f2038005c80 comp rx=0 tx=0).stop 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 shutdown_connections 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c890f70 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c87f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f20257fa700 0 client.25763 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:52.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c881900 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 --2- 172.21.15.5:0/3786615412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.011+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 >> 172.21.15.5:0/3786615412 conn(0x559f2c866be0 msgr2=0x559f2c86ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.012+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 shutdown_connections 2023-07-21T19:09:52.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.012+0000 7f2057c03b80 1 -- 172.21.15.5:0/3786615412 wait complete. 2023-07-21T19:09:52.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.012+0000 7f2057c03b80 20 client.25763 trim_cache size 0 max 16384 2023-07-21T19:09:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.028+0000 7f2057c03b80 1 Processor -- start 2023-07-21T19:09:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.028+0000 7f2057c03b80 1 -- start start 2023-07-21T19:09:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c886070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c71c490 con 0x559f2c87b6e0 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c86ca00 con 0x559f2c6cd6b0 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2057c03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c863c00 con 0x559f2c875990 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f204279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f204279c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37270/0 (socket says 172.21.15.5:37270) 2023-07-21T19:09:52.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f204279c700 1 -- 172.21.15.5:0/1718794640 learned_addr learned my addr 172.21.15.5:0/1718794640 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f2041f9b700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c886070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.029+0000 7f204179a700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 -- 172.21.15.5:0/1718794640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 msgr2=0x559f2c87c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 -- 172.21.15.5:0/1718794640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 msgr2=0x559f2c886070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c886070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 -- 172.21.15.5:0/1718794640 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f2c6171f0 con 0x559f2c87b6e0 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f2041f9b700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c886070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f204279c700 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f20340185e0 tx=0x7f2034015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec28f82b3ad981ce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f2030ff9700 1 -- 172.21.15.5:0/1718794640 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034016420 con 0x559f2c87b6e0 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f2030ff9700 1 -- 172.21.15.5:0/1718794640 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f203401c070 con 0x559f2c87b6e0 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.030+0000 7f2030ff9700 1 -- 172.21.15.5:0/1718794640 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2034024720 con 0x559f2c87b6e0 2023-07-21T19:09:52.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 -- 172.21.15.5:0/1718794640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 msgr2=0x559f2c8814a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f20340185e0 tx=0x7f2034015000 comp rx=0 tx=0).stop 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 -- 172.21.15.5:0/1718794640 shutdown_connections 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c87b6e0 0x559f2c8814a0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c6cd6b0 0x559f2c886070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 --2- 172.21.15.5:0/1718794640 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c875990 0x559f2c87c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 -- 172.21.15.5:0/1718794640 >> 172.21.15.5:0/1718794640 conn(0x559f2c866be0 msgr2=0x559f2c87bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 -- 172.21.15.5:0/1718794640 shutdown_connections 2023-07-21T19:09:52.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.031+0000 7f2057c03b80 1 -- 172.21.15.5:0/1718794640 wait complete. 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 -- start start 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 Processor -- start 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c886080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c8770e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c877620 con 0x559f2c875990 2023-07-21T19:09:52.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c877790 con 0x559f2c87b6e0 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f2c877950 con 0x559f2c6cd6b0 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.032+0000 7f2057c03b80 10 client.0 _ll_register_callbacks cb 0x7f20380094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37286/0 (socket says 172.21.15.5:37286) 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 -- 172.21.15.5:0/880932832 learned_addr learned my addr 172.21.15.5:0/880932832 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204279c700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c8770e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f2041f9b700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c886080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 -- 172.21.15.5:0/880932832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 msgr2=0x559f2c886080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c886080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 -- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 msgr2=0x559f2c8770e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c8770e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.033+0000 7f204179a700 1 -- 172.21.15.5:0/880932832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f2c5d9c00 con 0x559f2c875990 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f204179a700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f202c009230 tx=0x7f202c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25837e45a1a0ba5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:09:52.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f20257fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f202c0052e0 con 0x559f2c875990 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f202c0054b0 con 0x559f2c875990 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f202c02a040 con 0x559f2c875990 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f202c011250 con 0x559f2c875990 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f2057c03b80 10 client.16035 fetch_fsmap learned FSMap version 26 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.034+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f2034007e20 con 0x559f2c875990 2023-07-21T19:09:52.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f202c0054b0 con 0x559f2c875990 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f202c02b2a0 con 0x559f2c875990 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 10 client.16035 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 10 client.16035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 20 client.16035 populate_metadata read hostname 'smithi005' 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c8e5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 4 client.16035 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:09:52.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.035+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559f2c77dff0 con 0x559f2c88eb60 2023-07-21T19:09:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.036+0000 7f2041f9b700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c8e5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:09:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.036+0000 7f2041f9b700 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c8e5cf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=324a5f381839e512 server_cookie=cf806f0bb3f66128 in_seq=0 out_seq=0 2023-07-21T19:09:52.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.036+0000 7f20257fa700 10 client.16035 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f20257fa700 1 -- 172.21.15.5:0/880932832 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559f2c77dff0 con 0x559f2c88eb60 2023-07-21T19:09:52.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f20257fa700 10 client.16035 handle_command_reply: tid=0 2023-07-21T19:09:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 client.16035 shutdown 2023-07-21T19:09:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 msgr2=0x559f2c876ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f202c009230 tx=0x7f202c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 shutdown_connections 2023-07-21T19:09:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f2c87b6e0 0x559f2c8770e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f2c6cd6b0 0x559f2c886080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.053+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559f2c88eb60 0x559f2c8e5cf0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f2057c03b80 1 --2- 172.21.15.5:0/880932832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f2c875990 0x559f2c876ba0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f20257fa700 0 client.16035 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 >> 172.21.15.5:0/880932832 conn(0x559f2c866be0 msgr2=0x559f2c87c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 shutdown_connections 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f2057c03b80 1 -- 172.21.15.5:0/880932832 wait complete. 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f2057c03b80 20 client.16035 trim_cache size 0 max 16384 2023-07-21T19:09:52.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:09:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:09:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:09:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:09:52.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 msgr2=0x7f20280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.054+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 0x7f20280a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2034009e10 tx=0x7f2034009040 comp rx=0 tx=0).stop 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 msgr2=0x7f203c196ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2038000c00 tx=0x7f20380057c0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 shutdown_connections 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f203c104a30 0x7f203c1a1980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f203c0fea30 0x7f203c196ae0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20280a12c0 0x7f20280a3770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 --2- 172.21.15.5:0/1926905595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f203c0ff3e0 0x7f203c197020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:09:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 >> 172.21.15.5:0/1926905595 conn(0x7f203c074760 msgr2=0x7f203c1026c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:09:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 shutdown_connections 2023-07-21T19:09:52.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:09:52.055+0000 7f20441ff700 1 -- 172.21.15.5:0/1926905595 wait complete. 2023-07-21T19:09:52.473 INFO:teuthology.orchestra.run.smithi005.stderr:6600000 of 50000000 tuples (13%) done (elapsed 110.03 s, remaining 723.53 s) 2023-07-21T19:09:53.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:53 smithi005 ceph-mon[126460]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 34 MiB/s wr, 59 op/s 2023-07-21T19:09:53.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:53 smithi156 ceph-mon[128316]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 34 MiB/s wr, 59 op/s 2023-07-21T19:09:53.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:53 smithi101 ceph-mon[128136]: pgmap v71: 97 pgs: 97 active+clean; 1.5 GiB data, 985 MiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 34 MiB/s wr, 59 op/s 2023-07-21T19:09:54.366 INFO:teuthology.orchestra.run.smithi005.stderr:6700000 of 50000000 tuples (13%) done (elapsed 111.92 s, remaining 723.32 s) 2023-07-21T19:09:55.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:55 smithi005 ceph-mon[126460]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 45 op/s 2023-07-21T19:09:55.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:55 smithi156 ceph-mon[128316]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 45 op/s 2023-07-21T19:09:55.688 INFO:teuthology.orchestra.run.smithi005.stderr:6800000 of 50000000 tuples (13%) done (elapsed 113.24 s, remaining 719.43 s) 2023-07-21T19:09:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:55 smithi101 ceph-mon[128136]: pgmap v72: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 24 MiB/s wr, 45 op/s 2023-07-21T19:09:56.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:09:56.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:09:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:09:56.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:09:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:09:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:09:57.480 INFO:teuthology.orchestra.run.smithi005.stderr:6900000 of 50000000 tuples (13%) done (elapsed 115.04 s, remaining 718.56 s) 2023-07-21T19:09:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:57 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:09:57.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:57 smithi005 ceph-mon[126460]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 44 op/s 2023-07-21T19:09:57.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:57 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:09:57.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:57 smithi156 ceph-mon[128316]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 44 op/s 2023-07-21T19:09:57.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:57 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:09:57.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:57 smithi101 ceph-mon[128136]: pgmap v73: 97 pgs: 97 active+clean; 1.5 GiB data, 988 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 44 op/s 2023-07-21T19:09:58.815 INFO:teuthology.orchestra.run.smithi005.stderr:7000000 of 50000000 tuples (14%) done (elapsed 116.37 s, remaining 714.85 s) 2023-07-21T19:09:59.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:09:59 smithi005 ceph-mon[126460]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 35 MiB/s wr, 54 op/s 2023-07-21T19:09:59.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:09:59 smithi156 ceph-mon[128316]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 35 MiB/s wr, 54 op/s 2023-07-21T19:09:59.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:09:59 smithi101 ceph-mon[128136]: pgmap v74: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 79 KiB/s rd, 35 MiB/s wr, 54 op/s 2023-07-21T19:10:00.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:00 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T19:10:00.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:00 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T19:10:00.684 INFO:teuthology.orchestra.run.smithi005.stderr:7100000 of 50000000 tuples (14%) done (elapsed 118.24 s, remaining 714.44 s) 2023-07-21T19:10:00.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:00 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T19:10:01.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:01 smithi005 ceph-mon[126460]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:10:01.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:01 smithi156 ceph-mon[128316]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:10:01.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:01 smithi101 ceph-mon[128136]: pgmap v75: 97 pgs: 97 active+clean; 1.6 GiB data, 1018 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:10:02.705 INFO:teuthology.orchestra.run.smithi005.stderr:7200000 of 50000000 tuples (14%) done (elapsed 120.26 s, remaining 714.89 s) 2023-07-21T19:10:03.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:03 smithi005 ceph-mon[126460]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 29 op/s 2023-07-21T19:10:03.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:03 smithi156 ceph-mon[128316]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 29 op/s 2023-07-21T19:10:03.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:03 smithi101 ceph-mon[128136]: pgmap v76: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 31 MiB/s wr, 29 op/s 2023-07-21T19:10:03.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:10:04.077 INFO:teuthology.orchestra.run.smithi005.stderr:7300000 of 50000000 tuples (14%) done (elapsed 121.63 s, remaining 711.47 s) 2023-07-21T19:10:04.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 -- 172.21.15.5:0/2985912114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234102800 msgr2=0x7f52341067c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 --2- 172.21.15.5:0/2985912114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234102800 0x7f52341067c0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f52280099c0 tx=0x7f5228009cd0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 -- 172.21.15.5:0/2985912114 shutdown_connections 2023-07-21T19:10:04.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 --2- 172.21.15.5:0/2985912114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234106ea0 0x7f523410ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 --2- 172.21.15.5:0/2985912114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f5234102230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 --2- 172.21.15.5:0/2985912114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234102800 0x7f52341067c0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 -- 172.21.15.5:0/2985912114 >> 172.21.15.5:0/2985912114 conn(0x7f52340fd690 msgr2=0x7f52340ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 -- 172.21.15.5:0/2985912114 shutdown_connections 2023-07-21T19:10:04.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.193+0000 7f5238f9d700 1 -- 172.21.15.5:0/2985912114 wait complete. 2023-07-21T19:10:04.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.194+0000 7f5238f9d700 1 Processor -- start 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 -- start start 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234102800 0x7f523419dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5234197f70 con 0x7f5234106ea0 2023-07-21T19:10:04.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52341980e0 con 0x7f5234101e60 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5238f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52341982a0 con 0x7f5234102800 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f5233fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.195+0000 7f52337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f52337fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60848/0 (socket says 172.21.15.5:60848) 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45418/0 (socket says 172.21.15.5:45418) 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f52337fe700 1 -- 172.21.15.5:0/3732548296 learned_addr learned my addr 172.21.15.5:0/3732548296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234102800 0x7f523419dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 -- 172.21.15.5:0/3732548296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234102800 msgr2=0x7f523419dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234102800 0x7f523419dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 -- 172.21.15.5:0/3732548296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 msgr2=0x7f523419d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 -- 172.21.15.5:0/3732548296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52280095f0 con 0x7f5234106ea0 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f52337fe700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:04.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5233fff700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f522c00efc0 tx=0x7f522c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.196+0000 7f5230ff9700 1 -- 172.21.15.5:0/3732548296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c018070 con 0x7f5234106ea0 2023-07-21T19:10:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.197+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5234198520 con 0x7f5234106ea0 2023-07-21T19:10:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.197+0000 7f5230ff9700 1 -- 172.21.15.5:0/3732548296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f522c00f040 con 0x7f5234106ea0 2023-07-21T19:10:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.197+0000 7f5230ff9700 1 -- 172.21.15.5:0/3732548296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522c00cba0 con 0x7f5234106ea0 2023-07-21T19:10:04.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.197+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52341a4820 con 0x7f5234106ea0 2023-07-21T19:10:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.202+0000 7f5230ff9700 1 -- 172.21.15.5:0/3732548296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f522c003a00 con 0x7f5234106ea0 2023-07-21T19:10:04.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 Processor -- start 2023-07-21T19:10:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 -- start start 2023-07-21T19:10:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247856e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x56247857b110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478473cb0 con 0x56247856ee00 2023-07-21T19:10:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624784c6a40 con 0x562478578d30 2023-07-21T19:10:04.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f524da49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624784bfed0 con 0x56247856e4f0 2023-07-21T19:10:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f52337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.203+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247856e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45430/0 (socket says 172.21.15.5:45430) 2023-07-21T19:10:04.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 -- 172.21.15.5:0/644514473 learned_addr learned my addr 172.21.15.5:0/644514473 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f5232ffd700 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x56247857b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f5230ff9700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 0x7f521c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 -- 172.21.15.5:0/644514473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 msgr2=0x56247856e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247856e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 -- 172.21.15.5:0/644514473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 msgr2=0x56247857b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x56247857b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 -- 172.21.15.5:0/644514473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5624782edde0 con 0x56247856ee00 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f5232ffd700 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x56247857b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f5230ff9700 1 -- 172.21.15.5:0/3732548296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f522c0cc740 con 0x7f5234106ea0 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f5233fff700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 0x7f521c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.204+0000 7f52337fe700 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f52240117e0 tx=0x7f5224011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85437bba8dd487a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.205+0000 7f5219ffb700 1 -- 172.21.15.5:0/644514473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522400f480 con 0x56247856ee00 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.205+0000 7f5219ffb700 1 -- 172.21.15.5:0/644514473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f522400fa70 con 0x56247856ee00 2023-07-21T19:10:04.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.205+0000 7f5219ffb700 1 -- 172.21.15.5:0/644514473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224019950 con 0x56247856ee00 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- 172.21.15.5:0/644514473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 msgr2=0x5624785787f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f52240117e0 tx=0x7f5224011af0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f5233fff700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 0x7f521c0a3810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f522c0c9f70 tx=0x7f522c009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- 172.21.15.5:0/644514473 shutdown_connections 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x56247857b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247856e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 --2- 172.21.15.5:0/644514473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785787f0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- 172.21.15.5:0/644514473 >> 172.21.15.5:0/644514473 conn(0x562478563ff0 msgr2=0x562478569930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- 172.21.15.5:0/644514473 shutdown_connections 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- 172.21.15.5:0/644514473 wait complete. 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.206+0000 7f524da49b80 1 -- start start 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 Processor -- start 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856e4f0 0x562478578750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 0x562478576070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247845fac0 con 0x56247856ee00 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247845f7c0 con 0x56247856e4f0 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478343920 con 0x562478578d30 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f524da49b80 10 client.0 _ll_register_callbacks cb 0x56247845d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f52337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f52337fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45434/0 (socket says 172.21.15.5:45434) 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f52337fe700 1 -- 172.21.15.5:0/3129882252 learned_addr learned my addr 172.21.15.5:0/3129882252 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.207+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856e4f0 0x562478578750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 0x562478576070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 -- 172.21.15.5:0/3129882252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 msgr2=0x562478576070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 0x562478576070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 -- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856e4f0 msgr2=0x562478578750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856e4f0 0x562478578750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 -- 172.21.15.5:0/3129882252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5224011440 con 0x56247856ee00 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 0x562478576070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f52337fe700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fba5e6f8a34052b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.208+0000 7f5212ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224005250 con 0x56247856ee00 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5624784194f0 con 0x56247856ee00 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f52240053b0 con 0x56247856ee00 2023-07-21T19:10:04.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5224005570 con 0x56247856ee00 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f524da49b80 10 client.16047 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5624782d6c60 con 0x56247856ee00 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5624784194f0 con 0x56247856ee00 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5224022430 con 0x56247856ee00 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f524da49b80 10 client.16047 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.209+0000 7f524da49b80 10 client.16047 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:04.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.210+0000 7f524da49b80 20 client.16047 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:04.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.210+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785886f0 0x56247858ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.210+0000 7f524da49b80 4 client.16047 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:04.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.210+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624784b8eb0 con 0x5624785886f0 2023-07-21T19:10:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.210+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785886f0 0x56247858ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.211+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785886f0 0x56247858ab00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b94c2a5589bb4c1 server_cookie=e012d01665ef2056 in_seq=0 out_seq=0 2023-07-21T19:10:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.211+0000 7f5212ffd700 10 client.16047 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f5212ffd700 1 -- 172.21.15.5:0/3129882252 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624784b8eb0 con 0x5624785886f0 2023-07-21T19:10:04.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f5212ffd700 10 client.16047 handle_command_reply: tid=0 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 client.16047 shutdown 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 msgr2=0x5624785759d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f5224009d90 tx=0x7f522400f730 comp rx=0 tx=0).stop 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 shutdown_connections 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562478578d30 0x562478576070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856e4f0 0x562478578750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785886f0 0x56247858ab00 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 --2- 172.21.15.5:0/3129882252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856ee00 0x5624785759d0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f5212ffd700 0 client.16047 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 >> 172.21.15.5:0/3129882252 conn(0x562478563ff0 msgr2=0x562478569520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 shutdown_connections 2023-07-21T19:10:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 1 -- 172.21.15.5:0/3129882252 wait complete. 2023-07-21T19:10:04.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.212+0000 7f524da49b80 20 client.16047 trim_cache size 0 max 16384 2023-07-21T19:10:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 Processor -- start 2023-07-21T19:10:04.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 -- start start 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856ee00 0x562478584aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x562478583fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624784194f0 con 0x56247856e4f0 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562478563e00 con 0x562478578d30 2023-07-21T19:10:04.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.238+0000 7f524da49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785701a0 con 0x56247856ee00 2023-07-21T19:10:04.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45444/0 (socket says 172.21.15.5:45444) 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 -- 172.21.15.5:0/2407106586 learned_addr learned my addr 172.21.15.5:0/2407106586 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f52337fe700 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856ee00 0x562478584aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x562478583fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 -- 172.21.15.5:0/2407106586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856ee00 msgr2=0x562478584aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856ee00 0x562478584aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 -- 172.21.15.5:0/2407106586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 msgr2=0x562478583fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x562478583fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 -- 172.21.15.5:0/2407106586 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562478314250 con 0x56247856e4f0 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.239+0000 7f5232ffd700 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52280095c0 tx=0x7f522800b7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed40b454b102374 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.240+0000 7f521a7fc700 1 -- 172.21.15.5:0/2407106586 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522800bdd0 con 0x56247856e4f0 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.240+0000 7f521a7fc700 1 -- 172.21.15.5:0/2407106586 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f522801e040 con 0x56247856e4f0 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.240+0000 7f521a7fc700 1 -- 172.21.15.5:0/2407106586 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5228005300 con 0x56247856e4f0 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- 172.21.15.5:0/2407106586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 msgr2=0x56247857f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f52280095c0 tx=0x7f522800b7b0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- 172.21.15.5:0/2407106586 shutdown_connections 2023-07-21T19:10:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562478578d30 0x562478583fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247856e4f0 0x56247857f810 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 --2- 172.21.15.5:0/2407106586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856ee00 0x562478584aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- 172.21.15.5:0/2407106586 >> 172.21.15.5:0/2407106586 conn(0x562478563ff0 msgr2=0x562478568060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- 172.21.15.5:0/2407106586 shutdown_connections 2023-07-21T19:10:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- 172.21.15.5:0/2407106586 wait complete. 2023-07-21T19:10:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.241+0000 7f524da49b80 1 -- start start 2023-07-21T19:10:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 Processor -- start 2023-07-21T19:10:04.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247857f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 0x56247857de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 0x56247857e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247857ec50 con 0x562478578d30 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785dc5c0 con 0x56247856ee00 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624785dc700 con 0x56247856e4f0 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.242+0000 7f524da49b80 10 client.0 _ll_register_callbacks cb 0x7f5224005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 0x56247857e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f5232ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 0x56247857e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45448/0 (socket says 172.21.15.5:45448) 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f5232ffd700 1 -- 172.21.15.5:0/3548041891 learned_addr learned my addr 172.21.15.5:0/3548041891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 0x56247857de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247857f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 -- 172.21.15.5:0/3548041891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 msgr2=0x56247857f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247857f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 -- 172.21.15.5:0/3548041891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 msgr2=0x56247857e570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:04.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 0x56247857e570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.243+0000 7f52337fe700 1 -- 172.21.15.5:0/3548041891 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5228020040 con 0x56247856ee00 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f52337fe700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 0x56247857de30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52240117e0 tx=0x7f5224022960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4862983de0690cc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f5212ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5224021dc0 con 0x56247856ee00 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f522400fa70 con 0x56247856ee00 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5224022ab0 con 0x56247856ee00 2023-07-21T19:10:04.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5224022640 con 0x56247856ee00 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f524da49b80 10 client.25787 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.244+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5624782ec2b0 con 0x56247856ee00 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f522400fa70 con 0x56247856ee00 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5224021690 con 0x56247856ee00 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 10 client.25787 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 20 client.25787 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:04.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785dda30 0x5624785dfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 4 client.25787 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:10:04.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56247847b050 con 0x5624785dda30 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.245+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785dda30 0x5624785dfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f5212ffd700 10 client.25787 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f5232ffd700 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785dda30 0x5624785dfe40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=511570e968fb79c4 server_cookie=10bd06df9128aded in_seq=0 out_seq=0 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f5212ffd700 1 -- 172.21.15.5:0/3548041891 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56247847b050 con 0x5624785dda30 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f5212ffd700 10 client.25787 handle_command_reply: tid=0 2023-07-21T19:10:04.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 client.25787 shutdown 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 msgr2=0x56247857de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 0x56247857de30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f52240117e0 tx=0x7f5224022960 comp rx=0 tx=0).stop 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 shutdown_connections 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624785dda30 0x5624785dfe40 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562478578d30 0x56247857e570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f5212ffd700 0 client.25787 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247856e4f0 0x56247857f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 --2- 172.21.15.5:0/3548041891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247856ee00 0x56247857de30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.246+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 >> 172.21.15.5:0/3548041891 conn(0x562478563ff0 msgr2=0x562478583fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 shutdown_connections 2023-07-21T19:10:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f524da49b80 1 -- 172.21.15.5:0/3548041891 wait complete. 2023-07-21T19:10:04.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f524da49b80 20 client.25787 trim_cache size 0 max 16384 2023-07-21T19:10:04.262 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:04.263 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:10:04.263 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:10:04.263 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:04.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 msgr2=0x7f521c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 0x7f521c0a3810 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f522c0c9f70 tx=0x7f522c009040 comp rx=0 tx=0).stop 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 msgr2=0x7f5234197910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.247+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f522c00efc0 tx=0x7f522c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 shutdown_connections 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f521c0a1360 0x7f521c0a3810 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5234106ea0 0x7f5234197910 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5234101e60 0x7f523419d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 --2- 172.21.15.5:0/3732548296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5234102800 0x7f523419dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 >> 172.21.15.5:0/3732548296 conn(0x7f52340fd690 msgr2=0x7f52340fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 shutdown_connections 2023-07-21T19:10:04.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.248+0000 7f5238f9d700 1 -- 172.21.15.5:0/3732548296 wait complete. 2023-07-21T19:10:04.276 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:10:04.276 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:10:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 -- 172.21.15.5:0/1342692410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0728b0 msgr2=0x7f37bc072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1342692410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0728b0 0x7f37bc072c80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f37b0009990 tx=0x7f37b0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 -- 172.21.15.5:0/1342692410 shutdown_connections 2023-07-21T19:10:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1342692410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc10c310 0x7f37bc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1342692410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0728b0 0x7f37bc072c80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1342692410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc0731c0 0x7f37bc10bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.514+0000 7f37c31ff700 1 -- 172.21.15.5:0/1342692410 >> 172.21.15.5:0/1342692410 conn(0x7f37bc0fb590 msgr2=0x7f37bc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.515+0000 7f37c31ff700 1 -- 172.21.15.5:0/1342692410 shutdown_connections 2023-07-21T19:10:04.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.515+0000 7f37c31ff700 1 -- 172.21.15.5:0/1342692410 wait complete. 2023-07-21T19:10:04.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 Processor -- start 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 -- start start 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 0x7f37bc197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 0x7f37bc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc10c310 0x7f37bc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19cd00 con 0x7f37bc0731c0 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19ce70 con 0x7f37bc0728b0 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.516+0000 7f37c31ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37bc19d030 con 0x7f37bc10c310 2023-07-21T19:10:04.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 0x7f37bc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37c0f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 0x7f37bc197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37c0f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 0x7f37bc197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60920/0 (socket says 172.21.15.5:60920) 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37c0f9b700 1 -- 172.21.15.5:0/1193281524 learned_addr learned my addr 172.21.15.5:0/1193281524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37c179c700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc10c310 0x7f37bc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 -- 172.21.15.5:0/1193281524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc10c310 msgr2=0x7f37bc19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc10c310 0x7f37bc19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 -- 172.21.15.5:0/1193281524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 msgr2=0x7f37bc197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 0x7f37bc197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 -- 172.21.15.5:0/1193281524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37b00095f0 con 0x7f37bc0731c0 2023-07-21T19:10:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.517+0000 7f37bbfff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 0x7f37bc198260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d950 tx=0x7f37a800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.518+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a8009930 con 0x7f37bc0731c0 2023-07-21T19:10:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.518+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f37a8010460 con 0x7f37bc0731c0 2023-07-21T19:10:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.518+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37bc19d310 con 0x7f37bc0731c0 2023-07-21T19:10:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.518+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a800b5d0 con 0x7f37bc0731c0 2023-07-21T19:10:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.518+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37bc1a4b50 con 0x7f37bc0731c0 2023-07-21T19:10:04.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.519+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37bc067300 con 0x7f37bc0731c0 2023-07-21T19:10:04.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.522+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f37a8009a90 con 0x7f37bc0731c0 2023-07-21T19:10:04.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.524+0000 7f37b9ffb700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 0x7f37ac0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.524+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f37a80cc860 con 0x7f37bc0731c0 2023-07-21T19:10:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.524+0000 7f37c0f9b700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 0x7f37ac0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:04.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.525+0000 7f37c0f9b700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 0x7f37ac0a3830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b5c0 tx=0x7f37b0005fb0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:04.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.528+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f37a8095c40 con 0x7f37bc0731c0 2023-07-21T19:10:04.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.889+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f37bc199070 con 0x7f37bc0731c0 2023-07-21T19:10:04.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.890+0000 7f37b9ffb700 1 -- 172.21.15.5:0/1193281524 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f37a8013050 con 0x7f37bc0731c0 2023-07-21T19:10:04.899 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:10:04.900 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:09:18.477788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:10:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 msgr2=0x7f37ac0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 0x7f37ac0a3830 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f37b000b5c0 tx=0x7f37b0005fb0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 msgr2=0x7f37bc198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:04.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 0x7f37bc198260 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f37a800d950 tx=0x7f37a800dc60 comp rx=0 tx=0).stop 2023-07-21T19:10:04.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 shutdown_connections 2023-07-21T19:10:04.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f37bc10c310 0x7f37bc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f37bc0728b0 0x7f37bc197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f37ac0a1380 0x7f37ac0a3830 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 --2- 172.21.15.5:0/1193281524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f37bc0731c0 0x7f37bc198260 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 >> 172.21.15.5:0/1193281524 conn(0x7f37bc0fb590 msgr2=0x7f37bc10e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.893+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 shutdown_connections 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:04.894+0000 7f37c31ff700 1 -- 172.21.15.5:0/1193281524 wait complete. 2023-07-21T19:10:04.904 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2023-07-21T19:10:04.925 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:10:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 -- 172.21.15.5:0/3243333132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c81059e0 msgr2=0x7ff9c8105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/3243333132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c81059e0 0x7ff9c8105db0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7ff9c4009990 tx=0x7ff9c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 -- 172.21.15.5:0/3243333132 shutdown_connections 2023-07-21T19:10:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/3243333132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c8100380 0x7ff9c810c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/3243333132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c81059e0 0x7ff9c8105db0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/3243333132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c80ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 -- 172.21.15.5:0/3243333132 >> 172.21.15.5:0/3243333132 conn(0x7ff9c80fb4b0 msgr2=0x7ff9c80fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.151+0000 7ff9cf19d700 1 -- 172.21.15.5:0/3243333132 shutdown_connections 2023-07-21T19:10:05.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.152+0000 7ff9cf19d700 1 -- 172.21.15.5:0/3243333132 wait complete. 2023-07-21T19:10:05.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 Processor -- start 2023-07-21T19:10:05.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 -- start start 2023-07-21T19:10:05.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 0x7ff9c8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c81059e0 0x7ff9c81a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c8073580 con 0x7ff9c80ff9d0 2023-07-21T19:10:05.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81a3f10 con 0x7ff9c8100380 2023-07-21T19:10:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.153+0000 7ff9cf19d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9c81a4050 con 0x7ff9c81059e0 2023-07-21T19:10:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45478/0 (socket says 172.21.15.5:45478) 2023-07-21T19:10:05.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9ce19b700 1 -- 172.21.15.5:0/1935846950 learned_addr learned my addr 172.21.15.5:0/1935846950 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 0x7ff9c8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c81059e0 0x7ff9c81a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1935846950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c81059e0 msgr2=0x7ff9c81a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c81059e0 0x7ff9c81a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 msgr2=0x7ff9c8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1935846950 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9c40095f0 con 0x7ff9c8100380 2023-07-21T19:10:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.154+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 0x7ff9c8072d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.155+0000 7ff9bf7fe700 1 -- 172.21.15.5:0/1935846950 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b800eee0 con 0x7ff9c8100380 2023-07-21T19:10:05.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.155+0000 7ff9bf7fe700 1 -- 172.21.15.5:0/1935846950 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9b800cbd0 con 0x7ff9c8100380 2023-07-21T19:10:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.155+0000 7ff9bf7fe700 1 -- 172.21.15.5:0/1935846950 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b8018680 con 0x7ff9c8100380 2023-07-21T19:10:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.155+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9c81a42b0 con 0x7ff9c8100380 2023-07-21T19:10:05.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.155+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c81a4740 con 0x7ff9c8100380 2023-07-21T19:10:05.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.160+0000 7ff9bf7fe700 1 -- 172.21.15.5:0/1935846950 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff9b800cd40 con 0x7ff9c8100380 2023-07-21T19:10:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.161+0000 7ff9e3d77b80 1 Processor -- start 2023-07-21T19:10:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.161+0000 7ff9e3d77b80 1 -- start start 2023-07-21T19:10:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f93f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f942110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f83acb0 con 0x55da4f93fd30 2023-07-21T19:10:05.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f88da40 con 0x55da4f9354f0 2023-07-21T19:10:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f886ed0 con 0x55da4f935e00 2023-07-21T19:10:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9bf7fe700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 0x7ff9b40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9bf7fe700 1 -- 172.21.15.5:0/1935846950 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff9b8014070 con 0x7ff9c8100380 2023-07-21T19:10:05.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9ce99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9ce99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60946/0 (socket says 172.21.15.5:60946) 2023-07-21T19:10:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9ce99c700 1 -- 172.21.15.5:0/4219572076 learned_addr learned my addr 172.21.15.5:0/4219572076 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f942110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f93f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.162+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 0x7ff9b40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 -- 172.21.15.5:0/4219572076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 msgr2=0x55da4f93f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f93f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 -- 172.21.15.5:0/4219572076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 msgr2=0x55da4f942110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f942110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 -- 172.21.15.5:0/4219572076 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4f6b4de0 con 0x55da4f9354f0 2023-07-21T19:10:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f942110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.163+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0012ba0 tx=0x7ff9c0012eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efd53279ec78cd69 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.164+0000 7ff9bcbfe700 1 -- 172.21.15.5:0/4219572076 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c000e040 con 0x55da4f9354f0 2023-07-21T19:10:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.164+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 0x7ff9b40a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0021000 tx=0x7ff9c001c000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.164+0000 7ff9bcbfe700 1 -- 172.21.15.5:0/4219572076 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9c0017e30 con 0x55da4f9354f0 2023-07-21T19:10:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.164+0000 7ff9bcbfe700 1 -- 172.21.15.5:0/4219572076 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0022410 con 0x55da4f9354f0 2023-07-21T19:10:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/4219572076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 msgr2=0x55da4f9358c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0012ba0 tx=0x7ff9c0012eb0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/4219572076 shutdown_connections 2023-07-21T19:10:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f942110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f9354f0 0x55da4f9358c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/4219572076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f93f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/4219572076 >> 172.21.15.5:0/4219572076 conn(0x55da4f92aff0 msgr2=0x55da4f930930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/4219572076 shutdown_connections 2023-07-21T19:10:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/4219572076 wait complete. 2023-07-21T19:10:05.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.165+0000 7ff9e3d77b80 1 -- start start 2023-07-21T19:10:05.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 Processor -- start 2023-07-21T19:10:05.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f9354f0 0x55da4f93f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f93cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f826ac0 con 0x55da4f93fd30 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f8267c0 con 0x55da4f935e00 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f70a920 con 0x55da4f9354f0 2023-07-21T19:10:05.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.166+0000 7ff9e3d77b80 10 client.0 _ll_register_callbacks cb 0x55da4f824760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60954/0 (socket says 172.21.15.5:60954) 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1567478055 learned_addr learned my addr 172.21.15.5:0/1567478055 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f9354f0 0x55da4f93f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f93cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f9354f0 msgr2=0x55da4f93f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f9354f0 0x55da4f93f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1567478055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 msgr2=0x55da4f93cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f93cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9cd99a700 1 -- 172.21.15.5:0/1567478055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c81a4740 con 0x55da4f935e00 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.167+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f93cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80cac10 tx=0x7ff9b8099e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=536919436450342b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b809dfc0 con 0x55da4f935e00 2023-07-21T19:10:05.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da4f7e04f0 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff9b80a23e0 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9b80b8310 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9e3d77b80 10 client.25805 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.168+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7ff9c0012800 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55da4f7e04f0 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff9b8098ef0 con 0x55da4f935e00 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 10 client.25805 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 10 client.25805 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 20 client.25805 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f94f660 0x55da4f951a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 4 client.25805 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.169+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da4f87feb0 con 0x55da4f94f660 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.170+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f94f660 0x55da4f951a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.170+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f94f660 0x55da4f951a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=bb0c8f9eff22ae04 server_cookie=dd6b0a80c7547fe6 in_seq=0 out_seq=0 2023-07-21T19:10:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.170+0000 7ff9b1ffb700 10 client.25805 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.171+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/1567478055 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da4f87feb0 con 0x55da4f94f660 2023-07-21T19:10:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.171+0000 7ff9b1ffb700 10 client.25805 handle_command_reply: tid=0 2023-07-21T19:10:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.171+0000 7ff9e3d77b80 1 client.25805 shutdown 2023-07-21T19:10:05.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 msgr2=0x55da4f93c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff9b80cac10 tx=0x7ff9b8099e80 comp rx=0 tx=0).stop 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 shutdown_connections 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f93fd30 0x55da4f93cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f9354f0 0x55da4f93f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f94f660 0x55da4f951a70 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/1567478055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f93c970 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9b1ffb700 0 client.25805 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 >> 172.21.15.5:0/1567478055 conn(0x55da4f92aff0 msgr2=0x55da4f930520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 shutdown_connections 2023-07-21T19:10:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/1567478055 wait complete. 2023-07-21T19:10:05.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.172+0000 7ff9e3d77b80 20 client.25805 trim_cache size 0 max 16384 2023-07-21T19:10:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 Processor -- start 2023-07-21T19:10:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 -- start start 2023-07-21T19:10:05.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f937cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f93fd30 0x55da4f938230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f936f90 con 0x55da4f9354f0 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f94f440 con 0x55da4f93fd30 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f9316a0 con 0x55da4f935e00 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.188+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45510/0 (socket says 172.21.15.5:45510) 2023-07-21T19:10:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2680676106 learned_addr learned my addr 172.21.15.5:0/2680676106 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f937cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f93fd30 0x55da4f938230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2680676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 msgr2=0x55da4f937cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f937cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2680676106 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f93fd30 msgr2=0x55da4f938230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f93fd30 0x55da4f938230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2680676106 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da4f69dc60 con 0x55da4f9354f0 2023-07-21T19:10:05.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.189+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098e0 tx=0x7ff9c400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee7c90b2ae37ec7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.190+0000 7ff9bd3ff700 1 -- 172.21.15.5:0/2680676106 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c401d040 con 0x55da4f9354f0 2023-07-21T19:10:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.190+0000 7ff9bd3ff700 1 -- 172.21.15.5:0/2680676106 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9c401e040 con 0x55da4f9354f0 2023-07-21T19:10:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.190+0000 7ff9bd3ff700 1 -- 172.21.15.5:0/2680676106 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c4005590 con 0x55da4f9354f0 2023-07-21T19:10:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2680676106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 msgr2=0x55da4f93f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40098e0 tx=0x7ff9c400b260 comp rx=0 tx=0).stop 2023-07-21T19:10:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2680676106 shutdown_connections 2023-07-21T19:10:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f93fd30 0x55da4f938230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f93f320 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2680676106 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f935e00 0x55da4f937cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2680676106 >> 172.21.15.5:0/2680676106 conn(0x55da4f92aff0 msgr2=0x55da4f92f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2680676106 shutdown_connections 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2680676106 wait complete. 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.191+0000 7ff9e3d77b80 1 -- start start 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 Processor -- start 2023-07-21T19:10:05.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f944c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f93fd30 0x55da4f945160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f945840 con 0x55da4f9354f0 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f9459b0 con 0x55da4f935e00 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.192+0000 7ff9e3d77b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da4f9a3530 con 0x55da4f93fd30 2023-07-21T19:10:05.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9e3d77b80 10 client.0 _ll_register_callbacks cb 0x7ff9b8099ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9cd99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f944c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9ce19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45522/0 (socket says 172.21.15.5:45522) 2023-07-21T19:10:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2568782444 learned_addr learned my addr 172.21.15.5:0/2568782444 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.193+0000 7ff9ce99c700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f93fd30 0x55da4f945160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f93fd30 msgr2=0x55da4f945160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f93fd30 0x55da4f945160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2568782444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 msgr2=0x55da4f944c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f944c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 -- 172.21.15.5:0/2568782444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9c810d110 con 0x55da4f9354f0 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9ce19b700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9c402af60 tx=0x7ff9c402af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46f1eb82cd732a97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:05.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9c401d050 con 0x55da4f9354f0 2023-07-21T19:10:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff9c0017e30 con 0x55da4f9354f0 2023-07-21T19:10:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.194+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff9c4029380 con 0x55da4f9354f0 2023-07-21T19:10:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.195+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9c4029540 con 0x55da4f9354f0 2023-07-21T19:10:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.195+0000 7ff9e3d77b80 10 client.16077 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.195+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55da4f6db250 con 0x55da4f9354f0 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9c0017e30 con 0x55da4f9354f0 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff9c40449d0 con 0x55da4f9354f0 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 10 client.16077 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 10 client.16077 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 20 client.16077 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f9a4630 0x55da4f9a6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 4 client.16077 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:05.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.196+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55da4f842050 con 0x55da4f9a4630 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.197+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f9a4630 0x55da4f9a6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.197+0000 7ff9cd99a700 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f9a4630 0x55da4f9a6a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6de3a70b424beb7 server_cookie=67e000dda7334b6 in_seq=0 out_seq=0 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.197+0000 7ff9b1ffb700 10 client.16077 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9b1ffb700 1 -- 172.21.15.5:0/2568782444 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55da4f842050 con 0x55da4f9a4630 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9b1ffb700 10 client.16077 handle_command_reply: tid=0 2023-07-21T19:10:05.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 client.16077 shutdown 2023-07-21T19:10:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 msgr2=0x55da4f9467d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7ff9c402af60 tx=0x7ff9c402af90 comp rx=0 tx=0).stop 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 shutdown_connections 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da4f93fd30 0x55da4f945160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da4f9354f0 0x55da4f9467d0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55da4f9a4630 0x55da4f9a6a40 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 --2- 172.21.15.5:0/2568782444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da4f935e00 0x55da4f944c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9b1ffb700 0 client.16077 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.198+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 >> 172.21.15.5:0/2568782444 conn(0x55da4f92aff0 msgr2=0x55da4f937cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.199+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 shutdown_connections 2023-07-21T19:10:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.199+0000 7ff9e3d77b80 1 -- 172.21.15.5:0/2568782444 wait complete. 2023-07-21T19:10:05.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.199+0000 7ff9e3d77b80 20 client.16077 trim_cache size 0 max 16384 2023-07-21T19:10:05.214 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 msgr2=0x7ff9b40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 0x7ff9b40a3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0021000 tx=0x7ff9c001c000 comp rx=0 tx=0).stop 2023-07-21T19:10:05.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 msgr2=0x7ff9c8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 0x7ff9c8072d50 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff9b800e9f0 tx=0x7ff9b800ed00 comp rx=0 tx=0).stop 2023-07-21T19:10:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 shutdown_connections 2023-07-21T19:10:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9c81059e0 0x7ff9c81a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9b40a13e0 0x7ff9b40a3890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9c80ff9d0 0x7ff9c8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 --2- 172.21.15.5:0/1935846950 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9c8100380 0x7ff9c8072d50 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 >> 172.21.15.5:0/1935846950 conn(0x7ff9c80fb4b0 msgr2=0x7ff9c8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 shutdown_connections 2023-07-21T19:10:05.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.200+0000 7ff9cf19d700 1 -- 172.21.15.5:0/1935846950 wait complete. 2023-07-21T19:10:05.231 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:10:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 -- 172.21.15.5:0/355435245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c067c10 msgr2=0x7f5b6c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 --2- 172.21.15.5:0/355435245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c067c10 0x7f5b6c067fe0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b68009cd0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 -- 172.21.15.5:0/355435245 shutdown_connections 2023-07-21T19:10:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 --2- 172.21.15.5:0/355435245 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c104e50 0x7f5b6c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 --2- 172.21.15.5:0/355435245 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c067c10 0x7f5b6c067fe0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 --2- 172.21.15.5:0/355435245 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 0x7f5b6c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 -- 172.21.15.5:0/355435245 >> 172.21.15.5:0/355435245 conn(0x7f5b6c074760 msgr2=0x7f5b6c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.236+0000 7f5b745ff700 1 -- 172.21.15.5:0/355435245 shutdown_connections 2023-07-21T19:10:05.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.237+0000 7f5b745ff700 1 -- 172.21.15.5:0/355435245 wait complete. 2023-07-21T19:10:05.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.238+0000 7f5b745ff700 1 Processor -- start 2023-07-21T19:10:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.238+0000 7f5b745ff700 1 -- start start 2023-07-21T19:10:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.238+0000 7f5b745ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c067c10 0x7f5b6c19dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.238+0000 7f5b745ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 0x7f5b6c19e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.238+0000 7f5b745ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b745ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c1981a0 con 0x7f5b6c104e50 2023-07-21T19:10:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b745ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c198310 con 0x7f5b6c0685b0 2023-07-21T19:10:05.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b745ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6c1984d0 con 0x7f5b6c067c10 2023-07-21T19:10:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45540/0 (socket says 172.21.15.5:45540) 2023-07-21T19:10:05.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 -- 172.21.15.5:0/4283857047 learned_addr learned my addr 172.21.15.5:0/4283857047 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b7239b700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c067c10 0x7f5b6c19dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 0x7f5b6c19e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 -- 172.21.15.5:0/4283857047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c067c10 msgr2=0x7f5b6c19dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c067c10 0x7f5b6c19dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 -- 172.21.15.5:0/4283857047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 msgr2=0x7f5b6c19e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 0x7f5b6c19e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.239+0000 7f5b72b9c700 1 -- 172.21.15.5:0/4283857047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b5c009520 con 0x7f5b6c104e50 2023-07-21T19:10:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.240+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b64007ee0 tx=0x7f5b6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.240+0000 7f5b637fe700 1 -- 172.21.15.5:0/4283857047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64017070 con 0x7f5b6c104e50 2023-07-21T19:10:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.240+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b680095f0 con 0x7f5b6c104e50 2023-07-21T19:10:05.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.240+0000 7f5b637fe700 1 -- 172.21.15.5:0/4283857047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b6400f040 con 0x7f5b6c104e50 2023-07-21T19:10:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.240+0000 7f5b637fe700 1 -- 172.21.15.5:0/4283857047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64005570 con 0x7f5b6c104e50 2023-07-21T19:10:05.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.241+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c1a4ae0 con 0x7f5b6c104e50 2023-07-21T19:10:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.244+0000 7f5b637fe700 1 -- 172.21.15.5:0/4283857047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5b6400db20 con 0x7f5b6c104e50 2023-07-21T19:10:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.246+0000 7f5b87ff2b80 1 Processor -- start 2023-07-21T19:10:05.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.246+0000 7f5b637fe700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 0x7f5b580a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.246+0000 7f5b637fe700 1 -- 172.21.15.5:0/4283857047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5b640cc410 con 0x7f5b6c104e50 2023-07-21T19:10:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.246+0000 7f5b87ff2b80 1 -- start start 2023-07-21T19:10:05.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.246+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 0x7f5b580a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951819040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x5629518199d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295171ec50 con 0x56295181f2d0 2023-07-21T19:10:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629517719e0 con 0x5629516756b0 2023-07-21T19:10:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295176ae70 con 0x562951819580 2023-07-21T19:10:05.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b7239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b7239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60994/0 (socket says 172.21.15.5:60994) 2023-07-21T19:10:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b7239b700 1 -- 172.21.15.5:0/986733326 learned_addr learned my addr 172.21.15.5:0/986733326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951819040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.247+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x5629518199d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 -- 172.21.15.5:0/986733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 msgr2=0x5629518199d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x5629518199d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 -- 172.21.15.5:0/986733326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 msgr2=0x562951819040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951819040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 -- 172.21.15.5:0/986733326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562951598d80 con 0x5629516756b0 2023-07-21T19:10:05.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 0x7f5b580a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c106290 tx=0x7f5b5c011720 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b7239b700 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b6800b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8864acc0f1acd6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b60bfe700 1 -- 172.21.15.5:0/986733326 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6800bb00 con 0x5629516756b0 2023-07-21T19:10:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b60bfe700 1 -- 172.21.15.5:0/986733326 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b6801e040 con 0x5629516756b0 2023-07-21T19:10:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.248+0000 7f5b60bfe700 1 -- 172.21.15.5:0/986733326 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b680039c0 con 0x5629516756b0 2023-07-21T19:10:05.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/986733326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 msgr2=0x56295181ed90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b6800b790 comp rx=0 tx=0).stop 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/986733326 shutdown_connections 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x5629518199d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x56295181ed90 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/986733326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951819040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.249+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/986733326 >> 172.21.15.5:0/986733326 conn(0x56295180ebe0 msgr2=0x562951813f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.250+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/986733326 shutdown_connections 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.250+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/986733326 wait complete. 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.250+0000 7f5b87ff2b80 1 -- start start 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.250+0000 7f5b87ff2b80 1 Processor -- start 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 0x562951825660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 0x562951822870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295170aa60 con 0x562951819580 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295170a760 con 0x56295181f2d0 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629515ee8c0 con 0x5629516756b0 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b87ff2b80 10 client.0 _ll_register_callbacks cb 0x562951708700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60998/0 (socket says 172.21.15.5:60998) 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2435959956 learned_addr learned my addr 172.21.15.5:0/2435959956 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.251+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 0x562951822870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 0x562951825660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2435959956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 msgr2=0x562951825660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 0x562951825660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2435959956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 msgr2=0x562951822870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 0x562951822870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2435959956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6c10d210 con 0x56295181f2d0 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 0x562951822870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 0x562951825660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b64095a10 tx=0x7f5b64099050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6c8a032fffb306c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b64099510 con 0x56295181f2d0 2023-07-21T19:10:05.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5b6409d540 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.252+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629516c4490 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b640a1960 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 10 client.25823 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562951581c00 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5629516c4490 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b640af020 con 0x56295181f2d0 2023-07-21T19:10:05.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 10 client.25823 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 10 client.25823 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 20 client.25823 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562951832b90 0x562951834fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 4 client.25823 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.253+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562951763e50 con 0x562951832b90 2023-07-21T19:10:05.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.254+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562951832b90 0x562951834fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.254+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562951832b90 0x562951834fa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a36ab982d9d1089 server_cookie=679f1ddf604971fe in_seq=0 out_seq=0 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.254+0000 7f5b55ffb700 10 client.25823 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.255+0000 7f5b55ffb700 1 -- 172.21.15.5:0/2435959956 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562951763e50 con 0x562951832b90 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.255+0000 7f5b55ffb700 10 client.25823 handle_command_reply: tid=0 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.255+0000 7f5b87ff2b80 1 client.25823 shutdown 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 msgr2=0x562951822fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f5b64095a10 tx=0x7f5b64099050 comp rx=0 tx=0).stop 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 shutdown_connections 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56295181f2d0 0x562951822fb0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629516756b0 0x562951825660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562951832b90 0x562951834fa0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2435959956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562951819580 0x562951822870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 >> 172.21.15.5:0/2435959956 conn(0x56295180ebe0 msgr2=0x5629518134e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b55ffb700 0 client.25823 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 shutdown_connections 2023-07-21T19:10:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.256+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2435959956 wait complete. 2023-07-21T19:10:05.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.257+0000 7f5b87ff2b80 20 client.25823 trim_cache size 0 max 16384 2023-07-21T19:10:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.278+0000 7f5b87ff2b80 1 Processor -- start 2023-07-21T19:10:05.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.278+0000 7f5b87ff2b80 1 -- start start 2023-07-21T19:10:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x562951819040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x56295181e100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56295181f6a0 con 0x56295181f2d0 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629518301c0 con 0x5629516756b0 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562951810d00 con 0x562951819580 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45576/0 (socket says 172.21.15.5:45576) 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2401620544 learned_addr learned my addr 172.21.15.5:0/2401620544 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x562951819040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x56295181e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2401620544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 msgr2=0x56295181e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x56295181e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2401620544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 msgr2=0x562951819040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x562951819040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.279+0000 7f5b72b9c700 1 -- 172.21.15.5:0/2401620544 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562951839bd0 con 0x56295181f2d0 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.280+0000 7f5b7239b700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x562951819040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.280+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b640994a0 tx=0x7f5b6409d3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b235cf4b0f8619 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.280+0000 7f5b613ff700 1 -- 172.21.15.5:0/2401620544 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6409d540 con 0x56295181f2d0 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.280+0000 7f5b613ff700 1 -- 172.21.15.5:0/2401620544 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b640999b0 con 0x56295181f2d0 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.280+0000 7f5b613ff700 1 -- 172.21.15.5:0/2401620544 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b640a14d0 con 0x56295181f2d0 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2401620544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 msgr2=0x562951813c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f5b640994a0 tx=0x7f5b6409d3a0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2401620544 shutdown_connections 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56295181f2d0 0x562951813c60 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629516756b0 0x562951819040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/2401620544 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562951819580 0x56295181e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2401620544 >> 172.21.15.5:0/2401620544 conn(0x56295180ebe0 msgr2=0x56295181d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2401620544 shutdown_connections 2023-07-21T19:10:05.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.281+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/2401620544 wait complete. 2023-07-21T19:10:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.282+0000 7f5b87ff2b80 1 -- start start 2023-07-21T19:10:05.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.282+0000 7f5b87ff2b80 1 Processor -- start 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.282+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.282+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562951819580 0x5629518266d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.282+0000 7f5b87ff2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 0x562951885a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562951827010 con 0x5629516756b0 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562951885f90 con 0x562951819580 2023-07-21T19:10:05.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b87ff2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562951886150 con 0x56295181f2d0 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b87ff2b80 10 client.0 _ll_register_callbacks cb 0x7f5b64095850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45590/0 (socket says 172.21.15.5:45590) 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 -- 172.21.15.5:0/774933367 learned_addr learned my addr 172.21.15.5:0/774933367 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562951819580 0x5629518266d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 0x562951885a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 -- 172.21.15.5:0/774933367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 msgr2=0x562951885a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 0x562951885a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 -- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562951819580 msgr2=0x5629518266d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562951819580 0x5629518266d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b7239b700 1 -- 172.21.15.5:0/774933367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562951598d80 con 0x5629516756b0 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.283+0000 7f5b72b9c700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 0x562951885a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b7239b700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b68009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab7eeb218521d39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6802fcb0 con 0x5629516756b0 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f5b6802fe10 con 0x5629516756b0 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b6801e040 con 0x5629516756b0 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b6800b540 con 0x5629516756b0 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b87ff2b80 10 client.16095 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.284+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562951581c00 con 0x5629516756b0 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b6801e040 con 0x5629516756b0 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5b68003760 con 0x5629516756b0 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 10 client.16095 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 10 client.16095 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 20 client.16095 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629518874c0 0x5629518898d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 4 client.16095 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:10:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562951725ff0 con 0x5629518874c0 2023-07-21T19:10:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.285+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629518874c0 0x5629518898d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.286+0000 7f5b55ffb700 10 client.16095 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.286+0000 7f5b71b9a700 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629518874c0 0x5629518898d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=194d7fc678c9de7c server_cookie=b287a6560556db2d in_seq=0 out_seq=0 2023-07-21T19:10:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b55ffb700 1 -- 172.21.15.5:0/774933367 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562951725ff0 con 0x5629518874c0 2023-07-21T19:10:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b55ffb700 10 client.16095 handle_command_reply: tid=0 2023-07-21T19:10:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 client.16095 shutdown 2023-07-21T19:10:05.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 msgr2=0x562951826190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b68009310 comp rx=0 tx=0).stop 2023-07-21T19:10:05.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 shutdown_connections 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56295181f2d0 0x562951885a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629518874c0 0x5629518898d0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629516756b0 0x562951826190 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 --2- 172.21.15.5:0/774933367 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562951819580 0x5629518266d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 >> 172.21.15.5:0/774933367 conn(0x56295180ebe0 msgr2=0x56295181fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b55ffb700 0 client.16095 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.301+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 shutdown_connections 2023-07-21T19:10:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b87ff2b80 1 -- 172.21.15.5:0/774933367 wait complete. 2023-07-21T19:10:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b87ff2b80 20 client.16095 trim_cache size 0 max 16384 2023-07-21T19:10:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:10:05.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:10:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:10:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:10:05.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 msgr2=0x7f5b580a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 0x7f5b580a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c106290 tx=0x7f5b5c011720 comp rx=0 tx=0).stop 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 msgr2=0x7f5b6c197b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.302+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f5b64007ee0 tx=0x7f5b6400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 shutdown_connections 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b580a1340 0x7f5b580a37f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5b6c104e50 0x7f5b6c197b30 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5b6c067c10 0x7f5b6c19dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 --2- 172.21.15.5:0/4283857047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5b6c0685b0 0x7f5b6c19e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 >> 172.21.15.5:0/4283857047 conn(0x7f5b6c074760 msgr2=0x7f5b6c0fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 shutdown_connections 2023-07-21T19:10:05.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:05.303+0000 7f5b745ff700 1 -- 172.21.15.5:0/4283857047 wait complete. 2023-07-21T19:10:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 -- 172.21.15.5:0/2312457232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 msgr2=0x7fc51010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2312457232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51010c4b0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc50800b0a0 tx=0x7fc50800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 -- 172.21.15.5:0/2312457232 shutdown_connections 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2312457232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51010c4b0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2312457232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc5101010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2312457232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 0x7fc510109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 -- 172.21.15.5:0/2312457232 >> 172.21.15.5:0/2312457232 conn(0x7fc5100fa4b0 msgr2=0x7fc5100fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 -- 172.21.15.5:0/2312457232 shutdown_connections 2023-07-21T19:10:05.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.471+0000 7fc5181ff700 1 -- 172.21.15.5:0/2312457232 wait complete. 2023-07-21T19:10:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 Processor -- start 2023-07-21T19:10:05.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 -- start start 2023-07-21T19:10:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 0x7fc5101983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51019ce90 con 0x7fc51010a040 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51019d000 con 0x7fc510100d00 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.473+0000 7fc5181ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc51019d1c0 con 0x7fc510101610 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc51679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc51679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45598/0 (socket says 172.21.15.5:45598) 2023-07-21T19:10:05.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc51679c700 1 -- 172.21.15.5:0/2725414870 learned_addr learned my addr 172.21.15.5:0/2725414870 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32810/0 (socket says 172.21.15.5:32810) 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc51579a700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 0x7fc5101983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 -- 172.21.15.5:0/2725414870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 msgr2=0x7fc5101983f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 0x7fc5101983f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 -- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 msgr2=0x7fc51019c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 -- 172.21.15.5:0/2725414870 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc508009d00 con 0x7fc510100d00 2023-07-21T19:10:05.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc51679c700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.474+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00e9f0 tx=0x7fc50c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.475+0000 7fc506ffd700 1 -- 172.21.15.5:0/2725414870 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00c9c0 con 0x7fc510100d00 2023-07-21T19:10:05.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.475+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc51019d4a0 con 0x7fc510100d00 2023-07-21T19:10:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.475+0000 7fc506ffd700 1 -- 172.21.15.5:0/2725414870 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc50c010430 con 0x7fc510100d00 2023-07-21T19:10:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.475+0000 7fc506ffd700 1 -- 172.21.15.5:0/2725414870 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c018680 con 0x7fc510100d00 2023-07-21T19:10:05.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.475+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5101a4ce0 con 0x7fc510100d00 2023-07-21T19:10:05.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.479+0000 7fc506ffd700 1 -- 172.21.15.5:0/2725414870 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc50c0105a0 con 0x7fc510100d00 2023-07-21T19:10:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 Processor -- start 2023-07-21T19:10:05.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 -- start start 2023-07-21T19:10:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa02d0 0x5587b5aa06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b59a6cb0 con 0x5587b5aa0be0 2023-07-21T19:10:05.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b59f9a40 con 0x5587b5aabad0 2023-07-21T19:10:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.480+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b59f2ed0 con 0x5587b5aa02d0 2023-07-21T19:10:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45604/0 (socket says 172.21.15.5:45604) 2023-07-21T19:10:05.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32816/0 (socket says 172.21.15.5:32816) 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc515f9b700 1 -- 172.21.15.5:0/1860044884 learned_addr learned my addr 172.21.15.5:0/1860044884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc506ffd700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 0x7fc4fc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51679c700 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa02d0 0x5587b5aa06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc506ffd700 1 -- 172.21.15.5:0/2725414870 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc50c014070 con 0x7fc510100d00 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 -- 172.21.15.5:0/1860044884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa02d0 msgr2=0x5587b5aa06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa02d0 0x5587b5aa06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 -- 172.21.15.5:0/1860044884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 msgr2=0x5587b5aab590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.481+0000 7fc51579a700 1 -- 172.21.15.5:0/1860044884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b5820de0 con 0x5587b5aabad0 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.482+0000 7fc51679c700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 0x7fc4fc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.482+0000 7fc51579a700 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc500009c80 tx=0x7fc500009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9ad47e757e50b11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.482+0000 7fc4fbfff700 1 -- 172.21.15.5:0/1860044884 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500011cb0 con 0x5587b5aabad0 2023-07-21T19:10:05.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.482+0000 7fc4fbfff700 1 -- 172.21.15.5:0/1860044884 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc500011e10 con 0x5587b5aabad0 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.482+0000 7fc4fbfff700 1 -- 172.21.15.5:0/1860044884 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500005280 con 0x5587b5aabad0 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1860044884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 msgr2=0x5587b5aadeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc500009c80 tx=0x7fc500009f90 comp rx=0 tx=0).stop 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc51679c700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 0x7fc4fc0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc508009fd0 tx=0x7fc5080099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1860044884 shutdown_connections 2023-07-21T19:10:05.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aabad0 0x5587b5aadeb0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa02d0 0x5587b5aa06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1860044884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1860044884 >> 172.21.15.5:0/1860044884 conn(0x5587b5a96da0 msgr2=0x5587b5a9c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1860044884 shutdown_connections 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.483+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1860044884 wait complete. 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 -- start start 2023-07-21T19:10:05.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 Processor -- start 2023-07-21T19:10:05.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa0be0 0x5587b5aa8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aabad0 0x5587b5aa8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5992ac0 con 0x5587b5aabad0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b59927c0 con 0x5587b5aa02d0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5876920 con 0x5587b5aa0be0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.484+0000 7fc52bb59b80 10 client.0 _ll_register_callbacks cb 0x5587b5990760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc51679c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aabad0 0x5587b5aa8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32824/0 (socket says 172.21.15.5:32824) 2023-07-21T19:10:05.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 -- 172.21.15.5:0/2834270829 learned_addr learned my addr 172.21.15.5:0/2834270829 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc51579a700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa0be0 0x5587b5aa8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 -- 172.21.15.5:0/2834270829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa0be0 msgr2=0x5587b5aa8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa0be0 0x5587b5aa8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 -- 172.21.15.5:0/2834270829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aabad0 msgr2=0x5587b5aa8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aabad0 0x5587b5aa8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 -- 172.21.15.5:0/2834270829 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5000098e0 con 0x5587b5aa02d0 2023-07-21T19:10:05.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.485+0000 7fc515f9b700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0cabd0 tx=0x7fc50c009ef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8665d8cef764ee3d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c09a3b0 con 0x5587b5aa02d0 2023-07-21T19:10:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc50c010880 con 0x5587b5aa02d0 2023-07-21T19:10:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5587b594c4f0 con 0x5587b5aa02d0 2023-07-21T19:10:05.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc50c099a90 con 0x5587b5aa02d0 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc52bb59b80 10 client.25841 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5587b5809c60 con 0x5587b5aa02d0 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5587b594c4f0 con 0x5587b5aa02d0 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc50c0cad20 con 0x5587b5aa02d0 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.486+0000 7fc52bb59b80 10 client.25841 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc52bb59b80 10 client.25841 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc52bb59b80 20 client.25841 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5abb470 0x5587b5abd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc52bb59b80 4 client.25841 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5587b59ebeb0 con 0x5587b5abb470 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.487+0000 7fc51579a700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5abb470 0x5587b5abd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.488+0000 7fc4f97fa700 10 client.25841 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:05.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.488+0000 7fc51579a700 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5abb470 0x5587b5abd880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=72954f4b86bf52c server_cookie=928115d2677d682b in_seq=0 out_seq=0 2023-07-21T19:10:05.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.488+0000 7fc4f97fa700 1 -- 172.21.15.5:0/2834270829 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5587b59ebeb0 con 0x5587b5abb470 2023-07-21T19:10:05.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.488+0000 7fc4f97fa700 10 client.25841 handle_command_reply: tid=0 2023-07-21T19:10:05.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.488+0000 7fc52bb59b80 1 client.25841 shutdown 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 msgr2=0x5587b5aab540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc50c0cabd0 tx=0x7fc50c009ef0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 shutdown_connections 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aabad0 0x5587b5aa8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5abb470 0x5587b5abd880 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5aab540 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/2834270829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aa0be0 0x5587b5aa8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc4f97fa700 0 client.25841 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 >> 172.21.15.5:0/2834270829 conn(0x5587b5a96da0 msgr2=0x5587b5a9c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 shutdown_connections 2023-07-21T19:10:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 1 -- 172.21.15.5:0/2834270829 wait complete. 2023-07-21T19:10:05.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.489+0000 7fc52bb59b80 20 client.25841 trim_cache size 0 max 16384 2023-07-21T19:10:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 Processor -- start 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 -- start start 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5abd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5aa3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5aa1540 con 0x5587b5aa0be0 2023-07-21T19:10:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5ab8a60 con 0x5587b5aa02d0 2023-07-21T19:10:05.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.509+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5a9c8c0 con 0x5587b5aabad0 2023-07-21T19:10:05.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc51579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc51579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45626/0 (socket says 172.21.15.5:45626) 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc51579a700 1 -- 172.21.15.5:0/764621032 learned_addr learned my addr 172.21.15.5:0/764621032 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5abd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc51679c700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5aa3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 -- 172.21.15.5:0/764621032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 msgr2=0x5587b5aa3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5aa3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 -- 172.21.15.5:0/764621032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 msgr2=0x5587b5aab070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc515f9b700 1 -- 172.21.15.5:0/764621032 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5587b5847250 con 0x5587b5aa02d0 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.510+0000 7fc51579a700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.511+0000 7fc515f9b700 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5abd850 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc50c099c50 tx=0x7fc50c0ccfe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7435f1b9324b8b92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.511+0000 7fc504ff9700 1 -- 172.21.15.5:0/764621032 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c09a3b0 con 0x5587b5aa02d0 2023-07-21T19:10:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.511+0000 7fc504ff9700 1 -- 172.21.15.5:0/764621032 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc50c099a90 con 0x5587b5aa02d0 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.511+0000 7fc504ff9700 1 -- 172.21.15.5:0/764621032 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c0a2190 con 0x5587b5aa02d0 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- 172.21.15.5:0/764621032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 msgr2=0x5587b5abd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5abd850 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fc50c099c50 tx=0x7fc50c0ccfe0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- 172.21.15.5:0/764621032 shutdown_connections 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5aa3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5abd850 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/764621032 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5aab070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- 172.21.15.5:0/764621032 >> 172.21.15.5:0/764621032 conn(0x5587b5a96da0 msgr2=0x5587b5a9ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- 172.21.15.5:0/764621032 shutdown_connections 2023-07-21T19:10:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- 172.21.15.5:0/764621032 wait complete. 2023-07-21T19:10:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.512+0000 7fc52bb59b80 1 -- start start 2023-07-21T19:10:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 Processor -- start 2023-07-21T19:10:05.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5ab2400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5ab0490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5ab0a60 con 0x5587b5aa0be0 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5ab0ba0 con 0x5587b5aa02d0 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587b5ab0d60 con 0x5587b5aabad0 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.513+0000 7fc52bb59b80 10 client.0 _ll_register_callbacks cb 0x5587b5990760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc51579a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5ab2400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32846/0 (socket says 172.21.15.5:32846) 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 -- 172.21.15.5:0/1043362432 learned_addr learned my addr 172.21.15.5:0/1043362432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc51679c700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5ab0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 -- 172.21.15.5:0/1043362432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 msgr2=0x5587b5ab0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5ab0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 -- 172.21.15.5:0/1043362432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 msgr2=0x5587b5ab2400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5ab2400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 -- 172.21.15.5:0/1043362432 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc50001d040 con 0x5587b5aa02d0 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc515f9b700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc50c099530 tx=0x7fc50c095c90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd8f8e98817bfda server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.514+0000 7fc4f97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50c00cd20 con 0x5587b5aa02d0 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc500011e10 con 0x5587b5aa02d0 2023-07-21T19:10:05.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc50c092020 con 0x5587b5aa02d0 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc50c0998b0 con 0x5587b5aa02d0 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 10 client.25847 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5587b5809c60 con 0x5587b5aa02d0 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc50c0ca020 con 0x5587b5aa02d0 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc50c0d1040 con 0x5587b5aa02d0 2023-07-21T19:10:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 10 client.25847 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 10 client.25847 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.515+0000 7fc52bb59b80 20 client.25847 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5b10080 0x5587b5b12460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc52bb59b80 4 client.25847 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5587b59ae050 con 0x5587b5b10080 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc51579a700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5b10080 0x5587b5b12460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc4f97fa700 10 client.25847 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.516+0000 7fc51579a700 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5b10080 0x5587b5b12460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=add6974a8f759a8a server_cookie=b4c45524b855524f in_seq=0 out_seq=0 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc4f97fa700 1 -- 172.21.15.5:0/1043362432 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5587b59ae050 con 0x5587b5b10080 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc4f97fa700 10 client.25847 handle_command_reply: tid=0 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 client.25847 shutdown 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 msgr2=0x5587b5ab1ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc50c099530 tx=0x7fc50c095c90 comp rx=0 tx=0).stop 2023-07-21T19:10:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 shutdown_connections 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5587b5aabad0 0x5587b5ab0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5587b5b10080 0x5587b5b12460 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5587b5aa02d0 0x5587b5ab1ec0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 --2- 172.21.15.5:0/1043362432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5587b5aa0be0 0x5587b5ab2400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 >> 172.21.15.5:0/1043362432 conn(0x5587b5a96da0 msgr2=0x5587b5aa3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc4f97fa700 0 client.25847 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 shutdown_connections 2023-07-21T19:10:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.517+0000 7fc52bb59b80 1 -- 172.21.15.5:0/1043362432 wait complete. 2023-07-21T19:10:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc52bb59b80 20 client.25847 trim_cache size 0 max 16384 2023-07-21T19:10:05.536 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 msgr2=0x7fc4fc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 0x7fc4fc0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc508009fd0 tx=0x7fc5080099e0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 msgr2=0x7fc510197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fc50c00e9f0 tx=0x7fc50c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 shutdown_connections 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4fc0a12c0 0x7fc4fc0a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc51010a040 0x7fc51019c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc510100d00 0x7fc510197eb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 --2- 172.21.15.5:0/2725414870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc510101610 0x7fc5101983f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 >> 172.21.15.5:0/2725414870 conn(0x7fc5100fa4b0 msgr2=0x7fc51010c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.518+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 shutdown_connections 2023-07-21T19:10:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.519+0000 7fc5181ff700 1 -- 172.21.15.5:0/2725414870 wait complete. 2023-07-21T19:10:05.544 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:05 smithi005 ceph-mon[126460]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:10:05.545 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1193281524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:05.547 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:10:05.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:05 smithi156 ceph-mon[128316]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:10:05.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1193281524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:05.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.822+0000 7f49375ff700 1 -- 172.21.15.5:0/2872734926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4930100d20 msgr2=0x7f49301010f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.822+0000 7f49375ff700 1 --2- 172.21.15.5:0/2872734926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4930100d20 0x7f49301010f0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f492c0098f0 tx=0x7f492c009c00 comp rx=0 tx=0).stop 2023-07-21T19:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 -- 172.21.15.5:0/2872734926 shutdown_connections 2023-07-21T19:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 --2- 172.21.15.5:0/2872734926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f493010a060 0x7f493010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 --2- 172.21.15.5:0/2872734926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4930100d20 0x7f49301010f0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 --2- 172.21.15.5:0/2872734926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f4930109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 -- 172.21.15.5:0/2872734926 >> 172.21.15.5:0/2872734926 conn(0x7f49300fa4b0 msgr2=0x7f49300fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 -- 172.21.15.5:0/2872734926 shutdown_connections 2023-07-21T19:10:05.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.823+0000 7f49375ff700 1 -- 172.21.15.5:0/2872734926 wait complete. 2023-07-21T19:10:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 Processor -- start 2023-07-21T19:10:05.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 -- start start 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f49300feeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f493010a060 0x7f49301a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49300ff6e0 con 0x7f4930100d20 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301a42f0 con 0x7f4930101630 2023-07-21T19:10:05.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.824+0000 7f49375ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49301a4430 con 0x7f493010a060 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45648/0 (socket says 172.21.15.5:45648) 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 -- 172.21.15.5:0/588176597 learned_addr learned my addr 172.21.15.5:0/588176597 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f4934b9a700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f49300feeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 -- 172.21.15.5:0/588176597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f493010a060 msgr2=0x7f49301a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f493010a060 0x7f49301a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 -- 172.21.15.5:0/588176597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 msgr2=0x7f49300feeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f49300feeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 -- 172.21.15.5:0/588176597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4928009d00 con 0x7f4930100d20 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f4934b9a700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f49300feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.825+0000 7f493539b700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f492c009810 tx=0x7f492c00fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.826+0000 7f49267fc700 1 -- 172.21.15.5:0/588176597 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c01c070 con 0x7f4930100d20 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.826+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c009520 con 0x7f4930100d20 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.826+0000 7f49267fc700 1 -- 172.21.15.5:0/588176597 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f492c00bba0 con 0x7f4930100d20 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.826+0000 7f49267fc700 1 -- 172.21.15.5:0/588176597 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c0049c0 con 0x7f4930100d20 2023-07-21T19:10:05.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.826+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49301a4920 con 0x7f4930100d20 2023-07-21T19:10:05.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.830+0000 7f49267fc700 1 -- 172.21.15.5:0/588176597 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f492c020070 con 0x7f4930100d20 2023-07-21T19:10:05.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.831+0000 7f494af36b80 1 Processor -- start 2023-07-21T19:10:05.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.831+0000 7f49267fc700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 0x7f491c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.831+0000 7f494af36b80 1 -- start start 2023-07-21T19:10:05.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.831+0000 7f49267fc700 1 -- 172.21.15.5:0/588176597 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f492c0cda50 con 0x7f4930100d20 2023-07-21T19:10:05.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:05 smithi101 ceph-mon[128136]: pgmap v77: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:10:05.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1193281524' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbda66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f4935b9c700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 0x7f491c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbdb3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbcaccb0 con 0x562bcbda6be0 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbcffa40 con 0x562bcbdb1ad0 2023-07-21T19:10:05.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f494af36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbcf8ed0 con 0x562bcbda62d0 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45662/0 (socket says 172.21.15.5:45662) 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f4934b9a700 1 -- 172.21.15.5:0/621324660 learned_addr learned my addr 172.21.15.5:0/621324660 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4935b9c700 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbdb3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.832+0000 7f493539b700 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbda66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 -- 172.21.15.5:0/621324660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 msgr2=0x562bcbda66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbda66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 -- 172.21.15.5:0/621324660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 msgr2=0x562bcbdb3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbdb3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 -- 172.21.15.5:0/621324660 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcbb26de0 con 0x562bcbda6be0 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4935b9c700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 0x7f491c0a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4928009fd0 tx=0x7f492800bb60 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f4934b9a700 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4920009c90 tx=0x7f4920010370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30422a88cee51b67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.833+0000 7f491b7fe700 1 -- 172.21.15.5:0/621324660 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920004230 con 0x562bcbda6be0 2023-07-21T19:10:05.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f491b7fe700 1 -- 172.21.15.5:0/621324660 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4920010dc0 con 0x562bcbda6be0 2023-07-21T19:10:05.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f491b7fe700 1 -- 172.21.15.5:0/621324660 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920003cc0 con 0x562bcbda6be0 2023-07-21T19:10:05.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 -- 172.21.15.5:0/621324660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 msgr2=0x562bcbdb1590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4920009c90 tx=0x7f4920010370 comp rx=0 tx=0).stop 2023-07-21T19:10:05.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 -- 172.21.15.5:0/621324660 shutdown_connections 2023-07-21T19:10:05.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbdb3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbda66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 --2- 172.21.15.5:0/621324660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbdb1590 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 -- 172.21.15.5:0/621324660 >> 172.21.15.5:0/621324660 conn(0x562bcbd9cda0 msgr2=0x562bcbda26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.834+0000 7f494af36b80 1 -- 172.21.15.5:0/621324660 shutdown_connections 2023-07-21T19:10:05.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.835+0000 7f494af36b80 1 -- 172.21.15.5:0/621324660 wait complete. 2023-07-21T19:10:05.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.835+0000 7f494af36b80 1 -- start start 2023-07-21T19:10:05.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.835+0000 7f494af36b80 1 Processor -- start 2023-07-21T19:10:05.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbdae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 0x562bcbdaee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbc98ac0 con 0x562bcbda62d0 2023-07-21T19:10:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbc987c0 con 0x562bcbda6be0 2023-07-21T19:10:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbb7c920 con 0x562bcbdb1ad0 2023-07-21T19:10:05.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f494af36b80 10 client.0 _ll_register_callbacks cb 0x562bcbc96760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f493539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbdae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f493539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45676/0 (socket says 172.21.15.5:45676) 2023-07-21T19:10:05.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.836+0000 7f493539b700 1 -- 172.21.15.5:0/4112183503 learned_addr learned my addr 172.21.15.5:0/4112183503 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f4935b9c700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 0x562bcbdaee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 -- 172.21.15.5:0/4112183503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 msgr2=0x562bcbdaee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 0x562bcbdaee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 -- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 msgr2=0x562bcbdae7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbdae7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 -- 172.21.15.5:0/4112183503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49200098c0 con 0x562bcbda62d0 2023-07-21T19:10:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f4935b9c700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 0x562bcbdaee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:05.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f493539b700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f492c09ab10 tx=0x7f492c09ad00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e221ae4919ef7631 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.837+0000 7f4918ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f492c09afe0 con 0x562bcbda62d0 2023-07-21T19:10:05.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f492c098fe0 con 0x562bcbda62d0 2023-07-21T19:10:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bcbc524f0 con 0x562bcbda62d0 2023-07-21T19:10:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f492c0054c0 con 0x562bcbda62d0 2023-07-21T19:10:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 10 client.16125 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562bcbb0fc60 con 0x562bcbda62d0 2023-07-21T19:10:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bcbc524f0 con 0x562bcbda62d0 2023-07-21T19:10:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f492c0a2b40 con 0x562bcbda62d0 2023-07-21T19:10:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 10 client.16125 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 10 client.16125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 20 client.16125 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbdc98a0 0x562bcbdcbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 4 client.16125 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:05.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.838+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bcbcf1eb0 con 0x562bcbdc98a0 2023-07-21T19:10:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.839+0000 7f4934b9a700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbdc98a0 0x562bcbdcbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.840+0000 7f4934b9a700 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbdc98a0 0x562bcbdcbc80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0144d6f2980e07a server_cookie=69fc66104b316595 in_seq=0 out_seq=0 2023-07-21T19:10:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.840+0000 7f4918ff9700 10 client.16125 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.840+0000 7f4918ff9700 1 -- 172.21.15.5:0/4112183503 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bcbcf1eb0 con 0x562bcbdc98a0 2023-07-21T19:10:05.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.840+0000 7f4918ff9700 10 client.16125 handle_command_reply: tid=0 2023-07-21T19:10:05.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 client.16125 shutdown 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 msgr2=0x562bcbdb14c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f492c09ab10 tx=0x7f492c09ad00 comp rx=0 tx=0).stop 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 shutdown_connections 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbdb1ad0 0x562bcbdaee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbdc98a0 0x562bcbdcbc80 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda62d0 0x562bcbdb14c0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 --2- 172.21.15.5:0/4112183503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbdae7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 >> 172.21.15.5:0/4112183503 conn(0x562bcbd9cda0 msgr2=0x562bcbda2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f4918ff9700 0 client.16125 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 shutdown_connections 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 1 -- 172.21.15.5:0/4112183503 wait complete. 2023-07-21T19:10:05.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.841+0000 7f494af36b80 20 client.16125 trim_cache size 0 max 16384 2023-07-21T19:10:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.859+0000 7f494af36b80 1 Processor -- start 2023-07-21T19:10:05.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.859+0000 7f494af36b80 1 -- start start 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbdb84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbda9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbdaf1c0 con 0x562bcbda6be0 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbda7d60 con 0x562bcbdb1ad0 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f494af36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbd99f80 con 0x562bcbda62d0 2023-07-21T19:10:05.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45688/0 (socket says 172.21.15.5:45688) 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f4934b9a700 1 -- 172.21.15.5:0/3526493535 learned_addr learned my addr 172.21.15.5:0/3526493535 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f4935b9c700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbda9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.860+0000 7f493539b700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbdb84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 -- 172.21.15.5:0/3526493535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 msgr2=0x562bcbdb84f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbdb84f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 -- 172.21.15.5:0/3526493535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 msgr2=0x562bcbda9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbda9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 -- 172.21.15.5:0/3526493535 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bcbd9e5e0 con 0x562bcbda6be0 2023-07-21T19:10:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4935b9c700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbda9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:05.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f4934b9a700 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f49200046a0 tx=0x7f492001e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218f73589b7e46ce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f491bfff700 1 -- 172.21.15.5:0/3526493535 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920021070 con 0x562bcbda6be0 2023-07-21T19:10:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f491bfff700 1 -- 172.21.15.5:0/3526493535 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4920022040 con 0x562bcbda6be0 2023-07-21T19:10:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.861+0000 7f491bfff700 1 -- 172.21.15.5:0/3526493535 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200052a0 con 0x562bcbda6be0 2023-07-21T19:10:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 -- 172.21.15.5:0/3526493535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 msgr2=0x562bcbda8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f49200046a0 tx=0x7f492001e040 comp rx=0 tx=0).stop 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 -- 172.21.15.5:0/3526493535 shutdown_connections 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbdb1ad0 0x562bcbda9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbdb84f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 --2- 172.21.15.5:0/3526493535 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbda6be0 0x562bcbda8af0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 -- 172.21.15.5:0/3526493535 >> 172.21.15.5:0/3526493535 conn(0x562bcbd9cda0 msgr2=0x562bcbda83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 -- 172.21.15.5:0/3526493535 shutdown_connections 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.862+0000 7f494af36b80 1 -- 172.21.15.5:0/3526493535 wait complete. 2023-07-21T19:10:05.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.863+0000 7f494af36b80 1 -- start start 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.863+0000 7f494af36b80 1 Processor -- start 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbe118b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbe16c50 con 0x562bcbdb1ad0 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbe16d90 con 0x562bcbda6be0 2023-07-21T19:10:05.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bcbe16f50 con 0x562bcbda62d0 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f494af36b80 10 client.0 _ll_register_callbacks cb 0x7f492c09ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4935b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32878/0 (socket says 172.21.15.5:32878) 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4935b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45690/0 (socket says 172.21.15.5:45690) 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 -- 172.21.15.5:0/474950457 learned_addr learned my addr 172.21.15.5:0/474950457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f493539b700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbe118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 -- 172.21.15.5:0/474950457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 msgr2=0x562bcbe118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbe118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 -- 172.21.15.5:0/474950457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 msgr2=0x562bcbe16710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.864+0000 7f4934b9a700 1 -- 172.21.15.5:0/474950457 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bcbb26de0 con 0x562bcbda6be0 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4935b9c700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4934b9a700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4920005c50 tx=0x7f4920005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25be823ce5b74500 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:05.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4918ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:05.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920021040 con 0x562bcbda6be0 2023-07-21T19:10:05.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4920010dc0 con 0x562bcbda6be0 2023-07-21T19:10:05.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4920016070 con 0x562bcbda6be0 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49200289d0 con 0x562bcbda6be0 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f494af36b80 10 client.25862 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.865+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562bcbb4d250 con 0x562bcbda6be0 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4920010dc0 con 0x562bcbda6be0 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f492002f070 con 0x562bcbda6be0 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 10 client.25862 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:05.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 20 client.25862 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbe1e280 0x562bcbe20690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 4 client.25862 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.866+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562bcbcb4050 con 0x562bcbe1e280 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.867+0000 7f493539b700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbe1e280 0x562bcbe20690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.867+0000 7f493539b700 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbe1e280 0x562bcbe20690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3147776850c3b38e server_cookie=2b24cfafceeb86c8 in_seq=0 out_seq=0 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.867+0000 7f4918ff9700 10 client.25862 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.867+0000 7f4918ff9700 1 -- 172.21.15.5:0/474950457 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562bcbcb4050 con 0x562bcbe1e280 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f4918ff9700 10 client.25862 handle_command_reply: tid=0 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 client.25862 shutdown 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 msgr2=0x562bcbe11df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f4920005c50 tx=0x7f4920005770 comp rx=0 tx=0).stop 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 shutdown_connections 2023-07-21T19:10:05.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bcbe1e280 0x562bcbe20690 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bcbdb1ad0 0x562bcbe16710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bcbda62d0 0x562bcbe118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f4918ff9700 0 client.25862 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 --2- 172.21.15.5:0/474950457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bcbda6be0 0x562bcbe11df0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 >> 172.21.15.5:0/474950457 conn(0x562bcbd9cda0 msgr2=0x562bcbda8c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 shutdown_connections 2023-07-21T19:10:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 1 -- 172.21.15.5:0/474950457 wait complete. 2023-07-21T19:10:05.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.868+0000 7f494af36b80 20 client.25862 trim_cache size 0 max 16384 2023-07-21T19:10:05.885 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 msgr2=0x7f491c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 0x7f491c0a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4928009fd0 tx=0x7f492800bb60 comp rx=0 tx=0).stop 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 msgr2=0x7f49300fe970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f492c009810 tx=0x7f492c00fa00 comp rx=0 tx=0).stop 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 shutdown_connections 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f493010a060 0x7f49301a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f491c0a12c0 0x7f491c0a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4930100d20 0x7f49300fe970 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 --2- 172.21.15.5:0/588176597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4930101630 0x7f49300feeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:05.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 >> 172.21.15.5:0/588176597 conn(0x7f49300fa4b0 msgr2=0x7f49300fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:05.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 shutdown_connections 2023-07-21T19:10:05.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:05.869+0000 7f49375ff700 1 -- 172.21.15.5:0/588176597 wait complete. 2023-07-21T19:10:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:7400000 of 50000000 tuples (14%) done (elapsed 123.61 s, remaining 711.57 s) 2023-07-21T19:10:06.900 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:10:06.900 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:10:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 -- 172.21.15.5:0/4122192355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 msgr2=0x7f55cc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 --2- 172.21.15.5:0/4122192355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc0fff40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f55c0009990 tx=0x7f55c0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 -- 172.21.15.5:0/4122192355 shutdown_connections 2023-07-21T19:10:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 --2- 172.21.15.5:0/4122192355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55cc100480 0x7f55cc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 --2- 172.21.15.5:0/4122192355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc105ae0 0x7f55cc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 --2- 172.21.15.5:0/4122192355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc0fff40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 -- 172.21.15.5:0/4122192355 >> 172.21.15.5:0/4122192355 conn(0x7f55cc0fb5f0 msgr2=0x7f55cc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 -- 172.21.15.5:0/4122192355 shutdown_connections 2023-07-21T19:10:07.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.144+0000 7f55d0bff700 1 -- 172.21.15.5:0/4122192355 wait complete. 2023-07-21T19:10:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.145+0000 7f55d0bff700 1 Processor -- start 2023-07-21T19:10:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 -- start start 2023-07-21T19:10:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc100480 0x7f55cc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55cc105ae0 0x7f55cc195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc196000 con 0x7f55cc0ffad0 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc196170 con 0x7f55cc100480 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55d0bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55cc196330 con 0x7f55cc105ae0 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.146+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45334/0 (socket says 172.21.15.5:45334) 2023-07-21T19:10:07.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 -- 172.21.15.5:0/2658626088 learned_addr learned my addr 172.21.15.5:0/2658626088 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc100480 0x7f55cc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 -- 172.21.15.5:0/2658626088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55cc105ae0 msgr2=0x7f55cc195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55cc105ae0 0x7f55cc195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 -- 172.21.15.5:0/2658626088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc100480 msgr2=0x7f55cc19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc100480 0x7f55cc19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 -- 172.21.15.5:0/2658626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c00095f0 con 0x7f55cc0ffad0 2023-07-21T19:10:07.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.147+0000 7f55ca59c700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9a0 tx=0x7f55bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.148+0000 7f55bb7fe700 1 -- 172.21.15.5:0/2658626088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc009790 con 0x7f55cc0ffad0 2023-07-21T19:10:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.148+0000 7f55bb7fe700 1 -- 172.21.15.5:0/2658626088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55bc010430 con 0x7f55cc0ffad0 2023-07-21T19:10:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.148+0000 7f55bb7fe700 1 -- 172.21.15.5:0/2658626088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc018680 con 0x7f55cc0ffad0 2023-07-21T19:10:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.148+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55cc196610 con 0x7f55cc0ffad0 2023-07-21T19:10:07.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.148+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc1a28c0 con 0x7f55cc0ffad0 2023-07-21T19:10:07.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.152+0000 7f55bb7fe700 1 -- 172.21.15.5:0/2658626088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f55bc00cb50 con 0x7f55cc0ffad0 2023-07-21T19:10:07.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.154+0000 7f55e44bdb80 1 Processor -- start 2023-07-21T19:10:07.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.154+0000 7f55bb7fe700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 0x7f55b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.154+0000 7f55e44bdb80 1 -- start start 2023-07-21T19:10:07.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.154+0000 7f55bb7fe700 1 -- 172.21.15.5:0/2658626088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f55bc014070 con 0x7f55cc0ffad0 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 0x55fdeb66da20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 0x55fdeb670340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb564d30 con 0x55fdeb65ec60 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb5b7ac0 con 0x55fdeb66df60 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb5b0f50 con 0x55fdeb65e350 2023-07-21T19:10:07.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55cad9d700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 0x7f55b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42370/0 (socket says 172.21.15.5:42370) 2023-07-21T19:10:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55ca59c700 1 -- 172.21.15.5:0/3090644849 learned_addr learned my addr 172.21.15.5:0/3090644849 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.155+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 0x55fdeb66da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55cad9d700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 0x55fdeb670340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 -- 172.21.15.5:0/3090644849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 msgr2=0x55fdeb670340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 0x55fdeb670340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 -- 172.21.15.5:0/3090644849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 msgr2=0x55fdeb66da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 0x55fdeb66da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 -- 172.21.15.5:0/3090644849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdeb3dee60 con 0x55fdeb65e350 2023-07-21T19:10:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55cad9d700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 0x55fdeb670340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:07.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 0x55fdeb66da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:07.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55cad9d700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 0x7f55b40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fd0 tx=0x7f55c4009da0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55ca59c700 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009230 tx=0x7f55bc0999d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1d454b5cebd0655 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55b8ff9700 1 -- 172.21.15.5:0/3090644849 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc09dfd0 con 0x55fdeb65e350 2023-07-21T19:10:07.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55b8ff9700 1 -- 172.21.15.5:0/3090644849 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55bc0a23f0 con 0x55fdeb65e350 2023-07-21T19:10:07.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.156+0000 7f55b8ff9700 1 -- 172.21.15.5:0/3090644849 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc09a4d0 con 0x55fdeb65e350 2023-07-21T19:10:07.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.157+0000 7f55e44bdb80 1 -- 172.21.15.5:0/3090644849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 msgr2=0x55fdeb65e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.157+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009230 tx=0x7f55bc0999d0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 -- 172.21.15.5:0/3090644849 shutdown_connections 2023-07-21T19:10:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb66df60 0x55fdeb670340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65e350 0x55fdeb65e720 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/3090644849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65ec60 0x55fdeb66da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 -- 172.21.15.5:0/3090644849 >> 172.21.15.5:0/3090644849 conn(0x55fdeb654e20 msgr2=0x55fdeb65a720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 -- 172.21.15.5:0/3090644849 shutdown_connections 2023-07-21T19:10:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 -- 172.21.15.5:0/3090644849 wait complete. 2023-07-21T19:10:07.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.158+0000 7f55e44bdb80 1 -- start start 2023-07-21T19:10:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 Processor -- start 2023-07-21T19:10:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb66ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb66b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb550b40 con 0x55fdeb66df60 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb550840 con 0x55fdeb65e350 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb4349a0 con 0x55fdeb65ec60 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55e44bdb80 10 client.0 _ll_register_callbacks cb 0x55fdeb54e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.159+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45348/0 (socket says 172.21.15.5:45348) 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 -- 172.21.15.5:0/875860175 learned_addr learned my addr 172.21.15.5:0/875860175 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55ca59c700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb66ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb66b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 -- 172.21.15.5:0/875860175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 msgr2=0x55fdeb66b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb66b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 -- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 msgr2=0x55fdeb66ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb66ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.160+0000 7f55cad9d700 1 -- 172.21.15.5:0/875860175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55cc10d210 con 0x55fdeb66df60 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55cad9d700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55c4017a40 tx=0x7f55c4009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab0f6e3057f4cb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55c400e070 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fdeb50a570 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f55c4017e10 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55c4015840 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55e44bdb80 10 client.16143 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f55cc1a28c0 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.161+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fdeb50a570 con 0x55fdeb66df60 2023-07-21T19:10:07.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f55c4021990 con 0x55fdeb66df60 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 10 client.16143 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 20 client.16143 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb67d9a0 0x55fdeb67fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 4 client.16143 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fdeb5a9f30 con 0x55fdeb67d9a0 2023-07-21T19:10:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.162+0000 7f55ca59c700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb67d9a0 0x55fdeb67fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.163+0000 7f55ca59c700 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb67d9a0 0x55fdeb67fdb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc5d1f33d9e6138f server_cookie=9583a74016a67d68 in_seq=0 out_seq=0 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.163+0000 7f55b1ffb700 10 client.16143 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55b1ffb700 1 -- 172.21.15.5:0/875860175 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fdeb5a9f30 con 0x55fdeb67d9a0 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55b1ffb700 10 client.16143 handle_command_reply: tid=0 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55e44bdb80 1 client.16143 shutdown 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 msgr2=0x55fdeb66b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f55c4017a40 tx=0x7f55c4009920 comp rx=0 tx=0).stop 2023-07-21T19:10:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.164+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 shutdown_connections 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb67d9a0 0x55fdeb67fdb0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb66b780 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb66ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/875860175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb66b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55b1ffb700 0 client.16143 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 >> 172.21.15.5:0/875860175 conn(0x55fdeb654e20 msgr2=0x55fdeb65a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 shutdown_connections 2023-07-21T19:10:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.165+0000 7f55e44bdb80 1 -- 172.21.15.5:0/875860175 wait complete. 2023-07-21T19:10:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.166+0000 7f55e44bdb80 20 client.16143 trim_cache size 0 max 16384 2023-07-21T19:10:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 Processor -- start 2023-07-21T19:10:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 -- start start 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65ec60 0x55fdeb660b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb66df60 0x55fdeb667ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb656f40 con 0x55fdeb65e350 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb66dd40 con 0x55fdeb65ec60 2023-07-21T19:10:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.185+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb67af20 con 0x55fdeb66df60 2023-07-21T19:10:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45350/0 (socket says 172.21.15.5:45350) 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 -- 172.21.15.5:0/4140832274 learned_addr learned my addr 172.21.15.5:0/4140832274 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55cad9d700 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb66df60 0x55fdeb667ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65ec60 0x55fdeb660b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 -- 172.21.15.5:0/4140832274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb66df60 msgr2=0x55fdeb667ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb66df60 0x55fdeb667ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 -- 172.21.15.5:0/4140832274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65ec60 msgr2=0x55fdeb660b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65ec60 0x55fdeb660b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 -- 172.21.15.5:0/4140832274 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fdeb656660 con 0x55fdeb65e350 2023-07-21T19:10:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.186+0000 7f55ca59c700 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c5b0 tx=0x7f55bc095cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ced1a60310960b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.187+0000 7f55b97fa700 1 -- 172.21.15.5:0/4140832274 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0b8070 con 0x55fdeb65e350 2023-07-21T19:10:07.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.187+0000 7f55b97fa700 1 -- 172.21.15.5:0/4140832274 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55bc098340 con 0x55fdeb65e350 2023-07-21T19:10:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.187+0000 7f55b97fa700 1 -- 172.21.15.5:0/4140832274 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0996f0 con 0x55fdeb65e350 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- 172.21.15.5:0/4140832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 msgr2=0x55fdeb66a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00c5b0 tx=0x7f55bc095cb0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- 172.21.15.5:0/4140832274 shutdown_connections 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb66df60 0x55fdeb667ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb65e350 0x55fdeb66a520 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/4140832274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65ec60 0x55fdeb660b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- 172.21.15.5:0/4140832274 >> 172.21.15.5:0/4140832274 conn(0x55fdeb654e20 msgr2=0x55fdeb660460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- 172.21.15.5:0/4140832274 shutdown_connections 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- 172.21.15.5:0/4140832274 wait complete. 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.188+0000 7f55e44bdb80 1 -- start start 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 Processor -- start 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb672c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb6731b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb673880 con 0x55fdeb66df60 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb6739f0 con 0x55fdeb65e350 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fdeb6d1830 con 0x55fdeb65ec60 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.189+0000 7f55e44bdb80 10 client.0 _ll_register_callbacks cb 0x55fdeb54e7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb6731b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58828/0 (socket says 172.21.15.5:58828) 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 -- 172.21.15.5:0/611058190 learned_addr learned my addr 172.21.15.5:0/611058190 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb672c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 -- 172.21.15.5:0/611058190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 msgr2=0x55fdeb672c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb672c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 -- 172.21.15.5:0/611058190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 msgr2=0x55fdeb6731b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb6731b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 -- 172.21.15.5:0/611058190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fdeb3dee60 con 0x55fdeb65e350 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55cad9d700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb6731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55ca59c700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009230 tx=0x7f55bc0108e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=424ea01966a85cd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.190+0000 7f55b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55bc0b8040 con 0x55fdeb65e350 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f55bc09a200 con 0x55fdeb65e350 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f55bc0a23f0 con 0x55fdeb65e350 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55bc09a3c0 con 0x55fdeb65e350 2023-07-21T19:10:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 10 client.25877 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55fdeb3c7ce0 con 0x55fdeb65e350 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55bc0a23f0 con 0x55fdeb65e350 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f55bc0a1f60 con 0x55fdeb65e350 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 10 client.25877 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 10 client.25877 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 20 client.25877 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb6d29f0 0x55fdeb6d4e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 4 client.25877 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:10:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.191+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55fdeb56c0d0 con 0x55fdeb6d29f0 2023-07-21T19:10:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.192+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb6d29f0 0x55fdeb6d4e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.192+0000 7f55c9d9b700 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb6d29f0 0x55fdeb6d4e00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19c419574ba775e0 server_cookie=c333ebc3feaacb60 in_seq=0 out_seq=0 2023-07-21T19:10:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.192+0000 7f55b1ffb700 10 client.25877 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55b1ffb700 1 -- 172.21.15.5:0/611058190 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55fdeb56c0d0 con 0x55fdeb6d29f0 2023-07-21T19:10:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55b1ffb700 10 client.25877 handle_command_reply: tid=0 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 client.25877 shutdown 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 msgr2=0x55fdeb6749b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f55bc009230 tx=0x7f55bc0108e0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 shutdown_connections 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fdeb6d29f0 0x55fdeb6d4e00 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.193+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fdeb66df60 0x55fdeb6731b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fdeb65e350 0x55fdeb6749b0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55b1ffb700 0 client.25877 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 1 --2- 172.21.15.5:0/611058190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fdeb65ec60 0x55fdeb672c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 >> 172.21.15.5:0/611058190 conn(0x55fdeb654e20 msgr2=0x55fdeb660cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 shutdown_connections 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 1 -- 172.21.15.5:0/611058190 wait complete. 2023-07-21T19:10:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.194+0000 7f55e44bdb80 20 client.25877 trim_cache size 0 max 16384 2023-07-21T19:10:07.211 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:07.211 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:10:07.211 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "0067487d-244b-4f32-8048-1145bba8c5c8", 2023-07-21T19:10:07.212 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:10:07.212 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 msgr2=0x7f55b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 0x7f55b40a37f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f55c4009fd0 tx=0x7f55c4009da0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 msgr2=0x7f55cc19b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f55bc00e9a0 tx=0x7f55bc00ed60 comp rx=0 tx=0).stop 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 shutdown_connections 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55cc105ae0 0x7f55cc195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f55b40a1340 0x7f55b40a37f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f55cc0ffad0 0x7f55cc19b8d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 --2- 172.21.15.5:0/2658626088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55cc100480 0x7f55cc19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 >> 172.21.15.5:0/2658626088 conn(0x7f55cc0fb5f0 msgr2=0x7f55cc0fbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 shutdown_connections 2023-07-21T19:10:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.195+0000 7f55d0bff700 1 -- 172.21.15.5:0/2658626088 wait complete. 2023-07-21T19:10:07.225 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:10:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 -- 172.21.15.5:0/2304758274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 msgr2=0x7f58b8102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/2304758274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b8102230 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0099c0 tx=0x7f58ac009cd0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 -- 172.21.15.5:0/2304758274 shutdown_connections 2023-07-21T19:10:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/2304758274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8106ea0 0x7f58b810ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/2304758274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b8102230 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/2304758274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8102800 0x7f58b81067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.471+0000 7f58bdd9d700 1 -- 172.21.15.5:0/2304758274 >> 172.21.15.5:0/2304758274 conn(0x7f58b80fd690 msgr2=0x7f58b80ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.472+0000 7f58bdd9d700 1 -- 172.21.15.5:0/2304758274 shutdown_connections 2023-07-21T19:10:07.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.472+0000 7f58bdd9d700 1 -- 172.21.15.5:0/2304758274 wait complete. 2023-07-21T19:10:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 Processor -- start 2023-07-21T19:10:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 -- start start 2023-07-21T19:10:07.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 0x7f58b819de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8106ea0 0x7f58b8197970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8197fe0 con 0x7f58b8101e60 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8198150 con 0x7f58b8102800 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.473+0000 7f58bdd9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58b8198310 con 0x7f58b8106ea0 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45378/0 (socket says 172.21.15.5:45378) 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58bcd9b700 1 -- 172.21.15.5:0/3594472607 learned_addr learned my addr 172.21.15.5:0/3594472607 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 0x7f58b819de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8106ea0 0x7f58b8197970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 -- 172.21.15.5:0/3594472607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8106ea0 msgr2=0x7f58b8197970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8106ea0 0x7f58b8197970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 -- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 msgr2=0x7f58b819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58b7fff700 1 -- 172.21.15.5:0/3594472607 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac0095f0 con 0x7f58b8102800 2023-07-21T19:10:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.474+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:07.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58b7fff700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 0x7f58b819de40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58b8102bd0 tx=0x7f58a400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58b5ffb700 1 -- 172.21.15.5:0/3594472607 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a400cbd0 con 0x7f58b8102800 2023-07-21T19:10:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58b81985f0 con 0x7f58b8102800 2023-07-21T19:10:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58b5ffb700 1 -- 172.21.15.5:0/3594472607 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58a400ee80 con 0x7f58b8102800 2023-07-21T19:10:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58b5ffb700 1 -- 172.21.15.5:0/3594472607 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58a4018680 con 0x7f58b8102800 2023-07-21T19:10:07.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.475+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b81a48f0 con 0x7f58b8102800 2023-07-21T19:10:07.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.480+0000 7f58b5ffb700 1 -- 172.21.15.5:0/3594472607 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f58a4010430 con 0x7f58b8102800 2023-07-21T19:10:07.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 Processor -- start 2023-07-21T19:10:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58b5ffb700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 0x7f58a80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 -- start start 2023-07-21T19:10:07.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58b5ffb700 1 -- 172.21.15.5:0/3594472607 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f58a4014070 con 0x7f58b8102800 2023-07-21T19:10:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b1b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b1e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7a16cb0 con 0x555fa7b1bd30 2023-07-21T19:10:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7a69a40 con 0x555fa7b114f0 2023-07-21T19:10:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.482+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7a62ed0 con 0x555fa7b11e00 2023-07-21T19:10:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 0x7f58a80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58860/0 (socket says 172.21.15.5:58860) 2023-07-21T19:10:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bcd9b700 1 -- 172.21.15.5:0/2515510048 learned_addr learned my addr 172.21.15.5:0/2515510048 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bcd9b700 1 -- 172.21.15.5:0/2515510048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 msgr2=0x555fa7b1b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bd59c700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b1e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58b7fff700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b1b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.483+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b1b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bcd9b700 1 -- 172.21.15.5:0/2515510048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 msgr2=0x555fa7b1e110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b1e110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bcd9b700 1 -- 172.21.15.5:0/2515510048 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa7890de0 con 0x555fa7b114f0 2023-07-21T19:10:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bd59c700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:07.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58b7fff700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005ca0 tx=0x7f58ac005ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d734dc01c1e00bd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 0x7f58a80a3810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58b819e600 tx=0x7f58b0009050 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.484+0000 7f58a2ffd700 1 -- 172.21.15.5:0/2515510048 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0050a0 con 0x555fa7b114f0 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.485+0000 7f58a2ffd700 1 -- 172.21.15.5:0/2515510048 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58ac01e040 con 0x555fa7b114f0 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.485+0000 7f58a2ffd700 1 -- 172.21.15.5:0/2515510048 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac00f460 con 0x555fa7b114f0 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.485+0000 7f58d2a1db80 1 -- 172.21.15.5:0/2515510048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 msgr2=0x555fa7b118c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.485+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f58ac005ca0 tx=0x7f58ac005ee0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 -- 172.21.15.5:0/2515510048 shutdown_connections 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b1e110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b118c0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/2515510048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b1b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 -- 172.21.15.5:0/2515510048 >> 172.21.15.5:0/2515510048 conn(0x555fa7b06ff0 msgr2=0x555fa7b0c930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 -- 172.21.15.5:0/2515510048 shutdown_connections 2023-07-21T19:10:07.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.486+0000 7f58d2a1db80 1 -- 172.21.15.5:0/2515510048 wait complete. 2023-07-21T19:10:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.487+0000 7f58d2a1db80 1 -- start start 2023-07-21T19:10:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.487+0000 7f58d2a1db80 1 Processor -- start 2023-07-21T19:10:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 0x555fa7b1b650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 0x555fa7b188d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 0x555fa7b18e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7a02ac0 con 0x555fa7b11e00 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7a027c0 con 0x555fa7b1bd30 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa78e6920 con 0x555fa7b114f0 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58d2a1db80 10 client.0 _ll_register_callbacks cb 0x555fa7a00760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 0x555fa7b1b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58bd59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 0x555fa7b18e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 0x555fa7b188d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 0x555fa7b1b650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42420/0 (socket says 172.21.15.5:42420) 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 0x555fa7b188d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45394/0 (socket says 172.21.15.5:45394) 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.488+0000 7f58bcd9b700 1 -- 172.21.15.5:0/1654102652 learned_addr learned my addr 172.21.15.5:0/1654102652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 -- 172.21.15.5:0/1654102652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 msgr2=0x555fa7b1b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 0x555fa7b1b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 -- 172.21.15.5:0/1654102652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 msgr2=0x555fa7b188d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 0x555fa7b188d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 -- 172.21.15.5:0/1654102652 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58b810f2b0 con 0x555fa7b1bd30 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f58bd59c700 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 0x555fa7b18e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007b70 tx=0x7f58b0004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4de26f6762a2d78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f5897fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.489+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0025070 con 0x555fa7b1bd30 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fa79bc4f0 con 0x555fa7b1bd30 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f58b0017e30 con 0x555fa7b1bd30 2023-07-21T19:10:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58b0004bb0 con 0x555fa7b1bd30 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 10 client.25889 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555fa7879c60 con 0x555fa7b1bd30 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58b002d020 con 0x555fa7b1bd30 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58b0020070 con 0x555fa7b1bd30 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 10 client.25889 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 10 client.25889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.490+0000 7f58d2a1db80 20 client.25889 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.491+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b33ec0 0x555fa7b362d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.491+0000 7f58d2a1db80 4 client.25889 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.491+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fa7a5beb0 con 0x555fa7b33ec0 2023-07-21T19:10:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.491+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b33ec0 0x555fa7b362d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.492+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b33ec0 0x555fa7b362d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e08a271f19f7e46b server_cookie=199bf60bb5fd9f64 in_seq=0 out_seq=0 2023-07-21T19:10:07.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.492+0000 7f5897fff700 10 client.25889 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.493+0000 7f5897fff700 1 -- 172.21.15.5:0/1654102652 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fa7a5beb0 con 0x555fa7b33ec0 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.493+0000 7f5897fff700 10 client.25889 handle_command_reply: tid=0 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.493+0000 7f58d2a1db80 1 client.25889 shutdown 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.493+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 msgr2=0x555fa7b18e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.493+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 0x555fa7b18e70 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007b70 tx=0x7f58b0004ad0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 shutdown_connections 2023-07-21T19:10:07.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b1bd30 0x555fa7b18e70 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b114f0 0x555fa7b1b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b33ec0 0x555fa7b362d0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/1654102652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b11e00 0x555fa7b188d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 >> 172.21.15.5:0/1654102652 conn(0x555fa7b06ff0 msgr2=0x555fa7b0c520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f5897fff700 0 client.25889 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 shutdown_connections 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.494+0000 7f58d2a1db80 1 -- 172.21.15.5:0/1654102652 wait complete. 2023-07-21T19:10:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.495+0000 7f58d2a1db80 20 client.25889 trim_cache size 0 max 16384 2023-07-21T19:10:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 Processor -- start 2023-07-21T19:10:07.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 -- start start 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 0x555fa7b2b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b11e00 0x555fa7b13c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b12980 con 0x555fa7b114f0 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b19eb0 con 0x555fa7b11e00 2023-07-21T19:10:07.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.515+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b0d5e0 con 0x555fa7b1bd30 2023-07-21T19:10:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 0x555fa7b2b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42426/0 (socket says 172.21.15.5:42426) 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 -- 172.21.15.5:0/3096119763 learned_addr learned my addr 172.21.15.5:0/3096119763 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58b7fff700 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b11e00 0x555fa7b13c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 -- 172.21.15.5:0/3096119763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b11e00 msgr2=0x555fa7b13c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b11e00 0x555fa7b13c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 -- 172.21.15.5:0/3096119763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 msgr2=0x555fa7b2b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 0x555fa7b2b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bd59c700 1 -- 172.21.15.5:0/3096119763 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fa78b7250 con 0x555fa7b1bd30 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.516+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 0x555fa7b2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.517+0000 7f58bd59c700 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007cd0 tx=0x7f58b00166a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32390cc8b6090245 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.517+0000 7f58a37fe700 1 -- 172.21.15.5:0/3096119763 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0025070 con 0x555fa7b1bd30 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.517+0000 7f58a37fe700 1 -- 172.21.15.5:0/3096119763 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58b002d040 con 0x555fa7b1bd30 2023-07-21T19:10:07.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.517+0000 7f58a37fe700 1 -- 172.21.15.5:0/3096119763 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b0004070 con 0x555fa7b1bd30 2023-07-21T19:10:07.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 -- 172.21.15.5:0/3096119763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 msgr2=0x555fa7b141d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f58b0007cd0 tx=0x7f58b00166a0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 -- 172.21.15.5:0/3096119763 shutdown_connections 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b1bd30 0x555fa7b141d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b114f0 0x555fa7b2b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/3096119763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b11e00 0x555fa7b13c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 -- 172.21.15.5:0/3096119763 >> 172.21.15.5:0/3096119763 conn(0x555fa7b06ff0 msgr2=0x555fa7b1b280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 -- 172.21.15.5:0/3096119763 shutdown_connections 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.518+0000 7f58d2a1db80 1 -- 172.21.15.5:0/3096119763 wait complete. 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 -- start start 2023-07-21T19:10:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 Processor -- start 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b81750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b7ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b80530 con 0x555fa7b1bd30 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b806a0 con 0x555fa7b114f0 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fa7b80860 con 0x555fa7b11e00 2023-07-21T19:10:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.519+0000 7f58d2a1db80 10 client.0 _ll_register_callbacks cb 0x7f58b0017800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bd59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b7ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58896/0 (socket says 172.21.15.5:58896) 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 -- 172.21.15.5:0/115112921 learned_addr learned my addr 172.21.15.5:0/115112921 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58b7fff700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b81750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 -- 172.21.15.5:0/115112921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 msgr2=0x555fa7b81750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b81750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 -- 172.21.15.5:0/115112921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 msgr2=0x555fa7b7ff80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b7ff80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 -- 172.21.15.5:0/115112921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fa7890de0 con 0x555fa7b114f0 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58b7fff700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b81750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bd59c700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f58bcd9b700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58ac000c00 tx=0x7f58ac00bc70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2484dd888bc58bb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.520+0000 7f5897fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0050c0 con 0x555fa7b114f0 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f58ac01e040 con 0x555fa7b114f0 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f58ac005220 con 0x555fa7b114f0 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58ac0053e0 con 0x555fa7b114f0 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f58d2a1db80 10 client.25895 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555fa7879c60 con 0x555fa7b114f0 2023-07-21T19:10:07.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.521+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f58ac042020 con 0x555fa7b114f0 2023-07-21T19:10:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f58ac021070 con 0x555fa7b114f0 2023-07-21T19:10:07.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 10 client.25895 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 10 client.25895 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 20 client.25895 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b88d90 0x555fa7b8b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 4 client.25895 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555fa7a1e050 con 0x555fa7b88d90 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58b7fff700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b88d90 0x555fa7b8b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:07.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f58b7fff700 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b88d90 0x555fa7b8b1a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0f1e570ee6ae26 server_cookie=e4cd2ea9a233f653 in_seq=0 out_seq=0 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.522+0000 7f5897fff700 10 client.25895 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f5897fff700 1 -- 172.21.15.5:0/115112921 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555fa7a1e050 con 0x555fa7b88d90 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f5897fff700 10 client.25895 handle_command_reply: tid=0 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 client.25895 shutdown 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 msgr2=0x555fa7b81210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f58ac000c00 tx=0x7f58ac00bc70 comp rx=0 tx=0).stop 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 shutdown_connections 2023-07-21T19:10:07.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fa7b88d90 0x555fa7b8b1a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fa7b1bd30 0x555fa7b7ff80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fa7b114f0 0x555fa7b81210 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 --2- 172.21.15.5:0/115112921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fa7b11e00 0x555fa7b81750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 >> 172.21.15.5:0/115112921 conn(0x555fa7b06ff0 msgr2=0x555fa7b13c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f5897fff700 0 client.25895 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 shutdown_connections 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.523+0000 7f58d2a1db80 1 -- 172.21.15.5:0/115112921 wait complete. 2023-07-21T19:10:07.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58d2a1db80 20 client.25895 trim_cache size 0 max 16384 2023-07-21T19:10:07.540 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:07.540 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:10:07.540 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:10:07.540 INFO:teuthology.orchestra.run.smithi005.stdout: "0067487d-244b-4f32-8048-1145bba8c5c8": { 2023-07-21T19:10:07.540 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:10:07.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "0067487d-244b-4f32-8048-1145bba8c5c8", 2023-07-21T19:10:07.541 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:10:07.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:10:07.541 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:10:07.541 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:07.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 msgr2=0x7f58a80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 0x7f58a80a3810 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f58b819e600 tx=0x7f58b0009050 comp rx=0 tx=0).stop 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 msgr2=0x7f58b819de40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 0x7f58b819de40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f58b8102bd0 tx=0x7f58a400eca0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 shutdown_connections 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f58b8106ea0 0x7f58b8197970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58a80a1360 0x7f58a80a3810 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f58b8101e60 0x7f58b819d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 --2- 172.21.15.5:0/3594472607 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f58b8102800 0x7f58b819de40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 >> 172.21.15.5:0/3594472607 conn(0x7f58b80fd690 msgr2=0x7f58b80fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 shutdown_connections 2023-07-21T19:10:07.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:07.524+0000 7f58bdd9d700 1 -- 172.21.15.5:0/3594472607 wait complete. 2023-07-21T19:10:07.554 INFO:tasks.cephfs.filesystem:scrub status for tag:0067487d-244b-4f32-8048-1145bba8c5c8 - {'path': '/', 'tag': '0067487d-244b-4f32-8048-1145bba8c5c8', 'options': 'recursive,force'} 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 32 op/s 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:07.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 32 op/s 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: pgmap v78: 97 pgs: 97 active+clean; 1.7 GiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 23 MiB/s wr, 32 op/s 2023-07-21T19:10:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:07.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.842 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:07.966 INFO:teuthology.orchestra.run.smithi005.stderr:7500000 of 50000000 tuples (15%) done (elapsed 125.52 s, remaining 711.29 s) 2023-07-21T19:10:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:08 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:10:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:08 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:08 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:08.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:08 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:08 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:10:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:08 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:08.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:08 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:08.681 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:08 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:08.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:08 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:10:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:08 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:08 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:08.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:08 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:09.197 INFO:teuthology.orchestra.run.smithi005.stderr:7600000 of 50000000 tuples (15%) done (elapsed 126.75 s, remaining 707.15 s) 2023-07-21T19:10:09.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:09 smithi005 ceph-mon[126460]: pgmap v79: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-07-21T19:10:09.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:09 smithi156 ceph-mon[128316]: pgmap v79: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-07-21T19:10:09.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:09 smithi101 ceph-mon[128136]: pgmap v79: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 34 MiB/s wr, 53 op/s 2023-07-21T19:10:11.078 INFO:teuthology.orchestra.run.smithi005.stderr:7700000 of 50000000 tuples (15%) done (elapsed 128.63 s, remaining 706.66 s) 2023-07-21T19:10:11.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:11 smithi005 ceph-mon[126460]: pgmap v80: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:10:11.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:11 smithi156 ceph-mon[128316]: pgmap v80: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:10:11.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:11 smithi101 ceph-mon[128136]: pgmap v80: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 23 MiB/s wr, 43 op/s 2023-07-21T19:10:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:7800000 of 50000000 tuples (15%) done (elapsed 130.38 s, remaining 705.39 s) 2023-07-21T19:10:13.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:13 smithi005 ceph-mon[126460]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 32 MiB/s wr, 118 op/s 2023-07-21T19:10:13.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:13 smithi156 ceph-mon[128316]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 32 MiB/s wr, 118 op/s 2023-07-21T19:10:13.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:13 smithi101 ceph-mon[128136]: pgmap v81: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 32 MiB/s wr, 118 op/s 2023-07-21T19:10:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:7900000 of 50000000 tuples (15%) done (elapsed 131.72 s, remaining 701.93 s) 2023-07-21T19:10:15.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:15 smithi156 ceph-mon[128316]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 26 MiB/s wr, 128 op/s 2023-07-21T19:10:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:15 smithi101 ceph-mon[128136]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 26 MiB/s wr, 128 op/s 2023-07-21T19:10:15.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:15 smithi005 ceph-mon[126460]: pgmap v82: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 146 KiB/s rd, 26 MiB/s wr, 128 op/s 2023-07-21T19:10:16.035 INFO:teuthology.orchestra.run.smithi005.stderr:8000000 of 50000000 tuples (16%) done (elapsed 133.59 s, remaining 701.36 s) 2023-07-21T19:10:17.371 INFO:teuthology.orchestra.run.smithi005.stderr:8100000 of 50000000 tuples (16%) done (elapsed 134.93 s, remaining 697.96 s) 2023-07-21T19:10:17.680 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:17 smithi156 ceph-mon[128316]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 111 op/s 2023-07-21T19:10:17.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:17 smithi101 ceph-mon[128136]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 111 op/s 2023-07-21T19:10:17.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:17 smithi005 ceph-mon[126460]: pgmap v83: 97 pgs: 97 active+clean; 1.8 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 22 MiB/s wr, 111 op/s 2023-07-21T19:10:18.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 -- 172.21.15.5:0/975918941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f840ffd20 msgr2=0x7f3f841000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 --2- 172.21.15.5:0/975918941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f840ffd20 0x7f3f841000f0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f3f74009990 tx=0x7f3f74009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 -- 172.21.15.5:0/975918941 shutdown_connections 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 --2- 172.21.15.5:0/975918941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f84101e50 0x7f3f8410c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 --2- 172.21.15.5:0/975918941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f840ffd20 0x7f3f841000f0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 --2- 172.21.15.5:0/975918941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f841014a0 0x7f3f84101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 -- 172.21.15.5:0/975918941 >> 172.21.15.5:0/975918941 conn(0x7f3f840fb440 msgr2=0x7f3f840fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 -- 172.21.15.5:0/975918941 shutdown_connections 2023-07-21T19:10:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.481+0000 7f3f83fff700 1 -- 172.21.15.5:0/975918941 wait complete. 2023-07-21T19:10:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.482+0000 7f3f83fff700 1 Processor -- start 2023-07-21T19:10:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.482+0000 7f3f83fff700 1 -- start start 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f841014a0 0x7f3f8419dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f84101e50 0x7f3f84197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84198250 con 0x7f3f84101e50 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f841983c0 con 0x7f3f841014a0 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f83fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f84198580 con 0x7f3f840ffd20 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f84101e50 0x7f3f84197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39018/0 (socket says 172.21.15.5:39018) 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 -- 172.21.15.5:0/1253539168 learned_addr learned my addr 172.21.15.5:0/1253539168 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f827fc700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f841014a0 0x7f3f8419dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 -- 172.21.15.5:0/1253539168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f841014a0 msgr2=0x7f3f8419dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f841014a0 0x7f3f8419dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 -- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f84101e50 msgr2=0x7f3f84197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f84101e50 0x7f3f84197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.483+0000 7f3f82ffd700 1 -- 172.21.15.5:0/1253539168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f740095f0 con 0x7f3f840ffd20 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005910 tx=0x7f3f740047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f6bfff700 1 -- 172.21.15.5:0/1253539168 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7401d070 con 0x7f3f840ffd20 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f6bfff700 1 -- 172.21.15.5:0/1253539168 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f74021410 con 0x7f3f840ffd20 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f84198800 con 0x7f3f840ffd20 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f6bfff700 1 -- 172.21.15.5:0/1253539168 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f74017650 con 0x7f3f840ffd20 2023-07-21T19:10:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.484+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f841a4a20 con 0x7f3f840ffd20 2023-07-21T19:10:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.488+0000 7f3f6bfff700 1 -- 172.21.15.5:0/1253539168 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3f74021580 con 0x7f3f840ffd20 2023-07-21T19:10:18.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 Processor -- start 2023-07-21T19:10:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f6bfff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 0x7f3f6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 -- start start 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f6bfff700 1 -- 172.21.15.5:0/1253539168 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3f740cde00 con 0x7f3f840ffd20 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f837fe700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 0x7f3f6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b8559700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b845ec50 con 0x5616b855f470 2023-07-21T19:10:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b84b19e0 con 0x5616b83b56b0 2023-07-21T19:10:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.490+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b84aae70 con 0x5616b855f8c0 2023-07-21T19:10:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45572/0 (socket says 172.21.15.5:45572) 2023-07-21T19:10:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 -- 172.21.15.5:0/2882361939 learned_addr learned my addr 172.21.15.5:0/2882361939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f837fe700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 0x7f3f6c0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f8419e730 tx=0x7f3f7c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 -- 172.21.15.5:0/2882361939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 msgr2=0x5616b8559700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b8559700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 -- 172.21.15.5:0/2882361939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 msgr2=0x5616b855ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.491+0000 7f3f827fc700 1 -- 172.21.15.5:0/2882361939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616b82d8d80 con 0x5616b855f470 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.492+0000 7f3f827fc700 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009cf0 tx=0x7f3f78011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577e4304e6cc4eca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.492+0000 7f3f697fa700 1 -- 172.21.15.5:0/2882361939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78011d80 con 0x5616b855f470 2023-07-21T19:10:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.492+0000 7f3f697fa700 1 -- 172.21.15.5:0/2882361939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f780044e0 con 0x5616b855f470 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.492+0000 7f3f697fa700 1 -- 172.21.15.5:0/2882361939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78005280 con 0x5616b855f470 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/2882361939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 msgr2=0x5616b85591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009cf0 tx=0x7f3f78011440 comp rx=0 tx=0).stop 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/2882361939 shutdown_connections 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b8559700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/2882361939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b85591a0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/2882361939 >> 172.21.15.5:0/2882361939 conn(0x5616b854ed40 msgr2=0x5616b8554340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/2882361939 shutdown_connections 2023-07-21T19:10:18.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/2882361939 wait complete. 2023-07-21T19:10:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.493+0000 7f3f9ce1bb80 1 -- start start 2023-07-21T19:10:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 Processor -- start 2023-07-21T19:10:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 0x5616b8565830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 0x5616b85631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b844aa60 con 0x5616b855f470 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b844a760 con 0x5616b855f8c0 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b832e8c0 con 0x5616b83b56b0 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.494+0000 7f3f9ce1bb80 10 client.0 _ll_register_callbacks cb 0x5616b8448700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45580/0 (socket says 172.21.15.5:45580) 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 -- 172.21.15.5:0/717216048 learned_addr learned my addr 172.21.15.5:0/717216048 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:18.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f837fe700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 0x5616b85631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 0x5616b8565830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 -- 172.21.15.5:0/717216048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 msgr2=0x5616b8565830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 0x5616b8565830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 -- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 msgr2=0x5616b85631f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 0x5616b85631f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 -- 172.21.15.5:0/717216048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f78009920 con 0x5616b855f470 2023-07-21T19:10:18.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f837fe700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 0x5616b85631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 0x5616b8565830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.495+0000 7f3f827fc700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009cc0 tx=0x7f3f78011c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a4a23b0e9b336f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78004260 con 0x5616b855f470 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616b8404490 con 0x5616b855f470 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3f78003680 con 0x5616b855f470 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f78003880 con 0x5616b855f470 2023-07-21T19:10:18.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f9ce1bb80 10 client.16176 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5616b82c1c00 con 0x5616b855f470 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.496+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5616b8404490 con 0x5616b855f470 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f78005210 con 0x5616b855f470 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 10 client.16176 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 10 client.16176 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 20 client.16176 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:18.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b8572cc0 0x5616b85750d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 4 client.16176 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5616b84a3e50 con 0x5616b8572cc0 2023-07-21T19:10:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.497+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b8572cc0 0x5616b85750d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.498+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b8572cc0 0x5616b85750d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76e28f87ccf1d781 server_cookie=eb33032a97616d2d in_seq=0 out_seq=0 2023-07-21T19:10:18.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.498+0000 7f3f66ff5700 10 client.16176 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f66ff5700 1 -- 172.21.15.5:0/717216048 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5616b84a3e50 con 0x5616b8572cc0 2023-07-21T19:10:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f66ff5700 10 client.16176 handle_command_reply: tid=0 2023-07-21T19:10:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 client.16176 shutdown 2023-07-21T19:10:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 msgr2=0x5616b8562ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f3f78009cc0 tx=0x7f3f78011c10 comp rx=0 tx=0).stop 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 shutdown_connections 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b855f8c0 0x5616b85631f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b83b56b0 0x5616b8565830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b8572cc0 0x5616b85750d0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/717216048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b8562ab0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f66ff5700 0 client.16176 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:18.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 >> 172.21.15.5:0/717216048 conn(0x5616b854ed40 msgr2=0x5616b8553640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 shutdown_connections 2023-07-21T19:10:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/717216048 wait complete. 2023-07-21T19:10:18.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.499+0000 7f3f9ce1bb80 20 client.16176 trim_cache size 0 max 16384 2023-07-21T19:10:18.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.519+0000 7f3f9ce1bb80 1 Processor -- start 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.519+0000 7f3f9ce1bb80 1 -- start start 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.519+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b8569e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.519+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f470 0x5616b8565360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.519+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b855a200 con 0x5616b855f8c0 2023-07-21T19:10:18.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b854bd80 con 0x5616b83b56b0 2023-07-21T19:10:18.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b8550e60 con 0x5616b855f470 2023-07-21T19:10:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b8569e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45596/0 (socket says 172.21.15.5:45596) 2023-07-21T19:10:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f837fe700 1 -- 172.21.15.5:0/3424505572 learned_addr learned my addr 172.21.15.5:0/3424505572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:18.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f827fc700 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f470 0x5616b8565360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 -- 172.21.15.5:0/3424505572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f470 msgr2=0x5616b8565360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f470 0x5616b8565360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 -- 172.21.15.5:0/3424505572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 msgr2=0x5616b8553dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f82ffd700 1 -- 172.21.15.5:0/3424505572 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616b82ff1f0 con 0x5616b83b56b0 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.520+0000 7f3f837fe700 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:18.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.521+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b8569e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f740cb580 tx=0x7f3f7400bfd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51bc4e5ffc8b2230 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.521+0000 7f3f69ffb700 1 -- 172.21.15.5:0/3424505572 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f740a3350 con 0x5616b83b56b0 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.521+0000 7f3f69ffb700 1 -- 172.21.15.5:0/3424505572 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f740990d0 con 0x5616b83b56b0 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.521+0000 7f3f69ffb700 1 -- 172.21.15.5:0/3424505572 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f7409b450 con 0x5616b83b56b0 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/3424505572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 msgr2=0x5616b8569e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b8569e90 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3f740cb580 tx=0x7f3f7400bfd0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/3424505572 shutdown_connections 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f8c0 0x5616b8553dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b8569e90 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/3424505572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f470 0x5616b8565360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/3424505572 >> 172.21.15.5:0/3424505572 conn(0x5616b854ed40 msgr2=0x5616b855cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/3424505572 shutdown_connections 2023-07-21T19:10:18.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/3424505572 wait complete. 2023-07-21T19:10:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 -- start start 2023-07-21T19:10:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.522+0000 7f3f9ce1bb80 1 Processor -- start 2023-07-21T19:10:18.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b855a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b855c5c0 con 0x5616b855f470 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b855c730 con 0x5616b83b56b0 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616b855aed0 con 0x5616b855f8c0 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f9ce1bb80 10 client.0 _ll_register_callbacks cb 0x7f3f74005e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:18.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f827fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45604/0 (socket says 172.21.15.5:45604) 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.523+0000 7f3f827fc700 1 -- 172.21.15.5:0/1146097724 learned_addr learned my addr 172.21.15.5:0/1146097724 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f837fe700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b855a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 -- 172.21.15.5:0/1146097724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 msgr2=0x5616b855a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b855a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 -- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 msgr2=0x5616b855ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:18.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 -- 172.21.15.5:0/1146097724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f7c00fe10 con 0x5616b855f470 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f827fc700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5616b85640f0 tx=0x7f3f780046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c8841a51902898 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.524+0000 7f3f66ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f78022450 con 0x5616b855f470 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f780044e0 con 0x5616b855f470 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3f78000e40 con 0x5616b855f470 2023-07-21T19:10:18.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f78004c30 con 0x5616b855f470 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 10 client.16182 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f3f841a4a20 con 0x5616b855f470 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f78042020 con 0x5616b855f470 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f78022690 con 0x5616b855f470 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 10 client.16182 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 10 client.16182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 20 client.16182 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:18.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.525+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b85c74b0 0x5616b85c98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.526+0000 7f3f9ce1bb80 4 client.16182 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:10:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.526+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5616b8465ff0 con 0x5616b85c74b0 2023-07-21T19:10:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.526+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b85c74b0 0x5616b85c98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.526+0000 7f3f66ff5700 10 client.16182 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:18.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.526+0000 7f3f82ffd700 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b85c74b0 0x5616b85c98c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=560e423aac04b4ed server_cookie=1d69b69e19024524 in_seq=0 out_seq=0 2023-07-21T19:10:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f66ff5700 1 -- 172.21.15.5:0/1146097724 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5616b8465ff0 con 0x5616b85c74b0 2023-07-21T19:10:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f66ff5700 10 client.16182 handle_command_reply: tid=0 2023-07-21T19:10:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 client.16182 shutdown 2023-07-21T19:10:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 msgr2=0x5616b855bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x5616b85640f0 tx=0x7f3f780046d0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 shutdown_connections 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5616b855f8c0 0x5616b855a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5616b83b56b0 0x5616b855ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5616b85c74b0 0x5616b85c98c0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 --2- 172.21.15.5:0/1146097724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5616b855f470 0x5616b855bfc0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 >> 172.21.15.5:0/1146097724 conn(0x5616b854ed40 msgr2=0x5616b8560520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f66ff5700 0 client.16182 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 shutdown_connections 2023-07-21T19:10:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 1 -- 172.21.15.5:0/1146097724 wait complete. 2023-07-21T19:10:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.539+0000 7f3f9ce1bb80 20 client.16182 trim_cache size 0 max 16384 2023-07-21T19:10:18.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:10:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:10:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:10:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:10:18.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 msgr2=0x7f3f6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 0x7f3f6c0a37f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f8419e730 tx=0x7f3f7c00b040 comp rx=0 tx=0).stop 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 msgr2=0x7f3f8419daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3f74005910 tx=0x7f3f740047b0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 shutdown_connections 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f6c0a1340 0x7f3f6c0a37f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f84101e50 0x7f3f84197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f840ffd20 0x7f3f8419daa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 --2- 172.21.15.5:0/1253539168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f841014a0 0x7f3f8419dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 >> 172.21.15.5:0/1253539168 conn(0x7f3f840fb440 msgr2=0x7f3f84072980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 shutdown_connections 2023-07-21T19:10:18.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:18.540+0000 7f3f83fff700 1 -- 172.21.15.5:0/1253539168 wait complete. 2023-07-21T19:10:19.247 INFO:teuthology.orchestra.run.smithi005.stderr:8200000 of 50000000 tuples (16%) done (elapsed 136.80 s, remaining 697.36 s) 2023-07-21T19:10:19.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:19 smithi101 ceph-mon[128136]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 34 MiB/s wr, 122 op/s 2023-07-21T19:10:19.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:19 smithi005 ceph-mon[126460]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 34 MiB/s wr, 122 op/s 2023-07-21T19:10:19.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:19 smithi156 ceph-mon[128316]: pgmap v84: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 34 MiB/s wr, 122 op/s 2023-07-21T19:10:21.015 INFO:teuthology.orchestra.run.smithi005.stderr:8300000 of 50000000 tuples (16%) done (elapsed 138.57 s, remaining 696.20 s) 2023-07-21T19:10:21.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:21 smithi101 ceph-mon[128136]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 23 MiB/s wr, 101 op/s 2023-07-21T19:10:21.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:21.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:21.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:21.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:21 smithi005 ceph-mon[126460]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 23 MiB/s wr, 101 op/s 2023-07-21T19:10:21.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:21.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:21.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:21.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:21 smithi156 ceph-mon[128316]: pgmap v85: 97 pgs: 97 active+clean; 1.9 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 23 MiB/s wr, 101 op/s 2023-07-21T19:10:21.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:21.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:21.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:22.345 INFO:teuthology.orchestra.run.smithi005.stderr:8400000 of 50000000 tuples (16%) done (elapsed 139.90 s, remaining 692.84 s) 2023-07-21T19:10:23.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:23 smithi101 ceph-mon[128136]: pgmap v86: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 33 MiB/s wr, 108 op/s 2023-07-21T19:10:23.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:23 smithi005 ceph-mon[126460]: pgmap v86: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 33 MiB/s wr, 108 op/s 2023-07-21T19:10:23.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:23 smithi156 ceph-mon[128316]: pgmap v86: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 33 MiB/s wr, 108 op/s 2023-07-21T19:10:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:8500000 of 50000000 tuples (17%) done (elapsed 141.81 s, remaining 692.37 s) 2023-07-21T19:10:25.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:25 smithi101 ceph-mon[128136]: pgmap v87: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 25 MiB/s wr, 39 op/s 2023-07-21T19:10:25.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:25 smithi005 ceph-mon[126460]: pgmap v87: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 25 MiB/s wr, 39 op/s 2023-07-21T19:10:25.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:25 smithi156 ceph-mon[128316]: pgmap v87: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 25 MiB/s wr, 39 op/s 2023-07-21T19:10:26.317 INFO:teuthology.orchestra.run.smithi005.stderr:8600000 of 50000000 tuples (17%) done (elapsed 143.87 s, remaining 692.60 s) 2023-07-21T19:10:27.632 INFO:teuthology.orchestra.run.smithi005.stderr:8700000 of 50000000 tuples (17%) done (elapsed 145.19 s, remaining 689.23 s) 2023-07-21T19:10:27.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:27 smithi101 ceph-mon[128136]: pgmap v88: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 23 op/s 2023-07-21T19:10:27.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:27 smithi005 ceph-mon[126460]: pgmap v88: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 23 op/s 2023-07-21T19:10:27.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:27 smithi156 ceph-mon[128316]: pgmap v88: 97 pgs: 97 active+clean; 2.0 GiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 23 op/s 2023-07-21T19:10:29.547 INFO:teuthology.orchestra.run.smithi005.stderr:8800000 of 50000000 tuples (17%) done (elapsed 147.10 s, remaining 688.71 s) 2023-07-21T19:10:29.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:29 smithi101 ceph-mon[128136]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-07-21T19:10:29.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:29 smithi005 ceph-mon[126460]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-07-21T19:10:29.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:29 smithi156 ceph-mon[128316]: pgmap v89: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 35 MiB/s wr, 34 op/s 2023-07-21T19:10:31.442 INFO:teuthology.orchestra.run.smithi005.stderr:8900000 of 50000000 tuples (17%) done (elapsed 149.00 s, remaining 688.07 s) 2023-07-21T19:10:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 -- 172.21.15.5:0/804518473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 msgr2=0x7fc808067940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 --2- 172.21.15.5:0/804518473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc808067940 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc009990 tx=0x7fc7fc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 -- 172.21.15.5:0/804518473 shutdown_connections 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 --2- 172.21.15.5:0/804518473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc808068070 0x7fc80810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 --2- 172.21.15.5:0/804518473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8081059a0 0x7fc808105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 --2- 172.21.15.5:0/804518473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc808067940 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 -- 172.21.15.5:0/804518473 >> 172.21.15.5:0/804518473 conn(0x7fc808074770 msgr2=0x7fc808074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 -- 172.21.15.5:0/804518473 shutdown_connections 2023-07-21T19:10:31.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.708+0000 7fc80c99d700 1 -- 172.21.15.5:0/804518473 wait complete. 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.709+0000 7fc80c99d700 1 Processor -- start 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.709+0000 7fc80c99d700 1 -- start start 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8081059a0 0x7fc8080fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc8081a3bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8080ff7d0 con 0x7fc8081062b0 2023-07-21T19:10:31.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8081a4130 con 0x7fc8081059a0 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc80c99d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8081a4270 con 0x7fc808068070 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc8077fe700 1 -- 172.21.15.5:0/3155489307 learned_addr learned my addr 172.21.15.5:0/3155489307 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc807fff700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc8081a3bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc806ffd700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8081059a0 0x7fc8080fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc8077fe700 1 -- 172.21.15.5:0/3155489307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8081059a0 msgr2=0x7fc8080fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.710+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8081059a0 0x7fc8080fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc8077fe700 1 -- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 msgr2=0x7fc8081a3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc8081a3bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc8077fe700 1 -- 172.21.15.5:0/3155489307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc0095f0 con 0x7fc808068070 2023-07-21T19:10:31.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc807fff700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc8081a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d950 tx=0x7fc7f800dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc804ff9700 1 -- 172.21.15.5:0/3155489307 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8009940 con 0x7fc808068070 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc804ff9700 1 -- 172.21.15.5:0/3155489307 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7f8010460 con 0x7fc808068070 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc804ff9700 1 -- 172.21.15.5:0/3155489307 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f800f620 con 0x7fc808068070 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8081a44d0 con 0x7fc808068070 2023-07-21T19:10:31.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.711+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8081a4960 con 0x7fc808068070 2023-07-21T19:10:31.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.716+0000 7fc804ff9700 1 -- 172.21.15.5:0/3155489307 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc7f80105d0 con 0x7fc808068070 2023-07-21T19:10:31.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 Processor -- start 2023-07-21T19:10:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 -- start start 2023-07-21T19:10:31.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc804ff9700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 0x7fc7f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc804ff9700 1 -- 172.21.15.5:0/3155489307 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7f80ccb30 con 0x7fc808068070 2023-07-21T19:10:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b3e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630b3e700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.717+0000 7fc82145fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630a43c50 con 0x561630b44470 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc82145fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630a969e0 con 0x561630b448c0 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc82145fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630a8fe70 con 0x56163099a6b0 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc807fff700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 0x7fc7f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58788/0 (socket says 172.21.15.5:58788) 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc8077fe700 1 -- 172.21.15.5:0/2487557680 learned_addr learned my addr 172.21.15.5:0/2487557680 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:31.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc806ffd700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b3e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc807fff700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630b3e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.718+0000 7fc8077fe700 1 -- 172.21.15.5:0/2487557680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 msgr2=0x561630b3e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630b3e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc8077fe700 1 -- 172.21.15.5:0/2487557680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 msgr2=0x561630b3e1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b3e1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc8077fe700 1 -- 172.21.15.5:0/2487557680 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616308bdd80 con 0x56163099a6b0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc807fff700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630b3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc807fff700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 0x7fc7f00a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc808100b70 tx=0x7fc800009d10 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8096640 tx=0x7fc7f80995b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=800befe01f3545a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc7edffb700 1 -- 172.21.15.5:0/2487557680 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f809dc00 con 0x56163099a6b0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc7edffb700 1 -- 172.21.15.5:0/2487557680 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7f80a2020 con 0x56163099a6b0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.719+0000 7fc7edffb700 1 -- 172.21.15.5:0/2487557680 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8099320 con 0x56163099a6b0 2023-07-21T19:10:31.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 -- 172.21.15.5:0/2487557680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 msgr2=0x561630b43f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8096640 tx=0x7fc7f80995b0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 -- 172.21.15.5:0/2487557680 shutdown_connections 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630b3e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b43f30 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2487557680 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b3e1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 -- 172.21.15.5:0/2487557680 >> 172.21.15.5:0/2487557680 conn(0x561630b33d40 msgr2=0x561630b39340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 -- 172.21.15.5:0/2487557680 shutdown_connections 2023-07-21T19:10:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.720+0000 7fc82145fb80 1 -- 172.21.15.5:0/2487557680 wait complete. 2023-07-21T19:10:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.721+0000 7fc82145fb80 1 -- start start 2023-07-21T19:10:31.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.721+0000 7fc82145fb80 1 Processor -- start 2023-07-21T19:10:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b47ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630a2fa60 con 0x561630b44470 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630a2f760 con 0x56163099a6b0 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616309138c0 con 0x561630b448c0 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc82145fb80 10 client.0 _ll_register_callbacks cb 0x561630a2d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53356/0 (socket says 172.21.15.5:53356) 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc807fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b47ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 -- 172.21.15.5:0/3739837254 learned_addr learned my addr 172.21.15.5:0/3739837254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 -- 172.21.15.5:0/3739837254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 msgr2=0x561630b47ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b47ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 -- 172.21.15.5:0/3739837254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 msgr2=0x561630b4dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.722+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc8077fe700 1 -- 172.21.15.5:0/3739837254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8081a4960 con 0x56163099a6b0 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc807fff700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b47ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc806ffd700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc8077fe700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809c040 tx=0x7fc7f80992d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d0f99c9b234dfca6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc7e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f809dc00 con 0x56163099a6b0 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616309e9490 con 0x56163099a6b0 2023-07-21T19:10:31.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7f80b2360 con 0x56163099a6b0 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f8099e30 con 0x56163099a6b0 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc82145fb80 10 client.25925 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.723+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5616308a6c00 con 0x56163099a6b0 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5616309e9490 con 0x56163099a6b0 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc7f8098b30 con 0x56163099a6b0 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 10 client.25925 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 10 client.25925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:31.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 20 client.25925 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630b57790 0x561630b59ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 4 client.25925 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.724+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561630a88e50 con 0x561630b57790 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.725+0000 7fc806ffd700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630b57790 0x561630b59ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.725+0000 7fc806ffd700 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630b57790 0x561630b59ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d8140534b490b35 server_cookie=37311e9816db5bd9 in_seq=0 out_seq=0 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.725+0000 7fc7e6ffd700 10 client.25925 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:31.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/3739837254 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561630a88e50 con 0x561630b57790 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc7e6ffd700 10 client.25925 handle_command_reply: tid=0 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 client.25925 shutdown 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 msgr2=0x561630b4d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc7f809c040 tx=0x7fc7f80992d0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 shutdown_connections 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b47ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b4d7d0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630b57790 0x561630b59ba0 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 --2- 172.21.15.5:0/3739837254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc7e6ffd700 0 client.25925 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.726+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 >> 172.21.15.5:0/3739837254 conn(0x561630b33d40 msgr2=0x561630b38640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.727+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 shutdown_connections 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.727+0000 7fc82145fb80 1 -- 172.21.15.5:0/3739837254 wait complete. 2023-07-21T19:10:31.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.727+0000 7fc82145fb80 20 client.25925 trim_cache size 0 max 16384 2023-07-21T19:10:31.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.743+0000 7fc82145fb80 1 Processor -- start 2023-07-21T19:10:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 -- start start 2023-07-21T19:10:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b47c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b38dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630b43bf0 con 0x561630b44470 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630b3a9f0 con 0x56163099a6b0 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc82145fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630b3ea90 con 0x561630b448c0 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc8077fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b47c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57800/0 (socket says 172.21.15.5:57800) 2023-07-21T19:10:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc806ffd700 1 -- 172.21.15.5:0/522347550 learned_addr learned my addr 172.21.15.5:0/522347550 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:31.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.744+0000 7fc807fff700 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b38dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 -- 172.21.15.5:0/522347550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 msgr2=0x561630b38dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b38dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 -- 172.21.15.5:0/522347550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 msgr2=0x561630b47c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b47c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 -- 172.21.15.5:0/522347550 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5616308e41f0 con 0x561630b44470 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc806ffd700 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0036c0 tx=0x7fc7fc009510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc1a278441f0fbf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc7ee7fc700 1 -- 172.21.15.5:0/522347550 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc00b7d0 con 0x561630b44470 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc7ee7fc700 1 -- 172.21.15.5:0/522347550 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7fc01e040 con 0x561630b44470 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.745+0000 7fc7ee7fc700 1 -- 172.21.15.5:0/522347550 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7fc004bb0 con 0x561630b44470 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 -- 172.21.15.5:0/522347550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 msgr2=0x561630b432c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fc7fc0036c0 tx=0x7fc7fc009510 comp rx=0 tx=0).stop 2023-07-21T19:10:31.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 -- 172.21.15.5:0/522347550 shutdown_connections 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561630b448c0 0x561630b38dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56163099a6b0 0x561630b47c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 --2- 172.21.15.5:0/522347550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b432c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 -- 172.21.15.5:0/522347550 >> 172.21.15.5:0/522347550 conn(0x561630b33d40 msgr2=0x561630b42bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.746+0000 7fc82145fb80 1 -- 172.21.15.5:0/522347550 shutdown_connections 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.747+0000 7fc82145fb80 1 -- 172.21.15.5:0/522347550 wait complete. 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.747+0000 7fc82145fb80 1 -- start start 2023-07-21T19:10:31.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.747+0000 7fc82145fb80 1 Processor -- start 2023-07-21T19:10:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b4b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630bace30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630bad370 con 0x561630b44470 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630bad4e0 con 0x561630b448c0 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561630bad6a0 con 0x56163099a6b0 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc82145fb80 10 client.0 _ll_register_callbacks cb 0x7fc7fc009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc806ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57804/0 (socket says 172.21.15.5:57804) 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc806ffd700 1 -- 172.21.15.5:0/2704719175 learned_addr learned my addr 172.21.15.5:0/2704719175 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b4b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc807fff700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630bace30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 -- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 msgr2=0x561630bace30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630bace30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 -- 172.21.15.5:0/2704719175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 msgr2=0x561630b4bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc8077fe700 1 -- 172.21.15.5:0/2704719175 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80810d020 con 0x56163099a6b0 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc806ffd700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.748+0000 7fc807fff700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630bace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc8077fe700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b4b9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8099b60 tx=0x7fc7f8010950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94b90d21af71424 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc7e6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f809a030 con 0x56163099a6b0 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7f80b2360 con 0x56163099a6b0 2023-07-21T19:10:31.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7f80a2020 con 0x56163099a6b0 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7f8095f10 con 0x56163099a6b0 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc82145fb80 10 client.25888 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5616308bc250 con 0x56163099a6b0 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.749+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7f80a2020 con 0x56163099a6b0 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc7f807e080 con 0x56163099a6b0 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 10 client.25888 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 10 client.25888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:31.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 20 client.25888 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630bb49a0 0x561630bb6db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 4 client.25888 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561630a4aff0 con 0x561630bb49a0 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.750+0000 7fc806ffd700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630bb49a0 0x561630bb6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.751+0000 7fc806ffd700 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630bb49a0 0x561630bb6db0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5877286c8c5fe03 server_cookie=3244c563b8e3f1c5 in_seq=0 out_seq=0 2023-07-21T19:10:31.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.751+0000 7fc7e6ffd700 10 client.25888 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:31.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc7e6ffd700 1 -- 172.21.15.5:0/2704719175 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561630a4aff0 con 0x561630bb49a0 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc7e6ffd700 10 client.25888 handle_command_reply: tid=0 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc82145fb80 1 client.25888 shutdown 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 msgr2=0x561630b4b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b4b9a0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8099b60 tx=0x7fc7f8010950 comp rx=0 tx=0).stop 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 shutdown_connections 2023-07-21T19:10:31.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.765+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561630b448c0 0x561630bace30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56163099a6b0 0x561630b4b9a0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561630bb49a0 0x561630bb6db0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 --2- 172.21.15.5:0/2704719175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561630b44470 0x561630b4bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc7e6ffd700 0 client.25888 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 >> 172.21.15.5:0/2704719175 conn(0x561630b33d40 msgr2=0x561630b3d190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 shutdown_connections 2023-07-21T19:10:31.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 1 -- 172.21.15.5:0/2704719175 wait complete. 2023-07-21T19:10:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc82145fb80 20 client.25888 trim_cache size 0 max 16384 2023-07-21T19:10:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:10:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:10:31.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:10:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:10:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 msgr2=0x7fc7f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.766+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 0x7fc7f00a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fc808100b70 tx=0x7fc800009d10 comp rx=0 tx=0).stop 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 msgr2=0x7fc8080fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fc7f800d950 tx=0x7fc7f800dc60 comp rx=0 tx=0).stop 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 shutdown_connections 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7f00a12c0 0x7fc7f00a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8081062b0 0x7fc8081a3bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc808068070 0x7fc8080fea60 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 --2- 172.21.15.5:0/3155489307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8081059a0 0x7fc8080fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 >> 172.21.15.5:0/3155489307 conn(0x7fc808074770 msgr2=0x7fc8080fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 shutdown_connections 2023-07-21T19:10:31.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:31.767+0000 7fc80c99d700 1 -- 172.21.15.5:0/3155489307 wait complete. 2023-07-21T19:10:31.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:31 smithi101 ceph-mon[128136]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:31.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:31 smithi005 ceph-mon[126460]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:31.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:31 smithi156 ceph-mon[128316]: pgmap v90: 97 pgs: 97 active+clean; 2.0 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:32.687 INFO:teuthology.orchestra.run.smithi005.stderr:9000000 of 50000000 tuples (18%) done (elapsed 150.24 s, remaining 684.44 s) 2023-07-21T19:10:33.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:33 smithi101 ceph-mon[128136]: pgmap v91: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 30 op/s 2023-07-21T19:10:33.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:33 smithi005 ceph-mon[126460]: pgmap v91: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 30 op/s 2023-07-21T19:10:33.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:33 smithi156 ceph-mon[128316]: pgmap v91: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 MiB/s wr, 30 op/s 2023-07-21T19:10:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:9100000 of 50000000 tuples (18%) done (elapsed 152.12 s, remaining 683.70 s) 2023-07-21T19:10:35.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:35 smithi101 ceph-mon[128136]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 29 op/s 2023-07-21T19:10:35.893 INFO:teuthology.orchestra.run.smithi005.stderr:9200000 of 50000000 tuples (18%) done (elapsed 153.45 s, remaining 680.51 s) 2023-07-21T19:10:35.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:35 smithi005 ceph-mon[126460]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 29 op/s 2023-07-21T19:10:35.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:35 smithi156 ceph-mon[128316]: pgmap v92: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 25 MiB/s wr, 29 op/s 2023-07-21T19:10:37.555 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:10:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 -- 172.21.15.5:0/2532946091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d40101610 msgr2=0x7f1d40109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 --2- 172.21.15.5:0/2532946091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d40101610 0x7f1d40109b00 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f1d30009990 tx=0x7f1d30009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 -- 172.21.15.5:0/2532946091 shutdown_connections 2023-07-21T19:10:37.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 --2- 172.21.15.5:0/2532946091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d4010a040 0x7f1d4010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 --2- 172.21.15.5:0/2532946091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 0x7f1d401010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 --2- 172.21.15.5:0/2532946091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d40101610 0x7f1d40109b00 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.779+0000 7f1d479ff700 1 -- 172.21.15.5:0/2532946091 >> 172.21.15.5:0/2532946091 conn(0x7f1d400fa4b0 msgr2=0x7f1d400fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.780+0000 7f1d479ff700 1 -- 172.21.15.5:0/2532946091 shutdown_connections 2023-07-21T19:10:37.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.780+0000 7f1d479ff700 1 -- 172.21.15.5:0/2532946091 wait complete. 2023-07-21T19:10:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.781+0000 7f1d479ff700 1 Processor -- start 2023-07-21T19:10:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.781+0000 7f1d479ff700 1 -- start start 2023-07-21T19:10:37.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 0x7f1d400fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4010a040 0x7f1d401a3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d400ff730 con 0x7f1d40101610 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401a4310 con 0x7f1d40100d00 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d479ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401a4450 con 0x7f1d4010a040 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d44f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d44f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37592/0 (socket says 172.21.15.5:37592) 2023-07-21T19:10:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3466533492 learned_addr learned my addr 172.21.15.5:0/3466533492 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4010a040 0x7f1d401a3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.782+0000 7f1d4579b700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 0x7f1d400fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3466533492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4010a040 msgr2=0x7f1d401a3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4010a040 0x7f1d401a3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3466533492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 msgr2=0x7f1d400fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 0x7f1d400fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3466533492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d300095f0 con 0x7f1d40101610 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1d30004730 tx=0x7f1d30004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.795 INFO:teuthology.orchestra.run.smithi005.stderr:9300000 of 50000000 tuples (18%) done (elapsed 155.35 s, remaining 679.86 s) 2023-07-21T19:10:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.783+0000 7f1d367fc700 1 -- 172.21.15.5:0/3466533492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3001d070 con 0x7f1d40101610 2023-07-21T19:10:37.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.784+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401a46a0 con 0x7f1d40101610 2023-07-21T19:10:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.784+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401a4ba0 con 0x7f1d40101610 2023-07-21T19:10:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.784+0000 7f1d367fc700 1 -- 172.21.15.5:0/3466533492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d30022470 con 0x7f1d40101610 2023-07-21T19:10:37.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.784+0000 7f1d367fc700 1 -- 172.21.15.5:0/3466533492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30017650 con 0x7f1d40101610 2023-07-21T19:10:37.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.789+0000 7f1d367fc700 1 -- 172.21.15.5:0/3466533492 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1d300225e0 con 0x7f1d40101610 2023-07-21T19:10:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.790+0000 7f1d5b40ab80 1 Processor -- start 2023-07-21T19:10:37.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.790+0000 7f1d5b40ab80 1 -- start start 2023-07-21T19:10:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7962d0 0x55daca7966a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 0x55daca7a1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 0x55daca7a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca69ccb0 con 0x55daca7962d0 2023-07-21T19:10:37.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca6efa40 con 0x55daca7a1ad0 2023-07-21T19:10:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.791+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca6e8ed0 con 0x55daca796be0 2023-07-21T19:10:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d4579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 0x55daca7a1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d4579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 0x55daca7a1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37380/0 (socket says 172.21.15.5:37380) 2023-07-21T19:10:37.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d4579b700 1 -- 172.21.15.5:0/2847675236 learned_addr learned my addr 172.21.15.5:0/2847675236 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:37.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 0x55daca7a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7962d0 0x55daca7966a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.792+0000 7f1d367fc700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 0x7f1d2c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d44f9a700 1 -- 172.21.15.5:0/2847675236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 msgr2=0x55daca7a1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 0x55daca7a1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d44f9a700 1 -- 172.21.15.5:0/2847675236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7962d0 msgr2=0x55daca7966a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7962d0 0x55daca7966a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d44f9a700 1 -- 172.21.15.5:0/2847675236 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daca516de0 con 0x55daca7a1ad0 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 0x7f1d2c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.793+0000 7f1d367fc700 1 -- 172.21.15.5:0/3466533492 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1d300ce0e0 con 0x7f1d40101610 2023-07-21T19:10:37.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.794+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 0x55daca7a3eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d300cb900 tx=0x7f1d3002e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3a19a70dec1e13c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.794+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 0x7f1d2c0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009ec0 tx=0x7f1d38009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.794+0000 7f1d2b7fe700 1 -- 172.21.15.5:0/2847675236 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3009b0e0 con 0x55daca7a1ad0 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.794+0000 7f1d2b7fe700 1 -- 172.21.15.5:0/2847675236 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d3009ae50 con 0x55daca7a1ad0 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.794+0000 7f1d2b7fe700 1 -- 172.21.15.5:0/2847675236 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30015e20 con 0x55daca7a1ad0 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2847675236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 msgr2=0x55daca7a3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 0x55daca7a3eb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1d300cb900 tx=0x7f1d3002e000 comp rx=0 tx=0).stop 2023-07-21T19:10:37.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2847675236 shutdown_connections 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7a1ad0 0x55daca7a3eb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7962d0 0x55daca7966a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2847675236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca796be0 0x55daca7a1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2847675236 >> 172.21.15.5:0/2847675236 conn(0x55daca78cda0 msgr2=0x55daca7926a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2847675236 shutdown_connections 2023-07-21T19:10:37.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.795+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2847675236 wait complete. 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.796+0000 7f1d5b40ab80 1 -- start start 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.796+0000 7f1d5b40ab80 1 Processor -- start 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca7a6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca7a6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca688ac0 con 0x55daca7a1ad0 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca6887c0 con 0x55daca796be0 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca56c920 con 0x55daca7962d0 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d5b40ab80 10 client.0 _ll_register_callbacks cb 0x55daca686760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d45f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d45f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37604/0 (socket says 172.21.15.5:37604) 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.797+0000 7f1d45f9c700 1 -- 172.21.15.5:0/4065055003 learned_addr learned my addr 172.21.15.5:0/4065055003 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:37.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d4579b700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca7a6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca7a6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d45f9c700 1 -- 172.21.15.5:0/4065055003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 msgr2=0x55daca7a6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca7a6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d45f9c700 1 -- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 msgr2=0x55daca7a6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca7a6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d45f9c700 1 -- 172.21.15.5:0/4065055003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401a4ba0 con 0x55daca7a1ad0 2023-07-21T19:10:37.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d4579b700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca7a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:37.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.798+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca7a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:37.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d38018d80 tx=0x7f1d38004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4915e61358ba94 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:37.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3801c040 con 0x55daca7a1ad0 2023-07-21T19:10:37.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1d38022470 con 0x55daca7a1ad0 2023-07-21T19:10:37.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55daca6424f0 con 0x55daca7a1ad0 2023-07-21T19:10:37.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d3800e040 con 0x55daca7a1ad0 2023-07-21T19:10:37.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d5b40ab80 10 client.16212 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.799+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55daca4ffc60 con 0x55daca7a1ad0 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55daca6424f0 con 0x55daca7a1ad0 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d380214e0 con 0x55daca7a1ad0 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 10 client.16212 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 10 client.16212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 20 client.16212 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca7b1080 0x55daca7b3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 4 client.16212 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.800+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55daca6e1eb0 con 0x55daca7b1080 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.801+0000 7f1d4579b700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca7b1080 0x55daca7b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.801+0000 7f1d4579b700 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca7b1080 0x55daca7b3490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=159861b3380b7031 server_cookie=3219e4db3b9930a4 in_seq=0 out_seq=0 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.801+0000 7f1d28ff9700 10 client.16212 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d28ff9700 1 -- 172.21.15.5:0/4065055003 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55daca6e1eb0 con 0x55daca7b1080 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d28ff9700 10 client.16212 handle_command_reply: tid=0 2023-07-21T19:10:37.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 client.16212 shutdown 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 msgr2=0x55daca7a09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f1d38018d80 tx=0x7f1d38004ad0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 shutdown_connections 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca7b1080 0x55daca7b3490 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca7a09e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca7a6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/4065055003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca7a6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 >> 172.21.15.5:0/4065055003 conn(0x55daca78cda0 msgr2=0x55daca792290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.802+0000 7f1d28ff9700 0 client.16212 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:37.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.803+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 shutdown_connections 2023-07-21T19:10:37.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.803+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/4065055003 wait complete. 2023-07-21T19:10:37.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.803+0000 7f1d5b40ab80 20 client.16212 trim_cache size 0 max 16384 2023-07-21T19:10:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.824+0000 7f1d5b40ab80 1 Processor -- start 2023-07-21T19:10:37.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.824+0000 7f1d5b40ab80 1 -- start start 2023-07-21T19:10:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca79b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca79bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca78c970 con 0x55daca7a1ad0 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca7ad8d0 con 0x55daca796be0 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca79b7a0 con 0x55daca7962d0 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37618/0 (socket says 172.21.15.5:37618) 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 -- 172.21.15.5:0/2904582160 learned_addr learned my addr 172.21.15.5:0/2904582160 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:37.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca79bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d4579b700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca79b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 -- 172.21.15.5:0/2904582160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 msgr2=0x55daca79b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca79b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 -- 172.21.15.5:0/2904582160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 msgr2=0x55daca79bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca79bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d45f9c700 1 -- 172.21.15.5:0/2904582160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55daca53d250 con 0x55daca7a1ad0 2023-07-21T19:10:37.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.825+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca79bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.826+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d60 tx=0x7f1d38016a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b8473b71b816bf5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.826+0000 7f1d2bfff700 1 -- 172.21.15.5:0/2904582160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3800e040 con 0x55daca7a1ad0 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.826+0000 7f1d2bfff700 1 -- 172.21.15.5:0/2904582160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d3800f040 con 0x55daca7a1ad0 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.826+0000 7f1d2bfff700 1 -- 172.21.15.5:0/2904582160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d38021630 con 0x55daca7a1ad0 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2904582160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 msgr2=0x55daca79c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f1d38007d60 tx=0x7f1d38016a40 comp rx=0 tx=0).stop 2023-07-21T19:10:37.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2904582160 shutdown_connections 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca7a1ad0 0x55daca79c240 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7962d0 0x55daca79b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/2904582160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca796be0 0x55daca79bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2904582160 >> 172.21.15.5:0/2904582160 conn(0x55daca78cda0 msgr2=0x55daca7918b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2904582160 shutdown_connections 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.827+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/2904582160 wait complete. 2023-07-21T19:10:37.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.828+0000 7f1d5b40ab80 1 -- start start 2023-07-21T19:10:37.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:37 smithi101 ceph-mon[128136]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.828+0000 7f1d5b40ab80 1 Processor -- start 2023-07-21T19:10:37.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 0x55daca7fe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7a1ad0 0x55daca7fcf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca7fd4c0 con 0x55daca796be0 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca7fd630 con 0x55daca7962d0 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55daca7fd7f0 con 0x55daca7a1ad0 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d5b40ab80 10 client.0 _ll_register_callbacks cb 0x7f1d38018930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:37.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d44f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d44f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37626/0 (socket says 172.21.15.5:37626) 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3873852151 learned_addr learned my addr 172.21.15.5:0/3873852151 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d4579b700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 0x55daca7fe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.829+0000 7f1d45f9c700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7a1ad0 0x55daca7fcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3873852151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7a1ad0 msgr2=0x55daca7fcf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7a1ad0 0x55daca7fcf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 msgr2=0x55daca7fe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 0x55daca7fe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 -- 172.21.15.5:0/3873852151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55daca516de0 con 0x55daca796be0 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d4579b700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 0x55daca7fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d44f9a700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0ad20c42a408b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30015e20 con 0x55daca796be0 2023-07-21T19:10:37.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1d3009ae50 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1d300228c0 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1d3009b0e0 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d5b40ab80 10 client.16218 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.830+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55daca4ffc60 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1d3009ae50 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1d30097100 con 0x55daca796be0 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 10 client.16218 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:37.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 20 client.16218 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca805eb0 0x55daca8082c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 4 client.16218 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.831+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55daca6a4050 con 0x55daca805eb0 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d4579b700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca805eb0 0x55daca8082c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d4579b700 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca805eb0 0x55daca8082c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de89fb2a6f6cce8b server_cookie=cd31f53460ee21a5 in_seq=0 out_seq=0 2023-07-21T19:10:37.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d28ff9700 10 client.16218 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d28ff9700 1 -- 172.21.15.5:0/3873852151 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55daca6a4050 con 0x55daca805eb0 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d28ff9700 10 client.16218 handle_command_reply: tid=0 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.832+0000 7f1d5b40ab80 1 client.16218 shutdown 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 msgr2=0x55daca7fe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f1d30000c00 tx=0x7f1d30009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 shutdown_connections 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55daca7a1ad0 0x55daca7fcf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55daca7962d0 0x55daca7fe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55daca805eb0 0x55daca8082c0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 --2- 172.21.15.5:0/3873852151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55daca796be0 0x55daca7fe730 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 >> 172.21.15.5:0/3873852151 conn(0x55daca78cda0 msgr2=0x55daca79bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d28ff9700 0 client.16218 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 shutdown_connections 2023-07-21T19:10:37.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 1 -- 172.21.15.5:0/3873852151 wait complete. 2023-07-21T19:10:37.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.833+0000 7f1d5b40ab80 20 client.16218 trim_cache size 0 max 16384 2023-07-21T19:10:37.851 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:10:37.851 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:10:37.851 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:37.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 msgr2=0x7f1d2c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 0x7f1d2c0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1d38009ec0 tx=0x7f1d38009040 comp rx=0 tx=0).stop 2023-07-21T19:10:37.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 msgr2=0x7f1d400fee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:37.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f1d30004730 tx=0x7f1d30004760 comp rx=0 tx=0).stop 2023-07-21T19:10:37.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 shutdown_connections 2023-07-21T19:10:37.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1d4010a040 0x7f1d401a3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1d40100d00 0x7f1d400fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1d2c0a1340 0x7f1d2c0a37f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 --2- 172.21.15.5:0/3466533492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1d40101610 0x7f1d400fee90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:37.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 >> 172.21.15.5:0/3466533492 conn(0x7f1d400fa4b0 msgr2=0x7f1d400fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:37.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 shutdown_connections 2023-07-21T19:10:37.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:37.834+0000 7f1d479ff700 1 -- 172.21.15.5:0/3466533492 wait complete. 2023-07-21T19:10:37.860 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:37 smithi005 ceph-mon[126460]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:37.863 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:10:37.863 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:10:37.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:37 smithi156 ceph-mon[128316]: pgmap v93: 97 pgs: 97 active+clean; 2.1 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 23 op/s 2023-07-21T19:10:38.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 -- 172.21.15.5:0/182177402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 msgr2=0x7f05cc10c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 --2- 172.21.15.5:0/182177402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc10c4b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f05c400b0a0 tx=0x7f05c400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 -- 172.21.15.5:0/182177402 shutdown_connections 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 --2- 172.21.15.5:0/182177402 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc10c4b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 --2- 172.21.15.5:0/182177402 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc100ca0 0x7f05cc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 --2- 172.21.15.5:0/182177402 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc1015b0 0x7f05cc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 -- 172.21.15.5:0/182177402 >> 172.21.15.5:0/182177402 conn(0x7f05cc0fa490 msgr2=0x7f05cc0fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 -- 172.21.15.5:0/182177402 shutdown_connections 2023-07-21T19:10:38.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.084+0000 7f05d1fff700 1 -- 172.21.15.5:0/182177402 wait complete. 2023-07-21T19:10:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.085+0000 7f05d1fff700 1 Processor -- start 2023-07-21T19:10:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 -- start start 2023-07-21T19:10:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc100ca0 0x7f05cc19b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc1015b0 0x7f05cc19be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc196030 con 0x7f05cc10a070 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc1961a0 con 0x7f05cc100ca0 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05d1fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05cc196360 con 0x7f05cc1015b0 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37638/0 (socket says 172.21.15.5:37638) 2023-07-21T19:10:38.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05cbfff700 1 -- 172.21.15.5:0/1673445468 learned_addr learned my addr 172.21.15.5:0/1673445468 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.086+0000 7f05caffd700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc1015b0 0x7f05cc19be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cb7fe700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc100ca0 0x7f05cc19b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 -- 172.21.15.5:0/1673445468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc1015b0 msgr2=0x7f05cc19be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc1015b0 0x7f05cc19be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 -- 172.21.15.5:0/1673445468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc100ca0 msgr2=0x7f05cc19b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc100ca0 0x7f05cc19b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 -- 172.21.15.5:0/1673445468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c4009d00 con 0x7f05cc10a070 2023-07-21T19:10:38.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05cbfff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f05c400b070 tx=0x7f05c400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c400e040 con 0x7f05cc10a070 2023-07-21T19:10:38.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05cc1965e0 con 0x7f05cc10a070 2023-07-21T19:10:38.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05c4009480 con 0x7f05cc10a070 2023-07-21T19:10:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.087+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05c4005040 con 0x7f05cc10a070 2023-07-21T19:10:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.088+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05cc0fea00 con 0x7f05cc10a070 2023-07-21T19:10:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.089+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05cc067300 con 0x7f05cc10a070 2023-07-21T19:10:38.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.091+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f05c401d480 con 0x7f05cc10a070 2023-07-21T19:10:38.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.092+0000 7f05c8ff9700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 0x7f05b40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.092+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f05c40ce2b0 con 0x7f05cc10a070 2023-07-21T19:10:38.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.093+0000 7f05cb7fe700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 0x7f05b40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.093+0000 7f05cb7fe700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 0x7f05b40a37b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009bf0 tx=0x7f05bc009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.095+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f05c4097780 con 0x7f05cc10a070 2023-07-21T19:10:38.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.403+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05cc196ed0 con 0x7f05cc10a070 2023-07-21T19:10:38.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.404+0000 7f05c8ff9700 1 -- 172.21.15.5:0/1673445468 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f05c409ada0 con 0x7f05cc10a070 2023-07-21T19:10:38.414 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:10:38.414 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:09:18.477788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:10:38.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 msgr2=0x7f05b40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 0x7f05b40a37b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f05bc009bf0 tx=0x7f05bc009260 comp rx=0 tx=0).stop 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 msgr2=0x7f05cc1959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f05c400b070 tx=0x7f05c400bce0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 shutdown_connections 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05b40a1300 0x7f05b40a37b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f05cc10a070 0x7f05cc1959c0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05cc100ca0 0x7f05cc19b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 --2- 172.21.15.5:0/1673445468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05cc1015b0 0x7f05cc19be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 >> 172.21.15.5:0/1673445468 conn(0x7f05cc0fa490 msgr2=0x7f05cc10c1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.406+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 shutdown_connections 2023-07-21T19:10:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.407+0000 7f05d1fff700 1 -- 172.21.15.5:0/1673445468 wait complete. 2023-07-21T19:10:38.418 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2023-07-21T19:10:38.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:10:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 -- 172.21.15.5:0/572129874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f4100480 msgr2=0x7fc7f410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/572129874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f4100480 0x7fc7f410c910 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec00b0a0 tx=0x7fc7ec00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 -- 172.21.15.5:0/572129874 shutdown_connections 2023-07-21T19:10:38.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/572129874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f4100480 0x7fc7f410c910 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/572129874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4105ae0 0x7fc7f4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/572129874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f40ffad0 0x7fc7f40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 -- 172.21.15.5:0/572129874 >> 172.21.15.5:0/572129874 conn(0x7fc7f40fb5f0 msgr2=0x7fc7f40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 -- 172.21.15.5:0/572129874 shutdown_connections 2023-07-21T19:10:38.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.681+0000 7fc7f8fff700 1 -- 172.21.15.5:0/572129874 wait complete. 2023-07-21T19:10:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.683+0000 7fc7f8fff700 1 Processor -- start 2023-07-21T19:10:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.683+0000 7fc7f8fff700 1 -- start start 2023-07-21T19:10:38.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.683+0000 7fc7f8fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.683+0000 7fc7f8fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4100480 0x7fc7f41982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.683+0000 7fc7f8fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 0x7fc7f419c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f8fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f419cd90 con 0x7fc7f40ffad0 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f8fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f419cf00 con 0x7fc7f4105ae0 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f8fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7f419d0c0 con 0x7fc7f4100480 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37668/0 (socket says 172.21.15.5:37668) 2023-07-21T19:10:38.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f259c700 1 -- 172.21.15.5:0/3754051122 learned_addr learned my addr 172.21.15.5:0/3754051122 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 0x7fc7f419c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.684+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4100480 0x7fc7f41982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 -- 172.21.15.5:0/3754051122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4100480 msgr2=0x7fc7f41982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4100480 0x7fc7f41982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 -- 172.21.15.5:0/3754051122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 msgr2=0x7fc7f419c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 0x7fc7f419c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 -- 172.21.15.5:0/3754051122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009d00 con 0x7fc7f40ffad0 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 0x7fc7f419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:38.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f259c700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e9a0 tx=0x7fc7e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7e37fe700 1 -- 172.21.15.5:0/3754051122 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4009790 con 0x7fc7f40ffad0 2023-07-21T19:10:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7f419d3a0 con 0x7fc7f40ffad0 2023-07-21T19:10:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7e37fe700 1 -- 172.21.15.5:0/3754051122 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e4010430 con 0x7fc7f40ffad0 2023-07-21T19:10:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.685+0000 7fc7e37fe700 1 -- 172.21.15.5:0/3754051122 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e400cea0 con 0x7fc7f40ffad0 2023-07-21T19:10:38.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.686+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f41a4be0 con 0x7fc7f40ffad0 2023-07-21T19:10:38.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.690+0000 7fc7e37fe700 1 -- 172.21.15.5:0/3754051122 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc7e40105a0 con 0x7fc7f40ffad0 2023-07-21T19:10:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.691+0000 7fc80c953b80 1 Processor -- start 2023-07-21T19:10:38.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.691+0000 7fc80c953b80 1 -- start start 2023-07-21T19:10:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46f2d0 0x55761c46f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 0x55761c47a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 0x55761c47ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c375cb0 con 0x55761c46fbe0 2023-07-21T19:10:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c3c8a40 con 0x55761c47aad0 2023-07-21T19:10:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc80c953b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c3c1ed0 con 0x55761c46f2d0 2023-07-21T19:10:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 0x55761c47a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 0x55761c47a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37670/0 (socket says 172.21.15.5:37670) 2023-07-21T19:10:38.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7f259c700 1 -- 172.21.15.5:0/3508517213 learned_addr learned my addr 172.21.15.5:0/3508517213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 0x55761c47ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7e37fe700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 0x7fc7dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.692+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46f2d0 0x55761c46f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7e37fe700 1 -- 172.21.15.5:0/3754051122 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7e4014070 con 0x7fc7f40ffad0 2023-07-21T19:10:38.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/3508517213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46f2d0 msgr2=0x55761c46f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46f2d0 0x55761c46f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/3508517213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 msgr2=0x55761c47a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 0x55761c47a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/3508517213 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55761c1efde0 con 0x55761c47aad0 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 0x7fc7dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 0x55761c47ceb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009c80 tx=0x7fc7e8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f790f8debf687c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.693+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 0x7fc7dc0a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0062a0 tx=0x7fc7ec0099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.694+0000 7fc7e0ff9700 1 -- 172.21.15.5:0/3508517213 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8011cb0 con 0x55761c47aad0 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.694+0000 7fc7e0ff9700 1 -- 172.21.15.5:0/3508517213 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e8011e10 con 0x55761c47aad0 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.694+0000 7fc7e0ff9700 1 -- 172.21.15.5:0/3508517213 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8005280 con 0x55761c47aad0 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.694+0000 7fc80c953b80 1 -- 172.21.15.5:0/3508517213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 msgr2=0x55761c47ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.694+0000 7fc80c953b80 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 0x55761c47ceb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7e8009c80 tx=0x7fc7e8009f90 comp rx=0 tx=0).stop 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 -- 172.21.15.5:0/3508517213 shutdown_connections 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c47aad0 0x55761c47ceb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46f2d0 0x55761c46f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 --2- 172.21.15.5:0/3508517213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46fbe0 0x55761c47a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 -- 172.21.15.5:0/3508517213 >> 172.21.15.5:0/3508517213 conn(0x55761c465da0 msgr2=0x55761c46b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 -- 172.21.15.5:0/3508517213 shutdown_connections 2023-07-21T19:10:38.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 -- 172.21.15.5:0/3508517213 wait complete. 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.695+0000 7fc80c953b80 1 -- start start 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 Processor -- start 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c477810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c477f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c361ac0 con 0x55761c46f2d0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c3617c0 con 0x55761c46fbe0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c245920 con 0x55761c47aad0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.696+0000 7fc80c953b80 10 client.0 _ll_register_callbacks cb 0x55761c35f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37676/0 (socket says 172.21.15.5:37676) 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 -- 172.21.15.5:0/1812533061 learned_addr learned my addr 172.21.15.5:0/1812533061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c477810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c477f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 -- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 msgr2=0x55761c477f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c477f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 -- 172.21.15.5:0/1812533061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 msgr2=0x55761c477810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c477810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 -- 172.21.15.5:0/1812533061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7e80098e0 con 0x55761c46f2d0 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c477f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:38.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c2d0 tx=0x7fc7e4099aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bac25f9173411d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.697+0000 7fc7d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4010880 con 0x55761c46f2d0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7e4099ee0 con 0x55761c46f2d0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55761c31b4f0 con 0x55761c46f2d0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e409df10 con 0x55761c46f2d0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc80c953b80 10 client.16236 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55761c1d8c60 con 0x55761c46f2d0 2023-07-21T19:10:38.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.698+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e40c8070 con 0x55761c46f2d0 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc7e40d2030 con 0x55761c46f2d0 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 10 client.16236 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 10 client.16236 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 20 client.16236 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c48a420 0x55761c48c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 4 client.16236 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55761c3baeb0 con 0x55761c48a420 2023-07-21T19:10:38.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.699+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c48a420 0x55761c48c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.700+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c48a420 0x55761c48c830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=54a93af8921992c5 server_cookie=77c144cd8414cefc in_seq=0 out_seq=0 2023-07-21T19:10:38.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.700+0000 7fc7d9ffb700 10 client.16236 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/1812533061 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55761c3baeb0 con 0x55761c48a420 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc7d9ffb700 10 client.16236 handle_command_reply: tid=0 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 client.16236 shutdown 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 msgr2=0x55761c47a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c2d0 tx=0x7fc7e4099aa0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 shutdown_connections 2023-07-21T19:10:38.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c477f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c47a520 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c48a420 0x55761c48c830 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1812533061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c477810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 >> 172.21.15.5:0/1812533061 conn(0x55761c465da0 msgr2=0x55761c46b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc7d9ffb700 0 client.16236 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 shutdown_connections 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.701+0000 7fc80c953b80 1 -- 172.21.15.5:0/1812533061 wait complete. 2023-07-21T19:10:38.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.702+0000 7fc80c953b80 20 client.16236 trim_cache size 0 max 16384 2023-07-21T19:10:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 Processor -- start 2023-07-21T19:10:38.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 -- start start 2023-07-21T19:10:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 0x55761c47a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46fbe0 0x55761c476fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c462fa0 con 0x55761c47aad0 2023-07-21T19:10:38.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c465970 con 0x55761c46f2d0 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.717+0000 7fc80c953b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c486e00 con 0x55761c46fbe0 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f2d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37684/0 (socket says 172.21.15.5:37684) 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f2d9d700 1 -- 172.21.15.5:0/1649328017 learned_addr learned my addr 172.21.15.5:0/1649328017 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 0x55761c47a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46fbe0 0x55761c476fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 -- 172.21.15.5:0/1649328017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46fbe0 msgr2=0x55761c476fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46fbe0 0x55761c476fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 -- 172.21.15.5:0/1649328017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 msgr2=0x55761c477510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 -- 172.21.15.5:0/1649328017 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55761c216250 con 0x55761c46f2d0 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:38.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.718+0000 7fc7f259c700 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 0x55761c47a0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40963e0 tx=0x7fc7e40ca4a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2522cdae8b506955 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.719+0000 7fc7e17fa700 1 -- 172.21.15.5:0/1649328017 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4099840 con 0x55761c46f2d0 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.719+0000 7fc7e17fa700 1 -- 172.21.15.5:0/1649328017 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e40c8020 con 0x55761c46f2d0 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.719+0000 7fc7e17fa700 1 -- 172.21.15.5:0/1649328017 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e409a5e0 con 0x55761c46f2d0 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.719+0000 7fc80c953b80 1 -- 172.21.15.5:0/1649328017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 msgr2=0x55761c47a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.719+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 0x55761c47a0c0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7e40963e0 tx=0x7fc7e40ca4a0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 -- 172.21.15.5:0/1649328017 shutdown_connections 2023-07-21T19:10:38.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c47aad0 0x55761c477510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46f2d0 0x55761c47a0c0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 --2- 172.21.15.5:0/1649328017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c46fbe0 0x55761c476fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 -- 172.21.15.5:0/1649328017 >> 172.21.15.5:0/1649328017 conn(0x55761c465da0 msgr2=0x55761c472b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 -- 172.21.15.5:0/1649328017 shutdown_connections 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 -- 172.21.15.5:0/1649328017 wait complete. 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 -- start start 2023-07-21T19:10:38.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.720+0000 7fc80c953b80 1 Processor -- start 2023-07-21T19:10:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c47f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c47fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c4800c0 con 0x55761c46f2d0 2023-07-21T19:10:38.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c480230 con 0x55761c46fbe0 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55761c4803f0 con 0x55761c47aad0 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc80c953b80 10 client.0 _ll_register_callbacks cb 0x55761c35f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc7f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37700/0 (socket says 172.21.15.5:37700) 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc7f259c700 1 -- 172.21.15.5:0/2436378229 learned_addr learned my addr 172.21.15.5:0/2436378229 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c47f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.721+0000 7fc7f2d9d700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c47fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 msgr2=0x55761c47fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c47fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/2436378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 msgr2=0x55761c481310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 -- 172.21.15.5:0/2436378229 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7f410d210 con 0x55761c46fbe0 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f259c700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7f1d9b700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c47f640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55761c478d50 tx=0x7fc7e80096f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3fe7b4ea0ab8599 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:38.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.722+0000 7fc7d9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e8029040 con 0x55761c46fbe0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7e8011e10 con 0x55761c46fbe0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc7e8005ce0 con 0x55761c46fbe0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7e8009260 con 0x55761c46fbe0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc80c953b80 10 client.25973 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55761c1d8c60 con 0x55761c46fbe0 2023-07-21T19:10:38.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.723+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7e8011e10 con 0x55761c46fbe0 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc7e802c070 con 0x55761c46fbe0 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 10 client.25973 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 10 client.25973 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 20 client.25973 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c4df760 0x55761c4e1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 4 client.25973 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55761c37d050 con 0x55761c4df760 2023-07-21T19:10:38.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.724+0000 7fc7f259c700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c4df760 0x55761c4e1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc7d9ffb700 10 client.25973 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc7f259c700 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c4df760 0x55761c4e1b70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7296be31c2e84e3d server_cookie=a83bdfeec4e88032 in_seq=0 out_seq=0 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/2436378229 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55761c37d050 con 0x55761c4df760 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc7d9ffb700 10 client.25973 handle_command_reply: tid=0 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 client.25973 shutdown 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 msgr2=0x55761c47f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c47f640 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x55761c478d50 tx=0x7fc7e80096f0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 shutdown_connections 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55761c47aad0 0x55761c47fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55761c46f2d0 0x55761c481310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55761c4df760 0x55761c4e1b70 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 --2- 172.21.15.5:0/2436378229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55761c46fbe0 0x55761c47f640 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 >> 172.21.15.5:0/2436378229 conn(0x55761c465da0 msgr2=0x55761c4712b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc7d9ffb700 0 client.25973 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:10:38.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 shutdown_connections 2023-07-21T19:10:38.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.725+0000 7fc80c953b80 1 -- 172.21.15.5:0/2436378229 wait complete. 2023-07-21T19:10:38.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc80c953b80 20 client.25973 trim_cache size 0 max 16384 2023-07-21T19:10:38.743 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 msgr2=0x7fc7dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 0x7fc7dc0a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0062a0 tx=0x7fc7ec0099e0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 msgr2=0x7fc7f4197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400e9a0 tx=0x7fc7e400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 shutdown_connections 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7f4105ae0 0x7fc7f419c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7dc0a1340 0x7fc7dc0a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7f40ffad0 0x7fc7f4197db0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 --2- 172.21.15.5:0/3754051122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7f4100480 0x7fc7f41982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 >> 172.21.15.5:0/3754051122 conn(0x7fc7f40fb5f0 msgr2=0x7fc7f410ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 shutdown_connections 2023-07-21T19:10:38.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:38.726+0000 7fc7f8fff700 1 -- 172.21.15.5:0/3754051122 wait complete. 2023-07-21T19:10:38.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:38 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1673445468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:38.755 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:10:38.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:38 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1673445468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:38.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:38 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1673445468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:10:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 -- 172.21.15.5:0/265545761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 msgr2=0x7f1c4410e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 --2- 172.21.15.5:0/265545761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4410e810 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b0a0 tx=0x7f1c3c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 -- 172.21.15.5:0/265545761 shutdown_connections 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 --2- 172.21.15.5:0/265545761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4410e810 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 --2- 172.21.15.5:0/265545761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440fe880 0x7f1c440fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 --2- 172.21.15.5:0/265545761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440ff190 0x7f1c4410bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 -- 172.21.15.5:0/265545761 >> 172.21.15.5:0/265545761 conn(0x7f1c440fa4b0 msgr2=0x7f1c440fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 -- 172.21.15.5:0/265545761 shutdown_connections 2023-07-21T19:10:39.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.013+0000 7f1c49dff700 1 -- 172.21.15.5:0/265545761 wait complete. 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.014+0000 7f1c49dff700 1 Processor -- start 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.014+0000 7f1c49dff700 1 -- start start 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 0x7f1c44197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440ff190 0x7f1c44198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4419ce00 con 0x7f1c4410c430 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4419cf70 con 0x7f1c440ff190 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c49dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4419d130 con 0x7f1c440fe880 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440ff190 0x7f1c44198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c43fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37716/0 (socket says 172.21.15.5:37716) 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c43fff700 1 -- 172.21.15.5:0/639689697 learned_addr learned my addr 172.21.15.5:0/639689697 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.015+0000 7f1c437fe700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 0x7f1c44197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 -- 172.21.15.5:0/639689697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 msgr2=0x7f1c44197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 0x7f1c44197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 -- 172.21.15.5:0/639689697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440ff190 msgr2=0x7f1c44198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440ff190 0x7f1c44198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 -- 172.21.15.5:0/639689697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c3c009d00 con 0x7f1c4410c430 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c437fe700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 0x7f1c44197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:39.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c43fff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b780 tx=0x7f1c3c00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c40ff9700 1 -- 172.21.15.5:0/639689697 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c00e040 con 0x7f1c4410c430 2023-07-21T19:10:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c40ff9700 1 -- 172.21.15.5:0/639689697 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c3c009480 con 0x7f1c4410c430 2023-07-21T19:10:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4419d3b0 con 0x7f1c4410c430 2023-07-21T19:10:39.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c40ff9700 1 -- 172.21.15.5:0/639689697 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3c0052a0 con 0x7f1c4410c430 2023-07-21T19:10:39.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.016+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4419d630 con 0x7f1c4410c430 2023-07-21T19:10:39.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.021+0000 7f1c40ff9700 1 -- 172.21.15.5:0/639689697 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1c3c005400 con 0x7f1c4410c430 2023-07-21T19:10:39.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.022+0000 7f1c5d76db80 1 Processor -- start 2023-07-21T19:10:39.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.022+0000 7f1c5d76db80 1 -- start start 2023-07-21T19:10:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047c2d0 0x55aea047c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0489eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea0382cb0 con 0x55aea047cbe0 2023-07-21T19:10:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea03d5a40 con 0x55aea0487ad0 2023-07-21T19:10:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea03ceed0 con 0x55aea047c2d0 2023-07-21T19:10:39.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c40ff9700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 0x7f1c2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c437fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37718/0 (socket says 172.21.15.5:37718) 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c437fe700 1 -- 172.21.15.5:0/3882046113 learned_addr learned my addr 172.21.15.5:0/3882046113 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047c2d0 0x55aea047c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0489eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.023+0000 7f1c40ff9700 1 -- 172.21.15.5:0/639689697 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1c3c0cde50 con 0x7f1c4410c430 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c43fff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 0x7f1c2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 -- 172.21.15.5:0/3882046113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047c2d0 msgr2=0x55aea047c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047c2d0 0x55aea047c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 -- 172.21.15.5:0/3882046113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 msgr2=0x55aea0489eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0489eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 -- 172.21.15.5:0/3882046113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aea01fcde0 con 0x55aea047cbe0 2023-07-21T19:10:39.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c437fe700 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009de0 tx=0x7f1c340116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d9b5cfdafb66e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c43fff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 0x7f1c2c0a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0cb680 tx=0x7f1c3c009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.024+0000 7f1c29ffb700 1 -- 172.21.15.5:0/3882046113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c34011dc0 con 0x55aea047cbe0 2023-07-21T19:10:39.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c29ffb700 1 -- 172.21.15.5:0/3882046113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c340128e0 con 0x55aea047cbe0 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c29ffb700 1 -- 172.21.15.5:0/3882046113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3400f640 con 0x55aea047cbe0 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 -- 172.21.15.5:0/3882046113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 msgr2=0x55aea0487590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f1c34009de0 tx=0x7f1c340116c0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 -- 172.21.15.5:0/3882046113 shutdown_connections 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0489eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047c2d0 0x55aea047c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/3882046113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047cbe0 0x55aea0487590 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 -- 172.21.15.5:0/3882046113 >> 172.21.15.5:0/3882046113 conn(0x55aea0472da0 msgr2=0x55aea04786a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.025+0000 7f1c5d76db80 1 -- 172.21.15.5:0/3882046113 shutdown_connections 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.026+0000 7f1c5d76db80 1 -- 172.21.15.5:0/3882046113 wait complete. 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.026+0000 7f1c5d76db80 1 -- start start 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.026+0000 7f1c5d76db80 1 Processor -- start 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea0484760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0484e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea036eac0 con 0x55aea047c2d0 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea036e7c0 con 0x55aea0487ad0 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea0252920 con 0x55aea047cbe0 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c5d76db80 10 client.0 _ll_register_callbacks cb 0x55aea036c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37724/0 (socket says 172.21.15.5:37724) 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 -- 172.21.15.5:0/4175779363 learned_addr learned my addr 172.21.15.5:0/4175779363 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c437fe700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea0484760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 -- 172.21.15.5:0/4175779363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 msgr2=0x55aea0484760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea0484760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 -- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 msgr2=0x55aea0484e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0484e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.027+0000 7f1c42ffd700 1 -- 172.21.15.5:0/4175779363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c34009a10 con 0x55aea047c2d0 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c38004590 tx=0x7f1c38003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58a6c205f57e8944 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:39.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38004830 con 0x55aea047c2d0 2023-07-21T19:10:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aea03284f0 con 0x55aea047c2d0 2023-07-21T19:10:39.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1c38004990 con 0x55aea047c2d0 2023-07-21T19:10:39.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c38004b50 con 0x55aea047c2d0 2023-07-21T19:10:39.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c5d76db80 10 client.16254 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.028+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aea01e5c60 con 0x55aea047c2d0 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aea03284f0 con 0x55aea047c2d0 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1c38028530 con 0x55aea047c2d0 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 10 client.16254 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 10 client.16254 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 20 client.16254 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:39.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea0497430 0x55aea0499840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 4 client.16254 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.029+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aea03c7eb0 con 0x55aea0497430 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.030+0000 7f1c437fe700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea0497430 0x55aea0499840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.030+0000 7f1c437fe700 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea0497430 0x55aea0499840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=96db9a207b5b952e server_cookie=2c89606bac0d928e in_seq=0 out_seq=0 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.030+0000 7f1c22ffd700 10 client.16254 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c22ffd700 1 -- 172.21.15.5:0/4175779363 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aea03c7eb0 con 0x55aea0497430 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c22ffd700 10 client.16254 handle_command_reply: tid=0 2023-07-21T19:10:39.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 client.16254 shutdown 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 msgr2=0x55aea04874e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f1c38004590 tx=0x7f1c38003960 comp rx=0 tx=0).stop 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 shutdown_connections 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea0497430 0x55aea0499840 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea0487ad0 0x55aea0484e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea04874e0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c22ffd700 0 client.16254 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:39.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/4175779363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea0484760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.031+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 >> 172.21.15.5:0/4175779363 conn(0x55aea0472da0 msgr2=0x55aea0478230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.032+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 shutdown_connections 2023-07-21T19:10:39.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.032+0000 7f1c5d76db80 1 -- 172.21.15.5:0/4175779363 wait complete. 2023-07-21T19:10:39.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.032+0000 7f1c5d76db80 20 client.16254 trim_cache size 0 max 16384 2023-07-21T19:10:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 Processor -- start 2023-07-21T19:10:39.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 -- start start 2023-07-21T19:10:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047cbe0 0x55aea0493780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea0487ad0 0x55aea0487080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea0485d10 con 0x55aea047c2d0 2023-07-21T19:10:39.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea0494ae0 con 0x55aea047cbe0 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.049+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea0482cc0 con 0x55aea0487ad0 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37740/0 (socket says 172.21.15.5:37740) 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 -- 172.21.15.5:0/942600129 learned_addr learned my addr 172.21.15.5:0/942600129 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea0487ad0 0x55aea0487080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c437fe700 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047cbe0 0x55aea0493780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 -- 172.21.15.5:0/942600129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea0487ad0 msgr2=0x55aea0487080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea0487ad0 0x55aea0487080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 -- 172.21.15.5:0/942600129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047cbe0 msgr2=0x55aea0493780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047cbe0 0x55aea0493780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 -- 172.21.15.5:0/942600129 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aea0223250 con 0x55aea047c2d0 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.050+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1c38007f20 tx=0x7f1c38007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d779017b3914b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.051+0000 7f1c2a7fc700 1 -- 172.21.15.5:0/942600129 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c38028b30 con 0x55aea047c2d0 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.051+0000 7f1c2a7fc700 1 -- 172.21.15.5:0/942600129 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c38028450 con 0x55aea047c2d0 2023-07-21T19:10:39.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.051+0000 7f1c2a7fc700 1 -- 172.21.15.5:0/942600129 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c380047c0 con 0x55aea047c2d0 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- 172.21.15.5:0/942600129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 msgr2=0x55aea0484dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1c38007f20 tx=0x7f1c38007e90 comp rx=0 tx=0).stop 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- 172.21.15.5:0/942600129 shutdown_connections 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea0487ad0 0x55aea0487080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea047c2d0 0x55aea0484dc0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/942600129 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047cbe0 0x55aea0493780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- 172.21.15.5:0/942600129 >> 172.21.15.5:0/942600129 conn(0x55aea0472da0 msgr2=0x55aea049e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- 172.21.15.5:0/942600129 shutdown_connections 2023-07-21T19:10:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- 172.21.15.5:0/942600129 wait complete. 2023-07-21T19:10:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.052+0000 7f1c5d76db80 1 -- start start 2023-07-21T19:10:39.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 Processor -- start 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047c2d0 0x55aea048c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea048c850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea04ea7f0 con 0x55aea0487ad0 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea04ea960 con 0x55aea047c2d0 2023-07-21T19:10:39.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aea04eab20 con 0x55aea047cbe0 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.053+0000 7f1c5d76db80 10 client.0 _ll_register_callbacks cb 0x7f1c38004060 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37754/0 (socket says 172.21.15.5:37754) 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 -- 172.21.15.5:0/898763919 learned_addr learned my addr 172.21.15.5:0/898763919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047c2d0 0x55aea048c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c437fe700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea048c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 -- 172.21.15.5:0/898763919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 msgr2=0x55aea048c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea048c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 -- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047c2d0 msgr2=0x55aea048c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047c2d0 0x55aea048c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.054+0000 7f1c42ffd700 1 -- 172.21.15.5:0/898763919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aea01e5c60 con 0x55aea0487ad0 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c42ffd700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1c38007430 tx=0x7f1c38005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52eec9592a101a64 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c22ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c3800fe30 con 0x55aea0487ad0 2023-07-21T19:10:39.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1c38000e40 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1c340128e0 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1c3800f3f0 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c5d76db80 10 client.16260 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55aea01fb2b0 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.055+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1c340128e0 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1c38005470 con 0x55aea0487ad0 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 10 client.16260 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:39.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 10 client.16260 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 20 client.16260 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea04ebdf0 0x55aea04ee200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 4 client.16260 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aea038a050 con 0x55aea04ebdf0 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.056+0000 7f1c437fe700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea04ebdf0 0x55aea04ee200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c437fe700 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea04ebdf0 0x55aea04ee200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9687648ccb999db5 server_cookie=702f4e4022b919d1 in_seq=0 out_seq=0 2023-07-21T19:10:39.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c22ffd700 10 client.16260 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c22ffd700 1 -- 172.21.15.5:0/898763919 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aea038a050 con 0x55aea04ebdf0 2023-07-21T19:10:39.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c22ffd700 10 client.16260 handle_command_reply: tid=0 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 client.16260 shutdown 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 msgr2=0x55aea04ea2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f1c38007430 tx=0x7f1c38005a40 comp rx=0 tx=0).stop 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 shutdown_connections 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aea0487ad0 0x55aea04ea2b0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aea04ebdf0 0x55aea04ee200 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aea047c2d0 0x55aea048c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c5d76db80 1 --2- 172.21.15.5:0/898763919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aea047cbe0 0x55aea048c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.057+0000 7f1c22ffd700 0 client.16260 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.058+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 >> 172.21.15.5:0/898763919 conn(0x55aea0472da0 msgr2=0x55aea0493780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.058+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 shutdown_connections 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.058+0000 7f1c5d76db80 1 -- 172.21.15.5:0/898763919 wait complete. 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.058+0000 7f1c5d76db80 20 client.16260 trim_cache size 0 max 16384 2023-07-21T19:10:39.074 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 msgr2=0x7f1c2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 0x7f1c2c0a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c0cb680 tx=0x7f1c3c009040 comp rx=0 tx=0).stop 2023-07-21T19:10:39.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 msgr2=0x7f1c4419c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c00b780 tx=0x7f1c3c00bce0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 shutdown_connections 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1c2c0a12c0 0x7f1c2c0a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1c4410c430 0x7f1c4419c7c0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1c440fe880 0x7f1c44197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 --2- 172.21.15.5:0/639689697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1c440ff190 0x7f1c44198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 >> 172.21.15.5:0/639689697 conn(0x7f1c440fa4b0 msgr2=0x7f1c4410d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 shutdown_connections 2023-07-21T19:10:39.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.059+0000 7f1c49dff700 1 -- 172.21.15.5:0/639689697 wait complete. 2023-07-21T19:10:39.088 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:10:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.305+0000 7feb9a5ff700 1 -- 172.21.15.5:0/4102762902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb940ffae0 msgr2=0x7feb940fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.305+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/4102762902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb940ffae0 0x7feb940fff50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7feb88009990 tx=0x7feb88009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 -- 172.21.15.5:0/4102762902 shutdown_connections 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/4102762902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb9410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/4102762902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb94105af0 0x7feb94105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/4102762902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb940ffae0 0x7feb940fff50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 -- 172.21.15.5:0/4102762902 >> 172.21.15.5:0/4102762902 conn(0x7feb940fb5e0 msgr2=0x7feb940fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 -- 172.21.15.5:0/4102762902 shutdown_connections 2023-07-21T19:10:39.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.306+0000 7feb9a5ff700 1 -- 172.21.15.5:0/4102762902 wait complete. 2023-07-21T19:10:39.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 Processor -- start 2023-07-21T19:10:39.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 -- start start 2023-07-21T19:10:39.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb940ffae0 0x7feb94072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb94072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb94073580 con 0x7feb94105af0 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb941a4290 con 0x7feb940ffae0 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.307+0000 7feb9a5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb941a43d0 con 0x7feb94100490 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37776/0 (socket says 172.21.15.5:37776) 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 -- 172.21.15.5:0/2456465728 learned_addr learned my addr 172.21.15.5:0/2456465728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb937fe700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb94072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb93fff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb940ffae0 0x7feb94072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 -- 172.21.15.5:0/2456465728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 msgr2=0x7feb94072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb94072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 -- 172.21.15.5:0/2456465728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb940ffae0 msgr2=0x7feb94072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb940ffae0 0x7feb94072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb98b9c700 1 -- 172.21.15.5:0/2456465728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb880095f0 con 0x7feb94105af0 2023-07-21T19:10:39.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.308+0000 7feb937fe700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb94072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb98b9c700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ef30 tx=0x7feb8c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb917fa700 1 -- 172.21.15.5:0/2456465728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c018070 con 0x7feb94105af0 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb917fa700 1 -- 172.21.15.5:0/2456465728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb8c00f040 con 0x7feb94105af0 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb941a45d0 con 0x7feb94105af0 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb917fa700 1 -- 172.21.15.5:0/2456465728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c00cbc0 con 0x7feb94105af0 2023-07-21T19:10:39.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.309+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb941a4ab0 con 0x7feb94105af0 2023-07-21T19:10:39.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.313+0000 7feb917fa700 1 -- 172.21.15.5:0/2456465728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7feb8c00cd80 con 0x7feb94105af0 2023-07-21T19:10:39.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.314+0000 7febadeb2b80 1 Processor -- start 2023-07-21T19:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 -- start start 2023-07-21T19:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646893242d0 0x5646893246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x564689331eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468922acb0 con 0x564689324be0 2023-07-21T19:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468927da40 con 0x56468932fad0 2023-07-21T19:10:39.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7febadeb2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689276ed0 con 0x5646893242d0 2023-07-21T19:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.315+0000 7feb93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb93fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37788/0 (socket says 172.21.15.5:37788) 2023-07-21T19:10:39.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x564689331eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb93fff700 1 -- 172.21.15.5:0/2306587743 learned_addr learned my addr 172.21.15.5:0/2306587743 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646893242d0 0x5646893246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb917fa700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 0x7feb7c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 -- 172.21.15.5:0/2306587743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646893242d0 msgr2=0x5646893246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646893242d0 0x5646893246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 -- 172.21.15.5:0/2306587743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 msgr2=0x56468932f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 -- 172.21.15.5:0/2306587743 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5646890a4de0 con 0x56468932fad0 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb917fa700 1 -- 172.21.15.5:0/2456465728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7feb8c0cc810 con 0x7feb94105af0 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb98b9c700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 0x7feb7c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.316+0000 7feb937fe700 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x564689331eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb88005910 tx=0x7feb8801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdc178775a95cf78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.317+0000 7feb93fff700 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.317+0000 7feb7a7fc700 1 -- 172.21.15.5:0/2306587743 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800f460 con 0x56468932fad0 2023-07-21T19:10:39.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.317+0000 7feb7a7fc700 1 -- 172.21.15.5:0/2306587743 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb8800fa50 con 0x56468932fad0 2023-07-21T19:10:39.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.317+0000 7feb7a7fc700 1 -- 172.21.15.5:0/2306587743 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb880177b0 con 0x56468932fad0 2023-07-21T19:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- 172.21.15.5:0/2306587743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 msgr2=0x564689331eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7feb98b9c700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 0x7feb7c0a37f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0c9f90 tx=0x7feb8c004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x564689331eb0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7feb88005910 tx=0x7feb8801a040 comp rx=0 tx=0).stop 2023-07-21T19:10:39.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- 172.21.15.5:0/2306587743 shutdown_connections 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x564689331eb0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5646893242d0 0x5646893246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2306587743 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564689324be0 0x56468932f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- 172.21.15.5:0/2306587743 >> 172.21.15.5:0/2306587743 conn(0x56468931ada0 msgr2=0x5646893206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- 172.21.15.5:0/2306587743 shutdown_connections 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- 172.21.15.5:0/2306587743 wait complete. 2023-07-21T19:10:39.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.318+0000 7febadeb2b80 1 -- start start 2023-07-21T19:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 Processor -- start 2023-07-21T19:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564689324be0 0x56468932c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x56468932cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689216ac0 con 0x5646893242d0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646892167c0 con 0x56468932fad0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646890fa920 con 0x564689324be0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7febadeb2b80 10 client.0 _ll_register_callbacks cb 0x564689214760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.319+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37792/0 (socket says 172.21.15.5:37792) 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 -- 172.21.15.5:0/830109089 learned_addr learned my addr 172.21.15.5:0/830109089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb93fff700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564689324be0 0x56468932c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x56468932cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 -- 172.21.15.5:0/830109089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564689324be0 msgr2=0x56468932c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564689324be0 0x56468932c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 -- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 msgr2=0x56468932cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x56468932cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 -- 172.21.15.5:0/830109089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb840078a0 con 0x5646893242d0 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb937fe700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feb88017430 tx=0x7feb88005600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=546fdebda752fa3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.320+0000 7feb737fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800f830 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5646891d04f0 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7feb88017490 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb88017650 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7febadeb2b80 10 client.16272 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7feb88020040 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.321+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb88043080 con 0x5646893242d0 2023-07-21T19:10:39.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feb8803f7e0 con 0x5646893242d0 2023-07-21T19:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 10 client.16272 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 20 client.16272 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56468933eec0 0x5646893412a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 4 client.16272 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:39.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56468926feb0 con 0x56468933eec0 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.322+0000 7feb93fff700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56468933eec0 0x5646893412a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.323+0000 7feb737fe700 10 client.16272 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.323+0000 7feb93fff700 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56468933eec0 0x5646893412a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42db86f9a98a4ab server_cookie=f741ea25b3e2f0bf in_seq=0 out_seq=0 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7feb737fe700 1 -- 172.21.15.5:0/830109089 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56468926feb0 con 0x56468933eec0 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7feb737fe700 10 client.16272 handle_command_reply: tid=0 2023-07-21T19:10:39.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 client.16272 shutdown 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 msgr2=0x56468932f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7feb88017430 tx=0x7feb88005600 comp rx=0 tx=0).stop 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 shutdown_connections 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56468932fad0 0x56468932cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56468933eec0 0x5646893412a0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x56468932f570 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 --2- 172.21.15.5:0/830109089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564689324be0 0x56468932c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 >> 172.21.15.5:0/830109089 conn(0x56468931ada0 msgr2=0x564689320230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7feb737fe700 0 client.16272 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:39.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 shutdown_connections 2023-07-21T19:10:39.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.324+0000 7febadeb2b80 1 -- 172.21.15.5:0/830109089 wait complete. 2023-07-21T19:10:39.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.325+0000 7febadeb2b80 20 client.16272 trim_cache size 0 max 16384 2023-07-21T19:10:39.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.340+0000 7febadeb2b80 1 Processor -- start 2023-07-21T19:10:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 -- start start 2023-07-21T19:10:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x56468932f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689327af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56468933bd80 con 0x5646893242d0 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689325540 con 0x564689324be0 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7febadeb2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689321330 con 0x56468932fad0 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37796/0 (socket says 172.21.15.5:37796) 2023-07-21T19:10:39.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7feb93fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x56468932f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.341+0000 7feb937fe700 1 -- 172.21.15.5:0/2399251582 learned_addr learned my addr 172.21.15.5:0/2399251582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 -- 172.21.15.5:0/2399251582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 msgr2=0x564689327af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689327af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 -- 172.21.15.5:0/2399251582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 msgr2=0x56468932f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x56468932f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 -- 172.21.15.5:0/2399251582 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56468908dc60 con 0x5646893242d0 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb937fe700 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7feb88005670 tx=0x7feb8800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0e602b4d5f837c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb7affd700 1 -- 172.21.15.5:0/2399251582 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8800f830 con 0x5646893242d0 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb7affd700 1 -- 172.21.15.5:0/2399251582 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5646891d04f0 con 0x5646893242d0 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.342+0000 7feb7affd700 1 -- 172.21.15.5:0/2399251582 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb88029720 con 0x5646893242d0 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 -- 172.21.15.5:0/2399251582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 msgr2=0x5646893364b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7feb88005670 tx=0x7feb8800b260 comp rx=0 tx=0).stop 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 -- 172.21.15.5:0/2399251582 shutdown_connections 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689327af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893364b0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2399251582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x56468932f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 -- 172.21.15.5:0/2399251582 >> 172.21.15.5:0/2399251582 conn(0x56468931ada0 msgr2=0x56468931ee10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 -- 172.21.15.5:0/2399251582 shutdown_connections 2023-07-21T19:10:39.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.343+0000 7febadeb2b80 1 -- 172.21.15.5:0/2399251582 wait complete. 2023-07-21T19:10:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.344+0000 7febadeb2b80 1 -- start start 2023-07-21T19:10:39.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.344+0000 7febadeb2b80 1 Processor -- start 2023-07-21T19:10:39.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x5646893345d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689334b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689335050 con 0x5646893242d0 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5646893351c0 con 0x564689324be0 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564689335380 con 0x56468932fad0 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7febadeb2b80 10 client.0 _ll_register_callbacks cb 0x7feb84007710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37810/0 (socket says 172.21.15.5:37810) 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 -- 172.21.15.5:0/2957594407 learned_addr learned my addr 172.21.15.5:0/2957594407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb93fff700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x5646893345d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689334b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 -- 172.21.15.5:0/2957594407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 msgr2=0x564689334b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689334b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 -- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 msgr2=0x5646893345d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x5646893345d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.345+0000 7feb937fe700 1 -- 172.21.15.5:0/2957594407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb88020040 con 0x5646893242d0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7feb937fe700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feb88005850 tx=0x7feb88038040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3a3c338ba3f5eef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7feb737fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8803fdc0 con 0x5646893242d0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7feb88000e40 con 0x5646893242d0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb8800fa50 con 0x5646893242d0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb8800f830 con 0x5646893242d0 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7febadeb2b80 10 client.16278 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:39.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.346+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5646890cb250 con 0x5646893242d0 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb8800fa50 con 0x5646893242d0 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7feb88036490 con 0x5646893242d0 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 10 client.16278 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 20 client.16278 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564689393c10 0x564689395ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:39.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 4 client.16278 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564689232050 con 0x564689393c10 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.347+0000 7feb93fff700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564689393c10 0x564689395ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7feb93fff700 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564689393c10 0x564689395ff0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6421a3cca88cad91 server_cookie=5021769f49614166 in_seq=0 out_seq=0 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7feb737fe700 10 client.16278 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7feb737fe700 1 -- 172.21.15.5:0/2957594407 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564689232050 con 0x564689393c10 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7feb737fe700 10 client.16278 handle_command_reply: tid=0 2023-07-21T19:10:39.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7febadeb2b80 1 client.16278 shutdown 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 msgr2=0x5646893362a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7feb88005850 tx=0x7feb88038040 comp rx=0 tx=0).stop 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.348+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 shutdown_connections 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56468932fad0 0x564689334b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564689393c10 0x564689395ff0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5646893242d0 0x5646893362a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 --2- 172.21.15.5:0/2957594407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564689324be0 0x5646893345d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7feb737fe700 0 client.16278 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 >> 172.21.15.5:0/2957594407 conn(0x56468931ada0 msgr2=0x564689327af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 shutdown_connections 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 1 -- 172.21.15.5:0/2957594407 wait complete. 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7febadeb2b80 20 client.16278 trim_cache size 0 max 16384 2023-07-21T19:10:39.366 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:10:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 msgr2=0x7feb7c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 0x7feb7c0a37f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb8c0c9f90 tx=0x7feb8c004ad0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 msgr2=0x7feb941a3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.349+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7feb8c00ef30 tx=0x7feb8c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 shutdown_connections 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb7c0a1340 0x7feb7c0a37f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb94105af0 0x7feb941a3d50 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb940ffae0 0x7feb94072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 --2- 172.21.15.5:0/2456465728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb94100490 0x7feb94072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:39.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 >> 172.21.15.5:0/2456465728 conn(0x7feb940fb5e0 msgr2=0x7feb941035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:39.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 shutdown_connections 2023-07-21T19:10:39.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:39.350+0000 7feb9a5ff700 1 -- 172.21.15.5:0/2456465728 wait complete. 2023-07-21T19:10:39.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:39 smithi005 ceph-mon[126460]: pgmap v94: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-07-21T19:10:39.764 INFO:teuthology.orchestra.run.smithi005.stderr:9400000 of 50000000 tuples (18%) done (elapsed 157.32 s, remaining 679.49 s) 2023-07-21T19:10:39.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:39 smithi101 ceph-mon[128136]: pgmap v94: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-07-21T19:10:39.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:39 smithi156 ceph-mon[128316]: pgmap v94: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 33 op/s 2023-07-21T19:10:40.379 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:10:40.380 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:10:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 -- 172.21.15.5:0/955938444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec105ae0 msgr2=0x7fdaec105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/955938444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec105ae0 0x7fdaec105eb0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fdadc009990 tx=0x7fdadc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 -- 172.21.15.5:0/955938444 shutdown_connections 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/955938444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec100480 0x7fdaec10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/955938444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec105ae0 0x7fdaec105eb0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/955938444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 0x7fdaec0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 -- 172.21.15.5:0/955938444 >> 172.21.15.5:0/955938444 conn(0x7fdaec0fb5f0 msgr2=0x7fdaec0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 -- 172.21.15.5:0/955938444 shutdown_connections 2023-07-21T19:10:40.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.608+0000 7fdaf1bff700 1 -- 172.21.15.5:0/955938444 wait complete. 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.609+0000 7fdaf1bff700 1 Processor -- start 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.609+0000 7fdaf1bff700 1 -- start start 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 0x7fdaec197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec105ae0 0x7fdaec19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec19cd00 con 0x7fdaec100480 2023-07-21T19:10:40.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec19ce70 con 0x7fdaec0ffad0 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaf1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaec19d030 con 0x7fdaec105ae0 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37832/0 (socket says 172.21.15.5:37832) 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaeaffd700 1 -- 172.21.15.5:0/856339650 learned_addr learned my addr 172.21.15.5:0/856339650 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 0x7fdaec197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.610+0000 7fdaebfff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec105ae0 0x7fdaec19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 -- 172.21.15.5:0/856339650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec105ae0 msgr2=0x7fdaec19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec105ae0 0x7fdaec19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 -- 172.21.15.5:0/856339650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 msgr2=0x7fdaec197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 0x7fdaec197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 -- 172.21.15.5:0/856339650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdadc0095f0 con 0x7fdaec100480 2023-07-21T19:10:40.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9a0 tx=0x7fdae000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdae8ff9700 1 -- 172.21.15.5:0/856339650 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0009790 con 0x7fdaec100480 2023-07-21T19:10:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaec19d310 con 0x7fdaec100480 2023-07-21T19:10:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdae8ff9700 1 -- 172.21.15.5:0/856339650 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdae0010430 con 0x7fdaec100480 2023-07-21T19:10:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdae8ff9700 1 -- 172.21.15.5:0/856339650 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae000cea0 con 0x7fdaec100480 2023-07-21T19:10:40.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.611+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec1a4b50 con 0x7fdaec100480 2023-07-21T19:10:40.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.616+0000 7fdae8ff9700 1 -- 172.21.15.5:0/856339650 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdae00105a0 con 0x7fdaec100480 2023-07-21T19:10:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.617+0000 7fdb0553db80 1 Processor -- start 2023-07-21T19:10:40.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.617+0000 7fdb0553db80 1 -- start start 2023-07-21T19:10:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405d350 0x55bde405d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405dc60 0x55bde4068610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3f63d30 con 0x55bde4068b50 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3fb6ac0 con 0x55bde405d350 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdb0553db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3faff50 con 0x55bde405dc60 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37836/0 (socket says 172.21.15.5:37836) 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdaeaffd700 1 -- 172.21.15.5:0/734929723 learned_addr learned my addr 172.21.15.5:0/734929723 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdae8ff9700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 0x7fdad40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.618+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405dc60 0x55bde4068610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaebfff700 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405d350 0x55bde405d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaebfff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 0x7fdad40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdae8ff9700 1 -- 172.21.15.5:0/856339650 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdae0014070 con 0x7fdaec100480 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 -- 172.21.15.5:0/734929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405dc60 msgr2=0x55bde4068610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405dc60 0x55bde4068610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 -- 172.21.15.5:0/734929723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405d350 msgr2=0x55bde405d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405d350 0x55bde405d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 -- 172.21.15.5:0/734929723 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde3ddde60 con 0x55bde4068b50 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdae0099d40 tx=0x7fdae0095d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d92be67c417378 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdaebfff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 0x7fdad40a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009ec0 tx=0x7fdae400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.619+0000 7fdad1ffb700 1 -- 172.21.15.5:0/734929723 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00999a0 con 0x55bde4068b50 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.620+0000 7fdad1ffb700 1 -- 172.21.15.5:0/734929723 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdae0010880 con 0x55bde4068b50 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.620+0000 7fdad1ffb700 1 -- 172.21.15.5:0/734929723 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae009a2a0 con 0x55bde4068b50 2023-07-21T19:10:40.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.620+0000 7fdb0553db80 1 -- 172.21.15.5:0/734929723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 msgr2=0x55bde406af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.620+0000 7fdb0553db80 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fdae0099d40 tx=0x7fdae0095d80 comp rx=0 tx=0).stop 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 -- 172.21.15.5:0/734929723 shutdown_connections 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde4068b50 0x55bde406af30 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405d350 0x55bde405d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 --2- 172.21.15.5:0/734929723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405dc60 0x55bde4068610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 -- 172.21.15.5:0/734929723 >> 172.21.15.5:0/734929723 conn(0x55bde4053e20 msgr2=0x55bde4059720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 -- 172.21.15.5:0/734929723 shutdown_connections 2023-07-21T19:10:40.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 -- 172.21.15.5:0/734929723 wait complete. 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.621+0000 7fdb0553db80 1 -- start start 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 Processor -- start 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde4065810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4068b50 0x55bde4065f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3f4fb40 con 0x55bde405d350 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3f4f840 con 0x55bde405dc60 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde3e339a0 con 0x55bde4068b50 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdb0553db80 10 client.0 _ll_register_callbacks cb 0x55bde3f4d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.622+0000 7fdaeb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37846/0 (socket says 172.21.15.5:37846) 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 -- 172.21.15.5:0/2270930371 learned_addr learned my addr 172.21.15.5:0/2270930371 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaebfff700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4068b50 0x55bde4065f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde4065810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 -- 172.21.15.5:0/2270930371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4068b50 msgr2=0x55bde4065f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4068b50 0x55bde4065f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 -- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 msgr2=0x55bde4065810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde4065810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 -- 172.21.15.5:0/2270930371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaec1a4b50 con 0x55bde405d350 2023-07-21T19:10:40.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdadc015000 tx=0x7fdadc0057e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f63b8469a1ffb60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.623+0000 7fdacaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdadc01d070 con 0x55bde405d350 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bde3f09570 con 0x55bde405d350 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdadc004500 con 0x55bde405d350 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdadc0046c0 con 0x55bde405d350 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdb0553db80 10 client.16290 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:40.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bde3dc6ce0 con 0x55bde405d350 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bde3f09570 con 0x55bde405d350 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdadc00f790 con 0x55bde405d350 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.624+0000 7fdb0553db80 10 client.16290 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdb0553db80 10 client.16290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdb0553db80 20 client.16290 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4078530 0x55bde407a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdb0553db80 4 client.16290 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:40.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bde3fa8f30 con 0x55bde4078530 2023-07-21T19:10:40.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.625+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4078530 0x55bde407a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.626+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4078530 0x55bde407a940 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7b47d7316617994 server_cookie=4210a5e91db77ab7 in_seq=0 out_seq=0 2023-07-21T19:10:40.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.626+0000 7fdacaffd700 10 client.16290 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.626+0000 7fdacaffd700 1 -- 172.21.15.5:0/2270930371 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bde3fa8f30 con 0x55bde4078530 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.626+0000 7fdacaffd700 10 client.16290 handle_command_reply: tid=0 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.626+0000 7fdb0553db80 1 client.16290 shutdown 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 msgr2=0x55bde4068590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fdadc015000 tx=0x7fdadc0057e0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 shutdown_connections 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4068b50 0x55bde4065f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4078530 0x55bde407a940 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde4068590 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdacaffd700 0 client.16290 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2270930371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde4065810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 >> 172.21.15.5:0/2270930371 conn(0x55bde4053e20 msgr2=0x55bde4059310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 shutdown_connections 2023-07-21T19:10:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 1 -- 172.21.15.5:0/2270930371 wait complete. 2023-07-21T19:10:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.627+0000 7fdb0553db80 20 client.16290 trim_cache size 0 max 16384 2023-07-21T19:10:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.647+0000 7fdb0553db80 1 Processor -- start 2023-07-21T19:10:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.647+0000 7fdb0553db80 1 -- start start 2023-07-21T19:10:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde4053e20 0x55bde406f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 0x55bde405fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 0x55bde40600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde4066790 con 0x55bde405dc60 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde405f220 con 0x55bde4053e20 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdb0553db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde405a550 con 0x55bde405d350 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 0x55bde40600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 0x55bde405fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 0x55bde40600b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37852/0 (socket says 172.21.15.5:37852) 2023-07-21T19:10:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaebfff700 1 -- 172.21.15.5:0/2326727310 learned_addr learned my addr 172.21.15.5:0/2326727310 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde4053e20 0x55bde406f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2326727310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde4053e20 msgr2=0x55bde406f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde4053e20 0x55bde406f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2326727310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 msgr2=0x55bde40600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 0x55bde40600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.648+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2326727310 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bde3e042d0 con 0x55bde405d350 2023-07-21T19:10:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.649+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 0x55bde405fb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdae0098390 tx=0x7fdae0095ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d16851d66a0bd2cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.649+0000 7fdad27fc700 1 -- 172.21.15.5:0/2326727310 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae0096380 con 0x55bde405d350 2023-07-21T19:10:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.649+0000 7fdad27fc700 1 -- 172.21.15.5:0/2326727310 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdae009e070 con 0x55bde405d350 2023-07-21T19:10:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.649+0000 7fdad27fc700 1 -- 172.21.15.5:0/2326727310 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae009a2a0 con 0x55bde405d350 2023-07-21T19:10:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- 172.21.15.5:0/2326727310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 msgr2=0x55bde405fb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 0x55bde405fb70 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fdae0098390 tx=0x7fdae0095ce0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- 172.21.15.5:0/2326727310 shutdown_connections 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405dc60 0x55bde40600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde4053e20 0x55bde406f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2326727310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde405d350 0x55bde405fb70 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- 172.21.15.5:0/2326727310 >> 172.21.15.5:0/2326727310 conn(0x55bde4055680 msgr2=0x55bde405f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- 172.21.15.5:0/2326727310 shutdown_connections 2023-07-21T19:10:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- 172.21.15.5:0/2326727310 wait complete. 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.650+0000 7fdb0553db80 1 -- start start 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 Processor -- start 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4053e20 0x55bde406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde40c63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde40c6930 con 0x55bde405d350 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde40c6aa0 con 0x55bde405dc60 2023-07-21T19:10:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bde40c6c60 con 0x55bde4053e20 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.651+0000 7fdb0553db80 10 client.0 _ll_register_callbacks cb 0x7fdadc005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4053e20 0x55bde406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37860/0 (socket says 172.21.15.5:37860) 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2526470997 learned_addr learned my addr 172.21.15.5:0/2526470997 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaebfff700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde40c63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2526470997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4053e20 msgr2=0x55bde406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4053e20 0x55bde406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 msgr2=0x55bde40c63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde40c63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 -- 172.21.15.5:0/2526470997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bde3ddde60 con 0x55bde405d350 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdaeaffd700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdae0099fd0 tx=0x7fdae009a0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a33e7328250fecf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.652+0000 7fdacaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdae00cab40 con 0x55bde405d350 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdae00ce070 con 0x55bde405d350 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdae0010880 con 0x55bde405d350 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdae0099530 con 0x55bde405d350 2023-07-21T19:10:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 10 client.16296 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55bde3ddc330 con 0x55bde405d350 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdae0010880 con 0x55bde405d350 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdae00a2090 con 0x55bde405d350 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 10 client.16296 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 10 client.16296 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 20 client.16296 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4068b50 0x55bde40d0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 4 client.16296 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.653+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55bde3f6b0d0 con 0x55bde4068b50 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.654+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4068b50 0x55bde40d0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.654+0000 7fdaeb7fe700 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4068b50 0x55bde40d0020 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b61597c3aeff7d2 server_cookie=9c7ed1a221f12c8b in_seq=0 out_seq=0 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.654+0000 7fdacaffd700 10 client.16296 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdacaffd700 1 -- 172.21.15.5:0/2526470997 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55bde3f6b0d0 con 0x55bde4068b50 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdacaffd700 10 client.16296 handle_command_reply: tid=0 2023-07-21T19:10:40.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 client.16296 shutdown 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 msgr2=0x55bde406e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7fdae0099fd0 tx=0x7fdae009a0d0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 shutdown_connections 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bde405dc60 0x55bde40c63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bde4053e20 0x55bde406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bde4068b50 0x55bde40d0020 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 --2- 172.21.15.5:0/2526470997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bde405d350 0x55bde406e940 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 >> 172.21.15.5:0/2526470997 conn(0x55bde4055680 msgr2=0x55bde405fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdacaffd700 0 client.16296 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 shutdown_connections 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.656+0000 7fdb0553db80 1 -- 172.21.15.5:0/2526470997 wait complete. 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdb0553db80 20 client.16296 trim_cache size 0 max 16384 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:40.672 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:10:40.673 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "02eecd3d-7822-4eb0-a264-ef03395623d7", 2023-07-21T19:10:40.673 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:10:40.673 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:40.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 msgr2=0x7fdad40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 0x7fdad40a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdae4009ec0 tx=0x7fdae400b040 comp rx=0 tx=0).stop 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 msgr2=0x7fdaec198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fdae000e9a0 tx=0x7fdae000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 shutdown_connections 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdaec105ae0 0x7fdaec19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdaec0ffad0 0x7fdaec197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdad40a12c0 0x7fdad40a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 --2- 172.21.15.5:0/856339650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdaec100480 0x7fdaec198260 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.657+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 >> 172.21.15.5:0/856339650 conn(0x7fdaec0fb5f0 msgr2=0x7fdaec0fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.658+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 shutdown_connections 2023-07-21T19:10:40.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.658+0000 7fdaf1bff700 1 -- 172.21.15.5:0/856339650 wait complete. 2023-07-21T19:10:40.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:10:40.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 -- 172.21.15.5:0/4258952896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b580ffad0 msgr2=0x7f3b580fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4258952896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b580ffad0 0x7f3b580fff40 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f3b48009990 tx=0x7f3b48009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 -- 172.21.15.5:0/4258952896 shutdown_connections 2023-07-21T19:10:40.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4258952896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4258952896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b58105ae0 0x7f3b58105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4258952896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b580ffad0 0x7f3b580fff40 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 -- 172.21.15.5:0/4258952896 >> 172.21.15.5:0/4258952896 conn(0x7f3b580fb5f0 msgr2=0x7f3b580fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.941+0000 7f3b605ff700 1 -- 172.21.15.5:0/4258952896 shutdown_connections 2023-07-21T19:10:40.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.942+0000 7f3b605ff700 1 -- 172.21.15.5:0/4258952896 wait complete. 2023-07-21T19:10:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 Processor -- start 2023-07-21T19:10:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 -- start start 2023-07-21T19:10:40.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b580ffad0 0x7f3b5819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5819be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58196030 con 0x7f3b58105ae0 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b581961a0 con 0x7f3b58100480 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.943+0000 7f3b605ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b58196360 con 0x7f3b580ffad0 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5db9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5819be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37872/0 (socket says 172.21.15.5:37872) 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4093775113 learned_addr learned my addr 172.21.15.5:0/4093775113 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b580ffad0 0x7f3b5819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4093775113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b580ffad0 msgr2=0x7f3b5819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b580ffad0 0x7f3b5819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4093775113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 msgr2=0x7f3b5819be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5819be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4093775113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b480095f0 con 0x7f3b58105ae0 2023-07-21T19:10:40.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.944+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:40.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b970 tx=0x7f3b5000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/4093775113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000d860 con 0x7f3b58105ae0 2023-07-21T19:10:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/4093775113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b5000de50 con 0x7f3b58105ae0 2023-07-21T19:10:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b58196640 con 0x7f3b58105ae0 2023-07-21T19:10:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/4093775113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50017400 con 0x7f3b58105ae0 2023-07-21T19:10:40.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.945+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b581a2940 con 0x7f3b58105ae0 2023-07-21T19:10:40.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.950+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/4093775113 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3b5000d9c0 con 0x7f3b58105ae0 2023-07-21T19:10:40.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.951+0000 7f3b73eaab80 1 Processor -- start 2023-07-21T19:10:40.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.951+0000 7f3b73eaab80 1 -- start start 2023-07-21T19:10:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b4f7fe700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 0x7f3b440a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fed2d0 0x55afc4fed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ffc9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4ef3cb0 con 0x55afc4fed2d0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4f46a40 con 0x55afc4fedbe0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4f3fed0 con 0x55afc4ffcee0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/4093775113 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3b50006e40 con 0x7f3b58105ae0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 0x7f3b440a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5db9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fed2d0 0x55afc4fed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37656/0 (socket says 172.21.15.5:37656) 2023-07-21T19:10:40.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4228762859 learned_addr learned my addr 172.21.15.5:0/4228762859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ffc9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.952+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4228762859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 msgr2=0x55afc4ffc9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ffc9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4228762859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fed2d0 msgr2=0x55afc4fed6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fed2d0 0x55afc4fed6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/4228762859 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afc4d6dde0 con 0x55afc4ffcee0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ffc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b50095ec0 tx=0x7f3b50004be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ed2d3514b1531ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 0x7f3b440a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009c60 tx=0x7f3b54009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b4cbfe700 1 -- 172.21.15.5:0/4228762859 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5004e0c0 con 0x55afc4ffcee0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b4cbfe700 1 -- 172.21.15.5:0/4228762859 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b50052070 con 0x55afc4ffcee0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.953+0000 7f3b4cbfe700 1 -- 172.21.15.5:0/4228762859 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000e430 con 0x55afc4ffcee0 2023-07-21T19:10:40.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 -- 172.21.15.5:0/4228762859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 msgr2=0x55afc4fff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f3b50095ec0 tx=0x7f3b50004be0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 -- 172.21.15.5:0/4228762859 shutdown_connections 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4ffcee0 0x55afc4fff2c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fed2d0 0x55afc4fed6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/4228762859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ffc9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 -- 172.21.15.5:0/4228762859 >> 172.21.15.5:0/4228762859 conn(0x55afc4fe3da0 msgr2=0x55afc4fe96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 -- 172.21.15.5:0/4228762859 shutdown_connections 2023-07-21T19:10:40.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.954+0000 7f3b73eaab80 1 -- 172.21.15.5:0/4228762859 wait complete. 2023-07-21T19:10:40.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.955+0000 7f3b73eaab80 1 -- start start 2023-07-21T19:10:40.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.955+0000 7f3b73eaab80 1 Processor -- start 2023-07-21T19:10:40.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc4ffc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ff9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4edfac0 con 0x55afc4ffcee0 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4edf7c0 con 0x55afc4fedbe0 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4dc3920 con 0x55afc4fed2d0 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b73eaab80 10 client.0 _ll_register_callbacks cb 0x55afc4edd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.956+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37900/0 (socket says 172.21.15.5:37900) 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/1108621882 learned_addr learned my addr 172.21.15.5:0/1108621882 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ff9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc4ffc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 -- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 msgr2=0x55afc4ff9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ff9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 -- 172.21.15.5:0/1108621882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 msgr2=0x55afc4ffa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:40.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 -- 172.21.15.5:0/1108621882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b581a2940 con 0x55afc4fed2d0 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ff9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc4ffc8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400eb70 tx=0x7f3b540047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b55ee910feb5d925 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.957+0000 7f3b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401c9b0 con 0x55afc4fed2d0 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3b54023070 con 0x55afc4fed2d0 2023-07-21T19:10:40.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55afc4e994f0 con 0x55afc4fed2d0 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b5400ee80 con 0x55afc4fed2d0 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b73eaab80 10 client.25990 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.958+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afc4d56c60 con 0x55afc4fed2d0 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55afc4e994f0 con 0x55afc4fed2d0 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3b5401d410 con 0x55afc4fed2d0 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 10 client.25990 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 10 client.25990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:40.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 20 client.25990 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc500c6c0 0x55afc500ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 4 client.25990 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55afc4f38eb0 con 0x55afc500c6c0 2023-07-21T19:10:40.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.959+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc500c6c0 0x55afc500ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.960+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc500c6c0 0x55afc500ead0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966772c069a212f7 server_cookie=3044eb95b87a4e86 in_seq=0 out_seq=0 2023-07-21T19:10:40.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.960+0000 7f3b41ffb700 10 client.25990 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1108621882 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55afc4f38eb0 con 0x55afc500c6c0 2023-07-21T19:10:40.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b41ffb700 10 client.25990 handle_command_reply: tid=0 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 client.25990 shutdown 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 msgr2=0x55afc4ffc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc4ffc8c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400eb70 tx=0x7f3b540047c0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 shutdown_connections 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc500c6c0 0x55afc500ead0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4ffa1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc4ffc8c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1108621882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fedbe0 0x55afc4ff9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 >> 172.21.15.5:0/1108621882 conn(0x55afc4fe3da0 msgr2=0x55afc4fe9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b41ffb700 0 client.25990 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 shutdown_connections 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1108621882 wait complete. 2023-07-21T19:10:40.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.961+0000 7f3b73eaab80 20 client.25990 trim_cache size 0 max 16384 2023-07-21T19:10:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.987+0000 7f3b73eaab80 1 Processor -- start 2023-07-21T19:10:40.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.987+0000 7f3b73eaab80 1 -- start start 2023-07-21T19:10:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 0x55afc50012e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fedbe0 0x55afc4fe91b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:40.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4fead20 con 0x55afc4ffcee0 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc5009d10 con 0x55afc4fed2d0 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc4fee540 con 0x55afc4fedbe0 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37912/0 (socket says 172.21.15.5:37912) 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5e39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 0x55afc50012e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/2210653094 learned_addr learned my addr 172.21.15.5:0/2210653094 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:40.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fedbe0 0x55afc4fe91b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/2210653094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fedbe0 msgr2=0x55afc4fe91b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fedbe0 0x55afc4fe91b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/2210653094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 msgr2=0x55afc50012e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.988+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 0x55afc50012e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b5eb9c700 1 -- 172.21.15.5:0/2210653094 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55afc4fe55e0 con 0x55afc4ffcee0 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 0x55afc50012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:41.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3b50095ec0 tx=0x7f3b50099440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83b17fbd65aef642 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b4d3ff700 1 -- 172.21.15.5:0/2210653094 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000e630 con 0x55afc4ffcee0 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b4d3ff700 1 -- 172.21.15.5:0/2210653094 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b5000e430 con 0x55afc4ffcee0 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.989+0000 7f3b4d3ff700 1 -- 172.21.15.5:0/2210653094 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5009d9f0 con 0x55afc4ffcee0 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 -- 172.21.15.5:0/2210653094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 msgr2=0x55afc4fe96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f3b50095ec0 tx=0x7f3b50099440 comp rx=0 tx=0).stop 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 -- 172.21.15.5:0/2210653094 shutdown_connections 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4ffcee0 0x55afc4fe96f0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4fed2d0 0x55afc50012e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/2210653094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fedbe0 0x55afc4fe91b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 -- 172.21.15.5:0/2210653094 >> 172.21.15.5:0/2210653094 conn(0x55afc4fe3da0 msgr2=0x55afc4ff4ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 -- 172.21.15.5:0/2210653094 shutdown_connections 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.990+0000 7f3b73eaab80 1 -- 172.21.15.5:0/2210653094 wait complete. 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 -- start start 2023-07-21T19:10:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 Processor -- start 2023-07-21T19:10:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc50014d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 0x55afc4fffa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc5000110 con 0x55afc4fedbe0 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc505f540 con 0x55afc4ffcee0 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55afc505f680 con 0x55afc4fed2d0 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.991+0000 7f3b73eaab80 10 client.0 _ll_register_callbacks cb 0x7f3b5400e750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:41.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37928/0 (socket says 172.21.15.5:37928) 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 -- 172.21.15.5:0/1760260444 learned_addr learned my addr 172.21.15.5:0/1760260444 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc50014d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 -- 172.21.15.5:0/1760260444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 msgr2=0x55afc50014d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc50014d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 0x55afc4fffa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:41.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 -- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 msgr2=0x55afc4fffa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 0x55afc4fffa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5db9a700 1 -- 172.21.15.5:0/1760260444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55afc4d56c60 con 0x55afc4fedbe0 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc50014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.992+0000 7f3b5eb9c700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 0x55afc4fffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b5db9a700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005910 tx=0x7f3b48005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da134020e364dcb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b41ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4802e030 con 0x55afc4fedbe0 2023-07-21T19:10:41.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b50052070 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f3b4802f070 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b4801d040 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b73eaab80 10 client.16314 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55afc4d6dde0 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.993+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b50052070 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3b480048d0 con 0x55afc4fedbe0 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 10 client.16314 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:41.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 10 client.16314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 20 client.16314 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc5060dc0 0x55afc50631d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 4 client.16314 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55afc4efb050 con 0x55afc5060dc0 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.994+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc5060dc0 0x55afc50631d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b5e39b700 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc5060dc0 0x55afc50631d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13464e660c7130e8 server_cookie=f5c344f376be571d in_seq=0 out_seq=0 2023-07-21T19:10:41.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b41ffb700 10 client.16314 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b41ffb700 1 -- 172.21.15.5:0/1760260444 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55afc4efb050 con 0x55afc5060dc0 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b41ffb700 10 client.16314 handle_command_reply: tid=0 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 client.16314 shutdown 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 msgr2=0x55afc4fff2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f3b48005910 tx=0x7f3b48005510 comp rx=0 tx=0).stop 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 shutdown_connections 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55afc4ffcee0 0x55afc4fffa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55afc4fed2d0 0x55afc50014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55afc5060dc0 0x55afc50631d0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 --2- 172.21.15.5:0/1760260444 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55afc4fedbe0 0x55afc4fff2c0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 >> 172.21.15.5:0/1760260444 conn(0x55afc4fe3da0 msgr2=0x55afc4fe9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.995+0000 7f3b41ffb700 0 client.16314 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 shutdown_connections 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b73eaab80 1 -- 172.21.15.5:0/1760260444 wait complete. 2023-07-21T19:10:41.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b73eaab80 20 client.16314 trim_cache size 0 max 16384 2023-07-21T19:10:41.013 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:10:41.013 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:10:41.013 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:10:41.013 INFO:teuthology.orchestra.run.smithi005.stdout: "02eecd3d-7822-4eb0-a264-ef03395623d7": { 2023-07-21T19:10:41.013 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:10:41.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "02eecd3d-7822-4eb0-a264-ef03395623d7", 2023-07-21T19:10:41.014 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:10:41.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:10:41.014 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:10:41.014 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 msgr2=0x7f3b440a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 0x7f3b440a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009c60 tx=0x7f3b54009260 comp rx=0 tx=0).stop 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 msgr2=0x7f3b581959c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.996+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000b970 tx=0x7f3b5000bc80 comp rx=0 tx=0).stop 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 shutdown_connections 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b440a12c0 0x7f3b440a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b58105ae0 0x7f3b581959c0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b580ffad0 0x7f3b5819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 --2- 172.21.15.5:0/4093775113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b58100480 0x7f3b5819be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 >> 172.21.15.5:0/4093775113 conn(0x7f3b580fb5f0 msgr2=0x7f3b580fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 shutdown_connections 2023-07-21T19:10:41.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:10:40.997+0000 7f3b605ff700 1 -- 172.21.15.5:0/4093775113 wait complete. 2023-07-21T19:10:41.027 INFO:tasks.cephfs.filesystem:scrub status for tag:02eecd3d-7822-4eb0-a264-ef03395623d7 - {'path': '/', 'tag': '02eecd3d-7822-4eb0-a264-ef03395623d7', 'options': 'recursive,force'} 2023-07-21T19:10:41.090 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:10:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:10:40.686+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...5,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v1630 ap=2 f(v20 m2023-07-21T19:10:02.394795+0000 3=3+0) n(v2 rc2023-07-21T19:10:02.394795+0000 b3321 4=3+1) old_inodes=3 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@62} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v1630>,<0x10000000005/data v792>,<0x10000000003/postgres v251>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v266>,<0x10000000001/sv_0 v284>,<0x10000000000/_nogroup v267>,<0x1/volumes v254>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:10:02.381229+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:10:02.394795+0000 b3321 4=3+1)","memory_value.dirstat":"f(v20 m2023-07-21T19:10:02.394795+0000 3=3+0)","memory_value.rstat":"n(v2 rc2023-07-21T19:10:02.394795+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:10:41.138 INFO:teuthology.orchestra.run.smithi005.stderr:9500000 of 50000000 tuples (19%) done (elapsed 158.70 s, remaining 676.54 s) 2023-07-21T19:10:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: pgmap v95: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:10:41.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:41 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: pgmap v95: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:41 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: pgmap v95: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:41.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:41 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:42 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:10:42.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:42 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:10:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:42 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:10:42.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:42 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:10:42.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:42 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:10:42.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:42 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:10:42.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:42 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:10:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:42 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:10:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:42 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:10:42.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:10:42.965 INFO:teuthology.orchestra.run.smithi005.stderr:9600000 of 50000000 tuples (19%) done (elapsed 160.52 s, remaining 675.53 s) 2023-07-21T19:10:43.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:43 smithi101 ceph-mon[128136]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 101 op/s 2023-07-21T19:10:43.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:43 smithi005 ceph-mon[126460]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 101 op/s 2023-07-21T19:10:43.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:43 smithi156 ceph-mon[128316]: pgmap v96: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 33 MiB/s wr, 101 op/s 2023-07-21T19:10:44.715 INFO:teuthology.orchestra.run.smithi005.stderr:9700000 of 50000000 tuples (19%) done (elapsed 162.27 s, remaining 674.18 s) 2023-07-21T19:10:44.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 -- 172.21.15.5:0/2087037333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 msgr2=0x7fa14c105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 --2- 172.21.15.5:0/2087037333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c105d70 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fa13c009990 tx=0x7fa13c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 -- 172.21.15.5:0/2087037333 shutdown_connections 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 --2- 172.21.15.5:0/2087037333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c10c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 --2- 172.21.15.5:0/2087037333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c105d70 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 --2- 172.21.15.5:0/2087037333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 0x7fa14c067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 -- 172.21.15.5:0/2087037333 >> 172.21.15.5:0/2087037333 conn(0x7fa14c074770 msgr2=0x7fa14c074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:44.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.947+0000 7fa150b9d700 1 -- 172.21.15.5:0/2087037333 shutdown_connections 2023-07-21T19:10:44.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.948+0000 7fa150b9d700 1 -- 172.21.15.5:0/2087037333 wait complete. 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.949+0000 7fa150b9d700 1 Processor -- start 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.949+0000 7fa150b9d700 1 -- start start 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.949+0000 7fa150b9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.949+0000 7fa150b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa150b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 0x7fa14c1a3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa150b9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c0ff7d0 con 0x7fa14c068070 2023-07-21T19:10:44.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa150b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c1a4120 con 0x7fa14c1062b0 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa150b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa14c1a4260 con 0x7fa14c1059a0 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37708/0 (socket says 172.21.15.5:37708) 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37958/0 (socket says 172.21.15.5:37958) 2023-07-21T19:10:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 -- 172.21.15.5:0/1838360558 learned_addr learned my addr 172.21.15.5:0/1838360558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14bfff700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 0x7fa14c1a3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 -- 172.21.15.5:0/1838360558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 msgr2=0x7fa14c1a3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 0x7fa14c1a3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 -- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 msgr2=0x7fa14c0fea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14affd700 1 -- 172.21.15.5:0/1838360558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa13c0095f0 con 0x7fa14c1059a0 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.950+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa14affd700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa14000e970 tx=0x7fa14000ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa148ff9700 1 -- 172.21.15.5:0/1838360558 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140009790 con 0x7fa14c1059a0 2023-07-21T19:10:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa148ff9700 1 -- 172.21.15.5:0/1838360558 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa14000c7e0 con 0x7fa14c1059a0 2023-07-21T19:10:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa14c1a44c0 con 0x7fa14c1059a0 2023-07-21T19:10:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa148ff9700 1 -- 172.21.15.5:0/1838360558 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140005440 con 0x7fa14c1059a0 2023-07-21T19:10:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.951+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c1a49a0 con 0x7fa14c1059a0 2023-07-21T19:10:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.955+0000 7fa148ff9700 1 -- 172.21.15.5:0/1838360558 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa14000c950 con 0x7fa14c1059a0 2023-07-21T19:10:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.956+0000 7fa16578db80 1 Processor -- start 2023-07-21T19:10:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.956+0000 7fa16578db80 1 -- start start 2023-07-21T19:10:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90aef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90a91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90a9700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b8faec50 con 0x5565b90af8c0 2023-07-21T19:10:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90019e0 con 0x5565b90af470 2023-07-21T19:10:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa16578db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b8ffae70 con 0x5565b8f056b0 2023-07-21T19:10:44.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa14bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90aef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa14bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90aef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37722/0 (socket says 172.21.15.5:37722) 2023-07-21T19:10:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa14bfff700 1 -- 172.21.15.5:0/3646530618 learned_addr learned my addr 172.21.15.5:0/3646530618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90a91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa14affd700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.957+0000 7fa148ff9700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 0x7fa1340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 -- 172.21.15.5:0/3646530618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 msgr2=0x5565b90aef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90aef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 -- 172.21.15.5:0/3646530618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 msgr2=0x5565b90a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa148ff9700 1 -- 172.21.15.5:0/1838360558 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa140014070 con 0x7fa14c1059a0 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14bfff700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 0x7fa1340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 -- 172.21.15.5:0/3646530618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b8e28d80 con 0x5565b90af470 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14affd700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90a91a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098e0 tx=0x7fa13c005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b532241288f6abe6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:44.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa14bfff700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 0x7fa1340a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa144009fd0 tx=0x7fa144004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.958+0000 7fa131ffb700 1 -- 172.21.15.5:0/3646530618 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c01d050 con 0x5565b90af470 2023-07-21T19:10:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.959+0000 7fa131ffb700 1 -- 172.21.15.5:0/3646530618 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa13c01e040 con 0x5565b90af470 2023-07-21T19:10:44.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.959+0000 7fa131ffb700 1 -- 172.21.15.5:0/3646530618 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c0054c0 con 0x5565b90af470 2023-07-21T19:10:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.959+0000 7fa16578db80 1 -- 172.21.15.5:0/3646530618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 msgr2=0x5565b90a91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.959+0000 7fa16578db80 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90a91a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa13c0098e0 tx=0x7fa13c005ec0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 -- 172.21.15.5:0/3646530618 shutdown_connections 2023-07-21T19:10:44.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90aef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 --2- 172.21.15.5:0/3646530618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90a91a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 -- 172.21.15.5:0/3646530618 >> 172.21.15.5:0/3646530618 conn(0x5565b909ed40 msgr2=0x5565b90a4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 -- 172.21.15.5:0/3646530618 shutdown_connections 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 -- 172.21.15.5:0/3646530618 wait complete. 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.960+0000 7fa16578db80 1 -- start start 2023-07-21T19:10:44.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 Processor -- start 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 0x5565b90b57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b90af8c0 0x5565b90b2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b8f9aa60 con 0x5565b90af470 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b8f9a760 con 0x5565b8f056b0 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b8e7e8c0 con 0x5565b90af8c0 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa16578db80 10 client.0 _ll_register_callbacks cb 0x5565b8f98700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:44.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa14affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa14b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 0x5565b90b57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa14affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37976/0 (socket says 172.21.15.5:37976) 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.961+0000 7fa14affd700 1 -- 172.21.15.5:0/2665301376 learned_addr learned my addr 172.21.15.5:0/2665301376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14bfff700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b90af8c0 0x5565b90b2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 -- 172.21.15.5:0/2665301376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b90af8c0 msgr2=0x5565b90b2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b90af8c0 0x5565b90b2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 -- 172.21.15.5:0/2665301376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 msgr2=0x5565b90b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 -- 172.21.15.5:0/2665301376 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c10d020 con 0x5565b8f056b0 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14affd700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 0x5565b90b57c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa13c000c00 tx=0x7fa13c024720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b81b06820c27319d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.962+0000 7fa12affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa13c024dd0 con 0x5565b8f056b0 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa13c0047e0 con 0x5565b8f056b0 2023-07-21T19:10:44.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565b8f54490 con 0x5565b8f056b0 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa13c0049a0 con 0x5565b8f056b0 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa16578db80 10 client.26045 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5565b8e11c00 con 0x5565b8f056b0 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.963+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5565b8f54490 con 0x5565b8f056b0 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa13c033070 con 0x5565b8f056b0 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 10 client.26045 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 10 client.26045 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:44.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 20 client.26045 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b90c2890 0x5565b90c4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 4 client.26045 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565b8ff3e50 con 0x5565b90c2890 2023-07-21T19:10:44.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.964+0000 7fa14affd700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b90c2890 0x5565b90c4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.965+0000 7fa14affd700 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b90c2890 0x5565b90c4ca0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=373d019faf781436 server_cookie=ec8edfd17dd8abd8 in_seq=0 out_seq=0 2023-07-21T19:10:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.965+0000 7fa12affd700 10 client.26045 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.965+0000 7fa12affd700 1 -- 172.21.15.5:0/2665301376 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565b8ff3e50 con 0x5565b90c2890 2023-07-21T19:10:44.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.965+0000 7fa12affd700 10 client.26045 handle_command_reply: tid=0 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 client.26045 shutdown 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 msgr2=0x5565b90b57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 0x5565b90b57c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa13c000c00 tx=0x7fa13c024720 comp rx=0 tx=0).stop 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 shutdown_connections 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b90af8c0 0x5565b90b2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b8f056b0 0x5565b90b57c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b90c2890 0x5565b90c4ca0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 --2- 172.21.15.5:0/2665301376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa12affd700 0 client.26045 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 >> 172.21.15.5:0/2665301376 conn(0x5565b909ed40 msgr2=0x5565b90a3640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 shutdown_connections 2023-07-21T19:10:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 1 -- 172.21.15.5:0/2665301376 wait complete. 2023-07-21T19:10:44.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.966+0000 7fa16578db80 20 client.26045 trim_cache size 0 max 16384 2023-07-21T19:10:44.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 Processor -- start 2023-07-21T19:10:44.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 -- start start 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90b29d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90b52f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90b35c0 con 0x5565b90af8c0 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90a59c0 con 0x5565b90af470 2023-07-21T19:10:44.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa16578db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90b0f80 con 0x5565b8f056b0 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.985+0000 7fa14affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60800/0 (socket says 172.21.15.5:60800) 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 -- 172.21.15.5:0/817914014 learned_addr learned my addr 172.21.15.5:0/817914014 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90b29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14bfff700 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90b52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 -- 172.21.15.5:0/817914014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 msgr2=0x5565b90b29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90b29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 -- 172.21.15.5:0/817914014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 msgr2=0x5565b90b52f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90b52f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 -- 172.21.15.5:0/817914014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b8e4f1f0 con 0x5565b90af470 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa14affd700 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa140096970 tx=0x7fa140099b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ceaad64d2932a49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.986+0000 7fa1327fc700 1 -- 172.21.15.5:0/817914014 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa14000cc50 con 0x5565b90af470 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa1327fc700 1 -- 172.21.15.5:0/817914014 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa140099f50 con 0x5565b90af470 2023-07-21T19:10:44.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa1327fc700 1 -- 172.21.15.5:0/817914014 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa140005710 con 0x5565b90af470 2023-07-21T19:10:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 -- 172.21.15.5:0/817914014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 msgr2=0x5565b90b9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:45.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fa140096970 tx=0x7fa140099b40 comp rx=0 tx=0).stop 2023-07-21T19:10:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 -- 172.21.15.5:0/817914014 shutdown_connections 2023-07-21T19:10:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af8c0 0x5565b90b52f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90b29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 --2- 172.21.15.5:0/817914014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af470 0x5565b90b9e00 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 -- 172.21.15.5:0/817914014 >> 172.21.15.5:0/817914014 conn(0x5565b909ed40 msgr2=0x5565b90b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.987+0000 7fa16578db80 1 -- 172.21.15.5:0/817914014 shutdown_connections 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.988+0000 7fa16578db80 1 -- 172.21.15.5:0/817914014 wait complete. 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.988+0000 7fa16578db80 1 -- start start 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.988+0000 7fa16578db80 1 Processor -- start 2023-07-21T19:10:45.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af8c0 0x5565b90aaed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90ba300 con 0x5565b90af470 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90ab410 con 0x5565b90af8c0 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b90ab5d0 con 0x5565b8f056b0 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa16578db80 10 client.0 _ll_register_callbacks cb 0x7fa144009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:45.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa14affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa14affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37990/0 (socket says 172.21.15.5:37990) 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa14affd700 1 -- 172.21.15.5:0/1165377075 learned_addr learned my addr 172.21.15.5:0/1165377075 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa14bfff700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af8c0 0x5565b90aaed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.989+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 -- 172.21.15.5:0/1165377075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 msgr2=0x5565b90ac0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90ac0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 -- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af8c0 msgr2=0x5565b90aaed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af8c0 0x5565b90aaed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 -- 172.21.15.5:0/1165377075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa14c1a49a0 con 0x5565b90af470 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa14affd700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa140000c00 tx=0x7fa14009e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c50586cac7a69bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa12affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1400b8210 con 0x5565b90af470 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fa1400055a0 con 0x5565b90af470 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa13c01e040 con 0x5565b90af470 2023-07-21T19:10:45.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa140005760 con 0x5565b90af470 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa16578db80 10 client.16332 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.990+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5565b8e27250 con 0x5565b90af470 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa13c01e040 con 0x5565b90af470 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa140099440 con 0x5565b90af470 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 10 client.16332 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 10 client.16332 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:45.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 20 client.16332 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b9117530 0x5565b9119940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 4 client.16332 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5565b8fb5ff0 con 0x5565b9117530 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.991+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b9117530 0x5565b9119940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.992+0000 7fa14b7fe700 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b9117530 0x5565b9119940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d80a4828b5e944f2 server_cookie=f4bf24e80f27303f in_seq=0 out_seq=0 2023-07-21T19:10:45.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:44.992+0000 7fa12affd700 10 client.16332 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.008+0000 7fa12affd700 1 -- 172.21.15.5:0/1165377075 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5565b8fb5ff0 con 0x5565b9117530 2023-07-21T19:10:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.008+0000 7fa12affd700 10 client.16332 handle_command_reply: tid=0 2023-07-21T19:10:45.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.008+0000 7fa16578db80 1 client.16332 shutdown 2023-07-21T19:10:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 msgr2=0x5565b90aa990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fa140000c00 tx=0x7fa14009e000 comp rx=0 tx=0).stop 2023-07-21T19:10:45.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 shutdown_connections 2023-07-21T19:10:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b90af8c0 0x5565b90aaed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b8f056b0 0x5565b90ac0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b9117530 0x5565b9119940 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 --2- 172.21.15.5:0/1165377075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b90af470 0x5565b90aa990 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa12affd700 0 client.16332 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 >> 172.21.15.5:0/1165377075 conn(0x5565b909ed40 msgr2=0x5565b90afc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 shutdown_connections 2023-07-21T19:10:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 1 -- 172.21.15.5:0/1165377075 wait complete. 2023-07-21T19:10:45.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.009+0000 7fa16578db80 20 client.16332 trim_cache size 0 max 16384 2023-07-21T19:10:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:10:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:10:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:10:45.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 msgr2=0x7fa1340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 0x7fa1340a3770 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa144009fd0 tx=0x7fa144004ad0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 msgr2=0x7fa14c0fefa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fa14000e970 tx=0x7fa14000ec80 comp rx=0 tx=0).stop 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 shutdown_connections 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa14c1062b0 0x7fa14c1a3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa1340a12c0 0x7fa1340a3770 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa14c068070 0x7fa14c0fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 --2- 172.21.15.5:0/1838360558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa14c1059a0 0x7fa14c0fefa0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 >> 172.21.15.5:0/1838360558 conn(0x7fa14c074770 msgr2=0x7fa14c0fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 shutdown_connections 2023-07-21T19:10:45.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:45.010+0000 7fa150b9d700 1 -- 172.21.15.5:0/1838360558 wait complete. 2023-07-21T19:10:45.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:45 smithi101 ceph-mon[128136]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 23 MiB/s wr, 110 op/s 2023-07-21T19:10:45.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:45 smithi005 ceph-mon[126460]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 23 MiB/s wr, 110 op/s 2023-07-21T19:10:45.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:45 smithi156 ceph-mon[128316]: pgmap v97: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 23 MiB/s wr, 110 op/s 2023-07-21T19:10:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:9800000 of 50000000 tuples (19%) done (elapsed 163.67 s, remaining 671.37 s) 2023-07-21T19:10:47.514 INFO:teuthology.orchestra.run.smithi005.stderr:9900000 of 50000000 tuples (19%) done (elapsed 165.07 s, remaining 668.62 s) 2023-07-21T19:10:47.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:47 smithi101 ceph-mon[128136]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 104 op/s 2023-07-21T19:10:47.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:47 smithi005 ceph-mon[126460]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 104 op/s 2023-07-21T19:10:47.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:47 smithi156 ceph-mon[128316]: pgmap v98: 97 pgs: 97 active+clean; 2.2 GiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 22 MiB/s wr, 104 op/s 2023-07-21T19:10:48.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:48 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:10:48.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:48 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:10:48.918 INFO:teuthology.orchestra.run.smithi005.stderr:10000000 of 50000000 tuples (20%) done (elapsed 166.48 s, remaining 665.90 s) 2023-07-21T19:10:48.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:48 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:10:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:49 smithi101 ceph-mon[128136]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 34 MiB/s wr, 135 op/s 2023-07-21T19:10:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:10:49.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:10:49.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:10:49.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:49 smithi005 ceph-mon[126460]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 34 MiB/s wr, 135 op/s 2023-07-21T19:10:49.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:10:49.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:10:49.918 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:10:49.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:49 smithi156 ceph-mon[128316]: pgmap v99: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 34 MiB/s wr, 135 op/s 2023-07-21T19:10:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:10:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:10:49.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:10:50.241 INFO:teuthology.orchestra.run.smithi005.stderr:10100000 of 50000000 tuples (20%) done (elapsed 167.80 s, remaining 662.89 s) 2023-07-21T19:10:51.526 INFO:teuthology.orchestra.run.smithi005.stderr:10200000 of 50000000 tuples (20%) done (elapsed 169.08 s, remaining 659.76 s) 2023-07-21T19:10:51.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:51 smithi101 ceph-mon[128136]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 23 MiB/s wr, 125 op/s 2023-07-21T19:10:51.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:51 smithi005 ceph-mon[126460]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 23 MiB/s wr, 125 op/s 2023-07-21T19:10:51.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:51 smithi156 ceph-mon[128316]: pgmap v100: 97 pgs: 97 active+clean; 2.3 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 175 KiB/s rd, 23 MiB/s wr, 125 op/s 2023-07-21T19:10:52.908 INFO:teuthology.orchestra.run.smithi005.stderr:10300000 of 50000000 tuples (20%) done (elapsed 170.47 s, remaining 657.04 s) 2023-07-21T19:10:53.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:53 smithi101 ceph-mon[128136]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 33 MiB/s wr, 132 op/s 2023-07-21T19:10:53.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:53 smithi005 ceph-mon[126460]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 33 MiB/s wr, 132 op/s 2023-07-21T19:10:53.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:53 smithi156 ceph-mon[128316]: pgmap v101: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 33 MiB/s wr, 132 op/s 2023-07-21T19:10:54.186 INFO:teuthology.orchestra.run.smithi005.stderr:10400000 of 50000000 tuples (20%) done (elapsed 171.74 s, remaining 653.95 s) 2023-07-21T19:10:55.480 INFO:teuthology.orchestra.run.smithi005.stderr:10500000 of 50000000 tuples (21%) done (elapsed 173.04 s, remaining 650.95 s) 2023-07-21T19:10:55.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:55 smithi101 ceph-mon[128136]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 56 op/s 2023-07-21T19:10:55.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:55 smithi005 ceph-mon[126460]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 56 op/s 2023-07-21T19:10:55.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:55 smithi156 ceph-mon[128316]: pgmap v102: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 24 MiB/s wr, 56 op/s 2023-07-21T19:10:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:56.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:10:56.844 INFO:teuthology.orchestra.run.smithi005.stderr:10600000 of 50000000 tuples (21%) done (elapsed 174.40 s, remaining 648.24 s) 2023-07-21T19:10:56.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:56.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:56.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:10:56.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:56.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:56.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:10:57.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:57 smithi101 ceph-mon[128136]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:10:57.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:57 smithi005 ceph-mon[126460]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:10:57.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:57 smithi156 ceph-mon[128316]: pgmap v103: 97 pgs: 97 active+clean; 2.4 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:10:58.144 INFO:teuthology.orchestra.run.smithi005.stderr:10700000 of 50000000 tuples (21%) done (elapsed 175.70 s, remaining 645.33 s) 2023-07-21T19:10:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 -- 172.21.15.5:0/2146308337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 msgr2=0x7f84c8104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 --2- 172.21.15.5:0/2146308337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c8104770 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f84b40099c0 tx=0x7f84b4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 -- 172.21.15.5:0/2146308337 shutdown_connections 2023-07-21T19:10:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 --2- 172.21.15.5:0/2146308337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8104e50 0x7f84c810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 --2- 172.21.15.5:0/2146308337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8067c10 0x7f84c8067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 --2- 172.21.15.5:0/2146308337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c8104770 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 -- 172.21.15.5:0/2146308337 >> 172.21.15.5:0/2146308337 conn(0x7f84c8074760 msgr2=0x7f84c8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.185+0000 7f84cedff700 1 -- 172.21.15.5:0/2146308337 shutdown_connections 2023-07-21T19:10:58.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.186+0000 7f84cedff700 1 -- 172.21.15.5:0/2146308337 wait complete. 2023-07-21T19:10:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.186+0000 7f84cedff700 1 Processor -- start 2023-07-21T19:10:58.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 -- start start 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8067c10 0x7f84c8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 0x7f84c819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c819cfa0 con 0x7f84c80685b0 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c819d110 con 0x7f84c8104e50 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84cedff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84c819d2d0 con 0x7f84c8067c10 2023-07-21T19:10:58.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.187+0000 7f84c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54708/0 (socket says 172.21.15.5:54708) 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 -- 172.21.15.5:0/2667078043 learned_addr learned my addr 172.21.15.5:0/2667078043 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8067c10 0x7f84c8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84cd39c700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 0x7f84c819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 -- 172.21.15.5:0/2667078043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8067c10 msgr2=0x7f84c8197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8067c10 0x7f84c8197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 -- 172.21.15.5:0/2667078043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 msgr2=0x7f84c819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 0x7f84c819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 -- 172.21.15.5:0/2667078043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84b40095f0 con 0x7f84c80685b0 2023-07-21T19:10:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84cd39c700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 0x7f84c819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:10:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.188+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f84b4006010 tx=0x7f84b400f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.189+0000 7f84c5ffb700 1 -- 172.21.15.5:0/2667078043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b401d070 con 0x7f84c80685b0 2023-07-21T19:10:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.189+0000 7f84c5ffb700 1 -- 172.21.15.5:0/2667078043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b4005030 con 0x7f84c80685b0 2023-07-21T19:10:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.189+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84c819d550 con 0x7f84c80685b0 2023-07-21T19:10:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.189+0000 7f84c5ffb700 1 -- 172.21.15.5:0/2667078043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4004a70 con 0x7f84c80685b0 2023-07-21T19:10:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.189+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c81a4df0 con 0x7f84c80685b0 2023-07-21T19:10:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.192+0000 7f84c5ffb700 1 -- 172.21.15.5:0/2667078043 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f84b40051a0 con 0x7f84c80685b0 2023-07-21T19:10:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.194+0000 7f84e26d9b80 1 Processor -- start 2023-07-21T19:10:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.194+0000 7f84e26d9b80 1 -- start start 2023-07-21T19:10:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84c5ffb700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 0x7f84b80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c426fbc50 con 0x564c426526b0 2023-07-21T19:10:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4274e9e0 con 0x564c427f6580 2023-07-21T19:10:58.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c42747e70 con 0x564c427fc2d0 2023-07-21T19:10:58.205 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84c5ffb700 1 -- 172.21.15.5:0/2667078043 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84b40cd900 con 0x7f84c80685b0 2023-07-21T19:10:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 0x7f84b80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54716/0 (socket says 172.21.15.5:54716) 2023-07-21T19:10:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84cd39c700 1 -- 172.21.15.5:0/1034283122 learned_addr learned my addr 172.21.15.5:0/1034283122 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:58.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.195+0000 7f84c7fff700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84cd39c700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 -- 172.21.15.5:0/1034283122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 msgr2=0x564c427f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 -- 172.21.15.5:0/1034283122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 msgr2=0x564c427fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 -- 172.21.15.5:0/1034283122 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c42575d80 con 0x564c427fc2d0 2023-07-21T19:10:58.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84cd39c700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 0x7f84b80a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009c60 tx=0x7f84bc009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84c7fff700 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427f6040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cb020 tx=0x7f84b409a410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58c6dbaaebe56217 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84b2ffd700 1 -- 172.21.15.5:0/1034283122 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b409ea60 con 0x564c427fc2d0 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84b2ffd700 1 -- 172.21.15.5:0/1034283122 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b40a2df0 con 0x564c427fc2d0 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.196+0000 7f84b2ffd700 1 -- 172.21.15.5:0/1034283122 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4005480 con 0x564c427fc2d0 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1034283122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 msgr2=0x564c427f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427f6040 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f84b40cb020 tx=0x7f84b409a410 comp rx=0 tx=0).stop 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1034283122 shutdown_connections 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c426526b0 0x564c427fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1034283122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427f6040 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1034283122 >> 172.21.15.5:0/1034283122 conn(0x564c427ebbe0 msgr2=0x564c427f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1034283122 shutdown_connections 2023-07-21T19:10:58.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.197+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1034283122 wait complete. 2023-07-21T19:10:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.198+0000 7f84e26d9b80 1 -- start start 2023-07-21T19:10:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.198+0000 7f84e26d9b80 1 Processor -- start 2023-07-21T19:10:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c42805390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c426e7a60 con 0x564c427f6580 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c426e7760 con 0x564c426526b0 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c425cb8c0 con 0x564c427fc2d0 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84e26d9b80 10 client.0 _ll_register_callbacks cb 0x564c426e5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84c7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54724/0 (socket says 172.21.15.5:54724) 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84c7fff700 1 -- 172.21.15.5:0/2851936272 learned_addr learned my addr 172.21.15.5:0/2851936272 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.199+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c42805390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84cd39c700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 -- 172.21.15.5:0/2851936272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 msgr2=0x564c427ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 -- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 msgr2=0x564c42805390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c42805390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 -- 172.21.15.5:0/2851936272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0007500 con 0x564c427f6580 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84c7fff700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84b409a730 tx=0x7f84b407e930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7eb323ec4b6312e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84b4082070 con 0x564c427f6580 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c426a1490 con 0x564c427f6580 2023-07-21T19:10:58.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.200+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84b4005480 con 0x564c427f6580 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84b4005640 con 0x564c427f6580 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 10 client.16344 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f84c81a4df0 con 0x564c427f6580 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c426a1490 con 0x564c427f6580 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84b409ac90 con 0x564c427f6580 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 10 client.16344 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 10 client.16344 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 20 client.16344 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c4280f360 0x564c42811740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 4 client.16344 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.201+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c42740e50 con 0x564c4280f360 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.202+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c4280f360 0x564c42811740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.202+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c4280f360 0x564c42811740 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cb6839c521bd1e7 server_cookie=518aeec92a5b3b57 in_seq=0 out_seq=0 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.202+0000 7f84a7fff700 10 client.16344 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84a7fff700 1 -- 172.21.15.5:0/2851936272 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c42740e50 con 0x564c4280f360 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84a7fff700 10 client.16344 handle_command_reply: tid=0 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 client.16344 shutdown 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 msgr2=0x564c428058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f84b409a730 tx=0x7f84b407e930 comp rx=0 tx=0).stop 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 shutdown_connections 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427fc2d0 0x564c427ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c42805390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c4280f360 0x564c42811740 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/2851936272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427f6580 0x564c428058d0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 >> 172.21.15.5:0/2851936272 conn(0x564c427ebbe0 msgr2=0x564c427f04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.203+0000 7f84a7fff700 0 client.16344 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.204+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 shutdown_connections 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.204+0000 7f84e26d9b80 1 -- 172.21.15.5:0/2851936272 wait complete. 2023-07-21T19:10:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.204+0000 7f84e26d9b80 20 client.16344 trim_cache size 0 max 16384 2023-07-21T19:10:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.221+0000 7f84e26d9b80 1 Processor -- start 2023-07-21T19:10:58.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.221+0000 7f84e26d9b80 1 -- start start 2023-07-21T19:10:58.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.221+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 0x564c427fe870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c427fc6a0 con 0x564c427fc2d0 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c427f1af0 con 0x564c427f6580 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c42800bc0 con 0x564c426526b0 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84ccb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 0x564c427fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54736/0 (socket says 172.21.15.5:54736) 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84ccb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 0x564c427fe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34854/0 (socket says 172.21.15.5:34854) 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 -- 172.21.15.5:0/3043942005 learned_addr learned my addr 172.21.15.5:0/3043942005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84c7fff700 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 -- 172.21.15.5:0/3043942005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 msgr2=0x564c427fe870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 0x564c427fe870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 -- 172.21.15.5:0/3043942005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 msgr2=0x564c427f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.222+0000 7f84cd39c700 1 -- 172.21.15.5:0/3043942005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4255ec00 con 0x564c427fc2d0 2023-07-21T19:10:58.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.223+0000 7f84cd39c700 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006130 tx=0x7f84c0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3e6b540ef95fbb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.223+0000 7f84b37fe700 1 -- 172.21.15.5:0/3043942005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000e040 con 0x564c427fc2d0 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.223+0000 7f84b37fe700 1 -- 172.21.15.5:0/3043942005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84c000f040 con 0x564c427fc2d0 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.223+0000 7f84b37fe700 1 -- 172.21.15.5:0/3043942005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c00177c0 con 0x564c427fc2d0 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 -- 172.21.15.5:0/3043942005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 msgr2=0x564c427f87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006130 tx=0x7f84c0009920 comp rx=0 tx=0).stop 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 -- 172.21.15.5:0/3043942005 shutdown_connections 2023-07-21T19:10:58.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c427f87a0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c426526b0 0x564c427fe870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/3043942005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c427f6580 0x564c427f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 -- 172.21.15.5:0/3043942005 >> 172.21.15.5:0/3043942005 conn(0x564c427ebbe0 msgr2=0x564c427f6950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 -- 172.21.15.5:0/3043942005 shutdown_connections 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.224+0000 7f84e26d9b80 1 -- 172.21.15.5:0/3043942005 wait complete. 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 -- start start 2023-07-21T19:10:58.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 Processor -- start 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c4280b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427f6580 0x564c4280bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c428088c0 con 0x564c427fc2d0 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c42808a30 con 0x564c426526b0 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.225+0000 7f84e26d9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c42808bf0 con 0x564c427f6580 2023-07-21T19:10:58.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84e26d9b80 10 client.0 _ll_register_callbacks cb 0x7f84c0009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54748/0 (socket says 172.21.15.5:54748) 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 -- 172.21.15.5:0/1975276510 learned_addr learned my addr 172.21.15.5:0/1975276510 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c4280b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84c7fff700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427f6580 0x564c4280bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 -- 172.21.15.5:0/1975276510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427f6580 msgr2=0x564c4280bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427f6580 0x564c4280bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 -- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 msgr2=0x564c4280b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:10:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c4280b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.226+0000 7f84cd39c700 1 -- 172.21.15.5:0/1975276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84c0007690 con 0x564c427fc2d0 2023-07-21T19:10:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84cd39c700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ee0 tx=0x7f84c00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd5ea4dd2bbcd68f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:10:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84a7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:10:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c000e030 con 0x564c427fc2d0 2023-07-21T19:10:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f84b40a2df0 con 0x564c427fc2d0 2023-07-21T19:10:58.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f84c0023440 con 0x564c427fc2d0 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f84c0023600 con 0x564c427fc2d0 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84e26d9b80 10 client.16350 fetch_fsmap learned FSMap version 26 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.227+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564c4259c1f0 con 0x564c427fc2d0 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f84b40a2df0 con 0x564c427fc2d0 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f84c0020070 con 0x564c427fc2d0 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 10 client.16350 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:10:58.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 10 client.16350 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 20 client.16350 populate_metadata read hostname 'smithi005' 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c42864320 0x564c42866730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 4 client.16350 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c42702ff0 con 0x564c42864320 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.228+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c42864320 0x564c42866730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.229+0000 7f84ccb9b700 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c42864320 0x564c42866730 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bce7025cd104b92 server_cookie=47ef03e79a17e7cd in_seq=0 out_seq=0 2023-07-21T19:10:58.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.229+0000 7f84a7fff700 10 client.16350 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84a7fff700 1 -- 172.21.15.5:0/1975276510 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c42702ff0 con 0x564c42864320 2023-07-21T19:10:58.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84a7fff700 10 client.16350 handle_command_reply: tid=0 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 client.16350 shutdown 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 msgr2=0x564c428081b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f84c0007ee0 tx=0x7f84c00196b0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 shutdown_connections 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c42864320 0x564c42866730 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c427fc2d0 0x564c428081b0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c426526b0 0x564c4280b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84a7fff700 0 client.16350 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:10:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 --2- 172.21.15.5:0/1975276510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c427f6580 0x564c4280bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 >> 172.21.15.5:0/1975276510 conn(0x564c427ebbe0 msgr2=0x564c427fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 shutdown_connections 2023-07-21T19:10:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.245+0000 7f84e26d9b80 1 -- 172.21.15.5:0/1975276510 wait complete. 2023-07-21T19:10:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84e26d9b80 20 client.16350 trim_cache size 0 max 16384 2023-07-21T19:10:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:10:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:10:58.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:10:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:10:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 msgr2=0x7f84b80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 0x7f84b80a37f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f84bc009c60 tx=0x7f84bc009260 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 msgr2=0x7f84c81982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f84b4006010 tx=0x7f84b400f670 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 shutdown_connections 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f84c8104e50 0x7f84c819c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f84c8067c10 0x7f84c8197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f84b80a1340 0x7f84b80a37f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 --2- 172.21.15.5:0/2667078043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f84c80685b0 0x7f84c81982f0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:10:58.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 >> 172.21.15.5:0/2667078043 conn(0x7f84c8074760 msgr2=0x7f84c80fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:10:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.246+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 shutdown_connections 2023-07-21T19:10:58.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:10:58.247+0000 7f84cedff700 1 -- 172.21.15.5:0/2667078043 wait complete. 2023-07-21T19:10:59.431 INFO:teuthology.orchestra.run.smithi005.stderr:10800000 of 50000000 tuples (21%) done (elapsed 176.99 s, remaining 642.40 s) 2023-07-21T19:10:59.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:10:59 smithi101 ceph-mon[128136]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-07-21T19:10:59.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:10:59 smithi005 ceph-mon[126460]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-07-21T19:10:59.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:10:59 smithi156 ceph-mon[128316]: pgmap v104: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 34 MiB/s wr, 49 op/s 2023-07-21T19:11:00.718 INFO:teuthology.orchestra.run.smithi005.stderr:10900000 of 50000000 tuples (21%) done (elapsed 178.27 s, remaining 639.50 s) 2023-07-21T19:11:01.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:01 smithi101 ceph-mon[128136]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:11:01.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:01 smithi005 ceph-mon[126460]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:11:01.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:01 smithi156 ceph-mon[128316]: pgmap v105: 97 pgs: 97 active+clean; 2.5 GiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:11:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:11000000 of 50000000 tuples (22%) done (elapsed 179.81 s, remaining 637.50 s) 2023-07-21T19:11:03.684 INFO:teuthology.orchestra.run.smithi005.stderr:11100000 of 50000000 tuples (22%) done (elapsed 181.24 s, remaining 635.16 s) 2023-07-21T19:11:03.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:03 smithi101 ceph-mon[128136]: pgmap v106: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 38 op/s 2023-07-21T19:11:03.917 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:03 smithi005 ceph-mon[126460]: pgmap v106: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 38 op/s 2023-07-21T19:11:03.929 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:03 smithi156 ceph-mon[128316]: pgmap v106: 97 pgs: 97 active+clean; 2.5 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 38 op/s 2023-07-21T19:11:04.968 INFO:teuthology.orchestra.run.smithi005.stderr:11200000 of 50000000 tuples (22%) done (elapsed 182.53 s, remaining 632.32 s) 2023-07-21T19:11:05.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:05 smithi101 ceph-mon[128136]: pgmap v107: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:11:05.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:05 smithi005 ceph-mon[126460]: pgmap v107: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:11:05.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:05 smithi156 ceph-mon[128316]: pgmap v107: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:11:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:11300000 of 50000000 tuples (22%) done (elapsed 183.83 s, remaining 629.58 s) 2023-07-21T19:11:07.570 INFO:teuthology.orchestra.run.smithi005.stderr:11400000 of 50000000 tuples (22%) done (elapsed 185.13 s, remaining 626.84 s) 2023-07-21T19:11:07.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:07 smithi101 ceph-mon[128136]: pgmap v108: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 33 op/s 2023-07-21T19:11:07.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:07 smithi005 ceph-mon[126460]: pgmap v108: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 33 op/s 2023-07-21T19:11:07.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:07 smithi156 ceph-mon[128316]: pgmap v108: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 22 MiB/s wr, 33 op/s 2023-07-21T19:11:08.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:08 smithi101 ceph-mon[128136]: pgmap v109: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 33 MiB/s wr, 44 op/s 2023-07-21T19:11:08.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:08 smithi005 ceph-mon[126460]: pgmap v109: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 33 MiB/s wr, 44 op/s 2023-07-21T19:11:08.928 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:08 smithi156 ceph-mon[128316]: pgmap v109: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 33 MiB/s wr, 44 op/s 2023-07-21T19:11:08.952 INFO:teuthology.orchestra.run.smithi005.stderr:11500000 of 50000000 tuples (23%) done (elapsed 186.51 s, remaining 624.40 s) 2023-07-21T19:11:10.215 INFO:teuthology.orchestra.run.smithi005.stderr:11600000 of 50000000 tuples (23%) done (elapsed 187.77 s, remaining 621.59 s) 2023-07-21T19:11:11.028 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:11:11.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.270+0000 7f75d1bff700 1 -- 172.21.15.5:0/2334268153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc10a070 msgr2=0x7f75cc10c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.270+0000 7f75d1bff700 1 --2- 172.21.15.5:0/2334268153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc10a070 0x7f75cc10c4b0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b0a0 tx=0x7f75c400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 -- 172.21.15.5:0/2334268153 shutdown_connections 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 --2- 172.21.15.5:0/2334268153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc10a070 0x7f75cc10c4b0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 --2- 172.21.15.5:0/2334268153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc100ca0 0x7f75cc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 --2- 172.21.15.5:0/2334268153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc1015b0 0x7f75cc109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 -- 172.21.15.5:0/2334268153 >> 172.21.15.5:0/2334268153 conn(0x7f75cc0fa4b0 msgr2=0x7f75cc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 -- 172.21.15.5:0/2334268153 shutdown_connections 2023-07-21T19:11:11.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.271+0000 7f75d1bff700 1 -- 172.21.15.5:0/2334268153 wait complete. 2023-07-21T19:11:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.272+0000 7f75d1bff700 1 Processor -- start 2023-07-21T19:11:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.272+0000 7f75d1bff700 1 -- start start 2023-07-21T19:11:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.272+0000 7f75d1bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75d1bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc1015b0 0x7f75cc198400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75d1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 0x7f75cc19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75d1bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc19cea0 con 0x7f75cc100ca0 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75d1bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc19d010 con 0x7f75cc1015b0 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75d1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc19d1d0 con 0x7f75cc10a070 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46634/0 (socket says 172.21.15.5:46634) 2023-07-21T19:11:11.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 -- 172.21.15.5:0/150903831 learned_addr learned my addr 172.21.15.5:0/150903831 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cbfff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 0x7f75cc19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75caffd700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc1015b0 0x7f75cc198400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 -- 172.21.15.5:0/150903831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 msgr2=0x7f75cc19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 0x7f75cc19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 -- 172.21.15.5:0/150903831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc1015b0 msgr2=0x7f75cc198400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc1015b0 0x7f75cc198400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cb7fe700 1 -- 172.21.15.5:0/150903831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75c4009d00 con 0x7f75cc100ca0 2023-07-21T19:11:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.273+0000 7f75cbfff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 0x7f75cc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00e9a0 tx=0x7f75bc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75c8ff9700 1 -- 172.21.15.5:0/150903831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc009790 con 0x7f75cc100ca0 2023-07-21T19:11:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc19d4b0 con 0x7f75cc100ca0 2023-07-21T19:11:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75c8ff9700 1 -- 172.21.15.5:0/150903831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75bc00cbd0 con 0x7f75cc100ca0 2023-07-21T19:11:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75c8ff9700 1 -- 172.21.15.5:0/150903831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc018680 con 0x7f75cc100ca0 2023-07-21T19:11:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.274+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1a4cf0 con 0x7f75cc100ca0 2023-07-21T19:11:11.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.278+0000 7f75c8ff9700 1 -- 172.21.15.5:0/150903831 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f75bc00cd40 con 0x7f75cc100ca0 2023-07-21T19:11:11.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.280+0000 7f75c8ff9700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 0x7f75b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.280+0000 7f75c8ff9700 1 -- 172.21.15.5:0/150903831 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f75bc014070 con 0x7f75cc100ca0 2023-07-21T19:11:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.280+0000 7f75caffd700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 0x7f75b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.281+0000 7f75caffd700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 0x7f75b40a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1994d0 tx=0x7f75c0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.281+0000 7f75e5581b80 1 Processor -- start 2023-07-21T19:11:11.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 -- start start 2023-07-21T19:11:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6fe76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6ff2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6eedcb0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6f40a40 con 0x5582a6fe72d0 2023-07-21T19:11:11.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75e5581b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6f39ed0 con 0x5582a6fe7be0 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.282+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6ff2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46644/0 (socket says 172.21.15.5:46644) 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 -- 172.21.15.5:0/637138752 learned_addr learned my addr 172.21.15.5:0/637138752 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6fe76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 -- 172.21.15.5:0/637138752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 msgr2=0x5582a6ff2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6ff2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 -- 172.21.15.5:0/637138752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 msgr2=0x5582a6fe76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6fe76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 -- 172.21.15.5:0/637138752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582a6d67de0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6fe76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.283+0000 7f75cbfff700 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f75c4015000 tx=0x7f75c40092e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04648984b797587 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.284+0000 7f75b1ffb700 1 -- 172.21.15.5:0/637138752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400e040 con 0x5582a6ff2ad0 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.284+0000 7f75b1ffb700 1 -- 172.21.15.5:0/637138752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75c4007d90 con 0x5582a6ff2ad0 2023-07-21T19:11:11.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.284+0000 7f75b1ffb700 1 -- 172.21.15.5:0/637138752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c4005440 con 0x5582a6ff2ad0 2023-07-21T19:11:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 -- 172.21.15.5:0/637138752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 msgr2=0x5582a6ff4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f75c4015000 tx=0x7f75c40092e0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 -- 172.21.15.5:0/637138752 shutdown_connections 2023-07-21T19:11:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff4eb0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6fe76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 --2- 172.21.15.5:0/637138752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6ff2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 -- 172.21.15.5:0/637138752 >> 172.21.15.5:0/637138752 conn(0x5582a6fddda0 msgr2=0x5582a6fe36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 -- 172.21.15.5:0/637138752 shutdown_connections 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.285+0000 7f75e5581b80 1 -- 172.21.15.5:0/637138752 wait complete. 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.286+0000 7f75e5581b80 1 -- start start 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.286+0000 7f75e5581b80 1 Processor -- start 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.286+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6fef7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6ed9ac0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6ed97c0 con 0x5582a6fe72d0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6dbd920 con 0x5582a6fe7be0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75e5581b80 10 client.0 _ll_register_callbacks cb 0x5582a6ed7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46656/0 (socket says 172.21.15.5:46656) 2023-07-21T19:11:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43174/0 (socket says 172.21.15.5:43174) 2023-07-21T19:11:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75cbfff700 1 -- 172.21.15.5:0/402553270 learned_addr learned my addr 172.21.15.5:0/402553270 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.287+0000 7f75caffd700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6fef7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 -- 172.21.15.5:0/402553270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 msgr2=0x5582a6fef7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6fef7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 -- 172.21.15.5:0/402553270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 msgr2=0x5582a6fefe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 -- 172.21.15.5:0/402553270 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc10d170 con 0x5582a6fe72d0 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cbfff700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0ca780 tx=0x7f75bc09a1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9643c49e1d361559 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75bc099800 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f75bc0a20c0 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.288+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5582a6e934f0 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75bc0a2280 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75e5581b80 10 client.26081 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582a6d50c60 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75bc0c8020 con 0x5582a6fe72d0 2023-07-21T19:11:11.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f75bc096010 con 0x5582a6fe72d0 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75e5581b80 10 client.26081 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.289+0000 7f75e5581b80 10 client.26081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.290+0000 7f75e5581b80 20 client.26081 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.290+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a70023b0 0x5582a70047c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.290+0000 7f75e5581b80 4 client.26081 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.290+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5582a6f32eb0 con 0x5582a70023b0 2023-07-21T19:11:11.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.290+0000 7f75caffd700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a70023b0 0x5582a70047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.291+0000 7f75aaffd700 10 client.26081 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.291+0000 7f75caffd700 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a70023b0 0x5582a70047c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e3b4e90165a0e87 server_cookie=22802eb690f58af1 in_seq=0 out_seq=0 2023-07-21T19:11:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75aaffd700 1 -- 172.21.15.5:0/402553270 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5582a6f32eb0 con 0x5582a70023b0 2023-07-21T19:11:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75aaffd700 10 client.26081 handle_command_reply: tid=0 2023-07-21T19:11:11.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 client.26081 shutdown 2023-07-21T19:11:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 msgr2=0x5582a6ff24c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f75bc0ca780 tx=0x7f75bc09a1f0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 shutdown_connections 2023-07-21T19:11:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a70023b0 0x5582a70047c0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fefe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff24c0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75aaffd700 0 client.26081 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 --2- 172.21.15.5:0/402553270 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6fef7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.292+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 >> 172.21.15.5:0/402553270 conn(0x5582a6fddda0 msgr2=0x5582a6fe3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.293+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 shutdown_connections 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.293+0000 7f75e5581b80 1 -- 172.21.15.5:0/402553270 wait complete. 2023-07-21T19:11:11.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.293+0000 7f75e5581b80 20 client.26081 trim_cache size 0 max 16384 2023-07-21T19:11:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.330+0000 7f75e5581b80 1 Processor -- start 2023-07-21T19:11:11.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.330+0000 7f75e5581b80 1 -- start start 2023-07-21T19:11:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff2060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6feef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6fe8e30 con 0x5582a6ff2ad0 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6fff9f0 con 0x5582a6fe72d0 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75e5581b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6fe8540 con 0x5582a6fe7be0 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46670/0 (socket says 172.21.15.5:46670) 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75cbfff700 1 -- 172.21.15.5:0/3748179898 learned_addr learned my addr 172.21.15.5:0/3748179898 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75caffd700 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6feef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.331+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 -- 172.21.15.5:0/3748179898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 msgr2=0x5582a6feef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6feef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 -- 172.21.15.5:0/3748179898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 msgr2=0x5582a6ff2060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff2060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 -- 172.21.15.5:0/3748179898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5582a6d8e250 con 0x5582a6ff2ad0 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.332+0000 7f75cbfff700 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009ca0 tx=0x7f75c40079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aad5e386d5482d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.333+0000 7f75b27fc700 1 -- 172.21.15.5:0/3748179898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400e040 con 0x5582a6ff2ad0 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.333+0000 7f75b27fc700 1 -- 172.21.15.5:0/3748179898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75c40077b0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.333+0000 7f75b27fc700 1 -- 172.21.15.5:0/3748179898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c40052f0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 -- 172.21.15.5:0/3748179898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 msgr2=0x5582a6fef4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f75c4009ca0 tx=0x7f75c40079c0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 -- 172.21.15.5:0/3748179898 shutdown_connections 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6ff2ad0 0x5582a6fef4d0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff2060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 --2- 172.21.15.5:0/3748179898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6fe7be0 0x5582a6feef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 -- 172.21.15.5:0/3748179898 >> 172.21.15.5:0/3748179898 conn(0x5582a6fddda0 msgr2=0x5582a6ffe6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 -- 172.21.15.5:0/3748179898 shutdown_connections 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.334+0000 7f75e5581b80 1 -- 172.21.15.5:0/3748179898 wait complete. 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.335+0000 7f75e5581b80 1 -- start start 2023-07-21T19:11:11.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.335+0000 7f75e5581b80 1 Processor -- start 2023-07-21T19:11:11.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff7490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6ff7a70 con 0x5582a6fe7be0 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6ff7be0 con 0x5582a6fe72d0 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5582a6ff7da0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75e5581b80 10 client.0 _ll_register_callbacks cb 0x7f75bc099a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46678/0 (socket says 172.21.15.5:46678) 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.336+0000 7f75caffd700 1 -- 172.21.15.5:0/1266537869 learned_addr learned my addr 172.21.15.5:0/1266537869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 -- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 msgr2=0x5582a6ff8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 -- 172.21.15.5:0/1266537869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 msgr2=0x5582a6ff9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75cbfff700 1 -- 172.21.15.5:0/1266537869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5582a6d67de0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.337+0000 7f75caffd700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:11.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75cbfff700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff7490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b3f0 tx=0x7f75c4003d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=916cb517aa19c001 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75c400e050 con 0x5582a6ff2ad0 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f75c40050b0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f75c4007d90 con 0x5582a6ff2ad0 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75c4005270 con 0x5582a6ff2ad0 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75e5581b80 10 client.26050 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:11.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.338+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5582a6d662b0 con 0x5582a6ff2ad0 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75c4007d90 con 0x5582a6ff2ad0 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f75c4030070 con 0x5582a6ff2ad0 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 10 client.26050 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 10 client.26050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 20 client.26050 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a705f8a0 0x5582a7061cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 4 client.26050 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.339+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5582a6ef5050 con 0x5582a705f8a0 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.340+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a705f8a0 0x5582a7061cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.340+0000 7f75aaffd700 10 client.26050 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.340+0000 7f75cb7fe700 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a705f8a0 0x5582a7061cb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=411304308aa550de server_cookie=6d49086b766d29eb in_seq=0 out_seq=0 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75aaffd700 1 -- 172.21.15.5:0/1266537869 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5582a6ef5050 con 0x5582a705f8a0 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75aaffd700 10 client.26050 handle_command_reply: tid=0 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 client.26050 shutdown 2023-07-21T19:11:11.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 msgr2=0x5582a6ff7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff7490 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f75c400b3f0 tx=0x7f75c4003d70 comp rx=0 tx=0).stop 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 shutdown_connections 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5582a6ff2ad0 0x5582a6ff7490 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5582a6fe72d0 0x5582a6ff8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5582a705f8a0 0x5582a7061cb0 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 --2- 172.21.15.5:0/1266537869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5582a6fe7be0 0x5582a6ff9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75aaffd700 0 client.26050 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 >> 172.21.15.5:0/1266537869 conn(0x5582a6fddda0 msgr2=0x5582a6fe93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 shutdown_connections 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 1 -- 172.21.15.5:0/1266537869 wait complete. 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.341+0000 7f75e5581b80 20 client.26050 trim_cache size 0 max 16384 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:11:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:11:11.356 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:11.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 msgr2=0x7f75b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 0x7f75b40a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f75cc1994d0 tx=0x7f75c0009260 comp rx=0 tx=0).stop 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 msgr2=0x7f75cc197ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00e9a0 tx=0x7f75bc00ed60 comp rx=0 tx=0).stop 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 shutdown_connections 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75cc10a070 0x7f75cc19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f75b40a12c0 0x7f75b40a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f75cc100ca0 0x7f75cc197ec0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 --2- 172.21.15.5:0/150903831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75cc1015b0 0x7f75cc198400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 >> 172.21.15.5:0/150903831 conn(0x7f75cc0fa4b0 msgr2=0x7f75cc10c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 shutdown_connections 2023-07-21T19:11:11.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.342+0000 7f75d1bff700 1 -- 172.21.15.5:0/150903831 wait complete. 2023-07-21T19:11:11.374 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:11 smithi005 ceph-mon[126460]: pgmap v110: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:11:11.376 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:11:11.377 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:11:11.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 -- 172.21.15.5:0/1871989749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9780685b0 msgr2=0x7ff978104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/1871989749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9780685b0 0x7ff978104770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098f0 tx=0x7ff968009c00 comp rx=0 tx=0).stop 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 -- 172.21.15.5:0/1871989749 shutdown_connections 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/1871989749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff978104e50 0x7ff97810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/1871989749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978067c10 0x7ff978067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/1871989749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9780685b0 0x7ff978104770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.447+0000 7ff97f7ff700 1 -- 172.21.15.5:0/1871989749 >> 172.21.15.5:0/1871989749 conn(0x7ff978074760 msgr2=0x7ff978074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.448+0000 7ff97f7ff700 1 -- 172.21.15.5:0/1871989749 shutdown_connections 2023-07-21T19:11:11.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.448+0000 7ff97f7ff700 1 -- 172.21.15.5:0/1871989749 wait complete. 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.449+0000 7ff97f7ff700 1 Processor -- start 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.449+0000 7ff97f7ff700 1 -- start start 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.449+0000 7ff97f7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff978067c10 0x7ff9780fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.449+0000 7ff97f7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 0x7ff9780ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97f7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97f7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a41e0 con 0x7ff978104e50 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97f7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a4320 con 0x7ff9780685b0 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97f7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9781a4490 con 0x7ff978067c10 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 0x7ff9780ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 0x7ff9780ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43210/0 (socket says 172.21.15.5:43210) 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97cd9a700 1 -- 172.21.15.5:0/4139776685 learned_addr learned my addr 172.21.15.5:0/4139776685 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46698/0 (socket says 172.21.15.5:46698) 2023-07-21T19:11:11.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97d59b700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff978067c10 0x7ff9780fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 -- 172.21.15.5:0/4139776685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff978067c10 msgr2=0x7ff9780fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff978067c10 0x7ff9780fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 -- 172.21.15.5:0/4139776685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 msgr2=0x7ff9780ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 0x7ff9780ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.450+0000 7ff97dd9c700 1 -- 172.21.15.5:0/4139776685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9740095f0 con 0x7ff978104e50 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff97dd9c700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff97000cc50 tx=0x7ff97000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff96e7fc700 1 -- 172.21.15.5:0/4139776685 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff97000aa90 con 0x7ff978104e50 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff968009520 con 0x7ff978104e50 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff96e7fc700 1 -- 172.21.15.5:0/4139776685 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff97000abf0 con 0x7ff978104e50 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff96e7fc700 1 -- 172.21.15.5:0/4139776685 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9700115e0 con 0x7ff978104e50 2023-07-21T19:11:11.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.451+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9781a4b40 con 0x7ff978104e50 2023-07-21T19:11:11.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.456+0000 7ff96e7fc700 1 -- 172.21.15.5:0/4139776685 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff970012420 con 0x7ff978104e50 2023-07-21T19:11:11.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.457+0000 7ff9931b3b80 1 Processor -- start 2023-07-21T19:11:11.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.457+0000 7ff9931b3b80 1 -- start start 2023-07-21T19:11:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d683ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d6835040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d673ac50 con 0x5584d6835580 2023-07-21T19:11:11.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d678d9e0 con 0x5584d683b2d0 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6786e70 con 0x5584d66916b0 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46714/0 (socket says 172.21.15.5:46714) 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff97cd9a700 1 -- 172.21.15.5:0/1471114788 learned_addr learned my addr 172.21.15.5:0/1471114788 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d683ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.458+0000 7ff96e7fc700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 0x7ff9640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff96e7fc700 1 -- 172.21.15.5:0/4139776685 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff9700ccd30 con 0x7ff978104e50 2023-07-21T19:11:11.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97d59b700 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d6835040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 -- 172.21.15.5:0/1471114788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 msgr2=0x5584d683ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d683ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 -- 172.21.15.5:0/1471114788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 msgr2=0x5584d6835040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d6835040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 -- 172.21.15.5:0/1471114788 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d65b4d80 con 0x5584d6835580 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97dd9c700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 0x7ff9640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff968005950 tx=0x7ff9680058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fef26ff1c5226e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff9637fe700 1 -- 172.21.15.5:0/1471114788 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800b560 con 0x5584d6835580 2023-07-21T19:11:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff9637fe700 1 -- 172.21.15.5:0/1471114788 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96800bb50 con 0x5584d6835580 2023-07-21T19:11:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.459+0000 7ff9637fe700 1 -- 172.21.15.5:0/1471114788 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968003680 con 0x5584d6835580 2023-07-21T19:11:11.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 -- 172.21.15.5:0/1471114788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 msgr2=0x5584d68359d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff97dd9c700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 0x7ff9640a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff97000cfd0 tx=0x7ff970012ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ff968005950 tx=0x7ff9680058e0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 -- 172.21.15.5:0/1471114788 shutdown_connections 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d68359d0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d683ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/1471114788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d6835040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.460+0000 7ff9931b3b80 1 -- 172.21.15.5:0/1471114788 >> 172.21.15.5:0/1471114788 conn(0x5584d682abe0 msgr2=0x5584d682ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.461+0000 7ff9931b3b80 1 -- 172.21.15.5:0/1471114788 shutdown_connections 2023-07-21T19:11:11.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.461+0000 7ff9931b3b80 1 -- 172.21.15.5:0/1471114788 wait complete. 2023-07-21T19:11:11.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.461+0000 7ff9931b3b80 1 -- start start 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 Processor -- start 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d683e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d683ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6726a60 con 0x5584d6835580 2023-07-21T19:11:11.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6726760 con 0x5584d683b2d0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d660a8c0 con 0x5584d66916b0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.462+0000 7ff9931b3b80 10 client.0 _ll_register_callbacks cb 0x5584d6724700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.463+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d683ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.463+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.463+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41624/0 (socket says 172.21.15.5:41624) 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.463+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3280153357 learned_addr learned my addr 172.21.15.5:0/3280153357 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.463+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d683e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 msgr2=0x5584d683ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d683ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3280153357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 msgr2=0x5584d683e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d683e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3280153357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff97810d210 con 0x5584d66916b0 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff96800bd20 tx=0x7ff968005e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb01c90f7869d9fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff960ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:11.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968005580 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584d66e0490 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.464+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff96800b560 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.465+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff9680056e0 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.465+0000 7ff9931b3b80 10 client.26062 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.465+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5584d659dc00 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.465+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff96803a020 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.465+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff968015020 con 0x5584d66916b0 2023-07-21T19:11:11.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 10 client.26062 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:11.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 10 client.26062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 20 client.26062 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d684e630 0x5584d6850a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 4 client.26062 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584d677fe50 con 0x5584d684e630 2023-07-21T19:11:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.466+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d684e630 0x5584d6850a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.467+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d684e630 0x5584d6850a40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c92ba7f4fd944aa9 server_cookie=cdc7a588256dd4e1 in_seq=0 out_seq=0 2023-07-21T19:11:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.467+0000 7ff960ff9700 10 client.26062 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff960ff9700 1 -- 172.21.15.5:0/3280153357 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5584d677fe50 con 0x5584d684e630 2023-07-21T19:11:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff960ff9700 10 client.26062 handle_command_reply: tid=0 2023-07-21T19:11:11.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 client.26062 shutdown 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 msgr2=0x5584d6841590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff96800bd20 tx=0x7ff968005e50 comp rx=0 tx=0).stop 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 shutdown_connections 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d683b2d0 0x5584d683ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d66916b0 0x5584d6841590 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d684e630 0x5584d6850a40 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3280153357 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d683e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 >> 172.21.15.5:0/3280153357 conn(0x5584d682abe0 msgr2=0x5584d682f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff960ff9700 0 client.26062 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 shutdown_connections 2023-07-21T19:11:11.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.468+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3280153357 wait complete. 2023-07-21T19:11:11.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.469+0000 7ff9931b3b80 20 client.26062 trim_cache size 0 max 16384 2023-07-21T19:11:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 Processor -- start 2023-07-21T19:11:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 -- start start 2023-07-21T19:11:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 0x5584d6835040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 0x5584d6845c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d683a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d677fe50 con 0x5584d66916b0 2023-07-21T19:11:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6830d50 con 0x5584d6835580 2023-07-21T19:11:11.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d684b500 con 0x5584d683b2d0 2023-07-21T19:11:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.487+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 0x5584d6835040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 0x5584d6835040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46734/0 (socket says 172.21.15.5:46734) 2023-07-21T19:11:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3980245642 learned_addr learned my addr 172.21.15.5:0/3980245642 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 0x5584d6845c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d683a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 -- 172.21.15.5:0/3980245642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 msgr2=0x5584d683a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d683a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 -- 172.21.15.5:0/3980245642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 msgr2=0x5584d6835040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 0x5584d6835040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 -- 172.21.15.5:0/3980245642 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584d6835c20 con 0x5584d6835580 2023-07-21T19:11:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d683a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:11.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.488+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 0x5584d6845c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5584d6851350 tx=0x7ff974009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b0e71f1167c9640 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.489+0000 7ff963fff700 1 -- 172.21.15.5:0/3980245642 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974017040 con 0x5584d6835580 2023-07-21T19:11:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.489+0000 7ff963fff700 1 -- 172.21.15.5:0/3980245642 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff974018040 con 0x5584d6835580 2023-07-21T19:11:11.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.489+0000 7ff963fff700 1 -- 172.21.15.5:0/3980245642 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff974004420 con 0x5584d6835580 2023-07-21T19:11:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.489+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3980245642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 msgr2=0x5584d6845c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 0x5584d6845c60 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x5584d6851350 tx=0x7ff974009310 comp rx=0 tx=0).stop 2023-07-21T19:11:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3980245642 shutdown_connections 2023-07-21T19:11:11.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d683a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d66916b0 0x5584d6835040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3980245642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d6835580 0x5584d6845c60 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3980245642 >> 172.21.15.5:0/3980245642 conn(0x5584d682abe0 msgr2=0x5584d68399f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3980245642 shutdown_connections 2023-07-21T19:11:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3980245642 wait complete. 2023-07-21T19:11:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.490+0000 7ff9931b3b80 1 -- start start 2023-07-21T19:11:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 Processor -- start 2023-07-21T19:11:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d66916b0 0x5584d6837880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d6837dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6838350 con 0x5584d6835580 2023-07-21T19:11:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d68384c0 con 0x5584d66916b0 2023-07-21T19:11:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584d6836cd0 con 0x5584d683b2d0 2023-07-21T19:11:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.491+0000 7ff9931b3b80 10 client.0 _ll_register_callbacks cb 0x7ff9680051d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41646/0 (socket says 172.21.15.5:41646) 2023-07-21T19:11:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3257644761 learned_addr learned my addr 172.21.15.5:0/3257644761 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d66916b0 0x5584d6837880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d6837dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d66916b0 msgr2=0x5584d6837880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d66916b0 0x5584d6837880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3257644761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 msgr2=0x5584d6837dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d6837dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 -- 172.21.15.5:0/3257644761 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff974009d50 con 0x5584d683b2d0 2023-07-21T19:11:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff97cd9a700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098c0 tx=0x7ff968009fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=904cc3037dfd8912 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.492+0000 7ff960ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968005b00 con 0x5584d683b2d0 2023-07-21T19:11:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff968000e40 con 0x5584d683b2d0 2023-07-21T19:11:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff96800bb50 con 0x5584d683b2d0 2023-07-21T19:11:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff968005580 con 0x5584d683b2d0 2023-07-21T19:11:11.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 10 client.26068 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5584d65db1f0 con 0x5584d683b2d0 2023-07-21T19:11:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff968045080 con 0x5584d683b2d0 2023-07-21T19:11:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff96803d040 con 0x5584d683b2d0 2023-07-21T19:11:11.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 10 client.26068 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 10 client.26068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.493+0000 7ff9931b3b80 20 client.26068 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.494+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d68ab440 0x5584d68ad850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.494+0000 7ff9931b3b80 4 client.26068 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:11:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.494+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584d6741ff0 con 0x5584d68ab440 2023-07-21T19:11:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.494+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d68ab440 0x5584d68ad850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.495+0000 7ff97d59b700 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d68ab440 0x5584d68ad850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da99c5b9e0b0925 server_cookie=24be15fee434400d in_seq=0 out_seq=0 2023-07-21T19:11:11.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.495+0000 7ff960ff9700 10 client.26068 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.509+0000 7ff960ff9700 1 -- 172.21.15.5:0/3257644761 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584d6741ff0 con 0x5584d68ab440 2023-07-21T19:11:11.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.509+0000 7ff960ff9700 10 client.26068 handle_command_reply: tid=0 2023-07-21T19:11:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.509+0000 7ff9931b3b80 1 client.26068 shutdown 2023-07-21T19:11:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 msgr2=0x5584d6836790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098c0 tx=0x7ff968009fd0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 shutdown_connections 2023-07-21T19:11:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584d683b2d0 0x5584d6836790 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584d66916b0 0x5584d6837880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584d68ab440 0x5584d68ad850 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 --2- 172.21.15.5:0/3257644761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584d6835580 0x5584d6837dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 >> 172.21.15.5:0/3257644761 conn(0x5584d682abe0 msgr2=0x5584d683a280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff960ff9700 0 client.26068 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 shutdown_connections 2023-07-21T19:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 1 -- 172.21.15.5:0/3257644761 wait complete. 2023-07-21T19:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.510+0000 7ff9931b3b80 20 client.26068 trim_cache size 0 max 16384 2023-07-21T19:11:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:11:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:11:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:11:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:11:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 msgr2=0x7ff9640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 0x7ff9640a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff97000cfd0 tx=0x7ff970012ad0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 msgr2=0x7ff9781a3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ff97000cc50 tx=0x7ff97000cf60 comp rx=0 tx=0).stop 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 shutdown_connections 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9640a12c0 0x7ff9640a3770 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff978104e50 0x7ff9781a3ca0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff978067c10 0x7ff9780fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 --2- 172.21.15.5:0/4139776685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9780685b0 0x7ff9780ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 >> 172.21.15.5:0/4139776685 conn(0x7ff978074760 msgr2=0x7ff9780fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 shutdown_connections 2023-07-21T19:11:11.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:11.511+0000 7ff97f7ff700 1 -- 172.21.15.5:0/4139776685 wait complete. 2023-07-21T19:11:11.552 INFO:teuthology.orchestra.run.smithi005.stderr:11700000 of 50000000 tuples (23%) done (elapsed 189.11 s, remaining 619.05 s) 2023-07-21T19:11:11.589 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:11 smithi101 ceph-mon[128136]: pgmap v110: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:11:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/1857376904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc103910 msgr2=0x7fe2bc103ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/1857376904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc103910 0x7fe2bc103ce0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fe2b00099c0 tx=0x7fe2b0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/1857376904 shutdown_connections 2023-07-21T19:11:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/1857376904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc104f80 0x7fe2bc10b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/1857376904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc103910 0x7fe2bc103ce0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/1857376904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 0x7fe2bc104a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/1857376904 >> 172.21.15.5:0/1857376904 conn(0x7fe2bc074a30 msgr2=0x7fe2bc074e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/1857376904 shutdown_connections 2023-07-21T19:11:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.626+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/1857376904 wait complete. 2023-07-21T19:11:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.627+0000 7fe2c1d9d700 1 Processor -- start 2023-07-21T19:11:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 -- start start 2023-07-21T19:11:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 0x7fe2bc19de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 0x7fe2bc197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc197ff0 con 0x7fe2bc104f80 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc198160 con 0x7fe2bc103910 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.628+0000 7fe2c1d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2bc198320 con 0x7fe2bc1045d0 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 0x7fe2bc197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43240/0 (socket says 172.21.15.5:43240) 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 -- 172.21.15.5:0/813780615 learned_addr learned my addr 172.21.15.5:0/813780615 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2bbfff700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 0x7fe2bc19de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 -- 172.21.15.5:0/813780615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 msgr2=0x7fe2bc19de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 0x7fe2bc19de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 -- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 msgr2=0x7fe2bc197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 0x7fe2bc197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 -- 172.21.15.5:0/813780615 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b00095f0 con 0x7fe2bc103910 2023-07-21T19:11:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c159c700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 0x7fe2bc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.629+0000 7fe2c0d9b700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0005910 tx=0x7fe2b000f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.630+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b001d070 con 0x7fe2bc103910 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.630+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2bc1985a0 con 0x7fe2bc103910 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.630+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe2b0005050 con 0x7fe2bc103910 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.630+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b0004a00 con 0x7fe2bc103910 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.630+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2bc1a4900 con 0x7fe2bc103910 2023-07-21T19:11:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.631+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2bc067220 con 0x7fe2bc103910 2023-07-21T19:11:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.633+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe2b00051c0 con 0x7fe2bc103910 2023-07-21T19:11:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.635+0000 7fe2b9ffb700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 0x7fe2ac0a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.635+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fe2b00cdad0 con 0x7fe2bc103910 2023-07-21T19:11:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.636+0000 7fe2bbfff700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 0x7fe2ac0a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.636+0000 7fe2bbfff700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 0x7fe2ac0a3860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc075290 tx=0x7fe2a8009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.637+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fe2b0096f40 con 0x7fe2bc103910 2023-07-21T19:11:11.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:11 smithi156 ceph-mon[128316]: pgmap v110: 97 pgs: 97 active+clean; 2.6 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:11:11.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.934+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2bc198e90 con 0x7fe2bc103910 2023-07-21T19:11:11.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.935+0000 7fe2b9ffb700 1 -- 172.21.15.5:0/813780615 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10839 (secure 0 0 0) 0x7fe2b009a560 con 0x7fe2bc103910 2023-07-21T19:11:11.944 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:11:11.945 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:09:18.477788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:11:11.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 msgr2=0x7fe2ac0a3860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 0x7fe2ac0a3860 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc075290 tx=0x7fe2a8009260 comp rx=0 tx=0).stop 2023-07-21T19:11:11.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 msgr2=0x7fe2bc19d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0005910 tx=0x7fe2b000f690 comp rx=0 tx=0).stop 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 shutdown_connections 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe2ac0a13b0 0x7fe2ac0a3860 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe2bc104f80 0x7fe2bc197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe2bc103910 0x7fe2bc19d910 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 --2- 172.21.15.5:0/813780615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe2bc1045d0 0x7fe2bc19de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:11.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 >> 172.21.15.5:0/813780615 conn(0x7fe2bc074a30 msgr2=0x7fe2bc109bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:11.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 shutdown_connections 2023-07-21T19:11:11.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:11.938+0000 7fe2c1d9d700 1 -- 172.21.15.5:0/813780615 wait complete. 2023-07-21T19:11:11.949 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2023-07-21T19:11:11.968 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:11:12.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.196+0000 7f20ba9ff700 1 -- 172.21.15.5:0/2295423931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b40ffae0 msgr2=0x7f20b40fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.196+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/2295423931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b40ffae0 0x7f20b40fff50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f20a8009990 tx=0x7f20a8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 -- 172.21.15.5:0/2295423931 shutdown_connections 2023-07-21T19:11:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/2295423931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b4100490 0x7f20b410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/2295423931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/2295423931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b40ffae0 0x7f20b40fff50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 -- 172.21.15.5:0/2295423931 >> 172.21.15.5:0/2295423931 conn(0x7f20b40fb5e0 msgr2=0x7f20b40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 -- 172.21.15.5:0/2295423931 shutdown_connections 2023-07-21T19:11:12.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.197+0000 7f20ba9ff700 1 -- 172.21.15.5:0/2295423931 wait complete. 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 Processor -- start 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 -- start start 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 0x7f20b4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b419cd30 con 0x7f20b4100490 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b419cea0 con 0x7f20b4105af0 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.198+0000 7f20ba9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20b419d060 con 0x7f20b40ffae0 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46792/0 (socket says 172.21.15.5:46792) 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 -- 172.21.15.5:0/3105764041 learned_addr learned my addr 172.21.15.5:0/3105764041 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b3fff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 0x7f20b4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 -- 172.21.15.5:0/3105764041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 msgr2=0x7f20b4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 0x7f20b4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 -- 172.21.15.5:0/3105764041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 msgr2=0x7f20b419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b37fe700 1 -- 172.21.15.5:0/3105764041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20a80095f0 con 0x7f20b4100490 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.199+0000 7f20b3fff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 0x7f20b4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f20a8005770 tx=0x7f20a80057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20b17fa700 1 -- 172.21.15.5:0/3105764041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a801d070 con 0x7f20b4100490 2023-07-21T19:11:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20b17fa700 1 -- 172.21.15.5:0/3105764041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20a8021410 con 0x7f20b4100490 2023-07-21T19:11:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20b17fa700 1 -- 172.21.15.5:0/3105764041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a8017650 con 0x7f20b4100490 2023-07-21T19:11:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20b419d2e0 con 0x7f20b4100490 2023-07-21T19:11:12.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.200+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20b419d560 con 0x7f20b4100490 2023-07-21T19:11:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.205+0000 7f20b17fa700 1 -- 172.21.15.5:0/3105764041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f20a80217f0 con 0x7f20b4100490 2023-07-21T19:11:12.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.206+0000 7f20ce3c7b80 1 Processor -- start 2023-07-21T19:11:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.206+0000 7f20ce3c7b80 1 -- start start 2023-07-21T19:11:12.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c186a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c23590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c25eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8b1ecb0 con 0x563bc8c23ad0 2023-07-21T19:11:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8b71a40 con 0x563bc8c182d0 2023-07-21T19:11:12.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8b6aed0 con 0x563bc8c18be0 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c25eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c25eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46808/0 (socket says 172.21.15.5:46808) 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20b37fe700 1 -- 172.21.15.5:0/1299445558 learned_addr learned my addr 172.21.15.5:0/1299445558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20b3fff700 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c23590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.207+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c186a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b17fa700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 -- 172.21.15.5:0/1299445558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 msgr2=0x563bc8c186a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c186a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 -- 172.21.15.5:0/1299445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 msgr2=0x563bc8c25eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c25eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 -- 172.21.15.5:0/1299445558 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc8998de0 con 0x563bc8c18be0 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b17fa700 1 -- 172.21.15.5:0/3105764041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f20a80cdec0 con 0x7f20b4100490 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.208+0000 7f20b3fff700 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c23590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20a4011440 tx=0x7f20a4011750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44accd95c7283ac1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.209+0000 7f209a7fc700 1 -- 172.21.15.5:0/1299445558 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004230 con 0x563bc8c18be0 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.209+0000 7f209a7fc700 1 -- 172.21.15.5:0/1299445558 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20a4004cc0 con 0x563bc8c18be0 2023-07-21T19:11:12.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.209+0000 7f209a7fc700 1 -- 172.21.15.5:0/1299445558 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a40055e0 con 0x563bc8c18be0 2023-07-21T19:11:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/1299445558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 msgr2=0x563bc8c23590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 0x7f209c0a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009110 tx=0x7f20ac004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c23590 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f20a4011440 tx=0x7f20a4011750 comp rx=0 tx=0).stop 2023-07-21T19:11:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/1299445558 shutdown_connections 2023-07-21T19:11:12.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c25eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c186a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/1299445558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c23590 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/1299445558 >> 172.21.15.5:0/1299445558 conn(0x563bc8c0eda0 msgr2=0x563bc8c146a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/1299445558 shutdown_connections 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.210+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/1299445558 wait complete. 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.211+0000 7f20ce3c7b80 1 -- start start 2023-07-21T19:11:12.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.211+0000 7f20ce3c7b80 1 Processor -- start 2023-07-21T19:11:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c20760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8b0aac0 con 0x563bc8c23ad0 2023-07-21T19:11:12.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8b0a7c0 con 0x563bc8c18be0 2023-07-21T19:11:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc89ee920 con 0x563bc8c182d0 2023-07-21T19:11:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20ce3c7b80 10 client.0 _ll_register_callbacks cb 0x563bc8b08760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b8f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41688/0 (socket says 172.21.15.5:41688) 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 -- 172.21.15.5:0/3660655504 learned_addr learned my addr 172.21.15.5:0/3660655504 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c20760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 -- 172.21.15.5:0/3660655504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 msgr2=0x563bc8c20760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c20760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 -- 172.21.15.5:0/3660655504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 msgr2=0x563bc8c20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c20e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.212+0000 7f20b3fff700 1 -- 172.21.15.5:0/3660655504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a4009c90 con 0x563bc8c182d0 2023-07-21T19:11:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.213+0000 7f20b3fff700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009790 tx=0x7f20a40053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=80aa1bcc3d67bc7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.213+0000 7f20937fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.216+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4005000 con 0x563bc8c182d0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.216+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f20a4005ce0 con 0x563bc8c182d0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.216+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563bc8ac44f0 con 0x563bc8c182d0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a4022470 con 0x563bc8c182d0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 10 client.26086 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x563bc8981c60 con 0x563bc8c182d0 2023-07-21T19:11:12.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563bc8ac44f0 con 0x563bc8c182d0 2023-07-21T19:11:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f20a4020070 con 0x563bc8c182d0 2023-07-21T19:11:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 10 client.26086 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 10 client.26086 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:11:12.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 20 client.26086 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c332f0 0x563bc8c35700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 4 client.26086 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.217+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563bc8b63eb0 con 0x563bc8c332f0 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.218+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c332f0 0x563bc8c35700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.219+0000 7f20937fe700 10 client.26086 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.219+0000 7f20b37fe700 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c332f0 0x563bc8c35700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=aefd0fb49432a85c server_cookie=9aaf93488f2116ab in_seq=0 out_seq=0 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20937fe700 1 -- 172.21.15.5:0/3660655504 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563bc8b63eb0 con 0x563bc8c332f0 2023-07-21T19:11:12.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20937fe700 10 client.26086 handle_command_reply: tid=0 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 client.26086 shutdown 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 msgr2=0x563bc8c23550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009790 tx=0x7f20a40053f0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 shutdown_connections 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c182d0 0x563bc8c23550 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c332f0 0x563bc8c35700 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/3660655504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c20760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20937fe700 0 client.26086 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:12.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 >> 172.21.15.5:0/3660655504 conn(0x563bc8c0eda0 msgr2=0x563bc8c14290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 shutdown_connections 2023-07-21T19:11:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.220+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/3660655504 wait complete. 2023-07-21T19:11:12.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.221+0000 7f20ce3c7b80 20 client.26086 trim_cache size 0 max 16384 2023-07-21T19:11:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 Processor -- start 2023-07-21T19:11:12.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 -- start start 2023-07-21T19:11:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c12e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c2a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c13350 con 0x563bc8c23ad0 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c2aa20 con 0x563bc8c182d0 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.235+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c1b1b0 con 0x563bc8c18be0 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c2a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46828/0 (socket says 172.21.15.5:46828) 2023-07-21T19:11:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 -- 172.21.15.5:0/2600306164 learned_addr learned my addr 172.21.15.5:0/2600306164 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b3fff700 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c12e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 -- 172.21.15.5:0/2600306164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 msgr2=0x563bc8c2a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c2a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 -- 172.21.15.5:0/2600306164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 msgr2=0x563bc8c12e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c12e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 -- 172.21.15.5:0/2600306164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563bc89bf250 con 0x563bc8c23ad0 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b3fff700 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.236+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007500 tx=0x7f20ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2028f57a4175078d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.237+0000 7f209affd700 1 -- 172.21.15.5:0/2600306164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac007dd0 con 0x563bc8c23ad0 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.237+0000 7f209affd700 1 -- 172.21.15.5:0/2600306164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20ac00f040 con 0x563bc8c23ad0 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.237+0000 7f209affd700 1 -- 172.21.15.5:0/2600306164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20ac017660 con 0x563bc8c23ad0 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2600306164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 msgr2=0x563bc8c23080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f20ac007500 tx=0x7f20ac017000 comp rx=0 tx=0).stop 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2600306164 shutdown_connections 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c23ad0 0x563bc8c23080 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c182d0 0x563bc8c12e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2600306164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c18be0 0x563bc8c2a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2600306164 >> 172.21.15.5:0/2600306164 conn(0x563bc8c0eda0 msgr2=0x563bc8c3a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2600306164 shutdown_connections 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2600306164 wait complete. 2023-07-21T19:11:12.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.238+0000 7f20ce3c7b80 1 -- start start 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 Processor -- start 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c1e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c23ad0 0x563bc8c1cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c1d2c0 con 0x563bc8c182d0 2023-07-21T19:11:12.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c1d430 con 0x563bc8c18be0 2023-07-21T19:11:12.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563bc8c1d5f0 con 0x563bc8c23ad0 2023-07-21T19:11:12.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20ce3c7b80 10 client.0 _ll_register_callbacks cb 0x7f20a8097330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46830/0 (socket says 172.21.15.5:46830) 2023-07-21T19:11:12.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20b3fff700 1 -- 172.21.15.5:0/2789361558 learned_addr learned my addr 172.21.15.5:0/2789361558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.239+0000 7f20b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c1e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b8f9c700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c23ad0 0x563bc8c1cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 -- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c23ad0 msgr2=0x563bc8c1cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c23ad0 0x563bc8c1cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 -- 172.21.15.5:0/2789361558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 msgr2=0x563bc8c1e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c1e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 -- 172.21.15.5:0/2789361558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563bc8981c60 con 0x563bc8c182d0 2023-07-21T19:11:12.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b37fe700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20b3fff700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e7f0 tx=0x7f20a4009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59fd9cee4a9ecb59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20937fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20a4005ce0 con 0x563bc8c182d0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f20a4004cc0 con 0x563bc8c182d0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.240+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f20a4000e40 con 0x563bc8c182d0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f20a400ea60 con 0x563bc8c182d0 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 10 client.16410 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f20ac004720 con 0x563bc8c182d0 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20a4004cc0 con 0x563bc8c182d0 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f20a4020070 con 0x563bc8c182d0 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 10 client.16410 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 10 client.16410 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 20 client.16410 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c87ec0 0x563bc8c8a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 4 client.16410 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.241+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563bc8b26050 con 0x563bc8c87ec0 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.242+0000 7f20b37fe700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c87ec0 0x563bc8c8a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.242+0000 7f20937fe700 10 client.16410 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.242+0000 7f20b37fe700 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c87ec0 0x563bc8c8a2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=201df8619ec454ed server_cookie=960091c826c5e138 in_seq=0 out_seq=0 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20937fe700 1 -- 172.21.15.5:0/2789361558 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563bc8b26050 con 0x563bc8c87ec0 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20937fe700 10 client.16410 handle_command_reply: tid=0 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 client.16410 shutdown 2023-07-21T19:11:12.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 msgr2=0x563bc8c1def0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f20a400e7f0 tx=0x7f20a4009fd0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 shutdown_connections 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563bc8c23ad0 0x563bc8c1cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563bc8c182d0 0x563bc8c1def0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563bc8c87ec0 0x563bc8c8a2a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 --2- 172.21.15.5:0/2789361558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563bc8c18be0 0x563bc8c1e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20937fe700 0 client.16410 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:12.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 >> 172.21.15.5:0/2789361558 conn(0x563bc8c0eda0 msgr2=0x563bc8c1ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 shutdown_connections 2023-07-21T19:11:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 1 -- 172.21.15.5:0/2789361558 wait complete. 2023-07-21T19:11:12.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.243+0000 7f20ce3c7b80 20 client.16410 trim_cache size 0 max 16384 2023-07-21T19:11:12.259 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 msgr2=0x7f209c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 0x7f209c0a3770 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f20ac009110 tx=0x7f20ac004ad0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 msgr2=0x7f20b4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f20a8005770 tx=0x7f20a80057a0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 shutdown_connections 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20b4105af0 0x7f20b419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20b40ffae0 0x7f20b4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f209c0a12c0 0x7f209c0a3770 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 --2- 172.21.15.5:0/3105764041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20b4100490 0x7f20b4198290 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 >> 172.21.15.5:0/3105764041 conn(0x7f20b40fb5e0 msgr2=0x7f20b40fbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 shutdown_connections 2023-07-21T19:11:12.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.244+0000 7f20ba9ff700 1 -- 172.21.15.5:0/3105764041 wait complete. 2023-07-21T19:11:12.271 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:11:12.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 -- 172.21.15.5:0/2509985505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07380ffae0 msgr2=0x7f07380fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 --2- 172.21.15.5:0/2509985505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07380ffae0 0x7f07380fff50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f072c0098f0 tx=0x7f072c009c00 comp rx=0 tx=0).stop 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 -- 172.21.15.5:0/2509985505 shutdown_connections 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 --2- 172.21.15.5:0/2509985505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0738100490 0x7f073810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 --2- 172.21.15.5:0/2509985505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738105af0 0x7f0738105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 --2- 172.21.15.5:0/2509985505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f07380ffae0 0x7f07380fff50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.497+0000 7f073c9ff700 1 -- 172.21.15.5:0/2509985505 >> 172.21.15.5:0/2509985505 conn(0x7f07380fb5e0 msgr2=0x7f07380fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.498+0000 7f073c9ff700 1 -- 172.21.15.5:0/2509985505 shutdown_connections 2023-07-21T19:11:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.498+0000 7f073c9ff700 1 -- 172.21.15.5:0/2509985505 wait complete. 2023-07-21T19:11:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.499+0000 7f073c9ff700 1 Processor -- start 2023-07-21T19:11:12.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.499+0000 7f073c9ff700 1 -- start start 2023-07-21T19:11:12.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 0x7f0738198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0738105af0 0x7f073819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073819cdf0 con 0x7f07380ffae0 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073819cf60 con 0x7f0738100490 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073c9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073819d120 con 0x7f0738105af0 2023-07-21T19:11:12.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46842/0 (socket says 172.21.15.5:46842) 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f073659c700 1 -- 172.21.15.5:0/4125640205 learned_addr learned my addr 172.21.15.5:0/4125640205 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.500+0000 7f0736d9d700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0738105af0 0x7f073819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f0735d9b700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 0x7f0738198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 -- 172.21.15.5:0/4125640205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0738105af0 msgr2=0x7f073819c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0738105af0 0x7f073819c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 -- 172.21.15.5:0/4125640205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 msgr2=0x7f0738198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 0x7f0738198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 -- 172.21.15.5:0/4125640205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0730009d00 con 0x7f07380ffae0 2023-07-21T19:11:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f0735d9b700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 0x7f0738198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073659c700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9a0 tx=0x7f072800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f07277fe700 1 -- 172.21.15.5:0/4125640205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728009790 con 0x7f07380ffae0 2023-07-21T19:11:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f07277fe700 1 -- 172.21.15.5:0/4125640205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f072800cbd0 con 0x7f07380ffae0 2023-07-21T19:11:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c009520 con 0x7f07380ffae0 2023-07-21T19:11:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.501+0000 7f07277fe700 1 -- 172.21.15.5:0/4125640205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728018680 con 0x7f07380ffae0 2023-07-21T19:11:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.502+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073819d5f0 con 0x7f07380ffae0 2023-07-21T19:11:12.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.505+0000 7f07277fe700 1 -- 172.21.15.5:0/4125640205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f072800cd40 con 0x7f07380ffae0 2023-07-21T19:11:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.506+0000 7f07502b8b80 1 Processor -- start 2023-07-21T19:11:12.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 -- start start 2023-07-21T19:11:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07277fe700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 0x7f07200a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7faeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea6f3cb0 con 0x555eea7f8ad0 2023-07-21T19:11:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea746a40 con 0x555eea7edbe0 2023-07-21T19:11:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07502b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea73fed0 con 0x555eea7ed2d0 2023-07-21T19:11:12.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.507+0000 7f07277fe700 1 -- 172.21.15.5:0/4125640205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0728014070 con 0x7f07380ffae0 2023-07-21T19:11:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7faeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41752/0 (socket says 172.21.15.5:41752) 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 -- 172.21.15.5:0/766177843 learned_addr learned my addr 172.21.15.5:0/766177843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f0735d9b700 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f0736d9d700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 0x7f07200a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 -- 172.21.15.5:0/766177843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 msgr2=0x555eea7f8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 -- 172.21.15.5:0/766177843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 msgr2=0x555eea7faeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7faeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 -- 172.21.15.5:0/766177843 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eea56dde0 con 0x555eea7ed2d0 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f0736d9d700 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.508+0000 7f073659c700 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07280969b0 tx=0x7f0728099920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e2de686563b9970a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.509+0000 7f0724ff9700 1 -- 172.21.15.5:0/766177843 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728099f40 con 0x555eea7ed2d0 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.509+0000 7f0724ff9700 1 -- 172.21.15.5:0/766177843 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f072809df70 con 0x555eea7ed2d0 2023-07-21T19:11:12.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.509+0000 7f0724ff9700 1 -- 172.21.15.5:0/766177843 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072809a1a0 con 0x555eea7ed2d0 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f0736d9d700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 0x7f07200a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0730009610 tx=0x7f073000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- 172.21.15.5:0/766177843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 msgr2=0x555eea7ed6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f07280969b0 tx=0x7f0728099920 comp rx=0 tx=0).stop 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- 172.21.15.5:0/766177843 shutdown_connections 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7faeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ed6a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 --2- 172.21.15.5:0/766177843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- 172.21.15.5:0/766177843 >> 172.21.15.5:0/766177843 conn(0x555eea7e3da0 msgr2=0x555eea7e96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- 172.21.15.5:0/766177843 shutdown_connections 2023-07-21T19:11:12.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- 172.21.15.5:0/766177843 wait complete. 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.510+0000 7f07502b8b80 1 -- start start 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 Processor -- start 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea6dfac0 con 0x555eea7f8ad0 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea6df7c0 con 0x555eea7edbe0 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea5c3920 con 0x555eea7ed2d0 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f07502b8b80 10 client.0 _ll_register_callbacks cb 0x555eea6dd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.511+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46872/0 (socket says 172.21.15.5:46872) 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 -- 172.21.15.5:0/2878135051 learned_addr learned my addr 172.21.15.5:0/2878135051 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f073659c700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0735d9b700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 -- 172.21.15.5:0/2878135051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 msgr2=0x555eea7f8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 -- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 msgr2=0x555eea7f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 -- 172.21.15.5:0/2878135051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073819d5f0 con 0x555eea7f8ad0 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f073659c700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0735d9b700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f0736d9d700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0730019b30 tx=0x7f0730004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb7aeb54185843 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.512+0000 7f071dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0730003c10 con 0x555eea7f8ad0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555eea6994f0 con 0x555eea7f8ad0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0730003d70 con 0x555eea7f8ad0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0730018840 con 0x555eea7f8ad0 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f07502b8b80 10 client.16422 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.513+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x555eea556c60 con 0x555eea7f8ad0 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0730032020 con 0x555eea7f8ad0 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0730004480 con 0x555eea7f8ad0 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 10 client.16422 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 10 client.16422 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 20 client.16422 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea8084d0 0x555eea80a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 4 client.16422 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.514+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555eea738eb0 con 0x555eea8084d0 2023-07-21T19:11:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.515+0000 7f073659c700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea8084d0 0x555eea80a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.515+0000 7f071dffb700 10 client.16422 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:12.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.515+0000 7f073659c700 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea8084d0 0x555eea80a8e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ecd811368747a255 server_cookie=80e681cdb70ffdaa in_seq=0 out_seq=0 2023-07-21T19:11:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f071dffb700 1 -- 172.21.15.5:0/2878135051 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555eea738eb0 con 0x555eea8084d0 2023-07-21T19:11:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f071dffb700 10 client.16422 handle_command_reply: tid=0 2023-07-21T19:11:12.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 client.16422 shutdown 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 msgr2=0x555eea7f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f0730019b30 tx=0x7f0730004b40 comp rx=0 tx=0).stop 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 shutdown_connections 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7f8ad0 0x555eea7f5f50 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea8084d0 0x555eea80a8e0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2878135051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7edbe0 0x555eea7f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f071dffb700 0 client.16422 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.516+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 >> 172.21.15.5:0/2878135051 conn(0x555eea7e3da0 msgr2=0x555eea7e9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.517+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 shutdown_connections 2023-07-21T19:11:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.517+0000 7f07502b8b80 1 -- 172.21.15.5:0/2878135051 wait complete. 2023-07-21T19:11:12.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.517+0000 7f07502b8b80 20 client.16422 trim_cache size 0 max 16384 2023-07-21T19:11:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 Processor -- start 2023-07-21T19:11:12.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 -- start start 2023-07-21T19:11:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7f4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7f0660 con 0x555eea7edbe0 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7ef1a0 con 0x555eea7f8ad0 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.534+0000 7f07502b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7ea4d0 con 0x555eea7ed2d0 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46876/0 (socket says 172.21.15.5:46876) 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 -- 172.21.15.5:0/2050564047 learned_addr learned my addr 172.21.15.5:0/2050564047 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f073659c700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0736d9d700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7f4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 -- 172.21.15.5:0/2050564047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 msgr2=0x555eea7f0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 -- 172.21.15.5:0/2050564047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 msgr2=0x555eea7f4ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7f4ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f0735d9b700 1 -- 172.21.15.5:0/2050564047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555eea594250 con 0x555eea7edbe0 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.535+0000 7f073659c700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.536+0000 7f0735d9b700 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f072c0098f0 tx=0x7f072c00bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a03bf066fd6ee4f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.536+0000 7f07257fa700 1 -- 172.21.15.5:0/2050564047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c01c070 con 0x555eea7edbe0 2023-07-21T19:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.536+0000 7f07257fa700 1 -- 172.21.15.5:0/2050564047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f072c01d040 con 0x555eea7edbe0 2023-07-21T19:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.536+0000 7f07257fa700 1 -- 172.21.15.5:0/2050564047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c0052a0 con 0x555eea7edbe0 2023-07-21T19:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- 172.21.15.5:0/2050564047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 msgr2=0x555eea7f1070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f072c0098f0 tx=0x7f072c00bf70 comp rx=0 tx=0).stop 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- 172.21.15.5:0/2050564047 shutdown_connections 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7f4ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7f0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 --2- 172.21.15.5:0/2050564047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7f1070 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- 172.21.15.5:0/2050564047 >> 172.21.15.5:0/2050564047 conn(0x555eea7e3da0 msgr2=0x555eea8047c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- 172.21.15.5:0/2050564047 shutdown_connections 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- 172.21.15.5:0/2050564047 wait complete. 2023-07-21T19:11:12.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.537+0000 7f07502b8b80 1 -- start start 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 Processor -- start 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ff340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7fdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7fe0b0 con 0x555eea7edbe0 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7fe220 con 0x555eea7f8ad0 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.538+0000 7f07502b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555eea7fe3e0 con 0x555eea7ed2d0 2023-07-21T19:11:12.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f07502b8b80 10 client.0 _ll_register_callbacks cb 0x7f0730019450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46888/0 (socket says 172.21.15.5:46888) 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 -- 172.21.15.5:0/318193278 learned_addr learned my addr 172.21.15.5:0/318193278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f073659c700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ff340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0736d9d700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7fdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 -- 172.21.15.5:0/318193278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 msgr2=0x555eea7ff340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ff340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 -- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 msgr2=0x555eea7fdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7fdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.539+0000 7f0735d9b700 1 -- 172.21.15.5:0/318193278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555eea56dde0 con 0x555eea7edbe0 2023-07-21T19:11:12.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f0735d9b700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f072c009c40 tx=0x7f072c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf831f70750079a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f071dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f072c01c040 con 0x555eea7edbe0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f072809df70 con 0x555eea7edbe0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f072c005240 con 0x555eea7edbe0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f072c005400 con 0x555eea7edbe0 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f07502b8b80 10 client.16428 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f073810d220 con 0x555eea7edbe0 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.540+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f072809df70 con 0x555eea7edbe0 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f072c015020 con 0x555eea7edbe0 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 10 client.16428 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 10 client.16428 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 20 client.16428 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea85d4b0 0x555eea85f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 4 client.16428 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:12.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x555eea6fb050 con 0x555eea85d4b0 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.541+0000 7f073659c700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea85d4b0 0x555eea85f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f071dffb700 10 client.16428 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f073659c700 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea85d4b0 0x555eea85f8c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=56faca627a1047ca server_cookie=5d87d9208d620d14 in_seq=0 out_seq=0 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f071dffb700 1 -- 172.21.15.5:0/318193278 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x555eea6fb050 con 0x555eea85d4b0 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f071dffb700 10 client.16428 handle_command_reply: tid=0 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f07502b8b80 1 client.16428 shutdown 2023-07-21T19:11:12.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 msgr2=0x555eea7fd630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.542+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f072c009c40 tx=0x7f072c00b810 comp rx=0 tx=0).stop 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 shutdown_connections 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x555eea85d4b0 0x555eea85f8c0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555eea7f8ad0 0x555eea7fdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555eea7ed2d0 0x555eea7ff340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 --2- 172.21.15.5:0/318193278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555eea7edbe0 0x555eea7fd630 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f071dffb700 0 client.16428 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:12.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 >> 172.21.15.5:0/318193278 conn(0x555eea7e3da0 msgr2=0x555eea7f0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 shutdown_connections 2023-07-21T19:11:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 1 -- 172.21.15.5:0/318193278 wait complete. 2023-07-21T19:11:12.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.543+0000 7f07502b8b80 20 client.16428 trim_cache size 0 max 16384 2023-07-21T19:11:12.558 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 msgr2=0x7f07200a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 0x7f07200a37f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f0730009610 tx=0x7f073000b040 comp rx=0 tx=0).stop 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 msgr2=0x7f0738197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f072800e9a0 tx=0x7f072800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 shutdown_connections 2023-07-21T19:11:12.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0738105af0 0x7f073819c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f07200a1340 0x7f07200a37f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f07380ffae0 0x7f0738197e10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 --2- 172.21.15.5:0/4125640205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0738100490 0x7f0738198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 >> 172.21.15.5:0/4125640205 conn(0x7f07380fb5e0 msgr2=0x7f07380fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 shutdown_connections 2023-07-21T19:11:12.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.544+0000 7f073c9ff700 1 -- 172.21.15.5:0/4125640205 wait complete. 2023-07-21T19:11:12.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:12 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/813780615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:12.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:11:12.589 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:12 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/813780615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:12.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:12 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/813780615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:12.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.811+0000 7fc84fbff700 1 -- 172.21.15.5:0/1696439889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8480731c0 msgr2=0x7fc84810bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.811+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1696439889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8480731c0 0x7fc84810bdd0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fc838009990 tx=0x7fc838009ca0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 -- 172.21.15.5:0/1696439889 shutdown_connections 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1696439889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc84810c310 0x7fc84810e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1696439889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1696439889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8480731c0 0x7fc84810bdd0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 -- 172.21.15.5:0/1696439889 >> 172.21.15.5:0/1696439889 conn(0x7fc8480fb5f0 msgr2=0x7fc8480fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 -- 172.21.15.5:0/1696439889 shutdown_connections 2023-07-21T19:11:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.812+0000 7fc84fbff700 1 -- 172.21.15.5:0/1696439889 wait complete. 2023-07-21T19:11:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.813+0000 7fc84fbff700 1 Processor -- start 2023-07-21T19:11:12.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.813+0000 7fc84fbff700 1 -- start start 2023-07-21T19:11:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8480731c0 0x7fc848198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84819cd00 con 0x7fc84810c310 2023-07-21T19:11:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84819ce70 con 0x7fc8480731c0 2023-07-21T19:11:12.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84fbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84819d030 con 0x7fc8480728b0 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84e19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41796/0 (socket says 172.21.15.5:41796) 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84e19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46906/0 (socket says 172.21.15.5:46906) 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.814+0000 7fc84d99b700 1 -- 172.21.15.5:0/1289978101 learned_addr learned my addr 172.21.15.5:0/1289978101 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84d19a700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8480731c0 0x7fc848198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 -- 172.21.15.5:0/1289978101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 msgr2=0x7fc848197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 -- 172.21.15.5:0/1289978101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8480731c0 msgr2=0x7fc848198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8480731c0 0x7fc848198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 -- 172.21.15.5:0/1289978101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380095f0 con 0x7fc84810c310 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84d99b700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc840007f10 tx=0x7fc84000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc83effd700 1 -- 172.21.15.5:0/1289978101 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840004db0 con 0x7fc84810c310 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc83effd700 1 -- 172.21.15.5:0/1289978101 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc84000f040 con 0x7fc84810c310 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc84819d310 con 0x7fc84810c310 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.815+0000 7fc83effd700 1 -- 172.21.15.5:0/1289978101 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840005960 con 0x7fc84810c310 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.816+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8481a4b50 con 0x7fc84810c310 2023-07-21T19:11:12.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.819+0000 7fc83effd700 1 -- 172.21.15.5:0/1289978101 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc84000db20 con 0x7fc84810c310 2023-07-21T19:11:12.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.821+0000 7fc863557b80 1 Processor -- start 2023-07-21T19:11:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc83effd700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 0x7fc8340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 -- start start 2023-07-21T19:11:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc83effd700 1 -- 172.21.15.5:0/1289978101 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc8400cc5b0 con 0x7fc84810c310 2023-07-21T19:11:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc84d19a700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 0x7fc8340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba9463590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9463ad0 0x560ba9465eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba935ecb0 con 0x560ba94582d0 2023-07-21T19:11:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba93b1a40 con 0x560ba9458be0 2023-07-21T19:11:12.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.822+0000 7fc863557b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba93aaed0 con 0x560ba9463ad0 2023-07-21T19:11:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46908/0 (socket says 172.21.15.5:46908) 2023-07-21T19:11:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 -- 172.21.15.5:0/3704125277 learned_addr learned my addr 172.21.15.5:0/3704125277 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84e19c700 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9463ad0 0x560ba9465eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d19a700 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba9463590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 -- 172.21.15.5:0/3704125277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9463ad0 msgr2=0x560ba9465eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9463ad0 0x560ba9465eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 -- 172.21.15.5:0/3704125277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 msgr2=0x560ba9463590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba9463590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 -- 172.21.15.5:0/3704125277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba91d8de0 con 0x560ba94582d0 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d19a700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 0x7fc8340a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc848199330 tx=0x7fc8380051a0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.823+0000 7fc84d99b700 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc8440117e0 tx=0x7fc844011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c739ad079a3a3783 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.824+0000 7fc833fff700 1 -- 172.21.15.5:0/3704125277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844019410 con 0x560ba94582d0 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.824+0000 7fc833fff700 1 -- 172.21.15.5:0/3704125277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc844019a00 con 0x560ba94582d0 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.824+0000 7fc833fff700 1 -- 172.21.15.5:0/3704125277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc844018900 con 0x560ba94582d0 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.824+0000 7fc863557b80 1 -- 172.21.15.5:0/3704125277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 msgr2=0x560ba94586a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.824+0000 7fc863557b80 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fc8440117e0 tx=0x7fc844011af0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 -- 172.21.15.5:0/3704125277 shutdown_connections 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9463ad0 0x560ba9465eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba94586a0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 --2- 172.21.15.5:0/3704125277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba9463590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 -- 172.21.15.5:0/3704125277 >> 172.21.15.5:0/3704125277 conn(0x560ba944eda0 msgr2=0x560ba94546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 -- 172.21.15.5:0/3704125277 shutdown_connections 2023-07-21T19:11:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 -- 172.21.15.5:0/3704125277 wait complete. 2023-07-21T19:11:12.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.825+0000 7fc863557b80 1 -- start start 2023-07-21T19:11:12.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.826+0000 7fc863557b80 1 Processor -- start 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.826+0000 7fc863557b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba94607b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba934aac0 con 0x560ba9463ad0 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba934a7c0 con 0x560ba9458be0 2023-07-21T19:11:12.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba922e920 con 0x560ba94582d0 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc863557b80 10 client.0 _ll_register_callbacks cb 0x560ba9348760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46924/0 (socket says 172.21.15.5:46924) 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 -- 172.21.15.5:0/1814021902 learned_addr learned my addr 172.21.15.5:0/1814021902 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41812/0 (socket says 172.21.15.5:41812) 2023-07-21T19:11:12.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84d19a700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba94607b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 -- 172.21.15.5:0/1814021902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 msgr2=0x560ba9463530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 -- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 msgr2=0x560ba94607b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba94607b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84e19c700 1 -- 172.21.15.5:0/1814021902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844011440 con 0x560ba9463ad0 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.827+0000 7fc84d99b700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc84e19c700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc84000d700 tx=0x7fc840099190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bb58f4dcae7355 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840005000 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ba93044f0 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc840099650 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc84009d680 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc863557b80 10 client.16440 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x560ba91c1c60 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.828+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ba93044f0 con 0x560ba9463ad0 2023-07-21T19:11:12.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc840099940 con 0x560ba9463ad0 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 10 client.16440 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 10 client.16440 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 20 client.16440 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94733c0 0x560ba94757d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 4 client.16440 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ba93a3eb0 con 0x560ba94733c0 2023-07-21T19:11:12.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.829+0000 7fc84d99b700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94733c0 0x560ba94757d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.830+0000 7fc8317fa700 10 client.16440 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.830+0000 7fc84d99b700 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94733c0 0x560ba94757d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a561fd7af31191b server_cookie=15019745e2dfa87 in_seq=0 out_seq=0 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc8317fa700 1 -- 172.21.15.5:0/1814021902 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560ba93a3eb0 con 0x560ba94733c0 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc8317fa700 10 client.16440 handle_command_reply: tid=0 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 client.16440 shutdown 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 msgr2=0x560ba9460ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fc84000d700 tx=0x7fc840099190 comp rx=0 tx=0).stop 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 shutdown_connections 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94733c0 0x560ba94757d0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460ef0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba94582d0 0x560ba9463530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 --2- 172.21.15.5:0/1814021902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9458be0 0x560ba94607b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc8317fa700 0 client.16440 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 >> 172.21.15.5:0/1814021902 conn(0x560ba944eda0 msgr2=0x560ba9454290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 shutdown_connections 2023-07-21T19:11:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 1 -- 172.21.15.5:0/1814021902 wait complete. 2023-07-21T19:11:12.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.831+0000 7fc863557b80 20 client.16440 trim_cache size 0 max 16384 2023-07-21T19:11:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.848+0000 7fc863557b80 1 Processor -- start 2023-07-21T19:11:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.848+0000 7fc863557b80 1 -- start start 2023-07-21T19:11:12.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 0x560ba9463060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba9455d60 con 0x560ba9463ad0 2023-07-21T19:11:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba9470ae0 con 0x560ba94582d0 2023-07-21T19:11:12.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc863557b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba9459590 con 0x560ba9458be0 2023-07-21T19:11:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc84d19a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc84d19a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41828/0 (socket says 172.21.15.5:41828) 2023-07-21T19:11:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc84d19a700 1 -- 172.21.15.5:0/849280737 learned_addr learned my addr 172.21.15.5:0/849280737 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.849+0000 7fc84e19c700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d99b700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 0x560ba9463060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 -- 172.21.15.5:0/849280737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 msgr2=0x560ba9463060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 0x560ba9463060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 -- 172.21.15.5:0/849280737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 msgr2=0x560ba9460590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 -- 172.21.15.5:0/849280737 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba91ff250 con 0x560ba9458be0 2023-07-21T19:11:12.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d99b700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 0x560ba9463060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:12.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84e19c700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.850+0000 7fc84d19a700 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc838009990 tx=0x7fc838022000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94f90a340037f362 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.851+0000 7fc83cbff700 1 -- 172.21.15.5:0/849280737 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8380045c0 con 0x560ba9458be0 2023-07-21T19:11:12.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.851+0000 7fc83cbff700 1 -- 172.21.15.5:0/849280737 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc83801e040 con 0x560ba9458be0 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.851+0000 7fc83cbff700 1 -- 172.21.15.5:0/849280737 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83800fa70 con 0x560ba9458be0 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.851+0000 7fc863557b80 1 -- 172.21.15.5:0/849280737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 msgr2=0x560ba9460050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.851+0000 7fc863557b80 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fc838009990 tx=0x7fc838022000 comp rx=0 tx=0).stop 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 -- 172.21.15.5:0/849280737 shutdown_connections 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba9463ad0 0x560ba9460590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba94582d0 0x560ba9463060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 --2- 172.21.15.5:0/849280737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9460050 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 -- 172.21.15.5:0/849280737 >> 172.21.15.5:0/849280737 conn(0x560ba944eda0 msgr2=0x560ba946f760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 -- 172.21.15.5:0/849280737 shutdown_connections 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 -- 172.21.15.5:0/849280737 wait complete. 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 -- start start 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.852+0000 7fc863557b80 1 Processor -- start 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9468800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 0x560ba9468d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba9469450 con 0x560ba94582d0 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba94695c0 con 0x560ba9463ad0 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba94c7290 con 0x560ba9458be0 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc863557b80 10 client.0 _ll_register_callbacks cb 0x560ba9348760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc84d99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46938/0 (socket says 172.21.15.5:46938) 2023-07-21T19:11:12.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc84d99b700 1 -- 172.21.15.5:0/770661142 learned_addr learned my addr 172.21.15.5:0/770661142 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc84d19a700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9468800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.853+0000 7fc84e19c700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 0x560ba9468d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 -- 172.21.15.5:0/770661142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 msgr2=0x560ba9468800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9468800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 -- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 msgr2=0x560ba9468d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 0x560ba9468d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 -- 172.21.15.5:0/770661142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560ba91c1c60 con 0x560ba94582d0 2023-07-21T19:11:12.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84e19c700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 0x560ba9468d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:12.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc84d99b700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc844011730 tx=0x7fc844019e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f856986a1915a7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc8317fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.854+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84402a070 con 0x560ba94582d0 2023-07-21T19:11:12.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc844019a00 con 0x560ba94582d0 2023-07-21T19:11:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc844011e40 con 0x560ba94582d0 2023-07-21T19:11:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8440050b0 con 0x560ba94582d0 2023-07-21T19:11:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 10 client.16446 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:12.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fc84810f390 con 0x560ba94582d0 2023-07-21T19:11:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc844019a00 con 0x560ba94582d0 2023-07-21T19:11:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc844003a50 con 0x560ba94582d0 2023-07-21T19:11:12.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 10 client.16446 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 10 client.16446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 20 client.16446 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.855+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94c8900 0x560ba94cad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:12.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.856+0000 7fc863557b80 4 client.16446 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:12.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.856+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560ba9366050 con 0x560ba94c8900 2023-07-21T19:11:12.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.856+0000 7fc84d19a700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94c8900 0x560ba94cad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:12.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.856+0000 7fc84d19a700 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94c8900 0x560ba94cad10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52eb2dea50b4d09 server_cookie=d1454f1b75875799 in_seq=0 out_seq=0 2023-07-21T19:11:12.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.856+0000 7fc8317fa700 10 client.16446 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:12.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc8317fa700 1 -- 172.21.15.5:0/770661142 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560ba9366050 con 0x560ba94c8900 2023-07-21T19:11:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc8317fa700 10 client.16446 handle_command_reply: tid=0 2023-07-21T19:11:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 client.16446 shutdown 2023-07-21T19:11:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 msgr2=0x560ba946a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fc844011730 tx=0x7fc844019e80 comp rx=0 tx=0).stop 2023-07-21T19:11:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 shutdown_connections 2023-07-21T19:11:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba9463ad0 0x560ba9468d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba94c8900 0x560ba94cad10 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba94582d0 0x560ba946a4d0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 --2- 172.21.15.5:0/770661142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba9458be0 0x560ba9468800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc8317fa700 0 client.16446 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 >> 172.21.15.5:0/770661142 conn(0x560ba944eda0 msgr2=0x560ba945a3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 shutdown_connections 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 1 -- 172.21.15.5:0/770661142 wait complete. 2023-07-21T19:11:12.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.857+0000 7fc863557b80 20 client.16446 trim_cache size 0 max 16384 2023-07-21T19:11:12.879 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:12.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 msgr2=0x7fc8340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 0x7fc8340a3770 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc848199330 tx=0x7fc8380051a0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 msgr2=0x7fc84819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fc840007f10 tx=0x7fc84000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 shutdown_connections 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8340a12c0 0x7fc8340a3770 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc84810c310 0x7fc84819c6c0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8480728b0 0x7fc848197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 --2- 172.21.15.5:0/1289978101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8480731c0 0x7fc848198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 >> 172.21.15.5:0/1289978101 conn(0x7fc8480fb5f0 msgr2=0x7fc84810e470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 shutdown_connections 2023-07-21T19:11:12.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:12.858+0000 7fc84fbff700 1 -- 172.21.15.5:0/1289978101 wait complete. 2023-07-21T19:11:13.054 INFO:teuthology.orchestra.run.smithi005.stderr:11800000 of 50000000 tuples (23%) done (elapsed 190.61 s, remaining 617.07 s) 2023-07-21T19:11:13.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:13 smithi005 ceph-mon[126460]: pgmap v111: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:11:13.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:13 smithi156 ceph-mon[128316]: pgmap v111: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:11:13.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:13 smithi101 ceph-mon[128136]: pgmap v111: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:11:13.885 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:11:13.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:11:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/1888395569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc1003a0 msgr2=0x7f6adc10c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/1888395569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc1003a0 0x7f6adc10c830 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f6ad400b0a0 tx=0x7f6ad400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/1888395569 shutdown_connections 2023-07-21T19:11:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/1888395569 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc1003a0 0x7f6adc10c830 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/1888395569 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 0x7f6adc105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/1888395569 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc0ffe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/1888395569 >> 172.21.15.5:0/1888395569 conn(0x7f6adc0fb510 msgr2=0x7f6adc0fd920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/1888395569 shutdown_connections 2023-07-21T19:11:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.118+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/1888395569 wait complete. 2023-07-21T19:11:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.120+0000 7f6ae1d9d700 1 Processor -- start 2023-07-21T19:11:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.120+0000 7f6ae1d9d700 1 -- start start 2023-07-21T19:11:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc1003a0 0x7f6adc19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 0x7f6adc195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc195e80 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc195ff0 con 0x7f6adc105a00 2023-07-21T19:11:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae1d9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6adc1961b0 con 0x7f6adc1003a0 2023-07-21T19:11:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46964/0 (socket says 172.21.15.5:46964) 2023-07-21T19:11:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/2269204841 learned_addr learned my addr 172.21.15.5:0/2269204841 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae159c700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 0x7f6adc195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc1003a0 0x7f6adc19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/2269204841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc1003a0 msgr2=0x7f6adc19bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.121+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc1003a0 0x7f6adc19bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/2269204841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 msgr2=0x7f6adc195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 0x7f6adc195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/2269204841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ad4009d00 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000e970 tx=0x7f6ad000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ad9ffb700 1 -- 172.21.15.5:0/2269204841 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0009790 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ad9ffb700 1 -- 172.21.15.5:0/2269204841 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ad000c7e0 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6adc196490 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ad9ffb700 1 -- 172.21.15.5:0/2269204841 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad0003bd0 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.122+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a2790 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.126+0000 7f6ad9ffb700 1 -- 172.21.15.5:0/2269204841 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6ad000c950 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.127+0000 7f6af69beb80 1 Processor -- start 2023-07-21T19:11:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 -- start start 2023-07-21T19:11:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d6413b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64145870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64148190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d64040d30 con 0x557d6413be80 2023-07-21T19:11:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d64093ac0 con 0x557d64145db0 2023-07-21T19:11:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6af69beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6408cf50 con 0x557d6413b570 2023-07-21T19:11:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.128+0000 7f6ad9ffb700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 0x7f6acc0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ae159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d6413b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ad9ffb700 1 -- 172.21.15.5:0/2269204841 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ad0014070 con 0x7f6adc0ff9f0 2023-07-21T19:11:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64145870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64148190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64145870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46976/0 (socket says 172.21.15.5:46976) 2023-07-21T19:11:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/2051224001 learned_addr learned my addr 172.21.15.5:0/2051224001 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6ae159c700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 0x7f6acc0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 -- 172.21.15.5:0/2051224001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 msgr2=0x557d6413b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d6413b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 -- 172.21.15.5:0/2051224001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 msgr2=0x557d64145870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64145870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.129+0000 7f6adbfff700 1 -- 172.21.15.5:0/2051224001 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d63ebae60 con 0x557d64145db0 2023-07-21T19:11:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.130+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64148190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009c80 tx=0x7f6ac8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2dad1980c816d2cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.130+0000 7f6ae159c700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 0x7f6acc0a3810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40062a0 tx=0x7f6ad4006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.130+0000 7f6ac6ffd700 1 -- 172.21.15.5:0/2051224001 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8004310 con 0x557d64145db0 2023-07-21T19:11:14.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.130+0000 7f6ac6ffd700 1 -- 172.21.15.5:0/2051224001 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ac800fe00 con 0x557d64145db0 2023-07-21T19:11:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.130+0000 7f6ac6ffd700 1 -- 172.21.15.5:0/2051224001 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8005020 con 0x557d64145db0 2023-07-21T19:11:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 -- 172.21.15.5:0/2051224001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 msgr2=0x557d64148190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64148190 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8009c80 tx=0x7f6ac8009f90 comp rx=0 tx=0).stop 2023-07-21T19:11:14.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 -- 172.21.15.5:0/2051224001 shutdown_connections 2023-07-21T19:11:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64148190 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d6413b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2051224001 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64145870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 -- 172.21.15.5:0/2051224001 >> 172.21.15.5:0/2051224001 conn(0x557d64131070 msgr2=0x557d641369b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 -- 172.21.15.5:0/2051224001 shutdown_connections 2023-07-21T19:11:14.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.131+0000 7f6af69beb80 1 -- 172.21.15.5:0/2051224001 wait complete. 2023-07-21T19:11:14.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.132+0000 7f6af69beb80 1 -- start start 2023-07-21T19:11:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.132+0000 7f6af69beb80 1 Processor -- start 2023-07-21T19:11:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d641456f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64142f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6402cb40 con 0x557d6413be80 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6402c840 con 0x557d64145db0 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d63f109a0 con 0x557d6413b570 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.133+0000 7f6af69beb80 10 client.0 _ll_register_callbacks cb 0x557d6402a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d641456f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d641456f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41890/0 (socket says 172.21.15.5:41890) 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46992/0 (socket says 172.21.15.5:46992) 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/3990307260 learned_addr learned my addr 172.21.15.5:0/3990307260 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6ae159c700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64142f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 -- 172.21.15.5:0/3990307260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 msgr2=0x557d641456f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d641456f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 -- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 msgr2=0x557d64142f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64142f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.134+0000 7f6adbfff700 1 -- 172.21.15.5:0/3990307260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ac80098e0 con 0x557d6413be80 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6adbfff700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80097c0 tx=0x7f6ac800fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fd83eca1cbd6ac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6abbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac8003680 con 0x557d6413be80 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ac80037e0 con 0x557d6413be80 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d63fe6570 con 0x557d6413be80 2023-07-21T19:11:14.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.135+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac80039a0 con 0x557d6413be80 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 10 client.16458 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557d63ea3ce0 con 0x557d6413be80 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557d63fe6570 con 0x557d6413be80 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6ac8029040 con 0x557d6413be80 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 10 client.16458 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 10 client.16458 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 20 client.16458 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d64155620 0x557d64157a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 4 client.16458 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.136+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d64085f30 con 0x557d64155620 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.137+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d64155620 0x557d64157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.138+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d64155620 0x557d64157a30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5226f9b9b10f76fc server_cookie=77904989655a6e6 in_seq=0 out_seq=0 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.138+0000 7f6abbfff700 10 client.16458 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6abbfff700 1 -- 172.21.15.5:0/3990307260 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d64085f30 con 0x557d64155620 2023-07-21T19:11:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6abbfff700 10 client.16458 handle_command_reply: tid=0 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 client.16458 shutdown 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 msgr2=0x557d64142970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f6ac80097c0 tx=0x7f6ac800fba0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 shutdown_connections 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d64145db0 0x557d64142f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d6413b570 0x557d641456f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d64155620 0x557d64157a30 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3990307260 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d64142970 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 >> 172.21.15.5:0/3990307260 conn(0x557d64131070 msgr2=0x557d641365a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.141+0000 7f6abbfff700 0 client.16458 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.142+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 shutdown_connections 2023-07-21T19:11:14.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.142+0000 7f6af69beb80 1 -- 172.21.15.5:0/3990307260 wait complete. 2023-07-21T19:11:14.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.142+0000 7f6af69beb80 20 client.16458 trim_cache size 0 max 16384 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 Processor -- start 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 -- start start 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413be80 0x557d64142670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6413fc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6412e1c0 con 0x557d6413b570 2023-07-21T19:11:14.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6413d220 con 0x557d6413be80 2023-07-21T19:11:14.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6af69beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d64140ed0 con 0x557d64145db0 2023-07-21T19:11:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.166+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47000/0 (socket says 172.21.15.5:47000) 2023-07-21T19:11:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/3923579193 learned_addr learned my addr 172.21.15.5:0/3923579193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae159c700 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6413fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6adbfff700 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413be80 0x557d64142670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/3923579193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 msgr2=0x557d6413fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6413fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/3923579193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413be80 msgr2=0x557d64142670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413be80 0x557d64142670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 -- 172.21.15.5:0/3923579193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d63ee12d0 con 0x557d6413b570 2023-07-21T19:11:14.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000cc00 tx=0x7f6ad00bb040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d7eb2573be46ed4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.167+0000 7f6ac77fe700 1 -- 172.21.15.5:0/3923579193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00a2390 con 0x557d6413b570 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6ac77fe700 1 -- 172.21.15.5:0/3923579193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ad00982e0 con 0x557d6413b570 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6ac77fe700 1 -- 172.21.15.5:0/3923579193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ad00caa70 con 0x557d6413b570 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 -- 172.21.15.5:0/3923579193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 msgr2=0x557d64142130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000cc00 tx=0x7f6ad00bb040 comp rx=0 tx=0).stop 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 -- 172.21.15.5:0/3923579193 shutdown_connections 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6413fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413b570 0x557d64142130 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 --2- 172.21.15.5:0/3923579193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413be80 0x557d64142670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.168+0000 7f6af69beb80 1 -- 172.21.15.5:0/3923579193 >> 172.21.15.5:0/3923579193 conn(0x557d64131070 msgr2=0x557d6413dc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.169+0000 7f6af69beb80 1 -- 172.21.15.5:0/3923579193 shutdown_connections 2023-07-21T19:11:14.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.169+0000 7f6af69beb80 1 -- 172.21.15.5:0/3923579193 wait complete. 2023-07-21T19:11:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.169+0000 7f6af69beb80 1 -- start start 2023-07-21T19:11:14.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.169+0000 7f6af69beb80 1 Processor -- start 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413b570 0x557d641a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6414b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6414bf10 con 0x557d6413be80 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6414c080 con 0x557d6413b570 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d6414c240 con 0x557d64145db0 2023-07-21T19:11:14.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6af69beb80 10 client.0 _ll_register_callbacks cb 0x7f6ad000ce40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6adbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47012/0 (socket says 172.21.15.5:47012) 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6adbfff700 1 -- 172.21.15.5:0/2089147076 learned_addr learned my addr 172.21.15.5:0/2089147076 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6ae159c700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6414b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.170+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413b570 0x557d641a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 -- 172.21.15.5:0/2089147076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 msgr2=0x557d6414b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6414b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 -- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413b570 msgr2=0x557d641a27a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413b570 0x557d641a27a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 -- 172.21.15.5:0/2089147076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6adc1a2790 con 0x557d6413be80 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6adbfff700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac8005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05bb55f8855a1e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6abbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ac802b870 con 0x557d6413be80 2023-07-21T19:11:14.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ac800fe00 con 0x557d6413be80 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6ac802b9d0 con 0x557d6413be80 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ac8021430 con 0x557d6413be80 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6af69beb80 10 client.16464 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.171+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x557d63eb9330 con 0x557d6413be80 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ac800fe00 con 0x557d6413be80 2023-07-21T19:11:14.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6ac8029070 con 0x557d6413be80 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 10 client.16464 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 10 client.16464 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 20 client.16464 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d641ab420 0x557d641ad830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 4 client.16464 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x557d640480d0 con 0x557d641ab420 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.172+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d641ab420 0x557d641ad830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.173+0000 7f6ae0d9b700 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d641ab420 0x557d641ad830 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29fcf3d613e5ac3a server_cookie=93b02e3ad897f117 in_seq=0 out_seq=0 2023-07-21T19:11:14.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.173+0000 7f6abbfff700 10 client.16464 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.174+0000 7f6abbfff700 1 -- 172.21.15.5:0/2089147076 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x557d640480d0 con 0x557d641ab420 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.174+0000 7f6abbfff700 10 client.16464 handle_command_reply: tid=0 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.174+0000 7f6af69beb80 1 client.16464 shutdown 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 msgr2=0x557d641a2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ac8000c00 tx=0x7f6ac8005c90 comp rx=0 tx=0).stop 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 shutdown_connections 2023-07-21T19:11:14.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d64145db0 0x557d6414b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d6413b570 0x557d641a27a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d641ab420 0x557d641ad830 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 --2- 172.21.15.5:0/2089147076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d6413be80 0x557d641a2ce0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6abbfff700 0 client.16464 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 >> 172.21.15.5:0/2089147076 conn(0x557d64131070 msgr2=0x557d64144280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 shutdown_connections 2023-07-21T19:11:14.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 1 -- 172.21.15.5:0/2089147076 wait complete. 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.175+0000 7f6af69beb80 20 client.16464 trim_cache size 0 max 16384 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5a1fdd3e-5eec-438c-8908-57b0bf74d5af", 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:11:14.191 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 msgr2=0x7f6acc0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 0x7f6acc0a3810 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6ad40062a0 tx=0x7f6ad4006210 comp rx=0 tx=0).stop 2023-07-21T19:11:14.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 msgr2=0x7f6adc19b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f6ad000e970 tx=0x7f6ad000ec80 comp rx=0 tx=0).stop 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 shutdown_connections 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6adc105a00 0x7f6adc195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6acc0a1360 0x7f6acc0a3810 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6adc0ff9f0 0x7f6adc19b7a0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 --2- 172.21.15.5:0/2269204841 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6adc1003a0 0x7f6adc19bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 >> 172.21.15.5:0/2269204841 conn(0x7f6adc0fb510 msgr2=0x7f6adc0fca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 shutdown_connections 2023-07-21T19:11:14.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.176+0000 7f6ae1d9d700 1 -- 172.21.15.5:0/2269204841 wait complete. 2023-07-21T19:11:14.207 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:11:14.407 INFO:teuthology.orchestra.run.smithi005.stderr:11900000 of 50000000 tuples (23%) done (elapsed 191.97 s, remaining 614.61 s) 2023-07-21T19:11:14.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 -- 172.21.15.5:0/949511604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 msgr2=0x7fdecc109b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 --2- 172.21.15.5:0/949511604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc109b40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fdebc009990 tx=0x7fdebc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 -- 172.21.15.5:0/949511604 shutdown_connections 2023-07-21T19:11:14.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 --2- 172.21.15.5:0/949511604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc10a080 0x7fdecc10c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 --2- 172.21.15.5:0/949511604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc100b60 0x7fdecc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 --2- 172.21.15.5:0/949511604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc109b40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 -- 172.21.15.5:0/949511604 >> 172.21.15.5:0/949511604 conn(0x7fdecc0fa4b0 msgr2=0x7fdecc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 -- 172.21.15.5:0/949511604 shutdown_connections 2023-07-21T19:11:14.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.455+0000 7fded45ff700 1 -- 172.21.15.5:0/949511604 wait complete. 2023-07-21T19:11:14.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 Processor -- start 2023-07-21T19:11:14.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 -- start start 2023-07-21T19:11:14.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 0x7fdecc197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc10a080 0x7fdecc19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc19cd50 con 0x7fdecc101470 2023-07-21T19:11:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc19cec0 con 0x7fdecc10a080 2023-07-21T19:11:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.456+0000 7fded45ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdecc19d080 con 0x7fdecc100b60 2023-07-21T19:11:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47042/0 (socket says 172.21.15.5:47042) 2023-07-21T19:11:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 -- 172.21.15.5:0/288327541 learned_addr learned my addr 172.21.15.5:0/288327541 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded239b700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 0x7fdecc197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded2b9c700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc10a080 0x7fdecc19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 -- 172.21.15.5:0/288327541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 msgr2=0x7fdecc197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 0x7fdecc197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 -- 172.21.15.5:0/288327541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc10a080 msgr2=0x7fdecc19c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc10a080 0x7fdecc19c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded1b9a700 1 -- 172.21.15.5:0/288327541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0095f0 con 0x7fdecc101470 2023-07-21T19:11:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.457+0000 7fded239b700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 0x7fdecc197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:14.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fded1b9a700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005740 tx=0x7fdebc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fdec37fe700 1 -- 172.21.15.5:0/288327541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc01d070 con 0x7fdecc101470 2023-07-21T19:11:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdecc19d300 con 0x7fdecc101470 2023-07-21T19:11:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fdec37fe700 1 -- 172.21.15.5:0/288327541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdebc022470 con 0x7fdecc101470 2023-07-21T19:11:14.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fdec37fe700 1 -- 172.21.15.5:0/288327541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc017650 con 0x7fdecc101470 2023-07-21T19:11:14.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.458+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdecc1a4ba0 con 0x7fdecc101470 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.462+0000 7fdec37fe700 1 -- 172.21.15.5:0/288327541 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdebc00fb00 con 0x7fdecc101470 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.463+0000 7fdec37fe700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 0x7fdeb80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.463+0000 7fdec37fe700 1 -- 172.21.15.5:0/288327541 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdebc0ce270 con 0x7fdecc101470 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.463+0000 7fdee7ec6b80 1 Processor -- start 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fded239b700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 0x7fdeb80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 -- start start 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b58f4cb0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b5947a40 con 0x55b5b59f9ad0 2023-07-21T19:11:14.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b5940ed0 con 0x55b5b59eebe0 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.464+0000 7fded239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47050/0 (socket says 172.21.15.5:47050) 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded2b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 -- 172.21.15.5:0/910929801 learned_addr learned my addr 172.21.15.5:0/910929801 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 0x7fdeb80a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdec8005950 tx=0x7fdec800b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded1b9a700 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 -- 172.21.15.5:0/910929801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 msgr2=0x55b5b59f9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 -- 172.21.15.5:0/910929801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 msgr2=0x55b5b59fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 -- 172.21.15.5:0/910929801 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b576ede0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.465+0000 7fded239b700 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdec8011760 tx=0x7fdec8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2998f7d54d1fdc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.466+0000 7fdec0bfe700 1 -- 172.21.15.5:0/910929801 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec8004220 con 0x55b5b59ee2d0 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.466+0000 7fdec0bfe700 1 -- 172.21.15.5:0/910929801 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdec8015070 con 0x55b5b59ee2d0 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.466+0000 7fdec0bfe700 1 -- 172.21.15.5:0/910929801 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec801e680 con 0x55b5b59ee2d0 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.466+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/910929801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 msgr2=0x55b5b59ee6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.466+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7fdec8011760 tx=0x7fdec8013000 comp rx=0 tx=0).stop 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/910929801 shutdown_connections 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59ee6a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/910929801 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/910929801 >> 172.21.15.5:0/910929801 conn(0x55b5b59e4da0 msgr2=0x55b5b59ea6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/910929801 shutdown_connections 2023-07-21T19:11:14.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/910929801 wait complete. 2023-07-21T19:11:14.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 -- start start 2023-07-21T19:11:14.484 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.467+0000 7fdee7ec6b80 1 Processor -- start 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b58e0ac0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b58e07c0 con 0x55b5b59f9ad0 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b57c4920 con 0x55b5b59eebe0 2023-07-21T19:11:14.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fdee7ec6b80 10 client.0 _ll_register_callbacks cb 0x55b5b58de760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:14.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fded239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fded239b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47056/0 (socket says 172.21.15.5:47056) 2023-07-21T19:11:14.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.468+0000 7fded239b700 1 -- 172.21.15.5:0/3204958824 learned_addr learned my addr 172.21.15.5:0/3204958824 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded1b9a700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 -- 172.21.15.5:0/3204958824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 msgr2=0x55b5b59f6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 -- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 msgr2=0x55b5b59f6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded2b9c700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f6ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 -- 172.21.15.5:0/3204958824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec8011440 con 0x55b5b59ee2d0 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded1b9a700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:14.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fded239b700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdec8010c00 tx=0x7fdec8003960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59abb454c8eb858 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fdeb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.469+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec802c070 con 0x55b5b59ee2d0 2023-07-21T19:11:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b5b589a4f0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdec8017070 con 0x55b5b59ee2d0 2023-07-21T19:11:14.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdec8010ea0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 10 client.16476 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55b5b5757c60 con 0x55b5b59ee2d0 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b5b589a4f0 con 0x55b5b59ee2d0 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdec8024030 con 0x55b5b59ee2d0 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 10 client.16476 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 20 client.16476 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:14.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a09350 0x55b5b5a0b760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 4 client.16476 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.470+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5b5939eb0 con 0x55b5b5a09350 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.471+0000 7fded1b9a700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a09350 0x55b5b5a0b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.471+0000 7fded1b9a700 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a09350 0x55b5b5a0b760 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0cc6a6c158fbff1 server_cookie=7daec0f7c14478da in_seq=0 out_seq=0 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.471+0000 7fdeb5ffb700 10 client.16476 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/3204958824 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b5b5939eb0 con 0x55b5b5a09350 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdeb5ffb700 10 client.16476 handle_command_reply: tid=0 2023-07-21T19:11:14.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 client.16476 shutdown 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 msgr2=0x55b5b59f9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7fdec8010c00 tx=0x7fdec8003960 comp rx=0 tx=0).stop 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 shutdown_connections 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a09350 0x55b5b5a0b760 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f9500 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/3204958824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59eebe0 0x55b5b59f6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.472+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 >> 172.21.15.5:0/3204958824 conn(0x55b5b59e4da0 msgr2=0x55b5b59ea230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.473+0000 7fdeb5ffb700 0 client.16476 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.473+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 shutdown_connections 2023-07-21T19:11:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.473+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/3204958824 wait complete. 2023-07-21T19:11:14.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.473+0000 7fdee7ec6b80 20 client.16476 trim_cache size 0 max 16384 2023-07-21T19:11:14.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 Processor -- start 2023-07-21T19:11:14.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 -- start start 2023-07-21T19:11:14.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f1fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b59e6ec0 con 0x55b5b59f9ad0 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b59ebd20 con 0x55b5b59eebe0 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.493+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b59f7d30 con 0x55b5b59ee2d0 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43484/0 (socket says 172.21.15.5:43484) 2023-07-21T19:11:14.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 -- 172.21.15.5:0/1065699412 learned_addr learned my addr 172.21.15.5:0/1065699412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41956/0 (socket says 172.21.15.5:41956) 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded2b9c700 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f1fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 -- 172.21.15.5:0/1065699412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 msgr2=0x55b5b59f90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 -- 172.21.15.5:0/1065699412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 msgr2=0x55b5b59f1fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f1fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded1b9a700 1 -- 172.21.15.5:0/1065699412 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5b5795250 con 0x55b5b59eebe0 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.494+0000 7fded239b700 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.495+0000 7fded1b9a700 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0cb9f0 tx=0x7fdebc0a1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3990c4fdb8f623 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.495+0000 7fdec13ff700 1 -- 172.21.15.5:0/1065699412 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc09f3d0 con 0x55b5b59eebe0 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.495+0000 7fdec13ff700 1 -- 172.21.15.5:0/1065699412 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdebc0a3760 con 0x55b5b59eebe0 2023-07-21T19:11:14.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.495+0000 7fdec13ff700 1 -- 172.21.15.5:0/1065699412 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc099740 con 0x55b5b59eebe0 2023-07-21T19:11:14.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/1065699412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 msgr2=0x55b5b59f1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdebc0cb9f0 tx=0x7fdebc0a1040 comp rx=0 tx=0).stop 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/1065699412 shutdown_connections 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b59f1fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59f90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/1065699412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59f1a90 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/1065699412 >> 172.21.15.5:0/1065699412 conn(0x55b5b59e4da0 msgr2=0x55b5b5a05760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/1065699412 shutdown_connections 2023-07-21T19:11:14.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.496+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/1065699412 wait complete. 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.497+0000 7fdee7ec6b80 1 -- start start 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.497+0000 7fdee7ec6b80 1 Processor -- start 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59fe370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59fe8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b5a56740 con 0x55b5b59f9ad0 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b5a56880 con 0x55b5b59eebe0 2023-07-21T19:11:14.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5b5a56a40 con 0x55b5b59ee2d0 2023-07-21T19:11:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.498+0000 7fdee7ec6b80 10 client.0 _ll_register_callbacks cb 0x7fdec4009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:14.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47066/0 (socket says 172.21.15.5:47066) 2023-07-21T19:11:14.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 -- 172.21.15.5:0/958657578 learned_addr learned my addr 172.21.15.5:0/958657578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded1b9a700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59fe8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded239b700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59fe370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 -- 172.21.15.5:0/958657578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 msgr2=0x55b5b59fe370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59fe370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 -- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 msgr2=0x55b5b59fe8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59fe8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.499+0000 7fded2b9c700 1 -- 172.21.15.5:0/958657578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5b5757c60 con 0x55b5b59f9ad0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.500+0000 7fded2b9c700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdec4004660 tx=0x7fdec4004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435e7c29d2da6c32 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.500+0000 7fdeb5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.500+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec400e040 con 0x55b5b59f9ad0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.500+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdec8015070 con 0x55b5b59f9ad0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.500+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdec4017820 con 0x55b5b59f9ad0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.501+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdec40179e0 con 0x55b5b59f9ad0 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.501+0000 7fdee7ec6b80 10 client.16482 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:14.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.501+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fdecc1a4ba0 con 0x55b5b59f9ad0 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.501+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdec8015070 con 0x55b5b59f9ad0 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.501+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fdec402ab90 con 0x55b5b59f9ad0 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 10 client.16482 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 20 client.16482 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a5e2a0 0x55b5b5a606b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:14.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 4 client.16482 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:11:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b5b58fc050 con 0x55b5b5a5e2a0 2023-07-21T19:11:14.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.502+0000 7fded239b700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a5e2a0 0x55b5b5a606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.503+0000 7fded239b700 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a5e2a0 0x55b5b5a606b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6214bd87f6fccc84 server_cookie=f4d2cc9ba3a684b6 in_seq=0 out_seq=0 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.503+0000 7fdeb5ffb700 10 client.16482 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdeb5ffb700 1 -- 172.21.15.5:0/958657578 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b5b58fc050 con 0x55b5b5a5e2a0 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdeb5ffb700 10 client.16482 handle_command_reply: tid=0 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 client.16482 shutdown 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 msgr2=0x55b5b5a56200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fdec4004660 tx=0x7fdec4004690 comp rx=0 tx=0).stop 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 shutdown_connections 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5b5a5e2a0 0x55b5b5a606b0 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b5b59f9ad0 0x55b5b5a56200 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b5b59ee2d0 0x55b5b59fe370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 --2- 172.21.15.5:0/958657578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b5b59eebe0 0x55b5b59fe8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 >> 172.21.15.5:0/958657578 conn(0x55b5b59e4da0 msgr2=0x55b5b59f1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdeb5ffb700 0 client.16482 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 shutdown_connections 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.504+0000 7fdee7ec6b80 1 -- 172.21.15.5:0/958657578 wait complete. 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fdee7ec6b80 20 client.16482 trim_cache size 0 max 16384 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:11:14.518 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "5a1fdd3e-5eec-438c-8908-57b0bf74d5af": { 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5a1fdd3e-5eec-438c-8908-57b0bf74d5af", 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:11:14.519 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 msgr2=0x7fdeb80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 0x7fdeb80a3770 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fdec8005950 tx=0x7fdec800b410 comp rx=0 tx=0).stop 2023-07-21T19:11:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 msgr2=0x7fdecc1982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:14.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7fdebc005740 tx=0x7fdebc005770 comp rx=0 tx=0).stop 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 shutdown_connections 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdecc10a080 0x7fdecc19c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdecc100b60 0x7fdecc197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdeb80a12c0 0x7fdeb80a3770 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 --2- 172.21.15.5:0/288327541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdecc101470 0x7fdecc1982b0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.505+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 >> 172.21.15.5:0/288327541 conn(0x7fdecc0fa4b0 msgr2=0x7fdecc10c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:14.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.506+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 shutdown_connections 2023-07-21T19:11:14.522 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:14.506+0000 7fded45ff700 1 -- 172.21.15.5:0/288327541 wait complete. 2023-07-21T19:11:14.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:14 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:11:14.532 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:14 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:14.535 INFO:tasks.cephfs.filesystem:scrub status for tag:5a1fdd3e-5eec-438c-8908-57b0bf74d5af - {'path': '/', 'tag': '5a1fdd3e-5eec-438c-8908-57b0bf74d5af', 'options': 'recursive,force'} 2023-07-21T19:11:14.589 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:14 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:11:14.589 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:14 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:14.589 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:11:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:11:14.196+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...5,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v1924 ap=2 f(v2 m2023-07-21T19:10:44.888225+0000 3=1+2) n(v1 rc2023-07-21T19:10:44.888225+0000 b8 4=1+3) old_inodes=3 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@62} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v1924>,<0x10000000005/data v1011>,<0x10000000003/postgres v289>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v307>,<0x10000000001/sv_0 v325>,<0x10000000000/_nogroup v308>,<0x1/volumes v296>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:10:44.872088+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:10:44.888225+0000 b8 4=1+3)","memory_value.dirstat":"f(v2 m2023-07-21T19:10:44.888225+0000 3=1+2)","memory_value.rstat":"n(v1 rc2023-07-21T19:10:44.888225+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:11:14.589 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:11:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:11:14.197+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...6,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v1958 ap=2 f(v30 m2023-07-21T19:11:02.256537+0000 3=3+0) n(v3 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1) old_inodes=4 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@86} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v1958>,<0x10000000005/data v1011>,<0x10000000003/postgres v289>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v307>,<0x10000000001/sv_0 v325>,<0x10000000000/_nogroup v308>,<0x1/volumes v296>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:11:02.244983+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1)","memory_value.dirstat":"f(v30 m2023-07-21T19:11:02.256537+0000 3=3+0)","memory_value.rstat":"n(v3 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:11:14.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:14 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:11:14.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:14 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: pgmap v112: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.667 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.668 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:15 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: pgmap v112: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:11:15.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:11:15.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.679 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:15 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.707 INFO:teuthology.orchestra.run.smithi005.stderr:12000000 of 50000000 tuples (24%) done (elapsed 193.27 s, remaining 612.01 s) 2023-07-21T19:11:15.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: pgmap v112: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 26 op/s 2023-07-21T19:11:15.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:11:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:11:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:11:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.840 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:15.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:15.841 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:15 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:17.087 INFO:teuthology.orchestra.run.smithi005.stderr:12100000 of 50000000 tuples (24%) done (elapsed 194.65 s, remaining 609.67 s) 2023-07-21T19:11:17.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:17 smithi005 ceph-mon[126460]: pgmap v113: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 22 op/s 2023-07-21T19:11:17.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:17 smithi156 ceph-mon[128316]: pgmap v113: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 22 op/s 2023-07-21T19:11:17.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:17 smithi101 ceph-mon[128136]: pgmap v113: 97 pgs: 97 active+clean; 2.7 GiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 22 MiB/s wr, 22 op/s 2023-07-21T19:11:18.357 INFO:teuthology.orchestra.run.smithi005.stderr:12200000 of 50000000 tuples (24%) done (elapsed 195.91 s, remaining 607.01 s) 2023-07-21T19:11:19.646 INFO:teuthology.orchestra.run.smithi005.stderr:12300000 of 50000000 tuples (24%) done (elapsed 197.20 s, remaining 604.44 s) 2023-07-21T19:11:19.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:19 smithi005 ceph-mon[126460]: pgmap v114: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 33 MiB/s wr, 113 op/s 2023-07-21T19:11:19.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:19 smithi156 ceph-mon[128316]: pgmap v114: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 33 MiB/s wr, 113 op/s 2023-07-21T19:11:19.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:19 smithi101 ceph-mon[128136]: pgmap v114: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 117 KiB/s rd, 33 MiB/s wr, 113 op/s 2023-07-21T19:11:20.923 INFO:teuthology.orchestra.run.smithi005.stderr:12400000 of 50000000 tuples (24%) done (elapsed 198.48 s, remaining 601.85 s) 2023-07-21T19:11:21.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:21 smithi005 ceph-mon[126460]: pgmap v115: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:11:21.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:11:21.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:11:21.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:21 smithi156 ceph-mon[128316]: pgmap v115: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:11:21.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:11:21.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:11:21.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:21 smithi101 ceph-mon[128136]: pgmap v115: 97 pgs: 97 active+clean; 2.8 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:11:21.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:11:21.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:11:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:12500000 of 50000000 tuples (25%) done (elapsed 199.87 s, remaining 599.62 s) 2023-07-21T19:11:22.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:22 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:11:22.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:22 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:11:22.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:22 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:11:23.580 INFO:teuthology.orchestra.run.smithi005.stderr:12600000 of 50000000 tuples (25%) done (elapsed 201.14 s, remaining 597.03 s) 2023-07-21T19:11:23.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:23 smithi005 ceph-mon[126460]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 33 MiB/s wr, 109 op/s 2023-07-21T19:11:23.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:23 smithi156 ceph-mon[128316]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 33 MiB/s wr, 109 op/s 2023-07-21T19:11:23.839 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:23 smithi101 ceph-mon[128136]: pgmap v116: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 103 KiB/s rd, 33 MiB/s wr, 109 op/s 2023-07-21T19:11:24.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 -- 172.21.15.5:0/719247283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 msgr2=0x7f808010e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 --2- 172.21.15.5:0/719247283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808010e7d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f807800b0a0 tx=0x7f807800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 -- 172.21.15.5:0/719247283 shutdown_connections 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 --2- 172.21.15.5:0/719247283 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808010e7d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 --2- 172.21.15.5:0/719247283 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 --2- 172.21.15.5:0/719247283 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 0x7f808010beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 -- 172.21.15.5:0/719247283 >> 172.21.15.5:0/719247283 conn(0x7f80800fb650 msgr2=0x7f80800fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 -- 172.21.15.5:0/719247283 shutdown_connections 2023-07-21T19:11:24.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.669+0000 7f80871ff700 1 -- 172.21.15.5:0/719247283 wait complete. 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 Processor -- start 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 -- start start 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 0x7f80801983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808019d090 con 0x7f808010c3f0 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808019d200 con 0x7f8080072850 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.671+0000 7f80871ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808019d3c0 con 0x7f8080073160 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f808579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35438/0 (socket says 172.21.15.5:35438) 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f808579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57094/0 (socket says 172.21.15.5:57094) 2023-07-21T19:11:24.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 -- 172.21.15.5:0/972947585 learned_addr learned my addr 172.21.15.5:0/972947585 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f807ffff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 0x7f80801983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 -- 172.21.15.5:0/972947585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 msgr2=0x7f80801983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 0x7f80801983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 -- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 msgr2=0x7f808019ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 -- 172.21.15.5:0/972947585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8078009d00 con 0x7f8080072850 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f808579c700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:24.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.672+0000 7f8084f9b700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f807400e9a0 tx=0x7f807400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.673+0000 7f807dffb700 1 -- 172.21.15.5:0/972947585 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8074009790 con 0x7f8080072850 2023-07-21T19:11:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.673+0000 7f807dffb700 1 -- 172.21.15.5:0/972947585 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8074010430 con 0x7f8080072850 2023-07-21T19:11:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.673+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808019d6a0 con 0x7f8080072850 2023-07-21T19:11:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.673+0000 7f807dffb700 1 -- 172.21.15.5:0/972947585 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807400cea0 con 0x7f8080072850 2023-07-21T19:11:24.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.673+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80801a4ee0 con 0x7f8080072850 2023-07-21T19:11:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 Processor -- start 2023-07-21T19:11:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f807dffb700 1 -- 172.21.15.5:0/972947585 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f80740105a0 con 0x7f8080072850 2023-07-21T19:11:24.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 -- start start 2023-07-21T19:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eae9ffe040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eae9ffe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9f03c50 con 0x55eaea0042d0 2023-07-21T19:11:24.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9f569e0 con 0x55eae9e5a6b0 2023-07-21T19:11:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.678+0000 7f809abbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9f4fe70 con 0x55eae9ffe580 2023-07-21T19:11:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35448/0 (socket says 172.21.15.5:35448) 2023-07-21T19:11:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 -- 172.21.15.5:0/2322293646 learned_addr learned my addr 172.21.15.5:0/2322293646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:24.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f807ffff700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eae9ffe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f8084f9b700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eae9ffe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 -- 172.21.15.5:0/2322293646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 msgr2=0x55eae9ffe9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eae9ffe9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 -- 172.21.15.5:0/2322293646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 msgr2=0x55eae9ffe040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eae9ffe040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 -- 172.21.15.5:0/2322293646 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eae9d7dd80 con 0x55eae9e5a6b0 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f8084f9b700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eae9ffe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f807dffb700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 0x7f80700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.679+0000 7f808579c700 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8080074920 tx=0x7f8078009c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be363d7a3d264536 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.680+0000 7f807dffb700 1 -- 172.21.15.5:0/972947585 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8074014070 con 0x7f8080072850 2023-07-21T19:11:24.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.680+0000 7f806affd700 1 -- 172.21.15.5:0/2322293646 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800e040 con 0x55eae9e5a6b0 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.680+0000 7f806affd700 1 -- 172.21.15.5:0/2322293646 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8078009460 con 0x55eae9e5a6b0 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.680+0000 7f806affd700 1 -- 172.21.15.5:0/2322293646 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8078005690 con 0x55eae9e5a6b0 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.680+0000 7f8084f9b700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 0x7f80700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 -- 172.21.15.5:0/2322293646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 msgr2=0x55eaea003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8080074920 tx=0x7f8078009c50 comp rx=0 tx=0).stop 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f8084f9b700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 0x7f80700a37f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80740ca680 tx=0x7f80740ccf50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 -- 172.21.15.5:0/2322293646 shutdown_connections 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eae9ffe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea003d90 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 --2- 172.21.15.5:0/2322293646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eae9ffe040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 -- 172.21.15.5:0/2322293646 >> 172.21.15.5:0/2322293646 conn(0x55eae9ff3be0 msgr2=0x55eae9ff8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 -- 172.21.15.5:0/2322293646 shutdown_connections 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.681+0000 7f809abbab80 1 -- 172.21.15.5:0/2322293646 wait complete. 2023-07-21T19:11:24.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.682+0000 7f809abbab80 1 -- start start 2023-07-21T19:11:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.682+0000 7f809abbab80 1 Processor -- start 2023-07-21T19:11:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eaea007840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9eefa60 con 0x55eaea0042d0 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9eef760 con 0x55eae9e5a6b0 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9dd38c0 con 0x55eae9ffe580 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.683+0000 7f809abbab80 10 client.0 _ll_register_callbacks cb 0x55eae9eed700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f808579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f808579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35456/0 (socket says 172.21.15.5:35456) 2023-07-21T19:11:24.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f808579c700 1 -- 172.21.15.5:0/2230878869 learned_addr learned my addr 172.21.15.5:0/2230878869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f808579c700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eaea007840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 -- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 msgr2=0x55eaea00a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 -- 172.21.15.5:0/2230878869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 msgr2=0x55eaea007e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f807ffff700 1 -- 172.21.15.5:0/2230878869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808010f410 con 0x55eae9ffe580 2023-07-21T19:11:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.684+0000 7f808579c700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.685+0000 7f807ffff700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eaea007840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f806c005950 tx=0x7f806c011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6ed44634a977817 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.685+0000 7f805ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.685+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c016070 con 0x55eae9ffe580 2023-07-21T19:11:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.685+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f806c004830 con 0x55eae9ffe580 2023-07-21T19:11:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.685+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eae9ea9490 con 0x55eae9ffe580 2023-07-21T19:11:24.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.686+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f806c0049f0 con 0x55eae9ffe580 2023-07-21T19:11:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.686+0000 7f809abbab80 10 client.26173 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.686+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eae9d66c00 con 0x55eae9ffe580 2023-07-21T19:11:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eae9ea9490 con 0x55eae9ffe580 2023-07-21T19:11:24.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f806c01a710 con 0x55eae9ffe580 2023-07-21T19:11:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 10 client.26173 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 10 client.26173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:24.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 20 client.26173 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea017c40 0x55eaea01a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 4 client.26173 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:24.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.687+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eae9f48e50 con 0x55eaea017c40 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.688+0000 7f808579c700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea017c40 0x55eaea01a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.688+0000 7f808579c700 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea017c40 0x55eaea01a050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cc5a41cf41eb3bc server_cookie=b493152376580d2d in_seq=0 out_seq=0 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.688+0000 7f805ffff700 10 client.26173 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.689+0000 7f805ffff700 1 -- 172.21.15.5:0/2230878869 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eae9f48e50 con 0x55eaea017c40 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.689+0000 7f805ffff700 10 client.26173 handle_command_reply: tid=0 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.689+0000 7f809abbab80 1 client.26173 shutdown 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.689+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 msgr2=0x55eaea007840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.689+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eaea007840 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f806c005950 tx=0x7f806c011720 comp rx=0 tx=0).stop 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 shutdown_connections 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea017c40 0x55eaea01a050 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 --2- 172.21.15.5:0/2230878869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eae9ffe580 0x55eaea007840 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 >> 172.21.15.5:0/2230878869 conn(0x55eae9ff3be0 msgr2=0x55eae9ff7c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f805ffff700 0 client.26173 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 shutdown_connections 2023-07-21T19:11:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 1 -- 172.21.15.5:0/2230878869 wait complete. 2023-07-21T19:11:24.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.690+0000 7f809abbab80 20 client.26173 trim_cache size 0 max 16384 2023-07-21T19:11:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.712+0000 7f809abbab80 1 Processor -- start 2023-07-21T19:11:24.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.712+0000 7f809abbab80 1 -- start start 2023-07-21T19:11:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea007f10 0x55eae9ff8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9ffaa80 con 0x55eaea0042d0 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaea005dc0 con 0x55eae9e5a6b0 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f809abbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaea0025e0 con 0x55eaea007f10 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57106/0 (socket says 172.21.15.5:57106) 2023-07-21T19:11:24.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 -- 172.21.15.5:0/166699877 learned_addr learned my addr 172.21.15.5:0/166699877 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f808579c700 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 -- 172.21.15.5:0/166699877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea007f10 msgr2=0x55eae9ff8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea007f10 0x55eae9ff8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 -- 172.21.15.5:0/166699877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 msgr2=0x55eaea00ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.713+0000 7f807ffff700 1 -- 172.21.15.5:0/166699877 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eae9da41f0 con 0x55eaea0042d0 2023-07-21T19:11:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.714+0000 7f807ffff700 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f806c000c00 tx=0x7f806c011af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94382cdb9733ad87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.714+0000 7f806b7fe700 1 -- 172.21.15.5:0/166699877 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c016070 con 0x55eaea0042d0 2023-07-21T19:11:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.714+0000 7f806b7fe700 1 -- 172.21.15.5:0/166699877 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f806c00b480 con 0x55eaea0042d0 2023-07-21T19:11:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.714+0000 7f806b7fe700 1 -- 172.21.15.5:0/166699877 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806c010700 con 0x55eaea0042d0 2023-07-21T19:11:24.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- 172.21.15.5:0/166699877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 msgr2=0x55eaea007020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f806c000c00 tx=0x7f806c011af0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- 172.21.15.5:0/166699877 shutdown_connections 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea007f10 0x55eae9ff8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eae9e5a6b0 0x55eaea00ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 --2- 172.21.15.5:0/166699877 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eaea0042d0 0x55eaea007020 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- 172.21.15.5:0/166699877 >> 172.21.15.5:0/166699877 conn(0x55eae9ff3be0 msgr2=0x55eaea007800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- 172.21.15.5:0/166699877 shutdown_connections 2023-07-21T19:11:24.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- 172.21.15.5:0/166699877 wait complete. 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.715+0000 7f809abbab80 1 -- start start 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 Processor -- start 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 0x55eaea001080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaea007f10 0x55eae9fff860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9fffe40 con 0x55eae9e5a6b0 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eae9ffffb0 con 0x55eaea007f10 2023-07-21T19:11:24.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eaea000170 con 0x55eaea0042d0 2023-07-21T19:11:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.716+0000 7f809abbab80 10 client.0 _ll_register_callbacks cb 0x7f806c00b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57116/0 (socket says 172.21.15.5:57116) 2023-07-21T19:11:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 -- 172.21.15.5:0/3035406231 learned_addr learned my addr 172.21.15.5:0/3035406231 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:24.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f807ffff700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 0x55eaea001080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 -- 172.21.15.5:0/3035406231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 msgr2=0x55eaea001080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 0x55eaea001080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 -- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaea007f10 msgr2=0x55eae9fff860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaea007f10 0x55eae9fff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 -- 172.21.15.5:0/3035406231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eae9d66c00 con 0x55eae9e5a6b0 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f807ffff700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 0x55eaea001080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:24.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f808579c700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f807800b0a0 tx=0x7f80780052b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1142878408940b83 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.717+0000 7f805ffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f807800e040 con 0x55eae9e5a6b0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8078009460 con 0x55eae9e5a6b0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8078005750 con 0x55eae9e5a6b0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8078029040 con 0x55eae9e5a6b0 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f809abbab80 10 client.16500 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:24.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55eae9d7c250 con 0x55eae9e5a6b0 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8078009460 con 0x55eae9e5a6b0 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8078019040 con 0x55eae9e5a6b0 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f809abbab80 10 client.16500 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.718+0000 7f809abbab80 10 client.16500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.719+0000 7f809abbab80 20 client.16500 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.719+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea06c820 0x55eaea06ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:24.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.719+0000 7f809abbab80 4 client.16500 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:11:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.719+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eae9f0aff0 con 0x55eaea06c820 2023-07-21T19:11:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.719+0000 7f807ffff700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea06c820 0x55eaea06ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.720+0000 7f805ffff700 10 client.16500 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:24.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.720+0000 7f807ffff700 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea06c820 0x55eaea06ec30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52baf25d941184b server_cookie=5a9166779578288a in_seq=0 out_seq=0 2023-07-21T19:11:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.732+0000 7f805ffff700 1 -- 172.21.15.5:0/3035406231 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eae9f0aff0 con 0x55eaea06c820 2023-07-21T19:11:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.732+0000 7f805ffff700 10 client.16500 handle_command_reply: tid=0 2023-07-21T19:11:24.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.732+0000 7f809abbab80 1 client.16500 shutdown 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.732+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 msgr2=0x55eaea000b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.732+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f807800b0a0 tx=0x7f80780052b0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 shutdown_connections 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eaea007f10 0x55eae9fff860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eaea06c820 0x55eaea06ec30 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eae9e5a6b0 0x55eaea000b40 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 --2- 172.21.15.5:0/3035406231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eaea0042d0 0x55eaea001080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 >> 172.21.15.5:0/3035406231 conn(0x55eae9ff3be0 msgr2=0x55eaea003280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f805ffff700 0 client.16500 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 shutdown_connections 2023-07-21T19:11:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 1 -- 172.21.15.5:0/3035406231 wait complete. 2023-07-21T19:11:24.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.733+0000 7f809abbab80 20 client.16500 trim_cache size 0 max 16384 2023-07-21T19:11:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:11:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:11:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:11:24.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:11:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 msgr2=0x7f80700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 0x7f80700a37f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f80740ca680 tx=0x7f80740ccf50 comp rx=0 tx=0).stop 2023-07-21T19:11:24.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 msgr2=0x7f8080197ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f807400e9a0 tx=0x7f807400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 shutdown_connections 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80700a1340 0x7f80700a37f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f808010c3f0 0x7f808019ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8080072850 0x7f8080197ea0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 --2- 172.21.15.5:0/972947585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8080073160 0x7f80801983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 >> 172.21.15.5:0/972947585 conn(0x7f80800fb650 msgr2=0x7f808010e4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 shutdown_connections 2023-07-21T19:11:24.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:24.734+0000 7f80871ff700 1 -- 172.21.15.5:0/972947585 wait complete. 2023-07-21T19:11:24.880 INFO:teuthology.orchestra.run.smithi005.stderr:12700000 of 50000000 tuples (25%) done (elapsed 202.44 s, remaining 594.56 s) 2023-07-21T19:11:25.666 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:25 smithi005 ceph-mon[126460]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 101 op/s 2023-07-21T19:11:25.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:25 smithi156 ceph-mon[128316]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 101 op/s 2023-07-21T19:11:25.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:25 smithi101 ceph-mon[128136]: pgmap v117: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 24 MiB/s wr, 101 op/s 2023-07-21T19:11:26.175 INFO:teuthology.orchestra.run.smithi005.stderr:12800000 of 50000000 tuples (25%) done (elapsed 203.73 s, remaining 592.10 s) 2023-07-21T19:11:27.564 INFO:teuthology.orchestra.run.smithi005.stderr:12900000 of 50000000 tuples (25%) done (elapsed 205.12 s, remaining 589.93 s) 2023-07-21T19:11:27.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:27 smithi005 ceph-mon[126460]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:11:27.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:27 smithi156 ceph-mon[128316]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:11:27.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:27 smithi101 ceph-mon[128136]: pgmap v118: 97 pgs: 97 active+clean; 2.9 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:11:28.853 INFO:teuthology.orchestra.run.smithi005.stderr:13000000 of 50000000 tuples (26%) done (elapsed 206.41 s, remaining 587.48 s) 2023-07-21T19:11:29.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:29 smithi005 ceph-mon[126460]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 34 MiB/s wr, 109 op/s 2023-07-21T19:11:29.678 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:29 smithi156 ceph-mon[128316]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 34 MiB/s wr, 109 op/s 2023-07-21T19:11:29.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:29 smithi101 ceph-mon[128136]: pgmap v119: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 34 MiB/s wr, 109 op/s 2023-07-21T19:11:30.134 INFO:teuthology.orchestra.run.smithi005.stderr:13100000 of 50000000 tuples (26%) done (elapsed 207.69 s, remaining 585.03 s) 2023-07-21T19:11:31.521 INFO:teuthology.orchestra.run.smithi005.stderr:13200000 of 50000000 tuples (26%) done (elapsed 209.08 s, remaining 582.89 s) 2023-07-21T19:11:31.669 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:31 smithi005 ceph-mon[126460]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:31.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:31 smithi156 ceph-mon[128316]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:31.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:31 smithi101 ceph-mon[128136]: pgmap v120: 97 pgs: 97 active+clean; 3.0 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:32.791 INFO:teuthology.orchestra.run.smithi005.stderr:13300000 of 50000000 tuples (26%) done (elapsed 210.35 s, remaining 580.44 s) 2023-07-21T19:11:33.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:33 smithi005 ceph-mon[126460]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 24 op/s 2023-07-21T19:11:33.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:33 smithi156 ceph-mon[128316]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 24 op/s 2023-07-21T19:11:33.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:33 smithi101 ceph-mon[128136]: pgmap v121: 97 pgs: 97 active+clean; 3.1 GiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 34 MiB/s wr, 24 op/s 2023-07-21T19:11:34.060 INFO:teuthology.orchestra.run.smithi005.stderr:13400000 of 50000000 tuples (26%) done (elapsed 211.62 s, remaining 578.00 s) 2023-07-21T19:11:35.665 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:35 smithi005 ceph-mon[126460]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:35.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:35 smithi156 ceph-mon[128316]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:35.812 INFO:teuthology.orchestra.run.smithi005.stderr:13500000 of 50000000 tuples (27%) done (elapsed 213.37 s, remaining 576.89 s) 2023-07-21T19:11:35.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:35 smithi101 ceph-mon[128136]: pgmap v122: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:11:37.076 INFO:teuthology.orchestra.run.smithi005.stderr:13600000 of 50000000 tuples (27%) done (elapsed 214.64 s, remaining 574.47 s) 2023-07-21T19:11:37.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:37 smithi156 ceph-mon[128316]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:11:37.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:37 smithi101 ceph-mon[128136]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:11:37.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:37 smithi005 ceph-mon[126460]: pgmap v123: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:11:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.912+0000 7f6a23bff700 1 -- 172.21.15.5:0/3087376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c105b10 msgr2=0x7f6a1c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.912+0000 7f6a23bff700 1 --2- 172.21.15.5:0/3087376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c105b10 0x7f6a1c105ee0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f6a18009990 tx=0x7f6a18009ca0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 -- 172.21.15.5:0/3087376533 shutdown_connections 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 --2- 172.21.15.5:0/3087376533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 0x7f6a1c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 --2- 172.21.15.5:0/3087376533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c105b10 0x7f6a1c105ee0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 --2- 172.21.15.5:0/3087376533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 -- 172.21.15.5:0/3087376533 >> 172.21.15.5:0/3087376533 conn(0x7f6a1c0fb600 msgr2=0x7f6a1c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 -- 172.21.15.5:0/3087376533 shutdown_connections 2023-07-21T19:11:37.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.913+0000 7f6a23bff700 1 -- 172.21.15.5:0/3087376533 wait complete. 2023-07-21T19:11:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.914+0000 7f6a23bff700 1 Processor -- start 2023-07-21T19:11:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.914+0000 7f6a23bff700 1 -- start start 2023-07-21T19:11:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 0x7f6a1c198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c105b10 0x7f6a1c19c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c19cf40 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c19d0b0 con 0x7f6a1c1004b0 2023-07-21T19:11:37.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a23bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a1c19d270 con 0x7f6a1c105b10 2023-07-21T19:11:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45728/0 (socket says 172.21.15.5:45728) 2023-07-21T19:11:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a2199b700 1 -- 172.21.15.5:0/856408784 learned_addr learned my addr 172.21.15.5:0/856408784 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a2119a700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 0x7f6a1c198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.915+0000 7f6a2219c700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c105b10 0x7f6a1c19c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 -- 172.21.15.5:0/856408784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c105b10 msgr2=0x7f6a1c19c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c105b10 0x7f6a1c19c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 -- 172.21.15.5:0/856408784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 msgr2=0x7f6a1c198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 0x7f6a1c198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 -- 172.21.15.5:0/856408784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a180095f0 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a2199b700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000c00 tx=0x7f6a18004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a12ffd700 1 -- 172.21.15.5:0/856408784 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1801d070 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a12ffd700 1 -- 172.21.15.5:0/856408784 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a18022470 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a12ffd700 1 -- 172.21.15.5:0/856408784 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a18017650 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.916+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a1c19d4f0 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.917+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1a4d90 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.921+0000 7f6a12ffd700 1 -- 172.21.15.5:0/856408784 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6a180225e0 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.922+0000 7f6a375d1b80 1 Processor -- start 2023-07-21T19:11:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.922+0000 7f6a375d1b80 1 -- start start 2023-07-21T19:11:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.922+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a42d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.922+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859a3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.922+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a3d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859942c50 con 0x55c859a432d0 2023-07-21T19:11:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8599959e0 con 0x55c859a3d580 2023-07-21T19:11:37.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c85998ee70 con 0x55c8598996b0 2023-07-21T19:11:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2219c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a42d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a12ffd700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 0x7f6a080a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2219c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a42d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45778/0 (socket says 172.21.15.5:45778) 2023-07-21T19:11:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2219c700 1 -- 172.21.15.5:0/2037852176 learned_addr learned my addr 172.21.15.5:0/2037852176 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2199b700 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859a3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2119a700 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a3d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a12ffd700 1 -- 172.21.15.5:0/856408784 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6a180ce070 con 0x7f6a1c0ffb00 2023-07-21T19:11:37.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2119a700 1 -- 172.21.15.5:0/2037852176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 msgr2=0x55c859a42d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2119a700 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a42d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2119a700 1 -- 172.21.15.5:0/2037852176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 msgr2=0x55c859a3d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.923+0000 7f6a2119a700 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859a3d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a2119a700 1 -- 172.21.15.5:0/2037852176 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8597bcd80 con 0x55c859a3d580 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a2219c700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 0x7f6a080a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a2119a700 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a3d9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009c80 tx=0x7f6a0c009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ea10ed7a526e173 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a07fff700 1 -- 172.21.15.5:0/2037852176 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c011cb0 con 0x55c859a3d580 2023-07-21T19:11:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a07fff700 1 -- 172.21.15.5:0/2037852176 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a0c011e10 con 0x55c859a3d580 2023-07-21T19:11:37.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.924+0000 7f6a07fff700 1 -- 172.21.15.5:0/2037852176 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a0c005280 con 0x55c859a3d580 2023-07-21T19:11:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 -- 172.21.15.5:0/2037852176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 msgr2=0x55c859a3d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a2219c700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 0x7f6a080a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a140094b0 tx=0x7f6a14004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a3d9d0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6a0c009c80 tx=0x7f6a0c009f90 comp rx=0 tx=0).stop 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 -- 172.21.15.5:0/2037852176 shutdown_connections 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a3d9d0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a42d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/2037852176 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859a3d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 -- 172.21.15.5:0/2037852176 >> 172.21.15.5:0/2037852176 conn(0x55c859a32be0 msgr2=0x55c859a37f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 -- 172.21.15.5:0/2037852176 shutdown_connections 2023-07-21T19:11:37.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.925+0000 7f6a375d1b80 1 -- 172.21.15.5:0/2037852176 wait complete. 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.926+0000 7f6a375d1b80 1 -- start start 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.926+0000 7f6a375d1b80 1 Processor -- start 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a49610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a46820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 0x55c859a46f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c85992ea60 con 0x55c8598996b0 2023-07-21T19:11:37.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c85992e760 con 0x55c859a3d580 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8598128c0 con 0x55c859a432d0 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a375d1b80 10 client.0 _ll_register_callbacks cb 0x55c85992c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a49610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a2119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a46820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a2219c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 0x55c859a46f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a49610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45742/0 (socket says 172.21.15.5:45742) 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.927+0000 7f6a2199b700 1 -- 172.21.15.5:0/1643927935 learned_addr learned my addr 172.21.15.5:0/1643927935 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:37.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 -- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 msgr2=0x55c859a46820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a46820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 -- 172.21.15.5:0/1643927935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 msgr2=0x55c859a49610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a49610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 -- 172.21.15.5:0/1643927935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a0c0098e0 con 0x55c859a432d0 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a2219c700 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 0x55c859a46f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a14007550 tx=0x7f6a1400b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f15de3415071ff27 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.928+0000 7f6a057fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1402b070 con 0x55c859a432d0 2023-07-21T19:11:37.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6a14017e30 con 0x55c859a432d0 2023-07-21T19:11:37.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8598e8490 con 0x55c859a432d0 2023-07-21T19:11:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a1400e040 con 0x55c859a432d0 2023-07-21T19:11:37.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a375d1b80 10 client.26188 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.929+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c8597a5c00 con 0x55c859a432d0 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c8598e8490 con 0x55c859a432d0 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a14030460 con 0x55c859a432d0 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 10 client.26188 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 10 client.26188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 20 client.26188 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:37.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859a56bc0 0x55c859a58fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 4 client.26188 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c859987e50 con 0x55c859a56bc0 2023-07-21T19:11:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.930+0000 7f6a2199b700 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859a56bc0 0x55c859a58fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.931+0000 7f6a2199b700 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859a56bc0 0x55c859a58fd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7df62ea36340535 server_cookie=b630205f4a7f511 in_seq=0 out_seq=0 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.931+0000 7f6a057fa700 10 client.26188 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a057fa700 1 -- 172.21.15.5:0/1643927935 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c859987e50 con 0x55c859a56bc0 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a057fa700 10 client.26188 handle_command_reply: tid=0 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 client.26188 shutdown 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 msgr2=0x55c859a46f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 0x55c859a46f60 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6a14007550 tx=0x7f6a1400b040 comp rx=0 tx=0).stop 2023-07-21T19:11:37.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 shutdown_connections 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a432d0 0x55c859a46f60 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859a56bc0 0x55c859a58fd0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a49610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a057fa700 0 client.26188 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/1643927935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a46820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 >> 172.21.15.5:0/1643927935 conn(0x55c859a32be0 msgr2=0x55c859a374e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.932+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 shutdown_connections 2023-07-21T19:11:37.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.933+0000 7f6a375d1b80 1 -- 172.21.15.5:0/1643927935 wait complete. 2023-07-21T19:11:37.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.933+0000 7f6a375d1b80 20 client.26188 trim_cache size 0 max 16384 2023-07-21T19:11:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.951+0000 7f6a375d1b80 1 Processor -- start 2023-07-21T19:11:37.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.951+0000 7f6a375d1b80 1 -- start start 2023-07-21T19:11:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 0x55c859a4dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 0x55c859a49140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859a53520 con 0x55c8598996b0 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859a2fbb0 con 0x55c859a432d0 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859a39800 con 0x55c859a3d580 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45748/0 (socket says 172.21.15.5:45748) 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.952+0000 7f6a2199b700 1 -- 172.21.15.5:0/3558403623 learned_addr learned my addr 172.21.15.5:0/3558403623 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2219c700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 0x55c859a49140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 -- 172.21.15.5:0/3558403623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 msgr2=0x55c859a4dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2119a700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 0x55c859a4dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 0x55c859a4dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 -- 172.21.15.5:0/3558403623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 msgr2=0x55c859a49140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 0x55c859a49140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 -- 172.21.15.5:0/3558403623 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8597e31f0 con 0x55c8598996b0 2023-07-21T19:11:37.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2119a700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 0x55c859a4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2219c700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 0x55c859a49140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a2199b700 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6a180a11f0 tx=0x7f6a180a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a6ab3077c8a9ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.953+0000 7f6a10bff700 1 -- 172.21.15.5:0/3558403623 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1809ab00 con 0x55c8598996b0 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.954+0000 7f6a10bff700 1 -- 172.21.15.5:0/3558403623 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a180228c0 con 0x55c8598996b0 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.954+0000 7f6a10bff700 1 -- 172.21.15.5:0/3558403623 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a180cba10 con 0x55c8598996b0 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.954+0000 7f6a375d1b80 1 -- 172.21.15.5:0/3558403623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 msgr2=0x55c859a3d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.954+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f6a180a11f0 tx=0x7f6a180a1040 comp rx=0 tx=0).stop 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 -- 172.21.15.5:0/3558403623 shutdown_connections 2023-07-21T19:11:37.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a432d0 0x55c859a49140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8598996b0 0x55c859a3d040 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/3558403623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c859a3d580 0x55c859a4dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 -- 172.21.15.5:0/3558403623 >> 172.21.15.5:0/3558403623 conn(0x55c859a32be0 msgr2=0x55c859a3d950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 -- 172.21.15.5:0/3558403623 shutdown_connections 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 -- 172.21.15.5:0/3558403623 wait complete. 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 -- start start 2023-07-21T19:11:37.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.955+0000 7f6a375d1b80 1 Processor -- start 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a4c380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859aa9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859aa9fc0 con 0x55c859a432d0 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859aaa100 con 0x55c859a3d580 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c859aaa240 con 0x55c8598996b0 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a375d1b80 10 client.0 _ll_register_callbacks cb 0x7f6a14026d60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:37.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a2199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45822/0 (socket says 172.21.15.5:45822) 2023-07-21T19:11:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a2199b700 1 -- 172.21.15.5:0/4214918751 learned_addr learned my addr 172.21.15.5:0/4214918751 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.956+0000 7f6a2119a700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a4c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2219c700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859aa9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 -- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 msgr2=0x55c859a4c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a4c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 -- 172.21.15.5:0/4214918751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 msgr2=0x55c859aa9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859aa9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 -- 172.21.15.5:0/4214918751 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a1c1a4d90 con 0x55c8598996b0 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a2199b700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a180976a0 tx=0x7f6a1809f030 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f58cc54203bc6bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a057fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a1809b1a0 con 0x55c8598996b0 2023-07-21T19:11:37.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.957+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6a1809f1d0 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a0c011e10 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a180a3560 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 10 client.26191 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55c8597bb250 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a180cb020 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6a1809a690 con 0x55c8598996b0 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 10 client.26191 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:37.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 10 client.26191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 20 client.26191 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859aab3a0 0x55c859aad780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 4 client.26191 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.958+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c859949ff0 con 0x55c859aab3a0 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.959+0000 7f6a2119a700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859aab3a0 0x55c859aad780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.959+0000 7f6a2119a700 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859aab3a0 0x55c859aad780 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56d88ba76a107499 server_cookie=17892df4002b7e53 in_seq=0 out_seq=0 2023-07-21T19:11:37.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.959+0000 7f6a057fa700 10 client.26191 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.972+0000 7f6a057fa700 1 -- 172.21.15.5:0/4214918751 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c859949ff0 con 0x55c859aab3a0 2023-07-21T19:11:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a057fa700 10 client.26191 handle_command_reply: tid=0 2023-07-21T19:11:37.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 client.26191 shutdown 2023-07-21T19:11:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 msgr2=0x55c859a4be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6a180976a0 tx=0x7f6a1809f030 comp rx=0 tx=0).stop 2023-07-21T19:11:37.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 shutdown_connections 2023-07-21T19:11:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c859aab3a0 0x55c859aad780 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c859a432d0 0x55c859aa9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8598996b0 0x55c859a4be40 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 --2- 172.21.15.5:0/4214918751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c859a3d580 0x55c859a4c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a057fa700 0 client.26191 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 >> 172.21.15.5:0/4214918751 conn(0x55c859a32be0 msgr2=0x55c859a46030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 shutdown_connections 2023-07-21T19:11:37.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 1 -- 172.21.15.5:0/4214918751 wait complete. 2023-07-21T19:11:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.973+0000 7f6a375d1b80 20 client.26191 trim_cache size 0 max 16384 2023-07-21T19:11:37.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:11:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:11:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:11:37.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:11:37.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 msgr2=0x7f6a080a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 0x7f6a080a37f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f6a140094b0 tx=0x7f6a14004ad0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 msgr2=0x7f6a1c197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:37.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f6a18000c00 tx=0x7f6a18004730 comp rx=0 tx=0).stop 2023-07-21T19:11:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 shutdown_connections 2023-07-21T19:11:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a1c105b10 0x7f6a1c19c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a080a1340 0x7f6a080a37f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a1c0ffb00 0x7f6a1c197d50 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 --2- 172.21.15.5:0/856408784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a1c1004b0 0x7f6a1c198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 >> 172.21.15.5:0/856408784 conn(0x7f6a1c0fb600 msgr2=0x7f6a1c10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:37.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 shutdown_connections 2023-07-21T19:11:37.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:37.974+0000 7f6a23bff700 1 -- 172.21.15.5:0/856408784 wait complete. 2023-07-21T19:11:38.537 INFO:teuthology.orchestra.run.smithi005.stderr:13700000 of 50000000 tuples (27%) done (elapsed 216.10 s, remaining 572.57 s) 2023-07-21T19:11:39.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:39 smithi156 ceph-mon[128316]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:11:39.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:39 smithi101 ceph-mon[128136]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:11:39.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:39 smithi005 ceph-mon[126460]: pgmap v124: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:11:40.001 INFO:teuthology.orchestra.run.smithi005.stderr:13800000 of 50000000 tuples (27%) done (elapsed 217.56 s, remaining 570.70 s) 2023-07-21T19:11:41.286 INFO:teuthology.orchestra.run.smithi005.stderr:13900000 of 50000000 tuples (27%) done (elapsed 218.84 s, remaining 568.37 s) 2023-07-21T19:11:41.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:41 smithi156 ceph-mon[128316]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:11:41.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:41 smithi101 ceph-mon[128136]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:11:41.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:41 smithi005 ceph-mon[126460]: pgmap v125: 97 pgs: 97 active+clean; 3.1 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:11:42.578 INFO:teuthology.orchestra.run.smithi005.stderr:14000000 of 50000000 tuples (28%) done (elapsed 220.14 s, remaining 566.07 s) 2023-07-21T19:11:43.677 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:43 smithi156 ceph-mon[128316]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 31 op/s 2023-07-21T19:11:43.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:43 smithi101 ceph-mon[128136]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 31 op/s 2023-07-21T19:11:43.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:43 smithi005 ceph-mon[126460]: pgmap v126: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 31 MiB/s wr, 31 op/s 2023-07-21T19:11:44.536 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:11:44.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 -- 172.21.15.5:0/1244957080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034810c3c0 msgr2=0x7f034810e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 --2- 172.21.15.5:0/1244957080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034810c3c0 0x7f034810e7a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f034000b0a0 tx=0x7f034000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 -- 172.21.15.5:0/1244957080 shutdown_connections 2023-07-21T19:11:44.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 --2- 172.21.15.5:0/1244957080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f034810c3c0 0x7f034810e7a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 --2- 172.21.15.5:0/1244957080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 --2- 172.21.15.5:0/1244957080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0348073140 0x7f034810be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 -- 172.21.15.5:0/1244957080 >> 172.21.15.5:0/1244957080 conn(0x7f03480fb600 msgr2=0x7f03480fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 -- 172.21.15.5:0/1244957080 shutdown_connections 2023-07-21T19:11:44.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.749+0000 7f034fdff700 1 -- 172.21.15.5:0/1244957080 wait complete. 2023-07-21T19:11:44.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.750+0000 7f034fdff700 1 Processor -- start 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.750+0000 7f034fdff700 1 -- start start 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f034810c3c0 0x7f034819c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034819cdd0 con 0x7f0348073140 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034819cf40 con 0x7f0348072830 2023-07-21T19:11:44.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f034819d100 con 0x7f034810c3c0 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45772/0 (socket says 172.21.15.5:45772) 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 -- 172.21.15.5:0/25163202 learned_addr learned my addr 172.21.15.5:0/25163202 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034e39c700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f034810c3c0 0x7f034819c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034db9b700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 -- 172.21.15.5:0/25163202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f034810c3c0 msgr2=0x7f034819c790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f034810c3c0 0x7f034819c790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 -- 172.21.15.5:0/25163202 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 msgr2=0x7f0348197df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348197df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034d39a700 1 -- 172.21.15.5:0/25163202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0340009d00 con 0x7f0348073140 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.751+0000 7f034db9b700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f034d39a700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f033800e9a0 tx=0x7f033800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f033effd700 1 -- 172.21.15.5:0/25163202 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338009790 con 0x7f0348073140 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f033effd700 1 -- 172.21.15.5:0/25163202 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f033800cbd0 con 0x7f0348073140 2023-07-21T19:11:44.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f033effd700 1 -- 172.21.15.5:0/25163202 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338018680 con 0x7f0348073140 2023-07-21T19:11:44.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f034819d3e0 con 0x7f0348073140 2023-07-21T19:11:44.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.752+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03481a4c20 con 0x7f0348073140 2023-07-21T19:11:44.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.755+0000 7f033effd700 1 -- 172.21.15.5:0/25163202 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f033800cd40 con 0x7f0348073140 2023-07-21T19:11:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 Processor -- start 2023-07-21T19:11:44.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f033effd700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 0x7f03340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 -- start start 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f033effd700 1 -- 172.21.15.5:0/25163202 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0338014070 con 0x7f0348073140 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f034e39c700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 0x7f03340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbfc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbfeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babaf7cb0 con 0x562babbf12d0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babb4aa40 con 0x562babbf1be0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.757+0000 7f036369db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babb43ed0 con 0x562babbfcad0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45782/0 (socket says 172.21.15.5:45782) 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 -- 172.21.15.5:0/3909439839 learned_addr learned my addr 172.21.15.5:0/3909439839 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034d39a700 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbfc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034e39c700 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbfeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034e39c700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 0x7f03340a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f034000b780 tx=0x7f03400061f0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 -- 172.21.15.5:0/3909439839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 msgr2=0x562babbfeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbfeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 -- 172.21.15.5:0/3909439839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 msgr2=0x562babbfc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbfc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 -- 172.21.15.5:0/3909439839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bab971de0 con 0x562babbf12d0 2023-07-21T19:11:44.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.758+0000 7f034db9b700 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0344009ce0 tx=0x7f034400e2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=968ec5991464cf34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f0333fff700 1 -- 172.21.15.5:0/3909439839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f034400eb10 con 0x562babbf12d0 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f0333fff700 1 -- 172.21.15.5:0/3909439839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f034400ec70 con 0x562babbf12d0 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f0333fff700 1 -- 172.21.15.5:0/3909439839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344010630 con 0x562babbf12d0 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 -- 172.21.15.5:0/3909439839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 msgr2=0x562babbf16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0344009ce0 tx=0x7f034400e2f0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 -- 172.21.15.5:0/3909439839 shutdown_connections 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbfeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbf16a0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 --2- 172.21.15.5:0/3909439839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbfc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 -- 172.21.15.5:0/3909439839 >> 172.21.15.5:0/3909439839 conn(0x562babbe7da0 msgr2=0x562babbed6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.759+0000 7f036369db80 1 -- 172.21.15.5:0/3909439839 shutdown_connections 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.760+0000 7f036369db80 1 -- 172.21.15.5:0/3909439839 wait complete. 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.760+0000 7f036369db80 1 -- start start 2023-07-21T19:11:44.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.760+0000 7f036369db80 1 Processor -- start 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbf9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babae3ac0 con 0x562babbf12d0 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babae37c0 con 0x562babbf1be0 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bab9c7920 con 0x562babbfcad0 2023-07-21T19:11:44.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f036369db80 10 client.0 _ll_register_callbacks cb 0x562babae1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45786/0 (socket says 172.21.15.5:45786) 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 -- 172.21.15.5:0/2420799342 learned_addr learned my addr 172.21.15.5:0/2420799342 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034e39c700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034d39a700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbf9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 -- 172.21.15.5:0/2420799342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 msgr2=0x562babbf9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 -- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 msgr2=0x562babbf9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.761+0000 7f034db9b700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbf9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f034db9b700 1 -- 172.21.15.5:0/2420799342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0344009940 con 0x562babbf12d0 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f034d39a700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbf9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f034db9b700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0344011b40 tx=0x7f0344011f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ce65ba41b83ed1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f03317fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344020070 con 0x562babbf12d0 2023-07-21T19:11:44.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0344011680 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562baba9d4f0 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0344011840 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f036369db80 10 client.16530 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.762+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562bab95ac60 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0344032020 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f0344004b70 con 0x562babbf12d0 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 10 client.16530 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:44.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 10 client.16530 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 20 client.16530 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc0c410 0x562babc0e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 4 client.16530 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:44.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.763+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562babb3ceb0 con 0x562babc0c410 2023-07-21T19:11:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.764+0000 7f034d39a700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc0c410 0x562babc0e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.765+0000 7f034d39a700 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc0c410 0x562babc0e820 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df5a283032a3e51d server_cookie=f9b8629889c45bde in_seq=0 out_seq=0 2023-07-21T19:11:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.765+0000 7f03317fa700 10 client.16530 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:44.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f03317fa700 1 -- 172.21.15.5:0/2420799342 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562babb3ceb0 con 0x562babc0c410 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f03317fa700 10 client.16530 handle_command_reply: tid=0 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 client.16530 shutdown 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 msgr2=0x562babbfc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f0344011b40 tx=0x7f0344011f50 comp rx=0 tx=0).stop 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 shutdown_connections 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc0c410 0x562babc0e820 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babbfc510 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 --2- 172.21.15.5:0/2420799342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf1be0 0x562babbf9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f03317fa700 0 client.16530 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 >> 172.21.15.5:0/2420799342 conn(0x562babbe7da0 msgr2=0x562babbed290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 shutdown_connections 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 1 -- 172.21.15.5:0/2420799342 wait complete. 2023-07-21T19:11:44.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.766+0000 7f036369db80 20 client.16530 trim_cache size 0 max 16384 2023-07-21T19:11:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 Processor -- start 2023-07-21T19:11:44.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 -- start start 2023-07-21T19:11:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 0x562babbfc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babbfa7f0 con 0x562babbf1be0 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babbe79a0 con 0x562babbf12d0 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f036369db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babbfad70 con 0x562babbfcad0 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f034d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f034d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45802/0 (socket says 172.21.15.5:45802) 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.784+0000 7f034d39a700 1 -- 172.21.15.5:0/2727784826 learned_addr learned my addr 172.21.15.5:0/2727784826 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:44.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034db9b700 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 0x562babbfc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 -- 172.21.15.5:0/2727784826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 msgr2=0x562babbf9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 -- 172.21.15.5:0/2727784826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 msgr2=0x562babbfc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 0x562babbfc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 -- 172.21.15.5:0/2727784826 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bab998250 con 0x562babbf1be0 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034db9b700 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 0x562babbfc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:44.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f034d39a700 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03380ca750 tx=0x7f0338010b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5982553d917cf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f033cbff700 1 -- 172.21.15.5:0/2727784826 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03380b6070 con 0x562babbf1be0 2023-07-21T19:11:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f033cbff700 1 -- 172.21.15.5:0/2727784826 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03380b1070 con 0x562babbf1be0 2023-07-21T19:11:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.785+0000 7f033cbff700 1 -- 172.21.15.5:0/2727784826 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0338099560 con 0x562babbf1be0 2023-07-21T19:11:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- 172.21.15.5:0/2727784826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 msgr2=0x562babbf8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f03380ca750 tx=0x7f0338010b20 comp rx=0 tx=0).stop 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- 172.21.15.5:0/2727784826 shutdown_connections 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbfcad0 0x562babbf9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbf12d0 0x562babbfc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 --2- 172.21.15.5:0/2727784826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf1be0 0x562babbf8fc0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- 172.21.15.5:0/2727784826 >> 172.21.15.5:0/2727784826 conn(0x562babbe7da0 msgr2=0x562babbf4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- 172.21.15.5:0/2727784826 shutdown_connections 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- 172.21.15.5:0/2727784826 wait complete. 2023-07-21T19:11:44.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.786+0000 7f036369db80 1 -- start start 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 Processor -- start 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbf1be0 0x562babc016f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbfcad0 0x562babc01c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babc021c0 con 0x562babbf12d0 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babc02330 con 0x562babbfcad0 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562babc024f0 con 0x562babbf1be0 2023-07-21T19:11:44.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f036369db80 10 client.0 _ll_register_callbacks cb 0x7f0338099e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f034db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45804/0 (socket says 172.21.15.5:45804) 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.787+0000 7f034db9b700 1 -- 172.21.15.5:0/3758049443 learned_addr learned my addr 172.21.15.5:0/3758049443 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034e39c700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbfcad0 0x562babc01c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034d39a700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbf1be0 0x562babc016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 -- 172.21.15.5:0/3758049443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbf1be0 msgr2=0x562babc016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbf1be0 0x562babc016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 -- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbfcad0 msgr2=0x562babc01c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbfcad0 0x562babc01c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 -- 172.21.15.5:0/3758049443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03481a4c20 con 0x562babbf12d0 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f034db9b700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03440059d0 tx=0x7f0344011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1a00afdbde6f168 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f03317fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0344020040 con 0x562babbf12d0 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f034400ec70 con 0x562babbf12d0 2023-07-21T19:11:44.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.788+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f0344000e40 con 0x562babbf12d0 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f034400e640 con 0x562babbf12d0 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 10 client.16536 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562bab9702b0 con 0x562babbf12d0 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f034400ec70 con 0x562babbf12d0 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f034402d610 con 0x562babbf12d0 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 10 client.16536 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 10 client.16536 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:44.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 20 client.16536 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc61370 0x562babc63780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 4 client.16536 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:11:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.789+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562babaff050 con 0x562babc61370 2023-07-21T19:11:44.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.790+0000 7f034d39a700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc61370 0x562babc63780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.790+0000 7f034d39a700 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc61370 0x562babc63780 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d2894ead1effe8d server_cookie=4b3060b77d7b2f44 in_seq=0 out_seq=0 2023-07-21T19:11:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.790+0000 7f03317fa700 10 client.16536 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.790+0000 7f03317fa700 1 -- 172.21.15.5:0/3758049443 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x562babaff050 con 0x562babc61370 2023-07-21T19:11:44.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.790+0000 7f03317fa700 10 client.16536 handle_command_reply: tid=0 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 client.16536 shutdown 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 msgr2=0x562babc03470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f03440059d0 tx=0x7f0344011930 comp rx=0 tx=0).stop 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 shutdown_connections 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562babbfcad0 0x562babc01c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562babc61370 0x562babc63780 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562babbf12d0 0x562babc03470 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 --2- 172.21.15.5:0/3758049443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562babbf1be0 0x562babc016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 >> 172.21.15.5:0/3758049443 conn(0x562babbe7da0 msgr2=0x562babbec8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f03317fa700 0 client.16536 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 shutdown_connections 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 1 -- 172.21.15.5:0/3758049443 wait complete. 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.791+0000 7f036369db80 20 client.16536 trim_cache size 0 max 16384 2023-07-21T19:11:44.805 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:44.806 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:11:44.806 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:11:44.806 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 msgr2=0x7f03340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 0x7f03340a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f034000b780 tx=0x7f03400061f0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 msgr2=0x7f0348198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f033800e9a0 tx=0x7f033800ed60 comp rx=0 tx=0).stop 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 shutdown_connections 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f034810c3c0 0x7f034819c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0348072830 0x7f0348197df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03340a1340 0x7f03340a37f0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 --2- 172.21.15.5:0/25163202 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0348073140 0x7f0348198330 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 >> 172.21.15.5:0/25163202 conn(0x7f03480fb600 msgr2=0x7f034810e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 shutdown_connections 2023-07-21T19:11:44.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:44.792+0000 7f034fdff700 1 -- 172.21.15.5:0/25163202 wait complete. 2023-07-21T19:11:44.815 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:11:44.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:11:45.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.013+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1296628556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc538100480 msgr2=0x7fc53810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.013+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1296628556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc538100480 0x7fc53810c910 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc53000b0a0 tx=0x7fc53000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1296628556 shutdown_connections 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1296628556 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc538100480 0x7fc53810c910 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1296628556 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 0x7fc538105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1296628556 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5380ffad0 0x7fc5380fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1296628556 >> 172.21.15.5:0/1296628556 conn(0x7fc5380fb590 msgr2=0x7fc5380fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1296628556 shutdown_connections 2023-07-21T19:11:45.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.014+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1296628556 wait complete. 2023-07-21T19:11:45.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 Processor -- start 2023-07-21T19:11:45.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 -- start start 2023-07-21T19:11:45.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 0x7fc538072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 0x7fc5381a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc538073580 con 0x7fc5380ffad0 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381a4260 con 0x7fc538105ae0 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc53e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5381a43a0 con 0x7fc538100480 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc5377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc5377fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45816/0 (socket says 172.21.15.5:45816) 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.015+0000 7fc5377fe700 1 -- 172.21.15.5:0/1147665578 learned_addr learned my addr 172.21.15.5:0/1147665578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc536ffd700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 0x7fc538072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc537fff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 0x7fc5381a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 -- 172.21.15.5:0/1147665578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 msgr2=0x7fc538072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 0x7fc538072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 -- 172.21.15.5:0/1147665578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 msgr2=0x7fc5381a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 0x7fc5381a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 -- 172.21.15.5:0/1147665578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc530009d00 con 0x7fc5380ffad0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc536ffd700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 0x7fc538072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc5377fe700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e9a0 tx=0x7fc52800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528009790 con 0x7fc5380ffad0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5381a4600 con 0x7fc5380ffad0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc52800cbd0 con 0x7fc5380ffad0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.016+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528018680 con 0x7fc5380ffad0 2023-07-21T19:11:45.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.017+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5381a4b30 con 0x7fc5380ffad0 2023-07-21T19:11:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.017+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc538067300 con 0x7fc5380ffad0 2023-07-21T19:11:45.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.021+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc52800cd40 con 0x7fc5380ffad0 2023-07-21T19:11:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.023+0000 7fc534ff9700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 0x7fc5200a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.023+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fc528014070 con 0x7fc5380ffad0 2023-07-21T19:11:45.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.023+0000 7fc536ffd700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 0x7fc5200a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.024+0000 7fc536ffd700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 0x7fc5200a37b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc538074000 tx=0x7fc52c009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.025+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc5280966f0 con 0x7fc5380ffad0 2023-07-21T19:11:45.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.282+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc5381a4de0 con 0x7fc5380ffad0 2023-07-21T19:11:45.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.283+0000 7fc534ff9700 1 -- 172.21.15.5:0/1147665578 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+10839 (secure 0 0 0) 0x7fc528099d10 con 0x7fc5380ffad0 2023-07-21T19:11:45.289 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:11:45.289 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":26,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:09:18.477788+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 msgr2=0x7fc5200a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 0x7fc5200a37b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc538074000 tx=0x7fc52c009260 comp rx=0 tx=0).stop 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 msgr2=0x7fc538072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fc52800e9a0 tx=0x7fc52800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 shutdown_connections 2023-07-21T19:11:45.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc538105ae0 0x7fc5381a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5200a1300 0x7fc5200a37b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5380ffad0 0x7fc538072810 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 --2- 172.21.15.5:0/1147665578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc538100480 0x7fc538072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 >> 172.21.15.5:0/1147665578 conn(0x7fc5380fb590 msgr2=0x7fc5381035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 shutdown_connections 2023-07-21T19:11:45.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.285+0000 7fc53e1ff700 1 -- 172.21.15.5:0/1147665578 wait complete. 2023-07-21T19:11:45.294 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 26 2023-07-21T19:11:45.313 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:11:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 -- 172.21.15.5:0/2304695654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 msgr2=0x7f02b410c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/2304695654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b410c4b0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00b0a0 tx=0x7f02ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 -- 172.21.15.5:0/2304695654 shutdown_connections 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/2304695654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b410c4b0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/2304695654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4100d00 0x7f02b41010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/2304695654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4101610 0x7f02b4109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 -- 172.21.15.5:0/2304695654 >> 172.21.15.5:0/2304695654 conn(0x7f02b40fa4b0 msgr2=0x7f02b40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 -- 172.21.15.5:0/2304695654 shutdown_connections 2023-07-21T19:11:45.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.513+0000 7f02bc5ff700 1 -- 172.21.15.5:0/2304695654 wait complete. 2023-07-21T19:11:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.514+0000 7f02bc5ff700 1 Processor -- start 2023-07-21T19:11:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.514+0000 7f02bc5ff700 1 -- start start 2023-07-21T19:11:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4100d00 0x7f02b419b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4101610 0x7f02b419bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41960a0 con 0x7f02b410a040 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b4196210 con 0x7f02b4100d00 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b41963d0 con 0x7f02b4101610 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02ba39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4100d00 0x7f02b419b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02b9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4101610 0x7f02b419bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45822/0 (socket says 172.21.15.5:45822) 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.515+0000 7f02bab9c700 1 -- 172.21.15.5:0/1339360454 learned_addr learned my addr 172.21.15.5:0/1339360454 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 -- 172.21.15.5:0/1339360454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4101610 msgr2=0x7f02b419bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4101610 0x7f02b419bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 -- 172.21.15.5:0/1339360454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4100d00 msgr2=0x7f02b419b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4100d00 0x7f02b419b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 -- 172.21.15.5:0/1339360454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02ac009d00 con 0x7f02b410a040 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00b070 tx=0x7f02ac00bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02ab7fe700 1 -- 172.21.15.5:0/1339360454 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac00e040 con 0x7f02b410a040 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b4196650 con 0x7f02b410a040 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02ab7fe700 1 -- 172.21.15.5:0/1339360454 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac003e10 con 0x7f02b410a040 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02ab7fe700 1 -- 172.21.15.5:0/1339360454 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac01dea0 con 0x7f02b410a040 2023-07-21T19:11:45.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.516+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b41a29b0 con 0x7f02b410a040 2023-07-21T19:11:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.519+0000 7f02ab7fe700 1 -- 172.21.15.5:0/1339360454 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f02ac01d480 con 0x7f02b410a040 2023-07-21T19:11:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 Processor -- start 2023-07-21T19:11:45.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 -- start start 2023-07-21T19:11:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02ab7fe700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 0x7f02a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02ab7fe700 1 -- 172.21.15.5:0/1339360454 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f02ac0cdfb0 con 0x7f02b410a040 2023-07-21T19:11:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84ecbd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ebc3cb0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ec16a40 con 0x55a84ecbdbe0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ec0fed0 con 0x55a84ecbd2d0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.521+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 0x7f02a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02ba39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84ecbd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45838/0 (socket says 172.21.15.5:45838) 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 -- 172.21.15.5:0/2970525113 learned_addr learned my addr 172.21.15.5:0/2970525113 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 -- 172.21.15.5:0/2970525113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 msgr2=0x55a84ecbd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84ecbd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 -- 172.21.15.5:0/2970525113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 msgr2=0x55a84ecc8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 -- 172.21.15.5:0/2970525113 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84ea3dde0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 0x7f02a00a3770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f02b410b680 tx=0x7f02a4009f80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.522+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0a2f70 tx=0x7f02ac09af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18af8430f396362 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02a8bfe700 1 -- 172.21.15.5:0/2970525113 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac09b0e0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02a8bfe700 1 -- 172.21.15.5:0/2970525113 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac09f110 con 0x55a84ecc8ad0 2023-07-21T19:11:45.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02a8bfe700 1 -- 172.21.15.5:0/2970525113 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac09b340 con 0x55a84ecc8ad0 2023-07-21T19:11:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2970525113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 msgr2=0x55a84eccaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f02ac0a2f70 tx=0x7f02ac09af80 comp rx=0 tx=0).stop 2023-07-21T19:11:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2970525113 shutdown_connections 2023-07-21T19:11:45.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84eccaeb0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84ecbd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2970525113 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2970525113 >> 172.21.15.5:0/2970525113 conn(0x55a84ecb3da0 msgr2=0x55a84ecb96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2970525113 shutdown_connections 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.523+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2970525113 wait complete. 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.524+0000 7f02cfe42b80 1 -- start start 2023-07-21T19:11:45.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.524+0000 7f02cfe42b80 1 Processor -- start 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 0x55a84ecd3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ebafac0 con 0x55a84ecbdbe0 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ebaf7c0 con 0x55a84ecbd2d0 2023-07-21T19:11:45.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ea93920 con 0x55a84ecc8ad0 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02cfe42b80 10 client.0 _ll_register_callbacks cb 0x55a84ebad760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45842/0 (socket says 172.21.15.5:45842) 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 -- 172.21.15.5:0/2162767999 learned_addr learned my addr 172.21.15.5:0/2162767999 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 0x55a84ecd3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 -- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 msgr2=0x55a84ecd3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 0x55a84ecd3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 -- 172.21.15.5:0/2162767999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 msgr2=0x55a84ecc8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02b9b9a700 1 -- 172.21.15.5:0/2162767999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b41a29b0 con 0x55a84ecbdbe0 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.525+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 0x55a84ecd3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f02a400f9a0 tx=0x7f02a40047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e0ee62a5029a25 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f029dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:45.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a4028660 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f02a4011e10 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a84eb694f0 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02a40287c0 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f02cfe42b80 10 client.16554 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.526+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a84ea26c60 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a84eb694f0 con 0x55a84ecbdbe0 2023-07-21T19:11:45.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f02a4031b10 con 0x55a84ecbdbe0 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 10 client.16554 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 10 client.16554 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 20 client.16554 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ecd8200 0x55a84ecda610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 4 client.16554 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a84ec08eb0 con 0x55a84ecd8200 2023-07-21T19:11:45.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.527+0000 7f02ba39b700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ecd8200 0x55a84ecda610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.528+0000 7f02ba39b700 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ecd8200 0x55a84ecda610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ba527ea292a88ec8 server_cookie=2407c382634d426 in_seq=0 out_seq=0 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.528+0000 7f029dffb700 10 client.16554 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.528+0000 7f029dffb700 1 -- 172.21.15.5:0/2162767999 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a84ec08eb0 con 0x55a84ecd8200 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f029dffb700 10 client.16554 handle_command_reply: tid=0 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 client.16554 shutdown 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 msgr2=0x55a84ecc93e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f02a400f9a0 tx=0x7f02a40047c0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 shutdown_connections 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecc8ad0 0x55a84ecd3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ecd8200 0x55a84ecda610 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2162767999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f029dffb700 0 client.16554 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:45.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 >> 172.21.15.5:0/2162767999 conn(0x55a84ecb3da0 msgr2=0x55a84ecb9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 shutdown_connections 2023-07-21T19:11:45.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2162767999 wait complete. 2023-07-21T19:11:45.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.529+0000 7f02cfe42b80 20 client.16554 trim_cache size 0 max 16384 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.542+0000 7f02cfe42b80 1 Processor -- start 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 -- start start 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ecba530 con 0x55a84ecc8ad0 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ecb3970 con 0x55a84ecbd2d0 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ecc8310 con 0x55a84ecbdbe0 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45858/0 (socket says 172.21.15.5:45858) 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02bab9c700 1 -- 172.21.15.5:0/2590491608 learned_addr learned my addr 172.21.15.5:0/2590491608 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02ba39b700 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.543+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 -- 172.21.15.5:0/2590491608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbdbe0 msgr2=0x55a84ecc93e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 -- 172.21.15.5:0/2590491608 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 msgr2=0x55a84ecc8ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 -- 172.21.15.5:0/2590491608 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a84ea64250 con 0x55a84ecc8ad0 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02bab9c700 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f02ac09ade0 tx=0x7f02ac007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8100bdbcab8801f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02a93ff700 1 -- 172.21.15.5:0/2590491608 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac09b160 con 0x55a84ecc8ad0 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02a93ff700 1 -- 172.21.15.5:0/2590491608 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac0a34a0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.544+0000 7f02a93ff700 1 -- 172.21.15.5:0/2590491608 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac09a830 con 0x55a84ecc8ad0 2023-07-21T19:11:45.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2590491608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 msgr2=0x55a84ecc3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f02ac09ade0 tx=0x7f02ac007930 comp rx=0 tx=0).stop 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2590491608 shutdown_connections 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ecc3030 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbd2d0 0x55a84ecc8ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/2590491608 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbdbe0 0x55a84ecc93e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2590491608 >> 172.21.15.5:0/2590491608 conn(0x55a84ecb3da0 msgr2=0x55a84ecd4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2590491608 shutdown_connections 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- 172.21.15.5:0/2590491608 wait complete. 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.545+0000 7f02cfe42b80 1 -- start start 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 Processor -- start 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84eccc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84eccc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ed255f0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ed25730 con 0x55a84ecbdbe0 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.546+0000 7f02cfe42b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a84ed258f0 con 0x55a84ecbd2d0 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02cfe42b80 10 client.0 _ll_register_callbacks cb 0x7f02a400f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45864/0 (socket says 172.21.15.5:45864) 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 -- 172.21.15.5:0/1802927945 learned_addr learned my addr 172.21.15.5:0/1802927945 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02ba39b700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84eccc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02b9b9a700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84eccc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 -- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 msgr2=0x55a84eccc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84eccc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 -- 172.21.15.5:0/1802927945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 msgr2=0x55a84eccc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84eccc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 -- 172.21.15.5:0/1802927945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a84ea3dde0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f02bab9c700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f02ac097b50 tx=0x7f02ac0099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2425b90c409949d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.547+0000 7f029dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02ac0b9280 con 0x55a84ecc8ad0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f02ac0b93e0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02ac09f110 con 0x55a84ecc8ad0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02ac066070 con 0x55a84ecc8ad0 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 10 client.16560 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:45.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55a84ea3c2b0 con 0x55a84ecc8ad0 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02ac09f110 con 0x55a84ecc8ad0 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f02ac0a3010 con 0x55a84ecc8ad0 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 10 client.16560 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 10 client.16560 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 20 client.16560 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:45.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ed2cb80 0x55a84ed2ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 4 client.16560 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.548+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a84ebcb050 con 0x55a84ed2cb80 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.549+0000 7f02ba39b700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ed2cb80 0x55a84ed2ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.549+0000 7f02ba39b700 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ed2cb80 0x55a84ed2ef60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=247187a4213aca16 server_cookie=ba365a4523ea74ca in_seq=0 out_seq=0 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.549+0000 7f029dffb700 10 client.16560 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:45.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f029dffb700 1 -- 172.21.15.5:0/1802927945 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a84ebcb050 con 0x55a84ed2cb80 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f029dffb700 10 client.16560 handle_command_reply: tid=0 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 client.16560 shutdown 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 msgr2=0x55a84ed250b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f02ac097b50 tx=0x7f02ac0099e0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 shutdown_connections 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a84ecc8ad0 0x55a84ed250b0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a84ecbd2d0 0x55a84eccc050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a84ed2cb80 0x55a84ed2ef60 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 --2- 172.21.15.5:0/1802927945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a84ecbdbe0 0x55a84eccc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 >> 172.21.15.5:0/1802927945 conn(0x55a84ecb3da0 msgr2=0x55a84ecc8ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f029dffb700 0 client.16560 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 shutdown_connections 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.550+0000 7f02cfe42b80 1 -- 172.21.15.5:0/1802927945 wait complete. 2023-07-21T19:11:45.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02cfe42b80 20 client.16560 trim_cache size 0 max 16384 2023-07-21T19:11:45.566 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:45.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 msgr2=0x7f02a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 0x7f02a00a3770 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f02b410b680 tx=0x7f02a4009f80 comp rx=0 tx=0).stop 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 msgr2=0x7f02b4195a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f02ac00b070 tx=0x7f02ac00bce0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 shutdown_connections 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a12c0 0x7f02a00a3770 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b410a040 0x7f02b4195a30 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b4100d00 0x7f02b419b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 --2- 172.21.15.5:0/1339360454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b4101610 0x7f02b419bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 >> 172.21.15.5:0/1339360454 conn(0x7f02b40fa4b0 msgr2=0x7f02b410c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 shutdown_connections 2023-07-21T19:11:45.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.551+0000 7f02bc5ff700 1 -- 172.21.15.5:0/1339360454 wait complete. 2023-07-21T19:11:45.572 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:45 smithi005 ceph-mon[126460]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:11:45.573 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:45 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1147665578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:45.575 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:11:45.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 -- 172.21.15.5:0/4161104046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af410c380 msgr2=0x7f7af410e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 --2- 172.21.15.5:0/4161104046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af410c380 0x7f7af410e760 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f7aec00b0a0 tx=0x7f7aec00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 -- 172.21.15.5:0/4161104046 shutdown_connections 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 --2- 172.21.15.5:0/4161104046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af410c380 0x7f7af410e760 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 --2- 172.21.15.5:0/4161104046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4072850 0x7f7af4072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 --2- 172.21.15.5:0/4161104046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af4073160 0x7f7af410be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 -- 172.21.15.5:0/4161104046 >> 172.21.15.5:0/4161104046 conn(0x7f7af40fb5e0 msgr2=0x7f7af40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 -- 172.21.15.5:0/4161104046 shutdown_connections 2023-07-21T19:11:45.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.775+0000 7f7afbfff700 1 -- 172.21.15.5:0/4161104046 wait complete. 2023-07-21T19:11:45.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.776+0000 7f7afbfff700 1 Processor -- start 2023-07-21T19:11:45.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.776+0000 7f7afbfff700 1 -- start start 2023-07-21T19:11:45.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4073160 0x7f7af4198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af410c380 0x7f7af419c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419ce00 con 0x7f7af4072850 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419cf70 con 0x7f7af4073160 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af419d130 con 0x7f7af410c380 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45882/0 (socket says 172.21.15.5:45882) 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 -- 172.21.15.5:0/1570844728 learned_addr learned my addr 172.21.15.5:0/1570844728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af959a700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4073160 0x7f7af4198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7afa59c700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af410c380 0x7f7af419c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 -- 172.21.15.5:0/1570844728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af410c380 msgr2=0x7f7af419c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af410c380 0x7f7af419c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 -- 172.21.15.5:0/1570844728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4073160 msgr2=0x7f7af4198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4073160 0x7f7af4198360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.777+0000 7f7af9d9b700 1 -- 172.21.15.5:0/1570844728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aec009d00 con 0x7f7af4072850 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7af000d950 tx=0x7f7af000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7aeaffd700 1 -- 172.21.15.5:0/1570844728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000f890 con 0x7f7af4072850 2023-07-21T19:11:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af419d410 con 0x7f7af4072850 2023-07-21T19:11:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7aeaffd700 1 -- 172.21.15.5:0/1570844728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7af000fe80 con 0x7f7af4072850 2023-07-21T19:11:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7aeaffd700 1 -- 172.21.15.5:0/1570844728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000e3b0 con 0x7f7af4072850 2023-07-21T19:11:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.778+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af41a4c50 con 0x7f7af4072850 2023-07-21T19:11:45.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.782+0000 7f7aeaffd700 1 -- 172.21.15.5:0/1570844728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7af0010460 con 0x7f7af4072850 2023-07-21T19:11:45.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 Processor -- start 2023-07-21T19:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 -- start start 2023-07-21T19:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7aeaffd700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 0x7f7ae00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7aeaffd700 1 -- 172.21.15.5:0/1570844728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7af00ccaf0 con 0x7f7af4072850 2023-07-21T19:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bc2d0 0x5597701bc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597700c2cb0 con 0x5597701bcbe0 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559770115a40 con 0x5597701bc2d0 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.783+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55977010eed0 con 0x5597701c7ad0 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7afa59c700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 0x7f7ae00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45888/0 (socket says 172.21.15.5:45888) 2023-07-21T19:11:45.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 -- 172.21.15.5:0/3658979165 learned_addr learned my addr 172.21.15.5:0/3658979165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bc2d0 0x5597701bc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 -- 172.21.15.5:0/3658979165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 msgr2=0x5597701c9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 -- 172.21.15.5:0/3658979165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bc2d0 msgr2=0x5597701bc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bc2d0 0x5597701bc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 -- 172.21.15.5:0/3658979165 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55976ff3cde0 con 0x5597701bcbe0 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7afa59c700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 0x7f7ae00a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec009300 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.784+0000 7f7af959a700 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009ca0 tx=0x7f7ae4011440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1110150f296d167d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.785+0000 7f7adffff700 1 -- 172.21.15.5:0/3658979165 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4011d20 con 0x5597701bcbe0 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.785+0000 7f7adffff700 1 -- 172.21.15.5:0/3658979165 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae4011e80 con 0x5597701bcbe0 2023-07-21T19:11:45.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.785+0000 7f7adffff700 1 -- 172.21.15.5:0/3658979165 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae400e6e0 con 0x5597701bcbe0 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.785+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/3658979165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 msgr2=0x5597701c7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.785+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4009ca0 tx=0x7f7ae4011440 comp rx=0 tx=0).stop 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/3658979165 shutdown_connections 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bc2d0 0x5597701bc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/3658979165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701c7590 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/3658979165 >> 172.21.15.5:0/3658979165 conn(0x5597701b2da0 msgr2=0x5597701b86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/3658979165 shutdown_connections 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/3658979165 wait complete. 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 -- start start 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.786+0000 7f7b0f8a7b80 1 Processor -- start 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bcbe0 0x5597701c4790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701c4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597700aeac0 con 0x5597701bc2d0 2023-07-21T19:11:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597700ae7c0 con 0x5597701c7ad0 2023-07-21T19:11:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55976ff92920 con 0x5597701bcbe0 2023-07-21T19:11:45.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7b0f8a7b80 10 client.0 _ll_register_callbacks cb 0x5597700ac760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45902/0 (socket says 172.21.15.5:45902) 2023-07-21T19:11:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7af9d9b700 1 -- 172.21.15.5:0/2064719416 learned_addr learned my addr 172.21.15.5:0/2064719416 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.787+0000 7f7af959a700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bcbe0 0x5597701c4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7afa59c700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701c4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 -- 172.21.15.5:0/2064719416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bcbe0 msgr2=0x5597701c4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bcbe0 0x5597701c4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 -- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 msgr2=0x5597701c4ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701c4ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 -- 172.21.15.5:0/2064719416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ae4009900 con 0x5597701bc2d0 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000c00 tx=0x7f7af00108a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ae060ec967d189 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7add7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:45.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af00a1fe0 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7af00b2200 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559770107eb0 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7af0099080 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7b0f8a7b80 10 client.16572 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.788+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55976ff25c60 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559770107eb0 con 0x5597701bc2d0 2023-07-21T19:11:45.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7af0095980 con 0x5597701bc2d0 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 10 client.16572 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 20 client.16572 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5597701d7400 0x5597701d9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 4 client.16572 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.789+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597700ca050 con 0x5597701d7400 2023-07-21T19:11:45.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.790+0000 7f7af959a700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5597701d7400 0x5597701d9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.790+0000 7f7af959a700 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5597701d7400 0x5597701d9810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7a99159ec65262f6 server_cookie=6cbbac07f7498a17 in_seq=0 out_seq=0 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.790+0000 7f7add7fa700 10 client.16572 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7add7fa700 1 -- 172.21.15.5:0/2064719416 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5597700ca050 con 0x5597701d7400 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7add7fa700 10 client.16572 handle_command_reply: tid=0 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 client.16572 shutdown 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 msgr2=0x5597701c7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f7af0000c00 tx=0x7f7af00108a0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 shutdown_connections 2023-07-21T19:11:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5597701d7400 0x5597701d9810 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701c4ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7add7fa700 0 client.16572 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:45.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c7580 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2064719416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bcbe0 0x5597701c4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 >> 172.21.15.5:0/2064719416 conn(0x5597701b2da0 msgr2=0x5597701b8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.791+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 shutdown_connections 2023-07-21T19:11:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.792+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2064719416 wait complete. 2023-07-21T19:11:45.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.792+0000 7f7b0f8a7b80 20 client.16572 trim_cache size 0 max 16384 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.805+0000 7f7b0f8a7b80 1 Processor -- start 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.805+0000 7f7b0f8a7b80 1 -- start start 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.805+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.805+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 0x5597701c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701b9d70 con 0x5597701bc2d0 2023-07-21T19:11:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701bd570 con 0x5597701bcbe0 2023-07-21T19:11:45.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701d4b10 con 0x5597701c7ad0 2023-07-21T19:11:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45914/0 (socket says 172.21.15.5:45914) 2023-07-21T19:11:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af9d9b700 1 -- 172.21.15.5:0/2245992517 learned_addr learned my addr 172.21.15.5:0/2245992517 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 0x5597701c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7afa59c700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 -- 172.21.15.5:0/2245992517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 msgr2=0x5597701c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 -- 172.21.15.5:0/2245992517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 msgr2=0x5597701c70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af959a700 1 -- 172.21.15.5:0/2245992517 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55976ff63250 con 0x5597701bcbe0 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7afa59c700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:45.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.806+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.807+0000 7f7af959a700 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 0x5597701c4030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae40096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7357ddb54ddbb586 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.807+0000 7f7ae8ff9700 1 -- 172.21.15.5:0/2245992517 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4011e60 con 0x5597701bcbe0 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.807+0000 7f7ae8ff9700 1 -- 172.21.15.5:0/2245992517 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7ae4005550 con 0x5597701bcbe0 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.807+0000 7f7ae8ff9700 1 -- 172.21.15.5:0/2245992517 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4019b10 con 0x5597701bcbe0 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2245992517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 msgr2=0x5597701c4030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 0x5597701c4030 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4000c00 tx=0x7f7ae40096d0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2245992517 shutdown_connections 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701c7ad0 0x5597701c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bc2d0 0x5597701c70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/2245992517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701bcbe0 0x5597701c4030 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2245992517 >> 172.21.15.5:0/2245992517 conn(0x5597701b2da0 msgr2=0x5597701bfa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2245992517 shutdown_connections 2023-07-21T19:11:45.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/2245992517 wait complete. 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 -- start start 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.808+0000 7f7b0f8a7b80 1 Processor -- start 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bc2d0 0x5597701ce480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701ccc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701cd250 con 0x5597701bcbe0 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701cd390 con 0x5597701c7ad0 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597701cd550 con 0x5597701bc2d0 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7b0f8a7b80 10 client.0 _ll_register_callbacks cb 0x7f7af0010760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45926/0 (socket says 172.21.15.5:45926) 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 -- 172.21.15.5:0/1931372537 learned_addr learned my addr 172.21.15.5:0/1931372537 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:45.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bc2d0 0x5597701ce480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 -- 172.21.15.5:0/1931372537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bc2d0 msgr2=0x5597701ce480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7afa59c700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701ccc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bc2d0 0x5597701ce480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 -- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 msgr2=0x5597701ccc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701ccc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.809+0000 7f7af959a700 1 -- 172.21.15.5:0/1931372537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55976ff25c60 con 0x5597701bcbe0 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7afa59c700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:45.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7af959a700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005740 tx=0x7f7ae40094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db054d7e06335192 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7add7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ae4037070 con 0x5597701bcbe0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597700684f0 con 0x5597701bcbe0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7ae4011250 con 0x5597701bcbe0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ae4005050 con 0x5597701bcbe0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7b0f8a7b80 10 client.16578 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.810+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x55976ff3cde0 con 0x5597701bcbe0 2023-07-21T19:11:45.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5597700684f0 con 0x5597701bcbe0 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7ae403a030 con 0x5597701bcbe0 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 10 client.16578 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 20 client.16578 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55977022c450 0x55977022e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 4 client.16578 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:45.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55976ff5df70 con 0x55977022c450 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.811+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55977022c450 0x55977022e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7add7fa700 10 client.16578 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7af9d9b700 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55977022c450 0x55977022e860 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=103fd76f5db2a70d server_cookie=c0562b0d30f36e89 in_seq=0 out_seq=0 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7add7fa700 1 -- 172.21.15.5:0/1931372537 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55976ff5df70 con 0x55977022c450 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7add7fa700 10 client.16578 handle_command_reply: tid=0 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 client.16578 shutdown 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 msgr2=0x5597701cc730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f7ae4005740 tx=0x7f7ae40094b0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 shutdown_connections 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55977022c450 0x55977022e860 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597701c7ad0 0x5597701ccc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597701bc2d0 0x5597701ce480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 --2- 172.21.15.5:0/1931372537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597701bcbe0 0x5597701cc730 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 >> 172.21.15.5:0/1931372537 conn(0x5597701b2da0 msgr2=0x5597701be3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7add7fa700 0 client.16578 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:11:45.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 shutdown_connections 2023-07-21T19:11:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.812+0000 7f7b0f8a7b80 1 -- 172.21.15.5:0/1931372537 wait complete. 2023-07-21T19:11:45.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7b0f8a7b80 20 client.16578 trim_cache size 0 max 16384 2023-07-21T19:11:45.827 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 msgr2=0x7f7ae00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 0x7f7ae00a3770 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f7aec009fd0 tx=0x7f7aec009300 comp rx=0 tx=0).stop 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 msgr2=0x7f7af4197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f7af000d950 tx=0x7f7af000dc60 comp rx=0 tx=0).stop 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 shutdown_connections 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7af410c380 0x7f7af419c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ae00a12c0 0x7f7ae00a3770 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7af4072850 0x7f7af4197e20 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 --2- 172.21.15.5:0/1570844728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7af4073160 0x7f7af4198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 >> 172.21.15.5:0/1570844728 conn(0x7f7af40fb5e0 msgr2=0x7f7af410e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 shutdown_connections 2023-07-21T19:11:45.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:45.813+0000 7f7afbfff700 1 -- 172.21.15.5:0/1570844728 wait complete. 2023-07-21T19:11:45.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:11:45.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:45 smithi101 ceph-mon[128136]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:11:45.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:45 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1147665578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:45.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:45 smithi156 ceph-mon[128316]: pgmap v127: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 88 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:11:45.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:45 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1147665578' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:11:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 -- 172.21.15.5:0/981674166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87600ffad0 msgr2=0x7f87600fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 --2- 172.21.15.5:0/981674166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87600ffad0 0x7f87600fff40 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f8754009990 tx=0x7f8754009ca0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 -- 172.21.15.5:0/981674166 shutdown_connections 2023-07-21T19:11:46.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 --2- 172.21.15.5:0/981674166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760100480 0x7f876010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 --2- 172.21.15.5:0/981674166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8760105ae0 0x7f8760105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 --2- 172.21.15.5:0/981674166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f87600ffad0 0x7f87600fff40 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 -- 172.21.15.5:0/981674166 >> 172.21.15.5:0/981674166 conn(0x7f87600fb5f0 msgr2=0x7f87600fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 -- 172.21.15.5:0/981674166 shutdown_connections 2023-07-21T19:11:46.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.037+0000 7f8765fff700 1 -- 172.21.15.5:0/981674166 wait complete. 2023-07-21T19:11:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.039+0000 7f8765fff700 1 Processor -- start 2023-07-21T19:11:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.039+0000 7f8765fff700 1 -- start start 2023-07-21T19:11:46.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87600ffad0 0x7f8760106990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105ae0 0x7f8760107410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87601a83b0 con 0x7f8760100480 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87601a8520 con 0x7f8760105ae0 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f8765fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87601a86e0 con 0x7f87600ffad0 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45944/0 (socket says 172.21.15.5:45944) 2023-07-21T19:11:46.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 -- 172.21.15.5:0/2104018340 learned_addr learned my addr 172.21.15.5:0/2104018340 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:46.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875f7fe700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87600ffad0 0x7f8760106990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875ffff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105ae0 0x7f8760107410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 -- 172.21.15.5:0/2104018340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87600ffad0 msgr2=0x7f8760106990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87600ffad0 0x7f8760106990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 -- 172.21.15.5:0/2104018340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105ae0 msgr2=0x7f8760107410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105ae0 0x7f8760107410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.040+0000 7f875effd700 1 -- 172.21.15.5:0/2104018340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87540095f0 con 0x7f8760100480 2023-07-21T19:11:46.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f875effd700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f87540036c0 tx=0x7f8754005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f875cff9700 1 -- 172.21.15.5:0/2104018340 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875401d070 con 0x7f8760100480 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f875cff9700 1 -- 172.21.15.5:0/2104018340 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8754003f40 con 0x7f8760100480 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87601a8960 con 0x7f8760100480 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f875cff9700 1 -- 172.21.15.5:0/2104018340 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8754022c40 con 0x7f8760100480 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.041+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87601a8d80 con 0x7f8760100480 2023-07-21T19:11:46.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.044+0000 7f875cff9700 1 -- 172.21.15.5:0/2104018340 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8754022470 con 0x7f8760100480 2023-07-21T19:11:46.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.045+0000 7f87798fbb80 1 Processor -- start 2023-07-21T19:11:46.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 -- start start 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875cff9700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 0x7f87480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeaba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeac5ad0 0x561afeac7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe9c0cb0 con 0x561afeababe0 2023-07-21T19:11:46.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afea13a40 con 0x561afeaba2d0 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875cff9700 1 -- 172.21.15.5:0/2104018340 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f87540ce220 con 0x7f8760100480 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f87798fbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afea0ced0 con 0x561afeac5ad0 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875ffff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 0x7f87480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeaba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45948/0 (socket says 172.21.15.5:45948) 2023-07-21T19:11:46.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.046+0000 7f875effd700 1 -- 172.21.15.5:0/2211245453 learned_addr learned my addr 172.21.15.5:0/2211245453 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:46.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875ffff700 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeac5ad0 0x561afeac7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 -- 172.21.15.5:0/2211245453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeac5ad0 msgr2=0x561afeac7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeac5ad0 0x561afeac7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 -- 172.21.15.5:0/2211245453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 msgr2=0x561afeaba6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeaba6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 -- 172.21.15.5:0/2211245453 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561afe83ade0 con 0x561afeababe0 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875f7fe700 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeaba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:46.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875effd700 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8754097850 tx=0x7f875409acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a9149b4e6b94700 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f875ffff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 0x7f87480a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87580060b0 tx=0x7f8758007a80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f8745ffb700 1 -- 172.21.15.5:0/2211245453 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87540ce030 con 0x561afeababe0 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f8745ffb700 1 -- 172.21.15.5:0/2211245453 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f875409b350 con 0x561afeababe0 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.047+0000 7f8745ffb700 1 -- 172.21.15.5:0/2211245453 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875409b5b0 con 0x561afeababe0 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 -- 172.21.15.5:0/2211245453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 msgr2=0x561afeac5590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f8754097850 tx=0x7f875409acb0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 -- 172.21.15.5:0/2211245453 shutdown_connections 2023-07-21T19:11:46.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeac5ad0 0x561afeac7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeaba6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 --2- 172.21.15.5:0/2211245453 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeababe0 0x561afeac5590 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 -- 172.21.15.5:0/2211245453 >> 172.21.15.5:0/2211245453 conn(0x561afeab0da0 msgr2=0x561afeab66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 -- 172.21.15.5:0/2211245453 shutdown_connections 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.048+0000 7f87798fbb80 1 -- 172.21.15.5:0/2211245453 wait complete. 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 -- start start 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 Processor -- start 2023-07-21T19:11:46.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeac5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeababe0 0x561afeac63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe9acac0 con 0x561afeac5ad0 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe9ac7c0 con 0x561afeaba2d0 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.049+0000 7f87798fbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afe890920 con 0x561afeababe0 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f87798fbb80 10 client.0 _ll_register_callbacks cb 0x561afe9aa760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:46.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45954/0 (socket says 172.21.15.5:45954) 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 -- 172.21.15.5:0/929291921 learned_addr learned my addr 172.21.15.5:0/929291921 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875f7fe700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeac5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875effd700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeababe0 0x561afeac63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 -- 172.21.15.5:0/929291921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeababe0 msgr2=0x561afeac63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeababe0 0x561afeac63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 -- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 msgr2=0x561afeac5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeac5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 -- 172.21.15.5:0/929291921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87500077c0 con 0x561afeac5ad0 2023-07-21T19:11:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f875ffff700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8758019b60 tx=0x7f8758004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=948066af21ab9f74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.050+0000 7f873effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:46.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758009110 con 0x561afeac5ad0 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561afe9664f0 con 0x561afeac5ad0 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8758018a00 con 0x561afeac5ad0 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8758019e70 con 0x561afeac5ad0 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f87798fbb80 10 client.16590 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7f87601a8d80 con 0x561afeac5ad0 2023-07-21T19:11:46.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561afe9664f0 con 0x561afeac5ad0 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.051+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8758024ca0 con 0x561afeac5ad0 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 10 client.16590 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 20 client.16590 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afead5190 0x561afead75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 4 client.16590 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561afea05eb0 con 0x561afead5190 2023-07-21T19:11:46.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.052+0000 7f875f7fe700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afead5190 0x561afead75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.053+0000 7f875f7fe700 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afead5190 0x561afead75a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3f4264c542fb1ed server_cookie=f546205ec8d9c630 in_seq=0 out_seq=0 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.053+0000 7f873effd700 10 client.16590 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.053+0000 7f873effd700 1 -- 172.21.15.5:0/929291921 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561afea05eb0 con 0x561afead5190 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f873effd700 10 client.16590 handle_command_reply: tid=0 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 client.16590 shutdown 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 msgr2=0x561afeac6920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f8758019b60 tx=0x7f8758004b40 comp rx=0 tx=0).stop 2023-07-21T19:11:46.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 shutdown_connections 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afead5190 0x561afead75a0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac6920 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeaba2d0 0x561afeac5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 --2- 172.21.15.5:0/929291921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeababe0 0x561afeac63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 >> 172.21.15.5:0/929291921 conn(0x561afeab0da0 msgr2=0x561afeab6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f873effd700 0 client.16590 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:46.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 shutdown_connections 2023-07-21T19:11:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 1 -- 172.21.15.5:0/929291921 wait complete. 2023-07-21T19:11:46.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.054+0000 7f87798fbb80 20 client.16590 trim_cache size 0 max 16384 2023-07-21T19:11:46.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 Processor -- start 2023-07-21T19:11:46.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 -- start start 2023-07-21T19:11:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afead0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeabbe70 con 0x561afeac5ad0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeab74d0 con 0x561afeababe0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f87798fbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeac5140 con 0x561afeaba2d0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45968/0 (socket says 172.21.15.5:45968) 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.068+0000 7f875ffff700 1 -- 172.21.15.5:0/3966003805 learned_addr learned my addr 172.21.15.5:0/3966003805 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875f7fe700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afead0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875effd700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 -- 172.21.15.5:0/3966003805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 msgr2=0x561afead0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afead0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 -- 172.21.15.5:0/3966003805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 msgr2=0x561afeac42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 -- 172.21.15.5:0/3966003805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561afe823c60 con 0x561afeac5ad0 2023-07-21T19:11:46.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875f7fe700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afead0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:46.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f875ffff700 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8758019b30 tx=0x7f8758009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ae156aad47d4e20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f87467fc700 1 -- 172.21.15.5:0/3966003805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758009a60 con 0x561afeac5ad0 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f87467fc700 1 -- 172.21.15.5:0/3966003805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f875800f070 con 0x561afeac5ad0 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.069+0000 7f87467fc700 1 -- 172.21.15.5:0/3966003805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758024670 con 0x561afeac5ad0 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- 172.21.15.5:0/3966003805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 msgr2=0x561afeac1da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f8758019b30 tx=0x7f8758009600 comp rx=0 tx=0).stop 2023-07-21T19:11:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- 172.21.15.5:0/3966003805 shutdown_connections 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac1da0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afead0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 --2- 172.21.15.5:0/3966003805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- 172.21.15.5:0/3966003805 >> 172.21.15.5:0/3966003805 conn(0x561afeab0da0 msgr2=0x561afeac6730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- 172.21.15.5:0/3966003805 shutdown_connections 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- 172.21.15.5:0/3966003805 wait complete. 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.070+0000 7f87798fbb80 1 -- start start 2023-07-21T19:11:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 Processor -- start 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afeacabc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeacb490 con 0x561afeac5ad0 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeac98c0 con 0x561afeababe0 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561afeac9a50 con 0x561afeaba2d0 2023-07-21T19:11:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f87798fbb80 10 client.0 _ll_register_callbacks cb 0x7f8758019710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.071+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:45976/0 (socket says 172.21.15.5:45976) 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 -- 172.21.15.5:0/1223363242 learned_addr learned my addr 172.21.15.5:0/1223363242 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875effd700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875f7fe700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afeacabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 -- 172.21.15.5:0/1223363242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 msgr2=0x561afeacabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afeacabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 -- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 msgr2=0x561afeac8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 -- 172.21.15.5:0/1223363242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87601a8d80 con 0x561afeac5ad0 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875effd700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f875ffff700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f87580093f0 tx=0x7f875802d970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf74cc9b3c79db17 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f873effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.072+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758009ae0 con 0x561afeac5ad0 2023-07-21T19:11:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f875409b350 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8758018a00 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8758018bc0 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 10 client.16596 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x561afe861250 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f875409b350 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8758005870 con 0x561afeac5ad0 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 20 client.16596 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afeb2a060 0x561afeb2c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 4 client.16596 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:11:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.073+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561afe9c8050 con 0x561afeb2a060 2023-07-21T19:11:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.074+0000 7f875f7fe700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afeb2a060 0x561afeb2c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.074+0000 7f875f7fe700 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afeb2a060 0x561afeb2c470 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e433f4ae0d3d8083 server_cookie=176108f4e9a24f74 in_seq=0 out_seq=0 2023-07-21T19:11:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.074+0000 7f873effd700 10 client.16596 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f873effd700 1 -- 172.21.15.5:0/1223363242 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561afe9c8050 con 0x561afeb2a060 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f873effd700 10 client.16596 handle_command_reply: tid=0 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 client.16596 shutdown 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 msgr2=0x561afeac9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f87580093f0 tx=0x7f875802d970 comp rx=0 tx=0).stop 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 shutdown_connections 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561afeb2a060 0x561afeb2c470 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561afeac5ad0 0x561afeac9380 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561afeaba2d0 0x561afeacabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f873effd700 0 client.16596 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 --2- 172.21.15.5:0/1223363242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561afeababe0 0x561afeac8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 >> 172.21.15.5:0/1223363242 conn(0x561afeab0da0 msgr2=0x561afeabe3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 shutdown_connections 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 1 -- 172.21.15.5:0/1223363242 wait complete. 2023-07-21T19:11:46.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.075+0000 7f87798fbb80 20 client.16596 trim_cache size 0 max 16384 2023-07-21T19:11:46.090 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:11:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 msgr2=0x7f87480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 0x7f87480a3770 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f87580060b0 tx=0x7f8758007a80 comp rx=0 tx=0).stop 2023-07-21T19:11:46.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 msgr2=0x7f8760106ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f87540036c0 tx=0x7f8754005740 comp rx=0 tx=0).stop 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 shutdown_connections 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8760105ae0 0x7f8760107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f87600ffad0 0x7f8760106990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87480a12c0 0x7f87480a3770 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 --2- 172.21.15.5:0/2104018340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8760100480 0x7f8760106ed0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 >> 172.21.15.5:0/2104018340 conn(0x7f87600fb5f0 msgr2=0x7f876010aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 shutdown_connections 2023-07-21T19:11:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:46.076+0000 7f8765fff700 1 -- 172.21.15.5:0/2104018340 wait complete. 2023-07-21T19:11:47.100 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:11:47.100 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 -- 172.21.15.5:0/3061691583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 msgr2=0x7f679c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 --2- 172.21.15.5:0/3061691583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c10c910 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f679400b0a0 tx=0x7f679400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 -- 172.21.15.5:0/3061691583 shutdown_connections 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 --2- 172.21.15.5:0/3061691583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c10c910 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 --2- 172.21.15.5:0/3061691583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 0x7f679c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 --2- 172.21.15.5:0/3061691583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 -- 172.21.15.5:0/3061691583 >> 172.21.15.5:0/3061691583 conn(0x7f679c0fb5f0 msgr2=0x7f679c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 -- 172.21.15.5:0/3061691583 shutdown_connections 2023-07-21T19:11:47.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.314+0000 7f67a17ff700 1 -- 172.21.15.5:0/3061691583 wait complete. 2023-07-21T19:11:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.315+0000 7f67a17ff700 1 Processor -- start 2023-07-21T19:11:47.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.315+0000 7f67a17ff700 1 -- start start 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 0x7f679c19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c19ce00 con 0x7f679c100480 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c19cf70 con 0x7f679c0ffad0 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f67a17ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f679c19d130 con 0x7f679c105ae0 2023-07-21T19:11:47.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47788/0 (socket says 172.21.15.5:47788) 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 -- 172.21.15.5:0/1552368197 learned_addr learned my addr 172.21.15.5:0/1552368197 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f679b7fe700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 0x7f679c19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f679affd700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 -- 172.21.15.5:0/1552368197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 msgr2=0x7f679c19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 0x7f679c19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 -- 172.21.15.5:0/1552368197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 msgr2=0x7f679c197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f6793fff700 1 -- 172.21.15.5:0/1552368197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6794009d00 con 0x7f679c100480 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.316+0000 7f679affd700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f6793fff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f678400d950 tx=0x7f678400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f6798ff9700 1 -- 172.21.15.5:0/1552368197 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784009940 con 0x7f679c100480 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f6798ff9700 1 -- 172.21.15.5:0/1552368197 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6784010460 con 0x7f679c100480 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f679c19d410 con 0x7f679c100480 2023-07-21T19:11:47.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f6798ff9700 1 -- 172.21.15.5:0/1552368197 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678400f620 con 0x7f679c100480 2023-07-21T19:11:47.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.317+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c1a4c50 con 0x7f679c100480 2023-07-21T19:11:47.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.320+0000 7f6798ff9700 1 -- 172.21.15.5:0/1552368197 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f67840105d0 con 0x7f679c100480 2023-07-21T19:11:47.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.322+0000 7f67b520fb80 1 Processor -- start 2023-07-21T19:11:47.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.322+0000 7f67b520fb80 1 -- start start 2023-07-21T19:11:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 0x564a315a8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315aaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a314a3d30 con 0x564a3159d350 2023-07-21T19:11:47.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a314f6ac0 con 0x564a3159dc60 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f67b520fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a314eff50 con 0x564a315a8b50 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f6798ff9700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 0x7f67880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f6798ff9700 1 -- 172.21.15.5:0/1552368197 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f67840ccb00 con 0x7f679c100480 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f679b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f679b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47800/0 (socket says 172.21.15.5:47800) 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f679b7fe700 1 -- 172.21.15.5:0/3827211674 learned_addr learned my addr 172.21.15.5:0/3827211674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f679b7fe700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 0x7f67880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.323+0000 7f679affd700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 0x564a315a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f6793fff700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315aaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 -- 172.21.15.5:0/3827211674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 msgr2=0x564a315aaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315aaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 -- 172.21.15.5:0/3827211674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 msgr2=0x564a315a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 0x564a315a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 -- 172.21.15.5:0/3827211674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a3131de60 con 0x564a3159d350 2023-07-21T19:11:47.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679affd700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 0x564a315a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:47.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6794015000 tx=0x7f6794009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5e0e49d8801655e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f679b7fe700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 0x7f67880a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6794024000 tx=0x7f6794026000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f67917fa700 1 -- 172.21.15.5:0/3827211674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679400e040 con 0x564a3159d350 2023-07-21T19:11:47.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f67917fa700 1 -- 172.21.15.5:0/3827211674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6794009030 con 0x564a3159d350 2023-07-21T19:11:47.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.324+0000 7f67917fa700 1 -- 172.21.15.5:0/3827211674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f679401d690 con 0x564a3159d350 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 -- 172.21.15.5:0/3827211674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 msgr2=0x564a3159d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f6794015000 tx=0x7f6794009420 comp rx=0 tx=0).stop 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 -- 172.21.15.5:0/3827211674 shutdown_connections 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315aaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159d350 0x564a3159d720 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3827211674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159dc60 0x564a315a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 -- 172.21.15.5:0/3827211674 >> 172.21.15.5:0/3827211674 conn(0x564a31593e20 msgr2=0x564a31599720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 -- 172.21.15.5:0/3827211674 shutdown_connections 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.325+0000 7f67b520fb80 1 -- 172.21.15.5:0/3827211674 wait complete. 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.326+0000 7f67b520fb80 1 -- start start 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.326+0000 7f67b520fb80 1 Processor -- start 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.326+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a315a8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.326+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315a5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67b520fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3148fb40 con 0x564a3159dc60 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67b520fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3148f840 con 0x564a315a8b50 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67b520fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a313739a0 con 0x564a3159d350 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67b520fb80 10 client.0 _ll_register_callbacks cb 0x564a3148d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47816/0 (socket says 172.21.15.5:47816) 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 -- 172.21.15.5:0/40675891 learned_addr learned my addr 172.21.15.5:0/40675891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f679affd700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a315a8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f679b7fe700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315a5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 -- 172.21.15.5:0/40675891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 msgr2=0x564a315a8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a315a8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 -- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 msgr2=0x564a315a5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315a5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 -- 172.21.15.5:0/40675891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f679c10d210 con 0x564a3159dc60 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f6793fff700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6784010880 tx=0x7f6784010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540713fae9469458 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.327+0000 7f67827fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:47.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840ca6b0 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a31449570 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f678409dbd0 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67840b7140 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 10 client.16608 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a31306ce0 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a31449570 con 0x564a3159dc60 2023-07-21T19:11:47.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6784099090 con 0x564a3159dc60 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 10 client.16608 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 20 client.16608 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a315b82f0 0x564a315ba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 4 client.16608 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:47.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.328+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a314e8f30 con 0x564a315b82f0 2023-07-21T19:11:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.329+0000 7f679affd700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a315b82f0 0x564a315ba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.329+0000 7f679affd700 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a315b82f0 0x564a315ba700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cab13373c4905a49 server_cookie=4c6e5d1c118d9317 in_seq=0 out_seq=0 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.329+0000 7f67827fc700 10 client.16608 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.330+0000 7f67827fc700 1 -- 172.21.15.5:0/40675891 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a314e8f30 con 0x564a315b82f0 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.330+0000 7f67827fc700 10 client.16608 handle_command_reply: tid=0 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.330+0000 7f67b520fb80 1 client.16608 shutdown 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.330+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 msgr2=0x564a315a5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.330+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6784010880 tx=0x7f6784010950 comp rx=0 tx=0).stop 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 shutdown_connections 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315a5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a315a8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a315b82f0 0x564a315ba700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 --2- 172.21.15.5:0/40675891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a5840 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 >> 172.21.15.5:0/40675891 conn(0x564a31593e20 msgr2=0x564a31599310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67827fc700 0 client.16608 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 shutdown_connections 2023-07-21T19:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 1 -- 172.21.15.5:0/40675891 wait complete. 2023-07-21T19:11:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.331+0000 7f67b520fb80 20 client.16608 trim_cache size 0 max 16384 2023-07-21T19:11:47.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.348+0000 7f67b520fb80 1 Processor -- start 2023-07-21T19:11:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.348+0000 7f67b520fb80 1 -- start start 2023-07-21T19:11:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 0x564a315af530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315a5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a314e8f30 con 0x564a3159dc60 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a31449570 con 0x564a3159d350 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f67b520fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a3159ada0 con 0x564a315a8b50 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47818/0 (socket says 172.21.15.5:47818) 2023-07-21T19:11:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f6793fff700 1 -- 172.21.15.5:0/3757400913 learned_addr learned my addr 172.21.15.5:0/3757400913 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315a5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.349+0000 7f679affd700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 0x564a315af530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 -- 172.21.15.5:0/3757400913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 msgr2=0x564a315a5040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315a5040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 -- 172.21.15.5:0/3757400913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 msgr2=0x564a315af530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 0x564a315af530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 -- 172.21.15.5:0/3757400913 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a313442d0 con 0x564a3159dc60 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f679affd700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 0x564a315af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6793fff700 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6784009520 tx=0x7f6784000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2e11bf4c663ed08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6791ffb700 1 -- 172.21.15.5:0/3757400913 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6784099e00 con 0x564a3159dc60 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6791ffb700 1 -- 172.21.15.5:0/3757400913 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f67840ca6b0 con 0x564a3159dc60 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.350+0000 7f6791ffb700 1 -- 172.21.15.5:0/3757400913 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67840b2200 con 0x564a3159dc60 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- 172.21.15.5:0/3757400913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 msgr2=0x564a315a77f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f6784009520 tx=0x7f6784000bc0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- 172.21.15.5:0/3757400913 shutdown_connections 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a315a8b50 0x564a315a5040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a3159d350 0x564a315af530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3757400913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a315a77f0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- 172.21.15.5:0/3757400913 >> 172.21.15.5:0/3757400913 conn(0x564a31593e20 msgr2=0x564a315a70e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- 172.21.15.5:0/3757400913 shutdown_connections 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- 172.21.15.5:0/3757400913 wait complete. 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 -- start start 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.351+0000 7f67b520fb80 1 Processor -- start 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a316055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315ae550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a315aec60 con 0x564a3159dc60 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a315aedd0 con 0x564a315a8b50 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a315aef90 con 0x564a3159d350 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f67b520fb80 10 client.0 _ll_register_callbacks cb 0x7f67840996e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f6793fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47824/0 (socket says 172.21.15.5:47824) 2023-07-21T19:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f6793fff700 1 -- 172.21.15.5:0/3089014435 learned_addr learned my addr 172.21.15.5:0/3089014435 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f6793fff700 1 -- 172.21.15.5:0/3089014435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 msgr2=0x564a316055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f6793fff700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a316055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f6793fff700 1 -- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 msgr2=0x564a315ae550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.352+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315ae550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f6793fff700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315ae550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f6793fff700 1 -- 172.21.15.5:0/3089014435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a31306ce0 con 0x564a3159dc60 2023-07-21T19:11:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f679b7fe700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f6793fff700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6784099d60 tx=0x7f67840b7400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785d523e0c1143a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67827fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f678409a000 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6794009030 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f67840a1b60 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f67840a1d60 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67b520fb80 10 client.16614 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.353+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x564a3131c330 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6794009030 con 0x564a3159dc60 2023-07-21T19:11:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f678407f070 con 0x564a3159dc60 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 10 client.16614 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 20 client.16614 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a3160e170 0x564a31610580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 4 client.16614 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x564a314ab0d0 con 0x564a3160e170 2023-07-21T19:11:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.354+0000 7f679affd700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a3160e170 0x564a31610580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.355+0000 7f679affd700 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a3160e170 0x564a31610580 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b314d8c7eef7c6b2 server_cookie=2f603d26711d1486 in_seq=0 out_seq=0 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.355+0000 7f67827fc700 10 client.16614 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67827fc700 1 -- 172.21.15.5:0/3089014435 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x564a314ab0d0 con 0x564a3160e170 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67827fc700 10 client.16614 handle_command_reply: tid=0 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 client.16614 shutdown 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 msgr2=0x564a31605ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6784099d60 tx=0x7f67840b7400 comp rx=0 tx=0).stop 2023-07-21T19:11:47.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 shutdown_connections 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a315a8b50 0x564a315ae550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a3159d350 0x564a316055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a3160e170 0x564a31610580 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 --2- 172.21.15.5:0/3089014435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a3159dc60 0x564a31605ae0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 >> 172.21.15.5:0/3089014435 conn(0x564a31593e20 msgr2=0x564a3159fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67827fc700 0 client.16614 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 shutdown_connections 2023-07-21T19:11:47.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.356+0000 7f67b520fb80 1 -- 172.21.15.5:0/3089014435 wait complete. 2023-07-21T19:11:47.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67b520fb80 20 client.16614 trim_cache size 0 max 16384 2023-07-21T19:11:47.370 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:11:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "40bb24d1-67b6-48e7-9782-221c8b1b6ed3", 2023-07-21T19:11:47.370 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:11:47.371 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 msgr2=0x7f67880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 0x7f67880a37f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6794024000 tx=0x7f6794026000 comp rx=0 tx=0).stop 2023-07-21T19:11:47.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 msgr2=0x7f679c198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f678400d950 tx=0x7f678400dc60 comp rx=0 tx=0).stop 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 shutdown_connections 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f679c105ae0 0x7f679c19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f679c0ffad0 0x7f679c197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f67880a1340 0x7f67880a37f0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 --2- 172.21.15.5:0/1552368197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f679c100480 0x7f679c198360 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 >> 172.21.15.5:0/1552368197 conn(0x7f679c0fb5f0 msgr2=0x7f679c10aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 shutdown_connections 2023-07-21T19:11:47.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.357+0000 7f67a17ff700 1 -- 172.21.15.5:0/1552368197 wait complete. 2023-07-21T19:11:47.382 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3364852835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878100480 msgr2=0x7fb87810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3364852835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878100480 0x7fb87810c910 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b0a0 tx=0x7fb87000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3364852835 shutdown_connections 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3364852835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878100480 0x7fb87810c910 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3364852835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878105ae0 0x7fb878105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3364852835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 0x7fb8780fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3364852835 >> 172.21.15.5:0/3364852835 conn(0x7fb8780fb5f0 msgr2=0x7fb8780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3364852835 shutdown_connections 2023-07-21T19:11:47.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.598+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3364852835 wait complete. 2023-07-21T19:11:47.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.599+0000 7fb87e5ff700 1 Processor -- start 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.599+0000 7fb87e5ff700 1 -- start start 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 0x7fb878072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878100480 0x7fb878072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a4010 con 0x7fb878105ae0 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a4180 con 0x7fb878100480 2023-07-21T19:11:47.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8781a4340 con 0x7fb8780ffad0 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47834/0 (socket says 172.21.15.5:47834) 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3091003843 learned_addr learned my addr 172.21.15.5:0/3091003843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb877fff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 0x7fb878072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb8777fe700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878100480 0x7fb878072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3091003843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 msgr2=0x7fb878072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 0x7fb878072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3091003843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878100480 msgr2=0x7fb878072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.600+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878100480 0x7fb878072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3091003843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb870009d00 con 0x7fb878105ae0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b070 tx=0x7fb87000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb8757fa700 1 -- 172.21.15.5:0/3091003843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87000e040 con 0x7fb878105ae0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb8757fa700 1 -- 172.21.15.5:0/3091003843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb870003e10 con 0x7fb878105ae0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb8757fa700 1 -- 172.21.15.5:0/3091003843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8700052a0 con 0x7fb878105ae0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8781a45c0 con 0x7fb878105ae0 2023-07-21T19:11:47.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.601+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781a4990 con 0x7fb878105ae0 2023-07-21T19:11:47.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.604+0000 7fb8757fa700 1 -- 172.21.15.5:0/3091003843 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb870005400 con 0x7fb878105ae0 2023-07-21T19:11:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 Processor -- start 2023-07-21T19:11:47.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb8757fa700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 0x7fb8600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 -- start start 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb8757fa700 1 -- 172.21.15.5:0/3091003843 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb8700ce080 con 0x7fb878105ae0 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb8777fe700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 0x7fb8600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4b46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c3bacb0 con 0x562c3c4b4be0 2023-07-21T19:11:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c40da40 con 0x562c3c4bfad0 2023-07-21T19:11:47.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.606+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c406ed0 con 0x562c3c4b42d0 2023-07-21T19:11:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47838/0 (socket says 172.21.15.5:47838) 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb8777fe700 1 -- 172.21.15.5:0/3254377161 learned_addr learned my addr 172.21.15.5:0/3254377161 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb877fff700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4b46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb8777fe700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 0x7fb8600a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb86c013510 tx=0x7fb86c0138d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3254377161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 msgr2=0x562c3c4b46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4b46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3254377161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 msgr2=0x562c3c4bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb87cb9c700 1 -- 172.21.15.5:0/3254377161 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c3c234de0 con 0x562c3c4bfad0 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.607+0000 7fb8777fe700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.608+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c1eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cb7a0 tx=0x7fb8700099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=89191008f47595c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.608+0000 7fb85e7fc700 1 -- 172.21.15.5:0/3254377161 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87009b1f0 con 0x562c3c4bfad0 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.608+0000 7fb85e7fc700 1 -- 172.21.15.5:0/3254377161 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb87009f1e0 con 0x562c3c4bfad0 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.608+0000 7fb85e7fc700 1 -- 172.21.15.5:0/3254377161 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb87009a6a0 con 0x562c3c4bfad0 2023-07-21T19:11:47.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- 172.21.15.5:0/3254377161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 msgr2=0x562c3c4c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c1eb0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fb8700cb7a0 tx=0x7fb8700099e0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- 172.21.15.5:0/3254377161 shutdown_connections 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c1eb0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4b46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/3254377161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- 172.21.15.5:0/3254377161 >> 172.21.15.5:0/3254377161 conn(0x562c3c4aada0 msgr2=0x562c3c4b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- 172.21.15.5:0/3254377161 shutdown_connections 2023-07-21T19:11:47.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- 172.21.15.5:0/3254377161 wait complete. 2023-07-21T19:11:47.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.609+0000 7fb891e7ab80 1 -- start start 2023-07-21T19:11:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 Processor -- start 2023-07-21T19:11:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4c03e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c0920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c3a6ac0 con 0x562c3c4b4be0 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c3a67c0 con 0x562c3c4bfad0 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c28a920 con 0x562c3c4b42d0 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb891e7ab80 10 client.0 _ll_register_callbacks cb 0x562c3c3a4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb877fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb8777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4c03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb877fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35030/0 (socket says 172.21.15.5:35030) 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb877fff700 1 -- 172.21.15.5:0/1227758700 learned_addr learned my addr 172.21.15.5:0/1227758700 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.610+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c0920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 -- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 msgr2=0x562c3c4c0920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c0920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 -- 172.21.15.5:0/1227758700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 msgr2=0x562c3c4c03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4c03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 -- 172.21.15.5:0/1227758700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb86c00f3d0 con 0x562c3c4b42d0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb8777fe700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb877fff700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb868011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76df03587b2f0342 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868016070 con 0x562c3c4b42d0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c3c3604f0 con 0x562c3c4b42d0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.611+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb868010640 con 0x562c3c4b42d0 2023-07-21T19:11:47.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb868010800 con 0x562c3c4b42d0 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 10 client.26290 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x7fb8781a4990 con 0x562c3c4b42d0 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562c3c3604f0 con 0x562c3c4b42d0 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb868011d20 con 0x562c3c4b42d0 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 10 client.26290 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 10 client.26290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 20 client.26290 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:47.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c4cf2c0 0x562c3c4d16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 4 client.26290 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c3c3ffeb0 con 0x562c3c4cf2c0 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.612+0000 7fb8777fe700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c4cf2c0 0x562c3c4d16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.613+0000 7fb8777fe700 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c4cf2c0 0x562c3c4d16d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85b14a5cde4f28ea server_cookie=3cb309e8a9d3d9ea in_seq=0 out_seq=0 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.613+0000 7fb8577fe700 10 client.26290 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb8577fe700 1 -- 172.21.15.5:0/1227758700 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c3c3ffeb0 con 0x562c3c4cf2c0 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb8577fe700 10 client.26290 handle_command_reply: tid=0 2023-07-21T19:11:47.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 client.26290 shutdown 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 msgr2=0x562c3c4bfea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb868007fd0 tx=0x7fb868011720 comp rx=0 tx=0).stop 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 shutdown_connections 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4c0920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b42d0 0x562c3c4bfea0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c4cf2c0 0x562c3c4d16d0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1227758700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b4be0 0x562c3c4c03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb8577fe700 0 client.26290 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 >> 172.21.15.5:0/1227758700 conn(0x562c3c4aada0 msgr2=0x562c3c4b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 shutdown_connections 2023-07-21T19:11:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.614+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1227758700 wait complete. 2023-07-21T19:11:47.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.615+0000 7fb891e7ab80 20 client.26290 trim_cache size 0 max 16384 2023-07-21T19:11:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 Processor -- start 2023-07-21T19:11:47.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 -- start start 2023-07-21T19:11:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 0x562c3c4bbda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4bc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c4cafc0 con 0x562c3c4b42d0 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c4aa970 con 0x562c3c4bfad0 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c4bec60 con 0x562c3c4b4be0 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb877fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb877fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47862/0 (socket says 172.21.15.5:47862) 2023-07-21T19:11:47.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.632+0000 7fb877fff700 1 -- 172.21.15.5:0/1722320298 learned_addr learned my addr 172.21.15.5:0/1722320298 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4bc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb8777fe700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 0x562c3c4bbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 -- 172.21.15.5:0/1722320298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 msgr2=0x562c3c4bbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 0x562c3c4bbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 -- 172.21.15.5:0/1722320298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 msgr2=0x562c3c4bc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4bc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 -- 172.21.15.5:0/1722320298 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c3c4ac5e0 con 0x562c3c4b42d0 2023-07-21T19:11:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb8777fe700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 0x562c3c4bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb877fff700 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb868011a70 tx=0x7fb868011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f891534437c6ddd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb85effd700 1 -- 172.21.15.5:0/1722320298 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868016070 con 0x562c3c4b42d0 2023-07-21T19:11:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb85effd700 1 -- 172.21.15.5:0/1722320298 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb868017040 con 0x562c3c4b42d0 2023-07-21T19:11:47.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.633+0000 7fb85effd700 1 -- 172.21.15.5:0/1722320298 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8680038e0 con 0x562c3c4b42d0 2023-07-21T19:11:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1722320298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 msgr2=0x562c3c4ca660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb868011a70 tx=0x7fb868011aa0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1722320298 shutdown_connections 2023-07-21T19:11:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4bfad0 0x562c3c4bc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4ca660 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/1722320298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4b4be0 0x562c3c4bbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1722320298 >> 172.21.15.5:0/1722320298 conn(0x562c3c4aada0 msgr2=0x562c3c4b9e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1722320298 shutdown_connections 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- 172.21.15.5:0/1722320298 wait complete. 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.634+0000 7fb891e7ab80 1 -- start start 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 Processor -- start 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 0x562c3c4c3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4bfad0 0x562c3c51c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c51c6c0 con 0x562c3c4b42d0 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c51c830 con 0x562c3c4b4be0 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c3c51c9f0 con 0x562c3c4bfad0 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb891e7ab80 10 client.0 _ll_register_callbacks cb 0x7fb868007aa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb877fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb877fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47878/0 (socket says 172.21.15.5:47878) 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb877fff700 1 -- 172.21.15.5:0/915944042 learned_addr learned my addr 172.21.15.5:0/915944042 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.635+0000 7fb8777fe700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 0x562c3c4c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb87cb9c700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4bfad0 0x562c3c51c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 -- 172.21.15.5:0/915944042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4bfad0 msgr2=0x562c3c51c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4bfad0 0x562c3c51c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 -- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 msgr2=0x562c3c4c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 0x562c3c4c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 -- 172.21.15.5:0/915944042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8781a4990 con 0x562c3c4b42d0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb8777fe700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 0x562c3c4c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb877fff700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb86802f040 tx=0x7fb868005630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2168cb7f486d7547 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb8577fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb868016040 con 0x562c3c4b42d0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb87009f1e0 con 0x562c3c4b42d0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.636+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fb8680053a0 con 0x562c3c4b42d0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb86801b720 con 0x562c3c4b42d0 2023-07-21T19:11:47.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 10 client.16632 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x562c3c21dc60 con 0x562c3c4b42d0 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb87009f1e0 con 0x562c3c4b42d0 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb8680108a0 con 0x562c3c4b42d0 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 10 client.16632 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 10 client.16632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 20 client.16632 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:47.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c523de0 0x562c3c5261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 4 client.16632 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c3c3c2050 con 0x562c3c523de0 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.637+0000 7fb8777fe700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c523de0 0x562c3c5261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb8777fe700 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c523de0 0x562c3c5261f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e55b299ff2d911 server_cookie=829000b56dff24c9 in_seq=0 out_seq=0 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb8577fe700 10 client.16632 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb8577fe700 1 -- 172.21.15.5:0/915944042 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562c3c3c2050 con 0x562c3c523de0 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb8577fe700 10 client.16632 handle_command_reply: tid=0 2023-07-21T19:11:47.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 client.16632 shutdown 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 msgr2=0x562c3c4c2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb86802f040 tx=0x7fb868005630 comp rx=0 tx=0).stop 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 shutdown_connections 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c3c4bfad0 0x562c3c51c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c3c523de0 0x562c3c5261f0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c3c4b42d0 0x562c3c4c2e10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 --2- 172.21.15.5:0/915944042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c3c4b4be0 0x562c3c4c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb8577fe700 0 client.16632 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.638+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 >> 172.21.15.5:0/915944042 conn(0x562c3c4aada0 msgr2=0x562c3c4b8270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 shutdown_connections 2023-07-21T19:11:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb891e7ab80 1 -- 172.21.15.5:0/915944042 wait complete. 2023-07-21T19:11:47.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb891e7ab80 20 client.16632 trim_cache size 0 max 16384 2023-07-21T19:11:47.653 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:11:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:11:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:11:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "40bb24d1-67b6-48e7-9782-221c8b1b6ed3": { 2023-07-21T19:11:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:11:47.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "40bb24d1-67b6-48e7-9782-221c8b1b6ed3", 2023-07-21T19:11:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:11:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:11:47.655 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:11:47.655 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 msgr2=0x7fb8600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 0x7fb8600a37f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb86c013510 tx=0x7fb86c0138d0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 msgr2=0x7fb878073290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb87000b070 tx=0x7fb87000bce0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 shutdown_connections 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb8600a1340 0x7fb8600a37f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb878105ae0 0x7fb878073290 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb8780ffad0 0x7fb878072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 --2- 172.21.15.5:0/3091003843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb878100480 0x7fb878072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 >> 172.21.15.5:0/3091003843 conn(0x7fb8780fb5f0 msgr2=0x7fb8781035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 shutdown_connections 2023-07-21T19:11:47.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:11:47.639+0000 7fb87e5ff700 1 -- 172.21.15.5:0/3091003843 wait complete. 2023-07-21T19:11:47.663 INFO:tasks.cephfs.filesystem:scrub status for tag:40bb24d1-67b6-48e7-9782-221c8b1b6ed3 - {'path': '/', 'tag': '40bb24d1-67b6-48e7-9782-221c8b1b6ed3', 'options': 'recursive,force'} 2023-07-21T19:11:47.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:47 smithi101 ceph-mon[128136]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:11:47.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:47 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:11:47.839 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:11:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:11:47.383+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...6,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v1958 ap=2 f(v30 m2023-07-21T19:11:02.256537+0000 3=3+0) n(v3 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1) old_inodes=4 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@86} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v1958>,<0x10000000005/data v1188>,<0x10000000003/postgres v314>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v330>,<0x10000000001/sv_0 v349>,<0x10000000000/_nogroup v333>,<0x1/volumes v319>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:11:02.244983+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1)","memory_value.dirstat":"f(v30 m2023-07-21T19:11:02.256537+0000 3=3+0)","memory_value.rstat":"n(v3 rc2023-07-21T19:11:02.256537+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:11:47.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:47 smithi005 ceph-mon[126460]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:11:47.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:47 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:11:47.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:47 smithi156 ceph-mon[128316]: pgmap v128: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:11:47.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:47 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:11:48.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:48 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:11:48.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:48 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:48.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:48 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:11:48.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:48 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:48.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:48 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:48.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:48 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:11:48.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:48 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:48.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:48 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:11:48.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:48 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:48.916 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:48 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:48.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:48 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:11:48.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:48 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:48.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:48 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:11:48.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:48 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:11:48.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:48 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:11:49.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:49 smithi101 ceph-mon[128136]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 24 MiB/s wr, 42 op/s 2023-07-21T19:11:49.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:49 smithi005 ceph-mon[126460]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 24 MiB/s wr, 42 op/s 2023-07-21T19:11:49.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:49 smithi156 ceph-mon[128316]: pgmap v129: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 24 MiB/s wr, 42 op/s 2023-07-21T19:11:50.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:50 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:11:50.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:50 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:11:50.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:50 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:11:50.838 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:50 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:11:50.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:50 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:11:50.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:50 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:11:50.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:50 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:11:50.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:50 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:11:50.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:50 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:11:50.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:50 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:11:50.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:50 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:11:50.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:50 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:11:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 -- 172.21.15.5:0/3425184253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c41004e0 msgr2=0x7f59c410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/3425184253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c41004e0 0x7f59c410c920 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f59bc00b0a0 tx=0x7f59bc00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 -- 172.21.15.5:0/3425184253 shutdown_connections 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/3425184253 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c41004e0 0x7f59c410c920 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/3425184253 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c4105b60 0x7f59c4105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/3425184253 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c40ffb30 0x7f59c40fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 -- 172.21.15.5:0/3425184253 >> 172.21.15.5:0/3425184253 conn(0x7f59c40fb5f0 msgr2=0x7f59c40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 -- 172.21.15.5:0/3425184253 shutdown_connections 2023-07-21T19:11:51.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.127+0000 7f59ca5ff700 1 -- 172.21.15.5:0/3425184253 wait complete. 2023-07-21T19:11:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 Processor -- start 2023-07-21T19:11:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 -- start start 2023-07-21T19:11:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 0x7f59c41982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 0x7f59c419c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c419cf90 con 0x7f59c4105b60 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c419d100 con 0x7f59c40ffb30 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.128+0000 7f59ca5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c419d2c0 con 0x7f59c41004e0 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43094/0 (socket says 172.21.15.5:43094) 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 -- 172.21.15.5:0/1666838046 learned_addr learned my addr 172.21.15.5:0/1666838046 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:51.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 0x7f59c419c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59bbfff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 0x7f59c41982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 -- 172.21.15.5:0/1666838046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 msgr2=0x7f59c41982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 0x7f59c41982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 -- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 msgr2=0x7f59c419c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 0x7f59c419c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 -- 172.21.15.5:0/1666838046 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59bc009d00 con 0x7f59c40ffb30 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59bbfff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 0x7f59c41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:51.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 0x7f59c419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.129+0000 7f59c3fff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59b400e9a0 tx=0x7f59b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.130+0000 7f59c1ffb700 1 -- 172.21.15.5:0/1666838046 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4009790 con 0x7f59c40ffb30 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.130+0000 7f59c1ffb700 1 -- 172.21.15.5:0/1666838046 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b400cbd0 con 0x7f59c40ffb30 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.130+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c419d5a0 con 0x7f59c40ffb30 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.130+0000 7f59c1ffb700 1 -- 172.21.15.5:0/1666838046 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4018680 con 0x7f59c40ffb30 2023-07-21T19:11:51.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.130+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c41a4de0 con 0x7f59c40ffb30 2023-07-21T19:11:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.133+0000 7f59c1ffb700 1 -- 172.21.15.5:0/1666838046 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f59b400cd40 con 0x7f59c40ffb30 2023-07-21T19:11:51.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.134+0000 7f59ddebeb80 1 Processor -- start 2023-07-21T19:11:51.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.134+0000 7f59ddebeb80 1 -- start start 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.134+0000 7f59c1ffb700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 0x7f59b00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c1ffb700 1 -- 172.21.15.5:0/1666838046 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f59b4014070 con 0x7f59c40ffb30 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecaa8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 0x7f59b00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecaa89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ec9adc50 con 0x5613ecaa8580 2023-07-21T19:11:51.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613eca009e0 con 0x5613ecaae2d0 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ec9f9e70 con 0x5613ec9046b0 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35060/0 (socket says 172.21.15.5:35060) 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 -- 172.21.15.5:0/2555842707 learned_addr learned my addr 172.21.15.5:0/2555842707 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59bbfff700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecaa8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecaa89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 -- 172.21.15.5:0/2555842707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 msgr2=0x5613ecaa8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecaa8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 -- 172.21.15.5:0/2555842707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 msgr2=0x5613ecaa89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecaa89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59c3fff700 1 -- 172.21.15.5:0/2555842707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ec827d80 con 0x5613ec9046b0 2023-07-21T19:11:51.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.135+0000 7f59bbfff700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecaa8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.136+0000 7f59c3fff700 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59b40a1f00 tx=0x7f59b4099c40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dd1ecb2eacf5b2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.136+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 0x7f59b00a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59bc006130 tx=0x7f59bc007cc0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.136+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2555842707 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b4099fe0 con 0x5613ec9046b0 2023-07-21T19:11:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.136+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2555842707 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b409e010 con 0x5613ec9046b0 2023-07-21T19:11:51.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.136+0000 7f59ba7fc700 1 -- 172.21.15.5:0/2555842707 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40996f0 con 0x5613ec9046b0 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- 172.21.15.5:0/2555842707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 msgr2=0x5613ecaadd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f59b40a1f00 tx=0x7f59b4099c40 comp rx=0 tx=0).stop 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- 172.21.15.5:0/2555842707 shutdown_connections 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecaa89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecaadd90 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/2555842707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecaa8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- 172.21.15.5:0/2555842707 >> 172.21.15.5:0/2555842707 conn(0x5613eca9dbe0 msgr2=0x5613ecaa2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- 172.21.15.5:0/2555842707 shutdown_connections 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- 172.21.15.5:0/2555842707 wait complete. 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 -- start start 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.137+0000 7f59ddebeb80 1 Processor -- start 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ecaa8580 0x5613ecab1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ec999a60 con 0x5613ec9046b0 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ec999760 con 0x5613ecaae2d0 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ec87d8c0 con 0x5613ecaa8580 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59ddebeb80 10 client.0 _ll_register_callbacks cb 0x5613ec997700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59c3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47906/0 (socket says 172.21.15.5:47906) 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59c3fff700 1 -- 172.21.15.5:0/3942232251 learned_addr learned my addr 172.21.15.5:0/3942232251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ecaa8580 0x5613ecab1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.138+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 -- 172.21.15.5:0/3942232251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ecaa8580 msgr2=0x5613ecab1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ecaa8580 0x5613ecab1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 -- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 msgr2=0x5613ecab1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 -- 172.21.15.5:0/3942232251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c41a4de0 con 0x5613ec9046b0 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:51.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f59b4096a50 tx=0x7f59b409a4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8db260493d0159b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b40a2430 con 0x5613ec9046b0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5613ec953490 con 0x5613ec9046b0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f59b4099fe0 con 0x5613ec9046b0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59b40a21b0 con 0x5613ec9046b0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59ddebeb80 10 client.16644 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.139+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5613ec810c00 con 0x5613ec9046b0 2023-07-21T19:11:51.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b407c080 con 0x5613ec9046b0 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f59b4095dd0 con 0x5613ec9046b0 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 10 client.16644 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 20 client.16644 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecac19e0 0x5613ecac3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 4 client.16644 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.140+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5613ec9f2e50 con 0x5613ecac19e0 2023-07-21T19:11:51.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.141+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecac19e0 0x5613ecac3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.141+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecac19e0 0x5613ecac3df0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d8e43f357e6a573 server_cookie=e6a5de018c90afa8 in_seq=0 out_seq=0 2023-07-21T19:11:51.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.141+0000 7f59a37fe700 10 client.16644 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59a37fe700 1 -- 172.21.15.5:0/3942232251 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5613ec9f2e50 con 0x5613ecac19e0 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59a37fe700 10 client.16644 handle_command_reply: tid=0 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 client.16644 shutdown 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 msgr2=0x5613ecab4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f59b4096a50 tx=0x7f59b409a4f0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 shutdown_connections 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecac19e0 0x5613ecac3df0 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ec9046b0 0x5613ecab4590 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3942232251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ecaa8580 0x5613ecab1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 >> 172.21.15.5:0/3942232251 conn(0x5613eca9dbe0 msgr2=0x5613ecaa24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59a37fe700 0 client.16644 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 shutdown_connections 2023-07-21T19:11:51.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.142+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3942232251 wait complete. 2023-07-21T19:11:51.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.143+0000 7f59ddebeb80 20 client.16644 trim_cache size 0 max 16384 2023-07-21T19:11:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 Processor -- start 2023-07-21T19:11:51.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 -- start start 2023-07-21T19:11:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab1920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaa8580 0x5613ecaad110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecaac5f0 con 0x5613ecaae2d0 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecac4110 con 0x5613ecaa8580 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecaae070 con 0x5613ec9046b0 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.158+0000 7f59c8b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47910/0 (socket says 172.21.15.5:47910) 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 -- 172.21.15.5:0/3973760160 learned_addr learned my addr 172.21.15.5:0/3973760160 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:51.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaa8580 0x5613ecaad110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 -- 172.21.15.5:0/3973760160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 msgr2=0x5613ecab1920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab1920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 -- 172.21.15.5:0/3973760160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaa8580 msgr2=0x5613ecaad110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaa8580 0x5613ecaad110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 -- 172.21.15.5:0/3973760160 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5613ec84e1f0 con 0x5613ecaae2d0 2023-07-21T19:11:51.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:11:51.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.159+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009460 tx=0x7f59bc023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebff26ffc8b9c890 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59baffd700 1 -- 172.21.15.5:0/3973760160 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc00e040 con 0x5613ecaae2d0 2023-07-21T19:11:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59baffd700 1 -- 172.21.15.5:0/3973760160 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59bc003c00 con 0x5613ecaae2d0 2023-07-21T19:11:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59baffd700 1 -- 172.21.15.5:0/3973760160 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59bc005040 con 0x5613ecaae2d0 2023-07-21T19:11:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3973760160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 msgr2=0x5613ecab4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f59bc009460 tx=0x7f59bc023000 comp rx=0 tx=0).stop 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3973760160 shutdown_connections 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaae2d0 0x5613ecab4130 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab1920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3973760160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaa8580 0x5613ecaad110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3973760160 >> 172.21.15.5:0/3973760160 conn(0x5613eca9dbe0 msgr2=0x5613ecaaca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3973760160 shutdown_connections 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.160+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3973760160 wait complete. 2023-07-21T19:11:51.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.161+0000 7f59ddebeb80 1 -- start start 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.161+0000 7f59ddebeb80 1 Processor -- start 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab6c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecab7200 con 0x5613ecaa8580 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecab7370 con 0x5613ecaae2d0 2023-07-21T19:11:51.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5613ecab7530 con 0x5613ec9046b0 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59ddebeb80 10 client.0 _ll_register_callbacks cb 0x7f59b4099b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:47914/0 (socket says 172.21.15.5:47914) 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 -- 172.21.15.5:0/3674190440 learned_addr learned my addr 172.21.15.5:0/3674190440 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 -- 172.21.15.5:0/3674190440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 msgr2=0x5613ecab85d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab85d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 -- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 msgr2=0x5613ecab6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.162+0000 7f59bbfff700 1 -- 172.21.15.5:0/3674190440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59bc025040 con 0x5613ecaa8580 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59c8b9c700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59bbfff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5613ecab2dc0 tx=0x7f59ac01d680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=554bb00762f616c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59a37fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:11:51.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59ac018040 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b409e010 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f59ac01dd60 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59ac003680 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59ddebeb80 10 client.16650 fetch_fsmap learned FSMap version 26 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.163+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=4+}) v3 -- 0x5613ec826250 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59b409e010 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 26) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f59ac02f860 con 0x5613ecaa8580 2023-07-21T19:11:51.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 10 client.16650 fetch_fsmap finished waiting for FSMap version 26 2023-07-21T19:11:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:11:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 20 client.16650 populate_metadata read hostname 'smithi005' 2023-07-21T19:11:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecb16b40 0x5613ecb18f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:11:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 4 client.16650 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:11:51.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5613ec9b4ff0 con 0x5613ecb16b40 2023-07-21T19:11:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.164+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecb16b40 0x5613ecb18f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:11:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.165+0000 7f59c3fff700 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecb16b40 0x5613ecb18f50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bde8cfc307ab733 server_cookie=b90e47094c7d3042 in_seq=0 out_seq=0 2023-07-21T19:11:51.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.165+0000 7f59a37fe700 10 client.16650 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.177+0000 7f59a37fe700 1 -- 172.21.15.5:0/3674190440 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5613ec9b4ff0 con 0x5613ecb16b40 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.177+0000 7f59a37fe700 10 client.16650 handle_command_reply: tid=0 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.177+0000 7f59ddebeb80 1 client.16650 shutdown 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 msgr2=0x5613ecab8b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x5613ecab2dc0 tx=0x7f59ac01d680 comp rx=0 tx=0).stop 2023-07-21T19:11:51.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 shutdown_connections 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5613ecaae2d0 0x5613ecab6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5613ec9046b0 0x5613ecab85d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5613ecb16b40 0x5613ecb18f50 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 --2- 172.21.15.5:0/3674190440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5613ecaa8580 0x5613ecab8b10 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59a37fe700 0 client.16650 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 >> 172.21.15.5:0/3674190440 conn(0x5613eca9dbe0 msgr2=0x5613ecaad290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 shutdown_connections 2023-07-21T19:11:51.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 1 -- 172.21.15.5:0/3674190440 wait complete. 2023-07-21T19:11:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.178+0000 7f59ddebeb80 20 client.16650 trim_cache size 0 max 16384 2023-07-21T19:11:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:11:51.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:11:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:11:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:11:51.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 msgr2=0x7f59b00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 0x7f59b00a37f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f59bc006130 tx=0x7f59bc007cc0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 msgr2=0x7f59c4197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f59b400e9a0 tx=0x7f59b400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 shutdown_connections 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59b00a1340 0x7f59b00a37f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59c4105b60 0x7f59c419c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59c40ffb30 0x7f59c4197da0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 --2- 172.21.15.5:0/1666838046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59c41004e0 0x7f59c41982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:11:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 >> 172.21.15.5:0/1666838046 conn(0x7f59c40fb5f0 msgr2=0x7f59c410ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:11:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 shutdown_connections 2023-07-21T19:11:51.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:11:51.179+0000 7f59ca5ff700 1 -- 172.21.15.5:0/1666838046 wait complete. 2023-07-21T19:11:51.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:51 smithi101 ceph-mon[128136]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-21T19:11:51.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:51 smithi005 ceph-mon[126460]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-21T19:11:51.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:51 smithi156 ceph-mon[128316]: pgmap v130: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 13 MiB/s wr, 34 op/s 2023-07-21T19:11:53.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:53 smithi101 ceph-mon[128136]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 13 MiB/s wr, 111 op/s 2023-07-21T19:11:53.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:53 smithi005 ceph-mon[126460]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 13 MiB/s wr, 111 op/s 2023-07-21T19:11:53.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:53 smithi156 ceph-mon[128316]: pgmap v131: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 170 KiB/s rd, 13 MiB/s wr, 111 op/s 2023-07-21T19:11:54.003 INFO:teuthology.orchestra.run.smithi005.stderr:14100000 of 50000000 tuples (28%) done (elapsed 231.56 s, remaining 589.58 s) 2023-07-21T19:11:55.390 INFO:teuthology.orchestra.run.smithi005.stderr:14200000 of 50000000 tuples (28%) done (elapsed 232.95 s, remaining 587.30 s) 2023-07-21T19:11:55.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:55 smithi101 ceph-mon[128136]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 107 op/s 2023-07-21T19:11:55.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:55 smithi005 ceph-mon[126460]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 107 op/s 2023-07-21T19:11:55.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:55 smithi156 ceph-mon[128316]: pgmap v132: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 4.7 MiB/s wr, 107 op/s 2023-07-21T19:11:56.667 INFO:teuthology.orchestra.run.smithi005.stderr:14300000 of 50000000 tuples (28%) done (elapsed 234.23 s, remaining 584.75 s) 2023-07-21T19:11:57.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:57 smithi101 ceph-mon[128136]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2023-07-21T19:11:57.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:57 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:11:57.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:57 smithi005 ceph-mon[126460]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2023-07-21T19:11:57.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:57 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:11:57.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:57 smithi156 ceph-mon[128316]: pgmap v133: 97 pgs: 97 active+clean; 3.2 GiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 2.6 MiB/s wr, 93 op/s 2023-07-21T19:11:57.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:57 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:11:57.968 INFO:teuthology.orchestra.run.smithi005.stderr:14400000 of 50000000 tuples (28%) done (elapsed 235.53 s, remaining 582.28 s) 2023-07-21T19:11:59.266 INFO:teuthology.orchestra.run.smithi005.stderr:14500000 of 50000000 tuples (29%) done (elapsed 236.83 s, remaining 579.81 s) 2023-07-21T19:11:59.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:11:59 smithi101 ceph-mon[128136]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 102 op/s 2023-07-21T19:11:59.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:11:59 smithi005 ceph-mon[126460]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 102 op/s 2023-07-21T19:11:59.927 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:11:59 smithi156 ceph-mon[128316]: pgmap v134: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 11 MiB/s wr, 102 op/s 2023-07-21T19:12:00.693 INFO:teuthology.orchestra.run.smithi005.stderr:14600000 of 50000000 tuples (29%) done (elapsed 238.25 s, remaining 577.68 s) 2023-07-21T19:12:01.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:01 smithi101 ceph-mon[128136]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.2 MiB/s wr, 97 op/s 2023-07-21T19:12:01.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:01 smithi005 ceph-mon[126460]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.2 MiB/s wr, 97 op/s 2023-07-21T19:12:01.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:01 smithi156 ceph-mon[128316]: pgmap v135: 97 pgs: 97 active+clean; 3.3 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 9.2 MiB/s wr, 97 op/s 2023-07-21T19:12:01.968 INFO:teuthology.orchestra.run.smithi005.stderr:14700000 of 50000000 tuples (29%) done (elapsed 239.53 s, remaining 575.19 s) 2023-07-21T19:12:03.280 INFO:teuthology.orchestra.run.smithi005.stderr:14800000 of 50000000 tuples (29%) done (elapsed 240.84 s, remaining 572.81 s) 2023-07-21T19:12:03.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:03 smithi101 ceph-mon[128136]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 115 op/s 2023-07-21T19:12:03.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:03 smithi005 ceph-mon[126460]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 115 op/s 2023-07-21T19:12:03.930 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:03 smithi156 ceph-mon[128316]: pgmap v136: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 19 MiB/s wr, 115 op/s 2023-07-21T19:12:04.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.359+0000 7f89083ff700 1 -- 172.21.15.5:0/3575987930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 msgr2=0x7f89000fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.359+0000 7f89083ff700 1 --2- 172.21.15.5:0/3575987930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f89000fff70 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f88f00098f0 tx=0x7f88f0009c00 comp rx=0 tx=0).stop 2023-07-21T19:12:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 -- 172.21.15.5:0/3575987930 shutdown_connections 2023-07-21T19:12:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 --2- 172.21.15.5:0/3575987930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f89001004b0 0x7f890010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 --2- 172.21.15.5:0/3575987930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8900105b10 0x7f8900105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 --2- 172.21.15.5:0/3575987930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f89000fff70 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 -- 172.21.15.5:0/3575987930 >> 172.21.15.5:0/3575987930 conn(0x7f89000fb600 msgr2=0x7f89000fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 -- 172.21.15.5:0/3575987930 shutdown_connections 2023-07-21T19:12:04.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.360+0000 7f89083ff700 1 -- 172.21.15.5:0/3575987930 wait complete. 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.361+0000 7f89083ff700 1 Processor -- start 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.361+0000 7f89083ff700 1 -- start start 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89001004b0 0x7f8900198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900105b10 0x7f890019c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890019d030 con 0x7f89001004b0 2023-07-21T19:12:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890019d1a0 con 0x7f89000ffb00 2023-07-21T19:12:04.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f89083ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890019d360 con 0x7f8900105b10 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47624/0 (socket says 172.21.15.5:47624) 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.362+0000 7f890619b700 1 -- 172.21.15.5:0/1602847355 learned_addr learned my addr 172.21.15.5:0/1602847355 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890599a700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89001004b0 0x7f8900198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890699c700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900105b10 0x7f890019c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 -- 172.21.15.5:0/1602847355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900105b10 msgr2=0x7f890019c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900105b10 0x7f890019c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 -- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89001004b0 msgr2=0x7f8900198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89001004b0 0x7f8900198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 -- 172.21.15.5:0/1602847355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88fc0095f0 con 0x7f89000ffb00 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.363+0000 7f890619b700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00ec10 tx=0x7f88fc00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.364+0000 7f88f77fe700 1 -- 172.21.15.5:0/1602847355 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc00cd40 con 0x7f89000ffb00 2023-07-21T19:12:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.364+0000 7f88f77fe700 1 -- 172.21.15.5:0/1602847355 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88fc0044b0 con 0x7f89000ffb00 2023-07-21T19:12:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.364+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88f0009520 con 0x7f89000ffb00 2023-07-21T19:12:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.364+0000 7f88f77fe700 1 -- 172.21.15.5:0/1602847355 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc010430 con 0x7f89000ffb00 2023-07-21T19:12:04.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.364+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89001a4ca0 con 0x7f89000ffb00 2023-07-21T19:12:04.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.368+0000 7f88f77fe700 1 -- 172.21.15.5:0/1602847355 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f88fc004000 con 0x7f89000ffb00 2023-07-21T19:12:04.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.369+0000 7f891be1fb80 1 Processor -- start 2023-07-21T19:12:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.369+0000 7f891be1fb80 1 -- start start 2023-07-21T19:12:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 0x562616f02540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efcdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616df9c50 con 0x562616d506b0 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616e4c9e0 con 0x562616f02a80 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f891be1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616e45e70 con 0x562616f02ed0 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f88f77fe700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 0x7f88ec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.370+0000 7f88f77fe700 1 -- 172.21.15.5:0/1602847355 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f88fc014070 con 0x7f89000ffb00 2023-07-21T19:12:04.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890699c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 0x562616f02540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890699c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 0x562616f02540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34770/0 (socket says 172.21.15.5:34770) 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890699c700 1 -- 172.21.15.5:0/559207800 learned_addr learned my addr 172.21.15.5:0/559207800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47632/0 (socket says 172.21.15.5:47632) 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890599a700 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efcdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890699c700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 0x7f88ec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 -- 172.21.15.5:0/559207800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 msgr2=0x562616efcdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efcdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 -- 172.21.15.5:0/559207800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 msgr2=0x562616f02540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 0x562616f02540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 -- 172.21.15.5:0/559207800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562616c73d80 con 0x562616f02a80 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.371+0000 7f890619b700 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0ca870 tx=0x7f88fc099d30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3acd8657ab289d0e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.372+0000 7f890699c700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 0x7f88ec0a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f88f8012f00 tx=0x7f88f800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.372+0000 7f88f4bfe700 1 -- 172.21.15.5:0/559207800 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc09a1f0 con 0x562616f02a80 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.372+0000 7f88f4bfe700 1 -- 172.21.15.5:0/559207800 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88fc09e220 con 0x562616f02a80 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.372+0000 7f88f4bfe700 1 -- 172.21.15.5:0/559207800 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc086210 con 0x562616f02a80 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 -- 172.21.15.5:0/559207800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 msgr2=0x562616efc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0ca870 tx=0x7f88fc099d30 comp rx=0 tx=0).stop 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 -- 172.21.15.5:0/559207800 shutdown_connections 2023-07-21T19:12:04.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efcdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616d506b0 0x562616f02540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 --2- 172.21.15.5:0/559207800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616f02a80 0x562616efc850 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 -- 172.21.15.5:0/559207800 >> 172.21.15.5:0/559207800 conn(0x562616ee9be0 msgr2=0x562616eeef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.373+0000 7f891be1fb80 1 -- 172.21.15.5:0/559207800 shutdown_connections 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.374+0000 7f891be1fb80 1 -- 172.21.15.5:0/559207800 wait complete. 2023-07-21T19:12:04.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.374+0000 7f891be1fb80 1 -- start start 2023-07-21T19:12:04.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.374+0000 7f891be1fb80 1 Processor -- start 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616f060a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616f066b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616de5a60 con 0x562616f02a80 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616de5760 con 0x562616d506b0 2023-07-21T19:12:04.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616cc98c0 con 0x562616f02ed0 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f891be1fb80 10 client.0 _ll_register_callbacks cb 0x562616de3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47636/0 (socket says 172.21.15.5:47636) 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f890619b700 1 -- 172.21.15.5:0/819834223 learned_addr learned my addr 172.21.15.5:0/819834223 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f890699c700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616f066b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.375+0000 7f890599a700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616f060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 -- 172.21.15.5:0/819834223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 msgr2=0x562616f066b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616f066b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 -- 172.21.15.5:0/819834223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 msgr2=0x562616f060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616f060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 -- 172.21.15.5:0/819834223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89001a4ca0 con 0x562616d506b0 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890599a700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616f060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f890619b700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0a2110 tx=0x7f88fc003900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=23eedb0e9d2465d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f88e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc09e030 con 0x562616d506b0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f88fc09a1f0 con 0x562616d506b0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562616d9f490 con 0x562616d506b0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.376+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88fc0042b0 con 0x562616d506b0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 10 client.26351 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562616c5cc00 con 0x562616d506b0 2023-07-21T19:12:04.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562616d9f490 con 0x562616d506b0 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f88fc09a4e0 con 0x562616d506b0 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 10 client.26351 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 10 client.26351 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 20 client.26351 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f16000 0x562616f183e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 4 client.26351 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:04.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.377+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562616e3ee50 con 0x562616f16000 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.378+0000 7f890599a700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f16000 0x562616f183e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.378+0000 7f88e9ffb700 10 client.26351 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.378+0000 7f890599a700 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f16000 0x562616f183e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4253a1a1f7e3e976 server_cookie=1b520e0a62e7cc36 in_seq=0 out_seq=0 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f88e9ffb700 1 -- 172.21.15.5:0/819834223 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562616e3ee50 con 0x562616f16000 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f88e9ffb700 10 client.26351 handle_command_reply: tid=0 2023-07-21T19:12:04.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 client.26351 shutdown 2023-07-21T19:12:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 msgr2=0x562616f08df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f88fc0a2110 tx=0x7f88fc003900 comp rx=0 tx=0).stop 2023-07-21T19:12:04.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 shutdown_connections 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616f066b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f08df0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f16000 0x562616f183e0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 --2- 172.21.15.5:0/819834223 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616f060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f88e9ffb700 0 client.26351 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 >> 172.21.15.5:0/819834223 conn(0x562616ee9be0 msgr2=0x562616eedc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.379+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 shutdown_connections 2023-07-21T19:12:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.380+0000 7f891be1fb80 1 -- 172.21.15.5:0/819834223 wait complete. 2023-07-21T19:12:04.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.380+0000 7f891be1fb80 20 client.26351 trim_cache size 0 max 16384 2023-07-21T19:12:04.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 Processor -- start 2023-07-21T19:12:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 -- start start 2023-07-21T19:12:04.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02a80 0x562616f01870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02ed0 0x562616eeec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616eba060 con 0x562616f02ed0 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616f06cb0 con 0x562616d506b0 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.396+0000 7f891be1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616ee6b80 con 0x562616f02a80 2023-07-21T19:12:04.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890699c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02ed0 0x562616eeec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47644/0 (socket says 172.21.15.5:47644) 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 -- 172.21.15.5:0/3088947161 learned_addr learned my addr 172.21.15.5:0/3088947161 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890599a700 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02a80 0x562616f01870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 -- 172.21.15.5:0/3088947161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02a80 msgr2=0x562616f01870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02a80 0x562616f01870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 -- 172.21.15.5:0/3088947161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02ed0 msgr2=0x562616eeec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02ed0 0x562616eeec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 -- 172.21.15.5:0/3088947161 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562616f1cfe0 con 0x562616d506b0 2023-07-21T19:12:04.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.397+0000 7f890619b700 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f88fc098540 tx=0x7f88fc00ef90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=956183a0148d5105 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f88f53ff700 1 -- 172.21.15.5:0/3088947161 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc009a10 con 0x562616d506b0 2023-07-21T19:12:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f88f53ff700 1 -- 172.21.15.5:0/3088947161 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88fc09e030 con 0x562616d506b0 2023-07-21T19:12:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f88f53ff700 1 -- 172.21.15.5:0/3088947161 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88fc0caaf0 con 0x562616d506b0 2023-07-21T19:12:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 -- 172.21.15.5:0/3088947161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 msgr2=0x562616f0d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f88fc098540 tx=0x7f88fc00ef90 comp rx=0 tx=0).stop 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 -- 172.21.15.5:0/3088947161 shutdown_connections 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02ed0 0x562616eeec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616f0d4a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 --2- 172.21.15.5:0/3088947161 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02a80 0x562616f01870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 -- 172.21.15.5:0/3088947161 >> 172.21.15.5:0/3088947161 conn(0x562616ee9be0 msgr2=0x562616f01160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.398+0000 7f891be1fb80 1 -- 172.21.15.5:0/3088947161 shutdown_connections 2023-07-21T19:12:04.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.399+0000 7f891be1fb80 1 -- 172.21.15.5:0/3088947161 wait complete. 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.399+0000 7f891be1fb80 1 -- start start 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.399+0000 7f891be1fb80 1 Processor -- start 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616efdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616eff130 con 0x562616f02a80 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616eff2a0 con 0x562616d506b0 2023-07-21T19:12:04.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562616eff460 con 0x562616f02ed0 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f891be1fb80 10 client.0 _ll_register_callbacks cb 0x7f88f8012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34806/0 (socket says 172.21.15.5:34806) 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 -- 172.21.15.5:0/604580385 learned_addr learned my addr 172.21.15.5:0/604580385 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890619b700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616efdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890699c700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 -- 172.21.15.5:0/604580385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 msgr2=0x562616efea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 -- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 msgr2=0x562616efdfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616efdfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890599a700 1 -- 172.21.15.5:0/604580385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562616c5cc00 con 0x562616f02a80 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.400+0000 7f890619b700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616efdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f890599a700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f88f00092a0 tx=0x7f88f000f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9159c8c0f5b12b6d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:04.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f88e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f0026cc0 con 0x562616f02a80 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f88f0026e20 con 0x562616f02a80 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f88fc09e220 con 0x562616f02a80 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f88f0026040 con 0x562616f02a80 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f891be1fb80 10 client.16668 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.401+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x562616c9a1f0 con 0x562616f02a80 2023-07-21T19:12:04.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f88fc09e220 con 0x562616f02a80 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f88f0015020 con 0x562616f02a80 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 10 client.16668 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 10 client.16668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 20 client.16668 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f6aa50 0x562616f6ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 4 client.16668 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:12:04.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562616e00ff0 con 0x562616f6aa50 2023-07-21T19:12:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.402+0000 7f890619b700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f6aa50 0x562616f6ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.403+0000 7f890619b700 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f6aa50 0x562616f6ce60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16d2676cc3bf0592 server_cookie=87f0a0f76df2159 in_seq=0 out_seq=0 2023-07-21T19:12:04.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.403+0000 7f88e9ffb700 10 client.16668 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.416+0000 7f88e9ffb700 1 -- 172.21.15.5:0/604580385 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562616e00ff0 con 0x562616f6aa50 2023-07-21T19:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.416+0000 7f88e9ffb700 10 client.16668 handle_command_reply: tid=0 2023-07-21T19:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.416+0000 7f891be1fb80 1 client.16668 shutdown 2023-07-21T19:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 msgr2=0x562616efe4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f88f00092a0 tx=0x7f88f000f460 comp rx=0 tx=0).stop 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 shutdown_connections 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562616f02ed0 0x562616efea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562616d506b0 0x562616efdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562616f6aa50 0x562616f6ce60 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 --2- 172.21.15.5:0/604580385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562616f02a80 0x562616efe4e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 >> 172.21.15.5:0/604580385 conn(0x562616ee9be0 msgr2=0x562616ef38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f88e9ffb700 0 client.16668 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 shutdown_connections 2023-07-21T19:12:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 1 -- 172.21.15.5:0/604580385 wait complete. 2023-07-21T19:12:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.417+0000 7f891be1fb80 20 client.16668 trim_cache size 0 max 16384 2023-07-21T19:12:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:12:04.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:12:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:12:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:12:04.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 msgr2=0x7f88ec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 0x7f88ec0a37f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f88f8012f00 tx=0x7f88f800b040 comp rx=0 tx=0).stop 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 msgr2=0x7f8900197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f88fc00ec10 tx=0x7f88fc00ef20 comp rx=0 tx=0).stop 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 shutdown_connections 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8900105b10 0x7f890019c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f89000ffb00 0x7f8900197e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88ec0a1340 0x7f88ec0a37f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 --2- 172.21.15.5:0/1602847355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f89001004b0 0x7f8900198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 >> 172.21.15.5:0/1602847355 conn(0x7f89000fb600 msgr2=0x7f890010afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 shutdown_connections 2023-07-21T19:12:04.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:04.418+0000 7f89083ff700 1 -- 172.21.15.5:0/1602847355 wait complete. 2023-07-21T19:12:04.671 INFO:teuthology.orchestra.run.smithi005.stderr:14900000 of 50000000 tuples (29%) done (elapsed 242.23 s, remaining 570.62 s) 2023-07-21T19:12:05.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:05 smithi101 ceph-mon[128136]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 40 op/s 2023-07-21T19:12:05.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:05 smithi005 ceph-mon[126460]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 40 op/s 2023-07-21T19:12:05.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:05 smithi156 ceph-mon[128316]: pgmap v137: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 20 MiB/s wr, 40 op/s 2023-07-21T19:12:05.971 INFO:teuthology.orchestra.run.smithi005.stderr:15000000 of 50000000 tuples (30%) done (elapsed 243.53 s, remaining 568.24 s) 2023-07-21T19:12:07.564 INFO:teuthology.orchestra.run.smithi005.stderr:15100000 of 50000000 tuples (30%) done (elapsed 245.12 s, remaining 566.54 s) 2023-07-21T19:12:07.837 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:07 smithi101 ceph-mon[128136]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 28 op/s 2023-07-21T19:12:07.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:07 smithi005 ceph-mon[126460]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 28 op/s 2023-07-21T19:12:07.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:07 smithi156 ceph-mon[128316]: pgmap v138: 97 pgs: 97 active+clean; 3.4 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 MiB/s wr, 28 op/s 2023-07-21T19:12:08.876 INFO:teuthology.orchestra.run.smithi005.stderr:15200000 of 50000000 tuples (30%) done (elapsed 246.44 s, remaining 564.21 s) 2023-07-21T19:12:09.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:09 smithi101 ceph-mon[128136]: pgmap v139: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 42 op/s 2023-07-21T19:12:09.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:09 smithi005 ceph-mon[126460]: pgmap v139: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 42 op/s 2023-07-21T19:12:09.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:09 smithi156 ceph-mon[128316]: pgmap v139: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 30 MiB/s wr, 42 op/s 2023-07-21T19:12:10.259 INFO:teuthology.orchestra.run.smithi005.stderr:15300000 of 50000000 tuples (30%) done (elapsed 247.82 s, remaining 562.05 s) 2023-07-21T19:12:11.554 INFO:teuthology.orchestra.run.smithi005.stderr:15400000 of 50000000 tuples (30%) done (elapsed 249.11 s, remaining 559.70 s) 2023-07-21T19:12:11.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:11 smithi101 ceph-mon[128136]: pgmap v140: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:11.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:11 smithi005 ceph-mon[126460]: pgmap v140: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:11.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:11 smithi156 ceph-mon[128316]: pgmap v140: 97 pgs: 97 active+clean; 3.5 GiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:12.842 INFO:teuthology.orchestra.run.smithi005.stderr:15500000 of 50000000 tuples (31%) done (elapsed 250.40 s, remaining 557.35 s) 2023-07-21T19:12:13.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:13 smithi101 ceph-mon[128136]: pgmap v141: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 40 op/s 2023-07-21T19:12:13.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:13 smithi005 ceph-mon[126460]: pgmap v141: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 40 op/s 2023-07-21T19:12:13.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:13 smithi156 ceph-mon[128316]: pgmap v141: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 40 op/s 2023-07-21T19:12:14.212 INFO:teuthology.orchestra.run.smithi005.stderr:15600000 of 50000000 tuples (31%) done (elapsed 251.77 s, remaining 555.19 s) 2023-07-21T19:12:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:15700000 of 50000000 tuples (31%) done (elapsed 253.04 s, remaining 552.82 s) 2023-07-21T19:12:15.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:15 smithi101 ceph-mon[128136]: pgmap v142: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:12:15.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:15 smithi005 ceph-mon[126460]: pgmap v142: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:12:15.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:15 smithi156 ceph-mon[128316]: pgmap v142: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:12:16.772 INFO:teuthology.orchestra.run.smithi005.stderr:15800000 of 50000000 tuples (31%) done (elapsed 254.33 s, remaining 550.52 s) 2023-07-21T19:12:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.592+0000 7f8c183ff700 1 -- 172.21.15.5:0/1682382719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c10105b10 msgr2=0x7f8c10105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.592+0000 7f8c183ff700 1 --2- 172.21.15.5:0/1682382719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c10105b10 0x7f8c10105ee0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009990 tx=0x7f8c0c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 -- 172.21.15.5:0/1682382719 shutdown_connections 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 --2- 172.21.15.5:0/1682382719 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c101004b0 0x7f8c1010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 --2- 172.21.15.5:0/1682382719 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c10105b10 0x7f8c10105ee0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 --2- 172.21.15.5:0/1682382719 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c100fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 -- 172.21.15.5:0/1682382719 >> 172.21.15.5:0/1682382719 conn(0x7f8c100fb600 msgr2=0x7f8c100fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 -- 172.21.15.5:0/1682382719 shutdown_connections 2023-07-21T19:12:17.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.593+0000 7f8c183ff700 1 -- 172.21.15.5:0/1682382719 wait complete. 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.594+0000 7f8c183ff700 1 Processor -- start 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.594+0000 7f8c183ff700 1 -- start start 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c10197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c10105b10 0x7f8c1019c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1019cf60 con 0x7f8c100ffb00 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1019d0d0 con 0x7f8c101004b0 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c183ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c1019d290 con 0x7f8c10105b10 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c10197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55430/0 (socket says 172.21.15.5:55430) 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 -- 172.21.15.5:0/4105385547 learned_addr learned my addr 172.21.15.5:0/4105385547 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1699c700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c10105b10 0x7f8c1019c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 -- 172.21.15.5:0/4105385547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c10105b10 msgr2=0x7f8c1019c920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c10105b10 0x7f8c1019c920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 -- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 msgr2=0x7f8c10197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c10197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1599a700 1 -- 172.21.15.5:0/4105385547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00009520 con 0x7f8c101004b0 2023-07-21T19:12:17.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.595+0000 7f8c1619b700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c10197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:17.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c1599a700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000eb30 tx=0x7f8c0000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c077fe700 1 -- 172.21.15.5:0/4105385547 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00009880 con 0x7f8c101004b0 2023-07-21T19:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c077fe700 1 -- 172.21.15.5:0/4105385547 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c000044b0 con 0x7f8c101004b0 2023-07-21T19:12:17.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c0c0095f0 con 0x7f8c101004b0 2023-07-21T19:12:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c077fe700 1 -- 172.21.15.5:0/4105385547 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00003ce0 con 0x7f8c101004b0 2023-07-21T19:12:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.596+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c101a4bd0 con 0x7f8c101004b0 2023-07-21T19:12:17.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.600+0000 7f8c077fe700 1 -- 172.21.15.5:0/4105385547 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8c00003e40 con 0x7f8c101004b0 2023-07-21T19:12:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.601+0000 7f8c2bcfcb80 1 Processor -- start 2023-07-21T19:12:17.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.601+0000 7f8c2bcfcb80 1 -- start start 2023-07-21T19:12:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.601+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78b1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78ac580 0x55c3d78ac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d77b1c50 con 0x55c3d77086b0 2023-07-21T19:12:17.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78049e0 con 0x55c3d78b22d0 2023-07-21T19:12:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d77fde70 con 0x55c3d78ac580 2023-07-21T19:12:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1699c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78b1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55438/0 (socket says 172.21.15.5:55438) 2023-07-21T19:12:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 -- 172.21.15.5:0/448264881 learned_addr learned my addr 172.21.15.5:0/448264881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c077fe700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 0x7f8bfc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1599a700 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78ac580 0x55c3d78ac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c077fe700 1 -- 172.21.15.5:0/4105385547 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8c00014070 con 0x7f8c101004b0 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 -- 172.21.15.5:0/448264881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78ac580 msgr2=0x55c3d78ac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78ac580 0x55c3d78ac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 -- 172.21.15.5:0/448264881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 msgr2=0x55c3d78b1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78b1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1619b700 1 -- 172.21.15.5:0/448264881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d762bd80 con 0x55c3d78b22d0 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.602+0000 7f8c1699c700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 0x7f8bfc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.603+0000 7f8c1619b700 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c015000 tx=0x7f8c0c005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4482027154d5d488 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.603+0000 7f8c04bfe700 1 -- 172.21.15.5:0/448264881 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c01d050 con 0x55c3d78b22d0 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.603+0000 7f8c1699c700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 0x7f8bfc0a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ec0 tx=0x7f8c08004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.603+0000 7f8c04bfe700 1 -- 172.21.15.5:0/448264881 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c01e040 con 0x55c3d78b22d0 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.603+0000 7f8c04bfe700 1 -- 172.21.15.5:0/448264881 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0053b0 con 0x55c3d78b22d0 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/448264881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 msgr2=0x55c3d78ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c015000 tx=0x7f8c0c005ec0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/448264881 shutdown_connections 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78ac580 0x55c3d78ac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78b1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/448264881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ac040 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/448264881 >> 172.21.15.5:0/448264881 conn(0x55c3d78a1be0 msgr2=0x55c3d78a6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/448264881 shutdown_connections 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.604+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/448264881 wait complete. 2023-07-21T19:12:17.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.605+0000 7f8c2bcfcb80 1 -- start start 2023-07-21T19:12:17.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.605+0000 7f8c2bcfcb80 1 Processor -- start 2023-07-21T19:12:17.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.605+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.605+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78b5780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.605+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b5d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d779da60 con 0x55c3d78ac580 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d779d760 con 0x55c3d77086b0 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d76818c0 con 0x55c3d78b22d0 2023-07-21T19:12:17.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c2bcfcb80 10 client.0 _ll_register_callbacks cb 0x55c3d779b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55440/0 (socket says 172.21.15.5:55440) 2023-07-21T19:12:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 -- 172.21.15.5:0/126050028 learned_addr learned my addr 172.21.15.5:0/126050028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1599a700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78b5780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1699c700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 -- 172.21.15.5:0/126050028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 msgr2=0x55c3d78b5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 -- 172.21.15.5:0/126050028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 msgr2=0x55c3d78b5780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78b5780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1619b700 1 -- 172.21.15.5:0/126050028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1010d240 con 0x55c3d77086b0 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.606+0000 7f8c1699c700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8c1619b700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006010 tx=0x7f8c0c004a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9d92fb7bc36738c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c0053a0 con 0x55c3d77086b0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8c0c005500 con 0x55c3d77086b0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3d7757490 con 0x55c3d77086b0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c0c01d020 con 0x55c3d77086b0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8c2bcfcb80 10 client.26369 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3d7614c00 con 0x55c3d77086b0 2023-07-21T19:12:17.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.607+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3d7757490 con 0x55c3d77086b0 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8c0c023cd0 con 0x55c3d77086b0 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 10 client.26369 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 10 client.26369 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 20 client.26369 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d78c55f0 0x55c3d78c79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 4 client.26369 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3d77f6e50 con 0x55c3d78c55f0 2023-07-21T19:12:17.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.608+0000 7f8c1599a700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d78c55f0 0x55c3d78c79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.609+0000 7f8c1599a700 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d78c55f0 0x55c3d78c79d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3f808acb22672ed server_cookie=c64dc02f1bd4f67c in_seq=0 out_seq=0 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.609+0000 7f8bf9ffb700 10 client.26369 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.609+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/126050028 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3d77f6e50 con 0x55c3d78c55f0 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8bf9ffb700 10 client.26369 handle_command_reply: tid=0 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 client.26369 shutdown 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 msgr2=0x55c3d78b8500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c006010 tx=0x7f8c0c004a10 comp rx=0 tx=0).stop 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 shutdown_connections 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d77086b0 0x55c3d78b8500 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d78c55f0 0x55c3d78c79d0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/126050028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78b5780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8bf9ffb700 0 client.26369 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 >> 172.21.15.5:0/126050028 conn(0x55c3d78a1be0 msgr2=0x55c3d78a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 shutdown_connections 2023-07-21T19:12:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/126050028 wait complete. 2023-07-21T19:12:17.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.610+0000 7f8c2bcfcb80 20 client.26369 trim_cache size 0 max 16384 2023-07-21T19:12:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 Processor -- start 2023-07-21T19:12:17.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 -- start start 2023-07-21T19:12:17.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 0x55c3d78bcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b80a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78b1a40 con 0x55c3d77086b0 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78a3d30 con 0x55c3d78ac580 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.629+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d789ec30 con 0x55c3d78b22d0 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36864/0 (socket says 172.21.15.5:36864) 2023-07-21T19:12:17.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 -- 172.21.15.5:0/1091828611 learned_addr learned my addr 172.21.15.5:0/1091828611 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1599a700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 0x55c3d78bcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1699c700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 -- 172.21.15.5:0/1091828611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 msgr2=0x55c3d78b80a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b80a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 -- 172.21.15.5:0/1091828611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 msgr2=0x55c3d78bcc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 0x55c3d78bcc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1619b700 1 -- 172.21.15.5:0/1091828611 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3d78a8880 con 0x55c3d77086b0 2023-07-21T19:12:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.630+0000 7f8c1599a700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 0x55c3d78bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:17.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c1619b700 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009350 tx=0x7f8c0c00bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61f645f856ffce6a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c053ff700 1 -- 172.21.15.5:0/1091828611 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c01d040 con 0x55c3d77086b0 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c053ff700 1 -- 172.21.15.5:0/1091828611 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c004a80 con 0x55c3d77086b0 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c053ff700 1 -- 172.21.15.5:0/1091828611 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0c005400 con 0x55c3d77086b0 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/1091828611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 msgr2=0x55c3d78ac040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.631+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f8c0c009350 tx=0x7f8c0c00bfd0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/1091828611 shutdown_connections 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d78b22d0 0x55c3d78b80a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d77086b0 0x55c3d78ac040 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/1091828611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78ac580 0x55c3d78bcc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/1091828611 >> 172.21.15.5:0/1091828611 conn(0x55c3d78a1be0 msgr2=0x55c3d78afa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/1091828611 shutdown_connections 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/1091828611 wait complete. 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.632+0000 7f8c2bcfcb80 1 -- start start 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 Processor -- start 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d77086b0 0x55c3d78ae810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ad790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78af630 con 0x55c3d78ac580 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78bcf00 con 0x55c3d78b22d0 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3d78bd0c0 con 0x55c3d77086b0 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.633+0000 7f8c2bcfcb80 10 client.0 _ll_register_callbacks cb 0x7f8c08009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36876/0 (socket says 172.21.15.5:36876) 2023-07-21T19:12:17.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 -- 172.21.15.5:0/454695824 learned_addr learned my addr 172.21.15.5:0/454695824 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1699c700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ad790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1619b700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d77086b0 0x55c3d78ae810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 -- 172.21.15.5:0/454695824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d77086b0 msgr2=0x55c3d78ae810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d77086b0 0x55c3d78ae810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 -- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 msgr2=0x55c3d78ad790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ad790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 -- 172.21.15.5:0/454695824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c101a4bd0 con 0x55c3d78ac580 2023-07-21T19:12:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8c1599a700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8c00096790 tx=0x7f8c00004620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=290ecd78716cbbbf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.634+0000 7f8bf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:12:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00094ea0 con 0x55c3d78ac580 2023-07-21T19:12:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f8c0009a290 con 0x55c3d78ac580 2023-07-21T19:12:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c0c01e040 con 0x55c3d78ac580 2023-07-21T19:12:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c000b6050 con 0x55c3d78ac580 2023-07-21T19:12:17.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 10 client.16686 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55c3d7614c00 con 0x55c3d78ac580 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c0c01e040 con 0x55c3d78ac580 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f8c0009a4f0 con 0x55c3d78ac580 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 10 client.16686 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 10 client.16686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 20 client.16686 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:17.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d7919de0 0x55c3d791c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 4 client.16686 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:12:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.635+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3d77b8ff0 con 0x55c3d7919de0 2023-07-21T19:12:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.636+0000 7f8c1619b700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d7919de0 0x55c3d791c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.636+0000 7f8bf9ffb700 10 client.16686 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.636+0000 7f8c1619b700 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d7919de0 0x55c3d791c1f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e38fc9985bc4395 server_cookie=500dbb2b2ac07424 in_seq=0 out_seq=0 2023-07-21T19:12:17.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8bf9ffb700 1 -- 172.21.15.5:0/454695824 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3d77b8ff0 con 0x55c3d7919de0 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8bf9ffb700 10 client.16686 handle_command_reply: tid=0 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 client.16686 shutdown 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 msgr2=0x55c3d78aed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f8c00096790 tx=0x7f8c00004620 comp rx=0 tx=0).stop 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 shutdown_connections 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c3d78b22d0 0x55c3d78ad790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c3d77086b0 0x55c3d78ae810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3d7919de0 0x55c3d791c1f0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 --2- 172.21.15.5:0/454695824 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c3d78ac580 0x55c3d78aed50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8bf9ffb700 0 client.16686 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 >> 172.21.15.5:0/454695824 conn(0x55c3d78a1be0 msgr2=0x55c3d78b0900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.648+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 shutdown_connections 2023-07-21T19:12:17.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c2bcfcb80 1 -- 172.21.15.5:0/454695824 wait complete. 2023-07-21T19:12:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c2bcfcb80 20 client.16686 trim_cache size 0 max 16384 2023-07-21T19:12:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:12:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:12:17.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:12:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:12:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 msgr2=0x7f8bfc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 0x7f8bfc0a3770 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8c08009ec0 tx=0x7f8c08004b40 comp rx=0 tx=0).stop 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 msgr2=0x7f8c101982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000eb30 tx=0x7f8c0000ee40 comp rx=0 tx=0).stop 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.649+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 shutdown_connections 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c10105b10 0x7f8c1019c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bfc0a12c0 0x7f8bfc0a3770 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c100ffb00 0x7f8c10197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 --2- 172.21.15.5:0/4105385547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c101004b0 0x7f8c101982b0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 >> 172.21.15.5:0/4105385547 conn(0x7f8c100fb600 msgr2=0x7f8c100fbc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 shutdown_connections 2023-07-21T19:12:17.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:17.650+0000 7f8c183ff700 1 -- 172.21.15.5:0/4105385547 wait complete. 2023-07-21T19:12:17.664 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:12:17.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:17 smithi101 ceph-mon[128136]: pgmap v143: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 21 op/s 2023-07-21T19:12:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 -- 172.21.15.5:0/354112097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc106fb0 msgr2=0x7f50cc10eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 --2- 172.21.15.5:0/354112097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc106fb0 0x7f50cc10eb80 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f50c400b0d0 tx=0x7f50c400b3e0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 -- 172.21.15.5:0/354112097 shutdown_connections 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 --2- 172.21.15.5:0/354112097 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc106fb0 0x7f50cc10eb80 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 --2- 172.21.15.5:0/354112097 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 0x7f50cc102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 --2- 172.21.15.5:0/354112097 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc102910 0x7f50cc1068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 -- 172.21.15.5:0/354112097 >> 172.21.15.5:0/354112097 conn(0x7f50cc0fd7e0 msgr2=0x7f50cc0ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 -- 172.21.15.5:0/354112097 shutdown_connections 2023-07-21T19:12:17.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.896+0000 7f50d11ff700 1 -- 172.21.15.5:0/354112097 wait complete. 2023-07-21T19:12:17.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.897+0000 7f50d11ff700 1 Processor -- start 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.897+0000 7f50d11ff700 1 -- start start 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 0x7f50cc197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 0x7f50cc19c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19ce00 con 0x7f50cc102910 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19cf70 con 0x7f50cc106fb0 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50d11ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19d130 con 0x7f50cc101f70 2023-07-21T19:12:17.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36892/0 (socket says 172.21.15.5:36892) 2023-07-21T19:12:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50ca59c700 1 -- 172.21.15.5:0/3512495585 learned_addr learned my addr 172.21.15.5:0/3512495585 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50cad9d700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 0x7f50cc197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.898+0000 7f50cb59e700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 0x7f50cc19c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 -- 172.21.15.5:0/3512495585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 msgr2=0x7f50cc197e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 0x7f50cc197e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 -- 172.21.15.5:0/3512495585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 msgr2=0x7f50cc19c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 0x7f50cc19c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 -- 172.21.15.5:0/3512495585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c4009d00 con 0x7f50cc102910 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50cb59e700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 0x7f50cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50ca59c700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f50c000e940 tx=0x7f50c000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50b3fff700 1 -- 172.21.15.5:0/3512495585 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0009730 con 0x7f50cc102910 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc19d410 con 0x7f50cc102910 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50b3fff700 1 -- 172.21.15.5:0/3512495585 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50c000ce70 con 0x7f50cc102910 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50b3fff700 1 -- 172.21.15.5:0/3512495585 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0018420 con 0x7f50cc102910 2023-07-21T19:12:17.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.899+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc1a4c50 con 0x7f50cc102910 2023-07-21T19:12:17.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.904+0000 7f50b3fff700 1 -- 172.21.15.5:0/3512495585 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f50c000c9e0 con 0x7f50cc102910 2023-07-21T19:12:17.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.905+0000 7f50e4b6eb80 1 Processor -- start 2023-07-21T19:12:17.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 -- start start 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202b2d0 0x558af202b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2036590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 0x558af2038eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1f31cb0 con 0x558af202bbe0 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1f84a40 con 0x558af2036ad0 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.906+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1f7ded0 con 0x558af202b2d0 2023-07-21T19:12:17.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50b3fff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 0x7f50b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 0x558af2038eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 0x558af2038eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55486/0 (socket says 172.21.15.5:55486) 2023-07-21T19:12:17.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50ca59c700 1 -- 172.21.15.5:0/3560931656 learned_addr learned my addr 172.21.15.5:0/3560931656 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50cad9d700 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2036590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50cb59e700 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202b2d0 0x558af202b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50b3fff700 1 -- 172.21.15.5:0/3512495585 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f50c0014070 con 0x7f50cc102910 2023-07-21T19:12:17.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.907+0000 7f50cb59e700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 0x7f50b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.915 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:17 smithi005 ceph-mon[126460]: pgmap v143: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 21 op/s 2023-07-21T19:12:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 -- 172.21.15.5:0/3560931656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202b2d0 msgr2=0x558af202b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202b2d0 0x558af202b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 -- 172.21.15.5:0/3560931656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 msgr2=0x558af2038eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 0x558af2038eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 -- 172.21.15.5:0/3560931656 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af1dabde0 con 0x558af202bbe0 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cad9d700 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2036590 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bc80 tx=0x7f50bc00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05913996ecaad7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.908+0000 7f50cb59e700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 0x7f50b40a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007f80 tx=0x7f50c4006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.909+0000 7f50b17fa700 1 -- 172.21.15.5:0/3560931656 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004230 con 0x558af202bbe0 2023-07-21T19:12:17.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.909+0000 7f50b17fa700 1 -- 172.21.15.5:0/3560931656 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50bc015070 con 0x558af202bbe0 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.909+0000 7f50b17fa700 1 -- 172.21.15.5:0/3560931656 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc004cf0 con 0x558af202bbe0 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.909+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/3560931656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 msgr2=0x558af2036590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.909+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2036590 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00bc80 tx=0x7f50bc00bf90 comp rx=0 tx=0).stop 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/3560931656 shutdown_connections 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af2036ad0 0x558af2038eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202b2d0 0x558af202b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/3560931656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2036590 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/3560931656 >> 172.21.15.5:0/3560931656 conn(0x558af2021da0 msgr2=0x558af20276a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/3560931656 shutdown_connections 2023-07-21T19:12:17.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/3560931656 wait complete. 2023-07-21T19:12:17.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.910+0000 7f50e4b6eb80 1 -- start start 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.911+0000 7f50e4b6eb80 1 Processor -- start 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2033d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20342d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1f1dac0 con 0x558af202bbe0 2023-07-21T19:12:17.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1f1d7c0 con 0x558af202b2d0 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af1e01920 con 0x558af2036ad0 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50e4b6eb80 10 client.0 _ll_register_callbacks cb 0x558af1f1b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55502/0 (socket says 172.21.15.5:55502) 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 -- 172.21.15.5:0/1965219038 learned_addr learned my addr 172.21.15.5:0/1965219038 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50ca59c700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2033d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 -- 172.21.15.5:0/1965219038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 msgr2=0x558af20342d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20342d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 -- 172.21.15.5:0/1965219038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 msgr2=0x558af2033d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2033d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50cad9d700 1 -- 172.21.15.5:0/1965219038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50bc00b8e0 con 0x558af202b2d0 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.912+0000 7f50ca59c700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2033d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50cad9d700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b2b0 tx=0x7f50bc010c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd22ccbb3ad69405 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:17.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc021070 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558af1ed74f0 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f50bc017070 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50bc010e90 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.913+0000 7f50e4b6eb80 10 client.26387 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558af1d94c60 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558af1ed74f0 con 0x558af202b2d0 2023-07-21T19:12:17.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f50bc01b780 con 0x558af202b2d0 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 10 client.26387 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 10 client.26387 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 20 client.26387 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af2046340 0x558af2048750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 4 client.26387 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558af1f76eb0 con 0x558af2046340 2023-07-21T19:12:17.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.914+0000 7f50ca59c700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af2046340 0x558af2048750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:17 smithi156 ceph-mon[128316]: pgmap v143: 97 pgs: 97 active+clean; 3.6 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 22 MiB/s wr, 21 op/s 2023-07-21T19:12:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.915+0000 7f50aeff5700 10 client.26387 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.915+0000 7f50ca59c700 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af2046340 0x558af2048750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fdc63471c83c282 server_cookie=3f0e11416ce64569 in_seq=0 out_seq=0 2023-07-21T19:12:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50aeff5700 1 -- 172.21.15.5:0/1965219038 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558af1f76eb0 con 0x558af2046340 2023-07-21T19:12:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50aeff5700 10 client.26387 handle_command_reply: tid=0 2023-07-21T19:12:17.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 client.26387 shutdown 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 msgr2=0x558af2033850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f50bc00b2b0 tx=0x7f50bc010c40 comp rx=0 tx=0).stop 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 shutdown_connections 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20342d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af2033850 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af2046340 0x558af2048750 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/1965219038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202bbe0 0x558af2033d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50aeff5700 0 client.26387 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.916+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 >> 172.21.15.5:0/1965219038 conn(0x558af2021da0 msgr2=0x558af2027230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.917+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 shutdown_connections 2023-07-21T19:12:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.917+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/1965219038 wait complete. 2023-07-21T19:12:17.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.917+0000 7f50e4b6eb80 20 client.26387 trim_cache size 0 max 16384 2023-07-21T19:12:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 Processor -- start 2023-07-21T19:12:17.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 -- start start 2023-07-21T19:12:17.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202bbe0 0x558af202eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af2036ad0 0x558af202f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af202d1e0 con 0x558af2036ad0 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af202c570 con 0x558af202b2d0 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.938+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af202b8f0 con 0x558af202bbe0 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af2036ad0 0x558af202f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55508/0 (socket says 172.21.15.5:55508) 2023-07-21T19:12:17.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 -- 172.21.15.5:0/172109440 learned_addr learned my addr 172.21.15.5:0/172109440 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50ca59c700 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202bbe0 0x558af202eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 -- 172.21.15.5:0/172109440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202bbe0 msgr2=0x558af202eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202bbe0 0x558af202eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 -- 172.21.15.5:0/172109440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af2036ad0 msgr2=0x558af202f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af2036ad0 0x558af202f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 -- 172.21.15.5:0/172109440 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558af1dd2250 con 0x558af202b2d0 2023-07-21T19:12:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.939+0000 7f50cad9d700 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50bc01bb90 tx=0x7f50bc01bcb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb0b7e909319e808 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50b1ffb700 1 -- 172.21.15.5:0/172109440 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc021070 con 0x558af202b2d0 2023-07-21T19:12:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50b1ffb700 1 -- 172.21.15.5:0/172109440 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50bc01b8b0 con 0x558af202b2d0 2023-07-21T19:12:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50b1ffb700 1 -- 172.21.15.5:0/172109440 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc0048b0 con 0x558af202b2d0 2023-07-21T19:12:17.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/172109440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 msgr2=0x558af203d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f50bc01bb90 tx=0x7f50bc01bcb0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/172109440 shutdown_connections 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af2036ad0 0x558af202f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202b2d0 0x558af203d4f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/172109440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af202bbe0 0x558af202eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/172109440 >> 172.21.15.5:0/172109440 conn(0x558af2021da0 msgr2=0x558af2042800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.940+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/172109440 shutdown_connections 2023-07-21T19:12:17.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.941+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/172109440 wait complete. 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.941+0000 7f50e4b6eb80 1 -- start start 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.941+0000 7f50e4b6eb80 1 Processor -- start 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 0x558af208e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af2092a80 con 0x558af202b2d0 2023-07-21T19:12:17.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af2092bf0 con 0x558af202bbe0 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558af2092db0 con 0x558af2036ad0 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50e4b6eb80 10 client.0 _ll_register_callbacks cb 0x7f50c0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50cad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 0x558af208e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50ca59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55520/0 (socket says 172.21.15.5:55520) 2023-07-21T19:12:17.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50ca59c700 1 -- 172.21.15.5:0/232790284 learned_addr learned my addr 172.21.15.5:0/232790284 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.942+0000 7f50cb59e700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 -- 172.21.15.5:0/232790284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 msgr2=0x558af20924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 -- 172.21.15.5:0/232790284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 msgr2=0x558af208e380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 0x558af208e380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 -- 172.21.15.5:0/232790284 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558af1d94c60 con 0x558af202bbe0 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50cad9d700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 0x558af208e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:17.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50ca59c700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50c0096340 tx=0x7f50c00cf050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38a99322a7502e8a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50aeff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0099e40 con 0x558af202bbe0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f50c00a2290 con 0x558af202bbe0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50bc015070 con 0x558af202bbe0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c009de70 con 0x558af202bbe0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50e4b6eb80 10 client.26393 fetch_fsmap learned FSMap version 27 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.943+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x558af1daa2b0 con 0x558af202bbe0 2023-07-21T19:12:17.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50bc015070 con 0x558af202bbe0 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 27) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f50c00993c0 con 0x558af202bbe0 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 10 client.26393 fetch_fsmap finished waiting for FSMap version 27 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 10 client.26393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 20 client.26393 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af209b7a0 0x558af209dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 4 client.26393 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:12:17.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558af1f39050 con 0x558af209b7a0 2023-07-21T19:12:17.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.944+0000 7f50cad9d700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af209b7a0 0x558af209dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50aeff5700 10 client.26393 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50cad9d700 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af209b7a0 0x558af209dbb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8bc970cc7d2cffa server_cookie=dcb3d78c534ca6fd in_seq=0 out_seq=0 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50aeff5700 1 -- 172.21.15.5:0/232790284 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x558af1f39050 con 0x558af209b7a0 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50aeff5700 10 client.26393 handle_command_reply: tid=0 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 client.26393 shutdown 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 msgr2=0x558af208e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f50c0096340 tx=0x7f50c00cf050 comp rx=0 tx=0).stop 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 shutdown_connections 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558af2036ad0 0x558af20924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558af209b7a0 0x558af209dbb0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558af202b2d0 0x558af208e380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 --2- 172.21.15.5:0/232790284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558af202bbe0 0x558af208e920 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50aeff5700 0 client.26393 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:17.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 >> 172.21.15.5:0/232790284 conn(0x558af2021da0 msgr2=0x558af202eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 shutdown_connections 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.945+0000 7f50e4b6eb80 1 -- 172.21.15.5:0/232790284 wait complete. 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50e4b6eb80 20 client.26393 trim_cache size 0 max 16384 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:12:17.960 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: "40bb24d1-67b6-48e7-9782-221c8b1b6ed3": { 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "40bb24d1-67b6-48e7-9782-221c8b1b6ed3", 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:12:17.961 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:12:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 msgr2=0x7f50b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 0x7f50b40a37f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f50c4007f80 tx=0x7f50c4006210 comp rx=0 tx=0).stop 2023-07-21T19:12:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 msgr2=0x7f50cc198360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:17.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f50c000e940 tx=0x7f50c000ec50 comp rx=0 tx=0).stop 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 shutdown_connections 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc106fb0 0x7f50cc19c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc101f70 0x7f50cc197e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50b40a1340 0x7f50b40a37f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 --2- 172.21.15.5:0/3512495585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc102910 0x7f50cc198360 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 >> 172.21.15.5:0/3512495585 conn(0x7f50cc0fd7e0 msgr2=0x7f50cc0fe230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 shutdown_connections 2023-07-21T19:12:17.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:17.946+0000 7f50d11ff700 1 -- 172.21.15.5:0/3512495585 wait complete. 2023-07-21T19:12:17.971 INFO:tasks.cephfs.filesystem:scrub status for tag:40bb24d1-67b6-48e7-9782-221c8b1b6ed3 - {'path': '/', 'tag': '40bb24d1-67b6-48e7-9782-221c8b1b6ed3', 'options': 'recursive,force'} 2023-07-21T19:12:18.223 INFO:teuthology.orchestra.run.smithi005.stderr:15900000 of 50000000 tuples (31%) done (elapsed 255.78 s, remaining 548.57 s) 2023-07-21T19:12:19.497 INFO:teuthology.orchestra.run.smithi005.stderr:16000000 of 50000000 tuples (32%) done (elapsed 257.06 s, remaining 546.25 s) 2023-07-21T19:12:19.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:19 smithi101 ceph-mon[128136]: pgmap v144: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 29 op/s 2023-07-21T19:12:19.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:19 smithi005 ceph-mon[126460]: pgmap v144: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 29 op/s 2023-07-21T19:12:19.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:19 smithi156 ceph-mon[128316]: pgmap v144: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 33 MiB/s wr, 29 op/s 2023-07-21T19:12:20.780 INFO:teuthology.orchestra.run.smithi005.stderr:16100000 of 50000000 tuples (32%) done (elapsed 258.34 s, remaining 543.96 s) 2023-07-21T19:12:21.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:21 smithi101 ceph-mon[128136]: pgmap v145: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:12:21.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:21 smithi005 ceph-mon[126460]: pgmap v145: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:12:21.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:21 smithi156 ceph-mon[128316]: pgmap v145: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 MiB/s wr, 15 op/s 2023-07-21T19:12:22.061 INFO:teuthology.orchestra.run.smithi005.stderr:16200000 of 50000000 tuples (32%) done (elapsed 259.62 s, remaining 541.68 s) 2023-07-21T19:12:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:16300000 of 50000000 tuples (32%) done (elapsed 261.01 s, remaining 539.63 s) 2023-07-21T19:12:23.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:23 smithi101 ceph-mon[128136]: pgmap v146: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:12:23.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:23 smithi005 ceph-mon[126460]: pgmap v146: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:12:23.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:23 smithi156 ceph-mon[128316]: pgmap v146: 97 pgs: 97 active+clean; 3.7 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:12:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:16400000 of 50000000 tuples (32%) done (elapsed 262.30 s, remaining 537.40 s) 2023-07-21T19:12:25.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:25 smithi101 ceph-mon[128136]: pgmap v147: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:12:25.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:25 smithi005 ceph-mon[126460]: pgmap v147: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:12:25.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:25 smithi156 ceph-mon[128316]: pgmap v147: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:12:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:16500000 of 50000000 tuples (33%) done (elapsed 263.61 s, remaining 535.20 s) 2023-07-21T19:12:27.341 INFO:teuthology.orchestra.run.smithi005.stderr:16600000 of 50000000 tuples (33%) done (elapsed 264.90 s, remaining 533.00 s) 2023-07-21T19:12:27.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:27 smithi101 ceph-mon[128136]: pgmap v148: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:27.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:27 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:27.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:27 smithi005 ceph-mon[126460]: pgmap v148: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:27.914 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:27 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:27.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:27 smithi156 ceph-mon[128316]: pgmap v148: 97 pgs: 97 active+clean; 3.8 GiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:27.926 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:27 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:28.729 INFO:teuthology.orchestra.run.smithi005.stderr:16700000 of 50000000 tuples (33%) done (elapsed 266.29 s, remaining 530.99 s) 2023-07-21T19:12:29.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:29 smithi101 ceph-mon[128136]: pgmap v149: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:29.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:29 smithi005 ceph-mon[126460]: pgmap v149: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:29.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:29 smithi156 ceph-mon[128316]: pgmap v149: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:29.988 INFO:teuthology.orchestra.run.smithi005.stderr:16800000 of 50000000 tuples (33%) done (elapsed 267.55 s, remaining 528.73 s) 2023-07-21T19:12:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 -- 172.21.15.5:0/2790380401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 msgr2=0x7f630410ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 --2- 172.21.15.5:0/2790380401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630410ca00 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f62fc00b0d0 tx=0x7f62fc00b3e0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 -- 172.21.15.5:0/2790380401 shutdown_connections 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 --2- 172.21.15.5:0/2790380401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630410ca00 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 --2- 172.21.15.5:0/2790380401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 --2- 172.21.15.5:0/2790380401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 -- 172.21.15.5:0/2790380401 >> 172.21.15.5:0/2790380401 conn(0x7f6304074760 msgr2=0x7f6304074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 -- 172.21.15.5:0/2790380401 shutdown_connections 2023-07-21T19:12:30.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.818+0000 7f630c5ff700 1 -- 172.21.15.5:0/2790380401 wait complete. 2023-07-21T19:12:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.819+0000 7f630c5ff700 1 Processor -- start 2023-07-21T19:12:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 -- start start 2023-07-21T19:12:30.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304197e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630419d040 con 0x7f6304104e50 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630419d1b0 con 0x7f63040685b0 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f630419d370 con 0x7f6304067c10 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f6309b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36136/0 (socket says 172.21.15.5:36136) 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f630ab9c700 1 -- 172.21.15.5:0/1358610844 learned_addr learned my addr 172.21.15.5:0/1358610844 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.820+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60964/0 (socket says 172.21.15.5:60964) 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f630a39b700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304197e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 -- 172.21.15.5:0/1358610844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 msgr2=0x7f6304197e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304197e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 -- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 msgr2=0x7f630419ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 -- 172.21.15.5:0/1358610844 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62f4009520 con 0x7f63040685b0 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f630a39b700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f630ab9c700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6304068980 tx=0x7f62f400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f62fb7fe700 1 -- 172.21.15.5:0/1358610844 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f400eee0 con 0x7f63040685b0 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f62fb7fe700 1 -- 172.21.15.5:0/1358610844 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62f4004020 con 0x7f63040685b0 2023-07-21T19:12:30.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62fc009d00 con 0x7f63040685b0 2023-07-21T19:12:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.821+0000 7f62fb7fe700 1 -- 172.21.15.5:0/1358610844 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f4003cc0 con 0x7f63040685b0 2023-07-21T19:12:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.822+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63041a4cb0 con 0x7f63040685b0 2023-07-21T19:12:30.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.825+0000 7f62fb7fe700 1 -- 172.21.15.5:0/1358610844 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f62f4004190 con 0x7f63040685b0 2023-07-21T19:12:30.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f62fb7fe700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 0x7f62f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f62fb7fe700 1 -- 172.21.15.5:0/1358610844 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f62f4014070 con 0x7f63040685b0 2023-07-21T19:12:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f631ffe2b80 1 Processor -- start 2023-07-21T19:12:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f630a39b700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 0x7f62f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f631ffe2b80 1 -- start start 2023-07-21T19:12:30.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e91d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.827+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e8c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96d91c50 con 0x55ec96e922d0 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96de49e0 con 0x55ec96e8c580 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96ddde70 con 0x55ec96ce86b0 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36144/0 (socket says 172.21.15.5:36144) 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 -- 172.21.15.5:0/736912698 learned_addr learned my addr 172.21.15.5:0/736912698 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f630a39b700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 0x7f62f00a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6300005950 tx=0x7f63000058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f630ab9c700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e8c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f630a39b700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e91d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 -- 172.21.15.5:0/736912698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 msgr2=0x55ec96e91d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e91d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 -- 172.21.15.5:0/736912698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 msgr2=0x55ec96e8c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e8c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f6309b9a700 1 -- 172.21.15.5:0/736912698 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec96c0bd80 con 0x55ec96e922d0 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.828+0000 7f630ab9c700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e8c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.829+0000 7f6309b9a700 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62f40a1fa0 tx=0x7f62f4004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a95d83bbcf94d65 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.829+0000 7f62f8bfe700 1 -- 172.21.15.5:0/736912698 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f409a080 con 0x55ec96e922d0 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.829+0000 7f62f8bfe700 1 -- 172.21.15.5:0/736912698 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62f409e0b0 con 0x55ec96e922d0 2023-07-21T19:12:30.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.829+0000 7f62f8bfe700 1 -- 172.21.15.5:0/736912698 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62f40997b0 con 0x55ec96e922d0 2023-07-21T19:12:30.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- 172.21.15.5:0/736912698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 msgr2=0x55ec96e8c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f62f40a1fa0 tx=0x7f62f4004510 comp rx=0 tx=0).stop 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- 172.21.15.5:0/736912698 shutdown_connections 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e8c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e91d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/736912698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e8c040 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- 172.21.15.5:0/736912698 >> 172.21.15.5:0/736912698 conn(0x55ec96e81be0 msgr2=0x55ec96e86f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- 172.21.15.5:0/736912698 shutdown_connections 2023-07-21T19:12:30.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- 172.21.15.5:0/736912698 wait complete. 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.830+0000 7f631ffe2b80 1 -- start start 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 Processor -- start 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96e8c580 0x55ec96e95ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e96400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96d7da60 con 0x55ec96ce86b0 2023-07-21T19:12:30.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96d7d760 con 0x55ec96e922d0 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96c618c0 con 0x55ec96e8c580 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f631ffe2b80 10 client.0 _ll_register_callbacks cb 0x55ec96d7b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f630a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f630a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36148/0 (socket says 172.21.15.5:36148) 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.831+0000 7f630a39b700 1 -- 172.21.15.5:0/4223890504 learned_addr learned my addr 172.21.15.5:0/4223890504 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630ab9c700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e96400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f6309b9a700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96e8c580 0x55ec96e95ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 -- 172.21.15.5:0/4223890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96e8c580 msgr2=0x55ec96e95ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96e8c580 0x55ec96e95ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 -- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 msgr2=0x55ec96e96400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e96400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 -- 172.21.15.5:0/4223890504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f630410d210 con 0x55ec96ce86b0 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f630a39b700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f63000040d0 tx=0x7f6300004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47f3ae4e7ea3de98 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f62edffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:12:30.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300017040 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.832+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ec96d37490 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6300003c10 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6300003dd0 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 10 client.16716 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x7f63041a4cb0 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ec96d37490 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f630001d050 con 0x55ec96ce86b0 2023-07-21T19:12:30.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 10 client.16716 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 10 client.16716 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 20 client.16716 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96ea5d20 0x55ec96ea8130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 4 client.16716 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.833+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ec96dd6e50 con 0x55ec96ea5d20 2023-07-21T19:12:30.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.834+0000 7f6309b9a700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96ea5d20 0x55ec96ea8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.834+0000 7f6309b9a700 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96ea5d20 0x55ec96ea8130 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f308ca64401fb9fb server_cookie=f90d91226f346d13 in_seq=0 out_seq=0 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.834+0000 7f62edffb700 10 client.16716 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f62edffb700 1 -- 172.21.15.5:0/4223890504 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ec96dd6e50 con 0x55ec96ea5d20 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f62edffb700 10 client.16716 handle_command_reply: tid=0 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 client.16716 shutdown 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 msgr2=0x55ec96e95980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f63000040d0 tx=0x7f6300004750 comp rx=0 tx=0).stop 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 shutdown_connections 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e96400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96ea5d20 0x55ec96ea8130 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96ce86b0 0x55ec96e95980 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f62edffb700 0 client.16716 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/4223890504 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96e8c580 0x55ec96e95ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.835+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 >> 172.21.15.5:0/4223890504 conn(0x55ec96e81be0 msgr2=0x55ec96e864e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.836+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 shutdown_connections 2023-07-21T19:12:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.836+0000 7f631ffe2b80 1 -- 172.21.15.5:0/4223890504 wait complete. 2023-07-21T19:12:30.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.836+0000 7f631ffe2b80 20 client.16716 trim_cache size 0 max 16384 2023-07-21T19:12:30.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.851+0000 7f631ffe2b80 1 Processor -- start 2023-07-21T19:12:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.851+0000 7f631ffe2b80 1 -- start start 2023-07-21T19:12:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e9cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e96230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96e8d050 con 0x55ec96e922d0 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96dd6e50 con 0x55ec96e8c580 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96e87d50 con 0x55ec96ce86b0 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f630a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f630ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e96230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f630a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46214/0 (socket says 172.21.15.5:46214) 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f630a39b700 1 -- 172.21.15.5:0/1889362010 learned_addr learned my addr 172.21.15.5:0/1889362010 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e9cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.852+0000 7f630a39b700 1 -- 172.21.15.5:0/1889362010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 msgr2=0x55ec96e9cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f630a39b700 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e9cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f630a39b700 1 -- 172.21.15.5:0/1889362010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 msgr2=0x55ec96e96230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f630a39b700 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e96230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f630a39b700 1 -- 172.21.15.5:0/1889362010 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ec96bf4c00 con 0x55ec96ce86b0 2023-07-21T19:12:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f630a39b700 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6300011520 tx=0x7f630000b9f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dec2c10a0d17a186 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f62f93ff700 1 -- 172.21.15.5:0/1889362010 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300011740 con 0x55ec96ce86b0 2023-07-21T19:12:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f62f93ff700 1 -- 172.21.15.5:0/1889362010 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63000118a0 con 0x55ec96ce86b0 2023-07-21T19:12:30.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.853+0000 7f62f93ff700 1 -- 172.21.15.5:0/1889362010 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63000207b0 con 0x55ec96ce86b0 2023-07-21T19:12:30.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1889362010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 msgr2=0x55ec96e8c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6300011520 tx=0x7f630000b9f0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1889362010 shutdown_connections 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e922d0 0x55ec96e96230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e8c040 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1889362010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e8c580 0x55ec96e9cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1889362010 >> 172.21.15.5:0/1889362010 conn(0x55ec96e81be0 msgr2=0x55ec96e913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1889362010 shutdown_connections 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.854+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1889362010 wait complete. 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.855+0000 7f631ffe2b80 1 -- start start 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.855+0000 7f631ffe2b80 1 Processor -- start 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 0x55ec96e9cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e9ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96e9b220 con 0x55ec96e8c580 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96e9b390 con 0x55ec96e922d0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ec96e9b550 con 0x55ec96ce86b0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f631ffe2b80 10 client.0 _ll_register_callbacks cb 0x7f630000fc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f6309b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 0x55ec96e9cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f630a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f630ab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e9ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f630a39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46218/0 (socket says 172.21.15.5:46218) 2023-07-21T19:12:30.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.856+0000 7f630a39b700 1 -- 172.21.15.5:0/1044184735 learned_addr learned my addr 172.21.15.5:0/1044184735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 -- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 msgr2=0x55ec96e9ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e9ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 -- 172.21.15.5:0/1044184735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 msgr2=0x55ec96e9cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 0x55ec96e9cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 -- 172.21.15.5:0/1044184735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63041a4cb0 con 0x55ec96ce86b0 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 0x55ec96e9cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:30.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f630a39b700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f630000b5b0 tx=0x7f630000b5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89b18f34299d59d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.857+0000 7f62edffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6300011c10 con 0x55ec96ce86b0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f6300011d70 con 0x55ec96ce86b0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f62f409e0b0 con 0x55ec96ce86b0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6300003ce0 con 0x55ec96ce86b0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f631ffe2b80 10 client.26377 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55ec96c321f0 con 0x55ec96ce86b0 2023-07-21T19:12:30.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.858+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f62f409e0b0 con 0x55ec96ce86b0 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6300004100 con 0x55ec96ce86b0 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 10 client.26377 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 10 client.26377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 20 client.26377 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96efa800 0x55ec96efcbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 4 client.26377 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:12:30.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ec96d98ff0 con 0x55ec96efa800 2023-07-21T19:12:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96efa800 0x55ec96efcbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f6309b9a700 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96efa800 0x55ec96efcbe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51b763a8770362a0 server_cookie=39ce797bddda5070 in_seq=0 out_seq=0 2023-07-21T19:12:30.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.859+0000 7f62edffb700 10 client.26377 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f62edffb700 1 -- 172.21.15.5:0/1044184735 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ec96d98ff0 con 0x55ec96efa800 2023-07-21T19:12:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f62edffb700 10 client.26377 handle_command_reply: tid=0 2023-07-21T19:12:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 client.26377 shutdown 2023-07-21T19:12:30.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 msgr2=0x55ec96e9c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f630000b5b0 tx=0x7f630000b5e0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 shutdown_connections 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ec96e922d0 0x55ec96e9ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ec96ce86b0 0x55ec96e9c6a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ec96efa800 0x55ec96efcbe0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 --2- 172.21.15.5:0/1044184735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ec96e8c580 0x55ec96e9cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 >> 172.21.15.5:0/1044184735 conn(0x55ec96e81be0 msgr2=0x55ec96e94130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f62edffb700 0 client.26377 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 shutdown_connections 2023-07-21T19:12:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.870+0000 7f631ffe2b80 1 -- 172.21.15.5:0/1044184735 wait complete. 2023-07-21T19:12:30.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f631ffe2b80 20 client.26377 trim_cache size 0 max 16384 2023-07-21T19:12:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:12:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:12:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:12:30.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:12:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 msgr2=0x7f62f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 0x7f62f00a3770 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f6300005950 tx=0x7f63000058e0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 msgr2=0x7f6304198390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6304068980 tx=0x7f62f400eca0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 shutdown_connections 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f62f00a12c0 0x7f62f00a3770 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6304104e50 0x7f630419ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6304067c10 0x7f6304197e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 --2- 172.21.15.5:0/1358610844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63040685b0 0x7f6304198390 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 >> 172.21.15.5:0/1358610844 conn(0x7f6304074760 msgr2=0x7f63040fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:30.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 shutdown_connections 2023-07-21T19:12:30.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:30.871+0000 7f630c5ff700 1 -- 172.21.15.5:0/1358610844 wait complete. 2023-07-21T19:12:31.270 INFO:teuthology.orchestra.run.smithi005.stderr:16900000 of 50000000 tuples (33%) done (elapsed 268.83 s, remaining 526.53 s) 2023-07-21T19:12:31.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:31 smithi101 ceph-mon[128136]: pgmap v150: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:31.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:31 smithi005 ceph-mon[126460]: pgmap v150: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:31.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:31 smithi156 ceph-mon[128316]: pgmap v150: 97 pgs: 97 active+clean; 3.8 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:32.645 INFO:teuthology.orchestra.run.smithi005.stderr:17000000 of 50000000 tuples (34%) done (elapsed 270.21 s, remaining 524.52 s) 2023-07-21T19:12:33.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:33 smithi101 ceph-mon[128136]: pgmap v151: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 26 op/s 2023-07-21T19:12:33.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:33 smithi005 ceph-mon[126460]: pgmap v151: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 26 op/s 2023-07-21T19:12:33.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:33 smithi156 ceph-mon[128316]: pgmap v151: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 MiB/s wr, 26 op/s 2023-07-21T19:12:33.940 INFO:teuthology.orchestra.run.smithi005.stderr:17100000 of 50000000 tuples (34%) done (elapsed 271.50 s, remaining 522.36 s) 2023-07-21T19:12:35.250 INFO:teuthology.orchestra.run.smithi005.stderr:17200000 of 50000000 tuples (34%) done (elapsed 272.81 s, remaining 520.24 s) 2023-07-21T19:12:35.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:35 smithi101 ceph-mon[128136]: pgmap v152: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:12:35.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:35 smithi005 ceph-mon[126460]: pgmap v152: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:12:35.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:35 smithi156 ceph-mon[128316]: pgmap v152: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:12:36.516 INFO:teuthology.orchestra.run.smithi005.stderr:17300000 of 50000000 tuples (34%) done (elapsed 274.08 s, remaining 518.05 s) 2023-07-21T19:12:37.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:37 smithi101 ceph-mon[128136]: pgmap v153: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:37.888 INFO:teuthology.orchestra.run.smithi005.stderr:17400000 of 50000000 tuples (34%) done (elapsed 275.45 s, remaining 516.07 s) 2023-07-21T19:12:37.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:37 smithi005 ceph-mon[126460]: pgmap v153: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:37.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:37 smithi156 ceph-mon[128316]: pgmap v153: 97 pgs: 97 active+clean; 3.9 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:12:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:17500000 of 50000000 tuples (35%) done (elapsed 276.72 s, remaining 513.91 s) 2023-07-21T19:12:39.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:39 smithi101 ceph-mon[128136]: pgmap v154: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:39.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:39 smithi005 ceph-mon[126460]: pgmap v154: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:39.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:39 smithi156 ceph-mon[128316]: pgmap v154: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 34 MiB/s wr, 26 op/s 2023-07-21T19:12:40.561 INFO:teuthology.orchestra.run.smithi005.stderr:17600000 of 50000000 tuples (35%) done (elapsed 278.12 s, remaining 512.00 s) 2023-07-21T19:12:41.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:41 smithi101 ceph-mon[128136]: pgmap v155: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:12:41.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:41 smithi005 ceph-mon[126460]: pgmap v155: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:12:41.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:41 smithi156 ceph-mon[128316]: pgmap v155: 97 pgs: 97 active+clean; 4.0 GiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:12:42.067 INFO:teuthology.orchestra.run.smithi005.stderr:17700000 of 50000000 tuples (35%) done (elapsed 279.63 s, remaining 510.28 s) 2023-07-21T19:12:43.357 INFO:teuthology.orchestra.run.smithi005.stderr:17800000 of 50000000 tuples (35%) done (elapsed 280.92 s, remaining 508.18 s) 2023-07-21T19:12:43.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:43 smithi101 ceph-mon[128136]: pgmap v156: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 32 op/s 2023-07-21T19:12:43.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:43 smithi005 ceph-mon[126460]: pgmap v156: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 32 op/s 2023-07-21T19:12:43.925 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:43 smithi156 ceph-mon[128316]: pgmap v156: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 33 MiB/s wr, 32 op/s 2023-07-21T19:12:44.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/2744050469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a14105b60 msgr2=0x7f2a14105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/2744050469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a14105b60 0x7f2a14105f30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2a10009990 tx=0x7f2a10009ca0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/2744050469 shutdown_connections 2023-07-21T19:12:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/2744050469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 0x7f2a1410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/2744050469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a14105b60 0x7f2a14105f30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/2744050469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a140ffb30 0x7f2a140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/2744050469 >> 172.21.15.5:0/2744050469 conn(0x7f2a140fb5f0 msgr2=0x7f2a140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/2744050469 shutdown_connections 2023-07-21T19:12:44.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.039+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/2744050469 wait complete. 2023-07-21T19:12:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.040+0000 7f2a1b9ff700 1 Processor -- start 2023-07-21T19:12:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 -- start start 2023-07-21T19:12:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 0x7f2a14198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14105b60 0x7f2a1419c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1419cf40 con 0x7f2a141004e0 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1419d0b0 con 0x7f2a140ffb30 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1419d270 con 0x7f2a14105b60 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52194/0 (socket says 172.21.15.5:52194) 2023-07-21T19:12:44.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a1979b700 1 -- 172.21.15.5:0/1497201421 learned_addr learned my addr 172.21.15.5:0/1497201421 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.041+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 0x7f2a14198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 -- 172.21.15.5:0/1497201421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14105b60 msgr2=0x7f2a1419c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a19f9c700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14105b60 0x7f2a1419c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14105b60 0x7f2a1419c900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 -- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 msgr2=0x7f2a14198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 0x7f2a14198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 -- 172.21.15.5:0/1497201421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0c009c40 con 0x7f2a140ffb30 2023-07-21T19:12:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098e0 tx=0x7f2a10009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a0a7fc700 1 -- 172.21.15.5:0/1497201421 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1001d070 con 0x7f2a140ffb30 2023-07-21T19:12:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a100095f0 con 0x7f2a140ffb30 2023-07-21T19:12:44.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a0a7fc700 1 -- 172.21.15.5:0/1497201421 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a10021410 con 0x7f2a140ffb30 2023-07-21T19:12:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.042+0000 7f2a0a7fc700 1 -- 172.21.15.5:0/1497201421 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a10021ea0 con 0x7f2a140ffb30 2023-07-21T19:12:44.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.043+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a141a4bb0 con 0x7f2a140ffb30 2023-07-21T19:12:44.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.047+0000 7f2a0a7fc700 1 -- 172.21.15.5:0/1497201421 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2a10021580 con 0x7f2a140ffb30 2023-07-21T19:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 Processor -- start 2023-07-21T19:12:44.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 -- start start 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x563871301d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638712fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x5638712fc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563871201c50 con 0x5638711586b0 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638712549e0 con 0x5638713022d0 2023-07-21T19:12:44.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.048+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56387124de70 con 0x5638712fc580 2023-07-21T19:12:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638712fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a19f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x563871301d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a19f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x563871301d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36594/0 (socket says 172.21.15.5:36594) 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a19f9c700 1 -- 172.21.15.5:0/3431686571 learned_addr learned my addr 172.21.15.5:0/3431686571 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a0a7fc700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 0x7f2a000a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x5638712fc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a0a7fc700 1 -- 172.21.15.5:0/1497201421 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2a100cded0 con 0x7f2a140ffb30 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 -- 172.21.15.5:0/3431686571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 msgr2=0x5638712fc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x5638712fc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 -- 172.21.15.5:0/3431686571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 msgr2=0x563871301d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x563871301d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a19f9c700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 0x7f2a000a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.049+0000 7f2a1979b700 1 -- 172.21.15.5:0/3431686571 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56387107bd80 con 0x5638713022d0 2023-07-21T19:12:44.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.050+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638712fc040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cb5f0 tx=0x7f2a10009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=961f26c555daab1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.050+0000 7f29ff7fe700 1 -- 172.21.15.5:0/3431686571 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100c9ea0 con 0x5638713022d0 2023-07-21T19:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.050+0000 7f29ff7fe700 1 -- 172.21.15.5:0/3431686571 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a1009ac60 con 0x5638713022d0 2023-07-21T19:12:44.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.050+0000 7f29ff7fe700 1 -- 172.21.15.5:0/3431686571 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100a3570 con 0x5638713022d0 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a19f9c700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 0x7f2a000a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0077e0 tx=0x7f2a0c004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3431686571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 msgr2=0x5638712fc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638712fc040 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2a100cb5f0 tx=0x7f2a10009310 comp rx=0 tx=0).stop 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3431686571 shutdown_connections 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x5638712fc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x563871301d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3431686571 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638712fc040 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3431686571 >> 172.21.15.5:0/3431686571 conn(0x5638712f1be0 msgr2=0x5638712f6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3431686571 shutdown_connections 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3431686571 wait complete. 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.051+0000 7f2a2f38bb80 1 -- start start 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 Processor -- start 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x56387130b820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638713055a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638711eda60 con 0x5638711586b0 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638711ed760 con 0x5638713022d0 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638710d18c0 con 0x5638712fc580 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a2f38bb80 10 client.0 _ll_register_callbacks cb 0x5638711eb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36600/0 (socket says 172.21.15.5:36600) 2023-07-21T19:12:44.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.052+0000 7f2a1979b700 1 -- 172.21.15.5:0/1329704351 learned_addr learned my addr 172.21.15.5:0/1329704351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a19f9c700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638713055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x56387130b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 -- 172.21.15.5:0/1329704351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 msgr2=0x56387130b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x56387130b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 -- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 msgr2=0x5638713055a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638713055a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 -- 172.21.15.5:0/1329704351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a141a4bb0 con 0x5638711586b0 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a19f9c700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638713055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f2a1979b700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a10021850 tx=0x7f2a10021b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94f2518d0dc91e15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f29fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.053+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1009b120 con 0x5638711586b0 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638711a7490 con 0x5638711586b0 2023-07-21T19:12:44.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2a100a34e0 con 0x5638711586b0 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a1009b590 con 0x5638711586b0 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 10 client.16734 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563871064c00 con 0x5638711586b0 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638711a7490 con 0x5638711586b0 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a1009a080 con 0x5638711586b0 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 10 client.16734 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:44.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 10 client.16734 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 20 client.16734 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563871315350 0x563871317730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 4 client.16734 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.054+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563871246e50 con 0x563871315350 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.055+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563871315350 0x563871317730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.055+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563871315350 0x563871317730 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e89ac4426f4f568 server_cookie=5dadc551803524bf in_seq=0 out_seq=0 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.055+0000 7f29fcff9700 10 client.16734 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:44.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f29fcff9700 1 -- 172.21.15.5:0/1329704351 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563871246e50 con 0x563871315350 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f29fcff9700 10 client.16734 handle_command_reply: tid=0 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 client.16734 shutdown 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 msgr2=0x56387130b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f2a10021850 tx=0x7f2a10021b50 comp rx=0 tx=0).stop 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 shutdown_connections 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638713022d0 0x5638713055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563871315350 0x563871317730 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.056+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638711586b0 0x56387130b2e0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/1329704351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x56387130b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 >> 172.21.15.5:0/1329704351 conn(0x5638712f1be0 msgr2=0x5638712f5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f29fcff9700 0 client.16734 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 shutdown_connections 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/1329704351 wait complete. 2023-07-21T19:12:44.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.057+0000 7f2a2f38bb80 20 client.16734 trim_cache size 0 max 16384 2023-07-21T19:12:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 Processor -- start 2023-07-21T19:12:44.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 -- start start 2023-07-21T19:12:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638711586b0 0x5638712fc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638712f6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638711a7490 con 0x5638713022d0 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563871306b10 con 0x5638712fc580 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563871300500 con 0x5638711586b0 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a19f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638712f6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.077+0000 7f2a18f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52236/0 (socket says 172.21.15.5:52236) 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a19f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638712f6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36610/0 (socket says 172.21.15.5:36610) 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 -- 172.21.15.5:0/2159164532 learned_addr learned my addr 172.21.15.5:0/2159164532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a1979b700 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638711586b0 0x5638712fc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 -- 172.21.15.5:0/2159164532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638711586b0 msgr2=0x5638712fc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638711586b0 0x5638712fc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 -- 172.21.15.5:0/2159164532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 msgr2=0x5638712f6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638712f6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 -- 172.21.15.5:0/2159164532 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638710a21f0 con 0x5638712fc580 2023-07-21T19:12:44.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.078+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x563871318010 tx=0x7f2a04009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53263a9e990d6934 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f29fffff700 1 -- 172.21.15.5:0/2159164532 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0401f310 con 0x5638712fc580 2023-07-21T19:12:44.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f29fffff700 1 -- 172.21.15.5:0/2159164532 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a04023070 con 0x5638712fc580 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f29fffff700 1 -- 172.21.15.5:0/2159164532 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a04005560 con 0x5638712fc580 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/2159164532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 msgr2=0x5638713056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x563871318010 tx=0x7f2a04009260 comp rx=0 tx=0).stop 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/2159164532 shutdown_connections 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638712f6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638711586b0 0x5638712fc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/2159164532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638712fc580 0x5638713056d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/2159164532 >> 172.21.15.5:0/2159164532 conn(0x5638712f1be0 msgr2=0x5638712fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/2159164532 shutdown_connections 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.079+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/2159164532 wait complete. 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.080+0000 7f2a2f38bb80 1 -- start start 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.080+0000 7f2a2f38bb80 1 Processor -- start 2023-07-21T19:12:44.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x563871309450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638713681b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563871309bf0 con 0x5638713022d0 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563871309d60 con 0x5638711586b0 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638713686f0 con 0x5638712fc580 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a2f38bb80 10 client.0 _ll_register_callbacks cb 0x7f2a0c009050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a19f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638713681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a1979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52252/0 (socket says 172.21.15.5:52252) 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.081+0000 7f2a1979b700 1 -- 172.21.15.5:0/3354991350 learned_addr learned my addr 172.21.15.5:0/3354991350 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x563871309450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 -- 172.21.15.5:0/3354991350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 msgr2=0x563871309450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x563871309450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 -- 172.21.15.5:0/3354991350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 msgr2=0x5638713681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638713681b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 -- 172.21.15.5:0/3354991350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04009d10 con 0x5638711586b0 2023-07-21T19:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a1979b700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a1009b780 tx=0x7f2a1009aef0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cec4c1a673fb9236 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f29fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a100a3050 con 0x5638711586b0 2023-07-21T19:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2a100ca020 con 0x5638711586b0 2023-07-21T19:12:44.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2a1009ac60 con 0x5638711586b0 2023-07-21T19:12:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.082+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2a100a34e0 con 0x5638711586b0 2023-07-21T19:12:44.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 10 client.26429 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56387107a250 con 0x5638711586b0 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2a1009ac60 con 0x5638711586b0 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2a1000fb00 con 0x5638711586b0 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 10 client.26429 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 10 client.26429 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 20 client.26429 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638713699a0 0x56387136bd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:44.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 4 client.26429 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:12:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563871208ff0 con 0x5638713699a0 2023-07-21T19:12:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.083+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638713699a0 0x56387136bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.084+0000 7f2a18f9a700 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638713699a0 0x56387136bd80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25b04f541d911e3 server_cookie=e599eb6821f94a0 in_seq=0 out_seq=0 2023-07-21T19:12:44.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.084+0000 7f29fcff9700 10 client.26429 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f29fcff9700 1 -- 172.21.15.5:0/3354991350 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563871208ff0 con 0x5638713699a0 2023-07-21T19:12:44.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f29fcff9700 10 client.26429 handle_command_reply: tid=0 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 client.26429 shutdown 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 msgr2=0x563871308f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a1009b780 tx=0x7f2a1009aef0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 shutdown_connections 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638713699a0 0x56387136bd80 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638713022d0 0x5638713681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638711586b0 0x563871308f10 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 --2- 172.21.15.5:0/3354991350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638712fc580 0x563871309450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 >> 172.21.15.5:0/3354991350 conn(0x5638712f1be0 msgr2=0x5638713028a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f29fcff9700 0 client.26429 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 shutdown_connections 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.097+0000 7f2a2f38bb80 1 -- 172.21.15.5:0/3354991350 wait complete. 2023-07-21T19:12:44.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a2f38bb80 20 client.26429 trim_cache size 0 max 16384 2023-07-21T19:12:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:12:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:12:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:12:44.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 msgr2=0x7f2a000a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 0x7f2a000a3770 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c0077e0 tx=0x7f2a0c004b40 comp rx=0 tx=0).stop 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 msgr2=0x7f2a14197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f2a100098e0 tx=0x7f2a10009580 comp rx=0 tx=0).stop 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 shutdown_connections 2023-07-21T19:12:44.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2a14105b60 0x7f2a1419c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2a140ffb30 0x7f2a14197d50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2a000a12c0 0x7f2a000a3770 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 --2- 172.21.15.5:0/1497201421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2a141004e0 0x7f2a14198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.098+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 >> 172.21.15.5:0/1497201421 conn(0x7f2a140fb5f0 msgr2=0x7f2a140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.099+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 shutdown_connections 2023-07-21T19:12:44.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:44.099+0000 7f2a1b9ff700 1 -- 172.21.15.5:0/1497201421 wait complete. 2023-07-21T19:12:44.655 INFO:teuthology.orchestra.run.smithi005.stderr:17900000 of 50000000 tuples (35%) done (elapsed 282.22 s, remaining 506.10 s) 2023-07-21T19:12:44.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:44 smithi005 ceph-mon[126460]: pgmap v157: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 26 MiB/s wr, 37 op/s 2023-07-21T19:12:44.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:44 smithi156 ceph-mon[128316]: pgmap v157: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 26 MiB/s wr, 37 op/s 2023-07-21T19:12:45.085 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:44 smithi101 ceph-mon[128136]: pgmap v157: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 26 MiB/s wr, 37 op/s 2023-07-21T19:12:46.036 INFO:teuthology.orchestra.run.smithi005.stderr:18000000 of 50000000 tuples (36%) done (elapsed 283.60 s, remaining 504.17 s) 2023-07-21T19:12:47.319 INFO:teuthology.orchestra.run.smithi005.stderr:18100000 of 50000000 tuples (36%) done (elapsed 284.88 s, remaining 502.08 s) 2023-07-21T19:12:47.585 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:47 smithi101 ceph-mon[128136]: pgmap v158: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:47.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:47 smithi005 ceph-mon[126460]: pgmap v158: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:47.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:47 smithi156 ceph-mon[128316]: pgmap v158: 97 pgs: 97 active+clean; 4.1 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 22 MiB/s wr, 34 op/s 2023-07-21T19:12:47.972 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:12:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.205+0000 7f855cfff700 1 -- 172.21.15.5:0/680250124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85580ffae0 msgr2=0x7f85580fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.205+0000 7f855cfff700 1 --2- 172.21.15.5:0/680250124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85580ffae0 0x7f85580fff50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f854c009990 tx=0x7f854c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 -- 172.21.15.5:0/680250124 shutdown_connections 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 --2- 172.21.15.5:0/680250124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8558100490 0x7f855810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 --2- 172.21.15.5:0/680250124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558105af0 0x7f8558105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 --2- 172.21.15.5:0/680250124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f85580ffae0 0x7f85580fff50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 -- 172.21.15.5:0/680250124 >> 172.21.15.5:0/680250124 conn(0x7f85580fb5e0 msgr2=0x7f85580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 -- 172.21.15.5:0/680250124 shutdown_connections 2023-07-21T19:12:48.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.206+0000 7f855cfff700 1 -- 172.21.15.5:0/680250124 wait complete. 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.207+0000 7f855cfff700 1 Processor -- start 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.207+0000 7f855cfff700 1 -- start start 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 0x7f8558072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 0x7f85581a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558073580 con 0x7f8558105af0 2023-07-21T19:12:48.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581a4240 con 0x7f85580ffae0 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855cfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85581a4380 con 0x7f8558100490 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f8555d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 0x7f8558072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f8556d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 0x7f85581a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55890/0 (socket says 172.21.15.5:55890) 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.208+0000 7f855659c700 1 -- 172.21.15.5:0/1303566944 learned_addr learned my addr 172.21.15.5:0/1303566944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:48.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 -- 172.21.15.5:0/1303566944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 msgr2=0x7f8558072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 0x7f8558072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 -- 172.21.15.5:0/1303566944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 msgr2=0x7f8558072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 -- 172.21.15.5:0/1303566944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8550009c40 con 0x7f8558105af0 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f855659c700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:48.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8555d9b700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 0x7f8558072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:12:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f8556d9d700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 0x7f85581a3d00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f855000efb0 tx=0x7f855000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f85477fe700 1 -- 172.21.15.5:0/1303566944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f855000ca50 con 0x7f8558105af0 2023-07-21T19:12:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f854c0095f0 con 0x7f8558105af0 2023-07-21T19:12:48.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f85477fe700 1 -- 172.21.15.5:0/1303566944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f855000f040 con 0x7f8558105af0 2023-07-21T19:12:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f85477fe700 1 -- 172.21.15.5:0/1303566944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85500045c0 con 0x7f8558105af0 2023-07-21T19:12:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.209+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85581a4930 con 0x7f8558105af0 2023-07-21T19:12:48.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.213+0000 7f85477fe700 1 -- 172.21.15.5:0/1303566944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f855000ccb0 con 0x7f8558105af0 2023-07-21T19:12:48.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.215+0000 7f8570a19b80 1 Processor -- start 2023-07-21T19:12:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.215+0000 7f8570a19b80 1 -- start start 2023-07-21T19:12:48.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.215+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab8f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.215+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.215+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f8570a19b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaa95cb0 con 0x5649aab8fbe0 2023-07-21T19:12:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f8570a19b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaae8a40 con 0x5649aab8f2d0 2023-07-21T19:12:48.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f8570a19b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaae1ed0 con 0x5649aab9aad0 2023-07-21T19:12:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f855659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f855659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33610/0 (socket says 172.21.15.5:33610) 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f855659c700 1 -- 172.21.15.5:0/1609554591 learned_addr learned my addr 172.21.15.5:0/1609554591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f8555d9b700 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f8555d9b700 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab8f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f855659c700 1 -- 172.21.15.5:0/1609554591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 msgr2=0x5649aab9ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.216+0000 7f855659c700 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab9ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f855659c700 1 -- 172.21.15.5:0/1609554591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 msgr2=0x5649aab8f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f855659c700 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab8f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f855659c700 1 -- 172.21.15.5:0/1609554591 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649aa90fde0 con 0x5649aab8fbe0 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f855659c700 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8548011650 tx=0x7f8548011960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ca634ac75391e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f8544ff9700 1 -- 172.21.15.5:0/1609554591 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548019630 con 0x5649aab8fbe0 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f8544ff9700 1 -- 172.21.15.5:0/1609554591 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8548019c20 con 0x5649aab8fbe0 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f8544ff9700 1 -- 172.21.15.5:0/1609554591 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800eb00 con 0x5649aab8fbe0 2023-07-21T19:12:48.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.217+0000 7f85477fe700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 0x7f85400a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.218+0000 7f85477fe700 1 -- 172.21.15.5:0/1303566944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f85500ccb50 con 0x7f8558105af0 2023-07-21T19:12:48.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.218+0000 7f8555d9b700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 0x7f85400a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.218+0000 7f8570a19b80 1 -- 172.21.15.5:0/1609554591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 msgr2=0x5649aab9a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.218+0000 7f8570a19b80 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f8548011650 tx=0x7f8548011960 comp rx=0 tx=0).stop 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 -- 172.21.15.5:0/1609554591 shutdown_connections 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab8f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 --2- 172.21.15.5:0/1609554591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab9a590 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 -- 172.21.15.5:0/1609554591 >> 172.21.15.5:0/1609554591 conn(0x5649aab85da0 msgr2=0x5649aab8b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 -- 172.21.15.5:0/1609554591 shutdown_connections 2023-07-21T19:12:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8555d9b700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 0x7f85400a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b2e0 tx=0x7f854c009ca0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 -- 172.21.15.5:0/1609554591 wait complete. 2023-07-21T19:12:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.219+0000 7f8570a19b80 1 -- start start 2023-07-21T19:12:48.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.220+0000 7f8570a19b80 1 Processor -- start 2023-07-21T19:12:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab9a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab977f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaa81ac0 con 0x5649aab8fbe0 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaa817c0 con 0x5649aab8f2d0 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aa965920 con 0x5649aab9aad0 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8570a19b80 10 client.0 _ll_register_callbacks cb 0x5649aaa7f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f855659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab9a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f8555d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f855659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab9a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55914/0 (socket says 172.21.15.5:55914) 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.221+0000 7f855659c700 1 -- 172.21.15.5:0/2960041413 learned_addr learned my addr 172.21.15.5:0/2960041413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8556d9d700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8555d9b700 1 -- 172.21.15.5:0/2960041413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 msgr2=0x5649aab97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8555d9b700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab97f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8555d9b700 1 -- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 msgr2=0x5649aab9a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8555d9b700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab9a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.222+0000 7f8555d9b700 1 -- 172.21.15.5:0/2960041413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8548009e10 con 0x5649aab8fbe0 2023-07-21T19:12:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f8555d9b700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab977f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f854c005910 tx=0x7f854c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d813a5bf03821457 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f853dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:12:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c02f070 con 0x5649aab8fbe0 2023-07-21T19:12:48.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649aaa3b4f0 con 0x5649aab8fbe0 2023-07-21T19:12:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f854c00fde0 con 0x5649aab8fbe0 2023-07-21T19:12:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f854c03d480 con 0x5649aab8fbe0 2023-07-21T19:12:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f8570a19b80 10 client.16752 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5649aa8f8c60 con 0x5649aab8fbe0 2023-07-21T19:12:48.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.223+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5649aaa3b4f0 con 0x5649aab8fbe0 2023-07-21T19:12:48.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f854c018400 con 0x5649aab8fbe0 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 10 client.16752 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 10 client.16752 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 20 client.16752 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aabb2c10 0x5649aabb5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 4 client.16752 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649aaadaeb0 con 0x5649aabb2c10 2023-07-21T19:12:48.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.224+0000 7f855659c700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aabb2c10 0x5649aabb5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.225+0000 7f855659c700 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aabb2c10 0x5649aabb5020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dae64e1320c748 server_cookie=c0434ba3832bdd77 in_seq=0 out_seq=0 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.225+0000 7f853dffb700 10 client.16752 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.226+0000 7f853dffb700 1 -- 172.21.15.5:0/2960041413 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649aaadaeb0 con 0x5649aabb2c10 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.226+0000 7f853dffb700 10 client.16752 handle_command_reply: tid=0 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.226+0000 7f8570a19b80 1 client.16752 shutdown 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.226+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 msgr2=0x5649aab977f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.226+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab977f0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f854c005910 tx=0x7f854c01a040 comp rx=0 tx=0).stop 2023-07-21T19:12:48.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 shutdown_connections 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab9aad0 0x5649aab97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8f2d0 0x5649aab9a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aabb2c10 0x5649aabb5020 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 --2- 172.21.15.5:0/2960041413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8fbe0 0x5649aab977f0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 >> 172.21.15.5:0/2960041413 conn(0x5649aab85da0 msgr2=0x5649aab8b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f853dffb700 0 client.16752 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:48.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 shutdown_connections 2023-07-21T19:12:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 1 -- 172.21.15.5:0/2960041413 wait complete. 2023-07-21T19:12:48.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.227+0000 7f8570a19b80 20 client.16752 trim_cache size 0 max 16384 2023-07-21T19:12:48.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 Processor -- start 2023-07-21T19:12:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 -- start start 2023-07-21T19:12:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8fbe0 0x5649aab96fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab9aad0 0x5649aab974f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aab90e30 con 0x5649aab8f2d0 2023-07-21T19:12:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aab987a0 con 0x5649aab9aad0 2023-07-21T19:12:48.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.246+0000 7f8570a19b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aab901c0 con 0x5649aab8fbe0 2023-07-21T19:12:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33636/0 (socket says 172.21.15.5:33636) 2023-07-21T19:12:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 -- 172.21.15.5:0/52512843 learned_addr learned my addr 172.21.15.5:0/52512843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f8556d9d700 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab9aad0 0x5649aab974f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f8555d9b700 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8fbe0 0x5649aab96fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 -- 172.21.15.5:0/52512843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8fbe0 msgr2=0x5649aab96fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8fbe0 0x5649aab96fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 -- 172.21.15.5:0/52512843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab9aad0 msgr2=0x5649aab974f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab9aad0 0x5649aab974f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 -- 172.21.15.5:0/52512843 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649aab875e0 con 0x5649aab8f2d0 2023-07-21T19:12:48.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.247+0000 7f855659c700 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f854800e420 tx=0x7f854800e450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ea221433217c90 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.248+0000 7f85457fa700 1 -- 172.21.15.5:0/52512843 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480198b0 con 0x5649aab8f2d0 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.248+0000 7f85457fa700 1 -- 172.21.15.5:0/52512843 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f854800e530 con 0x5649aab8f2d0 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.248+0000 7f85457fa700 1 -- 172.21.15.5:0/52512843 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854800e9b0 con 0x5649aab8f2d0 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.248+0000 7f8570a19b80 1 -- 172.21.15.5:0/52512843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 msgr2=0x5649aabb4c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.248+0000 7f8570a19b80 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f854800e420 tx=0x7f854800e450 comp rx=0 tx=0).stop 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 -- 172.21.15.5:0/52512843 shutdown_connections 2023-07-21T19:12:48.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab9aad0 0x5649aab974f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab8f2d0 0x5649aabb4c20 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 --2- 172.21.15.5:0/52512843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8fbe0 0x5649aab96fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 -- 172.21.15.5:0/52512843 >> 172.21.15.5:0/52512843 conn(0x5649aab85da0 msgr2=0x5649aab9a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 -- 172.21.15.5:0/52512843 shutdown_connections 2023-07-21T19:12:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 -- 172.21.15.5:0/52512843 wait complete. 2023-07-21T19:12:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 -- start start 2023-07-21T19:12:48.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.249+0000 7f8570a19b80 1 Processor -- start 2023-07-21T19:12:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8f2d0 0x5649aaba1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab9aad0 0x5649aab9fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaba0240 con 0x5649aab9aad0 2023-07-21T19:12:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaba0380 con 0x5649aab8fbe0 2023-07-21T19:12:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649aaba0540 con 0x5649aab8f2d0 2023-07-21T19:12:48.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8570a19b80 10 client.0 _ll_register_callbacks cb 0x7f854c00bd30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8555d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8555d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55922/0 (socket says 172.21.15.5:55922) 2023-07-21T19:12:48.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8555d9b700 1 -- 172.21.15.5:0/211181338 learned_addr learned my addr 172.21.15.5:0/211181338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f8556d9d700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab9aad0 0x5649aab9fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.250+0000 7f855659c700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8f2d0 0x5649aaba1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 -- 172.21.15.5:0/211181338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8f2d0 msgr2=0x5649aaba1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8f2d0 0x5649aaba1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 -- 172.21.15.5:0/211181338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab9aad0 msgr2=0x5649aab9fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab9aad0 0x5649aab9fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 -- 172.21.15.5:0/211181338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649aa90fde0 con 0x5649aab8fbe0 2023-07-21T19:12:48.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8555d9b700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f854c018810 tx=0x7f854c0052f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c6716c09671c5fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f853dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c02f040 con 0x5649aab8fbe0 2023-07-21T19:12:48.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8548019c20 con 0x5649aab8fbe0 2023-07-21T19:12:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.251+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f854c018a80 con 0x5649aab8fbe0 2023-07-21T19:12:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f854c005bc0 con 0x5649aab8fbe0 2023-07-21T19:12:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 10 client.26447 fetch_fsmap learned FSMap version 28 2023-07-21T19:12:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5649aa936250 con 0x5649aab8fbe0 2023-07-21T19:12:48.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8548019c20 con 0x5649aab8fbe0 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 28) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f854c03d480 con 0x5649aab8fbe0 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 10 client.26447 fetch_fsmap finished waiting for FSMap version 28 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 10 client.26447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 20 client.26447 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aac07bc0 0x5649aac09fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 4 client.26447 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.252+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5649aaa9d050 con 0x5649aac07bc0 2023-07-21T19:12:48.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f855659c700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aac07bc0 0x5649aac09fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f855659c700 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aac07bc0 0x5649aac09fd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e48550a246e09c9 server_cookie=869a34665165ad18 in_seq=0 out_seq=0 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f853dffb700 10 client.26447 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f853dffb700 1 -- 172.21.15.5:0/211181338 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5649aaa9d050 con 0x5649aac07bc0 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f853dffb700 10 client.26447 handle_command_reply: tid=0 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f8570a19b80 1 client.26447 shutdown 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 msgr2=0x5649aab9f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.253+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f854c018810 tx=0x7f854c0052f0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 shutdown_connections 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5649aac07bc0 0x5649aac09fd0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5649aab9aad0 0x5649aab9fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5649aab8f2d0 0x5649aaba1450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f853dffb700 0 client.26447 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 --2- 172.21.15.5:0/211181338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5649aab8fbe0 0x5649aab9f780 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 >> 172.21.15.5:0/211181338 conn(0x5649aab85da0 msgr2=0x5649aab913e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 shutdown_connections 2023-07-21T19:12:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 1 -- 172.21.15.5:0/211181338 wait complete. 2023-07-21T19:12:48.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.254+0000 7f8570a19b80 20 client.26447 trim_cache size 0 max 16384 2023-07-21T19:12:48.276 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:12:48.276 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:12:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:12:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: "40bb24d1-67b6-48e7-9782-221c8b1b6ed3": { 2023-07-21T19:12:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:12:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "40bb24d1-67b6-48e7-9782-221c8b1b6ed3", 2023-07-21T19:12:48.277 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:12:48.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:12:48.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:12:48.278 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 msgr2=0x7f85400a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 0x7f85400a3770 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f854c00b2e0 tx=0x7f854c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 msgr2=0x7f85581a3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 0x7f85581a3d00 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f855000efb0 tx=0x7f855000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 shutdown_connections 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f85400a12c0 0x7f85400a3770 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8558105af0 0x7f85581a3d00 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f85580ffae0 0x7f8558072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 --2- 172.21.15.5:0/1303566944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8558100490 0x7f8558072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 >> 172.21.15.5:0/1303566944 conn(0x7f85580fb5e0 msgr2=0x7f85581035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 shutdown_connections 2023-07-21T19:12:48.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:12:48.255+0000 7f855cfff700 1 -- 172.21.15.5:0/1303566944 wait complete. 2023-07-21T19:12:48.282 INFO:tasks.cephfs.filesystem:scrub status for tag:40bb24d1-67b6-48e7-9782-221c8b1b6ed3 - {'path': '/', 'tag': '40bb24d1-67b6-48e7-9782-221c8b1b6ed3', 'options': 'recursive,force'} 2023-07-21T19:12:48.635 INFO:teuthology.orchestra.run.smithi005.stderr:18200000 of 50000000 tuples (36%) done (elapsed 286.20 s, remaining 500.06 s) 2023-07-21T19:12:49.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:49 smithi005 ceph-mon[126460]: pgmap v159: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:12:49.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:49 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:49.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:49 smithi156 ceph-mon[128316]: pgmap v159: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:12:49.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:49 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:49.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:49 smithi101 ceph-mon[128136]: pgmap v159: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:12:49.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:49 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:49.950 INFO:teuthology.orchestra.run.smithi005.stderr:18300000 of 50000000 tuples (36%) done (elapsed 287.51 s, remaining 498.04 s) 2023-07-21T19:12:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:18400000 of 50000000 tuples (36%) done (elapsed 288.94 s, remaining 496.23 s) 2023-07-21T19:12:51.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:51 smithi005 ceph-mon[126460]: pgmap v160: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:12:51.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:51 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:12:51.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:51 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:51.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:51 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:12:51.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:51 smithi156 ceph-mon[128316]: pgmap v160: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:12:51.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:51 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:12:51.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:51 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:51.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:51 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:12:51.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:51 smithi101 ceph-mon[128136]: pgmap v160: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 134 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:12:51.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:51 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:12:51.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:51 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:51.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:51 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:12:52.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:52 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:12:52.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:52 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:12:52.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:52 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:12:52.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:52 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:12:52.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:52 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:12:52.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:52 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:12:52.683 INFO:teuthology.orchestra.run.smithi005.stderr:18500000 of 50000000 tuples (37%) done (elapsed 290.25 s, remaining 494.20 s) 2023-07-21T19:12:52.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:52 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:12:52.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:52 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:12:52.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:52 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:12:53.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:53 smithi005 ceph-mon[126460]: pgmap v161: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 31 MiB/s wr, 44 op/s 2023-07-21T19:12:53.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:53 smithi156 ceph-mon[128316]: pgmap v161: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 31 MiB/s wr, 44 op/s 2023-07-21T19:12:53.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:53 smithi101 ceph-mon[128136]: pgmap v161: 97 pgs: 97 active+clean; 4.2 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 31 MiB/s wr, 44 op/s 2023-07-21T19:12:53.980 INFO:teuthology.orchestra.run.smithi005.stderr:18600000 of 50000000 tuples (37%) done (elapsed 291.54 s, remaining 492.17 s) 2023-07-21T19:12:55.085 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.086 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.087 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:54 smithi101 ceph-mon[128136]: pgmap v162: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 31 op/s 2023-07-21T19:12:55.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:55.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:12:55.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:12:55.163 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:12:55.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.164 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:54 smithi005 ceph-mon[126460]: pgmap v162: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 31 op/s 2023-07-21T19:12:55.174 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.175 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.176 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:12:55.176 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:54 smithi156 ceph-mon[128316]: pgmap v162: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 91 KiB/s rd, 23 MiB/s wr, 31 op/s 2023-07-21T19:12:55.354 INFO:teuthology.orchestra.run.smithi005.stderr:18700000 of 50000000 tuples (37%) done (elapsed 292.92 s, remaining 490.28 s) 2023-07-21T19:12:56.655 INFO:teuthology.orchestra.run.smithi005.stderr:18800000 of 50000000 tuples (37%) done (elapsed 294.22 s, remaining 488.28 s) 2023-07-21T19:12:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 -- 172.21.15.5:0/4170172061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338106df0 msgr2=0x7f733810e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 --2- 172.21.15.5:0/4170172061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338106df0 0x7f733810e9c0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f733000b0d0 tx=0x7f733000b3e0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 -- 172.21.15.5:0/4170172061 shutdown_connections 2023-07-21T19:12:57.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 --2- 172.21.15.5:0/4170172061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338106df0 0x7f733810e9c0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 --2- 172.21.15.5:0/4170172061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338101db0 0x7f7338102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 --2- 172.21.15.5:0/4170172061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338102750 0x7f7338106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 -- 172.21.15.5:0/4170172061 >> 172.21.15.5:0/4170172061 conn(0x7f73380fd660 msgr2=0x7f73380ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 -- 172.21.15.5:0/4170172061 shutdown_connections 2023-07-21T19:12:57.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.272+0000 7f733e39d700 1 -- 172.21.15.5:0/4170172061 wait complete. 2023-07-21T19:12:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.273+0000 7f733e39d700 1 Processor -- start 2023-07-21T19:12:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 -- start start 2023-07-21T19:12:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338102750 0x7f733819df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338106df0 0x7f7338197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338198130 con 0x7f7338101db0 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73381982a0 con 0x7f7338106df0 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733e39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7338198460 con 0x7f7338102750 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59538/0 (socket says 172.21.15.5:59538) 2023-07-21T19:12:57.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.274+0000 7f733d39b700 1 -- 172.21.15.5:0/1585292771 learned_addr learned my addr 172.21.15.5:0/1585292771 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733cb9a700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338102750 0x7f733819df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733db9c700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338106df0 0x7f7338197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 -- 172.21.15.5:0/1585292771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338102750 msgr2=0x7f733819df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338102750 0x7f733819df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 -- 172.21.15.5:0/1585292771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338106df0 msgr2=0x7f7338197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338106df0 0x7f7338197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 -- 172.21.15.5:0/1585292771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7330009d00 con 0x7f7338101db0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733d39b700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9a0 tx=0x7f733400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f732e7fc700 1 -- 172.21.15.5:0/1585292771 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334009790 con 0x7f7338101db0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7338198740 con 0x7f7338101db0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f732e7fc700 1 -- 172.21.15.5:0/1585292771 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f733400cbd0 con 0x7f7338101db0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.275+0000 7f732e7fc700 1 -- 172.21.15.5:0/1585292771 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334018680 con 0x7f7338101db0 2023-07-21T19:12:57.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.276+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381a4a40 con 0x7f7338101db0 2023-07-21T19:12:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.279+0000 7f732e7fc700 1 -- 172.21.15.5:0/1585292771 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f733400cd40 con 0x7f7338101db0 2023-07-21T19:12:57.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.281+0000 7f7352fddb80 1 Processor -- start 2023-07-21T19:12:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.281+0000 7f7352fddb80 1 -- start start 2023-07-21T19:12:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.281+0000 7f732e7fc700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 0x7f73240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f732e7fc700 1 -- 172.21.15.5:0/1585292771 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f7334014070 con 0x7f7338101db0 2023-07-21T19:12:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476ccaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f733db9c700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 0x7f73240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476cc5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476bcac50 con 0x559476ccb470 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476c1d9e0 con 0x559476b216b0 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f7352fddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476c16e70 con 0x559476ccb8c0 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f733cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f733cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59552/0 (socket says 172.21.15.5:59552) 2023-07-21T19:12:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f733cb9a700 1 -- 172.21.15.5:0/3623810276 learned_addr learned my addr 172.21.15.5:0/3623810276 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:57.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.282+0000 7f733d39b700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476ccaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733db9c700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476cc5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 -- 172.21.15.5:0/3623810276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 msgr2=0x559476cc5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476cc5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 -- 172.21.15.5:0/3623810276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 msgr2=0x559476ccaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476ccaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 -- 172.21.15.5:0/3623810276 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559476a44d80 con 0x559476ccb470 2023-07-21T19:12:57.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733d39b700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476ccaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733db9c700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 0x7f73240a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73300060b0 tx=0x7f7330009560 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f733cb9a700 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f73280046c0 tx=0x7f732800de50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=428ad420c6af3dbf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f73237fe700 1 -- 172.21.15.5:0/3623810276 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7328004a30 con 0x559476ccb470 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f73237fe700 1 -- 172.21.15.5:0/3623810276 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f732800fce0 con 0x559476ccb470 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.283+0000 7f73237fe700 1 -- 172.21.15.5:0/3623810276 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73280054e0 con 0x559476ccb470 2023-07-21T19:12:57.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 -- 172.21.15.5:0/3623810276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 msgr2=0x559476cc51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f73280046c0 tx=0x7f732800de50 comp rx=0 tx=0).stop 2023-07-21T19:12:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 -- 172.21.15.5:0/3623810276 shutdown_connections 2023-07-21T19:12:57.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476cc5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476ccaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3623810276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476cc51a0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 -- 172.21.15.5:0/3623810276 >> 172.21.15.5:0/3623810276 conn(0x559476cbad40 msgr2=0x559476cc0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 -- 172.21.15.5:0/3623810276 shutdown_connections 2023-07-21T19:12:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.284+0000 7f7352fddb80 1 -- 172.21.15.5:0/3623810276 wait complete. 2023-07-21T19:12:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.285+0000 7f7352fddb80 1 -- start start 2023-07-21T19:12:57.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.285+0000 7f7352fddb80 1 Processor -- start 2023-07-21T19:12:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 0x559476cceb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476ccb470 0x559476ccf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476ccf580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476bb6a60 con 0x559476b216b0 2023-07-21T19:12:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476bb6760 con 0x559476ccb470 2023-07-21T19:12:57.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476a9a8c0 con 0x559476ccb8c0 2023-07-21T19:12:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f7352fddb80 10 client.0 _ll_register_callbacks cb 0x559476bb4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f733d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 0x559476cceb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f733d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 0x559476cceb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59560/0 (socket says 172.21.15.5:59560) 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f733d39b700 1 -- 172.21.15.5:0/2417657939 learned_addr learned my addr 172.21.15.5:0/2417657939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f733db9c700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476ccf580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.286+0000 7f733cb9a700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476ccb470 0x559476ccf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 -- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476ccb470 msgr2=0x559476ccf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476ccb470 0x559476ccf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 -- 172.21.15.5:0/2417657939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 msgr2=0x559476cceb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 0x559476cceb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 -- 172.21.15.5:0/2417657939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73381a4a40 con 0x559476ccb8c0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f733db9c700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476ccf580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f733819e750 tx=0x7f733000b3e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e595cd7f857962fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f7320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f733001d860 con 0x559476ccb8c0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.287+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559476b70490 con 0x559476ccb8c0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.288+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f733001d9c0 con 0x559476ccb8c0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.288+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73300078c0 con 0x559476ccb8c0 2023-07-21T19:12:57.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.288+0000 7f7352fddb80 10 client.26422 fetch_fsmap learned FSMap version 30 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.288+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f732800d7d0 con 0x559476ccb8c0 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.288+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559476b70490 con 0x559476ccb8c0 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f733001bac0 con 0x559476ccb8c0 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 10 client.26422 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 10 client.26422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 20 client.26422 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476cdee40 0x559476ce1250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 4 client.26422 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559476c0fe50 con 0x559476cdee40 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.289+0000 7f733d39b700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476cdee40 0x559476ce1250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.290+0000 7f733d39b700 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476cdee40 0x559476ce1250 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca4f6d791ffdeffc server_cookie=a10c9cf281f827fc in_seq=0 out_seq=0 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.290+0000 7f7320ff9700 10 client.26422 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.290+0000 7f7320ff9700 1 -- 172.21.15.5:0/2417657939 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559476c0fe50 con 0x559476cdee40 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.290+0000 7f7320ff9700 10 client.26422 handle_command_reply: tid=0 2023-07-21T19:12:57.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 client.26422 shutdown 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 msgr2=0x559476ccf580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476ccf580 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f733819e750 tx=0x7f733000b3e0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 shutdown_connections 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb8c0 0x559476ccf580 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476cdee40 0x559476ce1250 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476b216b0 0x559476cceb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 --2- 172.21.15.5:0/2417657939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476ccb470 0x559476ccf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7320ff9700 0 client.26422 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 >> 172.21.15.5:0/2417657939 conn(0x559476cbad40 msgr2=0x559476cbf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 shutdown_connections 2023-07-21T19:12:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 1 -- 172.21.15.5:0/2417657939 wait complete. 2023-07-21T19:12:57.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.291+0000 7f7352fddb80 20 client.26422 trim_cache size 0 max 16384 2023-07-21T19:12:57.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.316+0000 7f7352fddb80 1 Processor -- start 2023-07-21T19:12:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 -- start start 2023-07-21T19:12:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476b216b0 0x559476cd5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476ccf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476c0fe50 con 0x559476ccb470 2023-07-21T19:12:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476b70490 con 0x559476cbad40 2023-07-21T19:12:57.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.317+0000 7f7352fddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476cbc750 con 0x559476b216b0 2023-07-21T19:12:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476ccf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44436/0 (socket says 172.21.15.5:44436) 2023-07-21T19:12:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733d39b700 1 -- 172.21.15.5:0/3150706721 learned_addr learned my addr 172.21.15.5:0/3150706721 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:57.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733cb9a700 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476b216b0 0x559476cd5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733d39b700 1 -- 172.21.15.5:0/3150706721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476b216b0 msgr2=0x559476cd5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.318+0000 7f733d39b700 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476b216b0 0x559476cd5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f733d39b700 1 -- 172.21.15.5:0/3150706721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 msgr2=0x559476ccf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f733d39b700 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476ccf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f733d39b700 1 -- 172.21.15.5:0/3150706721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559476a2dc00 con 0x559476cbad40 2023-07-21T19:12:57.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f733d39b700 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f733400e970 tx=0x7f733409d040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8263393c7726edd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f7323fff700 1 -- 172.21.15.5:0/3150706721 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340984d0 con 0x559476cbad40 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f7323fff700 1 -- 172.21.15.5:0/3150706721 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73340b82f0 con 0x559476cbad40 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.319+0000 7f7323fff700 1 -- 172.21.15.5:0/3150706721 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7334099880 con 0x559476cbad40 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 -- 172.21.15.5:0/3150706721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 msgr2=0x559476cca400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f733400e970 tx=0x7f733409d040 comp rx=0 tx=0).stop 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 -- 172.21.15.5:0/3150706721 shutdown_connections 2023-07-21T19:12:57.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476ccb470 0x559476ccf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476cbad40 0x559476cca400 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 --2- 172.21.15.5:0/3150706721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476b216b0 0x559476cd5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.320+0000 7f7352fddb80 1 -- 172.21.15.5:0/3150706721 >> 172.21.15.5:0/3150706721 conn(0x559476cda910 msgr2=0x559476cc4190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.321+0000 7f7352fddb80 1 -- 172.21.15.5:0/3150706721 shutdown_connections 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.321+0000 7f7352fddb80 1 -- 172.21.15.5:0/3150706721 wait complete. 2023-07-21T19:12:57.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.321+0000 7f7352fddb80 1 -- start start 2023-07-21T19:12:57.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 Processor -- start 2023-07-21T19:12:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476cbad40 0x559476cc6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 0x559476cc6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476cc7510 con 0x559476cbad40 2023-07-21T19:12:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476cc7680 con 0x559476b216b0 2023-07-21T19:12:57.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.322+0000 7f7352fddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559476cc7840 con 0x559476ccb470 2023-07-21T19:12:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f7352fddb80 10 client.0 _ll_register_callbacks cb 0x7f733000baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:12:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733db9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 0x559476cc6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44452/0 (socket says 172.21.15.5:44452) 2023-07-21T19:12:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733d39b700 1 -- 172.21.15.5:0/39554696 learned_addr learned my addr 172.21.15.5:0/39554696 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:12:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733db9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 0x559476cc6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41752/0 (socket says 172.21.15.5:41752) 2023-07-21T19:12:57.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.323+0000 7f733cb9a700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476cbad40 0x559476cc6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 -- 172.21.15.5:0/39554696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 msgr2=0x559476cc6fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 0x559476cc6fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 -- 172.21.15.5:0/39554696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476cbad40 msgr2=0x559476cc6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:12:57.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476cbad40 0x559476cc6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 -- 172.21.15.5:0/39554696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7338198740 con 0x559476b216b0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f733d39b700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73340ca750 tx=0x7f7334005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=660d2fde3f502118 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f7320ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73340ca500 con 0x559476b216b0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f73340b6030 con 0x559476b216b0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.324+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f732800fce0 con 0x559476b216b0 2023-07-21T19:12:57.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f73340b8560 con 0x559476b216b0 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 10 client.26465 fetch_fsmap learned FSMap version 30 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559476a6b1f0 con 0x559476b216b0 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f732800fce0 con 0x559476b216b0 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7334095f20 con 0x559476b216b0 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 10 client.26465 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 10 client.26465 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 20 client.26465 populate_metadata read hostname 'smithi005' 2023-07-21T19:12:57.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476ccb8c0 0x559476d3e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:12:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 4 client.26465 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:12:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.325+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559476bd1ff0 con 0x559476ccb8c0 2023-07-21T19:12:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.326+0000 7f733cb9a700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476ccb8c0 0x559476d3e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:12:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.327+0000 7f7320ff9700 10 client.26465 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:57.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.327+0000 7f733cb9a700 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476ccb8c0 0x559476d3e740 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c83b0d2918d126ba server_cookie=db0cba27d4f3b8be in_seq=0 out_seq=0 2023-07-21T19:12:57.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7320ff9700 1 -- 172.21.15.5:0/39554696 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559476bd1ff0 con 0x559476ccb8c0 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7320ff9700 10 client.26465 handle_command_reply: tid=0 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 client.26465 shutdown 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 msgr2=0x559476cd5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f73340ca750 tx=0x7f7334005ce0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 shutdown_connections 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559476ccb470 0x559476cc6fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559476b216b0 0x559476cd5dd0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559476ccb8c0 0x559476d3e740 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 --2- 172.21.15.5:0/39554696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559476cbad40 0x559476cc6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 >> 172.21.15.5:0/39554696 conn(0x559476cda910 msgr2=0x559476cc4190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.341+0000 7f7320ff9700 0 client.26465 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:12:57.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 shutdown_connections 2023-07-21T19:12:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f7352fddb80 1 -- 172.21.15.5:0/39554696 wait complete. 2023-07-21T19:12:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f7352fddb80 20 client.26465 trim_cache size 0 max 16384 2023-07-21T19:12:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:12:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:12:57.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:12:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:12:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 msgr2=0x7f73240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 0x7f73240a3770 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f73300060b0 tx=0x7f7330009560 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 msgr2=0x7f733819da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.342+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f733400e9a0 tx=0x7f733400ed60 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 shutdown_connections 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7338106df0 0x7f7338197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73240a12c0 0x7f73240a3770 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7338101db0 0x7f733819da50 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 --2- 172.21.15.5:0/1585292771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7338102750 0x7f733819df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:12:57.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 >> 172.21.15.5:0/1585292771 conn(0x7f73380fd660 msgr2=0x7f73380fe0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:12:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 shutdown_connections 2023-07-21T19:12:57.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:12:57.343+0000 7f733e39d700 1 -- 172.21.15.5:0/1585292771 wait complete. 2023-07-21T19:12:57.585 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:57 smithi101 ceph-mon[128136]: pgmap v163: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:57.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:57 smithi005 ceph-mon[126460]: pgmap v163: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:57.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:57 smithi156 ceph-mon[128316]: pgmap v163: 97 pgs: 97 active+clean; 4.3 GiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:12:57.992 INFO:teuthology.orchestra.run.smithi005.stderr:18900000 of 50000000 tuples (37%) done (elapsed 295.55 s, remaining 486.33 s) 2023-07-21T19:12:59.282 INFO:teuthology.orchestra.run.smithi005.stderr:19000000 of 50000000 tuples (38%) done (elapsed 296.84 s, remaining 484.32 s) 2023-07-21T19:12:59.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:12:59 smithi005 ceph-mon[126460]: pgmap v164: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:12:59.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:12:59 smithi156 ceph-mon[128316]: pgmap v164: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:12:59.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:12:59 smithi101 ceph-mon[128136]: pgmap v164: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:13:00.696 INFO:teuthology.orchestra.run.smithi005.stderr:19100000 of 50000000 tuples (38%) done (elapsed 298.26 s, remaining 482.52 s) 2023-07-21T19:13:01.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:01 smithi005 ceph-mon[126460]: pgmap v165: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:13:01.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:01 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:13:01.663 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:01 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:13:01.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:01 smithi156 ceph-mon[128316]: pgmap v165: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:13:01.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:01 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:13:01.675 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:01 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:13:01.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:01 smithi101 ceph-mon[128136]: pgmap v165: 97 pgs: 97 active+clean; 4.3 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 23 MiB/s wr, 18 op/s 2023-07-21T19:13:01.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:01 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:13:01.836 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:01 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:13:01.994 INFO:teuthology.orchestra.run.smithi005.stderr:19200000 of 50000000 tuples (38%) done (elapsed 299.56 s, remaining 480.54 s) 2023-07-21T19:13:02.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:02 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:13:02.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:02 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:13:02.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:02 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:13:03.309 INFO:teuthology.orchestra.run.smithi005.stderr:19300000 of 50000000 tuples (38%) done (elapsed 300.87 s, remaining 478.59 s) 2023-07-21T19:13:03.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:03 smithi005 ceph-mon[126460]: pgmap v166: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 40 op/s 2023-07-21T19:13:03.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:03 smithi156 ceph-mon[128316]: pgmap v166: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 40 op/s 2023-07-21T19:13:03.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:03 smithi101 ceph-mon[128136]: pgmap v166: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 33 MiB/s wr, 40 op/s 2023-07-21T19:13:04.581 INFO:teuthology.orchestra.run.smithi005.stderr:19400000 of 50000000 tuples (38%) done (elapsed 302.14 s, remaining 476.58 s) 2023-07-21T19:13:05.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:05 smithi005 ceph-mon[126460]: pgmap v167: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 37 op/s 2023-07-21T19:13:05.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:05 smithi156 ceph-mon[128316]: pgmap v167: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 37 op/s 2023-07-21T19:13:05.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:05 smithi101 ceph-mon[128136]: pgmap v167: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 37 op/s 2023-07-21T19:13:05.987 INFO:teuthology.orchestra.run.smithi005.stderr:19500000 of 50000000 tuples (39%) done (elapsed 303.55 s, remaining 474.78 s) 2023-07-21T19:13:07.272 INFO:teuthology.orchestra.run.smithi005.stderr:19600000 of 50000000 tuples (39%) done (elapsed 304.83 s, remaining 472.80 s) 2023-07-21T19:13:07.662 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:07 smithi005 ceph-mon[126460]: pgmap v168: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:13:07.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:07 smithi156 ceph-mon[128316]: pgmap v168: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:13:07.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:07 smithi101 ceph-mon[128136]: pgmap v168: 97 pgs: 97 active+clean; 4.4 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 22 MiB/s wr, 35 op/s 2023-07-21T19:13:08.568 INFO:teuthology.orchestra.run.smithi005.stderr:19700000 of 50000000 tuples (39%) done (elapsed 306.13 s, remaining 470.85 s) 2023-07-21T19:13:09.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:09 smithi156 ceph-mon[128316]: pgmap v169: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 47 op/s 2023-07-21T19:13:09.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:09 smithi101 ceph-mon[128136]: pgmap v169: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 47 op/s 2023-07-21T19:13:09.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:09 smithi005 ceph-mon[126460]: pgmap v169: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 34 MiB/s wr, 47 op/s 2023-07-21T19:13:09.941 INFO:teuthology.orchestra.run.smithi005.stderr:19800000 of 50000000 tuples (39%) done (elapsed 307.50 s, remaining 469.02 s) 2023-07-21T19:13:10.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 -- 172.21.15.5:0/3240001282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 msgr2=0x7fab140fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 --2- 172.21.15.5:0/3240001282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140fff70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 -- 172.21.15.5:0/3240001282 shutdown_connections 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 --2- 172.21.15.5:0/3240001282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab141004b0 0x7fab1410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 --2- 172.21.15.5:0/3240001282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab14105b10 0x7fab14105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 --2- 172.21.15.5:0/3240001282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140fff70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 -- 172.21.15.5:0/3240001282 >> 172.21.15.5:0/3240001282 conn(0x7fab140fb600 msgr2=0x7fab140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 -- 172.21.15.5:0/3240001282 shutdown_connections 2023-07-21T19:13:10.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.511+0000 7fab197ff700 1 -- 172.21.15.5:0/3240001282 wait complete. 2023-07-21T19:13:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.512+0000 7fab197ff700 1 Processor -- start 2023-07-21T19:13:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.512+0000 7fab197ff700 1 -- start start 2023-07-21T19:13:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab141004b0 0x7fab14072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab14105b10 0x7fab141a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141a4290 con 0x7fab140ffb00 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141a43d0 con 0x7fab141004b0 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab197ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab141a4540 con 0x7fab14105b10 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43774/0 (socket says 172.21.15.5:43774) 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab12ffd700 1 -- 172.21.15.5:0/2333713572 learned_addr learned my addr 172.21.15.5:0/2333713572 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab137fe700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab14105b10 0x7fab141a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.513+0000 7fab127fc700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab141004b0 0x7fab14072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 -- 172.21.15.5:0/2333713572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab14105b10 msgr2=0x7fab141a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab14105b10 0x7fab141a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 -- 172.21.15.5:0/2333713572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab141004b0 msgr2=0x7fab14072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab141004b0 0x7fab14072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 -- 172.21.15.5:0/2333713572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab080095f0 con 0x7fab140ffb00 2023-07-21T19:13:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e9a0 tx=0x7fab0400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7faafbfff700 1 -- 172.21.15.5:0/2333713572 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04009790 con 0x7fab140ffb00 2023-07-21T19:13:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7faafbfff700 1 -- 172.21.15.5:0/2333713572 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0400cbd0 con 0x7fab140ffb00 2023-07-21T19:13:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7faafbfff700 1 -- 172.21.15.5:0/2333713572 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04018680 con 0x7fab140ffb00 2023-07-21T19:13:10.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab141a4820 con 0x7fab140ffb00 2023-07-21T19:13:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.514+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab141a4d30 con 0x7fab140ffb00 2023-07-21T19:13:10.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.518+0000 7faafbfff700 1 -- 172.21.15.5:0/2333713572 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fab0400cd40 con 0x7fab140ffb00 2023-07-21T19:13:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.519+0000 7faafbfff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 0x7faafc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.520+0000 7faafbfff700 1 -- 172.21.15.5:0/2333713572 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fab04014070 con 0x7fab140ffb00 2023-07-21T19:13:10.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.520+0000 7fab127fc700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 0x7faafc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.520+0000 7fab2d0adb80 1 Processor -- start 2023-07-21T19:13:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab127fc700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 0x7faafc0a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab08006010 tx=0x7fab0800b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 -- start start 2023-07-21T19:13:10.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f80d2d0 0x55dd1f807040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f8079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f70cc50 con 0x55dd1f6636b0 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f75f9e0 con 0x55dd1f807580 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.521+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f758e70 con 0x55dd1f80d2d0 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43788/0 (socket says 172.21.15.5:43788) 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 -- 172.21.15.5:0/1346838335 learned_addr learned my addr 172.21.15.5:0/1346838335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab137fe700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f8079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab127fc700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f80d2d0 0x55dd1f807040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 -- 172.21.15.5:0/1346838335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f80d2d0 msgr2=0x55dd1f807040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f80d2d0 0x55dd1f807040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 -- 172.21.15.5:0/1346838335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 msgr2=0x55dd1f8079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f8079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 -- 172.21.15.5:0/1346838335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1f586d80 con 0x55dd1f6636b0 2023-07-21T19:13:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab137fe700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f8079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:10.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.522+0000 7fab12ffd700 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fab04096ae0 tx=0x7fab0409a070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b622a49e604d79a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.523+0000 7faaf97fa700 1 -- 172.21.15.5:0/1346838335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040a24c0 con 0x55dd1f6636b0 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.523+0000 7faaf97fa700 1 -- 172.21.15.5:0/1346838335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0409a2d0 con 0x55dd1f6636b0 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.523+0000 7faaf97fa700 1 -- 172.21.15.5:0/1346838335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab040a2270 con 0x55dd1f6636b0 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.523+0000 7fab2d0adb80 1 -- 172.21.15.5:0/1346838335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 msgr2=0x55dd1f80cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.523+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fab04096ae0 tx=0x7fab0409a070 comp rx=0 tx=0).stop 2023-07-21T19:13:10.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 -- 172.21.15.5:0/1346838335 shutdown_connections 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f8079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f80cd90 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/1346838335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f80d2d0 0x55dd1f807040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 -- 172.21.15.5:0/1346838335 >> 172.21.15.5:0/1346838335 conn(0x55dd1f7fcbe0 msgr2=0x55dd1f801f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 -- 172.21.15.5:0/1346838335 shutdown_connections 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 -- 172.21.15.5:0/1346838335 wait complete. 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.524+0000 7fab2d0adb80 1 -- start start 2023-07-21T19:13:10.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 Processor -- start 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f813670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f6f8a60 con 0x55dd1f807580 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f6f8760 con 0x55dd1f80d2d0 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f5dc8c0 con 0x55dd1f6636b0 2023-07-21T19:13:10.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab2d0adb80 10 client.0 _ll_register_callbacks cb 0x55dd1f6f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43796/0 (socket says 172.21.15.5:43796) 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.525+0000 7fab127fc700 1 -- 172.21.15.5:0/2709800527 learned_addr learned my addr 172.21.15.5:0/2709800527 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab137fe700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f813670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 -- 172.21.15.5:0/2709800527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 msgr2=0x55dd1f813670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f813670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 -- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 msgr2=0x55dd1f8110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8110a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 -- 172.21.15.5:0/2709800527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab141a4d30 con 0x55dd1f807580 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7fab127fc700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fa5d50c41f5bfb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.526+0000 7faaf6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab08004ae0 con 0x55dd1f807580 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd1f6b2490 con 0x55dd1f807580 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fab08004c40 con 0x55dd1f807580 2023-07-21T19:13:10.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab08004e40 con 0x55dd1f807580 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7fab2d0adb80 10 client.16788 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd1f56fc00 con 0x55dd1f807580 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0803e020 con 0x55dd1f807580 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.527+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab0800f460 con 0x55dd1f807580 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 10 client.16788 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 10 client.16788 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:10.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 20 client.16788 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f820b70 0x55dd1f822f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 4 client.16788 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd1f751e50 con 0x55dd1f820b70 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.528+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f820b70 0x55dd1f822f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.529+0000 7faaf6ff5700 10 client.16788 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:10.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.529+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f820b70 0x55dd1f822f80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce49d01066d1fa5c server_cookie=d68eebbfa7f6d213 in_seq=0 out_seq=0 2023-07-21T19:13:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.529+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2709800527 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd1f751e50 con 0x55dd1f820b70 2023-07-21T19:13:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.529+0000 7faaf6ff5700 10 client.16788 handle_command_reply: tid=0 2023-07-21T19:13:10.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.529+0000 7fab2d0adb80 1 client.16788 shutdown 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 msgr2=0x55dd1f810960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fab08009990 tx=0x7fab08009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 shutdown_connections 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f813670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f820b70 0x55dd1f822f80 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2709800527 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f807580 0x55dd1f810960 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 >> 172.21.15.5:0/2709800527 conn(0x55dd1f7fcbe0 msgr2=0x55dd1f8014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7faaf6ff5700 0 client.16788 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 shutdown_connections 2023-07-21T19:13:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2709800527 wait complete. 2023-07-21T19:13:10.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.530+0000 7fab2d0adb80 20 client.16788 trim_cache size 0 max 16384 2023-07-21T19:13:10.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.546+0000 7fab2d0adb80 1 Processor -- start 2023-07-21T19:13:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.546+0000 7fab2d0adb80 1 -- start start 2023-07-21T19:13:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f816230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f80c230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f803a70 con 0x55dd1f80d2d0 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f7f9bc0 con 0x55dd1f807580 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f80b710 con 0x55dd1f6636b0 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43800/0 (socket says 172.21.15.5:43800) 2023-07-21T19:13:10.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 -- 172.21.15.5:0/2096904593 learned_addr learned my addr 172.21.15.5:0/2096904593 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab127fc700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f80c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f816230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 -- 172.21.15.5:0/2096904593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 msgr2=0x55dd1f816230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f816230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 -- 172.21.15.5:0/2096904593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 msgr2=0x55dd1f80c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f80c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.547+0000 7fab137fe700 1 -- 172.21.15.5:0/2096904593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd1f5ad1f0 con 0x55dd1f80d2d0 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7fab127fc700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f80c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f816230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7fab137fe700 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007630 tx=0x7fab0c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f5898dac5d5984b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7faaf9ffb700 1 -- 172.21.15.5:0/2096904593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c00e040 con 0x55dd1f80d2d0 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7faaf9ffb700 1 -- 172.21.15.5:0/2096904593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab0c00f040 con 0x55dd1f80d2d0 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.548+0000 7faaf9ffb700 1 -- 172.21.15.5:0/2096904593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab0c004280 con 0x55dd1f80d2d0 2023-07-21T19:13:10.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2096904593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 msgr2=0x55dd1f801c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fab0c007630 tx=0x7fab0c00b040 comp rx=0 tx=0).stop 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2096904593 shutdown_connections 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f80d2d0 0x55dd1f801c60 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f6636b0 0x55dd1f816230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2096904593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f807580 0x55dd1f80c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2096904593 >> 172.21.15.5:0/2096904593 conn(0x55dd1f7fcbe0 msgr2=0x55dd1f80bb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2096904593 shutdown_connections 2023-07-21T19:13:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.549+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2096904593 wait complete. 2023-07-21T19:13:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.550+0000 7fab2d0adb80 1 -- start start 2023-07-21T19:13:10.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.550+0000 7fab2d0adb80 1 Processor -- start 2023-07-21T19:13:10.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.550+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.550+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 0x55dd1f86e600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab2d0adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8689c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f868fa0 con 0x55dd1f6636b0 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f869110 con 0x55dd1f80d2d0 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab2d0adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd1f8692d0 con 0x55dd1f807580 2023-07-21T19:13:10.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab2d0adb80 10 client.0 _ll_register_callbacks cb 0x7fab0800ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43814/0 (socket says 172.21.15.5:43814) 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 -- 172.21.15.5:0/2351063751 learned_addr learned my addr 172.21.15.5:0/2351063751 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab127fc700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 0x55dd1f86e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab137fe700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8689c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 -- 172.21.15.5:0/2351063751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 msgr2=0x55dd1f86e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 0x55dd1f86e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 -- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 msgr2=0x55dd1f8689c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8689c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab12ffd700 1 -- 172.21.15.5:0/2351063751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab0c009360 con 0x55dd1f6636b0 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.551+0000 7fab127fc700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 0x55dd1f86e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7fab12ffd700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab040a1f90 tx=0x7fab04005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add36ec7f0d11d4e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7faaf6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab04099560 con 0x55dd1f6636b0 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fab040cae40 con 0x55dd1f6636b0 2023-07-21T19:13:10.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab0409a2d0 con 0x55dd1f6636b0 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab040996c0 con 0x55dd1f6636b0 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7fab2d0adb80 10 client.16794 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.552+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55dd1f585250 con 0x55dd1f6636b0 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab040ca020 con 0x55dd1f6636b0 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab04095ef0 con 0x55dd1f6636b0 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 10 client.16794 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:10.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 10 client.16794 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 20 client.16794 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f875c80 0x55dd1f878060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 4 client.16794 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dd1f713ff0 con 0x55dd1f875c80 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.553+0000 7fab127fc700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f875c80 0x55dd1f878060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.554+0000 7faaf6ff5700 10 client.16794 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:10.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.554+0000 7fab127fc700 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f875c80 0x55dd1f878060 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c44fabab03e37a49 server_cookie=b18048dd74612bca in_seq=0 out_seq=0 2023-07-21T19:13:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7faaf6ff5700 1 -- 172.21.15.5:0/2351063751 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dd1f713ff0 con 0x55dd1f875c80 2023-07-21T19:13:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7faaf6ff5700 10 client.16794 handle_command_reply: tid=0 2023-07-21T19:13:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 client.16794 shutdown 2023-07-21T19:13:10.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 msgr2=0x55dd1f86e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fab040a1f90 tx=0x7fab04005b40 comp rx=0 tx=0).stop 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 shutdown_connections 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd1f80d2d0 0x55dd1f8689c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dd1f875c80 0x55dd1f878060 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd1f6636b0 0x55dd1f86e0c0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 --2- 172.21.15.5:0/2351063751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd1f807580 0x55dd1f86e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7faaf6ff5700 0 client.16794 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:10.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 >> 172.21.15.5:0/2351063751 conn(0x55dd1f7fcbe0 msgr2=0x55dd1f80e320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 shutdown_connections 2023-07-21T19:13:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 1 -- 172.21.15.5:0/2351063751 wait complete. 2023-07-21T19:13:10.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.569+0000 7fab2d0adb80 20 client.16794 trim_cache size 0 max 16384 2023-07-21T19:13:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:13:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:13:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:13:10.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 msgr2=0x7faafc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 0x7faafc0a37f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab08006010 tx=0x7fab0800b260 comp rx=0 tx=0).stop 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 msgr2=0x7fab140729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fab0400e9a0 tx=0x7fab0400ed60 comp rx=0 tx=0).stop 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 shutdown_connections 2023-07-21T19:13:10.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab14105b10 0x7fab141a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faafc0a1340 0x7faafc0a37f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab140ffb00 0x7fab140729f0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 --2- 172.21.15.5:0/2333713572 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab141004b0 0x7fab14072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 >> 172.21.15.5:0/2333713572 conn(0x7fab140fb600 msgr2=0x7fab140fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 shutdown_connections 2023-07-21T19:13:10.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:10.570+0000 7fab197ff700 1 -- 172.21.15.5:0/2333713572 wait complete. 2023-07-21T19:13:11.244 INFO:teuthology.orchestra.run.smithi005.stderr:19900000 of 50000000 tuples (39%) done (elapsed 308.81 s, remaining 467.09 s) 2023-07-21T19:13:11.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:11 smithi156 ceph-mon[128316]: pgmap v170: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:13:11.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:11 smithi101 ceph-mon[128136]: pgmap v170: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:13:11.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:11 smithi005 ceph-mon[126460]: pgmap v170: 97 pgs: 97 active+clean; 4.5 GiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:13:12.545 INFO:teuthology.orchestra.run.smithi005.stderr:20000000 of 50000000 tuples (40%) done (elapsed 310.11 s, remaining 465.16 s) 2023-07-21T19:13:13.674 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:13 smithi156 ceph-mon[128316]: pgmap v171: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:13:13.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:13 smithi101 ceph-mon[128136]: pgmap v171: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:13:13.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:13 smithi005 ceph-mon[126460]: pgmap v171: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 33 MiB/s wr, 45 op/s 2023-07-21T19:13:14.279 INFO:teuthology.orchestra.run.smithi005.stderr:20100000 of 50000000 tuples (40%) done (elapsed 311.84 s, remaining 463.88 s) 2023-07-21T19:13:15.554 INFO:teuthology.orchestra.run.smithi005.stderr:20200000 of 50000000 tuples (40%) done (elapsed 313.12 s, remaining 461.92 s) 2023-07-21T19:13:15.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:15 smithi101 ceph-mon[128136]: pgmap v172: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:13:15.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:15 smithi005 ceph-mon[126460]: pgmap v172: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:13:15.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:15 smithi156 ceph-mon[128316]: pgmap v172: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 24 MiB/s wr, 26 op/s 2023-07-21T19:13:16.865 INFO:teuthology.orchestra.run.smithi005.stderr:20300000 of 50000000 tuples (40%) done (elapsed 314.43 s, remaining 460.03 s) 2023-07-21T19:13:17.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:17 smithi101 ceph-mon[128136]: pgmap v173: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:13:17.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:17 smithi005 ceph-mon[126460]: pgmap v173: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:13:17.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:17 smithi156 ceph-mon[128316]: pgmap v173: 97 pgs: 97 active+clean; 4.6 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 23 MiB/s wr, 22 op/s 2023-07-21T19:13:18.164 INFO:teuthology.orchestra.run.smithi005.stderr:20400000 of 50000000 tuples (40%) done (elapsed 315.73 s, remaining 458.11 s) 2023-07-21T19:13:18.283 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:13:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 -- 172.21.15.5:0/2132180739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8105ae0 msgr2=0x7f4dc8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/2132180739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8105ae0 0x7f4dc8105eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009990 tx=0x7f4db8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 -- 172.21.15.5:0/2132180739 shutdown_connections 2023-07-21T19:13:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/2132180739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8100480 0x7f4dc810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/2132180739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8105ae0 0x7f4dc8105eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/2132180739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 0x7f4dc80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 -- 172.21.15.5:0/2132180739 >> 172.21.15.5:0/2132180739 conn(0x7f4dc80fb5f0 msgr2=0x7f4dc80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 -- 172.21.15.5:0/2132180739 shutdown_connections 2023-07-21T19:13:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.521+0000 7f4dcdfff700 1 -- 172.21.15.5:0/2132180739 wait complete. 2023-07-21T19:13:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.522+0000 7f4dcdfff700 1 Processor -- start 2023-07-21T19:13:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 -- start start 2023-07-21T19:13:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 0x7f4dc8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8100480 0x7f4dc8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc8073580 con 0x7f4dc8105ae0 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81a4320 con 0x7f4dc80ffad0 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dcdfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc81a4460 con 0x7f4dc8100480 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.523+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54390/0 (socket says 172.21.15.5:54390) 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 -- 172.21.15.5:0/4112287184 learned_addr learned my addr 172.21.15.5:0/4112287184 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8100480 0x7f4dc8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 0x7f4dc8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 -- 172.21.15.5:0/4112287184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8100480 msgr2=0x7f4dc8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8100480 0x7f4dc8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 -- 172.21.15.5:0/4112287184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 msgr2=0x7f4dc8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 0x7f4dc8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 -- 172.21.15.5:0/4112287184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc0009c40 con 0x7f4dc8105ae0 2023-07-21T19:13:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.524+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000c370 tx=0x7f4dc000c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.525+0000 7f4dc4ff9700 1 -- 172.21.15.5:0/4112287184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0017070 con 0x7f4dc8105ae0 2023-07-21T19:13:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.525+0000 7f4dc4ff9700 1 -- 172.21.15.5:0/4112287184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc000f040 con 0x7f4dc8105ae0 2023-07-21T19:13:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.525+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4db80095f0 con 0x7f4dc8105ae0 2023-07-21T19:13:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.525+0000 7f4dc4ff9700 1 -- 172.21.15.5:0/4112287184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000e050 con 0x7f4dc8105ae0 2023-07-21T19:13:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.525+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc81a4a00 con 0x7f4dc8105ae0 2023-07-21T19:13:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.529+0000 7f4dc4ff9700 1 -- 172.21.15.5:0/4112287184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4dc000cb60 con 0x7f4dc8105ae0 2023-07-21T19:13:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 Processor -- start 2023-07-21T19:13:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 -- start start 2023-07-21T19:13:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef550652d0 0x55ef550656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 0x55ef55070590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef55072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54f6bcb0 con 0x55ef55070ad0 2023-07-21T19:13:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54fbea40 con 0x55ef550652d0 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.530+0000 7f4de195eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54fb7ed0 con 0x55ef55065be0 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc4ff9700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 0x7f4db00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef55072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc4ff9700 1 -- 172.21.15.5:0/4112287184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4dc00cc5d0 con 0x7f4dc8105ae0 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef55072eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54396/0 (socket says 172.21.15.5:54396) 2023-07-21T19:13:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/1491561159 learned_addr learned my addr 172.21.15.5:0/1491561159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 0x55ef55070590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef550652d0 0x55ef550656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 0x7f4db00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 -- 172.21.15.5:0/1491561159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef550652d0 msgr2=0x55ef550656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef550652d0 0x55ef550656a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 -- 172.21.15.5:0/1491561159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 msgr2=0x55ef55072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef55072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.531+0000 7f4dc77fe700 1 -- 172.21.15.5:0/1491561159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef54de5de0 con 0x55ef55065be0 2023-07-21T19:13:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.532+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 0x55ef55070590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4db80098e0 tx=0x7f4db8005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98fcf114287d95e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.532+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 0x7f4db00a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00a15f0 tx=0x7f4dc000e1b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.532+0000 7f4dadffb700 1 -- 172.21.15.5:0/1491561159 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db801d070 con 0x55ef55065be0 2023-07-21T19:13:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.532+0000 7f4dadffb700 1 -- 172.21.15.5:0/1491561159 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4db801e040 con 0x55ef55065be0 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.532+0000 7f4dadffb700 1 -- 172.21.15.5:0/1491561159 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004760 con 0x55ef55065be0 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 -- 172.21.15.5:0/1491561159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 msgr2=0x55ef55070590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 0x55ef55070590 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4db80098e0 tx=0x7f4db8005f00 comp rx=0 tx=0).stop 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 -- 172.21.15.5:0/1491561159 shutdown_connections 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef55072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef550652d0 0x55ef550656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 --2- 172.21.15.5:0/1491561159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55065be0 0x55ef55070590 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 -- 172.21.15.5:0/1491561159 >> 172.21.15.5:0/1491561159 conn(0x55ef5505bda0 msgr2=0x55ef550616a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 -- 172.21.15.5:0/1491561159 shutdown_connections 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.533+0000 7f4de195eb80 1 -- 172.21.15.5:0/1491561159 wait complete. 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.534+0000 7f4de195eb80 1 -- start start 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.534+0000 7f4de195eb80 1 Processor -- start 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef5506de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54f57ac0 con 0x55ef55070ad0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54f577c0 con 0x55ef55065be0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef54e3b920 con 0x55ef550652d0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4de195eb80 10 client.0 _ll_register_callbacks cb 0x55ef54f55760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef5506de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54838/0 (socket says 172.21.15.5:54838) 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.535+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef5506de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54402/0 (socket says 172.21.15.5:54402) 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 -- 172.21.15.5:0/3834788075 learned_addr learned my addr 172.21.15.5:0/3834788075 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 -- 172.21.15.5:0/3834788075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 msgr2=0x55ef55070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55070520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 -- 172.21.15.5:0/3834788075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 msgr2=0x55ef5506de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef5506de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 -- 172.21.15.5:0/3834788075 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc810d210 con 0x55ef55065be0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b920 tx=0x7f4db800b950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52fa50d413b934fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.536+0000 7f4da6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db801d030 con 0x55ef55065be0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef54f114f0 con 0x55ef55065be0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4db8004500 con 0x55ef55065be0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4db80046c0 con 0x55ef55065be0 2023-07-21T19:13:18.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4de195eb80 10 client.26495 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ef54dcec60 con 0x55ef55065be0 2023-07-21T19:13:18.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ef54f114f0 con 0x55ef55065be0 2023-07-21T19:13:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.537+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4db800f7f0 con 0x55ef55065be0 2023-07-21T19:13:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 10 client.26495 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:18.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 10 client.26495 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 20 client.26495 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef5507fec0 0x55ef550822d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 4 client.26495 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef54fb0eb0 con 0x55ef5507fec0 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.538+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef5507fec0 0x55ef550822d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.539+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef5507fec0 0x55ef550822d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed77396626d98f33 server_cookie=1606b4e84c839cf4 in_seq=0 out_seq=0 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.539+0000 7f4da6ffd700 10 client.26495 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:18.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.539+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3834788075 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef54fb0eb0 con 0x55ef5507fec0 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4da6ffd700 10 client.26495 handle_command_reply: tid=0 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 client.26495 shutdown 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 msgr2=0x55ef5506d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f4db800b920 tx=0x7f4db800b950 comp rx=0 tx=0).stop 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 shutdown_connections 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef5507fec0 0x55ef550822d0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55070ad0 0x55ef5506de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3834788075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef5506d730 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4da6ffd700 0 client.26495 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 >> 172.21.15.5:0/3834788075 conn(0x55ef5505bda0 msgr2=0x55ef55061230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 shutdown_connections 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 1 -- 172.21.15.5:0/3834788075 wait complete. 2023-07-21T19:13:18.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.540+0000 7f4de195eb80 20 client.26495 trim_cache size 0 max 16384 2023-07-21T19:13:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 Processor -- start 2023-07-21T19:13:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 -- start start 2023-07-21T19:13:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55086c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55070ad0 0x55ef55070050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef55066540 con 0x55ef55065be0 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef55065a20 con 0x55ef55070ad0 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.558+0000 7f4de195eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef5506e450 con 0x55ef550652d0 2023-07-21T19:13:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54408/0 (socket says 172.21.15.5:54408) 2023-07-21T19:13:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 -- 172.21.15.5:0/963867112 learned_addr learned my addr 172.21.15.5:0/963867112 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55086c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55070ad0 0x55ef55070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 -- 172.21.15.5:0/963867112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 msgr2=0x55ef55086c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55086c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 -- 172.21.15.5:0/963867112 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55070ad0 msgr2=0x55ef55070050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55070ad0 0x55ef55070050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 -- 172.21.15.5:0/963867112 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef54e0c250 con 0x55ef55065be0 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55086c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009ce0 tx=0x7f4db800fdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b94966c01ee23c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.559+0000 7f4dae7fc700 1 -- 172.21.15.5:0/963867112 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db801d070 con 0x55ef55065be0 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4dae7fc700 1 -- 172.21.15.5:0/963867112 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4db800b9c0 con 0x55ef55065be0 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4dae7fc700 1 -- 172.21.15.5:0/963867112 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db802cc10 con 0x55ef55065be0 2023-07-21T19:13:18.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 -- 172.21.15.5:0/963867112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 msgr2=0x55ef5507c6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009ce0 tx=0x7f4db800fdb0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 -- 172.21.15.5:0/963867112 shutdown_connections 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55070ad0 0x55ef55070050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef550652d0 0x55ef55086c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 --2- 172.21.15.5:0/963867112 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef55065be0 0x55ef5507c6e0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 -- 172.21.15.5:0/963867112 >> 172.21.15.5:0/963867112 conn(0x55ef5505bda0 msgr2=0x55ef5505fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 -- 172.21.15.5:0/963867112 shutdown_connections 2023-07-21T19:13:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.560+0000 7f4de195eb80 1 -- 172.21.15.5:0/963867112 wait complete. 2023-07-21T19:13:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.561+0000 7f4de195eb80 1 -- start start 2023-07-21T19:13:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.561+0000 7f4de195eb80 1 Processor -- start 2023-07-21T19:13:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef55075630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55070ad0 0x55ef55075b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef550760b0 con 0x55ef550652d0 2023-07-21T19:13:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef55076220 con 0x55ef55065be0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef550763e0 con 0x55ef55070ad0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4de195eb80 10 client.0 _ll_register_callbacks cb 0x7f4dbc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4dc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54414/0 (socket says 172.21.15.5:54414) 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3064784351 learned_addr learned my addr 172.21.15.5:0/3064784351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.562+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef55075630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55070ad0 0x55ef55075b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3064784351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55070ad0 msgr2=0x55ef55075b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55070ad0 0x55ef55075b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 msgr2=0x55ef55075630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef55075630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 -- 172.21.15.5:0/3064784351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef54dcec60 con 0x55ef550652d0 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef55075630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4dc6ffd700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005920 tx=0x7f4db8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a8e32fff1362814 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4da6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc016040 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4dbc025430 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.563+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4db801e040 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dbc0045b0 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 10 client.16812 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ef54de42b0 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4db801e040 con 0x55ef550652d0 2023-07-21T19:13:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4dbc003680 con 0x55ef550652d0 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 10 client.16812 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 10 client.16812 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 20 client.16812 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef550d5200 0x55ef550d7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 4 client.16812 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.564+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ef54f73050 con 0x55ef550d5200 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.565+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef550d5200 0x55ef550d7610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.565+0000 7f4dc77fe700 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef550d5200 0x55ef550d7610 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6d09a5017475a6e server_cookie=706f888193197582 in_seq=0 out_seq=0 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.565+0000 7f4da6ffd700 10 client.16812 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4da6ffd700 1 -- 172.21.15.5:0/3064784351 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ef54f73050 con 0x55ef550d5200 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4da6ffd700 10 client.16812 handle_command_reply: tid=0 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 client.16812 shutdown 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 msgr2=0x55ef55077300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc005920 tx=0x7f4db8009310 comp rx=0 tx=0).stop 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 shutdown_connections 2023-07-21T19:13:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ef55070ad0 0x55ef55075b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ef550d5200 0x55ef550d7610 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ef550652d0 0x55ef55077300 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 --2- 172.21.15.5:0/3064784351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ef55065be0 0x55ef55075630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 >> 172.21.15.5:0/3064784351 conn(0x55ef5505bda0 msgr2=0x55ef55068a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4da6ffd700 0 client.16812 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 shutdown_connections 2023-07-21T19:13:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 1 -- 172.21.15.5:0/3064784351 wait complete. 2023-07-21T19:13:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.566+0000 7f4de195eb80 20 client.16812 trim_cache size 0 max 16384 2023-07-21T19:13:18.576 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:13:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:13:18.577 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:13:18.577 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:13:18.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 msgr2=0x7f4db00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 0x7f4db00a37f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00a15f0 tx=0x7f4dc000e1b0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 msgr2=0x7f4dc81a3de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000c370 tx=0x7f4dc000c680 comp rx=0 tx=0).stop 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 shutdown_connections 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a1340 0x7f4db00a37f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dc8105ae0 0x7f4dc81a3de0 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dc80ffad0 0x7f4dc8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 --2- 172.21.15.5:0/4112287184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dc8100480 0x7f4dc8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.567+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 >> 172.21.15.5:0/4112287184 conn(0x7f4dc80fb5f0 msgr2=0x7f4dc80fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.568+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 shutdown_connections 2023-07-21T19:13:18.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.568+0000 7f4dcdfff700 1 -- 172.21.15.5:0/4112287184 wait complete. 2023-07-21T19:13:18.597 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:13:18.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:13:18.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 -- 172.21.15.5:0/1330565076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4010a0a0 msgr2=0x7f4f4010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 --2- 172.21.15.5:0/1330565076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4010a0a0 0x7f4f4010c4c0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f4f3800b0a0 tx=0x7f4f3800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 -- 172.21.15.5:0/1330565076 shutdown_connections 2023-07-21T19:13:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 --2- 172.21.15.5:0/1330565076 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f4010a0a0 0x7f4f4010c4c0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 --2- 172.21.15.5:0/1330565076 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40100b80 0x7f4f40100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 --2- 172.21.15.5:0/1330565076 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f40101490 0x7f4f40109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 -- 172.21.15.5:0/1330565076 >> 172.21.15.5:0/1330565076 conn(0x7f4f400fa490 msgr2=0x7f4f400fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:18.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.832+0000 7f4f47bff700 1 -- 172.21.15.5:0/1330565076 shutdown_connections 2023-07-21T19:13:18.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.833+0000 7f4f47bff700 1 -- 172.21.15.5:0/1330565076 wait complete. 2023-07-21T19:13:18.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 Processor -- start 2023-07-21T19:13:18.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 -- start start 2023-07-21T19:13:18.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40101490 0x7f4f401982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f4010a0a0 0x7f4f4019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4019cd90 con 0x7f4f40100b80 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4019cf00 con 0x7f4f4010a0a0 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f47bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f4019d0c0 con 0x7f4f40101490 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.834+0000 7f4f4599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54438/0 (socket says 172.21.15.5:54438) 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40101490 0x7f4f401982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 -- 172.21.15.5:0/422696837 learned_addr learned my addr 172.21.15.5:0/422696837 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4619c700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f4010a0a0 0x7f4f4019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 -- 172.21.15.5:0/422696837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40101490 msgr2=0x7f4f401982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40101490 0x7f4f401982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 -- 172.21.15.5:0/422696837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f4010a0a0 msgr2=0x7f4f4019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f4010a0a0 0x7f4f4019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:18.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 -- 172.21.15.5:0/422696837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f38009d00 con 0x7f4f40100b80 2023-07-21T19:13:18.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f4599b700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00e9a0 tx=0x7f4f3c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.835+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c009790 con 0x7f4f40100b80 2023-07-21T19:13:18.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.836+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f3c010430 con 0x7f4f40100b80 2023-07-21T19:13:18.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.836+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3c00cea0 con 0x7f4f40100b80 2023-07-21T19:13:18.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.836+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f4019d3a0 con 0x7f4f40100b80 2023-07-21T19:13:18.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.836+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f401a4be0 con 0x7f4f40100b80 2023-07-21T19:13:18.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.837+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f40067300 con 0x7f4f40100b80 2023-07-21T19:13:18.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.839+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4f3c0105a0 con 0x7f4f40100b80 2023-07-21T19:13:18.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.841+0000 7f4f36ffd700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 0x7f4f2c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:18.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.841+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4f3c014070 con 0x7f4f40100b80 2023-07-21T19:13:18.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.841+0000 7f4f4519a700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 0x7f4f2c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:18.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.842+0000 7f4f4519a700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 0x7f4f2c0a3830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f401993c0 tx=0x7f4f30009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:18.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:18.846+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4f3c051790 con 0x7f4f40100b80 2023-07-21T19:13:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.162+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4f40199100 con 0x7f4f40100b80 2023-07-21T19:13:19.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.163+0000 7f4f36ffd700 1 -- 172.21.15.5:0/422696837 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f4f3c0962c0 con 0x7f4f40100b80 2023-07-21T19:13:19.166 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:13:19.166 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:12:53.698175+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:13:19.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 msgr2=0x7f4f2c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 0x7f4f2c0a3830 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f4f401993c0 tx=0x7f4f30009260 comp rx=0 tx=0).stop 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 msgr2=0x7f4f40197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c00e9a0 tx=0x7f4f3c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 shutdown_connections 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f4010a0a0 0x7f4f4019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f2c0a1380 0x7f4f2c0a3830 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f40100b80 0x7f4f40197db0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 --2- 172.21.15.5:0/422696837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f40101490 0x7f4f401982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.165+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 >> 172.21.15.5:0/422696837 conn(0x7f4f400fa490 msgr2=0x7f4f4010c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.166+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 shutdown_connections 2023-07-21T19:13:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.166+0000 7f4f47bff700 1 -- 172.21.15.5:0/422696837 wait complete. 2023-07-21T19:13:19.170 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 30 2023-07-21T19:13:19.189 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:13:19.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3713583193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28100d00 msgr2=0x7f4b281010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3713583193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28100d00 0x7f4b281010d0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009990 tx=0x7f4b24009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3713583193 shutdown_connections 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3713583193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b2810a040 0x7f4b2810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3713583193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b28100d00 0x7f4b281010d0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3713583193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 0x7f4b28109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3713583193 >> 172.21.15.5:0/3713583193 conn(0x7f4b280fa4b0 msgr2=0x7f4b280fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3713583193 shutdown_connections 2023-07-21T19:13:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.414+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3713583193 wait complete. 2023-07-21T19:13:19.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.415+0000 7f4b2f5ff700 1 Processor -- start 2023-07-21T19:13:19.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 -- start start 2023-07-21T19:13:19.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b28100d00 0x7f4b280fe950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 0x7f4b280fee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b280ff6c0 con 0x7f4b2810a040 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b281a4190 con 0x7f4b28100d00 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b281a42d0 con 0x7f4b28101610 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54454/0 (socket says 172.21.15.5:54454) 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2db9c700 1 -- 172.21.15.5:0/3673797833 learned_addr learned my addr 172.21.15.5:0/3673797833 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.416+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b28100d00 0x7f4b280fe950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 0x7f4b280fee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 -- 172.21.15.5:0/3673797833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 msgr2=0x7f4b280fee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 0x7f4b280fee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 -- 172.21.15.5:0/3673797833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b28100d00 msgr2=0x7f4b280fe950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b28100d00 0x7f4b280fe950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 -- 172.21.15.5:0/3673797833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b240095f0 con 0x7f4b2810a040 2023-07-21T19:13:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2db9c700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007d70 tx=0x7f4b2000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b1e7fc700 1 -- 172.21.15.5:0/3673797833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2000ac90 con 0x7f4b2810a040 2023-07-21T19:13:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.417+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b281a4530 con 0x7f4b2810a040 2023-07-21T19:13:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.418+0000 7f4b1e7fc700 1 -- 172.21.15.5:0/3673797833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b2000adf0 con 0x7f4b2810a040 2023-07-21T19:13:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.418+0000 7f4b1e7fc700 1 -- 172.21.15.5:0/3673797833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b200115e0 con 0x7f4b2810a040 2023-07-21T19:13:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.418+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b281a4a60 con 0x7f4b2810a040 2023-07-21T19:13:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.421+0000 7f4b1e7fc700 1 -- 172.21.15.5:0/3673797833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4b20012420 con 0x7f4b2810a040 2023-07-21T19:13:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.422+0000 7f4b42eceb80 1 Processor -- start 2023-07-21T19:13:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 -- start start 2023-07-21T19:13:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f44b5cb0 con 0x5556f45baad0 2023-07-21T19:13:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f4508a40 con 0x5556f45af2d0 2023-07-21T19:13:19.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.423+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f4501ed0 con 0x5556f45afbe0 2023-07-21T19:13:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b1e7fc700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 0x7f4b140a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55180/0 (socket says 172.21.15.5:55180) 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 -- 172.21.15.5:0/1564053758 learned_addr learned my addr 172.21.15.5:0/1564053758 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b1e7fc700 1 -- 172.21.15.5:0/3673797833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4b200ccd60 con 0x7f4b2810a040 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2db9c700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 0x7f4b140a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 -- 172.21.15.5:0/1564053758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 msgr2=0x5556f45af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 -- 172.21.15.5:0/1564053758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 msgr2=0x5556f45bceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45bceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2d39b700 1 -- 172.21.15.5:0/1564053758 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f432fde0 con 0x5556f45afbe0 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.424+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:19.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.425+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b24015000 tx=0x7f4b24009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff2e0524501e16e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.425+0000 7f4b2db9c700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 0x7f4b140a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007d40 tx=0x7f4b20012b80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.425+0000 7f4b137fe700 1 -- 172.21.15.5:0/1564053758 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2401d070 con 0x5556f45afbe0 2023-07-21T19:13:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.425+0000 7f4b137fe700 1 -- 172.21.15.5:0/1564053758 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b2401e040 con 0x5556f45afbe0 2023-07-21T19:13:19.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.425+0000 7f4b137fe700 1 -- 172.21.15.5:0/1564053758 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24004760 con 0x5556f45afbe0 2023-07-21T19:13:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 -- 172.21.15.5:0/1564053758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 msgr2=0x5556f45ba590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4b24015000 tx=0x7f4b24009f90 comp rx=0 tx=0).stop 2023-07-21T19:13:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 -- 172.21.15.5:0/1564053758 shutdown_connections 2023-07-21T19:13:19.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45bceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/1564053758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45ba590 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 -- 172.21.15.5:0/1564053758 >> 172.21.15.5:0/1564053758 conn(0x5556f45a5da0 msgr2=0x5556f45ab6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 -- 172.21.15.5:0/1564053758 shutdown_connections 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.426+0000 7f4b42eceb80 1 -- 172.21.15.5:0/1564053758 wait complete. 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.427+0000 7f4b42eceb80 1 -- start start 2023-07-21T19:13:19.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.427+0000 7f4b42eceb80 1 Processor -- start 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45ba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45b7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f44a1ac0 con 0x5556f45baad0 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f44a17c0 con 0x5556f45af2d0 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f4385920 con 0x5556f45afbe0 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b42eceb80 10 client.0 _ll_register_callbacks cb 0x5556f449f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:19.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b2cb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45ba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b2cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45b7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55194/0 (socket says 172.21.15.5:55194) 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.428+0000 7f4b2d39b700 1 -- 172.21.15.5:0/355339874 learned_addr learned my addr 172.21.15.5:0/355339874 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 -- 172.21.15.5:0/355339874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 msgr2=0x5556f45ba500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45ba500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 -- 172.21.15.5:0/355339874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 msgr2=0x5556f45b7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45b7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 -- 172.21.15.5:0/355339874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b2810d170 con 0x5556f45afbe0 2023-07-21T19:13:19.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009350 tx=0x7f4b240053f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ead7551812cda3f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.429+0000 7f4b10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:13:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.430+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2401d030 con 0x5556f45afbe0 2023-07-21T19:13:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.430+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5556f445b4f0 con 0x5556f45afbe0 2023-07-21T19:13:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.430+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4b2400be50 con 0x5556f45afbe0 2023-07-21T19:13:19.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b2400bcb0 con 0x5556f45afbe0 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 10 client.26482 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5556f4318c60 con 0x5556f45afbe0 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5556f445b4f0 con 0x5556f45afbe0 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b2400f690 con 0x5556f45afbe0 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 10 client.26482 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 10 client.26482 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:13:19.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 20 client.26482 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f45c9f80 0x5556f45cc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 4 client.26482 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.431+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556f44faeb0 con 0x5556f45c9f80 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.432+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f45c9f80 0x5556f45cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.432+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f45c9f80 0x5556f45cc360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8ba1e43a34980189 server_cookie=3fa632f4721fe51 in_seq=0 out_seq=0 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.432+0000 7f4b10ff9700 10 client.26482 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.433+0000 7f4b10ff9700 1 -- 172.21.15.5:0/355339874 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5556f44faeb0 con 0x5556f45c9f80 2023-07-21T19:13:19.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.433+0000 7f4b10ff9700 10 client.26482 handle_command_reply: tid=0 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.433+0000 7f4b42eceb80 1 client.26482 shutdown 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 msgr2=0x5556f45b7710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009350 tx=0x7f4b240053f0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 shutdown_connections 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45baad0 0x5556f45b7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f45c9f80 0x5556f45cc360 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45af2d0 0x5556f45ba500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/355339874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45afbe0 0x5556f45b7710 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 >> 172.21.15.5:0/355339874 conn(0x5556f45a5da0 msgr2=0x5556f45ab230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b10ff9700 0 client.26482 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 shutdown_connections 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 1 -- 172.21.15.5:0/355339874 wait complete. 2023-07-21T19:13:19.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.434+0000 7f4b42eceb80 20 client.26482 trim_cache size 0 max 16384 2023-07-21T19:13:19.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.450+0000 7f4b42eceb80 1 Processor -- start 2023-07-21T19:13:19.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.450+0000 7f4b42eceb80 1 -- start start 2023-07-21T19:13:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45af2d0 0x5556f45b69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45baad0 0x5556f45a9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45b0540 con 0x5556f45afbe0 2023-07-21T19:13:19.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45aa350 con 0x5556f45baad0 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45b7e10 con 0x5556f45af2d0 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b2d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54494/0 (socket says 172.21.15.5:54494) 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b2d39b700 1 -- 172.21.15.5:0/2335161287 learned_addr learned my addr 172.21.15.5:0/2335161287 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45baad0 0x5556f45a9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.451+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45af2d0 0x5556f45b69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 -- 172.21.15.5:0/2335161287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45af2d0 msgr2=0x5556f45b69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45af2d0 0x5556f45b69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 -- 172.21.15.5:0/2335161287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45baad0 msgr2=0x5556f45a9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45baad0 0x5556f45a9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 -- 172.21.15.5:0/2335161287 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556f4356250 con 0x5556f45afbe0 2023-07-21T19:13:19.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009ce0 tx=0x7f4b24009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59002cc8d0f92efe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b13fff700 1 -- 172.21.15.5:0/2335161287 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b2401d070 con 0x5556f45afbe0 2023-07-21T19:13:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b13fff700 1 -- 172.21.15.5:0/2335161287 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b24005460 con 0x5556f45afbe0 2023-07-21T19:13:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.452+0000 7f4b13fff700 1 -- 172.21.15.5:0/2335161287 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b24003c40 con 0x5556f45afbe0 2023-07-21T19:13:19.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2335161287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 msgr2=0x5556f45bceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f4b24009ce0 tx=0x7f4b24009290 comp rx=0 tx=0).stop 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2335161287 shutdown_connections 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45baad0 0x5556f45a9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45af2d0 0x5556f45b69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2335161287 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45afbe0 0x5556f45bceb0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2335161287 >> 172.21.15.5:0/2335161287 conn(0x5556f45a5da0 msgr2=0x5556f45d0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2335161287 shutdown_connections 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.453+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2335161287 wait complete. 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 -- start start 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 Processor -- start 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 0x5556f45c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45baad0 0x5556f45bf530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45bfbd0 con 0x5556f45af2d0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45bfd40 con 0x5556f45afbe0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.454+0000 7f4b42eceb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556f45bff00 con 0x5556f45baad0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b42eceb80 10 client.0 _ll_register_callbacks cb 0x7f4b24009ee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54504/0 (socket says 172.21.15.5:54504) 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 -- 172.21.15.5:0/2426211594 learned_addr learned my addr 172.21.15.5:0/2426211594 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 0x5556f45c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 -- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45baad0 msgr2=0x5556f45bf530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45baad0 0x5556f45bf530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 -- 172.21.15.5:0/2426211594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 msgr2=0x5556f45c1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 0x5556f45c1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 -- 172.21.15.5:0/2426211594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5556f4318c60 con 0x5556f45af2d0 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 0x5556f45c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b2cb9a700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5556f45ccc40 tx=0x7f4b18015b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28094d0e555c24c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.455+0000 7f4b10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b18014190 con 0x5556f45af2d0 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b2401e040 con 0x5556f45af2d0 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f4b18022a10 con 0x5556f45af2d0 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b180142f0 con 0x5556f45af2d0 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b42eceb80 10 client.16836 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:19.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5556f432e2b0 con 0x5556f45af2d0 2023-07-21T19:13:19.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.456+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b2401e040 con 0x5556f45af2d0 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4b18003680 con 0x5556f45af2d0 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 10 client.16836 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 10 client.16836 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 20 client.16836 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f461ef10 0x5556f4621320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 4 client.16836 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5556f44bd050 con 0x5556f461ef10 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.457+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f461ef10 0x5556f4621320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b2d39b700 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f461ef10 0x5556f4621320 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=379b34197e86d835 server_cookie=93d7fcfb42e63665 in_seq=0 out_seq=0 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b10ff9700 10 client.16836 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b10ff9700 1 -- 172.21.15.5:0/2426211594 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5556f44bd050 con 0x5556f461ef10 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b10ff9700 10 client.16836 handle_command_reply: tid=0 2023-07-21T19:13:19.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 client.16836 shutdown 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 msgr2=0x5556f45c0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x5556f45ccc40 tx=0x7f4b18015b20 comp rx=0 tx=0).stop 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 shutdown_connections 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556f45baad0 0x5556f45bf530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556f45af2d0 0x5556f45c0ef0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5556f461ef10 0x5556f4621320 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 --2- 172.21.15.5:0/2426211594 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556f45afbe0 0x5556f45c1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b10ff9700 0 client.16836 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.458+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 >> 172.21.15.5:0/2426211594 conn(0x5556f45a5da0 msgr2=0x5556f45c6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 shutdown_connections 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b42eceb80 1 -- 172.21.15.5:0/2426211594 wait complete. 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b42eceb80 20 client.16836 trim_cache size 0 max 16384 2023-07-21T19:13:19.469 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:13:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 msgr2=0x7f4b140a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 0x7f4b140a3770 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007d40 tx=0x7f4b20012b80 comp rx=0 tx=0).stop 2023-07-21T19:13:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 msgr2=0x7f4b281a3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f4b20007d70 tx=0x7f4b2000a340 comp rx=0 tx=0).stop 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 shutdown_connections 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b140a12c0 0x7f4b140a3770 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b2810a040 0x7f4b281a3c50 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b28100d00 0x7f4b280fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 --2- 172.21.15.5:0/3673797833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b28101610 0x7f4b280fee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.459+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 >> 172.21.15.5:0/3673797833 conn(0x7f4b280fa4b0 msgr2=0x7f4b2810c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.460+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 shutdown_connections 2023-07-21T19:13:19.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.460+0000 7f4b2f5ff700 1 -- 172.21.15.5:0/3673797833 wait complete. 2023-07-21T19:13:19.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:19 smithi005 ceph-mon[126460]: pgmap v174: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:13:19.479 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:19 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/422696837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:13:19.482 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:13:19.643 INFO:teuthology.orchestra.run.smithi005.stderr:20500000 of 50000000 tuples (41%) done (elapsed 317.21 s, remaining 456.47 s) 2023-07-21T19:13:19.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 -- 172.21.15.5:0/2189123398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec105af0 msgr2=0x7f71ec105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2189123398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec105af0 0x7f71ec105ec0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009990 tx=0x7f71e0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 -- 172.21.15.5:0/2189123398 shutdown_connections 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2189123398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec100490 0x7f71ec10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2189123398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec105af0 0x7f71ec105ec0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2189123398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec0ffae0 0x7f71ec0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 -- 172.21.15.5:0/2189123398 >> 172.21.15.5:0/2189123398 conn(0x7f71ec0fb5e0 msgr2=0x7f71ec0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 -- 172.21.15.5:0/2189123398 shutdown_connections 2023-07-21T19:13:19.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.710+0000 7f71f2fff700 1 -- 172.21.15.5:0/2189123398 wait complete. 2023-07-21T19:13:19.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.711+0000 7f71f2fff700 1 Processor -- start 2023-07-21T19:13:19.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.711+0000 7f71f2fff700 1 -- start start 2023-07-21T19:13:19.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec100490 0x7f71ec072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 0x7f71ec1a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec073580 con 0x7f71ec0ffae0 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a4280 con 0x7f71ec100490 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f2fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec1a43c0 con 0x7f71ec105af0 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54520/0 (socket says 172.21.15.5:54520) 2023-07-21T19:13:19.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2060006578 learned_addr learned my addr 172.21.15.5:0/2060006578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.712+0000 7f71f159c700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 0x7f71ec1a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71ebfff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec100490 0x7f71ec072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2060006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 msgr2=0x7f71ec1a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 0x7f71ec1a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2060006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec100490 msgr2=0x7f71ec072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec100490 0x7f71ec072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2060006578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e00095f0 con 0x7f71ec0ffae0 2023-07-21T19:13:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f159c700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 0x7f71ec1a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000c00 tx=0x7f71e0009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71e9ffb700 1 -- 172.21.15.5:0/2060006578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001d070 con 0x7f71ec0ffae0 2023-07-21T19:13:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec1a45c0 con 0x7f71ec0ffae0 2023-07-21T19:13:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71e9ffb700 1 -- 172.21.15.5:0/2060006578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e0022470 con 0x7f71ec0ffae0 2023-07-21T19:13:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71e9ffb700 1 -- 172.21.15.5:0/2060006578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0017650 con 0x7f71ec0ffae0 2023-07-21T19:13:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.713+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1a4a40 con 0x7f71ec0ffae0 2023-07-21T19:13:19.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.717+0000 7f71e9ffb700 1 -- 172.21.15.5:0/2060006578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f71e000fb00 con 0x7f71ec0ffae0 2023-07-21T19:13:19.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.719+0000 7f7206a0ab80 1 Processor -- start 2023-07-21T19:13:19.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 -- start start 2023-07-21T19:13:19.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c7f2d0 0x55e681c7f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c8aad0 0x55e681c8ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f71e9ffb700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 0x7f71dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681b85cb0 con 0x55e681c7fbe0 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681bd8a40 con 0x55e681c7f2d0 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681bd1ed0 con 0x55e681c8aad0 2023-07-21T19:13:19.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f71e9ffb700 1 -- 172.21.15.5:0/2060006578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71e00ce160 con 0x7f71ec0ffae0 2023-07-21T19:13:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.720+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54532/0 (socket says 172.21.15.5:54532) 2023-07-21T19:13:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2126600959 learned_addr learned my addr 172.21.15.5:0/2126600959 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71ebfff700 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c8aad0 0x55e681c8ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f159c700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 0x7f71dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f159c700 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c7f2d0 0x55e681c7f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2126600959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c8aad0 msgr2=0x55e681c8ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c8aad0 0x55e681c8ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2126600959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c7f2d0 msgr2=0x55e681c7f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c7f2d0 0x55e681c7f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 -- 172.21.15.5:0/2126600959 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6819ffde0 con 0x55e681c7fbe0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f71e009b390 tx=0x7f71e009b1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31050e21c1b23169 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.721+0000 7f71f159c700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 0x7f71dc0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009550 tx=0x7f71e40090b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.722+0000 7f71d6ffd700 1 -- 172.21.15.5:0/2126600959 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0022850 con 0x55e681c7fbe0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.722+0000 7f71d6ffd700 1 -- 172.21.15.5:0/2126600959 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e009b400 con 0x55e681c7fbe0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.722+0000 7f71d6ffd700 1 -- 172.21.15.5:0/2126600959 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e009f120 con 0x55e681c7fbe0 2023-07-21T19:13:19.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.722+0000 7f7206a0ab80 1 -- 172.21.15.5:0/2126600959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 msgr2=0x55e681c8a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.722+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f71e009b390 tx=0x7f71e009b1d0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 -- 172.21.15.5:0/2126600959 shutdown_connections 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c8aad0 0x55e681c8ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c7f2d0 0x55e681c7f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/2126600959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8a590 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 -- 172.21.15.5:0/2126600959 >> 172.21.15.5:0/2126600959 conn(0x55e681c75da0 msgr2=0x55e681c7b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 -- 172.21.15.5:0/2126600959 shutdown_connections 2023-07-21T19:13:19.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 -- 172.21.15.5:0/2126600959 wait complete. 2023-07-21T19:13:19.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.723+0000 7f7206a0ab80 1 -- start start 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 Processor -- start 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8b920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681b71ac0 con 0x55e681c7fbe0 2023-07-21T19:13:19.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681b717c0 con 0x55e681c8aad0 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681a55920 con 0x55e681c7f2d0 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.724+0000 7f7206a0ab80 10 client.0 _ll_register_callbacks cb 0x55e681b6f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54548/0 (socket says 172.21.15.5:54548) 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71f0d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55254/0 (socket says 172.21.15.5:55254) 2023-07-21T19:13:19.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 -- 172.21.15.5:0/1108583440 learned_addr learned my addr 172.21.15.5:0/1108583440 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71f159c700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 -- 172.21.15.5:0/1108583440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 msgr2=0x55e681c8aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 -- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 msgr2=0x55e681c8b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 -- 172.21.15.5:0/1108583440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1a4a40 con 0x55e681c7fbe0 2023-07-21T19:13:19.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f71cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.725+0000 7f71ebfff700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71d8012700 tx=0x7f71d8012a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351c957e7ac3abd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8017070 con 0x55e681c7fbe0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71d8004260 con 0x55e681c7fbe0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e681b2b4f0 con 0x55e681c7fbe0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d8004420 con 0x55e681c7fbe0 2023-07-21T19:13:19.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f7206a0ab80 10 client.16848 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e6819e8c60 con 0x55e681c7fbe0 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.726+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e681b2b4f0 con 0x55e681c7fbe0 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71d800f6a0 con 0x55e681c7fbe0 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 10 client.16848 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 10 client.16848 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:13:19.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 20 client.16848 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681c9a1e0 0x55e681c9c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 4 client.16848 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e681bcaeb0 con 0x55e681c9a1e0 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.727+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681c9a1e0 0x55e681c9c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.728+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681c9a1e0 0x55e681c9c5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d8a9470d4071865 server_cookie=492d3979cb7d1b47 in_seq=0 out_seq=0 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.728+0000 7f71cbfff700 10 client.16848 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:13:19.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f71cbfff700 1 -- 172.21.15.5:0/1108583440 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e681bcaeb0 con 0x55e681c9a1e0 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f71cbfff700 10 client.16848 handle_command_reply: tid=0 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 client.16848 shutdown 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 msgr2=0x55e681c8b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f71d8012700 tx=0x7f71d8012a10 comp rx=0 tx=0).stop 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 shutdown_connections 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681c9a1e0 0x55e681c9c5f0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f71cbfff700 0 client.16848 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/1108583440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8b3e0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 >> 172.21.15.5:0/1108583440 conn(0x55e681c75da0 msgr2=0x55e681c7b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 shutdown_connections 2023-07-21T19:13:19.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 1 -- 172.21.15.5:0/1108583440 wait complete. 2023-07-21T19:13:19.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.729+0000 7f7206a0ab80 20 client.16848 trim_cache size 0 max 16384 2023-07-21T19:13:19.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.748+0000 7f7206a0ab80 1 Processor -- start 2023-07-21T19:13:19.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 -- start start 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c95660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c801c0 con 0x55e681c7fbe0 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c75970 con 0x55e681c8aad0 2023-07-21T19:13:19.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c811a0 con 0x55e681c7f2d0 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54558/0 (socket says 172.21.15.5:54558) 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f71ebfff700 1 -- 172.21.15.5:0/202856295 learned_addr learned my addr 172.21.15.5:0/202856295 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.749+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c95660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71f159c700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 -- 172.21.15.5:0/202856295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 msgr2=0x55e681c95660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c95660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 -- 172.21.15.5:0/202856295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 msgr2=0x55e681c872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 -- 172.21.15.5:0/202856295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e681a26250 con 0x55e681c7fbe0 2023-07-21T19:13:19.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71f159c700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:13:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71ebfff700 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f71d8000c00 tx=0x7f71d800fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67b98db86f2a1379 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.750+0000 7f71d77fe700 1 -- 172.21.15.5:0/202856295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8017040 con 0x55e681c7fbe0 2023-07-21T19:13:19.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.751+0000 7f71d77fe700 1 -- 172.21.15.5:0/202856295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71d800fd20 con 0x55e681c7fbe0 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.751+0000 7f71d77fe700 1 -- 172.21.15.5:0/202856295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d8003a20 con 0x55e681c7fbe0 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.751+0000 7f7206a0ab80 1 -- 172.21.15.5:0/202856295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 msgr2=0x55e681c86d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.751+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f71d8000c00 tx=0x7f71d800fc50 comp rx=0 tx=0).stop 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 -- 172.21.15.5:0/202856295 shutdown_connections 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c95660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/202856295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c86d80 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 -- 172.21.15.5:0/202856295 >> 172.21.15.5:0/202856295 conn(0x55e681c75da0 msgr2=0x55e681c893b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 -- 172.21.15.5:0/202856295 shutdown_connections 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 -- 172.21.15.5:0/202856295 wait complete. 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 -- start start 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.752+0000 7f7206a0ab80 1 Processor -- start 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c8e250 con 0x55e681c7fbe0 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c8e3c0 con 0x55e681c8aad0 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e681c8e580 con 0x55e681c7f2d0 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f7206a0ab80 10 client.0 _ll_register_callbacks cb 0x7f71d8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f71ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54566/0 (socket says 172.21.15.5:54566) 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f71ebfff700 1 -- 172.21.15.5:0/735121902 learned_addr learned my addr 172.21.15.5:0/735121902 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:19.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f71f159c700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.753+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 -- 172.21.15.5:0/735121902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 msgr2=0x55e681c8f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 -- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 msgr2=0x55e681c8dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 -- 172.21.15.5:0/735121902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6819e8c60 con 0x55e681c7fbe0 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71ebfff700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f71d802e000 tx=0x7f71d8009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68c01d12966a928a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:19.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800f640 con 0x55e681c7fbe0 2023-07-21T19:13:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e009b400 con 0x55e681c7fbe0 2023-07-21T19:13:19.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.754+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f71d800f7a0 con 0x55e681c7fbe0 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71d80093a0 con 0x55e681c7fbe0 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 10 client.16854 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55e6819fe2b0 con 0x55e681c7fbe0 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e009b400 con 0x55e681c7fbe0 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71d802e6e0 con 0x55e681c7fbe0 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 10 client.16854 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:19.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 10 client.16854 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 20 client.16854 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681ceef20 0x55e681cf1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 4 client.16854 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.755+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e681b8d050 con 0x55e681ceef20 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.756+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681ceef20 0x55e681cf1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.756+0000 7f71cbfff700 10 client.16854 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.756+0000 7f71f0d9b700 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681ceef20 0x55e681cf1330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=aa41c30fddd64e75 server_cookie=c888203f1fdced65 in_seq=0 out_seq=0 2023-07-21T19:13:19.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f71cbfff700 1 -- 172.21.15.5:0/735121902 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e681b8d050 con 0x55e681ceef20 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f71cbfff700 10 client.16854 handle_command_reply: tid=0 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 client.16854 shutdown 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 msgr2=0x55e681c8fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f71d802e000 tx=0x7f71d8009d90 comp rx=0 tx=0).stop 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 shutdown_connections 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e681ceef20 0x55e681cf1330 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e681c8aad0 0x55e681c8dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e681c7f2d0 0x55e681c8f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f71cbfff700 0 client.16854 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 --2- 172.21.15.5:0/735121902 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e681c7fbe0 0x55e681c8fd70 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 >> 172.21.15.5:0/735121902 conn(0x55e681c75da0 msgr2=0x55e681c882c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 shutdown_connections 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 1 -- 172.21.15.5:0/735121902 wait complete. 2023-07-21T19:13:19.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.757+0000 7f7206a0ab80 20 client.16854 trim_cache size 0 max 16384 2023-07-21T19:13:19.768 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:13:19.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 msgr2=0x7f71dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 0x7f71dc0a37f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f71e4009550 tx=0x7f71e40090b0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 msgr2=0x7f71ec072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f71e0000c00 tx=0x7f71e0009570 comp rx=0 tx=0).stop 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 shutdown_connections 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec105af0 0x7f71ec1a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71dc0a1340 0x7f71dc0a37f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0ffae0 0x7f71ec072810 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 --2- 172.21.15.5:0/2060006578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec100490 0x7f71ec072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 >> 172.21.15.5:0/2060006578 conn(0x7f71ec0fb5e0 msgr2=0x7f71ec0fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 shutdown_connections 2023-07-21T19:13:19.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:19.758+0000 7f71f2fff700 1 -- 172.21.15.5:0/2060006578 wait complete. 2023-07-21T19:13:19.782 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:13:19.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:19 smithi101 ceph-mon[128136]: pgmap v174: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:13:19.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:19 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/422696837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:13:19.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:19 smithi156 ceph-mon[128316]: pgmap v174: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 MiB/s wr, 29 op/s 2023-07-21T19:13:19.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:19 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/422696837' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:13:20.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 -- 172.21.15.5:0/3966283547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20100aa0 msgr2=0x7faf20100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 --2- 172.21.15.5:0/3966283547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20100aa0 0x7faf20100e70 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7faf1c009990 tx=0x7faf1c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 -- 172.21.15.5:0/3966283547 shutdown_connections 2023-07-21T19:13:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 --2- 172.21.15.5:0/3966283547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20109fc0 0x7faf2010c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 --2- 172.21.15.5:0/3966283547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20100aa0 0x7faf20100e70 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 --2- 172.21.15.5:0/3966283547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 0x7faf20109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 -- 172.21.15.5:0/3966283547 >> 172.21.15.5:0/3966283547 conn(0x7faf200745d0 msgr2=0x7faf200749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 -- 172.21.15.5:0/3966283547 shutdown_connections 2023-07-21T19:13:20.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.033+0000 7faf26b9d700 1 -- 172.21.15.5:0/3966283547 wait complete. 2023-07-21T19:13:20.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.034+0000 7faf26b9d700 1 Processor -- start 2023-07-21T19:13:20.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.034+0000 7faf26b9d700 1 -- start start 2023-07-21T19:13:20.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20100aa0 0x7faf2019b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 0x7faf2019bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20195dc0 con 0x7faf20109fc0 2023-07-21T19:13:20.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf20195f30 con 0x7faf201013b0 2023-07-21T19:13:20.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf26b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf201960f0 con 0x7faf20100aa0 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf2639c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf2639c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54580/0 (socket says 172.21.15.5:54580) 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.035+0000 7faf2639c700 1 -- 172.21.15.5:0/388647832 learned_addr learned my addr 172.21.15.5:0/388647832 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2539a700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 0x7faf2019bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf25b9b700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20100aa0 0x7faf2019b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 -- 172.21.15.5:0/388647832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20100aa0 msgr2=0x7faf2019b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20100aa0 0x7faf2019b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 -- 172.21.15.5:0/388647832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 msgr2=0x7faf2019bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 0x7faf2019bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 -- 172.21.15.5:0/388647832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c0095f0 con 0x7faf20109fc0 2023-07-21T19:13:20.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf2639c700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf18007ee0 tx=0x7faf1800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.036+0000 7faf16ffd700 1 -- 172.21.15.5:0/388647832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1800ddd0 con 0x7faf20109fc0 2023-07-21T19:13:20.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.037+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf201963d0 con 0x7faf20109fc0 2023-07-21T19:13:20.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.037+0000 7faf16ffd700 1 -- 172.21.15.5:0/388647832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1800f040 con 0x7faf20109fc0 2023-07-21T19:13:20.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.037+0000 7faf16ffd700 1 -- 172.21.15.5:0/388647832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf18005970 con 0x7faf20109fc0 2023-07-21T19:13:20.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.037+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf201a26d0 con 0x7faf20109fc0 2023-07-21T19:13:20.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.040+0000 7faf16ffd700 1 -- 172.21.15.5:0/388647832 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7faf18004fb0 con 0x7faf20109fc0 2023-07-21T19:13:20.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.041+0000 7faf3b81fb80 1 Processor -- start 2023-07-21T19:13:20.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 -- start start 2023-07-21T19:13:20.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 0x559c4e5f08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f0e00 0x559c4e5fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 0x559c4e5fd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e4f5cb0 con 0x559c4e5fad30 2023-07-21T19:13:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e548a40 con 0x559c4e5f04f0 2023-07-21T19:13:20.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e541ed0 con 0x559c4e5f0e00 2023-07-21T19:13:20.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf16ffd700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 0x7faf0c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf2539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 0x559c4e5fd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.042+0000 7faf2539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 0x559c4e5fd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54596/0 (socket says 172.21.15.5:54596) 2023-07-21T19:13:20.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf16ffd700 1 -- 172.21.15.5:0/388647832 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faf180cc540 con 0x7faf20109fc0 2023-07-21T19:13:20.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 -- 172.21.15.5:0/2833100280 learned_addr learned my addr 172.21.15.5:0/2833100280 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:20.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 0x559c4e5f08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2639c700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 0x7faf0c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf25b9b700 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f0e00 0x559c4e5fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 -- 172.21.15.5:0/2833100280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f0e00 msgr2=0x559c4e5fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f0e00 0x559c4e5fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 -- 172.21.15.5:0/2833100280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 msgr2=0x559c4e5fd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 0x559c4e5fd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 -- 172.21.15.5:0/2833100280 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c4e36fde0 con 0x559c4e5f04f0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2639c700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 0x7faf0c0a3890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf18095fb0 tx=0x7faf18009650 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.043+0000 7faf2539a700 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 0x559c4e5f08c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf10011880 tx=0x7faf10009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7be54a9625b78b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.044+0000 7faf0bfff700 1 -- 172.21.15.5:0/2833100280 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10011ea0 con 0x559c4e5f04f0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.044+0000 7faf0bfff700 1 -- 172.21.15.5:0/2833100280 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf10011540 con 0x559c4e5f04f0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.044+0000 7faf0bfff700 1 -- 172.21.15.5:0/2833100280 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1000e3c0 con 0x559c4e5f04f0 2023-07-21T19:13:20.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2833100280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 msgr2=0x559c4e5f08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 0x559c4e5f08c0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7faf10011880 tx=0x7faf10009260 comp rx=0 tx=0).stop 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2833100280 shutdown_connections 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5fad30 0x559c4e5fd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f04f0 0x559c4e5f08c0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2833100280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f0e00 0x559c4e5fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2833100280 >> 172.21.15.5:0/2833100280 conn(0x559c4e5e5ff0 msgr2=0x559c4e5eb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2833100280 shutdown_connections 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2833100280 wait complete. 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.045+0000 7faf3b81fb80 1 -- start start 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 Processor -- start 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f0e00 0x559c4e5f7920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5fad30 0x559c4e5f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e4e1ac0 con 0x559c4e5f04f0 2023-07-21T19:13:20.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e4e17c0 con 0x559c4e5f0e00 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e3c5920 con 0x559c4e5fad30 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.046+0000 7faf3b81fb80 10 client.0 _ll_register_callbacks cb 0x559c4e4df760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54602/0 (socket says 172.21.15.5:54602) 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 -- 172.21.15.5:0/1172867566 learned_addr learned my addr 172.21.15.5:0/1172867566 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf25b9b700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f0e00 0x559c4e5f7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 -- 172.21.15.5:0/1172867566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5fad30 msgr2=0x559c4e5f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5fad30 0x559c4e5f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 -- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f0e00 msgr2=0x559c4e5f7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f0e00 0x559c4e5f7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 -- 172.21.15.5:0/1172867566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf10019040 con 0x559c4e5f04f0 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf2539a700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faf100099d0 tx=0x7faf100054d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28fae14fb9d17cd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.047+0000 7faf097fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:20.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf10011ea0 con 0x559c4e5f04f0 2023-07-21T19:13:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c4e49b4f0 con 0x559c4e5f04f0 2023-07-21T19:13:20.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faf10011020 con 0x559c4e5f04f0 2023-07-21T19:13:20.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf10009ca0 con 0x559c4e5f04f0 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf3b81fb80 10 client.16866 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559c4e358c60 con 0x559c4e5f04f0 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c4e49b4f0 con 0x559c4e5f04f0 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.048+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf1001d070 con 0x559c4e5f04f0 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 10 client.16866 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 10 client.16866 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:20.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 20 client.16866 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e60a630 0x559c4e60ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 4 client.16866 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c4e53aeb0 con 0x559c4e60a630 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.049+0000 7faf25b9b700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e60a630 0x559c4e60ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.050+0000 7faf097fa700 10 client.16866 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.050+0000 7faf25b9b700 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e60a630 0x559c4e60ca40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a88bb4ac35ef88e server_cookie=2994658ff4deb993 in_seq=0 out_seq=0 2023-07-21T19:13:20.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.050+0000 7faf097fa700 1 -- 172.21.15.5:0/1172867566 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c4e53aeb0 con 0x559c4e60a630 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.050+0000 7faf097fa700 10 client.16866 handle_command_reply: tid=0 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 client.16866 shutdown 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 msgr2=0x559c4e5fa710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7faf100099d0 tx=0x7faf100054d0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 shutdown_connections 2023-07-21T19:13:20.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5fad30 0x559c4e5f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e60a630 0x559c4e60ca40 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f04f0 0x559c4e5fa710 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/1172867566 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5f0e00 0x559c4e5f7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf097fa700 0 client.16866 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 >> 172.21.15.5:0/1172867566 conn(0x559c4e5e5ff0 msgr2=0x559c4e5eb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 shutdown_connections 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 1 -- 172.21.15.5:0/1172867566 wait complete. 2023-07-21T19:13:20.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.051+0000 7faf3b81fb80 20 client.16866 trim_cache size 0 max 16384 2023-07-21T19:13:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.070+0000 7faf3b81fb80 1 Processor -- start 2023-07-21T19:13:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 -- start start 2023-07-21T19:13:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e601810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e5fa240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e5f1980 con 0x559c4e5f0e00 2023-07-21T19:13:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e5f1f50 con 0x559c4e5fad30 2023-07-21T19:13:20.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e5f11d0 con 0x559c4e5f04f0 2023-07-21T19:13:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf25b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf25b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54618/0 (socket says 172.21.15.5:54618) 2023-07-21T19:13:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf25b9b700 1 -- 172.21.15.5:0/3887583517 learned_addr learned my addr 172.21.15.5:0/3887583517 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.071+0000 7faf2539a700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e601810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf2539a700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e5fa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 -- 172.21.15.5:0/3887583517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 msgr2=0x559c4e601810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e601810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 -- 172.21.15.5:0/3887583517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 msgr2=0x559c4e5fa240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e5fa240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 -- 172.21.15.5:0/3887583517 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c4e396250 con 0x559c4e5f0e00 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf2539a700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e5fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:20.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf25b9b700 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000c00 tx=0x7faf1c005af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9f52b08f5e77ba9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf14bff700 1 -- 172.21.15.5:0/3887583517 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c028d50 con 0x559c4e5f0e00 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf14bff700 1 -- 172.21.15.5:0/3887583517 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf1c02c070 con 0x559c4e5f0e00 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.072+0000 7faf14bff700 1 -- 172.21.15.5:0/3887583517 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c00b970 con 0x559c4e5f0e00 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 -- 172.21.15.5:0/3887583517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 msgr2=0x559c4e605fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7faf1c000c00 tx=0x7faf1c005af0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 -- 172.21.15.5:0/3887583517 shutdown_connections 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e5fa240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e601810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/3887583517 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e605fb0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.073+0000 7faf3b81fb80 1 -- 172.21.15.5:0/3887583517 >> 172.21.15.5:0/3887583517 conn(0x559c4e5e5ff0 msgr2=0x559c4e606980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.074+0000 7faf3b81fb80 1 -- 172.21.15.5:0/3887583517 shutdown_connections 2023-07-21T19:13:20.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.074+0000 7faf3b81fb80 1 -- 172.21.15.5:0/3887583517 wait complete. 2023-07-21T19:13:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.074+0000 7faf3b81fb80 1 -- start start 2023-07-21T19:13:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.074+0000 7faf3b81fb80 1 Processor -- start 2023-07-21T19:13:20.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e5ffd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e6002b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e6007f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e601dc0 con 0x559c4e5f0e00 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e65e4c0 con 0x559c4e5fad30 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c4e662a10 con 0x559c4e5f04f0 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf3b81fb80 10 client.0 _ll_register_callbacks cb 0x7faf10009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf2539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e5ffd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf2539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e5ffd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55330/0 (socket says 172.21.15.5:55330) 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf2539a700 1 -- 172.21.15.5:0/2983399183 learned_addr learned my addr 172.21.15.5:0/2983399183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf25b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e6002b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.075+0000 7faf2539a700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e6007f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 -- 172.21.15.5:0/2983399183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 msgr2=0x559c4e5ffd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e5ffd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 -- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 msgr2=0x559c4e6007f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e6007f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 -- 172.21.15.5:0/2983399183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2010d0f0 con 0x559c4e5f0e00 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf25b9b700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e6002b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x559c4e5f8a10 tx=0x7faf1c0285b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab40f0703bd18d3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.076+0000 7faf097fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c036070 con 0x559c4e5f0e00 2023-07-21T19:13:20.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7faf1c028b50 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faf10011540 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf1c028d10 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf3b81fb80 10 client.16872 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559c4e36e2b0 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf10011540 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.077+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf1c00bbf0 con 0x559c4e5f0e00 2023-07-21T19:13:20.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 10 client.16872 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 10 client.16872 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 20 client.16872 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e65e8c0 0x559c4e660cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 4 client.16872 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559c4e4fd050 con 0x559c4e65e8c0 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.078+0000 7faf2539a700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e65e8c0 0x559c4e660cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf097fa700 10 client.16872 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:20.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf2539a700 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e65e8c0 0x559c4e660cd0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d9b77aa1d2f43ff server_cookie=d197a6452ff1da4 in_seq=0 out_seq=0 2023-07-21T19:13:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf097fa700 1 -- 172.21.15.5:0/2983399183 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559c4e4fd050 con 0x559c4e65e8c0 2023-07-21T19:13:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf097fa700 10 client.16872 handle_command_reply: tid=0 2023-07-21T19:13:20.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 client.16872 shutdown 2023-07-21T19:13:20.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 msgr2=0x559c4e6002b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e6002b0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x559c4e5f8a10 tx=0x7faf1c0285b0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 shutdown_connections 2023-07-21T19:13:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c4e5fad30 0x559c4e6007f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c4e5f04f0 0x559c4e5ffd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c4e65e8c0 0x559c4e660cd0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 --2- 172.21.15.5:0/2983399183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c4e5f0e00 0x559c4e6002b0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.079+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 >> 172.21.15.5:0/2983399183 conn(0x559c4e5e5ff0 msgr2=0x559c4e605fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf097fa700 0 client.16872 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 shutdown_connections 2023-07-21T19:13:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf3b81fb80 1 -- 172.21.15.5:0/2983399183 wait complete. 2023-07-21T19:13:20.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf3b81fb80 20 client.16872 trim_cache size 0 max 16384 2023-07-21T19:13:20.093 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:13:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 msgr2=0x7faf0c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 0x7faf0c0a3890 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7faf18095fb0 tx=0x7faf18009650 comp rx=0 tx=0).stop 2023-07-21T19:13:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.080+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 msgr2=0x7faf20195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7faf18007ee0 tx=0x7faf1800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 shutdown_connections 2023-07-21T19:13:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf0c0a13e0 0x7faf0c0a3890 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf20109fc0 0x7faf20195750 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf20100aa0 0x7faf2019b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 --2- 172.21.15.5:0/388647832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf201013b0 0x7faf2019bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 >> 172.21.15.5:0/388647832 conn(0x7faf200745d0 msgr2=0x7faf20101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 shutdown_connections 2023-07-21T19:13:20.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:20.081+0000 7faf26b9d700 1 -- 172.21.15.5:0/388647832 wait complete. 2023-07-21T19:13:20.954 INFO:teuthology.orchestra.run.smithi005.stderr:20600000 of 50000000 tuples (41%) done (elapsed 318.52 s, remaining 454.58 s) 2023-07-21T19:13:21.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:13:21.104 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:13:21.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 -- 172.21.15.5:0/2560317712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538100d00 msgr2=0x7ff5381010d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/2560317712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538100d00 0x7ff5381010d0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7ff534009990 tx=0x7ff534009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 -- 172.21.15.5:0/2560317712 shutdown_connections 2023-07-21T19:13:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/2560317712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff53810a040 0x7ff53810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/2560317712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538100d00 0x7ff5381010d0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/2560317712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff538101610 0x7ff538109b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.338+0000 7ff53f5ff700 1 -- 172.21.15.5:0/2560317712 >> 172.21.15.5:0/2560317712 conn(0x7ff5380fa4b0 msgr2=0x7ff5380fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.339+0000 7ff53f5ff700 1 -- 172.21.15.5:0/2560317712 shutdown_connections 2023-07-21T19:13:21.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.339+0000 7ff53f5ff700 1 -- 172.21.15.5:0/2560317712 wait complete. 2023-07-21T19:13:21.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 Processor -- start 2023-07-21T19:13:21.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 -- start start 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff538100d00 0x7ff538197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff53810a040 0x7ff53819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819cd30 con 0x7ff538101610 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819cea0 con 0x7ff53810a040 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.340+0000 7ff53f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff53819d060 con 0x7ff538100d00 2023-07-21T19:13:21.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54646/0 (socket says 172.21.15.5:54646) 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 -- 172.21.15.5:0/303562006 learned_addr learned my addr 172.21.15.5:0/303562006 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53db9c700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff53810a040 0x7ff53819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53d39b700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff538100d00 0x7ff538197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 -- 172.21.15.5:0/303562006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff538100d00 msgr2=0x7ff538197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff538100d00 0x7ff538197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 -- 172.21.15.5:0/303562006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff53810a040 msgr2=0x7ff53819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff53810a040 0x7ff53819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 -- 172.21.15.5:0/303562006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5340095f0 con 0x7ff538101610 2023-07-21T19:13:21.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.341+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d950 tx=0x7ff52800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.342+0000 7ff52e7fc700 1 -- 172.21.15.5:0/303562006 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52800f890 con 0x7ff538101610 2023-07-21T19:13:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.342+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff53819d340 con 0x7ff538101610 2023-07-21T19:13:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.342+0000 7ff52e7fc700 1 -- 172.21.15.5:0/303562006 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff52800fe80 con 0x7ff538101610 2023-07-21T19:13:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.342+0000 7ff52e7fc700 1 -- 172.21.15.5:0/303562006 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52800e3b0 con 0x7ff538101610 2023-07-21T19:13:21.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.342+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381a4b80 con 0x7ff538101610 2023-07-21T19:13:21.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.346+0000 7ff52e7fc700 1 -- 172.21.15.5:0/303562006 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff528010460 con 0x7ff538101610 2023-07-21T19:13:21.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.348+0000 7ff552e8eb80 1 Processor -- start 2023-07-21T19:13:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.348+0000 7ff552e8eb80 1 -- start start 2023-07-21T19:13:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917350 0x561c36917720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36924f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3681dd30 con 0x561c36917c60 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36870ac0 con 0x561c36922b50 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36869f50 con 0x561c36917350 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff52e7fc700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 0x7ff5240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff52e7fc700 1 -- 172.21.15.5:0/303562006 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff5280ccaf0 con 0x7ff538101610 2023-07-21T19:13:21.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53db9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917350 0x561c36917720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54658/0 (socket says 172.21.15.5:54658) 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53d39b700 1 -- 172.21.15.5:0/918807062 learned_addr learned my addr 172.21.15.5:0/918807062 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36924f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.349+0000 7ff53db9c700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 0x7ff5240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 -- 172.21.15.5:0/918807062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917350 msgr2=0x561c36917720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917350 0x561c36917720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 -- 172.21.15.5:0/918807062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 msgr2=0x561c36922610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 -- 172.21.15.5:0/918807062 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c36697e60 con 0x561c36922b50 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53d39b700 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36924f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ca210 tx=0x7ff5280994f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9487cb78ec621fe0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff53db9c700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 0x7ff5240a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff530007b50 tx=0x7ff530009100 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.350+0000 7ff5237fe700 1 -- 172.21.15.5:0/918807062 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52809eca0 con 0x561c36922b50 2023-07-21T19:13:21.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff5237fe700 1 -- 172.21.15.5:0/918807062 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff528010760 con 0x561c36922b50 2023-07-21T19:13:21.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff5237fe700 1 -- 172.21.15.5:0/918807062 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52809ee80 con 0x561c36922b50 2023-07-21T19:13:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 -- 172.21.15.5:0/918807062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 msgr2=0x561c36924f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36924f30 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff5280ca210 tx=0x7ff5280994f0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 -- 172.21.15.5:0/918807062 shutdown_connections 2023-07-21T19:13:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36924f30 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917350 0x561c36917720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/918807062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917c60 0x561c36922610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.351+0000 7ff552e8eb80 1 -- 172.21.15.5:0/918807062 >> 172.21.15.5:0/918807062 conn(0x561c3690de20 msgr2=0x561c36913720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.352+0000 7ff552e8eb80 1 -- 172.21.15.5:0/918807062 shutdown_connections 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.352+0000 7ff552e8eb80 1 -- 172.21.15.5:0/918807062 wait complete. 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.352+0000 7ff552e8eb80 1 -- start start 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.352+0000 7ff552e8eb80 1 Processor -- start 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c3691fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36920330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36809b40 con 0x561c36917350 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36809840 con 0x561c36922b50 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c366ed9a0 con 0x561c36917c60 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff552e8eb80 10 client.0 _ll_register_callbacks cb 0x561c368077e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff53d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff53d39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54660/0 (socket says 172.21.15.5:54660) 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.353+0000 7ff53d39b700 1 -- 172.21.15.5:0/1827005876 learned_addr learned my addr 172.21.15.5:0/1827005876 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53db9c700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36920330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c3691fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 -- 172.21.15.5:0/1827005876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 msgr2=0x561c3691fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c3691fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 -- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 msgr2=0x561c36920330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36920330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 -- 172.21.15.5:0/1827005876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff53810d170 con 0x561c36917350 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff53d39b700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff534005820 tx=0x7ff534005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=878d256468f1933 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.354+0000 7ff520ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53401d070 con 0x561c36917350 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c367c3570 con 0x561c36917350 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff53400bc60 con 0x561c36917350 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff53400be20 con 0x561c36917350 2023-07-21T19:13:21.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff552e8eb80 10 client.16884 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff5381a4b80 con 0x561c36917350 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561c367c3570 con 0x561c36917350 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.355+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5340038b0 con 0x561c36917350 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 10 client.16884 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 10 client.16884 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:21.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 20 client.16884 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:21.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36932360 0x561c36934770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 4 client.16884 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:21.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c36862f30 con 0x561c36932360 2023-07-21T19:13:21.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.356+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36932360 0x561c36934770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.357+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36932360 0x561c36934770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70cfa700de7d8540 server_cookie=25bdf9d71fc3e729 in_seq=0 out_seq=0 2023-07-21T19:13:21.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.357+0000 7ff520ff9700 10 client.16884 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.357+0000 7ff520ff9700 1 -- 172.21.15.5:0/1827005876 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c36862f30 con 0x561c36932360 2023-07-21T19:13:21.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.357+0000 7ff520ff9700 10 client.16884 handle_command_reply: tid=0 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.357+0000 7ff552e8eb80 1 client.16884 shutdown 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 msgr2=0x561c3691f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7ff534005820 tx=0x7ff534005c40 comp rx=0 tx=0).stop 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 shutdown_connections 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36922b50 0x561c36920330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36932360 0x561c36934770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36917350 0x561c3691f8b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/1827005876 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c3691fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 >> 172.21.15.5:0/1827005876 conn(0x561c3690de20 msgr2=0x561c36913310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff520ff9700 0 client.16884 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 shutdown_connections 2023-07-21T19:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.358+0000 7ff552e8eb80 1 -- 172.21.15.5:0/1827005876 wait complete. 2023-07-21T19:13:21.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.359+0000 7ff552e8eb80 20 client.16884 trim_cache size 0 max 16384 2023-07-21T19:13:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 Processor -- start 2023-07-21T19:13:21.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 -- start start 2023-07-21T19:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c3692df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36920160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3691b4e0 con 0x561c36922b50 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c3691a700 con 0x561c36917350 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.385+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36918310 con 0x561c36917c60 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54666/0 (socket says 172.21.15.5:54666) 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 -- 172.21.15.5:0/66884891 learned_addr learned my addr 172.21.15.5:0/66884891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53d39b700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c3692df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36920160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 -- 172.21.15.5:0/66884891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 msgr2=0x561c36920160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36920160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 -- 172.21.15.5:0/66884891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 msgr2=0x561c3692df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c3692df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53db9c700 1 -- 172.21.15.5:0/66884891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c36680ce0 con 0x561c36922b50 2023-07-21T19:13:21.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.386+0000 7ff53d39b700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c3692df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.387+0000 7ff53db9c700 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff530007980 tx=0x7ff53000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8de052894d169552 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.387+0000 7ff523fff700 1 -- 172.21.15.5:0/66884891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff530021050 con 0x561c36922b50 2023-07-21T19:13:21.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.387+0000 7ff523fff700 1 -- 172.21.15.5:0/66884891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff530022040 con 0x561c36922b50 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.387+0000 7ff523fff700 1 -- 172.21.15.5:0/66884891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53001bd00 con 0x561c36922b50 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- 172.21.15.5:0/66884891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 msgr2=0x561c36919950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7ff530007980 tx=0x7ff53000b040 comp rx=0 tx=0).stop 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- 172.21.15.5:0/66884891 shutdown_connections 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36919950 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c3692df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/66884891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36920160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- 172.21.15.5:0/66884891 >> 172.21.15.5:0/66884891 conn(0x561c3690de20 msgr2=0x561c36919240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- 172.21.15.5:0/66884891 shutdown_connections 2023-07-21T19:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- 172.21.15.5:0/66884891 wait complete. 2023-07-21T19:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.388+0000 7ff552e8eb80 1 -- start start 2023-07-21T19:13:21.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.389+0000 7ff552e8eb80 1 Processor -- start 2023-07-21T19:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c369254f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36925a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36986230 con 0x561c36922b50 2023-07-21T19:13:21.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c36986370 con 0x561c36917350 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c369864b0 con 0x561c36917c60 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff552e8eb80 10 client.0 _ll_register_callbacks cb 0x7ff534005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53db9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54670/0 (socket says 172.21.15.5:54670) 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c369254f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53db9c700 1 -- 172.21.15.5:0/4163644117 learned_addr learned my addr 172.21.15.5:0/4163644117 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36925a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 -- 172.21.15.5:0/4163644117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 msgr2=0x561c36925a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36925a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 -- 172.21.15.5:0/4163644117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 msgr2=0x561c36925f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53d39b700 1 -- 172.21.15.5:0/4163644117 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5381a4b80 con 0x561c36917350 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.390+0000 7ff53db9c700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff53d39b700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c369254f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff5340098e0 tx=0x7ff534009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a319b3f0cc84c982 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff520ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53401d040 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff528010760 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff534000e40 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff534003a20 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff552e8eb80 10 client.26573 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.391+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7ff530011040 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff528010760 con 0x561c36917350 2023-07-21T19:13:21.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff534004850 con 0x561c36917350 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 10 client.26573 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 10 client.26573 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 20 client.26573 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36987700 0x561c36989b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 4 client.26573 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:13:21.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x561c368250d0 con 0x561c36987700 2023-07-21T19:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.392+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36987700 0x561c36989b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.393+0000 7ff53cb9a700 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36987700 0x561c36989b10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7f6b27acaa70063 server_cookie=d7d28c062086f99a in_seq=0 out_seq=0 2023-07-21T19:13:21.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.393+0000 7ff520ff9700 10 client.26573 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff520ff9700 1 -- 172.21.15.5:0/4163644117 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x561c368250d0 con 0x561c36987700 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff520ff9700 10 client.26573 handle_command_reply: tid=0 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff552e8eb80 1 client.26573 shutdown 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 msgr2=0x561c369254f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c369254f0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff5340098e0 tx=0x7ff534009580 comp rx=0 tx=0).stop 2023-07-21T19:13:21.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.394+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 shutdown_connections 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561c36987700 0x561c36989b10 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561c36922b50 0x561c36925f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561c36917350 0x561c369254f0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 --2- 172.21.15.5:0/4163644117 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561c36917c60 0x561c36925a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff520ff9700 0 client.26573 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 >> 172.21.15.5:0/4163644117 conn(0x561c3690de20 msgr2=0x561c36919ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 shutdown_connections 2023-07-21T19:13:21.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 1 -- 172.21.15.5:0/4163644117 wait complete. 2023-07-21T19:13:21.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff552e8eb80 20 client.26573 trim_cache size 0 max 16384 2023-07-21T19:13:21.404 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:13:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:13:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "af538a69-c7b9-4588-bf48-434c0a1d68b9", 2023-07-21T19:13:21.404 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:13:21.405 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:13:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 msgr2=0x7ff5240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 0x7ff5240a3770 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff530007b50 tx=0x7ff530009100 comp rx=0 tx=0).stop 2023-07-21T19:13:21.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 msgr2=0x7ff538198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.395+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7ff52800d950 tx=0x7ff52800dc60 comp rx=0 tx=0).stop 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 shutdown_connections 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff53810a040 0x7ff53819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff538100d00 0x7ff538197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5240a12c0 0x7ff5240a3770 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 --2- 172.21.15.5:0/303562006 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff538101610 0x7ff538198290 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 >> 172.21.15.5:0/303562006 conn(0x7ff5380fa4b0 msgr2=0x7ff53810c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 shutdown_connections 2023-07-21T19:13:21.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.396+0000 7ff53f5ff700 1 -- 172.21.15.5:0/303562006 wait complete. 2023-07-21T19:13:21.420 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:13:21.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 -- 172.21.15.5:0/1987480533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc105ae0 msgr2=0x7f2fdc105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/1987480533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc105eb0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009990 tx=0x7f2fcc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 -- 172.21.15.5:0/1987480533 shutdown_connections 2023-07-21T19:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/1987480533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc100480 0x7f2fdc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/1987480533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc105eb0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/1987480533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 -- 172.21.15.5:0/1987480533 >> 172.21.15.5:0/1987480533 conn(0x7f2fdc0fb5f0 msgr2=0x7f2fdc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.678+0000 7f2fe1fff700 1 -- 172.21.15.5:0/1987480533 shutdown_connections 2023-07-21T19:13:21.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.679+0000 7f2fe1fff700 1 -- 172.21.15.5:0/1987480533 wait complete. 2023-07-21T19:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 Processor -- start 2023-07-21T19:13:21.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 -- start start 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc100480 0x7f2fdc072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc1a3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc073580 con 0x7f2fdc105ae0 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1a41d0 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.680+0000 7f2fe1fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fdc1a4310 con 0x7f2fdc100480 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc100480 0x7f2fdc072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55114/0 (socket says 172.21.15.5:55114) 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/2634974042 learned_addr learned my addr 172.21.15.5:0/2634974042 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc1a3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/2634974042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc100480 msgr2=0x7f2fdc072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc100480 0x7f2fdc072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 msgr2=0x7f2fdc1a3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc1a3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/2634974042 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0095f0 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc1a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.681+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc000c00 tx=0x7f2fcc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.682+0000 7f2fd8ff9700 1 -- 172.21.15.5:0/2634974042 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc01d070 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.682+0000 7f2fd8ff9700 1 -- 172.21.15.5:0/2634974042 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc022470 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.682+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fdc1a4510 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.682+0000 7f2fd8ff9700 1 -- 172.21.15.5:0/2634974042 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc017650 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.682+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1a4990 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.685+0000 7f2fd8ff9700 1 -- 172.21.15.5:0/2634974042 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2fcc0225e0 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.687+0000 7f2fd8ff9700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 0x7f2fc40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.687+0000 7f2fd8ff9700 1 -- 172.21.15.5:0/2634974042 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2fcc0ce110 con 0x7f2fdc0ffad0 2023-07-21T19:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.687+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 0x7f2fc40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.687+0000 7f2ff585cb80 1 Processor -- start 2023-07-21T19:13:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 -- start start 2023-07-21T19:13:21.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 0x7f2fc40a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc074000 tx=0x7f2fd0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 0x5630bf463590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf463ad0 0x5630bf465eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf35ecb0 con 0x5630bf4582d0 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf3b1a40 con 0x5630bf458be0 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf3aaed0 con 0x5630bf463ad0 2023-07-21T19:13:21.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54688/0 (socket says 172.21.15.5:54688) 2023-07-21T19:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/2361896134 learned_addr learned my addr 172.21.15.5:0/2361896134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 0x5630bf463590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.688+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf463ad0 0x5630bf465eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 -- 172.21.15.5:0/2361896134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf463ad0 msgr2=0x5630bf465eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf463ad0 0x5630bf465eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 -- 172.21.15.5:0/2361896134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 msgr2=0x5630bf4586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 -- 172.21.15.5:0/2361896134 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630bf1d8de0 con 0x5630bf458be0 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 0x5630bf463590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0011c90 tx=0x7f2fd0014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30bfc558396f5fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fc1ffb700 1 -- 172.21.15.5:0/2361896134 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0004230 con 0x5630bf458be0 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.689+0000 7f2fc1ffb700 1 -- 172.21.15.5:0/2361896134 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fd0021070 con 0x5630bf458be0 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2fc1ffb700 1 -- 172.21.15.5:0/2361896134 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd0003b20 con 0x5630bf458be0 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 -- 172.21.15.5:0/2361896134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 msgr2=0x5630bf463590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 0x5630bf463590 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0011c90 tx=0x7f2fd0014000 comp rx=0 tx=0).stop 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 -- 172.21.15.5:0/2361896134 shutdown_connections 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf463ad0 0x5630bf465eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf4586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/2361896134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf458be0 0x5630bf463590 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.690+0000 7f2ff585cb80 1 -- 172.21.15.5:0/2361896134 >> 172.21.15.5:0/2361896134 conn(0x5630bf44eda0 msgr2=0x5630bf4546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.691+0000 7f2ff585cb80 1 -- 172.21.15.5:0/2361896134 shutdown_connections 2023-07-21T19:13:21.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.691+0000 7f2ff585cb80 1 -- 172.21.15.5:0/2361896134 wait complete. 2023-07-21T19:13:21.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.691+0000 7f2ff585cb80 1 -- start start 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.691+0000 7f2ff585cb80 1 Processor -- start 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf460750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf34aac0 con 0x5630bf463ad0 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf34a7c0 con 0x5630bf4582d0 2023-07-21T19:13:21.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf22e920 con 0x5630bf458be0 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2ff585cb80 10 client.0 _ll_register_callbacks cb 0x5630bf348760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55126/0 (socket says 172.21.15.5:55126) 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.692+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1597295901 learned_addr learned my addr 172.21.15.5:0/1597295901 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf460750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1597295901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 msgr2=0x5630bf460750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf460750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1597295901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 msgr2=0x5630bf460e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1597295901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fdc1a4990 con 0x5630bf4582d0 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf460750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009350 tx=0x7f2fcc09aea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7b78b6a87699c84 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09a930 con 0x5630bf4582d0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2fcc0228c0 con 0x5630bf4582d0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.693+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5630bf3044f0 con 0x5630bf4582d0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc09b240 con 0x5630bf4582d0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 10 client.26585 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7f2fd0017070 con 0x5630bf4582d0 2023-07-21T19:13:21.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc0c9020 con 0x5630bf4582d0 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fcc0b3b90 con 0x5630bf4582d0 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 10 client.26585 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 10 client.26585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 20 client.26585 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf47b8e0 0x5630bf47dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 4 client.26585 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.694+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5630bf3a3eb0 con 0x5630bf47b8e0 2023-07-21T19:13:21.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.695+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf47b8e0 0x5630bf47dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.695+0000 7f2fbaffd700 10 client.26585 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.695+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf47b8e0 0x5630bf47dcf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=867a0962521631cf server_cookie=c81da8fa060e2406 in_seq=0 out_seq=0 2023-07-21T19:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2fbaffd700 1 -- 172.21.15.5:0/1597295901 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5630bf3a3eb0 con 0x5630bf47b8e0 2023-07-21T19:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2fbaffd700 10 client.26585 handle_command_reply: tid=0 2023-07-21T19:13:21.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 client.26585 shutdown 2023-07-21T19:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 msgr2=0x5630bf463540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc009350 tx=0x7f2fcc09aea0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 shutdown_connections 2023-07-21T19:13:21.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf47b8e0 0x5630bf47dcf0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1597295901 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf460750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 >> 172.21.15.5:0/1597295901 conn(0x5630bf44eda0 msgr2=0x5630bf454230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2fbaffd700 0 client.26585 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.696+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 shutdown_connections 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.697+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1597295901 wait complete. 2023-07-21T19:13:21.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.697+0000 7f2ff585cb80 20 client.26585 trim_cache size 0 max 16384 2023-07-21T19:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 Processor -- start 2023-07-21T19:13:21.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 -- start start 2023-07-21T19:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf45fff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf459540 con 0x5630bf463ad0 2023-07-21T19:13:21.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf459dd0 con 0x5630bf4582d0 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.720+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf44bef0 con 0x5630bf458be0 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf45fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf45fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55420/0 (socket says 172.21.15.5:55420) 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55136/0 (socket says 172.21.15.5:55136) 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdaffd700 1 -- 172.21.15.5:0/1345771407 learned_addr learned my addr 172.21.15.5:0/1345771407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1345771407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 msgr2=0x5630bf45fff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf45fff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1345771407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 msgr2=0x5630bf460530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/1345771407 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5630bf4505e0 con 0x5630bf4582d0 2023-07-21T19:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc099590 tx=0x7f2fcc0092b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebacbfca06179245 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.721+0000 7f2fc27fc700 1 -- 172.21.15.5:0/1345771407 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a3440 con 0x5630bf4582d0 2023-07-21T19:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2fc27fc700 1 -- 172.21.15.5:0/1345771407 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc0c9020 con 0x5630bf4582d0 2023-07-21T19:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2fc27fc700 1 -- 172.21.15.5:0/1345771407 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09f270 con 0x5630bf4582d0 2023-07-21T19:13:21.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1345771407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 msgr2=0x5630bf463070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc099590 tx=0x7f2fcc0092b0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1345771407 shutdown_connections 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf463ad0 0x5630bf460530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf4582d0 0x5630bf463070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/1345771407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf45fff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.722+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1345771407 >> 172.21.15.5:0/1345771407 conn(0x5630bf44eda0 msgr2=0x5630bf45ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.723+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1345771407 shutdown_connections 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.723+0000 7f2ff585cb80 1 -- 172.21.15.5:0/1345771407 wait complete. 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.723+0000 7f2ff585cb80 1 -- start start 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.723+0000 7f2ff585cb80 1 Processor -- start 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf470ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf463ad0 0x5630bf4ce7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf4cece0 con 0x5630bf4582d0 2023-07-21T19:13:21.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf4cee20 con 0x5630bf463ad0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5630bf4cef60 con 0x5630bf458be0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2ff585cb80 10 client.0 _ll_register_callbacks cb 0x5630bf348760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54702/0 (socket says 172.21.15.5:54702) 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/3941144462 learned_addr learned my addr 172.21.15.5:0/3941144462 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf470ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdbfff700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf463ad0 0x5630bf4ce7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/3941144462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 msgr2=0x5630bf470ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf470ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf463ad0 msgr2=0x5630bf4ce7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.724+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf463ad0 0x5630bf4ce7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fdb7fe700 1 -- 172.21.15.5:0/3941144462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd0017070 con 0x5630bf4582d0 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fdb7fe700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc097740 tx=0x7f2fcc096ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bfe1b3d1f7b47c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fbaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:21.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0a3370 con 0x5630bf4582d0 2023-07-21T19:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f2fcc0ca080 con 0x5630bf4582d0 2023-07-21T19:13:21.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fd0021070 con 0x5630bf4582d0 2023-07-21T19:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc0228c0 con 0x5630bf4582d0 2023-07-21T19:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2ff585cb80 10 client.16908 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.725+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x5630bf1c1c60 con 0x5630bf4582d0 2023-07-21T19:13:21.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fd0021070 con 0x5630bf4582d0 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2fcc09a7c0 con 0x5630bf4582d0 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 10 client.16908 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 10 client.16908 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 20 client.16908 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf4d0170 0x5630bf4d2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 4 client.16908 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:13:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5630bf366050 con 0x5630bf4d0170 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.726+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf4d0170 0x5630bf4d2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2fdaffd700 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf4d0170 0x5630bf4d2550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=795c13b2b2b40f79 server_cookie=d42dfb835d0268a5 in_seq=0 out_seq=0 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2fbaffd700 10 client.16908 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2fbaffd700 1 -- 172.21.15.5:0/3941144462 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5630bf366050 con 0x5630bf4d0170 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2fbaffd700 10 client.16908 handle_command_reply: tid=0 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 client.16908 shutdown 2023-07-21T19:13:21.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 msgr2=0x5630bf470ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc097740 tx=0x7f2fcc096ef0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 shutdown_connections 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5630bf463ad0 0x5630bf4ce7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5630bf4d0170 0x5630bf4d2550 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5630bf4582d0 0x5630bf470ab0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 --2- 172.21.15.5:0/3941144462 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5630bf458be0 0x5630bf470ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2fbaffd700 0 client.16908 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:21.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 >> 172.21.15.5:0/3941144462 conn(0x5630bf44eda0 msgr2=0x5630bf45a1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.727+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 shutdown_connections 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2ff585cb80 1 -- 172.21.15.5:0/3941144462 wait complete. 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2ff585cb80 20 client.16908 trim_cache size 0 max 16384 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:13:21.738 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "af538a69-c7b9-4588-bf48-434c0a1d68b9": { 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "af538a69-c7b9-4588-bf48-434c0a1d68b9", 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:13:21.739 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 msgr2=0x7f2fc40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 0x7f2fc40a37f0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2fdc074000 tx=0x7f2fd0009260 comp rx=0 tx=0).stop 2023-07-21T19:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 msgr2=0x7f2fdc072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:21.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc000c00 tx=0x7f2fcc004730 comp rx=0 tx=0).stop 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 shutdown_connections 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc40a1340 0x7f2fc40a37f0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fdc105ae0 0x7f2fdc1a3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fdc0ffad0 0x7f2fdc072810 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 --2- 172.21.15.5:0/2634974042 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fdc100480 0x7f2fdc072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.728+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 >> 172.21.15.5:0/2634974042 conn(0x7f2fdc0fb5f0 msgr2=0x7f2fdc1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.729+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 shutdown_connections 2023-07-21T19:13:21.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:21.729+0000 7f2fe1fff700 1 -- 172.21.15.5:0/2634974042 wait complete. 2023-07-21T19:13:21.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:21 smithi005 ceph-mon[126460]: pgmap v175: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:13:21.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:21 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:13:21.750 INFO:tasks.cephfs.filesystem:scrub status for tag:af538a69-c7b9-4588-bf48-434c0a1d68b9 - {'path': '/', 'tag': 'af538a69-c7b9-4588-bf48-434c0a1d68b9', 'options': 'recursive,force'} 2023-07-21T19:13:21.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:21 smithi101 ceph-mon[128136]: pgmap v175: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:13:21.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:21 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:13:21.835 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:13:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:13:21.411+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...7,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v2347 ap=2 f(v6 m2023-07-21T19:12:39.709618+0000 3=1+2) n(v3 rc2023-07-21T19:12:39.709618+0000 b8 4=1+3) old_inodes=5 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@74} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v2347>,<0x10000000005/data v1452>,<0x10000000003/postgres v386>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v399>,<0x10000000001/sv_0 v419>,<0x10000000000/_nogroup v402>,<0x1/volumes v386>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:12:39.694533+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:12:39.709618+0000 b8 4=1+3)","memory_value.dirstat":"f(v6 m2023-07-21T19:12:39.709618+0000 3=1+2)","memory_value.rstat":"n(v3 rc2023-07-21T19:12:39.709618+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:13:21.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:21 smithi156 ceph-mon[128316]: pgmap v175: 97 pgs: 97 active+clean; 4.7 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 22 MiB/s wr, 17 op/s 2023-07-21T19:13:21.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:21 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:13:22.308 INFO:teuthology.orchestra.run.smithi005.stderr:20700000 of 50000000 tuples (41%) done (elapsed 319.87 s, remaining 452.76 s) 2023-07-21T19:13:22.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:13:22.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:13:22.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:13:22.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:13:22.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:13:22.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:13:22.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:13:22.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.912 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:13:22.913 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:13:22.924 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:13:23.682 INFO:teuthology.orchestra.run.smithi005.stderr:20800000 of 50000000 tuples (41%) done (elapsed 321.24 s, remaining 450.98 s) 2023-07-21T19:13:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 -- 172.21.15.5:0/2647134335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd700685b0 msgr2=0x7fdd70104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 --2- 172.21.15.5:0/2647134335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd700685b0 0x7fdd70104770 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fdd640099c0 tx=0x7fdd64009cd0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 -- 172.21.15.5:0/2647134335 shutdown_connections 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 --2- 172.21.15.5:0/2647134335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70104e50 0x7fdd7010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 --2- 172.21.15.5:0/2647134335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd70067c10 0x7fdd70067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 --2- 172.21.15.5:0/2647134335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd700685b0 0x7fdd70104770 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 -- 172.21.15.5:0/2647134335 >> 172.21.15.5:0/2647134335 conn(0x7fdd70074760 msgr2=0x7fdd70074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 -- 172.21.15.5:0/2647134335 shutdown_connections 2023-07-21T19:13:23.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.734+0000 7fdd753ff700 1 -- 172.21.15.5:0/2647134335 wait complete. 2023-07-21T19:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.736+0000 7fdd753ff700 1 Processor -- start 2023-07-21T19:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.736+0000 7fdd753ff700 1 -- start start 2023-07-21T19:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70067c10 0x7fdd7019a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd70104e50 0x7fdd70193750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70193f20 con 0x7fdd700685b0 2023-07-21T19:13:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70194090 con 0x7fdd70104e50 2023-07-21T19:13:23.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.737+0000 7fdd753ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd70194250 con 0x7fdd70067c10 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54712/0 (socket says 172.21.15.5:54712) 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1101091131 learned_addr learned my addr 172.21.15.5:0/1101091131 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70067c10 0x7fdd7019a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6f7fe700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd70104e50 0x7fdd70193750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1101091131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70067c10 msgr2=0x7fdd7019a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70067c10 0x7fdd7019a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1101091131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd70104e50 msgr2=0x7fdd70193750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd70104e50 0x7fdd70193750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.738+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1101091131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd640095f0 con 0x7fdd700685b0 2023-07-21T19:13:23.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.739+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000c00 tx=0x7fdd6400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.739+0000 7fdd57fff700 1 -- 172.21.15.5:0/1101091131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6401d070 con 0x7fdd700685b0 2023-07-21T19:13:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.739+0000 7fdd57fff700 1 -- 172.21.15.5:0/1101091131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd64005050 con 0x7fdd700685b0 2023-07-21T19:13:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.739+0000 7fdd57fff700 1 -- 172.21.15.5:0/1101091131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd64004b00 con 0x7fdd700685b0 2023-07-21T19:13:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.739+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd701944d0 con 0x7fdd700685b0 2023-07-21T19:13:23.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.740+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd701986a0 con 0x7fdd700685b0 2023-07-21T19:13:23.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.745+0000 7fdd57fff700 1 -- 172.21.15.5:0/1101091131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdd64005430 con 0x7fdd700685b0 2023-07-21T19:13:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.746+0000 7fdd88d01b80 1 Processor -- start 2023-07-21T19:13:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.746+0000 7fdd88d01b80 1 -- start start 2023-07-21T19:13:23.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1aacd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aa79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd19acc50 con 0x55cdd19036b0 2023-07-21T19:13:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd19ff9e0 con 0x55cdd1aad2d0 2023-07-21T19:13:23.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd19f8e70 con 0x55cdd1aa7580 2023-07-21T19:13:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1aacd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aa79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55170/0 (socket says 172.21.15.5:55170) 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.747+0000 7fdd6effd700 1 -- 172.21.15.5:0/1617397872 learned_addr learned my addr 172.21.15.5:0/1617397872 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd57fff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 0x7fdd580a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 -- 172.21.15.5:0/1617397872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 msgr2=0x55cdd1aa79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aa79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 -- 172.21.15.5:0/1617397872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 msgr2=0x55cdd1aacd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1aacd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 -- 172.21.15.5:0/1617397872 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdd1826d80 con 0x55cdd1aad2d0 2023-07-21T19:13:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd57fff700 1 -- 172.21.15.5:0/1101091131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdd640cd9d0 con 0x7fdd700685b0 2023-07-21T19:13:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000bd70 tx=0x7fdd600106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97660364f73c1109 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.753 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd6f7fe700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 0x7fdd580a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.748+0000 7fdd557fa700 1 -- 172.21.15.5:0/1617397872 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60010e10 con 0x55cdd1aad2d0 2023-07-21T19:13:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.749+0000 7fdd557fa700 1 -- 172.21.15.5:0/1617397872 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd60015070 con 0x55cdd1aad2d0 2023-07-21T19:13:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.749+0000 7fdd557fa700 1 -- 172.21.15.5:0/1617397872 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd600116a0 con 0x55cdd1aad2d0 2023-07-21T19:13:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.749+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1617397872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 msgr2=0x55cdd1aa7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.749+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdd6000bd70 tx=0x7fdd600106b0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.749+0000 7fdd6f7fe700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 0x7fdd580a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd680075c0 tx=0x7fdd68009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1617397872 shutdown_connections 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aa79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1aacd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1617397872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aad2d0 0x55cdd1aa7040 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1617397872 >> 172.21.15.5:0/1617397872 conn(0x55cdd1a9cbe0 msgr2=0x55cdd1aa1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1617397872 shutdown_connections 2023-07-21T19:13:23.755 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1617397872 wait complete. 2023-07-21T19:13:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.750+0000 7fdd88d01b80 1 -- start start 2023-07-21T19:13:23.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.751+0000 7fdd88d01b80 1 Processor -- start 2023-07-21T19:13:23.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.751+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.751+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.751+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab1380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1998a60 con 0x55cdd19036b0 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1998760 con 0x55cdd1aa7580 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd187c8c0 con 0x55cdd1aad2d0 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd88d01b80 10 client.0 _ll_register_callbacks cb 0x55cdd1996700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55172/0 (socket says 172.21.15.5:55172) 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1409799775 learned_addr learned my addr 172.21.15.5:0/1409799775 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6f7fe700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1409799775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 msgr2=0x55cdd1ab1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1409799775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 msgr2=0x55cdd1ab0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.752+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1409799775 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd6000b9a0 con 0x55cdd1aa7580 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd640cb1b0 tx=0x7fdd640d11d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67cea3803f6775b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6409a6a0 con 0x55cdd1aa7580 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cdd1952490 con 0x55cdd1aa7580 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdd640cde40 con 0x55cdd1aa7580 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6409a4c0 con 0x55cdd1aa7580 2023-07-21T19:13:23.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd88d01b80 10 client.26603 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cdd180fc00 con 0x55cdd1aa7580 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.753+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cdd1952490 con 0x55cdd1aa7580 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd64004c60 con 0x55cdd1aa7580 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 10 client.26603 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 10 client.26603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 20 client.26603 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1ac07e0 0x55cdd1ac2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 4 client.26603 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cdd19f1e50 con 0x55cdd1ac07e0 2023-07-21T19:13:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.754+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1ac07e0 0x55cdd1ac2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.755+0000 7fdd6effd700 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1ac07e0 0x55cdd1ac2bf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3312f1157518c7 server_cookie=5ee0619c3ed0b4a8 in_seq=0 out_seq=0 2023-07-21T19:13:23.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.755+0000 7fdd52ff5700 10 client.26603 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.755+0000 7fdd52ff5700 1 -- 172.21.15.5:0/1409799775 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cdd19f1e50 con 0x55cdd1ac07e0 2023-07-21T19:13:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.755+0000 7fdd52ff5700 10 client.26603 handle_command_reply: tid=0 2023-07-21T19:13:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.755+0000 7fdd88d01b80 1 client.26603 shutdown 2023-07-21T19:13:23.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 msgr2=0x55cdd1ab0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdd640cb1b0 tx=0x7fdd640d11d0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 shutdown_connections 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1ac07e0 0x55cdd1ac2bf0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab0900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd52ff5700 0 client.26603 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:23.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1409799775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab0e40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 >> 172.21.15.5:0/1409799775 conn(0x55cdd1a9cbe0 msgr2=0x55cdd1aa0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 shutdown_connections 2023-07-21T19:13:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1409799775 wait complete. 2023-07-21T19:13:23.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.756+0000 7fdd88d01b80 20 client.26603 trim_cache size 0 max 16384 2023-07-21T19:13:23.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.772+0000 7fdd88d01b80 1 Processor -- start 2023-07-21T19:13:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 -- start start 2023-07-21T19:13:23.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd19036b0 0x55cdd1aa7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1a9ed00 con 0x55cdd1aad2d0 2023-07-21T19:13:23.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1a9e5f0 con 0x55cdd19036b0 2023-07-21T19:13:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1aa2b20 con 0x55cdd1aa7580 2023-07-21T19:13:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd19036b0 0x55cdd1aa7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55472/0 (socket says 172.21.15.5:55472) 2023-07-21T19:13:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.773+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1520739926 learned_addr learned my addr 172.21.15.5:0/1520739926 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1520739926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd19036b0 msgr2=0x55cdd1aa7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd19036b0 0x55cdd1aa7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1520739926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd1aad2d0 msgr2=0x55cdd1ab11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 -- 172.21.15.5:0/1520739926 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cdd1abd2a0 con 0x55cdd1aa7580 2023-07-21T19:13:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd64097060 tx=0x7fdd6400fe60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8276a1d26b1b8e11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd55ffb700 1 -- 172.21.15.5:0/1520739926 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd640d3f10 con 0x55cdd1aa7580 2023-07-21T19:13:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd55ffb700 1 -- 172.21.15.5:0/1520739926 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd640d1240 con 0x55cdd1aa7580 2023-07-21T19:13:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.774+0000 7fdd55ffb700 1 -- 172.21.15.5:0/1520739926 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6409ab60 con 0x55cdd1aa7580 2023-07-21T19:13:23.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1520739926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 msgr2=0x55cdd1aac330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fdd64097060 tx=0x7fdd6400fe60 comp rx=0 tx=0).stop 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1520739926 shutdown_connections 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd1aad2d0 0x55cdd1ab11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd19036b0 0x55cdd1aa7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/1520739926 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aa7580 0x55cdd1aac330 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1520739926 >> 172.21.15.5:0/1520739926 conn(0x55cdd1a9cbe0 msgr2=0x55cdd1aaee60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1520739926 shutdown_connections 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.775+0000 7fdd88d01b80 1 -- 172.21.15.5:0/1520739926 wait complete. 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.776+0000 7fdd88d01b80 1 -- start start 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.776+0000 7fdd88d01b80 1 Processor -- start 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.776+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.776+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab6190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd88d01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1b136a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1ab6ab0 con 0x55cdd19036b0 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1b13be0 con 0x55cdd1aa7580 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd88d01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cdd1b13d20 con 0x55cdd1aad2d0 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd88d01b80 10 client.0 _ll_register_callbacks cb 0x7fdd640056e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1b136a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54732/0 (socket says 172.21.15.5:54732) 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 -- 172.21.15.5:0/389996941 learned_addr learned my addr 172.21.15.5:0/389996941 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 -- 172.21.15.5:0/389996941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 msgr2=0x55cdd1b136a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1b136a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 -- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 msgr2=0x55cdd1ab6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6effd700 1 -- 172.21.15.5:0/389996941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cdd180fc00 con 0x55cdd19036b0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.777+0000 7fdd6f7fe700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1b136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd6effd700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55cdd1aa3b40 tx=0x7fdd60005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54788d559b2cf875 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd52ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd60020040 con 0x55cdd19036b0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd60015070 con 0x55cdd19036b0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fdd60017070 con 0x55cdd19036b0 2023-07-21T19:13:23.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd6001eed0 con 0x55cdd19036b0 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd88d01b80 10 client.16926 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.778+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55cdd184d1f0 con 0x55cdd19036b0 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd60015070 con 0x55cdd19036b0 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdd60005320 con 0x55cdd19036b0 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 10 client.16926 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 20 client.16926 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:23.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1b15420 0x55cdd1b17830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 4 client.16926 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:13:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cdd19b3ff0 con 0x55cdd1b15420 2023-07-21T19:13:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.779+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1b15420 0x55cdd1b17830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.780+0000 7fdd52ff5700 10 client.16926 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:23.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.780+0000 7fdd6e7fc700 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1b15420 0x55cdd1b17830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34102dbed416aeea server_cookie=cf9c6dbd8f6316b9 in_seq=0 out_seq=0 2023-07-21T19:13:23.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd52ff5700 1 -- 172.21.15.5:0/389996941 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cdd19b3ff0 con 0x55cdd1b15420 2023-07-21T19:13:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd52ff5700 10 client.16926 handle_command_reply: tid=0 2023-07-21T19:13:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 client.16926 shutdown 2023-07-21T19:13:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 msgr2=0x55cdd1ab5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x55cdd1aa3b40 tx=0x7fdd60005950 comp rx=0 tx=0).stop 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 shutdown_connections 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cdd1aad2d0 0x55cdd1b136a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cdd1b15420 0x55cdd1b17830 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cdd19036b0 0x55cdd1ab5c50 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 --2- 172.21.15.5:0/389996941 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cdd1aa7580 0x55cdd1ab6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd52ff5700 0 client.16926 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 >> 172.21.15.5:0/389996941 conn(0x55cdd1a9cbe0 msgr2=0x55cdd1aaf6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 shutdown_connections 2023-07-21T19:13:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 1 -- 172.21.15.5:0/389996941 wait complete. 2023-07-21T19:13:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.791+0000 7fdd88d01b80 20 client.16926 trim_cache size 0 max 16384 2023-07-21T19:13:23.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:13:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:13:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:13:23.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 msgr2=0x7fdd580a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 0x7fdd580a37d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdd680075c0 tx=0x7fdd68009040 comp rx=0 tx=0).stop 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 msgr2=0x7fdd7019aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fdd64000c00 tx=0x7fdd6400f690 comp rx=0 tx=0).stop 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 shutdown_connections 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd70104e50 0x7fdd70193750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd70067c10 0x7fdd7019a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdd580a1320 0x7fdd580a37d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 --2- 172.21.15.5:0/1101091131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd700685b0 0x7fdd7019aeb0 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 >> 172.21.15.5:0/1101091131 conn(0x7fdd70074760 msgr2=0x7fdd700fde60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 shutdown_connections 2023-07-21T19:13:23.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:23.792+0000 7fdd753ff700 1 -- 172.21.15.5:0/1101091131 wait complete. 2023-07-21T19:13:23.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:23 smithi101 ceph-mon[128136]: pgmap v176: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 97 op/s 2023-07-21T19:13:23.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:23 smithi005 ceph-mon[126460]: pgmap v176: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 97 op/s 2023-07-21T19:13:23.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:23 smithi156 ceph-mon[128316]: pgmap v176: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 30 MiB/s wr, 97 op/s 2023-07-21T19:13:24.969 INFO:teuthology.orchestra.run.smithi005.stderr:20900000 of 50000000 tuples (41%) done (elapsed 322.53 s, remaining 449.08 s) 2023-07-21T19:13:25.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:25 smithi101 ceph-mon[128136]: pgmap v177: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 23 MiB/s wr, 102 op/s 2023-07-21T19:13:25.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:25 smithi005 ceph-mon[126460]: pgmap v177: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 23 MiB/s wr, 102 op/s 2023-07-21T19:13:25.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:25 smithi156 ceph-mon[128316]: pgmap v177: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 23 MiB/s wr, 102 op/s 2023-07-21T19:13:26.258 INFO:teuthology.orchestra.run.smithi005.stderr:21000000 of 50000000 tuples (42%) done (elapsed 323.82 s, remaining 447.18 s) 2023-07-21T19:13:27.542 INFO:teuthology.orchestra.run.smithi005.stderr:21100000 of 50000000 tuples (42%) done (elapsed 325.10 s, remaining 445.29 s) 2023-07-21T19:13:27.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:27 smithi101 ceph-mon[128136]: pgmap v178: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:13:27.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:27 smithi005 ceph-mon[126460]: pgmap v178: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:13:27.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:27 smithi156 ceph-mon[128316]: pgmap v178: 97 pgs: 97 active+clean; 4.8 GiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 99 op/s 2023-07-21T19:13:28.950 INFO:teuthology.orchestra.run.smithi005.stderr:21200000 of 50000000 tuples (42%) done (elapsed 326.51 s, remaining 443.56 s) 2023-07-21T19:13:29.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:29 smithi101 ceph-mon[128136]: pgmap v179: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 33 MiB/s wr, 110 op/s 2023-07-21T19:13:29.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:29 smithi005 ceph-mon[126460]: pgmap v179: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 33 MiB/s wr, 110 op/s 2023-07-21T19:13:29.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:29 smithi156 ceph-mon[128316]: pgmap v179: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 33 MiB/s wr, 110 op/s 2023-07-21T19:13:30.252 INFO:teuthology.orchestra.run.smithi005.stderr:21300000 of 50000000 tuples (42%) done (elapsed 327.81 s, remaining 441.70 s) 2023-07-21T19:13:31.542 INFO:teuthology.orchestra.run.smithi005.stderr:21400000 of 50000000 tuples (42%) done (elapsed 329.10 s, remaining 439.83 s) 2023-07-21T19:13:31.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:31 smithi101 ceph-mon[128136]: pgmap v180: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:13:31.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:31 smithi005 ceph-mon[126460]: pgmap v180: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:13:31.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:31 smithi156 ceph-mon[128316]: pgmap v180: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 105 KiB/s rd, 22 MiB/s wr, 102 op/s 2023-07-21T19:13:32.872 INFO:teuthology.orchestra.run.smithi005.stderr:21500000 of 50000000 tuples (43%) done (elapsed 330.43 s, remaining 438.02 s) 2023-07-21T19:13:33.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:33 smithi101 ceph-mon[128136]: pgmap v181: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 32 MiB/s wr, 110 op/s 2023-07-21T19:13:33.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:33 smithi005 ceph-mon[126460]: pgmap v181: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 32 MiB/s wr, 110 op/s 2023-07-21T19:13:33.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:33 smithi156 ceph-mon[128316]: pgmap v181: 97 pgs: 97 active+clean; 4.9 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 32 MiB/s wr, 110 op/s 2023-07-21T19:13:35.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:35 smithi101 ceph-mon[128136]: pgmap v182: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 32 op/s 2023-07-21T19:13:35.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:35 smithi005 ceph-mon[126460]: pgmap v182: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 32 op/s 2023-07-21T19:13:35.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:35 smithi156 ceph-mon[128316]: pgmap v182: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 26 MiB/s wr, 32 op/s 2023-07-21T19:13:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.953+0000 7fef6af9d700 1 -- 172.21.15.5:0/1422741797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef640ff310 msgr2=0x7fef640ff6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.953+0000 7fef6af9d700 1 --2- 172.21.15.5:0/1422741797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef640ff310 0x7fef640ff6e0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fef600098f0 tx=0x7fef60009c00 comp rx=0 tx=0).stop 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 -- 172.21.15.5:0/1422741797 shutdown_connections 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 --2- 172.21.15.5:0/1422741797 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef6410c540 0x7fef6410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 --2- 172.21.15.5:0/1422741797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef640ff310 0x7fef640ff6e0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 --2- 172.21.15.5:0/1422741797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ffc20 0x7fef6410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 -- 172.21.15.5:0/1422741797 >> 172.21.15.5:0/1422741797 conn(0x7fef64074770 msgr2=0x7fef64074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 -- 172.21.15.5:0/1422741797 shutdown_connections 2023-07-21T19:13:36.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.954+0000 7fef6af9d700 1 -- 172.21.15.5:0/1422741797 wait complete. 2023-07-21T19:13:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.955+0000 7fef6af9d700 1 Processor -- start 2023-07-21T19:13:36.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.955+0000 7fef6af9d700 1 -- start start 2023-07-21T19:13:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 0x7fef6419deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 0x7fef64197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef641980b0 con 0x7fef6410c540 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef64198220 con 0x7fef640ffc20 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6af9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef641983e0 con 0x7fef640ff310 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46634/0 (socket says 172.21.15.5:46634) 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef69f9b700 1 -- 172.21.15.5:0/2589324186 learned_addr learned my addr 172.21.15.5:0/2589324186 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:36.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6979a700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 0x7fef6419deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.956+0000 7fef6a79c700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 0x7fef64197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 -- 172.21.15.5:0/2589324186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 msgr2=0x7fef6419deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 0x7fef6419deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 -- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 msgr2=0x7fef64197a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 0x7fef64197a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 -- 172.21.15.5:0/2589324186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef5c009d00 con 0x7fef640ff310 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef6979a700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 0x7fef6419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:36.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef6a79c700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 0x7fef64197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.957+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef60009230 tx=0x7fef6000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.958+0000 7fef5affd700 1 -- 172.21.15.5:0/2589324186 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6001c070 con 0x7fef640ff310 2023-07-21T19:13:36.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.958+0000 7fef5affd700 1 -- 172.21.15.5:0/2589324186 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef600058e0 con 0x7fef640ff310 2023-07-21T19:13:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.958+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef60009520 con 0x7fef640ff310 2023-07-21T19:13:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.958+0000 7fef5affd700 1 -- 172.21.15.5:0/2589324186 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef600036e0 con 0x7fef640ff310 2023-07-21T19:13:36.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.958+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641a4780 con 0x7fef640ff310 2023-07-21T19:13:36.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.962+0000 7fef5affd700 1 -- 172.21.15.5:0/2589324186 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fef60020030 con 0x7fef640ff310 2023-07-21T19:13:36.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.963+0000 7fef7faf5b80 1 Processor -- start 2023-07-21T19:13:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.963+0000 7fef7faf5b80 1 -- start start 2023-07-21T19:13:36.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.963+0000 7fef5affd700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 0x7fef500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef5affd700 1 -- 172.21.15.5:0/2589324186 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fef600cdb40 con 0x7fef640ff310 2023-07-21T19:13:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d77700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25c7cc50 con 0x557d25d7d8c0 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25ccf9e0 con 0x557d25d7d470 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25cc8e70 con 0x557d25bd36b0 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef6a79c700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 0x7fef500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef6979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46648/0 (socket says 172.21.15.5:46648) 2023-07-21T19:13:36.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef69f9b700 1 -- 172.21.15.5:0/32661201 learned_addr learned my addr 172.21.15.5:0/32661201 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef6979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53522/0 (socket says 172.21.15.5:53522) 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.964+0000 7fef6a79c700 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d77700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 -- 172.21.15.5:0/32661201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 msgr2=0x557d25d7cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 -- 172.21.15.5:0/32661201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 msgr2=0x557d25d77700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d77700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 -- 172.21.15.5:0/32661201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d25af6d80 con 0x557d25d7d470 2023-07-21T19:13:36.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef69f9b700 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6a79c700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 0x7fef500a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c00ebb0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef6979a700 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef540114c0 tx=0x7fef540117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daab50b3dfd4e0d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef4ffff700 1 -- 172.21.15.5:0/32661201 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef540112f0 con 0x557d25d7d470 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef4ffff700 1 -- 172.21.15.5:0/32661201 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef540044c0 con 0x557d25d7d470 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.965+0000 7fef4ffff700 1 -- 172.21.15.5:0/32661201 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54010430 con 0x557d25d7d470 2023-07-21T19:13:36.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 -- 172.21.15.5:0/32661201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 msgr2=0x557d25d771a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fef540114c0 tx=0x7fef540117d0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 -- 172.21.15.5:0/32661201 shutdown_connections 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d77700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d7cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/32661201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d470 0x557d25d771a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 -- 172.21.15.5:0/32661201 >> 172.21.15.5:0/32661201 conn(0x557d25d6cd40 msgr2=0x557d25d72340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 -- 172.21.15.5:0/32661201 shutdown_connections 2023-07-21T19:13:36.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.966+0000 7fef7faf5b80 1 -- 172.21.15.5:0/32661201 wait complete. 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.967+0000 7fef7faf5b80 1 -- start start 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.967+0000 7fef7faf5b80 1 Processor -- start 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 0x557d25d80a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d811d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25c68a60 con 0x557d25d7d470 2023-07-21T19:13:36.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25c68760 con 0x557d25d7d8c0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25b4c8c0 con 0x557d25bd36b0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef7faf5b80 10 client.0 _ll_register_callbacks cb 0x557d25c66700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46664/0 (socket says 172.21.15.5:46664) 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef69f9b700 1 -- 172.21.15.5:0/3278896427 learned_addr learned my addr 172.21.15.5:0/3278896427 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef6a79c700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d811d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.968+0000 7fef6979a700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 0x557d25d80a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 -- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 msgr2=0x557d25d811d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d811d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 -- 172.21.15.5:0/3278896427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 msgr2=0x557d25d80a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 0x557d25d80a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 -- 172.21.15.5:0/3278896427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef641a4780 con 0x557d25bd36b0 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef6979a700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 0x557d25d80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef69f9b700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef600098c0 tx=0x7fef6009ad60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51bfa9e7b559edde server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:13:36.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef60097380 con 0x557d25bd36b0 2023-07-21T19:13:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fef600974e0 con 0x557d25bd36b0 2023-07-21T19:13:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557d25c22490 con 0x557d25bd36b0 2023-07-21T19:13:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef6009aed0 con 0x557d25bd36b0 2023-07-21T19:13:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef7faf5b80 10 client.26584 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:36.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.969+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fef54009c90 con 0x557d25bd36b0 2023-07-21T19:13:36.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef600c9080 con 0x557d25bd36b0 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef600a2ba0 con 0x557d25bd36b0 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 10 client.26584 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 10 client.26584 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 20 client.26584 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25d90a30 0x557d25d92e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 4 client.26584 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557d25cc1e50 con 0x557d25d90a30 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.970+0000 7fef6979a700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25d90a30 0x557d25d92e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.971+0000 7fef4d7fa700 10 client.26584 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.971+0000 7fef6979a700 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25d90a30 0x557d25d92e10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eecfd8fd643b5a7 server_cookie=d8407429fe168073 in_seq=0 out_seq=0 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef4d7fa700 1 -- 172.21.15.5:0/3278896427 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557d25cc1e50 con 0x557d25d90a30 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef4d7fa700 10 client.26584 handle_command_reply: tid=0 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 client.26584 shutdown 2023-07-21T19:13:36.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 msgr2=0x557d25d83880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef600098c0 tx=0x7fef6009ad60 comp rx=0 tx=0).stop 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 shutdown_connections 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d811d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25bd36b0 0x557d25d83880 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25d90a30 0x557d25d92e10 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/3278896427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d470 0x557d25d80a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 >> 172.21.15.5:0/3278896427 conn(0x557d25d6cd40 msgr2=0x557d25d71640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:36.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef4d7fa700 0 client.26584 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 shutdown_connections 2023-07-21T19:13:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.972+0000 7fef7faf5b80 1 -- 172.21.15.5:0/3278896427 wait complete. 2023-07-21T19:13:36.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.973+0000 7fef7faf5b80 20 client.26584 trim_cache size 0 max 16384 2023-07-21T19:13:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.989+0000 7fef7faf5b80 1 Processor -- start 2023-07-21T19:13:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.989+0000 7fef7faf5b80 1 -- start start 2023-07-21T19:13:36.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 0x557d25d809b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d7c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d863d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25d781b0 con 0x557d25bd36b0 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25d73b30 con 0x557d25d7d8c0 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25d7cb20 con 0x557d25d7d470 2023-07-21T19:13:36.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 0x557d25d809b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d863d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 0x557d25d809b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35500/0 (socket says 172.21.15.5:35500) 2023-07-21T19:13:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d7c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:36.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef69f9b700 1 -- 172.21.15.5:0/1934157530 learned_addr learned my addr 172.21.15.5:0/1934157530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6979a700 1 -- 172.21.15.5:0/1934157530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 msgr2=0x557d25d863d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6979a700 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d863d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6979a700 1 -- 172.21.15.5:0/1934157530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 msgr2=0x557d25d809b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.990+0000 7fef6979a700 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 0x557d25d809b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef6979a700 1 -- 172.21.15.5:0/1934157530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557d25adfc00 con 0x557d25d7d470 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef6a79c700 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef6979a700 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d7c3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef54009790 tx=0x7fef54011270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c8cda39ac62cb89f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef58ff9700 1 -- 172.21.15.5:0/1934157530 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef5401e070 con 0x557d25d7d470 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef58ff9700 1 -- 172.21.15.5:0/1934157530 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef54005ce0 con 0x557d25d7d470 2023-07-21T19:13:36.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.991+0000 7fef58ff9700 1 -- 172.21.15.5:0/1934157530 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef54005280 con 0x557d25d7d470 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- 172.21.15.5:0/1934157530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 msgr2=0x557d25d7c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d7c3e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fef54009790 tx=0x7fef54011270 comp rx=0 tx=0).stop 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- 172.21.15.5:0/1934157530 shutdown_connections 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25d7d8c0 0x557d25d863d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25bd36b0 0x557d25d809b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/1934157530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d7c3e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- 172.21.15.5:0/1934157530 >> 172.21.15.5:0/1934157530 conn(0x557d25d6cd40 msgr2=0x557d25d7dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:36.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- 172.21.15.5:0/1934157530 shutdown_connections 2023-07-21T19:13:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- 172.21.15.5:0/1934157530 wait complete. 2023-07-21T19:13:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.992+0000 7fef7faf5b80 1 -- start start 2023-07-21T19:13:36.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 Processor -- start 2023-07-21T19:13:37.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d84820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d84f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25d85650 con 0x557d25d7d8c0 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25de48d0 con 0x557d25bd36b0 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557d25de4a10 con 0x557d25d7d470 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.993+0000 7fef7faf5b80 10 client.0 _ll_register_callbacks cb 0x557d25c66700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:37.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef6979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d84820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef6979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d84820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46680/0 (socket says 172.21.15.5:46680) 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef6a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d84f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53544/0 (socket says 172.21.15.5:53544) 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef6979a700 1 -- 172.21.15.5:0/2261569010 learned_addr learned my addr 172.21.15.5:0/2261569010 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 -- 172.21.15.5:0/2261569010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 msgr2=0x557d25d84820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:37.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d84820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 -- 172.21.15.5:0/2261569010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 msgr2=0x557d25d84f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d84f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 -- 172.21.15.5:0/2261569010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef5c0092c0 con 0x557d25bd36b0 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef69f9b700 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef60009810 tx=0x7fef6009ac70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19a54156fa3505d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.994+0000 7fef4d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef6009a360 con 0x557d25bd36b0 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fef540044c0 con 0x557d25bd36b0 2023-07-21T19:13:37.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7fef6009aed0 con 0x557d25bd36b0 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fef600cb020 con 0x557d25bd36b0 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 10 client.26627 fetch_fsmap learned FSMap version 30 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x7fef54021040 con 0x557d25bd36b0 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fef60064080 con 0x557d25bd36b0 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 30) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fef60096a60 con 0x557d25bd36b0 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 10 client.26627 fetch_fsmap finished waiting for FSMap version 30 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 10 client.26627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:37.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 20 client.26627 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25de5c30 0x557d25de8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 4 client.26627 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.995+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557d25c83ff0 con 0x557d25de5c30 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.996+0000 7fef6979a700 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25de5c30 0x557d25de8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.996+0000 7fef6979a700 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25de5c30 0x557d25de8040 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbf18d8d8f80f1e server_cookie=adb1b721a617d38a in_seq=0 out_seq=0 2023-07-21T19:13:37.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:36.996+0000 7fef4d7fa700 10 client.26627 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:37.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef4d7fa700 1 -- 172.21.15.5:0/2261569010 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557d25c83ff0 con 0x557d25de5c30 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef4d7fa700 10 client.26627 handle_command_reply: tid=0 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 client.26627 shutdown 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 msgr2=0x557d25d86350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fef60009810 tx=0x7fef6009ac70 comp rx=0 tx=0).stop 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 shutdown_connections 2023-07-21T19:13:37.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557d25de5c30 0x557d25de8040 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557d25d7d8c0 0x557d25d84f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557d25bd36b0 0x557d25d86350 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 --2- 172.21.15.5:0/2261569010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557d25d7d470 0x557d25d84820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef4d7fa700 0 client.26627 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:37.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.009+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 >> 172.21.15.5:0/2261569010 conn(0x557d25d6cd40 msgr2=0x557d25d7e590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 shutdown_connections 2023-07-21T19:13:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef7faf5b80 1 -- 172.21.15.5:0/2261569010 wait complete. 2023-07-21T19:13:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef7faf5b80 20 client.26627 trim_cache size 0 max 16384 2023-07-21T19:13:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:13:37.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:13:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:13:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:13:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 msgr2=0x7fef500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:37.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 0x7fef500a3770 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fef5c009fd0 tx=0x7fef5c00ebb0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 msgr2=0x7fef6419d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fef60009230 tx=0x7fef6000fa00 comp rx=0 tx=0).stop 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 shutdown_connections 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fef500a12c0 0x7fef500a3770 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fef6410c540 0x7fef64197a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fef640ff310 0x7fef6419d970 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.010+0000 7fef6af9d700 1 --2- 172.21.15.5:0/2589324186 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fef640ffc20 0x7fef6419deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.011+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 >> 172.21.15.5:0/2589324186 conn(0x7fef64074770 msgr2=0x7fef640fc3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.011+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 shutdown_connections 2023-07-21T19:13:37.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:37.011+0000 7fef6af9d700 1 -- 172.21.15.5:0/2589324186 wait complete. 2023-07-21T19:13:37.254 INFO:teuthology.orchestra.run.smithi005.stderr:21600000 of 50000000 tuples (43%) done (elapsed 334.82 s, remaining 440.22 s) 2023-07-21T19:13:37.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:37 smithi101 ceph-mon[128136]: pgmap v183: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:13:37.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:37 smithi005 ceph-mon[126460]: pgmap v183: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:13:37.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:37 smithi156 ceph-mon[128316]: pgmap v183: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 19 op/s 2023-07-21T19:13:39.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:39 smithi101 ceph-mon[128136]: pgmap v184: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:13:39.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:39 smithi005 ceph-mon[126460]: pgmap v184: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:13:39.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:39 smithi156 ceph-mon[128316]: pgmap v184: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:13:41.445 INFO:teuthology.orchestra.run.smithi005.stderr:21700000 of 50000000 tuples (43%) done (elapsed 339.01 s, remaining 442.12 s) 2023-07-21T19:13:41.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:41 smithi101 ceph-mon[128136]: pgmap v185: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:13:41.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:41 smithi005 ceph-mon[126460]: pgmap v185: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:13:41.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:41 smithi156 ceph-mon[128316]: pgmap v185: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:13:42.783 INFO:teuthology.orchestra.run.smithi005.stderr:21800000 of 50000000 tuples (43%) done (elapsed 340.35 s, remaining 440.26 s) 2023-07-21T19:13:43.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:43 smithi101 ceph-mon[128136]: pgmap v186: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 31 op/s 2023-07-21T19:13:43.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:43 smithi005 ceph-mon[126460]: pgmap v186: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 31 op/s 2023-07-21T19:13:43.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:43 smithi156 ceph-mon[128316]: pgmap v186: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 20 MiB/s wr, 31 op/s 2023-07-21T19:13:44.188 INFO:teuthology.orchestra.run.smithi005.stderr:21900000 of 50000000 tuples (43%) done (elapsed 341.75 s, remaining 438.50 s) 2023-07-21T19:13:45.512 INFO:teuthology.orchestra.run.smithi005.stderr:22000000 of 50000000 tuples (44%) done (elapsed 343.08 s, remaining 436.64 s) 2023-07-21T19:13:45.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:45 smithi101 ceph-mon[128136]: pgmap v187: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 26 op/s 2023-07-21T19:13:45.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:45 smithi005 ceph-mon[126460]: pgmap v187: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 26 op/s 2023-07-21T19:13:45.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:45 smithi156 ceph-mon[128316]: pgmap v187: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 10 MiB/s wr, 26 op/s 2023-07-21T19:13:46.913 INFO:teuthology.orchestra.run.smithi005.stderr:22100000 of 50000000 tuples (44%) done (elapsed 344.48 s, remaining 434.88 s) 2023-07-21T19:13:47.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:47 smithi101 ceph-mon[128136]: pgmap v188: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.7 MiB/s wr, 24 op/s 2023-07-21T19:13:47.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:47 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:47.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:47 smithi005 ceph-mon[126460]: pgmap v188: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.7 MiB/s wr, 24 op/s 2023-07-21T19:13:47.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:47 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:47.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:47 smithi156 ceph-mon[128316]: pgmap v188: 97 pgs: 97 active+clean; 5.0 GiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 8.7 MiB/s wr, 24 op/s 2023-07-21T19:13:47.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:47 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:48.319 INFO:teuthology.orchestra.run.smithi005.stderr:22200000 of 50000000 tuples (44%) done (elapsed 345.88 s, remaining 433.13 s) 2023-07-21T19:13:49.632 INFO:teuthology.orchestra.run.smithi005.stderr:22300000 of 50000000 tuples (44%) done (elapsed 347.20 s, remaining 431.27 s) 2023-07-21T19:13:49.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:49 smithi101 ceph-mon[128136]: pgmap v189: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 19 MiB/s wr, 39 op/s 2023-07-21T19:13:49.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:49 smithi005 ceph-mon[126460]: pgmap v189: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 19 MiB/s wr, 39 op/s 2023-07-21T19:13:49.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:49 smithi156 ceph-mon[128316]: pgmap v189: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 19 MiB/s wr, 39 op/s 2023-07-21T19:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 -- 172.21.15.5:0/2660116598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a00685b0 msgr2=0x7f18a0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 --2- 172.21.15.5:0/2660116598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a00685b0 0x7f18a0104770 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f1894009990 tx=0x7f1894009ca0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 -- 172.21.15.5:0/2660116598 shutdown_connections 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 --2- 172.21.15.5:0/2660116598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0104e50 0x7f18a010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 --2- 172.21.15.5:0/2660116598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a0067c10 0x7f18a0067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 --2- 172.21.15.5:0/2660116598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a00685b0 0x7f18a0104770 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.174+0000 7f18a5dff700 1 -- 172.21.15.5:0/2660116598 >> 172.21.15.5:0/2660116598 conn(0x7f18a0074760 msgr2=0x7f18a0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.175+0000 7f18a5dff700 1 -- 172.21.15.5:0/2660116598 shutdown_connections 2023-07-21T19:13:50.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.175+0000 7f18a5dff700 1 -- 172.21.15.5:0/2660116598 wait complete. 2023-07-21T19:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 Processor -- start 2023-07-21T19:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 -- start start 2023-07-21T19:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a0104e50 0x7f18a01a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01a4250 con 0x7f18a0067c10 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01a4390 con 0x7f18a00685b0 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f18a5dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18a01a4500 con 0x7f18a0104e50 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f189effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52884/0 (socket says 172.21.15.5:52884) 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f189effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51858/0 (socket says 172.21.15.5:51858) 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.176+0000 7f189f7fe700 1 -- 172.21.15.5:0/3809777404 learned_addr learned my addr 172.21.15.5:0/3809777404 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189ffff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a0104e50 0x7f18a01a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 -- 172.21.15.5:0/3809777404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a0104e50 msgr2=0x7f18a01a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a0104e50 0x7f18a01a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 -- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 msgr2=0x7f18a00fec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 -- 172.21.15.5:0/3809777404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940095f0 con 0x7f18a00685b0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.177+0000 7f189effd700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18940098e0 tx=0x7f1894004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.178+0000 7f189cff9700 1 -- 172.21.15.5:0/3809777404 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f189401d070 con 0x7f18a00685b0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.178+0000 7f189cff9700 1 -- 172.21.15.5:0/3809777404 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1894021410 con 0x7f18a00685b0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.178+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18a01a4780 con 0x7f18a00685b0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.178+0000 7f189cff9700 1 -- 172.21.15.5:0/3809777404 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18940173f0 con 0x7f18a00685b0 2023-07-21T19:13:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.178+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18a01a4c00 con 0x7f18a00685b0 2023-07-21T19:13:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.182+0000 7f189cff9700 1 -- 172.21.15.5:0/3809777404 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f189400fa90 con 0x7f18a00685b0 2023-07-21T19:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.183+0000 7f18b97fdb80 1 Processor -- start 2023-07-21T19:13:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.183+0000 7f18b97fdb80 1 -- start start 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 0x5617c4a4dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 0x5617c4a48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c494dc50 con 0x5617c4a4e2d0 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c49a09e0 con 0x5617c4a48580 2023-07-21T19:13:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4999e70 con 0x5617c48a46b0 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 0x5617c4a48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 0x5617c4a4dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 0x5617c4a4dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:43492/0 (socket says 172.21.15.5:43492) 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189ffff700 1 -- 172.21.15.5:0/3428307991 learned_addr learned my addr 172.21.15.5:0/3428307991 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189cff9700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 0x7f18880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.184+0000 7f189effd700 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189cff9700 1 -- 172.21.15.5:0/3809777404 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f18940cdde0 con 0x7f18a00685b0 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 -- 172.21.15.5:0/3428307991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 msgr2=0x5617c4a4dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189ffff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 0x7f18880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 0x5617c4a4dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 -- 172.21.15.5:0/3428307991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 msgr2=0x5617c4a489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 -- 172.21.15.5:0/3428307991 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c47c7d80 con 0x5617c4a4e2d0 2023-07-21T19:13:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 0x5617c4a48040 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1890011570 tx=0x7f1890011880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2221be7d2274182 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f189ffff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 0x7f18880a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f189800fb80 tx=0x7f1898004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.185+0000 7f1885ffb700 1 -- 172.21.15.5:0/3428307991 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890004230 con 0x5617c4a4e2d0 2023-07-21T19:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f1885ffb700 1 -- 172.21.15.5:0/3428307991 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1890004390 con 0x5617c4a4e2d0 2023-07-21T19:13:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f1885ffb700 1 -- 172.21.15.5:0/3428307991 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18900197e0 con 0x5617c4a4e2d0 2023-07-21T19:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3428307991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 msgr2=0x5617c4a48040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 0x5617c4a48040 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f1890011570 tx=0x7f1890011880 comp rx=0 tx=0).stop 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3428307991 shutdown_connections 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c48a46b0 0x5617c4a4dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3428307991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a4e2d0 0x5617c4a48040 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3428307991 >> 172.21.15.5:0/3428307991 conn(0x5617c4a3dbe0 msgr2=0x5617c4a42f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.186+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3428307991 shutdown_connections 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.187+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3428307991 wait complete. 2023-07-21T19:13:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.187+0000 7f18b97fdb80 1 -- start start 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.187+0000 7f18b97fdb80 1 Processor -- start 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a51e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a52340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4939a60 con 0x5617c48a46b0 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4939760 con 0x5617c4a48580 2023-07-21T19:13:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c481d8c0 con 0x5617c4a4e2d0 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f18b97fdb80 10 client.0 _ll_register_callbacks cb 0x5617c4937700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52900/0 (socket says 172.21.15.5:52900) 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f189f7fe700 1 -- 172.21.15.5:0/3365770130 learned_addr learned my addr 172.21.15.5:0/3365770130 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.188+0000 7f189effd700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a51e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189ffff700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a52340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 -- 172.21.15.5:0/3365770130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 msgr2=0x5617c4a52340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a52340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 -- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 msgr2=0x5617c4a51e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a51e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 -- 172.21.15.5:0/3365770130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1890009d60 con 0x5617c48a46b0 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189effd700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a51e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1890009850 tx=0x7f18900052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8357979349e45bc2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.189+0000 7f187effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18900056e0 con 0x5617c48a46b0 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1890003680 con 0x5617c48a46b0 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5617c48f3490 con 0x5617c48a46b0 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1890003880 con 0x5617c48a46b0 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f18b97fdb80 10 client.16956 fetch_fsmap learned FSMap version 31 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5617c47b0c00 con 0x5617c48a46b0 2023-07-21T19:13:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.190+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5617c48f3490 con 0x5617c48a46b0 2023-07-21T19:13:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f1890021040 con 0x5617c48a46b0 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 10 client.16956 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 10 client.16956 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 20 client.16956 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4a61b30 0x5617c4a63f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 4 client.16956 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5617c4992e50 con 0x5617c4a61b30 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.191+0000 7f189effd700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4a61b30 0x5617c4a63f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.192+0000 7f189effd700 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4a61b30 0x5617c4a63f40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b16602947ad65ef server_cookie=61be78c4b8dd22ce in_seq=0 out_seq=0 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.192+0000 7f187effd700 10 client.16956 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.192+0000 7f187effd700 1 -- 172.21.15.5:0/3365770130 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5617c4992e50 con 0x5617c4a61b30 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.192+0000 7f187effd700 10 client.16956 handle_command_reply: tid=0 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.192+0000 7f18b97fdb80 1 client.16956 shutdown 2023-07-21T19:13:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 msgr2=0x5617c4a518c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f1890009850 tx=0x7f18900052f0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 shutdown_connections 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a52340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4a61b30 0x5617c4a63f40 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a518c0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3365770130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a48580 0x5617c4a51e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f187effd700 0 client.16956 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 >> 172.21.15.5:0/3365770130 conn(0x5617c4a3dbe0 msgr2=0x5617c4a424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 shutdown_connections 2023-07-21T19:13:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3365770130 wait complete. 2023-07-21T19:13:50.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.193+0000 7f18b97fdb80 20 client.16956 trim_cache size 0 max 16384 2023-07-21T19:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 Processor -- start 2023-07-21T19:13:50.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 -- start start 2023-07-21T19:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a48580 0x5617c4a48ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a4e2d0 0x5617c4a50a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4a4c4c0 con 0x5617c48a46b0 2023-07-21T19:13:50.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4a0e060 con 0x5617c4a4e2d0 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.214+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4a4e6a0 con 0x5617c4a48580 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52916/0 (socket says 172.21.15.5:52916) 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 -- 172.21.15.5:0/2151474163 learned_addr learned my addr 172.21.15.5:0/2151474163 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189effd700 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a48580 0x5617c4a48ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189ffff700 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a4e2d0 0x5617c4a50a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 -- 172.21.15.5:0/2151474163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a48580 msgr2=0x5617c4a48ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a48580 0x5617c4a48ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 -- 172.21.15.5:0/2151474163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a4e2d0 msgr2=0x5617c4a50a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a4e2d0 0x5617c4a50a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 -- 172.21.15.5:0/2151474163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5617c47ee1f0 con 0x5617c48a46b0 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.215+0000 7f189f7fe700 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1890023460 tx=0x7f18900114f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cdc854070040d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.216+0000 7f18867fc700 1 -- 172.21.15.5:0/2151474163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890005360 con 0x5617c48a46b0 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.216+0000 7f18867fc700 1 -- 172.21.15.5:0/2151474163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f18900054c0 con 0x5617c48a46b0 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.216+0000 7f18867fc700 1 -- 172.21.15.5:0/2151474163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890003c50 con 0x5617c48a46b0 2023-07-21T19:13:50.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 -- 172.21.15.5:0/2151474163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 msgr2=0x5617c4a48040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f1890023460 tx=0x7f18900114f0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 -- 172.21.15.5:0/2151474163 shutdown_connections 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c4a4e2d0 0x5617c4a50a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c48a46b0 0x5617c4a48040 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/2151474163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a48580 0x5617c4a48ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 -- 172.21.15.5:0/2151474163 >> 172.21.15.5:0/2151474163 conn(0x5617c4a3dbe0 msgr2=0x5617c4a4d0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 -- 172.21.15.5:0/2151474163 shutdown_connections 2023-07-21T19:13:50.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.217+0000 7f18b97fdb80 1 -- 172.21.15.5:0/2151474163 wait complete. 2023-07-21T19:13:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.218+0000 7f18b97fdb80 1 -- start start 2023-07-21T19:13:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.218+0000 7f18b97fdb80 1 Processor -- start 2023-07-21T19:13:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 0x5617c4a56480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a56f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4ab4a90 con 0x5617c4a48580 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4ab4c00 con 0x5617c48a46b0 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5617c4ab4dc0 con 0x5617c4a4e2d0 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f18b97fdb80 10 client.0 _ll_register_callbacks cb 0x7f189409ab70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 0x5617c4a56480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52920/0 (socket says 172.21.15.5:52920) 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189effd700 1 -- 172.21.15.5:0/3423674131 learned_addr learned my addr 172.21.15.5:0/3423674131 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189ffff700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a56f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 -- 172.21.15.5:0/3423674131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 msgr2=0x5617c4a56f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a56f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 -- 172.21.15.5:0/3423674131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 msgr2=0x5617c4a569c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.219+0000 7f189f7fe700 1 -- 172.21.15.5:0/3423674131 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5617c47b0c00 con 0x5617c48a46b0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f189effd700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f189f7fe700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 0x5617c4a56480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1890023560 tx=0x7f1890005d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21bb3fe491d30b73 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f187effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1890023e20 con 0x5617c48a46b0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1890004390 con 0x5617c48a46b0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.220+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f1890000e40 con 0x5617c48a46b0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1890003680 con 0x5617c48a46b0 2023-07-21T19:13:50.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 10 client.26645 fetch_fsmap learned FSMap version 31 2023-07-21T19:13:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=4+}) v3 -- 0x5617c47c6250 con 0x5617c48a46b0 2023-07-21T19:13:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1890004390 con 0x5617c48a46b0 2023-07-21T19:13:50.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 31) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f18900118f0 con 0x5617c48a46b0 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 10 client.26645 fetch_fsmap finished waiting for FSMap version 31 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 10 client.26645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 20 client.26645 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4ab6000 0x5617c4ab83e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 4 client.26645 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.221+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5617c4954ff0 con 0x5617c4ab6000 2023-07-21T19:13:50.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.222+0000 7f189effd700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4ab6000 0x5617c4ab83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.222+0000 7f187effd700 10 client.26645 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:50.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.222+0000 7f189effd700 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4ab6000 0x5617c4ab83e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c760104fccaef0 server_cookie=93b6318e6e08a1c0 in_seq=0 out_seq=0 2023-07-21T19:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.239+0000 7f187effd700 1 -- 172.21.15.5:0/3423674131 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5617c4954ff0 con 0x5617c4ab6000 2023-07-21T19:13:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.239+0000 7f187effd700 10 client.26645 handle_command_reply: tid=0 2023-07-21T19:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.239+0000 7f18b97fdb80 1 client.26645 shutdown 2023-07-21T19:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 msgr2=0x5617c4a56480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 0x5617c4a56480 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f1890023560 tx=0x7f1890005d20 comp rx=0 tx=0).stop 2023-07-21T19:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 shutdown_connections 2023-07-21T19:13:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5617c4a4e2d0 0x5617c4a56f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5617c48a46b0 0x5617c4a56480 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5617c4ab6000 0x5617c4ab83e0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 --2- 172.21.15.5:0/3423674131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5617c4a48580 0x5617c4a569c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 >> 172.21.15.5:0/3423674131 conn(0x5617c4a3dbe0 msgr2=0x5617c4a521b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f187effd700 0 client.26645 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.240+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 shutdown_connections 2023-07-21T19:13:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.241+0000 7f18b97fdb80 1 -- 172.21.15.5:0/3423674131 wait complete. 2023-07-21T19:13:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.241+0000 7f18b97fdb80 20 client.26645 trim_cache size 0 max 16384 2023-07-21T19:13:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:13:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:13:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:13:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:13:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 msgr2=0x7f18880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 0x7f18880a37f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f189800fb80 tx=0x7f1898004b40 comp rx=0 tx=0).stop 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 msgr2=0x7f18a00ff180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f18940098e0 tx=0x7f1894004730 comp rx=0 tx=0).stop 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 shutdown_connections 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f18a0104e50 0x7f18a01a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f18880a1340 0x7f18880a37f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f18a0067c10 0x7f18a00fec40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 --2- 172.21.15.5:0/3809777404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f18a00685b0 0x7f18a00ff180 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.242+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 >> 172.21.15.5:0/3809777404 conn(0x7f18a0074760 msgr2=0x7f18a00fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.243+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 shutdown_connections 2023-07-21T19:13:50.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:13:50.243+0000 7f18a5dff700 1 -- 172.21.15.5:0/3809777404 wait complete. 2023-07-21T19:13:51.025 INFO:teuthology.orchestra.run.smithi005.stderr:22400000 of 50000000 tuples (44%) done (elapsed 348.59 s, remaining 429.51 s) 2023-07-21T19:13:51.751 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:13:51.819 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:51 smithi005 ceph-mon[126460]: pgmap v190: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-21T19:13:51.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:51 smithi101 ceph-mon[128136]: pgmap v190: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-21T19:13:51.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:51 smithi156 ceph-mon[128316]: pgmap v190: 97 pgs: 97 active+clean; 5.1 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 83 KiB/s rd, 15 MiB/s wr, 26 op/s 2023-07-21T19:13:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 -- 172.21.15.5:0/3715810348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 msgr2=0x7f76f4109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/3715810348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4109aa0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f76e40098f0 tx=0x7f76e4009c00 comp rx=0 tx=0).stop 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 -- 172.21.15.5:0/3715810348 shutdown_connections 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/3715810348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f410a070 0x7f76f410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/3715810348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f4100ca0 0x7f76f4101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/3715810348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4109aa0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 -- 172.21.15.5:0/3715810348 >> 172.21.15.5:0/3715810348 conn(0x7f76f40fa4b0 msgr2=0x7f76f40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 -- 172.21.15.5:0/3715810348 shutdown_connections 2023-07-21T19:13:52.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.998+0000 7f76fb9ff700 1 -- 172.21.15.5:0/3715810348 wait complete. 2023-07-21T19:13:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.999+0000 7f76fb9ff700 1 Processor -- start 2023-07-21T19:13:52.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:51.999+0000 7f76fb9ff700 1 -- start start 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 0x7f76f419c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419cda0 con 0x7f76f410a070 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419cf10 con 0x7f76f4100ca0 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76fb9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f419d0d0 con 0x7f76f41015b0 2023-07-21T19:13:52.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76f979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76f9f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 0x7f76f419c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76f979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51922/0 (socket says 172.21.15.5:51922) 2023-07-21T19:13:52.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.000+0000 7f76f979b700 1 -- 172.21.15.5:0/4089925561 learned_addr learned my addr 172.21.15.5:0/4089925561 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 -- 172.21.15.5:0/4089925561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 msgr2=0x7f76f4198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 -- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 msgr2=0x7f76f419c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 0x7f76f419c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 -- 172.21.15.5:0/4089925561 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76e4009520 con 0x7f76f4100ca0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 0x7f76f419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76f979b700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e9c0 tx=0x7f76f000ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76ea7fc700 1 -- 172.21.15.5:0/4089925561 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0004a30 con 0x7f76f4100ca0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76ea7fc700 1 -- 172.21.15.5:0/4089925561 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f000cc50 con 0x7f76f4100ca0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f419d3b0 con 0x7f76f4100ca0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76ea7fc700 1 -- 172.21.15.5:0/4089925561 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f0005410 con 0x7f76f4100ca0 2023-07-21T19:13:52.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.001+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f41a4bf0 con 0x7f76f4100ca0 2023-07-21T19:13:52.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.005+0000 7f76ea7fc700 1 -- 172.21.15.5:0/4089925561 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f76f0005570 con 0x7f76f4100ca0 2023-07-21T19:13:52.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.007+0000 7f770f3ddb80 1 Processor -- start 2023-07-21T19:13:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 -- start start 2023-07-21T19:13:52.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76ea7fc700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 0x7f76e00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76ea7fc700 1 -- 172.21.15.5:0/4089925561 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f76f0014070 con 0x7f76f4100ca0 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c61b1cb0 con 0x5643c62b6ad0 2023-07-21T19:13:52.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c6204a40 con 0x5643c62abbe0 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c61fded0 con 0x5643c62ab2d0 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 0x7f76e00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76f979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51926/0 (socket says 172.21.15.5:51926) 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.008+0000 7f76f8f9a700 1 -- 172.21.15.5:0/3416032165 learned_addr learned my addr 172.21.15.5:0/3416032165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:52.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 -- 172.21.15.5:0/3416032165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 msgr2=0x5643c62ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 -- 172.21.15.5:0/3416032165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 msgr2=0x5643c62b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 -- 172.21.15.5:0/3416032165 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c602bde0 con 0x5643c62abbe0 2023-07-21T19:13:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76e4000c00 tx=0x7f76e4005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b67279cbd16d614e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 0x7f76e00a37f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76df7fe700 1 -- 172.21.15.5:0/3416032165 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400fe60 con 0x5643c62abbe0 2023-07-21T19:13:52.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76df7fe700 1 -- 172.21.15.5:0/3416032165 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76e401d040 con 0x5643c62abbe0 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.009+0000 7f76df7fe700 1 -- 172.21.15.5:0/3416032165 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e40052a0 con 0x5643c62abbe0 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3416032165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 msgr2=0x5643c62b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f76e4000c00 tx=0x7f76e4005dc0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3416032165 shutdown_connections 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3416032165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b6590 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3416032165 >> 172.21.15.5:0/3416032165 conn(0x5643c62a1da0 msgr2=0x5643c62a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3416032165 shutdown_connections 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.010+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3416032165 wait complete. 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.011+0000 7f770f3ddb80 1 -- start start 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.011+0000 7f770f3ddb80 1 Processor -- start 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62b6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c619dac0 con 0x5643c62b6ad0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c619d7c0 con 0x5643c62abbe0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c6081920 con 0x5643c62ab2d0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f770f3ddb80 10 client.0 _ll_register_callbacks cb 0x5643c619b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62b6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51938/0 (socket says 172.21.15.5:51938) 2023-07-21T19:13:52.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f8f9a700 1 -- 172.21.15.5:0/1328389009 learned_addr learned my addr 172.21.15.5:0/1328389009 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62b6570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:43560/0 (socket says 172.21.15.5:43560) 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.012+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 -- 172.21.15.5:0/1328389009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 msgr2=0x5643c62b6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62b6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 -- 172.21.15.5:0/1328389009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 msgr2=0x5643c62b3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 -- 172.21.15.5:0/1328389009 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f410d170 con 0x5643c62abbe0 2023-07-21T19:13:52.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:13:52.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e400b870 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7fd1a5553a67932 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400f870 con 0x5643c62abbe0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f76e4010ea0 con 0x5643c62abbe0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.013+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5643c61574f0 con 0x5643c62abbe0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76e40050c0 con 0x5643c62abbe0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 10 client.26657 fetch_fsmap learned FSMap version 32 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x5643c6014c60 con 0x5643c62abbe0 2023-07-21T19:13:52.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5643c61574f0 con 0x5643c62abbe0 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f76e4015020 con 0x5643c62abbe0 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 10 client.26657 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 10 client.26657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 20 client.26657 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c62ce7c0 0x5643c62d0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 4 client.26657 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:13:52.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.014+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5643c61f6eb0 con 0x5643c62ce7c0 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.015+0000 7f76f979b700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c62ce7c0 0x5643c62d0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.015+0000 7f76dcff9700 10 client.26657 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.015+0000 7f76f979b700 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c62ce7c0 0x5643c62d0bd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b003b602c744b37 server_cookie=d398b3f6ce3ed3ef in_seq=0 out_seq=0 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f76dcff9700 1 -- 172.21.15.5:0/1328389009 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5643c61f6eb0 con 0x5643c62ce7c0 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f76dcff9700 10 client.26657 handle_command_reply: tid=0 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 client.26657 shutdown 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 msgr2=0x5643c62b37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e400b870 comp rx=0 tx=0).stop 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 shutdown_connections 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c62ce7c0 0x5643c62d0bd0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62b6ad0 0x5643c62b3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f76dcff9700 0 client.26657 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62ab2d0 0x5643c62b6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/1328389009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62abbe0 0x5643c62b37f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 >> 172.21.15.5:0/1328389009 conn(0x5643c62a1da0 msgr2=0x5643c62a7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 shutdown_connections 2023-07-21T19:13:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.016+0000 7f770f3ddb80 1 -- 172.21.15.5:0/1328389009 wait complete. 2023-07-21T19:13:52.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.017+0000 7f770f3ddb80 20 client.26657 trim_cache size 0 max 16384 2023-07-21T19:13:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.036+0000 7f770f3ddb80 1 Processor -- start 2023-07-21T19:13:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 -- start start 2023-07-21T19:13:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62abbe0 0x5643c62d54f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 0x5643c62c5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62a8d60 con 0x5643c62abbe0 2023-07-21T19:13:52.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62ac540 con 0x5643c62ab2d0 2023-07-21T19:13:52.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62b4cb0 con 0x5643c62b6ad0 2023-07-21T19:13:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f76f979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f76f9f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 0x5643c62c5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62abbe0 0x5643c62d54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f76f979b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51954/0 (socket says 172.21.15.5:51954) 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.037+0000 7f76f979b700 1 -- 172.21.15.5:0/3365775567 learned_addr learned my addr 172.21.15.5:0/3365775567 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 -- 172.21.15.5:0/3365775567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 msgr2=0x5643c62c5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 0x5643c62c5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 -- 172.21.15.5:0/3365775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62abbe0 msgr2=0x5643c62d54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62abbe0 0x5643c62d54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 -- 172.21.15.5:0/3365775567 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5643c6052250 con 0x5643c62ab2d0 2023-07-21T19:13:52.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f9f9c700 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 0x5643c62c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76f979b700 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f76f000f010 tx=0x7f76f00962f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25f3a103a3d1ffbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76dffff700 1 -- 172.21.15.5:0/3365775567 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00b82b0 con 0x5643c62ab2d0 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76dffff700 1 -- 172.21.15.5:0/3365775567 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f009deb0 con 0x5643c62ab2d0 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.038+0000 7f76dffff700 1 -- 172.21.15.5:0/3365775567 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f009a0e0 con 0x5643c62ab2d0 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3365775567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 msgr2=0x5643c62d0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f76f000f010 tx=0x7f76f00962f0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3365775567 shutdown_connections 2023-07-21T19:13:52.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62b6ad0 0x5643c62c5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62ab2d0 0x5643c62d0ba0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/3365775567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62abbe0 0x5643c62d54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3365775567 >> 172.21.15.5:0/3365775567 conn(0x5643c62a1da0 msgr2=0x5643c62a3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3365775567 shutdown_connections 2023-07-21T19:13:52.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- 172.21.15.5:0/3365775567 wait complete. 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.039+0000 7f770f3ddb80 1 -- start start 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 Processor -- start 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62abbe0 0x5643c62c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62b6ad0 0x5643c62c4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62c4570 con 0x5643c62ab2d0 2023-07-21T19:13:52.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62c46e0 con 0x5643c62b6ad0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5643c62c48a0 con 0x5643c62abbe0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f770f3ddb80 10 client.0 _ll_register_callbacks cb 0x7f76f00961c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.040+0000 7f76f979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53000/0 (socket says 172.21.15.5:53000) 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f9f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62b6ad0 0x5643c62c4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f8f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62abbe0 0x5643c62c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 -- 172.21.15.5:0/2108111166 learned_addr learned my addr 172.21.15.5:0/2108111166 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 -- 172.21.15.5:0/2108111166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62abbe0 msgr2=0x5643c62c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62abbe0 0x5643c62c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 -- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62b6ad0 msgr2=0x5643c62c4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62b6ad0 0x5643c62c4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 -- 172.21.15.5:0/2108111166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76ec008d70 con 0x5643c62ab2d0 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76f979b700 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009590 tx=0x7f76f00999c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29ddf405ae080855 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76dcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:13:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.041+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76f00ca980 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7f76f0004b90 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f76e401d040 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7f76f0004d90 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 10 client.16980 fetch_fsmap learned FSMap version 32 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x7f76f41a4bf0 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f76f007e020 con 0x5643c62ab2d0 2023-07-21T19:13:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f76f009a200 con 0x5643c62ab2d0 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 10 client.16980 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 10 client.16980 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 20 client.16980 populate_metadata read hostname 'smithi005' 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c6323650 0x5643c6325a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 4 client.16980 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.042+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5643c61b9050 con 0x5643c6323650 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c6323650 0x5643c6325a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:13:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f76f8f9a700 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c6323650 0x5643c6325a60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d65c6d9442ce16cc server_cookie=be6409a1b9db7f50 in_seq=0 out_seq=0 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f76dcff9700 10 client.16980 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f76dcff9700 1 -- 172.21.15.5:0/2108111166 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5643c61b9050 con 0x5643c6323650 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f76dcff9700 10 client.16980 handle_command_reply: tid=0 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.043+0000 7f770f3ddb80 1 client.16980 shutdown 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 msgr2=0x5643c62c57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f76f0009590 tx=0x7f76f00999c0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 shutdown_connections 2023-07-21T19:13:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5643c62b6ad0 0x5643c62c4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5643c6323650 0x5643c6325a60 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5643c62ab2d0 0x5643c62c57c0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 --2- 172.21.15.5:0/2108111166 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5643c62abbe0 0x5643c62c3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 >> 172.21.15.5:0/2108111166 conn(0x5643c62a1da0 msgr2=0x5643c62a68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f76dcff9700 0 client.16980 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 shutdown_connections 2023-07-21T19:13:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 1 -- 172.21.15.5:0/2108111166 wait complete. 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.044+0000 7f770f3ddb80 20 client.16980 trim_cache size 0 max 16384 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stdout: "af538a69-c7b9-4588-bf48-434c0a1d68b9": { 2023-07-21T19:13:52.055 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:13:52.056 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "af538a69-c7b9-4588-bf48-434c0a1d68b9", 2023-07-21T19:13:52.056 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:13:52.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:13:52.056 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:13:52.056 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:13:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 msgr2=0x7f76e00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 0x7f76e00a37f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f76ec009fd0 tx=0x7f76ec009920 comp rx=0 tx=0).stop 2023-07-21T19:13:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 msgr2=0x7f76f4197dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:13:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f76f000e9c0 tx=0x7f76f000ecd0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 shutdown_connections 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f76e00a1340 0x7f76e00a37f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f76f410a070 0x7f76f419c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f76f4100ca0 0x7f76f4197dc0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 --2- 172.21.15.5:0/4089925561 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f76f41015b0 0x7f76f4198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 >> 172.21.15.5:0/4089925561 conn(0x7f76f40fa4b0 msgr2=0x7f76f410c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 shutdown_connections 2023-07-21T19:13:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:13:52.045+0000 7f76fb9ff700 1 -- 172.21.15.5:0/4089925561 wait complete. 2023-07-21T19:13:52.069 INFO:tasks.cephfs.filesystem:scrub status for tag:af538a69-c7b9-4588-bf48-434c0a1d68b9 - {'path': '/', 'tag': 'af538a69-c7b9-4588-bf48-434c0a1d68b9', 'options': 'recursive,force'} 2023-07-21T19:13:52.397 INFO:teuthology.orchestra.run.smithi005.stderr:22500000 of 50000000 tuples (45%) done (elapsed 349.96 s, remaining 427.73 s) 2023-07-21T19:13:52.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:52 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:52.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:52 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:13:52.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:52 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:52.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:52 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:13:52.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:52 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:13:52.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:52 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:13:53.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:53 smithi101 ceph-mon[128136]: pgmap v191: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 26 MiB/s wr, 50 op/s 2023-07-21T19:13:53.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:13:53.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:13:53.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:13:53.876 INFO:teuthology.orchestra.run.smithi005.stderr:22600000 of 50000000 tuples (45%) done (elapsed 351.44 s, remaining 426.08 s) 2023-07-21T19:13:53.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:53 smithi005 ceph-mon[126460]: pgmap v191: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 26 MiB/s wr, 50 op/s 2023-07-21T19:13:53.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:13:53.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:13:53.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:13:53.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:53 smithi156 ceph-mon[128316]: pgmap v191: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 26 MiB/s wr, 50 op/s 2023-07-21T19:13:53.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:13:53.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:13:53.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:13:55.200 INFO:teuthology.orchestra.run.smithi005.stderr:22700000 of 50000000 tuples (45%) done (elapsed 352.76 s, remaining 424.25 s) 2023-07-21T19:13:55.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:55 smithi101 ceph-mon[128136]: pgmap v192: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:13:55.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:55 smithi005 ceph-mon[126460]: pgmap v192: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:13:55.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:55 smithi156 ceph-mon[128316]: pgmap v192: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 111 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:13:56.517 INFO:teuthology.orchestra.run.smithi005.stderr:22800000 of 50000000 tuples (45%) done (elapsed 354.08 s, remaining 422.41 s) 2023-07-21T19:13:57.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:57 smithi101 ceph-mon[128136]: pgmap v193: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 44 op/s 2023-07-21T19:13:57.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:57 smithi005 ceph-mon[126460]: pgmap v193: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 44 op/s 2023-07-21T19:13:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:22900000 of 50000000 tuples (45%) done (elapsed 355.48 s, remaining 420.67 s) 2023-07-21T19:13:57.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:57 smithi156 ceph-mon[128316]: pgmap v193: 97 pgs: 97 active+clean; 5.2 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 106 KiB/s rd, 22 MiB/s wr, 44 op/s 2023-07-21T19:13:59.192 INFO:teuthology.orchestra.run.smithi005.stderr:23000000 of 50000000 tuples (46%) done (elapsed 356.76 s, remaining 418.80 s) 2023-07-21T19:13:59.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:13:59 smithi101 ceph-mon[128136]: pgmap v194: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 33 MiB/s wr, 72 op/s 2023-07-21T19:13:59.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:13:59 smithi005 ceph-mon[126460]: pgmap v194: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 33 MiB/s wr, 72 op/s 2023-07-21T19:13:59.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:13:59 smithi156 ceph-mon[128316]: pgmap v194: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 132 KiB/s rd, 33 MiB/s wr, 72 op/s 2023-07-21T19:14:00.514 INFO:teuthology.orchestra.run.smithi005.stderr:23100000 of 50000000 tuples (46%) done (elapsed 358.08 s, remaining 416.98 s) 2023-07-21T19:14:01.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:01 smithi101 ceph-mon[128136]: pgmap v195: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:14:01.856 INFO:teuthology.orchestra.run.smithi005.stderr:23200000 of 50000000 tuples (46%) done (elapsed 359.42 s, remaining 415.19 s) 2023-07-21T19:14:01.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:01 smithi005 ceph-mon[126460]: pgmap v195: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:14:01.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:01 smithi156 ceph-mon[128316]: pgmap v195: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:14:03.253 INFO:teuthology.orchestra.run.smithi005.stderr:23300000 of 50000000 tuples (46%) done (elapsed 360.82 s, remaining 413.47 s) 2023-07-21T19:14:03.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 -- 172.21.15.5:0/1194204972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c0ff950 msgr2=0x7ff37c0ffdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 --2- 172.21.15.5:0/1194204972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c0ff950 0x7ff37c0ffdc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff370009990 tx=0x7ff370009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 -- 172.21.15.5:0/1194204972 shutdown_connections 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 --2- 172.21.15.5:0/1194204972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 0x7ff37c10c790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 --2- 172.21.15.5:0/1194204972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c105960 0x7ff37c105d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 --2- 172.21.15.5:0/1194204972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c0ff950 0x7ff37c0ffdc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.427+0000 7ff37bfff700 1 -- 172.21.15.5:0/1194204972 >> 172.21.15.5:0/1194204972 conn(0x7ff37c0fb470 msgr2=0x7ff37c0fd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.428+0000 7ff37bfff700 1 -- 172.21.15.5:0/1194204972 shutdown_connections 2023-07-21T19:14:03.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.428+0000 7ff37bfff700 1 -- 172.21.15.5:0/1194204972 wait complete. 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 Processor -- start 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 -- start start 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 0x7ff37c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c105960 0x7ff37c19a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19a910 con 0x7ff37c100300 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19aa80 con 0x7ff37c0ff950 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.429+0000 7ff37bfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff37c19ac40 con 0x7ff37c105960 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 0x7ff37c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38884/0 (socket says 172.21.15.5:38884) 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 -- 172.21.15.5:0/810837378 learned_addr learned my addr 172.21.15.5:0/810837378 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:03.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37b7fe700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c105960 0x7ff37c19a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 -- 172.21.15.5:0/810837378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c105960 msgr2=0x7ff37c19a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c105960 0x7ff37c19a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 -- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 msgr2=0x7ff37c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 0x7ff37c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 -- 172.21.15.5:0/810837378 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff374009c40 con 0x7ff37c0ff950 2023-07-21T19:14:03.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.430+0000 7ff37affd700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00ea50 tx=0x7ff36c00ed60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.431+0000 7ff363fff700 1 -- 172.21.15.5:0/810837378 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00eee0 con 0x7ff37c0ff950 2023-07-21T19:14:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.431+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff36c009520 con 0x7ff37c0ff950 2023-07-21T19:14:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.431+0000 7ff363fff700 1 -- 172.21.15.5:0/810837378 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff36c00cbd0 con 0x7ff37c0ff950 2023-07-21T19:14:03.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.431+0000 7ff363fff700 1 -- 172.21.15.5:0/810837378 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c018680 con 0x7ff37c0ff950 2023-07-21T19:14:03.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.431+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3700095f0 con 0x7ff37c0ff950 2023-07-21T19:14:03.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.436+0000 7ff363fff700 1 -- 172.21.15.5:0/810837378 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff36c010430 con 0x7ff37c0ff950 2023-07-21T19:14:03.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 Processor -- start 2023-07-21T19:14:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 -- start start 2023-07-21T19:14:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0411f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 0x55e1f040c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f0311c50 con 0x55e1f0412470 2023-07-21T19:14:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f03649e0 con 0x55e1f04128c0 2023-07-21T19:14:03.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.437+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f035de70 con 0x55e1f02686b0 2023-07-21T19:14:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0411f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0411f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54968/0 (socket says 172.21.15.5:54968) 2023-07-21T19:14:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37b7fe700 1 -- 172.21.15.5:0/2526565563 learned_addr learned my addr 172.21.15.5:0/2526565563 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:03.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff363fff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 0x7ff3640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 0x55e1f040c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37affd700 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff363fff700 1 -- 172.21.15.5:0/810837378 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff36c014070 con 0x7ff37c0ff950 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37b7fe700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 0x7ff3640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 -- 172.21.15.5:0/2526565563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 msgr2=0x55e1f0411f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0411f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 -- 172.21.15.5:0/2526565563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f0412470 msgr2=0x55e1f040c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.438+0000 7ff37a7fc700 1 -- 172.21.15.5:0/2526565563 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1f018bd80 con 0x55e1f04128c0 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.439+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 0x55e1f040c700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098e0 tx=0x7ff370005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f17f4dd7f3beb23 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.439+0000 7ff37b7fe700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 0x7ff3640a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff374009f80 tx=0x7ff374004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.439+0000 7ff3617fa700 1 -- 172.21.15.5:0/2526565563 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37001d070 con 0x55e1f04128c0 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.439+0000 7ff3617fa700 1 -- 172.21.15.5:0/2526565563 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff37001e040 con 0x55e1f04128c0 2023-07-21T19:14:03.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.439+0000 7ff3617fa700 1 -- 172.21.15.5:0/2526565563 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370004760 con 0x55e1f04128c0 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2526565563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 msgr2=0x55e1f040c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 0x55e1f040c700 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098e0 tx=0x7ff370005f00 comp rx=0 tx=0).stop 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2526565563 shutdown_connections 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f04128c0 0x55e1f040c700 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0411f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2526565563 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2526565563 >> 172.21.15.5:0/2526565563 conn(0x55e1f0401d40 msgr2=0x55e1f0407340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2526565563 shutdown_connections 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.440+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2526565563 wait complete. 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 -- start start 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 Processor -- start 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 0x55e1f041b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f04128c0 0x55e1f04159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f02fda60 con 0x55e1f02686b0 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f02fd760 con 0x55e1f0412470 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.441+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f01e18c0 con 0x55e1f04128c0 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff394cc7b80 10 client.0 _ll_register_callbacks cb 0x55e1f02fb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38902/0 (socket says 172.21.15.5:38902) 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37a7fc700 1 -- 172.21.15.5:0/3744907816 learned_addr learned my addr 172.21.15.5:0/3744907816 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:03.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 0x55e1f041b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37b7fe700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f04128c0 0x55e1f04159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 -- 172.21.15.5:0/3744907816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f04128c0 msgr2=0x55e1f04159c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f04128c0 0x55e1f04159c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 -- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 msgr2=0x55e1f041bc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 -- 172.21.15.5:0/3744907816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff37c10d090 con 0x55e1f02686b0 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:03.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff37affd700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 0x55e1f041b730 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ca8d0 tx=0x7ff36c00ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0b3fdf00efcf000 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.442+0000 7ff35eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c065910 con 0x55e1f02686b0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1f02b7490 con 0x55e1f02686b0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff36c007e40 con 0x55e1f02686b0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36c09a250 con 0x55e1f02686b0 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff394cc7b80 10 client.16992 fetch_fsmap learned FSMap version 32 2023-07-21T19:14:03.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e1f0174c00 con 0x55e1f02686b0 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.443+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1f02b7490 con 0x55e1f02686b0 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff36c09a4b0 con 0x55e1f02686b0 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 10 client.16992 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 10 client.16992 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 20 client.16992 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f0425c30 0x55e1f0428040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 4 client.16992 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:03.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1f0356e50 con 0x55e1f0425c30 2023-07-21T19:14:03.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.444+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f0425c30 0x55e1f0428040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.445+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f0425c30 0x55e1f0428040 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a521aeb7f8296d0 server_cookie=cecdf10c29e43616 in_seq=0 out_seq=0 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.445+0000 7ff35eff5700 10 client.16992 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.445+0000 7ff35eff5700 1 -- 172.21.15.5:0/3744907816 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e1f0356e50 con 0x55e1f0425c30 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.445+0000 7ff35eff5700 10 client.16992 handle_command_reply: tid=0 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 client.16992 shutdown 2023-07-21T19:14:03.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 msgr2=0x55e1f041b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 0x55e1f041b730 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ca8d0 tx=0x7ff36c00ff90 comp rx=0 tx=0).stop 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 shutdown_connections 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f04128c0 0x55e1f04159c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f0425c30 0x55e1f0428040 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f02686b0 0x55e1f041b730 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3744907816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f041bc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 >> 172.21.15.5:0/3744907816 conn(0x55e1f0401d40 msgr2=0x55e1f0406640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff35eff5700 0 client.16992 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 shutdown_connections 2023-07-21T19:14:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3744907816 wait complete. 2023-07-21T19:14:03.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.446+0000 7ff394cc7b80 20 client.16992 trim_cache size 0 max 16384 2023-07-21T19:14:03.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 Processor -- start 2023-07-21T19:14:03.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 -- start start 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0413520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f0406dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f040c6e0 con 0x55e1f04128c0 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f040d3c0 con 0x55e1f0412470 2023-07-21T19:14:03.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f0403e60 con 0x55e1f02686b0 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff37affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0413520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.464+0000 7ff37b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f0406dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38918/0 (socket says 172.21.15.5:38918) 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 -- 172.21.15.5:0/3361452677 learned_addr learned my addr 172.21.15.5:0/3361452677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0413520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54978/0 (socket says 172.21.15.5:54978) 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 -- 172.21.15.5:0/3361452677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 msgr2=0x55e1f0413520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0413520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 -- 172.21.15.5:0/3361452677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 msgr2=0x55e1f0406dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f0406dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 -- 172.21.15.5:0/3361452677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1f01b21f0 con 0x55e1f0412470 2023-07-21T19:14:03.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098e0 tx=0x7ff37001a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2edc83bf6329c739 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.465+0000 7ff361ffb700 1 -- 172.21.15.5:0/3361452677 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37001d040 con 0x55e1f0412470 2023-07-21T19:14:03.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff361ffb700 1 -- 172.21.15.5:0/3361452677 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff37000be80 con 0x55e1f0412470 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff361ffb700 1 -- 172.21.15.5:0/3361452677 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff370004500 con 0x55e1f0412470 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3361452677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 msgr2=0x55e1f040c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff3700098e0 tx=0x7ff37001a040 comp rx=0 tx=0).stop 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3361452677 shutdown_connections 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f0406dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f02686b0 0x55e1f0413520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/3361452677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f0412470 0x55e1f040c1a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3361452677 >> 172.21.15.5:0/3361452677 conn(0x55e1f0401d40 msgr2=0x55e1f04158e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.466+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3361452677 shutdown_connections 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.467+0000 7ff394cc7b80 1 -- 172.21.15.5:0/3361452677 wait complete. 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.467+0000 7ff394cc7b80 1 -- start start 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.467+0000 7ff394cc7b80 1 Processor -- start 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f0412470 0x55e1f0419de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f047cec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f041a560 con 0x55e1f04128c0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f047d430 con 0x55e1f02686b0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1f047d5f0 con 0x55e1f0412470 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff394cc7b80 10 client.0 _ll_register_callbacks cb 0x7ff36c096590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f047cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f0412470 0x55e1f0419de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38922/0 (socket says 172.21.15.5:38922) 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 -- 172.21.15.5:0/2105990167 learned_addr learned my addr 172.21.15.5:0/2105990167 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 -- 172.21.15.5:0/2105990167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f0412470 msgr2=0x55e1f0419de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f0412470 0x55e1f0419de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 -- 172.21.15.5:0/2105990167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 msgr2=0x55e1f047cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.468+0000 7ff37affd700 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f047cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff37affd700 1 -- 172.21.15.5:0/2105990167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e1f018bd80 con 0x55e1f02686b0 2023-07-21T19:14:03.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff37affd700 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c5b0 tx=0x7ff36c099c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c67e10c9e317605 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff35eff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff36c099d90 con 0x55e1f02686b0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(88..88 src has 1..88) v4 ==== 8469+0+0 (secure 0 0 0) 0x7ff36c099980 con 0x55e1f02686b0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff37001e040 con 0x55e1f02686b0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff36c09a250 con 0x55e1f02686b0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff394cc7b80 10 client.26681 fetch_fsmap learned FSMap version 32 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.469+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) v3 -- 0x55e1f018a250 con 0x55e1f02686b0 2023-07-21T19:14:03.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff37001e040 con 0x55e1f02686b0 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 32) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff36c0b82d0 con 0x55e1f02686b0 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 10 client.26681 fetch_fsmap finished waiting for FSMap version 32 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 10 client.26681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 20 client.26681 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f047ea50 0x55e1f0480e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 4 client.26681 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:14:03.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1f0318ff0 con 0x55e1f047ea50 2023-07-21T19:14:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.470+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f047ea50 0x55e1f0480e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.471+0000 7ff37a7fc700 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f047ea50 0x55e1f0480e60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=933e9aa0993b3679 server_cookie=39f2f9e7961e7228 in_seq=0 out_seq=0 2023-07-21T19:14:03.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.471+0000 7ff35eff5700 10 client.26681 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff35eff5700 1 -- 172.21.15.5:0/2105990167 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1f0318ff0 con 0x55e1f047ea50 2023-07-21T19:14:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff35eff5700 10 client.26681 handle_command_reply: tid=0 2023-07-21T19:14:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 client.26681 shutdown 2023-07-21T19:14:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 msgr2=0x55e1f04198a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c5b0 tx=0x7ff36c099c70 comp rx=0 tx=0).stop 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 shutdown_connections 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1f047ea50 0x55e1f0480e60 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1f04128c0 0x55e1f047cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1f02686b0 0x55e1f04198a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 --2- 172.21.15.5:0/2105990167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1f0412470 0x55e1f0419de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff35eff5700 0 client.26681 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:03.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 >> 172.21.15.5:0/2105990167 conn(0x55e1f0401d40 msgr2=0x55e1f0414170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.486+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 shutdown_connections 2023-07-21T19:14:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff394cc7b80 1 -- 172.21.15.5:0/2105990167 wait complete. 2023-07-21T19:14:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff394cc7b80 20 client.26681 trim_cache size 0 max 16384 2023-07-21T19:14:03.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:14:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:14:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:14:03.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 msgr2=0x7ff3640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 0x7ff3640a37f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff374009f80 tx=0x7ff374004b40 comp rx=0 tx=0).stop 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 msgr2=0x7ff37c195970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00ea50 tx=0x7ff36c00ed60 comp rx=0 tx=0).stop 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 shutdown_connections 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff37c105960 0x7ff37c19a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff37c0ff950 0x7ff37c195970 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3640a1340 0x7ff3640a37f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.487+0000 7ff37bfff700 1 --2- 172.21.15.5:0/810837378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff37c100300 0x7ff37c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.488+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 >> 172.21.15.5:0/810837378 conn(0x7ff37c0fb470 msgr2=0x7ff37c0fca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.488+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 shutdown_connections 2023-07-21T19:14:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:03.488+0000 7ff37bfff700 1 -- 172.21.15.5:0/810837378 wait complete. 2023-07-21T19:14:03.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:03 smithi101 ceph-mon[128136]: pgmap v196: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:14:03.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:03 smithi005 ceph-mon[126460]: pgmap v196: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:14:03.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:03 smithi156 ceph-mon[128316]: pgmap v196: 97 pgs: 97 active+clean; 5.3 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:14:04.568 INFO:teuthology.orchestra.run.smithi005.stderr:23400000 of 50000000 tuples (46%) done (elapsed 362.13 s, remaining 411.65 s) 2023-07-21T19:14:05.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:05 smithi101 ceph-mon[128136]: pgmap v197: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:14:05.884 INFO:teuthology.orchestra.run.smithi005.stderr:23500000 of 50000000 tuples (47%) done (elapsed 363.45 s, remaining 409.85 s) 2023-07-21T19:14:05.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:05 smithi005 ceph-mon[126460]: pgmap v197: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:14:05.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:05 smithi156 ceph-mon[128316]: pgmap v197: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:14:07.310 INFO:teuthology.orchestra.run.smithi005.stderr:23600000 of 50000000 tuples (47%) done (elapsed 364.87 s, remaining 408.16 s) 2023-07-21T19:14:07.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: pgmap v198: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 65 op/s 2023-07-21T19:14:07.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T19:14:07.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:07.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:07.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:07.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.834 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: pgmap v198: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 65 op/s 2023-07-21T19:14:07.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T19:14:07.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:07.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:07.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:07.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: pgmap v198: 97 pgs: 97 active+clean; 5.4 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 65 op/s 2023-07-21T19:14:07.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: osdmap e89: 12 total, 12 up, 12 in 2023-07-21T19:14:07.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:07.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:07.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:07.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:07.923 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:08.587 INFO:teuthology.orchestra.run.smithi005.stderr:23700000 of 50000000 tuples (47%) done (elapsed 366.15 s, remaining 406.32 s) 2023-07-21T19:14:09.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:09 smithi101 ceph-mon[128136]: pgmap v200: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:09.860 INFO:teuthology.orchestra.run.smithi005.stderr:23800000 of 50000000 tuples (47%) done (elapsed 367.42 s, remaining 404.48 s) 2023-07-21T19:14:09.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:09 smithi005 ceph-mon[126460]: pgmap v200: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:09.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:09 smithi156 ceph-mon[128316]: pgmap v200: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:11.150 INFO:teuthology.orchestra.run.smithi005.stderr:23900000 of 50000000 tuples (47%) done (elapsed 368.71 s, remaining 402.65 s) 2023-07-21T19:14:11.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:11 smithi101 ceph-mon[128136]: pgmap v201: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:11.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:11 smithi005 ceph-mon[126460]: pgmap v201: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:11.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:11 smithi156 ceph-mon[128316]: pgmap v201: 97 pgs: 97 active+clean; 5.4 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 27 MiB/s wr, 77 op/s 2023-07-21T19:14:12.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:12.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:12 smithi101 ceph-mon[128136]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T19:14:12.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:12.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:12 smithi005 ceph-mon[126460]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T19:14:12.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:14:12.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:12 smithi156 ceph-mon[128316]: osdmap e90: 12 total, 12 up, 12 in 2023-07-21T19:14:13.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:13 smithi101 ceph-mon[128136]: pgmap v202: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 29 MiB/s wr, 58 op/s 2023-07-21T19:14:13.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:13 smithi005 ceph-mon[126460]: pgmap v202: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 29 MiB/s wr, 58 op/s 2023-07-21T19:14:13.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:13 smithi156 ceph-mon[128316]: pgmap v202: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 29 MiB/s wr, 58 op/s 2023-07-21T19:14:15.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:15 smithi101 ceph-mon[128136]: pgmap v204: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:14:15.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:15 smithi005 ceph-mon[126460]: pgmap v204: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:14:15.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:15 smithi156 ceph-mon[128316]: pgmap v204: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:14:16.527 INFO:teuthology.orchestra.run.smithi005.stderr:24000000 of 50000000 tuples (48%) done (elapsed 374.09 s, remaining 405.27 s) 2023-07-21T19:14:16.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 -- 172.21.15.5:0/4149268427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 msgr2=0x7fd274105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 --2- 172.21.15.5:0/4149268427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd274105f30 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd264009990 tx=0x7fd264009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 -- 172.21.15.5:0/4149268427 shutdown_connections 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 --2- 172.21.15.5:0/4149268427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2741004e0 0x7fd27410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 --2- 172.21.15.5:0/4149268427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd274105f30 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 --2- 172.21.15.5:0/4149268427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2740ffb30 0x7fd2740fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 -- 172.21.15.5:0/4149268427 >> 172.21.15.5:0/4149268427 conn(0x7fd2740fb5f0 msgr2=0x7fd2740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 -- 172.21.15.5:0/4149268427 shutdown_connections 2023-07-21T19:14:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.655+0000 7fd2797ff700 1 -- 172.21.15.5:0/4149268427 wait complete. 2023-07-21T19:14:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.657+0000 7fd2797ff700 1 Processor -- start 2023-07-21T19:14:16.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.657+0000 7fd2797ff700 1 -- start start 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.657+0000 7fd2797ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2740ffb30 0x7fd2740729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.657+0000 7fd2797ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 0x7fd274072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.657+0000 7fd2797ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2797ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741a41b0 con 0x7fd274105b60 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2797ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741a42f0 con 0x7fd2740ffb30 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2797ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2741a4460 con 0x7fd2741004e0 2023-07-21T19:14:16.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60194/0 (socket says 172.21.15.5:60194) 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 -- 172.21.15.5:0/1362776844 learned_addr learned my addr 172.21.15.5:0/1362776844 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2740ffb30 0x7fd2740729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 0x7fd274072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 -- 172.21.15.5:0/1362776844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 msgr2=0x7fd274072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 0x7fd274072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 -- 172.21.15.5:0/1362776844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2740ffb30 msgr2=0x7fd2740729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2740ffb30 0x7fd2740729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2737fe700 1 -- 172.21.15.5:0/1362776844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2640095f0 con 0x7fd274105b60 2023-07-21T19:14:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.658+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 0x7fd274072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd2737fe700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00ef30 tx=0x7fd26c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd26bfff700 1 -- 172.21.15.5:0/1362776844 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c018070 con 0x7fd274105b60 2023-07-21T19:14:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2741a46e0 con 0x7fd274105b60 2023-07-21T19:14:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd26bfff700 1 -- 172.21.15.5:0/1362776844 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26c00f040 con 0x7fd274105b60 2023-07-21T19:14:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd26bfff700 1 -- 172.21.15.5:0/1362776844 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00cc30 con 0x7fd274105b60 2023-07-21T19:14:16.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.659+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2741a4c40 con 0x7fd274105b60 2023-07-21T19:14:16.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.663+0000 7fd26bfff700 1 -- 172.21.15.5:0/1362776844 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd26c007500 con 0x7fd274105b60 2023-07-21T19:14:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.664+0000 7fd26bfff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 0x7fd2600a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.664+0000 7fd26bfff700 1 -- 172.21.15.5:0/1362776844 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd26c0cc830 con 0x7fd274105b60 2023-07-21T19:14:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.664+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 0x7fd2600a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.665+0000 7fd28d184b80 1 Processor -- start 2023-07-21T19:14:16.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.665+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 0x7fd2600a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd264006010 tx=0x7fd26400b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.665+0000 7fd28d184b80 1 -- start start 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf7d9c50 con 0x55f0bf8da2d0 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf82c9e0 con 0x55f0bf7306b0 2023-07-21T19:14:16.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd28d184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf825e70 con 0x55f0bf8d4580 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd2727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd2727fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60210/0 (socket says 172.21.15.5:60210) 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd2727fc700 1 -- 172.21.15.5:0/1126006327 learned_addr learned my addr 172.21.15.5:0/1126006327 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.666+0000 7fd2737fe700 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 -- 172.21.15.5:0/1126006327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 msgr2=0x55f0bf8d49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 -- 172.21.15.5:0/1126006327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf7306b0 msgr2=0x55f0bf8d9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 -- 172.21.15.5:0/1126006327 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0bf653d80 con 0x55f0bf8da2d0 2023-07-21T19:14:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d80 tx=0x7fd25c0116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bfe9f2ca3203b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2697fa700 1 -- 172.21.15.5:0/1126006327 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c011e20 con 0x55f0bf8da2d0 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2697fa700 1 -- 172.21.15.5:0/1126006327 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd25c0128e0 con 0x55f0bf8da2d0 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.667+0000 7fd2697fa700 1 -- 172.21.15.5:0/1126006327 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25c00f730 con 0x55f0bf8da2d0 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 -- 172.21.15.5:0/1126006327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 msgr2=0x55f0bf8d4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009d80 tx=0x7fd25c0116c0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 -- 172.21.15.5:0/1126006327 shutdown_connections 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1126006327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d4040 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 -- 172.21.15.5:0/1126006327 >> 172.21.15.5:0/1126006327 conn(0x55f0bf8c9be0 msgr2=0x55f0bf8cef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 -- 172.21.15.5:0/1126006327 shutdown_connections 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.668+0000 7fd28d184b80 1 -- 172.21.15.5:0/1126006327 wait complete. 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.669+0000 7fd28d184b80 1 -- start start 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.669+0000 7fd28d184b80 1 Processor -- start 2023-07-21T19:14:16.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8de030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf7c5a60 con 0x55f0bf7306b0 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf7c5760 con 0x55f0bf8da2d0 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf6a98c0 con 0x55f0bf8d4580 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd28d184b80 10 client.0 _ll_register_callbacks cb 0x55f0bf7c3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60214/0 (socket says 172.21.15.5:60214) 2023-07-21T19:14:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd272ffd700 1 -- 172.21.15.5:0/4188511649 learned_addr learned my addr 172.21.15.5:0/4188511649 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd2727fc700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.670+0000 7fd2737fe700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8de030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 -- 172.21.15.5:0/4188511649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 msgr2=0x55f0bf8dd8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8dd8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 -- 172.21.15.5:0/4188511649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 msgr2=0x55f0bf8e0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 -- 172.21.15.5:0/4188511649 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd25c0099b0 con 0x55f0bf8da2d0 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd272ffd700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd2737fe700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8de030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0c9f80 tx=0x7fd26c09da90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ce4b54862e2ecef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd25a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26c00cdf0 con 0x55f0bf8da2d0 2023-07-21T19:14:16.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd26c084040 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.671+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f0bf77f490 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd26c097d30 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 10 client.26693 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f0bf63cc00 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f0bf77f490 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd26c095780 con 0x55f0bf8da2d0 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 10 client.26693 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:16.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 10 client.26693 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 20 client.26693 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf8edb90 0x55f0bf8effa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 4 client.26693 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.672+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f0bf81ee50 con 0x55f0bf8edb90 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.673+0000 7fd272ffd700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf8edb90 0x55f0bf8effa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.673+0000 7fd25a7fc700 10 client.26693 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:16.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.673+0000 7fd272ffd700 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf8edb90 0x55f0bf8effa0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15d970be158c28fc server_cookie=5fdaef223da93065 in_seq=0 out_seq=0 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd25a7fc700 1 -- 172.21.15.5:0/4188511649 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f0bf81ee50 con 0x55f0bf8edb90 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd25a7fc700 10 client.26693 handle_command_reply: tid=0 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 client.26693 shutdown 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 msgr2=0x55f0bf8de030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8de030 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd26c0c9f80 tx=0x7fd26c09da90 comp rx=0 tx=0).stop 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 shutdown_connections 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8de030 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf8edb90 0x55f0bf8effa0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 --2- 172.21.15.5:0/4188511649 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8dd8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd25a7fc700 0 client.26693 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.674+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 >> 172.21.15.5:0/4188511649 conn(0x55f0bf8c9be0 msgr2=0x55f0bf8cdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.675+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 shutdown_connections 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.675+0000 7fd28d184b80 1 -- 172.21.15.5:0/4188511649 wait complete. 2023-07-21T19:14:16.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.675+0000 7fd28d184b80 20 client.26693 trim_cache size 0 max 16384 2023-07-21T19:14:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 Processor -- start 2023-07-21T19:14:16.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 -- start start 2023-07-21T19:14:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d91c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8cec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8d0990 con 0x55f0bf7306b0 2023-07-21T19:14:16.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8d99c0 con 0x55f0bf8d4580 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.692+0000 7fd28d184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8cb5f0 con 0x55f0bf8da2d0 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60230/0 (socket says 172.21.15.5:60230) 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 -- 172.21.15.5:0/2084368613 learned_addr learned my addr 172.21.15.5:0/2084368613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd2727fc700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d91c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd2737fe700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8cec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 -- 172.21.15.5:0/2084368613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 msgr2=0x55f0bf8cec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8cec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 -- 172.21.15.5:0/2084368613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 msgr2=0x55f0bf8d91c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d91c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 -- 172.21.15.5:0/2084368613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0bf8f4b90 con 0x55f0bf7306b0 2023-07-21T19:14:16.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd2737fe700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd2727fc700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.693+0000 7fd272ffd700 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55f0bf8f0840 tx=0x7fd264023000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c3e6852d96d6216 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.694+0000 7fd269ffb700 1 -- 172.21.15.5:0/2084368613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264005a00 con 0x55f0bf7306b0 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.694+0000 7fd269ffb700 1 -- 172.21.15.5:0/2084368613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26401e040 con 0x55f0bf7306b0 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.694+0000 7fd269ffb700 1 -- 172.21.15.5:0/2084368613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd264033420 con 0x55f0bf7306b0 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- 172.21.15.5:0/2084368613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 msgr2=0x55f0bf8e4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x55f0bf8f0840 tx=0x7fd264023000 comp rx=0 tx=0).stop 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- 172.21.15.5:0/2084368613 shutdown_connections 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8cec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8e4c60 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 --2- 172.21.15.5:0/2084368613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d91c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- 172.21.15.5:0/2084368613 >> 172.21.15.5:0/2084368613 conn(0x55f0bf8c9be0 msgr2=0x55f0bf8d8ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- 172.21.15.5:0/2084368613 shutdown_connections 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- 172.21.15.5:0/2084368613 wait complete. 2023-07-21T19:14:16.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 -- start start 2023-07-21T19:14:16.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.695+0000 7fd28d184b80 1 Processor -- start 2023-07-21T19:14:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d5860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8d76e0 con 0x55f0bf7306b0 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8d5da0 con 0x55f0bf8da2d0 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0bf8d5f30 con 0x55f0bf8d4580 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd28d184b80 10 client.0 _ll_register_callbacks cb 0x7fd26c0993e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd272ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60242/0 (socket says 172.21.15.5:60242) 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.696+0000 7fd272ffd700 1 -- 172.21.15.5:0/1795737128 learned_addr learned my addr 172.21.15.5:0/1795737128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd2737fe700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 -- 172.21.15.5:0/1795737128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 msgr2=0x55f0bf8d6fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d6fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 -- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 msgr2=0x55f0bf8d5860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d5860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 -- 172.21.15.5:0/1795737128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd27410d220 con 0x55f0bf7306b0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd272ffd700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd26400bd80 tx=0x7fd264005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4c1ea52daf8a989 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd25a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.697+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd26400fe00 con 0x55f0bf7306b0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd25c0128e0 con 0x55f0bf7306b0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd264036160 con 0x55f0bf7306b0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd264036320 con 0x55f0bf7306b0 2023-07-21T19:14:16.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 10 client.17016 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f0bf67a1f0 con 0x55f0bf7306b0 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd26404d080 con 0x55f0bf7306b0 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fd26402f030 con 0x55f0bf7306b0 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 10 client.17016 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 10 client.17016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 20 client.17016 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf942560 0x55f0bf944970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:16.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 4 client.17016 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:14:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.698+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f0bf7e0ff0 con 0x55f0bf942560 2023-07-21T19:14:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.699+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf942560 0x55f0bf944970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.699+0000 7fd2727fc700 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf942560 0x55f0bf944970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee5a827536ef5b72 server_cookie=44c67bbc72b28722 in_seq=0 out_seq=0 2023-07-21T19:14:16.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.699+0000 7fd25a7fc700 10 client.17016 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd25a7fc700 1 -- 172.21.15.5:0/1795737128 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f0bf7e0ff0 con 0x55f0bf942560 2023-07-21T19:14:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd25a7fc700 10 client.17016 handle_command_reply: tid=0 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 client.17016 shutdown 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 msgr2=0x55f0bf8d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fd26400bd80 tx=0x7fd264005aa0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 shutdown_connections 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f0bf8da2d0 0x55f0bf8d5860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f0bf942560 0x55f0bf944970 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f0bf7306b0 0x55f0bf8d6aa0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 --2- 172.21.15.5:0/1795737128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f0bf8d4580 0x55f0bf8d6fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd25a7fc700 0 client.17016 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 >> 172.21.15.5:0/1795737128 conn(0x55f0bf8c9be0 msgr2=0x55f0bf8da900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 shutdown_connections 2023-07-21T19:14:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 1 -- 172.21.15.5:0/1795737128 wait complete. 2023-07-21T19:14:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.713+0000 7fd28d184b80 20 client.17016 trim_cache size 0 max 16384 2023-07-21T19:14:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:14:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:14:16.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:14:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:14:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 msgr2=0x7fd2600a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 0x7fd2600a3770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fd264006010 tx=0x7fd26400b260 comp rx=0 tx=0).stop 2023-07-21T19:14:16.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 msgr2=0x7fd2741a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd26c00ef30 tx=0x7fd26c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 shutdown_connections 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2600a12c0 0x7fd2600a3770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd274105b60 0x7fd2741a3c70 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd2740ffb30 0x7fd2740729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 --2- 172.21.15.5:0/1362776844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd2741004e0 0x7fd274072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:16.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 >> 172.21.15.5:0/1362776844 conn(0x7fd2740fb5f0 msgr2=0x7fd274103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 shutdown_connections 2023-07-21T19:14:16.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:16.714+0000 7fd2797ff700 1 -- 172.21.15.5:0/1362776844 wait complete. 2023-07-21T19:14:16.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:14:16.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:14:16.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:14:16.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:14:16.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:14:16.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:14:16.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:14:16.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:14:16.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:14:17.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:17 smithi101 ceph-mon[128136]: pgmap v205: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-07-21T19:14:17.856 INFO:teuthology.orchestra.run.smithi005.stderr:24100000 of 50000000 tuples (48%) done (elapsed 375.42 s, remaining 403.46 s) 2023-07-21T19:14:17.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:17 smithi005 ceph-mon[126460]: pgmap v205: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-07-21T19:14:17.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:17 smithi156 ceph-mon[128316]: pgmap v205: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 28 MiB/s wr, 55 op/s 2023-07-21T19:14:18.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:18 smithi101 ceph-mon[128136]: pgmap v206: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:18.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:18 smithi005 ceph-mon[126460]: pgmap v206: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:18.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:18 smithi156 ceph-mon[128316]: pgmap v206: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:19.153 INFO:teuthology.orchestra.run.smithi005.stderr:24200000 of 50000000 tuples (48%) done (elapsed 376.72 s, remaining 401.62 s) 2023-07-21T19:14:20.528 INFO:teuthology.orchestra.run.smithi005.stderr:24300000 of 50000000 tuples (48%) done (elapsed 378.09 s, remaining 399.88 s) 2023-07-21T19:14:21.582 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:21 smithi101 ceph-mon[128136]: pgmap v207: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:21.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:21 smithi005 ceph-mon[126460]: pgmap v207: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:21.671 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:21 smithi156 ceph-mon[128316]: pgmap v207: 97 pgs: 97 active+clean; 5.5 GiB data, 2.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 16 MiB/s wr, 35 op/s 2023-07-21T19:14:21.816 INFO:teuthology.orchestra.run.smithi005.stderr:24400000 of 50000000 tuples (48%) done (elapsed 379.38 s, remaining 398.04 s) 2023-07-21T19:14:22.070 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:14:22.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 -- 172.21.15.5:0/2797801985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 msgr2=0x7f9e1c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 --2- 172.21.15.5:0/2797801985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c072c80 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c009990 tx=0x7f9e0c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 -- 172.21.15.5:0/2797801985 shutdown_connections 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 --2- 172.21.15.5:0/2797801985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c110af0 0x7f9e1c112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 --2- 172.21.15.5:0/2797801985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c072c80 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 --2- 172.21.15.5:0/2797801985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c0731c0 0x7f9e1c1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 -- 172.21.15.5:0/2797801985 >> 172.21.15.5:0/2797801985 conn(0x7f9e1c0fb5e0 msgr2=0x7f9e1c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 -- 172.21.15.5:0/2797801985 shutdown_connections 2023-07-21T19:14:22.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.307+0000 7f9e221ff700 1 -- 172.21.15.5:0/2797801985 wait complete. 2023-07-21T19:14:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 Processor -- start 2023-07-21T19:14:22.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 -- start start 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c0731c0 0x7f9e1c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 0x7f9e1c1a0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a1140 con 0x7f9e1c0728b0 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a12b0 con 0x7f9e1c110af0 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.309+0000 7f9e221ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e1c1a1470 con 0x7f9e1c0731c0 2023-07-21T19:14:22.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60270/0 (socket says 172.21.15.5:60270) 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/3871092416 learned_addr learned my addr 172.21.15.5:0/3871092416 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 0x7f9e1c1a0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1affd700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c0731c0 0x7f9e1c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/3871092416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c0731c0 msgr2=0x7f9e1c19c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c0731c0 0x7f9e1c19c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/3871092416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 msgr2=0x7f9e1c1a0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 0x7f9e1c1a0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/3871092416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e0c0095f0 con 0x7f9e1c0728b0 2023-07-21T19:14:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 0x7f9e1c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.310+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005740 tx=0x7f9e0c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.311+0000 7f9e18ff9700 1 -- 172.21.15.5:0/3871092416 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c01d070 con 0x7f9e1c0728b0 2023-07-21T19:14:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.311+0000 7f9e18ff9700 1 -- 172.21.15.5:0/3871092416 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e0c022470 con 0x7f9e1c0728b0 2023-07-21T19:14:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.311+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e1c1a16f0 con 0x7f9e1c0728b0 2023-07-21T19:14:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.311+0000 7f9e18ff9700 1 -- 172.21.15.5:0/3871092416 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c017650 con 0x7f9e1c0728b0 2023-07-21T19:14:22.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.311+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e1c1a8f90 con 0x7f9e1c0728b0 2023-07-21T19:14:22.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.315+0000 7f9e18ff9700 1 -- 172.21.15.5:0/3871092416 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9e0c0225e0 con 0x7f9e1c0728b0 2023-07-21T19:14:22.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.316+0000 7f9e35b56b80 1 Processor -- start 2023-07-21T19:14:22.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.316+0000 7f9e18ff9700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 0x7f9e040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e18ff9700 1 -- 172.21.15.5:0/3871092416 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9e0c0ce140 con 0x7f9e1c0728b0 2023-07-21T19:14:22.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 -- start start 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 0x7f9e040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c93c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 0x55585c947590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c949eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c842cb0 con 0x55585c93c2d0 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c895a40 con 0x55585c93cbe0 2023-07-21T19:14:22.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.317+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c88eed0 con 0x55585c947ad0 2023-07-21T19:14:22.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c93c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c93c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60284/0 (socket says 172.21.15.5:60284) 2023-07-21T19:14:22.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/709380846 learned_addr learned my addr 172.21.15.5:0/709380846 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c949eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 0x55585c947590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 0x7f9e040a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c069a80 tx=0x7f9e14009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 -- 172.21.15.5:0/709380846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 msgr2=0x55585c949eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c949eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 -- 172.21.15.5:0/709380846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 msgr2=0x55585c93c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c93c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.318+0000 7f9e1affd700 1 -- 172.21.15.5:0/709380846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55585c6bcde0 con 0x55585c93cbe0 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.319+0000 7f9e1affd700 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 0x55585c947590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009bd0 tx=0x7f9e10009ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab58fe2511c922a5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.319+0000 7f9e01ffb700 1 -- 172.21.15.5:0/709380846 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10011b90 con 0x55585c93cbe0 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.319+0000 7f9e01ffb700 1 -- 172.21.15.5:0/709380846 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e10011cf0 con 0x55585c93cbe0 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.319+0000 7f9e01ffb700 1 -- 172.21.15.5:0/709380846 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1000f640 con 0x55585c93cbe0 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- 172.21.15.5:0/709380846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 msgr2=0x55585c947590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 0x55585c947590 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9e10009bd0 tx=0x7f9e10009ee0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- 172.21.15.5:0/709380846 shutdown_connections 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c949eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c93c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/709380846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93cbe0 0x55585c947590 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- 172.21.15.5:0/709380846 >> 172.21.15.5:0/709380846 conn(0x55585c932da0 msgr2=0x55585c9386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- 172.21.15.5:0/709380846 shutdown_connections 2023-07-21T19:14:22.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- 172.21.15.5:0/709380846 wait complete. 2023-07-21T19:14:22.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.320+0000 7f9e35b56b80 1 -- start start 2023-07-21T19:14:22.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 Processor -- start 2023-07-21T19:14:22.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c9474e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c944e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c82eac0 con 0x55585c93cbe0 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c82e7c0 con 0x55585c93c2d0 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c712920 con 0x55585c947ad0 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.321+0000 7f9e35b56b80 10 client.0 _ll_register_callbacks cb 0x55585c82c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60292/0 (socket says 172.21.15.5:60292) 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 -- 172.21.15.5:0/359023439 learned_addr learned my addr 172.21.15.5:0/359023439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c9474e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c944e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 -- 172.21.15.5:0/359023439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 msgr2=0x55585c944e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c944e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 -- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 msgr2=0x55585c9474e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c9474e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1affd700 1 -- 172.21.15.5:0/359023439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e10009830 con 0x55585c93cbe0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.322+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c944e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9e1affd700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9e1001c040 tx=0x7f9e10012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c701eaad81b1fb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9dfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e10005ce0 con 0x55585c93cbe0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55585c7e84f0 con 0x55585c93cbe0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9e10012b50 con 0x55585c93cbe0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e10005250 con 0x55585c93cbe0 2023-07-21T19:14:22.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9e35b56b80 10 client.17028 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.323+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55585c6a5c60 con 0x55585c93cbe0 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55585c7e84f0 con 0x55585c93cbe0 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9e10022070 con 0x55585c93cbe0 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 10 client.17028 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 10 client.17028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 20 client.17028 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c957390 0x55585c9597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 4 client.17028 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:22.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.324+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55585c887eb0 con 0x55585c957390 2023-07-21T19:14:22.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.325+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c957390 0x55585c9597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.325+0000 7f9dfaffd700 10 client.17028 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:22.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.325+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c957390 0x55585c9597a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d648e858bffc2105 server_cookie=4087efebb712255d in_seq=0 out_seq=0 2023-07-21T19:14:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9dfaffd700 1 -- 172.21.15.5:0/359023439 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55585c887eb0 con 0x55585c957390 2023-07-21T19:14:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9dfaffd700 10 client.17028 handle_command_reply: tid=0 2023-07-21T19:14:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 client.17028 shutdown 2023-07-21T19:14:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 msgr2=0x55585c9447d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f9e1001c040 tx=0x7f9e10012730 comp rx=0 tx=0).stop 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 shutdown_connections 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c947ad0 0x55585c944e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c9474e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c957390 0x55585c9597a0 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/359023439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93cbe0 0x55585c9447d0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 >> 172.21.15.5:0/359023439 conn(0x55585c932da0 msgr2=0x55585c938230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9dfaffd700 0 client.17028 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:22.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 shutdown_connections 2023-07-21T19:14:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.326+0000 7f9e35b56b80 1 -- 172.21.15.5:0/359023439 wait complete. 2023-07-21T19:14:22.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.327+0000 7f9e35b56b80 20 client.17028 trim_cache size 0 max 16384 2023-07-21T19:14:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.344+0000 7f9e35b56b80 1 Processor -- start 2023-07-21T19:14:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.344+0000 7f9e35b56b80 1 -- start start 2023-07-21T19:14:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.344+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c94e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c9593a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 0x55585c93faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c939d60 con 0x55585c947ad0 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c954a50 con 0x55585c93c2d0 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c93d590 con 0x55585c93cbe0 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 0x55585c93faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c9593a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 0x55585c93faf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60296/0 (socket says 172.21.15.5:60296) 2023-07-21T19:14:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1bfff700 1 -- 172.21.15.5:0/211334446 learned_addr learned my addr 172.21.15.5:0/211334446 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c94e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 -- 172.21.15.5:0/211334446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 msgr2=0x55585c94e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c94e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 -- 172.21.15.5:0/211334446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 msgr2=0x55585c93faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 0x55585c93faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.345+0000 7f9e1affd700 1 -- 172.21.15.5:0/211334446 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55585c9370e0 con 0x55585c93cbe0 2023-07-21T19:14:22.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.346+0000 7f9e1affd700 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c9593a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e10004f20 tx=0x7f9e10024520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4935f49045bfb351 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.346+0000 7f9e027fc700 1 -- 172.21.15.5:0/211334446 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e100245f0 con 0x55585c93cbe0 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.346+0000 7f9e027fc700 1 -- 172.21.15.5:0/211334446 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e1001aad0 con 0x55585c93cbe0 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.346+0000 7f9e027fc700 1 -- 172.21.15.5:0/211334446 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e1001a3f0 con 0x55585c93cbe0 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 -- 172.21.15.5:0/211334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 msgr2=0x55585c9593a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c9593a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9e10004f20 tx=0x7f9e10024520 comp rx=0 tx=0).stop 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 -- 172.21.15.5:0/211334446 shutdown_connections 2023-07-21T19:14:22.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c947ad0 0x55585c93faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c93c2d0 0x55585c94e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/211334446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c9593a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 -- 172.21.15.5:0/211334446 >> 172.21.15.5:0/211334446 conn(0x55585c932da0 msgr2=0x55585c9536c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 -- 172.21.15.5:0/211334446 shutdown_connections 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.347+0000 7f9e35b56b80 1 -- 172.21.15.5:0/211334446 wait complete. 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.348+0000 7f9e35b56b80 1 -- start start 2023-07-21T19:14:22.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.348+0000 7f9e35b56b80 1 Processor -- start 2023-07-21T19:14:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c99f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c947ad0 0x55585c99fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c9aa300 con 0x55585c93c2d0 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c9aa440 con 0x55585c947ad0 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55585c9aa5d0 con 0x55585c93cbe0 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e35b56b80 10 client.0 _ll_register_callbacks cb 0x7f9e0c0094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60308/0 (socket says 172.21.15.5:60308) 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/1871535428 learned_addr learned my addr 172.21.15.5:0/1871535428 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1bfff700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c947ad0 0x55585c99fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1affd700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c99f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/1871535428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 msgr2=0x55585c99f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c99f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c947ad0 msgr2=0x55585c99fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:22.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c947ad0 0x55585c99fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.349+0000 7f9e1b7fe700 1 -- 172.21.15.5:0/1871535428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55585c6a5c60 con 0x55585c93c2d0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9e1b7fe700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c09b050 tx=0x7f9e0c086040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd8384bb34f37259 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9dfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e0c09f380 con 0x55585c93c2d0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9e0c0a3710 con 0x55585c93c2d0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e10011cf0 con 0x55585c93c2d0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e0c0cbab0 con 0x55585c93c2d0 2023-07-21T19:14:22.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9e35b56b80 10 client.17034 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55585c6e3250 con 0x55585c93c2d0 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.350+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e0c0c9080 con 0x55585c93c2d0 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f9e0c09b5b0 con 0x55585c93c2d0 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 10 client.17034 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 20 client.17034 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c9ab9d0 0x55585c9adde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 4 client.17034 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55585c84a050 con 0x55585c9ab9d0 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.351+0000 7f9e1affd700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c9ab9d0 0x55585c9adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9e1affd700 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c9ab9d0 0x55585c9adde0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abded2f6b463b50 server_cookie=1661651e4ee62eed in_seq=0 out_seq=0 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9dfaffd700 10 client.17034 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9dfaffd700 1 -- 172.21.15.5:0/1871535428 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55585c84a050 con 0x55585c9ab9d0 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9dfaffd700 10 client.17034 handle_command_reply: tid=0 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9e35b56b80 1 client.17034 shutdown 2023-07-21T19:14:22.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 msgr2=0x55585c99f1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.352+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c09b050 tx=0x7f9e0c086040 comp rx=0 tx=0).stop 2023-07-21T19:14:22.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 shutdown_connections 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55585c947ad0 0x55585c99fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55585c9ab9d0 0x55585c9adde0 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55585c93c2d0 0x55585c99f1a0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 --2- 172.21.15.5:0/1871535428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55585c93cbe0 0x55585c99f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 >> 172.21.15.5:0/1871535428 conn(0x55585c932da0 msgr2=0x55585c952c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9dfaffd700 0 client.17034 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 shutdown_connections 2023-07-21T19:14:22.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 1 -- 172.21.15.5:0/1871535428 wait complete. 2023-07-21T19:14:22.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.353+0000 7f9e35b56b80 20 client.17034 trim_cache size 0 max 16384 2023-07-21T19:14:22.362 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:14:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:14:22.362 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:14:22.362 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:14:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 msgr2=0x7f9e040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 0x7f9e040a37f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c069a80 tx=0x7f9e14009920 comp rx=0 tx=0).stop 2023-07-21T19:14:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 msgr2=0x7f9e1c19c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f9e0c005740 tx=0x7f9e0c00bf50 comp rx=0 tx=0).stop 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 shutdown_connections 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9e1c110af0 0x7f9e1c1a0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e040a1340 0x7f9e040a37f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9e1c0728b0 0x7f9e1c19c160 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 --2- 172.21.15.5:0/3871092416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9e1c0731c0 0x7f9e1c19c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 >> 172.21.15.5:0/3871092416 conn(0x7f9e1c0fb5e0 msgr2=0x7f9e1c0fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 shutdown_connections 2023-07-21T19:14:22.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.354+0000 7f9e221ff700 1 -- 172.21.15.5:0/3871092416 wait complete. 2023-07-21T19:14:22.374 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:14:22.374 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:14:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 -- 172.21.15.5:0/875204240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7980fea90 msgr2=0x7fa7980fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 --2- 172.21.15.5:0/875204240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7980fea90 0x7fa7980fee60 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fa788009990 tx=0x7fa788009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 -- 172.21.15.5:0/875204240 shutdown_connections 2023-07-21T19:14:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 --2- 172.21.15.5:0/875204240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa798103e40 0x7fa7981042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 --2- 172.21.15.5:0/875204240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa7980fea90 0x7fa7980fee60 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 --2- 172.21.15.5:0/875204240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 0x7fa798103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 -- 172.21.15.5:0/875204240 >> 172.21.15.5:0/875204240 conn(0x7fa7980fa490 msgr2=0x7fa7980fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.612+0000 7fa79ddff700 1 -- 172.21.15.5:0/875204240 shutdown_connections 2023-07-21T19:14:22.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.613+0000 7fa79ddff700 1 -- 172.21.15.5:0/875204240 wait complete. 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.614+0000 7fa79ddff700 1 Processor -- start 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.614+0000 7fa79ddff700 1 -- start start 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 0x7fa79819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 0x7fa79819c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a1210 con 0x7fa798103e40 2023-07-21T19:14:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a1380 con 0x7fa7980fea90 2023-07-21T19:14:22.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa79ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7981a1540 con 0x7fa7980ff3a0 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa797fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 0x7fa79819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa797fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60324/0 (socket says 172.21.15.5:60324) 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa7977fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 0x7fa79819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43674/0 (socket says 172.21.15.5:43674) 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa797fff700 1 -- 172.21.15.5:0/2519394910 learned_addr learned my addr 172.21.15.5:0/2519394910 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:22.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.615+0000 7fa796ffd700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 0x7fa79819c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 -- 172.21.15.5:0/2519394910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 msgr2=0x7fa79819c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 0x7fa79819c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 -- 172.21.15.5:0/2519394910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 msgr2=0x7fa79819c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 0x7fa79819c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 -- 172.21.15.5:0/2519394910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880095f0 con 0x7fa798103e40 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa797fff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa79000ef30 tx=0x7fa79000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa790018070 con 0x7fa798103e40 2023-07-21T19:14:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa79000f040 con 0x7fa798103e40 2023-07-21T19:14:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7981a17c0 con 0x7fa798103e40 2023-07-21T19:14:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.616+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa79000cbc0 con 0x7fa798103e40 2023-07-21T19:14:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.617+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7981a9060 con 0x7fa798103e40 2023-07-21T19:14:22.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.618+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa798067300 con 0x7fa798103e40 2023-07-21T19:14:22.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.620+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa79000cd40 con 0x7fa798103e40 2023-07-21T19:14:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.622+0000 7fa794ff9700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 0x7fa7800a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:22.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.622+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa7900cccf0 con 0x7fa798103e40 2023-07-21T19:14:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.622+0000 7fa7977fe700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 0x7fa7800a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:22.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.623+0000 7fa7977fe700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 0x7fa7800a3830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098e0 tx=0x7fa78800b2a0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:22.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.625+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fa790096190 con 0x7fa798103e40 2023-07-21T19:14:22.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.921+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa79819d580 con 0x7fa798103e40 2023-07-21T19:14:22.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.922+0000 7fa794ff9700 1 -- 172.21.15.5:0/2519394910 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+10842 (secure 0 0 0) 0x7fa7900997b0 con 0x7fa798103e40 2023-07-21T19:14:22.924 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:14:22.924 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":34,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:14:11.523937+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:14:22.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 msgr2=0x7fa7800a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 0x7fa7800a3830 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa7880098e0 tx=0x7fa78800b2a0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 msgr2=0x7fa7981a0bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7fa79000ef30 tx=0x7fa79000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 shutdown_connections 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7800a1380 0x7fa7800a3830 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa798103e40 0x7fa7981a0bd0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7980fea90 0x7fa79819c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 --2- 172.21.15.5:0/2519394910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7980ff3a0 0x7fa79819c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:22.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 >> 172.21.15.5:0/2519394910 conn(0x7fa7980fa490 msgr2=0x7fa798102960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 shutdown_connections 2023-07-21T19:14:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:22.924+0000 7fa79ddff700 1 -- 172.21.15.5:0/2519394910 wait complete. 2023-07-21T19:14:22.928 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 34 2023-07-21T19:14:22.946 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:14:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:24500000 of 50000000 tuples (49%) done (elapsed 380.74 s, remaining 396.28 s) 2023-07-21T19:14:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.196+0000 7f889efff700 1 -- 172.21.15.5:0/3437288570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898105ae0 msgr2=0x7f8898105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.196+0000 7f889efff700 1 --2- 172.21.15.5:0/3437288570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898105ae0 0x7f8898105eb0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f888c009990 tx=0x7f888c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 -- 172.21.15.5:0/3437288570 shutdown_connections 2023-07-21T19:14:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 --2- 172.21.15.5:0/3437288570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8898100480 0x7f889810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 --2- 172.21.15.5:0/3437288570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898105ae0 0x7f8898105eb0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 --2- 172.21.15.5:0/3437288570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f88980ffad0 0x7f88980fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 -- 172.21.15.5:0/3437288570 >> 172.21.15.5:0/3437288570 conn(0x7f88980fb5f0 msgr2=0x7f88980fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 -- 172.21.15.5:0/3437288570 shutdown_connections 2023-07-21T19:14:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.197+0000 7f889efff700 1 -- 172.21.15.5:0/3437288570 wait complete. 2023-07-21T19:14:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.198+0000 7f889efff700 1 Processor -- start 2023-07-21T19:14:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.198+0000 7f889efff700 1 -- start start 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.198+0000 7f889efff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 0x7f8898072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.198+0000 7f889efff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889efff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8898105ae0 0x7f88981a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889efff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8898073580 con 0x7f8898100480 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889efff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981a4240 con 0x7f8898105ae0 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889efff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88981a4380 con 0x7f88980ffad0 2023-07-21T19:14:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60346/0 (socket says 172.21.15.5:60346) 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 -- 172.21.15.5:0/312113248 learned_addr learned my addr 172.21.15.5:0/312113248 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889cd9b700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 0x7f8898072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889d59c700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8898105ae0 0x7f88981a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 -- 172.21.15.5:0/312113248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 msgr2=0x7f8898072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 0x7f8898072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 -- 172.21.15.5:0/312113248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8898105ae0 msgr2=0x7f88981a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8898105ae0 0x7f88981a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f8897fff700 1 -- 172.21.15.5:0/312113248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0095f0 con 0x7f8898100480 2023-07-21T19:14:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.199+0000 7f889cd9b700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 0x7f8898072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f8897fff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9a0 tx=0x7f888400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f8895ffb700 1 -- 172.21.15.5:0/312113248 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8884009790 con 0x7f8898100480 2023-07-21T19:14:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f8895ffb700 1 -- 172.21.15.5:0/312113248 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8884010430 con 0x7f8898100480 2023-07-21T19:14:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88981a45e0 con 0x7f8898100480 2023-07-21T19:14:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f8895ffb700 1 -- 172.21.15.5:0/312113248 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888400cea0 con 0x7f8898100480 2023-07-21T19:14:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.200+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88981a4ac0 con 0x7f8898100480 2023-07-21T19:14:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.204+0000 7f8895ffb700 1 -- 172.21.15.5:0/312113248 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f88840105a0 con 0x7f8898100480 2023-07-21T19:14:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.205+0000 7f8895ffb700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 0x7f88880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.205+0000 7f88b2966b80 1 Processor -- start 2023-07-21T19:14:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f8895ffb700 1 -- 172.21.15.5:0/312113248 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8884014070 con 0x7f8898100480 2023-07-21T19:14:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 -- start start 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f889cd9b700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 0x7f88880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f3c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f47590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118e42cb0 con 0x55a118f47ad0 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118e95a40 con 0x55a118f3cbe0 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f88b2966b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118e8eed0 con 0x55a118f3c2d0 2023-07-21T19:14:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f889d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f889d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60348/0 (socket says 172.21.15.5:60348) 2023-07-21T19:14:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f889d59c700 1 -- 172.21.15.5:0/1299279018 learned_addr learned my addr 172.21.15.5:0/1299279018 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.206+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f3c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f8897fff700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f47590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889cd9b700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 0x7f88880a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f888c006010 tx=0x7f888c00b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 -- 172.21.15.5:0/1299279018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 msgr2=0x55a118f3c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f3c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 -- 172.21.15.5:0/1299279018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 msgr2=0x55a118f47590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f47590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 -- 172.21.15.5:0/1299279018 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a118cbcde0 con 0x55a118f47ad0 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f3c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f889d59c700 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8890003f40 tx=0x7f8890009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e39caf74fa7c08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.207+0000 7f8882ffd700 1 -- 172.21.15.5:0/1299279018 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890019070 con 0x55a118f47ad0 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f8882ffd700 1 -- 172.21.15.5:0/1299279018 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f889000f040 con 0x55a118f47ad0 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f8882ffd700 1 -- 172.21.15.5:0/1299279018 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890007940 con 0x55a118f47ad0 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 -- 172.21.15.5:0/1299279018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 msgr2=0x55a118f49eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f8890003f40 tx=0x7f8890009ba0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 -- 172.21.15.5:0/1299279018 shutdown_connections 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f49eb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f3c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1299279018 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f47590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 -- 172.21.15.5:0/1299279018 >> 172.21.15.5:0/1299279018 conn(0x55a118f32da0 msgr2=0x55a118f386a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.208+0000 7f88b2966b80 1 -- 172.21.15.5:0/1299279018 shutdown_connections 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.209+0000 7f88b2966b80 1 -- 172.21.15.5:0/1299279018 wait complete. 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.209+0000 7f88b2966b80 1 -- start start 2023-07-21T19:14:23.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.209+0000 7f88b2966b80 1 Processor -- start 2023-07-21T19:14:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f474f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f44700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118e2eac0 con 0x55a118f47ad0 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118e2e7c0 con 0x55a118f3cbe0 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118d12920 con 0x55a118f3c2d0 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f88b2966b80 10 client.0 _ll_register_callbacks cb 0x55a118e2c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f889d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f889d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60350/0 (socket says 172.21.15.5:60350) 2023-07-21T19:14:23.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f889d59c700 1 -- 172.21.15.5:0/1086467241 learned_addr learned my addr 172.21.15.5:0/1086467241 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f8897fff700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f44700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.210+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f474f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 -- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 msgr2=0x55a118f474f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f474f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 -- 172.21.15.5:0/1086467241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 msgr2=0x55a118f44700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f44700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 -- 172.21.15.5:0/1086467241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8890003b70 con 0x55a118f47ad0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f8897fff700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f889d59c700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8890004de0 tx=0x7f88900095e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7192c3aceecc9a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f8877fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8890019040 con 0x55a118f47ad0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a118de84f0 con 0x55a118f47ad0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.211+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f88900079b0 con 0x55a118f47ad0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8890007b70 con 0x55a118f47ad0 2023-07-21T19:14:23.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 10 client.17052 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55a118ca5c60 con 0x55a118f47ad0 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a118de84f0 con 0x55a118f47ad0 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f8890015b70 con 0x55a118f47ad0 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 10 client.17052 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 10 client.17052 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 20 client.17052 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118f57410 0x55a118f59820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 4 client.17052 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.212+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a118e87eb0 con 0x55a118f57410 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.213+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118f57410 0x55a118f59820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.213+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118f57410 0x55a118f59820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e25957ef25d67266 server_cookie=9a0fec52ae8b3291 in_seq=0 out_seq=0 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.213+0000 7f8877fff700 10 client.17052 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.214+0000 7f8877fff700 1 -- 172.21.15.5:0/1086467241 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a118e87eb0 con 0x55a118f57410 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.214+0000 7f8877fff700 10 client.17052 handle_command_reply: tid=0 2023-07-21T19:14:23.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.214+0000 7f88b2966b80 1 client.17052 shutdown 2023-07-21T19:14:23.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 msgr2=0x55a118f44d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f8890004de0 tx=0x7f88900095e0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 shutdown_connections 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f47ad0 0x55a118f44d30 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3c2d0 0x55a118f474f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118f57410 0x55a118f59820 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1086467241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3cbe0 0x55a118f44700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 >> 172.21.15.5:0/1086467241 conn(0x55a118f32da0 msgr2=0x55a118f38290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f8877fff700 0 client.17052 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:14:23.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 shutdown_connections 2023-07-21T19:14:23.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 1 -- 172.21.15.5:0/1086467241 wait complete. 2023-07-21T19:14:23.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.215+0000 7f88b2966b80 20 client.17052 trim_cache size 0 max 16384 2023-07-21T19:14:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 Processor -- start 2023-07-21T19:14:23.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 -- start start 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 0x55a118f59420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f47ad0 0x55a118f3fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118f329a0 con 0x55a118f3cbe0 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118f3dde0 con 0x55a118f3c2d0 2023-07-21T19:14:23.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f88b2966b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118f45310 con 0x55a118f47ad0 2023-07-21T19:14:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.235+0000 7f8897fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60360/0 (socket says 172.21.15.5:60360) 2023-07-21T19:14:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 -- 172.21.15.5:0/2855698092 learned_addr learned my addr 172.21.15.5:0/2855698092 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f889cd9b700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 0x55a118f59420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f889d59c700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f47ad0 0x55a118f3fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 -- 172.21.15.5:0/2855698092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f47ad0 msgr2=0x55a118f3fa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f47ad0 0x55a118f3fa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 -- 172.21.15.5:0/2855698092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 msgr2=0x55a118f59420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 0x55a118f59420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 -- 172.21.15.5:0/2855698092 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a118f442c0 con 0x55a118f3cbe0 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f889cd9b700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 0x55a118f59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.236+0000 7f8897fff700 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8884099d40 tx=0x7f88840cc040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef15365376f23603 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.237+0000 7f88837fe700 1 -- 172.21.15.5:0/2855698092 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888409a040 con 0x55a118f3cbe0 2023-07-21T19:14:23.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.237+0000 7f88837fe700 1 -- 172.21.15.5:0/2855698092 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f888409e070 con 0x55a118f3cbe0 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.237+0000 7f88837fe700 1 -- 172.21.15.5:0/2855698092 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88840a2260 con 0x55a118f3cbe0 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.237+0000 7f88b2966b80 1 -- 172.21.15.5:0/2855698092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 msgr2=0x55a118f47020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.237+0000 7f88b2966b80 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f8884099d40 tx=0x7f88840cc040 comp rx=0 tx=0).stop 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 -- 172.21.15.5:0/2855698092 shutdown_connections 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f47ad0 0x55a118f3fa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f3c2d0 0x55a118f59420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 --2- 172.21.15.5:0/2855698092 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3cbe0 0x55a118f47020 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 -- 172.21.15.5:0/2855698092 >> 172.21.15.5:0/2855698092 conn(0x55a118f32da0 msgr2=0x55a118f5e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 -- 172.21.15.5:0/2855698092 shutdown_connections 2023-07-21T19:14:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 -- 172.21.15.5:0/2855698092 wait complete. 2023-07-21T19:14:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 -- start start 2023-07-21T19:14:23.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.238+0000 7f88b2966b80 1 Processor -- start 2023-07-21T19:14:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3cbe0 0x55a118f4ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 0x55a118f4cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118faa2d0 con 0x55a118f3c2d0 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118faa410 con 0x55a118f47ad0 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a118faa550 con 0x55a118f3cbe0 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f88b2966b80 10 client.0 _ll_register_callbacks cb 0x7f8884096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f889cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f889cd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60370/0 (socket says 172.21.15.5:60370) 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f889cd9b700 1 -- 172.21.15.5:0/1879006008 learned_addr learned my addr 172.21.15.5:0/1879006008 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.239+0000 7f8897fff700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3cbe0 0x55a118f4ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889d59c700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 0x55a118f4cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 -- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3cbe0 msgr2=0x55a118f4ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3cbe0 0x55a118f4ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 -- 172.21.15.5:0/1879006008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 msgr2=0x55a118f4cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 0x55a118f4cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 -- 172.21.15.5:0/1879006008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889810d210 con 0x55a118f3c2d0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889d59c700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 0x55a118f4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f889cd9b700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f888c009990 tx=0x7f888c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e5f78c22266ddb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f8877fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f888c034ad0 con 0x55a118f3c2d0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f889000f040 con 0x55a118f3c2d0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.240+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f888c034c30 con 0x55a118f3c2d0 2023-07-21T19:14:23.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f888c034df0 con 0x55a118f3c2d0 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 10 client.17058 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f88981a4ac0 con 0x55a118f3c2d0 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f889000f040 con 0x55a118f3c2d0 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f888c03a020 con 0x55a118f3c2d0 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 10 client.17058 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 10 client.17058 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 20 client.17058 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118fabb80 0x55a118fadf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 4 client.17058 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.241+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a118e4a050 con 0x55a118fabb80 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f8897fff700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118fabb80 0x55a118fadf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f8897fff700 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118fabb80 0x55a118fadf90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6f079943fba1d3ca server_cookie=e087f75bfae50b65 in_seq=0 out_seq=0 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f8877fff700 10 client.17058 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f8877fff700 1 -- 172.21.15.5:0/1879006008 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a118e4a050 con 0x55a118fabb80 2023-07-21T19:14:23.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f8877fff700 10 client.17058 handle_command_reply: tid=0 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f88b2966b80 1 client.17058 shutdown 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.242+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 msgr2=0x55a118f4c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f888c009990 tx=0x7f888c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 shutdown_connections 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55a118fabb80 0x55a118fadf90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a118f47ad0 0x55a118f4cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a118f3c2d0 0x55a118f4c4d0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 --2- 172.21.15.5:0/1879006008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a118f3cbe0 0x55a118f4ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 >> 172.21.15.5:0/1879006008 conn(0x55a118f32da0 msgr2=0x55a118f59420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f8877fff700 0 client.17058 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 shutdown_connections 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 1 -- 172.21.15.5:0/1879006008 wait complete. 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.243+0000 7f88b2966b80 20 client.17058 trim_cache size 0 max 16384 2023-07-21T19:14:23.252 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:14:23.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 msgr2=0x7f88880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 0x7f88880a3770 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f888c006010 tx=0x7f888c00b260 comp rx=0 tx=0).stop 2023-07-21T19:14:23.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 msgr2=0x7f8898072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f888400e9a0 tx=0x7f888400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 shutdown_connections 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8898105ae0 0x7f88981a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f88980ffad0 0x7f8898072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f88880a12c0 0x7f88880a3770 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 --2- 172.21.15.5:0/312113248 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8898100480 0x7f8898072d50 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 >> 172.21.15.5:0/312113248 conn(0x7f88980fb5f0 msgr2=0x7f88980fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 shutdown_connections 2023-07-21T19:14:23.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.244+0000 7f889efff700 1 -- 172.21.15.5:0/312113248 wait complete. 2023-07-21T19:14:23.265 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:14:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 -- 172.21.15.5:0/2924417676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc74072850 msgr2=0x7fcc74072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2924417676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc74072850 0x7fcc74072c20 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7fcc64009990 tx=0x7fcc64009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 -- 172.21.15.5:0/2924417676 shutdown_connections 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2924417676 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc7410c3a0 0x7fcc7410e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2924417676 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc74072850 0x7fcc74072c20 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2924417676 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 0x7fcc7410be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 -- 172.21.15.5:0/2924417676 >> 172.21.15.5:0/2924417676 conn(0x7fcc740fb600 msgr2=0x7fcc740fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 -- 172.21.15.5:0/2924417676 shutdown_connections 2023-07-21T19:14:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.500+0000 7fcc79fff700 1 -- 172.21.15.5:0/2924417676 wait complete. 2023-07-21T19:14:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.501+0000 7fcc79fff700 1 Processor -- start 2023-07-21T19:14:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 -- start start 2023-07-21T19:14:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 0x7fcc74198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc7410c3a0 0x7fcc7419c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419ce20 con 0x7fcc7410c3a0 2023-07-21T19:14:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419cf90 con 0x7fcc74072850 2023-07-21T19:14:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.502+0000 7fcc79fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc7419d150 con 0x7fcc74073160 2023-07-21T19:14:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc7410c3a0 0x7fcc7419c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43730/0 (socket says 172.21.15.5:43730) 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 -- 172.21.15.5:0/2185264410 learned_addr learned my addr 172.21.15.5:0/2185264410 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 0x7fcc74198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 -- 172.21.15.5:0/2185264410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 msgr2=0x7fcc74198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 0x7fcc74198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 -- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc7410c3a0 msgr2=0x7fcc7419c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc7410c3a0 0x7fcc7419c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 -- 172.21.15.5:0/2185264410 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640095f0 con 0x7fcc74072850 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.503+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005740 tx=0x7fcc64005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.504+0000 7fcc70ff9700 1 -- 172.21.15.5:0/2185264410 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6401d070 con 0x7fcc74072850 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.504+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc7419d3d0 con 0x7fcc74072850 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.504+0000 7fcc70ff9700 1 -- 172.21.15.5:0/2185264410 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc64022470 con 0x7fcc74072850 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.504+0000 7fcc70ff9700 1 -- 172.21.15.5:0/2185264410 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64017650 con 0x7fcc74072850 2023-07-21T19:14:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.504+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc741a4c70 con 0x7fcc74072850 2023-07-21T19:14:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.508+0000 7fcc70ff9700 1 -- 172.21.15.5:0/2185264410 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcc640225e0 con 0x7fcc74072850 2023-07-21T19:14:23.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.509+0000 7fcc8d8f8b80 1 Processor -- start 2023-07-21T19:14:23.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.509+0000 7fcc8d8f8b80 1 -- start start 2023-07-21T19:14:23.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a0640f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a06411eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0630acb0 con 0x561a0640fad0 2023-07-21T19:14:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a0635da40 con 0x561a064042d0 2023-07-21T19:14:23.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a06356ed0 con 0x561a06404be0 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc70ff9700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 0x7fcc5c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43732/0 (socket says 172.21.15.5:43732) 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc73fff700 1 -- 172.21.15.5:0/1910940788 learned_addr learned my addr 172.21.15.5:0/1910940788 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a06411eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.510+0000 7fcc737fe700 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a0640f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc70ff9700 1 -- 172.21.15.5:0/2185264410 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcc640cdfb0 con 0x7fcc74072850 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc73fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 0x7fcc5c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 -- 172.21.15.5:0/1910940788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 msgr2=0x561a064046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 -- 172.21.15.5:0/1910940788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 msgr2=0x561a06411eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a06411eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 -- 172.21.15.5:0/1910940788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a06184de0 con 0x561a06404be0 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc737fe700 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a0640f590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc640cb700 tx=0x7fcc64009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=58993088801f426f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc73fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 0x7fcc5c0a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c013c10 tx=0x7fcc6c004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.511+0000 7fcc59ffb700 1 -- 172.21.15.5:0/1910940788 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6409b110 con 0x561a06404be0 2023-07-21T19:14:23.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc59ffb700 1 -- 172.21.15.5:0/1910940788 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc6409f140 con 0x561a06404be0 2023-07-21T19:14:23.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc59ffb700 1 -- 172.21.15.5:0/1910940788 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc640228c0 con 0x561a06404be0 2023-07-21T19:14:23.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1910940788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 msgr2=0x561a0640f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a0640f590 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcc640cb700 tx=0x7fcc64009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1910940788 shutdown_connections 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a06411eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1910940788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a0640f590 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.512+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1910940788 >> 172.21.15.5:0/1910940788 conn(0x561a063fada0 msgr2=0x561a064006a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.513+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1910940788 shutdown_connections 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.513+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1910940788 wait complete. 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.513+0000 7fcc8d8f8b80 1 -- start start 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 Processor -- start 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a06404be0 0x561a064103e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a0640fad0 0x561a06410920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a062f6ac0 con 0x561a064042d0 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a062f67c0 con 0x561a06404be0 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.514+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a061da920 con 0x561a0640fad0 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc8d8f8b80 10 client.0 _ll_register_callbacks cb 0x561a062f4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60396/0 (socket says 172.21.15.5:60396) 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 -- 172.21.15.5:0/3270904143 learned_addr learned my addr 172.21.15.5:0/3270904143 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a06404be0 0x561a064103e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc73fff700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a0640fad0 0x561a06410920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 -- 172.21.15.5:0/3270904143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a0640fad0 msgr2=0x561a06410920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a0640fad0 0x561a06410920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 -- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a06404be0 msgr2=0x561a064103e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a06404be0 0x561a064103e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.515+0000 7fcc737fe700 1 -- 172.21.15.5:0/3270904143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc741a4c70 con 0x561a064042d0 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc737fe700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcc640cbc80 tx=0x7fcc64009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a91e6fc176243d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:23.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc64022ad0 con 0x561a064042d0 2023-07-21T19:14:23.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a062b04f0 con 0x561a064042d0 2023-07-21T19:14:23.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcc64089070 con 0x561a064042d0 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6409b110 con 0x561a064042d0 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc8d8f8b80 10 client.17070 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561a0616dc60 con 0x561a064042d0 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.516+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561a062b04f0 con 0x561a064042d0 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc6409a600 con 0x561a064042d0 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 10 client.17070 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 10 client.17070 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 20 client.17070 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a0641eb70 0x561a06420f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 4 client.17070 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a0634feb0 con 0x561a0641eb70 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.517+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a0641eb70 0x561a06420f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.518+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a0641eb70 0x561a06420f50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=24b0c8b091ccfd74 server_cookie=d932e33a53000db1 in_seq=0 out_seq=0 2023-07-21T19:14:23.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.518+0000 7fcc52ffd700 10 client.17070 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.518+0000 7fcc52ffd700 1 -- 172.21.15.5:0/3270904143 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a0634feb0 con 0x561a0641eb70 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.518+0000 7fcc52ffd700 10 client.17070 handle_command_reply: tid=0 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 client.17070 shutdown 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 msgr2=0x561a0640fea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fcc640cbc80 tx=0x7fcc64009310 comp rx=0 tx=0).stop 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 shutdown_connections 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a0640fad0 0x561a06410920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a064042d0 0x561a0640fea0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a0641eb70 0x561a06420f50 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/3270904143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a06404be0 0x561a064103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 >> 172.21.15.5:0/3270904143 conn(0x561a063fada0 msgr2=0x561a06400290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc52ffd700 0 client.17070 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 shutdown_connections 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/3270904143 wait complete. 2023-07-21T19:14:23.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.519+0000 7fcc8d8f8b80 20 client.17070 trim_cache size 0 max 16384 2023-07-21T19:14:23.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 Processor -- start 2023-07-21T19:14:23.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 -- start start 2023-07-21T19:14:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a064042d0 0x561a0641a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a0640fad0 0x561a0640a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a06405580 con 0x561a06404be0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a063fa970 con 0x561a0640fad0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a064061a0 con 0x561a064042d0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc72ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.538+0000 7fcc72ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60402/0 (socket says 172.21.15.5:60402) 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 -- 172.21.15.5:0/1135858860 learned_addr learned my addr 172.21.15.5:0/1135858860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc737fe700 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a064042d0 0x561a0641a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc73fff700 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a0640fad0 0x561a0640a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 -- 172.21.15.5:0/1135858860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a064042d0 msgr2=0x561a0641a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a064042d0 0x561a0641a660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 -- 172.21.15.5:0/1135858860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a0640fad0 msgr2=0x561a0640a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a0640fad0 0x561a0640a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 -- 172.21.15.5:0/1135858860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a061ab250 con 0x561a06404be0 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.539+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcc74199450 tx=0x7fcc68009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d081b502a561dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc5a7fc700 1 -- 172.21.15.5:0/1135858860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68016070 con 0x561a06404be0 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc5a7fc700 1 -- 172.21.15.5:0/1135858860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc68017040 con 0x561a06404be0 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc5a7fc700 1 -- 172.21.15.5:0/1135858860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc68010430 con 0x561a06404be0 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1135858860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 msgr2=0x561a06409de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fcc74199450 tx=0x7fcc68009260 comp rx=0 tx=0).stop 2023-07-21T19:14:23.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1135858860 shutdown_connections 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a0640fad0 0x561a0640a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a064042d0 0x561a0641a660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/1135858860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a06404be0 0x561a06409de0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.540+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1135858860 >> 172.21.15.5:0/1135858860 conn(0x561a063fada0 msgr2=0x561a06420b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.541+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1135858860 shutdown_connections 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.541+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/1135858860 wait complete. 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.541+0000 7fcc8d8f8b80 1 -- start start 2023-07-21T19:14:23.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.541+0000 7fcc8d8f8b80 1 Processor -- start 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064130b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a064135f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a06414bb0 con 0x561a0640fad0 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a06414d20 con 0x561a064042d0 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a06414ee0 con 0x561a06404be0 2023-07-21T19:14:23.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc8d8f8b80 10 client.0 _ll_register_callbacks cb 0x7fcc68009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc73fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60418/0 (socket says 172.21.15.5:60418) 2023-07-21T19:14:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.542+0000 7fcc73fff700 1 -- 172.21.15.5:0/2791532524 learned_addr learned my addr 172.21.15.5:0/2791532524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc72ffd700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a064135f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064130b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 -- 172.21.15.5:0/2791532524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 msgr2=0x561a064135f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a064135f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 -- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 msgr2=0x561a064130b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064130b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 -- 172.21.15.5:0/2791532524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc68005ce0 con 0x561a0640fad0 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc73fff700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c018d20 tx=0x7fcc6c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a54b3523440d313 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.543+0000 7fcc52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc6c021030 con 0x561a0640fad0 2023-07-21T19:14:23.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc6409f140 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcc6c007890 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc6c00e040 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc8d8f8b80 10 client.17076 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.544+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x561a061832b0 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc6409f140 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcc6c004690 con 0x561a0640fad0 2023-07-21T19:14:23.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 10 client.17076 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 10 client.17076 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 20 client.17076 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a06473c10 0x561a06476020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 4 client.17076 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561a06312050 con 0x561a06473c10 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.545+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a06473c10 0x561a06476020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc52ffd700 10 client.17076 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:14:23.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc737fe700 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a06473c10 0x561a06476020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=49ae246e68fc659e server_cookie=3353a150a1b98ab7 in_seq=0 out_seq=0 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc52ffd700 1 -- 172.21.15.5:0/2791532524 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561a06312050 con 0x561a06473c10 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc52ffd700 10 client.17076 handle_command_reply: tid=0 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 client.17076 shutdown 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 msgr2=0x561a064145e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c018d20 tx=0x7fcc6c004ad0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 shutdown_connections 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a0640fad0 0x561a064145e0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x561a06473c10 0x561a06476020 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a064042d0 0x561a064130b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 --2- 172.21.15.5:0/2791532524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a06404be0 0x561a064135f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 >> 172.21.15.5:0/2791532524 conn(0x561a063fada0 msgr2=0x561a06409de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.546+0000 7fcc52ffd700 0 client.17076 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 shutdown_connections 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc8d8f8b80 1 -- 172.21.15.5:0/2791532524 wait complete. 2023-07-21T19:14:23.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc8d8f8b80 20 client.17076 trim_cache size 0 max 16384 2023-07-21T19:14:23.556 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:14:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 msgr2=0x7fcc5c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 0x7fcc5c0a37f0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fcc6c013c10 tx=0x7fcc6c004b40 comp rx=0 tx=0).stop 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 msgr2=0x7fcc74197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fcc64005740 tx=0x7fcc64005770 comp rx=0 tx=0).stop 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 shutdown_connections 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc5c0a1340 0x7fcc5c0a37f0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc7410c3a0 0x7fcc7419c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc74072850 0x7fcc74197e40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 --2- 172.21.15.5:0/2185264410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc74073160 0x7fcc74198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.547+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 >> 172.21.15.5:0/2185264410 conn(0x7fcc740fb600 msgr2=0x7fcc7410e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.548+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 shutdown_connections 2023-07-21T19:14:23.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.548+0000 7fcc79fff700 1 -- 172.21.15.5:0/2185264410 wait complete. 2023-07-21T19:14:23.567 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:23 smithi005 ceph-mon[126460]: pgmap v208: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-21T19:14:23.568 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:23 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2519394910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:14:23.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:14:23.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:23 smithi156 ceph-mon[128316]: pgmap v208: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-21T19:14:23.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:23 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2519394910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:14:23.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 -- 172.21.15.5:0/569948500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 msgr2=0x7f739c10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 --2- 172.21.15.5:0/569948500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c10c3e0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f739400b0a0 tx=0x7f739400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 -- 172.21.15.5:0/569948500 shutdown_connections 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 --2- 172.21.15.5:0/569948500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c10c3e0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 --2- 172.21.15.5:0/569948500 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c100aa0 0x7f739c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 --2- 172.21.15.5:0/569948500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c1013b0 0x7f739c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 -- 172.21.15.5:0/569948500 >> 172.21.15.5:0/569948500 conn(0x7f739c0745d0 msgr2=0x7f739c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 -- 172.21.15.5:0/569948500 shutdown_connections 2023-07-21T19:14:23.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.821+0000 7f73a179d700 1 -- 172.21.15.5:0/569948500 wait complete. 2023-07-21T19:14:23.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 Processor -- start 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 -- start start 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c1013b0 0x7f739c195ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c19a930 con 0x7f739c109fc0 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c19aaa0 con 0x7f739c1013b0 2023-07-21T19:14:23.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.823+0000 7f73a179d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f739c19ac60 con 0x7f739c100aa0 2023-07-21T19:14:23.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60448/0 (socket says 172.21.15.5:60448) 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41416/0 (socket says 172.21.15.5:41416) 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f73a0f9c700 1 -- 172.21.15.5:0/475604447 learned_addr learned my addr 172.21.15.5:0/475604447 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739b7fe700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c1013b0 0x7f739c195ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 -- 172.21.15.5:0/475604447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c1013b0 msgr2=0x7f739c195ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c1013b0 0x7f739c195ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 -- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 msgr2=0x7f739c19a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f739bfff700 1 -- 172.21.15.5:0/475604447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7394009d00 con 0x7f739c100aa0 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.824+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f739bfff700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d950 tx=0x7f738c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f73997fa700 1 -- 172.21.15.5:0/475604447 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00f890 con 0x7f739c100aa0 2023-07-21T19:14:23.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f739c19af40 con 0x7f739c100aa0 2023-07-21T19:14:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f73997fa700 1 -- 172.21.15.5:0/475604447 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f738c00fe80 con 0x7f739c100aa0 2023-07-21T19:14:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f73997fa700 1 -- 172.21.15.5:0/475604447 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c00e3b0 con 0x7f739c100aa0 2023-07-21T19:14:23.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.825+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c1a27f0 con 0x7f739c100aa0 2023-07-21T19:14:23.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.829+0000 7f73997fa700 1 -- 172.21.15.5:0/475604447 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f738c00f9f0 con 0x7f739c100aa0 2023-07-21T19:14:23.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.830+0000 7f73997fa700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 0x7f73840a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 Processor -- start 2023-07-21T19:14:23.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73997fa700 1 -- 172.21.15.5:0/475604447 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f738c0cca80 con 0x7f739c100aa0 2023-07-21T19:14:23.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:23 smithi101 ceph-mon[128136]: pgmap v208: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 15 MiB/s wr, 32 op/s 2023-07-21T19:14:23.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:23 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2519394910' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:14:23.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f739b7fe700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 0x7f73840a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 -- start start 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 0x55597c447190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4476d0 0x55597c433360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c33dcb0 con 0x55597c446dc0 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c390a40 con 0x55597c4338a0 2023-07-21T19:14:23.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.831+0000 7f73b6260b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c389ed0 con 0x55597c4476d0 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f739bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 0x55597c447190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43794/0 (socket says 172.21.15.5:43794) 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 -- 172.21.15.5:0/762402993 learned_addr learned my addr 172.21.15.5:0/762402993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f739b7fe700 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4476d0 0x55597c433360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f739b7fe700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 0x7f73840a3810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f739c196fa0 tx=0x7f7390009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 -- 172.21.15.5:0/762402993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4476d0 msgr2=0x55597c433360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4476d0 0x55597c433360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 -- 172.21.15.5:0/762402993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 msgr2=0x55597c447190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 0x55597c447190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f73a0f9c700 1 -- 172.21.15.5:0/762402993 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597c1b7de0 con 0x55597c4338a0 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.832+0000 7f739bfff700 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 0x55597c447190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.833+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f739400b070 tx=0x7f7394009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae6d1f40428b889b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.833+0000 7f73827fc700 1 -- 172.21.15.5:0/762402993 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400e040 con 0x55597c4338a0 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.833+0000 7f73827fc700 1 -- 172.21.15.5:0/762402993 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f73940074e0 con 0x55597c4338a0 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.833+0000 7f73827fc700 1 -- 172.21.15.5:0/762402993 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940052a0 con 0x55597c4338a0 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 -- 172.21.15.5:0/762402993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 msgr2=0x55597c433cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f739400b070 tx=0x7f7394009c40 comp rx=0 tx=0).stop 2023-07-21T19:14:23.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 -- 172.21.15.5:0/762402993 shutdown_connections 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c433cf0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c446dc0 0x55597c447190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 --2- 172.21.15.5:0/762402993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4476d0 0x55597c433360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 -- 172.21.15.5:0/762402993 >> 172.21.15.5:0/762402993 conn(0x55597c42dff0 msgr2=0x55597c4389d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 -- 172.21.15.5:0/762402993 shutdown_connections 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.834+0000 7f73b6260b80 1 -- 172.21.15.5:0/762402993 wait complete. 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.835+0000 7f73b6260b80 1 -- start start 2023-07-21T19:14:23.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.835+0000 7f73b6260b80 1 Processor -- start 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.835+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 0x55597c44d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.835+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4476d0 0x55597c44b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73b6260b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c329ac0 con 0x55597c4338a0 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73b6260b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c3297c0 con 0x55597c4476d0 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73b6260b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c20d920 con 0x55597c446dc0 2023-07-21T19:14:23.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73b6260b80 10 client.0 _ll_register_callbacks cb 0x55597c327760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 0x55597c44d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41422/0 (socket says 172.21.15.5:41422) 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 0x55597c44d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60472/0 (socket says 172.21.15.5:60472) 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 -- 172.21.15.5:0/1809495829 learned_addr learned my addr 172.21.15.5:0/1809495829 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4476d0 0x55597c44b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 -- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4476d0 msgr2=0x55597c44b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4476d0 0x55597c44b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 -- 172.21.15.5:0/1809495829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 msgr2=0x55597c44d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 0x55597c44d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.836+0000 7f739b7fe700 1 -- 172.21.15.5:0/1809495829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c1a27f0 con 0x55597c446dc0 2023-07-21T19:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f739b7fe700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f739001c550 tx=0x7f73900047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9946f8931fb9bde8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f737b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7390021070 con 0x55597c446dc0 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55597c2e34f0 con 0x55597c446dc0 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7390011a10 con 0x55597c446dc0 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7390011bd0 con 0x55597c446dc0 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f73b6260b80 10 client.26731 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.837+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f739c10d0f0 con 0x55597c446dc0 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7390031020 con 0x55597c446dc0 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f739001cd20 con 0x55597c446dc0 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 10 client.26731 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 10 client.26731 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 20 client.26731 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c45af10 0x55597c45d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 4 client.26731 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.838+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55597c382eb0 con 0x55597c45af10 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.839+0000 7f739bfff700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c45af10 0x55597c45d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.839+0000 7f737b7fe700 10 client.26731 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.839+0000 7f739bfff700 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c45af10 0x55597c45d320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=419863fc713824b9 server_cookie=a291b4908d86d737 in_seq=0 out_seq=0 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.840+0000 7f737b7fe700 1 -- 172.21.15.5:0/1809495829 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55597c382eb0 con 0x55597c45af10 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.840+0000 7f737b7fe700 10 client.26731 handle_command_reply: tid=0 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.840+0000 7f73b6260b80 1 client.26731 shutdown 2023-07-21T19:14:23.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 msgr2=0x55597c44a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f739001c550 tx=0x7f73900047c0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 shutdown_connections 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4476d0 0x55597c44b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c45af10 0x55597c45d320 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4338a0 0x55597c44d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 --2- 172.21.15.5:0/1809495829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44a990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 >> 172.21.15.5:0/1809495829 conn(0x55597c42dff0 msgr2=0x55597c432b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f737b7fe700 0 client.26731 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 shutdown_connections 2023-07-21T19:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 1 -- 172.21.15.5:0/1809495829 wait complete. 2023-07-21T19:14:23.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.841+0000 7f73b6260b80 20 client.26731 trim_cache size 0 max 16384 2023-07-21T19:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.856+0000 7f73b6260b80 1 Processor -- start 2023-07-21T19:14:23.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.856+0000 7f73b6260b80 1 -- start start 2023-07-21T19:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c44d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44d780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c4578f0 con 0x55597c4476d0 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c434dd0 con 0x55597c4338a0 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73b6260b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c448f10 con 0x55597c446dc0 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60488/0 (socket says 172.21.15.5:60488) 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f73a0f9c700 1 -- 172.21.15.5:0/652060305 learned_addr learned my addr 172.21.15.5:0/652060305 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f739b7fe700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.857+0000 7f739bfff700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c44d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 -- 172.21.15.5:0/652060305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 msgr2=0x55597c44d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 -- 172.21.15.5:0/652060305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 msgr2=0x55597c44d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c44d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 -- 172.21.15.5:0/652060305 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55597c1a0c60 con 0x55597c4476d0 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f739bfff700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c44d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fd0 tx=0x7f7394000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2178103c1d3bf26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f7382ffd700 1 -- 172.21.15.5:0/652060305 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f739400e040 con 0x55597c4476d0 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f7382ffd700 1 -- 172.21.15.5:0/652060305 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7394007850 con 0x55597c4476d0 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.858+0000 7f7382ffd700 1 -- 172.21.15.5:0/652060305 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73940054b0 con 0x55597c4476d0 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 -- 172.21.15.5:0/652060305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 msgr2=0x55597c444b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f7394009fd0 tx=0x7f7394000f40 comp rx=0 tx=0).stop 2023-07-21T19:14:23.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 -- 172.21.15.5:0/652060305 shutdown_connections 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c444b90 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c4338a0 0x55597c44d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 --2- 172.21.15.5:0/652060305 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c446dc0 0x55597c44d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 -- 172.21.15.5:0/652060305 >> 172.21.15.5:0/652060305 conn(0x55597c42dff0 msgr2=0x55597c432060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 -- 172.21.15.5:0/652060305 shutdown_connections 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.859+0000 7f73b6260b80 1 -- 172.21.15.5:0/652060305 wait complete. 2023-07-21T19:14:23.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.860+0000 7f73b6260b80 1 -- start start 2023-07-21T19:14:23.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.860+0000 7f73b6260b80 1 Processor -- start 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.860+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c446dc0 0x55597c44e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c44ee10 con 0x55597c4476d0 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c4ae300 con 0x55597c446dc0 2023-07-21T19:14:23.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55597c4ae440 con 0x55597c4338a0 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73b6260b80 10 client.0 _ll_register_callbacks cb 0x7f7390012f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73a0f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60494/0 (socket says 172.21.15.5:60494) 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73a0f9c700 1 -- 172.21.15.5:0/679645725 learned_addr learned my addr 172.21.15.5:0/679645725 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41436/0 (socket says 172.21.15.5:41436) 2023-07-21T19:14:23.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739b7fe700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c446dc0 0x55597c44e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 -- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c446dc0 msgr2=0x55597c44e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c446dc0 0x55597c44e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 -- 172.21.15.5:0/679645725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 msgr2=0x55597c4addc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f739bfff700 1 -- 172.21.15.5:0/679645725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739c19af40 con 0x55597c4338a0 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.861+0000 7f73a0f9c700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f739bfff700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f738c0ca170 tx=0x7f738c0cc040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c266ff02e64797d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:23.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f737b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738c09db20 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f738c0a1f40 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f73940074e0 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f738c0b2170 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f73b6260b80 10 client.26737 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55597c1b7de0 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.862+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f738c063020 con 0x55597c4338a0 2023-07-21T19:14:23.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f738c0a1ab0 con 0x55597c4338a0 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 10 client.26737 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 10 client.26737 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 20 client.26737 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c4af990 0x55597c4b1da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 4 client.26737 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55597c345050 con 0x55597c4af990 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.863+0000 7f739b7fe700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c4af990 0x55597c4b1da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:23.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f737b7fe700 10 client.26737 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f739b7fe700 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c4af990 0x55597c4b1da0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7eaaacdbf3cc55c4 server_cookie=b3d3e722ea8a7ec4 in_seq=0 out_seq=0 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f737b7fe700 1 -- 172.21.15.5:0/679645725 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55597c345050 con 0x55597c4af990 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f737b7fe700 10 client.26737 handle_command_reply: tid=0 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f73b6260b80 1 client.26737 shutdown 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 msgr2=0x55597c44df90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.864+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f738c0ca170 tx=0x7f738c0cc040 comp rx=0 tx=0).stop 2023-07-21T19:14:23.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 shutdown_connections 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55597c4af990 0x55597c4b1da0 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55597c4476d0 0x55597c4addc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55597c4338a0 0x55597c44df90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 --2- 172.21.15.5:0/679645725 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55597c446dc0 0x55597c44e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f737b7fe700 0 client.26737 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 >> 172.21.15.5:0/679645725 conn(0x55597c42dff0 msgr2=0x55597c444b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 shutdown_connections 2023-07-21T19:14:23.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 1 -- 172.21.15.5:0/679645725 wait complete. 2023-07-21T19:14:23.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73b6260b80 20 client.26737 trim_cache size 0 max 16384 2023-07-21T19:14:23.874 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 msgr2=0x7f73840a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 0x7f73840a3810 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f739c196fa0 tx=0x7f7390009260 comp rx=0 tx=0).stop 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 msgr2=0x7f739c195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.865+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f738c00d950 tx=0x7f738c00dc60 comp rx=0 tx=0).stop 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 shutdown_connections 2023-07-21T19:14:23.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f73840a1360 0x7f73840a3810 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f739c109fc0 0x7f739c19a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f739c100aa0 0x7f739c195990 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 --2- 172.21.15.5:0/475604447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f739c1013b0 0x7f739c195ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 >> 172.21.15.5:0/475604447 conn(0x7f739c0745d0 msgr2=0x7f739c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 shutdown_connections 2023-07-21T19:14:23.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:23.866+0000 7f73a179d700 1 -- 172.21.15.5:0/475604447 wait complete. 2023-07-21T19:14:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:24600000 of 50000000 tuples (49%) done (elapsed 382.11 s, remaining 394.54 s) 2023-07-21T19:14:24.888 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:14:24.889 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:14:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 -- 172.21.15.5:0/13171205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c109fc0 msgr2=0x7f6b0c10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 --2- 172.21.15.5:0/13171205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c10c3e0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b0a0 tx=0x7f6b0400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 -- 172.21.15.5:0/13171205 shutdown_connections 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 --2- 172.21.15.5:0/13171205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c10c3e0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 --2- 172.21.15.5:0/13171205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 --2- 172.21.15.5:0/13171205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1013b0 0x7f6b0c109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 -- 172.21.15.5:0/13171205 >> 172.21.15.5:0/13171205 conn(0x7f6b0c0745d0 msgr2=0x7f6b0c0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 -- 172.21.15.5:0/13171205 shutdown_connections 2023-07-21T19:14:25.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.127+0000 7f6b1279d700 1 -- 172.21.15.5:0/13171205 wait complete. 2023-07-21T19:14:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 Processor -- start 2023-07-21T19:14:25.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 -- start start 2023-07-21T19:14:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1013b0 0x7f6b0c19deb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c1979e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c198050 con 0x7f6b0c100aa0 2023-07-21T19:14:25.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c1981c0 con 0x7f6b0c109fc0 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.129+0000 7f6b1279d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c198380 con 0x7f6b0c1013b0 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60512/0 (socket says 172.21.15.5:60512) 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 -- 172.21.15.5:0/3907765735 learned_addr learned my addr 172.21.15.5:0/3907765735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1013b0 0x7f6b0c19deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c1979e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 -- 172.21.15.5:0/3907765735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1013b0 msgr2=0x7f6b0c19deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1013b0 0x7f6b0c19deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 -- 172.21.15.5:0/3907765735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c109fc0 msgr2=0x7f6b0c1979e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c1979e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.130+0000 7f6b1179b700 1 -- 172.21.15.5:0/3907765735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b04009d00 con 0x7f6b0c100aa0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b1179b700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800d950 tx=0x7f6b0800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b027fc700 1 -- 172.21.15.5:0/3907765735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800f890 con 0x7f6b0c100aa0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c198660 con 0x7f6b0c100aa0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b027fc700 1 -- 172.21.15.5:0/3907765735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0800fe80 con 0x7f6b0c100aa0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b027fc700 1 -- 172.21.15.5:0/3907765735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0800e3b0 con 0x7f6b0c100aa0 2023-07-21T19:14:25.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.131+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1a4960 con 0x7f6b0c100aa0 2023-07-21T19:14:25.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.135+0000 7f6b027fc700 1 -- 172.21.15.5:0/3907765735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6b08010460 con 0x7f6b0c100aa0 2023-07-21T19:14:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.136+0000 7f6b272e3b80 1 Processor -- start 2023-07-21T19:14:25.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 -- start start 2023-07-21T19:14:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846b0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4845b5d30 con 0x55f4846b0570 2023-07-21T19:14:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f484608ac0 con 0x55f4846b0e80 2023-07-21T19:14:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f484601f50 con 0x55f4846badb0 2023-07-21T19:14:25.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.137+0000 7f6b027fc700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 0x7f6af80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b11f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846b0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41460/0 (socket says 172.21.15.5:41460) 2023-07-21T19:14:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3371213436 learned_addr learned my addr 172.21.15.5:0/3371213436 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b1179b700 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 0x7f6af80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b027fc700 1 -- 172.21.15.5:0/3907765735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6b080ccb20 con 0x7f6b0c100aa0 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3371213436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 msgr2=0x55f4846ba870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846ba870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3371213436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 msgr2=0x55f4846b0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846b0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3371213436 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f48442fe60 con 0x55f4846badb0 2023-07-21T19:14:25.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.138+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009ca0 tx=0x7f6afc011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd94abf0e0f8d651 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.139+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 0x7f6af80a3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b780 tx=0x7f6b04006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.139+0000 7f6af77fe700 1 -- 172.21.15.5:0/3371213436 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc011d80 con 0x55f4846badb0 2023-07-21T19:14:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.139+0000 7f6af77fe700 1 -- 172.21.15.5:0/3371213436 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc004cc0 con 0x55f4846badb0 2023-07-21T19:14:25.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.139+0000 7f6af77fe700 1 -- 172.21.15.5:0/3371213436 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc019430 con 0x55f4846badb0 2023-07-21T19:14:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3371213436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 msgr2=0x55f4846bd190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009ca0 tx=0x7f6afc011440 comp rx=0 tx=0).stop 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3371213436 shutdown_connections 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846bd190 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846b0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3371213436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846ba870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3371213436 >> 172.21.15.5:0/3371213436 conn(0x55f4846a6070 msgr2=0x55f4846ab9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3371213436 shutdown_connections 2023-07-21T19:14:25.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3371213436 wait complete. 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.140+0000 7f6b272e3b80 1 -- start start 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 Processor -- start 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846b0570 0x55f4846ba770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 0x55f4846b7f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4845a1b40 con 0x55f4846b0e80 2023-07-21T19:14:25.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4845a1840 con 0x55f4846badb0 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4844859a0 con 0x55f4846b0570 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.141+0000 7f6b272e3b80 10 client.0 _ll_register_callbacks cb 0x55f48459f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 0x55f4846b7f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60536/0 (socket says 172.21.15.5:60536) 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b10f9a700 1 -- 172.21.15.5:0/1972529279 learned_addr learned my addr 172.21.15.5:0/1972529279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846b0570 0x55f4846ba770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 -- 172.21.15.5:0/1972529279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846b0570 msgr2=0x55f4846ba770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846b0570 0x55f4846ba770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 -- 172.21.15.5:0/1972529279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 msgr2=0x55f4846b7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b11f9c700 1 -- 172.21.15.5:0/1972529279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc009900 con 0x55f4846badb0 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.142+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 0x55f4846b7f90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b070 tx=0x7f6b040099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9e827c60979a3b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6af4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400e070 con 0x55f4846badb0 2023-07-21T19:14:25.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f48455b570 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6b04009460 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b0401f640 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6b272e3b80 10 client.26786 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.143+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f484418ce0 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b04039080 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b04029070 con 0x55f4846badb0 2023-07-21T19:14:25.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 10 client.26786 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 10 client.26786 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 20 client.26786 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4846ca760 0x55f4846ccb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 4 client.26786 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.144+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4845faf30 con 0x55f4846ca760 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.145+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4846ca760 0x55f4846ccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.145+0000 7f6af4ff9700 10 client.26786 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.145+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4846ca760 0x55f4846ccb70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3aaf38512078bb8e server_cookie=72af1d33c291caea in_seq=0 out_seq=0 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1972529279 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4845faf30 con 0x55f4846ca760 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6af4ff9700 10 client.26786 handle_command_reply: tid=0 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 client.26786 shutdown 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 msgr2=0x55f4846b7f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 0x55f4846b7f90 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b070 tx=0x7f6b040099e0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 shutdown_connections 2023-07-21T19:14:25.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846badb0 0x55f4846b7f90 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846b0570 0x55f4846ba770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4846ca760 0x55f4846ccb70 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1972529279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f4846b7980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 >> 172.21.15.5:0/1972529279 conn(0x55f4846a6070 msgr2=0x55f4846ab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.146+0000 7f6af4ff9700 0 client.26786 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.147+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 shutdown_connections 2023-07-21T19:14:25.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.147+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1972529279 wait complete. 2023-07-21T19:14:25.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.147+0000 7f6b272e3b80 20 client.26786 trim_cache size 0 max 16384 2023-07-21T19:14:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 Processor -- start 2023-07-21T19:14:25.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 -- start start 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846b5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846b85d0 con 0x55f4846b0570 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846abf60 con 0x55f4846b0e80 2023-07-21T19:14:25.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.168+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846aa760 con 0x55f4846badb0 2023-07-21T19:14:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b11f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846b5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60550/0 (socket says 172.21.15.5:60550) 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b1179b700 1 -- 172.21.15.5:0/3907254338 learned_addr learned my addr 172.21.15.5:0/3907254338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43874/0 (socket says 172.21.15.5:43874) 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3907254338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 msgr2=0x55f4846b5130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846b5130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3907254338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 msgr2=0x55f4846ba2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b10f9a700 1 -- 172.21.15.5:0/3907254338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4844562d0 con 0x55f4846b0e80 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.169+0000 7f6b1179b700 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.170+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f4846b9030 tx=0x7f6afc009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3d5a5ffcfb7c840 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.170+0000 7f6af7fff700 1 -- 172.21.15.5:0/3907254338 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc005d60 con 0x55f4846b0e80 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.170+0000 7f6af7fff700 1 -- 172.21.15.5:0/3907254338 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc011d80 con 0x55f4846b0e80 2023-07-21T19:14:25.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.170+0000 7f6af7fff700 1 -- 172.21.15.5:0/3907254338 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc003b20 con 0x55f4846b0e80 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3907254338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 msgr2=0x55f4846b4bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x55f4846b9030 tx=0x7f6afc009710 comp rx=0 tx=0).stop 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3907254338 shutdown_connections 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846b5130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0570 0x55f4846ba2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/3907254338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0e80 0x55f4846b4bf0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3907254338 >> 172.21.15.5:0/3907254338 conn(0x55f4846a6070 msgr2=0x55f4846b2c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3907254338 shutdown_connections 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.171+0000 7f6b272e3b80 1 -- 172.21.15.5:0/3907254338 wait complete. 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.172+0000 7f6b272e3b80 1 -- start start 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.172+0000 7f6b272e3b80 1 Processor -- start 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f484718360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846c0d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846c1420 con 0x55f4846b0e80 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846c1590 con 0x55f4846b0570 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4846c1750 con 0x55f4846badb0 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b272e3b80 10 client.0 _ll_register_callbacks cb 0x55f48459f7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b10f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f484718360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b1179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43890/0 (socket says 172.21.15.5:43890) 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b1179b700 1 -- 172.21.15.5:0/1874207996 learned_addr learned my addr 172.21.15.5:0/1874207996 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.173+0000 7f6b11f9c700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846c0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 -- 172.21.15.5:0/1874207996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 msgr2=0x55f4846c0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846c0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 -- 172.21.15.5:0/1874207996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 msgr2=0x55f484718360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f484718360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 -- 172.21.15.5:0/1874207996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c1a4960 con 0x55f4846b0570 2023-07-21T19:14:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b1179b700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b08096600 tx=0x7f6b08099a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22955e276e5bd30b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6af4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:25.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080b2370 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6b0809dbc0 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.174+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6afc004cc0 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b080a1fe0 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 10 client.26792 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55f48442fe60 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b08065020 con 0x55f4846b0570 2023-07-21T19:14:25.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b08099180 con 0x55f4846b0570 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 10 client.26792 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 10 client.26792 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 20 client.26792 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f484720cf0 0x55f484723100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 4 client.26792 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f4845bd0d0 con 0x55f484720cf0 2023-07-21T19:14:25.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.175+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f484720cf0 0x55f484723100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.176+0000 7f6b10f9a700 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f484720cf0 0x55f484723100 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24445be77706f35 server_cookie=6adc8555bbfb012 in_seq=0 out_seq=0 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.176+0000 7f6af4ff9700 10 client.26792 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6af4ff9700 1 -- 172.21.15.5:0/1874207996 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f4845bd0d0 con 0x55f484720cf0 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6af4ff9700 10 client.26792 handle_command_reply: tid=0 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 client.26792 shutdown 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 msgr2=0x55f484717e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6b08096600 tx=0x7f6b08099a80 comp rx=0 tx=0).stop 2023-07-21T19:14:25.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 shutdown_connections 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4846badb0 0x55f4846c0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4846b0570 0x55f484717e20 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f484720cf0 0x55f484723100 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 --2- 172.21.15.5:0/1874207996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4846b0e80 0x55f484718360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6af4ff9700 0 client.26792 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 >> 172.21.15.5:0/1874207996 conn(0x55f4846a6070 msgr2=0x55f4846af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 shutdown_connections 2023-07-21T19:14:25.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.178+0000 7f6b272e3b80 1 -- 172.21.15.5:0/1874207996 wait complete. 2023-07-21T19:14:25.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b272e3b80 20 client.26792 trim_cache size 0 max 16384 2023-07-21T19:14:25.186 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:14:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:14:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:14:25.186 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:14:25.187 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:14:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 msgr2=0x7f6af80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 0x7f6af80a3810 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400b780 tx=0x7f6b04006210 comp rx=0 tx=0).stop 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 msgr2=0x7f6b0c19d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f6b0800d950 tx=0x7f6b0800dc60 comp rx=0 tx=0).stop 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 shutdown_connections 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c109fc0 0x7f6b0c1979e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1360 0x7f6af80a3810 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c100aa0 0x7f6b0c19d970 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 --2- 172.21.15.5:0/3907765735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1013b0 0x7f6b0c19deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.179+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 >> 172.21.15.5:0/3907765735 conn(0x7f6b0c0745d0 msgr2=0x7f6b0c0fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.180+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 shutdown_connections 2023-07-21T19:14:25.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.180+0000 7f6b1279d700 1 -- 172.21.15.5:0/3907765735 wait complete. 2023-07-21T19:14:25.206 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:14:25.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 -- 172.21.15.5:0/646982211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b00ffad0 msgr2=0x7fc9b00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/646982211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b00ffad0 0x7fc9b00fff40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009990 tx=0x7fc9a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 -- 172.21.15.5:0/646982211 shutdown_connections 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/646982211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0100480 0x7fc9b010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/646982211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/646982211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b00ffad0 0x7fc9b00fff40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 -- 172.21.15.5:0/646982211 >> 172.21.15.5:0/646982211 conn(0x7fc9b00fb5f0 msgr2=0x7fc9b00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 -- 172.21.15.5:0/646982211 shutdown_connections 2023-07-21T19:14:25.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.451+0000 7fc9b5bff700 1 -- 172.21.15.5:0/646982211 wait complete. 2023-07-21T19:14:25.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.452+0000 7fc9b5bff700 1 Processor -- start 2023-07-21T19:14:25.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 -- start start 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 0x7fc9b019be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0195fc0 con 0x7fc9b00ffad0 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b0196130 con 0x7fc9b0100480 2023-07-21T19:14:25.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9b5bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9b01962f0 con 0x7fc9b0105ae0 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 0x7fc9b019be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60586/0 (socket says 172.21.15.5:60586) 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.453+0000 7fc9af7fe700 1 -- 172.21.15.5:0/889878629 learned_addr learned my addr 172.21.15.5:0/889878629 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 -- 172.21.15.5:0/889878629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 msgr2=0x7fc9b019be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 0x7fc9b019be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 -- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 msgr2=0x7fc9b019b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 -- 172.21.15.5:0/889878629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9a40095f0 con 0x7fc9b0105ae0 2023-07-21T19:14:25.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 0x7fc9b019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:25.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:25.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.454+0000 7fc9affff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0195950 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800efc0 tx=0x7fc9a800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.455+0000 7fc9acff9700 1 -- 172.21.15.5:0/889878629 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a8018070 con 0x7fc9b0105ae0 2023-07-21T19:14:25.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.455+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9b0196570 con 0x7fc9b0105ae0 2023-07-21T19:14:25.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.455+0000 7fc9acff9700 1 -- 172.21.15.5:0/889878629 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9a800f040 con 0x7fc9b0105ae0 2023-07-21T19:14:25.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.455+0000 7fc9acff9700 1 -- 172.21.15.5:0/889878629 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a800cba0 con 0x7fc9b0105ae0 2023-07-21T19:14:25.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.455+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b01a28d0 con 0x7fc9b0105ae0 2023-07-21T19:14:25.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.459+0000 7fc9acff9700 1 -- 172.21.15.5:0/889878629 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc9a800cd00 con 0x7fc9b0105ae0 2023-07-21T19:14:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.460+0000 7fc9acff9700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 0x7fc9980a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9acff9700 1 -- 172.21.15.5:0/889878629 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc9a80cc7b0 con 0x7fc9b0105ae0 2023-07-21T19:14:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 Processor -- start 2023-07-21T19:14:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 0x7fc9980a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 -- start start 2023-07-21T19:14:25.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f72ee0 0x55b919f752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919e69cb0 con 0x55b919f632d0 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919ebca40 con 0x55b919f63be0 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.461+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919eb5ed0 con 0x55b919f72ee0 2023-07-21T19:14:25.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43914/0 (socket says 172.21.15.5:43914) 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9aeffd700 1 -- 172.21.15.5:0/3962121099 learned_addr learned my addr 172.21.15.5:0/3962121099 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 0x7fc9980a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0011550 tx=0x7fc9a0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9affff700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f72ee0 0x55b919f752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3962121099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f72ee0 msgr2=0x55b919f752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f72ee0 0x55b919f752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3962121099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 msgr2=0x55b919f729a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3962121099 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b919ce3de0 con 0x55b919f632d0 2023-07-21T19:14:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.462+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:25.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.463+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f636a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e5c0 tx=0x7fc9a0014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caab90ca60b83d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.463+0000 7fc995ffb700 1 -- 172.21.15.5:0/3962121099 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0026670 con 0x55b919f632d0 2023-07-21T19:14:25.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.463+0000 7fc995ffb700 1 -- 172.21.15.5:0/3962121099 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9a0021070 con 0x55b919f632d0 2023-07-21T19:14:25.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.463+0000 7fc995ffb700 1 -- 172.21.15.5:0/3962121099 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a001d9d0 con 0x55b919f632d0 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3962121099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 msgr2=0x55b919f636a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f636a0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fc9a000e5c0 tx=0x7fc9a0014000 comp rx=0 tx=0).stop 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3962121099 shutdown_connections 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f72ee0 0x55b919f752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f636a0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3962121099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f729a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3962121099 >> 172.21.15.5:0/3962121099 conn(0x55b919f59da0 msgr2=0x55b919f5f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3962121099 shutdown_connections 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.464+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3962121099 wait complete. 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 -- start start 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 Processor -- start 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f63be0 0x55b919f6fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 0x55b919f701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919e55ac0 con 0x55b919f632d0 2023-07-21T19:14:25.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919e557c0 con 0x55b919f72ee0 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919d39920 con 0x55b919f63be0 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.465+0000 7fc9c9442b80 10 client.0 _ll_register_callbacks cb 0x55b919e53760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60610/0 (socket says 172.21.15.5:60610) 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3172535937 learned_addr learned my addr 172.21.15.5:0/3172535937 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f63be0 0x55b919f6fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9affff700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 0x55b919f701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3172535937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f63be0 msgr2=0x55b919f6fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f63be0 0x55b919f6fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 msgr2=0x55b919f701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 0x55b919f701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9af7fe700 1 -- 172.21.15.5:0/3172535937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9b010d210 con 0x55b919f632d0 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.466+0000 7fc9affff700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 0x55b919f701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00046e0 tx=0x7fc9a0005510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576e94e35c28d48b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc98effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a0004950 con 0x55b919f632d0 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc9a000ee20 con 0x55b919f632d0 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b919e0f4f0 con 0x55b919f632d0 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a0004ab0 con 0x55b919f632d0 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc9c9442b80 10 client.17124 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:25.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.467+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7fc9a0017070 con 0x55b919f632d0 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9a003c080 con 0x55b919f632d0 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc9a0036030 con 0x55b919f632d0 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 10 client.17124 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 10 client.17124 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 20 client.17124 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919f827a0 0x55b919f84bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 4 client.17124 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b919eaeeb0 con 0x55b919f827a0 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.468+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919f827a0 0x55b919f84bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.469+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919f827a0 0x55b919f84bb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=162b946eb58e0d7b server_cookie=30614470c2245097 in_seq=0 out_seq=0 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.469+0000 7fc98effd700 10 client.17124 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc98effd700 1 -- 172.21.15.5:0/3172535937 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b919eaeeb0 con 0x55b919f827a0 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc98effd700 10 client.17124 handle_command_reply: tid=0 2023-07-21T19:14:25.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 client.17124 shutdown 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 msgr2=0x55b919f728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fc9a00046e0 tx=0x7fc9a0005510 comp rx=0 tx=0).stop 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 shutdown_connections 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f72ee0 0x55b919f701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919f827a0 0x55b919f84bb0 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f632d0 0x55b919f728c0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/3172535937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f63be0 0x55b919f6fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc98effd700 0 client.17124 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 >> 172.21.15.5:0/3172535937 conn(0x55b919f59da0 msgr2=0x55b919f5f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 shutdown_connections 2023-07-21T19:14:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 1 -- 172.21.15.5:0/3172535937 wait complete. 2023-07-21T19:14:25.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.470+0000 7fc9c9442b80 20 client.17124 trim_cache size 0 max 16384 2023-07-21T19:14:25.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 Processor -- start 2023-07-21T19:14:25.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 -- start start 2023-07-21T19:14:25.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f84b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f72460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f662f0 con 0x55b919f72ee0 2023-07-21T19:14:25.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f6b730 con 0x55b919f63be0 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.492+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f7f130 con 0x55b919f632d0 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9affff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f72460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43924/0 (socket says 172.21.15.5:43924) 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 -- 172.21.15.5:0/684133971 learned_addr learned my addr 172.21.15.5:0/684133971 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f84b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 -- 172.21.15.5:0/684133971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 msgr2=0x55b919f84b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f84b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 -- 172.21.15.5:0/684133971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 msgr2=0x55b919f72460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f72460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 -- 172.21.15.5:0/684133971 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b919cccc60 con 0x55b919f63be0 2023-07-21T19:14:25.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.493+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009990 tx=0x7fc9a400b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=933fa58a221dda79 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9967fc700 1 -- 172.21.15.5:0/684133971 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a401d040 con 0x55b919f63be0 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9967fc700 1 -- 172.21.15.5:0/684133971 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9a401e040 con 0x55b919f63be0 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9967fc700 1 -- 172.21.15.5:0/684133971 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a40056a0 con 0x55b919f63be0 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 -- 172.21.15.5:0/684133971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 msgr2=0x55b919f7ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4009990 tx=0x7fc9a400b260 comp rx=0 tx=0).stop 2023-07-21T19:14:25.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 -- 172.21.15.5:0/684133971 shutdown_connections 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f72460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f84b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/684133971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f7ebf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.494+0000 7fc9c9442b80 1 -- 172.21.15.5:0/684133971 >> 172.21.15.5:0/684133971 conn(0x55b919f59da0 msgr2=0x55b919f89510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.495+0000 7fc9c9442b80 1 -- 172.21.15.5:0/684133971 shutdown_connections 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.495+0000 7fc9c9442b80 1 -- 172.21.15.5:0/684133971 wait complete. 2023-07-21T19:14:25.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.495+0000 7fc9c9442b80 1 -- start start 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.495+0000 7fc9c9442b80 1 Processor -- start 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f798a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f780a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f78670 con 0x55b919f72ee0 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f787b0 con 0x55b919f63be0 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b919f78970 con 0x55b919f632d0 2023-07-21T19:14:25.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9c9442b80 10 client.0 _ll_register_callbacks cb 0x7fc9a0005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9aeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43938/0 (socket says 172.21.15.5:43938) 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9aeffd700 1 -- 172.21.15.5:0/1187308646 learned_addr learned my addr 172.21.15.5:0/1187308646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9affff700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f780a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.496+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f798a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 -- 172.21.15.5:0/1187308646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 msgr2=0x55b919f798a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f798a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 -- 172.21.15.5:0/1187308646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 msgr2=0x55b919f780a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f780a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 -- 172.21.15.5:0/1187308646 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9a400be80 con 0x55b919f63be0 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9affff700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f798a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9aeffd700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c00 tx=0x7fc9a4029c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b842a467dd678842 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc98effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9a401d040 con 0x55b919f63be0 2023-07-21T19:14:25.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc9a40050e0 con 0x55b919f63be0 2023-07-21T19:14:25.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.497+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9a0021070 con 0x55b919f63be0 2023-07-21T19:14:25.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9a4029620 con 0x55b919f63be0 2023-07-21T19:14:25.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 10 client.26810 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:25.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55b919d0a250 con 0x55b919f63be0 2023-07-21T19:14:25.506 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc9a0021070 con 0x55b919f63be0 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fc9a4038020 con 0x55b919f63be0 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 10 client.26810 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 10 client.26810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 20 client.26810 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919fd77b0 0x55b919fd9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 4 client.26810 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:14:25.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.498+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b919e71050 con 0x55b919fd77b0 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919fd77b0 0x55b919fd9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc9af7fe700 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919fd77b0 0x55b919fd9bc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69d53611324e08b7 server_cookie=ee7bfc54e1fa3655 in_seq=0 out_seq=0 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc98effd700 10 client.26810 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc98effd700 1 -- 172.21.15.5:0/1187308646 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b919e71050 con 0x55b919fd77b0 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc98effd700 10 client.26810 handle_command_reply: tid=0 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.499+0000 7fc9c9442b80 1 client.26810 shutdown 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 msgr2=0x55b919f77b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fc9a4000c00 tx=0x7fc9a4029c10 comp rx=0 tx=0).stop 2023-07-21T19:14:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 shutdown_connections 2023-07-21T19:14:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b919fd77b0 0x55b919fd9bc0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b919f72ee0 0x55b919f780a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b919f632d0 0x55b919f798a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 --2- 172.21.15.5:0/1187308646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b919f63be0 0x55b919f77b60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc98effd700 0 client.26810 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 >> 172.21.15.5:0/1187308646 conn(0x55b919f59da0 msgr2=0x55b919f652e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 shutdown_connections 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 1 -- 172.21.15.5:0/1187308646 wait complete. 2023-07-21T19:14:25.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.500+0000 7fc9c9442b80 20 client.26810 trim_cache size 0 max 16384 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:14:25.511 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:14:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:14:25.512 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:14:25.512 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:14:25.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 msgr2=0x7fc9980a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 0x7fc9980a3770 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc9a0011550 tx=0x7fc9a0009260 comp rx=0 tx=0).stop 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 msgr2=0x7fc9b0195950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0195950 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc9a800efc0 tx=0x7fc9a800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 shutdown_connections 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9b0105ae0 0x7fc9b0195950 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9980a12c0 0x7fc9980a3770 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9b00ffad0 0x7fc9b019b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 --2- 172.21.15.5:0/889878629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9b0100480 0x7fc9b019be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 >> 172.21.15.5:0/889878629 conn(0x7fc9b00fb5f0 msgr2=0x7fc9b00fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 shutdown_connections 2023-07-21T19:14:25.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:25.501+0000 7fc9b5bff700 1 -- 172.21.15.5:0/889878629 wait complete. 2023-07-21T19:14:25.523 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:14:25.582 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-mon[128136]: pgmap v209: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-21T19:14:25.582 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:25.583 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:25.583 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:25.583 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:25.583 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:14:25.203+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...8,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v2460 ap=2 f(v8 m2023-07-21T19:13:32.128478+0000 3=1+2) n(v4 rc2023-07-21T19:13:32.128478+0000 b8 4=1+3) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@80} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v2460>,<0x10000000005/data v1657>,<0x10000000003/postgres v445>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v462>,<0x10000000001/sv_0 v480>,<0x10000000000/_nogroup v462>,<0x1/volumes v436>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:13:32.114929+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:13:32.128478+0000 b8 4=1+3)","memory_value.dirstat":"f(v8 m2023-07-21T19:13:32.128478+0000 3=1+2)","memory_value.rstat":"n(v4 rc2023-07-21T19:13:32.128478+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:14:25.583 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:14:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:14:25.204+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...a,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v2663 ap=2 f(v60 m2023-07-21T19:14:02.426131+0000 3=3+0) n(v6 rc2023-07-21T19:14:02.426131+0000 b3321 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@158} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v2663>,<0x10000000005/data v1657>,<0x10000000003/postgres v445>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v462>,<0x10000000001/sv_0 v480>,<0x10000000000/_nogroup v462>,<0x1/volumes v436>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:14:02.413002+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:14:02.426131+0000 b3321 4=3+1)","memory_value.dirstat":"f(v60 m2023-07-21T19:14:02.426131+0000 3=3+0)","memory_value.rstat":"n(v6 rc2023-07-21T19:14:02.426131+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:14:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:25 smithi005 ceph-mon[126460]: pgmap v209: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-21T19:14:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:25 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:25 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:25.660 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:25.671 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:25 smithi156 ceph-mon[128316]: pgmap v209: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 14 MiB/s wr, 28 op/s 2023-07-21T19:14:25.671 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:25 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:25.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:25.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:25 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:25.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:24700000 of 50000000 tuples (49%) done (elapsed 383.58 s, remaining 392.89 s) 2023-07-21T19:14:26.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:14:26.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:26.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:14:26.672 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.673 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.673 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:26 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:14:26.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:26.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:14:26.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.833 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:26 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:14:26.911 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:26 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:14:27.289 INFO:teuthology.orchestra.run.smithi005.stderr:24800000 of 50000000 tuples (49%) done (elapsed 384.85 s, remaining 391.06 s) 2023-07-21T19:14:27.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:27 smithi101 ceph-mon[128136]: pgmap v210: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-21T19:14:27.910 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:27 smithi005 ceph-mon[126460]: pgmap v210: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-21T19:14:27.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:27 smithi156 ceph-mon[128316]: pgmap v210: 97 pgs: 97 active+clean; 5.6 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 13 MiB/s wr, 28 op/s 2023-07-21T19:14:28.567 INFO:teuthology.orchestra.run.smithi005.stderr:24900000 of 50000000 tuples (49%) done (elapsed 386.13 s, remaining 389.23 s) 2023-07-21T19:14:29.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:29 smithi101 ceph-mon[128136]: pgmap v211: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 24 MiB/s wr, 132 op/s 2023-07-21T19:14:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 -- 172.21.15.5:0/4030218874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 msgr2=0x7f5bc0104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/4030218874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0104720 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f5bb40099c0 tx=0x7f5bb4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 -- 172.21.15.5:0/4030218874 shutdown_connections 2023-07-21T19:14:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/4030218874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/4030218874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/4030218874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0104720 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 -- 172.21.15.5:0/4030218874 >> 172.21.15.5:0/4030218874 conn(0x7f5bc00fb650 msgr2=0x7f5bc00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 -- 172.21.15.5:0/4030218874 shutdown_connections 2023-07-21T19:14:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.880+0000 7f5bc57ff700 1 -- 172.21.15.5:0/4030218874 wait complete. 2023-07-21T19:14:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.881+0000 7f5bc57ff700 1 Processor -- start 2023-07-21T19:14:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.881+0000 7f5bc57ff700 1 -- start start 2023-07-21T19:14:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc019cfc0 con 0x7f5bc0100760 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc019d130 con 0x7f5bc0104e00 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bc57ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc019d2f0 con 0x7f5bc00ffdc0 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bbe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34624/0 (socket says 172.21.15.5:34624) 2023-07-21T19:14:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bbf7fe700 1 -- 172.21.15.5:0/3687792613 learned_addr learned my addr 172.21.15.5:0/3687792613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.882+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 -- 172.21.15.5:0/3687792613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 msgr2=0x7f5bc0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0197dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 -- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 msgr2=0x7f5bc0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 -- 172.21.15.5:0/3687792613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb40095f0 con 0x7f5bc0104e00 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800ef30 tx=0x7f5bb800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5ba7fff700 1 -- 172.21.15.5:0/3687792613 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8018070 con 0x7f5bc0104e00 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc019d570 con 0x7f5bc0104e00 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5ba7fff700 1 -- 172.21.15.5:0/3687792613 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bb800f040 con 0x7f5bc0104e00 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5ba7fff700 1 -- 172.21.15.5:0/3687792613 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb800cc30 con 0x7f5bc0104e00 2023-07-21T19:14:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.883+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc01a4e10 con 0x7f5bc0104e00 2023-07-21T19:14:29.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.887+0000 7f5ba7fff700 1 -- 172.21.15.5:0/3687792613 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5bb8007500 con 0x7f5bc0104e00 2023-07-21T19:14:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.888+0000 7f5bd90acb80 1 Processor -- start 2023-07-21T19:14:29.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.888+0000 7f5bd90acb80 1 -- start start 2023-07-21T19:14:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558000f7b6b0 0x558001124d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55800111f580 0x55800111f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5ba7fff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 0x7f5ba80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001024c50 con 0x5580011252d0 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580010779e0 con 0x55800111f580 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001070e70 con 0x558000f7b6b0 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5ba7fff700 1 -- 172.21.15.5:0/3687792613 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5bb80cc840 con 0x7f5bc0104e00 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40084/0 (socket says 172.21.15.5:40084) 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1976338446 learned_addr learned my addr 172.21.15.5:0/1976338446 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:29.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 0x7f5ba80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55800111f580 0x55800111f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558000f7b6b0 0x558001124d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1976338446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558000f7b6b0 msgr2=0x558001124d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558000f7b6b0 0x558001124d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1976338446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55800111f580 msgr2=0x55800111f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.889+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55800111f580 0x55800111f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1976338446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558000e9ed80 con 0x5580011252d0 2023-07-21T19:14:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009de0 tx=0x7f5bb00116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e7d75b783825510 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 0x7f5ba80a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009990 tx=0x7f5bb40058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5ba57fa700 1 -- 172.21.15.5:0/1976338446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0011e20 con 0x5580011252d0 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5ba57fa700 1 -- 172.21.15.5:0/1976338446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bb00128e0 con 0x5580011252d0 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.890+0000 7f5ba57fa700 1 -- 172.21.15.5:0/1976338446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb000f7b0 con 0x5580011252d0 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1976338446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 msgr2=0x55800111f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f5bb0009de0 tx=0x7f5bb00116c0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1976338446 shutdown_connections 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55800111f580 0x55800111f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558000f7b6b0 0x558001124d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1976338446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x55800111f040 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1976338446 >> 172.21.15.5:0/1976338446 conn(0x558001114be0 msgr2=0x558001119f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1976338446 shutdown_connections 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.891+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1976338446 wait complete. 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.892+0000 7f5bd90acb80 1 -- start start 2023-07-21T19:14:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.892+0000 7f5bd90acb80 1 Processor -- start 2023-07-21T19:14:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x5580011287c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x558001128dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001010a60 con 0x55800111f580 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001010760 con 0x558000f7b6b0 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558000ef48c0 con 0x5580011252d0 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bd90acb80 10 client.0 _ll_register_callbacks cb 0x55800100e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34652/0 (socket says 172.21.15.5:34652) 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1551834713 learned_addr learned my addr 172.21.15.5:0/1551834713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x5580011287c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.893+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x558001128dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1551834713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 msgr2=0x558001128dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x558001128dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1551834713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 msgr2=0x5580011287c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x5580011287c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 -- 172.21.15.5:0/1551834713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bb0009a10 con 0x558000f7b6b0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bb001c040 tx=0x7f5bb0012c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c94d7109513da693 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.894+0000 7f5ba2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb0012660 con 0x558000f7b6b0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558000fca490 con 0x558000f7b6b0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5bb0004040 con 0x558000f7b6b0 2023-07-21T19:14:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb0004200 con 0x558000f7b6b0 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5bd90acb80 10 client.26822 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x558000e87c00 con 0x558000f7b6b0 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558000fca490 con 0x558000f7b6b0 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.895+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5bb0022070 con 0x558000f7b6b0 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 10 client.26822 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 10 client.26822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 20 client.26822 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558001138af0 0x55800113af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 4 client.26822 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558001069e50 con 0x558001138af0 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.896+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558001138af0 0x55800113af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.897+0000 7f5ba2ff5700 10 client.26822 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.897+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558001138af0 0x55800113af00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84da6317a7d7ff1c server_cookie=7d8a6b459ee99d41 in_seq=0 out_seq=0 2023-07-21T19:14:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.897+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/1551834713 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558001069e50 con 0x558001138af0 2023-07-21T19:14:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.897+0000 7f5ba2ff5700 10 client.26822 handle_command_reply: tid=0 2023-07-21T19:14:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.897+0000 7f5bd90acb80 1 client.26822 shutdown 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 msgr2=0x55800112b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5bb001c040 tx=0x7f5bb0012c80 comp rx=0 tx=0).stop 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 shutdown_connections 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x558001128dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112b5b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558001138af0 0x55800113af00 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1551834713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x5580011287c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 >> 172.21.15.5:0/1551834713 conn(0x558001114be0 msgr2=0x5580011194e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5ba2ff5700 0 client.26822 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 shutdown_connections 2023-07-21T19:14:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1551834713 wait complete. 2023-07-21T19:14:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.898+0000 7f5bd90acb80 20 client.26822 trim_cache size 0 max 16384 2023-07-21T19:14:29.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:29 smithi005 ceph-mon[126460]: pgmap v211: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 24 MiB/s wr, 132 op/s 2023-07-21T19:14:29.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.916+0000 7f5bd90acb80 1 Processor -- start 2023-07-21T19:14:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 -- start start 2023-07-21T19:14:29.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x55800112fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001129900 con 0x55800111f580 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580011256a0 con 0x558000f7b6b0 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001129400 con 0x5580011252d0 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40094/0 (socket says 172.21.15.5:40094) 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbe7fc700 1 -- 172.21.15.5:0/1570513662 learned_addr learned my addr 172.21.15.5:0/1570513662 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34662/0 (socket says 172.21.15.5:34662) 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.917+0000 7f5bbf7fe700 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x55800112fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 -- 172.21.15.5:0/1570513662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 msgr2=0x55800112fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x55800112fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 -- 172.21.15.5:0/1570513662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 msgr2=0x55800111f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 -- 172.21.15.5:0/1570513662 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55800113faf0 con 0x55800111f580 2023-07-21T19:14:29.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:29.922 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:29 smithi156 ceph-mon[128316]: pgmap v211: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 24 MiB/s wr, 132 op/s 2023-07-21T19:14:29.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4005950 tx=0x7f5bb4020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5181bb72726d20cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/1570513662 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb402c070 con 0x55800111f580 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/1570513662 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bb402d040 con 0x55800111f580 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.918+0000 7f5ba5ffb700 1 -- 172.21.15.5:0/1570513662 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb40053c0 con 0x55800111f580 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1570513662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 msgr2=0x558001128770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4005950 tx=0x7f5bb4020000 comp rx=0 tx=0).stop 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1570513662 shutdown_connections 2023-07-21T19:14:29.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5580011252d0 0x55800112fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800111f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/1570513662 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55800111f580 0x558001128770 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1570513662 >> 172.21.15.5:0/1570513662 conn(0x558001114be0 msgr2=0x558001128060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1570513662 shutdown_connections 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.919+0000 7f5bd90acb80 1 -- 172.21.15.5:0/1570513662 wait complete. 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 -- start start 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 Processor -- start 2023-07-21T19:14:29.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55800111f580 0x558001126e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x558001127340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001127880 con 0x5580011252d0 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5580011279f0 con 0x558000f7b6b0 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558001127bb0 con 0x55800111f580 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.920+0000 7f5bd90acb80 10 client.0 _ll_register_callbacks cb 0x7f5bb8095c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:29.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x558001127340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34676/0 (socket says 172.21.15.5:34676) 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 -- 172.21.15.5:0/18762577 learned_addr learned my addr 172.21.15.5:0/18762577 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55800111f580 0x558001126e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 -- 172.21.15.5:0/18762577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55800111f580 msgr2=0x558001126e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55800111f580 0x558001126e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 -- 172.21.15.5:0/18762577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 msgr2=0x558001127340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x558001127340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.921+0000 7f5bbeffd700 1 -- 172.21.15.5:0/18762577 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558000e87c00 con 0x558000f7b6b0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5bbeffd700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00059d0 tx=0x7f5bb0011fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=740ea3f58a23c4c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5ba2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb001acd0 con 0x558000f7b6b0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bb00128e0 con 0x558000f7b6b0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5bb0000e40 con 0x558000f7b6b0 2023-07-21T19:14:29.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bb001ae30 con 0x558000f7b6b0 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5bd90acb80 10 client.26828 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f5bc010d210 con 0x558000f7b6b0 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.922+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5bb00128e0 con 0x558000f7b6b0 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f5bb0022030 con 0x558000f7b6b0 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 10 client.26828 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 10 client.26828 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:29.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 20 client.26828 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55800118daa0 0x55800118fe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 4 client.26828 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:14:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55800102bff0 con 0x55800118daa0 2023-07-21T19:14:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.923+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55800118daa0 0x55800118fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:29.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.924+0000 7f5ba2ff5700 10 client.26828 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:29.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.924+0000 7f5bbe7fc700 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55800118daa0 0x55800118fe80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8f6665d8dc3698 server_cookie=6a191c5484c32300 in_seq=0 out_seq=0 2023-07-21T19:14:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.935+0000 7f5ba2ff5700 1 -- 172.21.15.5:0/18762577 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55800102bff0 con 0x55800118daa0 2023-07-21T19:14:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.935+0000 7f5ba2ff5700 10 client.26828 handle_command_reply: tid=0 2023-07-21T19:14:29.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.935+0000 7f5bd90acb80 1 client.26828 shutdown 2023-07-21T19:14:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 msgr2=0x55800112fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f5bb00059d0 tx=0x7f5bb0011fa0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 shutdown_connections 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55800118daa0 0x55800118fe80 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5580011252d0 0x558001127340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558000f7b6b0 0x55800112fc70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 --2- 172.21.15.5:0/18762577 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55800111f580 0x558001126e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 >> 172.21.15.5:0/18762577 conn(0x558001114be0 msgr2=0x558001126190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5ba2ff5700 0 client.26828 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 shutdown_connections 2023-07-21T19:14:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 1 -- 172.21.15.5:0/18762577 wait complete. 2023-07-21T19:14:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.936+0000 7f5bd90acb80 20 client.26828 trim_cache size 0 max 16384 2023-07-21T19:14:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:14:29.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:14:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:14:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:14:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 msgr2=0x7f5ba80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 0x7f5ba80a3770 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f5bb4009990 tx=0x7f5bb40058e0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 msgr2=0x7f5bc019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5bb800ef30 tx=0x7f5bb800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 shutdown_connections 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5bc0104e00 0x7f5bc019c980 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5bc00ffdc0 0x7f5bc0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ba80a12c0 0x7f5ba80a3770 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 --2- 172.21.15.5:0/3687792613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5bc0100760 0x7f5bc0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 >> 172.21.15.5:0/3687792613 conn(0x7f5bc00fb650 msgr2=0x7f5bc00fc160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 shutdown_connections 2023-07-21T19:14:29.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:29.937+0000 7f5bc57ff700 1 -- 172.21.15.5:0/3687792613 wait complete. 2023-07-21T19:14:29.975 INFO:teuthology.orchestra.run.smithi005.stderr:25000000 of 50000000 tuples (50%) done (elapsed 387.54 s, remaining 387.54 s) 2023-07-21T19:14:31.243 INFO:teuthology.orchestra.run.smithi005.stderr:25100000 of 50000000 tuples (50%) done (elapsed 388.81 s, remaining 385.71 s) 2023-07-21T19:14:31.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:31 smithi101 ceph-mon[128136]: pgmap v212: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 22 MiB/s wr, 120 op/s 2023-07-21T19:14:31.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:31 smithi005 ceph-mon[126460]: pgmap v212: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 22 MiB/s wr, 120 op/s 2023-07-21T19:14:31.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:31 smithi156 ceph-mon[128316]: pgmap v212: 97 pgs: 97 active+clean; 5.7 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 108 KiB/s rd, 22 MiB/s wr, 120 op/s 2023-07-21T19:14:32.547 INFO:teuthology.orchestra.run.smithi005.stderr:25200000 of 50000000 tuples (50%) done (elapsed 390.11 s, remaining 383.92 s) 2023-07-21T19:14:33.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:33 smithi101 ceph-mon[128136]: pgmap v213: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 132 op/s 2023-07-21T19:14:33.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:33 smithi005 ceph-mon[126460]: pgmap v213: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 132 op/s 2023-07-21T19:14:33.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:33 smithi156 ceph-mon[128316]: pgmap v213: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 132 op/s 2023-07-21T19:14:33.947 INFO:teuthology.orchestra.run.smithi005.stderr:25300000 of 50000000 tuples (50%) done (elapsed 391.51 s, remaining 382.23 s) 2023-07-21T19:14:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:25400000 of 50000000 tuples (50%) done (elapsed 392.84 s, remaining 380.47 s) 2023-07-21T19:14:35.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:35 smithi101 ceph-mon[128136]: pgmap v214: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 122 op/s 2023-07-21T19:14:35.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:35 smithi005 ceph-mon[126460]: pgmap v214: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 122 op/s 2023-07-21T19:14:35.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:35 smithi156 ceph-mon[128316]: pgmap v214: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 116 KiB/s rd, 24 MiB/s wr, 122 op/s 2023-07-21T19:14:37.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:37 smithi101 ceph-mon[128136]: pgmap v215: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 118 op/s 2023-07-21T19:14:37.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:37 smithi005 ceph-mon[126460]: pgmap v215: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 118 op/s 2023-07-21T19:14:37.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:37 smithi156 ceph-mon[128316]: pgmap v215: 97 pgs: 97 active+clean; 5.8 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 115 KiB/s rd, 22 MiB/s wr, 118 op/s 2023-07-21T19:14:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:25500000 of 50000000 tuples (51%) done (elapsed 396.67 s, remaining 381.12 s) 2023-07-21T19:14:39.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:39 smithi101 ceph-mon[128136]: pgmap v216: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 31 MiB/s wr, 134 op/s 2023-07-21T19:14:39.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:39 smithi005 ceph-mon[126460]: pgmap v216: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 31 MiB/s wr, 134 op/s 2023-07-21T19:14:39.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:39 smithi156 ceph-mon[128316]: pgmap v216: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 131 KiB/s rd, 31 MiB/s wr, 134 op/s 2023-07-21T19:14:40.443 INFO:teuthology.orchestra.run.smithi005.stderr:25600000 of 50000000 tuples (51%) done (elapsed 398.01 s, remaining 379.35 s) 2023-07-21T19:14:41.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:41 smithi101 ceph-mon[128136]: pgmap v217: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 30 op/s 2023-07-21T19:14:41.896 INFO:teuthology.orchestra.run.smithi005.stderr:25700000 of 50000000 tuples (51%) done (elapsed 399.46 s, remaining 377.70 s) 2023-07-21T19:14:41.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:41 smithi005 ceph-mon[126460]: pgmap v217: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 30 op/s 2023-07-21T19:14:41.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:41 smithi156 ceph-mon[128316]: pgmap v217: 97 pgs: 97 active+clean; 5.9 GiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 20 MiB/s wr, 30 op/s 2023-07-21T19:14:43.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 -- 172.21.15.5:0/1922610353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c0ffb00 msgr2=0x7f927c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 --2- 172.21.15.5:0/1922610353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c0ffb00 0x7f927c0fff70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9270009990 tx=0x7f9270009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 -- 172.21.15.5:0/1922610353 shutdown_connections 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 --2- 172.21.15.5:0/1922610353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 --2- 172.21.15.5:0/1922610353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c105b10 0x7f927c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 --2- 172.21.15.5:0/1922610353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c0ffb00 0x7f927c0fff70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.112+0000 7f92809ff700 1 -- 172.21.15.5:0/1922610353 >> 172.21.15.5:0/1922610353 conn(0x7f927c0fb600 msgr2=0x7f927c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.113+0000 7f92809ff700 1 -- 172.21.15.5:0/1922610353 shutdown_connections 2023-07-21T19:14:43.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.113+0000 7f92809ff700 1 -- 172.21.15.5:0/1922610353 wait complete. 2023-07-21T19:14:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.114+0000 7f92809ff700 1 Processor -- start 2023-07-21T19:14:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.114+0000 7f92809ff700 1 -- start start 2023-07-21T19:14:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.114+0000 7f92809ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f92809ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f92809ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c105b10 0x7f927c1a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f92809ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c1a4290 con 0x7f927c1004b0 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f92809ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c1a43d0 con 0x7f927c105b10 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f92809ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927c1a4540 con 0x7f927c0ffb00 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f9279d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f9279d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36558/0 (socket says 172.21.15.5:36558) 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48248/0 (socket says 172.21.15.5:48248) 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f9279d9b700 1 -- 172.21.15.5:0/1456033321 learned_addr learned my addr 172.21.15.5:0/1456033321 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927ad9d700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c105b10 0x7f927c1a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 -- 172.21.15.5:0/1456033321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c105b10 msgr2=0x7f927c1a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c105b10 0x7f927c1a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 -- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 msgr2=0x7f927c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f927a59c700 1 -- 172.21.15.5:0/1456033321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92700095f0 con 0x7f927c0ffb00 2023-07-21T19:14:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.115+0000 7f9279d9b700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f927a59c700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d950 tx=0x7f926c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f926b7fe700 1 -- 172.21.15.5:0/1456033321 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00f890 con 0x7f927c0ffb00 2023-07-21T19:14:43.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f926b7fe700 1 -- 172.21.15.5:0/1456033321 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f926c00fe80 con 0x7f927c0ffb00 2023-07-21T19:14:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f926b7fe700 1 -- 172.21.15.5:0/1456033321 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00e3b0 con 0x7f927c0ffb00 2023-07-21T19:14:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f927c1a4820 con 0x7f927c0ffb00 2023-07-21T19:14:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.116+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c1a4ce0 con 0x7f927c0ffb00 2023-07-21T19:14:43.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.120+0000 7f926b7fe700 1 -- 172.21.15.5:0/1456033321 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f926c00f9f0 con 0x7f927c0ffb00 2023-07-21T19:14:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.121+0000 7f92942b4b80 1 Processor -- start 2023-07-21T19:14:43.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 -- start start 2023-07-21T19:14:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7d9e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7ca3c50 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7cf69e0 con 0x55c9a7da42d0 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f92942b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7cefe70 con 0x55c9a7d9e580 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f926b7fe700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 0x7f92640a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f927a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f927a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53216/0 (socket says 172.21.15.5:53216) 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f927a59c700 1 -- 172.21.15.5:0/624066687 learned_addr learned my addr 172.21.15.5:0/624066687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f926b7fe700 1 -- 172.21.15.5:0/1456033321 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f926c0cc970 con 0x7f927c0ffb00 2023-07-21T19:14:43.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f927ad9d700 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.122+0000 7f9279d9b700 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7d9e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927ad9d700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 0x7f92640a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 -- 172.21.15.5:0/624066687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 msgr2=0x55c9a7d9e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7d9e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 -- 172.21.15.5:0/624066687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 msgr2=0x55c9a7da3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 -- 172.21.15.5:0/624066687 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a7b1dd80 con 0x55c9a7da42d0 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927a59c700 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f926c000c00 tx=0x7f926c064e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82fc66f485a73214 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f9268ff9700 1 -- 172.21.15.5:0/624066687 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c0106d0 con 0x55c9a7da42d0 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f927ad9d700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 0x7f92640a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9274009ec0 tx=0x7f927400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f9268ff9700 1 -- 172.21.15.5:0/624066687 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f926c00fca0 con 0x55c9a7da42d0 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.123+0000 7f9268ff9700 1 -- 172.21.15.5:0/624066687 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c0104d0 con 0x55c9a7da42d0 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 -- 172.21.15.5:0/624066687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 msgr2=0x55c9a7d9e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f926c000c00 tx=0x7f926c064e30 comp rx=0 tx=0).stop 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 -- 172.21.15.5:0/624066687 shutdown_connections 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7d9e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 --2- 172.21.15.5:0/624066687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7d9e040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 -- 172.21.15.5:0/624066687 >> 172.21.15.5:0/624066687 conn(0x55c9a7d93be0 msgr2=0x55c9a7d98f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 -- 172.21.15.5:0/624066687 shutdown_connections 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.124+0000 7f92942b4b80 1 -- 172.21.15.5:0/624066687 wait complete. 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.125+0000 7f92942b4b80 1 -- start start 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.125+0000 7f92942b4b80 1 Processor -- start 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da7940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7da7e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7c8fa60 con 0x55c9a7da42d0 2023-07-21T19:14:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7c8f760 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7b738c0 con 0x55c9a7d9e580 2023-07-21T19:14:43.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f92942b4b80 10 client.0 _ll_register_callbacks cb 0x55c9a7c8d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f927ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f927a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f927ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36580/0 (socket says 172.21.15.5:36580) 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f927ad9d700 1 -- 172.21.15.5:0/3319513080 learned_addr learned my addr 172.21.15.5:0/3319513080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.126+0000 7f9279d9b700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7da7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 -- 172.21.15.5:0/3319513080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 msgr2=0x55c9a7da7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7da7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 -- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 msgr2=0x55c9a7da7940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da7940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 -- 172.21.15.5:0/3319513080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f927c1a4ce0 con 0x55c9a7da42d0 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927a59c700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f927ad9d700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9274003f80 tx=0x7f9274004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617e21c4b9befc8c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f9261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:43.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9274022070 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c9a7c49490 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.127+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9274004240 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f927400e070 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 10 client.17160 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55c9a7b06c00 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9274033020 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f927401ba60 con 0x55c9a7da42d0 2023-07-21T19:14:43.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 10 client.17160 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 10 client.17160 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 20 client.17160 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7db7c70 0x55c9a7dba080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 4 client.17160 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.128+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c9a7ce8e50 con 0x55c9a7db7c70 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.129+0000 7f927a59c700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7db7c70 0x55c9a7dba080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.129+0000 7f927a59c700 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7db7c70 0x55c9a7dba080 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b5ddbfa90b47c49 server_cookie=af8e2c888020b8 in_seq=0 out_seq=0 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.129+0000 7f9261ffb700 10 client.17160 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f9261ffb700 1 -- 172.21.15.5:0/3319513080 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c9a7ce8e50 con 0x55c9a7db7c70 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f9261ffb700 10 client.17160 handle_command_reply: tid=0 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 client.17160 shutdown 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 msgr2=0x55c9a7da83c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f9274003f80 tx=0x7f9274004ad0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 shutdown_connections 2023-07-21T19:14:43.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7db7c70 0x55c9a7dba080 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7da42d0 0x55c9a7da83c0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7da7940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f9261ffb700 0 client.17160 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3319513080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7d9e580 0x55c9a7da7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.130+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 >> 172.21.15.5:0/3319513080 conn(0x55c9a7d93be0 msgr2=0x55c9a7d97c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.131+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 shutdown_connections 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.131+0000 7f92942b4b80 1 -- 172.21.15.5:0/3319513080 wait complete. 2023-07-21T19:14:43.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.131+0000 7f92942b4b80 20 client.17160 trim_cache size 0 max 16384 2023-07-21T19:14:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.149+0000 7f92942b4b80 1 Processor -- start 2023-07-21T19:14:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.149+0000 7f92942b4b80 1 -- start start 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7d9e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7da81f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7db52c0 con 0x55c9a7d9e580 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7d955f0 con 0x55c9a7da42d0 2023-07-21T19:14:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f92942b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7da5ea0 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f9279d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f927ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7da81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f9279d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36592/0 (socket says 172.21.15.5:36592) 2023-07-21T19:14:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f9279d9b700 1 -- 172.21.15.5:0/1740310390 learned_addr learned my addr 172.21.15.5:0/1740310390 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.150+0000 7f927a59c700 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7d9e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 -- 172.21.15.5:0/1740310390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7bfa6b0 msgr2=0x55c9a7d9e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7d9e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 -- 172.21.15.5:0/1740310390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 msgr2=0x55c9a7da81f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7da81f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 -- 172.21.15.5:0/1740310390 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c9a7b441f0 con 0x55c9a7d9e580 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f927ad9d700 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7da81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f9279d9b700 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f927000bd00 tx=0x7f927000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=260348772b0881b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f92697fa700 1 -- 172.21.15.5:0/1740310390 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f927001d030 con 0x55c9a7d9e580 2023-07-21T19:14:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f92697fa700 1 -- 172.21.15.5:0/1740310390 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f927001e040 con 0x55c9a7d9e580 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.151+0000 7f92697fa700 1 -- 172.21.15.5:0/1740310390 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270004b50 con 0x55c9a7d9e580 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 -- 172.21.15.5:0/1740310390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 msgr2=0x55c9a7daec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f927000bd00 tx=0x7f927000b260 comp rx=0 tx=0).stop 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 -- 172.21.15.5:0/1740310390 shutdown_connections 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7da42d0 0x55c9a7da81f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7d9e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 --2- 172.21.15.5:0/1740310390 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7d9e580 0x55c9a7daec60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 -- 172.21.15.5:0/1740310390 >> 172.21.15.5:0/1740310390 conn(0x55c9a7d93be0 msgr2=0x55c9a7da6980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 -- 172.21.15.5:0/1740310390 shutdown_connections 2023-07-21T19:14:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.152+0000 7f92942b4b80 1 -- 172.21.15.5:0/1740310390 wait complete. 2023-07-21T19:14:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.153+0000 7f92942b4b80 1 -- start start 2023-07-21T19:14:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.153+0000 7f92942b4b80 1 Processor -- start 2023-07-21T19:14:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7d9e580 0x55c9a7daebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7da42d0 0x55c9a7dacc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7dad260 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7dad3d0 con 0x55c9a7d9e580 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c9a7dad590 con 0x55c9a7da42d0 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f92942b4b80 10 client.0 _ll_register_callbacks cb 0x7f9274008410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36604/0 (socket says 172.21.15.5:36604) 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 -- 172.21.15.5:0/3626021776 learned_addr learned my addr 172.21.15.5:0/3626021776 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f9279d9b700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7d9e580 0x55c9a7daebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927ad9d700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7da42d0 0x55c9a7dacc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 -- 172.21.15.5:0/3626021776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7da42d0 msgr2=0x55c9a7dacc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7da42d0 0x55c9a7dacc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 -- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7d9e580 msgr2=0x55c9a7daebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7d9e580 0x55c9a7daebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.154+0000 7f927a59c700 1 -- 172.21.15.5:0/3626021776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c9a7b06c00 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f927a59c700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f926c005950 tx=0x7f926c010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad0a11d4ac0ba39c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f9261ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c099520 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f926c09ae40 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f926c00fca0 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f926c0a1e30 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f92942b4b80 10 client.17166 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f927c10d240 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.155+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f926c00fca0 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f926c0ca2e0 con 0x55c9a7bfa6b0 2023-07-21T19:14:43.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 10 client.17166 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 10 client.17166 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 20 client.17166 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7e0ca00 0x55c9a7e0ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 4 client.17166 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c9a7caaff0 con 0x55c9a7e0ca00 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.156+0000 7f9279d9b700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7e0ca00 0x55c9a7e0ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.157+0000 7f9279d9b700 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7e0ca00 0x55c9a7e0ede0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=91f4589339f77665 server_cookie=4b0566ae731c4d7e in_seq=0 out_seq=0 2023-07-21T19:14:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.157+0000 7f9261ffb700 10 client.17166 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:43.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f9261ffb700 1 -- 172.21.15.5:0/3626021776 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c9a7caaff0 con 0x55c9a7e0ca00 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f9261ffb700 10 client.17166 handle_command_reply: tid=0 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 client.17166 shutdown 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 msgr2=0x55c9a7dae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f926c005950 tx=0x7f926c010f90 comp rx=0 tx=0).stop 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 shutdown_connections 2023-07-21T19:14:43.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c9a7da42d0 0x55c9a7dacc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c9a7e0ca00 0x55c9a7e0ede0 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c9a7bfa6b0 0x55c9a7dae6b0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 --2- 172.21.15.5:0/3626021776 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c9a7d9e580 0x55c9a7daebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 >> 172.21.15.5:0/3626021776 conn(0x55c9a7d93be0 msgr2=0x55c9a7da2ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f9261ffb700 0 client.17166 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 shutdown_connections 2023-07-21T19:14:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.170+0000 7f92942b4b80 1 -- 172.21.15.5:0/3626021776 wait complete. 2023-07-21T19:14:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92942b4b80 20 client.17166 trim_cache size 0 max 16384 2023-07-21T19:14:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:14:43.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:14:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:14:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:14:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 msgr2=0x7f92640a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 0x7f92640a37f0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9274009ec0 tx=0x7f927400b040 comp rx=0 tx=0).stop 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 msgr2=0x7f927c0729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f926c00d950 tx=0x7f926c00dc60 comp rx=0 tx=0).stop 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 shutdown_connections 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f927c105b10 0x7f927c1a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f927c0ffb00 0x7f927c0729f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92640a1340 0x7f92640a37f0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 --2- 172.21.15.5:0/1456033321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f927c1004b0 0x7f927c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.171+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 >> 172.21.15.5:0/1456033321 conn(0x7f927c0fb600 msgr2=0x7f927c0fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.172+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 shutdown_connections 2023-07-21T19:14:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:43.172+0000 7f92809ff700 1 -- 172.21.15.5:0/1456033321 wait complete. 2023-07-21T19:14:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:25800000 of 50000000 tuples (51%) done (elapsed 400.81 s, remaining 375.95 s) 2023-07-21T19:14:43.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:43 smithi101 ceph-mon[128136]: pgmap v218: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 27 MiB/s wr, 59 op/s 2023-07-21T19:14:43.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:43 smithi005 ceph-mon[126460]: pgmap v218: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 27 MiB/s wr, 59 op/s 2023-07-21T19:14:43.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:43 smithi156 ceph-mon[128316]: pgmap v218: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 27 MiB/s wr, 59 op/s 2023-07-21T19:14:44.564 INFO:teuthology.orchestra.run.smithi005.stderr:25900000 of 50000000 tuples (51%) done (elapsed 402.13 s, remaining 374.18 s) 2023-07-21T19:14:45.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:45 smithi101 ceph-mon[128136]: pgmap v219: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 53 op/s 2023-07-21T19:14:45.866 INFO:teuthology.orchestra.run.smithi005.stderr:26000000 of 50000000 tuples (52%) done (elapsed 403.43 s, remaining 372.40 s) 2023-07-21T19:14:45.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:45 smithi005 ceph-mon[126460]: pgmap v219: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 53 op/s 2023-07-21T19:14:45.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:45 smithi156 ceph-mon[128316]: pgmap v219: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 18 MiB/s wr, 53 op/s 2023-07-21T19:14:47.262 INFO:teuthology.orchestra.run.smithi005.stderr:26100000 of 50000000 tuples (52%) done (elapsed 404.83 s, remaining 370.70 s) 2023-07-21T19:14:47.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:47 smithi101 ceph-mon[128136]: pgmap v220: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 51 op/s 2023-07-21T19:14:47.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:47 smithi005 ceph-mon[126460]: pgmap v220: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 51 op/s 2023-07-21T19:14:47.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:47 smithi156 ceph-mon[128316]: pgmap v220: 97 pgs: 97 active+clean; 5.9 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 17 MiB/s wr, 51 op/s 2023-07-21T19:14:48.558 INFO:teuthology.orchestra.run.smithi005.stderr:26200000 of 50000000 tuples (52%) done (elapsed 406.12 s, remaining 368.92 s) 2023-07-21T19:14:49.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:49 smithi101 ceph-mon[128136]: pgmap v221: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:14:49.885 INFO:teuthology.orchestra.run.smithi005.stderr:26300000 of 50000000 tuples (52%) done (elapsed 407.45 s, remaining 367.17 s) 2023-07-21T19:14:49.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:49 smithi005 ceph-mon[126460]: pgmap v221: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:14:49.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:49 smithi156 ceph-mon[128316]: pgmap v221: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 80 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:14:51.310 INFO:teuthology.orchestra.run.smithi005.stderr:26400000 of 50000000 tuples (52%) done (elapsed 408.88 s, remaining 365.51 s) 2023-07-21T19:14:51.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:51 smithi101 ceph-mon[128136]: pgmap v222: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 48 op/s 2023-07-21T19:14:51.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:51 smithi005 ceph-mon[126460]: pgmap v222: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 48 op/s 2023-07-21T19:14:51.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:51 smithi156 ceph-mon[128316]: pgmap v222: 97 pgs: 97 active+clean; 6.0 GiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 19 MiB/s wr, 48 op/s 2023-07-21T19:14:52.638 INFO:teuthology.orchestra.run.smithi005.stderr:26500000 of 50000000 tuples (53%) done (elapsed 410.20 s, remaining 363.77 s) 2023-07-21T19:14:53.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:53 smithi101 ceph-mon[128136]: pgmap v223: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 74 op/s 2023-07-21T19:14:53.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:14:53.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:53 smithi005 ceph-mon[126460]: pgmap v223: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 74 op/s 2023-07-21T19:14:53.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:14:53.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:53 smithi156 ceph-mon[128316]: pgmap v223: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 30 MiB/s wr, 74 op/s 2023-07-21T19:14:53.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:14:54.112 INFO:teuthology.orchestra.run.smithi005.stderr:26600000 of 50000000 tuples (53%) done (elapsed 411.68 s, remaining 362.15 s) 2023-07-21T19:14:54.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:54 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:14:54.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:54 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:14:54.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:14:54.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:54 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:14:54.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:54 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:14:54.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:14:54.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:54 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:14:54.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:54 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:14:54.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:14:55.523 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:14:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.731+0000 7f117e9ff700 1 -- 172.21.15.5:0/3450716220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 msgr2=0x7f11780fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.731+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3450716220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f11780fff40 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f116c009990 tx=0x7f116c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 -- 172.21.15.5:0/3450716220 shutdown_connections 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3450716220 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178100480 0x7f117810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3450716220 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178105ae0 0x7f1178105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3450716220 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f11780fff40 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 -- 172.21.15.5:0/3450716220 >> 172.21.15.5:0/3450716220 conn(0x7f11780fb5f0 msgr2=0x7f11780fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 -- 172.21.15.5:0/3450716220 shutdown_connections 2023-07-21T19:14:55.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.732+0000 7f117e9ff700 1 -- 172.21.15.5:0/3450716220 wait complete. 2023-07-21T19:14:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 Processor -- start 2023-07-21T19:14:55.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 -- start start 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 0x7f117819e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178105ae0 0x7f1178197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178198300 con 0x7f11780ffad0 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178198470 con 0x7f1178105ae0 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.733+0000 7f117e9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1178198630 con 0x7f1178100480 2023-07-21T19:14:55.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36998/0 (socket says 172.21.15.5:36998) 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 -- 172.21.15.5:0/3614419488 learned_addr learned my addr 172.21.15.5:0/3614419488 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f117cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178105ae0 0x7f1178197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f11777fe700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 0x7f117819e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 -- 172.21.15.5:0/3614419488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 msgr2=0x7f117819e090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 0x7f117819e090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 -- 172.21.15.5:0/3614419488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178105ae0 msgr2=0x7f1178197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178105ae0 0x7f1178197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 -- 172.21.15.5:0/3614419488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f116c0095f0 con 0x7f11780ffad0 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f11777fe700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 0x7f117819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f1177fff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9a0 tx=0x7f116800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f11757fa700 1 -- 172.21.15.5:0/3614419488 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168009790 con 0x7f11780ffad0 2023-07-21T19:14:55.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.734+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1178198910 con 0x7f11780ffad0 2023-07-21T19:14:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.735+0000 7f11757fa700 1 -- 172.21.15.5:0/3614419488 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f116800cbd0 con 0x7f11780ffad0 2023-07-21T19:14:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.735+0000 7f11757fa700 1 -- 172.21.15.5:0/3614419488 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1168018680 con 0x7f11780ffad0 2023-07-21T19:14:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.735+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11781a4b20 con 0x7f11780ffad0 2023-07-21T19:14:55.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.738+0000 7f11757fa700 1 -- 172.21.15.5:0/3614419488 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f116800cd40 con 0x7f11780ffad0 2023-07-21T19:14:55.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.739+0000 7f1192419b80 1 Processor -- start 2023-07-21T19:14:55.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.739+0000 7f1192419b80 1 -- start start 2023-07-21T19:14:55.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f11757fa700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 0x7f11600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796ab2d0 0x5604796ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f11757fa700 1 -- 172.21.15.5:0/3614419488 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1168014070 con 0x7f11780ffad0 2023-07-21T19:14:55.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796b8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604795b1cb0 con 0x5604796abbe0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560479604a40 con 0x5604796b6ad0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1192419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604795fded0 con 0x5604796ab2d0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f117cf9c700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 0x7f11600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f11777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f117cf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796b8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f1177fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796ab2d0 0x5604796ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f11777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37008/0 (socket says 172.21.15.5:37008) 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.740+0000 7f11777fe700 1 -- 172.21.15.5:0/449712825 learned_addr learned my addr 172.21.15.5:0/449712825 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 -- 172.21.15.5:0/449712825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796ab2d0 msgr2=0x5604796ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796ab2d0 0x5604796ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 -- 172.21.15.5:0/449712825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 msgr2=0x5604796b8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796b8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 -- 172.21.15.5:0/449712825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56047942bde0 con 0x5604796abbe0 2023-07-21T19:14:55.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f117cf9c700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 0x7f11600a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1170009ec0 tx=0x7f11700092f0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f11777fe700 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f116c005e80 tx=0x7f116c005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd788d257516d379 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f115e7fc700 1 -- 172.21.15.5:0/449712825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c01e070 con 0x5604796abbe0 2023-07-21T19:14:55.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f115e7fc700 1 -- 172.21.15.5:0/449712825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f116c0054d0 con 0x5604796abbe0 2023-07-21T19:14:55.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.741+0000 7f115e7fc700 1 -- 172.21.15.5:0/449712825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c004760 con 0x5604796abbe0 2023-07-21T19:14:55.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 -- 172.21.15.5:0/449712825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 msgr2=0x5604796b6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f116c005e80 tx=0x7f116c005eb0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 -- 172.21.15.5:0/449712825 shutdown_connections 2023-07-21T19:14:55.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796b8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796ab2d0 0x5604796ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 --2- 172.21.15.5:0/449712825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796b6590 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 -- 172.21.15.5:0/449712825 >> 172.21.15.5:0/449712825 conn(0x5604796a1da0 msgr2=0x5604796a76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 -- 172.21.15.5:0/449712825 shutdown_connections 2023-07-21T19:14:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.742+0000 7f1192419b80 1 -- 172.21.15.5:0/449712825 wait complete. 2023-07-21T19:14:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 -- start start 2023-07-21T19:14:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 Processor -- start 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047959dac0 con 0x5604796ab2d0 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047959d7c0 con 0x5604796abbe0 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.743+0000 7f1192419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560479481920 con 0x5604796b6ad0 2023-07-21T19:14:55.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1192419b80 10 client.0 _ll_register_callbacks cb 0x56047959b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f11777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f11777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41452/0 (socket says 172.21.15.5:41452) 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f11777fe700 1 -- 172.21.15.5:0/443443363 learned_addr learned my addr 172.21.15.5:0/443443363 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37024/0 (socket says 172.21.15.5:37024) 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f117cf9c700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 -- 172.21.15.5:0/443443363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 msgr2=0x5604796b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 -- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 msgr2=0x5604796b37b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 -- 172.21.15.5:0/443443363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f117810d210 con 0x5604796ab2d0 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f117cf9c700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f11777fe700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:55.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f1177fff700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca5c0 tx=0x7f116809df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a99ce5a650e10f27 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.744+0000 7f11577fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11680b1070 con 0x5604796ab2d0 2023-07-21T19:14:55.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1168099640 con 0x5604796ab2d0 2023-07-21T19:14:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5604795574f0 con 0x5604796ab2d0 2023-07-21T19:14:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11680b8170 con 0x5604796ab2d0 2023-07-21T19:14:55.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 10 client.17178 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x560479414c60 con 0x5604796ab2d0 2023-07-21T19:14:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11680c8080 con 0x5604796ab2d0 2023-07-21T19:14:55.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f1168010550 con 0x5604796ab2d0 2023-07-21T19:14:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 10 client.17178 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 10 client.17178 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 20 client.17178 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:55.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5604796c62c0 0x5604796c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 4 client.17178 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.745+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5604795f6eb0 con 0x5604796c62c0 2023-07-21T19:14:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.746+0000 7f11777fe700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5604796c62c0 0x5604796c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.746+0000 7f11777fe700 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5604796c62c0 0x5604796c86d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8e6d95873f5d786 server_cookie=96dd5a7ee1505e3 in_seq=0 out_seq=0 2023-07-21T19:14:55.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.746+0000 7f11577fe700 10 client.17178 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.747+0000 7f11577fe700 1 -- 172.21.15.5:0/443443363 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5604795f6eb0 con 0x5604796c62c0 2023-07-21T19:14:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.747+0000 7f11577fe700 10 client.17178 handle_command_reply: tid=0 2023-07-21T19:14:55.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 client.17178 shutdown 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 msgr2=0x5604796b64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f11680ca5c0 tx=0x7f116809df70 comp rx=0 tx=0).stop 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 shutdown_connections 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5604796c62c0 0x5604796c86d0 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796b64c0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 --2- 172.21.15.5:0/443443363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796abbe0 0x5604796b37b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f11577fe700 0 client.17178 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 >> 172.21.15.5:0/443443363 conn(0x5604796a1da0 msgr2=0x5604796a7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 shutdown_connections 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 1 -- 172.21.15.5:0/443443363 wait complete. 2023-07-21T19:14:55.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.748+0000 7f1192419b80 20 client.17178 trim_cache size 0 max 16384 2023-07-21T19:14:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 Processor -- start 2023-07-21T19:14:55.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 -- start start 2023-07-21T19:14:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796aead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604796b4d50 con 0x5604796abbe0 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604796a7a10 con 0x5604796ab2d0 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.766+0000 7f1192419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5604796a19a0 con 0x5604796b6ad0 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f1177fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37028/0 (socket says 172.21.15.5:37028) 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 -- 172.21.15.5:0/852201972 learned_addr learned my addr 172.21.15.5:0/852201972 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f1177fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41462/0 (socket says 172.21.15.5:41462) 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f117cf9c700 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 -- 172.21.15.5:0/852201972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 msgr2=0x5604796aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 -- 172.21.15.5:0/852201972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 msgr2=0x5604796bd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 -- 172.21.15.5:0/852201972 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560479452250 con 0x5604796abbe0 2023-07-21T19:14:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f1177fff700 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:55.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.767+0000 7f11777fe700 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f116c0098e0 tx=0x7f116c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=649920506fce3834 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f115effd700 1 -- 172.21.15.5:0/852201972 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c031070 con 0x5604796abbe0 2023-07-21T19:14:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f115effd700 1 -- 172.21.15.5:0/852201972 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f116c00b2a0 con 0x5604796abbe0 2023-07-21T19:14:55.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f115effd700 1 -- 172.21.15.5:0/852201972 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c01e040 con 0x5604796abbe0 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 -- 172.21.15.5:0/852201972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 msgr2=0x5604796a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f116c0098e0 tx=0x7f116c009310 comp rx=0 tx=0).stop 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 -- 172.21.15.5:0/852201972 shutdown_connections 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796b6ad0 0x5604796aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796ab2d0 0x5604796bd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 --2- 172.21.15.5:0/852201972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796abbe0 0x5604796a68b0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 -- 172.21.15.5:0/852201972 >> 172.21.15.5:0/852201972 conn(0x5604796a1da0 msgr2=0x5604796a5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 -- 172.21.15.5:0/852201972 shutdown_connections 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.768+0000 7f1192419b80 1 -- 172.21.15.5:0/852201972 wait complete. 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 -- start start 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 Processor -- start 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796bb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796bbe40 0x5604796bc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047971a1e0 con 0x5604796ab2d0 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047971a350 con 0x5604796b6ad0 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.769+0000 7f1192419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56047971a510 con 0x5604796bbe40 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f1192419b80 10 client.0 _ll_register_callbacks cb 0x7f11680998a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796bb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f1177fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37042/0 (socket says 172.21.15.5:37042) 2023-07-21T19:14:55.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f1177fff700 1 -- 172.21.15.5:0/3973437618 learned_addr learned my addr 172.21.15.5:0/3973437618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f117cf9c700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796bbe40 0x5604796bc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 -- 172.21.15.5:0/3973437618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796bbe40 msgr2=0x5604796bc2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796bbe40 0x5604796bc2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 -- 172.21.15.5:0/3973437618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 msgr2=0x5604796bd580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 -- 172.21.15.5:0/3973437618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560479414c60 con 0x5604796b6ad0 2023-07-21T19:14:55.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f1177fff700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11777fe700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796bb900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5604796a7d80 tx=0x7f116c00bd30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c29ae0da01d7e51f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.770+0000 7f11577fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116c01e030 con 0x5604796b6ad0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f116c0054d0 con 0x5604796b6ad0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f116c004910 con 0x5604796b6ad0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f116c031040 con 0x5604796b6ad0 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 10 client.26864 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:55.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56047942bde0 con 0x5604796b6ad0 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f116c0054d0 con 0x5604796b6ad0 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f116c0099c0 con 0x5604796b6ad0 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 10 client.26864 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 10 client.26864 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.771+0000 7f1192419b80 20 client.26864 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56047971b7a0 0x56047971db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f1192419b80 4 client.26864 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:14:55.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5604795b9050 con 0x56047971b7a0 2023-07-21T19:14:55.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f1177fff700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56047971b7a0 0x56047971db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:55.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f1177fff700 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56047971b7a0 0x56047971db80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41cffa477331f7a0 server_cookie=fc3ab9ce7612360d in_seq=0 out_seq=0 2023-07-21T19:14:55.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.772+0000 7f11577fe700 10 client.26864 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f11577fe700 1 -- 172.21.15.5:0/3973437618 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x5604795b9050 con 0x56047971b7a0 2023-07-21T19:14:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f11577fe700 10 client.26864 handle_command_reply: tid=0 2023-07-21T19:14:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 client.26864 shutdown 2023-07-21T19:14:55.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 msgr2=0x5604796bb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796bb900 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x5604796a7d80 tx=0x7f116c00bd30 comp rx=0 tx=0).stop 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 shutdown_connections 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5604796bbe40 0x5604796bc2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56047971b7a0 0x56047971db80 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5604796ab2d0 0x5604796bd580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f11577fe700 0 client.26864 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:55.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 --2- 172.21.15.5:0/3973437618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5604796b6ad0 0x5604796bb900 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 >> 172.21.15.5:0/3973437618 conn(0x5604796a1da0 msgr2=0x5604796c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 shutdown_connections 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 1 -- 172.21.15.5:0/3973437618 wait complete. 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.773+0000 7f1192419b80 20 client.26864 trim_cache size 0 max 16384 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:14:55.782 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:14:55.783 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:14:55.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 msgr2=0x7f11600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 0x7f11600a37f0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1170009ec0 tx=0x7f11700092f0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 msgr2=0x7f117819db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:55.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f116800e9a0 tx=0x7f116800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 shutdown_connections 2023-07-21T19:14:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1178105ae0 0x7f1178197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11600a1340 0x7f11600a37f0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11780ffad0 0x7f117819db50 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 --2- 172.21.15.5:0/3614419488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1178100480 0x7f117819e090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 >> 172.21.15.5:0/3614419488 conn(0x7f11780fb5f0 msgr2=0x7f117810ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:55.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 shutdown_connections 2023-07-21T19:14:55.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:14:55.774+0000 7f117e9ff700 1 -- 172.21.15.5:0/3614419488 wait complete. 2023-07-21T19:14:55.789 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:55 smithi005 ceph-mon[126460]: pgmap v224: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:14:55.791 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:14:55.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:55 smithi101 ceph-mon[128136]: pgmap v224: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:14:55.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:55 smithi156 ceph-mon[128316]: pgmap v224: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:14:56.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 -- 172.21.15.5:0/145857514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 msgr2=0x7f5964105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 --2- 172.21.15.5:0/145857514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f5964105f30 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f5954009990 tx=0x7f5954009ca0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 -- 172.21.15.5:0/145857514 shutdown_connections 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 --2- 172.21.15.5:0/145857514 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 0x7f596410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 --2- 172.21.15.5:0/145857514 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f5964105f30 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.324+0000 7f596a7ff700 1 --2- 172.21.15.5:0/145857514 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f59640fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.325+0000 7f596a7ff700 1 -- 172.21.15.5:0/145857514 >> 172.21.15.5:0/145857514 conn(0x7f59640fb5f0 msgr2=0x7f59640fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.325+0000 7f596a7ff700 1 -- 172.21.15.5:0/145857514 shutdown_connections 2023-07-21T19:14:56.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.325+0000 7f596a7ff700 1 -- 172.21.15.5:0/145857514 wait complete. 2023-07-21T19:14:56.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 Processor -- start 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 -- start start 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 0x7f59641982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f596419c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596419cf80 con 0x7f5964105b60 2023-07-21T19:14:56.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596419d0f0 con 0x7f59640ffb30 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f596a7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596419d2b0 con 0x7f59641004e0 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f5963fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f5968d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f596419c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.326+0000 7f5963fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41486/0 (socket says 172.21.15.5:41486) 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5963fff700 1 -- 172.21.15.5:0/653873020 learned_addr learned my addr 172.21.15.5:0/653873020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:56.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f59637fe700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 0x7f59641982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 -- 172.21.15.5:0/653873020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 msgr2=0x7f59641982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 0x7f59641982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 -- 172.21.15.5:0/653873020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 msgr2=0x7f5964197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 -- 172.21.15.5:0/653873020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540095f0 con 0x7f5964105b60 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5963fff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:14:56.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f5968d9c700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f596419c940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f595c00be20 tx=0x7f595c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f59617fa700 1 -- 172.21.15.5:0/653873020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c00cb90 con 0x7f5964105b60 2023-07-21T19:14:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596419d590 con 0x7f5964105b60 2023-07-21T19:14:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.327+0000 7f59617fa700 1 -- 172.21.15.5:0/653873020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f595c00ccf0 con 0x7f5964105b60 2023-07-21T19:14:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.328+0000 7f59617fa700 1 -- 172.21.15.5:0/653873020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c0125a0 con 0x7f5964105b60 2023-07-21T19:14:56.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.328+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59641a4dd0 con 0x7f5964105b60 2023-07-21T19:14:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.331+0000 7f59617fa700 1 -- 172.21.15.5:0/653873020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f595c014440 con 0x7f5964105b60 2023-07-21T19:14:56.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.333+0000 7f597e156b80 1 Processor -- start 2023-07-21T19:14:56.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.333+0000 7f597e156b80 1 -- start start 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.333+0000 7f59617fa700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 0x7f594c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfe02d0 0x56260cfda040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfda9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cedfc50 con 0x56260ce366b0 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cf329e0 con 0x56260cfda580 2023-07-21T19:14:56.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f597e156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cf2be70 con 0x56260cfe02d0 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f59617fa700 1 -- 172.21.15.5:0/653873020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f595c006e40 con 0x7f5964105b60 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37072/0 (socket says 172.21.15.5:37072) 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 -- 172.21.15.5:0/2950459126 learned_addr learned my addr 172.21.15.5:0/2950459126 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f59637fe700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 0x7f594c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5968d9c700 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfda9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f59637fe700 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfe02d0 0x56260cfda040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 -- 172.21.15.5:0/2950459126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfe02d0 msgr2=0x56260cfda040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfe02d0 0x56260cfda040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 -- 172.21.15.5:0/2950459126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 msgr2=0x56260cfda9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfda9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.334+0000 7f5963fff700 1 -- 172.21.15.5:0/2950459126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56260cd59d80 con 0x56260ce366b0 2023-07-21T19:14:56.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.335+0000 7f5963fff700 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f595400bba0 tx=0x7f595400bbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a125d3286fff340c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.335+0000 7f59637fe700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 0x7f594c0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56260cfd6b40 tx=0x7f59580099b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.335+0000 7f594a7fc700 1 -- 172.21.15.5:0/2950459126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595401d070 con 0x56260ce366b0 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.335+0000 7f594a7fc700 1 -- 172.21.15.5:0/2950459126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f595401e040 con 0x56260ce366b0 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.335+0000 7f594a7fc700 1 -- 172.21.15.5:0/2950459126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5954004dd0 con 0x56260ce366b0 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 -- 172.21.15.5:0/2950459126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 msgr2=0x56260cfdfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f595400bba0 tx=0x7f595400bbd0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 -- 172.21.15.5:0/2950459126 shutdown_connections 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfda9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfdfd90 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 --2- 172.21.15.5:0/2950459126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfe02d0 0x56260cfda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 -- 172.21.15.5:0/2950459126 >> 172.21.15.5:0/2950459126 conn(0x56260cfcfbe0 msgr2=0x56260cfd4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 -- 172.21.15.5:0/2950459126 shutdown_connections 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.336+0000 7f597e156b80 1 -- 172.21.15.5:0/2950459126 wait complete. 2023-07-21T19:14:56.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 -- start start 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 Processor -- start 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfda580 0x56260cfe3860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260cfe3e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cecba60 con 0x56260ce366b0 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cecb760 con 0x56260cfe02d0 2023-07-21T19:14:56.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cdaf8c0 con 0x56260cfda580 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.337+0000 7f597e156b80 10 client.0 _ll_register_callbacks cb 0x56260cec9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37088/0 (socket says 172.21.15.5:37088) 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 -- 172.21.15.5:0/3129403961 learned_addr learned my addr 172.21.15.5:0/3129403961 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5968d9c700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260cfe3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 -- 172.21.15.5:0/3129403961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfda580 msgr2=0x56260cfe3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfda580 0x56260cfe3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 -- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 msgr2=0x56260cfe3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260cfe3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 -- 172.21.15.5:0/3129403961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596410d220 con 0x56260ce366b0 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f5963fff700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5954004800 tx=0x7f5954005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b159c5be354d92 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f59437fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:56.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595401d030 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56260ce85490 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.338+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5954005550 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5954005710 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 10 client.17196 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56260cd42c00 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56260ce85490 con 0x56260ce366b0 2023-07-21T19:14:56.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f595402c400 con 0x56260ce366b0 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 10 client.17196 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 10 client.17196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 20 client.17196 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260cff3770 0x56260cff5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 4 client.17196 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:14:56.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.339+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56260cf24e50 con 0x56260cff3770 2023-07-21T19:14:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.340+0000 7f59637fe700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260cff3770 0x56260cff5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.340+0000 7f59637fe700 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260cff3770 0x56260cff5b80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6818b09cb9bbb69 server_cookie=389e1477422bce6b in_seq=0 out_seq=0 2023-07-21T19:14:56.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.340+0000 7f59437fe700 10 client.17196 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.341+0000 7f59437fe700 1 -- 172.21.15.5:0/3129403961 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56260cf24e50 con 0x56260cff3770 2023-07-21T19:14:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.341+0000 7f59437fe700 10 client.17196 handle_command_reply: tid=0 2023-07-21T19:14:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.341+0000 7f597e156b80 1 client.17196 shutdown 2023-07-21T19:14:56.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 msgr2=0x56260cfe6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f5954004800 tx=0x7f5954005e80 comp rx=0 tx=0).stop 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 shutdown_connections 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260cfe3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260cff3770 0x56260cff5b80 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260ce366b0 0x56260cfe6540 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 --2- 172.21.15.5:0/3129403961 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260cfda580 0x56260cfe3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 >> 172.21.15.5:0/3129403961 conn(0x56260cfcfbe0 msgr2=0x56260cfd44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f59437fe700 0 client.17196 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 shutdown_connections 2023-07-21T19:14:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 1 -- 172.21.15.5:0/3129403961 wait complete. 2023-07-21T19:14:56.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.342+0000 7f597e156b80 20 client.17196 trim_cache size 0 max 16384 2023-07-21T19:14:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 Processor -- start 2023-07-21T19:14:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 -- start start 2023-07-21T19:14:56.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfda040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfe3810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.357+0000 7f597e156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cfe08a0 con 0x56260cfe02d0 2023-07-21T19:14:56.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f597e156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cfdfa10 con 0x56260cfda580 2023-07-21T19:14:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f597e156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260cfe06a0 con 0x56260ce366b0 2023-07-21T19:14:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37100/0 (socket says 172.21.15.5:37100) 2023-07-21T19:14:56.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 -- 172.21.15.5:0/1923583496 learned_addr learned my addr 172.21.15.5:0/1923583496 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f59637fe700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfe3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5963fff700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfda040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 -- 172.21.15.5:0/1923583496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 msgr2=0x56260cfda040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfda040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 -- 172.21.15.5:0/1923583496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 msgr2=0x56260cfe3810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfe3810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 -- 172.21.15.5:0/1923583496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56260cd801f0 con 0x56260cfe02d0 2023-07-21T19:14:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5963fff700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:14:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.358+0000 7f5968d9c700 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f595c00bfd0 tx=0x7f595c0b9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a2122dd626dc5c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f594affd700 1 -- 172.21.15.5:0/1923583496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c082ea0 con 0x56260cfe02d0 2023-07-21T19:14:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f594affd700 1 -- 172.21.15.5:0/1923583496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f595c09d6d0 con 0x56260cfe02d0 2023-07-21T19:14:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f594affd700 1 -- 172.21.15.5:0/1923583496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f595c099900 con 0x56260cfe02d0 2023-07-21T19:14:56.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 -- 172.21.15.5:0/1923583496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 msgr2=0x56260cfd4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f595c00bfd0 tx=0x7f595c0b9040 comp rx=0 tx=0).stop 2023-07-21T19:14:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 -- 172.21.15.5:0/1923583496 shutdown_connections 2023-07-21T19:14:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfe02d0 0x56260cfd4c60 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfda040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 --2- 172.21.15.5:0/1923583496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfda580 0x56260cfe3810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 -- 172.21.15.5:0/1923583496 >> 172.21.15.5:0/1923583496 conn(0x56260cfcfbe0 msgr2=0x56260cfe3100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 -- 172.21.15.5:0/1923583496 shutdown_connections 2023-07-21T19:14:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.359+0000 7f597e156b80 1 -- 172.21.15.5:0/1923583496 wait complete. 2023-07-21T19:14:56.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 -- start start 2023-07-21T19:14:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 Processor -- start 2023-07-21T19:14:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfe8e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260d0465f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260d046b30 con 0x56260cfda580 2023-07-21T19:14:56.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260d046c70 con 0x56260cfe02d0 2023-07-21T19:14:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56260d046e30 con 0x56260ce366b0 2023-07-21T19:14:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.360+0000 7f597e156b80 10 client.0 _ll_register_callbacks cb 0x7f595c095320 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:14:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37106/0 (socket says 172.21.15.5:37106) 2023-07-21T19:14:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 -- 172.21.15.5:0/1649031413 learned_addr learned my addr 172.21.15.5:0/1649031413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:14:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f5968d9c700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260d0465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f5963fff700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfe8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 -- 172.21.15.5:0/1649031413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 msgr2=0x56260cfe8e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfe8e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 -- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 msgr2=0x56260d0465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260d0465f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 -- 172.21.15.5:0/1649031413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59641a4dd0 con 0x56260cfda580 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f5963fff700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfe8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59637fe700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5958009fd0 tx=0x7f59580047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8762b879a6f1d609 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.361+0000 7f59437fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:14:56.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5958024040 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f595401e040 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(90..90 src has 1..90) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5958028510 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5958028780 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f597e156b80 10 client.17202 fetch_fsmap learned FSMap version 34 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x56260cd58250 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f595401e040 con 0x56260cfda580 2023-07-21T19:14:56.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.362+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f595802e030 con 0x56260cfda580 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 10 client.17202 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 10 client.17202 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 20 client.17202 populate_metadata read hostname 'smithi005' 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260d048550 0x56260d04a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 4 client.17202 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56260cee6ff0 con 0x56260d048550 2023-07-21T19:14:56.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f5963fff700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260d048550 0x56260d04a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:14:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f5963fff700 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260d048550 0x56260d04a960 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=46e6ebb3295e0a11 server_cookie=30ab9df3636ff81c in_seq=0 out_seq=0 2023-07-21T19:14:56.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.363+0000 7f59437fe700 10 client.17202 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.378+0000 7f59437fe700 1 -- 172.21.15.5:0/1649031413 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56260cee6ff0 con 0x56260d048550 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.378+0000 7f59437fe700 10 client.17202 handle_command_reply: tid=0 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.378+0000 7f597e156b80 1 client.17202 shutdown 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 msgr2=0x56260cfe9380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f5958009fd0 tx=0x7f59580047c0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 shutdown_connections 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56260cfe02d0 0x56260d0465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56260ce366b0 0x56260cfe8e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56260d048550 0x56260d04a960 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 --2- 172.21.15.5:0/1649031413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56260cfda580 0x56260cfe9380 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f59437fe700 0 client.17202 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 >> 172.21.15.5:0/1649031413 conn(0x56260cfcfbe0 msgr2=0x56260cfdea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 shutdown_connections 2023-07-21T19:14:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 1 -- 172.21.15.5:0/1649031413 wait complete. 2023-07-21T19:14:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.379+0000 7f597e156b80 20 client.17202 trim_cache size 0 max 16384 2023-07-21T19:14:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:14:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:14:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:14:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:14:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 msgr2=0x7f594c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 0x7f594c0a3770 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x56260cfd6b40 tx=0x7f59580099b0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 msgr2=0x7f596419c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f596419c940 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f595c00be20 tx=0x7f595c009ed0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 shutdown_connections 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f594c0a12c0 0x7f594c0a3770 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5964105b60 0x7f596419c940 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59640ffb30 0x7f5964197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 --2- 172.21.15.5:0/653873020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f59641004e0 0x7f59641982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:14:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 >> 172.21.15.5:0/653873020 conn(0x7f59640fb5f0 msgr2=0x7f5964072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:14:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 shutdown_connections 2023-07-21T19:14:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:14:56.380+0000 7f596a7ff700 1 -- 172.21.15.5:0/653873020 wait complete. 2023-07-21T19:14:57.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:57 smithi101 ceph-mon[128136]: pgmap v225: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 42 op/s 2023-07-21T19:14:57.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:57 smithi005 ceph-mon[126460]: pgmap v225: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 42 op/s 2023-07-21T19:14:57.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:57 smithi156 ceph-mon[128316]: pgmap v225: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 42 op/s 2023-07-21T19:14:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:26700000 of 50000000 tuples (53%) done (elapsed 415.97 s, remaining 363.00 s) 2023-07-21T19:14:59.744 INFO:teuthology.orchestra.run.smithi005.stderr:26800000 of 50000000 tuples (53%) done (elapsed 417.31 s, remaining 361.25 s) 2023-07-21T19:14:59.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:14:59 smithi101 ceph-mon[128136]: pgmap v226: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 68 op/s 2023-07-21T19:14:59.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:14:59 smithi005 ceph-mon[126460]: pgmap v226: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 68 op/s 2023-07-21T19:14:59.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:14:59 smithi156 ceph-mon[128316]: pgmap v226: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 68 op/s 2023-07-21T19:15:01.139 INFO:teuthology.orchestra.run.smithi005.stderr:26900000 of 50000000 tuples (53%) done (elapsed 418.70 s, remaining 359.56 s) 2023-07-21T19:15:01.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:01 smithi101 ceph-mon[128136]: pgmap v227: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 54 op/s 2023-07-21T19:15:01.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:01 smithi005 ceph-mon[126460]: pgmap v227: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 54 op/s 2023-07-21T19:15:01.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:01 smithi156 ceph-mon[128316]: pgmap v227: 97 pgs: 97 active+clean; 6.1 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 18 MiB/s wr, 54 op/s 2023-07-21T19:15:02.646 INFO:teuthology.orchestra.run.smithi005.stderr:27000000 of 50000000 tuples (54%) done (elapsed 420.21 s, remaining 357.96 s) 2023-07-21T19:15:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:02 smithi101 ceph-mon[128136]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T19:15:02.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:02 smithi005 ceph-mon[126460]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T19:15:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:02 smithi156 ceph-mon[128316]: osdmap e91: 12 total, 12 up, 12 in 2023-07-21T19:15:03.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:03 smithi101 ceph-mon[128136]: pgmap v229: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 81 op/s 2023-07-21T19:15:03.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:03 smithi005 ceph-mon[126460]: pgmap v229: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 81 op/s 2023-07-21T19:15:03.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:03 smithi156 ceph-mon[128316]: pgmap v229: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 82 KiB/s rd, 19 MiB/s wr, 81 op/s 2023-07-21T19:15:04.593 INFO:teuthology.orchestra.run.smithi005.stderr:27100000 of 50000000 tuples (54%) done (elapsed 422.16 s, remaining 356.73 s) 2023-07-21T19:15:05.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:05 smithi101 ceph-mon[128136]: pgmap v230: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:05.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:05 smithi005 ceph-mon[126460]: pgmap v230: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:05.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:05 smithi156 ceph-mon[128316]: pgmap v230: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:05.946 INFO:teuthology.orchestra.run.smithi005.stderr:27200000 of 50000000 tuples (54%) done (elapsed 423.51 s, remaining 355.00 s) 2023-07-21T19:15:07.256 INFO:teuthology.orchestra.run.smithi005.stderr:27300000 of 50000000 tuples (54%) done (elapsed 424.82 s, remaining 353.24 s) 2023-07-21T19:15:07.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:07 smithi101 ceph-mon[128136]: pgmap v231: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:07.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:07 smithi101 ceph-mon[128136]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T19:15:07.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:07 smithi005 ceph-mon[126460]: pgmap v231: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:07.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:07 smithi005 ceph-mon[126460]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T19:15:07.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:07 smithi156 ceph-mon[128316]: pgmap v231: 97 pgs: 97 active+clean; 6.2 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 18 MiB/s wr, 87 op/s 2023-07-21T19:15:07.921 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:07 smithi156 ceph-mon[128316]: osdmap e92: 12 total, 12 up, 12 in 2023-07-21T19:15:08.632 INFO:teuthology.orchestra.run.smithi005.stderr:27400000 of 50000000 tuples (54%) done (elapsed 426.20 s, remaining 351.54 s) 2023-07-21T19:15:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.561+0000 7f57ecdff700 1 -- 172.21.15.5:0/1308027241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e8072830 msgr2=0x7f57e8072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.561+0000 7f57ecdff700 1 --2- 172.21.15.5:0/1308027241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e8072830 0x7f57e8072c00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009990 tx=0x7f57d8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 -- 172.21.15.5:0/1308027241 shutdown_connections 2023-07-21T19:15:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 --2- 172.21.15.5:0/1308027241 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e810c410 0x7f57e810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 --2- 172.21.15.5:0/1308027241 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e8072830 0x7f57e8072c00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 --2- 172.21.15.5:0/1308027241 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e810bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 -- 172.21.15.5:0/1308027241 >> 172.21.15.5:0/1308027241 conn(0x7f57e80fb650 msgr2=0x7f57e80fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 -- 172.21.15.5:0/1308027241 shutdown_connections 2023-07-21T19:15:09.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.562+0000 7f57ecdff700 1 -- 172.21.15.5:0/1308027241 wait complete. 2023-07-21T19:15:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.563+0000 7f57ecdff700 1 Processor -- start 2023-07-21T19:15:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.563+0000 7f57ecdff700 1 -- start start 2023-07-21T19:15:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e8072830 0x7f57e8197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 0x7f57e819c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e819cf30 con 0x7f57e8073140 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e819d0a0 con 0x7f57e810c410 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57ecdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e819d260 con 0x7f57e8072830 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42696/0 (socket says 172.21.15.5:42696) 2023-07-21T19:15:09.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2882037833 learned_addr learned my addr 172.21.15.5:0/2882037833 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 0x7f57e819c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.564+0000 7f57e659c700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e8072830 0x7f57e8197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2882037833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e8072830 msgr2=0x7f57e8197d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e8072830 0x7f57e8197d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2882037833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 msgr2=0x7f57e819c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 0x7f57e819c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2882037833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57d80095f0 con 0x7f57e8073140 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 0x7f57e819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9a0 tx=0x7f57dc00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57d77fe700 1 -- 172.21.15.5:0/2882037833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc009790 con 0x7f57e8073140 2023-07-21T19:15:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e819d540 con 0x7f57e8073140 2023-07-21T19:15:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57d77fe700 1 -- 172.21.15.5:0/2882037833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57dc010430 con 0x7f57e8073140 2023-07-21T19:15:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57d77fe700 1 -- 172.21.15.5:0/2882037833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc00cea0 con 0x7f57e8073140 2023-07-21T19:15:09.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.565+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e81a4d80 con 0x7f57e8073140 2023-07-21T19:15:09.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.569+0000 7f57d77fe700 1 -- 172.21.15.5:0/2882037833 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f57dc0105a0 con 0x7f57e8073140 2023-07-21T19:15:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.570+0000 7f5800651b80 1 Processor -- start 2023-07-21T19:15:09.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.570+0000 7f5800651b80 1 -- start start 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57d77fe700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 0x7f57d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 0x55804f775d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f675c50 con 0x55804f770580 2023-07-21T19:15:09.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f6c89e0 con 0x55804f7762d0 2023-07-21T19:15:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f5800651b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f6c1e70 con 0x55804f5cc6b0 2023-07-21T19:15:09.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57d77fe700 1 -- 172.21.15.5:0/2882037833 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57dc014070 con 0x7f57e8073140 2023-07-21T19:15:09.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:40256/0 (socket says 172.21.15.5:40256) 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57e659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 0x55804f775d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57e5d9b700 1 -- 172.21.15.5:0/941106722 learned_addr learned my addr 172.21.15.5:0/941106722 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.571+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 0x7f57d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 -- 172.21.15.5:0/941106722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 msgr2=0x55804f770040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 -- 172.21.15.5:0/941106722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 msgr2=0x55804f7709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 -- 172.21.15.5:0/941106722 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55804f4efd80 con 0x55804f5cc6b0 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e659c700 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 0x55804f775d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57d8005910 tx=0x7f57d8005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566ea85a1c241745 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 0x7f57d00a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57e0007c20 tx=0x7f57e000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57d4ff9700 1 -- 172.21.15.5:0/941106722 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d801d070 con 0x55804f5cc6b0 2023-07-21T19:15:09.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57d4ff9700 1 -- 172.21.15.5:0/941106722 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d801e040 con 0x55804f5cc6b0 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.572+0000 7f57d4ff9700 1 -- 172.21.15.5:0/941106722 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8004830 con 0x55804f5cc6b0 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 -- 172.21.15.5:0/941106722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 msgr2=0x55804f775d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 0x55804f775d90 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f57d8005910 tx=0x7f57d8005f00 comp rx=0 tx=0).stop 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 -- 172.21.15.5:0/941106722 shutdown_connections 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f5cc6b0 0x55804f775d90 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 --2- 172.21.15.5:0/941106722 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f770040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 -- 172.21.15.5:0/941106722 >> 172.21.15.5:0/941106722 conn(0x55804f765be0 msgr2=0x55804f76af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 -- 172.21.15.5:0/941106722 shutdown_connections 2023-07-21T19:15:09.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.573+0000 7f5800651b80 1 -- 172.21.15.5:0/941106722 wait complete. 2023-07-21T19:15:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.574+0000 7f5800651b80 1 -- start start 2023-07-21T19:15:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.574+0000 7f5800651b80 1 Processor -- start 2023-07-21T19:15:09.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f77c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f779e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f661a60 con 0x55804f770580 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f661760 con 0x55804f5cc6b0 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f5458c0 con 0x55804f7762d0 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f5800651b80 10 client.0 _ll_register_callbacks cb 0x55804f65f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:09.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42728/0 (socket says 172.21.15.5:42728) 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f57e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f77c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2927212696 learned_addr learned my addr 172.21.15.5:0/2927212696 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.575+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f779e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2927212696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 msgr2=0x55804f779e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f779e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 msgr2=0x55804f77c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f77c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 -- 172.21.15.5:0/2927212696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e81a4d80 con 0x55804f770580 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f57dc010c30 tx=0x7f57dc099bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520380897fcadef8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57dc09a3b0 con 0x55804f770580 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57dc00bb10 con 0x55804f770580 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55804f61b490 con 0x55804f770580 2023-07-21T19:15:09.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57dc014230 con 0x55804f770580 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f5800651b80 10 client.17214 fetch_fsmap learned FSMap version 34 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.576+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x7f57e810f3f0 con 0x55804f770580 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55804f61b490 con 0x55804f770580 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57dc0cad20 con 0x55804f770580 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 10 client.17214 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 10 client.17214 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 20 client.17214 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:09.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f789ad0 0x55804f78bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 4 client.17214 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55804f6bae50 con 0x55804f789ad0 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.577+0000 7f57e659c700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f789ad0 0x55804f78bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.578+0000 7f57e659c700 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f789ad0 0x55804f78bee0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22af491db915e1e7 server_cookie=e2737cca8e416c8 in_seq=0 out_seq=0 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.578+0000 7f57cdffb700 10 client.17214 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f57cdffb700 1 -- 172.21.15.5:0/2927212696 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55804f6bae50 con 0x55804f789ad0 2023-07-21T19:15:09.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f57cdffb700 10 client.17214 handle_command_reply: tid=0 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 client.17214 shutdown 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 msgr2=0x55804f7797e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f57dc010c30 tx=0x7f57dc099bb0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 shutdown_connections 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f779e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f77c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f789ad0 0x55804f78bee0 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 --2- 172.21.15.5:0/2927212696 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f7797e0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f57cdffb700 0 client.17214 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 >> 172.21.15.5:0/2927212696 conn(0x55804f765be0 msgr2=0x55804f769c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 shutdown_connections 2023-07-21T19:15:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.579+0000 7f5800651b80 1 -- 172.21.15.5:0/2927212696 wait complete. 2023-07-21T19:15:09.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.580+0000 7f5800651b80 20 client.17214 trim_cache size 0 max 16384 2023-07-21T19:15:09.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.597+0000 7f5800651b80 1 Processor -- start 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.597+0000 7f5800651b80 1 -- start start 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f770580 0x55804f7771a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f77c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f76ca70 con 0x55804f5cc6b0 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f76bfb0 con 0x55804f7762d0 2023-07-21T19:15:09.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f5800651b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f787150 con 0x55804f770580 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f57e659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f57e6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f77c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f57e659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42744/0 (socket says 172.21.15.5:42744) 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f57e659c700 1 -- 172.21.15.5:0/1647983290 learned_addr learned my addr 172.21.15.5:0/1647983290 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.598+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f770580 0x55804f7771a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 -- 172.21.15.5:0/1647983290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f770580 msgr2=0x55804f7771a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f770580 0x55804f7771a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 -- 172.21.15.5:0/1647983290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 msgr2=0x55804f77c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f77c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 -- 172.21.15.5:0/1647983290 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55804f790b30 con 0x55804f5cc6b0 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f77c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57e659c700 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71844c4f834c7a2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57d57fa700 1 -- 172.21.15.5:0/1647983290 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d8032070 con 0x55804f5cc6b0 2023-07-21T19:15:09.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57d57fa700 1 -- 172.21.15.5:0/1647983290 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d8034860 con 0x55804f5cc6b0 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.599+0000 7f57d57fa700 1 -- 172.21.15.5:0/1647983290 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d801d040 con 0x55804f5cc6b0 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 -- 172.21.15.5:0/1647983290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 msgr2=0x55804f780b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f57d8000c00 tx=0x7f57d801a040 comp rx=0 tx=0).stop 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 -- 172.21.15.5:0/1647983290 shutdown_connections 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f7762d0 0x55804f77c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f5cc6b0 0x55804f780b80 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 --2- 172.21.15.5:0/1647983290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f770580 0x55804f7771a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 -- 172.21.15.5:0/1647983290 >> 172.21.15.5:0/1647983290 conn(0x55804f765be0 msgr2=0x55804f776a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 -- 172.21.15.5:0/1647983290 shutdown_connections 2023-07-21T19:15:09.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.600+0000 7f5800651b80 1 -- 172.21.15.5:0/1647983290 wait complete. 2023-07-21T19:15:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 -- start start 2023-07-21T19:15:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 Processor -- start 2023-07-21T19:15:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f772dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f771790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f773390 con 0x55804f770580 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f773500 con 0x55804f5cc6b0 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55804f771cd0 con 0x55804f7762d0 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.601+0000 7f5800651b80 10 client.0 _ll_register_callbacks cb 0x55804f65f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:40288/0 (socket says 172.21.15.5:40288) 2023-07-21T19:15:09.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f772dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 -- 172.21.15.5:0/4272658582 learned_addr learned my addr 172.21.15.5:0/4272658582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e6d9d700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f771790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 -- 172.21.15.5:0/4272658582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 msgr2=0x55804f771790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f771790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 -- 172.21.15.5:0/4272658582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 msgr2=0x55804f772dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f772dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.602+0000 7f57e659c700 1 -- 172.21.15.5:0/4272658582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0011040 con 0x55804f5cc6b0 2023-07-21T19:15:09.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57e659c700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009320 tx=0x7f57d800b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=732078f8e54c2150 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d801d030 con 0x55804f5cc6b0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57d801e040 con 0x55804f5cc6b0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57d80344d0 con 0x55804f5cc6b0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57d8032040 con 0x55804f5cc6b0 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f5800651b80 10 client.26900 fetch_fsmap learned FSMap version 34 2023-07-21T19:15:09.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) v3 -- 0x55804f4efd80 con 0x55804f5cc6b0 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.603+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57d801e040 con 0x55804f5cc6b0 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 34) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f57d8004730 con 0x55804f5cc6b0 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 10 client.26900 fetch_fsmap finished waiting for FSMap version 34 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 10 client.26900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 20 client.26900 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f7de130 0x55804f7e0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 4 client.26900 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:15:09.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55804f67cff0 con 0x55804f7de130 2023-07-21T19:15:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.604+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f7de130 0x55804f7e0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.605+0000 7f57e5d9b700 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f7de130 0x55804f7e0510 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2c82424d85cca65 server_cookie=e6f8c1c60f14144a in_seq=0 out_seq=0 2023-07-21T19:15:09.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.605+0000 7f57cdffb700 10 client.26900 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.623+0000 7f57cdffb700 1 -- 172.21.15.5:0/4272658582 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55804f67cff0 con 0x55804f7de130 2023-07-21T19:15:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.623+0000 7f57cdffb700 10 client.26900 handle_command_reply: tid=0 2023-07-21T19:15:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.623+0000 7f5800651b80 1 client.26900 shutdown 2023-07-21T19:15:09.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.623+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 msgr2=0x55804f772880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.623+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f57d8009320 tx=0x7f57d800b390 comp rx=0 tx=0).stop 2023-07-21T19:15:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 shutdown_connections 2023-07-21T19:15:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55804f7762d0 0x55804f771790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55804f5cc6b0 0x55804f772880 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55804f7de130 0x55804f7e0510 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 --2- 172.21.15.5:0/4272658582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55804f770580 0x55804f772dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f57cdffb700 0 client.26900 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 >> 172.21.15.5:0/4272658582 conn(0x55804f765be0 msgr2=0x55804f777320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 shutdown_connections 2023-07-21T19:15:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 1 -- 172.21.15.5:0/4272658582 wait complete. 2023-07-21T19:15:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.624+0000 7f5800651b80 20 client.26900 trim_cache size 0 max 16384 2023-07-21T19:15:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:15:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:15:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:15:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:15:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 msgr2=0x7f57d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 0x7f57d00a37f0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f57e0007c20 tx=0x7f57e000b040 comp rx=0 tx=0).stop 2023-07-21T19:15:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 msgr2=0x7f57e8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:09.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f57dc00e9a0 tx=0x7f57dc00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 shutdown_connections 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57e810c410 0x7f57e819c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57e8072830 0x7f57e8197d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57d00a1340 0x7f57d00a37f0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 --2- 172.21.15.5:0/2882037833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57e8073140 0x7f57e8198280 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 >> 172.21.15.5:0/2882037833 conn(0x7f57e80fb650 msgr2=0x7f57e810e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:09.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 shutdown_connections 2023-07-21T19:15:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:09.625+0000 7f57ecdff700 1 -- 172.21.15.5:0/2882037833 wait complete. 2023-07-21T19:15:09.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:09 smithi101 ceph-mon[128136]: pgmap v233: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 27 MiB/s wr, 133 op/s 2023-07-21T19:15:09.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:09 smithi005 ceph-mon[126460]: pgmap v233: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 27 MiB/s wr, 133 op/s 2023-07-21T19:15:09.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:09 smithi156 ceph-mon[128316]: pgmap v233: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 27 MiB/s wr, 133 op/s 2023-07-21T19:15:09.954 INFO:teuthology.orchestra.run.smithi005.stderr:27500000 of 50000000 tuples (55%) done (elapsed 427.52 s, remaining 349.79 s) 2023-07-21T19:15:11.223 INFO:teuthology.orchestra.run.smithi005.stderr:27600000 of 50000000 tuples (55%) done (elapsed 428.79 s, remaining 348.00 s) 2023-07-21T19:15:11.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:11 smithi101 ceph-mon[128136]: pgmap v234: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 25 MiB/s wr, 122 op/s 2023-07-21T19:15:11.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:11 smithi005 ceph-mon[126460]: pgmap v234: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 25 MiB/s wr, 122 op/s 2023-07-21T19:15:11.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:11 smithi156 ceph-mon[128316]: pgmap v234: 97 pgs: 97 active+clean; 6.3 GiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 188 KiB/s rd, 25 MiB/s wr, 122 op/s 2023-07-21T19:15:12.525 INFO:teuthology.orchestra.run.smithi005.stderr:27700000 of 50000000 tuples (55%) done (elapsed 430.09 s, remaining 346.25 s) 2023-07-21T19:15:13.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:13 smithi101 ceph-mon[128136]: pgmap v235: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 25 MiB/s wr, 83 op/s 2023-07-21T19:15:13.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:13 smithi005 ceph-mon[126460]: pgmap v235: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 25 MiB/s wr, 83 op/s 2023-07-21T19:15:13.913 INFO:teuthology.orchestra.run.smithi005.stderr:27800000 of 50000000 tuples (55%) done (elapsed 431.48 s, remaining 344.56 s) 2023-07-21T19:15:13.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:13 smithi156 ceph-mon[128316]: pgmap v235: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 137 KiB/s rd, 25 MiB/s wr, 83 op/s 2023-07-21T19:15:14.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:14 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:14.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:14 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:14.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:14 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:15.234 INFO:teuthology.orchestra.run.smithi005.stderr:27900000 of 50000000 tuples (55%) done (elapsed 432.80 s, remaining 342.83 s) 2023-07-21T19:15:15.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:15 smithi101 ceph-mon[128136]: pgmap v236: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:15.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:15 smithi005 ceph-mon[126460]: pgmap v236: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:15.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:15 smithi156 ceph-mon[128316]: pgmap v236: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:16.560 INFO:teuthology.orchestra.run.smithi005.stderr:28000000 of 50000000 tuples (56%) done (elapsed 434.13 s, remaining 341.10 s) 2023-07-21T19:15:17.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:17 smithi101 ceph-mon[128136]: pgmap v237: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:17.862 INFO:teuthology.orchestra.run.smithi005.stderr:28100000 of 50000000 tuples (56%) done (elapsed 435.43 s, remaining 339.35 s) 2023-07-21T19:15:17.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:17 smithi005 ceph-mon[126460]: pgmap v237: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:17.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:17 smithi156 ceph-mon[128316]: pgmap v237: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 25 MiB/s wr, 76 op/s 2023-07-21T19:15:19.266 INFO:teuthology.orchestra.run.smithi005.stderr:28200000 of 50000000 tuples (56%) done (elapsed 436.83 s, remaining 337.69 s) 2023-07-21T19:15:19.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:19 smithi101 ceph-mon[128136]: pgmap v238: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 42 op/s 2023-07-21T19:15:19.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:19 smithi005 ceph-mon[126460]: pgmap v238: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 42 op/s 2023-07-21T19:15:19.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:19 smithi156 ceph-mon[128316]: pgmap v238: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 MiB/s wr, 42 op/s 2023-07-21T19:15:20.566 INFO:teuthology.orchestra.run.smithi005.stderr:28300000 of 50000000 tuples (56%) done (elapsed 438.13 s, remaining 335.95 s) 2023-07-21T19:15:21.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:21 smithi101 ceph-mon[128136]: pgmap v239: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:15:21.844 INFO:teuthology.orchestra.run.smithi005.stderr:28400000 of 50000000 tuples (56%) done (elapsed 439.41 s, remaining 334.20 s) 2023-07-21T19:15:21.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:21 smithi005 ceph-mon[126460]: pgmap v239: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:15:21.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:21 smithi156 ceph-mon[128316]: pgmap v239: 97 pgs: 97 active+clean; 6.4 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 22 MiB/s wr, 38 op/s 2023-07-21T19:15:22.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.788+0000 7f8b6219d700 1 -- 172.21.15.5:0/1328164335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c0ffd20 msgr2=0x7f8b5c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.788+0000 7f8b6219d700 1 --2- 172.21.15.5:0/1328164335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c1000f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009990 tx=0x7f8b58009ca0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 -- 172.21.15.5:0/1328164335 shutdown_connections 2023-07-21T19:15:22.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 --2- 172.21.15.5:0/1328164335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c101e50 0x7f8b5c10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 --2- 172.21.15.5:0/1328164335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c1000f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 --2- 172.21.15.5:0/1328164335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c1014a0 0x7f8b5c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 -- 172.21.15.5:0/1328164335 >> 172.21.15.5:0/1328164335 conn(0x7f8b5c0fb440 msgr2=0x7f8b5c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 -- 172.21.15.5:0/1328164335 shutdown_connections 2023-07-21T19:15:22.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.789+0000 7f8b6219d700 1 -- 172.21.15.5:0/1328164335 wait complete. 2023-07-21T19:15:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 Processor -- start 2023-07-21T19:15:22.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 -- start start 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c1014a0 0x7f8b5c19df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 0x7f8b5c197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1980b0 con 0x7f8b5c1014a0 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c198220 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.790+0000 7f8b6219d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1983e0 con 0x7f8b5c101e50 2023-07-21T19:15:22.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49854/0 (socket says 172.21.15.5:49854) 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 -- 172.21.15.5:0/3119773753 learned_addr learned my addr 172.21.15.5:0/3119773753 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6099a700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c1014a0 0x7f8b5c19df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6199c700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 0x7f8b5c197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 -- 172.21.15.5:0/3119773753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 msgr2=0x7f8b5c197a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 0x7f8b5c197a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 -- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c1014a0 msgr2=0x7f8b5c19df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c1014a0 0x7f8b5c19df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 -- 172.21.15.5:0/3119773753 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580095f0 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6199c700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 0x7f8b5c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.791+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005740 tx=0x7f8b58005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.792+0000 7f8b527fc700 1 -- 172.21.15.5:0/3119773753 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5801d070 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.792+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c198660 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.792+0000 7f8b527fc700 1 -- 172.21.15.5:0/3119773753 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b58022470 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.792+0000 7f8b527fc700 1 -- 172.21.15.5:0/3119773753 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b58017650 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.792+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a49c0 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.795+0000 7f8b527fc700 1 -- 172.21.15.5:0/3119773753 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8b5800fb00 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.796+0000 7f8b76d03b80 1 Processor -- start 2023-07-21T19:15:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 -- start start 2023-07-21T19:15:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b527fc700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 0x7f8b480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825151a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d382515700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38241ac50 con 0x55d38251b8c0 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38246d9e0 con 0x55d3823716b0 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382466e70 con 0x55d38251b470 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b527fc700 1 -- 172.21.15.5:0/3119773753 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8b580ce2f0 con 0x7f8b5c0ffd20 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49856/0 (socket says 172.21.15.5:49856) 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b6119b700 1 -- 172.21.15.5:0/3679422213 learned_addr learned my addr 172.21.15.5:0/3679422213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.797+0000 7f8b6199c700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 0x7f8b480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6199c700 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d382515700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6099a700 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825151a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 -- 172.21.15.5:0/3679422213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 msgr2=0x55d3825151a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825151a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 -- 172.21.15.5:0/3679422213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 msgr2=0x55d382515700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d382515700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 -- 172.21.15.5:0/3679422213 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d382294d80 con 0x55d3823716b0 2023-07-21T19:15:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6119b700 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b580a3280 tx=0x7f8b58005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=30af84ebe3c1e575 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b6199c700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 0x7f8b480a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55d38251bcd0 tx=0x7f8b54009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b477fe700 1 -- 172.21.15.5:0/3679422213 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580cd020 con 0x55d3823716b0 2023-07-21T19:15:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b477fe700 1 -- 172.21.15.5:0/3679422213 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b5806c2b0 con 0x55d3823716b0 2023-07-21T19:15:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.798+0000 7f8b477fe700 1 -- 172.21.15.5:0/3679422213 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580a3820 con 0x55d3823716b0 2023-07-21T19:15:22.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 -- 172.21.15.5:0/3679422213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 msgr2=0x55d38251af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8b580a3280 tx=0x7f8b58005910 comp rx=0 tx=0).stop 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 -- 172.21.15.5:0/3679422213 shutdown_connections 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d382515700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d3823716b0 0x55d38251af30 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/3679422213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825151a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 -- 172.21.15.5:0/3679422213 >> 172.21.15.5:0/3679422213 conn(0x55d38250ad40 msgr2=0x55d382510340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 -- 172.21.15.5:0/3679422213 shutdown_connections 2023-07-21T19:15:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.799+0000 7f8b76d03b80 1 -- 172.21.15.5:0/3679422213 wait complete. 2023-07-21T19:15:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.800+0000 7f8b76d03b80 1 -- start start 2023-07-21T19:15:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.800+0000 7f8b76d03b80 1 Processor -- start 2023-07-21T19:15:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 0x55d382524480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d38251e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382406a60 con 0x55d38251b8c0 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382406760 con 0x55d38251b470 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d3822ea8c0 con 0x55d3823716b0 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b76d03b80 10 client.0 _ll_register_callbacks cb 0x55d382404700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6199c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d38251e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 0x55d382524480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49862/0 (socket says 172.21.15.5:49862) 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6099a700 1 -- 172.21.15.5:0/1408195278 learned_addr learned my addr 172.21.15.5:0/1408195278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.801+0000 7f8b6119b700 1 -- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 msgr2=0x55d3825249c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6119b700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6119b700 1 -- 172.21.15.5:0/1408195278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 msgr2=0x55d38251e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6119b700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d38251e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6119b700 1 -- 172.21.15.5:0/1408195278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c1a49c0 con 0x55d3823716b0 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6099a700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6199c700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d38251e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:22.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b6119b700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 0x55d382524480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005f50 tx=0x7f8b58097130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d488aba65f5b3c20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580cd070 con 0x55d3823716b0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8b5809b3f0 con 0x55d3823716b0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d3823c0490 con 0x55d3823716b0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.802+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b580a37b0 con 0x55d3823716b0 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 10 client.26860 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d38227dc00 con 0x55d3823716b0 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d3823c0490 con 0x55d3823716b0 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8b580cbc60 con 0x55d3823716b0 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 10 client.26860 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 10 client.26860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 20 client.26860 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d38252e4f0 0x55d382530900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 4 client.26860 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d38245fe50 con 0x55d38252e4f0 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.803+0000 7f8b6099a700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d38252e4f0 0x55d382530900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.804+0000 7f8b6099a700 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d38252e4f0 0x55d382530900 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=307efd1100358ef5 server_cookie=b20f805ee72e7c4c in_seq=0 out_seq=0 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.804+0000 7f8b44ff9700 10 client.26860 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b44ff9700 1 -- 172.21.15.5:0/1408195278 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d38245fe50 con 0x55d38252e4f0 2023-07-21T19:15:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b44ff9700 10 client.26860 handle_command_reply: tid=0 2023-07-21T19:15:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 client.26860 shutdown 2023-07-21T19:15:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 msgr2=0x55d382524480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 0x55d382524480 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005f50 tx=0x7f8b58097130 comp rx=0 tx=0).stop 2023-07-21T19:15:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 shutdown_connections 2023-07-21T19:15:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d38252e4f0 0x55d382530900 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d38251b8c0 0x55d38251e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d3823716b0 0x55d382524480 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b44ff9700 0 client.26860 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/1408195278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d3825249c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 >> 172.21.15.5:0/1408195278 conn(0x55d38250ad40 msgr2=0x55d38250edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 shutdown_connections 2023-07-21T19:15:22.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 1 -- 172.21.15.5:0/1408195278 wait complete. 2023-07-21T19:15:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.805+0000 7f8b76d03b80 20 client.26860 trim_cache size 0 max 16384 2023-07-21T19:15:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.827+0000 7f8b76d03b80 1 Processor -- start 2023-07-21T19:15:22.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 -- start start 2023-07-21T19:15:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d38251a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 0x55d38250fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382511120 con 0x55d3823716b0 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d38251fcb0 con 0x55d38251b470 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382516180 con 0x55d38251b8c0 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d38251a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b6099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b6099a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49866/0 (socket says 172.21.15.5:49866) 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.828+0000 7f8b6099a700 1 -- 172.21.15.5:0/886125968 learned_addr learned my addr 172.21.15.5:0/886125968 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:22.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6199c700 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 0x55d38250fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 -- 172.21.15.5:0/886125968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 msgr2=0x55d38250fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 0x55d38250fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 -- 172.21.15.5:0/886125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 msgr2=0x55d38251a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d38251a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 -- 172.21.15.5:0/886125968 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d382535530 con 0x55d38251b470 2023-07-21T19:15:22.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6199c700 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 0x55d38250fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:15:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b6099a700 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55d3825311e0 tx=0x7f8b4c01cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0299fef791d65f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b47fff700 1 -- 172.21.15.5:0/886125968 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c017040 con 0x55d38251b470 2023-07-21T19:15:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b47fff700 1 -- 172.21.15.5:0/886125968 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b4c020070 con 0x55d38251b470 2023-07-21T19:15:22.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.829+0000 7f8b47fff700 1 -- 172.21.15.5:0/886125968 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c005280 con 0x55d38251b470 2023-07-21T19:15:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 -- 172.21.15.5:0/886125968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 msgr2=0x55d38251d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x55d3825311e0 tx=0x7f8b4c01cc30 comp rx=0 tx=0).stop 2023-07-21T19:15:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 -- 172.21.15.5:0/886125968 shutdown_connections 2023-07-21T19:15:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b8c0 0x55d38250fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d38251a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/886125968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b470 0x55d38251d5b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 -- 172.21.15.5:0/886125968 >> 172.21.15.5:0/886125968 conn(0x55d38250ad40 msgr2=0x55d38251cea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.830+0000 7f8b76d03b80 1 -- 172.21.15.5:0/886125968 shutdown_connections 2023-07-21T19:15:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.831+0000 7f8b76d03b80 1 -- 172.21.15.5:0/886125968 wait complete. 2023-07-21T19:15:22.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.831+0000 7f8b76d03b80 1 -- start start 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.831+0000 7f8b76d03b80 1 Processor -- start 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825246b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b8c0 0x55d3825226d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382522ca0 con 0x55d3823716b0 2023-07-21T19:15:22.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382522e10 con 0x55d38251b8c0 2023-07-21T19:15:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d382522fd0 con 0x55d38251b470 2023-07-21T19:15:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b76d03b80 10 client.0 _ll_register_callbacks cb 0x7f8b5809b250 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b6119b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2023-07-21T19:15:22.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b6119b700 1 -- 172.21.15.5:0/4246758855 learned_addr learned my addr 172.21.15.5:0/4246758855 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b6199c700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b8c0 0x55d3825226d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.832+0000 7f8b6099a700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825246b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 -- 172.21.15.5:0/4246758855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 msgr2=0x55d3825246b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825246b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 -- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b8c0 msgr2=0x55d3825226d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b8c0 0x55d3825226d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 -- 172.21.15.5:0/4246758855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b4c0044d0 con 0x55d3823716b0 2023-07-21T19:15:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b6119b700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005950 tx=0x7f8b580094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd29866d8dd8476 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:22.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b44ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b580cd020 con 0x55d3823716b0 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8b5809b3f0 con 0x55d3823716b0 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.833+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8b5806c2b0 con 0x55d3823716b0 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b5809f420 con 0x55d3823716b0 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 10 client.17238 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:22.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d3822bb1f0 con 0x55d3823716b0 2023-07-21T19:15:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b5806c2b0 con 0x55d3823716b0 2023-07-21T19:15:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f8b580a3320 con 0x55d3823716b0 2023-07-21T19:15:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 10 client.17238 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 10 client.17238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:22.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 20 client.17238 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d382583030 0x55d382585410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 4 client.17238 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:15:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.834+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d382421ff0 con 0x55d382583030 2023-07-21T19:15:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.835+0000 7f8b6099a700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d382583030 0x55d382585410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:22.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.835+0000 7f8b44ff9700 10 client.17238 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:22.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.835+0000 7f8b6099a700 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d382583030 0x55d382585410 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32b146c6de7701d2 server_cookie=329ab2dbca3efecd in_seq=0 out_seq=0 2023-07-21T19:15:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b44ff9700 1 -- 172.21.15.5:0/4246758855 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d382421ff0 con 0x55d382583030 2023-07-21T19:15:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b44ff9700 10 client.17238 handle_command_reply: tid=0 2023-07-21T19:15:22.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 client.17238 shutdown 2023-07-21T19:15:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 msgr2=0x55d382524170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005950 tx=0x7f8b580094b0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 shutdown_connections 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d38251b8c0 0x55d3825226d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d382583030 0x55d382585410 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d3823716b0 0x55d382524170 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 --2- 172.21.15.5:0/4246758855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d38251b470 0x55d3825246b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b44ff9700 0 client.17238 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:22.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.853+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 >> 172.21.15.5:0/4246758855 conn(0x55d38250ad40 msgr2=0x55d38251d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.854+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 shutdown_connections 2023-07-21T19:15:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.854+0000 7f8b76d03b80 1 -- 172.21.15.5:0/4246758855 wait complete. 2023-07-21T19:15:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.854+0000 7f8b76d03b80 20 client.17238 trim_cache size 0 max 16384 2023-07-21T19:15:22.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:15:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:15:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:15:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:15:22.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 msgr2=0x7f8b480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 0x7f8b480a37f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x55d38251bcd0 tx=0x7f8b54009920 comp rx=0 tx=0).stop 2023-07-21T19:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 msgr2=0x7f8b5c19d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8b58005740 tx=0x7f8b58005770 comp rx=0 tx=0).stop 2023-07-21T19:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 shutdown_connections 2023-07-21T19:15:22.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b5c101e50 0x7f8b5c197a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b5c0ffd20 0x7f8b5c19d9d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b480a1340 0x7f8b480a37f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 --2- 172.21.15.5:0/3119773753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b5c1014a0 0x7f8b5c19df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 >> 172.21.15.5:0/3119773753 conn(0x7f8b5c0fb440 msgr2=0x7f8b5c072960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.855+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 shutdown_connections 2023-07-21T19:15:22.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:22.856+0000 7f8b6219d700 1 -- 172.21.15.5:0/3119773753 wait complete. 2023-07-21T19:15:23.227 INFO:teuthology.orchestra.run.smithi005.stderr:28500000 of 50000000 tuples (57%) done (elapsed 440.79 s, remaining 332.53 s) 2023-07-21T19:15:23.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:23 smithi101 ceph-mon[128136]: pgmap v240: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:15:23.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:23 smithi005 ceph-mon[126460]: pgmap v240: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:15:23.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:23 smithi156 ceph-mon[128316]: pgmap v240: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 32 MiB/s wr, 44 op/s 2023-07-21T19:15:24.546 INFO:teuthology.orchestra.run.smithi005.stderr:28600000 of 50000000 tuples (57%) done (elapsed 442.11 s, remaining 330.81 s) 2023-07-21T19:15:25.792 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:15:25.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:25 smithi101 ceph-mon[128136]: pgmap v241: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 29 op/s 2023-07-21T19:15:25.857 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:25 smithi005 ceph-mon[126460]: pgmap v241: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 29 op/s 2023-07-21T19:15:25.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:25 smithi156 ceph-mon[128316]: pgmap v241: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 23 MiB/s wr, 29 op/s 2023-07-21T19:15:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 -- 172.21.15.5:0/1118258012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 msgr2=0x7f156410c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 --2- 172.21.15.5:0/1118258012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156410c810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f155c009f90 tx=0x7f155c00b330 comp rx=0 tx=0).stop 2023-07-21T19:15:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 -- 172.21.15.5:0/1118258012 shutdown_connections 2023-07-21T19:15:26.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 --2- 172.21.15.5:0/1118258012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156410c810 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 --2- 172.21.15.5:0/1118258012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 --2- 172.21.15.5:0/1118258012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f15640ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 -- 172.21.15.5:0/1118258012 >> 172.21.15.5:0/1118258012 conn(0x7f15640fb4b0 msgr2=0x7f15640fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 -- 172.21.15.5:0/1118258012 shutdown_connections 2023-07-21T19:15:26.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.008+0000 7f1563fff700 1 -- 172.21.15.5:0/1118258012 wait complete. 2023-07-21T19:15:26.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 Processor -- start 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 -- start start 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f156419b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564195e80 con 0x7f1564100380 2023-07-21T19:15:26.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1564195ff0 con 0x7f15641059e0 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.010+0000 7f1563fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15641961b0 con 0x7f15640ff9d0 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38288/0 (socket says 172.21.15.5:38288) 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 -- 172.21.15.5:0/662548214 learned_addr learned my addr 172.21.15.5:0/662548214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f1562ffd700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f156419b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 -- 172.21.15.5:0/662548214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 msgr2=0x7f156419b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f156419b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 -- 172.21.15.5:0/662548214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 msgr2=0x7f1564195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15627fc700 1 -- 172.21.15.5:0/662548214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f155c009c40 con 0x7f1564100380 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f15637fe700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:26.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.011+0000 7f1562ffd700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f156419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:15:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f15627fc700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9a0 tx=0x7f155800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f154bfff700 1 -- 172.21.15.5:0/662548214 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558009790 con 0x7f1564100380 2023-07-21T19:15:26.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1564196490 con 0x7f1564100380 2023-07-21T19:15:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f154bfff700 1 -- 172.21.15.5:0/662548214 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155800cbd0 con 0x7f1564100380 2023-07-21T19:15:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f154bfff700 1 -- 172.21.15.5:0/662548214 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1558018680 con 0x7f1564100380 2023-07-21T19:15:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.012+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15641a2790 con 0x7f1564100380 2023-07-21T19:15:26.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.016+0000 7f154bfff700 1 -- 172.21.15.5:0/662548214 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f155800cd40 con 0x7f1564100380 2023-07-21T19:15:26.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 Processor -- start 2023-07-21T19:15:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 -- start start 2023-07-21T19:15:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cbf3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cbfdc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc00520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9caf4cb0 con 0x55db9cbfe140 2023-07-21T19:15:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cb47a40 con 0x55db9cbf4210 2023-07-21T19:15:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.017+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cb40ed0 con 0x55db9cbf3900 2023-07-21T19:15:26.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f154bfff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 0x7f154c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cbfdc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc00520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cbfdc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49896/0 (socket says 172.21.15.5:49896) 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f1562ffd700 1 -- 172.21.15.5:0/3659854104 learned_addr learned my addr 172.21.15.5:0/3659854104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15637fe700 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cbf3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f154bfff700 1 -- 172.21.15.5:0/662548214 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1558014070 con 0x7f1564100380 2023-07-21T19:15:26.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15637fe700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 0x7f154c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 -- 172.21.15.5:0/3659854104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 msgr2=0x55db9cbf3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cbf3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 -- 172.21.15.5:0/3659854104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 msgr2=0x55db9cbfdc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cbfdc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.018+0000 7f15627fc700 1 -- 172.21.15.5:0/3659854104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db9c96ede0 con 0x55db9cbfe140 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.019+0000 7f15627fc700 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc00520 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f15580ca690 tx=0x7f1558069490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff84be02c9e7e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.019+0000 7f15637fe700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 0x7f154c0a3890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c10 tx=0x7f155c0096c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.019+0000 7f15497fa700 1 -- 172.21.15.5:0/3659854104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155809a010 con 0x55db9cbfe140 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.019+0000 7f15497fa700 1 -- 172.21.15.5:0/3659854104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f155809e040 con 0x55db9cbfe140 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.019+0000 7f15497fa700 1 -- 172.21.15.5:0/3659854104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f155809a270 con 0x55db9cbfe140 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3659854104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 msgr2=0x55db9cc00520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc00520 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f15580ca690 tx=0x7f1558069490 comp rx=0 tx=0).stop 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3659854104 shutdown_connections 2023-07-21T19:15:26.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc00520 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cbf3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3659854104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cbfdc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3659854104 >> 172.21.15.5:0/3659854104 conn(0x55db9cbe4ff0 msgr2=0x55db9cbea880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3659854104 shutdown_connections 2023-07-21T19:15:26.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.020+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3659854104 wait complete. 2023-07-21T19:15:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.021+0000 7f157ccf8b80 1 -- start start 2023-07-21T19:15:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.021+0000 7f157ccf8b80 1 Processor -- start 2023-07-21T19:15:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.021+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.021+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbf6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.021+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbf6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cae0ac0 con 0x55db9cbfe140 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cae07c0 con 0x55db9cbf3900 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9c9c4920 con 0x55db9cbf4210 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f157ccf8b80 10 client.0 _ll_register_callbacks cb 0x55db9cade760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49906/0 (socket says 172.21.15.5:49906) 2023-07-21T19:15:26.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 -- 172.21.15.5:0/2518823683 learned_addr learned my addr 172.21.15.5:0/2518823683 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f15637fe700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbf6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f15627fc700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbf6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 -- 172.21.15.5:0/2518823683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 msgr2=0x55db9cbf6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbf6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 -- 172.21.15.5:0/2518823683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 msgr2=0x55db9cbf6ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbf6ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.022+0000 7f1562ffd700 1 -- 172.21.15.5:0/2518823683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15641a2790 con 0x55db9cbf3900 2023-07-21T19:15:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f1562ffd700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15540092b0 tx=0x7f15540129b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51eca1fac54b9e30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f1546ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554017070 con 0x55db9cbf3900 2023-07-21T19:15:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db9ca9a4f0 con 0x55db9cbf3900 2023-07-21T19:15:26.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1554004830 con 0x55db9cbf3900 2023-07-21T19:15:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15540049f0 con 0x55db9cbf3900 2023-07-21T19:15:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f157ccf8b80 10 client.26930 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:26.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.023+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55db9c957c60 con 0x55db9cbf3900 2023-07-21T19:15:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55db9ca9a4f0 con 0x55db9cbf3900 2023-07-21T19:15:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f155401c810 con 0x55db9cbf3900 2023-07-21T19:15:26.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 10 client.26930 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 10 client.26930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 20 client.26930 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc12010 0x55db9cc14420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 4 client.26930 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:26.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db9cb39eb0 con 0x55db9cc12010 2023-07-21T19:15:26.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.024+0000 7f15627fc700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc12010 0x55db9cc14420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.025+0000 7f15627fc700 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc12010 0x55db9cc14420 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=33f9850db53758a9 server_cookie=3785bcfafc5dcdce in_seq=0 out_seq=0 2023-07-21T19:15:26.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.025+0000 7f1546ff5700 10 client.26930 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:26.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.025+0000 7f1546ff5700 1 -- 172.21.15.5:0/2518823683 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db9cb39eb0 con 0x55db9cc12010 2023-07-21T19:15:26.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f1546ff5700 10 client.26930 handle_command_reply: tid=0 2023-07-21T19:15:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 client.26930 shutdown 2023-07-21T19:15:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 msgr2=0x55db9cbfd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f15540092b0 tx=0x7f15540129b0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 shutdown_connections 2023-07-21T19:15:26.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc12010 0x55db9cc14420 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbf6ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbfd740 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/2518823683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbf6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f1546ff5700 0 client.26930 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 >> 172.21.15.5:0/2518823683 conn(0x55db9cbe4ff0 msgr2=0x55db9cbea410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 shutdown_connections 2023-07-21T19:15:26.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 1 -- 172.21.15.5:0/2518823683 wait complete. 2023-07-21T19:15:26.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.026+0000 7f157ccf8b80 20 client.26930 trim_cache size 0 max 16384 2023-07-21T19:15:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.045+0000 7f157ccf8b80 1 Processor -- start 2023-07-21T19:15:26.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 -- start start 2023-07-21T19:15:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbfd2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbefdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cbf45e0 con 0x55db9cbfe140 2023-07-21T19:15:26.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cbefb80 con 0x55db9cbf3900 2023-07-21T19:15:26.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cbebef0 con 0x55db9cbf4210 2023-07-21T19:15:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f1562ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49912/0 (socket says 172.21.15.5:49912) 2023-07-21T19:15:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f1562ffd700 1 -- 172.21.15.5:0/74055101 learned_addr learned my addr 172.21.15.5:0/74055101 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:26.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f15637fe700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbefdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.046+0000 7f15627fc700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbfd2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 -- 172.21.15.5:0/74055101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 msgr2=0x55db9cbfd2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbfd2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 -- 172.21.15.5:0/74055101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 msgr2=0x55db9cbefdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbefdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 -- 172.21.15.5:0/74055101 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db9c995250 con 0x55db9cbf3900 2023-07-21T19:15:26.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f15627fc700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbfd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1562ffd700 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15540092e0 tx=0x7f1554012d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8cae2631c7e4baa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1549ffb700 1 -- 172.21.15.5:0/74055101 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1554017070 con 0x55db9cbf3900 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1549ffb700 1 -- 172.21.15.5:0/74055101 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1554018040 con 0x55db9cbf3900 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.047+0000 7f1549ffb700 1 -- 172.21.15.5:0/74055101 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15540036e0 con 0x55db9cbf3900 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 -- 172.21.15.5:0/74055101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 msgr2=0x55db9cbf6b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f15540092e0 tx=0x7f1554012d00 comp rx=0 tx=0).stop 2023-07-21T19:15:26.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 -- 172.21.15.5:0/74055101 shutdown_connections 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cbefdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf3900 0x55db9cbf6b50 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/74055101 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf4210 0x55db9cbfd2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 -- 172.21.15.5:0/74055101 >> 172.21.15.5:0/74055101 conn(0x55db9cbe4ff0 msgr2=0x55db9cbec990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 -- 172.21.15.5:0/74055101 shutdown_connections 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.048+0000 7f157ccf8b80 1 -- 172.21.15.5:0/74055101 wait complete. 2023-07-21T19:15:26.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.049+0000 7f157ccf8b80 1 -- start start 2023-07-21T19:15:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.049+0000 7f157ccf8b80 1 Processor -- start 2023-07-21T19:15:26.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.049+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cc0ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.049+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f157ccf8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc0b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cc0bc90 con 0x55db9cbfe140 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cc0be00 con 0x55db9cbf4210 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f157ccf8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db9cc65e90 con 0x55db9cbf3900 2023-07-21T19:15:26.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f157ccf8b80 10 client.0 _ll_register_callbacks cb 0x7f1554009d20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc0b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49916/0 (socket says 172.21.15.5:49916) 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 -- 172.21.15.5:0/3641456693 learned_addr learned my addr 172.21.15.5:0/3641456693 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f1562ffd700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cc0ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 -- 172.21.15.5:0/3641456693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 msgr2=0x55db9cc0ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cc0ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 -- 172.21.15.5:0/3641456693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 msgr2=0x55db9cc0b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc0b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.050+0000 7f15627fc700 1 -- 172.21.15.5:0/3641456693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55db9c957c60 con 0x55db9cbf4210 2023-07-21T19:15:26.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f15627fc700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1558098360 tx=0x7f155809a190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=63fa1758e03eb7e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15580a2460 con 0x55db9cbf4210 2023-07-21T19:15:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f155809e040 con 0x55db9cbf4210 2023-07-21T19:15:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1558069070 con 0x55db9cbf4210 2023-07-21T19:15:26.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1558099740 con 0x55db9cbf4210 2023-07-21T19:15:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f157ccf8b80 10 client.26936 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55db9c96d2b0 con 0x55db9cbf4210 2023-07-21T19:15:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15580b6080 con 0x55db9cbf4210 2023-07-21T19:15:26.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f15580a1fd0 con 0x55db9cbf4210 2023-07-21T19:15:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.051+0000 7f157ccf8b80 10 client.26936 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f157ccf8b80 10 client.26936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:26.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f157ccf8b80 20 client.26936 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc672f0 0x55db9cc69700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f157ccf8b80 4 client.26936 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:15:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55db9cafc050 con 0x55db9cc672f0 2023-07-21T19:15:26.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.052+0000 7f1562ffd700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc672f0 0x55db9cc69700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f1562ffd700 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc672f0 0x55db9cc69700 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4f0b81cbbcbfff00 server_cookie=6d6e7b842b25f8a6 in_seq=0 out_seq=0 2023-07-21T19:15:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f1546ff5700 10 client.26936 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:26.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f1546ff5700 1 -- 172.21.15.5:0/3641456693 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55db9cafc050 con 0x55db9cc672f0 2023-07-21T19:15:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f1546ff5700 10 client.26936 handle_command_reply: tid=0 2023-07-21T19:15:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 client.26936 shutdown 2023-07-21T19:15:26.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 msgr2=0x55db9cc0b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1558098360 tx=0x7f155809a190 comp rx=0 tx=0).stop 2023-07-21T19:15:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 shutdown_connections 2023-07-21T19:15:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55db9cc672f0 0x55db9cc69700 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55db9cbfe140 0x55db9cc0b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55db9cbf3900 0x55db9cc0ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f1546ff5700 0 client.26936 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 --2- 172.21.15.5:0/3641456693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55db9cbf4210 0x55db9cc0b090 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 >> 172.21.15.5:0/3641456693 conn(0x55db9cbe4ff0 msgr2=0x55db9cbefdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 shutdown_connections 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 1 -- 172.21.15.5:0/3641456693 wait complete. 2023-07-21T19:15:26.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.053+0000 7f157ccf8b80 20 client.26936 trim_cache size 0 max 16384 2023-07-21T19:15:26.067 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:15:26.067 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:15:26.067 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:15:26.067 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:15:26.067 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:15:26.068 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:15:26.068 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:15:26.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:15:26.068 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:15:26.068 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:15:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 msgr2=0x7f154c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 0x7f154c0a3890 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f155c009c10 tx=0x7f155c0096c0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 msgr2=0x7f156419bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:26.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f155800e9a0 tx=0x7f155800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 shutdown_connections 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15641059e0 0x7f1564195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15640ff9d0 0x7f156419b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f154c0a13e0 0x7f154c0a3890 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 --2- 172.21.15.5:0/662548214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1564100380 0x7f156419bce0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 >> 172.21.15.5:0/662548214 conn(0x7f15640fb4b0 msgr2=0x7f15640fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:26.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 shutdown_connections 2023-07-21T19:15:26.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:26.054+0000 7f1563fff700 1 -- 172.21.15.5:0/662548214 wait complete. 2023-07-21T19:15:26.074 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:15:26.218 INFO:teuthology.orchestra.run.smithi005.stderr:28700000 of 50000000 tuples (57%) done (elapsed 443.78 s, remaining 329.36 s) 2023-07-21T19:15:27.597 INFO:teuthology.orchestra.run.smithi005.stderr:28800000 of 50000000 tuples (57%) done (elapsed 445.16 s, remaining 327.69 s) 2023-07-21T19:15:27.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:27 smithi101 ceph-mon[128136]: pgmap v242: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 27 op/s 2023-07-21T19:15:27.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:27 smithi005 ceph-mon[126460]: pgmap v242: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 27 op/s 2023-07-21T19:15:27.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:27 smithi156 ceph-mon[128316]: pgmap v242: 97 pgs: 97 active+clean; 6.5 GiB data, 2.9 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 22 MiB/s wr, 27 op/s 2023-07-21T19:15:28.884 INFO:teuthology.orchestra.run.smithi005.stderr:28900000 of 50000000 tuples (57%) done (elapsed 446.45 s, remaining 325.95 s) 2023-07-21T19:15:29.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:29 smithi101 ceph-mon[128136]: pgmap v243: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-07-21T19:15:29.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:29 smithi005 ceph-mon[126460]: pgmap v243: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-07-21T19:15:29.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:29 smithi156 ceph-mon[128316]: pgmap v243: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 33 MiB/s wr, 48 op/s 2023-07-21T19:15:30.177 INFO:teuthology.orchestra.run.smithi005.stderr:29000000 of 50000000 tuples (58%) done (elapsed 447.74 s, remaining 324.23 s) 2023-07-21T19:15:31.572 INFO:teuthology.orchestra.run.smithi005.stderr:29100000 of 50000000 tuples (58%) done (elapsed 449.14 s, remaining 322.58 s) 2023-07-21T19:15:31.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:31 smithi101 ceph-mon[128136]: pgmap v244: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:15:31.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:31 smithi005 ceph-mon[126460]: pgmap v244: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:15:31.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:31 smithi156 ceph-mon[128316]: pgmap v244: 97 pgs: 97 active+clean; 6.6 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:15:32.876 INFO:teuthology.orchestra.run.smithi005.stderr:29200000 of 50000000 tuples (58%) done (elapsed 450.44 s, remaining 320.86 s) 2023-07-21T19:15:33.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:33 smithi101 ceph-mon[128136]: pgmap v245: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 45 op/s 2023-07-21T19:15:33.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:33 smithi005 ceph-mon[126460]: pgmap v245: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 45 op/s 2023-07-21T19:15:33.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:33 smithi156 ceph-mon[128316]: pgmap v245: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 31 MiB/s wr, 45 op/s 2023-07-21T19:15:34.195 INFO:teuthology.orchestra.run.smithi005.stderr:29300000 of 50000000 tuples (58%) done (elapsed 451.76 s, remaining 319.16 s) 2023-07-21T19:15:35.499 INFO:teuthology.orchestra.run.smithi005.stderr:29400000 of 50000000 tuples (58%) done (elapsed 453.06 s, remaining 317.45 s) 2023-07-21T19:15:35.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:35 smithi101 ceph-mon[128136]: pgmap v246: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:15:35.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:35 smithi005 ceph-mon[126460]: pgmap v246: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:15:35.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:35 smithi156 ceph-mon[128316]: pgmap v246: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 MiB/s wr, 40 op/s 2023-07-21T19:15:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.031+0000 7fce609fe700 1 -- 172.21.15.5:0/3013567936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c0ffd20 msgr2=0x7fce5c1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.031+0000 7fce609fe700 1 --2- 172.21.15.5:0/3013567936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c0ffd20 0x7fce5c1000f0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fce4c009990 tx=0x7fce4c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 -- 172.21.15.5:0/3013567936 shutdown_connections 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 --2- 172.21.15.5:0/3013567936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c101e50 0x7fce5c10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 --2- 172.21.15.5:0/3013567936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c0ffd20 0x7fce5c1000f0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 --2- 172.21.15.5:0/3013567936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 -- 172.21.15.5:0/3013567936 >> 172.21.15.5:0/3013567936 conn(0x7fce5c0fb440 msgr2=0x7fce5c0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 -- 172.21.15.5:0/3013567936 shutdown_connections 2023-07-21T19:15:36.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.032+0000 7fce609fe700 1 -- 172.21.15.5:0/3013567936 wait complete. 2023-07-21T19:15:36.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 Processor -- start 2023-07-21T19:15:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 -- start start 2023-07-21T19:15:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c0ffd20 0x7fce5c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c19df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 0x7fce5c197a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1980c0 con 0x7fce5c1014a0 2023-07-21T19:15:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c198230 con 0x7fce5c101e50 2023-07-21T19:15:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.033+0000 7fce609fe700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce5c1983f0 con 0x7fce5c0ffd20 2023-07-21T19:15:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c19df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 0x7fce5c197a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c19df20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42954/0 (socket says 172.21.15.5:42954) 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5ad9d700 1 -- 172.21.15.5:0/3345503478 learned_addr learned my addr 172.21.15.5:0/3345503478 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c0ffd20 0x7fce5c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 -- 172.21.15.5:0/3345503478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c0ffd20 msgr2=0x7fce5c19d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c0ffd20 0x7fce5c19d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 -- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 msgr2=0x7fce5c19df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c19df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.034+0000 7fce5b59e700 1 -- 172.21.15.5:0/3345503478 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce4c0095f0 con 0x7fce5c101e50 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce5b59e700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 0x7fce5c197a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fce54007f10 tx=0x7fce5400d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce58d99700 1 -- 172.21.15.5:0/3345503478 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54009ea0 con 0x7fce5c101e50 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce58d99700 1 -- 172.21.15.5:0/3345503478 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce5400f040 con 0x7fce5c101e50 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5c1986d0 con 0x7fce5c101e50 2023-07-21T19:15:36.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce58d99700 1 -- 172.21.15.5:0/3345503478 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce54004280 con 0x7fce5c101e50 2023-07-21T19:15:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.035+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c1a49d0 con 0x7fce5c101e50 2023-07-21T19:15:36.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.040+0000 7fce58d99700 1 -- 172.21.15.5:0/3345503478 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fce5400db20 con 0x7fce5c101e50 2023-07-21T19:15:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 Processor -- start 2023-07-21T19:15:36.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 -- start start 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958318f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d9583131a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958313700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958218c50 con 0x55d95816f6b0 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95826b9e0 con 0x55d958319470 2023-07-21T19:15:36.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.041+0000 7fce74be6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958264e70 con 0x55d9583198c0 2023-07-21T19:15:36.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958318f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d9583131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958318f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42970/0 (socket says 172.21.15.5:42970) 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce5ad9d700 1 -- 172.21.15.5:0/3124990291 learned_addr learned my addr 172.21.15.5:0/3124990291 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce58d99700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 0x7fce480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958313700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce58d99700 1 -- 172.21.15.5:0/3345503478 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fce540cc530 con 0x7fce5c101e50 2023-07-21T19:15:36.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 -- 172.21.15.5:0/3124990291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 msgr2=0x55d958313700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958313700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce5b59e700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 0x7fce480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 -- 172.21.15.5:0/3124990291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 msgr2=0x55d958318f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958318f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.042+0000 7fce52d9d700 1 -- 172.21.15.5:0/3124990291 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d958092d80 con 0x55d958319470 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.043+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d9583131a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098e0 tx=0x7fce4c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b439156f4e7b06c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.043+0000 7fce5159a700 1 -- 172.21.15.5:0/3124990291 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c01d070 con 0x55d958319470 2023-07-21T19:15:36.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.043+0000 7fce5b59e700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 0x7fce480a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce540c9d30 tx=0x7fce54009530 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.043+0000 7fce5159a700 1 -- 172.21.15.5:0/3124990291 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce4c01e040 con 0x55d958319470 2023-07-21T19:15:36.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.043+0000 7fce5159a700 1 -- 172.21.15.5:0/3124990291 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c004740 con 0x55d958319470 2023-07-21T19:15:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 -- 172.21.15.5:0/3124990291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 msgr2=0x55d9583131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d9583131a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fce4c0098e0 tx=0x7fce4c005f00 comp rx=0 tx=0).stop 2023-07-21T19:15:36.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 -- 172.21.15.5:0/3124990291 shutdown_connections 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958313700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958318f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3124990291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d9583131a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 -- 172.21.15.5:0/3124990291 >> 172.21.15.5:0/3124990291 conn(0x55d958308d40 msgr2=0x55d95830e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 -- 172.21.15.5:0/3124990291 shutdown_connections 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.044+0000 7fce74be6b80 1 -- 172.21.15.5:0/3124990291 wait complete. 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.045+0000 7fce74be6b80 1 -- start start 2023-07-21T19:15:36.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.045+0000 7fce74be6b80 1 Processor -- start 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d958322c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d95831c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958204a60 con 0x55d95816f6b0 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958204760 con 0x55d958319470 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9580e88c0 con 0x55d9583198c0 2023-07-21T19:15:36.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce74be6b80 10 client.0 _ll_register_callbacks cb 0x55d958202700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42974/0 (socket says 172.21.15.5:42974) 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d958322c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce5ad9d700 1 -- 172.21.15.5:0/3761628964 learned_addr learned my addr 172.21.15.5:0/3761628964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d95831c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 -- 172.21.15.5:0/3761628964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 msgr2=0x55d95831c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d95831c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 -- 172.21.15.5:0/3761628964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 msgr2=0x55d958322750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce52d9d700 1 -- 172.21.15.5:0/3761628964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce5c10d1a0 con 0x55d958319470 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.046+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d958322c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c00 tx=0x7fce4c00b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a0eaeb0eba56b52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce427fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c01d030 con 0x55d958319470 2023-07-21T19:15:36.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9581be490 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fce4c004700 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c0048c0 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.047+0000 7fce74be6b80 10 client.26948 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d95807bc00 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d9581be490 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fce4c00fb70 con 0x55d958319470 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 10 client.26948 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:36.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 10 client.26948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 20 client.26948 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d95832c8d0 0x55d95832ece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 4 client.26948 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:36.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.048+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d95825de50 con 0x55d95832c8d0 2023-07-21T19:15:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.049+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d95832c8d0 0x55d95832ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.049+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d95832c8d0 0x55d95832ece0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d153ecec7497aea server_cookie=d7a7762107df7ce in_seq=0 out_seq=0 2023-07-21T19:15:36.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.049+0000 7fce427fc700 10 client.26948 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce427fc700 1 -- 172.21.15.5:0/3761628964 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d95825de50 con 0x55d95832c8d0 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce427fc700 10 client.26948 handle_command_reply: tid=0 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 client.26948 shutdown 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 msgr2=0x55d958322c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d958322c90 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fce4c000c00 tx=0x7fce4c00b9d0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 shutdown_connections 2023-07-21T19:15:36.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d95831c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d95832c8d0 0x55d95832ece0 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958322750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3761628964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d958322c90 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce427fc700 0 client.26948 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.050+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 >> 172.21.15.5:0/3761628964 conn(0x55d958308d40 msgr2=0x55d95830d640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.051+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 shutdown_connections 2023-07-21T19:15:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.051+0000 7fce74be6b80 1 -- 172.21.15.5:0/3761628964 wait complete. 2023-07-21T19:15:36.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.051+0000 7fce74be6b80 20 client.26948 trim_cache size 0 max 16384 2023-07-21T19:15:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.069+0000 7fce74be6b80 1 Processor -- start 2023-07-21T19:15:36.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 -- start start 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958313980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958318110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958313f30 con 0x55d95816f6b0 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95831df80 con 0x55d958319470 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce74be6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d958318b80 con 0x55d9583198c0 2023-07-21T19:15:36.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958313980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce52d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce52d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60174/0 (socket says 172.21.15.5:60174) 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958313980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42988/0 (socket says 172.21.15.5:42988) 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.070+0000 7fce52d9d700 1 -- 172.21.15.5:0/545729670 learned_addr learned my addr 172.21.15.5:0/545729670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 -- 172.21.15.5:0/545729670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 msgr2=0x55d958318110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958318110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 -- 172.21.15.5:0/545729670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 msgr2=0x55d958313980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958313980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 -- 172.21.15.5:0/545729670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9580b91f0 con 0x55d958319470 2023-07-21T19:15:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce52d9d700 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00ff80 tx=0x7fce4c0098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10b3cce3d5d238b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce51d9b700 1 -- 172.21.15.5:0/545729670 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c01d070 con 0x55d958319470 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce51d9b700 1 -- 172.21.15.5:0/545729670 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce4c02c520 con 0x55d958319470 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.071+0000 7fce51d9b700 1 -- 172.21.15.5:0/545729670 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c02a910 con 0x55d958319470 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 -- 172.21.15.5:0/545729670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 msgr2=0x55d95831a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fce4c00ff80 tx=0x7fce4c0098c0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 -- 172.21.15.5:0/545729670 shutdown_connections 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958318110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d95816f6b0 0x55d958313980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 --2- 172.21.15.5:0/545729670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d958319470 0x55d95831a520 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 -- 172.21.15.5:0/545729670 >> 172.21.15.5:0/545729670 conn(0x55d958308d40 msgr2=0x55d958317a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 -- 172.21.15.5:0/545729670 shutdown_connections 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.072+0000 7fce74be6b80 1 -- 172.21.15.5:0/545729670 wait complete. 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 -- start start 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 Processor -- start 2023-07-21T19:15:36.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 0x55d9583202c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 0x55d958320800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958320d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95837f860 con 0x55d958319470 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95837f9d0 con 0x55d95816f6b0 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.073+0000 7fce74be6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d95837fb90 con 0x55d9583198c0 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce74be6b80 10 client.0 _ll_register_callbacks cb 0x7fce44009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 0x55d9583202c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 0x55d958320800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce5ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 0x55d9583202c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60180/0 (socket says 172.21.15.5:60180) 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce5ad9d700 1 -- 172.21.15.5:0/3142093576 learned_addr learned my addr 172.21.15.5:0/3142093576 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958320d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 -- 172.21.15.5:0/3142093576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 msgr2=0x55d958320d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958320d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 -- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 msgr2=0x55d9583202c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 0x55d9583202c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 -- 172.21.15.5:0/3142093576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d95807bc00 con 0x55d958319470 2023-07-21T19:15:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce52d9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 0x55d958320800 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55d95832a300 tx=0x7fce4c005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5683d36c7b2995 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:36.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.074+0000 7fce427fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:15:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4c01d070 con 0x55d958319470 2023-07-21T19:15:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fce4c000e40 con 0x55d958319470 2023-07-21T19:15:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fce4c01e040 con 0x55d958319470 2023-07-21T19:15:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fce4c02cc50 con 0x55d958319470 2023-07-21T19:15:36.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce74be6b80 10 client.17274 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55d958091250 con 0x55d958319470 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.075+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fce4c050020 con 0x55d958319470 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fce4c00f4f0 con 0x55d958319470 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 10 client.17274 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 10 client.17274 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 20 client.17274 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d958380e20 0x55d958383200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:36.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 4 client.17274 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:15:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d95821fff0 con 0x55d958380e20 2023-07-21T19:15:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.076+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d958380e20 0x55d958383200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.077+0000 7fce5ad9d700 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d958380e20 0x55d958383200 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18b5ee9abdb0a4ce server_cookie=869e114e4c8e402c in_seq=0 out_seq=0 2023-07-21T19:15:36.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.077+0000 7fce427fc700 10 client.17274 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce427fc700 1 -- 172.21.15.5:0/3142093576 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d95821fff0 con 0x55d958380e20 2023-07-21T19:15:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce427fc700 10 client.17274 handle_command_reply: tid=0 2023-07-21T19:15:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 client.17274 shutdown 2023-07-21T19:15:36.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 msgr2=0x55d958320800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 0x55d958320800 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x55d95832a300 tx=0x7fce4c005a00 comp rx=0 tx=0).stop 2023-07-21T19:15:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 shutdown_connections 2023-07-21T19:15:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9583198c0 0x55d958320d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d95816f6b0 0x55d9583202c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d958380e20 0x55d958383200 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 --2- 172.21.15.5:0/3142093576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d958319470 0x55d958320800 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 >> 172.21.15.5:0/3142093576 conn(0x55d958308d40 msgr2=0x55d958312a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce427fc700 0 client.17274 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 shutdown_connections 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.090+0000 7fce74be6b80 1 -- 172.21.15.5:0/3142093576 wait complete. 2023-07-21T19:15:36.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce74be6b80 20 client.17274 trim_cache size 0 max 16384 2023-07-21T19:15:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:15:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:15:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:15:36.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:15:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 msgr2=0x7fce480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 0x7fce480a37f0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fce540c9d30 tx=0x7fce54009530 comp rx=0 tx=0).stop 2023-07-21T19:15:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 msgr2=0x7fce5c197a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 0x7fce5c197a50 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fce54007f10 tx=0x7fce5400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 shutdown_connections 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fce5c101e50 0x7fce5c197a50 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fce5c0ffd20 0x7fce5c19d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fce480a1340 0x7fce480a37f0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 --2- 172.21.15.5:0/3345503478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fce5c1014a0 0x7fce5c19df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.091+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 >> 172.21.15.5:0/3345503478 conn(0x7fce5c0fb440 msgr2=0x7fce5c0fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.092+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 shutdown_connections 2023-07-21T19:15:36.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:36.092+0000 7fce609fe700 1 -- 172.21.15.5:0/3345503478 wait complete. 2023-07-21T19:15:36.942 INFO:teuthology.orchestra.run.smithi005.stderr:29500000 of 50000000 tuples (59%) done (elapsed 454.51 s, remaining 315.84 s) 2023-07-21T19:15:37.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:37 smithi101 ceph-mon[128136]: pgmap v247: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:15:37.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:37 smithi005 ceph-mon[126460]: pgmap v247: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:15:37.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:37 smithi156 ceph-mon[128316]: pgmap v247: 97 pgs: 97 active+clean; 6.7 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:15:38.258 INFO:teuthology.orchestra.run.smithi005.stderr:29600000 of 50000000 tuples (59%) done (elapsed 455.82 s, remaining 314.15 s) 2023-07-21T19:15:38.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:38 smithi005 ceph-mon[126460]: pgmap v248: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 58 op/s 2023-07-21T19:15:38.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:38 smithi156 ceph-mon[128316]: pgmap v248: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 58 op/s 2023-07-21T19:15:39.080 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:38 smithi101 ceph-mon[128136]: pgmap v248: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 58 op/s 2023-07-21T19:15:39.588 INFO:teuthology.orchestra.run.smithi005.stderr:29700000 of 50000000 tuples (59%) done (elapsed 457.15 s, remaining 312.47 s) 2023-07-21T19:15:41.005 INFO:teuthology.orchestra.run.smithi005.stderr:29800000 of 50000000 tuples (59%) done (elapsed 458.57 s, remaining 310.84 s) 2023-07-21T19:15:41.580 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:41 smithi101 ceph-mon[128136]: pgmap v249: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:15:41.658 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:41 smithi005 ceph-mon[126460]: pgmap v249: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:15:41.669 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:41 smithi156 ceph-mon[128316]: pgmap v249: 97 pgs: 97 active+clean; 6.8 GiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 22 MiB/s wr, 37 op/s 2023-07-21T19:15:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:29900000 of 50000000 tuples (59%) done (elapsed 459.87 s, remaining 309.14 s) 2023-07-21T19:15:43.624 INFO:teuthology.orchestra.run.smithi005.stderr:30000000 of 50000000 tuples (60%) done (elapsed 461.19 s, remaining 307.46 s) 2023-07-21T19:15:43.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:43 smithi101 ceph-mon[128136]: pgmap v250: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 56 op/s 2023-07-21T19:15:43.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:43 smithi005 ceph-mon[126460]: pgmap v250: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 56 op/s 2023-07-21T19:15:43.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:43 smithi156 ceph-mon[128316]: pgmap v250: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 33 MiB/s wr, 56 op/s 2023-07-21T19:15:44.943 INFO:teuthology.orchestra.run.smithi005.stderr:30100000 of 50000000 tuples (60%) done (elapsed 462.51 s, remaining 305.78 s) 2023-07-21T19:15:45.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:45 smithi101 ceph-mon[128136]: pgmap v251: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 43 op/s 2023-07-21T19:15:45.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:45 smithi005 ceph-mon[126460]: pgmap v251: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 43 op/s 2023-07-21T19:15:45.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:45 smithi156 ceph-mon[128316]: pgmap v251: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 24 MiB/s wr, 43 op/s 2023-07-21T19:15:46.384 INFO:teuthology.orchestra.run.smithi005.stderr:30200000 of 50000000 tuples (60%) done (elapsed 463.95 s, remaining 304.18 s) 2023-07-21T19:15:47.731 INFO:teuthology.orchestra.run.smithi005.stderr:30300000 of 50000000 tuples (60%) done (elapsed 465.30 s, remaining 302.52 s) 2023-07-21T19:15:47.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:47 smithi101 ceph-mon[128136]: pgmap v252: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 23 MiB/s wr, 42 op/s 2023-07-21T19:15:47.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:47 smithi005 ceph-mon[126460]: pgmap v252: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 23 MiB/s wr, 42 op/s 2023-07-21T19:15:47.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:47 smithi156 ceph-mon[128316]: pgmap v252: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 23 MiB/s wr, 42 op/s 2023-07-21T19:15:49.140 INFO:teuthology.orchestra.run.smithi005.stderr:30400000 of 50000000 tuples (60%) done (elapsed 466.71 s, remaining 300.90 s) 2023-07-21T19:15:49.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 -- 172.21.15.5:0/2115147353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0105b10 msgr2=0x7fd4d0105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/2115147353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0105b10 0x7fd4d0105ee0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc009990 tx=0x7fd4cc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 -- 172.21.15.5:0/2115147353 shutdown_connections 2023-07-21T19:15:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/2115147353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 0x7fd4d010c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/2115147353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d0105b10 0x7fd4d0105ee0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/2115147353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d00ffb00 0x7fd4d00fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 -- 172.21.15.5:0/2115147353 >> 172.21.15.5:0/2115147353 conn(0x7fd4d00fb600 msgr2=0x7fd4d00fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 -- 172.21.15.5:0/2115147353 shutdown_connections 2023-07-21T19:15:49.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.261+0000 7fd4d79ff700 1 -- 172.21.15.5:0/2115147353 wait complete. 2023-07-21T19:15:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.262+0000 7fd4d79ff700 1 Processor -- start 2023-07-21T19:15:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 -- start start 2023-07-21T19:15:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 0x7fd4d0198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 0x7fd4d019c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019cf40 con 0x7fd4d00ffb00 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019d0b0 con 0x7fd4d0105b10 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d79ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d019d270 con 0x7fd4d01004b0 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53374/0 (socket says 172.21.15.5:53374) 2023-07-21T19:15:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d579b700 1 -- 172.21.15.5:0/4051701881 learned_addr learned my addr 172.21.15.5:0/4051701881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d5f9c700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 0x7fd4d019c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.263+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 0x7fd4d0198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 -- 172.21.15.5:0/4051701881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 msgr2=0x7fd4d0198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 0x7fd4d0198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 -- 172.21.15.5:0/4051701881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 msgr2=0x7fd4d019c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 0x7fd4d019c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 -- 172.21.15.5:0/4051701881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4cc0095f0 con 0x7fd4d00ffb00 2023-07-21T19:15:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d5f9c700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 0x7fd4d019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4d579b700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005740 tx=0x7fd4cc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.264+0000 7fd4c67fc700 1 -- 172.21.15.5:0/4051701881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc01d070 con 0x7fd4d00ffb00 2023-07-21T19:15:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.265+0000 7fd4c67fc700 1 -- 172.21.15.5:0/4051701881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4cc022470 con 0x7fd4d00ffb00 2023-07-21T19:15:49.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.265+0000 7fd4c67fc700 1 -- 172.21.15.5:0/4051701881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc017650 con 0x7fd4d00ffb00 2023-07-21T19:15:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.265+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d019d4f0 con 0x7fd4d00ffb00 2023-07-21T19:15:49.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.265+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d019d770 con 0x7fd4d00ffb00 2023-07-21T19:15:49.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.269+0000 7fd4c67fc700 1 -- 172.21.15.5:0/4051701881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd4cc00fb00 con 0x7fd4d00ffb00 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.269+0000 7fd4eb255b80 1 Processor -- start 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 -- start start 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc2355d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2350040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc2255c50 con 0x55cbc23562d0 2023-07-21T19:15:49.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc22a89e0 con 0x55cbc2350580 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.270+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc22a1e70 con 0x55cbc21ac6b0 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58190/0 (socket says 172.21.15.5:58190) 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d5f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc2355d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/1855651414 learned_addr learned my addr 172.21.15.5:0/1855651414 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d579b700 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2350040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4c67fc700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 0x7fd4bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/1855651414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 msgr2=0x55cbc2355d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc2355d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/1855651414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 msgr2=0x55cbc2350040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2350040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/1855651414 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc20cfd80 con 0x55cbc2350580 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4c67fc700 1 -- 172.21.15.5:0/4051701881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4cc0ce0c0 con 0x7fd4d00ffb00 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d5f9c700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 0x7fd4bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.271+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009d20 tx=0x7fd4c0011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1d80cb9970b1ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.272+0000 7fd4bb7fe700 1 -- 172.21.15.5:0/1855651414 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0019630 con 0x55cbc2350580 2023-07-21T19:15:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.272+0000 7fd4bb7fe700 1 -- 172.21.15.5:0/1855651414 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c0019c20 con 0x55cbc2350580 2023-07-21T19:15:49.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.272+0000 7fd4bb7fe700 1 -- 172.21.15.5:0/1855651414 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c000eab0 con 0x55cbc2350580 2023-07-21T19:15:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- 172.21.15.5:0/1855651414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 msgr2=0x55cbc23509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4d5f9c700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 0x7fd4bc0a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009550 tx=0x7fd4c800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0009d20 tx=0x7fd4c0011720 comp rx=0 tx=0).stop 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- 172.21.15.5:0/1855651414 shutdown_connections 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc23509d0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc2355d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/1855651414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2350040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- 172.21.15.5:0/1855651414 >> 172.21.15.5:0/1855651414 conn(0x55cbc2345be0 msgr2=0x55cbc234af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- 172.21.15.5:0/1855651414 shutdown_connections 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- 172.21.15.5:0/1855651414 wait complete. 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.273+0000 7fd4eb255b80 1 -- start start 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.274+0000 7fd4eb255b80 1 Processor -- start 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2359f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc2241a60 con 0x55cbc23562d0 2023-07-21T19:15:49.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc2241760 con 0x55cbc2350580 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc21258c0 con 0x55cbc21ac6b0 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4eb255b80 10 client.0 _ll_register_callbacks cb 0x55cbc223f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d5f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2359f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55016/0 (socket says 172.21.15.5:55016) 2023-07-21T19:15:49.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58198/0 (socket says 172.21.15.5:58198) 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 -- 172.21.15.5:0/2722798256 learned_addr learned my addr 172.21.15.5:0/2722798256 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 -- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 msgr2=0x55cbc2359800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 -- 172.21.15.5:0/2722798256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 msgr2=0x55cbc2359f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2359f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d579b700 1 -- 172.21.15.5:0/2722798256 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4c0009980 con 0x55cbc21ac6b0 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.275+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:49.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4d579b700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0976c0 tx=0x7fd4cc0cbd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4974b11ef25f61e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc09ab20 con 0x55cbc21ac6b0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4cc09b1c0 con 0x55cbc21ac6b0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cbc21fb490 con 0x55cbc21ac6b0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4cc0b3030 con 0x55cbc21ac6b0 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4eb255b80 10 client.26908 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:49.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55cbc20b8c00 con 0x55cbc21ac6b0 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.276+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cbc21fb490 con 0x55cbc21ac6b0 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd4cc0d00d0 con 0x55cbc21ac6b0 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 10 client.26908 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 10 client.26908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 20 client.26908 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23697e0 0x55cbc236bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 4 client.26908 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:49.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cbc229ae50 con 0x55cbc23697e0 2023-07-21T19:15:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.277+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23697e0 0x55cbc236bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.278+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23697e0 0x55cbc236bbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a637272f45011845 server_cookie=e699a8d1c9d1bfbb in_seq=0 out_seq=0 2023-07-21T19:15:49.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.278+0000 7fd4b8ff9700 10 client.26908 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.278+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/2722798256 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cbc229ae50 con 0x55cbc23697e0 2023-07-21T19:15:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.278+0000 7fd4b8ff9700 10 client.26908 handle_command_reply: tid=0 2023-07-21T19:15:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.278+0000 7fd4eb255b80 1 client.26908 shutdown 2023-07-21T19:15:49.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 msgr2=0x55cbc235c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc0976c0 tx=0x7fd4cc0cbd10 comp rx=0 tx=0).stop 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 shutdown_connections 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23697e0 0x55cbc236bbf0 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc2359f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc21ac6b0 0x55cbc235c5f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2722798256 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 >> 172.21.15.5:0/2722798256 conn(0x55cbc2345be0 msgr2=0x55cbc234a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4b8ff9700 0 client.26908 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 shutdown_connections 2023-07-21T19:15:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2722798256 wait complete. 2023-07-21T19:15:49.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.279+0000 7fd4eb255b80 20 client.26908 trim_cache size 0 max 16384 2023-07-21T19:15:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.296+0000 7fd4eb255b80 1 Processor -- start 2023-07-21T19:15:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.296+0000 7fd4eb255b80 1 -- start start 2023-07-21T19:15:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 0x55cbc2350040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc23562d0 0x55cbc2360c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc2342c00 con 0x55cbc21ac6b0 2023-07-21T19:15:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc234bb70 con 0x55cbc2350580 2023-07-21T19:15:49.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc23559c0 con 0x55cbc23562d0 2023-07-21T19:15:49.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 0x55cbc2350040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4d4f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58210/0 (socket says 172.21.15.5:58210) 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/2545403820 learned_addr learned my addr 172.21.15.5:0/2545403820 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.297+0000 7fd4d5f9c700 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc23562d0 0x55cbc2360c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/2545403820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc23562d0 msgr2=0x55cbc2360c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc23562d0 0x55cbc2360c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/2545403820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 msgr2=0x55cbc2350040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 0x55cbc2350040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 -- 172.21.15.5:0/2545403820 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cbc20f61f0 con 0x55cbc2350580 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d579b700 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 0x55cbc2350040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005950 tx=0x7fd4c00098e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e8f299357aa4933 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4bbfff700 1 -- 172.21.15.5:0/2545403820 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c00199c0 con 0x55cbc2350580 2023-07-21T19:15:49.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4bbfff700 1 -- 172.21.15.5:0/2545403820 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4c0005ce0 con 0x55cbc2350580 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.298+0000 7fd4bbfff700 1 -- 172.21.15.5:0/2545403820 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0005040 con 0x55cbc2350580 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2545403820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 msgr2=0x55cbc2359800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd4c0005950 tx=0x7fd4c00098e0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2545403820 shutdown_connections 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc23562d0 0x55cbc2360c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc21ac6b0 0x55cbc2350040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/2545403820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc2350580 0x55cbc2359800 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2545403820 >> 172.21.15.5:0/2545403820 conn(0x55cbc2345be0 msgr2=0x55cbc23590f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2545403820 shutdown_connections 2023-07-21T19:15:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.299+0000 7fd4eb255b80 1 -- 172.21.15.5:0/2545403820 wait complete. 2023-07-21T19:15:49.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 -- start start 2023-07-21T19:15:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 Processor -- start 2023-07-21T19:15:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc2350580 0x55cbc235ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc235f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc235f7f0 con 0x55cbc23562d0 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.300+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc235f960 con 0x55cbc21ac6b0 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4eb255b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cbc235fb20 con 0x55cbc2350580 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4eb255b80 10 client.0 _ll_register_callbacks cb 0x55cbc223f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58222/0 (socket says 172.21.15.5:58222) 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d5f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc235f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 -- 172.21.15.5:0/559961518 learned_addr learned my addr 172.21.15.5:0/559961518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:49.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc2350580 0x55cbc235ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 -- 172.21.15.5:0/559961518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc2350580 msgr2=0x55cbc235ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc2350580 0x55cbc235ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 -- 172.21.15.5:0/559961518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 msgr2=0x55cbc235f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc235f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 -- 172.21.15.5:0/559961518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cbc20b8c00 con 0x55cbc21ac6b0 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4d579b700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc09a6d0 tx=0x7fd4cc09b790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e04c3c2258d28009 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:49.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.301+0000 7fd4b8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc0b3040 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4cc09b1c0 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4c0019c20 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4cc09a940 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 10 client.26972 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55cbc20cfd80 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4cc0c9080 con 0x55cbc21ac6b0 2023-07-21T19:15:49.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7fd4cc0a30f0 con 0x55cbc21ac6b0 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 10 client.26972 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 10 client.26972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.302+0000 7fd4eb255b80 20 client.26972 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23be640 0x55cbc23c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4eb255b80 4 client.26972 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cbc225cff0 con 0x55cbc23be640 2023-07-21T19:15:49.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23be640 0x55cbc23c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4d4f9a700 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23be640 0x55cbc23c0a20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c67c5149fd8bb97a server_cookie=759fb7f2bb4e4624 in_seq=0 out_seq=0 2023-07-21T19:15:49.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.303+0000 7fd4b8ff9700 10 client.26972 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4b8ff9700 1 -- 172.21.15.5:0/559961518 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cbc225cff0 con 0x55cbc23be640 2023-07-21T19:15:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4b8ff9700 10 client.26972 handle_command_reply: tid=0 2023-07-21T19:15:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 client.26972 shutdown 2023-07-21T19:15:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 msgr2=0x55cbc2360a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc09a6d0 tx=0x7fd4cc09b790 comp rx=0 tx=0).stop 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 shutdown_connections 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cbc23be640 0x55cbc23c0a20 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cbc23562d0 0x55cbc235f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cbc21ac6b0 0x55cbc2360a80 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4b8ff9700 0 client.26972 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:49.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 --2- 172.21.15.5:0/559961518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cbc2350580 0x55cbc235ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 >> 172.21.15.5:0/559961518 conn(0x55cbc2345be0 msgr2=0x55cbc2354a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 shutdown_connections 2023-07-21T19:15:49.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.320+0000 7fd4eb255b80 1 -- 172.21.15.5:0/559961518 wait complete. 2023-07-21T19:15:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4eb255b80 20 client.26972 trim_cache size 0 max 16384 2023-07-21T19:15:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:15:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:15:49.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:15:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:15:49.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 msgr2=0x7fd4bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 0x7fd4bc0a37f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd4c8009550 tx=0x7fd4c800b040 comp rx=0 tx=0).stop 2023-07-21T19:15:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 msgr2=0x7fd4d0197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7fd4cc005740 tx=0x7fd4cc005770 comp rx=0 tx=0).stop 2023-07-21T19:15:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 shutdown_connections 2023-07-21T19:15:49.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4d0105b10 0x7fd4d019c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4bc0a1340 0x7fd4bc0a37f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4d00ffb00 0x7fd4d0197d50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 --2- 172.21.15.5:0/4051701881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4d01004b0 0x7fd4d0198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.321+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 >> 172.21.15.5:0/4051701881 conn(0x7fd4d00fb600 msgr2=0x7fd4d010aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:49.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.322+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 shutdown_connections 2023-07-21T19:15:49.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:15:49.322+0000 7fd4d79ff700 1 -- 172.21.15.5:0/4051701881 wait complete. 2023-07-21T19:15:49.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:49 smithi101 ceph-mon[128136]: pgmap v253: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 34 MiB/s wr, 77 op/s 2023-07-21T19:15:49.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:49 smithi005 ceph-mon[126460]: pgmap v253: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 34 MiB/s wr, 77 op/s 2023-07-21T19:15:49.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:49 smithi156 ceph-mon[128316]: pgmap v253: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 34 MiB/s wr, 77 op/s 2023-07-21T19:15:50.469 INFO:teuthology.orchestra.run.smithi005.stderr:30500000 of 50000000 tuples (61%) done (elapsed 468.04 s, remaining 299.24 s) 2023-07-21T19:15:51.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:51 smithi101 ceph-mon[128136]: pgmap v254: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:15:51.892 INFO:teuthology.orchestra.run.smithi005.stderr:30600000 of 50000000 tuples (61%) done (elapsed 469.46 s, remaining 297.63 s) 2023-07-21T19:15:51.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:51 smithi005 ceph-mon[126460]: pgmap v254: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:15:51.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:51 smithi156 ceph-mon[128316]: pgmap v254: 97 pgs: 97 active+clean; 6.9 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:15:53.197 INFO:teuthology.orchestra.run.smithi005.stderr:30700000 of 50000000 tuples (61%) done (elapsed 470.76 s, remaining 295.95 s) 2023-07-21T19:15:53.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:53 smithi101 ceph-mon[128136]: pgmap v255: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 88 op/s 2023-07-21T19:15:53.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:53 smithi005 ceph-mon[126460]: pgmap v255: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 88 op/s 2023-07-21T19:15:53.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:53 smithi156 ceph-mon[128316]: pgmap v255: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 32 MiB/s wr, 88 op/s 2023-07-21T19:15:54.779 INFO:teuthology.orchestra.run.smithi005.stderr:30800000 of 50000000 tuples (61%) done (elapsed 472.35 s, remaining 294.45 s) 2023-07-21T19:15:55.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:55 smithi101 ceph-mon[128136]: pgmap v256: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-07-21T19:15:55.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:55 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:15:55.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:55 smithi005 ceph-mon[126460]: pgmap v256: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-07-21T19:15:55.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:55 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:15:55.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:55 smithi156 ceph-mon[128316]: pgmap v256: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 23 MiB/s wr, 85 op/s 2023-07-21T19:15:55.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:55 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:15:56.076 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:15:56.285 INFO:teuthology.orchestra.run.smithi005.stderr:30900000 of 50000000 tuples (61%) done (elapsed 473.85 s, remaining 292.90 s) 2023-07-21T19:15:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 -- 172.21.15.5:0/744169332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59900ffad0 msgr2=0x7f59900fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 --2- 172.21.15.5:0/744169332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59900ffad0 0x7f59900fff40 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f5980009990 tx=0x7f5980009ca0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 -- 172.21.15.5:0/744169332 shutdown_connections 2023-07-21T19:15:56.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 --2- 172.21.15.5:0/744169332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5990100480 0x7f599010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 --2- 172.21.15.5:0/744169332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 0x7f5990105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 --2- 172.21.15.5:0/744169332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f59900ffad0 0x7f59900fff40 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 -- 172.21.15.5:0/744169332 >> 172.21.15.5:0/744169332 conn(0x7f59900fb5f0 msgr2=0x7f59900fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 -- 172.21.15.5:0/744169332 shutdown_connections 2023-07-21T19:15:56.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.325+0000 7f59987ff700 1 -- 172.21.15.5:0/744169332 wait complete. 2023-07-21T19:15:56.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.326+0000 7f59987ff700 1 Processor -- start 2023-07-21T19:15:56.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 -- start start 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 0x7f5990072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 0x7f59901a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5990073580 con 0x7f5990100480 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901a4260 con 0x7f59900ffad0 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f59987ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59901a43a0 con 0x7f5990105ae0 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 0x7f5990072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f599659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f599659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58250/0 (socket says 172.21.15.5:58250) 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.327+0000 7f599659b700 1 -- 172.21.15.5:0/2829244064 learned_addr learned my addr 172.21.15.5:0/2829244064 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f5996d9c700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 0x7f59901a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 -- 172.21.15.5:0/2829244064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 msgr2=0x7f59901a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 0x7f59901a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 -- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 msgr2=0x7f5990072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 0x7f5990072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 -- 172.21.15.5:0/2829244064 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800095f0 con 0x7f59900ffad0 2023-07-21T19:15:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f5995d9a700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 0x7f5990072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:56.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f599659b700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.328+0000 7f59877fe700 1 -- 172.21.15.5:0/2829244064 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00f890 con 0x7f59900ffad0 2023-07-21T19:15:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.329+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59901a4600 con 0x7f59900ffad0 2023-07-21T19:15:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.329+0000 7f59877fe700 1 -- 172.21.15.5:0/2829244064 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f598c00fe80 con 0x7f59900ffad0 2023-07-21T19:15:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.329+0000 7f59877fe700 1 -- 172.21.15.5:0/2829244064 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c00e3b0 con 0x7f59900ffad0 2023-07-21T19:15:56.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.329+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59901a4b30 con 0x7f59900ffad0 2023-07-21T19:15:56.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.334+0000 7f59877fe700 1 -- 172.21.15.5:0/2829244064 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f598c010460 con 0x7f59900ffad0 2023-07-21T19:15:56.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.334+0000 7f59ac110b80 1 Processor -- start 2023-07-21T19:15:56.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 -- start start 2023-07-21T19:15:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395626a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c73956d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739468cb0 con 0x55c739562be0 2023-07-21T19:15:56.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7394bba40 con 0x55c73956dad0 2023-07-21T19:15:56.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.335+0000 7f59ac110b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7394b4ed0 con 0x55c7395622d0 2023-07-21T19:15:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f599659b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c73956d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58262/0 (socket says 172.21.15.5:58262) 2023-07-21T19:15:56.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 -- 172.21.15.5:0/846999378 learned_addr learned my addr 172.21.15.5:0/846999378 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f59877fe700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 0x7f597c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5996d9c700 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395626a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 -- 172.21.15.5:0/846999378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 msgr2=0x55c7395626a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5996d9c700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 0x7f597c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395626a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 -- 172.21.15.5:0/846999378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 msgr2=0x55c73956d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c73956d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f5995d9a700 1 -- 172.21.15.5:0/846999378 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c7392e2de0 con 0x55c73956dad0 2023-07-21T19:15:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.336+0000 7f59877fe700 1 -- 172.21.15.5:0/2829244064 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f598c0cca70 con 0x7f59900ffad0 2023-07-21T19:15:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.337+0000 7f5995d9a700 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5980009990 tx=0x7f5980005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907ae009a42bfefd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.337+0000 7f5996d9c700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 0x7f597c0a37f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5988009ec0 tx=0x7f598800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.337+0000 7f5984ff9700 1 -- 172.21.15.5:0/846999378 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598001d050 con 0x55c73956dad0 2023-07-21T19:15:56.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.337+0000 7f5984ff9700 1 -- 172.21.15.5:0/846999378 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f598001e040 con 0x55c73956dad0 2023-07-21T19:15:56.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.337+0000 7f5984ff9700 1 -- 172.21.15.5:0/846999378 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980004c60 con 0x55c73956dad0 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 -- 172.21.15.5:0/846999378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 msgr2=0x55c73956feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5980009990 tx=0x7f5980005c00 comp rx=0 tx=0).stop 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 -- 172.21.15.5:0/846999378 shutdown_connections 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c73956feb0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395626a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 --2- 172.21.15.5:0/846999378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c73956d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 -- 172.21.15.5:0/846999378 >> 172.21.15.5:0/846999378 conn(0x55c739558da0 msgr2=0x55c73955e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 -- 172.21.15.5:0/846999378 shutdown_connections 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.338+0000 7f59ac110b80 1 -- 172.21.15.5:0/846999378 wait complete. 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 -- start start 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 Processor -- start 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c73956d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c73956add0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739454ac0 con 0x55c73956dad0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7394547c0 con 0x55c739562be0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c739338920 con 0x55c7395622d0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.339+0000 7f59ac110b80 10 client.0 _ll_register_callbacks cb 0x55c739452760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5996d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c73956add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58278/0 (socket says 172.21.15.5:58278) 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 -- 172.21.15.5:0/4105946917 learned_addr learned my addr 172.21.15.5:0/4105946917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f599659b700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c73956d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 -- 172.21.15.5:0/4105946917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 msgr2=0x55c73956d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c73956d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 -- 172.21.15.5:0/4105946917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 msgr2=0x55c73956add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c73956add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5995d9a700 1 -- 172.21.15.5:0/4105946917 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599010d210 con 0x55c739562be0 2023-07-21T19:15:56.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.340+0000 7f5996d9c700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c73956add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f5995d9a700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5980023ac0 tx=0x7f5980023ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ebab9ddcf17ab87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f5979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598000f870 con 0x55c739562be0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c73940e4f0 con 0x55c739562be0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f598000f9d0 con 0x55c739562be0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5980005960 con 0x55c739562be0 2023-07-21T19:15:56.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f59ac110b80 10 client.26984 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.341+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c7392cbc60 con 0x55c739562be0 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c73940e4f0 con 0x55c739562be0 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f598002d8d0 con 0x55c739562be0 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 10 client.26984 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 10 client.26984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 20 client.26984 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:56.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c73957cee0 0x55c73957f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 4 client.26984 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c7394adeb0 con 0x55c73957cee0 2023-07-21T19:15:56.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.342+0000 7f599659b700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c73957cee0 0x55c73957f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.343+0000 7f599659b700 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c73957cee0 0x55c73957f2f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=403c6ac7fc1c3ca7 server_cookie=ad5af43c27480c5a in_seq=0 out_seq=0 2023-07-21T19:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.343+0000 7f5979ffb700 10 client.26984 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f5979ffb700 1 -- 172.21.15.5:0/4105946917 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c7394adeb0 con 0x55c73957cee0 2023-07-21T19:15:56.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f5979ffb700 10 client.26984 handle_command_reply: tid=0 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 client.26984 shutdown 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 msgr2=0x55c73956a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5980023ac0 tx=0x7f5980023ba0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 shutdown_connections 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c73957cee0 0x55c73957f2f0 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c73956add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c73956d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 --2- 172.21.15.5:0/4105946917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c739562be0 0x55c73956a750 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f5979ffb700 0 client.26984 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 >> 172.21.15.5:0/4105946917 conn(0x55c739558da0 msgr2=0x55c73955e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 shutdown_connections 2023-07-21T19:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 1 -- 172.21.15.5:0/4105946917 wait complete. 2023-07-21T19:15:56.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.344+0000 7f59ac110b80 20 client.26984 trim_cache size 0 max 16384 2023-07-21T19:15:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 Processor -- start 2023-07-21T19:15:56.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 -- start start 2023-07-21T19:15:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7395622d0 0x55c7395744b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73955aec0 con 0x55c73956dad0 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7395641a0 con 0x55c7395622d0 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.365+0000 7f59ac110b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c73955fd50 con 0x55c739562be0 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5996d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5996d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53444/0 (socket says 172.21.15.5:53444) 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55074/0 (socket says 172.21.15.5:55074) 2023-07-21T19:15:56.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5996d9c700 1 -- 172.21.15.5:0/3916644702 learned_addr learned my addr 172.21.15.5:0/3916644702 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f599659b700 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7395622d0 0x55c7395744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 -- 172.21.15.5:0/3916644702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7395622d0 msgr2=0x55c7395744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7395622d0 0x55c7395744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 -- 172.21.15.5:0/3916644702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 msgr2=0x55c739565f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5995d9a700 1 -- 172.21.15.5:0/3916644702 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c739309250 con 0x55c739562be0 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.366+0000 7f5996d9c700 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:15:56.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.367+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f598002dc60 tx=0x7f598000b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a89fa1536db516cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.367+0000 7f59857fa700 1 -- 172.21.15.5:0/3916644702 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598001d070 con 0x55c739562be0 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.367+0000 7f59857fa700 1 -- 172.21.15.5:0/3916644702 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5980005910 con 0x55c739562be0 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.367+0000 7f59857fa700 1 -- 172.21.15.5:0/3916644702 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980005680 con 0x55c739562be0 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- 172.21.15.5:0/3916644702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 msgr2=0x55c739565a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f598002dc60 tx=0x7f598000b260 comp rx=0 tx=0).stop 2023-07-21T19:15:56.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- 172.21.15.5:0/3916644702 shutdown_connections 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c73956dad0 0x55c739565f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c7395622d0 0x55c7395744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3916644702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c739562be0 0x55c739565a30 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- 172.21.15.5:0/3916644702 >> 172.21.15.5:0/3916644702 conn(0x55c739558da0 msgr2=0x55c739579660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- 172.21.15.5:0/3916644702 shutdown_connections 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- 172.21.15.5:0/3916644702 wait complete. 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.368+0000 7f59ac110b80 1 -- start start 2023-07-21T19:15:56.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.369+0000 7f59ac110b80 1 Processor -- start 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.369+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.369+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c7395c5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f59ac110b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c7395c57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f59ac110b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7395cfee0 con 0x55c739562be0 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f59ac110b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7395d0050 con 0x55c73956dad0 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f59ac110b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7395d0210 con 0x55c7395622d0 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f59ac110b80 10 client.0 _ll_register_callbacks cb 0x7f598c0998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:15:56.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2023-07-21T19:15:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 -- 172.21.15.5:0/3904211621 learned_addr learned my addr 172.21.15.5:0/3904211621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:15:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f5996d9c700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c7395c57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c7395c5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 -- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 msgr2=0x55c7395c57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c7395c57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 -- 172.21.15.5:0/3904211621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 msgr2=0x55c7395c5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:15:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c7395c5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.370+0000 7f599659b700 1 -- 172.21.15.5:0/3904211621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5988021040 con 0x55c7395622d0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f599659b700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f598c095fe0 tx=0x7f598c0ce040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5aba8378ab324fa4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f5979ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c014070 con 0x55c7395622d0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f598c09db10 con 0x55c7395622d0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f598001e040 con 0x55c7395622d0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c0a1f30 con 0x55c7395622d0 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f59ac110b80 10 client.26932 fetch_fsmap learned FSMap version 35 2023-07-21T19:15:56.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.371+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) v3 -- 0x55c7392cbc60 con 0x55c7395622d0 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f598001e040 con 0x55c7395622d0 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 35) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f598c0a1aa0 con 0x55c7395622d0 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 10 client.26932 fetch_fsmap finished waiting for FSMap version 35 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 10 client.26932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 20 client.26932 populate_metadata read hostname 'smithi005' 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c7395d17b0 0x55c7395d3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:15:56.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 4 client.26932 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c739470050 con 0x55c7395d17b0 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.372+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c7395d17b0 0x55c7395d3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f5995d9a700 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c7395d17b0 0x55c7395d3bc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54acbe185f36da09 server_cookie=6af1038f3e7eea46 in_seq=0 out_seq=0 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f5979ffb700 10 client.26932 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f5979ffb700 1 -- 172.21.15.5:0/3904211621 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55c739470050 con 0x55c7395d17b0 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f5979ffb700 10 client.26932 handle_command_reply: tid=0 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 client.26932 shutdown 2023-07-21T19:15:56.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 msgr2=0x55c7395c4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f598c095fe0 tx=0x7f598c0ce040 comp rx=0 tx=0).stop 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 shutdown_connections 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c73956dad0 0x55c7395c57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c7395622d0 0x55c7395c4d30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c7395d17b0 0x55c7395d3bc0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 --2- 172.21.15.5:0/3904211621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c739562be0 0x55c7395c5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f5979ffb700 0 client.26932 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:15:56.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.373+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 >> 172.21.15.5:0/3904211621 conn(0x55c739558da0 msgr2=0x55c739565a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 shutdown_connections 2023-07-21T19:15:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59ac110b80 1 -- 172.21.15.5:0/3904211621 wait complete. 2023-07-21T19:15:56.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59ac110b80 20 client.26932 trim_cache size 0 max 16384 2023-07-21T19:15:56.379 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:15:56.379 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:15:56.380 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:15:56.381 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:15:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 msgr2=0x7f597c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 0x7f597c0a37f0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5988009ec0 tx=0x7f598800b040 comp rx=0 tx=0).stop 2023-07-21T19:15:56.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 msgr2=0x7f5990072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f598c00d950 tx=0x7f598c00dc60 comp rx=0 tx=0).stop 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 shutdown_connections 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5990105ae0 0x7f59901a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f59900ffad0 0x7f5990072810 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f597c0a1340 0x7f597c0a37f0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 --2- 172.21.15.5:0/2829244064 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5990100480 0x7f5990072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:15:56.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.374+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 >> 172.21.15.5:0/2829244064 conn(0x7f59900fb5f0 msgr2=0x7f59900fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:15:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.375+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 shutdown_connections 2023-07-21T19:15:56.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:15:56.375+0000 7f59987ff700 1 -- 172.21.15.5:0/2829244064 wait complete. 2023-07-21T19:15:56.392 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:15:56.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:56 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:15:56.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:56 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:15:56.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:56 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:15:56.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:56 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:15:56.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:56 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:15:56.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:56 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:15:56.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:56 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:15:56.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:56 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:15:56.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:56 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:15:57.600 INFO:teuthology.orchestra.run.smithi005.stderr:31000000 of 50000000 tuples (62%) done (elapsed 475.17 s, remaining 291.23 s) 2023-07-21T19:15:57.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:57 smithi101 ceph-mon[128136]: pgmap v257: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 83 op/s 2023-07-21T19:15:57.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:57 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:57.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:57 smithi005 ceph-mon[126460]: pgmap v257: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 83 op/s 2023-07-21T19:15:57.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:57 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:57.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:57 smithi156 ceph-mon[128316]: pgmap v257: 97 pgs: 97 active+clean; 7.0 GiB data, 3.1 GiB used, 1.0 TiB / 1.0 TiB avail; 145 KiB/s rd, 22 MiB/s wr, 83 op/s 2023-07-21T19:15:57.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:57 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:15:58.909 INFO:teuthology.orchestra.run.smithi005.stderr:31100000 of 50000000 tuples (62%) done (elapsed 476.48 s, remaining 289.56 s) 2023-07-21T19:15:59.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:15:59 smithi101 ceph-mon[128136]: pgmap v258: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 107 op/s 2023-07-21T19:15:59.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:15:59 smithi005 ceph-mon[126460]: pgmap v258: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 107 op/s 2023-07-21T19:15:59.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:15:59 smithi156 ceph-mon[128316]: pgmap v258: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 151 KiB/s rd, 32 MiB/s wr, 107 op/s 2023-07-21T19:16:00.291 INFO:teuthology.orchestra.run.smithi005.stderr:31200000 of 50000000 tuples (62%) done (elapsed 477.86 s, remaining 287.94 s) 2023-07-21T19:16:01.666 INFO:teuthology.orchestra.run.smithi005.stderr:31300000 of 50000000 tuples (62%) done (elapsed 479.23 s, remaining 286.32 s) 2023-07-21T19:16:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:01 smithi101 ceph-mon[128136]: pgmap v259: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 21 MiB/s wr, 72 op/s 2023-07-21T19:16:01.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:01 smithi005 ceph-mon[126460]: pgmap v259: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 21 MiB/s wr, 72 op/s 2023-07-21T19:16:01.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:01 smithi156 ceph-mon[128316]: pgmap v259: 97 pgs: 97 active+clean; 7.1 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 21 MiB/s wr, 72 op/s 2023-07-21T19:16:02.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 -- 172.21.15.5:0/2692228894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481059a0 msgr2=0x7f3c48105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/2692228894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481059a0 0x7f3c48105d70 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0098f0 tx=0x7f3c3c009c00 comp rx=0 tx=0).stop 2023-07-21T19:16:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 -- 172.21.15.5:0/2692228894 shutdown_connections 2023-07-21T19:16:02.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/2692228894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c48068070 0x7f3c4810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/2692228894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481059a0 0x7f3c48105d70 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/2692228894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c481062b0 0x7f3c48067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.504+0000 7f3c4d99d700 1 -- 172.21.15.5:0/2692228894 >> 172.21.15.5:0/2692228894 conn(0x7f3c48074770 msgr2=0x7f3c48074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.505+0000 7f3c4d99d700 1 -- 172.21.15.5:0/2692228894 shutdown_connections 2023-07-21T19:16:02.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.505+0000 7f3c4d99d700 1 -- 172.21.15.5:0/2692228894 wait complete. 2023-07-21T19:16:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.506+0000 7f3c4d99d700 1 Processor -- start 2023-07-21T19:16:02.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 -- start start 2023-07-21T19:16:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 0x7f3c48106750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c481059a0 0x7f3c48106c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481a8310 con 0x7f3c481062b0 2023-07-21T19:16:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481a8480 con 0x7f3c481059a0 2023-07-21T19:16:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.507+0000 7f3c4d99d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c481a8640 con 0x7f3c48068070 2023-07-21T19:16:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.508+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 0x7f3c48106750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.508+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.508+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55060/0 (socket says 172.21.15.5:55060) 2023-07-21T19:16:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.508+0000 7f3c4d19c700 1 -- 172.21.15.5:0/3306197381 learned_addr learned my addr 172.21.15.5:0/3306197381 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:02.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c47fff700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c481059a0 0x7f3c48106c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3306197381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c481059a0 msgr2=0x7f3c48106c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c481059a0 0x7f3c48106c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 msgr2=0x7f3c481071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.509+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3306197381 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c3c009520 con 0x7f3c48068070 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c4d19c700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 0x7f3c48106750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009810 tx=0x7f3c3c00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c45ffb700 1 -- 172.21.15.5:0/3306197381 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c01c070 con 0x7f3c48068070 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c45ffb700 1 -- 172.21.15.5:0/3306197381 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c3c005040 con 0x7f3c48068070 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c45ffb700 1 -- 172.21.15.5:0/3306197381 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0039e0 con 0x7f3c48068070 2023-07-21T19:16:02.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.510+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c481a88c0 con 0x7f3c48068070 2023-07-21T19:16:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.511+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c481a8cf0 con 0x7f3c48068070 2023-07-21T19:16:02.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.516+0000 7f3c45ffb700 1 -- 172.21.15.5:0/3306197381 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3c3c020030 con 0x7f3c48068070 2023-07-21T19:16:02.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.517+0000 7f3c6249fb80 1 Processor -- start 2023-07-21T19:16:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.517+0000 7f3c6249fb80 1 -- start start 2023-07-21T19:16:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.517+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df80f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df7b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df7b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429de80c50 con 0x56429df81470 2023-07-21T19:16:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429ded39e0 con 0x56429ddd76b0 2023-07-21T19:16:02.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429decce70 con 0x56429df818c0 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df80f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df7b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df80f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37826/0 (socket says 172.21.15.5:37826) 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c4d19c700 1 -- 172.21.15.5:0/745769470 learned_addr learned my addr 172.21.15.5:0/745769470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df7b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c45ffb700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 0x7f3c380a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 -- 172.21.15.5:0/745769470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 msgr2=0x56429df80f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df80f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 -- 172.21.15.5:0/745769470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 msgr2=0x56429df7b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df7b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.518+0000 7f3c47fff700 1 -- 172.21.15.5:0/745769470 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429dcfad80 con 0x56429df818c0 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c45ffb700 1 -- 172.21.15.5:0/3306197381 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3c3c0cdc50 con 0x7f3c48068070 2023-07-21T19:16:02.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c4d19c700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 0x7f3c380a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c47fff700 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df7b700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c480fc8d0 tx=0x7f3c34009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a41521983e879f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c32ffd700 1 -- 172.21.15.5:0/745769470 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34004250 con 0x56429df818c0 2023-07-21T19:16:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c32ffd700 1 -- 172.21.15.5:0/745769470 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c34011d30 con 0x56429df818c0 2023-07-21T19:16:02.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.519+0000 7f3c32ffd700 1 -- 172.21.15.5:0/745769470 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c34003cc0 con 0x56429df818c0 2023-07-21T19:16:02.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c4d19c700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 0x7f3c380a38c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c400075c0 tx=0x7f3c40004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 -- 172.21.15.5:0/745769470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 msgr2=0x56429df7b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df7b700 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f3c480fc8d0 tx=0x7f3c34009e70 comp rx=0 tx=0).stop 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 -- 172.21.15.5:0/745769470 shutdown_connections 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df7b700 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df80f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/745769470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df7b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.520+0000 7f3c6249fb80 1 -- 172.21.15.5:0/745769470 >> 172.21.15.5:0/745769470 conn(0x56429df70d40 msgr2=0x56429df76340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.521+0000 7f3c6249fb80 1 -- 172.21.15.5:0/745769470 shutdown_connections 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.521+0000 7f3c6249fb80 1 -- 172.21.15.5:0/745769470 wait complete. 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.521+0000 7f3c6249fb80 1 -- start start 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 Processor -- start 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df877f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df84ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df85180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429de6ca60 con 0x56429df818c0 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429de6c760 con 0x56429ddd76b0 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429dd508c0 con 0x56429df81470 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.522+0000 7f3c6249fb80 10 client.0 _ll_register_callbacks cb 0x56429de6a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df85180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df85180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55074/0 (socket says 172.21.15.5:55074) 2023-07-21T19:16:02.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4d19c700 1 -- 172.21.15.5:0/3998874980 learned_addr learned my addr 172.21.15.5:0/3998874980 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df877f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c47fff700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df84ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3998874980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 msgr2=0x56429df84ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df84ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3998874980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 msgr2=0x56429df85180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df85180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.523+0000 7f3c4c99b700 1 -- 172.21.15.5:0/3998874980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c340097f0 con 0x56429ddd76b0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df877f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cb3a0 tx=0x7f3c3c0d1350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3bdfacdec45533b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c27fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c09adb0 con 0x56429ddd76b0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3c3c09ede0 con 0x56429ddd76b0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56429de26490 con 0x56429ddd76b0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.524+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c0a3170 con 0x56429ddd76b0 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.525+0000 7f3c6249fb80 10 client.27002 fetch_fsmap learned FSMap version 36 2023-07-21T19:16:02.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.525+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56429dce3c00 con 0x56429ddd76b0 2023-07-21T19:16:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.525+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56429de26490 con 0x56429ddd76b0 2023-07-21T19:16:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.525+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c3c09b010 con 0x56429ddd76b0 2023-07-21T19:16:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 10 client.27002 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T19:16:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 10 client.27002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:02.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 20 client.27002 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429df94a10 0x56429df96e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 4 client.27002 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56429dec5e50 con 0x56429df94a10 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.526+0000 7f3c47fff700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429df94a10 0x56429df96e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.527+0000 7f3c47fff700 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429df94a10 0x56429df96e20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fbca948372a7142 server_cookie=c2827ff90138c0fc in_seq=0 out_seq=0 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.527+0000 7f3c27fff700 10 client.27002 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:02.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c27fff700 1 -- 172.21.15.5:0/3998874980 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56429dec5e50 con 0x56429df94a10 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c27fff700 10 client.27002 handle_command_reply: tid=0 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 client.27002 shutdown 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 msgr2=0x56429df877f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df877f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0cb3a0 tx=0x7f3c3c0d1350 comp rx=0 tx=0).stop 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 shutdown_connections 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429df94a10 0x56429df96e20 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df85180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df877f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c27fff700 0 client.27002 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/3998874980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df84ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 >> 172.21.15.5:0/3998874980 conn(0x56429df70d40 msgr2=0x56429df75640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 shutdown_connections 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.529+0000 7f3c6249fb80 1 -- 172.21.15.5:0/3998874980 wait complete. 2023-07-21T19:16:02.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.530+0000 7f3c6249fb80 20 client.27002 trim_cache size 0 max 16384 2023-07-21T19:16:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 Processor -- start 2023-07-21T19:16:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 -- start start 2023-07-21T19:16:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df82410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df87390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df72ed0 con 0x56429df81470 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df86080 con 0x56429ddd76b0 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.552+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df830c0 con 0x56429df818c0 2023-07-21T19:16:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df82410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37836/0 (socket says 172.21.15.5:37836) 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4d19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df87390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 -- 172.21.15.5:0/4275218685 learned_addr learned my addr 172.21.15.5:0/4275218685 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 -- 172.21.15.5:0/4275218685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 msgr2=0x56429df87390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df87390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 -- 172.21.15.5:0/4275218685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 msgr2=0x56429df82410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df82410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c4c99b700 1 -- 172.21.15.5:0/4275218685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56429dd211f0 con 0x56429ddd76b0 2023-07-21T19:16:02.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.553+0000 7f3c47fff700 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df82410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.554+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0972b0 tx=0x7f3c3c09a610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=57528885f9b968e7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.554+0000 7f3c337fe700 1 -- 172.21.15.5:0/4275218685 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c09ede0 con 0x56429ddd76b0 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.554+0000 7f3c337fe700 1 -- 172.21.15.5:0/4275218685 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c3c09adb0 con 0x56429ddd76b0 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.554+0000 7f3c337fe700 1 -- 172.21.15.5:0/4275218685 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0cb620 con 0x56429ddd76b0 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- 172.21.15.5:0/4275218685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 msgr2=0x56429df8be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c0972b0 tx=0x7f3c3c09a610 comp rx=0 tx=0).stop 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- 172.21.15.5:0/4275218685 shutdown_connections 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df818c0 0x56429df87390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df8be00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/4275218685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df81470 0x56429df82410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- 172.21.15.5:0/4275218685 >> 172.21.15.5:0/4275218685 conn(0x56429df70d40 msgr2=0x56429df81d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- 172.21.15.5:0/4275218685 shutdown_connections 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- 172.21.15.5:0/4275218685 wait complete. 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.555+0000 7f3c6249fb80 1 -- start start 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.556+0000 7f3c6249fb80 1 Processor -- start 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.556+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.556+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c6249fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df7c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df7ced0 con 0x56429df818c0 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df7d040 con 0x56429ddd76b0 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c6249fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56429df7d200 con 0x56429df81470 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c6249fb80 10 client.0 _ll_register_callbacks cb 0x56429de6a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c47fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37840/0 (socket says 172.21.15.5:37840) 2023-07-21T19:16:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 -- 172.21.15.5:0/1335719203 learned_addr learned my addr 172.21.15.5:0/1335719203 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c47fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51958/0 (socket says 172.21.15.5:51958) 2023-07-21T19:16:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4d19c700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df7c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 -- 172.21.15.5:0/1335719203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 msgr2=0x56429df7e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 -- 172.21.15.5:0/1335719203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 msgr2=0x56429df7c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df7c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c4c99b700 1 -- 172.21.15.5:0/1335719203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56429dcfad80 con 0x56429ddd76b0 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.557+0000 7f3c47fff700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c4c99b700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c099100 tx=0x7f3c3c0d1330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=270d353c0c0fc2b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c27fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c3c0a3170 con 0x56429ddd76b0 2023-07-21T19:16:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3c34011d30 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(92..92 src has 1..92) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3c3c09ede0 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3c3c0cb620 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c6249fb80 10 client.27008 fetch_fsmap learned FSMap version 36 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x56429dce3c00 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.558+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3c34011d30 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3c3c0a2ce0 con 0x56429ddd76b0 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 10 client.27008 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T19:16:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 10 client.27008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 20 client.27008 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429dfe9120 0x56429dfeb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 4 client.27008 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56429de87ff0 con 0x56429dfe9120 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.559+0000 7f3c47fff700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429dfe9120 0x56429dfeb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.560+0000 7f3c47fff700 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429dfe9120 0x56429dfeb500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b35213dad5fca7d server_cookie=a0282384d5c9c345 in_seq=0 out_seq=0 2023-07-21T19:16:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.560+0000 7f3c27fff700 10 client.27008 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.576+0000 7f3c27fff700 1 -- 172.21.15.5:0/1335719203 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56429de87ff0 con 0x56429dfe9120 2023-07-21T19:16:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.576+0000 7f3c27fff700 10 client.27008 handle_command_reply: tid=0 2023-07-21T19:16:02.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.576+0000 7f3c6249fb80 1 client.27008 shutdown 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.576+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 msgr2=0x56429df7db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.576+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c099100 tx=0x7f3c3c0d1330 comp rx=0 tx=0).stop 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 shutdown_connections 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56429dfe9120 0x56429dfeb500 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56429df818c0 0x56429df7c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56429ddd76b0 0x56429df7db60 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c27fff700 0 client.27008 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 --2- 172.21.15.5:0/1335719203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56429df81470 0x56429df7e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 >> 172.21.15.5:0/1335719203 conn(0x56429df70d40 msgr2=0x56429df803c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 shutdown_connections 2023-07-21T19:16:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 1 -- 172.21.15.5:0/1335719203 wait complete. 2023-07-21T19:16:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.577+0000 7f3c6249fb80 20 client.27008 trim_cache size 0 max 16384 2023-07-21T19:16:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:16:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:16:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:16:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:16:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 msgr2=0x7f3c380a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 0x7f3c380a38c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f3c400075c0 tx=0x7f3c40004ad0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 msgr2=0x7f3c48106750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 0x7f3c48106750 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f3c3c009810 tx=0x7f3c3c00fa00 comp rx=0 tx=0).stop 2023-07-21T19:16:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 shutdown_connections 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3c380a1410 0x7f3c380a38c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3c481062b0 0x7f3c481071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3c48068070 0x7f3c48106750 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 --2- 172.21.15.5:0/3306197381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3c481059a0 0x7f3c48106c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 >> 172.21.15.5:0/3306197381 conn(0x7f3c48074770 msgr2=0x7f3c4810aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 shutdown_connections 2023-07-21T19:16:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:02.578+0000 7f3c4d99d700 1 -- 172.21.15.5:0/3306197381 wait complete. 2023-07-21T19:16:03.025 INFO:teuthology.orchestra.run.smithi005.stderr:31400000 of 50000000 tuples (62%) done (elapsed 480.59 s, remaining 284.68 s) 2023-07-21T19:16:03.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:03 smithi101 ceph-mon[128136]: pgmap v260: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 108 op/s 2023-07-21T19:16:03.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:03 smithi005 ceph-mon[126460]: pgmap v260: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 108 op/s 2023-07-21T19:16:03.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:03 smithi156 ceph-mon[128316]: pgmap v260: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 31 MiB/s wr, 108 op/s 2023-07-21T19:16:04.302 INFO:teuthology.orchestra.run.smithi005.stderr:31500000 of 50000000 tuples (63%) done (elapsed 481.87 s, remaining 283.00 s) 2023-07-21T19:16:05.693 INFO:teuthology.orchestra.run.smithi005.stderr:31600000 of 50000000 tuples (63%) done (elapsed 483.26 s, remaining 281.39 s) 2023-07-21T19:16:05.835 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:05 smithi101 ceph-mon[128136]: pgmap v261: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 23 MiB/s wr, 82 op/s 2023-07-21T19:16:05.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:05 smithi005 ceph-mon[126460]: pgmap v261: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 23 MiB/s wr, 82 op/s 2023-07-21T19:16:05.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:05 smithi156 ceph-mon[128316]: pgmap v261: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 23 MiB/s wr, 82 op/s 2023-07-21T19:16:07.055 INFO:teuthology.orchestra.run.smithi005.stderr:31700000 of 50000000 tuples (63%) done (elapsed 484.62 s, remaining 279.77 s) 2023-07-21T19:16:07.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:07 smithi101 ceph-mon[128136]: pgmap v262: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 66 op/s 2023-07-21T19:16:07.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:07 smithi101 ceph-mon[128136]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T19:16:07.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:07 smithi005 ceph-mon[126460]: pgmap v262: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 66 op/s 2023-07-21T19:16:07.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:07 smithi005 ceph-mon[126460]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T19:16:07.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:07 smithi156 ceph-mon[128316]: pgmap v262: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 21 MiB/s wr, 66 op/s 2023-07-21T19:16:07.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:07 smithi156 ceph-mon[128316]: osdmap e93: 12 total, 12 up, 12 in 2023-07-21T19:16:08.359 INFO:teuthology.orchestra.run.smithi005.stderr:31800000 of 50000000 tuples (63%) done (elapsed 485.93 s, remaining 278.11 s) 2023-07-21T19:16:09.762 INFO:teuthology.orchestra.run.smithi005.stderr:31900000 of 50000000 tuples (63%) done (elapsed 487.33 s, remaining 276.51 s) 2023-07-21T19:16:09.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:09 smithi101 ceph-mon[128136]: pgmap v264: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:09.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:09 smithi005 ceph-mon[126460]: pgmap v264: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:09.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:09 smithi156 ceph-mon[128316]: pgmap v264: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:11.048 INFO:teuthology.orchestra.run.smithi005.stderr:32000000 of 50000000 tuples (64%) done (elapsed 488.62 s, remaining 274.85 s) 2023-07-21T19:16:11.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:11 smithi101 ceph-mon[128136]: pgmap v265: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:11.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:11 smithi005 ceph-mon[126460]: pgmap v265: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:11.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:11 smithi156 ceph-mon[128316]: pgmap v265: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:16:12.354 INFO:teuthology.orchestra.run.smithi005.stderr:32100000 of 50000000 tuples (64%) done (elapsed 489.92 s, remaining 273.20 s) 2023-07-21T19:16:13.668 INFO:teuthology.orchestra.run.smithi005.stderr:32200000 of 50000000 tuples (64%) done (elapsed 491.24 s, remaining 271.55 s) 2023-07-21T19:16:13.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:13 smithi101 ceph-mon[128136]: pgmap v266: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 34 op/s 2023-07-21T19:16:13.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:13 smithi101 ceph-mon[128136]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T19:16:13.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:13 smithi005 ceph-mon[126460]: pgmap v266: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 34 op/s 2023-07-21T19:16:13.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:13 smithi005 ceph-mon[126460]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T19:16:13.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:13 smithi156 ceph-mon[128316]: pgmap v266: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 34 op/s 2023-07-21T19:16:13.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:13 smithi156 ceph-mon[128316]: osdmap e94: 12 total, 12 up, 12 in 2023-07-21T19:16:15.039 INFO:teuthology.orchestra.run.smithi005.stderr:32300000 of 50000000 tuples (64%) done (elapsed 492.61 s, remaining 269.94 s) 2023-07-21T19:16:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 -- 172.21.15.5:0/2740210057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c105b10 msgr2=0x7f8e4c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 --2- 172.21.15.5:0/2740210057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c105b10 0x7f8e4c105ee0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009990 tx=0x7f8e3c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 -- 172.21.15.5:0/2740210057 shutdown_connections 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 --2- 172.21.15.5:0/2740210057 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 --2- 172.21.15.5:0/2740210057 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c105b10 0x7f8e4c105ee0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 --2- 172.21.15.5:0/2740210057 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 0x7f8e4c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 -- 172.21.15.5:0/2740210057 >> 172.21.15.5:0/2740210057 conn(0x7f8e4c0fb600 msgr2=0x7f8e4c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 -- 172.21.15.5:0/2740210057 shutdown_connections 2023-07-21T19:16:15.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.758+0000 7f8e519ff700 1 -- 172.21.15.5:0/2740210057 wait complete. 2023-07-21T19:16:15.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.759+0000 7f8e519ff700 1 Processor -- start 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.759+0000 7f8e519ff700 1 -- start start 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 0x7f8e4c197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c105b10 0x7f8e4c19c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19cfa0 con 0x7f8e4c105b10 2023-07-21T19:16:15.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19d110 con 0x7f8e4c1004b0 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e519ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e4c19d2d0 con 0x7f8e4c0ffb00 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34288/0 (socket says 172.21.15.5:34288) 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/4250158067 learned_addr learned my addr 172.21.15.5:0/4250158067 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4affd700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 0x7f8e4c197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c105b10 0x7f8e4c19c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/4250158067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 msgr2=0x7f8e4c197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 0x7f8e4c197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c105b10 msgr2=0x7f8e4c19c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c105b10 0x7f8e4c19c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.760+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/4250158067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e44009c40 con 0x7f8e4c1004b0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e509fd700 1 -- 172.21.15.5:0/4250158067 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40009930 con 0x7f8e4c1004b0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e3c0095f0 con 0x7f8e4c1004b0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e509fd700 1 -- 172.21.15.5:0/4250158067 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e40010460 con 0x7f8e4c1004b0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e509fd700 1 -- 172.21.15.5:0/4250158067 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4000f620 con 0x7f8e4c1004b0 2023-07-21T19:16:15.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.761+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c19d7a0 con 0x7f8e4c1004b0 2023-07-21T19:16:15.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.765+0000 7f8e509fd700 1 -- 172.21.15.5:0/4250158067 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8e400105d0 con 0x7f8e4c1004b0 2023-07-21T19:16:15.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.766+0000 7f8e652e1b80 1 Processor -- start 2023-07-21T19:16:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 -- start start 2023-07-21T19:16:15.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e509fd700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 0x7f8e340a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e509fd700 1 -- 172.21.15.5:0/4250158067 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8e400cca10 con 0x7f8e4c1004b0 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb14257d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 0x7f8e340a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb14252580 0x55cb142529d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb14157c50 con 0x55cb142582d0 2023-07-21T19:16:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb141aa9e0 con 0x55cb14252580 2023-07-21T19:16:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.767+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb141a3e70 con 0x55cb140ae6b0 2023-07-21T19:16:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49234/0 (socket says 172.21.15.5:49234) 2023-07-21T19:16:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/1436665853 learned_addr learned my addr 172.21.15.5:0/1436665853 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:15.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4affd700 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb14257d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb14252580 0x55cb142529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/1436665853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 msgr2=0x55cb14257d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb14257d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/1436665853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb14252580 msgr2=0x55cb142529d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb14252580 0x55cb142529d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/1436665853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb13fd1d80 con 0x55cb142582d0 2023-07-21T19:16:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 0x7f8e340a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009fd0 tx=0x7f8e4400fdd0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.768+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8e400ca760 tx=0x7f8e40099440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eedbf8781bbf01d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.769+0000 7f8e31ffb700 1 -- 172.21.15.5:0/1436665853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099ae0 con 0x55cb142582d0 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.769+0000 7f8e31ffb700 1 -- 172.21.15.5:0/1436665853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e4009db10 con 0x55cb142582d0 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.769+0000 7f8e31ffb700 1 -- 172.21.15.5:0/1436665853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099230 con 0x55cb142582d0 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.769+0000 7f8e652e1b80 1 -- 172.21.15.5:0/1436665853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 msgr2=0x55cb14252040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.769+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f8e400ca760 tx=0x7f8e40099440 comp rx=0 tx=0).stop 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 -- 172.21.15.5:0/1436665853 shutdown_connections 2023-07-21T19:16:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb14252580 0x55cb142529d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb14257d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/1436665853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb14252040 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 -- 172.21.15.5:0/1436665853 >> 172.21.15.5:0/1436665853 conn(0x55cb14247be0 msgr2=0x55cb1424cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 -- 172.21.15.5:0/1436665853 shutdown_connections 2023-07-21T19:16:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 -- 172.21.15.5:0/1436665853 wait complete. 2023-07-21T19:16:15.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.770+0000 7f8e652e1b80 1 -- start start 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 Processor -- start 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb142614c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 0x55cb1425b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb14143a60 con 0x55cb14252580 2023-07-21T19:16:15.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb14143760 con 0x55cb142582d0 2023-07-21T19:16:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb140278c0 con 0x55cb140ae6b0 2023-07-21T19:16:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e652e1b80 10 client.0 _ll_register_callbacks cb 0x55cb14141700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49236/0 (socket says 172.21.15.5:49236) 2023-07-21T19:16:15.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/400750562 learned_addr learned my addr 172.21.15.5:0/400750562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.771+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 0x55cb1425b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4affd700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb142614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/400750562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 msgr2=0x55cb142614c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb142614c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 msgr2=0x55cb1425b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 0x55cb1425b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/400750562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c19d7a0 con 0x55cb14252580 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 0x55cb1425b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8e40095fe0 tx=0x7f8e40099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eeb2e9f40d734383 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e2affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099ae0 con 0x55cb14252580 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cb140fd490 con 0x55cb14252580 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.772+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8e400a1f30 con 0x55cb14252580 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e40099d40 con 0x55cb14252580 2023-07-21T19:16:15.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 10 client.17340 fetch_fsmap learned FSMap version 36 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cb13fbac00 con 0x55cb14252580 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cb140fd490 con 0x55cb14252580 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8e40098a40 con 0x55cb14252580 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 10 client.17340 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 20 client.17340 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb1426b520 0x55cb1426d930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 4 client.17340 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.773+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cb1419ce50 con 0x55cb1426b520 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.774+0000 7f8e4affd700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb1426b520 0x55cb1426d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.774+0000 7f8e4affd700 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb1426b520 0x55cb1426d930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598f18701ce0047c server_cookie=c97d40fdd679e0f1 in_seq=0 out_seq=0 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.774+0000 7f8e2affd700 10 client.17340 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e2affd700 1 -- 172.21.15.5:0/400750562 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cb1419ce50 con 0x55cb1426b520 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e2affd700 10 client.17340 handle_command_reply: tid=0 2023-07-21T19:16:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 client.17340 shutdown 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 msgr2=0x55cb14261a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f8e40095fe0 tx=0x7f8e40099280 comp rx=0 tx=0).stop 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 shutdown_connections 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb142582d0 0x55cb1425b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb140ae6b0 0x55cb142614c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb1426b520 0x55cb1426d930 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/400750562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb14261a00 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 >> 172.21.15.5:0/400750562 conn(0x55cb14247be0 msgr2=0x55cb1424c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.775+0000 7f8e2affd700 0 client.17340 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.776+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 shutdown_connections 2023-07-21T19:16:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.776+0000 7f8e652e1b80 1 -- 172.21.15.5:0/400750562 wait complete. 2023-07-21T19:16:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.776+0000 7f8e652e1b80 20 client.17340 trim_cache size 0 max 16384 2023-07-21T19:16:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 Processor -- start 2023-07-21T19:16:15.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 -- start start 2023-07-21T19:16:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14257280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb142582d0 0x55cb1424cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1425bcc0 con 0x55cb14252580 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb14257a10 con 0x55cb140ae6b0 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1424eb40 con 0x55cb142582d0 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e4a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49252/0 (socket says 172.21.15.5:49252) 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/18554257 learned_addr learned my addr 172.21.15.5:0/18554257 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.792+0000 7f8e4affd700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14257280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4b7fe700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb142582d0 0x55cb1424cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/18554257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb142582d0 msgr2=0x55cb1424cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb142582d0 0x55cb1424cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/18554257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 msgr2=0x55cb14257280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14257280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 -- 172.21.15.5:0/18554257 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cb1425c460 con 0x55cb14252580 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4affd700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14257280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:15.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8e40096550 tx=0x7f8e400ca140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1222d74f12769d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e327fc700 1 -- 172.21.15.5:0/18554257 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40099440 con 0x55cb14252580 2023-07-21T19:16:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e327fc700 1 -- 172.21.15.5:0/18554257 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e400a1f30 con 0x55cb14252580 2023-07-21T19:16:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.793+0000 7f8e327fc700 1 -- 172.21.15.5:0/18554257 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e40098a40 con 0x55cb14252580 2023-07-21T19:16:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 -- 172.21.15.5:0/18554257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 msgr2=0x55cb1425b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f8e40096550 tx=0x7f8e400ca140 comp rx=0 tx=0).stop 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 -- 172.21.15.5:0/18554257 shutdown_connections 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb142582d0 0x55cb1424cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14257280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/18554257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb14252580 0x55cb1425b780 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 -- 172.21.15.5:0/18554257 >> 172.21.15.5:0/18554257 conn(0x55cb14247be0 msgr2=0x55cb14259dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 -- 172.21.15.5:0/18554257 shutdown_connections 2023-07-21T19:16:15.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.794+0000 7f8e652e1b80 1 -- 172.21.15.5:0/18554257 wait complete. 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.795+0000 7f8e652e1b80 1 -- start start 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.795+0000 7f8e652e1b80 1 Processor -- start 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb14252580 0x55cb1425eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb1425f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1425fbd0 con 0x55cb142582d0 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb1425fd40 con 0x55cb140ae6b0 2023-07-21T19:16:15.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cb142be3d0 con 0x55cb14252580 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e652e1b80 10 client.0 _ll_register_callbacks cb 0x7f8e440097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb1425f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34340/0 (socket says 172.21.15.5:34340) 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 -- 172.21.15.5:0/964693180 learned_addr learned my addr 172.21.15.5:0/964693180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb14252580 0x55cb1425eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 -- 172.21.15.5:0/964693180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb14252580 msgr2=0x55cb1425eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb14252580 0x55cb1425eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 -- 172.21.15.5:0/964693180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 msgr2=0x55cb1425f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb1425f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.796+0000 7f8e4affd700 1 -- 172.21.15.5:0/964693180 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e4c10d240 con 0x55cb140ae6b0 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e4affd700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009990 tx=0x7f8e3c00b260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a2a16aff9c8f824 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e2affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e3c01d040 con 0x55cb140ae6b0 2023-07-21T19:16:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8e4009db10 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8e3c004500 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.797+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8e3c0056f0 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 10 client.27026 fetch_fsmap learned FSMap version 36 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) v3 -- 0x55cb13fbac00 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e4009db10 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 36) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8e3c03b5a0 con 0x55cb140ae6b0 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 10 client.27026 fetch_fsmap finished waiting for FSMap version 36 2023-07-21T19:16:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 10 client.27026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 20 client.27026 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb142bfb90 0x55cb142c1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 4 client.27026 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cb1415eff0 con 0x55cb142bfb90 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.798+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb142bfb90 0x55cb142c1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.799+0000 7f8e4a7fc700 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb142bfb90 0x55cb142c1fa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d414184b9641125a server_cookie=f3e25b02d404a42b in_seq=0 out_seq=0 2023-07-21T19:16:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.799+0000 7f8e2affd700 10 client.27026 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e2affd700 1 -- 172.21.15.5:0/964693180 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cb1415eff0 con 0x55cb142bfb90 2023-07-21T19:16:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e2affd700 10 client.27026 handle_command_reply: tid=0 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 client.27026 shutdown 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 msgr2=0x55cb14261940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c009990 tx=0x7f8e3c00b260 comp rx=0 tx=0).stop 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 shutdown_connections 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cb142bfb90 0x55cb142c1fa0 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cb142582d0 0x55cb1425f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e2affd700 0 client.27026 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cb140ae6b0 0x55cb14261940 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 --2- 172.21.15.5:0/964693180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cb14252580 0x55cb1425eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 >> 172.21.15.5:0/964693180 conn(0x55cb14247be0 msgr2=0x55cb14259dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 shutdown_connections 2023-07-21T19:16:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 1 -- 172.21.15.5:0/964693180 wait complete. 2023-07-21T19:16:15.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.815+0000 7f8e652e1b80 20 client.27026 trim_cache size 0 max 16384 2023-07-21T19:16:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:16:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:16:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:16:15.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:16:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 msgr2=0x7f8e340a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 0x7f8e340a37f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8e44009fd0 tx=0x7f8e4400fdd0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 msgr2=0x7f8e4c1982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f8e4000d950 tx=0x7f8e4000dc60 comp rx=0 tx=0).stop 2023-07-21T19:16:15.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 shutdown_connections 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e340a1340 0x7f8e340a37f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8e4c105b10 0x7f8e4c19c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8e4c0ffb00 0x7f8e4c197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 --2- 172.21.15.5:0/4250158067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8e4c1004b0 0x7f8e4c1982f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 >> 172.21.15.5:0/4250158067 conn(0x7f8e4c0fb600 msgr2=0x7f8e4c10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 shutdown_connections 2023-07-21T19:16:15.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:15.816+0000 7f8e519ff700 1 -- 172.21.15.5:0/4250158067 wait complete. 2023-07-21T19:16:15.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:15 smithi101 ceph-mon[128136]: pgmap v268: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:16:15.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:15 smithi005 ceph-mon[126460]: pgmap v268: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:16:15.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:15 smithi156 ceph-mon[128316]: pgmap v268: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 34 MiB/s wr, 36 op/s 2023-07-21T19:16:16.352 INFO:teuthology.orchestra.run.smithi005.stderr:32400000 of 50000000 tuples (64%) done (elapsed 493.92 s, remaining 268.30 s) 2023-07-21T19:16:17.649 INFO:teuthology.orchestra.run.smithi005.stderr:32500000 of 50000000 tuples (65%) done (elapsed 495.22 s, remaining 266.65 s) 2023-07-21T19:16:17.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:17 smithi101 ceph-mon[128136]: pgmap v269: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:16:17.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:17 smithi005 ceph-mon[126460]: pgmap v269: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:16:17.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:17 smithi156 ceph-mon[128316]: pgmap v269: 97 pgs: 97 active+clean; 7.2 GiB data, 3.2 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 28 MiB/s wr, 29 op/s 2023-07-21T19:16:18.933 INFO:teuthology.orchestra.run.smithi005.stderr:32600000 of 50000000 tuples (65%) done (elapsed 496.50 s, remaining 265.00 s) 2023-07-21T19:16:19.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:19 smithi101 ceph-mon[128136]: pgmap v270: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:19.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:19 smithi005 ceph-mon[126460]: pgmap v270: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:19.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:19 smithi156 ceph-mon[128316]: pgmap v270: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:20.311 INFO:teuthology.orchestra.run.smithi005.stderr:32700000 of 50000000 tuples (65%) done (elapsed 497.88 s, remaining 263.40 s) 2023-07-21T19:16:21.591 INFO:teuthology.orchestra.run.smithi005.stderr:32800000 of 50000000 tuples (65%) done (elapsed 499.16 s, remaining 261.75 s) 2023-07-21T19:16:21.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:21 smithi101 ceph-mon[128136]: pgmap v271: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:21.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:21 smithi005 ceph-mon[126460]: pgmap v271: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:21.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:21 smithi156 ceph-mon[128316]: pgmap v271: 97 pgs: 97 active+clean; 7.3 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 27 MiB/s wr, 24 op/s 2023-07-21T19:16:22.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:22 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:32900000 of 50000000 tuples (65%) done (elapsed 500.47 s, remaining 260.12 s) 2023-07-21T19:16:22.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:22 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:22.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:22 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:23.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:23 smithi101 ceph-mon[128136]: pgmap v272: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 28 op/s 2023-07-21T19:16:23.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:23 smithi005 ceph-mon[126460]: pgmap v272: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 28 op/s 2023-07-21T19:16:23.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:23 smithi156 ceph-mon[128316]: pgmap v272: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 28 op/s 2023-07-21T19:16:24.299 INFO:teuthology.orchestra.run.smithi005.stderr:33000000 of 50000000 tuples (66%) done (elapsed 501.87 s, remaining 258.54 s) 2023-07-21T19:16:25.592 INFO:teuthology.orchestra.run.smithi005.stderr:33100000 of 50000000 tuples (66%) done (elapsed 503.16 s, remaining 256.90 s) 2023-07-21T19:16:27.158 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:16:27.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:25 smithi156 ceph-mon[128316]: pgmap v273: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:27.160 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:25 smithi101 ceph-mon[128136]: pgmap v273: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:27.160 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:25 smithi005 ceph-mon[126460]: pgmap v273: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:27.160 INFO:teuthology.orchestra.run.smithi005.stderr:33200000 of 50000000 tuples (66%) done (elapsed 504.48 s, remaining 255.28 s) 2023-07-21T19:16:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 -- 172.21.15.5:0/3675103421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f81900ff110 msgr2=0x7f819010be70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 --2- 172.21.15.5:0/3675103421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f81900ff110 0x7f819010be70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f81800098f0 tx=0x7f8180009c00 comp rx=0 tx=0).stop 2023-07-21T19:16:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 -- 172.21.15.5:0/3675103421 shutdown_connections 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 --2- 172.21.15.5:0/3675103421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f819010c3b0 0x7f819010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 --2- 172.21.15.5:0/3675103421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 0x7f81900febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 --2- 172.21.15.5:0/3675103421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f81900ff110 0x7f819010be70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 -- 172.21.15.5:0/3675103421 >> 172.21.15.5:0/3675103421 conn(0x7f81900745d0 msgr2=0x7f81900749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 -- 172.21.15.5:0/3675103421 shutdown_connections 2023-07-21T19:16:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.411+0000 7f8196b9d700 1 -- 172.21.15.5:0/3675103421 wait complete. 2023-07-21T19:16:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 Processor -- start 2023-07-21T19:16:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 -- start start 2023-07-21T19:16:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 0x7f819019d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 0x7f8190197a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8190198080 con 0x7f81900ff110 2023-07-21T19:16:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81901981f0 con 0x7f819010c3b0 2023-07-21T19:16:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.413+0000 7f8196b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81901983b0 con 0x7f81900fe800 2023-07-21T19:16:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33488/0 (socket says 172.21.15.5:33488) 2023-07-21T19:16:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819539a700 1 -- 172.21.15.5:0/1078793904 learned_addr learned my addr 172.21.15.5:0/1078793904 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 0x7f8190197a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f8195b9b700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 0x7f819019d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 -- 172.21.15.5:0/1078793904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 msgr2=0x7f819019d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 0x7f819019d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 -- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 msgr2=0x7f819019dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 -- 172.21.15.5:0/1078793904 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8180009520 con 0x7f819010c3b0 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819539a700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.414+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 0x7f8190197a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f818800ef90 tx=0x7f818800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.415+0000 7f8186ffd700 1 -- 172.21.15.5:0/1078793904 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8188018070 con 0x7f819010c3b0 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.415+0000 7f8186ffd700 1 -- 172.21.15.5:0/1078793904 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818800f040 con 0x7f819010c3b0 2023-07-21T19:16:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.415+0000 7f8186ffd700 1 -- 172.21.15.5:0/1078793904 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818800cac0 con 0x7f819010c3b0 2023-07-21T19:16:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.415+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8190198630 con 0x7f819010c3b0 2023-07-21T19:16:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.415+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81901a4990 con 0x7f819010c3b0 2023-07-21T19:16:27.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.419+0000 7f8186ffd700 1 -- 172.21.15.5:0/1078793904 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f818800cce0 con 0x7f819010c3b0 2023-07-21T19:16:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 Processor -- start 2023-07-21T19:16:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 -- start start 2023-07-21T19:16:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 0x55cca44cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca43c5cb0 con 0x55cca44cad30 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca4418a40 con 0x55cca44c0e00 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.420+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca4411ed0 con 0x55cca44c04f0 2023-07-21T19:16:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f819539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f819539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 0x55cca44cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38454/0 (socket says 172.21.15.5:38454) 2023-07-21T19:16:27.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f819539a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 0x55cca44cd110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33496/0 (socket says 172.21.15.5:33496) 2023-07-21T19:16:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 -- 172.21.15.5:0/3602594005 learned_addr learned my addr 172.21.15.5:0/3602594005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8186ffd700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 0x7f817c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 -- 172.21.15.5:0/3602594005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 msgr2=0x55cca44c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 -- 172.21.15.5:0/3602594005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 msgr2=0x55cca44cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 0x55cca44cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8195b9b700 1 -- 172.21.15.5:0/3602594005 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca423fde0 con 0x55cca44c0e00 2023-07-21T19:16:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.421+0000 7f8186ffd700 1 -- 172.21.15.5:0/1078793904 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f81880cc830 con 0x7f819010c3b0 2023-07-21T19:16:27.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.422+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f818c009d80 tx=0x7f818c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fa7b51b7828d52a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.422+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 0x7f817c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.422+0000 7f817bfff700 1 -- 172.21.15.5:0/3602594005 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c004230 con 0x55cca44c0e00 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.422+0000 7f817bfff700 1 -- 172.21.15.5:0/3602594005 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818c004390 con 0x55cca44c0e00 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.422+0000 7f817bfff700 1 -- 172.21.15.5:0/3602594005 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c019800 con 0x55cca44c0e00 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3602594005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 msgr2=0x55cca44ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f818c009d80 tx=0x7f818c011720 comp rx=0 tx=0).stop 2023-07-21T19:16:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f819639c700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 0x7f817c0a3810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8188000f80 tx=0x7f8188009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3602594005 shutdown_connections 2023-07-21T19:16:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44cad30 0x55cca44cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3602594005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44c0e00 0x55cca44ca7f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3602594005 >> 172.21.15.5:0/3602594005 conn(0x55cca44b5ff0 msgr2=0x55cca44bb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3602594005 shutdown_connections 2023-07-21T19:16:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3602594005 wait complete. 2023-07-21T19:16:27.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.423+0000 7f81ab69bb80 1 -- start start 2023-07-21T19:16:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 Processor -- start 2023-07-21T19:16:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44ca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca43b1ac0 con 0x55cca44c0e00 2023-07-21T19:16:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca43b17c0 con 0x55cca44cad30 2023-07-21T19:16:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca4295920 con 0x55cca44c04f0 2023-07-21T19:16:27.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.424+0000 7f81ab69bb80 10 client.0 _ll_register_callbacks cb 0x55cca43af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33502/0 (socket says 172.21.15.5:33502) 2023-07-21T19:16:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 -- 172.21.15.5:0/3476999753 learned_addr learned my addr 172.21.15.5:0/3476999753 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f819539a700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f819539a700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44ca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 -- 172.21.15.5:0/3476999753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 msgr2=0x55cca44ca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44ca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 -- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 msgr2=0x55cca44c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f8195b9b700 1 -- 172.21.15.5:0/3476999753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f818c0099e0 con 0x55cca44c0e00 2023-07-21T19:16:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.425+0000 7f819539a700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:16:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f8195b9b700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f818c000c00 tx=0x7f818c0038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8e36cff9287af51 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81797fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:27.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c005450 con 0x55cca44c0e00 2023-07-21T19:16:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f818c005ce0 con 0x55cca44c0e00 2023-07-21T19:16:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cca436b4f0 con 0x55cca44c0e00 2023-07-21T19:16:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f818c009ed0 con 0x55cca44c0e00 2023-07-21T19:16:27.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81ab69bb80 10 client.17358 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.426+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cca4228c60 con 0x55cca44c0e00 2023-07-21T19:16:27.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cca436b4f0 con 0x55cca44c0e00 2023-07-21T19:16:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f818c021040 con 0x55cca44c0e00 2023-07-21T19:16:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 10 client.17358 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 20 client.17358 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:27.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca44da6c0 0x55cca44dcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 4 client.17358 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cca440aeb0 con 0x55cca44da6c0 2023-07-21T19:16:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.427+0000 7f819539a700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca44da6c0 0x55cca44dcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.428+0000 7f81797fa700 10 client.17358 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:27.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.428+0000 7f819539a700 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca44da6c0 0x55cca44dcad0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b001a707b90ec05 server_cookie=8b641b6b84fa4a19 in_seq=0 out_seq=0 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81797fa700 1 -- 172.21.15.5:0/3476999753 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cca440aeb0 con 0x55cca44da6c0 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81797fa700 10 client.17358 handle_command_reply: tid=0 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 client.17358 shutdown 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 msgr2=0x55cca44c7950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f818c000c00 tx=0x7f818c0038b0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 shutdown_connections 2023-07-21T19:16:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44ca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca44da6c0 0x55cca44dcad0 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/3476999753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44c7950 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 >> 172.21.15.5:0/3476999753 conn(0x55cca44b5ff0 msgr2=0x55cca44bb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81797fa700 0 client.17358 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 shutdown_connections 2023-07-21T19:16:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.429+0000 7f81ab69bb80 1 -- 172.21.15.5:0/3476999753 wait complete. 2023-07-21T19:16:27.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.430+0000 7f81ab69bb80 20 client.17358 trim_cache size 0 max 16384 2023-07-21T19:16:27.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.449+0000 7f81ab69bb80 1 Processor -- start 2023-07-21T19:16:27.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 -- start start 2023-07-21T19:16:27.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44ca300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44c1240 con 0x55cca44c0e00 2023-07-21T19:16:27.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44d7da0 con 0x55cca44cad30 2023-07-21T19:16:27.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44c1980 con 0x55cca44c04f0 2023-07-21T19:16:27.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33508/0 (socket says 172.21.15.5:33508) 2023-07-21T19:16:27.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f8195b9b700 1 -- 172.21.15.5:0/2895621196 learned_addr learned my addr 172.21.15.5:0/2895621196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:27.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.450+0000 7f819539a700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f819539a700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 -- 172.21.15.5:0/2895621196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 msgr2=0x55cca44d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 -- 172.21.15.5:0/2895621196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 msgr2=0x55cca44ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 -- 172.21.15.5:0/2895621196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cca4266250 con 0x55cca44c0e00 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f819539a700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f818c005a60 tx=0x7f818c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=433736612ecaa695 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8184bff700 1 -- 172.21.15.5:0/2895621196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0183d0 con 0x55cca44c0e00 2023-07-21T19:16:27.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8184bff700 1 -- 172.21.15.5:0/2895621196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818c0236a0 con 0x55cca44c0e00 2023-07-21T19:16:27.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.451+0000 7f8184bff700 1 -- 172.21.15.5:0/2895621196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c0054b0 con 0x55cca44c0e00 2023-07-21T19:16:27.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2895621196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 msgr2=0x55cca44ba060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7f818c005a60 tx=0x7f818c009260 comp rx=0 tx=0).stop 2023-07-21T19:16:27.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2895621196 shutdown_connections 2023-07-21T19:16:27.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44ca300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca44d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2895621196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca44ba060 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2895621196 >> 172.21.15.5:0/2895621196 conn(0x55cca44b5ff0 msgr2=0x55cca44d6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2895621196 shutdown_connections 2023-07-21T19:16:27.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.452+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2895621196 wait complete. 2023-07-21T19:16:27.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.453+0000 7f81ab69bb80 1 -- start start 2023-07-21T19:16:27.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.453+0000 7f81ab69bb80 1 Processor -- start 2023-07-21T19:16:27.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca4527140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44d0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44d1540 con 0x55cca44c0e00 2023-07-21T19:16:27.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44d16b0 con 0x55cca44cad30 2023-07-21T19:16:27.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cca44d1870 con 0x55cca44c04f0 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f81ab69bb80 10 client.0 _ll_register_callbacks cb 0x7f8188009880 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33516/0 (socket says 172.21.15.5:33516) 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 -- 172.21.15.5:0/2404122790 learned_addr learned my addr 172.21.15.5:0/2404122790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f819539a700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44d0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f819539a700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca4527140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 -- 172.21.15.5:0/2404122790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 msgr2=0x55cca4527140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca4527140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 -- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 msgr2=0x55cca44d0f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44d0f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f8195b9b700 1 -- 172.21.15.5:0/2404122790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cca4228c60 con 0x55cca44c0e00 2023-07-21T19:16:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.454+0000 7f819539a700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f8195b9b700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f818c005e50 tx=0x7f818c0183d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fb45d5615e6b117 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81797fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:27.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f818c003dd0 con 0x55cca44c0e00 2023-07-21T19:16:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f818c004390 con 0x55cca44c0e00 2023-07-21T19:16:27.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f818c000e40 con 0x55cca44c0e00 2023-07-21T19:16:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f818c005250 con 0x55cca44c0e00 2023-07-21T19:16:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.455+0000 7f81ab69bb80 10 client.17364 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55cca423e2b0 con 0x55cca44c0e00 2023-07-21T19:16:27.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f818c04d020 con 0x55cca44c0e00 2023-07-21T19:16:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f818c021030 con 0x55cca44c0e00 2023-07-21T19:16:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 10 client.17364 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 10 client.17364 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 20 client.17364 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:27.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca452ec00 0x55cca4531010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 4 client.17364 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:16:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cca43cd050 con 0x55cca452ec00 2023-07-21T19:16:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.456+0000 7f819539a700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca452ec00 0x55cca4531010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:27.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f819539a700 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca452ec00 0x55cca4531010 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8ad1659d498388 server_cookie=4ff5ef027666b6d7 in_seq=0 out_seq=0 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81797fa700 10 client.17364 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81797fa700 1 -- 172.21.15.5:0/2404122790 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55cca43cd050 con 0x55cca452ec00 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81797fa700 10 client.17364 handle_command_reply: tid=0 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 client.17364 shutdown 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 msgr2=0x55cca4527680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f818c005e50 tx=0x7f818c0183d0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 shutdown_connections 2023-07-21T19:16:27.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cca44cad30 0x55cca44d0f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cca44c04f0 0x55cca4527140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cca452ec00 0x55cca4531010 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 --2- 172.21.15.5:0/2404122790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cca44c0e00 0x55cca4527680 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.457+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 >> 172.21.15.5:0/2404122790 conn(0x55cca44b5ff0 msgr2=0x55cca44bab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.458+0000 7f81797fa700 0 client.17364 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.458+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 shutdown_connections 2023-07-21T19:16:27.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.458+0000 7f81ab69bb80 1 -- 172.21.15.5:0/2404122790 wait complete. 2023-07-21T19:16:27.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.458+0000 7f81ab69bb80 20 client.17364 trim_cache size 0 max 16384 2023-07-21T19:16:27.473 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:16:27.473 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:16:27.473 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:16:27.473 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:16:27.474 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:16:27.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:16:27.474 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:16:27.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:16:27.474 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:16:27.475 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:16:27.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 msgr2=0x7f817c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 0x7f817c0a3810 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f8188000f80 tx=0x7f8188009040 comp rx=0 tx=0).stop 2023-07-21T19:16:27.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 msgr2=0x7f8190197a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:27.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 0x7f8190197a10 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f818800ef90 tx=0x7f818800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 shutdown_connections 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f819010c3b0 0x7f8190197a10 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f81900fe800 0x7f819019d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f817c0a1360 0x7f817c0a3810 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 --2- 172.21.15.5:0/1078793904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f81900ff110 0x7f819019dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:27.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 >> 172.21.15.5:0/1078793904 conn(0x7f81900745d0 msgr2=0x7f81900fc430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 shutdown_connections 2023-07-21T19:16:27.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:27.459+0000 7f8196b9d700 1 -- 172.21.15.5:0/1078793904 wait complete. 2023-07-21T19:16:27.479 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:16:27.657 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:27 smithi005 ceph-mon[126460]: pgmap v274: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:27.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:27 smithi101 ceph-mon[128136]: pgmap v274: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:27.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:27 smithi156 ceph-mon[128316]: pgmap v274: 97 pgs: 97 active+clean; 7.4 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 MiB/s wr, 24 op/s 2023-07-21T19:16:28.339 INFO:teuthology.orchestra.run.smithi005.stderr:33300000 of 50000000 tuples (66%) done (elapsed 505.90 s, remaining 253.71 s) 2023-07-21T19:16:28.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 -- 172.21.15.5:0/3393694699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 msgr2=0x7fe1c0072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/3393694699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0072c00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc009990 tx=0x7fe1bc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 -- 172.21.15.5:0/3393694699 shutdown_connections 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/3393694699 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/3393694699 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0072c00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/3393694699 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c010bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 -- 172.21.15.5:0/3393694699 >> 172.21.15.5:0/3393694699 conn(0x7fe1c00fb650 msgr2=0x7fe1c00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:28.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 -- 172.21.15.5:0/3393694699 shutdown_connections 2023-07-21T19:16:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.980+0000 7fe1c79ff700 1 -- 172.21.15.5:0/3393694699 wait complete. 2023-07-21T19:16:28.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.981+0000 7fe1c79ff700 1 Processor -- start 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 -- start start 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c019c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c019cf80 con 0x7fe1c010c410 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c019d0f0 con 0x7fe1c0072830 2023-07-21T19:16:28.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.982+0000 7fe1c79ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c019d2b0 con 0x7fe1c0073140 2023-07-21T19:16:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c5f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c019c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38482/0 (socket says 172.21.15.5:38482) 2023-07-21T19:16:28.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c579b700 1 -- 172.21.15.5:0/2582682679 learned_addr learned my addr 172.21.15.5:0/2582682679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/2582682679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 msgr2=0x7fe1c0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 msgr2=0x7fe1c019c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c019c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/2582682679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1bc0095f0 con 0x7fe1c0073140 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.983+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c01982d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000d950 tx=0x7fe1b000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.984+0000 7fe1b67fc700 1 -- 172.21.15.5:0/2582682679 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000f890 con 0x7fe1c0073140 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.984+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c019d590 con 0x7fe1c0073140 2023-07-21T19:16:28.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.984+0000 7fe1b67fc700 1 -- 172.21.15.5:0/2582682679 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1b000fe80 con 0x7fe1c0073140 2023-07-21T19:16:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.984+0000 7fe1b67fc700 1 -- 172.21.15.5:0/2582682679 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b000e3b0 con 0x7fe1c0073140 2023-07-21T19:16:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.984+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c01a4dd0 con 0x7fe1c0073140 2023-07-21T19:16:28.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.988+0000 7fe1b67fc700 1 -- 172.21.15.5:0/2582682679 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe1b000f9f0 con 0x7fe1c0073140 2023-07-21T19:16:28.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 Processor -- start 2023-07-21T19:16:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 -- start start 2023-07-21T19:16:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 0x55e132437040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324379d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13233cc50 con 0x55e13243d2d0 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13238f9e0 con 0x55e1322936b0 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.989+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e132388e70 con 0x55e132437580 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c5f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 0x55e132437040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c5f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38488/0 (socket says 172.21.15.5:38488) 2023-07-21T19:16:28.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c5f9c700 1 -- 172.21.15.5:0/3734541447 learned_addr learned my addr 172.21.15.5:0/3734541447 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324379d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1b67fc700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1b67fc700 1 -- 172.21.15.5:0/2582682679 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe1b00cca20 con 0x7fe1c0073140 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 -- 172.21.15.5:0/3734541447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 msgr2=0x55e1324379d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324379d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 -- 172.21.15.5:0/3734541447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 msgr2=0x55e13243cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:28.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c579b700 1 -- 172.21.15.5:0/3734541447 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1321b6d80 con 0x55e13243d2d0 2023-07-21T19:16:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.990+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:16:28.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.991+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.991+0000 7fe1c579b700 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 0x55e132437040 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=617c66e3569d92db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.991+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3734541447 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc01d070 con 0x55e13243d2d0 2023-07-21T19:16:28.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.991+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3734541447 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1bc01e040 con 0x55e13243d2d0 2023-07-21T19:16:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.991+0000 7fe1ab7fe700 1 -- 172.21.15.5:0/3734541447 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc0048e0 con 0x55e13243d2d0 2023-07-21T19:16:28.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/3734541447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 msgr2=0x55e132437040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009fd0 tx=0x7fe1b8004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 0x55e132437040 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005f00 comp rx=0 tx=0).stop 2023-07-21T19:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/3734541447 shutdown_connections 2023-07-21T19:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324379d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e13243cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/3734541447 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e13243d2d0 0x55e132437040 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/3734541447 >> 172.21.15.5:0/3734541447 conn(0x55e13242cbe0 msgr2=0x55e132431f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/3734541447 shutdown_connections 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.992+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/3734541447 wait complete. 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.993+0000 7fe1db2b9b80 1 -- start start 2023-07-21T19:16:28.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.993+0000 7fe1db2b9b80 1 Processor -- start 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 0x55e132443540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324407c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e132328a60 con 0x55e1322936b0 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e132328760 con 0x55e13243d2d0 2023-07-21T19:16:28.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13220c8c0 con 0x55e132437580 2023-07-21T19:16:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1db2b9b80 10 client.0 _ll_register_callbacks cb 0x55e132326700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 0x55e132443540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c5f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c5f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38498/0 (socket says 172.21.15.5:38498) 2023-07-21T19:16:29.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c5f9c700 1 -- 172.21.15.5:0/2623055617 learned_addr learned my addr 172.21.15.5:0/2623055617 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c4f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c579b700 1 -- 172.21.15.5:0/2623055617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 msgr2=0x55e1324407c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c579b700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324407c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c579b700 1 -- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 msgr2=0x55e132440dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.994+0000 7fe1c579b700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1c579b700 1 -- 172.21.15.5:0/2623055617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b8003be0 con 0x55e1322936b0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1c579b700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 0x55e132443540 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318941420e125930 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc01d070 con 0x55e1322936b0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1322e2490 con 0x55e1322936b0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe1bc00bb10 con 0x55e1322936b0 2023-07-21T19:16:29.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.995+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1bc004910 con 0x55e1322936b0 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 10 client.17376 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x7fe1c010f3f0 con 0x55e1322936b0 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1322e2490 con 0x55e1322936b0 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe1bc00fbb0 con 0x55e1322936b0 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 10 client.17376 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:29.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 10 client.17376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 20 client.17376 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e132450730 0x55e132452b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 4 client.17376 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:29.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e132381e50 con 0x55e132450730 2023-07-21T19:16:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.996+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e132450730 0x55e132452b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.997+0000 7fe1a8ff9700 10 client.17376 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:29.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.997+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e132450730 0x55e132452b40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9571373721fd7fbb server_cookie=8cb151a5680ba633 in_seq=0 out_seq=0 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/2623055617 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e132381e50 con 0x55e132450730 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1a8ff9700 10 client.17376 handle_command_reply: tid=0 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 client.17376 shutdown 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 msgr2=0x55e132443540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 0x55e132443540 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc000c00 tx=0x7fe1bc005cd0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 shutdown_connections 2023-07-21T19:16:29.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e132440dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e132450730 0x55e132452b40 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1322936b0 0x55e132443540 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/2623055617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e132437580 0x55e1324407c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 >> 172.21.15.5:0/2623055617 conn(0x55e13242cbe0 msgr2=0x55e132430c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1a8ff9700 0 client.17376 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 shutdown_connections 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.998+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/2623055617 wait complete. 2023-07-21T19:16:29.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:28.999+0000 7fe1db2b9b80 20 client.17376 trim_cache size 0 max 16384 2023-07-21T19:16:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 Processor -- start 2023-07-21T19:16:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 -- start start 2023-07-21T19:16:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1322936b0 0x55e132431c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e13243c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13243b540 con 0x55e132437580 2023-07-21T19:16:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e132432e90 con 0x55e13243d2d0 2023-07-21T19:16:29.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.014+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e13244d420 con 0x55e1322936b0 2023-07-21T19:16:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33558/0 (socket says 172.21.15.5:33558) 2023-07-21T19:16:29.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/86548242 learned_addr learned my addr 172.21.15.5:0/86548242 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e13243c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c579b700 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1322936b0 0x55e132431c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/86548242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1322936b0 msgr2=0x55e132431c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1322936b0 0x55e132431c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/86548242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 msgr2=0x55e13243c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e13243c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/86548242 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e13219fc00 con 0x55e132437580 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.015+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00ca770 tx=0x7fe1b00cef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50dd3d8e96a0ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.016+0000 7fe1abfff700 1 -- 172.21.15.5:0/86548242 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0014070 con 0x55e132437580 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.016+0000 7fe1abfff700 1 -- 172.21.15.5:0/86548242 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1b00cce90 con 0x55e132437580 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.016+0000 7fe1abfff700 1 -- 172.21.15.5:0/86548242 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b009db20 con 0x55e132437580 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.016+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/86548242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 msgr2=0x55e1324430e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:29.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.016+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00ca770 tx=0x7fe1b00cef60 comp rx=0 tx=0).stop 2023-07-21T19:16:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/86548242 shutdown_connections 2023-07-21T19:16:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e13243d2d0 0x55e13243c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1322936b0 0x55e132431c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/86548242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e1324430e0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/86548242 >> 172.21.15.5:0/86548242 conn(0x55e13242cbe0 msgr2=0x55e13243b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/86548242 shutdown_connections 2023-07-21T19:16:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/86548242 wait complete. 2023-07-21T19:16:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 -- start start 2023-07-21T19:16:29.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.017+0000 7fe1db2b9b80 1 Processor -- start 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e132446380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e13243d2d0 0x55e1324a35b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e132446b00 con 0x55e132437580 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1324a3b20 con 0x55e1322936b0 2023-07-21T19:16:29.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1324a3ce0 con 0x55e13243d2d0 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1db2b9b80 10 client.0 _ll_register_callbacks cb 0x7fe1b8003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1c4f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e132446380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1c579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38514/0 (socket says 172.21.15.5:38514) 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1c579b700 1 -- 172.21.15.5:0/559317790 learned_addr learned my addr 172.21.15.5:0/559317790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:29.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.018+0000 7fe1c5f9c700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e13243d2d0 0x55e1324a35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/559317790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e13243d2d0 msgr2=0x55e1324a35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e13243d2d0 0x55e1324a35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 msgr2=0x55e132445e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 -- 172.21.15.5:0/559317790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c01a4dd0 con 0x55e132437580 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c579b700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:29.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1c4f9a700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e132446380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00106a0 tx=0x7fe1b00107b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dc53d7d897ba09d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1a8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0014070 con 0x55e132437580 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe1b0005b00 con 0x55e132437580 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe1bc01e040 con 0x55e132437580 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe1b00a1f40 con 0x55e132437580 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1db2b9b80 10 client.17382 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.019+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x55e1321dd1f0 con 0x55e132437580 2023-07-21T19:16:29.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe1bc01e040 con 0x55e132437580 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fe1b0099d50 con 0x55e132437580 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 10 client.17382 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 10 client.17382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 20 client.17382 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1324a5020 0x55e1324a7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 4 client.17382 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e132343ff0 con 0x55e1324a5020 2023-07-21T19:16:29.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.020+0000 7fe1c579b700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1324a5020 0x55e1324a7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.021+0000 7fe1c579b700 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1324a5020 0x55e1324a7430 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbfa5911f1f4c336 server_cookie=f49a4ff9ab8c1b0 in_seq=0 out_seq=0 2023-07-21T19:16:29.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.021+0000 7fe1a8ff9700 10 client.17382 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1a8ff9700 1 -- 172.21.15.5:0/559317790 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e132343ff0 con 0x55e1324a5020 2023-07-21T19:16:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1a8ff9700 10 client.17382 handle_command_reply: tid=0 2023-07-21T19:16:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 client.17382 shutdown 2023-07-21T19:16:29.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 msgr2=0x55e132446380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e132446380 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe1b00106a0 tx=0x7fe1b00107b0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 shutdown_connections 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e13243d2d0 0x55e1324a35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1322936b0 0x55e132445e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1324a5020 0x55e1324a7430 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 --2- 172.21.15.5:0/559317790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e132437580 0x55e132446380 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1a8ff9700 0 client.17382 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 >> 172.21.15.5:0/559317790 conn(0x55e13242cbe0 msgr2=0x55e13243c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 shutdown_connections 2023-07-21T19:16:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 1 -- 172.21.15.5:0/559317790 wait complete. 2023-07-21T19:16:29.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.034+0000 7fe1db2b9b80 20 client.17382 trim_cache size 0 max 16384 2023-07-21T19:16:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:16:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:16:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:16:29.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:16:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 msgr2=0x7fe1ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe1b8009fd0 tx=0x7fe1b8004ad0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 msgr2=0x7fe1c01982d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:29.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c01982d0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe1b000d950 tx=0x7fe1b000dc60 comp rx=0 tx=0).stop 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 shutdown_connections 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe1ac0a12c0 0x7fe1ac0a3770 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe1c010c410 0x7fe1c019c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe1c0072830 0x7fe1c0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 --2- 172.21.15.5:0/2582682679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe1c0073140 0x7fe1c01982d0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 >> 172.21.15.5:0/2582682679 conn(0x7fe1c00fb650 msgr2=0x7fe1c010e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 shutdown_connections 2023-07-21T19:16:29.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:29.035+0000 7fe1c79ff700 1 -- 172.21.15.5:0/2582682679 wait complete. 2023-07-21T19:16:29.663 INFO:teuthology.orchestra.run.smithi005.stderr:33400000 of 50000000 tuples (66%) done (elapsed 507.23 s, remaining 252.10 s) 2023-07-21T19:16:29.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:29 smithi101 ceph-mon[128136]: pgmap v275: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-07-21T19:16:29.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:29 smithi005 ceph-mon[126460]: pgmap v275: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-07-21T19:16:29.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:29 smithi156 ceph-mon[128316]: pgmap v275: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 34 MiB/s wr, 42 op/s 2023-07-21T19:16:31.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:31 smithi101 ceph-mon[128136]: pgmap v276: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:16:31.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:31 smithi005 ceph-mon[126460]: pgmap v276: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:16:31.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:31 smithi156 ceph-mon[128316]: pgmap v276: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 22 MiB/s wr, 31 op/s 2023-07-21T19:16:32.875 INFO:teuthology.orchestra.run.smithi005.stderr:33500000 of 50000000 tuples (67%) done (elapsed 510.44 s, remaining 251.41 s) 2023-07-21T19:16:33.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:33 smithi101 ceph-mon[128136]: pgmap v277: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:16:33.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:33 smithi005 ceph-mon[126460]: pgmap v277: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:16:33.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:33 smithi156 ceph-mon[128316]: pgmap v277: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:16:34.230 INFO:teuthology.orchestra.run.smithi005.stderr:33600000 of 50000000 tuples (67%) done (elapsed 511.80 s, remaining 249.81 s) 2023-07-21T19:16:35.659 INFO:teuthology.orchestra.run.smithi005.stderr:33700000 of 50000000 tuples (67%) done (elapsed 513.23 s, remaining 248.24 s) 2023-07-21T19:16:35.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:35 smithi101 ceph-mon[128136]: pgmap v278: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:16:35.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:35 smithi005 ceph-mon[126460]: pgmap v278: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:16:35.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:35 smithi156 ceph-mon[128316]: pgmap v278: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 21 MiB/s wr, 36 op/s 2023-07-21T19:16:36.967 INFO:teuthology.orchestra.run.smithi005.stderr:33800000 of 50000000 tuples (67%) done (elapsed 514.54 s, remaining 246.61 s) 2023-07-21T19:16:37.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:37 smithi101 ceph-mon[128136]: pgmap v279: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 18 MiB/s wr, 34 op/s 2023-07-21T19:16:37.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:37 smithi005 ceph-mon[126460]: pgmap v279: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 18 MiB/s wr, 34 op/s 2023-07-21T19:16:37.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:37 smithi156 ceph-mon[128316]: pgmap v279: 97 pgs: 97 active+clean; 7.5 GiB data, 3.3 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 18 MiB/s wr, 34 op/s 2023-07-21T19:16:38.290 INFO:teuthology.orchestra.run.smithi005.stderr:33900000 of 50000000 tuples (67%) done (elapsed 515.86 s, remaining 244.99 s) 2023-07-21T19:16:39.696 INFO:teuthology.orchestra.run.smithi005.stderr:34000000 of 50000000 tuples (68%) done (elapsed 517.26 s, remaining 243.42 s) 2023-07-21T19:16:39.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:39 smithi101 ceph-mon[128136]: pgmap v280: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 60 op/s 2023-07-21T19:16:39.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:39 smithi005 ceph-mon[126460]: pgmap v280: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 60 op/s 2023-07-21T19:16:39.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:39 smithi156 ceph-mon[128316]: pgmap v280: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 29 MiB/s wr, 60 op/s 2023-07-21T19:16:41.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:41 smithi101 ceph-mon[128136]: pgmap v281: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 42 op/s 2023-07-21T19:16:41.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:41 smithi005 ceph-mon[126460]: pgmap v281: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 42 op/s 2023-07-21T19:16:41.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:41 smithi156 ceph-mon[128316]: pgmap v281: 97 pgs: 97 active+clean; 7.6 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 18 MiB/s wr, 42 op/s 2023-07-21T19:16:42.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 -- 172.21.15.5:0/332464965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d00685b0 msgr2=0x7f94d0104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 --2- 172.21.15.5:0/332464965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d00685b0 0x7f94d0104770 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f94c40099c0 tx=0x7f94c4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 -- 172.21.15.5:0/332464965 shutdown_connections 2023-07-21T19:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 --2- 172.21.15.5:0/332464965 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d0104e50 0x7f94d010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 --2- 172.21.15.5:0/332464965 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0067c10 0x7f94d0067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 --2- 172.21.15.5:0/332464965 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d00685b0 0x7f94d0104770 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 -- 172.21.15.5:0/332464965 >> 172.21.15.5:0/332464965 conn(0x7f94d0074760 msgr2=0x7f94d0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 -- 172.21.15.5:0/332464965 shutdown_connections 2023-07-21T19:16:42.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.210+0000 7f94d63ff700 1 -- 172.21.15.5:0/332464965 wait complete. 2023-07-21T19:16:42.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 Processor -- start 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 -- start start 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d00685b0 0x7f94d0105e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 0x7f94d01a82b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01066d0 con 0x7f94d0067c10 2023-07-21T19:16:42.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01a87f0 con 0x7f94d0104e50 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94d63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94d01a8930 con 0x7f94d00685b0 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.212+0000 7f94cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94cffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49288/0 (socket says 172.21.15.5:49288) 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94cffff700 1 -- 172.21.15.5:0/3095237394 learned_addr learned my addr 172.21.15.5:0/3095237394 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 0x7f94d01a82b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d00685b0 0x7f94d0105e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 -- 172.21.15.5:0/3095237394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d00685b0 msgr2=0x7f94d0105e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d00685b0 0x7f94d0105e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 -- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 msgr2=0x7f94d0105930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 -- 172.21.15.5:0/3095237394 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40095f0 con 0x7f94d0104e50 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94cffff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.213+0000 7f94d499c700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 0x7f94d01a82b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94c800efc0 tx=0x7f94c800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.214+0000 7f94cd7fa700 1 -- 172.21.15.5:0/3095237394 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c8018070 con 0x7f94d0104e50 2023-07-21T19:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.214+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94d01a8b30 con 0x7f94d0104e50 2023-07-21T19:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.214+0000 7f94cd7fa700 1 -- 172.21.15.5:0/3095237394 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94c800f040 con 0x7f94d0104e50 2023-07-21T19:16:42.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.214+0000 7f94cd7fa700 1 -- 172.21.15.5:0/3095237394 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c800cba0 con 0x7f94d0104e50 2023-07-21T19:16:42.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.214+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d01a9060 con 0x7f94d0104e50 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.218+0000 7f94cd7fa700 1 -- 172.21.15.5:0/3095237394 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f94c8003a00 con 0x7f94d0104e50 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.219+0000 7f94cd7fa700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 0x7f94b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.219+0000 7f94e9d51b80 1 Processor -- start 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.219+0000 7f94cd7fa700 1 -- 172.21.15.5:0/3095237394 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f94c80cc820 con 0x7f94d0104e50 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.219+0000 7f94e9d51b80 1 -- start start 2023-07-21T19:16:42.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.219+0000 7f94cffff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 0x7f94b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818c9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a817c9c50 con 0x558a818c4580 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a8181c9e0 con 0x558a818ca2d0 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a81815e70 con 0x558a817206b0 2023-07-21T19:16:42.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60736/0 (socket says 172.21.15.5:60736) 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94cf7fe700 1 -- 172.21.15.5:0/174723598 learned_addr learned my addr 172.21.15.5:0/174723598 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94cffff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 0x7f94b80a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94c00115a0 tx=0x7f94c0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94d499c700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.220+0000 7f94cffff700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818c9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 -- 172.21.15.5:0/174723598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 msgr2=0x558a818c9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818c9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 -- 172.21.15.5:0/174723598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 msgr2=0x558a818c49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818c49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 -- 172.21.15.5:0/174723598 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a81643d80 con 0x558a818ca2d0 2023-07-21T19:16:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cffff700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:16:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b580 tx=0x7f94c4005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=71083900e18db97a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94b67fc700 1 -- 172.21.15.5:0/174723598 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c400ba10 con 0x558a818ca2d0 2023-07-21T19:16:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94b67fc700 1 -- 172.21.15.5:0/174723598 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94c401e040 con 0x558a818ca2d0 2023-07-21T19:16:42.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.221+0000 7f94b67fc700 1 -- 172.21.15.5:0/174723598 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c40039c0 con 0x558a818ca2d0 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 -- 172.21.15.5:0/174723598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 msgr2=0x558a818c4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f94c400b580 tx=0x7f94c4005c30 comp rx=0 tx=0).stop 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 -- 172.21.15.5:0/174723598 shutdown_connections 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818c49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818c9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/174723598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818c4040 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 -- 172.21.15.5:0/174723598 >> 172.21.15.5:0/174723598 conn(0x558a818b9be0 msgr2=0x558a818bef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 -- 172.21.15.5:0/174723598 shutdown_connections 2023-07-21T19:16:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.222+0000 7f94e9d51b80 1 -- 172.21.15.5:0/174723598 wait complete. 2023-07-21T19:16:42.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.223+0000 7f94e9d51b80 1 -- start start 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.223+0000 7f94e9d51b80 1 Processor -- start 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818cd890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818cdfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a817b5a60 con 0x558a818c4580 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a817b5760 con 0x558a818ca2d0 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a816998c0 con 0x558a817206b0 2023-07-21T19:16:42.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94e9d51b80 10 client.0 _ll_register_callbacks cb 0x558a817b3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94cffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94d499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818cdfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818cd890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94cffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36410/0 (socket says 172.21.15.5:36410) 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94cffff700 1 -- 172.21.15.5:0/60631046 learned_addr learned my addr 172.21.15.5:0/60631046 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.224+0000 7f94cffff700 1 -- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 msgr2=0x558a818cdfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cffff700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818cdfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cffff700 1 -- 172.21.15.5:0/60631046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 msgr2=0x558a818cd890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cffff700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818cd890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cffff700 1 -- 172.21.15.5:0/60631046 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94d010d210 con 0x558a817206b0 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94d499c700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94cffff700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94c00046e0 tx=0x7f94c00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42da33cc7b95e9c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:16:42.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000f870 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a8176f490 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.225+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f94c000f9d0 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94c000fb90 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 10 client.27016 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558a8162cc00 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94c0033080 con 0x558a817206b0 2023-07-21T19:16:42.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f94c002d030 con 0x558a817206b0 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 10 client.27016 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 10 client.27016 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 20 client.27016 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a818ddc80 0x558a818e0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 4 client.27016 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.226+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a8180ee50 con 0x558a818ddc80 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.227+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a818ddc80 0x558a818e0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.227+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a818ddc80 0x558a818e0090 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c1af5db4bd529e server_cookie=2ed608ad904ffdfe in_seq=0 out_seq=0 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.227+0000 7f94af7fe700 10 client.27016 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94af7fe700 1 -- 172.21.15.5:0/60631046 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a8180ee50 con 0x558a818ddc80 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94af7fe700 10 client.27016 handle_command_reply: tid=0 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 client.27016 shutdown 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 msgr2=0x558a818d0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f94c00046e0 tx=0x7f94c00047c0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 shutdown_connections 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818ca2d0 0x558a818cdfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d0610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a818ddc80 0x558a818e0090 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/60631046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818cd890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 >> 172.21.15.5:0/60631046 conn(0x558a818b9be0 msgr2=0x558a818bdc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.228+0000 7f94af7fe700 0 client.27016 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:42.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.229+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 shutdown_connections 2023-07-21T19:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.229+0000 7f94e9d51b80 1 -- 172.21.15.5:0/60631046 wait complete. 2023-07-21T19:16:42.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.229+0000 7f94e9d51b80 20 client.27016 trim_cache size 0 max 16384 2023-07-21T19:16:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.251+0000 7f94e9d51b80 1 Processor -- start 2023-07-21T19:16:42.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 -- start start 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a818ca2d0 0x558a818d01b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a818bbd40 con 0x558a818c4580 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a818c0990 con 0x558a817206b0 2023-07-21T19:16:42.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a818ce990 con 0x558a818ca2d0 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.252+0000 7f94cffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60750/0 (socket says 172.21.15.5:60750) 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49318/0 (socket says 172.21.15.5:49318) 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 -- 172.21.15.5:0/3299911929 learned_addr learned my addr 172.21.15.5:0/3299911929 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94d499c700 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a818ca2d0 0x558a818d01b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 -- 172.21.15.5:0/3299911929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a818ca2d0 msgr2=0x558a818d01b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a818ca2d0 0x558a818d01b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 -- 172.21.15.5:0/3299911929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 msgr2=0x558a818bec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 -- 172.21.15.5:0/3299911929 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a8166a1f0 con 0x558a817206b0 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:42.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.253+0000 7f94cffff700 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c0027e80 tx=0x7f94c0005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6610a22483c93cfd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94b6ffd700 1 -- 172.21.15.5:0/3299911929 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c000f970 con 0x558a817206b0 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94b6ffd700 1 -- 172.21.15.5:0/3299911929 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558a8176f490 con 0x558a817206b0 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94b6ffd700 1 -- 172.21.15.5:0/3299911929 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c001d540 con 0x558a817206b0 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 -- 172.21.15.5:0/3299911929 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 msgr2=0x558a818c91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f94c0027e80 tx=0x7f94c0005a40 comp rx=0 tx=0).stop 2023-07-21T19:16:42.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 -- 172.21.15.5:0/3299911929 shutdown_connections 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a818ca2d0 0x558a818d01b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a817206b0 0x558a818c91a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/3299911929 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818c4580 0x558a818bec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.254+0000 7f94e9d51b80 1 -- 172.21.15.5:0/3299911929 >> 172.21.15.5:0/3299911929 conn(0x558a818b9be0 msgr2=0x558a818c8a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.255+0000 7f94e9d51b80 1 -- 172.21.15.5:0/3299911929 shutdown_connections 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.255+0000 7f94e9d51b80 1 -- 172.21.15.5:0/3299911929 wait complete. 2023-07-21T19:16:42.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.255+0000 7f94e9d51b80 1 -- start start 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.255+0000 7f94e9d51b80 1 Processor -- start 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d2e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 0x558a818d3380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818ca2d0 0x558a81930b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a81931080 con 0x558a818ca2d0 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a819311c0 con 0x558a818c4580 2023-07-21T19:16:42.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a81931300 con 0x558a817206b0 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94e9d51b80 10 client.0 _ll_register_callbacks cb 0x7f94c001e8f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 0x558a818d3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94cffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94cf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 0x558a818d3380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60752/0 (socket says 172.21.15.5:60752) 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94cf7fe700 1 -- 172.21.15.5:0/2277326193 learned_addr learned my addr 172.21.15.5:0/2277326193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.256+0000 7f94d499c700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818ca2d0 0x558a81930b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 -- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 msgr2=0x558a818d3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 0x558a818d3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 -- 172.21.15.5:0/2277326193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818ca2d0 msgr2=0x558a81930b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818ca2d0 0x558a81930b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 -- 172.21.15.5:0/2277326193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a81643d80 con 0x558a817206b0 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94cffff700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d2e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94c0027f00 tx=0x7f94c000fd60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e56ff29a9b95573 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94af7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:16:42.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94c0004c90 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f94c401e040 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f94c0004df0 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=37) v2 ==== 24+0+0 (secure 0 0 0) 0x7f94c0005030 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94e9d51b80 10 client.27022 fetch_fsmap learned FSMap version 37 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.257+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=37,monmap=4+}) v3 -- 0x558a8162cc00 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f94c401e040 con 0x558a817206b0 2023-07-21T19:16:42.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 37) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f94c00278b0 con 0x558a817206b0 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 10 client.27022 fetch_fsmap finished waiting for FSMap version 37 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 10 client.27022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 20 client.27022 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a819326d0 0x558a81934ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 4 client.27022 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:16:42.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.258+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558a817d0ff0 con 0x558a819326d0 2023-07-21T19:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.259+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a819326d0 0x558a81934ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.259+0000 7f94cf7fe700 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a819326d0 0x558a81934ae0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d4f99033c4f1b5fb server_cookie=963d5accd3354a5 in_seq=0 out_seq=0 2023-07-21T19:16:42.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.259+0000 7f94af7fe700 10 client.27022 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:42.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.273+0000 7f94af7fe700 1 -- 172.21.15.5:0/2277326193 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558a817d0ff0 con 0x558a819326d0 2023-07-21T19:16:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.273+0000 7f94af7fe700 10 client.27022 handle_command_reply: tid=0 2023-07-21T19:16:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 client.27022 shutdown 2023-07-21T19:16:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 msgr2=0x558a818d2e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d2e40 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f94c0027f00 tx=0x7f94c000fd60 comp rx=0 tx=0).stop 2023-07-21T19:16:42.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 shutdown_connections 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a819326d0 0x558a81934ae0 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a818ca2d0 0x558a81930b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94af7fe700 0 client.27022 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a817206b0 0x558a818d2e40 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 --2- 172.21.15.5:0/2277326193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a818c4580 0x558a818d3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 >> 172.21.15.5:0/2277326193 conn(0x558a818b9be0 msgr2=0x558a818ca810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 shutdown_connections 2023-07-21T19:16:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 1 -- 172.21.15.5:0/2277326193 wait complete. 2023-07-21T19:16:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.274+0000 7f94e9d51b80 20 client.27022 trim_cache size 0 max 16384 2023-07-21T19:16:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:16:42.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:16:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:16:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:16:42.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 msgr2=0x7f94b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 0x7f94b80a3770 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f94c00115a0 tx=0x7f94c0009260 comp rx=0 tx=0).stop 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 msgr2=0x7f94d01a82b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 0x7f94d01a82b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f94c800efc0 tx=0x7f94c800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 shutdown_connections 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f94d0104e50 0x7f94d01a82b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f94b80a12c0 0x7f94b80a3770 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f94d0067c10 0x7f94d0105930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 --2- 172.21.15.5:0/3095237394 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f94d00685b0 0x7f94d0105e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 >> 172.21.15.5:0/3095237394 conn(0x7f94d0074760 msgr2=0x7f94d010bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 shutdown_connections 2023-07-21T19:16:42.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:42.275+0000 7f94d63ff700 1 -- 172.21.15.5:0/3095237394 wait complete. 2023-07-21T19:16:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:34100000 of 50000000 tuples (68%) done (elapsed 519.91 s, remaining 242.42 s) 2023-07-21T19:16:43.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:43 smithi101 ceph-mon[128136]: pgmap v282: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 54 op/s 2023-07-21T19:16:43.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:43 smithi005 ceph-mon[126460]: pgmap v282: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 54 op/s 2023-07-21T19:16:43.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:43 smithi156 ceph-mon[128316]: pgmap v282: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 126 KiB/s rd, 26 MiB/s wr, 54 op/s 2023-07-21T19:16:45.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:45 smithi101 ceph-mon[128136]: pgmap v283: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 43 op/s 2023-07-21T19:16:45.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:45 smithi005 ceph-mon[126460]: pgmap v283: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 43 op/s 2023-07-21T19:16:45.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:45 smithi156 ceph-mon[128316]: pgmap v283: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 19 MiB/s wr, 43 op/s 2023-07-21T19:16:47.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:47 smithi101 ceph-mon[128136]: pgmap v284: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 19 MiB/s wr, 40 op/s 2023-07-21T19:16:47.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:47 smithi005 ceph-mon[126460]: pgmap v284: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 19 MiB/s wr, 40 op/s 2023-07-21T19:16:47.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:47 smithi156 ceph-mon[128316]: pgmap v284: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 19 MiB/s wr, 40 op/s 2023-07-21T19:16:48.386 INFO:teuthology.orchestra.run.smithi005.stderr:34200000 of 50000000 tuples (68%) done (elapsed 525.95 s, remaining 242.98 s) 2023-07-21T19:16:49.732 INFO:teuthology.orchestra.run.smithi005.stderr:34300000 of 50000000 tuples (68%) done (elapsed 527.30 s, remaining 241.36 s) 2023-07-21T19:16:49.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:49 smithi101 ceph-mon[128136]: pgmap v285: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:16:49.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:49 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:49.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:49 smithi005 ceph-mon[126460]: pgmap v285: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:16:49.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:49 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:49.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:49 smithi156 ceph-mon[128316]: pgmap v285: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:16:49.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:49 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:16:51.145 INFO:teuthology.orchestra.run.smithi005.stderr:34400000 of 50000000 tuples (68%) done (elapsed 528.71 s, remaining 239.77 s) 2023-07-21T19:16:51.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:51 smithi101 ceph-mon[128136]: pgmap v286: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 30 op/s 2023-07-21T19:16:51.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:51 smithi005 ceph-mon[126460]: pgmap v286: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 30 op/s 2023-07-21T19:16:51.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:51 smithi156 ceph-mon[128316]: pgmap v286: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 11 MiB/s wr, 30 op/s 2023-07-21T19:16:52.457 INFO:teuthology.orchestra.run.smithi005.stderr:34500000 of 50000000 tuples (69%) done (elapsed 530.03 s, remaining 238.13 s) 2023-07-21T19:16:53.808 INFO:teuthology.orchestra.run.smithi005.stderr:34600000 of 50000000 tuples (69%) done (elapsed 531.38 s, remaining 236.51 s) 2023-07-21T19:16:53.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:53 smithi101 ceph-mon[128136]: pgmap v287: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 58 op/s 2023-07-21T19:16:53.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:53 smithi005 ceph-mon[126460]: pgmap v287: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 58 op/s 2023-07-21T19:16:53.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:53 smithi156 ceph-mon[128316]: pgmap v287: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 19 MiB/s wr, 58 op/s 2023-07-21T19:16:55.120 INFO:teuthology.orchestra.run.smithi005.stderr:34700000 of 50000000 tuples (69%) done (elapsed 532.69 s, remaining 234.87 s) 2023-07-21T19:16:55.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 -- 172.21.15.5:0/4032938688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea40ffc20 msgr2=0x7fbea410c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 --2- 172.21.15.5:0/4032938688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea40ffc20 0x7fbea410c000 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbe94009990 tx=0x7fbe94009ca0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 -- 172.21.15.5:0/4032938688 shutdown_connections 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 --2- 172.21.15.5:0/4032938688 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea410c540 0x7fbea410e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 --2- 172.21.15.5:0/4032938688 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea40ff6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 --2- 172.21.15.5:0/4032938688 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea40ffc20 0x7fbea410c000 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 -- 172.21.15.5:0/4032938688 >> 172.21.15.5:0/4032938688 conn(0x7fbea4074770 msgr2=0x7fbea4074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 -- 172.21.15.5:0/4032938688 shutdown_connections 2023-07-21T19:16:55.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.440+0000 7fbeab39d700 1 -- 172.21.15.5:0/4032938688 wait complete. 2023-07-21T19:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.441+0000 7fbeab39d700 1 Processor -- start 2023-07-21T19:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.441+0000 7fbeab39d700 1 -- start start 2023-07-21T19:16:55.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.441+0000 7fbeab39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.441+0000 7fbeab39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea40ffc20 0x7fbea419def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.441+0000 7fbeab39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea410c540 0x7fbea4197a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeab39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4198090 con 0x7fbea410c540 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeab39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea4198200 con 0x7fbea40ff310 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeab39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbea41983c0 con 0x7fbea40ffc20 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58670/0 (socket says 172.21.15.5:58670) 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 -- 172.21.15.5:0/3638292589 learned_addr learned my addr 172.21.15.5:0/3638292589 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea410c540 0x7fbea4197a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea40ffc20 0x7fbea419def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 -- 172.21.15.5:0/3638292589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea40ffc20 msgr2=0x7fbea419def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea40ffc20 0x7fbea419def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 -- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea410c540 msgr2=0x7fbea4197a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea410c540 0x7fbea4197a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.442+0000 7fbeaa39b700 1 -- 172.21.15.5:0/3638292589 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbea0009520 con 0x7fbea40ff310 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbea000eb30 tx=0x7fbea000ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbe9b7fe700 1 -- 172.21.15.5:0/3638292589 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0009880 con 0x7fbea40ff310 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbe9b7fe700 1 -- 172.21.15.5:0/3638292589 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea00044b0 con 0x7fbea40ff310 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbe9b7fe700 1 -- 172.21.15.5:0/3638292589 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0003ce0 con 0x7fbea40ff310 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe940095f0 con 0x7fbea40ff310 2023-07-21T19:16:55.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.443+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea41a47c0 con 0x7fbea40ff310 2023-07-21T19:16:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.447+0000 7fbe9b7fe700 1 -- 172.21.15.5:0/3638292589 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbea0003e40 con 0x7fbea40ff310 2023-07-21T19:16:55.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.448+0000 7fbebff51b80 1 Processor -- start 2023-07-21T19:16:55.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 -- start start 2023-07-21T19:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d87f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7c87c50 con 0x55e0e7bde6b0 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7cda9e0 con 0x55e0e7d888c0 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbebff51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7cd3e70 con 0x55e0e7d88470 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.449+0000 7fbe9b7fe700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 0x7fbe900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbe9b7fe700 1 -- 172.21.15.5:0/3638292589 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbea0014070 con 0x7fbea40ff310 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d87f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58678/0 (socket says 172.21.15.5:58678) 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2998098279 learned_addr learned my addr 172.21.15.5:0/2998098279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d87f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44326/0 (socket says 172.21.15.5:44326) 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 0x7fbe900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2998098279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 msgr2=0x55e0e7d821a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d821a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2998098279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 msgr2=0x55e0e7d87f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d87f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2998098279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e7b01d80 con 0x55e0e7d888c0 2023-07-21T19:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.450+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe94005ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ee6aa0eefb3098c7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.451+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 0x7fbe900a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009550 tx=0x7fbe9c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.451+0000 7fbe98bfe700 1 -- 172.21.15.5:0/2998098279 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94026070 con 0x55e0e7d888c0 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.451+0000 7fbe98bfe700 1 -- 172.21.15.5:0/2998098279 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe9401e040 con 0x55e0e7d888c0 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.451+0000 7fbe98bfe700 1 -- 172.21.15.5:0/2998098279 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9401d070 con 0x55e0e7d888c0 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 -- 172.21.15.5:0/2998098279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 msgr2=0x55e0e7d82700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbe94000c00 tx=0x7fbe94005ec0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 -- 172.21.15.5:0/2998098279 shutdown_connections 2023-07-21T19:16:55.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d82700 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d87f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2998098279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d821a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 -- 172.21.15.5:0/2998098279 >> 172.21.15.5:0/2998098279 conn(0x55e0e7d77d40 msgr2=0x55e0e7d7d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 -- 172.21.15.5:0/2998098279 shutdown_connections 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.452+0000 7fbebff51b80 1 -- 172.21.15.5:0/2998098279 wait complete. 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.453+0000 7fbebff51b80 1 -- start start 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.453+0000 7fbebff51b80 1 Processor -- start 2023-07-21T19:16:55.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d918e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d91e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7c73a60 con 0x55e0e7d88470 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7c73760 con 0x55e0e7d888c0 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7b578c0 con 0x55e0e7bde6b0 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbebff51b80 10 client.0 _ll_register_callbacks cb 0x55e0e7c71700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:55.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbea9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d91e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58684/0 (socket says 172.21.15.5:58684) 2023-07-21T19:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbeaab9c700 1 -- 172.21.15.5:0/2136117541 learned_addr learned my addr 172.21.15.5:0/2136117541 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:55.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.454+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d918e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 -- 172.21.15.5:0/2136117541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 msgr2=0x55e0e7d918e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d918e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 -- 172.21.15.5:0/2136117541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 msgr2=0x55e0e7d91e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d91e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 -- 172.21.15.5:0/2136117541 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea410f210 con 0x55e0e7d888c0 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d91e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:55.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c003fb0 tx=0x7fbe9c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f5790e43e75e5e3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.455+0000 7fbe8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c022070 con 0x55e0e7d888c0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbe9c01ae60 con 0x55e0e7d888c0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e0e7c2d490 con 0x55e0e7d888c0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe9c00e070 con 0x55e0e7d888c0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbebff51b80 10 client.27092 fetch_fsmap learned FSMap version 38 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x55e0e7aeac00 con 0x55e0e7d888c0 2023-07-21T19:16:55.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.456+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e0e7c2d490 con 0x55e0e7d888c0 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbe9c01b4e0 con 0x55e0e7d888c0 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 10 client.27092 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 10 client.27092 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 20 client.27092 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7d9bd50 0x55e0e7d9e160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 4 client.27092 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:55.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e0e7ccce50 con 0x55e0e7d9bd50 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.457+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7d9bd50 0x55e0e7d9e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.458+0000 7fbe8dffb700 10 client.27092 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.458+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7d9bd50 0x55e0e7d9e160 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8342402ed800ab1f server_cookie=aef3b54ac83a8954 in_seq=0 out_seq=0 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2136117541 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e0e7ccce50 con 0x55e0e7d9bd50 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbe8dffb700 10 client.27092 handle_command_reply: tid=0 2023-07-21T19:16:55.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 client.27092 shutdown 2023-07-21T19:16:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 msgr2=0x55e0e7d8bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c003fb0 tx=0x7fbe9c004ad0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 shutdown_connections 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d8bba0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d918e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7d9bd50 0x55e0e7d9e160 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2136117541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d91e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbe8dffb700 0 client.27092 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.459+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 >> 172.21.15.5:0/2136117541 conn(0x55e0e7d77d40 msgr2=0x55e0e7d7c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.460+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 shutdown_connections 2023-07-21T19:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.460+0000 7fbebff51b80 1 -- 172.21.15.5:0/2136117541 wait complete. 2023-07-21T19:16:55.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.460+0000 7fbebff51b80 20 client.27092 trim_cache size 0 max 16384 2023-07-21T19:16:55.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.477+0000 7fbebff51b80 1 Processor -- start 2023-07-21T19:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.477+0000 7fbebff51b80 1 -- start start 2023-07-21T19:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.477+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d821a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.477+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d7cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.477+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbebff51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d993d0 con 0x55e0e7d888c0 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbebff51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d8d140 con 0x55e0e7bde6b0 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbebff51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d7d8c0 con 0x55e0e7d88470 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d821a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44336/0 (socket says 172.21.15.5:44336) 2023-07-21T19:16:55.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaab9c700 1 -- 172.21.15.5:0/2749069347 learned_addr learned my addr 172.21.15.5:0/2749069347 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:55.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d7cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 -- 172.21.15.5:0/2749069347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 msgr2=0x55e0e7d7cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d7cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 -- 172.21.15.5:0/2749069347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 msgr2=0x55e0e7d7d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaa39b700 1 -- 172.21.15.5:0/2749069347 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e0e7b281f0 con 0x55e0e7bde6b0 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.478+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.479+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d821a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ca930 tx=0x7fbea0003740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8c56408bb3f86651 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.479+0000 7fbe993ff700 1 -- 172.21.15.5:0/2749069347 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea000cbd0 con 0x55e0e7bde6b0 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.479+0000 7fbe993ff700 1 -- 172.21.15.5:0/2749069347 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbea009a2b0 con 0x55e0e7bde6b0 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.479+0000 7fbe993ff700 1 -- 172.21.15.5:0/2749069347 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea0004350 con 0x55e0e7bde6b0 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- 172.21.15.5:0/2749069347 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 msgr2=0x55e0e7d821a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d821a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbea00ca930 tx=0x7fbea0003740 comp rx=0 tx=0).stop 2023-07-21T19:16:55.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- 172.21.15.5:0/2749069347 shutdown_connections 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d888c0 0x55e0e7d7d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d821a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2749069347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7d88470 0x55e0e7d7cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- 172.21.15.5:0/2749069347 >> 172.21.15.5:0/2749069347 conn(0x55e0e7d77d40 msgr2=0x55e0e7da2ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- 172.21.15.5:0/2749069347 shutdown_connections 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- 172.21.15.5:0/2749069347 wait complete. 2023-07-21T19:16:55.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.480+0000 7fbebff51b80 1 -- start start 2023-07-21T19:16:55.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 Processor -- start 2023-07-21T19:16:55.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d98040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d94b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d95090 con 0x55e0e7d88470 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d95200 con 0x55e0e7d888c0 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0e7d953c0 con 0x55e0e7bde6b0 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.481+0000 7fbebff51b80 10 client.0 _ll_register_callbacks cb 0x7fbe9c0083f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:55.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44340/0 (socket says 172.21.15.5:44340) 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2930812803 learned_addr learned my addr 172.21.15.5:0/2930812803 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d94b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d98040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2930812803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 msgr2=0x55e0e7d98040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d98040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 msgr2=0x55e0e7d94b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d94b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 -- 172.21.15.5:0/2930812803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e0e7aeac00 con 0x55e0e7d88470 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbeaab9c700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d94b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbea9b9a700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbe94006010 tx=0x7fbe9400bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a26521d4b18e38b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.482+0000 7fbe8dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94026070 con 0x55e0e7d88470 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe9401e040 con 0x55e0e7d88470 2023-07-21T19:16:55.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbe94004910 con 0x55e0e7d88470 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe9401d040 con 0x55e0e7d88470 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbebff51b80 10 client.17418 fetch_fsmap learned FSMap version 38 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7fbea41a47c0 con 0x55e0e7d88470 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.483+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe9401e040 con 0x55e0e7d88470 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbe9400fb00 con 0x55e0e7d88470 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 10 client.17418 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:16:55.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 10 client.17418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 20 client.17418 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7df0b50 0x55e0e7df2f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 4 client.17418 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e0e7c8eff0 con 0x55e0e7df0b50 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.484+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7df0b50 0x55e0e7df2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.485+0000 7fbeaa39b700 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7df0b50 0x55e0e7df2f60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c7097bdd6337c57 server_cookie=a10989fe6e64a3a in_seq=0 out_seq=0 2023-07-21T19:16:55.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.485+0000 7fbe8dffb700 10 client.17418 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.499+0000 7fbe8dffb700 1 -- 172.21.15.5:0/2930812803 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e0e7c8eff0 con 0x55e0e7df0b50 2023-07-21T19:16:55.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.499+0000 7fbe8dffb700 10 client.17418 handle_command_reply: tid=0 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 client.17418 shutdown 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 msgr2=0x55e0e7d94610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbe94006010 tx=0x7fbe9400bf70 comp rx=0 tx=0).stop 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 shutdown_connections 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0e7d888c0 0x55e0e7d94b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e0e7bde6b0 0x55e0e7d98040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0e7df0b50 0x55e0e7df2f60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 --2- 172.21.15.5:0/2930812803 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0e7d88470 0x55e0e7d94610 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbe8dffb700 0 client.17418 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 >> 172.21.15.5:0/2930812803 conn(0x55e0e7d77d40 msgr2=0x55e0e7d88d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 shutdown_connections 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 1 -- 172.21.15.5:0/2930812803 wait complete. 2023-07-21T19:16:55.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.500+0000 7fbebff51b80 20 client.17418 trim_cache size 0 max 16384 2023-07-21T19:16:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:16:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:16:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:16:55.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:16:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 msgr2=0x7fbe900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 0x7fbe900a3770 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c009550 tx=0x7fbe9c00b040 comp rx=0 tx=0).stop 2023-07-21T19:16:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 msgr2=0x7fbea419d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbea000eb30 tx=0x7fbea000ee40 comp rx=0 tx=0).stop 2023-07-21T19:16:55.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 shutdown_connections 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe900a12c0 0x7fbe900a3770 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbea410c540 0x7fbea4197a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbea40ff310 0x7fbea419d9b0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 --2- 172.21.15.5:0/3638292589 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbea40ffc20 0x7fbea419def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 >> 172.21.15.5:0/3638292589 conn(0x7fbea4074770 msgr2=0x7fbea40fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 shutdown_connections 2023-07-21T19:16:55.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:16:55.501+0000 7fbeab39d700 1 -- 172.21.15.5:0/3638292589 wait complete. 2023-07-21T19:16:55.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:55 smithi101 ceph-mon[128136]: pgmap v288: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-07-21T19:16:55.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:55 smithi005 ceph-mon[126460]: pgmap v288: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-07-21T19:16:55.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:55 smithi156 ceph-mon[128316]: pgmap v288: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 12 MiB/s wr, 47 op/s 2023-07-21T19:16:56.556 INFO:teuthology.orchestra.run.smithi005.stderr:34800000 of 50000000 tuples (69%) done (elapsed 534.12 s, remaining 233.30 s) 2023-07-21T19:16:56.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:56 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:16:56.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:56 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:16:56.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:56 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:16:57.480 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:16:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 -- 172.21.15.5:0/4183310915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4102910 msgr2=0x7f54d41068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/4183310915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4102910 0x7f54d41068d0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f54c40099c0 tx=0x7f54c4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 -- 172.21.15.5:0/4183310915 shutdown_connections 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/4183310915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4106fb0 0x7f54d410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/4183310915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4101f70 0x7f54d4102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/4183310915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4102910 0x7f54d41068d0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 -- 172.21.15.5:0/4183310915 >> 172.21.15.5:0/4183310915 conn(0x7f54d40fd7e0 msgr2=0x7f54d40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 -- 172.21.15.5:0/4183310915 shutdown_connections 2023-07-21T19:16:57.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.737+0000 7f54dc5ff700 1 -- 172.21.15.5:0/4183310915 wait complete. 2023-07-21T19:16:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.738+0000 7f54dc5ff700 1 Processor -- start 2023-07-21T19:16:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 -- start start 2023-07-21T19:16:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4102910 0x7f54d41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4106fb0 0x7f54d419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d419cd50 con 0x7f54d4102910 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d419cec0 con 0x7f54d4101f70 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d419d080 con 0x7f54d4106fb0 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54da39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54d9b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4102910 0x7f54d41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54da39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45026/0 (socket says 172.21.15.5:45026) 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54da39b700 1 -- 172.21.15.5:0/370942325 learned_addr learned my addr 172.21.15.5:0/370942325 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.739+0000 7f54dab9c700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4106fb0 0x7f54d419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 -- 172.21.15.5:0/370942325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4106fb0 msgr2=0x7f54d419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4106fb0 0x7f54d419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 -- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4102910 msgr2=0x7f54d41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4102910 0x7f54d41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 -- 172.21.15.5:0/370942325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c40095f0 con 0x7f54d4101f70 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54da39b700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9a0 tx=0x7f54d000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54cb7fe700 1 -- 172.21.15.5:0/370942325 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d0009790 con 0x7f54d4101f70 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54cb7fe700 1 -- 172.21.15.5:0/370942325 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54d0010430 con 0x7f54d4101f70 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d419d360 con 0x7f54d4101f70 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.740+0000 7f54cb7fe700 1 -- 172.21.15.5:0/370942325 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d000cea0 con 0x7f54d4101f70 2023-07-21T19:16:57.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.741+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d41a4ba0 con 0x7f54d4101f70 2023-07-21T19:16:57.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.744+0000 7f54cb7fe700 1 -- 172.21.15.5:0/370942325 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f54d00105a0 con 0x7f54d4101f70 2023-07-21T19:16:57.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.746+0000 7f54eff05b80 1 Processor -- start 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.746+0000 7f54eff05b80 1 -- start start 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.746+0000 7f54cb7fe700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 0x7f54c00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.746+0000 7f54cb7fe700 1 -- 172.21.15.5:0/370942325 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f54d0014070 con 0x7f54d4101f70 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.746+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 0x56475c641590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c641ad0 0x56475c643eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54dab9c700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 0x7f54c00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54eff05b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c53ccb0 con 0x56475c6362d0 2023-07-21T19:16:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54eff05b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c58fa40 con 0x56475c636be0 2023-07-21T19:16:57.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54eff05b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c588ed0 con 0x56475c641ad0 2023-07-21T19:16:57.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53480/0 (socket says 172.21.15.5:53480) 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54da39b700 1 -- 172.21.15.5:0/8808494 learned_addr learned my addr 172.21.15.5:0/8808494 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 0x56475c641590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54dab9c700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c641ad0 0x56475c643eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 -- 172.21.15.5:0/8808494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c641ad0 msgr2=0x56475c643eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c641ad0 0x56475c643eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 -- 172.21.15.5:0/8808494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 msgr2=0x56475c6366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54d9b9a700 1 -- 172.21.15.5:0/8808494 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56475c3b6de0 con 0x56475c636be0 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.747+0000 7f54da39b700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.748+0000 7f54dab9c700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 0x7f54c00a37f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00f540 tx=0x7f54cc009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.748+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 0x56475c641590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009fd0 tx=0x7f54c4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e9345abb33b95ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.748+0000 7f54c8bfe700 1 -- 172.21.15.5:0/8808494 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4005270 con 0x56475c636be0 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.748+0000 7f54c8bfe700 1 -- 172.21.15.5:0/8808494 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54c401e040 con 0x56475c636be0 2023-07-21T19:16:57.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.748+0000 7f54c8bfe700 1 -- 172.21.15.5:0/8808494 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c40039c0 con 0x56475c636be0 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 -- 172.21.15.5:0/8808494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 msgr2=0x56475c641590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 0x56475c641590 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f54c4009fd0 tx=0x7f54c4005c00 comp rx=0 tx=0).stop 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 -- 172.21.15.5:0/8808494 shutdown_connections 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c641ad0 0x56475c643eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c6366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 --2- 172.21.15.5:0/8808494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c636be0 0x56475c641590 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 -- 172.21.15.5:0/8808494 >> 172.21.15.5:0/8808494 conn(0x56475c62cda0 msgr2=0x56475c6326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 -- 172.21.15.5:0/8808494 shutdown_connections 2023-07-21T19:16:57.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.749+0000 7f54eff05b80 1 -- 172.21.15.5:0/8808494 wait complete. 2023-07-21T19:16:57.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.750+0000 7f54eff05b80 1 -- start start 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.750+0000 7f54eff05b80 1 Processor -- start 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c63e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c528ac0 con 0x56475c6362d0 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c5287c0 con 0x56475c641ad0 2023-07-21T19:16:57.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c40c920 con 0x56475c636be0 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54eff05b80 10 client.0 _ll_register_callbacks cb 0x56475c526760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54d9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c63e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53486/0 (socket says 172.21.15.5:53486) 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45052/0 (socket says 172.21.15.5:45052) 2023-07-21T19:16:57.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54da39b700 1 -- 172.21.15.5:0/3035019501 learned_addr learned my addr 172.21.15.5:0/3035019501 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 -- 172.21.15.5:0/3035019501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 msgr2=0x56475c63e7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c63e7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 -- 172.21.15.5:0/3035019501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 msgr2=0x56475c641500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54dab9c700 1 -- 172.21.15.5:0/3035019501 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d41a4ba0 con 0x56475c641ad0 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.751+0000 7f54da39b700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54dab9c700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f54cc019e30 tx=0x7f54cc004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10f730fce585f330 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc00e040 con 0x56475c641ad0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56475c4e24f0 con 0x56475c641ad0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f54cc017bb0 con 0x56475c641ad0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54cc017d70 con 0x56475c641ad0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54eff05b80 10 client.27110 fetch_fsmap learned FSMap version 38 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.752+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f54d410f3c0 con 0x56475c641ad0 2023-07-21T19:16:57.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56475c4e24f0 con 0x56475c641ad0 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54cc004650 con 0x56475c641ad0 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 10 client.27110 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 10 client.27110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 20 client.27110 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c6513a0 0x56475c6537b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 4 client.27110 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:16:57.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.753+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56475c581eb0 con 0x56475c6513a0 2023-07-21T19:16:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.754+0000 7f54da39b700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c6513a0 0x56475c6537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.754+0000 7f54da39b700 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c6513a0 0x56475c6537b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f832be24b5fbb66c server_cookie=80f399d26e233c84 in_seq=0 out_seq=0 2023-07-21T19:16:57.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.754+0000 7f54bdffb700 10 client.27110 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54bdffb700 1 -- 172.21.15.5:0/3035019501 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56475c581eb0 con 0x56475c6513a0 2023-07-21T19:16:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54bdffb700 10 client.27110 handle_command_reply: tid=0 2023-07-21T19:16:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 client.27110 shutdown 2023-07-21T19:16:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 msgr2=0x56475c63ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f54cc019e30 tx=0x7f54cc004b40 comp rx=0 tx=0).stop 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 shutdown_connections 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c641ad0 0x56475c63ef30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c6513a0 0x56475c6537b0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c6362d0 0x56475c641500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 --2- 172.21.15.5:0/3035019501 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c63e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 >> 172.21.15.5:0/3035019501 conn(0x56475c62cda0 msgr2=0x56475c632230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54bdffb700 0 client.27110 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:57.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 shutdown_connections 2023-07-21T19:16:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.755+0000 7f54eff05b80 1 -- 172.21.15.5:0/3035019501 wait complete. 2023-07-21T19:16:57.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.756+0000 7f54eff05b80 20 client.27110 trim_cache size 0 max 16384 2023-07-21T19:16:57.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.776+0000 7f54eff05b80 1 Processor -- start 2023-07-21T19:16:57.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 -- start start 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 0x56475c639af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c636be0 0x56475c63a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c637540 con 0x56475c636be0 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c629fa0 con 0x56475c62cda0 2023-07-21T19:16:57.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54eff05b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c63fb40 con 0x56475c6362d0 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54da39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54dab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c636be0 0x56475c63a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54da39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45056/0 (socket says 172.21.15.5:45056) 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.777+0000 7f54da39b700 1 -- 172.21.15.5:0/2931875142 learned_addr learned my addr 172.21.15.5:0/2931875142 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 0x56475c639af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 -- 172.21.15.5:0/2931875142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 msgr2=0x56475c639af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 0x56475c639af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 -- 172.21.15.5:0/2931875142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c636be0 msgr2=0x56475c63a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c636be0 0x56475c63a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 -- 172.21.15.5:0/2931875142 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56475c39fc60 con 0x56475c62cda0 2023-07-21T19:16:57.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 0x56475c639af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:16:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54da39b700 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f54d0099f10 tx=0x7f54d0011f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2eec266767eac3d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54c93ff700 1 -- 172.21.15.5:0/2931875142 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d00a23d0 con 0x56475c62cda0 2023-07-21T19:16:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54c93ff700 1 -- 172.21.15.5:0/2931875142 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54d0098320 con 0x56475c62cda0 2023-07-21T19:16:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.778+0000 7f54c93ff700 1 -- 172.21.15.5:0/2931875142 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54d00a1f40 con 0x56475c62cda0 2023-07-21T19:16:57.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 -- 172.21.15.5:0/2931875142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 msgr2=0x56475c6410a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f54d0099f10 tx=0x7f54d0011f90 comp rx=0 tx=0).stop 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 -- 172.21.15.5:0/2931875142 shutdown_connections 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c636be0 0x56475c63a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c62cda0 0x56475c6410a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 --2- 172.21.15.5:0/2931875142 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c6362d0 0x56475c639af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 -- 172.21.15.5:0/2931875142 >> 172.21.15.5:0/2931875142 conn(0x56475c63f0a0 msgr2=0x56475c64d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 -- 172.21.15.5:0/2931875142 shutdown_connections 2023-07-21T19:16:57.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.779+0000 7f54eff05b80 1 -- 172.21.15.5:0/2931875142 wait complete. 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 -- start start 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 Processor -- start 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 0x56475c646680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 0x56475c646bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c69e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c69e7e0 con 0x56475c62cda0 2023-07-21T19:16:57.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c69e920 con 0x56475c6362d0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56475c69eae0 con 0x56475c636be0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.780+0000 7f54eff05b80 10 client.0 _ll_register_callbacks cb 0x7f54cc019800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 0x56475c646680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 0x56475c646bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54da39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 0x56475c646680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53494/0 (socket says 172.21.15.5:53494) 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54da39b700 1 -- 172.21.15.5:0/513796056 learned_addr learned my addr 172.21.15.5:0/513796056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54dab9c700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c69e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 -- 172.21.15.5:0/513796056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 msgr2=0x56475c69e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c69e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 -- 172.21.15.5:0/513796056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 msgr2=0x56475c646680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 0x56475c646680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.781+0000 7f54d9b9a700 1 -- 172.21.15.5:0/513796056 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d419d360 con 0x56475c6362d0 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54d9b9a700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 0x56475c646bc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006010 tx=0x7f54c400bec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=828dd9d7537c4e9b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54bdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:16:57.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c4005450 con 0x56475c6362d0 2023-07-21T19:16:57.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(94..94 src has 1..94) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f54c40055b0 con 0x56475c6362d0 2023-07-21T19:16:57.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54c401e040 con 0x56475c6362d0 2023-07-21T19:16:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54c40059a0 con 0x56475c6362d0 2023-07-21T19:16:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54eff05b80 10 client.27116 fetch_fsmap learned FSMap version 38 2023-07-21T19:16:57.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x56475c3dd250 con 0x56475c6362d0 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.782+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54c4039080 con 0x56475c6362d0 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f54c400f760 con 0x56475c6362d0 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 10 client.27116 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 10 client.27116 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 20 client.27116 populate_metadata read hostname 'smithi005' 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c641ad0 0x56475c6a7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:16:57.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 4 client.27116 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56475c544050 con 0x56475c641ad0 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.783+0000 7f54da39b700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c641ad0 0x56475c6a7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54da39b700 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c641ad0 0x56475c6a7ef0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fec2a4d13eb4be5 server_cookie=62e715bf56eeb3e7 in_seq=0 out_seq=0 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54bdffb700 10 client.27116 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54bdffb700 1 -- 172.21.15.5:0/513796056 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x56475c544050 con 0x56475c641ad0 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54bdffb700 10 client.27116 handle_command_reply: tid=0 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 client.27116 shutdown 2023-07-21T19:16:57.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 msgr2=0x56475c646bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 0x56475c646bc0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f54c4006010 tx=0x7f54c400bec0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 shutdown_connections 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56475c636be0 0x56475c69e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56475c641ad0 0x56475c6a7ef0 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56475c62cda0 0x56475c646680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 --2- 172.21.15.5:0/513796056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56475c6362d0 0x56475c646bc0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 >> 172.21.15.5:0/513796056 conn(0x56475c63f0a0 msgr2=0x56475c639af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54bdffb700 0 client.27116 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:16:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 shutdown_connections 2023-07-21T19:16:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.784+0000 7f54eff05b80 1 -- 172.21.15.5:0/513796056 wait complete. 2023-07-21T19:16:57.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54eff05b80 20 client.27116 trim_cache size 0 max 16384 2023-07-21T19:16:57.790 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:16:57.790 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:16:57.791 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:16:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 msgr2=0x7f54c00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 0x7f54c00a37f0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f54cc00f540 tx=0x7f54cc009920 comp rx=0 tx=0).stop 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 msgr2=0x7f54d4197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9a0 tx=0x7f54d000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 shutdown_connections 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f54d4106fb0 0x7f54d419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f54d4101f70 0x7f54d4197d70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54c00a1340 0x7f54c00a37f0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 --2- 172.21.15.5:0/370942325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f54d4102910 0x7f54d41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:16:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 >> 172.21.15.5:0/370942325 conn(0x7f54d40fd7e0 msgr2=0x7f54d40fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:16:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 shutdown_connections 2023-07-21T19:16:57.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:16:57.785+0000 7f54dc5ff700 1 -- 172.21.15.5:0/370942325 wait complete. 2023-07-21T19:16:57.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:57 smithi005 ceph-mon[126460]: pgmap v289: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 45 op/s 2023-07-21T19:16:57.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:57 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:16:57.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:57 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:16:57.805 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:57 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:16:57.808 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:16:57.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:57 smithi101 ceph-mon[128136]: pgmap v289: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 45 op/s 2023-07-21T19:16:57.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:57 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:16:57.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:57 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:16:57.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:57 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:16:57.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:57 smithi156 ceph-mon[128316]: pgmap v289: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 12 MiB/s wr, 45 op/s 2023-07-21T19:16:57.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:57 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:16:57.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:57 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:16:57.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:57 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:16:57.922 INFO:teuthology.orchestra.run.smithi005.stderr:34900000 of 50000000 tuples (69%) done (elapsed 535.49 s, remaining 231.69 s) 2023-07-21T19:16:59.214 INFO:teuthology.orchestra.run.smithi005.stderr:35000000 of 50000000 tuples (70%) done (elapsed 536.78 s, remaining 230.05 s) 2023-07-21T19:16:59.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:16:59 smithi101 ceph-mon[128136]: pgmap v290: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 80 op/s 2023-07-21T19:16:59.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:16:59 smithi005 ceph-mon[126460]: pgmap v290: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 80 op/s 2023-07-21T19:16:59.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:16:59 smithi156 ceph-mon[128316]: pgmap v290: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 23 MiB/s wr, 80 op/s 2023-07-21T19:17:00.687 INFO:teuthology.orchestra.run.smithi005.stderr:35100000 of 50000000 tuples (70%) done (elapsed 538.26 s, remaining 228.49 s) 2023-07-21T19:17:01.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:01 smithi101 ceph-mon[128136]: pgmap v291: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 64 op/s 2023-07-21T19:17:01.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:01 smithi005 ceph-mon[126460]: pgmap v291: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 64 op/s 2023-07-21T19:17:01.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:01 smithi156 ceph-mon[128316]: pgmap v291: 97 pgs: 97 active+clean; 7.9 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 20 MiB/s wr, 64 op/s 2023-07-21T19:17:01.973 INFO:teuthology.orchestra.run.smithi005.stderr:35200000 of 50000000 tuples (70%) done (elapsed 539.54 s, remaining 226.85 s) 2023-07-21T19:17:03.317 INFO:teuthology.orchestra.run.smithi005.stderr:35300000 of 50000000 tuples (70%) done (elapsed 540.89 s, remaining 225.24 s) 2023-07-21T19:17:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:03 smithi101 ceph-mon[128136]: pgmap v292: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 91 op/s 2023-07-21T19:17:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:03 smithi005 ceph-mon[126460]: pgmap v292: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 91 op/s 2023-07-21T19:17:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:03 smithi156 ceph-mon[128316]: pgmap v292: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 29 MiB/s wr, 91 op/s 2023-07-21T19:17:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:35400000 of 50000000 tuples (70%) done (elapsed 542.28 s, remaining 223.65 s) 2023-07-21T19:17:05.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:05 smithi101 ceph-mon[128136]: pgmap v293: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:17:05.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:05 smithi005 ceph-mon[126460]: pgmap v293: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:17:05.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:05 smithi156 ceph-mon[128316]: pgmap v293: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 23 MiB/s wr, 70 op/s 2023-07-21T19:17:05.992 INFO:teuthology.orchestra.run.smithi005.stderr:35500000 of 50000000 tuples (71%) done (elapsed 543.56 s, remaining 222.02 s) 2023-07-21T19:17:07.608 INFO:teuthology.orchestra.run.smithi005.stderr:35600000 of 50000000 tuples (71%) done (elapsed 545.18 s, remaining 220.52 s) 2023-07-21T19:17:07.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:07 smithi101 ceph-mon[128136]: pgmap v294: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:17:07.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:07 smithi101 ceph-mon[128136]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T19:17:07.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:07 smithi005 ceph-mon[126460]: pgmap v294: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:17:07.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:07 smithi005 ceph-mon[126460]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T19:17:07.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:07 smithi156 ceph-mon[128316]: pgmap v294: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:17:07.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:07 smithi156 ceph-mon[128316]: osdmap e95: 12 total, 12 up, 12 in 2023-07-21T19:17:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 -- 172.21.15.5:0/446126684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 msgr2=0x7fe95410e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/446126684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95410e920 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b0a0 tx=0x7fe94c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 -- 172.21.15.5:0/446126684 shutdown_connections 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/446126684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95410e920 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/446126684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540679a0 0x7fe954067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/446126684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540682b0 0x7fe95410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 -- 172.21.15.5:0/446126684 >> 172.21.15.5:0/446126684 conn(0x7fe954074760 msgr2=0x7fe954074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 -- 172.21.15.5:0/446126684 shutdown_connections 2023-07-21T19:17:08.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.673+0000 7fe95c5ff700 1 -- 172.21.15.5:0/446126684 wait complete. 2023-07-21T19:17:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.674+0000 7fe95c5ff700 1 Processor -- start 2023-07-21T19:17:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.674+0000 7fe95c5ff700 1 -- start start 2023-07-21T19:17:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.674+0000 7fe95c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540679a0 0x7fe954197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.674+0000 7fe95c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 0x7fe954198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.674+0000 7fe95c5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95c5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95419d000 con 0x7fe95410c540 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95419d170 con 0x7fe9540682b0 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95419d330 con 0x7fe9540679a0 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 0x7fe954198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41742/0 (socket says 172.21.15.5:41742) 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95ab9c700 1 -- 172.21.15.5:0/2630983718 learned_addr learned my addr 172.21.15.5:0/2630983718 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95a39b700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540679a0 0x7fe954197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 -- 172.21.15.5:0/2630983718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540679a0 msgr2=0x7fe954197e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540679a0 0x7fe954197e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 -- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 msgr2=0x7fe95419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe959b9a700 1 -- 172.21.15.5:0/2630983718 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe94c009d00 con 0x7fe9540682b0 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.675+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 0x7fe954198350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9f0 tx=0x7fe94400ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe94b7fe700 1 -- 172.21.15.5:0/2630983718 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94400c9c0 con 0x7fe9540682b0 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95419d610 con 0x7fe9540682b0 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe94b7fe700 1 -- 172.21.15.5:0/2630983718 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe944010430 con 0x7fe9540682b0 2023-07-21T19:17:08.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe94b7fe700 1 -- 172.21.15.5:0/2630983718 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944018680 con 0x7fe9540682b0 2023-07-21T19:17:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.676+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9541a4e50 con 0x7fe9540682b0 2023-07-21T19:17:08.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.680+0000 7fe94b7fe700 1 -- 172.21.15.5:0/2630983718 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe9440105a0 con 0x7fe9540682b0 2023-07-21T19:17:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 Processor -- start 2023-07-21T19:17:08.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 -- start start 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c3dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c389d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032b3dc50 con 0x556032a946b0 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032b909e0 con 0x556032c3e2d0 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.681+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032b89e70 con 0x556032c38580 2023-07-21T19:17:08.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe94b7fe700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 0x7fe9400a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47450/0 (socket says 172.21.15.5:47450) 2023-07-21T19:17:08.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 -- 172.21.15.5:0/1253808268 learned_addr learned my addr 172.21.15.5:0/1253808268 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c3dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe94b7fe700 1 -- 172.21.15.5:0/2630983718 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe944014070 con 0x7fe9540682b0 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 0x7fe9400a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 -- 172.21.15.5:0/1253808268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 msgr2=0x556032c389d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe959b9a700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c389d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c389d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 -- 172.21.15.5:0/1253808268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 msgr2=0x556032c3dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c3dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95a39b700 1 -- 172.21.15.5:0/1253808268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5560329b7d80 con 0x556032c3e2d0 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.682+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.683+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d80 tx=0x7fe950011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8eecda418428f995 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.683+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 0x7fe9400a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b780 tx=0x7fe94c0099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.683+0000 7fe948bfe700 1 -- 172.21.15.5:0/1253808268 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950004230 con 0x556032c3e2d0 2023-07-21T19:17:08.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.683+0000 7fe948bfe700 1 -- 172.21.15.5:0/1253808268 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe950004390 con 0x556032c3e2d0 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.683+0000 7fe948bfe700 1 -- 172.21.15.5:0/1253808268 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe950019800 con 0x556032c3e2d0 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1253808268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 msgr2=0x556032c38040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe950009d80 tx=0x7fe950011720 comp rx=0 tx=0).stop 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1253808268 shutdown_connections 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c389d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c3dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1253808268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c38040 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1253808268 >> 172.21.15.5:0/1253808268 conn(0x556032c2dbe0 msgr2=0x556032c32f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1253808268 shutdown_connections 2023-07-21T19:17:08.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.684+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1253808268 wait complete. 2023-07-21T19:17:08.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 -- start start 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 Processor -- start 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c44620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c41f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032b29a60 con 0x556032c38580 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032b29760 con 0x556032c3e2d0 2023-07-21T19:17:08.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032a0d8c0 con 0x556032a946b0 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.685+0000 7fe96ff01b80 10 client.0 _ll_register_callbacks cb 0x556032b27700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41766/0 (socket says 172.21.15.5:41766) 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 -- 172.21.15.5:0/1196859752 learned_addr learned my addr 172.21.15.5:0/1196859752 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c41f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c44620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 -- 172.21.15.5:0/1196859752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 msgr2=0x556032c44620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c44620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 -- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 msgr2=0x556032c41f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c41f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 -- 172.21.15.5:0/1196859752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9500099e0 con 0x556032c38580 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c44620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe959b9a700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe944099e40 tx=0x7fe944099810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e4c45ec49f73b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.686+0000 7fe93dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944099eb0 con 0x556032c38580 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe94409dee0 con 0x556032c38580 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556032ae3490 con 0x556032c38580 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9440a2300 con 0x556032c38580 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe96ff01b80 10 client.17448 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5560329a0c00 con 0x556032c38580 2023-07-21T19:17:08.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9440c8080 con 0x556032c38580 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.687+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe9440c9f80 con 0x556032c38580 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 10 client.17448 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 10 client.17448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 20 client.17448 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032c51ae0 0x556032c53ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 4 client.17448 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556032b82e50 con 0x556032c51ae0 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.688+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032c51ae0 0x556032c53ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.689+0000 7fe93dffb700 10 client.17448 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.689+0000 7fe95a39b700 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032c51ae0 0x556032c53ef0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5cbd1bcf13e69e35 server_cookie=8d2f466800be73 in_seq=0 out_seq=0 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.689+0000 7fe93dffb700 1 -- 172.21.15.5:0/1196859752 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556032b82e50 con 0x556032c51ae0 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.689+0000 7fe93dffb700 10 client.17448 handle_command_reply: tid=0 2023-07-21T19:17:08.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 client.17448 shutdown 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 msgr2=0x556032c418a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe944099e40 tx=0x7fe944099810 comp rx=0 tx=0).stop 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 shutdown_connections 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c41f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c44620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032c51ae0 0x556032c53ef0 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/1196859752 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c38580 0x556032c418a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 >> 172.21.15.5:0/1196859752 conn(0x556032c2dbe0 msgr2=0x556032c31c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe93dffb700 0 client.17448 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 shutdown_connections 2023-07-21T19:17:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 1 -- 172.21.15.5:0/1196859752 wait complete. 2023-07-21T19:17:08.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.690+0000 7fe96ff01b80 20 client.17448 trim_cache size 0 max 16384 2023-07-21T19:17:08.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.707+0000 7fe96ff01b80 1 Processor -- start 2023-07-21T19:17:08.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 -- start start 2023-07-21T19:17:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c38040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c3e2d0 0x556032c44150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032c3da30 con 0x556032c3e2d0 2023-07-21T19:17:08.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032c2abb0 con 0x556032c38580 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032c4f220 con 0x556032a946b0 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe959b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe95ab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c3e2d0 0x556032c44150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe959b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47460/0 (socket says 172.21.15.5:47460) 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe959b9a700 1 -- 172.21.15.5:0/2048169099 learned_addr learned my addr 172.21.15.5:0/2048169099 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.708+0000 7fe95a39b700 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c38040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 -- 172.21.15.5:0/2048169099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 msgr2=0x556032c38040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c38040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 -- 172.21.15.5:0/2048169099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c3e2d0 msgr2=0x556032c44150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c3e2d0 0x556032c44150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 -- 172.21.15.5:0/2048169099 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556032c41410 con 0x556032c38580 2023-07-21T19:17:08.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe959b9a700 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe94409a3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc00167fe38dd5a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe9493ff700 1 -- 172.21.15.5:0/2048169099 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94409dee0 con 0x556032c38580 2023-07-21T19:17:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe9493ff700 1 -- 172.21.15.5:0/2048169099 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556032ae3490 con 0x556032c38580 2023-07-21T19:17:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.709+0000 7fe9493ff700 1 -- 172.21.15.5:0/2048169099 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe944095dc0 con 0x556032c38580 2023-07-21T19:17:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 -- 172.21.15.5:0/2048169099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 msgr2=0x556032c32c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe944000c00 tx=0x7fe94409a3f0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 -- 172.21.15.5:0/2048169099 shutdown_connections 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032c3e2d0 0x556032c44150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032a946b0 0x556032c38040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/2048169099 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c38580 0x556032c32c60 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 -- 172.21.15.5:0/2048169099 >> 172.21.15.5:0/2048169099 conn(0x556032c2dbe0 msgr2=0x556032c4de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 -- 172.21.15.5:0/2048169099 shutdown_connections 2023-07-21T19:17:08.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.710+0000 7fe96ff01b80 1 -- 172.21.15.5:0/2048169099 wait complete. 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.711+0000 7fe96ff01b80 1 -- start start 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.711+0000 7fe96ff01b80 1 Processor -- start 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c47190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c476d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032ca49e0 con 0x556032a946b0 2023-07-21T19:17:08.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032ca4b50 con 0x556032c3e2d0 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556032ca4d10 con 0x556032c38580 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe96ff01b80 10 client.0 _ll_register_callbacks cb 0x7fe944099c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41784/0 (socket says 172.21.15.5:41784) 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95a39b700 1 -- 172.21.15.5:0/818902944 learned_addr learned my addr 172.21.15.5:0/818902944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95ab9c700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c476d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe959b9a700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c47190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95a39b700 1 -- 172.21.15.5:0/818902944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 msgr2=0x556032c47190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.712+0000 7fe95a39b700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c47190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe95a39b700 1 -- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 msgr2=0x556032c476d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe95a39b700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c476d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe95a39b700 1 -- 172.21.15.5:0/818902944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5560329a0c00 con 0x556032a946b0 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe95a39b700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x556032c34b40 tx=0x7fe950004500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5757e4033949988 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:08.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe93dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:17:08.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95002b7f0 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(95..95 src has 1..95) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe95002a040 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.713+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe950004390 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe95002b950 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 10 client.17454 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x5560329de1f0 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe950004390 con 0x556032a946b0 2023-07-21T19:17:08.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe950031020 con 0x556032a946b0 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 10 client.17454 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 10 client.17454 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 20 client.17454 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032ca62c0 0x556032ca86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 4 client.17454 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.714+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556032b44ff0 con 0x556032ca62c0 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.715+0000 7fe959b9a700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032ca62c0 0x556032ca86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:08.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.715+0000 7fe93dffb700 10 client.17454 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:08.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.715+0000 7fe959b9a700 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032ca62c0 0x556032ca86d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b8dcaf1e5d7d40d server_cookie=cf409ec8f9f2262e in_seq=0 out_seq=0 2023-07-21T19:17:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe93dffb700 1 -- 172.21.15.5:0/818902944 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556032b44ff0 con 0x556032ca62c0 2023-07-21T19:17:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe93dffb700 10 client.17454 handle_command_reply: tid=0 2023-07-21T19:17:08.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 client.17454 shutdown 2023-07-21T19:17:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 msgr2=0x556032c46c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x556032c34b40 tx=0x7fe950004500 comp rx=0 tx=0).stop 2023-07-21T19:17:08.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 shutdown_connections 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556032c3e2d0 0x556032c476d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556032ca62c0 0x556032ca86d0 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556032a946b0 0x556032c46c50 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 --2- 172.21.15.5:0/818902944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556032c38580 0x556032c47190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 >> 172.21.15.5:0/818902944 conn(0x556032c2dbe0 msgr2=0x556032c3bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.729+0000 7fe93dffb700 0 client.17454 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.730+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 shutdown_connections 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.730+0000 7fe96ff01b80 1 -- 172.21.15.5:0/818902944 wait complete. 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.730+0000 7fe96ff01b80 20 client.17454 trim_cache size 0 max 16384 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:17:08.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:17:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:17:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 msgr2=0x7fe9400a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 0x7fe9400a37f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe94c00b780 tx=0x7fe94c0099e0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 msgr2=0x7fe954198350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 0x7fe954198350 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fe94400e9f0 tx=0x7fe94400ed00 comp rx=0 tx=0).stop 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 shutdown_connections 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9400a1340 0x7fe9400a37f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe95410c540 0x7fe95419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9540679a0 0x7fe954197e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 --2- 172.21.15.5:0/2630983718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9540682b0 0x7fe954198350 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:08.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 >> 172.21.15.5:0/2630983718 conn(0x7fe954074760 msgr2=0x7fe9540fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 shutdown_connections 2023-07-21T19:17:08.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:08.731+0000 7fe95c5ff700 1 -- 172.21.15.5:0/2630983718 wait complete. 2023-07-21T19:17:08.930 INFO:teuthology.orchestra.run.smithi005.stderr:35700000 of 50000000 tuples (71%) done (elapsed 546.50 s, remaining 218.91 s) 2023-07-21T19:17:09.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:09 smithi101 ceph-mon[128136]: pgmap v296: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:09 smithi005 ceph-mon[126460]: pgmap v296: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:09.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:09 smithi156 ceph-mon[128316]: pgmap v296: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:10.339 INFO:teuthology.orchestra.run.smithi005.stderr:35800000 of 50000000 tuples (71%) done (elapsed 547.91 s, remaining 217.33 s) 2023-07-21T19:17:11.665 INFO:teuthology.orchestra.run.smithi005.stderr:35900000 of 50000000 tuples (71%) done (elapsed 549.23 s, remaining 215.72 s) 2023-07-21T19:17:11.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:11 smithi101 ceph-mon[128136]: pgmap v297: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:11.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:11 smithi005 ceph-mon[126460]: pgmap v297: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:11.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:11 smithi156 ceph-mon[128316]: pgmap v297: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 26 MiB/s wr, 65 op/s 2023-07-21T19:17:12.982 INFO:teuthology.orchestra.run.smithi005.stderr:36000000 of 50000000 tuples (72%) done (elapsed 550.55 s, remaining 214.10 s) 2023-07-21T19:17:13.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:13 smithi101 ceph-mon[128136]: pgmap v298: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:17:13.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:13 smithi101 ceph-mon[128136]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T19:17:13.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:13 smithi005 ceph-mon[126460]: pgmap v298: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:17:13.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:13 smithi005 ceph-mon[126460]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T19:17:13.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:13 smithi156 ceph-mon[128316]: pgmap v298: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 26 MiB/s wr, 69 op/s 2023-07-21T19:17:13.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:13 smithi156 ceph-mon[128316]: osdmap e96: 12 total, 12 up, 12 in 2023-07-21T19:17:14.379 INFO:teuthology.orchestra.run.smithi005.stderr:36100000 of 50000000 tuples (72%) done (elapsed 551.95 s, remaining 212.52 s) 2023-07-21T19:17:15.664 INFO:teuthology.orchestra.run.smithi005.stderr:36200000 of 50000000 tuples (72%) done (elapsed 553.23 s, remaining 210.90 s) 2023-07-21T19:17:15.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:15 smithi101 ceph-mon[128136]: pgmap v300: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 32 MiB/s wr, 85 op/s 2023-07-21T19:17:15.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:15 smithi005 ceph-mon[126460]: pgmap v300: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 32 MiB/s wr, 85 op/s 2023-07-21T19:17:15.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:15 smithi156 ceph-mon[128316]: pgmap v300: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 32 MiB/s wr, 85 op/s 2023-07-21T19:17:16.955 INFO:teuthology.orchestra.run.smithi005.stderr:36300000 of 50000000 tuples (72%) done (elapsed 554.52 s, remaining 209.28 s) 2023-07-21T19:17:17.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:17 smithi101 ceph-mon[128136]: pgmap v301: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 70 op/s 2023-07-21T19:17:17.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:17 smithi005 ceph-mon[126460]: pgmap v301: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 70 op/s 2023-07-21T19:17:17.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:17 smithi156 ceph-mon[128316]: pgmap v301: 97 pgs: 97 active+clean; 7.7 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 26 MiB/s wr, 70 op/s 2023-07-21T19:17:18.368 INFO:teuthology.orchestra.run.smithi005.stderr:36400000 of 50000000 tuples (72%) done (elapsed 555.94 s, remaining 207.71 s) 2023-07-21T19:17:19.657 INFO:teuthology.orchestra.run.smithi005.stderr:36500000 of 50000000 tuples (73%) done (elapsed 557.23 s, remaining 206.10 s) 2023-07-21T19:17:19.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:19 smithi101 ceph-mon[128136]: pgmap v302: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:19.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:19 smithi005 ceph-mon[126460]: pgmap v302: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:19.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:19 smithi156 ceph-mon[128316]: pgmap v302: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:21.005 INFO:teuthology.orchestra.run.smithi005.stderr:36600000 of 50000000 tuples (73%) done (elapsed 558.57 s, remaining 204.51 s) 2023-07-21T19:17:21.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:21 smithi101 ceph-mon[128136]: pgmap v303: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:21.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 -- 172.21.15.5:0/2285668012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03141004e0 msgr2=0x7f031410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 --2- 172.21.15.5:0/2285668012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03141004e0 0x7f031410c920 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f030c009f90 tx=0x7f030c00b330 comp rx=0 tx=0).stop 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 -- 172.21.15.5:0/2285668012 shutdown_connections 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 --2- 172.21.15.5:0/2285668012 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03141004e0 0x7f031410c920 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 --2- 172.21.15.5:0/2285668012 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0314105b60 0x7f0314105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 --2- 172.21.15.5:0/2285668012 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03140ffb30 0x7f03140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 -- 172.21.15.5:0/2285668012 >> 172.21.15.5:0/2285668012 conn(0x7f03140fb5f0 msgr2=0x7f03140fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 -- 172.21.15.5:0/2285668012 shutdown_connections 2023-07-21T19:17:21.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.905+0000 7f0319fff700 1 -- 172.21.15.5:0/2285668012 wait complete. 2023-07-21T19:17:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.907+0000 7f0319fff700 1 Processor -- start 2023-07-21T19:17:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.907+0000 7f0319fff700 1 -- start start 2023-07-21T19:17:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.907+0000 7f0319fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0319fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 0x7f0314198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0319fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0314105b60 0x7f031419c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0319fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031419d000 con 0x7f0314105b60 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0319fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031419d170 con 0x7f03140ffb30 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0319fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f031419d330 con 0x7f03141004e0 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43496/0 (socket says 172.21.15.5:43496) 2023-07-21T19:17:21.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 -- 172.21.15.5:0/865642684 learned_addr learned my addr 172.21.15.5:0/865642684 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0313fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0314105b60 0x7f031419c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0312ffd700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 0x7f0314198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 -- 172.21.15.5:0/865642684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 msgr2=0x7f0314198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 0x7f0314198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 -- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0314105b60 msgr2=0x7f031419c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0314105b60 0x7f031419c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f03137fe700 1 -- 172.21.15.5:0/865642684 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03080095f0 con 0x7f03140ffb30 2023-07-21T19:17:21.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.908+0000 7f0312ffd700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 0x7f0314198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:21.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:21 smithi005 ceph-mon[126460]: pgmap v303: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f03137fe700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9a0 tx=0x7f030400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f0310ff9700 1 -- 172.21.15.5:0/865642684 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304009790 con 0x7f03140ffb30 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f0310ff9700 1 -- 172.21.15.5:0/865642684 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f030400cbd0 con 0x7f03140ffb30 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f0310ff9700 1 -- 172.21.15.5:0/865642684 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0304018680 con 0x7f03140ffb30 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f030c009c40 con 0x7f03140ffb30 2023-07-21T19:17:21.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.909+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03141a4c70 con 0x7f03140ffb30 2023-07-21T19:17:21.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.913+0000 7f0310ff9700 1 -- 172.21.15.5:0/865642684 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f030400cd40 con 0x7f03140ffb30 2023-07-21T19:17:21.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.914+0000 7f032d9dbb80 1 Processor -- start 2023-07-21T19:17:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f0310ff9700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 0x7f02fc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 -- start start 2023-07-21T19:17:21.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f0310ff9700 1 -- 172.21.15.5:0/865642684 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0304014070 con 0x7f03140ffb30 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f0313fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 0x7f02fc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 0x560a8641ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86415040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a864159d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8631ac50 con 0x560a862716b0 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8636d9e0 con 0x560a8641b2d0 2023-07-21T19:17:21.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a86366e70 con 0x560a86415580 2023-07-21T19:17:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.915+0000 7f03137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 0x560a8641ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f03137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 0x560a8641ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38944/0 (socket says 172.21.15.5:38944) 2023-07-21T19:17:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86415040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f03137fe700 1 -- 172.21.15.5:0/373557713 learned_addr learned my addr 172.21.15.5:0/373557713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:21.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0313fff700 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a864159d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 -- 172.21.15.5:0/373557713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 msgr2=0x560a864159d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a864159d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 -- 172.21.15.5:0/373557713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 msgr2=0x560a8641ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 0x560a8641ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 -- 172.21.15.5:0/373557713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a86194d80 con 0x560a8641b2d0 2023-07-21T19:17:21.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0313fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 0x7f02fc0a37f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f030c009790 tx=0x7f030c0074c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f0312ffd700 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86415040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0308005fd0 tx=0x7f03080116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b30e2333fe1c8df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.916+0000 7f02f9ffb700 1 -- 172.21.15.5:0/373557713 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308011ee0 con 0x560a8641b2d0 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f02f9ffb700 1 -- 172.21.15.5:0/373557713 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0308004cc0 con 0x560a8641b2d0 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f02f9ffb700 1 -- 172.21.15.5:0/373557713 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03080124b0 con 0x560a8641b2d0 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 -- 172.21.15.5:0/373557713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 msgr2=0x560a86415040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86415040 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0308005fd0 tx=0x7f03080116c0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 -- 172.21.15.5:0/373557713 shutdown_connections 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a864159d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a862716b0 0x560a8641ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.917+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/373557713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86415040 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.918+0000 7f032d9dbb80 1 -- 172.21.15.5:0/373557713 >> 172.21.15.5:0/373557713 conn(0x560a8640abe0 msgr2=0x560a8640ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.918+0000 7f032d9dbb80 1 -- 172.21.15.5:0/373557713 shutdown_connections 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.918+0000 7f032d9dbb80 1 -- 172.21.15.5:0/373557713 wait complete. 2023-07-21T19:17:21.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.918+0000 7f032d9dbb80 1 -- start start 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.918+0000 7f032d9dbb80 1 Processor -- start 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a8641e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 0x560a8641ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a86306a60 con 0x560a86415580 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a86306760 con 0x560a862716b0 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a861ea8c0 con 0x560a8641b2d0 2023-07-21T19:17:21.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f032d9dbb80 10 client.0 _ll_register_callbacks cb 0x560a86304700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:21.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:21 smithi156 ceph-mon[128316]: pgmap v303: 97 pgs: 97 active+clean; 7.8 GiB data, 3.4 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 26 MiB/s wr, 60 op/s 2023-07-21T19:17:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f0312ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a8641e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43522/0 (socket says 172.21.15.5:43522) 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.919+0000 7f03137fe700 1 -- 172.21.15.5:0/1146222690 learned_addr learned my addr 172.21.15.5:0/1146222690 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f0313fff700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 0x560a8641ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 -- 172.21.15.5:0/1146222690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 msgr2=0x560a8641ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 0x560a8641ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 -- 172.21.15.5:0/1146222690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 msgr2=0x560a8641e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a8641e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 -- 172.21.15.5:0/1146222690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0308009b90 con 0x560a862716b0 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f0312ffd700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a8641e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f0313fff700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 0x560a8641ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f03137fe700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f030400e970 tx=0x7f0304099ee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4dec25b0dd8dceb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f02f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040a2430 con 0x560a862716b0 2023-07-21T19:17:21.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f03040a1fa0 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a862c0490 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03040a2160 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f032d9dbb80 10 client.27146 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.920+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x560a8617dc00 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03040c8020 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0304095fd0 con 0x560a862716b0 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 10 client.27146 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:21.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 10 client.27146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 20 client.27146 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8642ec50 0x560a86431060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 4 client.27146 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a8635fe50 con 0x560a8642ec50 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.921+0000 7f0312ffd700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8642ec50 0x560a86431060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.922+0000 7f0312ffd700 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8642ec50 0x560a86431060 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfaf90477f97c11a server_cookie=bc2ebad24f1e8691 in_seq=0 out_seq=0 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.922+0000 7f02f2ffd700 10 client.27146 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:21.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f02f2ffd700 1 -- 172.21.15.5:0/1146222690 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a8635fe50 con 0x560a8642ec50 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f02f2ffd700 10 client.27146 handle_command_reply: tid=0 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 client.27146 shutdown 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 msgr2=0x560a86421630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f030400e970 tx=0x7f0304099ee0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 shutdown_connections 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a8641b2d0 0x560a8641ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a86421630 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8642ec50 0x560a86431060 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/1146222690 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a8641e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f02f2ffd700 0 client.27146 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 >> 172.21.15.5:0/1146222690 conn(0x560a8640abe0 msgr2=0x560a8640ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 shutdown_connections 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.923+0000 7f032d9dbb80 1 -- 172.21.15.5:0/1146222690 wait complete. 2023-07-21T19:17:21.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.924+0000 7f032d9dbb80 20 client.27146 trim_cache size 0 max 16384 2023-07-21T19:17:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.941+0000 7f032d9dbb80 1 Processor -- start 2023-07-21T19:17:21.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 -- start start 2023-07-21T19:17:21.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a862716b0 0x560a86415040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86421160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a86410a00 con 0x560a86415580 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8642c2a0 con 0x560a8641b2d0 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8640c5f0 con 0x560a862716b0 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f0312ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f0312ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38968/0 (socket says 172.21.15.5:38968) 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f0312ffd700 1 -- 172.21.15.5:0/2751086724 learned_addr learned my addr 172.21.15.5:0/2751086724 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f0313fff700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86421160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.942+0000 7f03137fe700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a862716b0 0x560a86415040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 -- 172.21.15.5:0/2751086724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a862716b0 msgr2=0x560a86415040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a862716b0 0x560a86415040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 -- 172.21.15.5:0/2751086724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 msgr2=0x560a86421160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86421160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 -- 172.21.15.5:0/2751086724 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a86435c90 con 0x560a86415580 2023-07-21T19:17:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0313fff700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86421160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:21.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f0312ffd700 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0308005850 tx=0x7f0308009fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c92e825d03117 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f02fa7fc700 1 -- 172.21.15.5:0/2751086724 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308005000 con 0x560a86415580 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f02fa7fc700 1 -- 172.21.15.5:0/2751086724 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0308011ce0 con 0x560a86415580 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.943+0000 7f02fa7fc700 1 -- 172.21.15.5:0/2751086724 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0308004290 con 0x560a86415580 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 -- 172.21.15.5:0/2751086724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 msgr2=0x560a86425c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f0308005850 tx=0x7f0308009fd0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 -- 172.21.15.5:0/2751086724 shutdown_connections 2023-07-21T19:17:21.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a8641b2d0 0x560a86421160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a862716b0 0x560a86415040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/2751086724 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a86415580 0x560a86425c70 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 -- 172.21.15.5:0/2751086724 >> 172.21.15.5:0/2751086724 conn(0x560a8640abe0 msgr2=0x560a86418ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 -- 172.21.15.5:0/2751086724 shutdown_connections 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.944+0000 7f032d9dbb80 1 -- 172.21.15.5:0/2751086724 wait complete. 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.945+0000 7f032d9dbb80 1 -- start start 2023-07-21T19:17:21.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.945+0000 7f032d9dbb80 1 Processor -- start 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.945+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a8647f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a8641b2d0 0x560a8648a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a864800b0 con 0x560a8641b2d0 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8648a870 con 0x560a862716b0 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a8648a9b0 con 0x560a86415580 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f032d9dbb80 10 client.0 _ll_register_callbacks cb 0x7f0304099940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:21.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43536/0 (socket says 172.21.15.5:43536) 2023-07-21T19:17:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 -- 172.21.15.5:0/4222672800 learned_addr learned my addr 172.21.15.5:0/4222672800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f0313fff700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a8641b2d0 0x560a8648a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f0312ffd700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a8647f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 -- 172.21.15.5:0/4222672800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 msgr2=0x560a8647f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a8647f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 -- 172.21.15.5:0/4222672800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a8641b2d0 msgr2=0x560a8648a330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a8641b2d0 0x560a8648a330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.946+0000 7f03137fe700 1 -- 172.21.15.5:0/4222672800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f031410d220 con 0x560a862716b0 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f03137fe700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f030400c5b0 tx=0x7f03040cabf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df916f3b8bf95129 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:21.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03040a21a0 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f03040c9080 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0308004cc0 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0304099130 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f032d9dbb80 10 client.27152 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x7f030801e040 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0308004cc0 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.947+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f030407e030 con 0x560a862716b0 2023-07-21T19:17:21.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 10 client.27152 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 10 client.27152 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 20 client.27152 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8648bd60 0x560a8648e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 4 client.27152 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560a86321ff0 con 0x560a8648bd60 2023-07-21T19:17:21.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.948+0000 7f0312ffd700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8648bd60 0x560a8648e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.949+0000 7f0312ffd700 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8648bd60 0x560a8648e170 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ec6cb7dbf99c4a6 server_cookie=f6ab12d9d81cd02c in_seq=0 out_seq=0 2023-07-21T19:17:21.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.949+0000 7f02f2ffd700 10 client.27152 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.960+0000 7f02f2ffd700 1 -- 172.21.15.5:0/4222672800 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560a86321ff0 con 0x560a8648bd60 2023-07-21T19:17:21.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.960+0000 7f02f2ffd700 10 client.27152 handle_command_reply: tid=0 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.960+0000 7f032d9dbb80 1 client.27152 shutdown 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 msgr2=0x560a8647f4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f030400c5b0 tx=0x7f03040cabf0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 shutdown_connections 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a8648bd60 0x560a8648e170 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a8641b2d0 0x560a8648a330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a862716b0 0x560a8647f4a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f02f2ffd700 0 client.27152 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 --2- 172.21.15.5:0/4222672800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a86415580 0x560a8647f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 >> 172.21.15.5:0/4222672800 conn(0x560a8640abe0 msgr2=0x560a8641e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 shutdown_connections 2023-07-21T19:17:21.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 1 -- 172.21.15.5:0/4222672800 wait complete. 2023-07-21T19:17:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.961+0000 7f032d9dbb80 20 client.27152 trim_cache size 0 max 16384 2023-07-21T19:17:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:17:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:17:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:17:21.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:17:21.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 msgr2=0x7f02fc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 0x7f02fc0a37f0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f030c009790 tx=0x7f030c0074c0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 msgr2=0x7f0314197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:21.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f030400e9a0 tx=0x7f030400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 shutdown_connections 2023-07-21T19:17:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02fc0a1340 0x7f02fc0a37f0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0314105b60 0x7f031419c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03140ffb30 0x7f0314197e10 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 --2- 172.21.15.5:0/865642684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03141004e0 0x7f0314198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 >> 172.21.15.5:0/865642684 conn(0x7f03140fb5f0 msgr2=0x7f031410ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 shutdown_connections 2023-07-21T19:17:21.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:21.962+0000 7f0319fff700 1 -- 172.21.15.5:0/865642684 wait complete. 2023-07-21T19:17:22.346 INFO:teuthology.orchestra.run.smithi005.stderr:36700000 of 50000000 tuples (73%) done (elapsed 559.92 s, remaining 202.91 s) 2023-07-21T19:17:23.727 INFO:teuthology.orchestra.run.smithi005.stderr:36800000 of 50000000 tuples (73%) done (elapsed 561.30 s, remaining 201.33 s) 2023-07-21T19:17:23.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:23 smithi101 ceph-mon[128136]: pgmap v304: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:17:23.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:23 smithi005 ceph-mon[126460]: pgmap v304: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:17:23.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:23 smithi156 ceph-mon[128316]: pgmap v304: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:17:25.025 INFO:teuthology.orchestra.run.smithi005.stderr:36900000 of 50000000 tuples (73%) done (elapsed 562.59 s, remaining 199.73 s) 2023-07-21T19:17:25.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:25 smithi101 ceph-mon[128136]: pgmap v305: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:17:25.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:25 smithi005 ceph-mon[126460]: pgmap v305: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:17:25.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:25 smithi156 ceph-mon[128316]: pgmap v305: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 MiB/s wr, 46 op/s 2023-07-21T19:17:26.316 INFO:teuthology.orchestra.run.smithi005.stderr:37000000 of 50000000 tuples (74%) done (elapsed 563.89 s, remaining 198.12 s) 2023-07-21T19:17:27.644 INFO:teuthology.orchestra.run.smithi005.stderr:37100000 of 50000000 tuples (74%) done (elapsed 565.21 s, remaining 196.53 s) 2023-07-21T19:17:27.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:17:27.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:27 smithi101 ceph-mon[128136]: pgmap v306: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 45 op/s 2023-07-21T19:17:27.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:27 smithi005 ceph-mon[126460]: pgmap v306: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 45 op/s 2023-07-21T19:17:27.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:27 smithi156 ceph-mon[128316]: pgmap v306: 97 pgs: 97 active+clean; 7.9 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 MiB/s wr, 45 op/s 2023-07-21T19:17:28.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/3693813089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c073160 msgr2=0x7fc39c10bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/3693813089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c073160 0x7fc39c10bd50 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009990 tx=0x7fc38c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/3693813089 shutdown_connections 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/3693813089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c10c290 0x7fc39c10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/3693813089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c072850 0x7fc39c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/3693813089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c073160 0x7fc39c10bd50 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/3693813089 >> 172.21.15.5:0/3693813089 conn(0x7fc39c0fb4f0 msgr2=0x7fc39c0fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/3693813089 shutdown_connections 2023-07-21T19:17:28.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.056+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/3693813089 wait complete. 2023-07-21T19:17:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.058+0000 7fc3a2b9d700 1 Processor -- start 2023-07-21T19:17:28.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.058+0000 7fc3a2b9d700 1 -- start start 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c073160 0x7fc39c19de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c10c290 0x7fc39c197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c197fb0 con 0x7fc39c072850 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c198120 con 0x7fc39c073160 2023-07-21T19:17:28.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a2b9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc39c1982e0 con 0x7fc39c10c290 2023-07-21T19:17:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.059+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37350/0 (socket says 172.21.15.5:37350) 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/423555446 learned_addr learned my addr 172.21.15.5:0/423555446 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a239c700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c10c290 0x7fc39c197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a139a700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c073160 0x7fc39c19de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/423555446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c10c290 msgr2=0x7fc39c197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c10c290 0x7fc39c197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/423555446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c073160 msgr2=0x7fc39c19de10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c073160 0x7fc39c19de10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.060+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/423555446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38c0095f0 con 0x7fc39c072850 2023-07-21T19:17:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.061+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc39800e9a0 tx=0x7fc39800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.061+0000 7fc392ffd700 1 -- 172.21.15.5:0/423555446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398009790 con 0x7fc39c072850 2023-07-21T19:17:28.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.061+0000 7fc392ffd700 1 -- 172.21.15.5:0/423555446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc39800cbd0 con 0x7fc39c072850 2023-07-21T19:17:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.061+0000 7fc392ffd700 1 -- 172.21.15.5:0/423555446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398018680 con 0x7fc39c072850 2023-07-21T19:17:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.062+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc39c1985c0 con 0x7fc39c072850 2023-07-21T19:17:28.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.062+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc39c1a48c0 con 0x7fc39c072850 2023-07-21T19:17:28.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.067+0000 7fc392ffd700 1 -- 172.21.15.5:0/423555446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc39800cd40 con 0x7fc39c072850 2023-07-21T19:17:28.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.069+0000 7fc3b77eab80 1 Processor -- start 2023-07-21T19:17:28.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.069+0000 7fc3b77eab80 1 -- start start 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43afc000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43afe920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f439eecb0 con 0x557f43ae9e00 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43a41a40 con 0x557f43ae94f0 2023-07-21T19:17:28.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43a3aed0 con 0x557f43afc540 2023-07-21T19:17:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.070+0000 7fc3a239c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56658/0 (socket says 172.21.15.5:56658) 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 -- 172.21.15.5:0/498033942 learned_addr learned my addr 172.21.15.5:0/498033942 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc392ffd700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 0x7fc3880a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a139a700 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43afe920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc392ffd700 1 -- 172.21.15.5:0/423555446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc398014070 con 0x7fc39c072850 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43afc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 0x7fc3880a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 -- 172.21.15.5:0/498033942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 msgr2=0x557f43afe920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43afe920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 -- 172.21.15.5:0/498033942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 msgr2=0x557f43afc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43afc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.071+0000 7fc3a239c700 1 -- 172.21.15.5:0/498033942 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f43868de0 con 0x557f43ae94f0 2023-07-21T19:17:28.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.072+0000 7fc3a239c700 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc394007990 tx=0x7fc394007ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87ab5c3be7bdb40d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.072+0000 7fc387fff700 1 -- 172.21.15.5:0/498033942 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc39400e070 con 0x557f43ae94f0 2023-07-21T19:17:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.072+0000 7fc387fff700 1 -- 172.21.15.5:0/498033942 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc39400f040 con 0x557f43ae94f0 2023-07-21T19:17:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.072+0000 7fc3a239c700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 0x7fc3880a3890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc394021000 tx=0x7fc394020000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.072+0000 7fc387fff700 1 -- 172.21.15.5:0/498033942 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc394023430 con 0x557f43ae94f0 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 -- 172.21.15.5:0/498033942 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 msgr2=0x557f43ae98c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc394007990 tx=0x7fc394007ca0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 -- 172.21.15.5:0/498033942 shutdown_connections 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43afe920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43ae98c0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/498033942 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43afc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 -- 172.21.15.5:0/498033942 >> 172.21.15.5:0/498033942 conn(0x557f43adeff0 msgr2=0x557f43ae4930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 -- 172.21.15.5:0/498033942 shutdown_connections 2023-07-21T19:17:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.073+0000 7fc3b77eab80 1 -- 172.21.15.5:0/498033942 wait complete. 2023-07-21T19:17:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.074+0000 7fc3b77eab80 1 -- start start 2023-07-21T19:17:28.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.074+0000 7fc3b77eab80 1 Processor -- start 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43af91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43af9810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f439daac0 con 0x557f43ae9e00 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f439da7c0 con 0x557f43ae94f0 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f438be920 con 0x557f43afc540 2023-07-21T19:17:28.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.075+0000 7fc3b77eab80 10 client.0 _ll_register_callbacks cb 0x557f439d8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56668/0 (socket says 172.21.15.5:56668) 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/2023265732 learned_addr learned my addr 172.21.15.5:0/2023265732 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a139a700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43af91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/2023265732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 msgr2=0x557f43af9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43af9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/2023265732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 msgr2=0x557f43af91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43af91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/2023265732 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc394007540 con 0x557f43ae94f0 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a139a700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43af91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c5b0 tx=0x7fc39809a010 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3403773ccf056749 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3857fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.076+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3980a2460 con 0x557f43ae94f0 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f439944f0 con 0x557f43ae94f0 2023-07-21T19:17:28.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc3980a1fd0 con 0x557f43ae94f0 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc3980a2190 con 0x557f43ae94f0 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 10 client.27164 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557f43851c60 con 0x557f43ae94f0 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557f439944f0 con 0x557f43ae94f0 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc398098f70 con 0x557f43ae94f0 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 10 client.27164 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:28.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 10 client.27164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 20 client.27164 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b0be50 0x557f43b0e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 4 client.27164 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.077+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f43a33eb0 con 0x557f43b0be50 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.078+0000 7fc3a139a700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b0be50 0x557f43b0e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.078+0000 7fc3857fa700 10 client.27164 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:28.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.078+0000 7fc3a139a700 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b0be50 0x557f43b0e260 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec2ce31d3d3541c6 server_cookie=4fd7ed4712d74405 in_seq=0 out_seq=0 2023-07-21T19:17:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.079+0000 7fc3857fa700 1 -- 172.21.15.5:0/2023265732 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f43a33eb0 con 0x557f43b0be50 2023-07-21T19:17:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.079+0000 7fc3857fa700 10 client.27164 handle_command_reply: tid=0 2023-07-21T19:17:28.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.079+0000 7fc3b77eab80 1 client.27164 shutdown 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 msgr2=0x557f43afbf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fc39800c5b0 tx=0x7fc39809a010 comp rx=0 tx=0).stop 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 shutdown_connections 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43af9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43afbf60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b0be50 0x557f43b0e260 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/2023265732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43af91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3857fa700 0 client.27164 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 >> 172.21.15.5:0/2023265732 conn(0x557f43adeff0 msgr2=0x557f43ae4520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 shutdown_connections 2023-07-21T19:17:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 1 -- 172.21.15.5:0/2023265732 wait complete. 2023-07-21T19:17:28.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.080+0000 7fc3b77eab80 20 client.27164 trim_cache size 0 max 16384 2023-07-21T19:17:28.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 Processor -- start 2023-07-21T19:17:28.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 -- start start 2023-07-21T19:17:28.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43ae9e00 0x557f43af8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 0x557f43af9060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43afa1b0 con 0x557f43ae94f0 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43af95a0 con 0x557f43afc540 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.101+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43adee00 con 0x557f43ae9e00 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37380/0 (socket says 172.21.15.5:37380) 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/101390839 learned_addr learned my addr 172.21.15.5:0/101390839 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a239c700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 0x557f43af9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a139a700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43ae9e00 0x557f43af8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/101390839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43ae9e00 msgr2=0x557f43af8b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43ae9e00 0x557f43af8b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/101390839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 msgr2=0x557f43af9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 0x557f43af9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 -- 172.21.15.5:0/101390839 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f4388f250 con 0x557f43ae94f0 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a239c700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 0x557f43af9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.102+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc3980ca740 tx=0x7fc39809e1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35177f94df14d9d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc390bff700 1 -- 172.21.15.5:0/101390839 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398099340 con 0x557f43ae94f0 2023-07-21T19:17:28.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc390bff700 1 -- 172.21.15.5:0/101390839 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc3980a2460 con 0x557f43ae94f0 2023-07-21T19:17:28.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc390bff700 1 -- 172.21.15.5:0/101390839 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc398018a40 con 0x557f43ae94f0 2023-07-21T19:17:28.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 -- 172.21.15.5:0/101390839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 msgr2=0x557f43afba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fc3980ca740 tx=0x7fc39809e1c0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 -- 172.21.15.5:0/101390839 shutdown_connections 2023-07-21T19:17:28.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43afc540 0x557f43af9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae94f0 0x557f43afba90 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/101390839 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43ae9e00 0x557f43af8b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.103+0000 7fc3b77eab80 1 -- 172.21.15.5:0/101390839 >> 172.21.15.5:0/101390839 conn(0x557f43adeff0 msgr2=0x557f43af4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.104+0000 7fc3b77eab80 1 -- 172.21.15.5:0/101390839 shutdown_connections 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.104+0000 7fc3b77eab80 1 -- 172.21.15.5:0/101390839 wait complete. 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.104+0000 7fc3b77eab80 1 -- start start 2023-07-21T19:17:28.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.104+0000 7fc3b77eab80 1 Processor -- start 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43b010f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43b01800 con 0x557f43ae9e00 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43b01970 con 0x557f43ae94f0 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f43b01b30 con 0x557f43afc540 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3b77eab80 10 client.0 _ll_register_callbacks cb 0x7fc398099b50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:28.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3a139a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3a139a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37396/0 (socket says 172.21.15.5:37396) 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3a1b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56682/0 (socket says 172.21.15.5:56682) 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.105+0000 7fc3a139a700 1 -- 172.21.15.5:0/1604923096 learned_addr learned my addr 172.21.15.5:0/1604923096 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a239c700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43b010f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 -- 172.21.15.5:0/1604923096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 msgr2=0x557f43b010f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43b010f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 -- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 msgr2=0x557f43b02c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 -- 172.21.15.5:0/1604923096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557f43868de0 con 0x557f43ae9e00 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3a139a700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009990 tx=0x7fc38c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea1248c5049ddc7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.106+0000 7fc3857fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38c01d070 con 0x557f43ae9e00 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc39400f040 con 0x557f43ae9e00 2023-07-21T19:17:28.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc38c028140 con 0x557f43ae9e00 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc38c028300 con 0x557f43ae9e00 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 10 client.17487 fetch_fsmap learned FSMap version 38 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) v3 -- 0x557f43851c60 con 0x557f43ae9e00 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc39400f040 con 0x557f43ae9e00 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 38) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc38c03a5a0 con 0x557f43ae9e00 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 10 client.17487 fetch_fsmap finished waiting for FSMap version 38 2023-07-21T19:17:28.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 10 client.17487 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:28.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 20 client.17487 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:28.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.107+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b61050 0x557f43b63460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:28.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.108+0000 7fc3b77eab80 4 client.17487 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:17:28.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.108+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557f439f6050 con 0x557f43b61050 2023-07-21T19:17:28.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.108+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b61050 0x557f43b63460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:28.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.108+0000 7fc3a1b9b700 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b61050 0x557f43b63460 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae2564ec7976f1af server_cookie=3cf8b35d63ad735e in_seq=0 out_seq=0 2023-07-21T19:17:28.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.108+0000 7fc3857fa700 10 client.17487 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:28.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3857fa700 1 -- 172.21.15.5:0/1604923096 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x557f439f6050 con 0x557f43b61050 2023-07-21T19:17:28.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3857fa700 10 client.17487 handle_command_reply: tid=0 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 client.17487 shutdown 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 msgr2=0x557f43b031b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fc38c009990 tx=0x7fc38c00b260 comp rx=0 tx=0).stop 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 shutdown_connections 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557f43afc540 0x557f43b010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557f43ae94f0 0x557f43b02c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557f43b61050 0x557f43b63460 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 --2- 172.21.15.5:0/1604923096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557f43ae9e00 0x557f43b031b0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 >> 172.21.15.5:0/1604923096 conn(0x557f43adeff0 msgr2=0x557f43af8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3857fa700 0 client.17487 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 shutdown_connections 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 1 -- 172.21.15.5:0/1604923096 wait complete. 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.109+0000 7fc3b77eab80 20 client.17487 trim_cache size 0 max 16384 2023-07-21T19:17:28.113 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93": { 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "42bafb64-7ad7-4a9a-a8f1-40eac9d80b93", 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:17:28.114 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:17:28.115 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:17:28.115 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:17:28.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 msgr2=0x7fc3880a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 0x7fc3880a3890 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fc394021000 tx=0x7fc394020000 comp rx=0 tx=0).stop 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 msgr2=0x7fc39c19d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fc39800e9a0 tx=0x7fc39800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 shutdown_connections 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc39c10c290 0x7fc39c197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc3880a13e0 0x7fc3880a3890 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc39c072850 0x7fc39c19d8d0 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 --2- 172.21.15.5:0/423555446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc39c073160 0x7fc39c19de10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 >> 172.21.15.5:0/423555446 conn(0x7fc39c0fb4f0 msgr2=0x7fc39c103cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 shutdown_connections 2023-07-21T19:17:28.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:28.110+0000 7fc3a2b9d700 1 -- 172.21.15.5:0/423555446 wait complete. 2023-07-21T19:17:28.126 INFO:tasks.cephfs.filesystem:scrub status for tag:42bafb64-7ad7-4a9a-a8f1-40eac9d80b93 - {'path': '/', 'tag': '42bafb64-7ad7-4a9a-a8f1-40eac9d80b93', 'options': 'recursive,force'} 2023-07-21T19:17:29.142 INFO:teuthology.orchestra.run.smithi005.stderr:37200000 of 50000000 tuples (74%) done (elapsed 566.71 s, remaining 195.00 s) 2023-07-21T19:17:29.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:29 smithi101 ceph-mon[128136]: pgmap v307: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:17:29.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:29 smithi005 ceph-mon[126460]: pgmap v307: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:17:29.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:29 smithi156 ceph-mon[128316]: pgmap v307: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 34 MiB/s wr, 66 op/s 2023-07-21T19:17:30.461 INFO:teuthology.orchestra.run.smithi005.stderr:37300000 of 50000000 tuples (74%) done (elapsed 568.03 s, remaining 193.40 s) 2023-07-21T19:17:31.818 INFO:teuthology.orchestra.run.smithi005.stderr:37400000 of 50000000 tuples (74%) done (elapsed 569.39 s, remaining 191.83 s) 2023-07-21T19:17:31.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:31 smithi101 ceph-mon[128136]: pgmap v308: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 51 op/s 2023-07-21T19:17:31.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:31 smithi005 ceph-mon[126460]: pgmap v308: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 51 op/s 2023-07-21T19:17:31.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:31 smithi156 ceph-mon[128316]: pgmap v308: 97 pgs: 97 active+clean; 8.0 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 23 MiB/s wr, 51 op/s 2023-07-21T19:17:33.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:33 smithi101 ceph-mon[128136]: pgmap v309: 97 pgs: 97 active+clean; 8.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-07-21T19:17:33.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:33 smithi005 ceph-mon[126460]: pgmap v309: 97 pgs: 97 active+clean; 8.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-07-21T19:17:33.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:33 smithi156 ceph-mon[128316]: pgmap v309: 97 pgs: 97 active+clean; 8.1 GiB data, 3.5 GiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 32 MiB/s wr, 79 op/s 2023-07-21T19:17:34.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:34 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:34.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:34 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:34.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:34 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 -- 172.21.15.5:0/2585848094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 msgr2=0x7f9370105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 --2- 172.21.15.5:0/2585848094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f9370105f30 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f9360009990 tx=0x7f9360009ca0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 -- 172.21.15.5:0/2585848094 shutdown_connections 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 --2- 172.21.15.5:0/2585848094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 0x7f937010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 --2- 172.21.15.5:0/2585848094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f9370105f30 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 --2- 172.21.15.5:0/2585848094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f93700fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 -- 172.21.15.5:0/2585848094 >> 172.21.15.5:0/2585848094 conn(0x7f93700fb5f0 msgr2=0x7f93700fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 -- 172.21.15.5:0/2585848094 shutdown_connections 2023-07-21T19:17:35.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.123+0000 7f9374bff700 1 -- 172.21.15.5:0/2585848094 wait complete. 2023-07-21T19:17:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.124+0000 7f9374bff700 1 Processor -- start 2023-07-21T19:17:35.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.124+0000 7f9374bff700 1 -- start start 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 0x7f9370198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f937019c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937019cf10 con 0x7f9370105b60 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937019d080 con 0x7f93700ffb30 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f9374bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937019d240 con 0x7f93701004e0 2023-07-21T19:17:35.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f937019c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56702/0 (socket says 172.21.15.5:56702) 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f937019c8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37416/0 (socket says 172.21.15.5:37416) 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 -- 172.21.15.5:0/3657319111 learned_addr learned my addr 172.21.15.5:0/3657319111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 0x7f9370198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 -- 172.21.15.5:0/3657319111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 msgr2=0x7f9370198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 0x7f9370198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 -- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 msgr2=0x7f937019c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f937019c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.125+0000 7f936e59c700 1 -- 172.21.15.5:0/3657319111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600095f0 con 0x7f93700ffb30 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f936e59c700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9360005740 tx=0x7f9360005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f935f7fe700 1 -- 172.21.15.5:0/3657319111 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936001d070 con 0x7f93700ffb30 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f935f7fe700 1 -- 172.21.15.5:0/3657319111 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9360022470 con 0x7f93700ffb30 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937019d4c0 con 0x7f93700ffb30 2023-07-21T19:17:35.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f935f7fe700 1 -- 172.21.15.5:0/3657319111 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360017650 con 0x7f93700ffb30 2023-07-21T19:17:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.126+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93701a4d60 con 0x7f93700ffb30 2023-07-21T19:17:35.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.129+0000 7f935f7fe700 1 -- 172.21.15.5:0/3657319111 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f936000fb00 con 0x7f93700ffb30 2023-07-21T19:17:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f935f7fe700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 0x7f93580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f938853eb80 1 Processor -- start 2023-07-21T19:17:35.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f935f7fe700 1 -- 172.21.15.5:0/3657319111 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f93600ce330 con 0x7f93700ffb30 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 0x7f93580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f938853eb80 1 -- start start 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.131+0000 7f938853eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f938853eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 0x563b7ddba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f938853eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddba9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f938853eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7dcbfc50 con 0x563b7dc166b0 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f938853eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7dd129e0 con 0x563b7ddc02d0 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f938853eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7dd0be70 con 0x563b7ddba580 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 0x7f93580a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9370199540 tx=0x7f9364009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37432/0 (socket says 172.21.15.5:37432) 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 -- 172.21.15.5:0/1778247852 learned_addr learned my addr 172.21.15.5:0/1778247852 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936dd9b700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 0x563b7ddba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936ed9d700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddba9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 -- 172.21.15.5:0/1778247852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 msgr2=0x563b7ddba9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddba9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 -- 172.21.15.5:0/1778247852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 msgr2=0x563b7ddba040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 0x563b7ddba040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936e59c700 1 -- 172.21.15.5:0/1778247852 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b7db39d80 con 0x563b7dc166b0 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.132+0000 7f936dd9b700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 0x563b7ddba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:35.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f936e59c700 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9360097ea0 tx=0x7f9360005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abe5fb706027fd89 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f935cff9700 1 -- 172.21.15.5:0/1778247852 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936009f460 con 0x563b7dc166b0 2023-07-21T19:17:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f935cff9700 1 -- 172.21.15.5:0/1778247852 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93600a37f0 con 0x563b7dc166b0 2023-07-21T19:17:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f935cff9700 1 -- 172.21.15.5:0/1778247852 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9360022850 con 0x563b7dc166b0 2023-07-21T19:17:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 -- 172.21.15.5:0/1778247852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 msgr2=0x563b7ddbfd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f9360097ea0 tx=0x7f9360005910 comp rx=0 tx=0).stop 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 -- 172.21.15.5:0/1778247852 shutdown_connections 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddba9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7ddbfd90 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.133+0000 7f938853eb80 1 --2- 172.21.15.5:0/1778247852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddc02d0 0x563b7ddba040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.134+0000 7f938853eb80 1 -- 172.21.15.5:0/1778247852 >> 172.21.15.5:0/1778247852 conn(0x563b7ddafbe0 msgr2=0x563b7ddb4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.134+0000 7f938853eb80 1 -- 172.21.15.5:0/1778247852 shutdown_connections 2023-07-21T19:17:35.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.134+0000 7f938853eb80 1 -- 172.21.15.5:0/1778247852 wait complete. 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.134+0000 7f938853eb80 1 -- start start 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.134+0000 7f938853eb80 1 Processor -- start 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddc36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7dcaba60 con 0x563b7ddc02d0 2023-07-21T19:17:35.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7dcab760 con 0x563b7dc166b0 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7db8f8c0 con 0x563b7ddba580 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f938853eb80 10 client.0 _ll_register_callbacks cb 0x563b7dca9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddc36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56710/0 (socket says 172.21.15.5:56710) 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936e59c700 1 -- 172.21.15.5:0/1282117617 learned_addr learned my addr 172.21.15.5:0/1282117617 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936ed9d700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddc36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37434/0 (socket says 172.21.15.5:37434) 2023-07-21T19:17:35.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.135+0000 7f936dd9b700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 -- 172.21.15.5:0/1282117617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 msgr2=0x563b7ddc9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 -- 172.21.15.5:0/1282117617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 msgr2=0x563b7ddc36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddc36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 -- 172.21.15.5:0/1282117617 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9364011a00 con 0x563b7dc166b0 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f936e59c700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f936009b3c0 tx=0x7f936009ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac2c2b985f166161 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f9355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936009f460 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9360022850 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b7dc65490 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f936009f120 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f938853eb80 10 client.27182 fetch_fsmap learned FSMap version 39 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f93701a4d60 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.136+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b7dc65490 con 0x563b7dc166b0 2023-07-21T19:17:35.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 39) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f93600cb070 con 0x563b7dc166b0 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 10 client.27182 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 10 client.27182 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 20 client.27182 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7ddd33d0 0x563b7ddd57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 4 client.27182 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b7dd04e50 con 0x563b7ddd33d0 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.137+0000 7f936dd9b700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7ddd33d0 0x563b7ddd57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.138+0000 7f9355ffb700 10 client.27182 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:35.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.138+0000 7f936dd9b700 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7ddd33d0 0x563b7ddd57e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=830b04640470c2de server_cookie=9df30d83082e5e6d in_seq=0 out_seq=0 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.138+0000 7f9355ffb700 1 -- 172.21.15.5:0/1282117617 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b7dd04e50 con 0x563b7ddd33d0 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.138+0000 7f9355ffb700 10 client.27182 handle_command_reply: tid=0 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.138+0000 7f938853eb80 1 client.27182 shutdown 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 msgr2=0x563b7ddc9350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f936009b3c0 tx=0x7f936009ba50 comp rx=0 tx=0).stop 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 shutdown_connections 2023-07-21T19:17:35.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7ddd33d0 0x563b7ddd57e0 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddc36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddc9350 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f9355ffb700 0 client.27182 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 --2- 172.21.15.5:0/1282117617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 >> 172.21.15.5:0/1282117617 conn(0x563b7ddafbe0 msgr2=0x563b7ddb44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 shutdown_connections 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 1 -- 172.21.15.5:0/1282117617 wait complete. 2023-07-21T19:17:35.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.139+0000 7f938853eb80 20 client.27182 trim_cache size 0 max 16384 2023-07-21T19:17:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 Processor -- start 2023-07-21T19:17:35.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 -- start start 2023-07-21T19:17:35.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc37e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddbf130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddbf9f0 con 0x563b7ddc02d0 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.154+0000 7f938853eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddbafd0 con 0x563b7dc166b0 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f938853eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddbe610 con 0x563b7ddba580 2023-07-21T19:17:35.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56724/0 (socket says 172.21.15.5:56724) 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 -- 172.21.15.5:0/3380144782 learned_addr learned my addr 172.21.15.5:0/3380144782 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936ed9d700 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddbf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 -- 172.21.15.5:0/3380144782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 msgr2=0x563b7ddc37e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 -- 172.21.15.5:0/3380144782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 msgr2=0x563b7ddbf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddbf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 -- 172.21.15.5:0/3380144782 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b7db22c00 con 0x563b7dc166b0 2023-07-21T19:17:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936ed9d700 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddbf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:35.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.155+0000 7f936e59c700 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9360005cd0 tx=0x7f93600999e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29033927812e6949 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f935d7fa700 1 -- 172.21.15.5:0/3380144782 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600a35e0 con 0x563b7dc166b0 2023-07-21T19:17:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f935d7fa700 1 -- 172.21.15.5:0/3380144782 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93600b1030 con 0x563b7dc166b0 2023-07-21T19:17:35.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f935d7fa700 1 -- 172.21.15.5:0/3380144782 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936009f120 con 0x563b7dc166b0 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 -- 172.21.15.5:0/3380144782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 msgr2=0x563b7ddba040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9360005cd0 tx=0x7f93600999e0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 -- 172.21.15.5:0/3380144782 shutdown_connections 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7ddc02d0 0x563b7ddbf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7dc166b0 0x563b7ddba040 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 --2- 172.21.15.5:0/3380144782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddba580 0x563b7ddc37e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 -- 172.21.15.5:0/3380144782 >> 172.21.15.5:0/3380144782 conn(0x563b7ddafbe0 msgr2=0x563b7ddbea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 -- 172.21.15.5:0/3380144782 shutdown_connections 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.156+0000 7f938853eb80 1 -- 172.21.15.5:0/3380144782 wait complete. 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 -- start start 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 Processor -- start 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 0x563b7ddc8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddc9210 con 0x563b7dc166b0 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddc9380 con 0x563b7ddba580 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b7ddc9540 con 0x563b7ddc02d0 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.157+0000 7f938853eb80 10 client.0 _ll_register_callbacks cb 0x7f9364009980 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37448/0 (socket says 172.21.15.5:37448) 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 -- 172.21.15.5:0/3773798376 learned_addr learned my addr 172.21.15.5:0/3773798376 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:35.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936ed9d700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 0x563b7ddc8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 -- 172.21.15.5:0/3773798376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 msgr2=0x563b7ddc8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 0x563b7ddc8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 -- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 msgr2=0x563b7de201e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 -- 172.21.15.5:0/3773798376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93680099f0 con 0x563b7dc166b0 2023-07-21T19:17:35.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936ed9d700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 0x563b7ddc8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f936e59c700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f936009b130 tx=0x7f9360009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71f0b5e91ce11f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.158+0000 7f9355ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93600a35e0 con 0x563b7dc166b0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f936009f460 con 0x563b7dc166b0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f93600a37f0 con 0x563b7dc166b0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9360067070 con 0x563b7dc166b0 2023-07-21T19:17:35.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 10 client.17505 fetch_fsmap learned FSMap version 39 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) v3 -- 0x7f93701a4d60 con 0x563b7dc166b0 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f93600a37f0 con 0x563b7dc166b0 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 39) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f936009a920 con 0x563b7dc166b0 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 10 client.17505 fetch_fsmap finished waiting for FSMap version 39 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 10 client.17505 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 20 client.17505 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:35.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7de276d0 0x563b7de29ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 4 client.17505 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:17:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.159+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b7dcc6ff0 con 0x563b7de276d0 2023-07-21T19:17:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.160+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7de276d0 0x563b7de29ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.160+0000 7f936dd9b700 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7de276d0 0x563b7de29ae0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b7a01523ff1a520 server_cookie=429b45edb5ac0a12 in_seq=0 out_seq=0 2023-07-21T19:17:35.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.160+0000 7f9355ffb700 10 client.17505 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f9355ffb700 1 -- 172.21.15.5:0/3773798376 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b7dcc6ff0 con 0x563b7de276d0 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f9355ffb700 10 client.17505 handle_command_reply: tid=0 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 client.17505 shutdown 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 msgr2=0x563b7de1fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f936009b130 tx=0x7f9360009310 comp rx=0 tx=0).stop 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 shutdown_connections 2023-07-21T19:17:35.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b7ddc02d0 0x563b7ddc8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b7de276d0 0x563b7de29ae0 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b7dc166b0 0x563b7de1fca0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 --2- 172.21.15.5:0/3773798376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b7ddba580 0x563b7de201e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f9355ffb700 0 client.17505 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 >> 172.21.15.5:0/3773798376 conn(0x563b7ddafbe0 msgr2=0x563b7ddc08a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 shutdown_connections 2023-07-21T19:17:35.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.172+0000 7f938853eb80 1 -- 172.21.15.5:0/3773798376 wait complete. 2023-07-21T19:17:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f938853eb80 20 client.17505 trim_cache size 0 max 16384 2023-07-21T19:17:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:17:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:17:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:17:35.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:17:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 msgr2=0x7f93580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 0x7f93580a3770 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f9370199540 tx=0x7f9364009260 comp rx=0 tx=0).stop 2023-07-21T19:17:35.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 msgr2=0x7f9370197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f9360005740 tx=0x7f9360005770 comp rx=0 tx=0).stop 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 shutdown_connections 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f93580a12c0 0x7f93580a3770 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9370105b60 0x7f937019c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f93700ffb30 0x7f9370197d20 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 --2- 172.21.15.5:0/3657319111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f93701004e0 0x7f9370198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 >> 172.21.15.5:0/3657319111 conn(0x7f93700fb5f0 msgr2=0x7f9370072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:35.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 shutdown_connections 2023-07-21T19:17:35.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:35.173+0000 7f9374bff700 1 -- 172.21.15.5:0/3657319111 wait complete. 2023-07-21T19:17:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:35 smithi101 ceph-mon[128136]: pgmap v310: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-07-21T19:17:35.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:35 smithi005 ceph-mon[126460]: pgmap v310: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-07-21T19:17:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:35 smithi156 ceph-mon[128316]: pgmap v310: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 23 MiB/s wr, 60 op/s 2023-07-21T19:17:36.876 INFO:teuthology.orchestra.run.smithi005.stderr:37500000 of 50000000 tuples (75%) done (elapsed 574.44 s, remaining 191.48 s) 2023-07-21T19:17:37.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:37 smithi101 ceph-mon[128136]: pgmap v311: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 55 op/s 2023-07-21T19:17:37.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:37 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:37.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:37 smithi005 ceph-mon[126460]: pgmap v311: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 55 op/s 2023-07-21T19:17:37.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:37 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:37.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:37 smithi156 ceph-mon[128316]: pgmap v311: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 73 KiB/s rd, 22 MiB/s wr, 55 op/s 2023-07-21T19:17:37.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:37 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:38.391 INFO:teuthology.orchestra.run.smithi005.stderr:37600000 of 50000000 tuples (75%) done (elapsed 575.96 s, remaining 189.94 s) 2023-07-21T19:17:39.720 INFO:teuthology.orchestra.run.smithi005.stderr:37700000 of 50000000 tuples (75%) done (elapsed 577.29 s, remaining 188.35 s) 2023-07-21T19:17:39.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:39 smithi101 ceph-mon[128136]: pgmap v312: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 72 op/s 2023-07-21T19:17:39.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:39 smithi005 ceph-mon[126460]: pgmap v312: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 72 op/s 2023-07-21T19:17:39.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:39 smithi156 ceph-mon[128316]: pgmap v312: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 25 MiB/s wr, 72 op/s 2023-07-21T19:17:41.135 INFO:teuthology.orchestra.run.smithi005.stderr:37800000 of 50000000 tuples (75%) done (elapsed 578.70 s, remaining 186.78 s) 2023-07-21T19:17:41.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:41 smithi101 ceph-mon[128136]: pgmap v313: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 51 op/s 2023-07-21T19:17:41.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:41 smithi005 ceph-mon[126460]: pgmap v313: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 51 op/s 2023-07-21T19:17:41.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:41 smithi156 ceph-mon[128316]: pgmap v313: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 14 MiB/s wr, 51 op/s 2023-07-21T19:17:42.586 INFO:teuthology.orchestra.run.smithi005.stderr:37900000 of 50000000 tuples (75%) done (elapsed 580.16 s, remaining 185.22 s) 2023-07-21T19:17:43.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:43 smithi101 ceph-mon[128136]: pgmap v314: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 72 op/s 2023-07-21T19:17:43.885 INFO:teuthology.orchestra.run.smithi005.stderr:38000000 of 50000000 tuples (76%) done (elapsed 581.45 s, remaining 183.62 s) 2023-07-21T19:17:43.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:43 smithi005 ceph-mon[126460]: pgmap v314: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 72 op/s 2023-07-21T19:17:43.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:43 smithi156 ceph-mon[128316]: pgmap v314: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 22 MiB/s wr, 72 op/s 2023-07-21T19:17:45.221 INFO:teuthology.orchestra.run.smithi005.stderr:38100000 of 50000000 tuples (76%) done (elapsed 582.79 s, remaining 182.03 s) 2023-07-21T19:17:45.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:45 smithi101 ceph-mon[128136]: pgmap v315: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 15 MiB/s wr, 52 op/s 2023-07-21T19:17:45.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:45 smithi005 ceph-mon[126460]: pgmap v315: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 15 MiB/s wr, 52 op/s 2023-07-21T19:17:45.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:45 smithi156 ceph-mon[128316]: pgmap v315: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 98 KiB/s rd, 15 MiB/s wr, 52 op/s 2023-07-21T19:17:46.665 INFO:teuthology.orchestra.run.smithi005.stderr:38200000 of 50000000 tuples (76%) done (elapsed 584.23 s, remaining 180.47 s) 2023-07-21T19:17:46.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:46 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:17:46.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:46.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:46 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:17:46.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:46.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:46.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:46 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:17:46.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:46.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:17:47.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:47 smithi101 ceph-mon[128136]: pgmap v316: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 46 op/s 2023-07-21T19:17:47.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:47 smithi005 ceph-mon[126460]: pgmap v316: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 46 op/s 2023-07-21T19:17:47.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:47 smithi156 ceph-mon[128316]: pgmap v316: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 78 KiB/s rd, 14 MiB/s wr, 46 op/s 2023-07-21T19:17:48.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3870573780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed940ffb00 msgr2=0x7fed940fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3870573780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed940ffb00 0x7fed940fff70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fed840098f0 tx=0x7fed84009c00 comp rx=0 tx=0).stop 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3870573780 shutdown_connections 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3870573780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed941004b0 0x7fed9410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3870573780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed94105b10 0x7fed94105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3870573780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed940ffb00 0x7fed940fff70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3870573780 >> 172.21.15.5:0/3870573780 conn(0x7fed940fb600 msgr2=0x7fed940fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3870573780 shutdown_connections 2023-07-21T19:17:48.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.353+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3870573780 wait complete. 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.354+0000 7fed9c1ff700 1 Processor -- start 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 -- start start 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed941004b0 0x7fed9419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 0x7fed94195a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94196220 con 0x7fed94105b10 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94196390 con 0x7fed941004b0 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed9c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94196550 con 0x7fed940ffb00 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.355+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54038/0 (socket says 172.21.15.5:54038) 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 -- 172.21.15.5:0/3924119217 learned_addr learned my addr 172.21.15.5:0/3924119217 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed9a79c700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 0x7fed94195a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed9979a700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed941004b0 0x7fed9419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 -- 172.21.15.5:0/3924119217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed941004b0 msgr2=0x7fed9419c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed941004b0 0x7fed9419c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 -- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 msgr2=0x7fed94195a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 0x7fed94195a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 -- 172.21.15.5:0/3924119217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed84009520 con 0x7fed940ffb00 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed9a79c700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 0x7fed94195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.356+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9c0 tx=0x7fed9000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.357+0000 7fed8affd700 1 -- 172.21.15.5:0/3924119217 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90004a30 con 0x7fed940ffb00 2023-07-21T19:17:48.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.357+0000 7fed8affd700 1 -- 172.21.15.5:0/3924119217 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed9000cc50 con 0x7fed940ffb00 2023-07-21T19:17:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.357+0000 7fed8affd700 1 -- 172.21.15.5:0/3924119217 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90005410 con 0x7fed940ffb00 2023-07-21T19:17:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.357+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed941967d0 con 0x7fed940ffb00 2023-07-21T19:17:48.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.357+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94072920 con 0x7fed940ffb00 2023-07-21T19:17:48.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.362+0000 7fed8affd700 1 -- 172.21.15.5:0/3924119217 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fed90005570 con 0x7fed940ffb00 2023-07-21T19:17:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.363+0000 7fedafb81b80 1 Processor -- start 2023-07-21T19:17:48.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.363+0000 7fedafb81b80 1 -- start start 2023-07-21T19:17:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.363+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bf0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3beb580 0x55deb3beb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fedafb81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3af0c50 con 0x55deb3beb580 2023-07-21T19:17:48.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fedafb81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3b439e0 con 0x55deb3bf12d0 2023-07-21T19:17:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fedafb81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3b3ce70 con 0x55deb3a476b0 2023-07-21T19:17:48.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51678/0 (socket says 172.21.15.5:51678) 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed99f9b700 1 -- 172.21.15.5:0/3429230321 learned_addr learned my addr 172.21.15.5:0/3429230321 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed9979a700 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3beb580 0x55deb3beb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed8affd700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 0x7fed800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.364+0000 7fed9a79c700 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bf0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed8affd700 1 -- 172.21.15.5:0/3924119217 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fed90014070 con 0x7fed940ffb00 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 -- 172.21.15.5:0/3429230321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 msgr2=0x55deb3bf0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bf0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 -- 172.21.15.5:0/3429230321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3beb580 msgr2=0x55deb3beb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed9a79c700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 0x7fed800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3beb580 0x55deb3beb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 -- 172.21.15.5:0/3429230321 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deb396ad80 con 0x55deb3bf12d0 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed99f9b700 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed900ca530 tx=0x7fed90096210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9980a9c8b615581a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed7ffff700 1 -- 172.21.15.5:0/3429230321 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900997b0 con 0x55deb3bf12d0 2023-07-21T19:17:48.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.365+0000 7fed7ffff700 1 -- 172.21.15.5:0/3429230321 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed9000cdc0 con 0x55deb3bf12d0 2023-07-21T19:17:48.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fed7ffff700 1 -- 172.21.15.5:0/3429230321 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed90096560 con 0x55deb3bf12d0 2023-07-21T19:17:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 -- 172.21.15.5:0/3429230321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 msgr2=0x55deb3beb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fed900ca530 tx=0x7fed90096210 comp rx=0 tx=0).stop 2023-07-21T19:17:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fed9a79c700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 0x7fed800a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0060b0 tx=0x7fed8c004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 -- 172.21.15.5:0/3429230321 shutdown_connections 2023-07-21T19:17:48.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3beb580 0x55deb3beb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bf0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 --2- 172.21.15.5:0/3429230321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3bf12d0 0x55deb3beb040 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.366+0000 7fedafb81b80 1 -- 172.21.15.5:0/3429230321 >> 172.21.15.5:0/3429230321 conn(0x55deb3be0be0 msgr2=0x55deb3be5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.367+0000 7fedafb81b80 1 -- 172.21.15.5:0/3429230321 shutdown_connections 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.367+0000 7fedafb81b80 1 -- 172.21.15.5:0/3429230321 wait complete. 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.367+0000 7fedafb81b80 1 -- start start 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.367+0000 7fedafb81b80 1 Processor -- start 2023-07-21T19:17:48.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 0x55deb3bf7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3bf12d0 0x55deb3bf4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3adca60 con 0x55deb3a476b0 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3adc760 con 0x55deb3beb580 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb39c08c0 con 0x55deb3bf12d0 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fedafb81b80 10 client.0 _ll_register_callbacks cb 0x55deb3ada700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:48.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fed9979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 0x55deb3bf7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fed9979a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51692/0 (socket says 172.21.15.5:51692) 2023-07-21T19:17:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.368+0000 7fed9979a700 1 -- 172.21.15.5:0/51816626 learned_addr learned my addr 172.21.15.5:0/51816626 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:48.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed9a79c700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3bf12d0 0x55deb3bf4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 -- 172.21.15.5:0/51816626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3bf12d0 msgr2=0x55deb3bf4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3bf12d0 0x55deb3bf4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 -- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 msgr2=0x55deb3bf47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 -- 172.21.15.5:0/51816626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed94072920 con 0x55deb3a476b0 2023-07-21T19:17:48.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed9979a700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed99f9b700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 0x55deb3bf7540 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55deb3be7b40 tx=0x7fed9000b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12c447a7dd680377 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.369+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900d0020 con 0x55deb3a476b0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fed900a22a0 con 0x55deb3a476b0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55deb3a96490 con 0x55deb3a476b0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed900b82a0 con 0x55deb3a476b0 2023-07-21T19:17:48.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 10 client.17517 fetch_fsmap learned FSMap version 40 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55deb3953c00 con 0x55deb3a476b0 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55deb3a96490 con 0x55deb3a476b0 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed9009c030 con 0x55deb3a476b0 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 10 client.17517 fetch_fsmap finished waiting for FSMap version 40 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 10 client.17517 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 20 client.17517 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c047f0 0x55deb3c06bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 4 client.17517 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.370+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55deb3b35e50 con 0x55deb3c047f0 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.371+0000 7fed9979a700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c047f0 0x55deb3c06bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.371+0000 7fed9979a700 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c047f0 0x55deb3c06bd0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a1ddd0454bb8d3e server_cookie=4923ba646cb6a02c in_seq=0 out_seq=0 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.371+0000 7fed7d7fa700 10 client.17517 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.372+0000 7fed7d7fa700 1 -- 172.21.15.5:0/51816626 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55deb3b35e50 con 0x55deb3c047f0 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.372+0000 7fed7d7fa700 10 client.17517 handle_command_reply: tid=0 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.372+0000 7fedafb81b80 1 client.17517 shutdown 2023-07-21T19:17:48.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.372+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 msgr2=0x55deb3bf7540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.372+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 0x55deb3bf7540 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x55deb3be7b40 tx=0x7fed9000b3e0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 shutdown_connections 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3bf12d0 0x55deb3bf4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c047f0 0x55deb3c06bd0 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3a476b0 0x55deb3bf7540 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 --2- 172.21.15.5:0/51816626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fed7d7fa700 0 client.17517 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 >> 172.21.15.5:0/51816626 conn(0x55deb3be0be0 msgr2=0x55deb3be4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 shutdown_connections 2023-07-21T19:17:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 1 -- 172.21.15.5:0/51816626 wait complete. 2023-07-21T19:17:48.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.373+0000 7fedafb81b80 20 client.17517 trim_cache size 0 max 16384 2023-07-21T19:17:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 Processor -- start 2023-07-21T19:17:48.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 -- start start 2023-07-21T19:17:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bfbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf2010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3a96490 con 0x55deb3bf12d0 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3be7b70 con 0x55deb3beb580 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fedafb81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3bb1060 con 0x55deb3a476b0 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fed9a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.398+0000 7fed9a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55630/0 (socket says 172.21.15.5:55630) 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 -- 172.21.15.5:0/1349015333 learned_addr learned my addr 172.21.15.5:0/1349015333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed99f9b700 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bfbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9979a700 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 -- 172.21.15.5:0/1349015333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 msgr2=0x55deb3bfbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bfbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 -- 172.21.15.5:0/1349015333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 msgr2=0x55deb3bf2010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf2010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 -- 172.21.15.5:0/1349015333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55deb3c0b800 con 0x55deb3bf12d0 2023-07-21T19:17:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.399+0000 7fed9a79c700 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009420 tx=0x7fed8c009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c402ec6f27d87ea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.400+0000 7fed88ff9700 1 -- 172.21.15.5:0/1349015333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c007700 con 0x55deb3bf12d0 2023-07-21T19:17:48.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.400+0000 7fed88ff9700 1 -- 172.21.15.5:0/1349015333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed8c022040 con 0x55deb3bf12d0 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.400+0000 7fed88ff9700 1 -- 172.21.15.5:0/1349015333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c026440 con 0x55deb3bf12d0 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- 172.21.15.5:0/1349015333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 msgr2=0x55deb3bf70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009420 tx=0x7fed8c009920 comp rx=0 tx=0).stop 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- 172.21.15.5:0/1349015333 shutdown_connections 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bf70e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3a476b0 0x55deb3bfbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 --2- 172.21.15.5:0/1349015333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3beb580 0x55deb3bf2010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- 172.21.15.5:0/1349015333 >> 172.21.15.5:0/1349015333 conn(0x55deb3be0be0 msgr2=0x55deb3bf1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- 172.21.15.5:0/1349015333 shutdown_connections 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- 172.21.15.5:0/1349015333 wait complete. 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.401+0000 7fedafb81b80 1 -- start start 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 Processor -- start 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 0x55deb3bee020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bec790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3beccd0 con 0x55deb3bf12d0 2023-07-21T19:17:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3bece40 con 0x55deb3a476b0 2023-07-21T19:17:48.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55deb3bed000 con 0x55deb3beb580 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.402+0000 7fedafb81b80 10 client.0 _ll_register_callbacks cb 0x55deb3ada700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51706/0 (socket says 172.21.15.5:51706) 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 -- 172.21.15.5:0/2251137927 learned_addr learned my addr 172.21.15.5:0/2251137927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed9a79c700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bec790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed9979a700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 0x55deb3bee020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 -- 172.21.15.5:0/2251137927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 msgr2=0x55deb3bee020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 0x55deb3bee020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 -- 172.21.15.5:0/2251137927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 msgr2=0x55deb3bec790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bec790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 -- 172.21.15.5:0/2251137927 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed8c009160 con 0x55deb3a476b0 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed9979a700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 0x55deb3bee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed9a79c700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:48.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed99f9b700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fed90099b50 tx=0x7fed900ccf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f37b84b896a8ba5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.403+0000 7fed7d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed900d0020 con 0x55deb3a476b0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fed900a22a0 con 0x55deb3a476b0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fed9000cdc0 con 0x55deb3a476b0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fed900b82a0 con 0x55deb3a476b0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 10 client.27206 fetch_fsmap learned FSMap version 40 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=4+}) v3 -- 0x55deb39911f0 con 0x55deb3a476b0 2023-07-21T19:17:48.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fed9000cdc0 con 0x55deb3a476b0 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 40) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fed90078d20 con 0x55deb3a476b0 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 10 client.27206 fetch_fsmap finished waiting for FSMap version 40 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 10 client.27206 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 20 client.27206 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c59020 0x55deb3c5b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 4 client.27206 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.404+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55deb3af7ff0 con 0x55deb3c59020 2023-07-21T19:17:48.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.405+0000 7fed9979a700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c59020 0x55deb3c5b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.405+0000 7fed9979a700 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c59020 0x55deb3c5b400 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1560dc4a3eba249e server_cookie=7cf9d118f63804aa in_seq=0 out_seq=0 2023-07-21T19:17:48.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.405+0000 7fed7d7fa700 10 client.27206 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fed7d7fa700 1 -- 172.21.15.5:0/2251137927 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55deb3af7ff0 con 0x55deb3c59020 2023-07-21T19:17:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fed7d7fa700 10 client.27206 handle_command_reply: tid=0 2023-07-21T19:17:48.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 client.27206 shutdown 2023-07-21T19:17:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 msgr2=0x55deb3bedae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fed90099b50 tx=0x7fed900ccf30 comp rx=0 tx=0).stop 2023-07-21T19:17:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 shutdown_connections 2023-07-21T19:17:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55deb3c59020 0x55deb3c5b400 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55deb3bf12d0 0x55deb3bec790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55deb3a476b0 0x55deb3bedae0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 --2- 172.21.15.5:0/2251137927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55deb3beb580 0x55deb3bee020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fed7d7fa700 0 client.27206 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 >> 172.21.15.5:0/2251137927 conn(0x55deb3be0be0 msgr2=0x55deb3bf2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 shutdown_connections 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.419+0000 7fedafb81b80 1 -- 172.21.15.5:0/2251137927 wait complete. 2023-07-21T19:17:48.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fedafb81b80 20 client.27206 trim_cache size 0 max 16384 2023-07-21T19:17:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:17:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:17:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:17:48.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:17:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 msgr2=0x7fed800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 0x7fed800a3770 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0060b0 tx=0x7fed8c004b40 comp rx=0 tx=0).stop 2023-07-21T19:17:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 msgr2=0x7fed9419bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:48.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed9000e9c0 tx=0x7fed9000ecd0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 shutdown_connections 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fed800a12c0 0x7fed800a3770 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fed94105b10 0x7fed94195a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fed940ffb00 0x7fed9419bc80 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 --2- 172.21.15.5:0/3924119217 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fed941004b0 0x7fed9419c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.420+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 >> 172.21.15.5:0/3924119217 conn(0x7fed940fb600 msgr2=0x7fed940fbce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.421+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 shutdown_connections 2023-07-21T19:17:48.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:17:48.421+0000 7fed9c1ff700 1 -- 172.21.15.5:0/3924119217 wait complete. 2023-07-21T19:17:48.916 INFO:teuthology.orchestra.run.smithi005.stderr:38300000 of 50000000 tuples (76%) done (elapsed 586.49 s, remaining 179.16 s) 2023-07-21T19:17:49.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:49 smithi101 ceph-mon[128136]: pgmap v317: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 25 MiB/s wr, 79 op/s 2023-07-21T19:17:49.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:49 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:49 smithi005 ceph-mon[126460]: pgmap v317: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 25 MiB/s wr, 79 op/s 2023-07-21T19:17:49.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:49 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:49.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:49 smithi156 ceph-mon[128316]: pgmap v317: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 25 MiB/s wr, 79 op/s 2023-07-21T19:17:49.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:49 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:50.229 INFO:teuthology.orchestra.run.smithi005.stderr:38400000 of 50000000 tuples (76%) done (elapsed 587.80 s, remaining 177.56 s) 2023-07-21T19:17:51.664 INFO:teuthology.orchestra.run.smithi005.stderr:38500000 of 50000000 tuples (77%) done (elapsed 589.23 s, remaining 176.00 s) 2023-07-21T19:17:51.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:51 smithi101 ceph-mon[128136]: pgmap v318: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 21 MiB/s wr, 62 op/s 2023-07-21T19:17:51.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:51 smithi005 ceph-mon[126460]: pgmap v318: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 21 MiB/s wr, 62 op/s 2023-07-21T19:17:51.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:51 smithi156 ceph-mon[128316]: pgmap v318: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 21 MiB/s wr, 62 op/s 2023-07-21T19:17:53.806 INFO:teuthology.orchestra.run.smithi005.stderr:38600000 of 50000000 tuples (77%) done (elapsed 591.38 s, remaining 174.65 s) 2023-07-21T19:17:53.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:53 smithi101 ceph-mon[128136]: pgmap v319: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 72 op/s 2023-07-21T19:17:53.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:53 smithi005 ceph-mon[126460]: pgmap v319: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 72 op/s 2023-07-21T19:17:53.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:53 smithi156 ceph-mon[128316]: pgmap v319: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 100 KiB/s rd, 28 MiB/s wr, 72 op/s 2023-07-21T19:17:55.137 INFO:teuthology.orchestra.run.smithi005.stderr:38700000 of 50000000 tuples (77%) done (elapsed 592.71 s, remaining 173.06 s) 2023-07-21T19:17:55.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:55 smithi101 ceph-mon[128136]: pgmap v320: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:17:55.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:55 smithi005 ceph-mon[126460]: pgmap v320: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:17:55.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:55 smithi156 ceph-mon[128316]: pgmap v320: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 95 KiB/s rd, 22 MiB/s wr, 57 op/s 2023-07-21T19:17:56.478 INFO:teuthology.orchestra.run.smithi005.stderr:38800000 of 50000000 tuples (77%) done (elapsed 594.05 s, remaining 171.48 s) 2023-07-21T19:17:56.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:17:56.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:17:56.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:17:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:17:56.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:17:56.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:17:56.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:17:56.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:17:56.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:17:57.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:57 smithi101 ceph-mon[128136]: pgmap v321: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 20 MiB/s wr, 49 op/s 2023-07-21T19:17:57.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:57 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:57.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:57 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:17:57.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:57 smithi005 ceph-mon[126460]: pgmap v321: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 20 MiB/s wr, 49 op/s 2023-07-21T19:17:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:57 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:57.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:57 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:17:57.913 INFO:teuthology.orchestra.run.smithi005.stderr:38900000 of 50000000 tuples (77%) done (elapsed 595.48 s, remaining 169.92 s) 2023-07-21T19:17:57.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:57 smithi156 ceph-mon[128316]: pgmap v321: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 20 MiB/s wr, 49 op/s 2023-07-21T19:17:57.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:57 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:17:57.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:57 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:17:58.127 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:17:58.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.397+0000 7f05353ff700 1 -- 172.21.15.5:0/3153639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530100480 msgr2=0x7f053010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.397+0000 7f05353ff700 1 --2- 172.21.15.5:0/3153639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530100480 0x7f053010c910 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f052800b0a0 tx=0x7f052800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 -- 172.21.15.5:0/3153639170 shutdown_connections 2023-07-21T19:17:58.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 --2- 172.21.15.5:0/3153639170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530100480 0x7f053010c910 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 --2- 172.21.15.5:0/3153639170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0530105ae0 0x7f0530105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 --2- 172.21.15.5:0/3153639170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f05300ffad0 0x7f05300fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 -- 172.21.15.5:0/3153639170 >> 172.21.15.5:0/3153639170 conn(0x7f05300fb5f0 msgr2=0x7f05300fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 -- 172.21.15.5:0/3153639170 shutdown_connections 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.398+0000 7f05353ff700 1 -- 172.21.15.5:0/3153639170 wait complete. 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.399+0000 7f05353ff700 1 Processor -- start 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.399+0000 7f05353ff700 1 -- start start 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05300ffad0 0x7f0530197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530105ae0 0x7f053019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053019cd90 con 0x7f0530105ae0 2023-07-21T19:17:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053019cf00 con 0x7f05300ffad0 2023-07-21T19:17:58.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f05353ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f053019d0c0 con 0x7f0530100480 2023-07-21T19:17:58.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f052effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05300ffad0 0x7f0530197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58426/0 (socket says 172.21.15.5:58426) 2023-07-21T19:17:58.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f052e7fc700 1 -- 172.21.15.5:0/1862051237 learned_addr learned my addr 172.21.15.5:0/1862051237 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.400+0000 7f052f7fe700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530105ae0 0x7f053019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 -- 172.21.15.5:0/1862051237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05300ffad0 msgr2=0x7f0530197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05300ffad0 0x7f0530197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 -- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530105ae0 msgr2=0x7f053019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530105ae0 0x7f053019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 -- 172.21.15.5:0/1862051237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0528009d00 con 0x7f0530100480 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f052400e9a0 tx=0x7f052400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f0517fff700 1 -- 172.21.15.5:0/1862051237 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524009790 con 0x7f0530100480 2023-07-21T19:17:58.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f053019d3a0 con 0x7f0530100480 2023-07-21T19:17:58.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f0517fff700 1 -- 172.21.15.5:0/1862051237 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0524010430 con 0x7f0530100480 2023-07-21T19:17:58.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.401+0000 7f0517fff700 1 -- 172.21.15.5:0/1862051237 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052400cea0 con 0x7f0530100480 2023-07-21T19:17:58.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.402+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05301a4be0 con 0x7f0530100480 2023-07-21T19:17:58.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.405+0000 7f0517fff700 1 -- 172.21.15.5:0/1862051237 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f05240105a0 con 0x7f0530100480 2023-07-21T19:17:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.407+0000 7f0548d03b80 1 Processor -- start 2023-07-21T19:17:58.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.407+0000 7f0548d03b80 1 -- start start 2023-07-21T19:17:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.407+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c4546a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c45fad0 0x55ce5c461eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0517fff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 0x7f05180a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0548d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c35acb0 con 0x55ce5c45fad0 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0548d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c3ada40 con 0x55ce5c4542d0 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0548d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c3a6ed0 con 0x55ce5c454be0 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f0517fff700 1 -- 172.21.15.5:0/1862051237 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0524014070 con 0x7f0530100480 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c4546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58436/0 (socket says 172.21.15.5:58436) 2023-07-21T19:17:58.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052effd700 1 -- 172.21.15.5:0/1162744531 learned_addr learned my addr 172.21.15.5:0/1162744531 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052f7fe700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 0x7f05180a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.408+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c45fad0 0x55ce5c461eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 -- 172.21.15.5:0/1162744531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 msgr2=0x55ce5c4546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c4546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 -- 172.21.15.5:0/1162744531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c45fad0 msgr2=0x55ce5c461eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c45fad0 0x55ce5c461eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 -- 172.21.15.5:0/1162744531 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce5c1d4de0 con 0x55ce5c454be0 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052effd700 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0520007c70 tx=0x7f0520007f80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6801cfc47dd94f3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f052f7fe700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 0x7f05180a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05280062a0 tx=0x7f0528009c50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.409+0000 7f05157fa700 1 -- 172.21.15.5:0/1162744531 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004220 con 0x55ce5c454be0 2023-07-21T19:17:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.410+0000 7f05157fa700 1 -- 172.21.15.5:0/1162744531 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0520015070 con 0x55ce5c454be0 2023-07-21T19:17:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.410+0000 7f05157fa700 1 -- 172.21.15.5:0/1162744531 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520003b30 con 0x55ce5c454be0 2023-07-21T19:17:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 -- 172.21.15.5:0/1162744531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 msgr2=0x55ce5c45f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f0520007c70 tx=0x7f0520007f80 comp rx=0 tx=0).stop 2023-07-21T19:17:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 -- 172.21.15.5:0/1162744531 shutdown_connections 2023-07-21T19:17:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c45fad0 0x55ce5c461eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c4546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1162744531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c454be0 0x55ce5c45f590 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 -- 172.21.15.5:0/1162744531 >> 172.21.15.5:0/1162744531 conn(0x55ce5c44ada0 msgr2=0x55ce5c4506a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 -- 172.21.15.5:0/1162744531 shutdown_connections 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.411+0000 7f0548d03b80 1 -- 172.21.15.5:0/1162744531 wait complete. 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.412+0000 7f0548d03b80 1 -- start start 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.412+0000 7f0548d03b80 1 Processor -- start 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c45c760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c346ac0 con 0x55ce5c454be0 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c3467c0 con 0x55ce5c4542d0 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c22a920 con 0x55ce5c45fad0 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f0548d03b80 10 client.0 _ll_register_callbacks cb 0x55ce5c344760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:58.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c45c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c45c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41924/0 (socket says 172.21.15.5:41924) 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f052e7fc700 1 -- 172.21.15.5:0/474942818 learned_addr learned my addr 172.21.15.5:0/474942818 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.413+0000 7f052effd700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052f7fe700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 -- 172.21.15.5:0/474942818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 msgr2=0x55ce5c45ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 -- 172.21.15.5:0/474942818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 msgr2=0x55ce5c45c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c45c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 -- 172.21.15.5:0/474942818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05200078a0 con 0x55ce5c4542d0 2023-07-21T19:17:58.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f052effd700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45f550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0520007c70 tx=0x7f0520010a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fb87966d173916f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f0512ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.414+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520004220 con 0x55ce5c4542d0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.415+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0520004380 con 0x55ce5c4542d0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.415+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ce5c3004f0 con 0x55ce5c4542d0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.415+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0520004540 con 0x55ce5c4542d0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.415+0000 7f0548d03b80 10 client.27218 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.415+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ce5c1bdc60 con 0x55ce5c4542d0 2023-07-21T19:17:58.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ce5c3004f0 con 0x55ce5c4542d0 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f052002a070 con 0x55ce5c4542d0 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 10 client.27218 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 10 client.27218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 20 client.27218 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4779a0 0x55ce5c479db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 4 client.27218 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:58.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.416+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ce5c39feb0 con 0x55ce5c4779a0 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.417+0000 7f052e7fc700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4779a0 0x55ce5c479db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.418+0000 7f052e7fc700 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4779a0 0x55ce5c479db0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba4b2760cee2d152 server_cookie=7157ed0dc1ba6c8e in_seq=0 out_seq=0 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.418+0000 7f0512ff5700 10 client.27218 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.418+0000 7f0512ff5700 1 -- 172.21.15.5:0/474942818 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ce5c39feb0 con 0x55ce5c4779a0 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.418+0000 7f0512ff5700 10 client.27218 handle_command_reply: tid=0 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 client.27218 shutdown 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 msgr2=0x55ce5c45f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45f550 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0520007c70 tx=0x7f0520010a10 comp rx=0 tx=0).stop 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 shutdown_connections 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45f550 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4779a0 0x55ce5c479db0 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 --2- 172.21.15.5:0/474942818 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c45c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0512ff5700 0 client.27218 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 >> 172.21.15.5:0/474942818 conn(0x55ce5c44ada0 msgr2=0x55ce5c450290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 shutdown_connections 2023-07-21T19:17:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 1 -- 172.21.15.5:0/474942818 wait complete. 2023-07-21T19:17:58.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.419+0000 7f0548d03b80 20 client.27218 trim_cache size 0 max 16384 2023-07-21T19:17:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 Processor -- start 2023-07-21T19:17:58.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 -- start start 2023-07-21T19:17:58.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c4542d0 0x55ce5c45b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c4622b0 con 0x55ce5c4542d0 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c455540 con 0x55ce5c454be0 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.440+0000 7f0548d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c454fe0 con 0x55ce5c45fad0 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46850/0 (socket says 172.21.15.5:46850) 2023-07-21T19:17:58.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 -- 172.21.15.5:0/4262876432 learned_addr learned my addr 172.21.15.5:0/4262876432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052effd700 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c4542d0 0x55ce5c45b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052f7fe700 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 -- 172.21.15.5:0/4262876432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 msgr2=0x55ce5c45f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 -- 172.21.15.5:0/4262876432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c4542d0 msgr2=0x55ce5c45b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c4542d0 0x55ce5c45b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.441+0000 7f052e7fc700 1 -- 172.21.15.5:0/4262876432 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ce5c1fb250 con 0x55ce5c454be0 2023-07-21T19:17:58.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.442+0000 7f052e7fc700 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0524099f80 tx=0x7f0524010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24c47aad2b92a605 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.442+0000 7f0515ffb700 1 -- 172.21.15.5:0/4262876432 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052409e070 con 0x55ce5c454be0 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.442+0000 7f0515ffb700 1 -- 172.21.15.5:0/4262876432 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f05240a2490 con 0x55ce5c454be0 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.442+0000 7f0515ffb700 1 -- 172.21.15.5:0/4262876432 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0524099530 con 0x55ce5c454be0 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- 172.21.15.5:0/4262876432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 msgr2=0x55ce5c46ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0524099f80 tx=0x7f0524010bf0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- 172.21.15.5:0/4262876432 shutdown_connections 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c45f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c4542d0 0x55ce5c45b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 --2- 172.21.15.5:0/4262876432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c454be0 0x55ce5c46ecf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- 172.21.15.5:0/4262876432 >> 172.21.15.5:0/4262876432 conn(0x55ce5c44ada0 msgr2=0x55ce5c473f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- 172.21.15.5:0/4262876432 shutdown_connections 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- 172.21.15.5:0/4262876432 wait complete. 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 -- start start 2023-07-21T19:17:58.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.443+0000 7f0548d03b80 1 Processor -- start 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c458f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c4594d0 con 0x55ce5c454be0 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c459640 con 0x55ce5c4542d0 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ce5c459800 con 0x55ce5c45fad0 2023-07-21T19:17:58.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f0548d03b80 10 client.0 _ll_register_callbacks cb 0x7f0524096380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41948/0 (socket says 172.21.15.5:41948) 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052e7fc700 1 -- 172.21.15.5:0/1743133503 learned_addr learned my addr 172.21.15.5:0/1743133503 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46852/0 (socket says 172.21.15.5:46852) 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.444+0000 7f052f7fe700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c458f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 -- 172.21.15.5:0/1743133503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 msgr2=0x55ce5c458f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c458f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 -- 172.21.15.5:0/1743133503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 msgr2=0x55ce5c4589e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 -- 172.21.15.5:0/1743133503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05301a4be0 con 0x55ce5c4542d0 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f052effd700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0520007b90 tx=0x7f0520004910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ccfdb3df14647e38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.445+0000 7f0512ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:58.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0520039070 con 0x55ce5c4542d0 2023-07-21T19:17:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0520000e40 con 0x55ce5c4542d0 2023-07-21T19:17:58.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0520015070 con 0x55ce5c4542d0 2023-07-21T19:17:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0520010b30 con 0x55ce5c4542d0 2023-07-21T19:17:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0548d03b80 10 client.27224 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:58.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.446+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ce5c1d32b0 con 0x55ce5c4542d0 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0520015070 con 0x55ce5c4542d0 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f052003c020 con 0x55ce5c4542d0 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 10 client.27224 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 10 client.27224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 20 client.27224 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4cca40 0x55ce5c4cee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 4 client.27224 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ce5c362050 con 0x55ce5c4cca40 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.447+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4cca40 0x55ce5c4cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f052e7fc700 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4cca40 0x55ce5c4cee20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba18f6ce9185cf45 server_cookie=a33f1f877d64284f in_seq=0 out_seq=0 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0512ff5700 10 client.27224 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0512ff5700 1 -- 172.21.15.5:0/1743133503 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55ce5c362050 con 0x55ce5c4cca40 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0512ff5700 10 client.27224 handle_command_reply: tid=0 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 client.27224 shutdown 2023-07-21T19:17:58.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 msgr2=0x55ce5c45a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0520007b90 tx=0x7f0520004910 comp rx=0 tx=0).stop 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 shutdown_connections 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ce5c45fad0 0x55ce5c458f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ce5c4542d0 0x55ce5c45a180 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ce5c4cca40 0x55ce5c4cee20 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 --2- 172.21.15.5:0/1743133503 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ce5c454be0 0x55ce5c4589e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0512ff5700 0 client.27224 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.448+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 >> 172.21.15.5:0/1743133503 conn(0x55ce5c44ada0 msgr2=0x55ce5c45c000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.449+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 shutdown_connections 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.449+0000 7f0548d03b80 1 -- 172.21.15.5:0/1743133503 wait complete. 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.449+0000 7f0548d03b80 20 client.27224 trim_cache size 0 max 16384 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:17:58.453 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:17:58.454 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:17:58.454 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:17:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.449+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 msgr2=0x7f05180a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 0x7f05180a3770 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f05280062a0 tx=0x7f0528009c50 comp rx=0 tx=0).stop 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 msgr2=0x7f05301982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f052400e9a0 tx=0x7f052400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 shutdown_connections 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f05180a12c0 0x7f05180a3770 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0530105ae0 0x7f053019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f05300ffad0 0x7f0530197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 --2- 172.21.15.5:0/1862051237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0530100480 0x7f05301982f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 >> 172.21.15.5:0/1862051237 conn(0x7f05300fb5f0 msgr2=0x7f053010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 shutdown_connections 2023-07-21T19:17:58.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.450+0000 7f05353ff700 1 -- 172.21.15.5:0/1862051237 wait complete. 2023-07-21T19:17:58.475 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:17:58.475 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.731+0000 7ff0cfbff700 1 -- 172.21.15.5:0/344007046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c80ff1c0 msgr2=0x7ff0c810be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.731+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/344007046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c810be90 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7ff0b80098f0 tx=0x7ff0b8009c00 comp rx=0 tx=0).stop 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 -- 172.21.15.5:0/344007046 shutdown_connections 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/344007046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c810c3d0 0x7ff0c810e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/344007046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80fe820 0x7ff0c80febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/344007046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c810be90 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 -- 172.21.15.5:0/344007046 >> 172.21.15.5:0/344007046 conn(0x7ff0c80fa490 msgr2=0x7ff0c80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 -- 172.21.15.5:0/344007046 shutdown_connections 2023-07-21T19:17:58.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.732+0000 7ff0cfbff700 1 -- 172.21.15.5:0/344007046 wait complete. 2023-07-21T19:17:58.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 Processor -- start 2023-07-21T19:17:58.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 -- start start 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 0x7ff0c819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819cd80 con 0x7ff0c80fe820 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819cef0 con 0x7ff0c80ff1c0 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.733+0000 7ff0cfbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c819d0b0 con 0x7ff0c810c3d0 2023-07-21T19:17:58.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41974/0 (socket says 172.21.15.5:41974) 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 -- 172.21.15.5:0/1113541303 learned_addr learned my addr 172.21.15.5:0/1113541303 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0ce19c700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 0x7ff0c819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd19a700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 -- 172.21.15.5:0/1113541303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 msgr2=0x7ff0c819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 0x7ff0c819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 -- 172.21.15.5:0/1113541303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 msgr2=0x7ff0c81982e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:58.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c81982e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 -- 172.21.15.5:0/1113541303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c40095f0 con 0x7ff0c80fe820 2023-07-21T19:17:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0ce19c700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 0x7ff0c819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd19a700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:58.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.734+0000 7ff0cd99b700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400ec10 tx=0x7ff0c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.735+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4009a10 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.735+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0c400cde0 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.735+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c0009c40 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.735+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4010430 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.735+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b8009520 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.736+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c81075f0 con 0x7ff0c80fe820 2023-07-21T19:17:58.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.739+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff0c4004000 con 0x7ff0c80fe820 2023-07-21T19:17:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.740+0000 7ff0beffd700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 0x7ff0b40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.740+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff0c4014070 con 0x7ff0c80fe820 2023-07-21T19:17:58.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.740+0000 7ff0cd19a700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 0x7ff0b40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:58.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.741+0000 7ff0cd19a700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 0x7ff0b40a3830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0c81993b0 tx=0x7ff0b8019040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:58.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:58.745+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7ff0c4096690 con 0x7ff0c80fe820 2023-07-21T19:17:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.072+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff0c8067300 con 0x7ff0c80fe820 2023-07-21T19:17:59.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.073+0000 7ff0beffd700 1 -- 172.21.15.5:0/1113541303 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10836 (secure 0 0 0) 0x7ff0c4099cb0 con 0x7ff0c80fe820 2023-07-21T19:17:59.071 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:17:59.071 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:17:56.540961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:17:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 msgr2=0x7ff0b40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 0x7ff0b40a3830 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7ff0c81993b0 tx=0x7ff0b8019040 comp rx=0 tx=0).stop 2023-07-21T19:17:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 msgr2=0x7ff0c8197da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7ff0c400ec10 tx=0x7ff0c400c2d0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 shutdown_connections 2023-07-21T19:17:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0c810c3d0 0x7ff0c819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0b40a1380 0x7ff0b40a3830 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0c80fe820 0x7ff0c8197da0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 --2- 172.21.15.5:0/1113541303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0c80ff1c0 0x7ff0c81982e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.077+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 >> 172.21.15.5:0/1113541303 conn(0x7ff0c80fa490 msgr2=0x7ff0c810e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.078+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 shutdown_connections 2023-07-21T19:17:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.078+0000 7ff0cfbff700 1 -- 172.21.15.5:0/1113541303 wait complete. 2023-07-21T19:17:59.078 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2023-07-21T19:17:59.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:17:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 -- 172.21.15.5:0/2712294820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 msgr2=0x7f50cc105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2712294820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc105ec0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f50c0009990 tx=0x7f50c0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 -- 172.21.15.5:0/2712294820 shutdown_connections 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2712294820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc100490 0x7f50cc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2712294820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc105ec0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2712294820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc0ffae0 0x7f50cc0fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 -- 172.21.15.5:0/2712294820 >> 172.21.15.5:0/2712294820 conn(0x7f50cc0fb5e0 msgr2=0x7f50cc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 -- 172.21.15.5:0/2712294820 shutdown_connections 2023-07-21T19:17:59.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.355+0000 7f50d2dff700 1 -- 172.21.15.5:0/2712294820 wait complete. 2023-07-21T19:17:59.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.356+0000 7f50d2dff700 1 Processor -- start 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.356+0000 7f50d2dff700 1 -- start start 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc100490 0x7f50cc198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc19c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19cd00 con 0x7f50cc105af0 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19ce70 con 0x7f50cc100490 2023-07-21T19:17:59.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d2dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50cc19d030 con 0x7f50cc0ffae0 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58494/0 (socket says 172.21.15.5:58494) 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2190304084 learned_addr learned my addr 172.21.15.5:0/2190304084 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50d139c700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc19c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.357+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc100490 0x7f50cc198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2190304084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc100490 msgr2=0x7f50cc198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc100490 0x7f50cc198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 msgr2=0x7f50cc19c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc19c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2190304084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c00095f0 con 0x7f50cc0ffae0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005740 tx=0x7f50c0005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50c9ffb700 1 -- 172.21.15.5:0/2190304084 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c001d070 con 0x7f50cc0ffae0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50c9ffb700 1 -- 172.21.15.5:0/2190304084 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50c0022470 con 0x7f50cc0ffae0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50cc19d2b0 con 0x7f50cc0ffae0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.358+0000 7f50c9ffb700 1 -- 172.21.15.5:0/2190304084 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c0017650 con 0x7f50cc0ffae0 2023-07-21T19:17:59.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.359+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50cc1a4b50 con 0x7f50cc0ffae0 2023-07-21T19:17:59.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.362+0000 7f50c9ffb700 1 -- 172.21.15.5:0/2190304084 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f50c00225e0 con 0x7f50cc0ffae0 2023-07-21T19:17:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.364+0000 7f50c9ffb700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 0x7f50bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.364+0000 7f50e67d2b80 1 Processor -- start 2023-07-21T19:17:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 -- start start 2023-07-21T19:17:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50c9ffb700 1 -- 172.21.15.5:0/2190304084 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50c00cdff0 con 0x7f50cc0ffae0 2023-07-21T19:17:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 0x7f50bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff726a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff7feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfe78cb0 con 0x5597dff7dad0 2023-07-21T19:17:59.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfecba40 con 0x5597dff72be0 2023-07-21T19:17:59.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfec4ed0 con 0x5597dff722d0 2023-07-21T19:17:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff726a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.365+0000 7f50d139c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff7feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff726a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58510/0 (socket says 172.21.15.5:58510) 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50d0b9b700 1 -- 172.21.15.5:0/4186369685 learned_addr learned my addr 172.21.15.5:0/4186369685 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 -- 172.21.15.5:0/4186369685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 msgr2=0x5597dff726a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff726a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 -- 172.21.15.5:0/4186369685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 msgr2=0x5597dff7feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff7feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 -- 172.21.15.5:0/4186369685 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597dfcf2de0 con 0x5597dff72be0 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50d139c700 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff7feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.366+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 0x7f50bc0a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f50cc199330 tx=0x7f50b8011720 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.367+0000 7f50cbfff700 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50b8011b20 tx=0x7f50b801e000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f884a2cd35aa08f5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.367+0000 7f50b6ffd700 1 -- 172.21.15.5:0/4186369685 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b801ed10 con 0x5597dff72be0 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.367+0000 7f50b6ffd700 1 -- 172.21.15.5:0/4186369685 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50b801ee70 con 0x5597dff72be0 2023-07-21T19:17:59.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.367+0000 7f50b6ffd700 1 -- 172.21.15.5:0/4186369685 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b800e5d0 con 0x5597dff72be0 2023-07-21T19:17:59.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 -- 172.21.15.5:0/4186369685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 msgr2=0x5597dff7d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f50b8011b20 tx=0x7f50b801e000 comp rx=0 tx=0).stop 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 -- 172.21.15.5:0/4186369685 shutdown_connections 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff7feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff726a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/4186369685 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 -- 172.21.15.5:0/4186369685 >> 172.21.15.5:0/4186369685 conn(0x5597dff68da0 msgr2=0x5597dff6e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 -- 172.21.15.5:0/4186369685 shutdown_connections 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.368+0000 7f50e67d2b80 1 -- 172.21.15.5:0/4186369685 wait complete. 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.369+0000 7f50e67d2b80 1 -- start start 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.369+0000 7f50e67d2b80 1 Processor -- start 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff72be0 0x5597dff7a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff7aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfe64ac0 con 0x5597dff72be0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfe647c0 con 0x5597dff7dad0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dfd48920 con 0x5597dff722d0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50e67d2b80 10 client.0 _ll_register_callbacks cb 0x5597dfe62760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff72be0 0x5597dff7a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58512/0 (socket says 172.21.15.5:58512) 2023-07-21T19:17:59.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2947866924 learned_addr learned my addr 172.21.15.5:0/2947866924 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d139c700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff7aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2947866924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 msgr2=0x5597dff7aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff7aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2947866924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff72be0 msgr2=0x5597dff7a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff72be0 0x5597dff7a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d0b9b700 1 -- 172.21.15.5:0/2947866924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4011230 con 0x5597dff722d0 2023-07-21T19:17:59.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.370+0000 7f50d139c700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff7aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50c0022fd0 tx=0x7f50c009acd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8dcfa4273f07c547 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50abfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:17:59.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50c009b570 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50c009f140 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5597dfe1e4f0 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50c0022ad0 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50e67d2b80 10 client.27169 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.371+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f50b8009de0 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50c00c9020 con 0x5597dff722d0 2023-07-21T19:17:59.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50c00a3040 con 0x5597dff722d0 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 10 client.27169 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 10 client.27169 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 20 client.27169 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dff8d090 0x5597dff8f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 4 client.27169 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.372+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5597dfebdeb0 con 0x5597dff8d090 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.373+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dff8d090 0x5597dff8f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.374+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dff8d090 0x5597dff8f4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=eec7c66bc04c24a3 server_cookie=cd750853c71a85df in_seq=0 out_seq=0 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.374+0000 7f50abfff700 10 client.27169 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.374+0000 7f50abfff700 1 -- 172.21.15.5:0/2947866924 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5597dfebdeb0 con 0x5597dff8d090 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.374+0000 7f50abfff700 10 client.27169 handle_command_reply: tid=0 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.374+0000 7f50e67d2b80 1 client.27169 shutdown 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 msgr2=0x5597dff7d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f50c0022fd0 tx=0x7f50c009acd0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 shutdown_connections 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dff8d090 0x5597dff8f4a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff7aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7d540 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50abfff700 0 client.27169 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2947866924 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff72be0 0x5597dff7a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 >> 172.21.15.5:0/2947866924 conn(0x5597dff68da0 msgr2=0x5597dff6e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 shutdown_connections 2023-07-21T19:17:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2947866924 wait complete. 2023-07-21T19:17:59.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.375+0000 7f50e67d2b80 20 client.27169 trim_cache size 0 max 16384 2023-07-21T19:17:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 Processor -- start 2023-07-21T19:17:59.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 -- start start 2023-07-21T19:17:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff75af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff78de0 con 0x5597dff7dad0 2023-07-21T19:17:59.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff75280 con 0x5597dff72be0 2023-07-21T19:17:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.392+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff8f6c0 con 0x5597dff722d0 2023-07-21T19:17:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50d139c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff75af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46934/0 (socket says 172.21.15.5:46934) 2023-07-21T19:17:59.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 -- 172.21.15.5:0/2494486040 learned_addr learned my addr 172.21.15.5:0/2494486040 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 -- 172.21.15.5:0/2494486040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 msgr2=0x5597dff7ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 -- 172.21.15.5:0/2494486040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 msgr2=0x5597dff75af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff75af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50cbfff700 1 -- 172.21.15.5:0/2494486040 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5597dfcdbc60 con 0x5597dff72be0 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.393+0000 7f50d139c700 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff75af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.394+0000 7f50cbfff700 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009740 tx=0x7f50b800fce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61fdbf2cac2eb51f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.394+0000 7f50b77fe700 1 -- 172.21.15.5:0/2494486040 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b802c040 con 0x5597dff72be0 2023-07-21T19:17:59.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.394+0000 7f50b77fe700 1 -- 172.21.15.5:0/2494486040 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50b802d040 con 0x5597dff72be0 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.394+0000 7f50b77fe700 1 -- 172.21.15.5:0/2494486040 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8033cc0 con 0x5597dff72be0 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2494486040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 msgr2=0x5597dff7d0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009740 tx=0x7f50b800fce0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2494486040 shutdown_connections 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff7dad0 0x5597dff75af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff722d0 0x5597dff7ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/2494486040 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff72be0 0x5597dff7d0e0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2494486040 >> 172.21.15.5:0/2494486040 conn(0x5597dff68da0 msgr2=0x5597dff6ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2494486040 shutdown_connections 2023-07-21T19:17:59.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.395+0000 7f50e67d2b80 1 -- 172.21.15.5:0/2494486040 wait complete. 2023-07-21T19:17:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.396+0000 7f50e67d2b80 1 -- start start 2023-07-21T19:17:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.396+0000 7f50e67d2b80 1 Processor -- start 2023-07-21T19:17:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 0x5597dff84140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 0x5597dff84680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff82690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff82d60 con 0x5597dff722d0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff82ed0 con 0x5597dff7dad0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5597dff83090 con 0x5597dff72be0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50e67d2b80 10 client.0 _ll_register_callbacks cb 0x7f50c009aee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 0x5597dff84140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 0x5597dff84680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50d0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 0x5597dff84140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42026/0 (socket says 172.21.15.5:42026) 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50d0b9b700 1 -- 172.21.15.5:0/1403496530 learned_addr learned my addr 172.21.15.5:0/1403496530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.397+0000 7f50d139c700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff82690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 -- 172.21.15.5:0/1403496530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 msgr2=0x5597dff82690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff82690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 -- 172.21.15.5:0/1403496530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 msgr2=0x5597dff84140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 0x5597dff84140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 -- 172.21.15.5:0/1403496530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c4009220 con 0x5597dff72be0 2023-07-21T19:17:59.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50d139c700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50cbfff700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 0x5597dff84680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009740 tx=0x7f50b80096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d79a23e5b2f08021 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50abfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.398+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b8033420 con 0x5597dff72be0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50b801ee70 con 0x5597dff72be0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50b8022070 con 0x5597dff72be0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50b8033580 con 0x5597dff72be0 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50e67d2b80 10 client.27172 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:59.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5597dfcf2de0 con 0x5597dff72be0 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50b801ee70 con 0x5597dff72be0 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.399+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50b800e800 con 0x5597dff72be0 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 10 client.27172 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 10 client.27172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 20 client.27172 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dffe1e60 0x5597dffe4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 4 client.27172 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:17:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5597dfe80050 con 0x5597dffe1e60 2023-07-21T19:17:59.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.400+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dffe1e60 0x5597dffe4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.401+0000 7f50d0b9b700 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dffe1e60 0x5597dffe4270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=42142383e7ffd8e3 server_cookie=c51a20ad584c678 in_seq=0 out_seq=0 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.401+0000 7f50abfff700 10 client.27172 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.401+0000 7f50abfff700 1 -- 172.21.15.5:0/1403496530 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5597dfe80050 con 0x5597dffe1e60 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.401+0000 7f50abfff700 10 client.27172 handle_command_reply: tid=0 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.401+0000 7f50e67d2b80 1 client.27172 shutdown 2023-07-21T19:17:59.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 msgr2=0x5597dff84680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 0x5597dff84680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f50b8009740 tx=0x7f50b80096d0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 shutdown_connections 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5597dffe1e60 0x5597dffe4270 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5597dff7dad0 0x5597dff82690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50abfff700 0 client.27172 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5597dff722d0 0x5597dff84140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 --2- 172.21.15.5:0/1403496530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5597dff72be0 0x5597dff84680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 >> 172.21.15.5:0/1403496530 conn(0x5597dff68da0 msgr2=0x5597dff75af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 shutdown_connections 2023-07-21T19:17:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 1 -- 172.21.15.5:0/1403496530 wait complete. 2023-07-21T19:17:59.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.402+0000 7f50e67d2b80 20 client.27172 trim_cache size 0 max 16384 2023-07-21T19:17:59.405 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:17:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 msgr2=0x7f50bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 0x7f50bc0a37f0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f50cc199330 tx=0x7f50b8011720 comp rx=0 tx=0).stop 2023-07-21T19:17:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 msgr2=0x7f50cc197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005740 tx=0x7f50c0005770 comp rx=0 tx=0).stop 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 shutdown_connections 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50bc0a1340 0x7f50bc0a37f0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50cc105af0 0x7f50cc19c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50cc0ffae0 0x7f50cc197d20 secure :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f50c0005740 tx=0x7f50c0005770 comp rx=0 tx=0).stop 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 --2- 172.21.15.5:0/2190304084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50cc100490 0x7f50cc198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.403+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 >> 172.21.15.5:0/2190304084 conn(0x7f50cc0fb5e0 msgr2=0x7f50cc10adc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.404+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 shutdown_connections 2023-07-21T19:17:59.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.404+0000 7f50d2dff700 1 -- 172.21.15.5:0/2190304084 wait complete. 2023-07-21T19:17:59.425 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:17:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:39000000 of 50000000 tuples (78%) done (elapsed 597.03 s, remaining 168.39 s) 2023-07-21T19:17:59.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 -- 172.21.15.5:0/1201539310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc1015b0 msgr2=0x7f00fc109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 --2- 172.21.15.5:0/1201539310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc1015b0 0x7f00fc109aa0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009990 tx=0x7f00ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 -- 172.21.15.5:0/1201539310 shutdown_connections 2023-07-21T19:17:59.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 --2- 172.21.15.5:0/1201539310 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 --2- 172.21.15.5:0/1201539310 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc100ca0 0x7f00fc101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 --2- 172.21.15.5:0/1201539310 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc1015b0 0x7f00fc109aa0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 -- 172.21.15.5:0/1201539310 >> 172.21.15.5:0/1201539310 conn(0x7f00fc0fa4b0 msgr2=0x7f00fc0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 -- 172.21.15.5:0/1201539310 shutdown_connections 2023-07-21T19:17:59.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.691+0000 7f01043ff700 1 -- 172.21.15.5:0/1201539310 wait complete. 2023-07-21T19:17:59.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.693+0000 7f01043ff700 1 Processor -- start 2023-07-21T19:17:59.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.693+0000 7f01043ff700 1 -- start start 2023-07-21T19:17:59.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc1015b0 0x7f00fc0fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc1a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ff660 con 0x7f00fc1015b0 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc1a42f0 con 0x7f00fc10a070 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f01043ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc1a4430 con 0x7f00fc100ca0 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f010219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f010199a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc1015b0 0x7f00fc0fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f010219b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58552/0 (socket says 172.21.15.5:58552) 2023-07-21T19:17:59.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.694+0000 7f010219b700 1 -- 172.21.15.5:0/77135153 learned_addr learned my addr 172.21.15.5:0/77135153 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010299c700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc1a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 -- 172.21.15.5:0/77135153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 msgr2=0x7f00fc1a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc1a3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 -- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc1015b0 msgr2=0x7f00fc0fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc1015b0 0x7f00fc0fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 -- 172.21.15.5:0/77135153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ec0095f0 con 0x7f00fc100ca0 2023-07-21T19:17:59.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010299c700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc1a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:17:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f010219b700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9a0 tx=0x7f00f800ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f00f37fe700 1 -- 172.21.15.5:0/77135153 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8009790 con 0x7f00fc100ca0 2023-07-21T19:17:59.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00fc1a4690 con 0x7f00fc100ca0 2023-07-21T19:17:59.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f00f37fe700 1 -- 172.21.15.5:0/77135153 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00f800cbd0 con 0x7f00fc100ca0 2023-07-21T19:17:59.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f00f37fe700 1 -- 172.21.15.5:0/77135153 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f8018680 con 0x7f00fc100ca0 2023-07-21T19:17:59.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.695+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc1a4bc0 con 0x7f00fc100ca0 2023-07-21T19:17:59.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.700+0000 7f00f37fe700 1 -- 172.21.15.5:0/77135153 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f00f800cd40 con 0x7f00fc100ca0 2023-07-21T19:17:59.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.700+0000 7f0117c5db80 1 Processor -- start 2023-07-21T19:17:59.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 -- start start 2023-07-21T19:17:59.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb7b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fa76cb0 con 0x55b66fb70be0 2023-07-21T19:17:59.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fac9a40 con 0x55b66fb702d0 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.701+0000 7f0117c5db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fac2ed0 con 0x55b66fb7bad0 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010219b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb7b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010199a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46980/0 (socket says 172.21.15.5:46980) 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 -- 172.21.15.5:0/3299856417 learned_addr learned my addr 172.21.15.5:0/3299856417 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 -- 172.21.15.5:0/3299856417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 msgr2=0x55b66fb7deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 -- 172.21.15.5:0/3299856417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 msgr2=0x55b66fb7b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb7b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010299c700 1 -- 172.21.15.5:0/3299856417 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66f8f0de0 con 0x55b66fb702d0 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.702+0000 7f010219b700 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.703+0000 7f010299c700 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009ca0 tx=0x7f00f4007500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=49fdd289461c4918 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.703+0000 7f00f0bfe700 1 -- 172.21.15.5:0/3299856417 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4014400 con 0x55b66fb702d0 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.703+0000 7f00f0bfe700 1 -- 172.21.15.5:0/3299856417 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00f40149f0 con 0x55b66fb702d0 2023-07-21T19:17:59.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.703+0000 7f00f0bfe700 1 -- 172.21.15.5:0/3299856417 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4013960 con 0x55b66fb702d0 2023-07-21T19:17:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.703+0000 7f00f37fe700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 0x7f00e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f00f37fe700 1 -- 172.21.15.5:0/77135153 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00f8014070 con 0x7f00fc100ca0 2023-07-21T19:17:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 -- 172.21.15.5:0/3299856417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 msgr2=0x55b66fb706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009ca0 tx=0x7f00f4007500 comp rx=0 tx=0).stop 2023-07-21T19:17:59.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f010219b700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 0x7f00e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 -- 172.21.15.5:0/3299856417 shutdown_connections 2023-07-21T19:17:59.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb706a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 --2- 172.21.15.5:0/3299856417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb7b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.704+0000 7f0117c5db80 1 -- 172.21.15.5:0/3299856417 >> 172.21.15.5:0/3299856417 conn(0x55b66fb66da0 msgr2=0x55b66fb6c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.705+0000 7f0117c5db80 1 -- 172.21.15.5:0/3299856417 shutdown_connections 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.705+0000 7f0117c5db80 1 -- 172.21.15.5:0/3299856417 wait complete. 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.705+0000 7f010219b700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 0x7f00e80a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f00f8096590 tx=0x7f00f801f000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.705+0000 7f0117c5db80 1 -- start start 2023-07-21T19:17:59.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.705+0000 7f0117c5db80 1 Processor -- start 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb78770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb7bad0 0x55b66fb78da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fa62ac0 con 0x55b66fb70be0 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fa627c0 con 0x55b66fb7bad0 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66f946920 con 0x55b66fb702d0 2023-07-21T19:17:59.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f0117c5db80 10 client.0 _ll_register_callbacks cb 0x55b66fa60760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.706+0000 7f010299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58568/0 (socket says 172.21.15.5:58568) 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 -- 172.21.15.5:0/1167387935 learned_addr learned my addr 172.21.15.5:0/1167387935 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb7bad0 0x55b66fb78da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010199a700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb78770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 -- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb7bad0 msgr2=0x55b66fb78da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb7bad0 0x55b66fb78da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 -- 172.21.15.5:0/1167387935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 msgr2=0x55b66fb78770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb78770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 -- 172.21.15.5:0/1167387935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f4009900 con 0x55b66fb702d0 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f010299c700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009890 tx=0x7f00f40097e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7549bc032287a26b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.707+0000 7f00e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:17:59.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f4004c30 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00f4004d90 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b66fa1c4f0 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00f40137f0 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f0117c5db80 10 client.27184 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b66f8d9c60 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.708+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b66fa1c4f0 con 0x55b66fb702d0 2023-07-21T19:17:59.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00f4005390 con 0x55b66fb702d0 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 10 client.27184 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 10 client.27184 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 20 client.27184 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fb93b00 0x55b66fb95f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 4 client.27184 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:17:59.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b66fabbeb0 con 0x55b66fb93b00 2023-07-21T19:17:59.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.709+0000 7f010199a700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fb93b00 0x55b66fb95f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.710+0000 7f010199a700 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fb93b00 0x55b66fb95f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=edb68aa09fb7f9dd server_cookie=440cf7dd4e970948 in_seq=0 out_seq=0 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.710+0000 7f00e5ffb700 10 client.27184 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f00e5ffb700 1 -- 172.21.15.5:0/1167387935 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b66fabbeb0 con 0x55b66fb93b00 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f00e5ffb700 10 client.27184 handle_command_reply: tid=0 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 client.27184 shutdown 2023-07-21T19:17:59.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 msgr2=0x55b66fb7b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00f4009890 tx=0x7f00f40097e0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 shutdown_connections 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fb93b00 0x55b66fb95f10 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb7bad0 0x55b66fb78da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb7b4f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 --2- 172.21.15.5:0/1167387935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb78770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 >> 172.21.15.5:0/1167387935 conn(0x55b66fb66da0 msgr2=0x55b66fb6c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.711+0000 7f00e5ffb700 0 client.27184 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:17:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.712+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 shutdown_connections 2023-07-21T19:17:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.712+0000 7f0117c5db80 1 -- 172.21.15.5:0/1167387935 wait complete. 2023-07-21T19:17:59.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.712+0000 7f0117c5db80 20 client.27184 trim_cache size 0 max 16384 2023-07-21T19:17:59.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 Processor -- start 2023-07-21T19:17:59.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 -- start start 2023-07-21T19:17:59.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb82550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fb91240 con 0x55b66fb70be0 2023-07-21T19:17:59.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fb71540 con 0x55b66fb702d0 2023-07-21T19:17:59.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.729+0000 7f0117c5db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fb721a0 con 0x55b66fb7bad0 2023-07-21T19:17:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010299c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb82550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42060/0 (socket says 172.21.15.5:42060) 2023-07-21T19:17:59.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 -- 172.21.15.5:0/2568854923 learned_addr learned my addr 172.21.15.5:0/2568854923 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010299c700 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 -- 172.21.15.5:0/2568854923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 msgr2=0x55b66fb7b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 -- 172.21.15.5:0/2568854923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 msgr2=0x55b66fb82550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb82550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 -- 172.21.15.5:0/2568854923 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b66f917250 con 0x55b66fb70be0 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.730+0000 7f010199a700 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f00ec015000 tx=0x7f00ec009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=449e56df5c3a8671 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.731+0000 7f00f13ff700 1 -- 172.21.15.5:0/2568854923 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec01d070 con 0x55b66fb70be0 2023-07-21T19:17:59.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.731+0000 7f00f13ff700 1 -- 172.21.15.5:0/2568854923 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00ec01e040 con 0x55b66fb70be0 2023-07-21T19:17:59.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.731+0000 7f00f13ff700 1 -- 172.21.15.5:0/2568854923 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec004cc0 con 0x55b66fb70be0 2023-07-21T19:17:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- 172.21.15.5:0/2568854923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 msgr2=0x55b66fb95b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f00ec015000 tx=0x7f00ec009310 comp rx=0 tx=0).stop 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- 172.21.15.5:0/2568854923 shutdown_connections 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb7bad0 0x55b66fb7b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb702d0 0x55b66fb82550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 --2- 172.21.15.5:0/2568854923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb70be0 0x55b66fb95b10 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- 172.21.15.5:0/2568854923 >> 172.21.15.5:0/2568854923 conn(0x55b66fb66da0 msgr2=0x55b66fb86cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- 172.21.15.5:0/2568854923 shutdown_connections 2023-07-21T19:17:59.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- 172.21.15.5:0/2568854923 wait complete. 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.732+0000 7f0117c5db80 1 -- start start 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 Processor -- start 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 0x55b66fb808a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb7bad0 0x55b66fbe69c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fbe6f00 con 0x55b66fb7bad0 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fbe7040 con 0x55b66fb70be0 2023-07-21T19:17:59.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b66fbe71b0 con 0x55b66fb702d0 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.733+0000 7f0117c5db80 10 client.0 _ll_register_callbacks cb 0x7f00f4017070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010299c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58594/0 (socket says 172.21.15.5:58594) 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010299c700 1 -- 172.21.15.5:0/402090043 learned_addr learned my addr 172.21.15.5:0/402090043 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 0x55b66fb808a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010299c700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb7bad0 0x55b66fbe69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 -- 172.21.15.5:0/402090043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 msgr2=0x55b66fb80360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 -- 172.21.15.5:0/402090043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb7bad0 msgr2=0x55b66fbe69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb7bad0 0x55b66fbe69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 -- 172.21.15.5:0/402090043 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc10d170 con 0x55b66fb70be0 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010299c700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f010199a700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 0x55b66fb808a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009990 tx=0x7f00ec005990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65fbc97b52e3df68 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.734+0000 7f00e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:17:59.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec01d040 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00ec004c70 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00f40149f0 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00ec004e70 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f0117c5db80 10 client.27263 fetch_fsmap learned FSMap version 42 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b66f8ef2b0 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.735+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00f40149f0 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f00ec00f690 con 0x55b66fb70be0 2023-07-21T19:17:59.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 10 client.27263 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 10 client.27263 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 20 client.27263 populate_metadata read hostname 'smithi005' 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fbe84e0 0x55b66fbea8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 4 client.27263 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b66fa7e050 con 0x55b66fbe84e0 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.736+0000 7f010299c700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fbe84e0 0x55b66fbea8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:17:59.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f010299c700 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fbe84e0 0x55b66fbea8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b50beec970a831f server_cookie=e95c5056b1cf3e02 in_seq=0 out_seq=0 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f00e5ffb700 10 client.27263 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f00e5ffb700 1 -- 172.21.15.5:0/402090043 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b66fa7e050 con 0x55b66fbe84e0 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f00e5ffb700 10 client.27263 handle_command_reply: tid=0 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 client.27263 shutdown 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 msgr2=0x55b66fb808a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 0x55b66fb808a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f00ec009990 tx=0x7f00ec005990 comp rx=0 tx=0).stop 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 shutdown_connections 2023-07-21T19:17:59.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b66fb7bad0 0x55b66fbe69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b66fb702d0 0x55b66fb80360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55b66fbe84e0 0x55b66fbea8f0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 --2- 172.21.15.5:0/402090043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b66fb70be0 0x55b66fb808a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 >> 172.21.15.5:0/402090043 conn(0x55b66fb66da0 msgr2=0x55b66fb87720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.737+0000 7f00e5ffb700 0 client.27263 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 shutdown_connections 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f0117c5db80 1 -- 172.21.15.5:0/402090043 wait complete. 2023-07-21T19:17:59.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f0117c5db80 20 client.27263 trim_cache size 0 max 16384 2023-07-21T19:17:59.744 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:17:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 msgr2=0x7f00e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 0x7f00e80a3770 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f00f8096590 tx=0x7f00f801f000 comp rx=0 tx=0).stop 2023-07-21T19:17:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 msgr2=0x7f00fc0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:17:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f00f800e9a0 tx=0x7f00f800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 shutdown_connections 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00fc10a070 0x7f00fc1a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00fc100ca0 0x7f00fc0fe8f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f00e80a12c0 0x7f00e80a3770 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 --2- 172.21.15.5:0/77135153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00fc1015b0 0x7f00fc0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.738+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 >> 172.21.15.5:0/77135153 conn(0x7f00fc0fa4b0 msgr2=0x7f00fc0fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.739+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 shutdown_connections 2023-07-21T19:17:59.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:17:59.739+0000 7f01043ff700 1 -- 172.21.15.5:0/77135153 wait complete. 2023-07-21T19:17:59.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:59 smithi005 ceph-mon[126460]: pgmap v322: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 29 MiB/s wr, 84 op/s 2023-07-21T19:17:59.752 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:17:59 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1113541303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:17:59.754 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:17:59.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:59 smithi101 ceph-mon[128136]: pgmap v322: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 29 MiB/s wr, 84 op/s 2023-07-21T19:17:59.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:17:59 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1113541303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:17:59.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:59 smithi156 ceph-mon[128316]: pgmap v322: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 29 MiB/s wr, 84 op/s 2023-07-21T19:17:59.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:17:59 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1113541303' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:18:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.045+0000 7f19281ff700 1 -- 172.21.15.5:0/2597404413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 msgr2=0x7f1920101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.045+0000 7f19281ff700 1 --2- 172.21.15.5:0/2597404413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f1920101070 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f191c009990 tx=0x7f191c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 -- 172.21.15.5:0/2597404413 shutdown_connections 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 --2- 172.21.15.5:0/2597404413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 0x7f192010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 --2- 172.21.15.5:0/2597404413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f1920101070 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 --2- 172.21.15.5:0/2597404413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f1920109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 -- 172.21.15.5:0/2597404413 >> 172.21.15.5:0/2597404413 conn(0x7f19200fa4b0 msgr2=0x7f19200fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 -- 172.21.15.5:0/2597404413 shutdown_connections 2023-07-21T19:18:00.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.046+0000 7f19281ff700 1 -- 172.21.15.5:0/2597404413 wait complete. 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.047+0000 7f19281ff700 1 Processor -- start 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.047+0000 7f19281ff700 1 -- start start 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f19200fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 0x7f19201a3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19200ff660 con 0x7f1920100ca0 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19201a4420 con 0x7f192010a070 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f19281ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19201a4560 con 0x7f19201015b0 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42086/0 (socket says 172.21.15.5:42086) 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f1925f9b700 1 -- 172.21.15.5:0/3816489996 learned_addr learned my addr 172.21.15.5:0/3816489996 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:00.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f192579a700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f19200fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.048+0000 7f192679c700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 0x7f19201a3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 -- 172.21.15.5:0/3816489996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 msgr2=0x7f19201a3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 0x7f19201a3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 -- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 msgr2=0x7f19200fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 -- 172.21.15.5:0/3816489996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c0095f0 con 0x7f19201015b0 2023-07-21T19:18:00.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f1925f9b700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.049+0000 7f192579a700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f19200fee30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f191000e9f0 tx=0x7f191000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.050+0000 7f1916ffd700 1 -- 172.21.15.5:0/3816489996 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191000c9c0 con 0x7f19201015b0 2023-07-21T19:18:00.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.050+0000 7f1916ffd700 1 -- 172.21.15.5:0/3816489996 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1910010430 con 0x7f19201015b0 2023-07-21T19:18:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.050+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19201a47c0 con 0x7f19201015b0 2023-07-21T19:18:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.050+0000 7f1916ffd700 1 -- 172.21.15.5:0/3816489996 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910018680 con 0x7f19201015b0 2023-07-21T19:18:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.050+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19201a4ca0 con 0x7f19201015b0 2023-07-21T19:18:00.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.053+0000 7f1916ffd700 1 -- 172.21.15.5:0/3816489996 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f19100105a0 con 0x7f19201015b0 2023-07-21T19:18:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.055+0000 7f193bc1ab80 1 Processor -- start 2023-07-21T19:18:00.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.055+0000 7f193bc1ab80 1 -- start start 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3c96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d2cfcb0 con 0x55ca2d3d4ad0 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d322a40 con 0x55ca2d3c92d0 2023-07-21T19:18:00.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d31bed0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f1916ffd700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 0x7f190c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f192679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3c96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f192679c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3c96a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47046/0 (socket says 172.21.15.5:47046) 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f192679c700 1 -- 172.21.15.5:0/4080134333 learned_addr learned my addr 172.21.15.5:0/4080134333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f1916ffd700 1 -- 172.21.15.5:0/3816489996 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1910014070 con 0x7f19201015b0 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.056+0000 7f192579a700 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 -- 172.21.15.5:0/4080134333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 msgr2=0x55ca2d3c96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3c96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 -- 172.21.15.5:0/4080134333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3d4ad0 msgr2=0x55ca2d3d6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f192679c700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 0x7f190c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 -- 172.21.15.5:0/4080134333 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2d149de0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f1925f9b700 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4590 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f191c005910 tx=0x7f191c005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d83f0e044669c91b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.057+0000 7f192679c700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 0x7f190c0a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1918009710 tx=0x7f1918004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.058+0000 7f190bfff700 1 -- 172.21.15.5:0/4080134333 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c01d070 con 0x55ca2d3c9be0 2023-07-21T19:18:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.058+0000 7f190bfff700 1 -- 172.21.15.5:0/4080134333 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f191c01e040 con 0x55ca2d3c9be0 2023-07-21T19:18:00.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.058+0000 7f190bfff700 1 -- 172.21.15.5:0/4080134333 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c004d20 con 0x55ca2d3c9be0 2023-07-21T19:18:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.058+0000 7f193bc1ab80 1 -- 172.21.15.5:0/4080134333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 msgr2=0x55ca2d3d4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.058+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4590 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f191c005910 tx=0x7f191c005c70 comp rx=0 tx=0).stop 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 -- 172.21.15.5:0/4080134333 shutdown_connections 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3c96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/4080134333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4590 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 -- 172.21.15.5:0/4080134333 >> 172.21.15.5:0/4080134333 conn(0x55ca2d3bfda0 msgr2=0x55ca2d3c56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 -- 172.21.15.5:0/4080134333 shutdown_connections 2023-07-21T19:18:00.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 -- 172.21.15.5:0/4080134333 wait complete. 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.059+0000 7f193bc1ab80 1 -- start start 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 Processor -- start 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d2bbac0 con 0x55ca2d3c92d0 2023-07-21T19:18:00.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d2bb7c0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.060+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d19f920 con 0x55ca2d3d4ad0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f193bc1ab80 10 client.0 _ll_register_callbacks cb 0x55ca2d2b9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42108/0 (socket says 172.21.15.5:42108) 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 -- 172.21.15.5:0/2139419286 learned_addr learned my addr 172.21.15.5:0/2139419286 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f192579a700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f192679c700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 -- 172.21.15.5:0/2139419286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 msgr2=0x55ca2d3d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 -- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 msgr2=0x55ca2d3d1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f1925f9b700 1 -- 172.21.15.5:0/2139419286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f192010d170 con 0x55ca2d3c92d0 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.061+0000 7f192679c700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f1925f9b700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f191c022a40 tx=0x7f191c022a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30c96e0f2d8c32ad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f19097fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191c01d030 con 0x55ca2d3c92d0 2023-07-21T19:18:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca2d2754f0 con 0x55ca2d3c92d0 2023-07-21T19:18:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f191c00f460 con 0x55ca2d3c92d0 2023-07-21T19:18:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191c004760 con 0x55ca2d3c92d0 2023-07-21T19:18:00.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f193bc1ab80 10 client.17592 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.062+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ca2d132c60 con 0x55ca2d3c92d0 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca2d2754f0 con 0x55ca2d3c92d0 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f191c02c600 con 0x55ca2d3c92d0 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 10 client.17592 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 10 client.17592 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 20 client.17592 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:00.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d3e4000 0x55ca2d3e63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 4 client.17592 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca2d314eb0 con 0x55ca2d3e4000 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.063+0000 7f192579a700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d3e4000 0x55ca2d3e63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.064+0000 7f192579a700 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d3e4000 0x55ca2d3e63e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42355e7cf2c84fff server_cookie=e1c58f92db973191 in_seq=0 out_seq=0 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.064+0000 7f19097fa700 10 client.17592 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f19097fa700 1 -- 172.21.15.5:0/2139419286 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca2d314eb0 con 0x55ca2d3e4000 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f19097fa700 10 client.17592 handle_command_reply: tid=0 2023-07-21T19:18:00.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 client.17592 shutdown 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 msgr2=0x55ca2d3d4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f191c022a40 tx=0x7f191c022a70 comp rx=0 tx=0).stop 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 shutdown_connections 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d3e4000 0x55ca2d3e63e0 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3d4480 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2139419286 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 >> 172.21.15.5:0/2139419286 conn(0x55ca2d3bfda0 msgr2=0x55ca2d3c5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f19097fa700 0 client.17592 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.065+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 shutdown_connections 2023-07-21T19:18:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.066+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2139419286 wait complete. 2023-07-21T19:18:00.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.066+0000 7f193bc1ab80 20 client.17592 trim_cache size 0 max 16384 2023-07-21T19:18:00.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 Processor -- start 2023-07-21T19:18:00.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 -- start start 2023-07-21T19:18:00.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3db4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3ce990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d3ca580 con 0x55ca2d3c92d0 2023-07-21T19:18:00.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d3d2ce0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.088+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d3c6ca0 con 0x55ca2d3d4ad0 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3db4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3db4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42110/0 (socket says 172.21.15.5:42110) 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f1925f9b700 1 -- 172.21.15.5:0/2140003652 learned_addr learned my addr 172.21.15.5:0/2140003652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:00.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192679c700 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3ce990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 -- 172.21.15.5:0/2140003652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 msgr2=0x55ca2d3ce990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3ce990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 -- 172.21.15.5:0/2140003652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 msgr2=0x55ca2d3db4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3db4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.089+0000 7f192579a700 1 -- 172.21.15.5:0/2140003652 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca2d170250 con 0x55ca2d3c9be0 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.090+0000 7f192579a700 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19100ca5f0 tx=0x7f19100cc040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=479fc427f5907ba1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.090+0000 7f1914ff9700 1 -- 172.21.15.5:0/2140003652 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1910099d40 con 0x55ca2d3c9be0 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.090+0000 7f1914ff9700 1 -- 172.21.15.5:0/2140003652 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1910010880 con 0x55ca2d3c9be0 2023-07-21T19:18:00.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.090+0000 7f1914ff9700 1 -- 172.21.15.5:0/2140003652 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f191009a430 con 0x55ca2d3c9be0 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2140003652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 msgr2=0x55ca2d3d4020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4020 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19100ca5f0 tx=0x7f19100cc040 comp rx=0 tx=0).stop 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2140003652 shutdown_connections 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d3ce990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d3db4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/2140003652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d3d4020 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2140003652 >> 172.21.15.5:0/2140003652 conn(0x55ca2d3bfda0 msgr2=0x55ca2d3e0720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2140003652 shutdown_connections 2023-07-21T19:18:00.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- 172.21.15.5:0/2140003652 wait complete. 2023-07-21T19:18:00.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.091+0000 7f193bc1ab80 1 -- start start 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.092+0000 7f193bc1ab80 1 Processor -- start 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d431660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d431ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d42be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d42c400 con 0x55ca2d3c92d0 2023-07-21T19:18:00.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d42c570 con 0x55ca2d3c9be0 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca2d42c730 con 0x55ca2d3d4ad0 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f193bc1ab80 10 client.0 _ll_register_callbacks cb 0x7f1910009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d431660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192579a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d431ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f1925f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d431660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42112/0 (socket says 172.21.15.5:42112) 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f1925f9b700 1 -- 172.21.15.5:0/836516299 learned_addr learned my addr 172.21.15.5:0/836516299 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:00.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192679c700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d42be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192579a700 1 -- 172.21.15.5:0/836516299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 msgr2=0x55ca2d42be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192579a700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d42be20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192579a700 1 -- 172.21.15.5:0/836516299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 msgr2=0x55ca2d431660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.093+0000 7f192579a700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d431660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f192579a700 1 -- 172.21.15.5:0/836516299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19201a4ca0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f192579a700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d431ba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f191000c5b0 tx=0x7f1910096330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c287cd9c397b9b1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f19097fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:00.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19100c8070 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1910099f70 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f191c01e040 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f191009dfa0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f193bc1ab80 10 client.27281 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.094+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55ca2d1482b0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1910065030 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f191009a1d0 con 0x55ca2d3c9be0 2023-07-21T19:18:00.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 10 client.27281 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 10 client.27281 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 20 client.27281 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d4391e0 0x55ca2d43b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 4 client.27281 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ca2d2d7050 con 0x55ca2d4391e0 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.095+0000 7f1925f9b700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d4391e0 0x55ca2d43b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.096+0000 7f1925f9b700 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d4391e0 0x55ca2d43b5f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4264961caa1745f server_cookie=f45938c0c9d8f791 in_seq=0 out_seq=0 2023-07-21T19:18:00.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.096+0000 7f19097fa700 10 client.27281 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:00.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.096+0000 7f19097fa700 1 -- 172.21.15.5:0/836516299 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ca2d2d7050 con 0x55ca2d4391e0 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.096+0000 7f19097fa700 10 client.27281 handle_command_reply: tid=0 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.096+0000 7f193bc1ab80 1 client.27281 shutdown 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 msgr2=0x55ca2d431ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d431ba0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f191000c5b0 tx=0x7f1910096330 comp rx=0 tx=0).stop 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 shutdown_connections 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca2d3d4ad0 0x55ca2d42be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca2d4391e0 0x55ca2d43b5f0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca2d3c92d0 0x55ca2d431660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 --2- 172.21.15.5:0/836516299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca2d3c9be0 0x55ca2d431ba0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f19097fa700 0 client.27281 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 >> 172.21.15.5:0/836516299 conn(0x55ca2d3bfda0 msgr2=0x55ca2d3cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 shutdown_connections 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 1 -- 172.21.15.5:0/836516299 wait complete. 2023-07-21T19:18:00.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.097+0000 7f193bc1ab80 20 client.27281 trim_cache size 0 max 16384 2023-07-21T19:18:00.101 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:18:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 msgr2=0x7f190c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 0x7f190c0a3770 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f1918009710 tx=0x7f1918004b40 comp rx=0 tx=0).stop 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 msgr2=0x7f19200fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f19200fee30 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f191000e9f0 tx=0x7f191000ed00 comp rx=0 tx=0).stop 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 shutdown_connections 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f192010a070 0x7f19201a3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f190c0a12c0 0x7f190c0a3770 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1920100ca0 0x7f19200fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 --2- 172.21.15.5:0/3816489996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19201015b0 0x7f19200fee30 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 >> 172.21.15.5:0/3816489996 conn(0x7f19200fa4b0 msgr2=0x7f192010c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 shutdown_connections 2023-07-21T19:18:00.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:00.098+0000 7f19281ff700 1 -- 172.21.15.5:0/3816489996 wait complete. 2023-07-21T19:18:00.899 INFO:teuthology.orchestra.run.smithi005.stderr:39100000 of 50000000 tuples (78%) done (elapsed 598.47 s, remaining 166.84 s) 2023-07-21T19:18:01.116 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:18:01.116 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:18:01.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 -- 172.21.15.5:0/602692485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880731c0 msgr2=0x7f46881105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 --2- 172.21.15.5:0/602692485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880731c0 0x7f46881105b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f4674009990 tx=0x7f4674009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 -- 172.21.15.5:0/602692485 shutdown_connections 2023-07-21T19:18:01.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 --2- 172.21.15.5:0/602692485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4688110af0 0x7f4688112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 --2- 172.21.15.5:0/602692485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f46880728b0 0x7f4688072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 --2- 172.21.15.5:0/602692485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880731c0 0x7f46881105b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 -- 172.21.15.5:0/602692485 >> 172.21.15.5:0/602692485 conn(0x7f46880fb5e0 msgr2=0x7f46880fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 -- 172.21.15.5:0/602692485 shutdown_connections 2023-07-21T19:18:01.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.372+0000 7f468f1ff700 1 -- 172.21.15.5:0/602692485 wait complete. 2023-07-21T19:18:01.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.373+0000 7f468f1ff700 1 Processor -- start 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 -- start start 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46880731c0 0x7f468819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4688110af0 0x7f46881a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881a1180 con 0x7f46880728b0 2023-07-21T19:18:01.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881a12f0 con 0x7f4688110af0 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46881a14b0 con 0x7f46880731c0 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42130/0 (socket says 172.21.15.5:42130) 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468cf9b700 1 -- 172.21.15.5:0/3199924674 learned_addr learned my addr 172.21.15.5:0/3199924674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.374+0000 7f468d79c700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4688110af0 0x7f46881a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f4687fff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46880731c0 0x7f468819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 -- 172.21.15.5:0/3199924674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46880731c0 msgr2=0x7f468819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46880731c0 0x7f468819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 -- 172.21.15.5:0/3199924674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4688110af0 msgr2=0x7f46881a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4688110af0 0x7f46881a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 -- 172.21.15.5:0/3199924674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46740095f0 con 0x7f46880728b0 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9a0 tx=0x7f467c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.375+0000 7f4685ffb700 1 -- 172.21.15.5:0/3199924674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c009790 con 0x7f46880728b0 2023-07-21T19:18:01.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.376+0000 7f4685ffb700 1 -- 172.21.15.5:0/3199924674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467c010430 con 0x7f46880728b0 2023-07-21T19:18:01.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.376+0000 7f4685ffb700 1 -- 172.21.15.5:0/3199924674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c00cea0 con 0x7f46880728b0 2023-07-21T19:18:01.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.376+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46881a1790 con 0x7f46880728b0 2023-07-21T19:18:01.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.376+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46881a8fd0 con 0x7f46880728b0 2023-07-21T19:18:01.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.379+0000 7f4685ffb700 1 -- 172.21.15.5:0/3199924674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f467c0105a0 con 0x7f46880728b0 2023-07-21T19:18:01.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 Processor -- start 2023-07-21T19:18:01.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 -- start start 2023-07-21T19:18:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f4685ffb700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 0x7f46780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9be6720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f4685ffb700 1 -- 172.21.15.5:0/3199924674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c014070 con 0x7f46880728b0 2023-07-21T19:18:01.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9bf3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9aecd30 con 0x5644b9be6350 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9b3fac0 con 0x5644b9be6c60 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.381+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9b38f50 con 0x5644b9bf1b50 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f468d79c700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 0x7f46780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47102/0 (socket says 172.21.15.5:47102) 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 -- 172.21.15.5:0/3517157692 learned_addr learned my addr 172.21.15.5:0/3517157692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9be6720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f468d79c700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9bf3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 -- 172.21.15.5:0/3517157692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 msgr2=0x5644b9bf3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9bf3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 -- 172.21.15.5:0/3517157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 msgr2=0x5644b9be6720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9be6720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f4687fff700 1 -- 172.21.15.5:0/3517157692 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b9966e60 con 0x5644b9be6c60 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.382+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9be6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.383+0000 7f4687fff700 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4674009990 tx=0x7f4674005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d22aa50d12ccc4d3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.383+0000 7f468d79c700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 0x7f46780a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ec0 tx=0x7f4680009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.383+0000 7f4672ffd700 1 -- 172.21.15.5:0/3517157692 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467401d070 con 0x5644b9be6c60 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.383+0000 7f4672ffd700 1 -- 172.21.15.5:0/3517157692 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467401e040 con 0x5644b9be6c60 2023-07-21T19:18:01.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.383+0000 7f4672ffd700 1 -- 172.21.15.5:0/3517157692 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740047f0 con 0x5644b9be6c60 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3517157692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 msgr2=0x5644b9bf1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f4674009990 tx=0x7f4674005f00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3517157692 shutdown_connections 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9bf3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9be6720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3517157692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bf1610 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3517157692 >> 172.21.15.5:0/3517157692 conn(0x5644b9bdce20 msgr2=0x5644b9be2720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3517157692 shutdown_connections 2023-07-21T19:18:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.384+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3517157692 wait complete. 2023-07-21T19:18:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.385+0000 7f46a2b01b80 1 -- start start 2023-07-21T19:18:01.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.385+0000 7f46a2b01b80 1 Processor -- start 2023-07-21T19:18:01.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.385+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.385+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bee7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9beee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9ad8b40 con 0x5644b9be6350 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9ad8840 con 0x5644b9be6c60 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b99bc9a0 con 0x5644b9bf1b50 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f46a2b01b80 10 client.0 _ll_register_callbacks cb 0x5644b9ad67e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f4687fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bee7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42160/0 (socket says 172.21.15.5:42160) 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 -- 172.21.15.5:0/63472883 learned_addr learned my addr 172.21.15.5:0/63472883 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468d79c700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9beee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 -- 172.21.15.5:0/63472883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 msgr2=0x5644b9beee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9beee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 -- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 msgr2=0x5644b9bee7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bee7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468cf9b700 1 -- 172.21.15.5:0/63472883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4688113790 con 0x5644b9be6350 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f468d79c700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9beee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.386+0000 7f4687fff700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f468cf9b700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f467c096450 tx=0x7f467c010970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfaefc83f40be10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f4667fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c099440 con 0x5644b9be6350 2023-07-21T19:18:01.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c0995a0 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5644b9a92570 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c0b8310 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f46a2b01b80 10 client.17610 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5644b994fce0 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.387+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467c0c8020 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f467c094b60 con 0x5644b9be6350 2023-07-21T19:18:01.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 10 client.17610 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 10 client.17610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 20 client.17610 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c01340 0x5644b9c03720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 4 client.17610 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5644b9b31f30 con 0x5644b9c01340 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.388+0000 7f4687fff700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c01340 0x5644b9c03720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.389+0000 7f4687fff700 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c01340 0x5644b9c03720 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e4217219126088 server_cookie=771188eb6423d8f4 in_seq=0 out_seq=0 2023-07-21T19:18:01.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.389+0000 7f4667fff700 10 client.17610 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f4667fff700 1 -- 172.21.15.5:0/63472883 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5644b9b31f30 con 0x5644b9c01340 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f4667fff700 10 client.17610 handle_command_reply: tid=0 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 client.17610 shutdown 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 msgr2=0x5644b9bf1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f467c096450 tx=0x7f467c010970 comp rx=0 tx=0).stop 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 shutdown_connections 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9bf1b50 0x5644b9beee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c01340 0x5644b9c03720 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9be6350 0x5644b9bf1530 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/63472883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6c60 0x5644b9bee7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 >> 172.21.15.5:0/63472883 conn(0x5644b9bdce20 msgr2=0x5644b9be22b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f4667fff700 0 client.17610 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 shutdown_connections 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.390+0000 7f46a2b01b80 1 -- 172.21.15.5:0/63472883 wait complete. 2023-07-21T19:18:01.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.391+0000 7f46a2b01b80 20 client.17610 trim_cache size 0 max 16384 2023-07-21T19:18:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.416+0000 7f46a2b01b80 1 Processor -- start 2023-07-21T19:18:01.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.416+0000 7f46a2b01b80 1 -- start start 2023-07-21T19:18:01.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bedfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9bd9fb0 con 0x5644b9bf1b50 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9becd70 con 0x5644b9be6350 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9c082d0 con 0x5644b9be6c60 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f468d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f468d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42164/0 (socket says 172.21.15.5:42164) 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f468d79c700 1 -- 172.21.15.5:0/1452194411 learned_addr learned my addr 172.21.15.5:0/1452194411 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.417+0000 7f4687fff700 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bedfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 -- 172.21.15.5:0/1452194411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 msgr2=0x5644b9bedfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bedfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 -- 172.21.15.5:0/1452194411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 msgr2=0x5644b9bf10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 -- 172.21.15.5:0/1452194411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5644b998d2d0 con 0x5644b9bf1b50 2023-07-21T19:18:01.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468cf9b700 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f468d79c700 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4680018950 tx=0x7f4680004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff669820d384b4a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f46737fe700 1 -- 172.21.15.5:0/1452194411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468000e040 con 0x5644b9bf1b50 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f46737fe700 1 -- 172.21.15.5:0/1452194411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4680021040 con 0x5644b9bf1b50 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.418+0000 7f46737fe700 1 -- 172.21.15.5:0/1452194411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4680008510 con 0x5644b9bf1b50 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 -- 172.21.15.5:0/1452194411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 msgr2=0x5644b9bee510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4680018950 tx=0x7f4680004b40 comp rx=0 tx=0).stop 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 -- 172.21.15.5:0/1452194411 shutdown_connections 2023-07-21T19:18:01.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9bee510 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/1452194411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bedfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 -- 172.21.15.5:0/1452194411 >> 172.21.15.5:0/1452194411 conn(0x5644b9bdce20 msgr2=0x5644b9be9b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 -- 172.21.15.5:0/1452194411 shutdown_connections 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.419+0000 7f46a2b01b80 1 -- 172.21.15.5:0/1452194411 wait complete. 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.420+0000 7f46a2b01b80 1 -- start start 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.420+0000 7f46a2b01b80 1 Processor -- start 2023-07-21T19:18:01.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bf78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9c551e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9c55720 con 0x5644b9bf1b50 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9c55860 con 0x5644b9be6350 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5644b9c559d0 con 0x5644b9be6c60 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f46a2b01b80 10 client.0 _ll_register_callbacks cb 0x7f467c099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f468cf9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47138/0 (socket says 172.21.15.5:47138) 2023-07-21T19:18:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f468cf9b700 1 -- 172.21.15.5:0/3225140258 learned_addr learned my addr 172.21.15.5:0/3225140258 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f4687fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bf78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.421+0000 7f468d79c700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9c551e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 -- 172.21.15.5:0/3225140258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 msgr2=0x5644b9bf78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bf78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 -- 172.21.15.5:0/3225140258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 msgr2=0x5644b9c551e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9c551e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 -- 172.21.15.5:0/3225140258 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5644b994fce0 con 0x5644b9be6350 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f4687fff700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bf78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f468cf9b700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f467c0a1e70 tx=0x7f467c010ae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5b73da36b82c347 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f4667fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0a2110 con 0x5644b9be6350 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f467401e040 con 0x5644b9be6350 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.422+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c0c9080 con 0x5644b9be6350 2023-07-21T19:18:01.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c0b8310 con 0x5644b9be6350 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 10 client.27299 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f46800186c0 con 0x5644b9be6350 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467401e040 con 0x5644b9be6350 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f467c095e40 con 0x5644b9be6350 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 10 client.27299 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 10 client.27299 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 20 client.27299 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c56d10 0x5644b9c59120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 4 client.27299 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.423+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5644b9af40d0 con 0x5644b9c56d10 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.424+0000 7f4687fff700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c56d10 0x5644b9c59120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.424+0000 7f4687fff700 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c56d10 0x5644b9c59120 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37b8320e5cdca025 server_cookie=6cc73b01e4cacc67 in_seq=0 out_seq=0 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.424+0000 7f4667fff700 10 client.27299 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f4667fff700 1 -- 172.21.15.5:0/3225140258 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5644b9af40d0 con 0x5644b9c56d10 2023-07-21T19:18:01.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f4667fff700 10 client.27299 handle_command_reply: tid=0 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f46a2b01b80 1 client.27299 shutdown 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 msgr2=0x5644b9bf7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f467c0a1e70 tx=0x7f467c010ae0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 shutdown_connections 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.425+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5644b9c56d10 0x5644b9c59120 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5644b9bf1b50 0x5644b9c551e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5644b9be6350 0x5644b9bf7370 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 --2- 172.21.15.5:0/3225140258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5644b9be6c60 0x5644b9bf78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 >> 172.21.15.5:0/3225140258 conn(0x5644b9bdce20 msgr2=0x5644b9be8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f4667fff700 0 client.27299 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 shutdown_connections 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 1 -- 172.21.15.5:0/3225140258 wait complete. 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.426+0000 7f46a2b01b80 20 client.27299 trim_cache size 0 max 16384 2023-07-21T19:18:01.429 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:18:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:18:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "83bdab96-51d1-4839-9d4c-a086f0feb3b9", 2023-07-21T19:18:01.430 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:18:01.430 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 msgr2=0x7f46780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 0x7f46780a37f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f4680009ec0 tx=0x7f4680009920 comp rx=0 tx=0).stop 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 msgr2=0x7f468819c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f467c00e9a0 tx=0x7f467c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 shutdown_connections 2023-07-21T19:18:01.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4688110af0 0x7f46881a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46780a1340 0x7f46780a37f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f46880728b0 0x7f468819c1a0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 --2- 172.21.15.5:0/3199924674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f46880731c0 0x7f468819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 >> 172.21.15.5:0/3199924674 conn(0x7f46880fb5e0 msgr2=0x7f46880fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 shutdown_connections 2023-07-21T19:18:01.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.427+0000 7f468f1ff700 1 -- 172.21.15.5:0/3199924674 wait complete. 2023-07-21T19:18:01.446 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:18:01.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/2138293837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf40ffb00 msgr2=0x7f5cf40fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/2138293837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf40ffb00 0x7f5cf40fff70 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f5ce400faf0 tx=0x7f5ce400fe00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/2138293837 shutdown_connections 2023-07-21T19:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/2138293837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf41004b0 0x7f5cf410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/2138293837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf4105b10 0x7f5cf4105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/2138293837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf40ffb00 0x7f5cf40fff70 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.639+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/2138293837 >> 172.21.15.5:0/2138293837 conn(0x7f5cf40fb600 msgr2=0x7f5cf40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.640+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/2138293837 shutdown_connections 2023-07-21T19:18:01.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.640+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/2138293837 wait complete. 2023-07-21T19:18:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.641+0000 7f5cfc7ff700 1 Processor -- start 2023-07-21T19:18:01.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.641+0000 7f5cfc7ff700 1 -- start start 2023-07-21T19:18:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 0x7f5cf4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf4105b10 0x7f5cf41937f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4193e60 con 0x7f5cf4105b10 2023-07-21T19:18:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4193fd0 con 0x7f5cf40ffb00 2023-07-21T19:18:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cf4194190 con 0x7f5cf41004b0 2023-07-21T19:18:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58718/0 (socket says 172.21.15.5:58718) 2023-07-21T19:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/258571916 learned_addr learned my addr 172.21.15.5:0/258571916 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf4105b10 0x7f5cf41937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.642+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 0x7f5cf4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 -- 172.21.15.5:0/258571916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 msgr2=0x7f5cf4199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 -- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf4105b10 msgr2=0x7f5cf41937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf4105b10 0x7f5cf41937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 -- 172.21.15.5:0/258571916 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cf0009520 con 0x7f5cf40ffb00 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 0x7f5cf4199990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9f0 tx=0x7f5cf000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/258571916 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf000cb70 con 0x7f5cf40ffb00 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce400f750 con 0x7f5cf40ffb00 2023-07-21T19:18:01.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.643+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/258571916 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cf000ee90 con 0x7f5cf40ffb00 2023-07-21T19:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.644+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/258571916 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf0003b60 con 0x7f5cf40ffb00 2023-07-21T19:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.644+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf4194660 con 0x7f5cf40ffb00 2023-07-21T19:18:01.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.648+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/258571916 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5cf0003d40 con 0x7f5cf40ffb00 2023-07-21T19:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.649+0000 7f5d10047b80 1 Processor -- start 2023-07-21T19:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 -- start start 2023-07-21T19:18:01.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f98aec50 con 0x5651f99a9580 2023-07-21T19:18:01.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99019e0 con 0x5651f99af2d0 2023-07-21T19:18:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5d10047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f98fae70 con 0x5651f98056b0 2023-07-21T19:18:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.650+0000 7f5ceb7fe700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 0x7f5ce00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cfad9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5ceb7fe700 1 -- 172.21.15.5:0/258571916 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5cf0014070 con 0x7f5cf40ffb00 2023-07-21T19:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42202/0 (socket says 172.21.15.5:42202) 2023-07-21T19:18:01.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/3691518575 learned_addr learned my addr 172.21.15.5:0/3691518575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cfad9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58722/0 (socket says 172.21.15.5:58722) 2023-07-21T19:18:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 0x7f5ce00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/3691518575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 msgr2=0x5651f99aed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/3691518575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 msgr2=0x5651f99a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/3691518575 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f9728d80 con 0x5651f99a9580 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.651+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.652+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4019040 tx=0x7f5ce4005f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bc98abb2a406adb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.652+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 0x7f5ce00a3770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cf419a690 tx=0x7f5cec009100 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.652+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3691518575 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce4003ba0 con 0x5651f99a9580 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.652+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3691518575 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ce4003d00 con 0x5651f99a9580 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.652+0000 7f5ce8ff9700 1 -- 172.21.15.5:0/3691518575 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce400e660 con 0x5651f99a9580 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 -- 172.21.15.5:0/3691518575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 msgr2=0x5651f99a99d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f5ce4019040 tx=0x7f5ce4005f00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 -- 172.21.15.5:0/3691518575 shutdown_connections 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99a99d0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99aed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 --2- 172.21.15.5:0/3691518575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 -- 172.21.15.5:0/3691518575 >> 172.21.15.5:0/3691518575 conn(0x5651f999ebe0 msgr2=0x5651f99a3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 -- 172.21.15.5:0/3691518575 shutdown_connections 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.653+0000 7f5d10047b80 1 -- 172.21.15.5:0/3691518575 wait complete. 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.654+0000 7f5d10047b80 1 -- start start 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.654+0000 7f5d10047b80 1 Processor -- start 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99a9580 0x5651f99b2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99b2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f989aa60 con 0x5651f99af2d0 2023-07-21T19:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f989a760 con 0x5651f99a9580 2023-07-21T19:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f977e8c0 con 0x5651f98056b0 2023-07-21T19:18:01.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5d10047b80 10 client.0 _ll_register_callbacks cb 0x5651f9898700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5cfa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5cfa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58732/0 (socket says 172.21.15.5:58732) 2023-07-21T19:18:01.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.655+0000 7f5cfa59b700 1 -- 172.21.15.5:0/396341881 learned_addr learned my addr 172.21.15.5:0/396341881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99b2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99a9580 0x5651f99b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfa59b700 1 -- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99a9580 msgr2=0x5651f99b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99a9580 0x5651f99b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfa59b700 1 -- 172.21.15.5:0/396341881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 msgr2=0x5651f99b2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99b2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfa59b700 1 -- 172.21.15.5:0/396341881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec011040 con 0x5651f98056b0 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.656+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.657+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9c0 tx=0x7f5cf009df00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcb022d5acc06299 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.657+0000 7f5cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.657+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b1070 con 0x5651f98056b0 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.657+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5cf009a080 con 0x5651f98056b0 2023-07-21T19:18:01.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.657+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651f9854490 con 0x5651f98056b0 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf009a280 con 0x5651f98056b0 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 10 client.27238 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f5cf410d240 con 0x5651f98056b0 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5cf00c8080 con 0x5651f98056b0 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cf0095c10 con 0x5651f98056b0 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 10 client.27238 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 10 client.27238 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 20 client.27238 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f99b99e0 0x5651f99bbdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 4 client.27238 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.658+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651f98f3e50 con 0x5651f99b99e0 2023-07-21T19:18:01.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.659+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f99b99e0 0x5651f99bbdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.660+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f99b99e0 0x5651f99bbdf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=786ff11018b8a268 server_cookie=a02cc756b6f593fe in_seq=0 out_seq=0 2023-07-21T19:18:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.660+0000 7f5cddffb700 10 client.27238 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5cddffb700 1 -- 172.21.15.5:0/396341881 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5651f98f3e50 con 0x5651f99b99e0 2023-07-21T19:18:01.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5cddffb700 10 client.27238 handle_command_reply: tid=0 2023-07-21T19:18:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5d10047b80 1 client.27238 shutdown 2023-07-21T19:18:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 msgr2=0x5651f99b55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9c0 tx=0x7f5cf009df00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 shutdown_connections 2023-07-21T19:18:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.661+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f99b99e0 0x5651f99bbdf0 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99b2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99b55c0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 --2- 172.21.15.5:0/396341881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99a9580 0x5651f99b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5cddffb700 0 client.27238 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 >> 172.21.15.5:0/396341881 conn(0x5651f999ebe0 msgr2=0x5651f99a2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 shutdown_connections 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 1 -- 172.21.15.5:0/396341881 wait complete. 2023-07-21T19:18:01.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.662+0000 7f5d10047b80 20 client.27238 trim_cache size 0 max 16384 2023-07-21T19:18:01.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.692+0000 7f5d10047b80 1 Processor -- start 2023-07-21T19:18:01.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 -- start start 2023-07-21T19:18:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99af6a0 con 0x5651f99a9580 2023-07-21T19:18:01.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99a4fb0 con 0x5651f99af2d0 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.693+0000 7f5d10047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99a5800 con 0x5651f98056b0 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58738/0 (socket says 172.21.15.5:58738) 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2330317475 learned_addr learned my addr 172.21.15.5:0/2330317475 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2330317475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 msgr2=0x5651f99a3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2330317475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 msgr2=0x5651f99ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2330317475 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651f9711c00 con 0x5651f98056b0 2023-07-21T19:18:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.694+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0099b20 tx=0x7f5cf0000bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9b580ad0f538b48 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5ce97fa700 1 -- 172.21.15.5:0/2330317475 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b1030 con 0x5651f98056b0 2023-07-21T19:18:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5ce97fa700 1 -- 172.21.15.5:0/2330317475 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651f9854490 con 0x5651f98056b0 2023-07-21T19:18:01.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5ce97fa700 1 -- 172.21.15.5:0/2330317475 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00a2040 con 0x5651f98056b0 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5d10047b80 1 -- 172.21.15.5:0/2330317475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 msgr2=0x5651f99a9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0099b20 tx=0x7f5cf0000bc0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.695+0000 7f5d10047b80 1 -- 172.21.15.5:0/2330317475 shutdown_connections 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f99af2d0 0x5651f99a3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f98056b0 0x5651f99a9040 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2330317475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99a9580 0x5651f99ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 -- 172.21.15.5:0/2330317475 >> 172.21.15.5:0/2330317475 conn(0x5651f999ebe0 msgr2=0x5651f99afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 -- 172.21.15.5:0/2330317475 shutdown_connections 2023-07-21T19:18:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 -- 172.21.15.5:0/2330317475 wait complete. 2023-07-21T19:18:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.696+0000 7f5d10047b80 1 -- start start 2023-07-21T19:18:01.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 Processor -- start 2023-07-21T19:18:01.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 0x5651f99abe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 0x5651f99aa790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99ac430 con 0x5651f99af2d0 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99ac5a0 con 0x5651f98056b0 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.697+0000 7f5d10047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651f99ab210 con 0x5651f99a9580 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5d10047b80 10 client.0 _ll_register_callbacks cb 0x7f5cf0099870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 0x5651f99aa790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5cf9d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 0x5651f99aa790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58744/0 (socket says 172.21.15.5:58744) 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5cf9d9a700 1 -- 172.21.15.5:0/2153669795 learned_addr learned my addr 172.21.15.5:0/2153669795 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 0x5651f99abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.698+0000 7f5cfad9c700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2153669795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 msgr2=0x5651f99aa790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 0x5651f99aa790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2153669795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 msgr2=0x5651f99aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 -- 172.21.15.5:0/2153669795 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cf410d240 con 0x5651f98056b0 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cfa59b700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 0x5651f99abe30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0099fe0 tx=0x7f5cf0099a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6409fe7a0846724 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cddffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cf00b1070 con 0x5651f98056b0 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5cf00a2270 con 0x5651f98056b0 2023-07-21T19:18:01.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.699+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ce4003d00 con 0x5651f98056b0 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5cf0099310 con 0x5651f98056b0 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 10 client.27314 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5651f974f1f0 con 0x5651f98056b0 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ce4003d00 con 0x5651f98056b0 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5cf0004140 con 0x5651f98056b0 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 10 client.27314 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 10 client.27314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 20 client.27314 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f9a28280 0x5651f9a2a660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 4 client.27314 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:18:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.700+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5651f98b5ff0 con 0x5651f9a28280 2023-07-21T19:18:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.701+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f9a28280 0x5651f9a2a660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.702+0000 7f5cf9d9a700 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f9a28280 0x5651f9a2a660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1cf7e989bc2b7f42 server_cookie=469eb789a1cdb948 in_seq=0 out_seq=0 2023-07-21T19:18:01.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.702+0000 7f5cddffb700 10 client.27314 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.718+0000 7f5cddffb700 1 -- 172.21.15.5:0/2153669795 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5651f98b5ff0 con 0x5651f9a28280 2023-07-21T19:18:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.718+0000 7f5cddffb700 10 client.27314 handle_command_reply: tid=0 2023-07-21T19:18:01.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.718+0000 7f5d10047b80 1 client.27314 shutdown 2023-07-21T19:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 msgr2=0x5651f99abe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 0x5651f99abe30 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0099fe0 tx=0x7f5cf0099a20 comp rx=0 tx=0).stop 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 shutdown_connections 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651f9a28280 0x5651f9a2a660 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651f99af2d0 0x5651f99aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651f98056b0 0x5651f99abe30 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5cddffb700 0 client.27314 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 --2- 172.21.15.5:0/2153669795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651f99a9580 0x5651f99aa790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 >> 172.21.15.5:0/2153669795 conn(0x5651f999ebe0 msgr2=0x5651f99b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.720+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 shutdown_connections 2023-07-21T19:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.721+0000 7f5d10047b80 1 -- 172.21.15.5:0/2153669795 wait complete. 2023-07-21T19:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.721+0000 7f5d10047b80 20 client.27314 trim_cache size 0 max 16384 2023-07-21T19:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:18:01.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 msgr2=0x7f5ce00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 0x7f5ce00a3770 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f5cf419a690 tx=0x7f5cec009100 comp rx=0 tx=0).stop 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 msgr2=0x7f5cf4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 0x7f5cf4199990 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f5cf000e9f0 tx=0x7f5cf000ed00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 shutdown_connections 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5ce00a12c0 0x7f5ce00a3770 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5cf4105b10 0x7f5cf41937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5cf40ffb00 0x7f5cf4199990 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 --2- 172.21.15.5:0/258571916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5cf41004b0 0x7f5cf4199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 >> 172.21.15.5:0/258571916 conn(0x7f5cf40fb600 msgr2=0x7f5cf40fbed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 shutdown_connections 2023-07-21T19:18:01.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:01.722+0000 7f5cfc7ff700 1 -- 172.21.15.5:0/258571916 wait complete. 2023-07-21T19:18:01.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 -- 172.21.15.5:0/61650862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff9d0 msgr2=0x7fbb000ffe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 --2- 172.21.15.5:0/61650862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff9d0 0x7fbb000ffe40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009990 tx=0x7fbaf4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 -- 172.21.15.5:0/61650862 shutdown_connections 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 --2- 172.21.15.5:0/61650862 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb00100380 0x7fbb0010c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 --2- 172.21.15.5:0/61650862 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb001059e0 0x7fbb00105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 --2- 172.21.15.5:0/61650862 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb000ff9d0 0x7fbb000ffe40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.762+0000 7fbafffff700 1 -- 172.21.15.5:0/61650862 >> 172.21.15.5:0/61650862 conn(0x7fbb000fb4b0 msgr2=0x7fbb000fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.763+0000 7fbafffff700 1 -- 172.21.15.5:0/61650862 shutdown_connections 2023-07-21T19:18:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.763+0000 7fbafffff700 1 -- 172.21.15.5:0/61650862 wait complete. 2023-07-21T19:18:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.764+0000 7fbafffff700 1 Processor -- start 2023-07-21T19:18:01.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.764+0000 7fbafffff700 1 -- start start 2023-07-21T19:18:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 0x7fbb0019b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb00100380 0x7fbb0019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00195db0 con 0x7fbb00100380 2023-07-21T19:18:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb00195f20 con 0x7fbb001059e0 2023-07-21T19:18:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.765+0000 7fbafffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb001960e0 con 0x7fbb000ff9d0 2023-07-21T19:18:01.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47220/0 (socket says 172.21.15.5:47220) 2023-07-21T19:18:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbaff7fe700 1 -- 172.21.15.5:0/2005536887 learned_addr learned my addr 172.21.15.5:0/2005536887 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 0x7fbb0019b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb00100380 0x7fbb0019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 -- 172.21.15.5:0/2005536887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 msgr2=0x7fbb00195740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 -- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb00100380 msgr2=0x7fbb0019bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb00100380 0x7fbb0019bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbafeffd700 1 -- 172.21.15.5:0/2005536887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbaf40095f0 con 0x7fbb000ff9d0 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.766+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbafeffd700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 0x7fbb0019b6d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e9c0 tx=0x7fbaf000ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbae7fff700 1 -- 172.21.15.5:0/2005536887 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0004220 con 0x7fbb000ff9d0 2023-07-21T19:18:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbae7fff700 1 -- 172.21.15.5:0/2005536887 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf0004380 con 0x7fbb000ff9d0 2023-07-21T19:18:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb001963c0 con 0x7fbb000ff9d0 2023-07-21T19:18:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbae7fff700 1 -- 172.21.15.5:0/2005536887 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0003b40 con 0x7fbb000ff9d0 2023-07-21T19:18:01.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.767+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb001a26c0 con 0x7fbb000ff9d0 2023-07-21T19:18:01.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.773+0000 7fbae7fff700 1 -- 172.21.15.5:0/2005536887 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbaf0003ca0 con 0x7fbb000ff9d0 2023-07-21T19:18:01.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.774+0000 7fbb1908fb80 1 Processor -- start 2023-07-21T19:18:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.774+0000 7fbb1908fb80 1 -- start start 2023-07-21T19:18:01.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.774+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.774+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c35037cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c3503917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c350281cb0 con 0x55c35037e2f0 2023-07-21T19:18:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3502d4a40 con 0x55c35038f3d0 2023-07-21T19:18:01.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3502cded0 con 0x55c35037ca30 2023-07-21T19:18:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42250/0 (socket says 172.21.15.5:42250) 2023-07-21T19:18:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbaff7fe700 1 -- 172.21.15.5:0/3268061141 learned_addr learned my addr 172.21.15.5:0/3268061141 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c3503917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.775+0000 7fbafeffd700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c35037cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbae7fff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 0x7fbae80a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 -- 172.21.15.5:0/3268061141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 msgr2=0x55c35037cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c35037cea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 -- 172.21.15.5:0/3268061141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 msgr2=0x55c35037c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 -- 172.21.15.5:0/3268061141 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3500fbde0 con 0x55c35038f3d0 2023-07-21T19:18:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbae7fff700 1 -- 172.21.15.5:0/2005536887 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbaf0014070 con 0x7fbb000ff9d0 2023-07-21T19:18:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c3503917b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005910 tx=0x7fbaf4005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37ebe858e2177163 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbae57fa700 1 -- 172.21.15.5:0/3268061141 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401d070 con 0x55c35038f3d0 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbae57fa700 1 -- 172.21.15.5:0/3268061141 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf401e040 con 0x55c35038f3d0 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.776+0000 7fbae57fa700 1 -- 172.21.15.5:0/3268061141 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf4004760 con 0x55c35038f3d0 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.777+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 0x7fbae80a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.777+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3268061141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 msgr2=0x55c3503917b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.777+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c3503917b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4005910 tx=0x7fbaf4005f00 comp rx=0 tx=0).stop 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3268061141 shutdown_connections 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c3503917b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35037c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3268061141 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c35037cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3268061141 >> 172.21.15.5:0/3268061141 conn(0x55c350371ff0 msgr2=0x55c350377930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3268061141 shutdown_connections 2023-07-21T19:18:01.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 0x7fbae80a3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007980 tx=0x7fbaf8004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.778+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3268061141 wait complete. 2023-07-21T19:18:01.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.779+0000 7fbb1908fb80 1 -- start start 2023-07-21T19:18:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.779+0000 7fbb1908fb80 1 Processor -- start 2023-07-21T19:18:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 0x55c350382210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 0x55c35037f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35038f3d0 0x55c35037fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35026dac0 con 0x55c35038f3d0 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35026d7c0 con 0x55c35037ca30 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c350151920 con 0x55c35037e2f0 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.780+0000 7fbb1908fb80 10 client.0 _ll_register_callbacks cb 0x55c35026b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 0x55c35037f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbaff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35038f3d0 0x55c35037fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 0x55c35037f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58792/0 (socket says 172.21.15.5:58792) 2023-07-21T19:18:01.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafe7fc700 1 -- 172.21.15.5:0/3908202074 learned_addr learned my addr 172.21.15.5:0/3908202074 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 0x55c350382210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 -- 172.21.15.5:0/3908202074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 msgr2=0x55c35037f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 0x55c35037f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 -- 172.21.15.5:0/3908202074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35038f3d0 msgr2=0x55c35037fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35038f3d0 0x55c35037fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.781+0000 7fbafeffd700 1 -- 172.21.15.5:0/3908202074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbaf8009170 con 0x55c35037ca30 2023-07-21T19:18:01.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbafeffd700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 0x55c350382210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ca580 tx=0x7fbaf0099ac0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f47503f08ae1298 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0016330 con 0x55c35037ca30 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbaf0099f00 con 0x55c35037ca30 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3502274f0 con 0x55c35037ca30 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf009df30 con 0x55c35037ca30 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbb1908fb80 10 client.27326 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fbb0010d110 con 0x55c35037ca30 2023-07-21T19:18:01.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.782+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c3502274f0 con 0x55c35037ca30 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbaf00a1ec0 con 0x55c35037ca30 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 10 client.27326 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 10 client.27326 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 20 client.27326 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c35039eef0 0x55c3503a1300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 4 client.27326 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3502c6eb0 con 0x55c35039eef0 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.783+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c35039eef0 0x55c3503a1300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.784+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c35039eef0 0x55c3503a1300 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1b443cdbcd11915 server_cookie=b9f02f2a27932db4 in_seq=0 out_seq=0 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.784+0000 7fbae2ff5700 10 client.27326 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbae2ff5700 1 -- 172.21.15.5:0/3908202074 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3502c6eb0 con 0x55c35039eef0 2023-07-21T19:18:01.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbae2ff5700 10 client.27326 handle_command_reply: tid=0 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 client.27326 shutdown 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 msgr2=0x55c350382210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 0x55c350382210 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbaf00ca580 tx=0x7fbaf0099ac0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 shutdown_connections 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c35039eef0 0x55c3503a1300 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35038f3d0 0x55c35037fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037ca30 0x55c350382210 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/3908202074 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037e2f0 0x55c35037f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbae2ff5700 0 client.27326 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 >> 172.21.15.5:0/3908202074 conn(0x55c350371ff0 msgr2=0x55c350377430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 shutdown_connections 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.785+0000 7fbb1908fb80 1 -- 172.21.15.5:0/3908202074 wait complete. 2023-07-21T19:18:01.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.786+0000 7fbb1908fb80 20 client.27326 trim_cache size 0 max 16384 2023-07-21T19:18:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 Processor -- start 2023-07-21T19:18:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 -- start start 2023-07-21T19:18:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c350381db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c35038edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c350377b10 con 0x55c35037e2f0 2023-07-21T19:18:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35039c600 con 0x55c35038f3d0 2023-07-21T19:18:01.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35037df50 con 0x55c35037ca30 2023-07-21T19:18:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42274/0 (socket says 172.21.15.5:42274) 2023-07-21T19:18:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.812+0000 7fbafe7fc700 1 -- 172.21.15.5:0/511921311 learned_addr learned my addr 172.21.15.5:0/511921311 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c35038edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafeffd700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c350381db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 -- 172.21.15.5:0/511921311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 msgr2=0x55c350381db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c350381db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 -- 172.21.15.5:0/511921311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 msgr2=0x55c35038edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c35038edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 -- 172.21.15.5:0/511921311 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c350373830 con 0x55c35037e2f0 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafeffd700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c350381db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:01.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009590 tx=0x7fbaf401a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1def0bd27a2682b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.813+0000 7fbae5ffb700 1 -- 172.21.15.5:0/511921311 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401d040 con 0x55c35037e2f0 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbae5ffb700 1 -- 172.21.15.5:0/511921311 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf4034860 con 0x55c35037e2f0 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbae5ffb700 1 -- 172.21.15.5:0/511921311 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf400bb40 con 0x55c35037e2f0 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 -- 172.21.15.5:0/511921311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 msgr2=0x55c35038e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009590 tx=0x7fbaf401a040 comp rx=0 tx=0).stop 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 -- 172.21.15.5:0/511921311 shutdown_connections 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35038f3d0 0x55c35038edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35037ca30 0x55c350381db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/511921311 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037e2f0 0x55c35038e880 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.814+0000 7fbb1908fb80 1 -- 172.21.15.5:0/511921311 >> 172.21.15.5:0/511921311 conn(0x55c350371ff0 msgr2=0x55c35038b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.815+0000 7fbb1908fb80 1 -- 172.21.15.5:0/511921311 shutdown_connections 2023-07-21T19:18:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.815+0000 7fbb1908fb80 1 -- 172.21.15.5:0/511921311 wait complete. 2023-07-21T19:18:01.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.815+0000 7fbb1908fb80 1 -- start start 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.815+0000 7fbb1908fb80 1 Processor -- start 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037ca30 0x55c350395c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35038f3d0 0x55c350394130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c350394840 con 0x55c35037ca30 2023-07-21T19:18:01.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3503949b0 con 0x55c35037e2f0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c350394b70 con 0x55c35038f3d0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbb1908fb80 10 client.0 _ll_register_callbacks cb 0x7fbaf0096330 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbafe7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47248/0 (socket says 172.21.15.5:47248) 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.816+0000 7fbafe7fc700 1 -- 172.21.15.5:0/1343139708 learned_addr learned my addr 172.21.15.5:0/1343139708 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbaff7fe700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35038f3d0 0x55c350394130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafeffd700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037ca30 0x55c350395c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 -- 172.21.15.5:0/1343139708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35038f3d0 msgr2=0x55c350394130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35038f3d0 0x55c350394130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 -- 172.21.15.5:0/1343139708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037ca30 msgr2=0x55c350395c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037ca30 0x55c350395c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 -- 172.21.15.5:0/1343139708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3500fbde0 con 0x55c35037e2f0 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbafe7fc700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009520 tx=0x7fbaf4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24a9524c3e0f56df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.817+0000 7fbae2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf401d040 con 0x55c35037e2f0 2023-07-21T19:18:01.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbaf401e040 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbaf4034bd0 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbaf4034dd0 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbb1908fb80 10 client.27332 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55c3500e4c60 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.818+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbaf401e040 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbaf4004870 con 0x55c35037e2f0 2023-07-21T19:18:01.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 10 client.27332 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 10 client.27332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 20 client.27332 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3503f3ba0 0x55c3503f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 4 client.27332 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c350289050 con 0x55c3503f3ba0 2023-07-21T19:18:01.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.819+0000 7fbafeffd700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3503f3ba0 0x55c3503f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:01.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbae2ff5700 10 client.27332 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbafeffd700 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3503f3ba0 0x55c3503f5fb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e8945fed2cba0 server_cookie=f307b040af6dbf91 in_seq=0 out_seq=0 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbae2ff5700 1 -- 172.21.15.5:0/1343139708 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c350289050 con 0x55c3503f3ba0 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbae2ff5700 10 client.27332 handle_command_reply: tid=0 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 client.27332 shutdown 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 msgr2=0x55c3503961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbaf4009520 tx=0x7fbaf4005770 comp rx=0 tx=0).stop 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 shutdown_connections 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c35038f3d0 0x55c350394130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c3503f3ba0 0x55c3503f5fb0 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c35037ca30 0x55c350395c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 --2- 172.21.15.5:0/1343139708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c35037e2f0 0x55c3503961b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbae2ff5700 0 client.27332 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:01.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.821+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 >> 172.21.15.5:0/1343139708 conn(0x55c350371ff0 msgr2=0x55c35038e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 shutdown_connections 2023-07-21T19:18:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbb1908fb80 1 -- 172.21.15.5:0/1343139708 wait complete. 2023-07-21T19:18:01.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbb1908fb80 20 client.27332 trim_cache size 0 max 16384 2023-07-21T19:18:01.825 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "83bdab96-51d1-4839-9d4c-a086f0feb3b9": { 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "83bdab96-51d1-4839-9d4c-a086f0feb3b9", 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:18:01.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:18:01.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:18:01.827 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:18:01.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 msgr2=0x7fbae80a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 0x7fbae80a3890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8007980 tx=0x7fbaf8004ad0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 msgr2=0x7fbb0019b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.822+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 0x7fbb0019b6d0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000e9c0 tx=0x7fbaf000ecd0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 shutdown_connections 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbb001059e0 0x7fbb00195740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbb000ff9d0 0x7fbb0019b6d0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbae80a13e0 0x7fbae80a3890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 --2- 172.21.15.5:0/2005536887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbb00100380 0x7fbb0019bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 >> 172.21.15.5:0/2005536887 conn(0x7fbb000fb4b0 msgr2=0x7fbb000fc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 shutdown_connections 2023-07-21T19:18:01.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:01.823+0000 7fbafffff700 1 -- 172.21.15.5:0/2005536887 wait complete. 2023-07-21T19:18:01.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: pgmap v323: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 50 op/s 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.830 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:18:01.444+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...10,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v3468 ap=2 f(v16 m2023-07-21T19:17:26.974402+0000 3=1+2) n(v8 rc2023-07-21T19:17:26.974402+0000 b8 4=1+3) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@104} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v3468>,<0x10000000005/data v2484>,<0x10000000003/postgres v656>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v679>,<0x10000000001/sv_0 v692>,<0x10000000000/_nogroup v673>,<0x1/volumes v646>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:17:26.958416+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:17:26.974402+0000 b8 4=1+3)","memory_value.dirstat":"f(v16 m2023-07-21T19:17:26.974402+0000 3=1+2)","memory_value.rstat":"n(v8 rc2023-07-21T19:17:26.974402+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:18:01.831 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:18:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:18:01.445+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...10,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v3410 ap=2 f(v90 m2023-07-21T19:17:02.614147+0000 3=3+0) n(v9 rc2023-07-21T19:17:02.614147+0000 b3321 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@231} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v3410>,<0x10000000005/data v2484>,<0x10000000003/postgres v656>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v679>,<0x10000000001/sv_0 v692>,<0x10000000000/_nogroup v673>,<0x1/volumes v646>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:17:02.599563+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:17:02.614147+0000 b3321 4=3+1)","memory_value.dirstat":"f(v90 m2023-07-21T19:17:02.614147+0000 3=3+0)","memory_value.rstat":"n(v9 rc2023-07-21T19:17:02.614147+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:18:01.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: pgmap v323: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 50 op/s 2023-07-21T19:18:01.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:01.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.843 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:01 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.845 INFO:tasks.cephfs.filesystem:scrub status for tag:83bdab96-51d1-4839-9d4c-a086f0feb3b9 - {'path': '/', 'tag': '83bdab96-51d1-4839-9d4c-a086f0feb3b9', 'options': 'recursive,force'} 2023-07-21T19:18:01.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: pgmap v323: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 90 KiB/s rd, 18 MiB/s wr, 50 op/s 2023-07-21T19:18:01.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:01.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:01.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:01 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:39200000 of 50000000 tuples (78%) done (elapsed 600.00 s, remaining 165.31 s) 2023-07-21T19:18:02.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:18:02.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:02.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:02.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.831 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.832 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:18:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:02.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.908 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.909 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:02.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:02.920 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:18:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:18:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: pgmap v324: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 157 op/s 2023-07-21T19:18:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:18:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:18:03.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:18:03.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:18:03.830 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:03 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:03.894 INFO:teuthology.orchestra.run.smithi005.stderr:39300000 of 50000000 tuples (78%) done (elapsed 601.46 s, remaining 163.76 s) 2023-07-21T19:18:03.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: pgmap v324: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 157 op/s 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.906 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:18:03.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:18:03.907 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:03 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:03.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:18:03.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:18:03.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: pgmap v324: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 176 KiB/s rd, 28 MiB/s wr, 157 op/s 2023-07-21T19:18:03.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:18:03.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:18:03.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.918 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:18:03.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:18:03.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:18:03.919 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:03 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:18:04.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:04 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:18:04.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:04 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:18:04.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:04 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:18:04.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:04 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:18:04.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:04 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:18:04.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:04 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:18:05.229 INFO:teuthology.orchestra.run.smithi005.stderr:39400000 of 50000000 tuples (78%) done (elapsed 602.80 s, remaining 162.17 s) 2023-07-21T19:18:05.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:05 smithi101 ceph-mon[128136]: pgmap v325: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 21 MiB/s wr, 165 op/s 2023-07-21T19:18:05.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:05 smithi005 ceph-mon[126460]: pgmap v325: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 21 MiB/s wr, 165 op/s 2023-07-21T19:18:05.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:05 smithi156 ceph-mon[128316]: pgmap v325: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 184 KiB/s rd, 21 MiB/s wr, 165 op/s 2023-07-21T19:18:06.519 INFO:teuthology.orchestra.run.smithi005.stderr:39500000 of 50000000 tuples (79%) done (elapsed 604.09 s, remaining 160.58 s) 2023-07-21T19:18:07.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:07 smithi101 ceph-mon[128136]: pgmap v326: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 19 MiB/s wr, 159 op/s 2023-07-21T19:18:07.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:07 smithi101 ceph-mon[128136]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T19:18:07.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:07 smithi005 ceph-mon[126460]: pgmap v326: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 19 MiB/s wr, 159 op/s 2023-07-21T19:18:07.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:07 smithi005 ceph-mon[126460]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T19:18:07.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:07 smithi156 ceph-mon[128316]: pgmap v326: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 19 MiB/s wr, 159 op/s 2023-07-21T19:18:07.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:07 smithi156 ceph-mon[128316]: osdmap e97: 12 total, 12 up, 12 in 2023-07-21T19:18:07.996 INFO:teuthology.orchestra.run.smithi005.stderr:39600000 of 50000000 tuples (79%) done (elapsed 605.57 s, remaining 159.04 s) 2023-07-21T19:18:08.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:08 smithi005 ceph-mon[126460]: pgmap v328: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:08.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:08 smithi156 ceph-mon[128316]: pgmap v328: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:09.077 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:08 smithi101 ceph-mon[128136]: pgmap v328: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:09.294 INFO:teuthology.orchestra.run.smithi005.stderr:39700000 of 50000000 tuples (79%) done (elapsed 606.86 s, remaining 157.45 s) 2023-07-21T19:18:10.618 INFO:teuthology.orchestra.run.smithi005.stderr:39800000 of 50000000 tuples (79%) done (elapsed 608.19 s, remaining 155.87 s) 2023-07-21T19:18:11.577 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:11 smithi101 ceph-mon[128136]: pgmap v329: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:11.578 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:11 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:11.578 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:11 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:11.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:11 smithi005 ceph-mon[126460]: pgmap v329: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:11.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:11 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:11.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:11 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:11.666 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:11 smithi156 ceph-mon[128316]: pgmap v329: 97 pgs: 97 active+clean; 8.1 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 25 MiB/s wr, 191 op/s 2023-07-21T19:18:11.666 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:11 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:11.667 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:11 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:12.218 INFO:teuthology.orchestra.run.smithi005.stderr:39900000 of 50000000 tuples (79%) done (elapsed 609.79 s, remaining 154.36 s) 2023-07-21T19:18:12.577 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:12 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:12.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:12 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:12.666 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:12 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:40000000 of 50000000 tuples (80%) done (elapsed 611.11 s, remaining 152.78 s) 2023-07-21T19:18:13.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:13 smithi101 ceph-mon[128136]: pgmap v330: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 93 op/s 2023-07-21T19:18:13.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:13 smithi101 ceph-mon[128136]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T19:18:13.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:13 smithi005 ceph-mon[126460]: pgmap v330: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 93 op/s 2023-07-21T19:18:13.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:13 smithi005 ceph-mon[126460]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T19:18:13.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:13 smithi156 ceph-mon[128316]: pgmap v330: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 25 MiB/s wr, 93 op/s 2023-07-21T19:18:13.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:13 smithi156 ceph-mon[128316]: osdmap e98: 12 total, 12 up, 12 in 2023-07-21T19:18:14.829 INFO:teuthology.orchestra.run.smithi005.stderr:40100000 of 50000000 tuples (80%) done (elapsed 612.40 s, remaining 151.19 s) 2023-07-21T19:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3669135303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28102950 msgr2=0x7fab28106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3669135303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28102950 0x7fab28106910 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fab180099c0 tx=0x7fab18009cd0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3669135303 shutdown_connections 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3669135303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28106ff0 0x7fab2810ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3669135303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 0x7fab28102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3669135303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28102950 0x7fab28106910 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3669135303 >> 172.21.15.5:0/3669135303 conn(0x7fab280fd800 msgr2=0x7fab280ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3669135303 shutdown_connections 2023-07-21T19:18:14.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.899+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3669135303 wait complete. 2023-07-21T19:18:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.900+0000 7fab2f1ff700 1 Processor -- start 2023-07-21T19:18:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.900+0000 7fab2f1ff700 1 -- start start 2023-07-21T19:18:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 0x7fab28197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 0x7fab281983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2819d050 con 0x7fab28106ff0 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2819d1c0 con 0x7fab28102950 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2f1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab2819d380 con 0x7fab28101fb0 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 0x7fab28197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35018/0 (socket says 172.21.15.5:35018) 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.901+0000 7fab2d79c700 1 -- 172.21.15.5:0/3149082989 learned_addr learned my addr 172.21.15.5:0/3149082989 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab1ffff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 0x7fab281983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 -- 172.21.15.5:0/3149082989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 msgr2=0x7fab28197e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 0x7fab28197e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 -- 172.21.15.5:0/3149082989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 msgr2=0x7fab281983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 0x7fab281983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 -- 172.21.15.5:0/3149082989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab180095f0 con 0x7fab28106ff0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab1ffff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 0x7fab281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fab20007ee0 tx=0x7fab2000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab1dffb700 1 -- 172.21.15.5:0/3149082989 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20017070 con 0x7fab28106ff0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab2819d660 con 0x7fab28106ff0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab1dffb700 1 -- 172.21.15.5:0/3149082989 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab2000f040 con 0x7fab28106ff0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.902+0000 7fab1dffb700 1 -- 172.21.15.5:0/3149082989 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20005710 con 0x7fab28106ff0 2023-07-21T19:18:14.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.903+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab281a4ea0 con 0x7fab28106ff0 2023-07-21T19:18:14.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.906+0000 7fab1dffb700 1 -- 172.21.15.5:0/3149082989 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fab20005870 con 0x7fab28106ff0 2023-07-21T19:18:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.908+0000 7fab42a3eb80 1 Processor -- start 2023-07-21T19:18:14.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.908+0000 7fab1dffb700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 0x7fab100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.908+0000 7fab1dffb700 1 -- 172.21.15.5:0/3149082989 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fab200cc520 con 0x7fab28106ff0 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 -- start start 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 0x7fab100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc673226b0 0x55bc674cbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc673cbc50 con 0x55bc674c6580 2023-07-21T19:18:14.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc6741e9e0 con 0x55bc673226b0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc67417e70 con 0x55bc674cc2d0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35026/0 (socket says 172.21.15.5:35026) 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.909+0000 7fab2d79c700 1 -- 172.21.15.5:0/3909506985 learned_addr learned my addr 172.21.15.5:0/3909506985 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab1ffff700 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 0x7fab100a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab24005950 tx=0x7fab2400b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc673226b0 0x55bc674cbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 -- 172.21.15.5:0/3909506985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 msgr2=0x55bc674c6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 -- 172.21.15.5:0/3909506985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc673226b0 msgr2=0x55bc674cbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc673226b0 0x55bc674cbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 -- 172.21.15.5:0/3909506985 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc67245d80 con 0x55bc674c6580 2023-07-21T19:18:14.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab2d79c700 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fab20096000 tx=0x7fab2000df60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb3c0c7859c1182c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.910+0000 7fab0effd700 1 -- 172.21.15.5:0/3909506985 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2000dc60 con 0x55bc674c6580 2023-07-21T19:18:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab0effd700 1 -- 172.21.15.5:0/3909506985 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab20099590 con 0x55bc674c6580 2023-07-21T19:18:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab0effd700 1 -- 172.21.15.5:0/3909506985 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20095ca0 con 0x55bc674c6580 2023-07-21T19:18:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 -- 172.21.15.5:0/3909506985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 msgr2=0x55bc674c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fab20096000 tx=0x7fab2000df60 comp rx=0 tx=0).stop 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 -- 172.21.15.5:0/3909506985 shutdown_connections 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674c6580 0x55bc674c69d0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc673226b0 0x55bc674cbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/3909506985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.911+0000 7fab42a3eb80 1 -- 172.21.15.5:0/3909506985 >> 172.21.15.5:0/3909506985 conn(0x55bc674bbbe0 msgr2=0x55bc674c0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.912+0000 7fab42a3eb80 1 -- 172.21.15.5:0/3909506985 shutdown_connections 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.912+0000 7fab42a3eb80 1 -- 172.21.15.5:0/3909506985 wait complete. 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.912+0000 7fab42a3eb80 1 -- start start 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 Processor -- start 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 0x55bc674d2610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674cf890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc673b7a60 con 0x55bc674cc2d0 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc673b7760 con 0x55bc674c6580 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc6729b8c0 con 0x55bc673226b0 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.913+0000 7fab42a3eb80 10 client.0 _ll_register_callbacks cb 0x55bc673b5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 0x55bc674d2610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35030/0 (socket says 172.21.15.5:35030) 2023-07-21T19:18:14.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 -- 172.21.15.5:0/2055988118 learned_addr learned my addr 172.21.15.5:0/2055988118 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab1ffff700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674cf890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 -- 172.21.15.5:0/2055988118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 msgr2=0x55bc674d2610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 0x55bc674d2610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 -- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 msgr2=0x55bc674cf890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674cf890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 -- 172.21.15.5:0/2055988118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab281a4ea0 con 0x55bc674cc2d0 2023-07-21T19:18:14.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 0x55bc674d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab1ffff700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:18:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab2d79c700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab20095a90 tx=0x7fab200ca0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b383a561c1b1b9a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.914+0000 7fab07fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:14.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab20084260 con 0x55bc674cc2d0 2023-07-21T19:18:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fab2000dc60 con 0x55bc674cc2d0 2023-07-21T19:18:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bc67371490 con 0x55bc674cc2d0 2023-07-21T19:18:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab200a19e0 con 0x55bc674cc2d0 2023-07-21T19:18:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab42a3eb80 10 client.17664 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:14.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bc6722ec00 con 0x55bc674cc2d0 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bc67371490 con 0x55bc674cc2d0 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.915+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab200c5d90 con 0x55bc674cc2d0 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 10 client.17664 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 10 client.17664 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 20 client.17664 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:14.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc674df6f0 0x55bc674e1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 4 client.17664 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bc67410e50 con 0x55bc674df6f0 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.916+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc674df6f0 0x55bc674e1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.917+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc674df6f0 0x55bc674e1ad0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d91c21bba7686389 server_cookie=a59ff4b33a770bd2 in_seq=0 out_seq=0 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.917+0000 7fab07fff700 10 client.17664 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.917+0000 7fab07fff700 1 -- 172.21.15.5:0/2055988118 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bc67410e50 con 0x55bc674df6f0 2023-07-21T19:18:14.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.917+0000 7fab07fff700 10 client.17664 handle_command_reply: tid=0 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 client.17664 shutdown 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 msgr2=0x55bc674cffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7fab20095a90 tx=0x7fab200ca0a0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 shutdown_connections 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc674df6f0 0x55bc674e1ad0 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc674cc2d0 0x55bc674cffd0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc673226b0 0x55bc674d2610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab07fff700 0 client.17664 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/2055988118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674cf890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 >> 172.21.15.5:0/2055988118 conn(0x55bc674bbbe0 msgr2=0x55bc674c04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 shutdown_connections 2023-07-21T19:18:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 1 -- 172.21.15.5:0/2055988118 wait complete. 2023-07-21T19:18:14.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.918+0000 7fab42a3eb80 20 client.17664 trim_cache size 0 max 16384 2023-07-21T19:18:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.935+0000 7fab42a3eb80 1 Processor -- start 2023-07-21T19:18:14.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.935+0000 7fab42a3eb80 1 -- start start 2023-07-21T19:18:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.935+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.935+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674d6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674c7060 con 0x55bc673226b0 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674cde10 con 0x55bc674c6580 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674ca680 con 0x55bc674cc2d0 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35036/0 (socket says 172.21.15.5:35036) 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 -- 172.21.15.5:0/1615069804 learned_addr learned my addr 172.21.15.5:0/1615069804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:14.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab1ffff700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674d6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 -- 172.21.15.5:0/1615069804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 msgr2=0x55bc674c0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2d79c700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 -- 172.21.15.5:0/1615069804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 msgr2=0x55bc674d6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674d6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2cf9b700 1 -- 172.21.15.5:0/1615069804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bc6726c1f0 con 0x55bc673226b0 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab2d79c700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:18:14.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.936+0000 7fab1ffff700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.937+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55bc674e23b0 tx=0x7fab24013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28c88091e3c08d56 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.937+0000 7fab0f7fe700 1 -- 172.21.15.5:0/1615069804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24024040 con 0x55bc673226b0 2023-07-21T19:18:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.937+0000 7fab0f7fe700 1 -- 172.21.15.5:0/1615069804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab24015070 con 0x55bc673226b0 2023-07-21T19:18:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.937+0000 7fab0f7fe700 1 -- 172.21.15.5:0/1615069804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab2401f760 con 0x55bc673226b0 2023-07-21T19:18:14.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- 172.21.15.5:0/1615069804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 msgr2=0x55bc674c6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x55bc674e23b0 tx=0x7fab24013000 comp rx=0 tx=0).stop 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- 172.21.15.5:0/1615069804 shutdown_connections 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674cc2d0 0x55bc674c0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c6040 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/1615069804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674c6580 0x55bc674d6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- 172.21.15.5:0/1615069804 >> 172.21.15.5:0/1615069804 conn(0x55bc674bbbe0 msgr2=0x55bc674d2240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- 172.21.15.5:0/1615069804 shutdown_connections 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- 172.21.15.5:0/1615069804 wait complete. 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 -- start start 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.938+0000 7fab42a3eb80 1 Processor -- start 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674c6580 0x55bc674c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674cc2d0 0x55bc674c7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674c7da0 con 0x55bc673226b0 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674c7f10 con 0x55bc674cc2d0 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bc674c80d0 con 0x55bc674c6580 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab42a3eb80 10 client.0 _ll_register_callbacks cb 0x7fab18009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab2cf9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35050/0 (socket says 172.21.15.5:35050) 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab2cf9b700 1 -- 172.21.15.5:0/756071439 learned_addr learned my addr 172.21.15.5:0/756071439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:14.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab1ffff700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674c6580 0x55bc674c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.939+0000 7fab2d79c700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674cc2d0 0x55bc674c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 -- 172.21.15.5:0/756071439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674c6580 msgr2=0x55bc674c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674c6580 0x55bc674c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 -- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674cc2d0 msgr2=0x55bc674c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674cc2d0 0x55bc674c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 -- 172.21.15.5:0/756071439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab2402a9c0 con 0x55bc673226b0 2023-07-21T19:18:14.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab2cf9b700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c010 tx=0x7fab2402a520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a81ac11c489b284 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab07fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24024070 con 0x55bc673226b0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fab20099590 con 0x55bc673226b0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.940+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fab24017070 con 0x55bc673226b0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fab24011aa0 con 0x55bc673226b0 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 10 client.17670 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:14.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55bc67244250 con 0x55bc673226b0 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fab24049080 con 0x55bc673226b0 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fab24043030 con 0x55bc673226b0 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 10 client.17670 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 10 client.17670 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 20 client.17670 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc67534310 0x55bc67536720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:14.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 4 client.17670 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:18:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.941+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bc673d2ff0 con 0x55bc67534310 2023-07-21T19:18:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.942+0000 7fab1ffff700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc67534310 0x55bc67536720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.942+0000 7fab1ffff700 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc67534310 0x55bc67536720 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a132cff0191a9dea server_cookie=fe6d6497061c2fc3 in_seq=0 out_seq=0 2023-07-21T19:18:14.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.942+0000 7fab07fff700 10 client.17670 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab07fff700 1 -- 172.21.15.5:0/756071439 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bc673d2ff0 con 0x55bc67534310 2023-07-21T19:18:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab07fff700 10 client.17670 handle_command_reply: tid=0 2023-07-21T19:18:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 client.17670 shutdown 2023-07-21T19:18:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 msgr2=0x55bc674c8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c010 tx=0x7fab2402a520 comp rx=0 tx=0).stop 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 shutdown_connections 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bc674cc2d0 0x55bc674c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bc67534310 0x55bc67536720 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bc673226b0 0x55bc674c8b80 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 --2- 172.21.15.5:0/756071439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bc674c6580 0x55bc674c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 >> 172.21.15.5:0/756071439 conn(0x55bc674bbbe0 msgr2=0x55bc674caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab07fff700 0 client.17670 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 shutdown_connections 2023-07-21T19:18:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.959+0000 7fab42a3eb80 1 -- 172.21.15.5:0/756071439 wait complete. 2023-07-21T19:18:14.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab42a3eb80 20 client.17670 trim_cache size 0 max 16384 2023-07-21T19:18:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:18:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:18:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:18:14.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 msgr2=0x7fab100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 0x7fab100a3770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fab24005950 tx=0x7fab2400b410 comp rx=0 tx=0).stop 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 msgr2=0x7fab2819ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fab20007ee0 tx=0x7fab2000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 shutdown_connections 2023-07-21T19:18:14.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fab100a12c0 0x7fab100a3770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fab28106ff0 0x7fab2819ca10 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fab28101fb0 0x7fab28197e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 --2- 172.21.15.5:0/3149082989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fab28102950 0x7fab281983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.960+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 >> 172.21.15.5:0/3149082989 conn(0x7fab280fd800 msgr2=0x7fab280fe310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.961+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 shutdown_connections 2023-07-21T19:18:14.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:14.961+0000 7fab2f1ff700 1 -- 172.21.15.5:0/3149082989 wait complete. 2023-07-21T19:18:15.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:15 smithi101 ceph-mon[128136]: pgmap v332: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 100 op/s 2023-07-21T19:18:15.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:15 smithi005 ceph-mon[126460]: pgmap v332: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 100 op/s 2023-07-21T19:18:15.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:15 smithi156 ceph-mon[128316]: pgmap v332: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 32 MiB/s wr, 100 op/s 2023-07-21T19:18:16.239 INFO:teuthology.orchestra.run.smithi005.stderr:40200000 of 50000000 tuples (80%) done (elapsed 613.81 s, remaining 149.63 s) 2023-07-21T19:18:17.540 INFO:teuthology.orchestra.run.smithi005.stderr:40300000 of 50000000 tuples (80%) done (elapsed 615.11 s, remaining 148.05 s) 2023-07-21T19:18:17.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:17 smithi101 ceph-mon[128136]: pgmap v333: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 83 op/s 2023-07-21T19:18:17.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:17 smithi005 ceph-mon[126460]: pgmap v333: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 83 op/s 2023-07-21T19:18:17.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:17 smithi156 ceph-mon[128316]: pgmap v333: 97 pgs: 97 active+clean; 8.2 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 27 MiB/s wr, 83 op/s 2023-07-21T19:18:18.860 INFO:teuthology.orchestra.run.smithi005.stderr:40400000 of 50000000 tuples (80%) done (elapsed 616.43 s, remaining 146.48 s) 2023-07-21T19:18:19.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:19 smithi101 ceph-mon[128136]: pgmap v334: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:19.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:19 smithi005 ceph-mon[126460]: pgmap v334: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:19.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:19 smithi156 ceph-mon[128316]: pgmap v334: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:20.187 INFO:teuthology.orchestra.run.smithi005.stderr:40500000 of 50000000 tuples (81%) done (elapsed 617.76 s, remaining 144.91 s) 2023-07-21T19:18:21.614 INFO:teuthology.orchestra.run.smithi005.stderr:40600000 of 50000000 tuples (81%) done (elapsed 619.18 s, remaining 143.36 s) 2023-07-21T19:18:21.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:21 smithi101 ceph-mon[128136]: pgmap v335: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:21.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:21 smithi005 ceph-mon[126460]: pgmap v335: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:21.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:21 smithi156 ceph-mon[128316]: pgmap v335: 97 pgs: 97 active+clean; 8.3 GiB data, 3.6 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 26 MiB/s wr, 57 op/s 2023-07-21T19:18:22.934 INFO:teuthology.orchestra.run.smithi005.stderr:40700000 of 50000000 tuples (81%) done (elapsed 620.50 s, remaining 141.79 s) 2023-07-21T19:18:23.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:23 smithi101 ceph-mon[128136]: pgmap v336: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:18:23.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:23 smithi005 ceph-mon[126460]: pgmap v336: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:18:23.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:23 smithi156 ceph-mon[128316]: pgmap v336: 97 pgs: 97 active+clean; 8.3 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 28 MiB/s wr, 64 op/s 2023-07-21T19:18:24.221 INFO:teuthology.orchestra.run.smithi005.stderr:40800000 of 50000000 tuples (81%) done (elapsed 621.79 s, remaining 140.21 s) 2023-07-21T19:18:25.630 INFO:teuthology.orchestra.run.smithi005.stderr:40900000 of 50000000 tuples (81%) done (elapsed 623.20 s, remaining 138.66 s) 2023-07-21T19:18:25.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:25 smithi101 ceph-mon[128136]: pgmap v337: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 48 op/s 2023-07-21T19:18:25.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:25 smithi005 ceph-mon[126460]: pgmap v337: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 48 op/s 2023-07-21T19:18:25.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:25 smithi156 ceph-mon[128316]: pgmap v337: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 23 MiB/s wr, 48 op/s 2023-07-21T19:18:26.981 INFO:teuthology.orchestra.run.smithi005.stderr:41000000 of 50000000 tuples (82%) done (elapsed 624.55 s, remaining 137.10 s) 2023-07-21T19:18:27.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:27 smithi101 ceph-mon[128136]: pgmap v338: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:18:27.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:27 smithi005 ceph-mon[126460]: pgmap v338: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:18:27.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:27 smithi156 ceph-mon[128316]: pgmap v338: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 22 MiB/s wr, 47 op/s 2023-07-21T19:18:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 -- 172.21.15.5:0/4225531350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff81062b0 msgr2=0x7f0ff8067940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/4225531350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff81062b0 0x7f0ff8067940 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f0fec0098f0 tx=0x7f0fec009c00 comp rx=0 tx=0).stop 2023-07-21T19:18:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 -- 172.21.15.5:0/4225531350 shutdown_connections 2023-07-21T19:18:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/4225531350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff8068070 0x7f0ff810c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/4225531350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 0x7f0ff8105d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/4225531350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff81062b0 0x7f0ff8067940 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.114+0000 7f0ffd39d700 1 -- 172.21.15.5:0/4225531350 >> 172.21.15.5:0/4225531350 conn(0x7f0ff8074770 msgr2=0x7f0ff8074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.115+0000 7f0ffd39d700 1 -- 172.21.15.5:0/4225531350 shutdown_connections 2023-07-21T19:18:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.115+0000 7f0ffd39d700 1 -- 172.21.15.5:0/4225531350 wait complete. 2023-07-21T19:18:28.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.116+0000 7f0ffd39d700 1 Processor -- start 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.116+0000 7f0ffd39d700 1 -- start start 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 0x7f0ff819df50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 0x7f0ff8197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff81980f0 con 0x7f0ff81059a0 2023-07-21T19:18:28.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8198260 con 0x7f0ff81062b0 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffd39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8198420 con 0x7f0ff8068070 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ff77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 0x7f0ff819df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56658/0 (socket says 172.21.15.5:56658) 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1826016461 learned_addr learned my addr 172.21.15.5:0/1826016461 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:28.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.117+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 0x7f0ff8197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1826016461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 msgr2=0x7f0ff8197a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 0x7f0ff8197a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 msgr2=0x7f0ff819df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 0x7f0ff819df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1826016461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009c40 con 0x7f0ff8068070 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 0x7f0ff8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800e9c0 tx=0x7f0fe800ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff57fa700 1 -- 172.21.15.5:0/1826016461 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8004a30 con 0x7f0ff8068070 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff57fa700 1 -- 172.21.15.5:0/1826016461 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe800cc50 con 0x7f0ff8068070 2023-07-21T19:18:28.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ff57fa700 1 -- 172.21.15.5:0/1826016461 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8005410 con 0x7f0ff8068070 2023-07-21T19:18:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec009520 con 0x7f0ff8068070 2023-07-21T19:18:28.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.118+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff81a4820 con 0x7f0ff8068070 2023-07-21T19:18:28.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.123+0000 7f0ff57fa700 1 -- 172.21.15.5:0/1826016461 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0fe8005570 con 0x7f0ff8068070 2023-07-21T19:18:28.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.124+0000 7f1011e36b80 1 Processor -- start 2023-07-21T19:18:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.124+0000 7f1011e36b80 1 -- start start 2023-07-21T19:18:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x5594307561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943065bc50 con 0x55943075c8c0 2023-07-21T19:18:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594306ae9e0 con 0x55943075c470 2023-07-21T19:18:28.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f1011e36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594306a7e70 con 0x5594305b26b0 2023-07-21T19:18:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f0ff77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f0ffcb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f0ffcb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56674/0 (socket says 172.21.15.5:56674) 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f0ffcb9c700 1 -- 172.21.15.5:0/1097979007 learned_addr learned my addr 172.21.15.5:0/1097979007 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.125+0000 7f0ff77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58866/0 (socket says 172.21.15.5:58866) 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x5594307561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ff57fa700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 0x7f0fe00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ff57fa700 1 -- 172.21.15.5:0/1826016461 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0fe8014070 con 0x7f0ff8068070 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 -- 172.21.15.5:0/1097979007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 msgr2=0x5594307561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x5594307561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 -- 172.21.15.5:0/1097979007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 msgr2=0x559430756700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 -- 172.21.15.5:0/1097979007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594304d5d80 con 0x5594305b26b0 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 0x7f0fe00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.126+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:28.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.127+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e6b0 tx=0x7f0ff000e9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=239888fbd830f302 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.127+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 0x7f0fe00a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0022000 tx=0x7f0ff0023000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.127+0000 7f0fde7fc700 1 -- 172.21.15.5:0/1097979007 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff00076c0 con 0x5594305b26b0 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.127+0000 7f0fde7fc700 1 -- 172.21.15.5:0/1097979007 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff000f040 con 0x5594305b26b0 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.127+0000 7f0fde7fc700 1 -- 172.21.15.5:0/1097979007 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff0023d10 con 0x5594305b26b0 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 -- 172.21.15.5:0/1097979007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 msgr2=0x55943075bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ff000e6b0 tx=0x7f0ff000e9c0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 -- 172.21.15.5:0/1097979007 shutdown_connections 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x559430756700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5594305b26b0 0x55943075bf30 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1097979007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x5594307561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 -- 172.21.15.5:0/1097979007 >> 172.21.15.5:0/1097979007 conn(0x55943074bd40 msgr2=0x559430751340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 -- 172.21.15.5:0/1097979007 shutdown_connections 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.128+0000 7f1011e36b80 1 -- 172.21.15.5:0/1097979007 wait complete. 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.129+0000 7f1011e36b80 1 -- start start 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.129+0000 7f1011e36b80 1 Processor -- start 2023-07-21T19:18:28.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 0x559430762790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 0x55943075fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x5594307600b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559430647a60 con 0x55943075c8c0 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559430647760 con 0x5594305b26b0 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943052b8c0 con 0x55943075c470 2023-07-21T19:18:28.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f1011e36b80 10 client.0 _ll_register_callbacks cb 0x559430645700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 0x559430762790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 0x559430762790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38124/0 (socket says 172.21.15.5:38124) 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.130+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1651722306 learned_addr learned my addr 172.21.15.5:0/1651722306 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 0x55943075fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x5594307600b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 -- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 msgr2=0x559430762790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 0x559430762790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 -- 172.21.15.5:0/1651722306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 msgr2=0x5594307600b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x5594307600b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.131+0000 7f0ff77fe700 1 -- 172.21.15.5:0/1651722306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff810d020 con 0x55943075c470 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 0x55943075fa80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009230 tx=0x7f0fec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dff2295ea72f08aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f0fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fec00b970 con 0x55943075c470 2023-07-21T19:18:28.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559430601490 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0fec00f8b0 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fec00bad0 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f1011e36b80 10 client.27289 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.132+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0ff00092b0 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559430601490 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fec015020 con 0x55943075c470 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 10 client.27289 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 10 client.27289 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 20 client.27289 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55943076fe00 0x559430772210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 4 client.27289 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.133+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5594306a0e50 con 0x55943076fe00 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.134+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55943076fe00 0x559430772210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.134+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55943076fe00 0x559430772210 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4fc4c8e8c1e673d server_cookie=467212153c3b47ed in_seq=0 out_seq=0 2023-07-21T19:18:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.134+0000 7f0fd77fe700 10 client.27289 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f0fd77fe700 1 -- 172.21.15.5:0/1651722306 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5594306a0e50 con 0x55943076fe00 2023-07-21T19:18:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f0fd77fe700 10 client.27289 handle_command_reply: tid=0 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 client.27289 shutdown 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 msgr2=0x55943075fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 0x55943075fa80 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f0fec009230 tx=0x7f0fec00f690 comp rx=0 tx=0).stop 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 shutdown_connections 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55943076fe00 0x559430772210 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55943075c8c0 0x5594307600b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5594305b26b0 0x559430762790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f0fd77fe700 0 client.27289 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1651722306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c470 0x55943075fa80 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.135+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 >> 172.21.15.5:0/1651722306 conn(0x55943074bd40 msgr2=0x559430750640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.136+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 shutdown_connections 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.136+0000 7f1011e36b80 1 -- 172.21.15.5:0/1651722306 wait complete. 2023-07-21T19:18:28.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.136+0000 7f1011e36b80 20 client.27289 trim_cache size 0 max 16384 2023-07-21T19:18:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.155+0000 7f1011e36b80 1 Processor -- start 2023-07-21T19:18:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.155+0000 7f1011e36b80 1 -- start start 2023-07-21T19:18:28.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x55943075b290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307561a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559430756bd0 con 0x5594305b26b0 2023-07-21T19:18:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559430752170 con 0x55943075c470 2023-07-21T19:18:28.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f1011e36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55943075bb30 con 0x55943075c8c0 2023-07-21T19:18:28.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58890/0 (socket says 172.21.15.5:58890) 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1959827968 learned_addr learned my addr 172.21.15.5:0/1959827968 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x55943075b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.156+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1959827968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 msgr2=0x5594307561a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307561a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1959827968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 msgr2=0x55943075b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x55943075b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 -- 172.21.15.5:0/1959827968 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5594304bec00 con 0x5594305b26b0 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x55943075b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8099b00 tx=0x7f0fe8009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96e82b7ced5fef78 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0fdeffd700 1 -- 172.21.15.5:0/1959827968 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8099e00 con 0x5594305b26b0 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0fdeffd700 1 -- 172.21.15.5:0/1959827968 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0fe809de30 con 0x5594305b26b0 2023-07-21T19:18:28.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.157+0000 7f0fdeffd700 1 -- 172.21.15.5:0/1959827968 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe80966a0 con 0x5594305b26b0 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 -- 172.21.15.5:0/1959827968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 msgr2=0x559430766e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f0fe8099b00 tx=0x7f0fe8009550 comp rx=0 tx=0).stop 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 -- 172.21.15.5:0/1959827968 shutdown_connections 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307561a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430766e90 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 --2- 172.21.15.5:0/1959827968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x55943075b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 -- 172.21.15.5:0/1959827968 >> 172.21.15.5:0/1959827968 conn(0x55943074bd40 msgr2=0x55943075ab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 -- 172.21.15.5:0/1959827968 shutdown_connections 2023-07-21T19:18:28.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.158+0000 7f1011e36b80 1 -- 172.21.15.5:0/1959827968 wait complete. 2023-07-21T19:18:28.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.159+0000 7f1011e36b80 1 -- start start 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.159+0000 7f1011e36b80 1 Processor -- start 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.159+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.159+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x559430757dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.159+0000 7f1011e36b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307584b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f1011e36b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559430758bc0 con 0x5594305b26b0 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f1011e36b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594307c3cc0 con 0x55943075c470 2023-07-21T19:18:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f1011e36b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5594307c3e00 con 0x55943075c8c0 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f1011e36b80 10 client.0 _ll_register_callbacks cb 0x7f0fec00fed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58892/0 (socket says 172.21.15.5:58892) 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 -- 172.21.15.5:0/3498256793 learned_addr learned my addr 172.21.15.5:0/3498256793 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x559430757dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ffcb9c700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307584b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 -- 172.21.15.5:0/3498256793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 msgr2=0x5594307584b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307584b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 -- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 msgr2=0x559430757dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x559430757dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 -- 172.21.15.5:0/3498256793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff00092b0 con 0x5594305b26b0 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0ff7fff700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80caa80 tx=0x7f0fe80a22c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e7b47b54556b703 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.160+0000 7f0fd77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe8096510 con 0x5594305b26b0 2023-07-21T19:18:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0fe80d1040 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ff000f040 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0fe8096670 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 10 client.17688 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0ff81a4820 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff000f040 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0fe80ca020 con 0x5594305b26b0 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 10 client.17688 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 20 client.17688 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5594307c5120 0x5594307c7530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 4 client.17688 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.161+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559430662ff0 con 0x5594307c5120 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.162+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5594307c5120 0x5594307c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.163+0000 7f0ff77fe700 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5594307c5120 0x5594307c7530 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc065d60bba2c129 server_cookie=1e846d4616330ad6 in_seq=0 out_seq=0 2023-07-21T19:18:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.163+0000 7f0fd77fe700 10 client.17688 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.174+0000 7f0fd77fe700 1 -- 172.21.15.5:0/3498256793 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559430662ff0 con 0x5594307c5120 2023-07-21T19:18:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.174+0000 7f0fd77fe700 10 client.17688 handle_command_reply: tid=0 2023-07-21T19:18:28.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.174+0000 7f1011e36b80 1 client.17688 shutdown 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 msgr2=0x559430767090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f0fe80caa80 tx=0x7f0fe80a22c0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 shutdown_connections 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55943075c8c0 0x5594307584b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5594307c5120 0x5594307c7530 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5594305b26b0 0x559430767090 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 --2- 172.21.15.5:0/3498256793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55943075c470 0x559430757dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 >> 172.21.15.5:0/3498256793 conn(0x55943074bd40 msgr2=0x55943075b410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f0fd77fe700 0 client.17688 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 shutdown_connections 2023-07-21T19:18:28.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 1 -- 172.21.15.5:0/3498256793 wait complete. 2023-07-21T19:18:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.175+0000 7f1011e36b80 20 client.17688 trim_cache size 0 max 16384 2023-07-21T19:18:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:18:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:18:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:18:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:18:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 msgr2=0x7f0fe00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 0x7f0fe00a37f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0ff0022000 tx=0x7f0ff0023000 comp rx=0 tx=0).stop 2023-07-21T19:18:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 msgr2=0x7f0ff819da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f0fe800e9c0 tx=0x7f0fe800ecd0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 shutdown_connections 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0ff81062b0 0x7f0ff8197a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0ff8068070 0x7f0ff819da10 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fe00a1340 0x7f0fe00a37f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 --2- 172.21.15.5:0/1826016461 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0ff81059a0 0x7f0ff819df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 >> 172.21.15.5:0/1826016461 conn(0x7f0ff8074770 msgr2=0x7f0ff80fcf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 shutdown_connections 2023-07-21T19:18:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:28.176+0000 7f0ffd39d700 1 -- 172.21.15.5:0/1826016461 wait complete. 2023-07-21T19:18:28.331 INFO:teuthology.orchestra.run.smithi005.stderr:41100000 of 50000000 tuples (82%) done (elapsed 625.90 s, remaining 135.54 s) 2023-07-21T19:18:29.663 INFO:teuthology.orchestra.run.smithi005.stderr:41200000 of 50000000 tuples (82%) done (elapsed 627.23 s, remaining 133.97 s) 2023-07-21T19:18:29.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:29 smithi101 ceph-mon[128136]: pgmap v339: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 64 op/s 2023-07-21T19:18:29.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:29 smithi005 ceph-mon[126460]: pgmap v339: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 64 op/s 2023-07-21T19:18:29.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:29 smithi156 ceph-mon[128316]: pgmap v339: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 33 MiB/s wr, 64 op/s 2023-07-21T19:18:31.086 INFO:teuthology.orchestra.run.smithi005.stderr:41300000 of 50000000 tuples (82%) done (elapsed 628.66 s, remaining 132.43 s) 2023-07-21T19:18:31.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:31 smithi101 ceph-mon[128136]: pgmap v340: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 49 op/s 2023-07-21T19:18:31.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:18:31.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:31 smithi005 ceph-mon[126460]: pgmap v340: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 49 op/s 2023-07-21T19:18:31.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:31 smithi156 ceph-mon[128316]: pgmap v340: 97 pgs: 97 active+clean; 8.4 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 49 op/s 2023-07-21T19:18:32.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 -- 172.21.15.5:0/560288928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 msgr2=0x7efd00101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 --2- 172.21.15.5:0/560288928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd00101070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7efcf00098f0 tx=0x7efcf0009c00 comp rx=0 tx=0).stop 2023-07-21T19:18:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 -- 172.21.15.5:0/560288928 shutdown_connections 2023-07-21T19:18:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 --2- 172.21.15.5:0/560288928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd0010a070 0x7efd0010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 --2- 172.21.15.5:0/560288928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd00101070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 --2- 172.21.15.5:0/560288928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd001015b0 0x7efd00109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 -- 172.21.15.5:0/560288928 >> 172.21.15.5:0/560288928 conn(0x7efd000fa4b0 msgr2=0x7efd000fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 -- 172.21.15.5:0/560288928 shutdown_connections 2023-07-21T19:18:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.082+0000 7efd067ff700 1 -- 172.21.15.5:0/560288928 wait complete. 2023-07-21T19:18:32.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 Processor -- start 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 -- start start 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd001015b0 0x7efd000fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd0010a070 0x7efd001a3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd000ff660 con 0x7efd001015b0 2023-07-21T19:18:32.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001a42c0 con 0x7efd0010a070 2023-07-21T19:18:32.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.084+0000 7efd067ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd001a4400 con 0x7efd00100ca0 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd001015b0 0x7efd000fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56720/0 (socket says 172.21.15.5:56720) 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 -- 172.21.15.5:0/971820386 learned_addr learned my addr 172.21.15.5:0/971820386 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efd04d9c700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd0010a070 0x7efd001a3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 -- 172.21.15.5:0/971820386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd0010a070 msgr2=0x7efd001a3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd0010a070 0x7efd001a3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 -- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd001015b0 msgr2=0x7efd000fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd001015b0 0x7efd000fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 -- 172.21.15.5:0/971820386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcf8009d00 con 0x7efd00100ca0 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.085+0000 7efcfffff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009810 tx=0x7efcf000fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.086+0000 7efcfd7fa700 1 -- 172.21.15.5:0/971820386 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf001c070 con 0x7efd00100ca0 2023-07-21T19:18:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.086+0000 7efcfd7fa700 1 -- 172.21.15.5:0/971820386 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf000baf0 con 0x7efd00100ca0 2023-07-21T19:18:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.086+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcf0009520 con 0x7efd00100ca0 2023-07-21T19:18:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.086+0000 7efcfd7fa700 1 -- 172.21.15.5:0/971820386 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf0004930 con 0x7efd00100ca0 2023-07-21T19:18:32.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.086+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd001a4940 con 0x7efd00100ca0 2023-07-21T19:18:32.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.090+0000 7efcfd7fa700 1 -- 172.21.15.5:0/971820386 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7efcf0020070 con 0x7efd00100ca0 2023-07-21T19:18:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.092+0000 7efd1a18fb80 1 Processor -- start 2023-07-21T19:18:32.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.092+0000 7efd1a18fb80 1 -- start start 2023-07-21T19:18:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.092+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c8d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c9aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2b93cb0 con 0x5638b2c98ad0 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2be6a40 con 0x5638b2c8dbe0 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2bdfed0 con 0x5638b2c8d2d0 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfd7fa700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 0x7efce80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38164/0 (socket says 172.21.15.5:38164) 2023-07-21T19:18:32.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 -- 172.21.15.5:0/1085411019 learned_addr learned my addr 172.21.15.5:0/1085411019 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd04d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c8d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcff7fe700 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c9aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efd04d9c700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 0x7efce80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 -- 172.21.15.5:0/1085411019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 msgr2=0x5638b2c8d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c8d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 -- 172.21.15.5:0/1085411019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 msgr2=0x5638b2c9aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.093+0000 7efcfffff700 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c9aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efcfffff700 1 -- 172.21.15.5:0/1085411019 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b2a0dde0 con 0x5638b2c8dbe0 2023-07-21T19:18:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efd04d9c700 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c8d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efcfd7fa700 1 -- 172.21.15.5:0/971820386 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7efcf00cd9a0 con 0x7efd00100ca0 2023-07-21T19:18:32.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efcfffff700 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efcf00a2930 tx=0x7efcf006cc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82f24405d2d8cbf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efce67fc700 1 -- 172.21.15.5:0/1085411019 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf009a400 con 0x5638b2c8dbe0 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efce67fc700 1 -- 172.21.15.5:0/1085411019 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf0082070 con 0x5638b2c8dbe0 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.094+0000 7efce67fc700 1 -- 172.21.15.5:0/1085411019 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf009ab30 con 0x5638b2c8dbe0 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 -- 172.21.15.5:0/1085411019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 msgr2=0x5638b2c98590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd04d9c700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 0x7efce80a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009610 tx=0x7efcf800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7efcf00a2930 tx=0x7efcf006cc30 comp rx=0 tx=0).stop 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 -- 172.21.15.5:0/1085411019 shutdown_connections 2023-07-21T19:18:32.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c9aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c8d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/1085411019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98590 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.095+0000 7efd1a18fb80 1 -- 172.21.15.5:0/1085411019 >> 172.21.15.5:0/1085411019 conn(0x5638b2c83da0 msgr2=0x5638b2c896a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.096+0000 7efd1a18fb80 1 -- 172.21.15.5:0/1085411019 shutdown_connections 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.096+0000 7efd1a18fb80 1 -- 172.21.15.5:0/1085411019 wait complete. 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.096+0000 7efd1a18fb80 1 -- start start 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 Processor -- start 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8d2d0 0x5638b2c97bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2b7fac0 con 0x5638b2c8dbe0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.097+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2b7f7c0 con 0x5638b2c8d2d0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2a63920 con 0x5638b2c98ad0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd1a18fb80 10 client.0 _ll_register_callbacks cb 0x5638b2b7d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efcff7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56730/0 (socket says 172.21.15.5:56730) 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 -- 172.21.15.5:0/985090193 learned_addr learned my addr 172.21.15.5:0/985090193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efcfffff700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8d2d0 0x5638b2c97bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 -- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8d2d0 msgr2=0x5638b2c97bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8d2d0 0x5638b2c97bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 -- 172.21.15.5:0/985090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8dbe0 msgr2=0x5638b2c98110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.098+0000 7efd04d9c700 1 -- 172.21.15.5:0/985090193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd001a4940 con 0x5638b2c98ad0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efd04d9c700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcf8007b70 tx=0x7efcf8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad3dbbb2aafda77 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efcdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf800e040 con 0x5638b2c98ad0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7efcf8003c10 con 0x5638b2c98ad0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638b2b394f0 con 0x5638b2c98ad0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf8007e80 con 0x5638b2c98ad0 2023-07-21T19:18:32.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.099+0000 7efd1a18fb80 10 client.27304 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5638b29f6c60 con 0x5638b2c98ad0 2023-07-21T19:18:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf8031080 con 0x5638b2c98ad0 2023-07-21T19:18:32.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efcf802b030 con 0x5638b2c98ad0 2023-07-21T19:18:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 10 client.27304 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 10 client.27304 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 20 client.27304 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:32.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2cb0890 0x5638b2cb2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 4 client.27304 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:32.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.100+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638b2bd8eb0 con 0x5638b2cb0890 2023-07-21T19:18:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.101+0000 7efcfffff700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2cb0890 0x5638b2cb2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.101+0000 7efcfffff700 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2cb0890 0x5638b2cb2ca0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c16d3122c9d9471 server_cookie=2c7c71ad0a2513f0 in_seq=0 out_seq=0 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.101+0000 7efcdf7fe700 10 client.27304 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.102+0000 7efcdf7fe700 1 -- 172.21.15.5:0/985090193 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638b2bd8eb0 con 0x5638b2cb0890 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.102+0000 7efcdf7fe700 10 client.27304 handle_command_reply: tid=0 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.102+0000 7efd1a18fb80 1 client.27304 shutdown 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 msgr2=0x5638b2ca4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7efcf8007b70 tx=0x7efcf8004ad0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 shutdown_connections 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c98ad0 0x5638b2ca4e90 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8d2d0 0x5638b2c97bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2cb0890 0x5638b2cb2ca0 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/985090193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8dbe0 0x5638b2c98110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 >> 172.21.15.5:0/985090193 conn(0x5638b2c83da0 msgr2=0x5638b2c89290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efcdf7fe700 0 client.27304 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:32.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 shutdown_connections 2023-07-21T19:18:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.103+0000 7efd1a18fb80 1 -- 172.21.15.5:0/985090193 wait complete. 2023-07-21T19:18:32.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.104+0000 7efd1a18fb80 20 client.27304 trim_cache size 0 max 16384 2023-07-21T19:18:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 Processor -- start 2023-07-21T19:18:32.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 -- start start 2023-07-21T19:18:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c98060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c97180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2c8ad20 con 0x5638b2c98ad0 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2c98910 con 0x5638b2c8dbe0 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.132+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2c8e570 con 0x5638b2c8d2d0 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efd04d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c97180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38194/0 (socket says 172.21.15.5:38194) 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efd04d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c97180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58950/0 (socket says 172.21.15.5:58950) 2023-07-21T19:18:32.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 -- 172.21.15.5:0/4069279349 learned_addr learned my addr 172.21.15.5:0/4069279349 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcfffff700 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c98060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 -- 172.21.15.5:0/4069279349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 msgr2=0x5638b2c98060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c98060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 -- 172.21.15.5:0/4069279349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 msgr2=0x5638b2c97180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c97180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.133+0000 7efcff7fe700 1 -- 172.21.15.5:0/4069279349 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638b2a34250 con 0x5638b2c8dbe0 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.134+0000 7efcff7fe700 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009d90 tx=0x7efcf4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e681eb8f1d3ea18 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.134+0000 7efce6ffd700 1 -- 172.21.15.5:0/4069279349 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4017070 con 0x5638b2c8dbe0 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.134+0000 7efce6ffd700 1 -- 172.21.15.5:0/4069279349 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf4018040 con 0x5638b2c8dbe0 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.134+0000 7efce6ffd700 1 -- 172.21.15.5:0/4069279349 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf4003a80 con 0x5638b2c8dbe0 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 -- 172.21.15.5:0/4069279349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 msgr2=0x5638b2ca67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7efcf4009d90 tx=0x7efcf4009260 comp rx=0 tx=0).stop 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 -- 172.21.15.5:0/4069279349 shutdown_connections 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c98ad0 0x5638b2c97180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8d2d0 0x5638b2c98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/4069279349 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca67a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 -- 172.21.15.5:0/4069279349 >> 172.21.15.5:0/4069279349 conn(0x5638b2c83da0 msgr2=0x5638b2c8fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 -- 172.21.15.5:0/4069279349 shutdown_connections 2023-07-21T19:18:32.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.135+0000 7efd1a18fb80 1 -- 172.21.15.5:0/4069279349 wait complete. 2023-07-21T19:18:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.136+0000 7efd1a18fb80 1 -- start start 2023-07-21T19:18:32.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.136+0000 7efd1a18fb80 1 Processor -- start 2023-07-21T19:18:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca99d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c98ad0 0x5638b2ca9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2d04830 con 0x5638b2c8d2d0 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2d049a0 con 0x5638b2c98ad0 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638b2d04b60 con 0x5638b2c8dbe0 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd1a18fb80 10 client.0 _ll_register_callbacks cb 0x7efcf8016e50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efcfffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efcfffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58964/0 (socket says 172.21.15.5:58964) 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efcfffff700 1 -- 172.21.15.5:0/52109887 learned_addr learned my addr 172.21.15.5:0/52109887 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efcff7fe700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca99d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.137+0000 7efd04d9c700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c98ad0 0x5638b2ca9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 -- 172.21.15.5:0/52109887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8dbe0 msgr2=0x5638b2ca99d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca99d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 -- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c98ad0 msgr2=0x5638b2ca9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c98ad0 0x5638b2ca9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 -- 172.21.15.5:0/52109887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcf4009a40 con 0x5638b2c8d2d0 2023-07-21T19:18:32.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcfffff700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efcf000bca0 tx=0x7efcf009a870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18b366c2f1a563c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.138+0000 7efcdf7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcf009ead0 con 0x5638b2c8d2d0 2023-07-21T19:18:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7efcf00a2e60 con 0x5638b2c8d2d0 2023-07-21T19:18:32.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efcf0082070 con 0x5638b2c8d2d0 2023-07-21T19:18:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7efcf009aaa0 con 0x5638b2c8d2d0 2023-07-21T19:18:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efd1a18fb80 10 client.17706 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:32.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5638b2a0c2b0 con 0x5638b2c8d2d0 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.139+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7efcf0082070 con 0x5638b2c8d2d0 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7efcf00cb310 con 0x5638b2c8d2d0 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 10 client.17706 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 10 client.17706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 20 client.17706 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2d05f30 0x5638b2d08310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 4 client.17706 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:18:32.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5638b2b9b050 con 0x5638b2d05f30 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.140+0000 7efcff7fe700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2d05f30 0x5638b2d08310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.141+0000 7efcdf7fe700 10 client.17706 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.141+0000 7efcff7fe700 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2d05f30 0x5638b2d08310 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b297776721e78db server_cookie=36dc6fbc28e67d90 in_seq=0 out_seq=0 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.141+0000 7efcdf7fe700 1 -- 172.21.15.5:0/52109887 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5638b2b9b050 con 0x5638b2d05f30 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.141+0000 7efcdf7fe700 10 client.17706 handle_command_reply: tid=0 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.141+0000 7efd1a18fb80 1 client.17706 shutdown 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 msgr2=0x5638b2ca9490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7efcf000bca0 tx=0x7efcf009a870 comp rx=0 tx=0).stop 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 shutdown_connections 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638b2c98ad0 0x5638b2ca9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638b2d05f30 0x5638b2d08310 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638b2c8d2d0 0x5638b2ca9490 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 --2- 172.21.15.5:0/52109887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638b2c8dbe0 0x5638b2ca99d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 >> 172.21.15.5:0/52109887 conn(0x5638b2c83da0 msgr2=0x5638b2c888b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efcdf7fe700 0 client.17706 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:32.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 shutdown_connections 2023-07-21T19:18:32.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 1 -- 172.21.15.5:0/52109887 wait complete. 2023-07-21T19:18:32.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.142+0000 7efd1a18fb80 20 client.17706 trim_cache size 0 max 16384 2023-07-21T19:18:32.144 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:18:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:18:32.144 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:18:32.145 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:18:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 msgr2=0x7efce80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 0x7efce80a37f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7efcf8009610 tx=0x7efcf800b040 comp rx=0 tx=0).stop 2023-07-21T19:18:32.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 msgr2=0x7efd000fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7efcf0009810 tx=0x7efcf000fa00 comp rx=0 tx=0).stop 2023-07-21T19:18:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 shutdown_connections 2023-07-21T19:18:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7efd0010a070 0x7efd001a3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7efd00100ca0 0x7efd000fe8f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7efce80a1340 0x7efce80a37f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 --2- 172.21.15.5:0/971820386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7efd001015b0 0x7efd000fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.143+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 >> 172.21.15.5:0/971820386 conn(0x7efd000fa4b0 msgr2=0x7efd000fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.144+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 shutdown_connections 2023-07-21T19:18:32.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.144+0000 7efd067ff700 1 -- 172.21.15.5:0/971820386 wait complete. 2023-07-21T19:18:32.166 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:18:32.167 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:18:32.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 -- 172.21.15.5:0/610379302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 msgr2=0x7f7414109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 --2- 172.21.15.5:0/610379302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f7414109b00 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f7400009990 tx=0x7f7400009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 -- 172.21.15.5:0/610379302 shutdown_connections 2023-07-21T19:18:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 --2- 172.21.15.5:0/610379302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 0x7f741410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 --2- 172.21.15.5:0/610379302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f74141010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 --2- 172.21.15.5:0/610379302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f7414109b00 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 -- 172.21.15.5:0/610379302 >> 172.21.15.5:0/610379302 conn(0x7f74140fa490 msgr2=0x7f74140fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 -- 172.21.15.5:0/610379302 shutdown_connections 2023-07-21T19:18:32.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.407+0000 7f741afff700 1 -- 172.21.15.5:0/610379302 wait complete. 2023-07-21T19:18:32.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.408+0000 7f741afff700 1 Processor -- start 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.408+0000 7f741afff700 1 -- start start 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f741419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 0x7f7414197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141981b0 con 0x7f7414101610 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7414198320 con 0x7f7414100d00 2023-07-21T19:18:32.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74141984e0 con 0x7f741410a040 2023-07-21T19:18:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f7413fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f7418d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f741419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f7413fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58976/0 (socket says 172.21.15.5:58976) 2023-07-21T19:18:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f7413fff700 1 -- 172.21.15.5:0/1273106195 learned_addr learned my addr 172.21.15.5:0/1273106195 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:32.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.409+0000 7f741959c700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 0x7f7414197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 -- 172.21.15.5:0/1273106195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 msgr2=0x7f7414197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 0x7f7414197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 -- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 msgr2=0x7f741419e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 -- 172.21.15.5:0/1273106195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7408009520 con 0x7f7414100d00 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7413fff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:32.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7418d9b700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f741419dad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f740800e9f0 tx=0x7f740800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f740800cb70 con 0x7f7414100d00 2023-07-21T19:18:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f740800ee90 con 0x7f7414100d00 2023-07-21T19:18:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7408003bc0 con 0x7f7414100d00 2023-07-21T19:18:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74000095f0 con 0x7f7414100d00 2023-07-21T19:18:32.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.410+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74141a48e0 con 0x7f7414100d00 2023-07-21T19:18:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.411+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7414067300 con 0x7f7414100d00 2023-07-21T19:18:32.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.414+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7408003d40 con 0x7f7414100d00 2023-07-21T19:18:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.416+0000 7f7411ffb700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 0x7f74040a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.416+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7408014070 con 0x7f7414100d00 2023-07-21T19:18:32.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.416+0000 7f7413fff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 0x7f74040a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:32.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.417+0000 7f7413fff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 0x7f74040a37b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f741410b650 tx=0x7f740000b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:32.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.418+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7408096560 con 0x7f7414100d00 2023-07-21T19:18:32.541 INFO:teuthology.orchestra.run.smithi005.stderr:41400000 of 50000000 tuples (82%) done (elapsed 630.11 s, remaining 130.89 s) 2023-07-21T19:18:32.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.736+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7414198fe0 con 0x7f7414100d00 2023-07-21T19:18:32.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.737+0000 7f7411ffb700 1 -- 172.21.15.5:0/1273106195 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f7408099b80 con 0x7f7414100d00 2023-07-21T19:18:32.734 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:18:32.734 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:17:56.540961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:18:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.739+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 msgr2=0x7f74040a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.739+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 0x7f74040a37b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f741410b650 tx=0x7f740000b260 comp rx=0 tx=0).stop 2023-07-21T19:18:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.739+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 msgr2=0x7f741419dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.739+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f741419dad0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f740800e9f0 tx=0x7f740800ed00 comp rx=0 tx=0).stop 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 shutdown_connections 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f741410a040 0x7f7414197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7414100d00 0x7f741419dad0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74040a1300 0x7f74040a37b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 --2- 172.21.15.5:0/1273106195 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7414101610 0x7f741419e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 >> 172.21.15.5:0/1273106195 conn(0x7f74140fa490 msgr2=0x7f741410c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 shutdown_connections 2023-07-21T19:18:32.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.740+0000 7f741afff700 1 -- 172.21.15.5:0/1273106195 wait complete. 2023-07-21T19:18:32.738 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2023-07-21T19:18:32.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:18:32.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 -- 172.21.15.5:0/4237764085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0100480 msgr2=0x7fb5d010c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/4237764085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0100480 0x7fb5d010c910 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b0a0 tx=0x7fb5c800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 -- 172.21.15.5:0/4237764085 shutdown_connections 2023-07-21T19:18:32.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/4237764085 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0100480 0x7fb5d010c910 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/4237764085 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0105ae0 0x7fb5d0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/4237764085 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 0x7fb5d00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 -- 172.21.15.5:0/4237764085 >> 172.21.15.5:0/4237764085 conn(0x7fb5d00fb5f0 msgr2=0x7fb5d00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 -- 172.21.15.5:0/4237764085 shutdown_connections 2023-07-21T19:18:32.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:32.999+0000 7fb5d7dff700 1 -- 172.21.15.5:0/4237764085 wait complete. 2023-07-21T19:18:32.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 Processor -- start 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 -- start start 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 0x7fb5d0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0100480 0x7fb5d01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d019cd90 con 0x7fb5d0105ae0 2023-07-21T19:18:33.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d019cf00 con 0x7fb5d0100480 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.001+0000 7fb5d7dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d019d0c0 con 0x7fb5d00ffad0 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58994/0 (socket says 172.21.15.5:58994) 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 0x7fb5d0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 -- 172.21.15.5:0/3457450898 learned_addr learned my addr 172.21.15.5:0/3457450898 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d539a700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0100480 0x7fb5d01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 -- 172.21.15.5:0/3457450898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 msgr2=0x7fb5d0197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 0x7fb5d0197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 -- 172.21.15.5:0/3457450898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0100480 msgr2=0x7fb5d01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0100480 0x7fb5d01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 -- 172.21.15.5:0/3457450898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5c8009d00 con 0x7fb5d0105ae0 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.002+0000 7fb5d639c700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b070 tx=0x7fb5c800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.003+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/3457450898 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c800e040 con 0x7fb5d0105ae0 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.003+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5d019d340 con 0x7fb5d0105ae0 2023-07-21T19:18:33.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.003+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/3457450898 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5c8009480 con 0x7fb5d0105ae0 2023-07-21T19:18:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.003+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/3457450898 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5c8005040 con 0x7fb5d0105ae0 2023-07-21T19:18:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.003+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5d01a4be0 con 0x7fb5d0105ae0 2023-07-21T19:18:33.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.007+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/3457450898 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb5c801d480 con 0x7fb5d0105ae0 2023-07-21T19:18:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.008+0000 7fb5eb79cb80 1 Processor -- start 2023-07-21T19:18:33.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.008+0000 7fb5eb79cb80 1 -- start start 2023-07-21T19:18:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f9686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f975eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f86ecb0 con 0x559f8f968be0 2023-07-21T19:18:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f8c1a40 con 0x559f8f973ad0 2023-07-21T19:18:33.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f8baed0 con 0x559f8f9682d0 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5c6ffd700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 0x7fb5bc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5d539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f9686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59000/0 (socket says 172.21.15.5:59000) 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2915394939 learned_addr learned my addr 172.21.15.5:0/2915394939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f975eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.009+0000 7fb5c6ffd700 1 -- 172.21.15.5:0/3457450898 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5c80cdd50 con 0x7fb5d0105ae0 2023-07-21T19:18:33.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d639c700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 0x7fb5bc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2915394939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 msgr2=0x559f8f9686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f9686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2915394939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 msgr2=0x559f8f975eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f975eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2915394939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f8f6e8de0 con 0x559f8f968be0 2023-07-21T19:18:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009d00 tx=0x7fb5cc011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84283b0e49db8f26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5d639c700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 0x7fb5bc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009670 tx=0x7fb5c8009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5bbfff700 1 -- 172.21.15.5:0/2915394939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc004230 con 0x559f8f968be0 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5bbfff700 1 -- 172.21.15.5:0/2915394939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5cc004390 con 0x559f8f968be0 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.010+0000 7fb5bbfff700 1 -- 172.21.15.5:0/2915394939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc0196a0 con 0x559f8f968be0 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2915394939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 msgr2=0x559f8f973590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc009d00 tx=0x7fb5cc011720 comp rx=0 tx=0).stop 2023-07-21T19:18:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2915394939 shutdown_connections 2023-07-21T19:18:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f975eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f9686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2915394939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f973590 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2915394939 >> 172.21.15.5:0/2915394939 conn(0x559f8f95eda0 msgr2=0x559f8f9646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2915394939 shutdown_connections 2023-07-21T19:18:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.011+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2915394939 wait complete. 2023-07-21T19:18:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.012+0000 7fb5eb79cb80 1 -- start start 2023-07-21T19:18:33.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.012+0000 7fb5eb79cb80 1 Processor -- start 2023-07-21T19:18:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f973540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f970e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f85aac0 con 0x559f8f968be0 2023-07-21T19:18:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f85a7c0 con 0x559f8f973ad0 2023-07-21T19:18:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f73e920 con 0x559f8f9682d0 2023-07-21T19:18:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5eb79cb80 10 client.0 _ll_register_callbacks cb 0x559f8f858760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59008/0 (socket says 172.21.15.5:59008) 2023-07-21T19:18:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2023033271 learned_addr learned my addr 172.21.15.5:0/2023033271 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f973540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.013+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f970e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 msgr2=0x559f8f973540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f973540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2023033271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 msgr2=0x559f8f970e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f970e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2023033271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5cc009960 con 0x559f8f968be0 2023-07-21T19:18:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f973540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0097c0 tx=0x7fb5cc0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b13ed5aa639a1bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc005ce0 con 0x559f8f968be0 2023-07-21T19:18:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f8f8144f0 con 0x559f8f968be0 2023-07-21T19:18:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.014+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5cc005250 con 0x559f8f968be0 2023-07-21T19:18:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5cc005450 con 0x559f8f968be0 2023-07-21T19:18:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 10 client.17724 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559f8f6d1c60 con 0x559f8f968be0 2023-07-21T19:18:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559f8f8144f0 con 0x559f8f968be0 2023-07-21T19:18:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5cc028020 con 0x559f8f968be0 2023-07-21T19:18:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 10 client.17724 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 10 client.17724 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:18:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 20 client.17724 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f983350 0x559f8f985760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 4 client.17724 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.015+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f8f8b3eb0 con 0x559f8f983350 2023-07-21T19:18:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.016+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f983350 0x559f8f985760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.016+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f983350 0x559f8f985760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5629f7d761f649c6 server_cookie=39997c9eb1242158 in_seq=0 out_seq=0 2023-07-21T19:18:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.016+0000 7fb5b97fa700 10 client.17724 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:18:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5b97fa700 1 -- 172.21.15.5:0/2023033271 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559f8f8b3eb0 con 0x559f8f983350 2023-07-21T19:18:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5b97fa700 10 client.17724 handle_command_reply: tid=0 2023-07-21T19:18:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 client.17724 shutdown 2023-07-21T19:18:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 msgr2=0x559f8f970750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0097c0 tx=0x7fb5cc0039e0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 shutdown_connections 2023-07-21T19:18:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f983350 0x559f8f985760 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f973ad0 0x559f8f970e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f9682d0 0x559f8f973540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5b97fa700 0 client.17724 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:18:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2023033271 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f970750 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.017+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 >> 172.21.15.5:0/2023033271 conn(0x559f8f95eda0 msgr2=0x559f8f964230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.018+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 shutdown_connections 2023-07-21T19:18:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.018+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2023033271 wait complete. 2023-07-21T19:18:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.018+0000 7fb5eb79cb80 20 client.17724 trim_cache size 0 max 16384 2023-07-21T19:18:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 Processor -- start 2023-07-21T19:18:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 -- start start 2023-07-21T19:18:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f970e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f973070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f98a5c0 con 0x559f8f968be0 2023-07-21T19:18:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f965c70 con 0x559f8f9682d0 2023-07-21T19:18:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.032+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f96fcc0 con 0x559f8f973ad0 2023-07-21T19:18:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59016/0 (socket says 172.21.15.5:59016) 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2172383800 learned_addr learned my addr 172.21.15.5:0/2172383800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f973070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f970e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2172383800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 msgr2=0x559f8f973070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f973070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2172383800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 msgr2=0x559f8f970e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f970e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/2172383800 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f8f70f250 con 0x559f8f968be0 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d539a700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f970e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.033+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000c00 tx=0x7fb5cc0336a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335d94c5bc2f0b39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.034+0000 7fb5c4bff700 1 -- 172.21.15.5:0/2172383800 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc009c30 con 0x559f8f968be0 2023-07-21T19:18:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.034+0000 7fb5c4bff700 1 -- 172.21.15.5:0/2172383800 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5cc033e40 con 0x559f8f968be0 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.034+0000 7fb5c4bff700 1 -- 172.21.15.5:0/2172383800 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc005760 con 0x559f8f968be0 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2172383800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 msgr2=0x559f8f98a080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc000c00 tx=0x7fb5cc0336a0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2172383800 shutdown_connections 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f973070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f970e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/2172383800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f98a080 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2172383800 >> 172.21.15.5:0/2172383800 conn(0x559f8f95eda0 msgr2=0x559f8f960310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2172383800 shutdown_connections 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/2172383800 wait complete. 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.035+0000 7fb5eb79cb80 1 -- start start 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 Processor -- start 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f977df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f978870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f9d6350 con 0x559f8f968be0 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f9d64c0 con 0x559f8f9682d0 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f8f9d6680 con 0x559f8f973ad0 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.036+0000 7fb5eb79cb80 10 client.0 _ll_register_callbacks cb 0x7fb5c0009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59022/0 (socket says 172.21.15.5:59022) 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/1329421184 learned_addr learned my addr 172.21.15.5:0/1329421184 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d539a700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f978870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d539a700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f977df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 msgr2=0x559f8f978870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f978870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/1329421184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 msgr2=0x559f8f977df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f977df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 -- 172.21.15.5:0/1329421184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559f8f6d1c60 con 0x559f8f968be0 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5d5b9b700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0095c0 tx=0x7fb5cc004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddb6ea2b30b766fd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.037+0000 7fb5b97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5cc005450 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5cc004390 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5cc000e40 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb5cc0055b0 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5eb79cb80 10 client.17730 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x559f8f6e72b0 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.038+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5cc049020 con 0x559f8f968be0 2023-07-21T19:18:33.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb5cc03a070 con 0x559f8f968be0 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 10 client.17730 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 10 client.17730 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 20 client.17730 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f9d78e0 0x559f8f9d9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 4 client.17730 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x559f8f876050 con 0x559f8f9d78e0 2023-07-21T19:18:33.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.039+0000 7fb5d539a700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f9d78e0 0x559f8f9d9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.041+0000 7fb5b97fa700 10 client.17730 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.041+0000 7fb5d539a700 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f9d78e0 0x559f8f9d9cf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=235093f37100c4c3 server_cookie=b15792c5936e40b0 in_seq=0 out_seq=0 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.041+0000 7fb5b97fa700 1 -- 172.21.15.5:0/1329421184 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x559f8f876050 con 0x559f8f9d78e0 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.041+0000 7fb5b97fa700 10 client.17730 handle_command_reply: tid=0 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.041+0000 7fb5eb79cb80 1 client.17730 shutdown 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 msgr2=0x559f8f978330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb5cc0095c0 tx=0x7fb5cc004320 comp rx=0 tx=0).stop 2023-07-21T19:18:33.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 shutdown_connections 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559f8f973ad0 0x559f8f978870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x559f8f9d78e0 0x559f8f9d9cf0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559f8f9682d0 0x559f8f977df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 --2- 172.21.15.5:0/1329421184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559f8f968be0 0x559f8f978330 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 >> 172.21.15.5:0/1329421184 conn(0x559f8f95eda0 msgr2=0x559f8f96ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5b97fa700 0 client.17730 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 shutdown_connections 2023-07-21T19:18:33.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 1 -- 172.21.15.5:0/1329421184 wait complete. 2023-07-21T19:18:33.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.042+0000 7fb5eb79cb80 20 client.17730 trim_cache size 0 max 16384 2023-07-21T19:18:33.053 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:18:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 msgr2=0x7fb5bc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 0x7fb5bc0a3770 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb5c8009670 tx=0x7fb5c8009040 comp rx=0 tx=0).stop 2023-07-21T19:18:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 msgr2=0x7fb5d019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7fb5c800b070 tx=0x7fb5c800bce0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 shutdown_connections 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5bc0a12c0 0x7fb5bc0a3770 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5d0105ae0 0x7fb5d019c750 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5d00ffad0 0x7fb5d0197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 --2- 172.21.15.5:0/3457450898 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5d0100480 0x7fb5d01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 >> 172.21.15.5:0/3457450898 conn(0x7fb5d00fb5f0 msgr2=0x7fb5d010ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.043+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 shutdown_connections 2023-07-21T19:18:33.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.044+0000 7fb5d7dff700 1 -- 172.21.15.5:0/3457450898 wait complete. 2023-07-21T19:18:33.058 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:18:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.301+0000 7f5e863ff700 1 -- 172.21.15.5:0/978912656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e800fea90 msgr2=0x7f5e800fee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.301+0000 7f5e863ff700 1 --2- 172.21.15.5:0/978912656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e800fea90 0x7f5e800fee60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f5e700098f0 tx=0x7f5e70009c00 comp rx=0 tx=0).stop 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 -- 172.21.15.5:0/978912656 shutdown_connections 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 --2- 172.21.15.5:0/978912656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e80103e40 0x7f5e801042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 --2- 172.21.15.5:0/978912656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e800fea90 0x7f5e800fee60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 --2- 172.21.15.5:0/978912656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80103870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 -- 172.21.15.5:0/978912656 >> 172.21.15.5:0/978912656 conn(0x7f5e800fa4b0 msgr2=0x7f5e800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 -- 172.21.15.5:0/978912656 shutdown_connections 2023-07-21T19:18:33.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.302+0000 7f5e863ff700 1 -- 172.21.15.5:0/978912656 wait complete. 2023-07-21T19:18:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.303+0000 7f5e863ff700 1 Processor -- start 2023-07-21T19:18:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.303+0000 7f5e863ff700 1 -- start start 2023-07-21T19:18:33.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80110c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e80103e40 0x7f5e8010b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8010bf90 con 0x7f5e800fea90 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8010c100 con 0x7f5e80103e40 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e863ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8010c2c0 con 0x7f5e800ff3a0 2023-07-21T19:18:33.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59038/0 (socket says 172.21.15.5:59038) 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e7ffff700 1 -- 172.21.15.5:0/2800412735 learned_addr learned my addr 172.21.15.5:0/2800412735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80110c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.304+0000 7f5e8499c700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e80103e40 0x7f5e8010b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2800412735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e80103e40 msgr2=0x7f5e8010b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e80103e40 0x7f5e8010b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 msgr2=0x7f5e80110730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2800412735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e70009520 con 0x7f5e800ff3a0 2023-07-21T19:18:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80110c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9f0 tx=0x7f5e7400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7d7fa700 1 -- 172.21.15.5:0/2800412735 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7400c9c0 con 0x7f5e800ff3a0 2023-07-21T19:18:33.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7d7fa700 1 -- 172.21.15.5:0/2800412735 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e74010430 con 0x7f5e800ff3a0 2023-07-21T19:18:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.305+0000 7f5e7d7fa700 1 -- 172.21.15.5:0/2800412735 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74018680 con 0x7f5e800ff3a0 2023-07-21T19:18:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.306+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8010c5a0 con 0x7f5e800ff3a0 2023-07-21T19:18:33.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.306+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8010f670 con 0x7f5e800ff3a0 2023-07-21T19:18:33.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.310+0000 7f5e7d7fa700 1 -- 172.21.15.5:0/2800412735 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5e740105a0 con 0x7f5e800ff3a0 2023-07-21T19:18:33.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 Processor -- start 2023-07-21T19:18:33.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 -- start start 2023-07-21T19:18:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e7d7fa700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 0x7f5e680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 0x55a9142476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a914252590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914252ad0 0x55a914254eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91414dcb0 con 0x55a9142472d0 2023-07-21T19:18:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9141a0a40 con 0x55a914247be0 2023-07-21T19:18:33.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.313+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914199ed0 con 0x55a914252ad0 2023-07-21T19:18:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e7d7fa700 1 -- 172.21.15.5:0/2800412735 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e74014070 con 0x7f5e800ff3a0 2023-07-21T19:18:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e8499c700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 0x7f5e680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e8499c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914252ad0 0x55a914254eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 0x55a9142476a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a914252590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 0x55a9142476a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59046/0 (socket says 172.21.15.5:59046) 2023-07-21T19:18:33.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.314+0000 7f5e7ffff700 1 -- 172.21.15.5:0/2906960858 learned_addr learned my addr 172.21.15.5:0/2906960858 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2906960858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914252ad0 msgr2=0x55a914254eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914252ad0 0x55a914254eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2906960858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 msgr2=0x55a9142476a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 0x55a9142476a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/2906960858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a913fc7de0 con 0x55a914247be0 2023-07-21T19:18:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e8499c700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 0x7f5e680a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e78003fd0 tx=0x7f5e780097f0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a914252590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ca5f0 tx=0x7f5e740963e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d94f49eb63c24ff8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e667fc700 1 -- 172.21.15.5:0/2906960858 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74077820 con 0x55a914247be0 2023-07-21T19:18:33.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e667fc700 1 -- 172.21.15.5:0/2906960858 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e74077640 con 0x55a914247be0 2023-07-21T19:18:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.315+0000 7f5e667fc700 1 -- 172.21.15.5:0/2906960858 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e740a23c0 con 0x55a914247be0 2023-07-21T19:18:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 -- 172.21.15.5:0/2906960858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 msgr2=0x55a914252590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a914252590 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ca5f0 tx=0x7f5e740963e0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 -- 172.21.15.5:0/2906960858 shutdown_connections 2023-07-21T19:18:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914252ad0 0x55a914254eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9142472d0 0x55a9142476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/2906960858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a914252590 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 -- 172.21.15.5:0/2906960858 >> 172.21.15.5:0/2906960858 conn(0x55a91423dda0 msgr2=0x55a9142436a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 -- 172.21.15.5:0/2906960858 shutdown_connections 2023-07-21T19:18:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.316+0000 7f5e99d10b80 1 -- 172.21.15.5:0/2906960858 wait complete. 2023-07-21T19:18:33.324 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.317+0000 7f5e99d10b80 1 -- start start 2023-07-21T19:18:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.317+0000 7f5e99d10b80 1 Processor -- start 2023-07-21T19:18:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a914252550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a91424fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914139ac0 con 0x55a914252ad0 2023-07-21T19:18:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9141397c0 con 0x55a914247be0 2023-07-21T19:18:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91401d920 con 0x55a9142472d0 2023-07-21T19:18:33.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e99d10b80 10 client.0 _ll_register_callbacks cb 0x55a914137760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38310/0 (socket says 172.21.15.5:38310) 2023-07-21T19:18:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.318+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3067042543 learned_addr learned my addr 172.21.15.5:0/3067042543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e8499c700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a91424fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3067042543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 msgr2=0x55a914252550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a914252550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3067042543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 msgr2=0x55a91424fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a91424fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.319+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3067042543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8010f670 con 0x55a914247be0 2023-07-21T19:18:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ca650 tx=0x7f5e74010bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f05b834a4ec28dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74075070 con 0x55a914247be0 2023-07-21T19:18:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e740a23c0 con 0x55a914247be0 2023-07-21T19:18:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a9140f34f0 con 0x55a914247be0 2023-07-21T19:18:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e7409dfa0 con 0x55a914247be0 2023-07-21T19:18:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e99d10b80 10 client.27416 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.320+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a913fb0c60 con 0x55a914247be0 2023-07-21T19:18:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e740b6020 con 0x55a914247be0 2023-07-21T19:18:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e740afd90 con 0x55a914247be0 2023-07-21T19:18:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 10 client.27416 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 10 client.27416 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:18:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 20 client.27416 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a914261fe0 0x55a9142643f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 4 client.27416 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a914192eb0 con 0x55a914261fe0 2023-07-21T19:18:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.321+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a914261fe0 0x55a9142643f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.322+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a914261fe0 0x55a9142643f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=874df266ff951d1b server_cookie=b756075784a2af0a in_seq=0 out_seq=0 2023-07-21T19:18:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.322+0000 7f5e5f7fe700 10 client.27416 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:18:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.323+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3067042543 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a914192eb0 con 0x55a914261fe0 2023-07-21T19:18:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.323+0000 7f5e5f7fe700 10 client.27416 handle_command_reply: tid=0 2023-07-21T19:18:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.323+0000 7f5e99d10b80 1 client.27416 shutdown 2023-07-21T19:18:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.323+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 msgr2=0x55a91424f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.323+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5e740ca650 tx=0x7f5e74010bf0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 shutdown_connections 2023-07-21T19:18:33.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a91424fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a914252550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a914261fe0 0x55a9142643f0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3067042543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914247be0 0x55a91424f760 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 >> 172.21.15.5:0/3067042543 conn(0x55a91423dda0 msgr2=0x55a914243290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e5f7fe700 0 client.27416 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:18:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 shutdown_connections 2023-07-21T19:18:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3067042543 wait complete. 2023-07-21T19:18:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.324+0000 7f5e99d10b80 20 client.27416 trim_cache size 0 max 16384 2023-07-21T19:18:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 Processor -- start 2023-07-21T19:18:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 -- start start 2023-07-21T19:18:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914247be0 0x55a914252080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 0x55a91424aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91424a1b0 con 0x55a914247be0 2023-07-21T19:18:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914247990 con 0x55a914252ad0 2023-07-21T19:18:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.344+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914268d10 con 0x55a9142472d0 2023-07-21T19:18:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56856/0 (socket says 172.21.15.5:56856) 2023-07-21T19:18:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914247be0 0x55a914252080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 -- 172.21.15.5:0/3484449180 learned_addr learned my addr 172.21.15.5:0/3484449180 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e8499c700 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 0x55a91424aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 -- 172.21.15.5:0/3484449180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 msgr2=0x55a91424aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 0x55a91424aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 -- 172.21.15.5:0/3484449180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914247be0 msgr2=0x55a914252080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914247be0 0x55a914252080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e7ffff700 1 -- 172.21.15.5:0/3484449180 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a913fee250 con 0x55a9142472d0 2023-07-21T19:18:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.345+0000 7f5e8499c700 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 0x55a91424aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:18:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.346+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005e90 tx=0x7f5e70009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87736022b18dc7f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.346+0000 7f5e66ffd700 1 -- 172.21.15.5:0/3484449180 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700249e0 con 0x55a9142472d0 2023-07-21T19:18:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.346+0000 7f5e66ffd700 1 -- 172.21.15.5:0/3484449180 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e7001d070 con 0x55a9142472d0 2023-07-21T19:18:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.346+0000 7f5e66ffd700 1 -- 172.21.15.5:0/3484449180 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e700052a0 con 0x55a9142472d0 2023-07-21T19:18:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3484449180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 msgr2=0x55a91424e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005e90 tx=0x7f5e70009840 comp rx=0 tx=0).stop 2023-07-21T19:18:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3484449180 shutdown_connections 2023-07-21T19:18:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a914252ad0 0x55a91424aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9142472d0 0x55a91424e9f0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3484449180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914247be0 0x55a914252080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3484449180 >> 172.21.15.5:0/3484449180 conn(0x55a91423dda0 msgr2=0x55a914263ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3484449180 shutdown_connections 2023-07-21T19:18:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.347+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3484449180 wait complete. 2023-07-21T19:18:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.348+0000 7f5e99d10b80 1 -- start start 2023-07-21T19:18:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.348+0000 7f5e99d10b80 1 Processor -- start 2023-07-21T19:18:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 0x55a914259170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 0x55a9142596b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a914257690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914257d60 con 0x55a914252ad0 2023-07-21T19:18:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914257ed0 con 0x55a9142472d0 2023-07-21T19:18:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a914258090 con 0x55a914247be0 2023-07-21T19:18:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e99d10b80 10 client.0 _ll_register_callbacks cb 0x7f5e74005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 0x55a914259170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 0x55a9142596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e7ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 0x55a914259170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38320/0 (socket says 172.21.15.5:38320) 2023-07-21T19:18:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e7ffff700 1 -- 172.21.15.5:0/3307532422 learned_addr learned my addr 172.21.15.5:0/3307532422 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.349+0000 7f5e8499c700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a914257690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 msgr2=0x55a914259170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 0x55a914259170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3307532422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 msgr2=0x55a914257690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a914257690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 -- 172.21.15.5:0/3307532422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e801137c0 con 0x55a914247be0 2023-07-21T19:18:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e7f7fe700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 0x55a9142596b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e74099c70 tx=0x7f5e7400c6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2349128d979e6ce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e5f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.350+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e74099f70 con 0x55a914247be0 2023-07-21T19:18:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e74077640 con 0x55a914247be0 2023-07-21T19:18:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e740b7080 con 0x55a914247be0 2023-07-21T19:18:33.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e740a21c0 con 0x55a914247be0 2023-07-21T19:18:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 10 client.27349 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55a913fc62b0 con 0x55a914247be0 2023-07-21T19:18:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e74077640 con 0x55a914247be0 2023-07-21T19:18:33.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e74095d60 con 0x55a914247be0 2023-07-21T19:18:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 10 client.27349 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 10 client.27349 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:18:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 20 client.27349 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a9142b6be0 0x55a9142b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 4 client.27349 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:18:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.351+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55a914155050 con 0x55a9142b6be0 2023-07-21T19:18:33.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a9142b6be0 0x55a9142b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e7ffff700 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a9142b6be0 0x55a9142b8ff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c27573224fe1892 server_cookie=11cdd0a577435441 in_seq=0 out_seq=0 2023-07-21T19:18:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e5f7fe700 10 client.27349 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:18:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e5f7fe700 1 -- 172.21.15.5:0/3307532422 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55a914155050 con 0x55a9142b6be0 2023-07-21T19:18:33.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e5f7fe700 10 client.27349 handle_command_reply: tid=0 2023-07-21T19:18:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.352+0000 7f5e99d10b80 1 client.27349 shutdown 2023-07-21T19:18:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 msgr2=0x55a9142596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 0x55a9142596b0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5e74099c70 tx=0x7f5e7400c6e0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 shutdown_connections 2023-07-21T19:18:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a914252ad0 0x55a914257690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55a9142b6be0 0x55a9142b8ff0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9142472d0 0x55a914259170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 --2- 172.21.15.5:0/3307532422 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a914247be0 0x55a9142596b0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 >> 172.21.15.5:0/3307532422 conn(0x55a91423dda0 msgr2=0x55a91424aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e5f7fe700 0 client.27349 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 shutdown_connections 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 1 -- 172.21.15.5:0/3307532422 wait complete. 2023-07-21T19:18:33.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.353+0000 7f5e99d10b80 20 client.27349 trim_cache size 0 max 16384 2023-07-21T19:18:33.369 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:18:33.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 msgr2=0x7f5e680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 0x7f5e680a3770 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5e78003fd0 tx=0x7f5e780097f0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 msgr2=0x7f5e80110c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80110c70 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400e9f0 tx=0x7f5e7400ed00 comp rx=0 tx=0).stop 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 shutdown_connections 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e80103e40 0x7f5e8010b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e680a12c0 0x7f5e680a3770 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e800fea90 0x7f5e80110730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 --2- 172.21.15.5:0/2800412735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e800ff3a0 0x7f5e80110c70 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 >> 172.21.15.5:0/2800412735 conn(0x7f5e800fa4b0 msgr2=0x7f5e80101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 shutdown_connections 2023-07-21T19:18:33.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.354+0000 7f5e863ff700 1 -- 172.21.15.5:0/2800412735 wait complete. 2023-07-21T19:18:33.372 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 -- 172.21.15.5:0/1798807870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 msgr2=0x7ff2b410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 --2- 172.21.15.5:0/1798807870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b410c920 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00b0a0 tx=0x7ff2ac00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 -- 172.21.15.5:0/1798807870 shutdown_connections 2023-07-21T19:18:33.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 --2- 172.21.15.5:0/1798807870 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b410c920 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 --2- 172.21.15.5:0/1798807870 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 0x7ff2b4105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 --2- 172.21.15.5:0/1798807870 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b40fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.617+0000 7ff2badff700 1 -- 172.21.15.5:0/1798807870 >> 172.21.15.5:0/1798807870 conn(0x7ff2b40fb5e0 msgr2=0x7ff2b40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.618+0000 7ff2badff700 1 -- 172.21.15.5:0/1798807870 shutdown_connections 2023-07-21T19:18:33.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.618+0000 7ff2badff700 1 -- 172.21.15.5:0/1798807870 wait complete. 2023-07-21T19:18:33.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.619+0000 7ff2badff700 1 Processor -- start 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.619+0000 7ff2badff700 1 -- start start 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.619+0000 7ff2badff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.619+0000 7ff2badff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b4198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2badff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 0x7ff2b419c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2badff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419cdc0 con 0x7ff2b4100490 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2badff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419cf30 con 0x7ff2b40ffae0 2023-07-21T19:18:33.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2badff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b419d0f0 con 0x7ff2b4105af0 2023-07-21T19:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38332/0 (socket says 172.21.15.5:38332) 2023-07-21T19:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/4018655389 learned_addr learned my addr 172.21.15.5:0/4018655389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b4198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b939c700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 0x7ff2b419c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/4018655389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 msgr2=0x7ff2b419c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 0x7ff2b419c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 msgr2=0x7ff2b4198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b4198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.620+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/4018655389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ac009d00 con 0x7ff2b40ffae0 2023-07-21T19:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e9a0 tx=0x7ff2a800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2b1ffb700 1 -- 172.21.15.5:0/4018655389 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a8009790 con 0x7ff2b40ffae0 2023-07-21T19:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2b1ffb700 1 -- 172.21.15.5:0/4018655389 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2a8010430 con 0x7ff2b40ffae0 2023-07-21T19:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b419d3d0 con 0x7ff2b40ffae0 2023-07-21T19:18:33.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2b1ffb700 1 -- 172.21.15.5:0/4018655389 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800cea0 con 0x7ff2b40ffae0 2023-07-21T19:18:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.621+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b41a4c10 con 0x7ff2b40ffae0 2023-07-21T19:18:33.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.624+0000 7ff2b1ffb700 1 -- 172.21.15.5:0/4018655389 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff2a80105a0 con 0x7ff2b40ffae0 2023-07-21T19:18:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.626+0000 7ff2ce725b80 1 Processor -- start 2023-07-21T19:18:33.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.626+0000 7ff2ce725b80 1 -- start start 2023-07-21T19:18:33.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.626+0000 7ff2b1ffb700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.626+0000 7ff2b1ffb700 1 -- 172.21.15.5:0/4018655389 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff2a8014070 con 0x7ff2b40ffae0 2023-07-21T19:18:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b939c700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aec86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91adcecb0 con 0x55b91aed3ad0 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91ae21a40 con 0x55b91aec82d0 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91ae1aed0 con 0x55b91aec8be0 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59110/0 (socket says 172.21.15.5:59110) 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b939c700 1 -- 172.21.15.5:0/343362958 learned_addr learned my addr 172.21.15.5:0/343362958 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.627+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aec86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 -- 172.21.15.5:0/343362958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 msgr2=0x55b91aec86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aec86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 -- 172.21.15.5:0/343362958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 msgr2=0x55b91aed5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 -- 172.21.15.5:0/343362958 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b91ac48de0 con 0x55b91aec8be0 2023-07-21T19:18:33.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b939c700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b939c700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 0x7ff2a40a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00d010 tx=0x7ff2ac007dd0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed3590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009c80 tx=0x7ff2a0009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7a460f55324c317 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff29effd700 1 -- 172.21.15.5:0/343362958 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0011c40 con 0x55b91aec8be0 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff29effd700 1 -- 172.21.15.5:0/343362958 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2a00044e0 con 0x55b91aec8be0 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.628+0000 7ff29effd700 1 -- 172.21.15.5:0/343362958 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005220 con 0x55b91aec8be0 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 -- 172.21.15.5:0/343362958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 msgr2=0x55b91aed3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed3590 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009c80 tx=0x7ff2a0009f90 comp rx=0 tx=0).stop 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 -- 172.21.15.5:0/343362958 shutdown_connections 2023-07-21T19:18:33.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aec86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/343362958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed3590 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 -- 172.21.15.5:0/343362958 >> 172.21.15.5:0/343362958 conn(0x55b91aebeda0 msgr2=0x55b91aec46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 -- 172.21.15.5:0/343362958 shutdown_connections 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.629+0000 7ff2ce725b80 1 -- 172.21.15.5:0/343362958 wait complete. 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 -- start start 2023-07-21T19:18:33.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 Processor -- start 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aed3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91adbaac0 con 0x55b91aed3ad0 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91adba7c0 con 0x55b91aec82d0 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91ac9e920 con 0x55b91aec8be0 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.630+0000 7ff2ce725b80 10 client.0 _ll_register_callbacks cb 0x55b91adb8760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aed3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56884/0 (socket says 172.21.15.5:56884) 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 -- 172.21.15.5:0/3766309096 learned_addr learned my addr 172.21.15.5:0/3766309096 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 -- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 msgr2=0x55b91aed3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aed3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 -- 172.21.15.5:0/3766309096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 msgr2=0x55b91aed0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b3fff700 1 -- 172.21.15.5:0/3766309096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2a00098e0 con 0x55b91aec8be0 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aed3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.631+0000 7ff2b939c700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005e40 tx=0x7ff2a0004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfaaaed97ed34d49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff293fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0011d50 con 0x55b91aec8be0 2023-07-21T19:18:33.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b91ad744f0 con 0x55b91aec8be0 2023-07-21T19:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff2a0003680 con 0x55b91aec8be0 2023-07-21T19:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a0003840 con 0x55b91aec8be0 2023-07-21T19:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff2ce725b80 10 client.27361 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.632+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b91ac31c60 con 0x55b91aec8be0 2023-07-21T19:18:33.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b91ad744f0 con 0x55b91aec8be0 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2a0022a20 con 0x55b91aec8be0 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 10 client.27361 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 10 client.27361 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 20 client.27361 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91aee3220 0x55b91aee5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 4 client.27361 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:33.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.633+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b91ae13eb0 con 0x55b91aee3220 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.634+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91aee3220 0x55b91aee5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.634+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91aee3220 0x55b91aee5630 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8efdf72a4ae69001 server_cookie=b5d896a765b60aa3 in_seq=0 out_seq=0 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.634+0000 7ff293fff700 10 client.27361 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff293fff700 1 -- 172.21.15.5:0/3766309096 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b91ae13eb0 con 0x55b91aee3220 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff293fff700 10 client.27361 handle_command_reply: tid=0 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 client.27361 shutdown 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 msgr2=0x55b91aed06c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0005e40 tx=0x7ff2a0004750 comp rx=0 tx=0).stop 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 shutdown_connections 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91aee3220 0x55b91aee5630 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aed3ad0 0x55b91aed0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec82d0 0x55b91aed3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff293fff700 0 client.27361 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3766309096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec8be0 0x55b91aed06c0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 >> 172.21.15.5:0/3766309096 conn(0x55b91aebeda0 msgr2=0x55b91aec4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 shutdown_connections 2023-07-21T19:18:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3766309096 wait complete. 2023-07-21T19:18:33.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.635+0000 7ff2ce725b80 20 client.27361 trim_cache size 0 max 16384 2023-07-21T19:18:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.651+0000 7ff2ce725b80 1 Processor -- start 2023-07-21T19:18:33.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.651+0000 7ff2ce725b80 1 -- start start 2023-07-21T19:18:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 0x55b91aeda4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aed3ad0 0x55b91aedf720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91aeea500 con 0x55b91aec8be0 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91aed12f0 con 0x55b91aed3ad0 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91aedaa20 con 0x55b91aec82d0 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 0x55b91aeda4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 0x55b91aeda4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59126/0 (socket says 172.21.15.5:59126) 2023-07-21T19:18:33.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2023-07-21T19:18:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b3fff700 1 -- 172.21.15.5:0/1460132279 learned_addr learned my addr 172.21.15.5:0/1460132279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b939c700 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aed3ad0 0x55b91aedf720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/1460132279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aed3ad0 msgr2=0x55b91aedf720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aed3ad0 0x55b91aedf720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/1460132279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 msgr2=0x55b91aeda4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 0x55b91aeda4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.652+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/1460132279 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b91ac6f250 con 0x55b91aec82d0 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.653+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80ca560 tx=0x7ff2a801a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b5f66d4fb3378c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.653+0000 7ff29f7fe700 1 -- 172.21.15.5:0/1460132279 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80b82c0 con 0x55b91aec82d0 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.653+0000 7ff29f7fe700 1 -- 172.21.15.5:0/1460132279 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2a80b1070 con 0x55b91aec82d0 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.653+0000 7ff29f7fe700 1 -- 172.21.15.5:0/1460132279 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80983a0 con 0x55b91aec82d0 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- 172.21.15.5:0/1460132279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 msgr2=0x55b91aee9f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff2a80ca560 tx=0x7ff2a801a040 comp rx=0 tx=0).stop 2023-07-21T19:18:33.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- 172.21.15.5:0/1460132279 shutdown_connections 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aed3ad0 0x55b91aedf720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aec82d0 0x55b91aee9f50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/1460132279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec8be0 0x55b91aeda4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- 172.21.15.5:0/1460132279 >> 172.21.15.5:0/1460132279 conn(0x55b91aebeda0 msgr2=0x55b91aec0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- 172.21.15.5:0/1460132279 shutdown_connections 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- 172.21.15.5:0/1460132279 wait complete. 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.654+0000 7ff2ce725b80 1 -- start start 2023-07-21T19:18:33.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 Processor -- start 2023-07-21T19:18:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 0x55b91aed6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 0x55b91af36090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91af365d0 con 0x55b91aec82d0 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91af36710 con 0x55b91aec8be0 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b91af36850 con 0x55b91aed3ad0 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.655+0000 7ff2ce725b80 10 client.0 _ll_register_callbacks cb 0x7ff2a0009b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59132/0 (socket says 172.21.15.5:59132) 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/3409197335 learned_addr learned my addr 172.21.15.5:0/3409197335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:33.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b939c700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 0x55b91af36090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 0x55b91aed6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/3409197335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 msgr2=0x55b91af36090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 0x55b91af36090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 msgr2=0x55b91aed6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 0x55b91aed6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 -- 172.21.15.5:0/3409197335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b41a4c10 con 0x55b91aec82d0 2023-07-21T19:18:33.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b939c700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 0x55b91af36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 0x55b91aed6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff2b8b9b700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800c2d0 tx=0x7ff2a80a2330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d01e4bc3a98a156 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.656+0000 7ff293fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a80adde0 con 0x55b91aec82d0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff2a809df10 con 0x55b91aec82d0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2a00044e0 con 0x55b91aec82d0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2a80ca300 con 0x55b91aec82d0 2023-07-21T19:18:33.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 10 client.17766 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55b91ac472b0 con 0x55b91aec82d0 2023-07-21T19:18:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2a00044e0 con 0x55b91aec82d0 2023-07-21T19:18:33.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff2a80a1fc0 con 0x55b91aec82d0 2023-07-21T19:18:33.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:33 smithi005 ceph-mon[126460]: pgmap v341: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:18:33.659 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:33 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1273106195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 10 client.17766 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 10 client.17766 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 20 client.17766 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.657+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91af37b80 0x55b91af39f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.658+0000 7ff2ce725b80 4 client.17766 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.658+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55b91add6050 con 0x55b91af37b80 2023-07-21T19:18:33.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.658+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91af37b80 0x55b91af39f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.658+0000 7ff2b3fff700 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91af37b80 0x55b91af39f90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3e9bbd09cafc819 server_cookie=37d2d3968537e990 in_seq=0 out_seq=0 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.658+0000 7ff293fff700 10 client.17766 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff293fff700 1 -- 172.21.15.5:0/3409197335 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55b91add6050 con 0x55b91af37b80 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff293fff700 10 client.17766 handle_command_reply: tid=0 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 client.17766 shutdown 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 msgr2=0x55b91aed66b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800c2d0 tx=0x7ff2a80a2330 comp rx=0 tx=0).stop 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 shutdown_connections 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b91aed3ad0 0x55b91af36090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b91af37b80 0x55b91af39f90 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b91aec82d0 0x55b91aed66b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 --2- 172.21.15.5:0/3409197335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b91aec8be0 0x55b91aed6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 >> 172.21.15.5:0/3409197335 conn(0x55b91aebeda0 msgr2=0x55b91aecba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff293fff700 0 client.17766 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:33.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 shutdown_connections 2023-07-21T19:18:33.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 1 -- 172.21.15.5:0/3409197335 wait complete. 2023-07-21T19:18:33.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.659+0000 7ff2ce725b80 20 client.17766 trim_cache size 0 max 16384 2023-07-21T19:18:33.664 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 msgr2=0x7ff2a40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 0x7ff2a40a37f0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7ff2ac00d010 tx=0x7ff2ac007dd0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 msgr2=0x7ff2b4197e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e9a0 tx=0x7ff2a800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 shutdown_connections 2023-07-21T19:18:33.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff2b4105af0 0x7ff2b419c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff2b40ffae0 0x7ff2b4197e30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2a40a1340 0x7ff2a40a37f0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 --2- 172.21.15.5:0/4018655389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff2b4100490 0x7ff2b4198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 >> 172.21.15.5:0/4018655389 conn(0x7ff2b40fb5e0 msgr2=0x7ff2b40fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.660+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 shutdown_connections 2023-07-21T19:18:33.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:33.661+0000 7ff2badff700 1 -- 172.21.15.5:0/4018655389 wait complete. 2023-07-21T19:18:33.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:33 smithi101 ceph-mon[128136]: pgmap v341: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:18:33.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:33 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1273106195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:18:33.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:33 smithi156 ceph-mon[128316]: pgmap v341: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:18:33.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:33 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1273106195' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:18:33.979 INFO:teuthology.orchestra.run.smithi005.stderr:41500000 of 50000000 tuples (83%) done (elapsed 631.55 s, remaining 129.35 s) 2023-07-21T19:18:34.676 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:18:34.677 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:18:34.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 -- 172.21.15.5:0/4129731947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 msgr2=0x7f1390105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 --2- 172.21.15.5:0/4129731947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390105eb0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f1380009990 tx=0x7f1380009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 -- 172.21.15.5:0/4129731947 shutdown_connections 2023-07-21T19:18:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 --2- 172.21.15.5:0/4129731947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 0x7f139010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 --2- 172.21.15.5:0/4129731947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390105eb0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 --2- 172.21.15.5:0/4129731947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f13900fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 -- 172.21.15.5:0/4129731947 >> 172.21.15.5:0/4129731947 conn(0x7f13900fb5f0 msgr2=0x7f13900fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 -- 172.21.15.5:0/4129731947 shutdown_connections 2023-07-21T19:18:34.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.918+0000 7f1394dff700 1 -- 172.21.15.5:0/4129731947 wait complete. 2023-07-21T19:18:34.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.919+0000 7f1394dff700 1 Processor -- start 2023-07-21T19:18:34.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.919+0000 7f1394dff700 1 -- start start 2023-07-21T19:18:34.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 0x7f139019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13901981c0 con 0x7f1390105ae0 2023-07-21T19:18:34.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1390198330 con 0x7f1390100480 2023-07-21T19:18:34.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f1394dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13901984f0 con 0x7f13900ffad0 2023-07-21T19:18:34.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56936/0 (socket says 172.21.15.5:56936) 2023-07-21T19:18:34.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 -- 172.21.15.5:0/2941270056 learned_addr learned my addr 172.21.15.5:0/2941270056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:34.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138dd9b700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 0x7f139019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 -- 172.21.15.5:0/2941270056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 msgr2=0x7f139019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 0x7f139019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 -- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 msgr2=0x7f1390197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.920+0000 7f138e59c700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f138e59c700 1 -- 172.21.15.5:0/2941270056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13800095f0 con 0x7f13900ffad0 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f138e59c700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1380005740 tx=0x7f1380005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f137f7fe700 1 -- 172.21.15.5:0/2941270056 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138001d070 con 0x7f13900ffad0 2023-07-21T19:18:34.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1390198770 con 0x7f13900ffad0 2023-07-21T19:18:34.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f137f7fe700 1 -- 172.21.15.5:0/2941270056 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1380022470 con 0x7f13900ffad0 2023-07-21T19:18:34.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f137f7fe700 1 -- 172.21.15.5:0/2941270056 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380017650 con 0x7f13900ffad0 2023-07-21T19:18:34.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.921+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901a4ad0 con 0x7f13900ffad0 2023-07-21T19:18:34.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.925+0000 7f137f7fe700 1 -- 172.21.15.5:0/2941270056 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f138000fb00 con 0x7f13900ffad0 2023-07-21T19:18:34.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.926+0000 7f13a873bb80 1 Processor -- start 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.926+0000 7f13a873bb80 1 -- start start 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bea350 0x55dca9bea720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 0x55dca9bf7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9af0d30 con 0x55dca9bea350 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9b43ac0 con 0x55dca9bf5b50 2023-07-21T19:18:34.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f13a873bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9b3cf50 con 0x55dca9beac60 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f137f7fe700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 0x7f13780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.927+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bea350 0x55dca9bea720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 0x55dca9bf7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56940/0 (socket says 172.21.15.5:56940) 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138e59c700 1 -- 172.21.15.5:0/274295338 learned_addr learned my addr 172.21.15.5:0/274295338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:34.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138ed9d700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 0x7f13780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f137f7fe700 1 -- 172.21.15.5:0/2941270056 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f13800ce2a0 con 0x7f13900ffad0 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 -- 172.21.15.5:0/274295338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 msgr2=0x55dca9bf5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 -- 172.21.15.5:0/274295338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bea350 msgr2=0x55dca9bea720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bea350 0x55dca9bea720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 -- 172.21.15.5:0/274295338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dca996ae60 con 0x55dca9bf5b50 2023-07-21T19:18:34.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138e59c700 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:34.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138dd9b700 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 0x55dca9bf7f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bc0 tx=0x7f1384007ed0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d53456dd6a733 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.928+0000 7f138ed9d700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 0x7f13780a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1388009ec0 tx=0x7f138800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.929+0000 7f137cff9700 1 -- 172.21.15.5:0/274295338 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384004250 con 0x55dca9bf5b50 2023-07-21T19:18:34.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.929+0000 7f137cff9700 1 -- 172.21.15.5:0/274295338 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1384010e10 con 0x55dca9bf5b50 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.929+0000 7f137cff9700 1 -- 172.21.15.5:0/274295338 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384005020 con 0x55dca9bf5b50 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.929+0000 7f13a873bb80 1 -- 172.21.15.5:0/274295338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 msgr2=0x55dca9bf7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.929+0000 7f13a873bb80 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 0x55dca9bf7f30 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bc0 tx=0x7f1384007ed0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 -- 172.21.15.5:0/274295338 shutdown_connections 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bf5b50 0x55dca9bf7f30 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bea350 0x55dca9bea720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 --2- 172.21.15.5:0/274295338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 -- 172.21.15.5:0/274295338 >> 172.21.15.5:0/274295338 conn(0x55dca9be0e20 msgr2=0x55dca9be6720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 -- 172.21.15.5:0/274295338 shutdown_connections 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 -- 172.21.15.5:0/274295338 wait complete. 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.930+0000 7f13a873bb80 1 -- start start 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 Processor -- start 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf2820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9bf2e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9adcb40 con 0x55dca9bf5b50 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9adc840 con 0x55dca9bea350 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca99c09a0 con 0x55dca9beac60 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.931+0000 7f13a873bb80 10 client.0 _ll_register_callbacks cb 0x55dca9ada7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9bf2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9bf2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59146/0 (socket says 172.21.15.5:59146) 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38388/0 (socket says 172.21.15.5:38388) 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138ed9d700 1 -- 172.21.15.5:0/3789232557 learned_addr learned my addr 172.21.15.5:0/3789232557 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 -- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 msgr2=0x55dca9bf5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 -- 172.21.15.5:0/3789232557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 msgr2=0x55dca9bf2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9bf2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138dd9b700 1 -- 172.21.15.5:0/3789232557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13901a4ad0 con 0x55dca9beac60 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.932+0000 7f138e59c700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f138dd9b700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf2820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bc0 tx=0x7f13840049b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96ee50ef9452eaa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f1375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138401f070 con 0x55dca9beac60 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dca9a96570 con 0x55dca9beac60 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1384004050 con 0x55dca9beac60 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1384004210 con 0x55dca9beac60 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f13a873bb80 10 client.27379 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:34.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.933+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f13840078a0 con 0x55dca9beac60 2023-07-21T19:18:34.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1384030020 con 0x55dca9beac60 2023-07-21T19:18:34.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1384011440 con 0x55dca9beac60 2023-07-21T19:18:34.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 10 client.27379 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 10 client.27379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 20 client.27379 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c05420 0x55dca9c07830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 4 client.27379 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dca9b35f30 con 0x55dca9c05420 2023-07-21T19:18:34.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.934+0000 7f138e59c700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c05420 0x55dca9c07830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.935+0000 7f1375ffb700 10 client.27379 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.935+0000 7f138e59c700 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c05420 0x55dca9c07830 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5efa53530d0138a9 server_cookie=f359dee2550c20ba in_seq=0 out_seq=0 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f1375ffb700 1 -- 172.21.15.5:0/3789232557 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dca9b35f30 con 0x55dca9c05420 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f1375ffb700 10 client.27379 handle_command_reply: tid=0 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 client.27379 shutdown 2023-07-21T19:18:34.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 msgr2=0x55dca9bf2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf2820 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f1384007bc0 tx=0x7f13840049b0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 shutdown_connections 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c05420 0x55dca9c07830 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9bf2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9bea350 0x55dca9bf5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3789232557 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9beac60 0x55dca9bf2820 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f1375ffb700 0 client.27379 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 >> 172.21.15.5:0/3789232557 conn(0x55dca9be0e20 msgr2=0x55dca9be6310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 shutdown_connections 2023-07-21T19:18:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 1 -- 172.21.15.5:0/3789232557 wait complete. 2023-07-21T19:18:34.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.936+0000 7f13a873bb80 20 client.27379 trim_cache size 0 max 16384 2023-07-21T19:18:34.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.954+0000 7f13a873bb80 1 Processor -- start 2023-07-21T19:18:34.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.954+0000 7f13a873bb80 1 -- start start 2023-07-21T19:18:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9beb5c0 con 0x55dca9bf5b50 2023-07-21T19:18:34.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9bde020 con 0x55dca9beac60 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f13a873bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9be09f0 con 0x55dca9bea350 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f138ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59160/0 (socket says 172.21.15.5:59160) 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f138ed9d700 1 -- 172.21.15.5:0/3824376836 learned_addr learned my addr 172.21.15.5:0/3824376836 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.955+0000 7f138dd9b700 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 -- 172.21.15.5:0/3824376836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 msgr2=0x55dca9bf2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bf2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 -- 172.21.15.5:0/3824376836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 msgr2=0x55dca9befb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 -- 172.21.15.5:0/3824376836 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dca9953ce0 con 0x55dca9bea350 2023-07-21T19:18:34.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138ed9d700 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f138e59c700 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bf2030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1380097e10 tx=0x7f138006e5e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=154c21941d3293e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f137d7fa700 1 -- 172.21.15.5:0/3824376836 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800a3760 con 0x55dca9bea350 2023-07-21T19:18:34.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f137d7fa700 1 -- 172.21.15.5:0/3824376836 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138009b3a0 con 0x55dca9bea350 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.956+0000 7f137d7fa700 1 -- 172.21.15.5:0/3824376836 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138009f120 con 0x55dca9bea350 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 -- 172.21.15.5:0/3824376836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 msgr2=0x55dca9bf2030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bf2030 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f1380097e10 tx=0x7f138006e5e0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 -- 172.21.15.5:0/3824376836 shutdown_connections 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9befb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bf2030 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 --2- 172.21.15.5:0/3824376836 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bf2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 -- 172.21.15.5:0/3824376836 >> 172.21.15.5:0/3824376836 conn(0x55dca9be0e20 msgr2=0x55dca9bedb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 -- 172.21.15.5:0/3824376836 shutdown_connections 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.957+0000 7f13a873bb80 1 -- 172.21.15.5:0/3824376836 wait complete. 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.958+0000 7f13a873bb80 1 -- start start 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.958+0000 7f13a873bb80 1 Processor -- start 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9c532f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9c53830 con 0x55dca9bf5b50 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9c539a0 con 0x55dca9beac60 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dca9c53b60 con 0x55dca9bea350 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f13a873bb80 10 client.0 _ll_register_callbacks cb 0x7f1388009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.959 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38412/0 (socket says 172.21.15.5:38412) 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138dd9b700 1 -- 172.21.15.5:0/202262431 learned_addr learned my addr 172.21.15.5:0/202262431 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138e59c700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56972/0 (socket says 172.21.15.5:56972) 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.959+0000 7f138ed9d700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9c532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 -- 172.21.15.5:0/202262431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 msgr2=0x55dca9bfb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 -- 172.21.15.5:0/202262431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 msgr2=0x55dca9c532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:34.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9c532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 -- 172.21.15.5:0/202262431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1388003a00 con 0x55dca9beac60 2023-07-21T19:18:34.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138ed9d700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9c532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:34.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138e59c700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f138dd9b700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13840106b0 tx=0x7f13840042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6bc09a81b4bea4b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f1375ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138401f070 con 0x55dca9beac60 2023-07-21T19:18:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1384031080 con 0x55dca9beac60 2023-07-21T19:18:34.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.960+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1384010e10 con 0x55dca9beac60 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1384007ed0 con 0x55dca9beac60 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 10 client.27455 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f1390198770 con 0x55dca9beac60 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1384041080 con 0x55dca9beac60 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f138403b030 con 0x55dca9beac60 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 10 client.27455 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:34.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 10 client.27455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 20 client.27455 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c5b090 0x55dca9c5d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 4 client.27455 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.961+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55dca9af80d0 con 0x55dca9c5b090 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.962+0000 7f138e59c700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c5b090 0x55dca9c5d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.962+0000 7f138e59c700 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c5b090 0x55dca9c5d4a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce64f79a53e7065c server_cookie=f8aa6cea17030ad8 in_seq=0 out_seq=0 2023-07-21T19:18:34.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.962+0000 7f1375ffb700 10 client.27455 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.963+0000 7f1375ffb700 1 -- 172.21.15.5:0/202262431 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55dca9af80d0 con 0x55dca9c5b090 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.963+0000 7f1375ffb700 10 client.27455 handle_command_reply: tid=0 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.963+0000 7f13a873bb80 1 client.27455 shutdown 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.963+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 msgr2=0x55dca9bfba30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.963+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f13840106b0 tx=0x7f13840042f0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 shutdown_connections 2023-07-21T19:18:34.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dca9c5b090 0x55dca9c5d4a0 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dca9bf5b50 0x55dca9c532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dca9bea350 0x55dca9bfb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 --2- 172.21.15.5:0/202262431 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dca9beac60 0x55dca9bfba30 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 >> 172.21.15.5:0/202262431 conn(0x55dca9be0e20 msgr2=0x55dca9bf40c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f1375ffb700 0 client.27455 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 shutdown_connections 2023-07-21T19:18:34.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 1 -- 172.21.15.5:0/202262431 wait complete. 2023-07-21T19:18:34.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.964+0000 7f13a873bb80 20 client.27455 trim_cache size 0 max 16384 2023-07-21T19:18:34.968 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:18:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:18:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "222784ff-b70f-4d01-950c-d88fade4db3d", 2023-07-21T19:18:34.968 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:18:34.969 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:18:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 msgr2=0x7f13780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 0x7f13780a3770 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f1388009ec0 tx=0x7f138800b040 comp rx=0 tx=0).stop 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 msgr2=0x7f139019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1380005740 tx=0x7f1380005770 comp rx=0 tx=0).stop 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 shutdown_connections 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13780a12c0 0x7f13780a3770 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1390105ae0 0x7f1390197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f13900ffad0 0x7f139019dae0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 --2- 172.21.15.5:0/2941270056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1390100480 0x7f139019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 >> 172.21.15.5:0/2941270056 conn(0x7f13900fb5f0 msgr2=0x7f139010adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 shutdown_connections 2023-07-21T19:18:34.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:34.965+0000 7f1394dff700 1 -- 172.21.15.5:0/2941270056 wait complete. 2023-07-21T19:18:34.985 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:18:35.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 -- 172.21.15.5:0/346368795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e4100ca0 msgr2=0x7f74e4101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/346368795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e4100ca0 0x7f74e4101070 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009990 tx=0x7f74e0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 -- 172.21.15.5:0/346368795 shutdown_connections 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/346368795 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e410a070 0x7f74e410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/346368795 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e4100ca0 0x7f74e4101070 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/346368795 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e41015b0 0x7f74e4109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 -- 172.21.15.5:0/346368795 >> 172.21.15.5:0/346368795 conn(0x7f74e40fa4b0 msgr2=0x7f74e40fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 -- 172.21.15.5:0/346368795 shutdown_connections 2023-07-21T19:18:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.262+0000 7f74ec3ff700 1 -- 172.21.15.5:0/346368795 wait complete. 2023-07-21T19:18:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.263+0000 7f74ec3ff700 1 Processor -- start 2023-07-21T19:18:35.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 -- start start 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e41015b0 0x7f74e419c020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e410a070 0x7f74e4195b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e41961c0 con 0x7f74e41015b0 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e4196330 con 0x7f74e4100ca0 2023-07-21T19:18:35.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ec3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74e41964f0 con 0x7f74e410a070 2023-07-21T19:18:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ea19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ea19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38428/0 (socket says 172.21.15.5:38428) 2023-07-21T19:18:35.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.264+0000 7f74ea19b700 1 -- 172.21.15.5:0/2456361621 learned_addr learned my addr 172.21.15.5:0/2456361621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea99c700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e410a070 0x7f74e4195b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74e999a700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e41015b0 0x7f74e419c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea19b700 1 -- 172.21.15.5:0/2456361621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e410a070 msgr2=0x7f74e4195b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea19b700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e410a070 0x7f74e4195b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea19b700 1 -- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e41015b0 msgr2=0x7f74e419c020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea19b700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e41015b0 0x7f74e419c020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.265+0000 7f74ea19b700 1 -- 172.21.15.5:0/2456361621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74e00095f0 con 0x7f74e4100ca0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74ea19b700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005740 tx=0x7f74e0005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74db7fe700 1 -- 172.21.15.5:0/2456361621 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e001d070 con 0x7f74e4100ca0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74db7fe700 1 -- 172.21.15.5:0/2456361621 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74e0022470 con 0x7f74e4100ca0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74e4196770 con 0x7f74e4100ca0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74db7fe700 1 -- 172.21.15.5:0/2456361621 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74e0017650 con 0x7f74e4100ca0 2023-07-21T19:18:35.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.266+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74e40fea00 con 0x7f74e4100ca0 2023-07-21T19:18:35.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.270+0000 7f74db7fe700 1 -- 172.21.15.5:0/2456361621 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f74e000fb00 con 0x7f74e4100ca0 2023-07-21T19:18:35.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.271+0000 7f74ffc4db80 1 Processor -- start 2023-07-21T19:18:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.271+0000 7f74ffc4db80 1 -- start start 2023-07-21T19:18:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dbb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073cc1cb0 con 0x560073dc6ad0 2023-07-21T19:18:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073d14a40 con 0x560073dbbbe0 2023-07-21T19:18:35.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073d0ded0 con 0x560073dbb2d0 2023-07-21T19:18:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ea19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ea19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38442/0 (socket says 172.21.15.5:38442) 2023-07-21T19:18:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ea19b700 1 -- 172.21.15.5:0/1852983179 learned_addr learned my addr 172.21.15.5:0/1852983179 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:35.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.272+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dbb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74e999a700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74ea99c700 1 -- 172.21.15.5:0/1852983179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 msgr2=0x560073dc6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74ea99c700 1 -- 172.21.15.5:0/1852983179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 msgr2=0x560073dc8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74ea99c700 1 -- 172.21.15.5:0/1852983179 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560073b3bde0 con 0x560073dbb2d0 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.273+0000 7f74e999a700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:35.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dbb6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f74e419c7e0 tx=0x7f74dc009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b26f8db0a9cc047 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74d8bfe700 1 -- 172.21.15.5:0/1852983179 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc019070 con 0x560073dbb2d0 2023-07-21T19:18:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74db7fe700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 0x7f74d00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74d8bfe700 1 -- 172.21.15.5:0/1852983179 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74dc00f040 con 0x560073dbb2d0 2023-07-21T19:18:35.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74d8bfe700 1 -- 172.21.15.5:0/1852983179 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc007830 con 0x560073dbb2d0 2023-07-21T19:18:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74db7fe700 1 -- 172.21.15.5:0/2456361621 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f74e00ce280 con 0x7f74e4100ca0 2023-07-21T19:18:35.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.274+0000 7f74ea19b700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 0x7f74d00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1852983179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 msgr2=0x560073dbb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dbb6a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f74e419c7e0 tx=0x7f74dc009ba0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ea19b700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 0x7f74d00a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009350 tx=0x7f74e0097040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1852983179 shutdown_connections 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dbb6a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1852983179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1852983179 >> 172.21.15.5:0/1852983179 conn(0x560073db1da0 msgr2=0x560073db76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1852983179 shutdown_connections 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.275+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1852983179 wait complete. 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.276+0000 7f74ffc4db80 1 -- start start 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.276+0000 7f74ffc4db80 1 Processor -- start 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dc6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073cadac0 con 0x560073dc6ad0 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073cad7c0 con 0x560073dbbbe0 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073b91920 con 0x560073dbb2d0 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ffc4db80 10 client.0 _ll_register_callbacks cb 0x560073cab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74e999a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74e999a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38456/0 (socket says 172.21.15.5:38456) 2023-07-21T19:18:35.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74e999a700 1 -- 172.21.15.5:0/1457311153 learned_addr learned my addr 172.21.15.5:0/1457311153 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ea99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dc6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.277+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 -- 172.21.15.5:0/1457311153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 msgr2=0x560073dc6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dc6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 -- 172.21.15.5:0/1457311153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 msgr2=0x560073dc3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 -- 172.21.15.5:0/1457311153 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d40098e0 con 0x560073dbbbe0 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dc6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:18:35.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74e999a700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74d4011950 tx=0x7f74d4011d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=affb7a9b0e718382 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4016070 con 0x560073dbbbe0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f74d4004300 con 0x560073dbbbe0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.278+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560073c674f0 con 0x560073dbbbe0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74d40044c0 con 0x560073dbbbe0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74ffc4db80 10 client.27467 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f74dc003ae0 con 0x560073dbbbe0 2023-07-21T19:18:35.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74d4027020 con 0x560073dbbbe0 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74d4003b20 con 0x560073dbbbe0 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.279+0000 7f74ffc4db80 10 client.27467 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ffc4db80 10 client.27467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ffc4db80 20 client.27467 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073ddec90 0x560073de10a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ffc4db80 4 client.27467 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560073d06eb0 con 0x560073ddec90 2023-07-21T19:18:35.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.280+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073ddec90 0x560073de10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.281+0000 7f74cdffb700 10 client.27467 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.281+0000 7f74ea99c700 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073ddec90 0x560073de10a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6532deb6c8ad744d server_cookie=b4c0662533e57df0 in_seq=0 out_seq=0 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.281+0000 7f74cdffb700 1 -- 172.21.15.5:0/1457311153 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560073d06eb0 con 0x560073ddec90 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.281+0000 7f74cdffb700 10 client.27467 handle_command_reply: tid=0 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.281+0000 7f74ffc4db80 1 client.27467 shutdown 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 msgr2=0x560073dc36b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f74d4011950 tx=0x7f74d4011d10 comp rx=0 tx=0).stop 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 shutdown_connections 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073ddec90 0x560073de10a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dc6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/1457311153 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dc36b0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 >> 172.21.15.5:0/1457311153 conn(0x560073db1da0 msgr2=0x560073db7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74cdffb700 0 client.27467 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:35.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 shutdown_connections 2023-07-21T19:18:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 1 -- 172.21.15.5:0/1457311153 wait complete. 2023-07-21T19:18:35.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.282+0000 7f74ffc4db80 20 client.27467 trim_cache size 0 max 16384 2023-07-21T19:18:35.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.305+0000 7f74ffc4db80 1 Processor -- start 2023-07-21T19:18:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.305+0000 7f74ffc4db80 1 -- start start 2023-07-21T19:18:35.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dc6ad0 0x560073dbe990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073db8ad0 con 0x560073dbbbe0 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073dc4270 con 0x560073dc6ad0 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073dbc540 con 0x560073dbb2d0 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74e999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74e999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59204/0 (socket says 172.21.15.5:59204) 2023-07-21T19:18:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ea99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74e999a700 1 -- 172.21.15.5:0/3663780388 learned_addr learned my addr 172.21.15.5:0/3663780388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:35.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.306+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dc6ad0 0x560073dbe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 -- 172.21.15.5:0/3663780388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dc6ad0 msgr2=0x560073dbe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dc6ad0 0x560073dbe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 -- 172.21.15.5:0/3663780388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 msgr2=0x560073de0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 -- 172.21.15.5:0/3663780388 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560073b24c60 con 0x560073dbb2d0 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74e999a700 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd540 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0093f0 tx=0x7f74dc004ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e276c7c23e9d48b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74d93ff700 1 -- 172.21.15.5:0/3663780388 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc019040 con 0x560073dbb2d0 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74d93ff700 1 -- 172.21.15.5:0/3663780388 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74dc004710 con 0x560073dbb2d0 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.307+0000 7f74d93ff700 1 -- 172.21.15.5:0/3663780388 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc0156e0 con 0x560073dbb2d0 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3663780388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 msgr2=0x560073dcd540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd540 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f74dc0093f0 tx=0x7f74dc004ed0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3663780388 shutdown_connections 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dc6ad0 0x560073dbe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd540 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3663780388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dbbbe0 0x560073de0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3663780388 >> 172.21.15.5:0/3663780388 conn(0x560073db1da0 msgr2=0x560073dd1cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.308+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3663780388 shutdown_connections 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.309+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3663780388 wait complete. 2023-07-21T19:18:35.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.309+0000 7f74ffc4db80 1 -- start start 2023-07-21T19:18:35.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.309+0000 7f74ffc4db80 1 Processor -- start 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dcd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073dc9d20 con 0x560073dc6ad0 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073dc9e90 con 0x560073dbbbe0 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560073dca050 con 0x560073dbb2d0 2023-07-21T19:18:35.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ffc4db80 10 client.0 _ll_register_callbacks cb 0x7f74d4011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ea99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ea99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ea99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57018/0 (socket says 172.21.15.5:57018) 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74ea99c700 1 -- 172.21.15.5:0/3182015159 learned_addr learned my addr 172.21.15.5:0/3182015159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.310+0000 7f74e999a700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dcd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 -- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 msgr2=0x560073dcd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dcd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 -- 172.21.15.5:0/3182015159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 msgr2=0x560073dc9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 -- 172.21.15.5:0/3182015159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc003ae0 con 0x560073dbb2d0 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ea99c700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009ef0 tx=0x7f74dc009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bff1fee5d1663290 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74cdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74dc019040 con 0x560073dbb2d0 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f74dc007790 con 0x560073dbb2d0 2023-07-21T19:18:35.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.311+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f74dc00f040 con 0x560073dbb2d0 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74dc007950 con 0x560073dbb2d0 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 10 client.27403 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f74dc02c040 con 0x560073dbb2d0 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74dc00f040 con 0x560073dbb2d0 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f74dc015ae0 con 0x560073dbb2d0 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 10 client.27403 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 10 client.27403 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:35.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 20 client.27403 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073e339b0 0x560073e35dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 4 client.27403 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.312+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560073cc9050 con 0x560073e339b0 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74e999a700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073e339b0 0x560073e35dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74e999a700 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073e339b0 0x560073e35dc0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a02574b2f58dc48e server_cookie=5bf8a6f1d34f7ce4 in_seq=0 out_seq=0 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74cdffb700 10 client.27403 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74cdffb700 1 -- 172.21.15.5:0/3182015159 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560073cc9050 con 0x560073e339b0 2023-07-21T19:18:35.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74cdffb700 10 client.27403 handle_command_reply: tid=0 2023-07-21T19:18:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.313+0000 7f74ffc4db80 1 client.27403 shutdown 2023-07-21T19:18:35.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 msgr2=0x560073dcd160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f74dc009ef0 tx=0x7f74dc009ba0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 shutdown_connections 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560073e339b0 0x560073e35dc0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560073dc6ad0 0x560073dc9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560073dbb2d0 0x560073dcd160 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74cdffb700 0 client.27403 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:35.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 --2- 172.21.15.5:0/3182015159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560073dbbbe0 0x560073dcd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 >> 172.21.15.5:0/3182015159 conn(0x560073db1da0 msgr2=0x560073dbe990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 shutdown_connections 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 1 -- 172.21.15.5:0/3182015159 wait complete. 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.314+0000 7f74ffc4db80 20 client.27403 trim_cache size 0 max 16384 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:18:35.317 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: "222784ff-b70f-4d01-950c-d88fade4db3d": { 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "222784ff-b70f-4d01-950c-d88fade4db3d", 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:18:35.318 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:18:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 msgr2=0x7f74d00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 0x7f74d00a37f0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f74e0009350 tx=0x7f74e0097040 comp rx=0 tx=0).stop 2023-07-21T19:18:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 msgr2=0x7f74e419bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:35.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f74e0005740 tx=0x7f74e0005770 comp rx=0 tx=0).stop 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 shutdown_connections 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f74e410a070 0x7f74e4195b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f74e4100ca0 0x7f74e419bae0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74d00a1340 0x7f74d00a37f0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 --2- 172.21.15.5:0/2456361621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f74e41015b0 0x7f74e419c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 >> 172.21.15.5:0/2456361621 conn(0x7f74e40fa4b0 msgr2=0x7f74e410c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 shutdown_connections 2023-07-21T19:18:35.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:18:35.315+0000 7f74ec3ff700 1 -- 172.21.15.5:0/2456361621 wait complete. 2023-07-21T19:18:35.326 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:18:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:18:34.985+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...12,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v3642 ap=2 f(v100 m2023-07-21T19:18:02.668439+0000 3=3+0) n(v10 rc2023-07-21T19:18:02.668439+0000 b3321 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@255} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v3642>,<0x10000000005/data v2613>,<0x10000000003/postgres v685>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v709>,<0x10000000001/sv_0 v722>,<0x10000000000/_nogroup v701>,<0x1/volumes v666>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:18:02.654676+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:18:02.668439+0000 b3321 4=3+1)","memory_value.dirstat":"f(v100 m2023-07-21T19:18:02.668439+0000 3=3+0)","memory_value.rstat":"n(v10 rc2023-07-21T19:18:02.668439+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:18:35.333 INFO:tasks.cephfs.filesystem:scrub status for tag:222784ff-b70f-4d01-950c-d88fade4db3d - {'path': '/', 'tag': '222784ff-b70f-4d01-950c-d88fade4db3d', 'options': 'recursive,force'} 2023-07-21T19:18:35.415 INFO:teuthology.orchestra.run.smithi005.stderr:41600000 of 50000000 tuples (83%) done (elapsed 632.98 s, remaining 127.81 s) 2023-07-21T19:18:35.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: pgmap v342: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 23 MiB/s wr, 64 op/s 2023-07-21T19:18:35.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:18:35.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.655 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.656 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:35 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: pgmap v342: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 23 MiB/s wr, 64 op/s 2023-07-21T19:18:35.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:18:35.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.829 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:35 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: pgmap v342: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 23 MiB/s wr, 64 op/s 2023-07-21T19:18:35.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:18:35.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:35.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:35.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:35 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:18:36.879 INFO:teuthology.orchestra.run.smithi005.stderr:41700000 of 50000000 tuples (83%) done (elapsed 634.45 s, remaining 126.28 s) 2023-07-21T19:18:37.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:37 smithi101 ceph-mon[128136]: pgmap v343: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 22 MiB/s wr, 62 op/s 2023-07-21T19:18:37.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:37 smithi005 ceph-mon[126460]: pgmap v343: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 22 MiB/s wr, 62 op/s 2023-07-21T19:18:37.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:37 smithi156 ceph-mon[128316]: pgmap v343: 97 pgs: 97 active+clean; 8.5 GiB data, 3.7 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 22 MiB/s wr, 62 op/s 2023-07-21T19:18:38.210 INFO:teuthology.orchestra.run.smithi005.stderr:41800000 of 50000000 tuples (83%) done (elapsed 635.78 s, remaining 124.72 s) 2023-07-21T19:18:39.525 INFO:teuthology.orchestra.run.smithi005.stderr:41900000 of 50000000 tuples (83%) done (elapsed 637.10 s, remaining 123.16 s) 2023-07-21T19:18:39.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:39 smithi101 ceph-mon[128136]: pgmap v344: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 32 MiB/s wr, 189 op/s 2023-07-21T19:18:39.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:39 smithi005 ceph-mon[126460]: pgmap v344: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 32 MiB/s wr, 189 op/s 2023-07-21T19:18:39.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:39 smithi156 ceph-mon[128316]: pgmap v344: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 278 KiB/s rd, 32 MiB/s wr, 189 op/s 2023-07-21T19:18:40.940 INFO:teuthology.orchestra.run.smithi005.stderr:42000000 of 50000000 tuples (84%) done (elapsed 638.51 s, remaining 121.62 s) 2023-07-21T19:18:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 -- 172.21.15.5:0/3344529720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 msgr2=0x7f0a0c0679a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 --2- 172.21.15.5:0/3344529720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c0679a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f09fc009990 tx=0x7f09fc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 -- 172.21.15.5:0/3344529720 shutdown_connections 2023-07-21T19:18:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 --2- 172.21.15.5:0/3344529720 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 0x7f0a0c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 --2- 172.21.15.5:0/3344529720 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c0679a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 --2- 172.21.15.5:0/3344529720 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 0x7f0a0c068350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 -- 172.21.15.5:0/3344529720 >> 172.21.15.5:0/3344529720 conn(0x7f0a0c074760 msgr2=0x7f0a0c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.342+0000 7f0a129ff700 1 -- 172.21.15.5:0/3344529720 shutdown_connections 2023-07-21T19:18:41.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.343+0000 7f0a129ff700 1 -- 172.21.15.5:0/3344529720 wait complete. 2023-07-21T19:18:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 Processor -- start 2023-07-21T19:18:41.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 -- start start 2023-07-21T19:18:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 0x7f0a0c19c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 0x7f0a0c19c630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a12e0 con 0x7f0a0c1024f0 2023-07-21T19:18:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a1450 con 0x7f0a0c104a60 2023-07-21T19:18:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.344+0000 7f0a129ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a0c1a1610 con 0x7f0a0c067ee0 2023-07-21T19:18:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47186/0 (socket says 172.21.15.5:47186) 2023-07-21T19:18:41.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2920160558 learned_addr learned my addr 172.21.15.5:0/2920160558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 0x7f0a0c19c630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 0x7f0a0c19c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2920160558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 msgr2=0x7f0a0c19c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 0x7f0a0c19c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 msgr2=0x7f0a0c19c630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 0x7f0a0c19c630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.345+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2920160558 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a04009c40 con 0x7f0a0c104a60 2023-07-21T19:18:41.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400c370 tx=0x7f0a0400c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a097fa700 1 -- 172.21.15.5:0/2920160558 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04017070 con 0x7f0a0c104a60 2023-07-21T19:18:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a097fa700 1 -- 172.21.15.5:0/2920160558 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a0400f040 con 0x7f0a0c104a60 2023-07-21T19:18:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09fc0095f0 con 0x7f0a0c104a60 2023-07-21T19:18:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a097fa700 1 -- 172.21.15.5:0/2920160558 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0400e050 con 0x7f0a0c104a60 2023-07-21T19:18:41.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.346+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c1a8f50 con 0x7f0a0c104a60 2023-07-21T19:18:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.352+0000 7f0a097fa700 1 -- 172.21.15.5:0/2920160558 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0a0400cb60 con 0x7f0a0c104a60 2023-07-21T19:18:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 Processor -- start 2023-07-21T19:18:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 -- start start 2023-07-21T19:18:41.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e051ec76b0 0x55e052070d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05206b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f70c50 con 0x55e051ec76b0 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fc39e0 con 0x55e0520712d0 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.353+0000 7f0a26315b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fbce70 con 0x55e05206b580 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47188/0 (socket says 172.21.15.5:47188) 2023-07-21T19:18:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e051ec76b0 0x55e052070d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 -- 172.21.15.5:0/4017661658 learned_addr learned my addr 172.21.15.5:0/4017661658 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:41.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05206b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 -- 172.21.15.5:0/4017661658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 msgr2=0x55e05206b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05206b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 -- 172.21.15.5:0/4017661658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e051ec76b0 msgr2=0x55e052070d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e051ec76b0 0x55e052070d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.354+0000 7f0a0bfff700 1 -- 172.21.15.5:0/4017661658 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e051dead80 con 0x55e0520712d0 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f0a097fa700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 0x7f09f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f09fc015000 tx=0x7f09fc005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693d11c8abc5023 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f0a097fa700 1 -- 172.21.15.5:0/2920160558 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0a040cc5c0 con 0x7f0a0c104a60 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 0x7f09f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f09f27fc700 1 -- 172.21.15.5:0/4017661658 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc01d070 con 0x55e0520712d0 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f09f27fc700 1 -- 172.21.15.5:0/4017661658 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09fc01e040 con 0x55e0520712d0 2023-07-21T19:18:41.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.355+0000 7f09f27fc700 1 -- 172.21.15.5:0/4017661658 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004740 con 0x55e0520712d0 2023-07-21T19:18:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 -- 172.21.15.5:0/4017661658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 msgr2=0x55e05206b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f09fc015000 tx=0x7f09fc005f00 comp rx=0 tx=0).stop 2023-07-21T19:18:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 0x7f09f40a37f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009fd0 tx=0x7f0a000058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 -- 172.21.15.5:0/4017661658 shutdown_connections 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05206b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e051ec76b0 0x55e052070d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 --2- 172.21.15.5:0/4017661658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e05206b040 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.356+0000 7f0a26315b80 1 -- 172.21.15.5:0/4017661658 >> 172.21.15.5:0/4017661658 conn(0x55e052060be0 msgr2=0x55e052065f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.357+0000 7f0a26315b80 1 -- 172.21.15.5:0/4017661658 shutdown_connections 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.357+0000 7f0a26315b80 1 -- 172.21.15.5:0/4017661658 wait complete. 2023-07-21T19:18:41.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.357+0000 7f0a26315b80 1 -- start start 2023-07-21T19:18:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 Processor -- start 2023-07-21T19:18:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e0520774c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e0520747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e052074d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f5ca60 con 0x55e05206b580 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f5c760 con 0x55e0520712d0 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.358+0000 7f0a26315b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051e408c0 con 0x55e051ec76b0 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a26315b80 10 client.0 _ll_register_callbacks cb 0x55e051f5a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:41.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e0520774c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a0b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e0520747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e0520774c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58134/0 (socket says 172.21.15.5:58134) 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a0bfff700 1 -- 172.21.15.5:0/1066134439 learned_addr learned my addr 172.21.15.5:0/1066134439 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.359+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e052074d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 -- 172.21.15.5:0/1066134439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 msgr2=0x55e0520774c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e0520774c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 -- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 msgr2=0x55e052074d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e052074d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 -- 172.21.15.5:0/1066134439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0c113930 con 0x55e05206b580 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e0520747b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005950 tx=0x7f0a000047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe198a5c854ccfe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f09eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00024070 con 0x55e05206b580 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.360+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0a0001f770 con 0x55e05206b580 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e051f16490 con 0x55e05206b580 2023-07-21T19:18:41.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a0001f930 con 0x55e05206b580 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f0a26315b80 10 client.17814 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55e051dd3c00 con 0x55e05206b580 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e051f16490 con 0x55e05206b580 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.361+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a0002aaa0 con 0x55e05206b580 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 10 client.17814 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 10 client.17814 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 20 client.17814 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:41.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e052084b10 0x55e052086f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 4 client.17814 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e051fb5e50 con 0x55e052084b10 2023-07-21T19:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.362+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e052084b10 0x55e052086f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.363+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e052084b10 0x55e052086f20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a6b953a90776558 server_cookie=23dbebbb67337d22 in_seq=0 out_seq=0 2023-07-21T19:18:41.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.363+0000 7f09eb7fe700 10 client.17814 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f09eb7fe700 1 -- 172.21.15.5:0/1066134439 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e051fb5e50 con 0x55e052084b10 2023-07-21T19:18:41.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f09eb7fe700 10 client.17814 handle_command_reply: tid=0 2023-07-21T19:18:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 client.17814 shutdown 2023-07-21T19:18:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 msgr2=0x55e0520747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e0520747b0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f0a00005950 tx=0x7f0a000047c0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 shutdown_connections 2023-07-21T19:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e052074d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e0520774c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e052084b10 0x55e052086f20 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 --2- 172.21.15.5:0/1066134439 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e0520747b0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 >> 172.21.15.5:0/1066134439 conn(0x55e052060be0 msgr2=0x55e052064c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.364+0000 7f09eb7fe700 0 client.17814 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.365+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 shutdown_connections 2023-07-21T19:18:41.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.365+0000 7f0a26315b80 1 -- 172.21.15.5:0/1066134439 wait complete. 2023-07-21T19:18:41.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.365+0000 7f0a26315b80 20 client.17814 trim_cache size 0 max 16384 2023-07-21T19:18:41.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.389+0000 7f0a26315b80 1 Processor -- start 2023-07-21T19:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.389+0000 7f0a26315b80 1 -- start start 2023-07-21T19:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05207bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 0x55e052065c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051fb5e50 con 0x55e0520712d0 2023-07-21T19:18:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e051f16490 con 0x55e051ec76b0 2023-07-21T19:18:41.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a26315b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e052062d50 con 0x55e05206b580 2023-07-21T19:18:41.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a0b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05207bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a0bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47210/0 (socket says 172.21.15.5:47210) 2023-07-21T19:18:41.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a0bfff700 1 -- 172.21.15.5:0/3887229370 learned_addr learned my addr 172.21.15.5:0/3887229370 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.390+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 0x55e052065c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 -- 172.21.15.5:0/3887229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 msgr2=0x55e05207bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05207bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 -- 172.21.15.5:0/3887229370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 msgr2=0x55e05206b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 -- 172.21.15.5:0/3887229370 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e051e111f0 con 0x55e0520712d0 2023-07-21T19:18:41.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 0x55e052065c60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0a040c9d10 tx=0x7f0a0400e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbcdff19538ef server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f09f2ffd700 1 -- 172.21.15.5:0/3887229370 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a040092e0 con 0x55e0520712d0 2023-07-21T19:18:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f09f2ffd700 1 -- 172.21.15.5:0/3887229370 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0a04082e90 con 0x55e0520712d0 2023-07-21T19:18:41.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.391+0000 7f09f2ffd700 1 -- 172.21.15.5:0/3887229370 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a04097a30 con 0x55e0520712d0 2023-07-21T19:18:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 -- 172.21.15.5:0/3887229370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 msgr2=0x55e052065c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 0x55e052065c60 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f0a040c9d10 tx=0x7f0a0400e1d0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 -- 172.21.15.5:0/3887229370 shutdown_connections 2023-07-21T19:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e0520712d0 0x55e052065c60 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e051ec76b0 0x55e05206b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 --2- 172.21.15.5:0/3887229370 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e05206b580 0x55e05207bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.392+0000 7f0a26315b80 1 -- 172.21.15.5:0/3887229370 >> 172.21.15.5:0/3887229370 conn(0x55e052060be0 msgr2=0x55e05206ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.393+0000 7f0a26315b80 1 -- 172.21.15.5:0/3887229370 shutdown_connections 2023-07-21T19:18:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.393+0000 7f0a26315b80 1 -- 172.21.15.5:0/3887229370 wait complete. 2023-07-21T19:18:41.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.393+0000 7f0a26315b80 1 -- start start 2023-07-21T19:18:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 Processor -- start 2023-07-21T19:18:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e052079e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e05207a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520d7f10 con 0x55e05206b580 2023-07-21T19:18:41.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520d8050 con 0x55e0520712d0 2023-07-21T19:18:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e0520d8190 con 0x55e051ec76b0 2023-07-21T19:18:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.394+0000 7f0a26315b80 10 client.0 _ll_register_callbacks cb 0x7f0a0000fa30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47216/0 (socket says 172.21.15.5:47216) 2023-07-21T19:18:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2643050294 learned_addr learned my addr 172.21.15.5:0/2643050294 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a0b7fe700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e05207a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e052079e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2643050294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 msgr2=0x55e052079e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e052079e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2643050294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 msgr2=0x55e05207a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:41.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e05207a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 -- 172.21.15.5:0/2643050294 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e051dead80 con 0x55e0520712d0 2023-07-21T19:18:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f0a10f9c700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a040ca290 tx=0x7f0a04095a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f385a52c482b504b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.395+0000 7f09eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:18:41.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0409d6c0 con 0x55e0520712d0 2023-07-21T19:18:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0a0400ce10 con 0x55e0520712d0 2023-07-21T19:18:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f09fc01e040 con 0x55e0520712d0 2023-07-21T19:18:41.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0a04099690 con 0x55e0520712d0 2023-07-21T19:18:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f0a26315b80 10 client.27491 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0a0c1a8f50 con 0x55e0520712d0 2023-07-21T19:18:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.396+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f09fc01e040 con 0x55e0520712d0 2023-07-21T19:18:41.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0a04004bd0 con 0x55e0520712d0 2023-07-21T19:18:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 10 client.27491 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 10 client.27491 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 20 client.27491 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:41.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0520d9560 0x55e0520db940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 4 client.27491 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:18:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e051f77ff0 con 0x55e0520d9560 2023-07-21T19:18:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.397+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0520d9560 0x55e0520db940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:41.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.398+0000 7f09eb7fe700 10 client.27491 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:41.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.398+0000 7f0a0bfff700 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0520d9560 0x55e0520db940 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54b9da40986975a6 server_cookie=c64987f9f1d99826 in_seq=0 out_seq=0 2023-07-21T19:18:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.414+0000 7f09eb7fe700 1 -- 172.21.15.5:0/2643050294 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e051f77ff0 con 0x55e0520d9560 2023-07-21T19:18:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.414+0000 7f09eb7fe700 10 client.27491 handle_command_reply: tid=0 2023-07-21T19:18:41.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.414+0000 7f0a26315b80 1 client.27491 shutdown 2023-07-21T19:18:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.414+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 msgr2=0x55e0520d79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.414+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0a040ca290 tx=0x7f0a04095a80 comp rx=0 tx=0).stop 2023-07-21T19:18:41.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 shutdown_connections 2023-07-21T19:18:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e0520712d0 0x55e0520d79d0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e051ec76b0 0x55e052079e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e0520d9560 0x55e0520db940 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 --2- 172.21.15.5:0/2643050294 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e05206b580 0x55e05207a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f09eb7fe700 0 client.27491 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:41.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 >> 172.21.15.5:0/2643050294 conn(0x55e052060be0 msgr2=0x55e0520720a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 shutdown_connections 2023-07-21T19:18:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 1 -- 172.21.15.5:0/2643050294 wait complete. 2023-07-21T19:18:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.415+0000 7f0a26315b80 20 client.27491 trim_cache size 0 max 16384 2023-07-21T19:18:41.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:18:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:18:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:18:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:18:41.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 msgr2=0x7f09f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 0x7f09f40a37f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0a00009fd0 tx=0x7f0a000058e0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 msgr2=0x7f0a0c1a0ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0a0400c370 tx=0x7f0a0400c680 comp rx=0 tx=0).stop 2023-07-21T19:18:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 shutdown_connections 2023-07-21T19:18:41.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0a0c104a60 0x7f0a0c1a0ca0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0a0c067ee0 0x7f0a0c19c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09f40a1340 0x7f09f40a37f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 --2- 172.21.15.5:0/2920160558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0a0c1024f0 0x7f0a0c19c630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 >> 172.21.15.5:0/2920160558 conn(0x7f0a0c074760 msgr2=0x7f0a0c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 shutdown_connections 2023-07-21T19:18:41.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:41.416+0000 7f0a129ff700 1 -- 172.21.15.5:0/2920160558 wait complete. 2023-07-21T19:18:41.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:41 smithi101 ceph-mon[128136]: pgmap v345: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 21 MiB/s wr, 172 op/s 2023-07-21T19:18:41.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:41 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:41.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:41 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:41.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:41 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:18:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:41 smithi005 ceph-mon[126460]: pgmap v345: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 21 MiB/s wr, 172 op/s 2023-07-21T19:18:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:41 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:41 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:41.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:41 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:18:41.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:41 smithi156 ceph-mon[128316]: pgmap v345: 97 pgs: 97 active+clean; 8.6 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 252 KiB/s rd, 21 MiB/s wr, 172 op/s 2023-07-21T19:18:41.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:41 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:41.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:41 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:41.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:41 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:18:42.284 INFO:teuthology.orchestra.run.smithi005.stderr:42100000 of 50000000 tuples (84%) done (elapsed 639.85 s, remaining 120.07 s) 2023-07-21T19:18:43.612 INFO:teuthology.orchestra.run.smithi005.stderr:42200000 of 50000000 tuples (84%) done (elapsed 641.18 s, remaining 118.51 s) 2023-07-21T19:18:43.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:43 smithi101 ceph-mon[128136]: pgmap v346: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 31 MiB/s wr, 193 op/s 2023-07-21T19:18:43.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:43 smithi005 ceph-mon[126460]: pgmap v346: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 31 MiB/s wr, 193 op/s 2023-07-21T19:18:43.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:43 smithi156 ceph-mon[128316]: pgmap v346: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 254 KiB/s rd, 31 MiB/s wr, 193 op/s 2023-07-21T19:18:45.075 INFO:teuthology.orchestra.run.smithi005.stderr:42300000 of 50000000 tuples (84%) done (elapsed 642.65 s, remaining 116.98 s) 2023-07-21T19:18:45.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:45 smithi101 ceph-mon[128136]: pgmap v347: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 23 MiB/s wr, 165 op/s 2023-07-21T19:18:45.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:45 smithi005 ceph-mon[126460]: pgmap v347: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 23 MiB/s wr, 165 op/s 2023-07-21T19:18:45.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:45 smithi156 ceph-mon[128316]: pgmap v347: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 234 KiB/s rd, 23 MiB/s wr, 165 op/s 2023-07-21T19:18:46.393 INFO:teuthology.orchestra.run.smithi005.stderr:42400000 of 50000000 tuples (84%) done (elapsed 643.96 s, remaining 115.43 s) 2023-07-21T19:18:47.734 INFO:teuthology.orchestra.run.smithi005.stderr:42500000 of 50000000 tuples (85%) done (elapsed 645.30 s, remaining 113.88 s) 2023-07-21T19:18:47.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:47 smithi101 ceph-mon[128136]: pgmap v348: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:18:47.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:47 smithi005 ceph-mon[126460]: pgmap v348: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:18:47.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:47 smithi156 ceph-mon[128316]: pgmap v348: 97 pgs: 97 active+clean; 8.7 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 178 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:18:49.032 INFO:teuthology.orchestra.run.smithi005.stderr:42600000 of 50000000 tuples (85%) done (elapsed 646.60 s, remaining 112.32 s) 2023-07-21T19:18:49.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:49 smithi101 ceph-mon[128136]: pgmap v349: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 33 MiB/s wr, 188 op/s 2023-07-21T19:18:49.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:49 smithi005 ceph-mon[126460]: pgmap v349: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 33 MiB/s wr, 188 op/s 2023-07-21T19:18:49.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:49 smithi156 ceph-mon[128316]: pgmap v349: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 213 KiB/s rd, 33 MiB/s wr, 188 op/s 2023-07-21T19:18:50.455 INFO:teuthology.orchestra.run.smithi005.stderr:42700000 of 50000000 tuples (85%) done (elapsed 648.03 s, remaining 110.79 s) 2023-07-21T19:18:51.757 INFO:teuthology.orchestra.run.smithi005.stderr:42800000 of 50000000 tuples (85%) done (elapsed 649.33 s, remaining 109.23 s) 2023-07-21T19:18:51.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:51 smithi101 ceph-mon[128136]: pgmap v350: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 61 op/s 2023-07-21T19:18:51.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:51 smithi005 ceph-mon[126460]: pgmap v350: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 61 op/s 2023-07-21T19:18:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:51 smithi156 ceph-mon[128316]: pgmap v350: 97 pgs: 97 active+clean; 8.8 GiB data, 3.8 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 22 MiB/s wr, 61 op/s 2023-07-21T19:18:53.054 INFO:teuthology.orchestra.run.smithi005.stderr:42900000 of 50000000 tuples (85%) done (elapsed 650.62 s, remaining 107.68 s) 2023-07-21T19:18:53.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:53 smithi101 ceph-mon[128136]: pgmap v351: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 77 op/s 2023-07-21T19:18:53.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:53 smithi005 ceph-mon[126460]: pgmap v351: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 77 op/s 2023-07-21T19:18:53.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:53 smithi156 ceph-mon[128316]: pgmap v351: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 32 MiB/s wr, 77 op/s 2023-07-21T19:18:54.449 INFO:teuthology.orchestra.run.smithi005.stderr:43000000 of 50000000 tuples (86%) done (elapsed 652.02 s, remaining 106.14 s) 2023-07-21T19:18:54.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 -- 172.21.15.5:0/4193281713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0685b0 msgr2=0x7f71ec104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 --2- 172.21.15.5:0/4193281713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0685b0 0x7f71ec104770 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f71e00099c0 tx=0x7f71e0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 -- 172.21.15.5:0/4193281713 shutdown_connections 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 --2- 172.21.15.5:0/4193281713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec104e50 0x7f71ec10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 --2- 172.21.15.5:0/4193281713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec067c10 0x7f71ec067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 --2- 172.21.15.5:0/4193281713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec0685b0 0x7f71ec104770 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 -- 172.21.15.5:0/4193281713 >> 172.21.15.5:0/4193281713 conn(0x7f71ec074760 msgr2=0x7f71ec074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 -- 172.21.15.5:0/4193281713 shutdown_connections 2023-07-21T19:18:54.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.594+0000 7f71f11ff700 1 -- 172.21.15.5:0/4193281713 wait complete. 2023-07-21T19:18:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.595+0000 7f71f11ff700 1 Processor -- start 2023-07-21T19:18:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 -- start start 2023-07-21T19:18:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec067c10 0x7f71ec19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0685b0 0x7f71ec19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec196020 con 0x7f71ec104e50 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec196190 con 0x7f71ec067c10 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71f11ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71ec196350 con 0x7f71ec0685b0 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.596+0000 7f71eb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49636/0 (socket says 172.21.15.5:49636) 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 -- 172.21.15.5:0/2006759207 learned_addr learned my addr 172.21.15.5:0/2006759207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:54.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71ead9d700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec067c10 0x7f71ec19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71ea59c700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0685b0 0x7f71ec19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 -- 172.21.15.5:0/2006759207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0685b0 msgr2=0x7f71ec19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0685b0 0x7f71ec19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 -- 172.21.15.5:0/2006759207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec067c10 msgr2=0x7f71ec19bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec067c10 0x7f71ec19bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 -- 172.21.15.5:0/2006759207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e00095f0 con 0x7f71ec104e50 2023-07-21T19:18:54.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.597+0000 7f71eb59e700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ef30 tx=0x7f71e400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.598+0000 7f71d3fff700 1 -- 172.21.15.5:0/2006759207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e4018070 con 0x7f71ec104e50 2023-07-21T19:18:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.598+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71ec1965d0 con 0x7f71ec104e50 2023-07-21T19:18:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.598+0000 7f71d3fff700 1 -- 172.21.15.5:0/2006759207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e400f040 con 0x7f71ec104e50 2023-07-21T19:18:54.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.598+0000 7f71d3fff700 1 -- 172.21.15.5:0/2006759207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400cb80 con 0x7f71ec104e50 2023-07-21T19:18:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.598+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec1a2b40 con 0x7f71ec104e50 2023-07-21T19:18:54.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.602+0000 7f71d3fff700 1 -- 172.21.15.5:0/2006759207 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f71e400cd60 con 0x7f71ec104e50 2023-07-21T19:18:54.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 Processor -- start 2023-07-21T19:18:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 -- start start 2023-07-21T19:18:54.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3680 0x558cfe8bd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3ad0 0x558cfe8bd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe7bec50 con 0x558cfe7156b0 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe8119e0 con 0x558cfe8c3680 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.603+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe80ae70 con 0x558cfe8c3ad0 2023-07-21T19:18:54.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49642/0 (socket says 172.21.15.5:49642) 2023-07-21T19:18:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 -- 172.21.15.5:0/3544512782 learned_addr learned my addr 172.21.15.5:0/3544512782 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ead9d700 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3680 0x558cfe8bd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71d3fff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 0x7f71d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 -- 172.21.15.5:0/3544512782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3ad0 msgr2=0x558cfe8bd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3ad0 0x558cfe8bd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 -- 172.21.15.5:0/3544512782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3680 msgr2=0x558cfe8bd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3680 0x558cfe8bd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 -- 172.21.15.5:0/3544512782 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cfe638d80 con 0x558cfe7156b0 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71d3fff700 1 -- 172.21.15.5:0/2006759207 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f71e40cc710 con 0x7f71ec104e50 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71eb59e700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 0x7f71d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.604+0000 7f71ea59c700 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f71e0006010 tx=0x7f71e0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8bbc9383e4382a3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.605+0000 7f71d17fa700 1 -- 172.21.15.5:0/3544512782 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001d070 con 0x558cfe7156b0 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.605+0000 7f71d17fa700 1 -- 172.21.15.5:0/3544512782 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e000be10 con 0x558cfe7156b0 2023-07-21T19:18:54.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.605+0000 7f71d17fa700 1 -- 172.21.15.5:0/3544512782 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0003780 con 0x558cfe7156b0 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 -- 172.21.15.5:0/3544512782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 msgr2=0x558cfe8c3140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f71eb59e700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 0x7f71d40a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71e40c9ec0 tx=0x7f71e4009ed0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7f71e0006010 tx=0x7f71e0005c80 comp rx=0 tx=0).stop 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 -- 172.21.15.5:0/3544512782 shutdown_connections 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3ad0 0x558cfe8bd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c3140 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/3544512782 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3680 0x558cfe8bd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 -- 172.21.15.5:0/3544512782 >> 172.21.15.5:0/3544512782 conn(0x558cfe8aebe0 msgr2=0x558cfe8b3f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 -- 172.21.15.5:0/3544512782 shutdown_connections 2023-07-21T19:18:54.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.606+0000 7f7204b4ab80 1 -- 172.21.15.5:0/3544512782 wait complete. 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.607+0000 7f7204b4ab80 1 -- start start 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.607+0000 7f7204b4ab80 1 Processor -- start 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8ccfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8c6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe7aaa60 con 0x558cfe7156b0 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe7aa760 con 0x558cfe8c3ad0 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe68e8c0 con 0x558cfe8c3680 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f7204b4ab80 10 client.0 _ll_register_callbacks cb 0x558cfe7a8700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49658/0 (socket says 172.21.15.5:49658) 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ea59c700 1 -- 172.21.15.5:0/716064983 learned_addr learned my addr 172.21.15.5:0/716064983 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:54.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ead9d700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8ccfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ea59c700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8c6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.608+0000 7f71ea59c700 1 -- 172.21.15.5:0/716064983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 msgr2=0x558cfe8ccfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ea59c700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8ccfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ea59c700 1 -- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 msgr2=0x558cfe8c6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ea59c700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8c6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ea59c700 1 -- 172.21.15.5:0/716064983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71ec10d210 con 0x558cfe7156b0 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ead9d700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:18:54.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ea59c700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009990 tx=0x7f71e0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cf9464cef0a25d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001d030 con 0x558cfe7156b0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f71e001e070 con 0x558cfe7156b0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558cfe764490 con 0x558cfe7156b0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0029070 con 0x558cfe7156b0 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f7204b4ab80 10 client.17832 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:54.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.609+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558cfe621c00 con 0x558cfe7156b0 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558cfe764490 con 0x558cfe7156b0 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71e000f750 con 0x558cfe7156b0 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 10 client.17832 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 10 client.17832 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 20 client.17832 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe8d6a10 0x558cfe8d8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 4 client.17832 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:18:54.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.610+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558cfe803e50 con 0x558cfe8d6a10 2023-07-21T19:18:54.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.611+0000 7f71ead9d700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe8d6a10 0x558cfe8d8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.611+0000 7f71ceff5700 10 client.17832 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.611+0000 7f71ead9d700 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe8d6a10 0x558cfe8d8df0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91cb4f1b2e6ef06 server_cookie=68d04a076218f06d in_seq=0 out_seq=0 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.612+0000 7f71ceff5700 1 -- 172.21.15.5:0/716064983 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558cfe803e50 con 0x558cfe8d6a10 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.612+0000 7f71ceff5700 10 client.17832 handle_command_reply: tid=0 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.612+0000 7f7204b4ab80 1 client.17832 shutdown 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.612+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 msgr2=0x558cfe8cca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.612+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f71e0009990 tx=0x7f71e0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 shutdown_connections 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8c6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe8d6a10 0x558cfe8d8df0 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8cca90 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/716064983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8ccfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 >> 172.21.15.5:0/716064983 conn(0x558cfe8aebe0 msgr2=0x558cfe8b34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f71ceff5700 0 client.17832 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 shutdown_connections 2023-07-21T19:18:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 1 -- 172.21.15.5:0/716064983 wait complete. 2023-07-21T19:18:54.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.613+0000 7f7204b4ab80 20 client.17832 trim_cache size 0 max 16384 2023-07-21T19:18:54.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 Processor -- start 2023-07-21T19:18:54.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 -- start start 2023-07-21T19:18:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8c2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8b3c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe764490 con 0x558cfe7156b0 2023-07-21T19:18:54.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe8abc00 con 0x558cfe8c3ad0 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.630+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe8bdf80 con 0x558cfe8c3680 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49670/0 (socket says 172.21.15.5:49670) 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 -- 172.21.15.5:0/841487381 learned_addr learned my addr 172.21.15.5:0/841487381 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ead9d700 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8c2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8b3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 -- 172.21.15.5:0/841487381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 msgr2=0x558cfe8c2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8c2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 -- 172.21.15.5:0/841487381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 msgr2=0x558cfe8b3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8b3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 -- 172.21.15.5:0/841487381 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558cfe65f1f0 con 0x558cfe7156b0 2023-07-21T19:18:54.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.631+0000 7f71ea59c700 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f71e000fb10 tx=0x7f71e0005430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ee16b3f070c4940 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f71d1ffb700 1 -- 172.21.15.5:0/841487381 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0029070 con 0x558cfe7156b0 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f71d1ffb700 1 -- 172.21.15.5:0/841487381 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e002a040 con 0x558cfe7156b0 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f71d1ffb700 1 -- 172.21.15.5:0/841487381 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001d040 con 0x558cfe7156b0 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 -- 172.21.15.5:0/841487381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 msgr2=0x558cfe8c6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f71e000fb10 tx=0x7f71e0005430 comp rx=0 tx=0).stop 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 -- 172.21.15.5:0/841487381 shutdown_connections 2023-07-21T19:18:54.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8b3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8c6e30 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/841487381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8c2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.632+0000 7f7204b4ab80 1 -- 172.21.15.5:0/841487381 >> 172.21.15.5:0/841487381 conn(0x558cfe8aebe0 msgr2=0x558cfe8c1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.633+0000 7f7204b4ab80 1 -- 172.21.15.5:0/841487381 shutdown_connections 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.633+0000 7f7204b4ab80 1 -- 172.21.15.5:0/841487381 wait complete. 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.633+0000 7f7204b4ab80 1 -- start start 2023-07-21T19:18:54.632 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.633+0000 7f7204b4ab80 1 Processor -- start 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8cab40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8cb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe929800 con 0x558cfe7156b0 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe929940 con 0x558cfe8c3ad0 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558cfe929b00 con 0x558cfe8c3680 2023-07-21T19:18:54.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f7204b4ab80 10 client.0 _ll_register_callbacks cb 0x7f71e0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f71ea59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49672/0 (socket says 172.21.15.5:49672) 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f71ea59c700 1 -- 172.21.15.5:0/4209098470 learned_addr learned my addr 172.21.15.5:0/4209098470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f71ead9d700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8cab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.634+0000 7f71ea59c700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8cb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 -- 172.21.15.5:0/4209098470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 msgr2=0x558cfe8cab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8cab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 -- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 msgr2=0x558cfe8cb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:18:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8cb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 -- 172.21.15.5:0/4209098470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558cfe621c00 con 0x558cfe7156b0 2023-07-21T19:18:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ea59c700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005a20 tx=0x7f71e0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acc61a7050cdf46f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:18:54.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ceff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:18:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e001d070 con 0x558cfe7156b0 2023-07-21T19:18:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f71e000be10 con 0x558cfe7156b0 2023-07-21T19:18:54.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f71e0000e40 con 0x558cfe7156b0 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f71e0029040 con 0x558cfe7156b0 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.635+0000 7f7204b4ab80 10 client.17838 fetch_fsmap learned FSMap version 42 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x558cfe637250 con 0x558cfe7156b0 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f71e004d080 con 0x558cfe7156b0 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f71e00306e0 con 0x558cfe7156b0 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 10 client.17838 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:18:54.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 10 client.17838 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 20 client.17838 populate_metadata read hostname 'smithi005' 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe92acd0 0x558cfe92d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 4 client.17838 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558cfe7c5ff0 con 0x558cfe92acd0 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.636+0000 7f71ead9d700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe92acd0 0x558cfe92d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.637+0000 7f71ead9d700 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe92acd0 0x558cfe92d0b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=defd80ca2e324f42 server_cookie=c6a6194eeafa1a17 in_seq=0 out_seq=0 2023-07-21T19:18:54.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.637+0000 7f71ceff5700 10 client.17838 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.656+0000 7f71ceff5700 1 -- 172.21.15.5:0/4209098470 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558cfe7c5ff0 con 0x558cfe92acd0 2023-07-21T19:18:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.656+0000 7f71ceff5700 10 client.17838 handle_command_reply: tid=0 2023-07-21T19:18:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.656+0000 7f7204b4ab80 1 client.17838 shutdown 2023-07-21T19:18:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.656+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 msgr2=0x558cfe8ca600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.656+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f71e0005a20 tx=0x7f71e0005ef0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 shutdown_connections 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558cfe8c3ad0 0x558cfe8cb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558cfe92acd0 0x558cfe92d0b0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558cfe7156b0 0x558cfe8ca600 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 --2- 172.21.15.5:0/4209098470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558cfe8c3680 0x558cfe8cab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f71ceff5700 0 client.17838 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:18:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 >> 172.21.15.5:0/4209098470 conn(0x558cfe8aebe0 msgr2=0x558cfe8b88c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 shutdown_connections 2023-07-21T19:18:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 1 -- 172.21.15.5:0/4209098470 wait complete. 2023-07-21T19:18:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.657+0000 7f7204b4ab80 20 client.17838 trim_cache size 0 max 16384 2023-07-21T19:18:54.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:18:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:18:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:18:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:18:54.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 msgr2=0x7f71d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 0x7f71d40a3770 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f71e40c9ec0 tx=0x7f71e4009ed0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 msgr2=0x7f71ec1959b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f71e400ef30 tx=0x7f71e400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 shutdown_connections 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f71d40a12c0 0x7f71d40a3770 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f71ec104e50 0x7f71ec1959b0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f71ec067c10 0x7f71ec19bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 --2- 172.21.15.5:0/2006759207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f71ec0685b0 0x7f71ec19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:18:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 >> 172.21.15.5:0/2006759207 conn(0x7f71ec074760 msgr2=0x7f71ec103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:18:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 shutdown_connections 2023-07-21T19:18:54.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:18:54.658+0000 7f71f11ff700 1 -- 172.21.15.5:0/2006759207 wait complete. 2023-07-21T19:18:55.753 INFO:teuthology.orchestra.run.smithi005.stderr:43100000 of 50000000 tuples (86%) done (elapsed 653.32 s, remaining 104.59 s) 2023-07-21T19:18:55.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:55 smithi101 ceph-mon[128136]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 MiB/s wr, 60 op/s 2023-07-21T19:18:55.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:55 smithi005 ceph-mon[126460]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 MiB/s wr, 60 op/s 2023-07-21T19:18:55.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:55 smithi156 ceph-mon[128316]: pgmap v352: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 24 MiB/s wr, 60 op/s 2023-07-21T19:18:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:43200000 of 50000000 tuples (86%) done (elapsed 654.66 s, remaining 103.05 s) 2023-07-21T19:18:57.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:57 smithi101 ceph-mon[128136]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:18:57.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:57 smithi005 ceph-mon[126460]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:18:57.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:57 smithi156 ceph-mon[128316]: pgmap v353: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 22 MiB/s wr, 56 op/s 2023-07-21T19:18:58.391 INFO:teuthology.orchestra.run.smithi005.stderr:43300000 of 50000000 tuples (86%) done (elapsed 655.96 s, remaining 101.50 s) 2023-07-21T19:18:59.794 INFO:teuthology.orchestra.run.smithi005.stderr:43400000 of 50000000 tuples (86%) done (elapsed 657.36 s, remaining 99.97 s) 2023-07-21T19:18:59.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:18:59 smithi101 ceph-mon[128136]: pgmap v354: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 75 op/s 2023-07-21T19:18:59.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:18:59 smithi005 ceph-mon[126460]: pgmap v354: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 75 op/s 2023-07-21T19:18:59.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:18:59 smithi156 ceph-mon[128316]: pgmap v354: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 75 op/s 2023-07-21T19:19:01.129 INFO:teuthology.orchestra.run.smithi005.stderr:43500000 of 50000000 tuples (87%) done (elapsed 658.70 s, remaining 98.43 s) 2023-07-21T19:19:01.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:01 smithi101 ceph-mon[128136]: pgmap v355: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 40 op/s 2023-07-21T19:19:01.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:01 smithi005 ceph-mon[126460]: pgmap v355: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 40 op/s 2023-07-21T19:19:01.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:01 smithi156 ceph-mon[128316]: pgmap v355: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 22 MiB/s wr, 40 op/s 2023-07-21T19:19:02.433 INFO:teuthology.orchestra.run.smithi005.stderr:43600000 of 50000000 tuples (87%) done (elapsed 660.00 s, remaining 96.88 s) 2023-07-21T19:19:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:43700000 of 50000000 tuples (87%) done (elapsed 661.39 s, remaining 95.35 s) 2023-07-21T19:19:03.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:03 smithi101 ceph-mon[128136]: pgmap v356: 97 pgs: 97 active+clean; 9.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 71 op/s 2023-07-21T19:19:03.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:03 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:19:03.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:03 smithi005 ceph-mon[126460]: pgmap v356: 97 pgs: 97 active+clean; 9.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 71 op/s 2023-07-21T19:19:03.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:03 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:19:03.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:03 smithi156 ceph-mon[128316]: pgmap v356: 97 pgs: 97 active+clean; 9.0 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 31 MiB/s wr, 71 op/s 2023-07-21T19:19:03.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:03 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:19:05.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:04 smithi101 ceph-mon[128136]: pgmap v357: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:05.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:19:05.351 INFO:teuthology.orchestra.run.smithi005.stderr:43800000 of 50000000 tuples (87%) done (elapsed 662.92 s, remaining 93.84 s) 2023-07-21T19:19:05.403 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.404 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.405 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:04 smithi005 ceph-mon[126460]: pgmap v357: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:05.415 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.415 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:05.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:04 smithi156 ceph-mon[128316]: pgmap v357: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 -- 172.21.15.5:0/2590076191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06600fe820 msgr2=0x7f06600febf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 --2- 172.21.15.5:0/2590076191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06600fe820 0x7f06600febf0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f0650009990 tx=0x7f0650009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 -- 172.21.15.5:0/2590076191 shutdown_connections 2023-07-21T19:19:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 --2- 172.21.15.5:0/2590076191 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f066010c3d0 0x7f066010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 --2- 172.21.15.5:0/2590076191 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06600fe820 0x7f06600febf0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 --2- 172.21.15.5:0/2590076191 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 -- 172.21.15.5:0/2590076191 >> 172.21.15.5:0/2590076191 conn(0x7f06600fa4b0 msgr2=0x7f06600fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 -- 172.21.15.5:0/2590076191 shutdown_connections 2023-07-21T19:19:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.575+0000 7f0665dff700 1 -- 172.21.15.5:0/2590076191 wait complete. 2023-07-21T19:19:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.576+0000 7f0665dff700 1 Processor -- start 2023-07-21T19:19:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.576+0000 7f0665dff700 1 -- start start 2023-07-21T19:19:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.576+0000 7f0665dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.576+0000 7f0665dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066019e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f0665dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 0x7f0660197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f0665dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06601981b0 con 0x7f066010c3d0 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f0665dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0660198320 con 0x7f06600ff1c0 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f0665dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06601984e0 con 0x7f06600fe820 2023-07-21T19:19:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066019e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 0x7f0660197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36426/0 (socket says 172.21.15.5:36426) 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066019e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60786/0 (socket says 172.21.15.5:60786) 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 -- 172.21.15.5:0/495508697 learned_addr learned my addr 172.21.15.5:0/495508697 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 -- 172.21.15.5:0/495508697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 msgr2=0x7f066019e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066019e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 -- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 msgr2=0x7f0660197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 0x7f0660197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065f7fe700 1 -- 172.21.15.5:0/495508697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06500095f0 con 0x7f06600fe820 2023-07-21T19:19:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.577+0000 7f065ffff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 0x7f0660197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f065f7fe700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0650006010 tx=0x7f0650005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f065cff9700 1 -- 172.21.15.5:0/495508697 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f065001d070 con 0x7f06600fe820 2023-07-21T19:19:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0660198760 con 0x7f06600fe820 2023-07-21T19:19:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f065cff9700 1 -- 172.21.15.5:0/495508697 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0650003f80 con 0x7f06600fe820 2023-07-21T19:19:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f065cff9700 1 -- 172.21.15.5:0/495508697 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0650022b90 con 0x7f06600fe820 2023-07-21T19:19:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.578+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06601a4ac0 con 0x7f06600fe820 2023-07-21T19:19:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.582+0000 7f065cff9700 1 -- 172.21.15.5:0/495508697 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0650003af0 con 0x7f06600fe820 2023-07-21T19:19:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 Processor -- start 2023-07-21T19:19:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 -- start start 2023-07-21T19:19:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df77c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df677cb0 con 0x5654df7712d0 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df6caa40 con 0x5654df771be0 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.583+0000 7f0679656b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df6c3ed0 con 0x5654df77cad0 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df77c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33252/0 (socket says 172.21.15.5:33252) 2023-07-21T19:19:05.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 -- 172.21.15.5:0/2999171621 learned_addr learned my addr 172.21.15.5:0/2999171621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065effd700 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 -- 172.21.15.5:0/2999171621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 msgr2=0x5654df77eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 -- 172.21.15.5:0/2999171621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 msgr2=0x5654df77c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df77c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065ffff700 1 -- 172.21.15.5:0/2999171621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654df4f1de0 con 0x5654df7712d0 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065effd700 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.584+0000 7f065cff9700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 0x7f06480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f065ffff700 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0658007680 tx=0x7f0658007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=159c81b5d59df86e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f0645ffb700 1 -- 172.21.15.5:0/2999171621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658003e10 con 0x5654df7712d0 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f065cff9700 1 -- 172.21.15.5:0/495508697 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f06500ce2a0 con 0x7f06600fe820 2023-07-21T19:19:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f0645ffb700 1 -- 172.21.15.5:0/2999171621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f065800f040 con 0x5654df7712d0 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f065f7fe700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 0x7f06480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.585+0000 7f0645ffb700 1 -- 172.21.15.5:0/2999171621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658015840 con 0x5654df7712d0 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- 172.21.15.5:0/2999171621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 msgr2=0x5654df7716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f065f7fe700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 0x7f06480a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f065009b330 tx=0x7f0650097040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f0658007680 tx=0x7f0658007990 comp rx=0 tx=0).stop 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- 172.21.15.5:0/2999171621 shutdown_connections 2023-07-21T19:19:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df7716a0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 --2- 172.21.15.5:0/2999171621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df77c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- 172.21.15.5:0/2999171621 >> 172.21.15.5:0/2999171621 conn(0x5654df767da0 msgr2=0x5654df76d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- 172.21.15.5:0/2999171621 shutdown_connections 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- 172.21.15.5:0/2999171621 wait complete. 2023-07-21T19:19:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.586+0000 7f0679656b80 1 -- start start 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 Processor -- start 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df779d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df663ac0 con 0x5654df7712d0 2023-07-21T19:19:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df6637c0 con 0x5654df771be0 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df547920 con 0x5654df77cad0 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.587+0000 7f0679656b80 10 client.0 _ll_register_callbacks cb 0x5654df661760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33258/0 (socket says 172.21.15.5:33258) 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 -- 172.21.15.5:0/3682221126 learned_addr learned my addr 172.21.15.5:0/3682221126 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065effd700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df779d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 -- 172.21.15.5:0/3682221126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 msgr2=0x5654df77a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 -- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 msgr2=0x5654df779d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df779d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 -- 172.21.15.5:0/3682221126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065400b8e0 con 0x5654df7712d0 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065effd700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df779d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:05.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f065ffff700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0658004a20 tx=0x7f0658005aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c47064e44e936a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.588+0000 7f063effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658004850 con 0x5654df7712d0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0658015890 con 0x5654df7712d0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5654df61d4f0 con 0x5654df7712d0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0658015a50 con 0x5654df7712d0 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f0679656b80 10 client.17850 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:05.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f0658009d50 con 0x5654df7712d0 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0658032020 con 0x5654df7712d0 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.589+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f065801ec90 con 0x5654df7712d0 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 10 client.17850 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 10 client.17850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 20 client.17850 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df78c420 0x5654df78e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 4 client.17850 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5654df6bceb0 con 0x5654df78c420 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.590+0000 7f065effd700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df78c420 0x5654df78e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.591+0000 7f065effd700 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df78c420 0x5654df78e830 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e31c406f1b3021e server_cookie=ecf588e5240a8664 in_seq=0 out_seq=0 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.591+0000 7f063effd700 10 client.17850 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f063effd700 1 -- 172.21.15.5:0/3682221126 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5654df6bceb0 con 0x5654df78c420 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f063effd700 10 client.17850 handle_command_reply: tid=0 2023-07-21T19:19:05.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 client.17850 shutdown 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 msgr2=0x5654df779820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f0658004a20 tx=0x7f0658005aa0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 shutdown_connections 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df77cad0 0x5654df77a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df78c420 0x5654df78e830 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df7712d0 0x5654df779820 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 --2- 172.21.15.5:0/3682221126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df771be0 0x5654df779d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 >> 172.21.15.5:0/3682221126 conn(0x5654df767da0 msgr2=0x5654df76d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f063effd700 0 client.17850 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:05.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 shutdown_connections 2023-07-21T19:19:05.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 1 -- 172.21.15.5:0/3682221126 wait complete. 2023-07-21T19:19:05.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.592+0000 7f0679656b80 20 client.17850 trim_cache size 0 max 16384 2023-07-21T19:19:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.611+0000 7f0679656b80 1 Processor -- start 2023-07-21T19:19:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.611+0000 7f0679656b80 1 -- start start 2023-07-21T19:19:05.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 0x5654df7834f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 0x5654df774b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df77cad0 0x5654df7750b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df769ec0 con 0x5654df77cad0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df772540 con 0x5654df7712d0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f0679656b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df77ae00 con 0x5654df771be0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f065ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df77cad0 0x5654df7750b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f065ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 0x5654df7834f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f065effd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 0x5654df774b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f065ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 0x5654df7834f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60826/0 (socket says 172.21.15.5:60826) 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.612+0000 7f065ffff700 1 -- 172.21.15.5:0/2705282790 learned_addr learned my addr 172.21.15.5:0/2705282790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 -- 172.21.15.5:0/2705282790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 msgr2=0x5654df7834f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 0x5654df7834f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 -- 172.21.15.5:0/2705282790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df77cad0 msgr2=0x5654df7750b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df77cad0 0x5654df7750b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 -- 172.21.15.5:0/2705282790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5654df7695e0 con 0x5654df771be0 2023-07-21T19:19:05.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f065effd700 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 0x5654df774b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5654df78f0d0 tx=0x7f065400b410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aaa581840520b863 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f06467fc700 1 -- 172.21.15.5:0/2705282790 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654016030 con 0x5654df771be0 2023-07-21T19:19:05.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f06467fc700 1 -- 172.21.15.5:0/2705282790 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0654017040 con 0x5654df771be0 2023-07-21T19:19:05.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.613+0000 7f06467fc700 1 -- 172.21.15.5:0/2705282790 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0654005280 con 0x5654df771be0 2023-07-21T19:19:05.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 -- 172.21.15.5:0/2705282790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 msgr2=0x5654df774b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 0x5654df774b70 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x5654df78f0d0 tx=0x7f065400b410 comp rx=0 tx=0).stop 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 -- 172.21.15.5:0/2705282790 shutdown_connections 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df77cad0 0x5654df7750b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df7712d0 0x5654df7834f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 --2- 172.21.15.5:0/2705282790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df771be0 0x5654df774b70 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 -- 172.21.15.5:0/2705282790 >> 172.21.15.5:0/2705282790 conn(0x5654df767da0 msgr2=0x5654df7887f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 -- 172.21.15.5:0/2705282790 shutdown_connections 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.614+0000 7f0679656b80 1 -- 172.21.15.5:0/2705282790 wait complete. 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.615+0000 7f0679656b80 1 -- start start 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.615+0000 7f0679656b80 1 Processor -- start 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 0x5654df7d4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 0x5654df7d4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df77cad0 0x5654df7d82e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df7d8890 con 0x5654df771be0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df7d8a00 con 0x5654df77cad0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5654df7d8bc0 con 0x5654df7712d0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f0679656b80 10 client.0 _ll_register_callbacks cb 0x7f06500d1be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f065effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 0x5654df7d4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f065ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 0x5654df7d4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f065effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 0x5654df7d4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33274/0 (socket says 172.21.15.5:33274) 2023-07-21T19:19:05.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f065effd700 1 -- 172.21.15.5:0/1884183174 learned_addr learned my addr 172.21.15.5:0/1884183174 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.616+0000 7f065ffff700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df77cad0 0x5654df7d82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 -- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df77cad0 msgr2=0x5654df7d82e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df77cad0 0x5654df7d82e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 -- 172.21.15.5:0/1884183174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 msgr2=0x5654df7d4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 0x5654df7d4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 -- 172.21.15.5:0/1884183174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f065401fab0 con 0x5654df7712d0 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f065ffff700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 0x5654df7d4420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0658026e70 tx=0x7f0658004360 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae9d7b270fc2223 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f063effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0658007ce0 con 0x5654df7712d0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0658033080 con 0x5654df7712d0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.617+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f065800f040 con 0x5654df7712d0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06580049e0 con 0x5654df7712d0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 10 client.27454 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x5654df4dac60 con 0x5654df7712d0 2023-07-21T19:19:05.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f065800f040 con 0x5654df7712d0 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0658015c30 con 0x5654df7712d0 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 10 client.27454 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 10 client.27454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 20 client.27454 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df7e1190 0x5654df7e3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 4 client.27454 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:19:05.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.618+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5654df67f050 con 0x5654df7e1190 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.619+0000 7f065effd700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df7e1190 0x5654df7e3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.619+0000 7f065effd700 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df7e1190 0x5654df7e3570 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2444d9633725281e server_cookie=d6cc4a76a12b9692 in_seq=0 out_seq=0 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.619+0000 7f063effd700 10 client.27454 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f063effd700 1 -- 172.21.15.5:0/1884183174 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5654df67f050 con 0x5654df7e1190 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f063effd700 10 client.27454 handle_command_reply: tid=0 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 client.27454 shutdown 2023-07-21T19:19:05.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 msgr2=0x5654df7d4420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 0x5654df7d4420 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0658026e70 tx=0x7f0658004360 comp rx=0 tx=0).stop 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 shutdown_connections 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5654df77cad0 0x5654df7d82e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5654df7712d0 0x5654df7d4420 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5654df7e1190 0x5654df7e3570 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 --2- 172.21.15.5:0/1884183174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5654df771be0 0x5654df7d4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 >> 172.21.15.5:0/1884183174 conn(0x5654df767da0 msgr2=0x5654df774b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f063effd700 0 client.27454 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 shutdown_connections 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 1 -- 172.21.15.5:0/1884183174 wait complete. 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.620+0000 7f0679656b80 20 client.27454 trim_cache size 0 max 16384 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:05.622 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:19:05.623 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:19:05.623 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:05.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 msgr2=0x7f06480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 0x7f06480a3770 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f065009b330 tx=0x7f0650097040 comp rx=0 tx=0).stop 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 msgr2=0x7f066019dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0650006010 tx=0x7f0650005740 comp rx=0 tx=0).stop 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 shutdown_connections 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f06480a12c0 0x7f06480a3770 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f066010c3d0 0x7f0660197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06600fe820 0x7f066019dad0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 --2- 172.21.15.5:0/495508697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06600ff1c0 0x7f066019e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 >> 172.21.15.5:0/495508697 conn(0x7f06600fa4b0 msgr2=0x7f066010e540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 shutdown_connections 2023-07-21T19:19:05.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.621+0000 7f0665dff700 1 -- 172.21.15.5:0/495508697 wait complete. 2023-07-21T19:19:05.636 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:19:05.637 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:19:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 -- 172.21.15.5:0/1595233568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb40ffad0 msgr2=0x7f3bb40fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/1595233568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb40ffad0 0x7f3bb40fff40 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f3ba8009990 tx=0x7f3ba8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 -- 172.21.15.5:0/1595233568 shutdown_connections 2023-07-21T19:19:05.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/1595233568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb4100480 0x7f3bb410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/1595233568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb4105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/1595233568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb40ffad0 0x7f3bb40fff40 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 -- 172.21.15.5:0/1595233568 >> 172.21.15.5:0/1595233568 conn(0x7f3bb40fb590 msgr2=0x7f3bb40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 -- 172.21.15.5:0/1595233568 shutdown_connections 2023-07-21T19:19:05.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.928+0000 7f3bb95ff700 1 -- 172.21.15.5:0/1595233568 wait complete. 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.930+0000 7f3bb95ff700 1 Processor -- start 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.930+0000 7f3bb95ff700 1 -- start start 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.930+0000 7f3bb95ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.930+0000 7f3bb95ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb4100480 0x7f3bb4198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.930+0000 7f3bb95ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb419b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb95ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb419bbd0 con 0x7f3bb4100480 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb95ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb419bd40 con 0x7f3bb40ffad0 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb95ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bb419bf00 con 0x7f3bb4105ae0 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60870/0 (socket says 172.21.15.5:60870) 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 -- 172.21.15.5:0/3465527088 learned_addr learned my addr 172.21.15.5:0/3465527088 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb27fc700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb4100480 0x7f3bb4198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb37fe700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb419b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 -- 172.21.15.5:0/3465527088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 msgr2=0x7f3bb419b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb419b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 -- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb4100480 msgr2=0x7f3bb4198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb4100480 0x7f3bb4198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb2ffd700 1 -- 172.21.15.5:0/3465527088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba80095f0 con 0x7f3bb40ffad0 2023-07-21T19:19:05.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.931+0000 7f3bb37fe700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb419b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:05.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3bb2ffd700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9a0 tx=0x7f3ba400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4009790 con 0x7f3bb40ffad0 2023-07-21T19:19:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ba400cbd0 con 0x7f3bb40ffad0 2023-07-21T19:19:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba4018680 con 0x7f3bb40ffad0 2023-07-21T19:19:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bb419c1e0 con 0x7f3bb40ffad0 2023-07-21T19:19:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.932+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bb41a4b40 con 0x7f3bb40ffad0 2023-07-21T19:19:05.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.933+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bb4067300 con 0x7f3bb40ffad0 2023-07-21T19:19:05.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.937+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3ba400cd40 con 0x7f3bb40ffad0 2023-07-21T19:19:05.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.939+0000 7f3b9bfff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 0x7f3b9c0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.939+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3ba4014070 con 0x7f3bb40ffad0 2023-07-21T19:19:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.939+0000 7f3bb27fc700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 0x7f3b9c0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:05.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.940+0000 7f3bb27fc700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 0x7f3b9c0a37b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4199350 tx=0x7f3ba800b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:05.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:05.943+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ba40964e0 con 0x7f3bb40ffad0 2023-07-21T19:19:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:05 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:19:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:05 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:19:06.154 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:05 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.315+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3bb4198ff0 con 0x7f3bb40ffad0 2023-07-21T19:19:06.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.316+0000 7f3b9bfff700 1 -- 172.21.15.5:0/3465527088 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f3ba4099b00 con 0x7f3bb40ffad0 2023-07-21T19:19:06.311 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:19:06.311 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":42,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":42,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:17:56.540961+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:19:06.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 msgr2=0x7f3b9c0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 0x7f3b9c0a37b0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3bb4199350 tx=0x7f3ba800b260 comp rx=0 tx=0).stop 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 msgr2=0x7f3bb4197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f3ba400e9a0 tx=0x7f3ba400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 shutdown_connections 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3bb4105ae0 0x7f3bb419b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3bb40ffad0 0x7f3bb4197ce0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b9c0a1300 0x7f3b9c0a37b0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 --2- 172.21.15.5:0/3465527088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3bb4100480 0x7f3bb4198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 >> 172.21.15.5:0/3465527088 conn(0x7f3bb40fb590 msgr2=0x7f3bb410aeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 shutdown_connections 2023-07-21T19:19:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.319+0000 7f3bb95ff700 1 -- 172.21.15.5:0/3465527088 wait complete. 2023-07-21T19:19:06.315 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 42 2023-07-21T19:19:06.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:05 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:19:06.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:05 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:19:06.327 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:05 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:06.334 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:19:06.415 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:05 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:19:06.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:05 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:19:06.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:05 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:19:06.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.569+0000 7fc47fdff700 1 -- 172.21.15.5:0/4235112945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 msgr2=0x7fc47810bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.569+0000 7fc47fdff700 1 --2- 172.21.15.5:0/4235112945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47810bde0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fc468009990 tx=0x7fc468009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 -- 172.21.15.5:0/4235112945 shutdown_connections 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 --2- 172.21.15.5:0/4235112945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 0x7fc47810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 --2- 172.21.15.5:0/4235112945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 0x7fc478072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 --2- 172.21.15.5:0/4235112945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47810bde0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 -- 172.21.15.5:0/4235112945 >> 172.21.15.5:0/4235112945 conn(0x7fc4780fb5e0 msgr2=0x7fc4780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 -- 172.21.15.5:0/4235112945 shutdown_connections 2023-07-21T19:19:06.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.570+0000 7fc47fdff700 1 -- 172.21.15.5:0/4235112945 wait complete. 2023-07-21T19:19:06.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.571+0000 7fc47fdff700 1 Processor -- start 2023-07-21T19:19:06.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.571+0000 7fc47fdff700 1 -- start start 2023-07-21T19:19:06.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.571+0000 7fc47fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 0x7fc47819dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.571+0000 7fc47fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47fdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 0x7fc478197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4781981b0 con 0x7fc4780731c0 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47fdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc478198320 con 0x7fc47810c320 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4781984e0 con 0x7fc4780728b0 2023-07-21T19:19:06.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56674/0 (socket says 172.21.15.5:56674) 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 -- 172.21.15.5:0/229341613 learned_addr learned my addr 172.21.15.5:0/229341613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47e39c700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 0x7fc478197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47db9b700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 0x7fc47819dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 -- 172.21.15.5:0/229341613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 msgr2=0x7fc47819dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 0x7fc47819dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 -- 172.21.15.5:0/229341613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 msgr2=0x7fc478197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 0x7fc478197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.572+0000 7fc47d39a700 1 -- 172.21.15.5:0/229341613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4680095f0 con 0x7fc4780731c0 2023-07-21T19:19:06.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc47d39a700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc468005740 tx=0x7fc468005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc46effd700 1 -- 172.21.15.5:0/229341613 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46801d070 con 0x7fc4780731c0 2023-07-21T19:19:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc46effd700 1 -- 172.21.15.5:0/229341613 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc46800fbb0 con 0x7fc4780731c0 2023-07-21T19:19:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc46effd700 1 -- 172.21.15.5:0/229341613 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4680173f0 con 0x7fc4780731c0 2023-07-21T19:19:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc478198760 con 0x7fc4780731c0 2023-07-21T19:19:06.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.573+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4781a4ac0 con 0x7fc4780731c0 2023-07-21T19:19:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.577+0000 7fc46effd700 1 -- 172.21.15.5:0/229341613 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc46800fd20 con 0x7fc4780731c0 2023-07-21T19:19:06.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.578+0000 7fc46effd700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 0x7fc4640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.578+0000 7fc46effd700 1 -- 172.21.15.5:0/229341613 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc4680ce130 con 0x7fc4780731c0 2023-07-21T19:19:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.578+0000 7fc47db9b700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 0x7fc4640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.578+0000 7fc4936adb80 1 Processor -- start 2023-07-21T19:19:06.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 -- start start 2023-07-21T19:19:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1abb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 0x55cee1ac6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee19c1cb0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1a14a40 con 0x55cee1abbbe0 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc4936adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1a0ded0 con 0x55cee1abb2d0 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.579+0000 7fc47db9b700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 0x7fc4640a3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc474009bf0 tx=0x7fc474009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56688/0 (socket says 172.21.15.5:56688) 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1abb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 -- 172.21.15.5:0/2364074677 learned_addr learned my addr 172.21.15.5:0/2364074677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47d39a700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 0x55cee1ac6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 -- 172.21.15.5:0/2364074677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 msgr2=0x55cee1abb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1abb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 -- 172.21.15.5:0/2364074677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 msgr2=0x55cee1ac6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 0x55cee1ac6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 -- 172.21.15.5:0/2364074677 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cee183bde0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47d39a700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 0x55cee1ac6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:06.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47db9b700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1abb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:06.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.580+0000 7fc47e39c700 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc470009d10 tx=0x7fc4700077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0e590b7bb6d11e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.581+0000 7fc463fff700 1 -- 172.21.15.5:0/2364074677 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470018070 con 0x55cee1ac6ad0 2023-07-21T19:19:06.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.581+0000 7fc463fff700 1 -- 172.21.15.5:0/2364074677 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc47000f050 con 0x55cee1ac6ad0 2023-07-21T19:19:06.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.581+0000 7fc463fff700 1 -- 172.21.15.5:0/2364074677 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470004640 con 0x55cee1ac6ad0 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.581+0000 7fc4936adb80 1 -- 172.21.15.5:0/2364074677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 msgr2=0x55cee1ac8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.581+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7fc470009d10 tx=0x7fc4700077e0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 -- 172.21.15.5:0/2364074677 shutdown_connections 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac8eb0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1abb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2364074677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abbbe0 0x55cee1ac6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 -- 172.21.15.5:0/2364074677 >> 172.21.15.5:0/2364074677 conn(0x55cee1ab1da0 msgr2=0x55cee1ab76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 -- 172.21.15.5:0/2364074677 shutdown_connections 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 -- 172.21.15.5:0/2364074677 wait complete. 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.582+0000 7fc4936adb80 1 -- start start 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 Processor -- start 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abb2d0 0x55cee1ac6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1ac3760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee19adac0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee19ad7c0 con 0x55cee1abb2d0 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1891920 con 0x55cee1abbbe0 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.583+0000 7fc4936adb80 10 client.0 _ll_register_callbacks cb 0x55cee19ab760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56700/0 (socket says 172.21.15.5:56700) 2023-07-21T19:19:06.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47db9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abb2d0 0x55cee1ac6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 -- 172.21.15.5:0/1524386341 learned_addr learned my addr 172.21.15.5:0/1524386341 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47d39a700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1ac3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 -- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 msgr2=0x55cee1ac3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1ac3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 -- 172.21.15.5:0/1524386341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abb2d0 msgr2=0x55cee1ac6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abb2d0 0x55cee1ac6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47e39c700 1 -- 172.21.15.5:0/1524386341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc474011a20 con 0x55cee1ac6ad0 2023-07-21T19:19:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.584+0000 7fc47d39a700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1ac3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc47e39c700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc470004140 tx=0x7fc4700093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=846e22862382e01e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4617fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:06.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc470018030 con 0x55cee1ac6ad0 2023-07-21T19:19:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cee19674f0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc4700045b0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc470004770 con 0x55cee1ac6ad0 2023-07-21T19:19:06.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4936adb80 10 client.17874 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:06.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fc470009940 con 0x55cee1ac6ad0 2023-07-21T19:19:06.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.585+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55cee19674f0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc4700144a0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 10 client.17874 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 10 client.17874 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 20 client.17874 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1ad6320 0x55cee1ad8730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 4 client.17874 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.586+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cee1a06eb0 con 0x55cee1ad6320 2023-07-21T19:19:06.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.587+0000 7fc47db9b700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1ad6320 0x55cee1ad8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.587+0000 7fc47db9b700 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1ad6320 0x55cee1ad8730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=40bc63c69d84b3e1 server_cookie=ede5778cb0853a45 in_seq=0 out_seq=0 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.587+0000 7fc4617fa700 10 client.17874 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4617fa700 1 -- 172.21.15.5:0/1524386341 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cee1a06eb0 con 0x55cee1ad6320 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4617fa700 10 client.17874 handle_command_reply: tid=0 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 client.17874 shutdown 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 msgr2=0x55cee1ac3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7fc470004140 tx=0x7fc4700093e0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 shutdown_connections 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1ac6ad0 0x55cee1ac3e50 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1ad6320 0x55cee1ad8730 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1abb2d0 0x55cee1ac6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 --2- 172.21.15.5:0/1524386341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1ac3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 >> 172.21.15.5:0/1524386341 conn(0x55cee1ab1da0 msgr2=0x55cee1ab7230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4617fa700 0 client.17874 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 shutdown_connections 2023-07-21T19:19:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.588+0000 7fc4936adb80 1 -- 172.21.15.5:0/1524386341 wait complete. 2023-07-21T19:19:06.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.589+0000 7fc4936adb80 20 client.17874 trim_cache size 0 max 16384 2023-07-21T19:19:06.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.604+0000 7fc4936adb80 1 Processor -- start 2023-07-21T19:19:06.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.604+0000 7fc4936adb80 1 -- start start 2023-07-21T19:19:06.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.604+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.604+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1add090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.604+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1ad8330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc4936adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1add5d0 con 0x55cee1abb2d0 2023-07-21T19:19:06.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc4936adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1ac2cd0 con 0x55cee1ac6ad0 2023-07-21T19:19:06.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc4936adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1ac2e90 con 0x55cee1abbbe0 2023-07-21T19:19:06.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56710/0 (socket says 172.21.15.5:56710) 2023-07-21T19:19:06.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 -- 172.21.15.5:0/2306820121 learned_addr learned my addr 172.21.15.5:0/2306820121 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47d39a700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1add090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47e39c700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1ad8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 -- 172.21.15.5:0/2306820121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 msgr2=0x55cee1add090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1add090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 -- 172.21.15.5:0/2306820121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 msgr2=0x55cee1ad8330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1ad8330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47db9b700 1 -- 172.21.15.5:0/2306820121 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cee1824c60 con 0x55cee1abb2d0 2023-07-21T19:19:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.605+0000 7fc47e39c700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1ad8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:06.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.606+0000 7fc47db9b700 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc47402bce0 tx=0x7fc474014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13065ef1c26d9d1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.606+0000 7fc46cbff700 1 -- 172.21.15.5:0/2306820121 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474025040 con 0x55cee1abb2d0 2023-07-21T19:19:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.606+0000 7fc46cbff700 1 -- 172.21.15.5:0/2306820121 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc474016070 con 0x55cee1abb2d0 2023-07-21T19:19:06.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.606+0000 7fc46cbff700 1 -- 172.21.15.5:0/2306820121 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474003d30 con 0x55cee1abb2d0 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- 172.21.15.5:0/2306820121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 msgr2=0x55cee1ac3d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7fc47402bce0 tx=0x7fc474014000 comp rx=0 tx=0).stop 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- 172.21.15.5:0/2306820121 shutdown_connections 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1ad8330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abb2d0 0x55cee1ac3d60 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 --2- 172.21.15.5:0/2306820121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abbbe0 0x55cee1add090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- 172.21.15.5:0/2306820121 >> 172.21.15.5:0/2306820121 conn(0x55cee1ab1da0 msgr2=0x55cee1ab3310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- 172.21.15.5:0/2306820121 shutdown_connections 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- 172.21.15.5:0/2306820121 wait complete. 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.607+0000 7fc4936adb80 1 -- start start 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 Processor -- start 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1b231b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1b236f0 con 0x55cee1abbbe0 2023-07-21T19:19:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1b23860 con 0x55cee1ac6ad0 2023-07-21T19:19:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cee1b23a20 con 0x55cee1abb2d0 2023-07-21T19:19:06.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc4936adb80 10 client.0 _ll_register_callbacks cb 0x7fc474009920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc47d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc47db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc47d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56722/0 (socket says 172.21.15.5:56722) 2023-07-21T19:19:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc47d39a700 1 -- 172.21.15.5:0/468446314 learned_addr learned my addr 172.21.15.5:0/468446314 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.608+0000 7fc47db9b700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57208/0 (socket says 172.21.15.5:57208) 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47e39c700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1b231b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 -- 172.21.15.5:0/468446314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 msgr2=0x55cee1b231b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1b231b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 -- 172.21.15.5:0/468446314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 msgr2=0x55cee1acbc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 -- 172.21.15.5:0/468446314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc47402b9c0 con 0x55cee1abb2d0 2023-07-21T19:19:06.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47d39a700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc47db9b700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc47402e040 tx=0x7fc474003700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4dae18078aa01ef2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.609+0000 7fc4617fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc474025040 con 0x55cee1abb2d0 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc47000f050 con 0x55cee1abb2d0 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc474018070 con 0x55cee1abb2d0 2023-07-21T19:19:06.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc47402b520 con 0x55cee1abb2d0 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4936adb80 10 client.27475 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55cee1862250 con 0x55cee1abb2d0 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.610+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc47000f050 con 0x55cee1abb2d0 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc474020990 con 0x55cee1abb2d0 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 10 client.27475 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:06.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 10 client.27475 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 20 client.27475 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1b2b100 0x55cee1b2d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 4 client.27475 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55cee19c9050 con 0x55cee1b2b100 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.611+0000 7fc47d39a700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1b2b100 0x55cee1b2d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4617fa700 10 client.27475 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc47d39a700 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1b2b100 0x55cee1b2d4e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d213469796b55644 server_cookie=80f7ebbd254e2b77 in_seq=0 out_seq=0 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4617fa700 1 -- 172.21.15.5:0/468446314 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55cee19c9050 con 0x55cee1b2b100 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4617fa700 10 client.27475 handle_command_reply: tid=0 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 client.27475 shutdown 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 msgr2=0x55cee1acb6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc47402e040 tx=0x7fc474003700 comp rx=0 tx=0).stop 2023-07-21T19:19:06.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 shutdown_connections 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55cee1b2b100 0x55cee1b2d4e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cee1ac6ad0 0x55cee1b231b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cee1abb2d0 0x55cee1acb6c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 --2- 172.21.15.5:0/468446314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cee1abbbe0 0x55cee1acbc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4617fa700 0 client.27475 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:06.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 >> 172.21.15.5:0/468446314 conn(0x55cee1ab1da0 msgr2=0x55cee1abea90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 shutdown_connections 2023-07-21T19:19:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.612+0000 7fc4936adb80 1 -- 172.21.15.5:0/468446314 wait complete. 2023-07-21T19:19:06.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc4936adb80 20 client.27475 trim_cache size 0 max 16384 2023-07-21T19:19:06.619 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:06.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 msgr2=0x7fc4640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 0x7fc4640a3770 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc474009bf0 tx=0x7fc474009260 comp rx=0 tx=0).stop 2023-07-21T19:19:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 msgr2=0x7fc47819e010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fc468005740 tx=0x7fc468005e80 comp rx=0 tx=0).stop 2023-07-21T19:19:06.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 shutdown_connections 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc47810c320 0x7fc478197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4780728b0 0x7fc47819dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4640a12c0 0x7fc4640a3770 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 --2- 172.21.15.5:0/229341613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4780731c0 0x7fc47819e010 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 >> 172.21.15.5:0/229341613 conn(0x7fc4780fb5e0 msgr2=0x7fc47810e410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 shutdown_connections 2023-07-21T19:19:06.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.613+0000 7fc47fdff700 1 -- 172.21.15.5:0/229341613 wait complete. 2023-07-21T19:19:06.628 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:19:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.861+0000 7f8c973ff700 1 -- 172.21.15.5:0/587523348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c901015b0 msgr2=0x7f8c90109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.861+0000 7f8c973ff700 1 --2- 172.21.15.5:0/587523348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c901015b0 0x7f8c90109aa0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c800098f0 tx=0x7f8c80009c00 comp rx=0 tx=0).stop 2023-07-21T19:19:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 -- 172.21.15.5:0/587523348 shutdown_connections 2023-07-21T19:19:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 --2- 172.21.15.5:0/587523348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c9010a070 0x7f8c9010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 --2- 172.21.15.5:0/587523348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 0x7f8c90101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 --2- 172.21.15.5:0/587523348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c901015b0 0x7f8c90109aa0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 -- 172.21.15.5:0/587523348 >> 172.21.15.5:0/587523348 conn(0x7f8c900fa4b0 msgr2=0x7f8c900fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 -- 172.21.15.5:0/587523348 shutdown_connections 2023-07-21T19:19:06.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.862+0000 7f8c973ff700 1 -- 172.21.15.5:0/587523348 wait complete. 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.863+0000 7f8c973ff700 1 Processor -- start 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.863+0000 7f8c973ff700 1 -- start start 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 0x7f8c9019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c9010a070 0x7f8c90195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90195fd0 con 0x7f8c90100ca0 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90196140 con 0x7f8c901015b0 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c973ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c90196300 con 0x7f8c9010a070 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9599c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c9010a070 0x7f8c90195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9519b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 0x7f8c9019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50454/0 (socket says 172.21.15.5:50454) 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 -- 172.21.15.5:0/180753959 learned_addr learned my addr 172.21.15.5:0/180753959 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 -- 172.21.15.5:0/180753959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c9010a070 msgr2=0x7f8c90195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c9010a070 0x7f8c90195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 -- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 msgr2=0x7f8c9019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 0x7f8c9019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.864+0000 7f8c9499a700 1 -- 172.21.15.5:0/180753959 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009520 con 0x7f8c901015b0 2023-07-21T19:19:06.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c9499a700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009230 tx=0x7f8c8000fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c867fc700 1 -- 172.21.15.5:0/180753959 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8001c070 con 0x7f8c901015b0 2023-07-21T19:19:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c867fc700 1 -- 172.21.15.5:0/180753959 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c80005040 con 0x7f8c901015b0 2023-07-21T19:19:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c90196580 con 0x7f8c901015b0 2023-07-21T19:19:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c867fc700 1 -- 172.21.15.5:0/180753959 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c800038e0 con 0x7f8c901015b0 2023-07-21T19:19:06.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.865+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900fea00 con 0x7f8c901015b0 2023-07-21T19:19:06.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.868+0000 7f8c867fc700 1 -- 172.21.15.5:0/180753959 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8c80020030 con 0x7f8c901015b0 2023-07-21T19:19:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.870+0000 7f8c867fc700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 0x7f8c7c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.870+0000 7f8c867fc700 1 -- 172.21.15.5:0/180753959 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8c800cdca0 con 0x7f8c901015b0 2023-07-21T19:19:06.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.870+0000 7f8caae06b80 1 Processor -- start 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.870+0000 7f8c9519b700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 0x7f8c7c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 -- start start 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c648a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6497eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c6390cb0 con 0x55e8c648abe0 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c63e3a40 con 0x55e8c6495ad0 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8caae06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c63dced0 con 0x55e8c648a2d0 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56756/0 (socket says 172.21.15.5:56756) 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8c9499a700 1 -- 172.21.15.5:0/2852158533 learned_addr learned my addr 172.21.15.5:0/2852158533 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.871+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6497eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c648a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9519b700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 0x7f8c7c0a3770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005fd0 tx=0x7f8c8c009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 -- 172.21.15.5:0/2852158533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 msgr2=0x55e8c648a6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c648a6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 -- 172.21.15.5:0/2852158533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 msgr2=0x55e8c6495590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 -- 172.21.15.5:0/2852158533 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c620ade0 con 0x55e8c6495ad0 2023-07-21T19:19:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9499a700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:06.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.872+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6497eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011550 tx=0x7f8c88009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=494995d656f323a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.873+0000 7f8c7b7fe700 1 -- 172.21.15.5:0/2852158533 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e040 con 0x55e8c6495ad0 2023-07-21T19:19:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.873+0000 7f8c7b7fe700 1 -- 172.21.15.5:0/2852158533 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c8800f040 con 0x55e8c6495ad0 2023-07-21T19:19:06.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.873+0000 7f8c7b7fe700 1 -- 172.21.15.5:0/2852158533 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88004320 con 0x55e8c6495ad0 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.873+0000 7f8caae06b80 1 -- 172.21.15.5:0/2852158533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 msgr2=0x55e8c6497eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.873+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6497eb0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011550 tx=0x7f8c88009920 comp rx=0 tx=0).stop 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 -- 172.21.15.5:0/2852158533 shutdown_connections 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6497eb0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c648a6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2852158533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c6495590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 -- 172.21.15.5:0/2852158533 >> 172.21.15.5:0/2852158533 conn(0x55e8c6480da0 msgr2=0x55e8c64866a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 -- 172.21.15.5:0/2852158533 shutdown_connections 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 -- 172.21.15.5:0/2852158533 wait complete. 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.874+0000 7f8caae06b80 1 -- start start 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 Processor -- start 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c6495500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c64927f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6492f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c637cac0 con 0x55e8c648abe0 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c637c7c0 con 0x55e8c6495ad0 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c6260920 con 0x55e8c648a2d0 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.875+0000 7f8caae06b80 10 client.0 _ll_register_callbacks cb 0x55e8c637a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9519b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c6495500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c64927f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9519b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c6495500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57238/0 (socket says 172.21.15.5:57238) 2023-07-21T19:19:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9519b700 1 -- 172.21.15.5:0/2028724543 learned_addr learned my addr 172.21.15.5:0/2028724543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6492f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 -- 172.21.15.5:0/2028724543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 msgr2=0x55e8c6495500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c6495500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 -- 172.21.15.5:0/2028724543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 msgr2=0x55e8c64927f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c64927f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9599c700 1 -- 172.21.15.5:0/2028724543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c900fea00 con 0x55e8c6495ad0 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.876+0000 7f8c9499a700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c64927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6492f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011550 tx=0x7f8c88011f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc59c3fea141973a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8c78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e040 con 0x55e8c6495ad0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e8c63364f0 con 0x55e8c6495ad0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8c88004240 con 0x55e8c6495ad0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c88004400 con 0x55e8c6495ad0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8caae06b80 10 client.27563 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.877+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f8c88011230 con 0x55e8c6495ad0 2023-07-21T19:19:06.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e8c63364f0 con 0x55e8c6495ad0 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c8801b020 con 0x55e8c6495ad0 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 10 client.27563 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 10 client.27563 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 20 client.27563 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c64a52d0 0x55e8c64a76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 4 client.27563 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.878+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e8c63d5eb0 con 0x55e8c64a52d0 2023-07-21T19:19:06.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.879+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c64a52d0 0x55e8c64a76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.879+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c64a52d0 0x55e8c64a76e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e105524c44aa66eb server_cookie=d3c34d77271bb31a in_seq=0 out_seq=0 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.879+0000 7f8c78ff9700 10 client.27563 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.880+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2028724543 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e8c63d5eb0 con 0x55e8c64a52d0 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.880+0000 7f8c78ff9700 10 client.27563 handle_command_reply: tid=0 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.880+0000 7f8caae06b80 1 client.27563 shutdown 2023-07-21T19:19:06.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.880+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 msgr2=0x55e8c6492f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.880+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6492f30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011550 tx=0x7f8c88011f10 comp rx=0 tx=0).stop 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 shutdown_connections 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c64a52d0 0x55e8c64a76e0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c6492f30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c6495500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2028724543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c64927f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 >> 172.21.15.5:0/2028724543 conn(0x55e8c6480da0 msgr2=0x55e8c6486230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8c78ff9700 0 client.27563 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 shutdown_connections 2023-07-21T19:19:06.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 1 -- 172.21.15.5:0/2028724543 wait complete. 2023-07-21T19:19:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.881+0000 7f8caae06b80 20 client.27563 trim_cache size 0 max 16384 2023-07-21T19:19:06.885 INFO:teuthology.orchestra.run.smithi005.stderr:43900000 of 50000000 tuples (87%) done (elapsed 664.45 s, remaining 92.33 s) 2023-07-21T19:19:06.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.900+0000 7f8caae06b80 1 Processor -- start 2023-07-21T19:19:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.900+0000 7f8caae06b80 1 -- start start 2023-07-21T19:19:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c64950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c648fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c648ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c648bdd0 con 0x55e8c648abe0 2023-07-21T19:19:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c6492df0 con 0x55e8c6495ad0 2023-07-21T19:19:06.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8caae06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c64ac3c0 con 0x55e8c648a2d0 2023-07-21T19:19:06.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c648fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8c9499a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c648fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56774/0 (socket says 172.21.15.5:56774) 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8c9499a700 1 -- 172.21.15.5:0/2963553278 learned_addr learned my addr 172.21.15.5:0/2963553278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c648ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.901+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c64950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 -- 172.21.15.5:0/2963553278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 msgr2=0x55e8c64950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c64950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 -- 172.21.15.5:0/2963553278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 msgr2=0x55e8c648fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c648fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 -- 172.21.15.5:0/2963553278 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e8c61f3c60 con 0x55e8c6495ad0 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c64950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c648ffc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011e90 tx=0x7f8c88011b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa95d3d67ab8bbac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c7bfff700 1 -- 172.21.15.5:0/2963553278 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8800e040 con 0x55e8c6495ad0 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c7bfff700 1 -- 172.21.15.5:0/2963553278 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c88003e30 con 0x55e8c6495ad0 2023-07-21T19:19:06.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.902+0000 7f8c7bfff700 1 -- 172.21.15.5:0/2963553278 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c88004930 con 0x55e8c6495ad0 2023-07-21T19:19:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 -- 172.21.15.5:0/2963553278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 msgr2=0x55e8c648ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c648ffc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8c88011e90 tx=0x7f8c88011b40 comp rx=0 tx=0).stop 2023-07-21T19:19:06.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 -- 172.21.15.5:0/2963553278 shutdown_connections 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c6495ad0 0x55e8c648ffc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648a2d0 0x55e8c64950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2963553278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c648abe0 0x55e8c648fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 -- 172.21.15.5:0/2963553278 >> 172.21.15.5:0/2963553278 conn(0x55e8c6480da0 msgr2=0x55e8c648daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 -- 172.21.15.5:0/2963553278 shutdown_connections 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.903+0000 7f8caae06b80 1 -- 172.21.15.5:0/2963553278 wait complete. 2023-07-21T19:19:06.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.904+0000 7f8caae06b80 1 -- start start 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.904+0000 7f8caae06b80 1 Processor -- start 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648abe0 0x55e8c649a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 0x55e8c649af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c64f8190 con 0x55e8c6495ad0 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c64f8300 con 0x55e8c648a2d0 2023-07-21T19:19:06.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e8c64f84c0 con 0x55e8c648abe0 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8caae06b80 10 client.0 _ll_register_callbacks cb 0x7f8c800970c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8c9519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8c9519b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50498/0 (socket says 172.21.15.5:50498) 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8c9519b700 1 -- 172.21.15.5:0/2111832682 learned_addr learned my addr 172.21.15.5:0/2111832682 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.905+0000 7f8c9499a700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648abe0 0x55e8c649a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 0x55e8c649af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 -- 172.21.15.5:0/2111832682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648abe0 msgr2=0x55e8c649a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648abe0 0x55e8c649a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 -- 172.21.15.5:0/2111832682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 msgr2=0x55e8c649af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 0x55e8c649af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 -- 172.21.15.5:0/2111832682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c8c017050 con 0x55e8c648a2d0 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9599c700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 0x55e8c649af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c9519b700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e8c64a7f80 tx=0x7f8c8c0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7bb91850faf44191 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:06.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.906+0000 7f8c78ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c021030 con 0x55e8c648a2d0 2023-07-21T19:19:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c8800f040 con 0x55e8c648a2d0 2023-07-21T19:19:06.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8c8c02a2a0 con 0x55e8c648a2d0 2023-07-21T19:19:06.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c8c02a460 con 0x55e8c648a2d0 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8caae06b80 10 client.27569 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f8c88011230 con 0x55e8c648a2d0 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8c8c03f080 con 0x55e8c648a2d0 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.907+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c8c033600 con 0x55e8c648a2d0 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 10 client.27569 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 10 client.27569 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:19:06.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 20 client.27569 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c65024a0 0x55e8c65048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 4 client.27569 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e8c6398050 con 0x55e8c65024a0 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.908+0000 7f8c9499a700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c65024a0 0x55e8c65048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8c9499a700 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c65024a0 0x55e8c65048b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3666436a0e0f75bd server_cookie=db1efe77b33067f1 in_seq=0 out_seq=0 2023-07-21T19:19:06.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8c78ff9700 10 client.27569 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8c78ff9700 1 -- 172.21.15.5:0/2111832682 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e8c6398050 con 0x55e8c65024a0 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8c78ff9700 10 client.27569 handle_command_reply: tid=0 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 client.27569 shutdown 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 msgr2=0x55e8c649a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55e8c64a7f80 tx=0x7f8c8c0047c0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 shutdown_connections 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e8c6495ad0 0x55e8c649af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55e8c65024a0 0x55e8c65048b0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e8c648a2d0 0x55e8c649a490 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 --2- 172.21.15.5:0/2111832682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e8c648abe0 0x55e8c649a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 >> 172.21.15.5:0/2111832682 conn(0x55e8c6480da0 msgr2=0x55e8c648c300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.909+0000 7f8c78ff9700 0 client.27569 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:06.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 shutdown_connections 2023-07-21T19:19:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8caae06b80 1 -- 172.21.15.5:0/2111832682 wait complete. 2023-07-21T19:19:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8caae06b80 20 client.27569 trim_cache size 0 max 16384 2023-07-21T19:19:06.912 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 msgr2=0x7f8c7c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 0x7f8c7c0a3770 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c005fd0 tx=0x7f8c8c009310 comp rx=0 tx=0).stop 2023-07-21T19:19:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 msgr2=0x7f8c9019be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.910+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009230 tx=0x7f8c8000fa00 comp rx=0 tx=0).stop 2023-07-21T19:19:06.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 shutdown_connections 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c9010a070 0x7f8c90195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8c7c0a12c0 0x7f8c7c0a3770 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c90100ca0 0x7f8c9019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 --2- 172.21.15.5:0/180753959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c901015b0 0x7f8c9019be30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 >> 172.21.15.5:0/180753959 conn(0x7f8c900fa4b0 msgr2=0x7f8c9010c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 shutdown_connections 2023-07-21T19:19:06.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:06.911+0000 7f8c973ff700 1 -- 172.21.15.5:0/180753959 wait complete. 2023-07-21T19:19:06.926 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:19:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 -- 172.21.15.5:0/1019585312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c100480 msgr2=0x7ff52c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 --2- 172.21.15.5:0/1019585312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c100480 0x7ff52c10c910 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ff52400b0a0 tx=0x7ff52400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 -- 172.21.15.5:0/1019585312 shutdown_connections 2023-07-21T19:19:07.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 --2- 172.21.15.5:0/1019585312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c100480 0x7ff52c10c910 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 --2- 172.21.15.5:0/1019585312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c105ae0 0x7ff52c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 --2- 172.21.15.5:0/1019585312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 -- 172.21.15.5:0/1019585312 >> 172.21.15.5:0/1019585312 conn(0x7ff52c0fb5f0 msgr2=0x7ff52c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 -- 172.21.15.5:0/1019585312 shutdown_connections 2023-07-21T19:19:07.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.187+0000 7ff5339ff700 1 -- 172.21.15.5:0/1019585312 wait complete. 2023-07-21T19:19:07.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.188+0000 7ff5339ff700 1 Processor -- start 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.188+0000 7ff5339ff700 1 -- start start 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c100480 0x7ff52c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c073580 con 0x7ff52c105ae0 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1a42d0 con 0x7ff52c100480 2023-07-21T19:19:07.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff5339ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff52c1a4410 con 0x7ff52c0ffad0 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff531f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff531f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56798/0 (socket says 172.21.15.5:56798) 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff531f9c700 1 -- 172.21.15.5:0/909412420 learned_addr learned my addr 172.21.15.5:0/909412420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff530f9a700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c100480 0x7ff52c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.189+0000 7ff53179b700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 -- 172.21.15.5:0/909412420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c100480 msgr2=0x7ff52c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c100480 0x7ff52c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 -- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 msgr2=0x7ff52c1a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 -- 172.21.15.5:0/909412420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff524009d00 con 0x7ff52c0ffad0 2023-07-21T19:19:07.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff531f9c700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff53179b700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c072810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9f0 tx=0x7ff52800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.190+0000 7ff5227fc700 1 -- 172.21.15.5:0/909412420 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff52800c9c0 con 0x7ff52c0ffad0 2023-07-21T19:19:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.191+0000 7ff5227fc700 1 -- 172.21.15.5:0/909412420 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff528010430 con 0x7ff52c0ffad0 2023-07-21T19:19:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.191+0000 7ff5227fc700 1 -- 172.21.15.5:0/909412420 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528018680 con 0x7ff52c0ffad0 2023-07-21T19:19:07.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.191+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff52c1a4670 con 0x7ff52c0ffad0 2023-07-21T19:19:07.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.191+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1a4b00 con 0x7ff52c0ffad0 2023-07-21T19:19:07.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.195+0000 7ff5227fc700 1 -- 172.21.15.5:0/909412420 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff5280105a0 con 0x7ff52c0ffad0 2023-07-21T19:19:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.196+0000 7ff5472aeb80 1 Processor -- start 2023-07-21T19:19:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.196+0000 7ff5472aeb80 1 -- start start 2023-07-21T19:19:07.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.196+0000 7ff5227fc700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 0x7ff5180a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5227fc700 1 -- 172.21.15.5:0/909412420 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff528014070 con 0x7ff52c0ffad0 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db2feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff531f9c700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 0x7ff5180a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743da28cb0 con 0x55743db222d0 2023-07-21T19:19:07.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743da7ba40 con 0x55743db22be0 2023-07-21T19:19:07.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743da74ed0 con 0x55743db2dad0 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff53179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff53179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56806/0 (socket says 172.21.15.5:56806) 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff53179b700 1 -- 172.21.15.5:0/1946066875 learned_addr learned my addr 172.21.15.5:0/1946066875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.197+0000 7ff530f9a700 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 -- 172.21.15.5:0/1946066875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 msgr2=0x55743db2feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db2feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 -- 172.21.15.5:0/1946066875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 msgr2=0x55743db2d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 -- 172.21.15.5:0/1946066875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55743d8a2de0 con 0x55743db222d0 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff53179b700 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff528096330 tx=0x7ff5280962a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=424f013d1879dd3f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff531f9c700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 0x7ff5180a37f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff524009610 tx=0x7ff52400bb50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff5177fe700 1 -- 172.21.15.5:0/1946066875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528099e30 con 0x55743db222d0 2023-07-21T19:19:07.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff5177fe700 1 -- 172.21.15.5:0/1946066875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff52809de60 con 0x55743db222d0 2023-07-21T19:19:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.198+0000 7ff5177fe700 1 -- 172.21.15.5:0/1946066875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff528099520 con 0x55743db222d0 2023-07-21T19:19:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1946066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 msgr2=0x55743db226a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7ff528096330 tx=0x7ff5280962a0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1946066875 shutdown_connections 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db2feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db222d0 0x55743db226a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1946066875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.199+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1946066875 >> 172.21.15.5:0/1946066875 conn(0x55743db18da0 msgr2=0x55743db1e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.200+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1946066875 shutdown_connections 2023-07-21T19:19:07.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.200+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1946066875 wait complete. 2023-07-21T19:19:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.201+0000 7ff5472aeb80 1 -- start start 2023-07-21T19:19:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.201+0000 7ff5472aeb80 1 Processor -- start 2023-07-21T19:19:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.201+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 0x55743db2d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743da14ac0 con 0x55743db2dad0 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743da147c0 con 0x55743db22be0 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743d8f8920 con 0x55743db222d0 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.202+0000 7ff5472aeb80 10 client.0 _ll_register_callbacks cb 0x55743da12760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.203+0000 7ff531f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.203+0000 7ff531f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56814/0 (socket says 172.21.15.5:56814) 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.203+0000 7ff531f9c700 1 -- 172.21.15.5:0/3310138247 learned_addr learned my addr 172.21.15.5:0/3310138247 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.203+0000 7ff53179b700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 0x55743db2d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.203+0000 7ff530f9a700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 -- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 msgr2=0x55743db2a7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2a7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 -- 172.21.15.5:0/3310138247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 msgr2=0x55743db2aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 -- 172.21.15.5:0/3310138247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff52c1a4b00 con 0x55743db222d0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff531f9c700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff53179b700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 0x55743db2d520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff528096330 tx=0x7ff52809a2f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ec774875649567b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280b8200 con 0x55743db222d0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff528099e30 con 0x55743db222d0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55743d9ce4f0 con 0x55743db222d0 2023-07-21T19:19:07.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5280a2280 con 0x55743db222d0 2023-07-21T19:19:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff5472aeb80 10 client.27502 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:07.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.204+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55743d88bc60 con 0x55743db222d0 2023-07-21T19:19:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.205+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55743d9ce4f0 con 0x55743db222d0 2023-07-21T19:19:07.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.205+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff5280adb30 con 0x55743db222d0 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 10 client.27502 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 10 client.27502 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 20 client.27502 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db458c0 0x55743db47cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 4 client.27502 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55743da6deb0 con 0x55743db458c0 2023-07-21T19:19:07.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.206+0000 7ff530f9a700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db458c0 0x55743db47cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.207+0000 7ff530f9a700 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db458c0 0x55743db47cd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59efc56331525c1d server_cookie=45cc0eac187f8fe9 in_seq=0 out_seq=0 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.207+0000 7ff514ff9700 10 client.27502 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.207+0000 7ff514ff9700 1 -- 172.21.15.5:0/3310138247 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55743da6deb0 con 0x55743db458c0 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff514ff9700 10 client.27502 handle_command_reply: tid=0 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 client.27502 shutdown 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 msgr2=0x55743db2d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 0x55743db2d520 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff528096330 tx=0x7ff52809a2f0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 shutdown_connections 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db458c0 0x55743db47cd0 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db2aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db222d0 0x55743db2d520 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/3310138247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db22be0 0x55743db2a7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff514ff9700 0 client.27502 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 >> 172.21.15.5:0/3310138247 conn(0x55743db18da0 msgr2=0x55743db1e230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 shutdown_connections 2023-07-21T19:19:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.208+0000 7ff5472aeb80 1 -- 172.21.15.5:0/3310138247 wait complete. 2023-07-21T19:19:07.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.209+0000 7ff5472aeb80 20 client.27502 trim_cache size 0 max 16384 2023-07-21T19:19:07.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.225+0000 7ff5472aeb80 1 Processor -- start 2023-07-21T19:19:07.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.225+0000 7ff5472aeb80 1 -- start start 2023-07-21T19:19:07.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 0x55743db27a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db27fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db2b790 con 0x55743db2dad0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db1fd30 con 0x55743db222d0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db25680 con 0x55743db22be0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff531f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db27fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff53179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff530f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 0x55743db27a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff530f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 0x55743db27a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57296/0 (socket says 172.21.15.5:57296) 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff53179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50548/0 (socket says 172.21.15.5:50548) 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.226+0000 7ff530f9a700 1 -- 172.21.15.5:0/2792711534 learned_addr learned my addr 172.21.15.5:0/2792711534 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 -- 172.21.15.5:0/2792711534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 msgr2=0x55743db27a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 0x55743db27a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 -- 172.21.15.5:0/2792711534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 msgr2=0x55743db27fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db27fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 -- 172.21.15.5:0/2792711534 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55743d8c9250 con 0x55743db222d0 2023-07-21T19:19:07.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff53179b700 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff528098180 tx=0x7ff528010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=37b465996a26798 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff517fff700 1 -- 172.21.15.5:0/2792711534 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280b8200 con 0x55743db222d0 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff517fff700 1 -- 172.21.15.5:0/2792711534 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff528099e30 con 0x55743db222d0 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.227+0000 7ff517fff700 1 -- 172.21.15.5:0/2792711534 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5280ca1a0 con 0x55743db222d0 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 -- 172.21.15.5:0/2792711534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 msgr2=0x55743db2d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff528098180 tx=0x7ff528010b80 comp rx=0 tx=0).stop 2023-07-21T19:19:07.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 -- 172.21.15.5:0/2792711534 shutdown_connections 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db2dad0 0x55743db27fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db2d0c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/2792711534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db22be0 0x55743db27a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 -- 172.21.15.5:0/2792711534 >> 172.21.15.5:0/2792711534 conn(0x55743db18da0 msgr2=0x55743db1ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 -- 172.21.15.5:0/2792711534 shutdown_connections 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.228+0000 7ff5472aeb80 1 -- 172.21.15.5:0/2792711534 wait complete. 2023-07-21T19:19:07.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.229+0000 7ff5472aeb80 1 -- start start 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.229+0000 7ff5472aeb80 1 Processor -- start 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.229+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db32680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.229+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.229+0000 7ff5472aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db98740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db98c80 con 0x55743db22be0 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db98dc0 con 0x55743db222d0 2023-07-21T19:19:07.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff5472aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55743db98f50 con 0x55743db2dad0 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff5472aeb80 10 client.0 _ll_register_callbacks cb 0x55743da12760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56824/0 (socket says 172.21.15.5:56824) 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 -- 172.21.15.5:0/1958985183 learned_addr learned my addr 172.21.15.5:0/1958985183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff531f9c700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db98740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff53179b700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db32680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 -- 172.21.15.5:0/1958985183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 msgr2=0x55743db98740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db98740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 -- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 msgr2=0x55743db32680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db32680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.230+0000 7ff530f9a700 1 -- 172.21.15.5:0/1958985183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55743d88bc60 con 0x55743db22be0 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff530f9a700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff52c074000 tx=0x7ff51c01e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446ec31475d92e38 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff514ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:07.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51c017070 con 0x55743db22be0 2023-07-21T19:19:07.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff52809de60 con 0x55743db22be0 2023-07-21T19:19:07.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff51c004030 con 0x55743db22be0 2023-07-21T19:19:07.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff51c01eea0 con 0x55743db22be0 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff5472aeb80 10 client.17913 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.231+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55743d8a12b0 con 0x55743db22be0 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff52809de60 con 0x55743db22be0 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff51c0037a0 con 0x55743db22be0 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 10 client.17913 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 10 client.17913 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:07.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 20 client.17913 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db9a6e0 0x55743db9caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 4 client.17913 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55743da30050 con 0x55743db9a6e0 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.232+0000 7ff53179b700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db9a6e0 0x55743db9caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff53179b700 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db9a6e0 0x55743db9caf0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3116c16cd58f6629 server_cookie=b6f48c7906954ed5 in_seq=0 out_seq=0 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff514ff9700 10 client.17913 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff514ff9700 1 -- 172.21.15.5:0/1958985183 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55743da30050 con 0x55743db9a6e0 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff514ff9700 10 client.17913 handle_command_reply: tid=0 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 client.17913 shutdown 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 msgr2=0x55743db32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7ff52c074000 tx=0x7ff51c01e670 comp rx=0 tx=0).stop 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 shutdown_connections 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55743db2dad0 0x55743db98740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55743db222d0 0x55743db32680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55743db9a6e0 0x55743db9caf0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 --2- 172.21.15.5:0/1958985183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55743db22be0 0x55743db32bc0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 >> 172.21.15.5:0/1958985183 conn(0x55743db18da0 msgr2=0x55743db243e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.233+0000 7ff514ff9700 0 client.17913 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 shutdown_connections 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5472aeb80 1 -- 172.21.15.5:0/1958985183 wait complete. 2023-07-21T19:19:07.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5472aeb80 20 client.17913 trim_cache size 0 max 16384 2023-07-21T19:19:07.237 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:07.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 msgr2=0x7ff5180a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 0x7ff5180a37f0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7ff524009610 tx=0x7ff52400bb50 comp rx=0 tx=0).stop 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 msgr2=0x7ff52c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c072810 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7ff52800e9f0 tx=0x7ff52800ed00 comp rx=0 tx=0).stop 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 shutdown_connections 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5180a1340 0x7ff5180a37f0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff52c105ae0 0x7ff52c1a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff52c0ffad0 0x7ff52c072810 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 --2- 172.21.15.5:0/909412420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff52c100480 0x7ff52c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.234+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 >> 172.21.15.5:0/909412420 conn(0x7ff52c0fb5f0 msgr2=0x7ff52c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.235+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 shutdown_connections 2023-07-21T19:19:07.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:07.235+0000 7ff5339ff700 1 -- 172.21.15.5:0/909412420 wait complete. 2023-07-21T19:19:07.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:07 smithi005 ceph-mon[126460]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T19:19:07.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:07 smithi005 ceph-mon[126460]: pgmap v359: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 70 op/s 2023-07-21T19:19:07.246 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3465527088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:07.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:07 smithi101 ceph-mon[128136]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T19:19:07.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:07 smithi101 ceph-mon[128136]: pgmap v359: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 70 op/s 2023-07-21T19:19:07.326 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3465527088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:07.415 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:07 smithi156 ceph-mon[128316]: osdmap e99: 12 total, 12 up, 12 in 2023-07-21T19:19:07.415 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:07 smithi156 ceph-mon[128316]: pgmap v359: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 27 MiB/s wr, 70 op/s 2023-07-21T19:19:07.416 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3465527088' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/3728740967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 msgr2=0x7f6a68104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/3728740967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a68104770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c009990 tx=0x7f6a5c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/3728740967 shutdown_connections 2023-07-21T19:19:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/3728740967 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a6810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/3728740967 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 0x7f6a68067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/3728740967 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a68104770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/3728740967 >> 172.21.15.5:0/3728740967 conn(0x7f6a68074760 msgr2=0x7f6a68074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/3728740967 shutdown_connections 2023-07-21T19:19:07.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.834+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/3728740967 wait complete. 2023-07-21T19:19:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.835+0000 7f6a6e5ff700 1 Processor -- start 2023-07-21T19:19:07.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 -- start start 2023-07-21T19:19:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 0x7f6a6819bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a6819c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68196000 con 0x7f6a68104e50 2023-07-21T19:19:07.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68196170 con 0x7f6a68067c10 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.836+0000 7f6a6e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a68196330 con 0x7f6a680685b0 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.837+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.837+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56842/0 (socket says 172.21.15.5:56842) 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.837+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2616113598 learned_addr learned my addr 172.21.15.5:0/2616113598 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.837+0000 7f6a67fff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 0x7f6a6819bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.837+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a6819c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2616113598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 msgr2=0x7f6a6819c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a6819c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2616113598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 msgr2=0x7f6a6819bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 0x7f6a6819bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2616113598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5c0095f0 con 0x7f6a68104e50 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000ef30 tx=0x7f6a6000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a657fa700 1 -- 172.21.15.5:0/2616113598 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60018070 con 0x7f6a68104e50 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a681965b0 con 0x7f6a68104e50 2023-07-21T19:19:07.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.838+0000 7f6a657fa700 1 -- 172.21.15.5:0/2616113598 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a6000f040 con 0x7f6a68104e50 2023-07-21T19:19:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.839+0000 7f6a657fa700 1 -- 172.21.15.5:0/2616113598 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6000cc70 con 0x7f6a68104e50 2023-07-21T19:19:07.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.839+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a681a2b20 con 0x7f6a68104e50 2023-07-21T19:19:07.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.843+0000 7f6a657fa700 1 -- 172.21.15.5:0/2616113598 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6a60007500 con 0x7f6a68104e50 2023-07-21T19:19:07.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.845+0000 7f6a81f87b80 1 Processor -- start 2023-07-21T19:19:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.845+0000 7f6a81f87b80 1 -- start start 2023-07-21T19:19:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.845+0000 7f6a657fa700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 0x7f6a500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a657fa700 1 -- 172.21.15.5:0/2616113598 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f6a600cc890 con 0x7f6a68104e50 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd85640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd89dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 0x7f6a500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bc86c50 con 0x55af0bbdd6b0 2023-07-21T19:19:07.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bcd99e0 con 0x55af0bd85ff0 2023-07-21T19:19:07.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bcd2e70 con 0x55af0bd85b80 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd85640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd85640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56858/0 (socket says 172.21.15.5:56858) 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50588/0 (socket says 172.21.15.5:50588) 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a67fff700 1 -- 172.21.15.5:0/2059078201 learned_addr learned my addr 172.21.15.5:0/2059078201 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.846+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd89dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2059078201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 msgr2=0x55af0bd89dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd89dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2059078201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 msgr2=0x55af0bd85640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd85640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/2059078201 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0bb00d80 con 0x55af0bd85ff0 2023-07-21T19:19:07.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 0x7f6a500a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b650 tx=0x7f6a5c0053c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c9fc0 tx=0x7f6a60095700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3cf432a494b938a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.847+0000 7f6a4e7fc700 1 -- 172.21.15.5:0/2059078201 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60099330 con 0x55af0bd85ff0 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a4e7fc700 1 -- 172.21.15.5:0/2059078201 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a600999d0 con 0x55af0bd85ff0 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a4e7fc700 1 -- 172.21.15.5:0/2059078201 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a60099e70 con 0x55af0bd85ff0 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2059078201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 msgr2=0x55af0bd8a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6a600c9fc0 tx=0x7f6a60095700 comp rx=0 tx=0).stop 2023-07-21T19:19:07.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2059078201 shutdown_connections 2023-07-21T19:19:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8a300 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd85640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2059078201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd89dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.848+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2059078201 >> 172.21.15.5:0/2059078201 conn(0x55af0bd76be0 msgr2=0x55af0bd7bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.849+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2059078201 shutdown_connections 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.849+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2059078201 wait complete. 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.849+0000 7f6a81f87b80 1 -- start start 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 Processor -- start 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd95060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bc72a60 con 0x55af0bbdd6b0 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bc72760 con 0x55af0bd85ff0 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.850+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bb568c0 con 0x55af0bd85b80 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a81f87b80 10 client.0 _ll_register_callbacks cb 0x55af0bc70700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56870/0 (socket says 172.21.15.5:56870) 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 -- 172.21.15.5:0/2629719192 learned_addr learned my addr 172.21.15.5:0/2629719192 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd95060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 -- 172.21.15.5:0/2629719192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 msgr2=0x55af0bd95060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd95060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 -- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 msgr2=0x55af0bd8ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a67fff700 1 -- 172.21.15.5:0/2629719192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a681a2b20 con 0x55af0bbdd6b0 2023-07-21T19:19:07.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.851+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a67fff700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6a580128d0 tx=0x7f6a58012be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a807c52f0c28c622 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a477fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a58017070 con 0x55af0bbdd6b0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f6a58004050 con 0x55af0bbdd6b0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55af0bc2c490 con 0x55af0bbdd6b0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a58004210 con 0x55af0bbdd6b0 2023-07-21T19:19:07.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a81f87b80 10 client.17925 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55af0bae9c00 con 0x55af0bbdd6b0 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.852+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55af0bc2c490 con 0x55af0bbdd6b0 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a5800f560 con 0x55af0bbdd6b0 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 10 client.17925 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 10 client.17925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 20 client.17925 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bd9f000 0x55af0bda1410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 4 client.17925 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:07.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55af0bccbe50 con 0x55af0bd9f000 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.853+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bd9f000 0x55af0bda1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.854+0000 7f6a477fe700 10 client.17925 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.854+0000 7f6a677fe700 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bd9f000 0x55af0bda1410 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8d79519ffa0d66e server_cookie=795047c3a5efcf57 in_seq=0 out_seq=0 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a477fe700 1 -- 172.21.15.5:0/2629719192 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55af0bccbe50 con 0x55af0bd9f000 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a477fe700 10 client.17925 handle_command_reply: tid=0 2023-07-21T19:19:07.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 client.17925 shutdown 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 msgr2=0x55af0bd94b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f6a580128d0 tx=0x7f6a58012be0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 shutdown_connections 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bd9f000 0x55af0bda1410 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bbdd6b0 0x55af0bd94b20 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/2629719192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bd85b80 0x55af0bd95060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a477fe700 0 client.17925 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 >> 172.21.15.5:0/2629719192 conn(0x55af0bd76be0 msgr2=0x55af0bd7b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 shutdown_connections 2023-07-21T19:19:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.855+0000 7f6a81f87b80 1 -- 172.21.15.5:0/2629719192 wait complete. 2023-07-21T19:19:07.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.856+0000 7f6a81f87b80 20 client.17925 trim_cache size 0 max 16384 2023-07-21T19:19:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.875+0000 7f6a81f87b80 1 Processor -- start 2023-07-21T19:19:07.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.875+0000 7f6a81f87b80 1 -- start start 2023-07-21T19:19:07.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd8ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bd8ab40 con 0x55af0bd85b80 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bccbe50 con 0x55af0bd85ff0 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bc2c490 con 0x55af0bbdd6b0 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56886/0 (socket says 172.21.15.5:56886) 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a677fe700 1 -- 172.21.15.5:0/4069478067 learned_addr learned my addr 172.21.15.5:0/4069478067 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.876+0000 7f6a67fff700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd8ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 -- 172.21.15.5:0/4069478067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 msgr2=0x55af0bd8ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd8ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 -- 172.21.15.5:0/4069478067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 msgr2=0x55af0bd8dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 -- 172.21.15.5:0/4069478067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55af0bda6020 con 0x55af0bd85b80 2023-07-21T19:19:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:07.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a677fe700 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6a68068980 tx=0x7f6a5c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b69e72dbe10847 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a4effd700 1 -- 172.21.15.5:0/4069478067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c02d070 con 0x55af0bd85b80 2023-07-21T19:19:07.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a4effd700 1 -- 172.21.15.5:0/4069478067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a5c01e030 con 0x55af0bd85b80 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.877+0000 7f6a4effd700 1 -- 172.21.15.5:0/4069478067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a5c031780 con 0x55af0bd85b80 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 -- 172.21.15.5:0/4069478067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 msgr2=0x55af0bd8a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f6a68068980 tx=0x7f6a5c026000 comp rx=0 tx=0).stop 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 -- 172.21.15.5:0/4069478067 shutdown_connections 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85ff0 0x55af0bd8dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd8ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/4069478067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85b80 0x55af0bd8a270 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 -- 172.21.15.5:0/4069478067 >> 172.21.15.5:0/4069478067 conn(0x55af0bd76be0 msgr2=0x55af0bd8d540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 -- 172.21.15.5:0/4069478067 shutdown_connections 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.878+0000 7f6a81f87b80 1 -- 172.21.15.5:0/4069478067 wait complete. 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 -- start start 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 Processor -- start 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd92850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 0x55af0bd92d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bd935f0 con 0x55af0bd85ff0 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bdf23e0 con 0x55af0bd85b80 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55af0bdf2520 con 0x55af0bbdd6b0 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.879+0000 7f6a81f87b80 10 client.0 _ll_register_callbacks cb 0x7f6a58009ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56900/0 (socket says 172.21.15.5:56900) 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/3537069177 learned_addr learned my addr 172.21.15.5:0/3537069177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a67fff700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd92850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a677fe700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 0x55af0bd92d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/3537069177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 msgr2=0x55af0bd92850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd92850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 msgr2=0x55af0bd92d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:07.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 0x55af0bd92d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 -- 172.21.15.5:0/3537069177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55af0bb00d80 con 0x55af0bd85ff0 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a677fe700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 0x55af0bd92d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a6cb9c700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6a60095ed0 tx=0x7f6a60099280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f96e408d84ee7a8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.880+0000 7f6a477fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a6009da00 con 0x55af0bd85ff0 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f6a60097b10 con 0x55af0bd85ff0 2023-07-21T19:19:07.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6a600999d0 con 0x55af0bd85ff0 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6a600a1d90 con 0x55af0bd85ff0 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 10 client.17931 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f6a5c020040 con 0x55af0bd85ff0 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6a600999d0 con 0x55af0bd85ff0 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6a600b9080 con 0x55af0bd85ff0 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 10 client.17931 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:07.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 10 client.17931 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 20 client.17931 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bdf36e0 0x55af0bdf5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 4 client.17931 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.881+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55af0bc8dff0 con 0x55af0bdf36e0 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.882+0000 7f6a67fff700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bdf36e0 0x55af0bdf5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:07.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.882+0000 7f6a67fff700 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bdf36e0 0x55af0bdf5af0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bfc308ae3bdea02 server_cookie=9fd5ffab35b31992 in_seq=0 out_seq=0 2023-07-21T19:19:07.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.882+0000 7f6a477fe700 10 client.17931 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.897+0000 7f6a477fe700 1 -- 172.21.15.5:0/3537069177 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55af0bc8dff0 con 0x55af0bdf36e0 2023-07-21T19:19:07.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.897+0000 7f6a477fe700 10 client.17931 handle_command_reply: tid=0 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.897+0000 7f6a81f87b80 1 client.17931 shutdown 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 msgr2=0x55af0bdf1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7f6a60095ed0 tx=0x7f6a60099280 comp rx=0 tx=0).stop 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 shutdown_connections 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55af0bdf36e0 0x55af0bdf5af0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55af0bd85ff0 0x55af0bdf1ea0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55af0bbdd6b0 0x55af0bd92850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 --2- 172.21.15.5:0/3537069177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55af0bd85b80 0x55af0bd92d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a477fe700 0 client.17931 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 >> 172.21.15.5:0/3537069177 conn(0x55af0bd76be0 msgr2=0x55af0bd8a0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 shutdown_connections 2023-07-21T19:19:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 1 -- 172.21.15.5:0/3537069177 wait complete. 2023-07-21T19:19:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.898+0000 7f6a81f87b80 20 client.17931 trim_cache size 0 max 16384 2023-07-21T19:19:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:19:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:19:07.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:19:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:19:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 msgr2=0x7f6a500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 0x7f6a500a3770 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f6a5c00b650 tx=0x7f6a5c0053c0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 msgr2=0x7f6a68195990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f6a6000ef30 tx=0x7f6a6000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 shutdown_connections 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6a500a12c0 0x7f6a500a3770 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6a68104e50 0x7f6a68195990 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6a68067c10 0x7f6a6819bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 --2- 172.21.15.5:0/2616113598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6a680685b0 0x7f6a6819c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 >> 172.21.15.5:0/2616113598 conn(0x7f6a68074760 msgr2=0x7f6a680fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 shutdown_connections 2023-07-21T19:19:07.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:07.899+0000 7f6a6e5ff700 1 -- 172.21.15.5:0/2616113598 wait complete. 2023-07-21T19:19:08.246 INFO:teuthology.orchestra.run.smithi005.stderr:44000000 of 50000000 tuples (88%) done (elapsed 665.82 s, remaining 90.79 s) 2023-07-21T19:19:08.249 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:19:08.249 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:19:08.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 -- 172.21.15.5:0/1959578540 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8102900 msgr2=0x7fcbf81068c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 --2- 172.21.15.5:0/1959578540 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8102900 0x7fcbf81068c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4009990 tx=0x7fcbe4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 -- 172.21.15.5:0/1959578540 shutdown_connections 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 --2- 172.21.15.5:0/1959578540 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8106fa0 0x7fcbf810eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 --2- 172.21.15.5:0/1959578540 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8101f60 0x7fcbf8102330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 --2- 172.21.15.5:0/1959578540 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8102900 0x7fcbf81068c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.494+0000 7fcbfebff700 1 -- 172.21.15.5:0/1959578540 >> 172.21.15.5:0/1959578540 conn(0x7fcbf80fd7d0 msgr2=0x7fcbf80ffbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.495+0000 7fcbfebff700 1 -- 172.21.15.5:0/1959578540 shutdown_connections 2023-07-21T19:19:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.495+0000 7fcbfebff700 1 -- 172.21.15.5:0/1959578540 wait complete. 2023-07-21T19:19:08.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.496+0000 7fcbfebff700 1 Processor -- start 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.496+0000 7fcbfebff700 1 -- start start 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.496+0000 7fcbfebff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.496+0000 7fcbfebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8102900 0x7fcbf8198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 0x7fcbf819c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfebff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf819cd30 con 0x7fcbf8101f60 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf819cea0 con 0x7fcbf8106fa0 2023-07-21T19:19:08.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbf819d060 con 0x7fcbf8102900 2023-07-21T19:19:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56910/0 (socket says 172.21.15.5:56910) 2023-07-21T19:19:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 -- 172.21.15.5:0/110313047 learned_addr learned my addr 172.21.15.5:0/110313047 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8102900 0x7fcbf8198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 0x7fcbf819c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 -- 172.21.15.5:0/110313047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8102900 msgr2=0x7fcbf8198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8102900 0x7fcbf8198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 -- 172.21.15.5:0/110313047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 msgr2=0x7fcbf819c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 0x7fcbf819c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfc99b700 1 -- 172.21.15.5:0/110313047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe40095f0 con 0x7fcbf8101f60 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.497+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 0x7fcbf819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e9a0 tx=0x7fcbec00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbf5ffb700 1 -- 172.21.15.5:0/110313047 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec009790 con 0x7fcbf8101f60 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbf819d340 con 0x7fcbf8101f60 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbf5ffb700 1 -- 172.21.15.5:0/110313047 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbec010430 con 0x7fcbf8101f60 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbf5ffb700 1 -- 172.21.15.5:0/110313047 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec00cea0 con 0x7fcbf8101f60 2023-07-21T19:19:08.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.498+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf81a4b80 con 0x7fcbf8101f60 2023-07-21T19:19:08.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.502+0000 7fcbf5ffb700 1 -- 172.21.15.5:0/110313047 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcbec0105a0 con 0x7fcbf8101f60 2023-07-21T19:19:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.504+0000 7fcbf5ffb700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 0x7fcbe80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.504+0000 7fcc1250bb80 1 Processor -- start 2023-07-21T19:19:08.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.504+0000 7fcbf5ffb700 1 -- 172.21.15.5:0/110313047 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcbec014070 con 0x7fcbf8101f60 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.504+0000 7fcc1250bb80 1 -- start start 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.504+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 0x7fcbe80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa68610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa6af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f963d30 con 0x55692fa5d350 2023-07-21T19:19:08.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f9b6ac0 con 0x55692fa68b50 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f9aff50 con 0x55692fa5dc60 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56922/0 (socket says 172.21.15.5:56922) 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbfc99b700 1 -- 172.21.15.5:0/1323506417 learned_addr learned my addr 172.21.15.5:0/1323506417 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 0x7fcbe80a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4000c00 tx=0x7fcbe400b280 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa6af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.505+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa68610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 -- 172.21.15.5:0/1323506417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 msgr2=0x55692fa68610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa68610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 -- 172.21.15.5:0/1323506417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 msgr2=0x55692fa6af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa6af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 -- 172.21.15.5:0/1323506417 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55692f7dde60 con 0x55692fa5d350 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa6af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcbec096ab0 tx=0x7fcbec099c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff6ba2cae96ec7d0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbe2ffd700 1 -- 172.21.15.5:0/1323506417 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec09e070 con 0x55692fa5d350 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbe2ffd700 1 -- 172.21.15.5:0/1323506417 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbec0a2490 con 0x55692fa5d350 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.506+0000 7fcbe2ffd700 1 -- 172.21.15.5:0/1323506417 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0a2000 con 0x55692fa5d350 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 -- 172.21.15.5:0/1323506417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 msgr2=0x55692fa5d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fcbec096ab0 tx=0x7fcbec099c90 comp rx=0 tx=0).stop 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 -- 172.21.15.5:0/1323506417 shutdown_connections 2023-07-21T19:19:08.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa6af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa5d720 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/1323506417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa68610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 -- 172.21.15.5:0/1323506417 >> 172.21.15.5:0/1323506417 conn(0x55692fa53e20 msgr2=0x55692fa59720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 -- 172.21.15.5:0/1323506417 shutdown_connections 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.507+0000 7fcc1250bb80 1 -- 172.21.15.5:0/1323506417 wait complete. 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.508+0000 7fcc1250bb80 1 -- start start 2023-07-21T19:19:08.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.508+0000 7fcc1250bb80 1 Processor -- start 2023-07-21T19:19:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fa65910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 0x55692fa65e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa66390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f94fb40 con 0x55692fa5dc60 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f94f840 con 0x55692fa68b50 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692f8339a0 con 0x55692fa5d350 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcc1250bb80 10 client.0 _ll_register_callbacks cb 0x55692f94d7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 0x55692fa65e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 0x55692fa65e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56938/0 (socket says 172.21.15.5:56938) 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcbf7fff700 1 -- 172.21.15.5:0/2337376574 learned_addr learned my addr 172.21.15.5:0/2337376574 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fa65910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.509+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa66390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 -- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 msgr2=0x55692fa66390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa66390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 -- 172.21.15.5:0/2337376574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 msgr2=0x55692fa65e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 0x55692fa65e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 -- 172.21.15.5:0/2337376574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf0009d10 con 0x55692fa5d350 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fa65910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbec096ab0 tx=0x7fcbec010bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=526cbb4377430728 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec09e070 con 0x55692fa5d350 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcbec0a2210 con 0x55692fa5d350 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.510+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55692f909570 con 0x55692fa5d350 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbec0b8200 con 0x55692fa5d350 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 10 client.27538 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fcbf81a4b80 con 0x55692fa5d350 2023-07-21T19:19:08.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbec063080 con 0x55692fa5d350 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbec095e30 con 0x55692fa5d350 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 10 client.27538 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 10 client.27538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 20 client.27538 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692fa780b0 0x55692fa7a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 4 client.27538 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.511+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55692f9a8f30 con 0x55692fa780b0 2023-07-21T19:19:08.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.512+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692fa780b0 0x55692fa7a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.512+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692fa780b0 0x55692fa7a4c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68fae475bb6fde64 server_cookie=b052c659d3702612 in_seq=0 out_seq=0 2023-07-21T19:19:08.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.512+0000 7fcbd7fff700 10 client.27538 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.513+0000 7fcbd7fff700 1 -- 172.21.15.5:0/2337376574 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55692f9a8f30 con 0x55692fa780b0 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.513+0000 7fcbd7fff700 10 client.27538 handle_command_reply: tid=0 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.513+0000 7fcc1250bb80 1 client.27538 shutdown 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 msgr2=0x55692fa65910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fa65910 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcbec096ab0 tx=0x7fcbec010bf0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 shutdown_connections 2023-07-21T19:19:08.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa66390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fa65910 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692fa780b0 0x55692fa7a4c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/2337376574 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5dc60 0x55692fa65e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcbd7fff700 0 client.27538 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 >> 172.21.15.5:0/2337376574 conn(0x55692fa53e20 msgr2=0x55692fa592b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 shutdown_connections 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 1 -- 172.21.15.5:0/2337376574 wait complete. 2023-07-21T19:19:08.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.514+0000 7fcc1250bb80 20 client.27538 trim_cache size 0 max 16384 2023-07-21T19:19:08.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 Processor -- start 2023-07-21T19:19:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 -- start start 2023-07-21T19:19:08.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa5fb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa62be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fa66e80 con 0x55692fa5d350 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fa539f0 con 0x55692fa68b50 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.534+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fa63eb0 con 0x55692fa5dc60 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa5fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56954/0 (socket says 172.21.15.5:56954) 2023-07-21T19:19:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 -- 172.21.15.5:0/4245004764 learned_addr learned my addr 172.21.15.5:0/4245004764 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa62be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 -- 172.21.15.5:0/4245004764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 msgr2=0x55692fa5fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa5fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 -- 172.21.15.5:0/4245004764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 msgr2=0x55692fa62be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa62be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfc99b700 1 -- 172.21.15.5:0/4245004764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55692fa55660 con 0x55692fa5d350 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.535+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa62be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.536+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00c2d0 tx=0x7fcbec0cac20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c18b03ba152b85a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.536+0000 7fcbe37fe700 1 -- 172.21.15.5:0/4245004764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec098fa0 con 0x55692fa5d350 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.536+0000 7fcbe37fe700 1 -- 172.21.15.5:0/4245004764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55692f909570 con 0x55692fa5d350 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.536+0000 7fcbe37fe700 1 -- 172.21.15.5:0/4245004764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbec0b2a20 con 0x55692fa5d350 2023-07-21T19:19:08.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 -- 172.21.15.5:0/4245004764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 msgr2=0x55692fa6f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00c2d0 tx=0x7fcbec0cac20 comp rx=0 tx=0).stop 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 -- 172.21.15.5:0/4245004764 shutdown_connections 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa68b50 0x55692fa62be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa5d350 0x55692fa6f530 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/4245004764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5dc60 0x55692fa5fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 -- 172.21.15.5:0/4245004764 >> 172.21.15.5:0/4245004764 conn(0x55692fa53e20 msgr2=0x55692fa5f460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 -- 172.21.15.5:0/4245004764 shutdown_connections 2023-07-21T19:19:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.537+0000 7fcc1250bb80 1 -- 172.21.15.5:0/4245004764 wait complete. 2023-07-21T19:19:08.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.538+0000 7fcc1250bb80 1 -- start start 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.538+0000 7fcc1250bb80 1 Processor -- start 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.538+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fac5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.538+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 0x55692fac63f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcc1250bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fac07f0 con 0x55692fa68b50 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fac0960 con 0x55692fa5dc60 2023-07-21T19:19:08.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcc1250bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55692fac0b20 con 0x55692fa5d350 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcc1250bb80 10 client.0 _ll_register_callbacks cb 0x7fcbf0009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbf7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 0x55692fac63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56956/0 (socket says 172.21.15.5:56956) 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 -- 172.21.15.5:0/809232855 learned_addr learned my addr 172.21.15.5:0/809232855 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fac5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 -- 172.21.15.5:0/809232855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 msgr2=0x55692fac5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fac5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 -- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 msgr2=0x55692fac63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 0x55692fac63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfd19c700 1 -- 172.21.15.5:0/809232855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbf810f3b0 con 0x55692fa68b50 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fac5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.539+0000 7fcbf7fff700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 0x55692fac63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcbfd19c700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0003dd0 tx=0x7fcbf0003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=733cee197da8e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcbd7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf0007e40 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcbf0021440 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcbec0a2490 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcbf0021640 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcc1250bb80 10 client.17949 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.540+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fcbf0003a00 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcbec0a2490 con 0x55692fa68b50 2023-07-21T19:19:08.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcbf0017790 con 0x55692fa68b50 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 10 client.17949 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 10 client.17949 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 20 client.17949 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692facddc0 0x55692fad01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 4 client.17949 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55692f96b0d0 con 0x55692facddc0 2023-07-21T19:19:08.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.541+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692facddc0 0x55692fad01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.542+0000 7fcbd7fff700 10 client.17949 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.542+0000 7fcbfc99b700 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692facddc0 0x55692fad01d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dcccec5730308ac5 server_cookie=25de66e640f21aa0 in_seq=0 out_seq=0 2023-07-21T19:19:08.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcbd7fff700 1 -- 172.21.15.5:0/809232855 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55692f96b0d0 con 0x55692facddc0 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcbd7fff700 10 client.17949 handle_command_reply: tid=0 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 client.17949 shutdown 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 msgr2=0x55692fac0210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fcbf0003dd0 tx=0x7fcbf0003e00 comp rx=0 tx=0).stop 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 shutdown_connections 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55692facddc0 0x55692fad01d0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55692fa68b50 0x55692fac0210 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55692fa5d350 0x55692fac5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcbd7fff700 0 client.17949 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 --2- 172.21.15.5:0/809232855 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55692fa5dc60 0x55692fac63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.543+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 >> 172.21.15.5:0/809232855 conn(0x55692fa53e20 msgr2=0x55692fa5fcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 shutdown_connections 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcc1250bb80 1 -- 172.21.15.5:0/809232855 wait complete. 2023-07-21T19:19:08.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcc1250bb80 20 client.17949 trim_cache size 0 max 16384 2023-07-21T19:19:08.545 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:08.545 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:19:08.545 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d40e713a-0c78-4d87-8252-5cbfdd3f2304", 2023-07-21T19:19:08.545 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:19:08.545 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 msgr2=0x7fcbe80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 0x7fcbe80a3770 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fcbe4000c00 tx=0x7fcbe400b280 comp rx=0 tx=0).stop 2023-07-21T19:19:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 msgr2=0x7fcbf8197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.544+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fcbec00e9a0 tx=0x7fcbec00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 shutdown_connections 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcbf8106fa0 0x7fcbf819c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcbe80a12c0 0x7fcbe80a3770 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcbf8101f60 0x7fcbf8197d50 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 --2- 172.21.15.5:0/110313047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcbf8102900 0x7fcbf8198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 >> 172.21.15.5:0/110313047 conn(0x7fcbf80fd7d0 msgr2=0x7fcbf80fe2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 shutdown_connections 2023-07-21T19:19:08.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.545+0000 7fcbfebff700 1 -- 172.21.15.5:0/110313047 wait complete. 2023-07-21T19:19:08.561 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:19:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 -- 172.21.15.5:0/4006014981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c105ae0 msgr2=0x7f2d5c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 --2- 172.21.15.5:0/4006014981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c105eb0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c009990 tx=0x7f2d4c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 -- 172.21.15.5:0/4006014981 shutdown_connections 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 --2- 172.21.15.5:0/4006014981 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c100480 0x7f2d5c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 --2- 172.21.15.5:0/4006014981 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c105eb0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 --2- 172.21.15.5:0/4006014981 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.815+0000 7f2d617ff700 1 -- 172.21.15.5:0/4006014981 >> 172.21.15.5:0/4006014981 conn(0x7f2d5c0fb5f0 msgr2=0x7f2d5c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.816+0000 7f2d617ff700 1 -- 172.21.15.5:0/4006014981 shutdown_connections 2023-07-21T19:19:08.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.816+0000 7f2d617ff700 1 -- 172.21.15.5:0/4006014981 wait complete. 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 Processor -- start 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 -- start start 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c197ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c198130 con 0x7f2d5c100480 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c1982a0 con 0x7f2d5c105ae0 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.817+0000 7f2d617ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d5c198460 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56976/0 (socket says 172.21.15.5:56976) 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/889516991 learned_addr learned my addr 172.21.15.5:0/889516991 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c197ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 -- 172.21.15.5:0/889516991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c105ae0 msgr2=0x7f2d5c197ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c197ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 -- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 msgr2=0x7f2d5c19df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 -- 172.21.15.5:0/889516991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d4c0095f0 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.818+0000 7f2d5affd700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c19da00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0057c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.819+0000 7f2d43fff700 1 -- 172.21.15.5:0/889516991 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c01d070 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.819+0000 7f2d43fff700 1 -- 172.21.15.5:0/889516991 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4c021410 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.819+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d5c1986e0 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.819+0000 7f2d43fff700 1 -- 172.21.15.5:0/889516991 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c017650 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.819+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5c1a4aa0 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.822+0000 7f2d43fff700 1 -- 172.21.15.5:0/889516991 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2d4c021580 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.824+0000 7f2d75130b80 1 Processor -- start 2023-07-21T19:19:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.824+0000 7f2d43fff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 0x7f2d440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.824+0000 7f2d75130b80 1 -- start start 2023-07-21T19:19:08.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.824+0000 7f2d43fff700 1 -- 172.21.15.5:0/889516991 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f2d4c0cdf70 con 0x7f2d5c0ffad0 2023-07-21T19:19:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.824+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 0x7f2d440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a730686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73068be0 0x561a73073590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72f6ecb0 con 0x561a73073ad0 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72fc1a40 con 0x561a73068be0 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d75130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72fbaed0 con 0x561a730682d0 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56984/0 (socket says 172.21.15.5:56984) 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d5b7fe700 1 -- 172.21.15.5:0/2435414983 learned_addr learned my addr 172.21.15.5:0/2435414983 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a730686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.825+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73068be0 0x561a73073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 -- 172.21.15.5:0/2435414983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73068be0 msgr2=0x561a73073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73068be0 0x561a73073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 -- 172.21.15.5:0/2435414983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 msgr2=0x561a73075eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 -- 172.21.15.5:0/2435414983 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a72de8de0 con 0x561a730682d0 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 0x7f2d440a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400f930 tx=0x7f2d5400fcf0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a730686a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0cb640 tx=0x7f2d4c09ab90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d211b5bdc9362b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d417fa700 1 -- 172.21.15.5:0/2435414983 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c09f080 con 0x561a730682d0 2023-07-21T19:19:08.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d417fa700 1 -- 172.21.15.5:0/2435414983 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4c0a3410 con 0x561a730682d0 2023-07-21T19:19:08.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.826+0000 7f2d417fa700 1 -- 172.21.15.5:0/2435414983 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c09a540 con 0x561a730682d0 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 -- 172.21.15.5:0/2435414983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 msgr2=0x561a730686a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a730686a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0cb640 tx=0x7f2d4c09ab90 comp rx=0 tx=0).stop 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 -- 172.21.15.5:0/2435414983 shutdown_connections 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73075eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a730686a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2435414983 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73068be0 0x561a73073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 -- 172.21.15.5:0/2435414983 >> 172.21.15.5:0/2435414983 conn(0x561a7305eda0 msgr2=0x561a730646a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 -- 172.21.15.5:0/2435414983 shutdown_connections 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.827+0000 7f2d75130b80 1 -- 172.21.15.5:0/2435414983 wait complete. 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.828+0000 7f2d75130b80 1 -- start start 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.828+0000 7f2d75130b80 1 Processor -- start 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a73079120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72f5aac0 con 0x561a73073ad0 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72f5a7c0 con 0x561a730682d0 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a72e3e920 con 0x561a73068be0 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d75130b80 10 client.0 _ll_register_callbacks cb 0x561a72f58760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a73079120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a73079120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57422/0 (socket says 172.21.15.5:57422) 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50696/0 (socket says 172.21.15.5:50696) 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.829+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/2903984486 learned_addr learned my addr 172.21.15.5:0/2903984486 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 -- 172.21.15.5:0/2903984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 msgr2=0x561a73079120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a73079120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 -- 172.21.15.5:0/2903984486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 msgr2=0x561a73072aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73072aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 -- 172.21.15.5:0/2903984486 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d5400fe70 con 0x561a730682d0 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d5affd700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0993a0 tx=0x7f2d4c021b30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df1e582085e1d1e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d4c09f080 con 0x561a730682d0 2023-07-21T19:19:08.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f2d4c09a750 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561a72f144f0 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d4c0cb330 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d75130b80 10 client.27635 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.830+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f2d5c1a4aa0 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4c07d020 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d4c0a2f80 con 0x561a730682d0 2023-07-21T19:19:08.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 10 client.27635 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 10 client.27635 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 20 client.27635 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a73082c90 0x561a73085070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 4 client.27635 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561a72fb3eb0 con 0x561a73082c90 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.831+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a73082c90 0x561a73085070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.832+0000 7f2d3eff5700 10 client.27635 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.834 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.832+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a73082c90 0x561a73085070 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1efe9ec040d6c697 server_cookie=12669215ce67d0e3 in_seq=0 out_seq=0 2023-07-21T19:19:08.834 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:19:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:19:08.562+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...12,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v3738 ap=2 f(v18 m2023-07-21T19:18:26.344563+0000 3=1+2) n(v9 rc2023-07-21T19:18:26.344563+0000 b8 4=1+3) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@110} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v3738>,<0x10000000005/data v2731>,<0x10000000003/postgres v721>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v745>,<0x10000000001/sv_0 v758>,<0x10000000000/_nogroup v736>,<0x1/volumes v700>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:18:26.326810+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:18:26.344563+0000 b8 4=1+3)","memory_value.dirstat":"f(v18 m2023-07-21T19:18:26.344563+0000 3=1+2)","memory_value.rstat":"n(v9 rc2023-07-21T19:18:26.344563+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:19:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d3eff5700 1 -- 172.21.15.5:0/2903984486 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561a72fb3eb0 con 0x561a73082c90 2023-07-21T19:19:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d3eff5700 10 client.27635 handle_command_reply: tid=0 2023-07-21T19:19:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 client.27635 shutdown 2023-07-21T19:19:08.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 msgr2=0x561a73078be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c0993a0 tx=0x7f2d4c021b30 comp rx=0 tx=0).stop 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 shutdown_connections 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a73082c90 0x561a73085070 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a73072aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a73078be0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 --2- 172.21.15.5:0/2903984486 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a73079120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 >> 172.21.15.5:0/2903984486 conn(0x561a7305eda0 msgr2=0x561a73064290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d3eff5700 0 client.27635 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 shutdown_connections 2023-07-21T19:19:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 1 -- 172.21.15.5:0/2903984486 wait complete. 2023-07-21T19:19:08.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.833+0000 7f2d75130b80 20 client.27635 trim_cache size 0 max 16384 2023-07-21T19:19:08.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 Processor -- start 2023-07-21T19:19:08.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 -- start start 2023-07-21T19:19:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a73071710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73073ad0 0x561a730787b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a73060ec0 con 0x561a73068be0 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a73072190 con 0x561a73073ad0 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.853+0000 7f2d75130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a73069330 con 0x561a730682d0 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a73071710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57012/0 (socket says 172.21.15.5:57012) 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/1845393110 learned_addr learned my addr 172.21.15.5:0/1845393110 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73073ad0 0x561a730787b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/1845393110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 msgr2=0x561a73071710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a73071710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/1845393110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73073ad0 msgr2=0x561a730787b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73073ad0 0x561a730787b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/1845393110 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561a72dd1c60 con 0x561a73068be0 2023-07-21T19:19:08.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.854+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x561a73085980 tx=0x7f2d50012b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21e5d51a687c8d95 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d41ffb700 1 -- 172.21.15.5:0/1845393110 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50017040 con 0x561a73068be0 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d41ffb700 1 -- 172.21.15.5:0/1845393110 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d50025070 con 0x561a73068be0 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d41ffb700 1 -- 172.21.15.5:0/1845393110 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50004bb0 con 0x561a73068be0 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d75130b80 1 -- 172.21.15.5:0/1845393110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 msgr2=0x561a73071c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d75130b80 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x561a73085980 tx=0x7f2d50012b10 comp rx=0 tx=0).stop 2023-07-21T19:19:08.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.855+0000 7f2d75130b80 1 -- 172.21.15.5:0/1845393110 shutdown_connections 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a73073ad0 0x561a730787b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a730682d0 0x561a73071710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 --2- 172.21.15.5:0/1845393110 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73068be0 0x561a73071c50 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 -- 172.21.15.5:0/1845393110 >> 172.21.15.5:0/1845393110 conn(0x561a7305eda0 msgr2=0x561a73071000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 -- 172.21.15.5:0/1845393110 shutdown_connections 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 -- 172.21.15.5:0/1845393110 wait complete. 2023-07-21T19:19:08.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 -- start start 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.856+0000 7f2d75130b80 1 Processor -- start 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a730cede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a730cfb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a730d0150 con 0x561a73073ad0 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a730d0290 con 0x561a730682d0 2023-07-21T19:19:08.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561a730d0450 con 0x561a73068be0 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d75130b80 10 client.0 _ll_register_callbacks cb 0x7f2d5401e050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d5a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57438/0 (socket says 172.21.15.5:57438) 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/4036345598 learned_addr learned my addr 172.21.15.5:0/4036345598 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d5b7fe700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a730cfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.857+0000 7f2d5affd700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a730cede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 msgr2=0x561a730cede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a730cede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/4036345598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 msgr2=0x561a730cfb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a730cfb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 -- 172.21.15.5:0/4036345598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d50021a50 con 0x561a73068be0 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5affd700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a730cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d5a7fc700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d500096b0 tx=0x7f2d50009e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d15926b1229c579 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:08.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d3eff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d50017070 con 0x561a73068be0 2023-07-21T19:19:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(99..99 src has 1..99) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f2d500212f0 con 0x561a73068be0 2023-07-21T19:19:08.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.858+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2d4c0a3410 con 0x561a73068be0 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2d500214b0 con 0x561a73068be0 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 10 client.27562 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x561a72e0f250 con 0x561a73068be0 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2d4c0a3410 con 0x561a73068be0 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2d50030020 con 0x561a73068be0 2023-07-21T19:19:08.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 10 client.27562 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 10 client.27562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 20 client.27562 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a730d7c60 0x561a730da070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 4 client.27562 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.859+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x561a72f76050 con 0x561a730d7c60 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.860+0000 7f2d5affd700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a730d7c60 0x561a730da070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:08.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.860+0000 7f2d3eff5700 10 client.27562 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.860+0000 7f2d5affd700 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a730d7c60 0x561a730da070 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d083cfbab969e1c8 server_cookie=6f15c81e41b4ecdc in_seq=0 out_seq=0 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d3eff5700 1 -- 172.21.15.5:0/4036345598 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x561a72f76050 con 0x561a730d7c60 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d3eff5700 10 client.27562 handle_command_reply: tid=0 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 client.27562 shutdown 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 msgr2=0x561a730cf320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f2d500096b0 tx=0x7f2d50009e80 comp rx=0 tx=0).stop 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 shutdown_connections 2023-07-21T19:19:08.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561a730d7c60 0x561a730da070 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561a73073ad0 0x561a730cfb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d3eff5700 0 client.27562 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561a730682d0 0x561a730cede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 --2- 172.21.15.5:0/4036345598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561a73068be0 0x561a730cf320 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 >> 172.21.15.5:0/4036345598 conn(0x561a7305eda0 msgr2=0x561a730729d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 shutdown_connections 2023-07-21T19:19:08.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 1 -- 172.21.15.5:0/4036345598 wait complete. 2023-07-21T19:19:08.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.861+0000 7f2d75130b80 20 client.27562 trim_cache size 0 max 16384 2023-07-21T19:19:08.864 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:19:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:19:08.864 INFO:teuthology.orchestra.run.smithi005.stdout: "d40e713a-0c78-4d87-8252-5cbfdd3f2304": { 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d40e713a-0c78-4d87-8252-5cbfdd3f2304", 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:19:08.865 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 msgr2=0x7f2d440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 0x7f2d440a37f0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f2d5400f930 tx=0x7f2d5400fcf0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 msgr2=0x7f2d5c19da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:08.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c19da00 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f2d4c000c00 tx=0x7f2d4c0057c0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 shutdown_connections 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2d5c105ae0 0x7f2d5c197ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2d5c0ffad0 0x7f2d5c19da00 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2d440a1340 0x7f2d440a37f0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 --2- 172.21.15.5:0/889516991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2d5c100480 0x7f2d5c19df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 >> 172.21.15.5:0/889516991 conn(0x7f2d5c0fb5f0 msgr2=0x7f2d5c10adf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:08.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 shutdown_connections 2023-07-21T19:19:08.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:08.862+0000 7f2d617ff700 1 -- 172.21.15.5:0/889516991 wait complete. 2023-07-21T19:19:08.875 INFO:tasks.cephfs.filesystem:scrub status for tag:d40e713a-0c78-4d87-8252-5cbfdd3f2304 - {'path': '/', 'tag': 'd40e713a-0c78-4d87-8252-5cbfdd3f2304', 'options': 'recursive,force'} 2023-07-21T19:19:09.731 INFO:teuthology.orchestra.run.smithi005.stderr:44100000 of 50000000 tuples (88%) done (elapsed 667.30 s, remaining 89.28 s) 2023-07-21T19:19:09.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: pgmap v360: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:09.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:19:09.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:09.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:09 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: pgmap v360: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:09 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: pgmap v360: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:09.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:19:09.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:09.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:09 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:11.026 INFO:teuthology.orchestra.run.smithi005.stderr:44200000 of 50000000 tuples (88%) done (elapsed 668.60 s, remaining 87.73 s) 2023-07-21T19:19:11.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:11 smithi101 ceph-mon[128136]: pgmap v361: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:11.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:11 smithi005 ceph-mon[126460]: pgmap v361: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:11.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:11 smithi156 ceph-mon[128316]: pgmap v361: 97 pgs: 97 active+clean; 8.7 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 26 MiB/s wr, 87 op/s 2023-07-21T19:19:12.303 INFO:teuthology.orchestra.run.smithi005.stderr:44300000 of 50000000 tuples (88%) done (elapsed 669.87 s, remaining 86.19 s) 2023-07-21T19:19:13.701 INFO:teuthology.orchestra.run.smithi005.stderr:44400000 of 50000000 tuples (88%) done (elapsed 671.27 s, remaining 84.66 s) 2023-07-21T19:19:13.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:13 smithi101 ceph-mon[128136]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 25 MiB/s wr, 164 op/s 2023-07-21T19:19:13.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:13 smithi101 ceph-mon[128136]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T19:19:13.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:13 smithi005 ceph-mon[126460]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 25 MiB/s wr, 164 op/s 2023-07-21T19:19:13.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:13 smithi005 ceph-mon[126460]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T19:19:13.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:13 smithi156 ceph-mon[128316]: pgmap v362: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 135 KiB/s rd, 25 MiB/s wr, 164 op/s 2023-07-21T19:19:13.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:13 smithi156 ceph-mon[128316]: osdmap e100: 12 total, 12 up, 12 in 2023-07-21T19:19:14.987 INFO:teuthology.orchestra.run.smithi005.stderr:44500000 of 50000000 tuples (89%) done (elapsed 672.56 s, remaining 83.13 s) 2023-07-21T19:19:15.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:15 smithi101 ceph-mon[128136]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 31 MiB/s wr, 213 op/s 2023-07-21T19:19:15.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:15 smithi005 ceph-mon[126460]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 31 MiB/s wr, 213 op/s 2023-07-21T19:19:15.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:15 smithi156 ceph-mon[128316]: pgmap v364: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 31 MiB/s wr, 213 op/s 2023-07-21T19:19:16.280 INFO:teuthology.orchestra.run.smithi005.stderr:44600000 of 50000000 tuples (89%) done (elapsed 673.85 s, remaining 81.59 s) 2023-07-21T19:19:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:17.584 INFO:teuthology.orchestra.run.smithi005.stderr:44700000 of 50000000 tuples (89%) done (elapsed 675.16 s, remaining 80.05 s) 2023-07-21T19:19:17.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:17 smithi101 ceph-mon[128136]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 25 MiB/s wr, 175 op/s 2023-07-21T19:19:17.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:17 smithi005 ceph-mon[126460]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 25 MiB/s wr, 175 op/s 2023-07-21T19:19:17.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:17 smithi156 ceph-mon[128316]: pgmap v365: 97 pgs: 97 active+clean; 8.8 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 25 MiB/s wr, 175 op/s 2023-07-21T19:19:19.007 INFO:teuthology.orchestra.run.smithi005.stderr:44800000 of 50000000 tuples (89%) done (elapsed 676.58 s, remaining 78.53 s) 2023-07-21T19:19:19.503 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:19 smithi005 ceph-mon[126460]: pgmap v366: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:19.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:19 smithi101 ceph-mon[128136]: pgmap v366: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:19.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:19 smithi156 ceph-mon[128316]: pgmap v366: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:20.359 INFO:teuthology.orchestra.run.smithi005.stderr:44900000 of 50000000 tuples (89%) done (elapsed 677.93 s, remaining 77.00 s) 2023-07-21T19:19:21.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3758846279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d48104a30 msgr2=0x7f6d48104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3758846279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d48104a30 0x7f6d48104e00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6d38009990 tx=0x7f6d38009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3758846279 shutdown_connections 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3758846279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d480ff3e0 0x7f6d4810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3758846279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d48104a30 0x7f6d48104e00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3758846279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480fea30 0x7f6d480feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3758846279 >> 172.21.15.5:0/3758846279 conn(0x7f6d48074760 msgr2=0x7f6d48074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3758846279 shutdown_connections 2023-07-21T19:19:21.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.072+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3758846279 wait complete. 2023-07-21T19:19:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.073+0000 7f6d4ddff700 1 Processor -- start 2023-07-21T19:19:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.073+0000 7f6d4ddff700 1 -- start start 2023-07-21T19:19:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480ff3e0 0x7f6d48198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 0x7f6d4819c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819cf10 con 0x7f6d48104a30 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819d080 con 0x7f6d480ff3e0 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d4ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d4819d240 con 0x7f6d480fea30 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d47fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 0x7f6d4819c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33178/0 (socket says 172.21.15.5:33178) 2023-07-21T19:19:21.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d477fe700 1 -- 172.21.15.5:0/3560813274 learned_addr learned my addr 172.21.15.5:0/3560813274 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.074+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480ff3e0 0x7f6d48198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 -- 172.21.15.5:0/3560813274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480ff3e0 msgr2=0x7f6d48198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480ff3e0 0x7f6d48198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 -- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 msgr2=0x7f6d4819c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 0x7f6d4819c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 -- 172.21.15.5:0/3560813274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d380095f0 con 0x7f6d480fea30 2023-07-21T19:19:21.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 0x7f6d4819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d477fe700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d380098e0 tx=0x7f6d38004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d44ff9700 1 -- 172.21.15.5:0/3560813274 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3801d070 con 0x7f6d480fea30 2023-07-21T19:19:21.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d44ff9700 1 -- 172.21.15.5:0/3560813274 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d38022470 con 0x7f6d480fea30 2023-07-21T19:19:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d4819d4c0 con 0x7f6d480fea30 2023-07-21T19:19:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d44ff9700 1 -- 172.21.15.5:0/3560813274 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d38017650 con 0x7f6d480fea30 2023-07-21T19:19:21.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.075+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d481a4d60 con 0x7f6d480fea30 2023-07-21T19:19:21.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.080+0000 7f6d44ff9700 1 -- 172.21.15.5:0/3560813274 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6d380225e0 con 0x7f6d480fea30 2023-07-21T19:19:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.081+0000 7f6d617fcb80 1 Processor -- start 2023-07-21T19:19:21.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 -- start start 2023-07-21T19:19:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x561455925d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x561455920040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561455825c50 con 0x561455920580 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614558789e0 con 0x5614559262d0 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561455871e70 con 0x56145577c6b0 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.082+0000 7f6d44ff9700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 0x7f6d300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d44ff9700 1 -- 172.21.15.5:0/3560813274 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6d380ce0e0 con 0x7f6d480fea30 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50298/0 (socket says 172.21.15.5:50298) 2023-07-21T19:19:21.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d46ffd700 1 -- 172.21.15.5:0/2583263787 learned_addr learned my addr 172.21.15.5:0/2583263787 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 0x7f6d300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x561455920040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d47fff700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x561455925d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 -- 172.21.15.5:0/2583263787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 msgr2=0x561455925d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x561455925d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 -- 172.21.15.5:0/2583263787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 msgr2=0x5614559209d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d477fe700 1 -- 172.21.15.5:0/2583263787 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56145569fd80 con 0x5614559262d0 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.083+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.084+0000 7f6d47fff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 0x7f6d300a37f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d400088a0 tx=0x7f6d400092d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.084+0000 7f6d477fe700 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x561455920040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d380cb7d0 tx=0x7f6d38005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3c49a5df1d56e59 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.084+0000 7f6d2dffb700 1 -- 172.21.15.5:0/2583263787 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809ab40 con 0x5614559262d0 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.084+0000 7f6d2dffb700 1 -- 172.21.15.5:0/2583263787 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d380a10b0 con 0x5614559262d0 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.084+0000 7f6d2dffb700 1 -- 172.21.15.5:0/2583263787 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809b6a0 con 0x5614559262d0 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 -- 172.21.15.5:0/2583263787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 msgr2=0x561455920040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x561455920040 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6d380cb7d0 tx=0x7f6d38005f10 comp rx=0 tx=0).stop 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 -- 172.21.15.5:0/2583263787 shutdown_connections 2023-07-21T19:19:21.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x5614559209d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x561455925d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/2583263787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x561455920040 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 -- 172.21.15.5:0/2583263787 >> 172.21.15.5:0/2583263787 conn(0x561455915be0 msgr2=0x56145591af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 -- 172.21.15.5:0/2583263787 shutdown_connections 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.085+0000 7f6d617fcb80 1 -- 172.21.15.5:0/2583263787 wait complete. 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.086+0000 7f6d617fcb80 1 -- start start 2023-07-21T19:19:21.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.086+0000 7f6d617fcb80 1 Processor -- start 2023-07-21T19:19:21.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561455920580 0x56145592f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x5614559294a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561455811a60 con 0x56145577c6b0 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561455811760 con 0x5614559262d0 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614556f58c0 con 0x561455920580 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d617fcb80 10 client.0 _ll_register_callbacks cb 0x56145580f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:21.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50308/0 (socket says 172.21.15.5:50308) 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561455920580 0x56145592f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.087+0000 7f6d477fe700 1 -- 172.21.15.5:0/1680647646 learned_addr learned my addr 172.21.15.5:0/1680647646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 -- 172.21.15.5:0/1680647646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561455920580 msgr2=0x56145592f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561455920580 0x56145592f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x5614559294a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 -- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 msgr2=0x5614559294a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x5614559294a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 -- 172.21.15.5:0/1680647646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c009910 con 0x56145577c6b0 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x5614559294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809b170 tx=0x7f6d3809b670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2669863f1a5ab08 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809ab40 con 0x56145577c6b0 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6d380228c0 con 0x56145577c6b0 2023-07-21T19:19:21.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614557cb490 con 0x56145577c6b0 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d380a35a0 con 0x56145577c6b0 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d617fcb80 10 client.17979 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.088+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f6d481a4d60 con 0x56145577c6b0 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5614557cb490 con 0x56145577c6b0 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d380970c0 con 0x56145577c6b0 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 10 client.17979 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 10 client.17979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:21.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 20 client.17979 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5614559392c0 0x56145593b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 4 client.17979 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.089+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56145586ae50 con 0x5614559392c0 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.090+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5614559392c0 0x56145593b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.091+0000 7f6d26ffd700 10 client.17979 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:21.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.091+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5614559392c0 0x56145593b6a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86300d0d6093c1f1 server_cookie=9f17b6c61f64d79d in_seq=0 out_seq=0 2023-07-21T19:19:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1680647646 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56145586ae50 con 0x5614559392c0 2023-07-21T19:19:21.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d26ffd700 10 client.17979 handle_command_reply: tid=0 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 client.17979 shutdown 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 msgr2=0x56145592f210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f6d3809b170 tx=0x7f6d3809b670 comp rx=0 tx=0).stop 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 shutdown_connections 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5614559262d0 0x5614559294a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5614559392c0 0x56145593b6a0 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56145577c6b0 0x56145592f210 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1680647646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561455920580 0x56145592f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d26ffd700 0 client.17979 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 >> 172.21.15.5:0/1680647646 conn(0x561455915be0 msgr2=0x561455919c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 shutdown_connections 2023-07-21T19:19:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.092+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1680647646 wait complete. 2023-07-21T19:19:21.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.093+0000 7f6d617fcb80 20 client.17979 trim_cache size 0 max 16384 2023-07-21T19:19:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.113+0000 7f6d617fcb80 1 Processor -- start 2023-07-21T19:19:21.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.113+0000 7f6d617fcb80 1 -- start start 2023-07-21T19:19:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.113+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 0x561455920040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.113+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614559262d0 0x561455924f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145592a610 con 0x561455920580 2023-07-21T19:19:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145586ae80 con 0x56145577c6b0 2023-07-21T19:19:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145592a930 con 0x5614559262d0 2023-07-21T19:19:21.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d46ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50324/0 (socket says 172.21.15.5:50324) 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d46ffd700 1 -- 172.21.15.5:0/204038780 learned_addr learned my addr 172.21.15.5:0/204038780 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 0x561455920040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.114+0000 7f6d47fff700 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614559262d0 0x561455924f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 -- 172.21.15.5:0/204038780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614559262d0 msgr2=0x561455924f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614559262d0 0x561455924f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 -- 172.21.15.5:0/204038780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 msgr2=0x561455927130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 -- 172.21.15.5:0/204038780 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561455940290 con 0x56145577c6b0 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d477fe700 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 0x561455920040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6d38097c50 tx=0x7f6d38005f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=45eb579423d95b19 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/204038780 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809a2b0 con 0x56145577c6b0 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/204038780 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d380228c0 con 0x56145577c6b0 2023-07-21T19:19:21.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.115+0000 7f6d2e7fc700 1 -- 172.21.15.5:0/204038780 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809f210 con 0x56145577c6b0 2023-07-21T19:19:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 -- 172.21.15.5:0/204038780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 msgr2=0x561455920040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 0x561455920040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f6d38097c50 tx=0x7f6d38005f10 comp rx=0 tx=0).stop 2023-07-21T19:19:21.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 -- 172.21.15.5:0/204038780 shutdown_connections 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5614559262d0 0x561455924f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56145577c6b0 0x561455920040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/204038780 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561455920580 0x561455927130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 -- 172.21.15.5:0/204038780 >> 172.21.15.5:0/204038780 conn(0x561455915be0 msgr2=0x561455924870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 -- 172.21.15.5:0/204038780 shutdown_connections 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.116+0000 7f6d617fcb80 1 -- 172.21.15.5:0/204038780 wait complete. 2023-07-21T19:19:21.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.117+0000 7f6d617fcb80 1 -- start start 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.117+0000 7f6d617fcb80 1 Processor -- start 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 0x56145592d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614559262d0 0x56145592d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145598c110 con 0x5614559262d0 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145598c280 con 0x561455920580 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56145598c440 con 0x56145577c6b0 2023-07-21T19:19:21.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d617fcb80 10 client.0 _ll_register_callbacks cb 0x56145580f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d477fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33206/0 (socket says 172.21.15.5:33206) 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d477fe700 1 -- 172.21.15.5:0/1220950908 learned_addr learned my addr 172.21.15.5:0/1220950908 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d47fff700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614559262d0 0x56145592d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.118+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 0x56145592d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 -- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 msgr2=0x56145592d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 0x56145592d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 -- 172.21.15.5:0/1220950908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614559262d0 msgr2=0x56145592d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614559262d0 0x56145592d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 -- 172.21.15.5:0/1220950908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d3c01f960 con 0x56145577c6b0 2023-07-21T19:19:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 0x56145592d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d477fe700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d38099530 tx=0x7f6d3809ad20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=123a682ca83136b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:21.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3809afb0 con 0x56145577c6b0 2023-07-21T19:19:21.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6d380a35a0 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.119+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6d380a10b0 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6d3809a2b0 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 10 client.27580 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7f6d481a4d60 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6d380a10b0 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6d380b3240 con 0x56145577c6b0 2023-07-21T19:19:21.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 10 client.27580 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 10 client.27580 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 20 client.27580 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56145598d680 0x56145598fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 4 client.27580 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.120+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56145582cff0 con 0x56145598d680 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.121+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56145598d680 0x56145598fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:21.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.121+0000 7f6d26ffd700 10 client.27580 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:21.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.121+0000 7f6d46ffd700 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56145598d680 0x56145598fa60 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ebb538fc5e8f7267 server_cookie=ec85bd07ce678110 in_seq=0 out_seq=0 2023-07-21T19:19:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d26ffd700 1 -- 172.21.15.5:0/1220950908 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56145582cff0 con 0x56145598d680 2023-07-21T19:19:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d26ffd700 10 client.27580 handle_command_reply: tid=0 2023-07-21T19:19:21.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 client.27580 shutdown 2023-07-21T19:19:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 msgr2=0x56145592cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6d38099530 tx=0x7f6d3809ad20 comp rx=0 tx=0).stop 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 shutdown_connections 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56145598d680 0x56145598fa60 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5614559262d0 0x56145592d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56145577c6b0 0x56145592cc10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 --2- 172.21.15.5:0/1220950908 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561455920580 0x56145592d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 >> 172.21.15.5:0/1220950908 conn(0x561455915be0 msgr2=0x561455926fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d26ffd700 0 client.27580 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:21.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 shutdown_connections 2023-07-21T19:19:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.137+0000 7f6d617fcb80 1 -- 172.21.15.5:0/1220950908 wait complete. 2023-07-21T19:19:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.138+0000 7f6d617fcb80 20 client.27580 trim_cache size 0 max 16384 2023-07-21T19:19:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:19:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:19:21.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:19:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:19:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.138+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 msgr2=0x7f6d300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.138+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 0x7f6d300a37f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6d400088a0 tx=0x7f6d400092d0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.138+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 msgr2=0x7f6d48197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.138+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6d380098e0 tx=0x7f6d38004730 comp rx=0 tx=0).stop 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 shutdown_connections 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6d300a1340 0x7f6d300a37f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6d48104a30 0x7f6d4819c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6d480fea30 0x7f6d48197d20 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 --2- 172.21.15.5:0/3560813274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6d480ff3e0 0x7f6d48198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 >> 172.21.15.5:0/3560813274 conn(0x7f6d48074760 msgr2=0x7f6d48067920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 shutdown_connections 2023-07-21T19:19:21.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:21.139+0000 7f6d4ddff700 1 -- 172.21.15.5:0/3560813274 wait complete. 2023-07-21T19:19:21.735 INFO:teuthology.orchestra.run.smithi005.stderr:45000000 of 50000000 tuples (90%) done (elapsed 679.31 s, remaining 75.48 s) 2023-07-21T19:19:21.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:21 smithi101 ceph-mon[128136]: pgmap v367: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:21.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:21 smithi005 ceph-mon[126460]: pgmap v367: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:21.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:21 smithi156 ceph-mon[128316]: pgmap v367: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 27 MiB/s wr, 148 op/s 2023-07-21T19:19:23.153 INFO:teuthology.orchestra.run.smithi005.stderr:45100000 of 50000000 tuples (90%) done (elapsed 680.72 s, remaining 73.96 s) 2023-07-21T19:19:23.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:23 smithi101 ceph-mon[128136]: pgmap v368: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 28 MiB/s wr, 66 op/s 2023-07-21T19:19:23.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:23 smithi005 ceph-mon[126460]: pgmap v368: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 28 MiB/s wr, 66 op/s 2023-07-21T19:19:23.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:23 smithi156 ceph-mon[128316]: pgmap v368: 97 pgs: 97 active+clean; 8.9 GiB data, 3.9 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 28 MiB/s wr, 66 op/s 2023-07-21T19:19:24.485 INFO:teuthology.orchestra.run.smithi005.stderr:45200000 of 50000000 tuples (90%) done (elapsed 682.06 s, remaining 72.43 s) 2023-07-21T19:19:25.794 INFO:teuthology.orchestra.run.smithi005.stderr:45300000 of 50000000 tuples (90%) done (elapsed 683.37 s, remaining 70.90 s) 2023-07-21T19:19:25.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:25 smithi101 ceph-mon[128136]: pgmap v369: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:19:25.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:25 smithi005 ceph-mon[126460]: pgmap v369: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:19:25.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:25 smithi156 ceph-mon[128316]: pgmap v369: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:19:27.113 INFO:teuthology.orchestra.run.smithi005.stderr:45400000 of 50000000 tuples (90%) done (elapsed 684.68 s, remaining 69.37 s) 2023-07-21T19:19:27.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:27 smithi101 ceph-mon[128136]: pgmap v370: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 48 op/s 2023-07-21T19:19:27.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:27 smithi005 ceph-mon[126460]: pgmap v370: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 48 op/s 2023-07-21T19:19:27.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:27 smithi156 ceph-mon[128316]: pgmap v370: 97 pgs: 97 active+clean; 8.9 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 22 MiB/s wr, 48 op/s 2023-07-21T19:19:28.616 INFO:teuthology.orchestra.run.smithi005.stderr:45500000 of 50000000 tuples (91%) done (elapsed 686.19 s, remaining 67.86 s) 2023-07-21T19:19:29.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:29 smithi101 ceph-mon[128136]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 72 op/s 2023-07-21T19:19:29.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:29 smithi005 ceph-mon[126460]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 72 op/s 2023-07-21T19:19:29.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:29 smithi156 ceph-mon[128316]: pgmap v371: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 87 KiB/s rd, 34 MiB/s wr, 72 op/s 2023-07-21T19:19:29.953 INFO:teuthology.orchestra.run.smithi005.stderr:45600000 of 50000000 tuples (91%) done (elapsed 687.52 s, remaining 66.34 s) 2023-07-21T19:19:31.316 INFO:teuthology.orchestra.run.smithi005.stderr:45700000 of 50000000 tuples (91%) done (elapsed 688.89 s, remaining 64.82 s) 2023-07-21T19:19:31.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:31 smithi101 ceph-mon[128136]: pgmap v372: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 60 op/s 2023-07-21T19:19:31.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:31 smithi005 ceph-mon[126460]: pgmap v372: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 60 op/s 2023-07-21T19:19:31.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:31 smithi156 ceph-mon[128316]: pgmap v372: 97 pgs: 97 active+clean; 9.0 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 22 MiB/s wr, 60 op/s 2023-07-21T19:19:32.652 INFO:teuthology.orchestra.run.smithi005.stderr:45800000 of 50000000 tuples (91%) done (elapsed 690.22 s, remaining 63.30 s) 2023-07-21T19:19:33.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:33 smithi101 ceph-mon[128136]: pgmap v373: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 31 MiB/s wr, 90 op/s 2023-07-21T19:19:33.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:33 smithi005 ceph-mon[126460]: pgmap v373: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 31 MiB/s wr, 90 op/s 2023-07-21T19:19:33.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:33 smithi156 ceph-mon[128316]: pgmap v373: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 31 MiB/s wr, 90 op/s 2023-07-21T19:19:34.054 INFO:teuthology.orchestra.run.smithi005.stderr:45900000 of 50000000 tuples (91%) done (elapsed 691.63 s, remaining 61.78 s) 2023-07-21T19:19:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 -- 172.21.15.5:0/152105861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c105b10 msgr2=0x7fbf8c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 --2- 172.21.15.5:0/152105861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c105b10 0x7fbf8c105ee0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbf88009990 tx=0x7fbf88009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 -- 172.21.15.5:0/152105861 shutdown_connections 2023-07-21T19:19:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 --2- 172.21.15.5:0/152105861 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 --2- 172.21.15.5:0/152105861 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c105b10 0x7fbf8c105ee0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 --2- 172.21.15.5:0/152105861 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c0ffb00 0x7fbf8c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 -- 172.21.15.5:0/152105861 >> 172.21.15.5:0/152105861 conn(0x7fbf8c0fb600 msgr2=0x7fbf8c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 -- 172.21.15.5:0/152105861 shutdown_connections 2023-07-21T19:19:34.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.302+0000 7fbf93dff700 1 -- 172.21.15.5:0/152105861 wait complete. 2023-07-21T19:19:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.303+0000 7fbf93dff700 1 Processor -- start 2023-07-21T19:19:34.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.303+0000 7fbf93dff700 1 -- start start 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c0ffb00 0x7fbf8c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 0x7fbf8c1a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1a4280 con 0x7fbf8c105b10 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1a43c0 con 0x7fbf8c1004b0 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf93dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8c1a4530 con 0x7fbf8c0ffb00 2023-07-21T19:19:34.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 0x7fbf8c1a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42820/0 (socket says 172.21.15.5:42820) 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf91b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c0ffb00 0x7fbf8c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 -- 172.21.15.5:0/3430491949 learned_addr learned my addr 172.21.15.5:0/3430491949 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 -- 172.21.15.5:0/3430491949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c0ffb00 msgr2=0x7fbf8c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c0ffb00 0x7fbf8c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 -- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 msgr2=0x7fbf8c1a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.304+0000 7fbf9139a700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 0x7fbf8c1a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf9139a700 1 -- 172.21.15.5:0/3430491949 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf7c009520 con 0x7fbf8c1004b0 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 0x7fbf8c1a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf9139a700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00eb30 tx=0x7fbf7c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf82ffd700 1 -- 172.21.15.5:0/3430491949 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c009880 con 0x7fbf8c1004b0 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf82ffd700 1 -- 172.21.15.5:0/3430491949 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf7c0044b0 con 0x7fbf8c1004b0 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf82ffd700 1 -- 172.21.15.5:0/3430491949 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c003ce0 con 0x7fbf8c1004b0 2023-07-21T19:19:34.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf880095f0 con 0x7fbf8c1004b0 2023-07-21T19:19:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.305+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8c1a4b40 con 0x7fbf8c1004b0 2023-07-21T19:19:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.309+0000 7fbf82ffd700 1 -- 172.21.15.5:0/3430491949 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbf7c003e40 con 0x7fbf8c1004b0 2023-07-21T19:19:34.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.310+0000 7fbfa77a5b80 1 Processor -- start 2023-07-21T19:19:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 -- start start 2023-07-21T19:19:34.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 0x55cf7a95d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a95d580 0x55cf7a95d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a862c50 con 0x55cf7a7b96b0 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a8b59e0 con 0x55cf7a95d580 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a8aee70 con 0x55cf7a9632d0 2023-07-21T19:19:34.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37490/0 (socket says 172.21.15.5:37490) 2023-07-21T19:19:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbf9239c700 1 -- 172.21.15.5:0/3766511344 learned_addr learned my addr 172.21.15.5:0/3766511344 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 0x55cf7a95d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.311+0000 7fbf82ffd700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 0x7fbf780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf9139a700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a95d580 0x55cf7a95d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf82ffd700 1 -- 172.21.15.5:0/3430491949 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbf7c014070 con 0x7fbf8c1004b0 2023-07-21T19:19:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 -- 172.21.15.5:0/3766511344 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a95d580 msgr2=0x55cf7a95d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a95d580 0x55cf7a95d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 -- 172.21.15.5:0/3766511344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 msgr2=0x55cf7a962d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 -- 172.21.15.5:0/3766511344 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf7a6dcd80 con 0x55cf7a9632d0 2023-07-21T19:19:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:34.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 0x7fbf780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.312+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 0x55cf7a95d040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005910 tx=0x7fbf88005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1932e7ed5565eea7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.313+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 0x7fbf780a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009fd0 tx=0x7fbf84004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.313+0000 7fbf77fff700 1 -- 172.21.15.5:0/3766511344 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf8801d070 con 0x55cf7a9632d0 2023-07-21T19:19:34.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.313+0000 7fbf77fff700 1 -- 172.21.15.5:0/3766511344 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf8801e040 con 0x55cf7a9632d0 2023-07-21T19:19:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.313+0000 7fbf77fff700 1 -- 172.21.15.5:0/3766511344 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf88004760 con 0x55cf7a9632d0 2023-07-21T19:19:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3766511344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 msgr2=0x55cf7a95d040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 0x55cf7a95d040 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fbf88005910 tx=0x7fbf88005f00 comp rx=0 tx=0).stop 2023-07-21T19:19:34.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3766511344 shutdown_connections 2023-07-21T19:19:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a95d580 0x55cf7a95d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a7b96b0 0x55cf7a962d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3766511344 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a9632d0 0x55cf7a95d040 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3766511344 >> 172.21.15.5:0/3766511344 conn(0x55cf7a952be0 msgr2=0x55cf7a957f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3766511344 shutdown_connections 2023-07-21T19:19:34.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.314+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3766511344 wait complete. 2023-07-21T19:19:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.315+0000 7fbfa77a5b80 1 -- start start 2023-07-21T19:19:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.315+0000 7fbfa77a5b80 1 Processor -- start 2023-07-21T19:19:34.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a7b96b0 0x55cf7a969580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 0x55cf7a966800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 0x55cf7a966e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a84ea60 con 0x55cf7a9632d0 2023-07-21T19:19:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a84e760 con 0x55cf7a7b96b0 2023-07-21T19:19:34.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a7328c0 con 0x55cf7a95d580 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbfa77a5b80 10 client.0 _ll_register_callbacks cb 0x55cf7a84c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 0x55cf7a966e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 0x55cf7a966e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37500/0 (socket says 172.21.15.5:37500) 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbf9139a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 0x55cf7a966800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbf9239c700 1 -- 172.21.15.5:0/789311194 learned_addr learned my addr 172.21.15.5:0/789311194 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.316+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a7b96b0 0x55cf7a969580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 -- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a7b96b0 msgr2=0x55cf7a969580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a7b96b0 0x55cf7a969580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 -- 172.21.15.5:0/789311194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 msgr2=0x55cf7a966e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 0x55cf7a966e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 -- 172.21.15.5:0/789311194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf84003be0 con 0x55cf7a95d580 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf9139a700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 0x55cf7a966800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ca8a0 tx=0x7fbf7c00cea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a09fe66d41251475 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf757fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf7c09a680 con 0x55cf7a95d580 2023-07-21T19:19:34.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbf7c09a220 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.317+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cf7a808490 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf7c004300 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 10 client.27592 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x7fbf8c10d240 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf7c0c8020 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf7c0b4d90 con 0x55cf7a95d580 2023-07-21T19:19:34.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 10 client.27592 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 10 client.27592 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 20 client.27592 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a976a70 0x55cf7a978e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 4 client.27592 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cf7a8a7e50 con 0x55cf7a976a70 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.318+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a976a70 0x55cf7a978e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.319+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a976a70 0x55cf7a978e80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=865e5a1c9f64f71c server_cookie=d190a6d19931998 in_seq=0 out_seq=0 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.319+0000 7fbf757fa700 10 client.27592 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbf757fa700 1 -- 172.21.15.5:0/789311194 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cf7a8a7e50 con 0x55cf7a976a70 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbf757fa700 10 client.27592 handle_command_reply: tid=0 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 client.27592 shutdown 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 msgr2=0x55cf7a966800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 0x55cf7a966800 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c0ca8a0 tx=0x7fbf7c00cea0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 shutdown_connections 2023-07-21T19:19:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a976a70 0x55cf7a978e80 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a9632d0 0x55cf7a966e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a7b96b0 0x55cf7a969580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbf757fa700 0 client.27592 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/789311194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a95d580 0x55cf7a966800 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 >> 172.21.15.5:0/789311194 conn(0x55cf7a952be0 msgr2=0x55cf7a9574e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 shutdown_connections 2023-07-21T19:19:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.320+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/789311194 wait complete. 2023-07-21T19:19:34.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.321+0000 7fbfa77a5b80 20 client.27592 trim_cache size 0 max 16384 2023-07-21T19:19:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 Processor -- start 2023-07-21T19:19:34.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 -- start start 2023-07-21T19:19:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a95d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a9641a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a967210 con 0x55cf7a95d580 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a959970 con 0x55cf7a952be0 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a967760 con 0x55cf7a7b96b0 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.343+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37514/0 (socket says 172.21.15.5:37514) 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 -- 172.21.15.5:0/3266074485 learned_addr learned my addr 172.21.15.5:0/3266074485 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a95d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9139a700 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a9641a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 -- 172.21.15.5:0/3266074485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 msgr2=0x55cf7a9641a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a9641a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 -- 172.21.15.5:0/3266074485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 msgr2=0x55cf7a95d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a95d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 -- 172.21.15.5:0/3266074485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cf7a95dc20 con 0x55cf7a95d580 2023-07-21T19:19:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.344+0000 7fbf9239c700 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e890 tx=0x7fbf84017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c158f14b6732777e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbf80bff700 1 -- 172.21.15.5:0/3266074485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84017640 con 0x55cf7a95d580 2023-07-21T19:19:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbf80bff700 1 -- 172.21.15.5:0/3266074485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf8400f040 con 0x55cf7a95d580 2023-07-21T19:19:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbf80bff700 1 -- 172.21.15.5:0/3266074485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf840259c0 con 0x55cf7a95d580 2023-07-21T19:19:34.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3266074485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 msgr2=0x55cf7a957c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e890 tx=0x7fbf84017000 comp rx=0 tx=0).stop 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3266074485 shutdown_connections 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a957c60 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a95d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/3266074485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a9641a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.345+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3266074485 >> 172.21.15.5:0/3266074485 conn(0x55cf7a9726b0 msgr2=0x55cf7a963a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.346+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3266074485 shutdown_connections 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.346+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/3266074485 wait complete. 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.346+0000 7fbfa77a5b80 1 -- start start 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.346+0000 7fbfa77a5b80 1 Processor -- start 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a96bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a96c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a96cc60 con 0x55cf7a95d580 2023-07-21T19:19:34.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a96cdd0 con 0x55cf7a952be0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cf7a9ca940 con 0x55cf7a7b96b0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbfa77a5b80 10 client.0 _ll_register_callbacks cb 0x7fbf7c099ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbf91b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbf9239c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a96c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbf91b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56500/0 (socket says 172.21.15.5:56500) 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbf91b9b700 1 -- 172.21.15.5:0/1039679205 learned_addr learned my addr 172.21.15.5:0/1039679205 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.347+0000 7fbf9139a700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a96bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 -- 172.21.15.5:0/1039679205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 msgr2=0x55cf7a96dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 -- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 msgr2=0x55cf7a96bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a96bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 -- 172.21.15.5:0/1039679205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf8400e4f0 con 0x55cf7a95d580 2023-07-21T19:19:34.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:34.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf9239c700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a96c520 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e780 tx=0x7fbf84025be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877e45d9c840d3e5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf757fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf84017640 con 0x55cf7a95d580 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbf8401d030 con 0x55cf7a95d580 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.348+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbf8801e040 con 0x55cf7a95d580 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=42) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbf840177a0 con 0x55cf7a95d580 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 10 client.18003 fetch_fsmap learned FSMap version 42 2023-07-21T19:19:34.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=42,monmap=4+}) v3 -- 0x55cf7a6c5c00 con 0x55cf7a95d580 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf8801e040 con 0x55cf7a95d580 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 42) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbf84036670 con 0x55cf7a95d580 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 10 client.18003 fetch_fsmap finished waiting for FSMap version 42 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 10 client.18003 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 20 client.18003 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a9632d0 0x55cf7a9cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 4 client.18003 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:19:34.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.349+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cf7a869ff0 con 0x55cf7a9632d0 2023-07-21T19:19:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.350+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a9632d0 0x55cf7a9cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.351+0000 7fbf91b9b700 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a9632d0 0x55cf7a9cdfe0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb3760229436d269 server_cookie=7b677c1ca06e7e36 in_seq=0 out_seq=0 2023-07-21T19:19:34.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.351+0000 7fbf757fa700 10 client.18003 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbf757fa700 1 -- 172.21.15.5:0/1039679205 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cf7a869ff0 con 0x55cf7a9632d0 2023-07-21T19:19:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbf757fa700 10 client.18003 handle_command_reply: tid=0 2023-07-21T19:19:34.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 client.18003 shutdown 2023-07-21T19:19:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 msgr2=0x55cf7a96c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a96c520 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7fbf8400e780 tx=0x7fbf84025be0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 shutdown_connections 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cf7a9632d0 0x55cf7a9cdfe0 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cf7a95d580 0x55cf7a96c520 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cf7a7b96b0 0x55cf7a96dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbf757fa700 0 client.18003 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 --2- 172.21.15.5:0/1039679205 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cf7a952be0 0x55cf7a96bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 >> 172.21.15.5:0/1039679205 conn(0x55cf7a9726b0 msgr2=0x55cf7a964320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 shutdown_connections 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.367+0000 7fbfa77a5b80 1 -- 172.21.15.5:0/1039679205 wait complete. 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbfa77a5b80 20 client.18003 trim_cache size 0 max 16384 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:19:34.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:19:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:19:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 msgr2=0x7fbf780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 0x7fbf780a37f0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fbf84009fd0 tx=0x7fbf84004ad0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 msgr2=0x7fbf8c072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbf7c00eb30 tx=0x7fbf7c00ee40 comp rx=0 tx=0).stop 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 shutdown_connections 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf780a1340 0x7fbf780a37f0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf8c105b10 0x7fbf8c1a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf8c0ffb00 0x7fbf8c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 --2- 172.21.15.5:0/3430491949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf8c1004b0 0x7fbf8c072f30 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.368+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 >> 172.21.15.5:0/3430491949 conn(0x7fbf8c0fb600 msgr2=0x7fbf8c0fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.369+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 shutdown_connections 2023-07-21T19:19:34.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:34.369+0000 7fbf93dff700 1 -- 172.21.15.5:0/3430491949 wait complete. 2023-07-21T19:19:35.349 INFO:teuthology.orchestra.run.smithi005.stderr:46000000 of 50000000 tuples (92%) done (elapsed 692.92 s, remaining 60.25 s) 2023-07-21T19:19:35.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:35 smithi101 ceph-mon[128136]: pgmap v374: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 24 MiB/s wr, 79 op/s 2023-07-21T19:19:35.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:35 smithi005 ceph-mon[126460]: pgmap v374: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 24 MiB/s wr, 79 op/s 2023-07-21T19:19:35.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:35 smithi156 ceph-mon[128316]: pgmap v374: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 152 KiB/s rd, 24 MiB/s wr, 79 op/s 2023-07-21T19:19:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:46100000 of 50000000 tuples (92%) done (elapsed 694.24 s, remaining 58.73 s) 2023-07-21T19:19:37.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:37 smithi101 ceph-mon[128136]: pgmap v375: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:19:37.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:37 smithi005 ceph-mon[126460]: pgmap v375: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:19:37.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:37 smithi156 ceph-mon[128316]: pgmap v375: 97 pgs: 97 active+clean; 9.1 GiB data, 4.0 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 22 MiB/s wr, 68 op/s 2023-07-21T19:19:38.102 INFO:teuthology.orchestra.run.smithi005.stderr:46200000 of 50000000 tuples (92%) done (elapsed 695.67 s, remaining 57.22 s) 2023-07-21T19:19:38.876 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:19:39.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 -- 172.21.15.5:0/3269721194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c101e60 msgr2=0x7f0c3c102230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/3269721194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c101e60 0x7f0c3c102230 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c0099c0 tx=0x7f0c2c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 -- 172.21.15.5:0/3269721194 shutdown_connections 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/3269721194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c106ea0 0x7f0c3c10ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/3269721194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c101e60 0x7f0c3c102230 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/3269721194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c102800 0x7f0c3c1067c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 -- 172.21.15.5:0/3269721194 >> 172.21.15.5:0/3269721194 conn(0x7f0c3c0fd690 msgr2=0x7f0c3c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 -- 172.21.15.5:0/3269721194 shutdown_connections 2023-07-21T19:19:39.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.112+0000 7f0c40f9d700 1 -- 172.21.15.5:0/3269721194 wait complete. 2023-07-21T19:19:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.113+0000 7f0c40f9d700 1 Processor -- start 2023-07-21T19:19:39.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.113+0000 7f0c40f9d700 1 -- start start 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c102800 0x7f0c3c19ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c106ea0 0x7f0c3c197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c197f70 con 0x7f0c3c106ea0 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1980e0 con 0x7f0c3c102800 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c40f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3c1982a0 con 0x7f0c3c101e60 2023-07-21T19:19:39.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37612/0 (socket says 172.21.15.5:37612) 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 -- 172.21.15.5:0/4179473293 learned_addr learned my addr 172.21.15.5:0/4179473293 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3affd700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c102800 0x7f0c3c19ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c106ea0 0x7f0c3c197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 -- 172.21.15.5:0/4179473293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c102800 msgr2=0x7f0c3c19ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c102800 0x7f0c3c19ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.114+0000 7f0c3b7fe700 1 -- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c106ea0 msgr2=0x7f0c3c197900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c3b7fe700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c106ea0 0x7f0c3c197900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c3b7fe700 1 -- 172.21.15.5:0/4179473293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c2c0095f0 con 0x7f0c3c101e60 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c3b7fe700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c005910 tx=0x7f0c2c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c38ff9700 1 -- 172.21.15.5:0/4179473293 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c01d070 con 0x7f0c3c101e60 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c3c198520 con 0x7f0c3c101e60 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c38ff9700 1 -- 172.21.15.5:0/4179473293 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c2c005050 con 0x7f0c3c101e60 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c38ff9700 1 -- 172.21.15.5:0/4179473293 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004a00 con 0x7f0c3c101e60 2023-07-21T19:19:39.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.115+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3c1a4880 con 0x7f0c3c101e60 2023-07-21T19:19:39.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.119+0000 7f0c38ff9700 1 -- 172.21.15.5:0/4179473293 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0c2c0051c0 con 0x7f0c3c101e60 2023-07-21T19:19:39.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.120+0000 7f0c55b9eb80 1 Processor -- start 2023-07-21T19:19:39.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.120+0000 7f0c55b9eb80 1 -- start start 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65afe00 0x5593f65b97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 0x5593f65bc110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f64b4cb0 con 0x5593f65afe00 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f6507a40 con 0x5593f65b9d30 2023-07-21T19:19:39.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f6500ed0 con 0x5593f65af4f0 2023-07-21T19:19:39.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65afe00 0x5593f65b97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37614/0 (socket says 172.21.15.5:37614) 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 -- 172.21.15.5:0/3767660505 learned_addr learned my addr 172.21.15.5:0/3767660505 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3affd700 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 0x5593f65bc110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 -- 172.21.15.5:0/3767660505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 msgr2=0x5593f65bc110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 0x5593f65bc110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 -- 172.21.15.5:0/3767660505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65afe00 msgr2=0x5593f65b97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65afe00 0x5593f65b97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.121+0000 7f0c3bfff700 1 -- 172.21.15.5:0/3767660505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f632ede0 con 0x5593f65af4f0 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c3affd700 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 0x5593f65bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003f40 tx=0x7f0c34009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e910f52f0e99b2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c38ff9700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 0x7f0c240a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c21ffb700 1 -- 172.21.15.5:0/3767660505 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34019070 con 0x5593f65af4f0 2023-07-21T19:19:39.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c38ff9700 1 -- 172.21.15.5:0/4179473293 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0c2c0cdb30 con 0x7f0c3c101e60 2023-07-21T19:19:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c21ffb700 1 -- 172.21.15.5:0/3767660505 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c3400f040 con 0x5593f65af4f0 2023-07-21T19:19:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c21ffb700 1 -- 172.21.15.5:0/3767660505 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34007970 con 0x5593f65af4f0 2023-07-21T19:19:39.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.122+0000 7f0c3b7fe700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 0x7f0c240a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/3767660505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 msgr2=0x5593f65af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c3b7fe700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 0x7f0c240a3810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b580 tx=0x7f0c2c0d15a0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0c34003f40 tx=0x7f0c34009ba0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/3767660505 shutdown_connections 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65b9d30 0x5593f65bc110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65af8c0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/3767660505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65afe00 0x5593f65b97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/3767660505 >> 172.21.15.5:0/3767660505 conn(0x5593f65a4ff0 msgr2=0x5593f65aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/3767660505 shutdown_connections 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.123+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/3767660505 wait complete. 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.124+0000 7f0c55b9eb80 1 -- start start 2023-07-21T19:19:39.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.124+0000 7f0c55b9eb80 1 Processor -- start 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 0x5593f65b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f64a0ac0 con 0x5593f65b9d30 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f64a07c0 con 0x5593f65af4f0 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f6384920 con 0x5593f65afe00 2023-07-21T19:19:39.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c55b9eb80 10 client.0 _ll_register_callbacks cb 0x5593f649e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 0x5593f65b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 0x5593f65b9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44242/0 (socket says 172.21.15.5:44242) 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37618/0 (socket says 172.21.15.5:37618) 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3bfff700 1 -- 172.21.15.5:0/1149356078 learned_addr learned my addr 172.21.15.5:0/1149356078 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.125+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 -- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 msgr2=0x5593f65b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 0x5593f65b9780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 -- 172.21.15.5:0/1149356078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 msgr2=0x5593f65b70d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b70d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 -- 172.21.15.5:0/1149356078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c34003b70 con 0x5593f65afe00 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c3affd700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0681c0 tx=0x7f0c300119a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=70f6f4228651f597 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c30016070 con 0x5593f65afe00 2023-07-21T19:19:39.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0c30004320 con 0x5593f65afe00 2023-07-21T19:19:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.126+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593f645a4f0 con 0x5593f65afe00 2023-07-21T19:19:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c300044e0 con 0x5593f65afe00 2023-07-21T19:19:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 10 client.27610 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5593f6317c60 con 0x5593f65afe00 2023-07-21T19:19:39.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5593f645a4f0 con 0x5593f65afe00 2023-07-21T19:19:39.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0c3001a6a0 con 0x5593f65afe00 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 10 client.27610 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 10 client.27610 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 20 client.27610 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f65c97c0 0x5593f65cbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 4 client.27610 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.127+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593f64f9eb0 con 0x5593f65c97c0 2023-07-21T19:19:39.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.128+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f65c97c0 0x5593f65cbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.129+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f65c97c0 0x5593f65cbbd0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=945c7dca371d5fc3 server_cookie=c9e0925d45d34572 in_seq=0 out_seq=0 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.129+0000 7f0c1affd700 10 client.27610 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.129+0000 7f0c1affd700 1 -- 172.21.15.5:0/1149356078 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593f64f9eb0 con 0x5593f65c97c0 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.129+0000 7f0c1affd700 10 client.27610 handle_command_reply: tid=0 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.129+0000 7f0c55b9eb80 1 client.27610 shutdown 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 msgr2=0x5593f65b6990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c0681c0 tx=0x7f0c300119a0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 shutdown_connections 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f65c97c0 0x5593f65cbbd0 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b70d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65af4f0 0x5593f65b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c1affd700 0 client.27610 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1149356078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65afe00 0x5593f65b6990 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 >> 172.21.15.5:0/1149356078 conn(0x5593f65a4ff0 msgr2=0x5593f65aa520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 shutdown_connections 2023-07-21T19:19:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1149356078 wait complete. 2023-07-21T19:19:39.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.130+0000 7f0c55b9eb80 20 client.27610 trim_cache size 0 max 16384 2023-07-21T19:19:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.148+0000 7f0c55b9eb80 1 Processor -- start 2023-07-21T19:19:39.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.148+0000 7f0c55b9eb80 1 -- start start 2023-07-21T19:19:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.148+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.148+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65b92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f645a4f0 con 0x5593f65b9d30 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f65b0980 con 0x5593f65afe00 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f65b1590 con 0x5593f65af4f0 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46058/0 (socket says 172.21.15.5:46058) 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 -- 172.21.15.5:0/1886359132 learned_addr learned my addr 172.21.15.5:0/1886359132 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3affd700 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65b92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 -- 172.21.15.5:0/1886359132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 msgr2=0x5593f65c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 -- 172.21.15.5:0/1886359132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 msgr2=0x5593f65b92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65b92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.149+0000 7f0c3bfff700 1 -- 172.21.15.5:0/1886359132 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593f6355250 con 0x5593f65b9d30 2023-07-21T19:19:39.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.150+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009380 tx=0x7f0c34004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e8bc189586faf3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.150+0000 7f0c227fc700 1 -- 172.21.15.5:0/1886359132 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34019040 con 0x5593f65b9d30 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.150+0000 7f0c227fc700 1 -- 172.21.15.5:0/1886359132 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c34004760 con 0x5593f65b9d30 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.150+0000 7f0c227fc700 1 -- 172.21.15.5:0/1886359132 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c340226b0 con 0x5593f65b9d30 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1886359132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 msgr2=0x5593f65b5f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f0c34009380 tx=0x7f0c34004d30 comp rx=0 tx=0).stop 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1886359132 shutdown_connections 2023-07-21T19:19:39.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65b9d30 0x5593f65b5f00 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65af4f0 0x5593f65c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/1886359132 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65b92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1886359132 >> 172.21.15.5:0/1886359132 conn(0x5593f65a4ff0 msgr2=0x5593f65d04f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1886359132 shutdown_connections 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/1886359132 wait complete. 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.151+0000 7f0c55b9eb80 1 -- start start 2023-07-21T19:19:39.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 Processor -- start 2023-07-21T19:19:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65bed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65b9d30 0x5593f65bf490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f65bfb70 con 0x5593f65af4f0 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f661d650 con 0x5593f65afe00 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593f661d790 con 0x5593f65b9d30 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c55b9eb80 10 client.0 _ll_register_callbacks cb 0x7f0c30009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46070/0 (socket says 172.21.15.5:46070) 2023-07-21T19:19:39.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.152+0000 7f0c3bfff700 1 -- 172.21.15.5:0/2315672134 learned_addr learned my addr 172.21.15.5:0/2315672134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3affd700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65bed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65b9d30 0x5593f65bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 -- 172.21.15.5:0/2315672134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65b9d30 msgr2=0x5593f65bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65b9d30 0x5593f65bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 -- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 msgr2=0x5593f65bed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65bed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 -- 172.21.15.5:0/2315672134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c3402b040 con 0x5593f65af4f0 2023-07-21T19:19:39.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c3bfff700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0c34004de0 tx=0x7f0c34009170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7eaece28123be87 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.153+0000 7f0c1affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c34019040 con 0x5593f65af4f0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0c3400f040 con 0x5593f65af4f0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0c340089c0 con 0x5593f65af4f0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0c34008b80 con 0x5593f65af4f0 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c55b9eb80 10 client.18021 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:39.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x5593f632d2b0 con 0x5593f65af4f0 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.154+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0c3400f040 con 0x5593f65af4f0 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f0c34022450 con 0x5593f65af4f0 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 10 client.18021 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 10 client.18021 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 20 client.18021 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f661eab0 0x5593f6620e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 4 client.18021 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:19:39.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5593f64bc050 con 0x5593f661eab0 2023-07-21T19:19:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.155+0000 7f0c3affd700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f661eab0 0x5593f6620e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.156+0000 7f0c3affd700 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f661eab0 0x5593f6620e90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1503b3caf83c7a5 server_cookie=3e4e0a8871f3a82a in_seq=0 out_seq=0 2023-07-21T19:19:39.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.156+0000 7f0c1affd700 10 client.18021 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.156+0000 7f0c1affd700 1 -- 172.21.15.5:0/2315672134 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x5593f64bc050 con 0x5593f661eab0 2023-07-21T19:19:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.156+0000 7f0c1affd700 10 client.18021 handle_command_reply: tid=0 2023-07-21T19:19:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.156+0000 7f0c55b9eb80 1 client.18021 shutdown 2023-07-21T19:19:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 msgr2=0x5593f65c0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7f0c34004de0 tx=0x7f0c34009170 comp rx=0 tx=0).stop 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 shutdown_connections 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593f65b9d30 0x5593f65bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593f661eab0 0x5593f6620e90 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593f65af4f0 0x5593f65c0a20 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 --2- 172.21.15.5:0/2315672134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593f65afe00 0x5593f65bed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 >> 172.21.15.5:0/2315672134 conn(0x5593f65a4ff0 msgr2=0x5593f65b1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c1affd700 0 client.18021 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:39.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 shutdown_connections 2023-07-21T19:19:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 1 -- 172.21.15.5:0/2315672134 wait complete. 2023-07-21T19:19:39.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.157+0000 7f0c55b9eb80 20 client.18021 trim_cache size 0 max 16384 2023-07-21T19:19:39.159 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:19:39.159 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:19:39.160 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 msgr2=0x7f0c240a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 0x7f0c240a3810 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00b580 tx=0x7f0c2c0d15a0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 msgr2=0x7f0c3c19d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c005910 tx=0x7f0c2c00f690 comp rx=0 tx=0).stop 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 shutdown_connections 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0c240a1360 0x7f0c240a3810 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0c3c106ea0 0x7f0c3c197900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0c3c101e60 0x7f0c3c19d890 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 --2- 172.21.15.5:0/4179473293 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0c3c102800 0x7f0c3c19ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 >> 172.21.15.5:0/4179473293 conn(0x7f0c3c0fd690 msgr2=0x7f0c3c0fe210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 shutdown_connections 2023-07-21T19:19:39.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.158+0000 7f0c40f9d700 1 -- 172.21.15.5:0/4179473293 wait complete. 2023-07-21T19:19:39.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:19:39.171 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:19:39.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/1046444070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 msgr2=0x7fb5f8101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/1046444070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f8101070 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8009990 tx=0x7fb5e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/1046444070 shutdown_connections 2023-07-21T19:19:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/1046444070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f810a070 0x7fb5f810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/1046444070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f8101070 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/1046444070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f81015b0 0x7fb5f8109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/1046444070 >> 172.21.15.5:0/1046444070 conn(0x7fb5f80fa490 msgr2=0x7fb5f80fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/1046444070 shutdown_connections 2023-07-21T19:19:39.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.402+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/1046444070 wait complete. 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.404+0000 7fb5fe3ff700 1 Processor -- start 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.404+0000 7fb5fe3ff700 1 -- start start 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.404+0000 7fb5fe3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.404+0000 7fb5fe3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 0x7fb5f80fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5fe3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f810a070 0x7fb5f81a3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5fe3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f80ff660 con 0x7fb5f8100ca0 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5fe3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81a4220 con 0x7fb5f810a070 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5fe3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5f81a4360 con 0x7fb5f81015b0 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46086/0 (socket says 172.21.15.5:46086) 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f7fff700 1 -- 172.21.15.5:0/772607413 learned_addr learned my addr 172.21.15.5:0/772607413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 0x7fb5f80fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5fc99c700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f810a070 0x7fb5f81a3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 -- 172.21.15.5:0/772607413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f810a070 msgr2=0x7fb5f81a3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f810a070 0x7fb5f81a3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 -- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 msgr2=0x7fb5f80fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f77fe700 1 -- 172.21.15.5:0/772607413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80095f0 con 0x7fb5f81015b0 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.405+0000 7fb5f7fff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5f77fe700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 0x7fb5f80fee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9f0 tx=0x7fb5ec00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec00c9c0 con 0x7fb5f81015b0 2023-07-21T19:19:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5f81a45c0 con 0x7fb5f81015b0 2023-07-21T19:19:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5ec010430 con 0x7fb5f81015b0 2023-07-21T19:19:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5ec018680 con 0x7fb5f81015b0 2023-07-21T19:19:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.406+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5f81a4aa0 con 0x7fb5f81015b0 2023-07-21T19:19:39.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.408+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5f8067300 con 0x7fb5f81015b0 2023-07-21T19:19:39.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.410+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb5ec0105a0 con 0x7fb5f81015b0 2023-07-21T19:19:39.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.411+0000 7fb5f57fa700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 0x7fb5e00a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:39.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.411+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5ec014070 con 0x7fb5f81015b0 2023-07-21T19:19:39.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.412+0000 7fb5f7fff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 0x7fb5e00a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:39.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.412+0000 7fb5f7fff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 0x7fb5e00a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8006010 tx=0x7fb5e800b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:39.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.416+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb5ec051790 con 0x7fb5f81015b0 2023-07-21T19:19:39.519 INFO:teuthology.orchestra.run.smithi005.stderr:46300000 of 50000000 tuples (92%) done (elapsed 697.09 s, remaining 55.71 s) 2023-07-21T19:19:39.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.754+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb5f80ffe20 con 0x7fb5f81015b0 2023-07-21T19:19:39.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.755+0000 7fb5f57fa700 1 -- 172.21.15.5:0/772607413 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 43 v43) v1 ==== 94+0+10834 (secure 0 0 0) 0x7fb5ec096290 con 0x7fb5f81015b0 2023-07-21T19:19:39.751 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:19:39.751 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":43,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":43,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:19:38.535031+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:19:39.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 msgr2=0x7fb5e00a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 0x7fb5e00a37b0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb5e8006010 tx=0x7fb5e800b260 comp rx=0 tx=0).stop 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 msgr2=0x7fb5f80fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 0x7fb5f80fee30 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb5ec00e9f0 tx=0x7fb5ec00ed00 comp rx=0 tx=0).stop 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 shutdown_connections 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb5f810a070 0x7fb5f81a3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5e00a1300 0x7fb5e00a37b0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb5f8100ca0 0x7fb5f80fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 --2- 172.21.15.5:0/772607413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb5f81015b0 0x7fb5f80fee30 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:39.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 >> 172.21.15.5:0/772607413 conn(0x7fb5f80fa490 msgr2=0x7fb5f810c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:39.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 shutdown_connections 2023-07-21T19:19:39.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:39.758+0000 7fb5fe3ff700 1 -- 172.21.15.5:0/772607413 wait complete. 2023-07-21T19:19:39.755 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 43 2023-07-21T19:19:39.772 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:19:39.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:39 smithi101 ceph-mon[128136]: pgmap v376: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 33 MiB/s wr, 94 op/s 2023-07-21T19:19:39.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:39 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:39.839 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:39 smithi005 ceph-mon[126460]: pgmap v376: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 33 MiB/s wr, 94 op/s 2023-07-21T19:19:39.839 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:39 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:39.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:39 smithi156 ceph-mon[128316]: pgmap v376: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 125 KiB/s rd, 33 MiB/s wr, 94 op/s 2023-07-21T19:19:39.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:39 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 -- 172.21.15.5:0/3171406364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670100ca0 msgr2=0x7fd670101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 --2- 172.21.15.5:0/3171406364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670100ca0 0x7fd670101070 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009990 tx=0x7fd66c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 -- 172.21.15.5:0/3171406364 shutdown_connections 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 --2- 172.21.15.5:0/3171406364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd67010a070 0x7fd67010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 --2- 172.21.15.5:0/3171406364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd670100ca0 0x7fd670101070 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 --2- 172.21.15.5:0/3171406364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd6701015b0 0x7fd670109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 -- 172.21.15.5:0/3171406364 >> 172.21.15.5:0/3171406364 conn(0x7fd6700fa4b0 msgr2=0x7fd6700fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 -- 172.21.15.5:0/3171406364 shutdown_connections 2023-07-21T19:19:40.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.028+0000 7fd677fff700 1 -- 172.21.15.5:0/3171406364 wait complete. 2023-07-21T19:19:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.029+0000 7fd677fff700 1 Processor -- start 2023-07-21T19:19:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 -- start start 2023-07-21T19:19:40.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd670100ca0 0x7fd67019b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67010a070 0x7fd670195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670195fd0 con 0x7fd6701015b0 2023-07-21T19:19:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670196140 con 0x7fd670100ca0 2023-07-21T19:19:40.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd677fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd670196300 con 0x7fd67010a070 2023-07-21T19:19:40.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.030+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46102/0 (socket says 172.21.15.5:46102) 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 -- 172.21.15.5:0/1987594056 learned_addr learned my addr 172.21.15.5:0/1987594056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd675d9b700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd670100ca0 0x7fd67019b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67659c700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67010a070 0x7fd670195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 -- 172.21.15.5:0/1987594056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67010a070 msgr2=0x7fd670195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67010a070 0x7fd670195960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 -- 172.21.15.5:0/1987594056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd670100ca0 msgr2=0x7fd67019b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd670100ca0 0x7fd67019b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 -- 172.21.15.5:0/1987594056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd66c0095f0 con 0x7fd6701015b0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.031+0000 7fd67559a700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd66000d980 tx=0x7fd66000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.032+0000 7fd666ffd700 1 -- 172.21.15.5:0/1987594056 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660004220 con 0x7fd6701015b0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.032+0000 7fd666ffd700 1 -- 172.21.15.5:0/1987594056 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd660007640 con 0x7fd6701015b0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.032+0000 7fd666ffd700 1 -- 172.21.15.5:0/1987594056 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660005660 con 0x7fd6701015b0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.032+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6701965e0 con 0x7fd6701015b0 2023-07-21T19:19:40.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.032+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6701a28e0 con 0x7fd6701015b0 2023-07-21T19:19:40.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.036+0000 7fd666ffd700 1 -- 172.21.15.5:0/1987594056 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd660004750 con 0x7fd6701015b0 2023-07-21T19:19:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.037+0000 7fd68b8b2b80 1 Processor -- start 2023-07-21T19:19:40.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.037+0000 7fd68b8b2b80 1 -- start start 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a606a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 0x55c392a6b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 0x55c392a6deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392966cb0 con 0x55c392a6bad0 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3929b9a40 con 0x55c392a602d0 2023-07-21T19:19:40.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3929b2ed0 con 0x55c392a60be0 2023-07-21T19:19:40.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd666ffd700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 0x7fd65c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd666ffd700 1 -- 172.21.15.5:0/1987594056 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd6600ccac0 con 0x7fd6701015b0 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd67659c700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 0x7fd65c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 0x55c392a6deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 0x55c392a6deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46114/0 (socket says 172.21.15.5:46114) 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd67559a700 1 -- 172.21.15.5:0/2533873159 learned_addr learned my addr 172.21.15.5:0/2533873159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 0x55c392a6b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.038+0000 7fd67659c700 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a606a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 -- 172.21.15.5:0/2533873159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 msgr2=0x55c392a606a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a606a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 -- 172.21.15.5:0/2533873159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 msgr2=0x55c392a6deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 0x55c392a6deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 -- 172.21.15.5:0/2533873159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3927e0de0 con 0x55c392a60be0 2023-07-21T19:19:40.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd67659c700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 0x7fd65c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd668009550 tx=0x7fd668009c10 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.039+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 0x55c392a6b590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0036c0 tx=0x7fd66c009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4363510ecd312569 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.040+0000 7fd65bfff700 1 -- 172.21.15.5:0/2533873159 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c01d070 con 0x55c392a60be0 2023-07-21T19:19:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.040+0000 7fd65bfff700 1 -- 172.21.15.5:0/2533873159 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66c01e040 con 0x55c392a60be0 2023-07-21T19:19:40.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.040+0000 7fd65bfff700 1 -- 172.21.15.5:0/2533873159 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c004500 con 0x55c392a60be0 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2533873159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 msgr2=0x55c392a6b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 0x55c392a6b590 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fd66c0036c0 tx=0x7fd66c009ee0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2533873159 shutdown_connections 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a6bad0 0x55c392a6deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a606a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2533873159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a60be0 0x55c392a6b590 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2533873159 >> 172.21.15.5:0/2533873159 conn(0x55c392a56da0 msgr2=0x55c392a5c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2533873159 shutdown_connections 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2533873159 wait complete. 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.041+0000 7fd68b8b2b80 1 -- start start 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 Processor -- start 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a6b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a68cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392952ac0 con 0x55c392a60be0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3929527c0 con 0x55c392a602d0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392836920 con 0x55c392a6bad0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.042+0000 7fd68b8b2b80 10 client.0 _ll_register_callbacks cb 0x55c392950760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a6b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46116/0 (socket says 172.21.15.5:46116) 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 -- 172.21.15.5:0/3931820123 learned_addr learned my addr 172.21.15.5:0/3931820123 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd675d9b700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a6b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44334/0 (socket says 172.21.15.5:44334) 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67659c700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a68cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 -- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 msgr2=0x55c392a68cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a68cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 -- 172.21.15.5:0/3931820123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 msgr2=0x55c392a6b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a6b460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 -- 172.21.15.5:0/3931820123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd67010d170 con 0x55c392a60be0 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd67559a700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd660007e80 tx=0x7fd6600996f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=993c8e17c839c3aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.043+0000 7fd6597fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd660022350 con 0x55c392a60be0 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd660099b30 con 0x55c392a60be0 2023-07-21T19:19:40.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c39290c4f0 con 0x55c392a60be0 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66009db60 con 0x55c392a60be0 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd68b8b2b80 10 client.18039 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.044+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c3927c9c60 con 0x55c392a60be0 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c39290c4f0 con 0x55c392a60be0 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd6600a1af0 con 0x55c392a60be0 2023-07-21T19:19:40.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 10 client.18039 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 10 client.18039 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 20 client.18039 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392a7f7a0 0x55c392a81bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 4 client.18039 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3929abeb0 con 0x55c392a7f7a0 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.045+0000 7fd675d9b700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392a7f7a0 0x55c392a81bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.046+0000 7fd6597fa700 10 client.18039 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.046+0000 7fd675d9b700 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392a7f7a0 0x55c392a81bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3bc3b7746d7465c7 server_cookie=3c9aad96ab8b341d in_seq=0 out_seq=0 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd6597fa700 1 -- 172.21.15.5:0/3931820123 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3929abeb0 con 0x55c392a7f7a0 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd6597fa700 10 client.18039 handle_command_reply: tid=0 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 client.18039 shutdown 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 msgr2=0x55c392a686e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fd660007e80 tx=0x7fd6600996f0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 shutdown_connections 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a68cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392a7f7a0 0x55c392a81bb0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a602d0 0x55c392a6b460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/3931820123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a686e0 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 >> 172.21.15.5:0/3931820123 conn(0x55c392a56da0 msgr2=0x55c392a5c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd6597fa700 0 client.18039 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 shutdown_connections 2023-07-21T19:19:40.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.047+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/3931820123 wait complete. 2023-07-21T19:19:40.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.048+0000 7fd68b8b2b80 20 client.18039 trim_cache size 0 max 16384 2023-07-21T19:19:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.064+0000 7fd68b8b2b80 1 Processor -- start 2023-07-21T19:19:40.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.064+0000 7fd68b8b2b80 1 -- start start 2023-07-21T19:19:40.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a7bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 0x55c392a6b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a61dd0 con 0x55c392a60be0 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a5d4c0 con 0x55c392a6bad0 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a66c40 con 0x55c392a602d0 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37708/0 (socket says 172.21.15.5:37708) 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd67559a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a7bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 -- 172.21.15.5:0/1257382187 learned_addr learned my addr 172.21.15.5:0/1257382187 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd67659c700 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 0x55c392a6b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 -- 172.21.15.5:0/1257382187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 msgr2=0x55c392a6b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 0x55c392a6b000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 -- 172.21.15.5:0/1257382187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 msgr2=0x55c392a7bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a7bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd675d9b700 1 -- 172.21.15.5:0/1257382187 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c392807250 con 0x55c392a602d0 2023-07-21T19:19:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.065+0000 7fd67659c700 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 0x55c392a6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:40.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.066+0000 7fd675d9b700 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00a010 tx=0x7fd66c0055f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b51182584cde1aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.066+0000 7fd664ff9700 1 -- 172.21.15.5:0/1257382187 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c01d040 con 0x55c392a602d0 2023-07-21T19:19:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.066+0000 7fd664ff9700 1 -- 172.21.15.5:0/1257382187 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd66c00b790 con 0x55c392a602d0 2023-07-21T19:19:40.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.066+0000 7fd664ff9700 1 -- 172.21.15.5:0/1257382187 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c00bd80 con 0x55c392a602d0 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/1257382187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 msgr2=0x55c392a67ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fd66c00a010 tx=0x7fd66c0055f0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/1257382187 shutdown_connections 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a6bad0 0x55c392a6b000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a602d0 0x55c392a67ea0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/1257382187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a60be0 0x55c392a7bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/1257382187 >> 172.21.15.5:0/1257382187 conn(0x55c392a56da0 msgr2=0x55c392a86510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/1257382187 shutdown_connections 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/1257382187 wait complete. 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.067+0000 7fd68b8b2b80 1 -- start start 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 Processor -- start 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a60be0 0x55c392a76bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a74bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a752c0 con 0x55c392a602d0 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a75430 con 0x55c392a60be0 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c392a755f0 con 0x55c392a6bad0 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd68b8b2b80 10 client.0 _ll_register_callbacks cb 0x55c392950760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd675d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46130/0 (socket says 172.21.15.5:46130) 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.068+0000 7fd675d9b700 1 -- 172.21.15.5:0/2713591605 learned_addr learned my addr 172.21.15.5:0/2713591605 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd67559a700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a60be0 0x55c392a76bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd67659c700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a74bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 -- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 msgr2=0x55c392a74bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a74bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 -- 172.21.15.5:0/2713591605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a60be0 msgr2=0x55c392a76bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a60be0 0x55c392a76bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 -- 172.21.15.5:0/2713591605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c3927e0de0 con 0x55c392a602d0 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd67659c700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:40.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd675d9b700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009ce0 tx=0x7fd66c0056f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee5c6bbcb1dc609f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.069+0000 7fd6597fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd66c01d040 con 0x55c392a602d0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c39296e050 con 0x55c392a602d0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd66c00b990 con 0x55c392a602d0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd66c0053c0 con 0x55c392a602d0 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 10 client.18045 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c3927df2b0 con 0x55c392a602d0 2023-07-21T19:19:40.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c39296e050 con 0x55c392a602d0 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fd66c042070 con 0x55c392a602d0 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 10 client.18045 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 10 client.18045 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.070+0000 7fd68b8b2b80 20 client.18045 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.071+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392ad4430 0x55c392ad6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.071+0000 7fd68b8b2b80 4 client.18045 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.071+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c392801f70 con 0x55c392ad4430 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.071+0000 7fd67559a700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392ad4430 0x55c392ad6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd67559a700 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392ad4430 0x55c392ad6840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2228eb1cd056b6e server_cookie=659eaa0c90e33d98 in_seq=0 out_seq=0 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd6597fa700 10 client.18045 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd6597fa700 1 -- 172.21.15.5:0/2713591605 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c392801f70 con 0x55c392ad4430 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd6597fa700 10 client.18045 handle_command_reply: tid=0 2023-07-21T19:19:40.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 client.18045 shutdown 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 msgr2=0x55c392a76680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7fd66c009ce0 tx=0x7fd66c0056f0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 shutdown_connections 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c392a6bad0 0x55c392a74bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c392a602d0 0x55c392a76680 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55c392ad4430 0x55c392ad6840 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 --2- 172.21.15.5:0/2713591605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c392a60be0 0x55c392a76bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 >> 172.21.15.5:0/2713591605 conn(0x55c392a56da0 msgr2=0x55c392a7bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd6597fa700 0 client.18045 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:19:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 shutdown_connections 2023-07-21T19:19:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.072+0000 7fd68b8b2b80 1 -- 172.21.15.5:0/2713591605 wait complete. 2023-07-21T19:19:40.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd68b8b2b80 20 client.18045 trim_cache size 0 max 16384 2023-07-21T19:19:40.075 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:40.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 msgr2=0x7fd65c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 0x7fd65c0a3770 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd668009550 tx=0x7fd668009c10 comp rx=0 tx=0).stop 2023-07-21T19:19:40.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 msgr2=0x7fd67019be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fd66000d980 tx=0x7fd66000dc90 comp rx=0 tx=0).stop 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 shutdown_connections 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd67010a070 0x7fd670195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd670100ca0 0x7fd67019b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd65c0a12c0 0x7fd65c0a3770 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 --2- 172.21.15.5:0/1987594056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd6701015b0 0x7fd67019be30 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 >> 172.21.15.5:0/1987594056 conn(0x7fd6700fa4b0 msgr2=0x7fd67010c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 shutdown_connections 2023-07-21T19:19:40.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.073+0000 7fd677fff700 1 -- 172.21.15.5:0/1987594056 wait complete. 2023-07-21T19:19:40.089 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:19:40.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 -- 172.21.15.5:0/3880349077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee41059e0 msgr2=0x7f6ee4105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/3880349077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee41059e0 0x7f6ee4105db0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0009990 tx=0x7f6ee0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 -- 172.21.15.5:0/3880349077 shutdown_connections 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/3880349077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 0x7f6ee410c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/3880349077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee41059e0 0x7f6ee4105db0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/3880349077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee40ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 -- 172.21.15.5:0/3880349077 >> 172.21.15.5:0/3880349077 conn(0x7f6ee40fb4b0 msgr2=0x7f6ee40fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 -- 172.21.15.5:0/3880349077 shutdown_connections 2023-07-21T19:19:40.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.334+0000 7f6eeb39d700 1 -- 172.21.15.5:0/3880349077 wait complete. 2023-07-21T19:19:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.335+0000 7f6eeb39d700 1 Processor -- start 2023-07-21T19:19:40.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.335+0000 7f6eeb39d700 1 -- start start 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 0x7f6ee419ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 0x7f6ee41978f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4197f60 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee41980d0 con 0x7f6ee41059e0 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeb39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee4198290 con 0x7f6ee4100380 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46150/0 (socket says 172.21.15.5:46150) 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 -- 172.21.15.5:0/751294990 learned_addr learned my addr 172.21.15.5:0/751294990 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 0x7f6ee419ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 0x7f6ee41978f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 -- 172.21.15.5:0/751294990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 msgr2=0x7f6ee419ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 0x7f6ee419ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 -- 172.21.15.5:0/751294990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 msgr2=0x7f6ee41978f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 0x7f6ee41978f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eea39b700 1 -- 172.21.15.5:0/751294990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee00095f0 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.336+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 0x7f6ee41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6eea39b700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005910 tx=0x7f6ee0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6edb7fe700 1 -- 172.21.15.5:0/751294990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee001d070 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6edb7fe700 1 -- 172.21.15.5:0/751294990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ee0022470 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee4198510 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6edb7fe700 1 -- 172.21.15.5:0/751294990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0017650 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.337+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41a4870 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.341+0000 7f6edb7fe700 1 -- 172.21.15.5:0/751294990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6ee00225e0 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.342+0000 7f6edb7fe700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 0x7f6ed00a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.343+0000 7f6edb7fe700 1 -- 172.21.15.5:0/751294990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ee00ce0a0 con 0x7f6ee40ff9d0 2023-07-21T19:19:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.343+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 0x7f6ed00a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.343+0000 7f6f0000db80 1 Processor -- start 2023-07-21T19:19:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.343+0000 7f6f0000db80 1 -- start start 2023-07-21T19:19:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1e97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1ec110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b0e4cb0 con 0x55c92b1dfe00 2023-07-21T19:19:40.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b137a40 con 0x55c92b1df4f0 2023-07-21T19:19:40.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6f0000db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b130ed0 con 0x55c92b1e9d30 2023-07-21T19:19:40.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 0x7f6ed00a3890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40fed00 tx=0x7f6ed4009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44384/0 (socket says 172.21.15.5:44384) 2023-07-21T19:19:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6eea39b700 1 -- 172.21.15.5:0/4270439843 learned_addr learned my addr 172.21.15.5:0/4270439843 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1ec110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.344+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1e97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 -- 172.21.15.5:0/4270439843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 msgr2=0x55c92b1ec110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1ec110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 -- 172.21.15.5:0/4270439843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 msgr2=0x55c92b1e97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1e97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 -- 172.21.15.5:0/4270439843 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92af5ede0 con 0x55c92b1df4f0 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:40.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1e97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6eea39b700 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0097d20 tx=0x7f6ee009ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed130518431e8640 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6ed8bfe700 1 -- 172.21.15.5:0/4270439843 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009f120 con 0x55c92b1df4f0 2023-07-21T19:19:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.345+0000 7f6ed8bfe700 1 -- 172.21.15.5:0/4270439843 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ee009f2e0 con 0x55c92b1df4f0 2023-07-21T19:19:40.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.346+0000 7f6ed8bfe700 1 -- 172.21.15.5:0/4270439843 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00229e0 con 0x55c92b1df4f0 2023-07-21T19:19:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.346+0000 7f6f0000db80 1 -- 172.21.15.5:0/4270439843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 msgr2=0x55c92b1df8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.346+0000 7f6f0000db80 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0097d20 tx=0x7f6ee009ae70 comp rx=0 tx=0).stop 2023-07-21T19:19:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 -- 172.21.15.5:0/4270439843 shutdown_connections 2023-07-21T19:19:40.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1ec110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1df8c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 --2- 172.21.15.5:0/4270439843 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1e97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 -- 172.21.15.5:0/4270439843 >> 172.21.15.5:0/4270439843 conn(0x55c92b1d4ff0 msgr2=0x55c92b1da930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 -- 172.21.15.5:0/4270439843 shutdown_connections 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.347+0000 7f6f0000db80 1 -- 172.21.15.5:0/4270439843 wait complete. 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.348+0000 7f6f0000db80 1 -- start start 2023-07-21T19:19:40.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.348+0000 7f6f0000db80 1 Processor -- start 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1dfe00 0x55c92b1f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1e9d30 0x55c92b1f3390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b0d0ac0 con 0x55c92b1df4f0 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b0d07c0 con 0x55c92b1e9d30 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92afb4920 con 0x55c92b1dfe00 2023-07-21T19:19:40.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6f0000db80 10 client.0 _ll_register_callbacks cb 0x55c92b0ce760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46164/0 (socket says 172.21.15.5:46164) 2023-07-21T19:19:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.349+0000 7f6eea39b700 1 -- 172.21.15.5:0/419371037 learned_addr learned my addr 172.21.15.5:0/419371037 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1e9d30 0x55c92b1f3390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1dfe00 0x55c92b1f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 -- 172.21.15.5:0/419371037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1dfe00 msgr2=0x55c92b1f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1dfe00 0x55c92b1f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 -- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1e9d30 msgr2=0x55c92b1f3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1e9d30 0x55c92b1f3390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 -- 172.21.15.5:0/419371037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc009050 con 0x55c92b1df4f0 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.350+0000 7f6eea39b700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009b240 tx=0x7f6ee0099860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1619ab8d86679ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6ecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:40.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00a3670 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ee009b510 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c92b08a4f0 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee009b6d0 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6f0000db80 10 client.18057 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.351+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f6ee41a4870 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c92b08a4f0 con 0x55c92b1df4f0 2023-07-21T19:19:40.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6ee00b9050 con 0x55c92b1df4f0 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 10 client.18057 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 10 client.18057 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 20 client.18057 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b201780 0x55c92b203b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 4 client.18057 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.352+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c92b129eb0 con 0x55c92b201780 2023-07-21T19:19:40.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.353+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b201780 0x55c92b203b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.353+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b201780 0x55c92b203b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91982c72f4be1321 server_cookie=c5329af1e5ee7e1 in_seq=0 out_seq=0 2023-07-21T19:19:40.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.353+0000 7f6ecdffb700 10 client.18057 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.354+0000 7f6ecdffb700 1 -- 172.21.15.5:0/419371037 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c92b129eb0 con 0x55c92b201780 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.354+0000 7f6ecdffb700 10 client.18057 handle_command_reply: tid=0 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.354+0000 7f6f0000db80 1 client.18057 shutdown 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 msgr2=0x55c92b1f2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009b240 tx=0x7f6ee0099860 comp rx=0 tx=0).stop 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 shutdown_connections 2023-07-21T19:19:40.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b201780 0x55c92b203b90 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1e9d30 0x55c92b1f3390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f2910 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 --2- 172.21.15.5:0/419371037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1dfe00 0x55c92b1f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 >> 172.21.15.5:0/419371037 conn(0x55c92b1d4ff0 msgr2=0x55c92b1da520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6ecdffb700 0 client.18057 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 shutdown_connections 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.355+0000 7f6f0000db80 1 -- 172.21.15.5:0/419371037 wait complete. 2023-07-21T19:19:40.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.356+0000 7f6f0000db80 20 client.18057 trim_cache size 0 max 16384 2023-07-21T19:19:40.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.374+0000 7f6f0000db80 1 Processor -- start 2023-07-21T19:19:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.374+0000 7f6f0000db80 1 -- start start 2023-07-21T19:19:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1dfe00 0x55c92b1d9b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1fd0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1d4b70 con 0x55c92b1df4f0 2023-07-21T19:19:40.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1e93f0 con 0x55c92b1dfe00 2023-07-21T19:19:40.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6f0000db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1e7340 con 0x55c92b1e9d30 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46178/0 (socket says 172.21.15.5:46178) 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6eea39b700 1 -- 172.21.15.5:0/2511117952 learned_addr learned my addr 172.21.15.5:0/2511117952 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1fd0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.375+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1dfe00 0x55c92b1d9b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 -- 172.21.15.5:0/2511117952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 msgr2=0x55c92b1fd0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1fd0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 -- 172.21.15.5:0/2511117952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1dfe00 msgr2=0x55c92b1d9b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1dfe00 0x55c92b1d9b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 -- 172.21.15.5:0/2511117952 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92af47c60 con 0x55c92b1df4f0 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.376+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0006010 tx=0x7f6ee0005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88e8d065b5f285d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.377+0000 7f6ed93ff700 1 -- 172.21.15.5:0/2511117952 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009adf0 con 0x55c92b1df4f0 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.377+0000 7f6ed93ff700 1 -- 172.21.15.5:0/2511117952 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ee00a3670 con 0x55c92b1df4f0 2023-07-21T19:19:40.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.377+0000 7f6ed93ff700 1 -- 172.21.15.5:0/2511117952 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009a7a0 con 0x55c92b1df4f0 2023-07-21T19:19:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.377+0000 7f6f0000db80 1 -- 172.21.15.5:0/2511117952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 msgr2=0x55c92b1f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.377+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0006010 tx=0x7f6ee0005c00 comp rx=0 tx=0).stop 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 -- 172.21.15.5:0/2511117952 shutdown_connections 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1fd0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6940 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2511117952 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1dfe00 0x55c92b1d9b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 -- 172.21.15.5:0/2511117952 >> 172.21.15.5:0/2511117952 conn(0x55c92b1d4ff0 msgr2=0x55c92b1d9060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 -- 172.21.15.5:0/2511117952 shutdown_connections 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 -- 172.21.15.5:0/2511117952 wait complete. 2023-07-21T19:19:40.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.378+0000 7f6f0000db80 1 -- start start 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 Processor -- start 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1f6890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1f4de0 con 0x55c92b1dfe00 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1f4f20 con 0x55c92b1df4f0 2023-07-21T19:19:40.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92b1f50e0 con 0x55c92b1e9d30 2023-07-21T19:19:40.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.379+0000 7f6f0000db80 10 client.0 _ll_register_callbacks cb 0x7f6edc009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eeab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eeab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37796/0 (socket says 172.21.15.5:37796) 2023-07-21T19:19:40.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eeab9c700 1 -- 172.21.15.5:0/2950156111 learned_addr learned my addr 172.21.15.5:0/2950156111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1f6890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 -- 172.21.15.5:0/2950156111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 msgr2=0x55c92b1f4810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 -- 172.21.15.5:0/2950156111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 msgr2=0x55c92b1f6890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1f6890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 -- 172.21.15.5:0/2950156111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee41a4870 con 0x55c92b1df4f0 2023-07-21T19:19:40.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eeab9c700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6eea39b700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009b240 tx=0x7f6ee00975f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2384343b6af1f1c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.380+0000 7f6ecdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee009b710 con 0x55c92b1df4f0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ee00b9320 con 0x55c92b1df4f0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee009f2e0 con 0x55c92b1df4f0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee009adf0 con 0x55c92b1df4f0 2023-07-21T19:19:40.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6f0000db80 10 client.27731 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55c92af85250 con 0x55c92b1df4f0 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.381+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ee009f2e0 con 0x55c92b1df4f0 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6ee00979c0 con 0x55c92b1df4f0 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 10 client.27731 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 10 client.27731 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 20 client.27731 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b2562a0 0x55c92b258680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 4 client.27731 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55c92b0ec050 con 0x55c92b2562a0 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.382+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b2562a0 0x55c92b258680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6ee9b9a700 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b2562a0 0x55c92b258680 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=315fbf08c46f898c server_cookie=3844ce8ff99822a4 in_seq=0 out_seq=0 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6ecdffb700 10 client.27731 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6ecdffb700 1 -- 172.21.15.5:0/2950156111 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55c92b0ec050 con 0x55c92b2562a0 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6ecdffb700 10 client.27731 handle_command_reply: tid=0 2023-07-21T19:19:40.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6f0000db80 1 client.27731 shutdown 2023-07-21T19:19:40.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 msgr2=0x55c92b1f6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.383+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6350 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f6ee009b240 tx=0x7f6ee00975f0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 shutdown_connections 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c92b1e9d30 0x55c92b1f4810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55c92b2562a0 0x55c92b258680 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c92b1df4f0 0x55c92b1f6350 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 --2- 172.21.15.5:0/2950156111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c92b1dfe00 0x55c92b1f6890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6ecdffb700 0 client.27731 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 >> 172.21.15.5:0/2950156111 conn(0x55c92b1d4ff0 msgr2=0x55c92b1e4120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 shutdown_connections 2023-07-21T19:19:40.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 1 -- 172.21.15.5:0/2950156111 wait complete. 2023-07-21T19:19:40.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.384+0000 7f6f0000db80 20 client.27731 trim_cache size 0 max 16384 2023-07-21T19:19:40.385 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 msgr2=0x7f6ed00a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 0x7f6ed00a3890 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f6ee40fed00 tx=0x7f6ed4009260 comp rx=0 tx=0).stop 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 msgr2=0x7f6ee419d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f6ee0005910 tx=0x7f6ee0005740 comp rx=0 tx=0).stop 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 shutdown_connections 2023-07-21T19:19:40.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6ee41059e0 0x7f6ee41978f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ed00a13e0 0x7f6ed00a3890 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6ee40ff9d0 0x7f6ee419d880 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 --2- 172.21.15.5:0/751294990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6ee4100380 0x7f6ee419ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 >> 172.21.15.5:0/751294990 conn(0x7f6ee40fb4b0 msgr2=0x7f6ee40fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 shutdown_connections 2023-07-21T19:19:40.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.385+0000 7f6eeb39d700 1 -- 172.21.15.5:0/751294990 wait complete. 2023-07-21T19:19:40.401 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:19:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 -- 172.21.15.5:0/2252549886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c073160 msgr2=0x7f345c10be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 --2- 172.21.15.5:0/2252549886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c073160 0x7f345c10be40 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f3450009990 tx=0x7f3450009ca0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 -- 172.21.15.5:0/2252549886 shutdown_connections 2023-07-21T19:19:40.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 --2- 172.21.15.5:0/2252549886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c10c380 0x7f345c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 --2- 172.21.15.5:0/2252549886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c072850 0x7f345c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 --2- 172.21.15.5:0/2252549886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c073160 0x7f345c10be40 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 -- 172.21.15.5:0/2252549886 >> 172.21.15.5:0/2252549886 conn(0x7f345c0fb5e0 msgr2=0x7f345c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 -- 172.21.15.5:0/2252549886 shutdown_connections 2023-07-21T19:19:40.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.647+0000 7f34617ff700 1 -- 172.21.15.5:0/2252549886 wait complete. 2023-07-21T19:19:40.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 Processor -- start 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 -- start start 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c072850 0x7f345c197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 0x7f345c1982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 0x7f345c19c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19cd40 con 0x7f345c10c380 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19ceb0 con 0x7f345c072850 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.649+0000 7f34617ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c19d070 con 0x7f345c073160 2023-07-21T19:19:40.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 0x7f345c19c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 0x7f345c1982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 0x7f345c19c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46212/0 (socket says 172.21.15.5:46212) 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345b7fe700 1 -- 172.21.15.5:0/3946609871 learned_addr learned my addr 172.21.15.5:0/3946609871 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345affd700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c072850 0x7f345c197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 -- 172.21.15.5:0/3946609871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c072850 msgr2=0x7f345c197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c072850 0x7f345c197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 -- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 msgr2=0x7f345c19c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 0x7f345c19c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.650+0000 7f345a7fc700 1 -- 172.21.15.5:0/3946609871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500095f0 con 0x7f345c073160 2023-07-21T19:19:40.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f345a7fc700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 0x7f345c1982a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3450005910 tx=0x7f3450005720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f3443fff700 1 -- 172.21.15.5:0/3946609871 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345001d070 con 0x7f345c073160 2023-07-21T19:19:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f3443fff700 1 -- 172.21.15.5:0/3946609871 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3450022470 con 0x7f345c073160 2023-07-21T19:19:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f3443fff700 1 -- 172.21.15.5:0/3946609871 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450017650 con 0x7f345c073160 2023-07-21T19:19:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c19d2f0 con 0x7f345c073160 2023-07-21T19:19:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.651+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c1a4b90 con 0x7f345c073160 2023-07-21T19:19:40.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.654+0000 7f3443fff700 1 -- 172.21.15.5:0/3946609871 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f34500225e0 con 0x7f345c073160 2023-07-21T19:19:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.657+0000 7f3443fff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.657+0000 7f34751b1b80 1 Processor -- start 2023-07-21T19:19:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.657+0000 7f3443fff700 1 -- 172.21.15.5:0/3946609871 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f34500cdff0 con 0x7f345c073160 2023-07-21T19:19:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.657+0000 7f34751b1b80 1 -- start start 2023-07-21T19:19:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.657+0000 7f345affd700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf64d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf64feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf548cb0 con 0x55d2cf6422d0 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf59ba40 con 0x55d2cf642be0 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f34751b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf594ed0 con 0x55d2cf64dad0 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f345affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f345affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46218/0 (socket says 172.21.15.5:46218) 2023-07-21T19:19:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f345affd700 1 -- 172.21.15.5:0/1815537082 learned_addr learned my addr 172.21.15.5:0/1815537082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf64feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.658+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf64d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345affd700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f344c009cb0 tx=0x7f344c011720 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 -- 172.21.15.5:0/1815537082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 msgr2=0x55d2cf64d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf64d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 -- 172.21.15.5:0/1815537082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 msgr2=0x55d2cf6426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 -- 172.21.15.5:0/1815537082 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2cf3c2de0 con 0x55d2cf64dad0 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345affd700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:40.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf64feb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3454009d30 tx=0x7f34540077e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab63b664112aa1e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f34417fa700 1 -- 172.21.15.5:0/1815537082 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454004bb0 con 0x55d2cf64dad0 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f34417fa700 1 -- 172.21.15.5:0/1815537082 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f345400f040 con 0x55d2cf64dad0 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.659+0000 7f34417fa700 1 -- 172.21.15.5:0/1815537082 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454005020 con 0x55d2cf64dad0 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 -- 172.21.15.5:0/1815537082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 msgr2=0x55d2cf64feb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf64feb0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3454009d30 tx=0x7f34540077e0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 -- 172.21.15.5:0/1815537082 shutdown_connections 2023-07-21T19:19:40.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf64feb0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf6426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1815537082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf64d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 -- 172.21.15.5:0/1815537082 >> 172.21.15.5:0/1815537082 conn(0x55d2cf638da0 msgr2=0x55d2cf63e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 -- 172.21.15.5:0/1815537082 shutdown_connections 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.660+0000 7f34751b1b80 1 -- 172.21.15.5:0/1815537082 wait complete. 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.661+0000 7f34751b1b80 1 -- start start 2023-07-21T19:19:40.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.661+0000 7f34751b1b80 1 Processor -- start 2023-07-21T19:19:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 0x55d2cf64aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf534ac0 con 0x55d2cf6422d0 2023-07-21T19:19:40.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf5347c0 con 0x55d2cf64dad0 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf418920 con 0x55d2cf642be0 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f34751b1b80 10 client.0 _ll_register_callbacks cb 0x55d2cf532760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f345a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f345affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f345a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37824/0 (socket says 172.21.15.5:37824) 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.662+0000 7f345a7fc700 1 -- 172.21.15.5:0/1100377505 learned_addr learned my addr 172.21.15.5:0/1100377505 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 0x55d2cf64aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 -- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 msgr2=0x55d2cf64aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 0x55d2cf64aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 -- 172.21.15.5:0/1100377505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 msgr2=0x55d2cf64d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 -- 172.21.15.5:0/1100377505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c011cd0 con 0x55d2cf642be0 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 0x55d2cf64aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:40.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3450097610 tx=0x7f345009aae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b6e7105b1c2c141b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f343eff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500b2070 con 0x55d2cf642be0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f34500a34d0 con 0x55d2cf642be0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d2cf4ee4f0 con 0x55d2cf642be0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.663+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3450022ad0 con 0x55d2cf642be0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 10 client.27670 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x7f3454009960 con 0x55d2cf642be0 2023-07-21T19:19:40.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34500c9080 con 0x55d2cf642be0 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f34500a3040 con 0x55d2cf642be0 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 10 client.27670 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 10 client.27670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 20 client.27670 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf65d030 0x55d2cf65f410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 4 client.27670 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.664+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2cf58deb0 con 0x55d2cf65d030 2023-07-21T19:19:40.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.665+0000 7f345affd700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf65d030 0x55d2cf65f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.666+0000 7f345affd700 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf65d030 0x55d2cf65f410 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e286be9c6e80a796 server_cookie=699c02a15c75290 in_seq=0 out_seq=0 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.666+0000 7f343eff5700 10 client.27670 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.666+0000 7f343eff5700 1 -- 172.21.15.5:0/1100377505 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2cf58deb0 con 0x55d2cf65d030 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.666+0000 7f343eff5700 10 client.27670 handle_command_reply: tid=0 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.666+0000 7f34751b1b80 1 client.27670 shutdown 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 msgr2=0x55d2cf64a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3450097610 tx=0x7f345009aae0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 shutdown_connections 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf64dad0 0x55d2cf64aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf65d030 0x55d2cf65f410 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1100377505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf642be0 0x55d2cf64a790 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f343eff5700 0 client.27670 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 >> 172.21.15.5:0/1100377505 conn(0x55d2cf638da0 msgr2=0x55d2cf63e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 shutdown_connections 2023-07-21T19:19:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 1 -- 172.21.15.5:0/1100377505 wait complete. 2023-07-21T19:19:40.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.667+0000 7f34751b1b80 20 client.27670 trim_cache size 0 max 16384 2023-07-21T19:19:40.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 Processor -- start 2023-07-21T19:19:40.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 -- start start 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf647a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf647f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf64b770 con 0x55d2cf6422d0 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf643570 con 0x55d2cf642be0 2023-07-21T19:19:40.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f34751b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf6462e0 con 0x55d2cf64dad0 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f345affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f345affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46242/0 (socket says 172.21.15.5:46242) 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.683+0000 7f345affd700 1 -- 172.21.15.5:0/3630280609 learned_addr learned my addr 172.21.15.5:0/3630280609 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345b7fe700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf647f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345a7fc700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf647a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 -- 172.21.15.5:0/3630280609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 msgr2=0x55d2cf647f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf647f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 -- 172.21.15.5:0/3630280609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 msgr2=0x55d2cf647a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf647a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 -- 172.21.15.5:0/3630280609 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2cf3abc60 con 0x55d2cf6422d0 2023-07-21T19:19:40.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345a7fc700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf647a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f345affd700 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f344c005c20 tx=0x7f344c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfa4308baa51225c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f3441ffb700 1 -- 172.21.15.5:0/3630280609 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c023040 con 0x55d2cf6422d0 2023-07-21T19:19:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.684+0000 7f3441ffb700 1 -- 172.21.15.5:0/3630280609 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f344c0284e0 con 0x55d2cf6422d0 2023-07-21T19:19:40.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f3441ffb700 1 -- 172.21.15.5:0/3630280609 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344c00fcc0 con 0x55d2cf6422d0 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 -- 172.21.15.5:0/3630280609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 msgr2=0x55d2cf64d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f344c005c20 tx=0x7f344c01c000 comp rx=0 tx=0).stop 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 -- 172.21.15.5:0/3630280609 shutdown_connections 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf647f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf6422d0 0x55d2cf64d0b0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 --2- 172.21.15.5:0/3630280609 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf642be0 0x55d2cf647a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 -- 172.21.15.5:0/3630280609 >> 172.21.15.5:0/3630280609 conn(0x55d2cf638da0 msgr2=0x55d2cf659700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 -- 172.21.15.5:0/3630280609 shutdown_connections 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.685+0000 7f34751b1b80 1 -- 172.21.15.5:0/3630280609 wait complete. 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.686+0000 7f34751b1b80 1 -- start start 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.686+0000 7f34751b1b80 1 Processor -- start 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf6422d0 0x55d2cf652b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf6535a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf6b0ed0 con 0x55d2cf642be0 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf6b1010 con 0x55d2cf6422d0 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2cf6b1150 con 0x55d2cf64dad0 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f34751b1b80 10 client.0 _ll_register_callbacks cb 0x7f345009af70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46254/0 (socket says 172.21.15.5:46254) 2023-07-21T19:19:40.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 -- 172.21.15.5:0/1526035817 learned_addr learned my addr 172.21.15.5:0/1526035817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345affd700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf6422d0 0x55d2cf652b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345b7fe700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf6535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 -- 172.21.15.5:0/1526035817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 msgr2=0x55d2cf6535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf6535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 -- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf6422d0 msgr2=0x55d2cf652b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf6422d0 0x55d2cf652b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.687+0000 7f345a7fc700 1 -- 172.21.15.5:0/1526035817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f344c00f470 con 0x55d2cf642be0 2023-07-21T19:19:40.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f345a7fc700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f345000b490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56bc073e4a107322 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f343eff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34500b2030 con 0x55d2cf642be0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f34500b3070 con 0x55d2cf642be0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f345400f040 con 0x55d2cf642be0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3450022ad0 con 0x55d2cf642be0 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f34751b1b80 10 client.18081 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:40.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55d2cf3e9250 con 0x55d2cf642be0 2023-07-21T19:19:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.688+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f345400f040 con 0x55d2cf642be0 2023-07-21T19:19:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f3450096f00 con 0x55d2cf642be0 2023-07-21T19:19:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 10 client.18081 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 10 client.18081 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:40.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 20 client.18081 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf6b2240 0x55d2cf6b4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 4 client.18081 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d2cf550050 con 0x55d2cf6b2240 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.689+0000 7f345affd700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf6b2240 0x55d2cf6b4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f345affd700 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf6b2240 0x55d2cf6b4620 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea24c6ad804ff8f6 server_cookie=78cebadd356320b1 in_seq=0 out_seq=0 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f343eff5700 10 client.18081 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:40.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f343eff5700 1 -- 172.21.15.5:0/1526035817 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d2cf550050 con 0x55d2cf6b2240 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f343eff5700 10 client.18081 handle_command_reply: tid=0 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 client.18081 shutdown 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 msgr2=0x55d2cf653060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3450006010 tx=0x7f345000b490 comp rx=0 tx=0).stop 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 shutdown_connections 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2cf64dad0 0x55d2cf6535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2cf6422d0 0x55d2cf652b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2cf6b2240 0x55d2cf6b4620 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 --2- 172.21.15.5:0/1526035817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2cf642be0 0x55d2cf653060 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 >> 172.21.15.5:0/1526035817 conn(0x55d2cf638da0 msgr2=0x55d2cf6443e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.690+0000 7f343eff5700 0 client.18081 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 shutdown_connections 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34751b1b80 1 -- 172.21.15.5:0/1526035817 wait complete. 2023-07-21T19:19:40.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34751b1b80 20 client.18081 trim_cache size 0 max 16384 2023-07-21T19:19:40.694 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:19:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 msgr2=0x7f34440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f344c009cb0 tx=0x7f344c011720 comp rx=0 tx=0).stop 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 msgr2=0x7f345c1982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 0x7f345c1982a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f3450005910 tx=0x7f3450005720 comp rx=0 tx=0).stop 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 shutdown_connections 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f345c10c380 0x7f345c19c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f345c072850 0x7f345c197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 --2- 172.21.15.5:0/3946609871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f345c073160 0x7f345c1982a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.691+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 >> 172.21.15.5:0/3946609871 conn(0x7f345c0fb5e0 msgr2=0x7f345c10e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.692+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 shutdown_connections 2023-07-21T19:19:40.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:40.692+0000 7f34617ff700 1 -- 172.21.15.5:0/3946609871 wait complete. 2023-07-21T19:19:40.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:40 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/772607413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:40.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:40 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/772607413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:40.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:40 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/772607413' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:19:41.003 INFO:teuthology.orchestra.run.smithi005.stderr:46400000 of 50000000 tuples (92%) done (elapsed 698.57 s, remaining 54.20 s) 2023-07-21T19:19:41.708 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:19:41.709 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:19:41.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:41 smithi101 ceph-mon[128136]: pgmap v377: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 22 MiB/s wr, 70 op/s 2023-07-21T19:19:41.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:41 smithi005 ceph-mon[126460]: pgmap v377: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 22 MiB/s wr, 70 op/s 2023-07-21T19:19:41.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:41 smithi156 ceph-mon[128316]: pgmap v377: 97 pgs: 97 active+clean; 9.2 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 22 MiB/s wr, 70 op/s 2023-07-21T19:19:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/154523083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 msgr2=0x7f6c7410e700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/154523083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7410e700 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00b0a0 tx=0x7f6c6c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/154523083 shutdown_connections 2023-07-21T19:19:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/154523083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7410e700 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/154523083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 0x7f6c74072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/154523083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c7410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/154523083 >> 172.21.15.5:0/154523083 conn(0x7f6c740fb5e0 msgr2=0x7f6c740fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:41.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/154523083 shutdown_connections 2023-07-21T19:19:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.955+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/154523083 wait complete. 2023-07-21T19:19:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 Processor -- start 2023-07-21T19:19:41.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 -- start start 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 0x7f6c74197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7419c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419ce60 con 0x7f6c7410c320 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419cfd0 con 0x7f6c740731c0 2023-07-21T19:19:41.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.957+0000 7f6c7a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7419d190 con 0x7f6c740728b0 2023-07-21T19:19:41.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 0x7f6c74197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44500/0 (socket says 172.21.15.5:44500) 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 -- 172.21.15.5:0/2873999599 learned_addr learned my addr 172.21.15.5:0/2873999599 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c73fff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7419c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 -- 172.21.15.5:0/2873999599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 msgr2=0x7f6c74197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 0x7f6c74197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 -- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 msgr2=0x7f6c7419c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7419c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.958+0000 7f6c72ffd700 1 -- 172.21.15.5:0/2873999599 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009d00 con 0x7f6c740731c0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e9a0 tx=0x7f6c6800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c70ff9700 1 -- 172.21.15.5:0/2873999599 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c68009790 con 0x7f6c740731c0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c70ff9700 1 -- 172.21.15.5:0/2873999599 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c68010430 con 0x7f6c740731c0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7419d470 con 0x7f6c740731c0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c70ff9700 1 -- 172.21.15.5:0/2873999599 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6800cea0 con 0x7f6c740731c0 2023-07-21T19:19:41.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.959+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741a4cb0 con 0x7f6c740731c0 2023-07-21T19:19:41.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.964+0000 7f6c70ff9700 1 -- 172.21.15.5:0/2873999599 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6c680105a0 con 0x7f6c740731c0 2023-07-21T19:19:41.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.965+0000 7f6c8da94b80 1 Processor -- start 2023-07-21T19:19:41.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.965+0000 7f6c8da94b80 1 -- start start 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c70ff9700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 0x7f6c5c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea43720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea56e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e949d30 con 0x55cd0ea43c60 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e99cac0 con 0x55cd0ea57360 2023-07-21T19:19:41.962 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e995f50 con 0x55cd0ea43350 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c70ff9700 1 -- 172.21.15.5:0/2873999599 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c68014070 con 0x7f6c740731c0 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.966+0000 7f6c73fff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 0x7f6c5c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea56e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44504/0 (socket says 172.21.15.5:44504) 2023-07-21T19:19:41.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 -- 172.21.15.5:0/3759642729 learned_addr learned my addr 172.21.15.5:0/3759642729 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:41.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c737fe700 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea43720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 -- 172.21.15.5:0/3759642729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 msgr2=0x55cd0ea43720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea43720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 -- 172.21.15.5:0/3759642729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 msgr2=0x55cd0ea56e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea56e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.967+0000 7f6c73fff700 1 -- 172.21.15.5:0/3759642729 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0e7c3e60 con 0x55cd0ea57360 2023-07-21T19:19:41.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c73fff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 0x7f6c5c0a37f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d010 tx=0x7f6c6c0094b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c737fe700 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea43720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c73fff700 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c015000 tx=0x7f6c6c025000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3231aa516c83a6d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c59ffb700 1 -- 172.21.15.5:0/3759642729 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c025070 con 0x55cd0ea57360 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c59ffb700 1 -- 172.21.15.5:0/3759642729 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c6c00bdc0 con 0x55cd0ea57360 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.968+0000 7f6c59ffb700 1 -- 172.21.15.5:0/3759642729 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c00e040 con 0x55cd0ea57360 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3759642729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 msgr2=0x55cd0ea59740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c015000 tx=0x7f6c6c025000 comp rx=0 tx=0).stop 2023-07-21T19:19:41.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3759642729 shutdown_connections 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea57360 0x55cd0ea59740 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea43720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3759642729 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea56e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.969+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3759642729 >> 172.21.15.5:0/3759642729 conn(0x55cd0ea39e20 msgr2=0x55cd0ea3f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.970+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3759642729 shutdown_connections 2023-07-21T19:19:41.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.970+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3759642729 wait complete. 2023-07-21T19:19:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.970+0000 7f6c8da94b80 1 -- start start 2023-07-21T19:19:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 Processor -- start 2023-07-21T19:19:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea53fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea57360 0x55cd0ea54620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e935b40 con 0x55cd0ea43c60 2023-07-21T19:19:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e935840 con 0x55cd0ea43350 2023-07-21T19:19:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0e8199a0 con 0x55cd0ea57360 2023-07-21T19:19:41.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.971+0000 7f6c8da94b80 10 client.0 _ll_register_callbacks cb 0x55cd0e9337e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea53fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44512/0 (socket says 172.21.15.5:44512) 2023-07-21T19:19:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c72ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea53fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46278/0 (socket says 172.21.15.5:46278) 2023-07-21T19:19:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 -- 172.21.15.5:0/585405015 learned_addr learned my addr 172.21.15.5:0/585405015 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c73fff700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea57360 0x55cd0ea54620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 -- 172.21.15.5:0/585405015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea57360 msgr2=0x55cd0ea54620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea57360 0x55cd0ea54620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 -- 172.21.15.5:0/585405015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 msgr2=0x55cd0ea53fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea53fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.972+0000 7f6c737fe700 1 -- 172.21.15.5:0/585405015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7410f3a0 con 0x55cd0ea43350 2023-07-21T19:19:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c737fe700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c640136e0 tx=0x7f6c640139f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=df9ccc745ff62d09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:41.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640118b0 con 0x55cd0ea43350 2023-07-21T19:19:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c64011a10 con 0x55cd0ea43350 2023-07-21T19:19:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd0e8ef570 con 0x55cd0ea43350 2023-07-21T19:19:41.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c64011bd0 con 0x55cd0ea43350 2023-07-21T19:19:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c8da94b80 10 client.27755 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.973+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55cd0e7acce0 con 0x55cd0ea43350 2023-07-21T19:19:41.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c64028020 con 0x55cd0ea43350 2023-07-21T19:19:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c6400f3c0 con 0x55cd0ea43350 2023-07-21T19:19:41.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 10 client.27755 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:41.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 10 client.27755 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:41.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 20 client.27755 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:41.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0ea66be0 0x55cd0ea68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 4 client.27755 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.974+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd0e98ef30 con 0x55cd0ea66be0 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.975+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0ea66be0 0x55cd0ea68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.975+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0ea66be0 0x55cd0ea68ff0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e890781ea73e1501 server_cookie=7e334eb6bc1e2f98 in_seq=0 out_seq=0 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.975+0000 7f6c52ffd700 10 client.27755 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c52ffd700 1 -- 172.21.15.5:0/585405015 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd0e98ef30 con 0x55cd0ea66be0 2023-07-21T19:19:41.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c52ffd700 10 client.27755 handle_command_reply: tid=0 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 client.27755 shutdown 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 msgr2=0x55cd0ea56d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f6c640136e0 tx=0x7f6c640139f0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 shutdown_connections 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea57360 0x55cd0ea54620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea56d40 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.976+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0ea66be0 0x55cd0ea68ff0 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/585405015 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea43c60 0x55cd0ea53fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c52ffd700 0 client.27755 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 >> 172.21.15.5:0/585405015 conn(0x55cd0ea39e20 msgr2=0x55cd0ea3f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 shutdown_connections 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c8da94b80 1 -- 172.21.15.5:0/585405015 wait complete. 2023-07-21T19:19:41.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.977+0000 7f6c8da94b80 20 client.27755 trim_cache size 0 max 16384 2023-07-21T19:19:41.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.997+0000 7f6c8da94b80 1 Processor -- start 2023-07-21T19:19:41.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.997+0000 7f6c8da94b80 1 -- start start 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea5dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43c60 0x55cd0ea3f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0ea52520 con 0x55cd0ea57360 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0ea46370 con 0x55cd0ea43350 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0ea5e280 con 0x55cd0ea43c60 2023-07-21T19:19:41.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c73fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46282/0 (socket says 172.21.15.5:46282) 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea5dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c73fff700 1 -- 172.21.15.5:0/896810731 learned_addr learned my addr 172.21.15.5:0/896810731 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.998+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43c60 0x55cd0ea3f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 -- 172.21.15.5:0/896810731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43c60 msgr2=0x55cd0ea3f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43c60 0x55cd0ea3f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 -- 172.21.15.5:0/896810731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 msgr2=0x55cd0ea3f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:41.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 -- 172.21.15.5:0/896810731 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd0ea3b660 con 0x55cd0ea43350 2023-07-21T19:19:41.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c73fff700 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:41.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c737fe700 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea5dd40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6c64016040 tx=0x7f6c64005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e6e7c33157f04ba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c5a7fc700 1 -- 172.21.15.5:0/896810731 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6400f910 con 0x55cd0ea43350 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c5a7fc700 1 -- 172.21.15.5:0/896810731 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c64028030 con 0x55cd0ea43350 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:41.999+0000 7f6c5a7fc700 1 -- 172.21.15.5:0/896810731 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640038e0 con 0x55cd0ea43350 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 -- 172.21.15.5:0/896810731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 msgr2=0x55cd0ea5dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea5dd40 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f6c64016040 tx=0x7f6c64005ce0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 -- 172.21.15.5:0/896810731 shutdown_connections 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0ea3f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43350 0x55cd0ea5dd40 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/896810731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43c60 0x55cd0ea3f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 -- 172.21.15.5:0/896810731 >> 172.21.15.5:0/896810731 conn(0x55cd0ea39e20 msgr2=0x55cd0ea4f320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 -- 172.21.15.5:0/896810731 shutdown_connections 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.000+0000 7f6c8da94b80 1 -- 172.21.15.5:0/896810731 wait complete. 2023-07-21T19:19:41.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.001+0000 7f6c8da94b80 1 -- start start 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.001+0000 7f6c8da94b80 1 Processor -- start 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 0x55cd0ea5d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0eab4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0eab4ff0 con 0x55cd0ea57360 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0eab5130 con 0x55cd0ea43c60 2023-07-21T19:19:42.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd0eab52f0 con 0x55cd0ea43350 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c8da94b80 10 client.0 _ll_register_callbacks cb 0x7f6c64009cf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c737fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37916/0 (socket says 172.21.15.5:37916) 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c737fe700 1 -- 172.21.15.5:0/3365518482 learned_addr learned my addr 172.21.15.5:0/3365518482 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.002+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 0x55cd0ea5d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c73fff700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0eab4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 -- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 msgr2=0x55cd0ea5d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 0x55cd0ea5d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 -- 172.21.15.5:0/3365518482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 msgr2=0x55cd0eab4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0eab4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 -- 172.21.15.5:0/3365518482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd0e7c3e60 con 0x55cd0ea43350 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 0x55cd0ea5d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c73fff700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0eab4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c737fe700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c6400f910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10fc41127e3799ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.003+0000 7f6c52ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c640050c0 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c64029080 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c6c00bdc0 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c64005220 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c8da94b80 10 client.27691 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55cd0e7acce0 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c64042020 con 0x55cd0ea43350 2023-07-21T19:19:42.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.004+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c640116b0 con 0x55cd0ea43350 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 10 client.27691 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 10 client.27691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 20 client.27691 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0eabc720 0x55cd0eabeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 4 client.27691 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55cd0e9510d0 con 0x55cd0eabc720 2023-07-21T19:19:42.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.005+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0eabc720 0x55cd0eabeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.006+0000 7f6c72ffd700 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0eabc720 0x55cd0eabeb30 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d6511715724d12c server_cookie=a85870096ae7f91e in_seq=0 out_seq=0 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.006+0000 7f6c52ffd700 10 client.27691 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c52ffd700 1 -- 172.21.15.5:0/3365518482 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55cd0e9510d0 con 0x55cd0eabc720 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c52ffd700 10 client.27691 handle_command_reply: tid=0 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 client.27691 shutdown 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 msgr2=0x55cd0ea5cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f6c64000c00 tx=0x7f6c6400f910 comp rx=0 tx=0).stop 2023-07-21T19:19:42.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 shutdown_connections 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd0eabc720 0x55cd0eabeb30 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd0ea57360 0x55cd0eab4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd0ea43350 0x55cd0ea5cb00 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 --2- 172.21.15.5:0/3365518482 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd0ea43c60 0x55cd0ea5d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 >> 172.21.15.5:0/3365518482 conn(0x55cd0ea39e20 msgr2=0x55cd0ea3f1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c52ffd700 0 client.27691 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 shutdown_connections 2023-07-21T19:19:42.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.007+0000 7f6c8da94b80 1 -- 172.21.15.5:0/3365518482 wait complete. 2023-07-21T19:19:42.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c8da94b80 20 client.27691 trim_cache size 0 max 16384 2023-07-21T19:19:42.009 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:42.009 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:19:42.010 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "52a4b12a-db17-4dcb-93d5-850c9b66c123", 2023-07-21T19:19:42.010 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:19:42.010 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:42.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 msgr2=0x7f6c5c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 0x7f6c5c0a37f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00d010 tx=0x7f6c6c0094b0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 msgr2=0x7f6c741983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6c6800e9a0 tx=0x7f6c6800ecb0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 shutdown_connections 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c5c0a1340 0x7f6c5c0a37f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c7410c320 0x7f6c7419c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c740728b0 0x7f6c74197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 --2- 172.21.15.5:0/2873999599 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c740731c0 0x7f6c741983c0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 >> 172.21.15.5:0/2873999599 conn(0x7f6c740fb5e0 msgr2=0x7f6c7410e450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.008+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 shutdown_connections 2023-07-21T19:19:42.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.009+0000 7f6c7a1ff700 1 -- 172.21.15.5:0/2873999599 wait complete. 2023-07-21T19:19:42.025 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:19:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.298+0000 7f30d55ff700 1 -- 172.21.15.5:0/2830585996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0100490 msgr2=0x7f30d010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.298+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2830585996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0100490 0x7f30d010c920 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f30c800b0a0 tx=0x7f30c800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 -- 172.21.15.5:0/2830585996 shutdown_connections 2023-07-21T19:19:42.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2830585996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0100490 0x7f30d010c920 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2830585996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d0105af0 0x7f30d0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2830585996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d00ffae0 0x7f30d00fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 -- 172.21.15.5:0/2830585996 >> 172.21.15.5:0/2830585996 conn(0x7f30d00fb5e0 msgr2=0x7f30d00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 -- 172.21.15.5:0/2830585996 shutdown_connections 2023-07-21T19:19:42.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.299+0000 7f30d55ff700 1 -- 172.21.15.5:0/2830585996 wait complete. 2023-07-21T19:19:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.300+0000 7f30d55ff700 1 Processor -- start 2023-07-21T19:19:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.300+0000 7f30d55ff700 1 -- start start 2023-07-21T19:19:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.300+0000 7f30d55ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30d55ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 0x7f30d019bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30d55ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 0x7f30d0195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30d55ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0196090 con 0x7f30d0105af0 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30d55ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d0196200 con 0x7f30d0100490 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30d55ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d01963c0 con 0x7f30d00ffae0 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ceffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 0x7f30d0195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ceffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37948/0 (socket says 172.21.15.5:37948) 2023-07-21T19:19:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ceffd700 1 -- 172.21.15.5:0/2141295446 learned_addr learned my addr 172.21.15.5:0/2141295446 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 0x7f30d019bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 -- 172.21.15.5:0/2141295446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 msgr2=0x7f30d019b9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 -- 172.21.15.5:0/2141295446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 msgr2=0x7f30d019bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 0x7f30d019bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30cf7fe700 1 -- 172.21.15.5:0/2141295446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30c8009d00 con 0x7f30d0105af0 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 0x7f30d019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:42.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.301+0000 7f30ceffd700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30cf7fe700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 0x7f30d0195a20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30b7fff700 1 -- 172.21.15.5:0/2141295446 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c800e040 con 0x7f30d0105af0 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30b7fff700 1 -- 172.21.15.5:0/2141295446 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30c800be40 con 0x7f30d0105af0 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30b7fff700 1 -- 172.21.15.5:0/2141295446 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c8005040 con 0x7f30d0105af0 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d0196640 con 0x7f30d0105af0 2023-07-21T19:19:42.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.302+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d0196890 con 0x7f30d0105af0 2023-07-21T19:19:42.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.306+0000 7f30b7fff700 1 -- 172.21.15.5:0/2141295446 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f30c801d480 con 0x7f30d0105af0 2023-07-21T19:19:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.307+0000 7f30e8eb0b80 1 Processor -- start 2023-07-21T19:19:42.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 -- start start 2023-07-21T19:19:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce7fab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 0x55a58ce8a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 0x55a58ce8d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cd81cb0 con 0x55a58ce8aee0 2023-07-21T19:19:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cdd4a40 con 0x55a58ce7f6e0 2023-07-21T19:19:42.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cdcded0 con 0x55a58ce7fff0 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.308+0000 7f30b7fff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 0x7f30b80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 0x55a58ce8d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 0x55a58ce8d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46324/0 (socket says 172.21.15.5:46324) 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ce7fc700 1 -- 172.21.15.5:0/3436805285 learned_addr learned my addr 172.21.15.5:0/3436805285 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 0x55a58ce8a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce7fab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30b7fff700 1 -- 172.21.15.5:0/2141295446 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f30c80cdde0 con 0x7f30d0105af0 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30cf7fe700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 0x7f30b80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 -- 172.21.15.5:0/3436805285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 msgr2=0x55a58ce7fab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce7fab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 -- 172.21.15.5:0/3436805285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 msgr2=0x55a58ce8d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 0x55a58ce8d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.309+0000 7f30ceffd700 1 -- 172.21.15.5:0/3436805285 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a58cbfbde0 con 0x55a58ce7fff0 2023-07-21T19:19:42.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.310+0000 7f30ceffd700 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 0x55a58ce8a9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30c00114c0 tx=0x7f30c00117d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a16820c2f26c69 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.310+0000 7f30cf7fe700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 0x7f30b80a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30c800d010 tx=0x7f30c8009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.310+0000 7f30b57fa700 1 -- 172.21.15.5:0/3436805285 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c0011310 con 0x55a58ce7fff0 2023-07-21T19:19:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.310+0000 7f30b57fa700 1 -- 172.21.15.5:0/3436805285 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30c00044c0 con 0x55a58ce7fff0 2023-07-21T19:19:42.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.310+0000 7f30b57fa700 1 -- 172.21.15.5:0/3436805285 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c001a6e0 con 0x55a58ce7fff0 2023-07-21T19:19:42.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3436805285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 msgr2=0x55a58ce8a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 0x55a58ce8a9a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f30c00114c0 tx=0x7f30c00117d0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3436805285 shutdown_connections 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce8aee0 0x55a58ce8d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce7fab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3436805285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7fff0 0x55a58ce8a9a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3436805285 >> 172.21.15.5:0/3436805285 conn(0x55a58ce71da0 msgr2=0x55a58ce776a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3436805285 shutdown_connections 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.311+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3436805285 wait complete. 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 -- start start 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 Processor -- start 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 0x55a58ce8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce882e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cd6dac0 con 0x55a58ce7f6e0 2023-07-21T19:19:42.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cd6d7c0 con 0x55a58ce7fff0 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.312+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58cc51920 con 0x55a58ce8aee0 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30e8eb0b80 10 client.0 _ll_register_callbacks cb 0x55a58cd6b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44570/0 (socket says 172.21.15.5:44570) 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 -- 172.21.15.5:0/772961329 learned_addr learned my addr 172.21.15.5:0/772961329 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 0x55a58ce8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce882e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 -- 172.21.15.5:0/772961329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 msgr2=0x55a58ce882e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce882e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 -- 172.21.15.5:0/772961329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 msgr2=0x55a58ce8a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 0x55a58ce8a920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ceffd700 1 -- 172.21.15.5:0/772961329 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c0009ce0 con 0x55a58ce7fff0 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.313+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 0x55a58ce8a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30ceffd700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c0005e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf866f92a7f74b24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c001f070 con 0x55a58ce7fff0 2023-07-21T19:19:42.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a58cd274f0 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f30c0003c90 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30c0003e50 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30e8eb0b80 10 client.27773 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.314+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a58cbe4c60 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30c0030020 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f30c0010430 con 0x55a58ce7fff0 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 10 client.27773 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:42.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 10 client.27773 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 20 client.27773 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ce9a780 0x55a58ce9cb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 4 client.27773 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a58cdc6eb0 con 0x55a58ce9a780 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.315+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ce9a780 0x55a58ce9cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.316+0000 7f30b2ff5700 10 client.27773 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.316+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ce9a780 0x55a58ce9cb90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f4e002716a0b418 server_cookie=871a733c969f9b1d in_seq=0 out_seq=0 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30b2ff5700 1 -- 172.21.15.5:0/772961329 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a58cdc6eb0 con 0x55a58ce9a780 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30b2ff5700 10 client.27773 handle_command_reply: tid=0 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 client.27773 shutdown 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 msgr2=0x55a58ce87ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f30c0000c00 tx=0x7f30c0005e20 comp rx=0 tx=0).stop 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 shutdown_connections 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce882e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ce9a780 0x55a58ce9cb90 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7f6e0 0x55a58ce8a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/772961329 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7fff0 0x55a58ce87ba0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30b2ff5700 0 client.27773 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 >> 172.21.15.5:0/772961329 conn(0x55a58ce71da0 msgr2=0x55a58ce77290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 shutdown_connections 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/772961329 wait complete. 2023-07-21T19:19:42.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.317+0000 7f30e8eb0b80 20 client.27773 trim_cache size 0 max 16384 2023-07-21T19:19:42.325 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:19:42.025+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1000000001b [...14,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical/ auth{1=1,2=1} v3968 ap=2 f(v20 m2023-07-21T19:19:19.915959+0000 3=1+2) n(v10 rc2023-07-21T19:19:19.915959+0000 b8 4=1+3) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@114} | dirtyscattered=0 request=0 lock=3 importing=0 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c798000]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x1000000001b:[<0x10000000006/pg_logical v481>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x1000000001b:[<0x10000000006/pg_logical v3968>,<0x10000000005/data v2898>,<0x10000000003/postgres v758>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v782>,<0x10000000001/sv_0 v795>,<0x10000000000/_nogroup v773>,<0x1/volumes v737>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:19:19.898114+0000 4=2+2)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:19:19.915959+0000 b8 4=1+3)","memory_value.dirstat":"f(v20 m2023-07-21T19:19:19.915959+0000 3=1+2)","memory_value.rstat":"n(v10 rc2023-07-21T19:19:19.915959+0000 b8 4=1+3)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:19:42.326 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:19:42.026+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...14,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v3907 ap=2 f(v110 m2023-07-21T19:19:02.733217+0000 3=3+0) n(v11 rc2023-07-21T19:19:02.733217+0000 b3321 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@280} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v3907>,<0x10000000005/data v2898>,<0x10000000003/postgres v758>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v782>,<0x10000000001/sv_0 v795>,<0x10000000000/_nogroup v773>,<0x1/volumes v737>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:19:02.719447+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:19:02.733217+0000 b3321 4=3+1)","memory_value.dirstat":"f(v110 m2023-07-21T19:19:02.733217+0000 3=3+0)","memory_value.rstat":"n(v11 rc2023-07-21T19:19:02.733217+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:19:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.338+0000 7f30e8eb0b80 1 Processor -- start 2023-07-21T19:19:42.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 -- start start 2023-07-21T19:19:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce96b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce8aee0 0x55a58ce8a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ce89180 con 0x55a58ce7fff0 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ce9d1f0 con 0x55a58ce8aee0 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ce86140 con 0x55a58ce7f6e0 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30ceffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce96b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37972/0 (socket says 172.21.15.5:37972) 2023-07-21T19:19:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.339+0000 7f30ce7fc700 1 -- 172.21.15.5:0/3328806829 learned_addr learned my addr 172.21.15.5:0/3328806829 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce8aee0 0x55a58ce8a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 -- 172.21.15.5:0/3328806829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce8aee0 msgr2=0x55a58ce8a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce8aee0 0x55a58ce8a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 -- 172.21.15.5:0/3328806829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 msgr2=0x55a58ce96b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce96b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 -- 172.21.15.5:0/3328806829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a58cc22250 con 0x55a58ce7f6e0 2023-07-21T19:19:42.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55a58ce9d430 tx=0x7f30c4026040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5dc7cb1058f004 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30b5ffb700 1 -- 172.21.15.5:0/3328806829 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4029070 con 0x55a58ce7f6e0 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30b5ffb700 1 -- 172.21.15.5:0/3328806829 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30c40045a0 con 0x55a58ce7f6e0 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.340+0000 7f30b5ffb700 1 -- 172.21.15.5:0/3328806829 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4003680 con 0x55a58ce7f6e0 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3328806829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 msgr2=0x55a58ce873a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x55a58ce9d430 tx=0x7f30c4026040 comp rx=0 tx=0).stop 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3328806829 shutdown_connections 2023-07-21T19:19:42.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce8aee0 0x55a58ce8a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce7f6e0 0x55a58ce873a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/3328806829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce96b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3328806829 >> 172.21.15.5:0/3328806829 conn(0x55a58ce71da0 msgr2=0x55a58ce75e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3328806829 shutdown_connections 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.341+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/3328806829 wait complete. 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 -- start start 2023-07-21T19:19:42.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 Processor -- start 2023-07-21T19:19:42.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce8ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce904e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ce90bf0 con 0x55a58ce7fff0 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ce90d60 con 0x55a58ce7f6e0 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.342+0000 7f30e8eb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a58ceee610 con 0x55a58ce8aee0 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30e8eb0b80 10 client.0 _ll_register_callbacks cb 0x7f30c4012450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44592/0 (socket says 172.21.15.5:44592) 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 -- 172.21.15.5:0/4237194338 learned_addr learned my addr 172.21.15.5:0/4237194338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ceffd700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce8ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 -- 172.21.15.5:0/4237194338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 msgr2=0x55a58ce904e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce904e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 -- 172.21.15.5:0/4237194338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 msgr2=0x55a58ce8ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:42.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce8ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 -- 172.21.15.5:0/4237194338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30c401a040 con 0x55a58ce7f6e0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ceffd700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce8ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30ce7fc700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009780 tx=0x7f30c4009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffcaa791a457feb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.343+0000 7f30b2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4029040 con 0x55a58ce7f6e0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30c00044c0 con 0x55a58ce7f6e0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f30c4012860 con 0x55a58ce7f6e0 2023-07-21T19:19:42.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30c4012a20 con 0x55a58ce7f6e0 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30e8eb0b80 10 client.27776 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55a58cbfa2b0 con 0x55a58ce7f6e0 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f30c00044c0 con 0x55a58ce7f6e0 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.344+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f30c4039020 con 0x55a58ce7f6e0 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 10 client.27776 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 10 client.27776 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:42.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 20 client.27776 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ceefcc0 0x55a58cef20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 4 client.27776 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a58cd89050 con 0x55a58ceefcc0 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30ceffd700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ceefcc0 0x55a58cef20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30ceffd700 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ceefcc0 0x55a58cef20d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf670f3d309a03dd server_cookie=53b065ef7528622c in_seq=0 out_seq=0 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.345+0000 7f30b2ff5700 10 client.27776 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30b2ff5700 1 -- 172.21.15.5:0/4237194338 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55a58cd89050 con 0x55a58ceefcc0 2023-07-21T19:19:42.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30b2ff5700 10 client.27776 handle_command_reply: tid=0 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 client.27776 shutdown 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 msgr2=0x55a58ce91a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f30c4009780 tx=0x7f30c4009260 comp rx=0 tx=0).stop 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 shutdown_connections 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a58ce8aee0 0x55a58ce904e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a58ce7f6e0 0x55a58ce91a20 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a58ceefcc0 0x55a58cef20d0 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 --2- 172.21.15.5:0/4237194338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a58ce7fff0 0x55a58ce8ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30b2ff5700 0 client.27776 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 >> 172.21.15.5:0/4237194338 conn(0x55a58ce71da0 msgr2=0x55a58ce82ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 shutdown_connections 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 1 -- 172.21.15.5:0/4237194338 wait complete. 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.346+0000 7f30e8eb0b80 20 client.27776 trim_cache size 0 max 16384 2023-07-21T19:19:42.348 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "52a4b12a-db17-4dcb-93d5-850c9b66c123": { 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "52a4b12a-db17-4dcb-93d5-850c9b66c123", 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:19:42.349 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:19:42.350 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:19:42.350 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:19:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 msgr2=0x7f30b80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 0x7f30b80a3770 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f30c800d010 tx=0x7f30c8009040 comp rx=0 tx=0).stop 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 msgr2=0x7f30d0195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 0x7f30d0195a20 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f30c8009fd0 tx=0x7f30c8007930 comp rx=0 tx=0).stop 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 shutdown_connections 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f30b80a12c0 0x7f30b80a3770 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f30d0105af0 0x7f30d0195a20 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f30d00ffae0 0x7f30d019b9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 --2- 172.21.15.5:0/2141295446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f30d0100490 0x7f30d019bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 >> 172.21.15.5:0/2141295446 conn(0x7f30d00fb5e0 msgr2=0x7f30d00fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 shutdown_connections 2023-07-21T19:19:42.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:19:42.347+0000 7f30d55ff700 1 -- 172.21.15.5:0/2141295446 wait complete. 2023-07-21T19:19:42.361 INFO:tasks.cephfs.filesystem:scrub status for tag:52a4b12a-db17-4dcb-93d5-850c9b66c123 - {'path': '/', 'tag': '52a4b12a-db17-4dcb-93d5-850c9b66c123', 'options': 'recursive,force'} 2023-07-21T19:19:42.533 INFO:teuthology.orchestra.run.smithi005.stderr:46500000 of 50000000 tuples (93%) done (elapsed 700.11 s, remaining 52.70 s) 2023-07-21T19:19:42.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:19:42.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:42.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:42.653 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.654 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:42 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:19:42.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:42 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:42.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: Scrub error on inode 0x1000000001b (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_logical) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:42.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:42 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:19:43.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:43 smithi101 ceph-mon[128136]: pgmap v378: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 32 MiB/s wr, 173 op/s 2023-07-21T19:19:43.849 INFO:teuthology.orchestra.run.smithi005.stderr:46600000 of 50000000 tuples (93%) done (elapsed 701.42 s, remaining 51.18 s) 2023-07-21T19:19:43.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:43 smithi005 ceph-mon[126460]: pgmap v378: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 32 MiB/s wr, 173 op/s 2023-07-21T19:19:43.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:43 smithi156 ceph-mon[128316]: pgmap v378: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 193 KiB/s rd, 32 MiB/s wr, 173 op/s 2023-07-21T19:19:45.170 INFO:teuthology.orchestra.run.smithi005.stderr:46700000 of 50000000 tuples (93%) done (elapsed 702.74 s, remaining 49.66 s) 2023-07-21T19:19:45.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:45 smithi101 ceph-mon[128136]: pgmap v379: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 23 MiB/s wr, 167 op/s 2023-07-21T19:19:45.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:45 smithi005 ceph-mon[126460]: pgmap v379: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 23 MiB/s wr, 167 op/s 2023-07-21T19:19:45.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:45 smithi156 ceph-mon[128316]: pgmap v379: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 250 KiB/s rd, 23 MiB/s wr, 167 op/s 2023-07-21T19:19:46.455 INFO:teuthology.orchestra.run.smithi005.stderr:46800000 of 50000000 tuples (93%) done (elapsed 704.03 s, remaining 48.14 s) 2023-07-21T19:19:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.550+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/1003571169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88067c10 msgr2=0x7f6c88067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.550+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/1003571169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88067c10 0x7f6c88067fe0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f6c780099c0 tx=0x7f6c78009cd0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/1003571169 shutdown_connections 2023-07-21T19:19:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/1003571169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88104e50 0x7f6c8810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/1003571169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88067c10 0x7f6c88067fe0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/1003571169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 0x7f6c88104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/1003571169 >> 172.21.15.5:0/1003571169 conn(0x7f6c88074760 msgr2=0x7f6c88074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/1003571169 shutdown_connections 2023-07-21T19:19:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.551+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/1003571169 wait complete. 2023-07-21T19:19:47.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.552+0000 7f6c8d5ff700 1 Processor -- start 2023-07-21T19:19:47.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.552+0000 7f6c8d5ff700 1 -- start start 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 0x7f6c8819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 0x7f6c8819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88196090 con 0x7f6c88104e50 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c88196200 con 0x7f6c880685b0 2023-07-21T19:19:47.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c8d5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c881963c0 con 0x7f6c88067c10 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55380/0 (socket says 172.21.15.5:55380) 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.553+0000 7f6c877fe700 1 -- 172.21.15.5:0/399931473 learned_addr learned my addr 172.21.15.5:0/399931473 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c867fc700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 0x7f6c8819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 0x7f6c8819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 -- 172.21.15.5:0/399931473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 msgr2=0x7f6c8819bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 0x7f6c8819bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 -- 172.21.15.5:0/399931473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 msgr2=0x7f6c8819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 0x7f6c8819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 -- 172.21.15.5:0/399931473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c780095f0 con 0x7f6c88104e50 2023-07-21T19:19:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 0x7f6c8819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:47.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c877fe700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007ee0 tx=0x7f6c8000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.554+0000 7f6c6ffff700 1 -- 172.21.15.5:0/399931473 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80017070 con 0x7f6c88104e50 2023-07-21T19:19:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.555+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c881966a0 con 0x7f6c88104e50 2023-07-21T19:19:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.555+0000 7f6c6ffff700 1 -- 172.21.15.5:0/399931473 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c8000f040 con 0x7f6c88104e50 2023-07-21T19:19:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.555+0000 7f6c6ffff700 1 -- 172.21.15.5:0/399931473 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c80005710 con 0x7f6c88104e50 2023-07-21T19:19:47.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.555+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c881a2bb0 con 0x7f6c88104e50 2023-07-21T19:19:47.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.558+0000 7f6c6ffff700 1 -- 172.21.15.5:0/399931473 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6c80005870 con 0x7f6c88104e50 2023-07-21T19:19:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.560+0000 7f6c6ffff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 0x7f6c700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.560+0000 7f6c6ffff700 1 -- 172.21.15.5:0/399931473 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c800cc510 con 0x7f6c88104e50 2023-07-21T19:19:47.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.560+0000 7f6ca0fc7b80 1 Processor -- start 2023-07-21T19:19:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 0x7f6c700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 -- start start 2023-07-21T19:19:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07536cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07427c50 con 0x55da0737e6b0 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da0747a9e0 con 0x55da07522040 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07473e70 con 0x55da075229d0 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6c867fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.561+0000 7f6c867fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34336/0 (socket says 172.21.15.5:34336) 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 -- 172.21.15.5:0/4108412437 learned_addr learned my addr 172.21.15.5:0/4108412437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c877fe700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07536cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 0x7f6c700a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c780036c0 tx=0x7f6c78009f90 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 -- 172.21.15.5:0/4108412437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 msgr2=0x55da07536cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07536cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 -- 172.21.15.5:0/4108412437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 msgr2=0x55da07525e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 -- 172.21.15.5:0/4108412437 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da072a1d80 con 0x55da07522040 2023-07-21T19:19:47.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:19:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.562+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00bcb0 tx=0x7f6c7c0103d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9e18c62260bc7d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.563+0000 7f6c6d7fa700 1 -- 172.21.15.5:0/4108412437 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c010d10 con 0x55da07522040 2023-07-21T19:19:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.563+0000 7f6c6d7fa700 1 -- 172.21.15.5:0/4108412437 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7c015070 con 0x55da07522040 2023-07-21T19:19:47.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.563+0000 7f6c6d7fa700 1 -- 172.21.15.5:0/4108412437 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7c005020 con 0x55da07522040 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.563+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4108412437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 msgr2=0x55da07522490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.563+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c00bcb0 tx=0x7f6c7c0103d0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4108412437 shutdown_connections 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07536cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4108412437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07522490 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4108412437 >> 172.21.15.5:0/4108412437 conn(0x55da07517be0 msgr2=0x55da0751cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4108412437 shutdown_connections 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4108412437 wait complete. 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.564+0000 7f6ca0fc7b80 1 -- start start 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 Processor -- start 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07533ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07534560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07413a60 con 0x55da0737e6b0 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07413760 con 0x55da07522040 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da072f78c0 con 0x55da075229d0 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6ca0fc7b80 10 client.0 _ll_register_callbacks cb 0x55da07411700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55394/0 (socket says 172.21.15.5:55394) 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.565+0000 7f6c86ffd700 1 -- 172.21.15.5:0/4037761893 learned_addr learned my addr 172.21.15.5:0/4037761893 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:47.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c877fe700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07534560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07533ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 -- 172.21.15.5:0/4037761893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 msgr2=0x55da07534560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07534560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 -- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 msgr2=0x55da07533ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07533ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 -- 172.21.15.5:0/4037761893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c00b8e0 con 0x55da0737e6b0 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c78015000 tx=0x7f6c7801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecdb2a151de6ae29 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.566+0000 7f6c6aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7802b040 con 0x55da0737e6b0 2023-07-21T19:19:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da073cd490 con 0x55da0737e6b0 2023-07-21T19:19:47.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c780040c0 con 0x55da0737e6b0 2023-07-21T19:19:47.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c7802f420 con 0x55da0737e6b0 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6ca0fc7b80 10 client.18129 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55da0728ac00 con 0x55da0737e6b0 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.567+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7803c020 con 0x55da0737e6b0 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c7800f5f0 con 0x55da0737e6b0 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 10 client.18129 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 10 client.18129 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 20 client.18129 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:47.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da075443d0 0x55da075467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 4 client.18129 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da0746ce50 con 0x55da075443d0 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.568+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da075443d0 0x55da075467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.569+0000 7f6c6aff5700 10 client.18129 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.569+0000 7f6c867fc700 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da075443d0 0x55da075467e0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0998e8b805748c0 server_cookie=a3e38f5ad22574f5 in_seq=0 out_seq=0 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6c6aff5700 1 -- 172.21.15.5:0/4037761893 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da0746ce50 con 0x55da075443d0 2023-07-21T19:19:47.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6c6aff5700 10 client.18129 handle_command_reply: tid=0 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 client.18129 shutdown 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 msgr2=0x55da07536bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f6c78015000 tx=0x7f6c7801a050 comp rx=0 tx=0).stop 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 shutdown_connections 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da075229d0 0x55da07534560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da075443d0 0x55da075467e0 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da0737e6b0 0x55da07536bf0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/4037761893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07533ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 >> 172.21.15.5:0/4037761893 conn(0x55da07517be0 msgr2=0x55da0751c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6c6aff5700 0 client.18129 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 shutdown_connections 2023-07-21T19:19:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/4037761893 wait complete. 2023-07-21T19:19:47.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.570+0000 7f6ca0fc7b80 20 client.18129 trim_cache size 0 max 16384 2023-07-21T19:19:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 Processor -- start 2023-07-21T19:19:47.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 -- start start 2023-07-21T19:19:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da0753b470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07536790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07534e70 con 0x55da075229d0 2023-07-21T19:19:47.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07535420 con 0x55da07522040 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.589+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07522da0 con 0x55da0737e6b0 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07536790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07536790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55402/0 (socket says 172.21.15.5:55402) 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c877fe700 1 -- 172.21.15.5:0/3807515237 learned_addr learned my addr 172.21.15.5:0/3807515237 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c867fc700 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da0753b470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 -- 172.21.15.5:0/3807515237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 msgr2=0x55da0753b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da0753b470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 -- 172.21.15.5:0/3807515237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 msgr2=0x55da07536790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07536790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 -- 172.21.15.5:0/3807515237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da072c81f0 con 0x55da0737e6b0 2023-07-21T19:19:47.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.590+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07525e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c78015000 tx=0x7f6c78009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38f16073fa23853 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.591+0000 7f6c6dffb700 1 -- 172.21.15.5:0/3807515237 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c7802b070 con 0x55da0737e6b0 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.591+0000 7f6c6dffb700 1 -- 172.21.15.5:0/3807515237 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c7803c070 con 0x55da0737e6b0 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.591+0000 7f6c6dffb700 1 -- 172.21.15.5:0/3807515237 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c780053b0 con 0x55da0737e6b0 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.591+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/3807515237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 msgr2=0x55da07525e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.591+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07525e20 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f6c78015000 tx=0x7f6c78009310 comp rx=0 tx=0).stop 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/3807515237 shutdown_connections 2023-07-21T19:19:47.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07536790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07525e20 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/3807515237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da0753b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/3807515237 >> 172.21.15.5:0/3807515237 conn(0x55da07517be0 msgr2=0x55da075406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/3807515237 shutdown_connections 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/3807515237 wait complete. 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 -- start start 2023-07-21T19:19:47.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.592+0000 7f6ca0fc7b80 1 Processor -- start 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07539640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07539b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da0753a300 con 0x55da075229d0 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da075977d0 con 0x55da07522040 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da07597940 con 0x55da0737e6b0 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6ca0fc7b80 10 client.0 _ll_register_callbacks cb 0x7f6c7800bb00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:19:47.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07539640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6c86ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07539640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57024/0 (socket says 172.21.15.5:57024) 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6c877fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55412/0 (socket says 172.21.15.5:55412) 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.593+0000 7f6c86ffd700 1 -- 172.21.15.5:0/844953232 learned_addr learned my addr 172.21.15.5:0/844953232 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c867fc700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07539b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 -- 172.21.15.5:0/844953232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 msgr2=0x55da07539640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07539640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 -- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 msgr2=0x55da07539b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:19:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07539b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 -- 172.21.15.5:0/844953232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da0728ac00 con 0x55da075229d0 2023-07-21T19:19:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c877fe700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6c80096050 tx=0x7f6c80007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5acce9cf9ae269dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:19:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c6aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:19:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c800ca0e0 con 0x55da075229d0 2023-07-21T19:19:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6c8009d610 con 0x55da075229d0 2023-07-21T19:19:47.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.594+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6c7c015070 con 0x55da075229d0 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=43) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c800a1a30 con 0x55da075229d0 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 10 client.18135 fetch_fsmap learned FSMap version 43 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=43,monmap=4+}) v3 -- 0x55da072a0250 con 0x55da075229d0 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6c800c7080 con 0x55da075229d0 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 43) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f6c800998b0 con 0x55da075229d0 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 10 client.18135 fetch_fsmap finished waiting for FSMap version 43 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 10 client.18135 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:19:47.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 20 client.18135 populate_metadata read hostname 'smithi005' 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da07599100 0x55da0759b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 4 client.18135 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.595+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da0742eff0 con 0x55da07599100 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.596+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da07599100 0x55da0759b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.596+0000 7f6c86ffd700 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da07599100 0x55da0759b510 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=778016749685ddff server_cookie=f89b5d415da54a94 in_seq=0 out_seq=0 2023-07-21T19:19:47.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.596+0000 7f6c6aff5700 10 client.18135 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6c6aff5700 1 -- 172.21.15.5:0/844953232 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da0742eff0 con 0x55da07599100 2023-07-21T19:19:47.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6c6aff5700 10 client.18135 handle_command_reply: tid=0 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 client.18135 shutdown 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 msgr2=0x55da07597290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f6c80096050 tx=0x7f6c80007f90 comp rx=0 tx=0).stop 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 shutdown_connections 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da07599100 0x55da0759b510 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da075229d0 0x55da07597290 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6c6aff5700 0 client.18135 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:19:47.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da0737e6b0 0x55da07539640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 --2- 172.21.15.5:0/844953232 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da07522040 0x55da07539b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 >> 172.21.15.5:0/844953232 conn(0x55da07517be0 msgr2=0x55da0752f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 shutdown_connections 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.612+0000 7f6ca0fc7b80 1 -- 172.21.15.5:0/844953232 wait complete. 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6ca0fc7b80 20 client.18135 trim_cache size 0 max 16384 2023-07-21T19:19:47.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:19:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:19:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:19:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:19:47.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 msgr2=0x7f6c700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 0x7f6c700a3770 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f6c780036c0 tx=0x7f6c78009f90 comp rx=0 tx=0).stop 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 msgr2=0x7f6c88195a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f6c80007ee0 tx=0x7f6c8000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 shutdown_connections 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6c700a12c0 0x7f6c700a3770 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6c88104e50 0x7f6c88195a20 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6c88067c10 0x7f6c8819bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 --2- 172.21.15.5:0/399931473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6c880685b0 0x7f6c8819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:19:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 >> 172.21.15.5:0/399931473 conn(0x7f6c88074760 msgr2=0x7f6c880fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:19:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 shutdown_connections 2023-07-21T19:19:47.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:19:47.613+0000 7f6c8d5ff700 1 -- 172.21.15.5:0/399931473 wait complete. 2023-07-21T19:19:47.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:47 smithi101 ceph-mon[128136]: pgmap v380: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 151 op/s 2023-07-21T19:19:47.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:47 smithi005 ceph-mon[126460]: pgmap v380: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 151 op/s 2023-07-21T19:19:47.906 INFO:teuthology.orchestra.run.smithi005.stderr:46900000 of 50000000 tuples (93%) done (elapsed 705.48 s, remaining 46.63 s) 2023-07-21T19:19:47.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:47 smithi156 ceph-mon[128316]: pgmap v380: 97 pgs: 97 active+clean; 9.3 GiB data, 4.1 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 151 op/s 2023-07-21T19:19:49.213 INFO:teuthology.orchestra.run.smithi005.stderr:47000000 of 50000000 tuples (94%) done (elapsed 706.78 s, remaining 45.11 s) 2023-07-21T19:19:49.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:49 smithi101 ceph-mon[128136]: pgmap v381: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 33 MiB/s wr, 185 op/s 2023-07-21T19:19:49.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:49 smithi005 ceph-mon[126460]: pgmap v381: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 33 MiB/s wr, 185 op/s 2023-07-21T19:19:49.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:49 smithi156 ceph-mon[128316]: pgmap v381: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 192 KiB/s rd, 33 MiB/s wr, 185 op/s 2023-07-21T19:19:50.513 INFO:teuthology.orchestra.run.smithi005.stderr:47100000 of 50000000 tuples (94%) done (elapsed 708.09 s, remaining 43.60 s) 2023-07-21T19:19:51.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:51 smithi101 ceph-mon[128136]: pgmap v382: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 160 op/s 2023-07-21T19:19:51.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:51 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:51.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:51 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:51.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:51 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:19:51.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:51 smithi005 ceph-mon[126460]: pgmap v382: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 160 op/s 2023-07-21T19:19:51.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:51 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:51.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:51 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:51.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:51 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:19:51.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:51 smithi156 ceph-mon[128316]: pgmap v382: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 21 MiB/s wr, 160 op/s 2023-07-21T19:19:51.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:51 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:51 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:51 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:19:51.921 INFO:teuthology.orchestra.run.smithi005.stderr:47200000 of 50000000 tuples (94%) done (elapsed 709.49 s, remaining 42.09 s) 2023-07-21T19:19:53.233 INFO:teuthology.orchestra.run.smithi005.stderr:47300000 of 50000000 tuples (94%) done (elapsed 710.80 s, remaining 40.57 s) 2023-07-21T19:19:53.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:53 smithi101 ceph-mon[128136]: pgmap v383: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 32 MiB/s wr, 186 op/s 2023-07-21T19:19:53.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:53 smithi005 ceph-mon[126460]: pgmap v383: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 32 MiB/s wr, 186 op/s 2023-07-21T19:19:53.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:53 smithi156 ceph-mon[128316]: pgmap v383: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 179 KiB/s rd, 32 MiB/s wr, 186 op/s 2023-07-21T19:19:54.551 INFO:teuthology.orchestra.run.smithi005.stderr:47400000 of 50000000 tuples (94%) done (elapsed 712.12 s, remaining 39.06 s) 2023-07-21T19:19:54.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:54 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:54.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:54 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:54.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:54 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:19:55.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:55 smithi005 ceph-mon[126460]: pgmap v384: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 23 MiB/s wr, 87 op/s 2023-07-21T19:19:55.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:55 smithi156 ceph-mon[128316]: pgmap v384: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 23 MiB/s wr, 87 op/s 2023-07-21T19:19:55.948 INFO:teuthology.orchestra.run.smithi005.stderr:47500000 of 50000000 tuples (95%) done (elapsed 713.52 s, remaining 37.55 s) 2023-07-21T19:19:56.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:55 smithi101 ceph-mon[128136]: pgmap v384: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 23 MiB/s wr, 87 op/s 2023-07-21T19:19:57.225 INFO:teuthology.orchestra.run.smithi005.stderr:47600000 of 50000000 tuples (95%) done (elapsed 714.80 s, remaining 36.04 s) 2023-07-21T19:19:57.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:57 smithi005 ceph-mon[126460]: pgmap v385: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:57.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:57 smithi156 ceph-mon[128316]: pgmap v385: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:58.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:57 smithi101 ceph-mon[128136]: pgmap v385: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 23 MiB/s wr, 63 op/s 2023-07-21T19:19:58.567 INFO:teuthology.orchestra.run.smithi005.stderr:47700000 of 50000000 tuples (95%) done (elapsed 716.14 s, remaining 34.53 s) 2023-07-21T19:19:58.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:19:58 smithi005 ceph-mon[126460]: pgmap v386: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 82 op/s 2023-07-21T19:19:58.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:19:58 smithi156 ceph-mon[128316]: pgmap v386: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 82 op/s 2023-07-21T19:19:59.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:19:58 smithi101 ceph-mon[128136]: pgmap v386: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 34 MiB/s wr, 82 op/s 2023-07-21T19:19:59.884 INFO:teuthology.orchestra.run.smithi005.stderr:47800000 of 50000000 tuples (95%) done (elapsed 717.46 s, remaining 33.02 s) 2023-07-21T19:20:00.325 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:00 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T19:20:00.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:00 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T19:20:00.414 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:00 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T19:20:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.805+0000 7f79b77ff700 1 -- 172.21.15.5:0/2210587953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00682b0 msgr2=0x7f79b010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.805+0000 7f79b77ff700 1 --2- 172.21.15.5:0/2210587953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00682b0 0x7f79b010c000 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f79a0009990 tx=0x7f79a0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 -- 172.21.15.5:0/2210587953 shutdown_connections 2023-07-21T19:20:00.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 --2- 172.21.15.5:0/2210587953 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b010c540 0x7f79b010e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 --2- 172.21.15.5:0/2210587953 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b00679a0 0x7f79b0067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 --2- 172.21.15.5:0/2210587953 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00682b0 0x7f79b010c000 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 -- 172.21.15.5:0/2210587953 >> 172.21.15.5:0/2210587953 conn(0x7f79b0074760 msgr2=0x7f79b0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 -- 172.21.15.5:0/2210587953 shutdown_connections 2023-07-21T19:20:00.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.806+0000 7f79b77ff700 1 -- 172.21.15.5:0/2210587953 wait complete. 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.807+0000 7f79b77ff700 1 Processor -- start 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.807+0000 7f79b77ff700 1 -- start start 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 0x7f79b019dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 0x7f79b0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01981c0 con 0x7f79b00679a0 2023-07-21T19:20:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0198330 con 0x7f79b00682b0 2023-07-21T19:20:00.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b77ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b01984f0 con 0x7f79b010c540 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 0x7f79b019dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b5d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 0x7f79b0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38416/0 (socket says 172.21.15.5:38416) 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.808+0000 7f79b4d9a700 1 -- 172.21.15.5:0/3392656412 learned_addr learned my addr 172.21.15.5:0/3392656412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:00.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 -- 172.21.15.5:0/3392656412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 msgr2=0x7f79b0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 0x7f79b0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 -- 172.21.15.5:0/3392656412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 msgr2=0x7f79b019e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 -- 172.21.15.5:0/3392656412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a00095f0 con 0x7f79b00679a0 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 0x7f79b0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:00.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.809+0000 7f79b559b700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 0x7f79b019dcf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.810+0000 7f79a67fc700 1 -- 172.21.15.5:0/3392656412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac00eee0 con 0x7f79b00679a0 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.810+0000 7f79a67fc700 1 -- 172.21.15.5:0/3392656412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79ac00cbd0 con 0x7f79b00679a0 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.810+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b01987d0 con 0x7f79b00679a0 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.810+0000 7f79a67fc700 1 -- 172.21.15.5:0/3392656412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac018680 con 0x7f79b00679a0 2023-07-21T19:20:00.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.810+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01a4ce0 con 0x7f79b00679a0 2023-07-21T19:20:00.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.815+0000 7f79a67fc700 1 -- 172.21.15.5:0/3392656412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f79ac00cd40 con 0x7f79b00679a0 2023-07-21T19:20:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.815+0000 7f79cb11bb80 1 Processor -- start 2023-07-21T19:20:00.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 -- start start 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6da1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6da21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6c9ec50 con 0x55c8f6da1d90 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6cf19e0 con 0x55c8f6da7ae0 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.816+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6ceae70 con 0x55c8f6bf56b0 2023-07-21T19:20:00.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b5d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b5d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34692/0 (socket says 172.21.15.5:34692) 2023-07-21T19:20:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6da1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b5d9c700 1 -- 172.21.15.5:0/1524100420 learned_addr learned my addr 172.21.15.5:0/1524100420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:00.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6da21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 -- 172.21.15.5:0/1524100420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 msgr2=0x55c8f6da75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 -- 172.21.15.5:0/1524100420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 msgr2=0x55c8f6da21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6da21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.817+0000 7f79b559b700 1 -- 172.21.15.5:0/1524100420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f6b18d80 con 0x55c8f6da7ae0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f79b559b700 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6da1850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0ca610 tx=0x7f79ac0d5410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=493589578ec2cb7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f79a67fc700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 0x7f799c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f799b7fe700 1 -- 172.21.15.5:0/1524100420 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0962d0 con 0x55c8f6da7ae0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f79a67fc700 1 -- 172.21.15.5:0/3392656412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f79ac014070 con 0x7f79b00679a0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f799b7fe700 1 -- 172.21.15.5:0/1524100420 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79ac099d60 con 0x55c8f6da7ae0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.818+0000 7f799b7fe700 1 -- 172.21.15.5:0/1524100420 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0998f0 con 0x55c8f6da7ae0 2023-07-21T19:20:00.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 0x7f799c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1524100420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 msgr2=0x55c8f6da1850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6da1850 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0ca610 tx=0x7f79ac0d5410 comp rx=0 tx=0).stop 2023-07-21T19:20:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1524100420 shutdown_connections 2023-07-21T19:20:00.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6da21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1524100420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6da1850 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.819+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1524100420 >> 172.21.15.5:0/1524100420 conn(0x55c8f6d8ebe0 msgr2=0x55c8f6d93f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.820+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 0x7f799c0a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009c10 tx=0x7f79a800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.820+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1524100420 shutdown_connections 2023-07-21T19:20:00.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.820+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1524100420 wait complete. 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.820+0000 7f79cb11bb80 1 -- start start 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 Processor -- start 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 0x55c8f6dade30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da7ae0 0x55c8f6dab810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6c8aa60 con 0x55c8f6da7ae0 2023-07-21T19:20:00.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6c8a760 con 0x55c8f6bf56b0 2023-07-21T19:20:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6b6e8c0 con 0x55c8f6da1d90 2023-07-21T19:20:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.821+0000 7f79cb11bb80 10 client.0 _ll_register_callbacks cb 0x55c8f6c88700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 0x55c8f6dade30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34706/0 (socket says 172.21.15.5:34706) 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2386364498 learned_addr learned my addr 172.21.15.5:0/2386364498 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 0x55c8f6dade30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38430/0 (socket says 172.21.15.5:38430) 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da7ae0 0x55c8f6dab810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 msgr2=0x55c8f6dade30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 0x55c8f6dade30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2386364498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da7ae0 msgr2=0x55c8f6dab810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da7ae0 0x55c8f6dab810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.822+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2386364498 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b01a4ce0 con 0x55c8f6da1d90 2023-07-21T19:20:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79a00052a0 tx=0x7f79a0005c90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68514ba2c135b7b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f7998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:00.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a001d070 con 0x55c8f6da1d90 2023-07-21T19:20:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f79a0004910 con 0x55c8f6da1d90 2023-07-21T19:20:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c8f6c44490 con 0x55c8f6da1d90 2023-07-21T19:20:00.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79a0004ad0 con 0x55c8f6da1d90 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.823+0000 7f79cb11bb80 10 client.27736 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c8f6b01c00 con 0x55c8f6da1d90 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c8f6c44490 con 0x55c8f6da1d90 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f79a000fcc0 con 0x55c8f6da1d90 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 10 client.27736 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 10 client.27736 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 20 client.27736 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:00.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6dbb490 0x55c8f6dbd8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 4 client.27736 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.824+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c8f6ce3e50 con 0x55c8f6dbb490 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.825+0000 7f79b559b700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6dbb490 0x55c8f6dbd8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.825+0000 7f79b559b700 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6dbb490 0x55c8f6dbd8a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24b4e4132d004f71 server_cookie=27f808f9fa4b3290 in_seq=0 out_seq=0 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.825+0000 7f7998ff9700 10 client.27736 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:00.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f7998ff9700 1 -- 172.21.15.5:0/2386364498 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c8f6ce3e50 con 0x55c8f6dbb490 2023-07-21T19:20:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f7998ff9700 10 client.27736 handle_command_reply: tid=0 2023-07-21T19:20:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 client.27736 shutdown 2023-07-21T19:20:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 msgr2=0x55c8f6dab120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f79a00052a0 tx=0x7f79a0005c90 comp rx=0 tx=0).stop 2023-07-21T19:20:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 shutdown_connections 2023-07-21T19:20:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6dbb490 0x55c8f6dbd8a0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da7ae0 0x55c8f6dab810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6bf56b0 0x55c8f6dade30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2386364498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da1d90 0x55c8f6dab120 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f7998ff9700 0 client.27736 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 >> 172.21.15.5:0/2386364498 conn(0x55c8f6d8ebe0 msgr2=0x55c8f6d92c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 shutdown_connections 2023-07-21T19:20:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.826+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2386364498 wait complete. 2023-07-21T19:20:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.827+0000 7f79cb11bb80 20 client.27736 trim_cache size 0 max 16384 2023-07-21T19:20:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 Processor -- start 2023-07-21T19:20:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 -- start start 2023-07-21T19:20:00.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 0x55c8f6db2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da7ae0 0x55c8f6daa930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6dac700 con 0x55c8f6bf56b0 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.846+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6da2810 con 0x55c8f6da1d90 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6da9700 con 0x55c8f6da7ae0 2023-07-21T19:20:00.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41416/0 (socket says 172.21.15.5:41416) 2023-07-21T19:20:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b559b700 1 -- 172.21.15.5:0/2131170401 learned_addr learned my addr 172.21.15.5:0/2131170401 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 0x55c8f6db2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da7ae0 0x55c8f6daa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2131170401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da7ae0 msgr2=0x55c8f6daa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da7ae0 0x55c8f6daa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2131170401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 msgr2=0x55c8f6da1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b4d9a700 1 -- 172.21.15.5:0/2131170401 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c8f6b3f1f0 con 0x55c8f6da1d90 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.847+0000 7f79b559b700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 0x55c8f6db2470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f79a0024450 tx=0x7f79a000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4d3f4b07b7e7751 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f799bfff700 1 -- 172.21.15.5:0/2131170401 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a001d040 con 0x55c8f6da1d90 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f799bfff700 1 -- 172.21.15.5:0/2131170401 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79a001e040 con 0x55c8f6da1d90 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f799bfff700 1 -- 172.21.15.5:0/2131170401 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0023660 con 0x55c8f6da1d90 2023-07-21T19:20:00.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2131170401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 msgr2=0x55c8f6db2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.848+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 0x55c8f6db2470 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f79a0024450 tx=0x7f79a000b410 comp rx=0 tx=0).stop 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2131170401 shutdown_connections 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6da7ae0 0x55c8f6daa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6bf56b0 0x55c8f6da1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/2131170401 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da1d90 0x55c8f6db2470 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2131170401 >> 172.21.15.5:0/2131170401 conn(0x55c8f6d8ebe0 msgr2=0x55c8f6dada60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2131170401 shutdown_connections 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 -- 172.21.15.5:0/2131170401 wait complete. 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 -- start start 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.849+0000 7f79cb11bb80 1 Processor -- start 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6db0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6e0e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6e0e890 con 0x55c8f6da1d90 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6e0e9d0 con 0x55c8f6da7ae0 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c8f6e0eb10 con 0x55c8f6bf56b0 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79cb11bb80 10 client.0 _ll_register_callbacks cb 0x7f79a000bf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79b4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41428/0 (socket says 172.21.15.5:41428) 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79b559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6db0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.850+0000 7f79b4d9a700 1 -- 172.21.15.5:0/1979225412 learned_addr learned my addr 172.21.15.5:0/1979225412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6e0e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 -- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 msgr2=0x55c8f6e0e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6e0e350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 -- 172.21.15.5:0/1979225412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 msgr2=0x55c8f6db0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 -- 172.21.15.5:0/1979225412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c8f6b18d80 con 0x55c8f6bf56b0 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:00.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b5d9c700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6e0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79b559b700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6db0650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0a1eb0 tx=0x7f79ac0cd020 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbaa08fae1c26b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f7998ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ac0962d0 con 0x55c8f6bf56b0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(100..100 src has 1..100) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f79ac0d04c0 con 0x55c8f6bf56b0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.851+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f79ac099d60 con 0x55c8f6bf56b0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79ac099ad0 con 0x55c8f6bf56b0 2023-07-21T19:20:00.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 10 client.27742 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55c8f6b01c00 con 0x55c8f6bf56b0 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f79ac099d60 con 0x55c8f6bf56b0 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f79ac0b8240 con 0x55c8f6bf56b0 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 10 client.27742 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 10 client.27742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 20 client.27742 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6e0fe10 0x55c8f6e12220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:00.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 4 client.27742 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:20:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c8f6ca5ff0 con 0x55c8f6e0fe10 2023-07-21T19:20:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.852+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6e0fe10 0x55c8f6e12220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.853+0000 7f79b4d9a700 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6e0fe10 0x55c8f6e12220 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d48006caf1a6ae server_cookie=30e8a2142e2fcb3f in_seq=0 out_seq=0 2023-07-21T19:20:00.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.853+0000 7f7998ff9700 10 client.27742 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.865+0000 7f7998ff9700 1 -- 172.21.15.5:0/1979225412 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c8f6ca5ff0 con 0x55c8f6e0fe10 2023-07-21T19:20:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.865+0000 7f7998ff9700 10 client.27742 handle_command_reply: tid=0 2023-07-21T19:20:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.865+0000 7f79cb11bb80 1 client.27742 shutdown 2023-07-21T19:20:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.865+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 msgr2=0x55c8f6db0650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.865+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6db0650 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f79ac0a1eb0 tx=0x7f79ac0cd020 comp rx=0 tx=0).stop 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 shutdown_connections 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c8f6da7ae0 0x55c8f6e0e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c8f6bf56b0 0x55c8f6db0650 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c8f6e0fe10 0x55c8f6e12220 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 --2- 172.21.15.5:0/1979225412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c8f6da1d90 0x55c8f6db0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f7998ff9700 0 client.27742 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 >> 172.21.15.5:0/1979225412 conn(0x55c8f6d8ebe0 msgr2=0x55c8f6da53c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 shutdown_connections 2023-07-21T19:20:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 1 -- 172.21.15.5:0/1979225412 wait complete. 2023-07-21T19:20:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.866+0000 7f79cb11bb80 20 client.27742 trim_cache size 0 max 16384 2023-07-21T19:20:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:20:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:20:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:20:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:20:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 msgr2=0x7f799c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 0x7f799c0a37f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f79a8009c10 tx=0x7f79a800b040 comp rx=0 tx=0).stop 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 msgr2=0x7f79b019dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 0x7f79b019dcf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f79ac00e9f0 tx=0x7f79ac00ed00 comp rx=0 tx=0).stop 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 shutdown_connections 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f79b010c540 0x7f79b0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f799c0a1340 0x7f799c0a37f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f79b00679a0 0x7f79b019dcf0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 --2- 172.21.15.5:0/3392656412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f79b00682b0 0x7f79b019e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 >> 172.21.15.5:0/3392656412 conn(0x7f79b0074760 msgr2=0x7f79b00fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 shutdown_connections 2023-07-21T19:20:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:00.867+0000 7f79b77ff700 1 -- 172.21.15.5:0/3392656412 wait complete. 2023-07-21T19:20:01.268 INFO:teuthology.orchestra.run.smithi005.stderr:47900000 of 50000000 tuples (95%) done (elapsed 718.84 s, remaining 31.51 s) 2023-07-21T19:20:01.325 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:01 smithi101 ceph-mon[128136]: pgmap v387: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:20:01.402 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:01 smithi005 ceph-mon[126460]: pgmap v387: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:20:01.414 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:01 smithi156 ceph-mon[128316]: pgmap v387: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 MiB/s wr, 49 op/s 2023-07-21T19:20:02.693 INFO:teuthology.orchestra.run.smithi005.stderr:48000000 of 50000000 tuples (96%) done (elapsed 720.27 s, remaining 30.01 s) 2023-07-21T19:20:03.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:03 smithi101 ceph-mon[128136]: pgmap v388: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:20:03.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:03 smithi005 ceph-mon[126460]: pgmap v388: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:20:03.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:03 smithi156 ceph-mon[128316]: pgmap v388: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 31 MiB/s wr, 88 op/s 2023-07-21T19:20:04.017 INFO:teuthology.orchestra.run.smithi005.stderr:48100000 of 50000000 tuples (96%) done (elapsed 721.59 s, remaining 28.50 s) 2023-07-21T19:20:05.336 INFO:teuthology.orchestra.run.smithi005.stderr:48200000 of 50000000 tuples (96%) done (elapsed 722.91 s, remaining 27.00 s) 2023-07-21T19:20:05.747 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:05 smithi005 ceph-mon[126460]: pgmap v389: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 MiB/s wr, 71 op/s 2023-07-21T19:20:05.748 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:05 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:20:05.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:05 smithi101 ceph-mon[128136]: pgmap v389: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 MiB/s wr, 71 op/s 2023-07-21T19:20:05.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:05 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:20:05.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:05 smithi156 ceph-mon[128316]: pgmap v389: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 23 MiB/s wr, 71 op/s 2023-07-21T19:20:05.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:05 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:20:06.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:06 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:20:06.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:06 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:20:06.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:06 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:20:06.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:06 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:20:06.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:06 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:20:06.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:06 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:20:06.912 INFO:teuthology.orchestra.run.smithi005.stderr:48300000 of 50000000 tuples (96%) done (elapsed 724.48 s, remaining 25.50 s) 2023-07-21T19:20:06.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:06 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:20:06.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:06 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:20:06.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:06 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:20:07.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:07 smithi101 ceph-mon[128136]: pgmap v390: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 67 op/s 2023-07-21T19:20:07.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:07 smithi101 ceph-mon[128136]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T19:20:07.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:07 smithi005 ceph-mon[126460]: pgmap v390: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 67 op/s 2023-07-21T19:20:07.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:07 smithi005 ceph-mon[126460]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T19:20:07.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:07 smithi156 ceph-mon[128316]: pgmap v390: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 22 MiB/s wr, 67 op/s 2023-07-21T19:20:07.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:07 smithi156 ceph-mon[128316]: osdmap e101: 12 total, 12 up, 12 in 2023-07-21T19:20:08.244 INFO:teuthology.orchestra.run.smithi005.stderr:48400000 of 50000000 tuples (96%) done (elapsed 725.82 s, remaining 23.99 s) 2023-07-21T19:20:09.565 INFO:teuthology.orchestra.run.smithi005.stderr:48500000 of 50000000 tuples (97%) done (elapsed 727.14 s, remaining 22.49 s) 2023-07-21T19:20:09.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:09 smithi101 ceph-mon[128136]: pgmap v392: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:09.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:09 smithi005 ceph-mon[126460]: pgmap v392: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:09.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:09 smithi156 ceph-mon[128316]: pgmap v392: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:11.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:11 smithi101 ceph-mon[128136]: pgmap v393: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:11.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:11 smithi005 ceph-mon[126460]: pgmap v393: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:11.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:11 smithi156 ceph-mon[128316]: pgmap v393: 97 pgs: 97 active+clean; 9.3 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 26 MiB/s wr, 101 op/s 2023-07-21T19:20:11.972 INFO:teuthology.orchestra.run.smithi005.stderr:48600000 of 50000000 tuples (97%) done (elapsed 729.54 s, remaining 21.02 s) 2023-07-21T19:20:12.362 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:20:12.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 -- 172.21.15.5:0/1536043529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55040ff9f0 msgr2=0x7f55040ffe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 --2- 172.21.15.5:0/1536043529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55040ff9f0 0x7f55040ffe60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f54f4009990 tx=0x7f54f4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 -- 172.21.15.5:0/1536043529 shutdown_connections 2023-07-21T19:20:12.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 --2- 172.21.15.5:0/1536043529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55041003a0 0x7f550410c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 --2- 172.21.15.5:0/1536043529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f5504105dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 --2- 172.21.15.5:0/1536043529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55040ff9f0 0x7f55040ffe60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.629+0000 7f550a99d700 1 -- 172.21.15.5:0/1536043529 >> 172.21.15.5:0/1536043529 conn(0x7f55040fb4f0 msgr2=0x7f55040fd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.630+0000 7f550a99d700 1 -- 172.21.15.5:0/1536043529 shutdown_connections 2023-07-21T19:20:12.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.630+0000 7f550a99d700 1 -- 172.21.15.5:0/1536043529 wait complete. 2023-07-21T19:20:12.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.632+0000 7f550a99d700 1 Processor -- start 2023-07-21T19:20:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.632+0000 7f550a99d700 1 -- start start 2023-07-21T19:20:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55041003a0 0x7f550410c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f55041090d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504109790 con 0x7f5504105a00 2023-07-21T19:20:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504109900 con 0x7f55041003a0 2023-07-21T19:20:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550a99d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5504109ac0 con 0x7f55040ff9f0 2023-07-21T19:20:12.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57172/0 (socket says 172.21.15.5:57172) 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.633+0000 7f550999b700 1 -- 172.21.15.5:0/2270247460 learned_addr learned my addr 172.21.15.5:0/2270247460 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550a19c700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f55041090d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550999b700 1 -- 172.21.15.5:0/2270247460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55041003a0 msgr2=0x7f550410c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550999b700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55041003a0 0x7f550410c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550999b700 1 -- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 msgr2=0x7f55041090d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550999b700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f55041090d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550999b700 1 -- 172.21.15.5:0/2270247460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f40095f0 con 0x7f55040ff9f0 2023-07-21T19:20:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.634+0000 7f550a19c700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f55041090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:12.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.635+0000 7f550999b700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f550000e970 tx=0x7f550000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.635+0000 7f54faffd700 1 -- 172.21.15.5:0/2270247460 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500009790 con 0x7f55040ff9f0 2023-07-21T19:20:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.635+0000 7f54faffd700 1 -- 172.21.15.5:0/2270247460 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f55000044b0 con 0x7f55040ff9f0 2023-07-21T19:20:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.635+0000 7f54faffd700 1 -- 172.21.15.5:0/2270247460 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5500003dd0 con 0x7f55040ff9f0 2023-07-21T19:20:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.636+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5504109da0 con 0x7f55040ff9f0 2023-07-21T19:20:12.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.636+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55041a8c10 con 0x7f55040ff9f0 2023-07-21T19:20:12.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.641+0000 7f54faffd700 1 -- 172.21.15.5:0/2270247460 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5500003f30 con 0x7f55040ff9f0 2023-07-21T19:20:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f551f54eb80 1 Processor -- start 2023-07-21T19:20:12.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f551f54eb80 1 -- start start 2023-07-21T19:20:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f54faffd700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 0x7f54f00a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e37b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f54faffd700 1 -- 172.21.15.5:0/2270247460 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5500014070 con 0x7f55040ff9f0 2023-07-21T19:20:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.643+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e388110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f551f54eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e280cb0 con 0x56176e385d30 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f551f54eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e2d3a40 con 0x56176e37be00 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f551f54eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e2cced0 con 0x56176e37b4f0 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550a19c700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 0x7f54f00a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59192/0 (socket says 172.21.15.5:59192) 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550919a700 1 -- 172.21.15.5:0/986453119 learned_addr learned my addr 172.21.15.5:0/986453119 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550a19c700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e388110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.644+0000 7f550999b700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e37b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 -- 172.21.15.5:0/986453119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 msgr2=0x56176e37b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e37b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 -- 172.21.15.5:0/986453119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 msgr2=0x56176e388110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e388110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 -- 172.21.15.5:0/986453119 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56176e0fade0 con 0x56176e37be00 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550a19c700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e388110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:12.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550a19c700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 0x7f54f00a3970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00f850 tx=0x7f54fc00fb60 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f550919a700 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54f40098e0 tx=0x7f54f4005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=61064559414e427e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f54effff700 1 -- 172.21.15.5:0/986453119 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f401d070 con 0x56176e37be00 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.645+0000 7f54effff700 1 -- 172.21.15.5:0/986453119 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54f400bba0 con 0x56176e37be00 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.646+0000 7f54effff700 1 -- 172.21.15.5:0/986453119 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f4022b60 con 0x56176e37be00 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.646+0000 7f551f54eb80 1 -- 172.21.15.5:0/986453119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 msgr2=0x56176e3857f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.646+0000 7f551f54eb80 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f54f40098e0 tx=0x7f54f4005c00 comp rx=0 tx=0).stop 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 -- 172.21.15.5:0/986453119 shutdown_connections 2023-07-21T19:20:12.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e388110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e37b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 --2- 172.21.15.5:0/986453119 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e3857f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 -- 172.21.15.5:0/986453119 >> 172.21.15.5:0/986453119 conn(0x56176e370ff0 msgr2=0x56176e376930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 -- 172.21.15.5:0/986453119 shutdown_connections 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.647+0000 7f551f54eb80 1 -- 172.21.15.5:0/986453119 wait complete. 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.648+0000 7f551f54eb80 1 -- start start 2023-07-21T19:20:12.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.648+0000 7f551f54eb80 1 Processor -- start 2023-07-21T19:20:12.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 0x56176e385640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e385d30 0x56176e382ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e26cac0 con 0x56176e37be00 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e26c7c0 con 0x56176e37b4f0 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e150920 con 0x56176e385d30 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f551f54eb80 10 client.0 _ll_register_callbacks cb 0x56176e26a760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:12.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f550919a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f550919a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35766/0 (socket says 172.21.15.5:35766) 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f550919a700 1 -- 172.21.15.5:0/3174433150 learned_addr learned my addr 172.21.15.5:0/3174433150 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f550a19c700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e385d30 0x56176e382ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.649+0000 7f550999b700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 0x56176e385640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 -- 172.21.15.5:0/3174433150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e385d30 msgr2=0x56176e382ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e385d30 0x56176e382ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 -- 172.21.15.5:0/3174433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 msgr2=0x56176e382930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 -- 172.21.15.5:0/3174433150 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f550410d130 con 0x56176e37b4f0 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550919a700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f550999b700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 0x56176e385640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f55000ca620 tx=0x7f5500016430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b6e519a81a5d2b5b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.650+0000 7f54ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000a23f0 con 0x56176e37b4f0 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f550000cbd0 con 0x56176e37b4f0 2023-07-21T19:20:12.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56176e2264f0 con 0x56176e37b4f0 2023-07-21T19:20:12.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55000041e0 con 0x56176e37b4f0 2023-07-21T19:20:12.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f551f54eb80 10 client.27818 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:12.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56176e0e3c60 con 0x56176e37b4f0 2023-07-21T19:20:12.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.651+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56176e2264f0 con 0x56176e37b4f0 2023-07-21T19:20:12.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55000ca230 con 0x56176e37b4f0 2023-07-21T19:20:12.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 10 client.27818 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:12.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 10 client.27818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 20 client.27818 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3954d0 0x56176e3978e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 4 client.27818 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56176e2c5eb0 con 0x56176e3954d0 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.652+0000 7f550919a700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3954d0 0x56176e3978e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.653+0000 7f550919a700 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3954d0 0x56176e3978e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b59abc8cad3e0cf server_cookie=640f27fd2dd9b503 in_seq=0 out_seq=0 2023-07-21T19:20:12.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.653+0000 7f54ed7fa700 10 client.27818 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f54ed7fa700 1 -- 172.21.15.5:0/3174433150 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56176e2c5eb0 con 0x56176e3954d0 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f54ed7fa700 10 client.27818 handle_command_reply: tid=0 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 client.27818 shutdown 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 msgr2=0x56176e385640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 0x56176e385640 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f55000ca620 tx=0x7f5500016430 comp rx=0 tx=0).stop 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 shutdown_connections 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e385d30 0x56176e382ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37b4f0 0x56176e385640 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3954d0 0x56176e3978e0 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 --2- 172.21.15.5:0/3174433150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37be00 0x56176e382930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f54ed7fa700 0 client.27818 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 >> 172.21.15.5:0/3174433150 conn(0x56176e370ff0 msgr2=0x56176e3764c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 shutdown_connections 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.654+0000 7f551f54eb80 1 -- 172.21.15.5:0/3174433150 wait complete. 2023-07-21T19:20:12.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.655+0000 7f551f54eb80 20 client.27818 trim_cache size 0 max 16384 2023-07-21T19:20:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.672+0000 7f551f54eb80 1 Processor -- start 2023-07-21T19:20:12.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.672+0000 7f551f54eb80 1 -- start start 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37be00 0x56176e3820f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e385d30 0x56176e382630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e37c980 con 0x56176e37b4f0 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e383ea0 con 0x56176e385d30 2023-07-21T19:20:12.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f551f54eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e370b70 con 0x56176e37be00 2023-07-21T19:20:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f550999b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f550999b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35782/0 (socket says 172.21.15.5:35782) 2023-07-21T19:20:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f550999b700 1 -- 172.21.15.5:0/2153194003 learned_addr learned my addr 172.21.15.5:0/2153194003 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f550919a700 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37be00 0x56176e3820f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.673+0000 7f550a19c700 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e385d30 0x56176e382630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 -- 172.21.15.5:0/2153194003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37be00 msgr2=0x56176e3820f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37be00 0x56176e3820f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 -- 172.21.15.5:0/2153194003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e385d30 msgr2=0x56176e382630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e385d30 0x56176e382630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 -- 172.21.15.5:0/2153194003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56176e121250 con 0x56176e37b4f0 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f550999b700 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5500000c00 tx=0x7f550009e150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cec6f52b951c171 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f54f8bff700 1 -- 172.21.15.5:0/2153194003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550009a350 con 0x56176e37b4f0 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f54f8bff700 1 -- 172.21.15.5:0/2153194003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5500099d70 con 0x56176e37b4f0 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.674+0000 7f54f8bff700 1 -- 172.21.15.5:0/2153194003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55000b8250 con 0x56176e37b4f0 2023-07-21T19:20:12.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 -- 172.21.15.5:0/2153194003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 msgr2=0x56176e3851e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f5500000c00 tx=0x7f550009e150 comp rx=0 tx=0).stop 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 -- 172.21.15.5:0/2153194003 shutdown_connections 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e385d30 0x56176e382630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e37b4f0 0x56176e3851e0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2153194003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37be00 0x56176e3820f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 -- 172.21.15.5:0/2153194003 >> 172.21.15.5:0/2153194003 conn(0x56176e370ff0 msgr2=0x56176e37dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 -- 172.21.15.5:0/2153194003 shutdown_connections 2023-07-21T19:20:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.675+0000 7f551f54eb80 1 -- 172.21.15.5:0/2153194003 wait complete. 2023-07-21T19:20:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 -- start start 2023-07-21T19:20:12.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 Processor -- start 2023-07-21T19:20:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e38ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e38b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e38b770 con 0x56176e385d30 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e38b8e0 con 0x56176e37be00 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56176e3e93a0 con 0x56176e37b4f0 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.676+0000 7f551f54eb80 10 client.0 _ll_register_callbacks cb 0x7f5500099b00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550a19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e38b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57202/0 (socket says 172.21.15.5:57202) 2023-07-21T19:20:12.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 -- 172.21.15.5:0/2305473593 learned_addr learned my addr 172.21.15.5:0/2305473593 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550919a700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e38ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 -- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 msgr2=0x56176e38ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e38ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 -- 172.21.15.5:0/2305473593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 msgr2=0x56176e38b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e38b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 -- 172.21.15.5:0/2305473593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56176e0e3c60 con 0x56176e37b4f0 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f550999b700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55000982f0 tx=0x7f55000043f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12657b7e3f87c297 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.677+0000 7f54ed7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550009a350 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f55000a23f0 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f54f400bba0 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f550000cbd0 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 10 client.27760 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56176e0f92b0 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f54f400bba0 con 0x56176e37b4f0 2023-07-21T19:20:12.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f55000bf020 con 0x56176e37b4f0 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 10 client.27760 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 10 client.27760 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 20 client.27760 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3ea590 0x56176e3ec970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 4 client.27760 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.678+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56176e288050 con 0x56176e3ea590 2023-07-21T19:20:12.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.679+0000 7f550919a700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3ea590 0x56176e3ec970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.679+0000 7f550919a700 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3ea590 0x56176e3ec970 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da8bcb64b580eadc server_cookie=72f80459a5eccc8b in_seq=0 out_seq=0 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.679+0000 7f54ed7fa700 10 client.27760 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f54ed7fa700 1 -- 172.21.15.5:0/2305473593 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x56176e288050 con 0x56176e3ea590 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f54ed7fa700 10 client.27760 handle_command_reply: tid=0 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 client.27760 shutdown 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 msgr2=0x56176e38c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f55000982f0 tx=0x7f55000043f0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 shutdown_connections 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56176e3ea590 0x56176e3ec970 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56176e385d30 0x56176e38b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56176e37b4f0 0x56176e38c880 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 --2- 172.21.15.5:0/2305473593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56176e37be00 0x56176e38ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f54ed7fa700 0 client.27760 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 >> 172.21.15.5:0/2305473593 conn(0x56176e370ff0 msgr2=0x56176e37a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 shutdown_connections 2023-07-21T19:20:12.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 1 -- 172.21.15.5:0/2305473593 wait complete. 2023-07-21T19:20:12.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.680+0000 7f551f54eb80 20 client.27760 trim_cache size 0 max 16384 2023-07-21T19:20:12.682 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:20:12.682 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:20:12.682 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:20:12.682 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:20:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 msgr2=0x7f54f00a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 0x7f54f00a3970 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f54fc00f850 tx=0x7f54fc00fb60 comp rx=0 tx=0).stop 2023-07-21T19:20:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 msgr2=0x7f550410bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f550000e970 tx=0x7f550000ed30 comp rx=0 tx=0).stop 2023-07-21T19:20:12.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 shutdown_connections 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54f00a14c0 0x7f54f00a3970 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5504105a00 0x7f55041090d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f55040ff9f0 0x7f550410bfa0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 --2- 172.21.15.5:0/2270247460 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f55041003a0 0x7f550410c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 >> 172.21.15.5:0/2270247460 conn(0x7f55040fb4f0 msgr2=0x7f5504103540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 shutdown_connections 2023-07-21T19:20:12.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.681+0000 7f550a99d700 1 -- 172.21.15.5:0/2270247460 wait complete. 2023-07-21T19:20:12.694 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:20:12.694 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:20:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 -- 172.21.15.5:0/4260025874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae8100480 msgr2=0x7f1ae810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 --2- 172.21.15.5:0/4260025874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae8100480 0x7f1ae810c910 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000b0a0 tx=0x7f1ae000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 -- 172.21.15.5:0/4260025874 shutdown_connections 2023-07-21T19:20:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 --2- 172.21.15.5:0/4260025874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae8100480 0x7f1ae810c910 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 --2- 172.21.15.5:0/4260025874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8105ae0 0x7f1ae8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 --2- 172.21.15.5:0/4260025874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae80ffad0 0x7f1ae80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 -- 172.21.15.5:0/4260025874 >> 172.21.15.5:0/4260025874 conn(0x7f1ae80fb590 msgr2=0x7f1ae80fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 -- 172.21.15.5:0/4260025874 shutdown_connections 2023-07-21T19:20:12.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.928+0000 7f1af05ff700 1 -- 172.21.15.5:0/4260025874 wait complete. 2023-07-21T19:20:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.929+0000 7f1af05ff700 1 Processor -- start 2023-07-21T19:20:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 -- start start 2023-07-21T19:20:12.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8100480 0x7f1ae81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae8105ae0 0x7f1ae819c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819cd50 con 0x7f1ae80ffad0 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819cec0 con 0x7f1ae8105ae0 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1af05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ae819d080 con 0x7f1ae8100480 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1aee39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1aee39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35822/0 (socket says 172.21.15.5:35822) 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1aee39b700 1 -- 172.21.15.5:0/1704480893 learned_addr learned my addr 172.21.15.5:0/1704480893 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.930+0000 7f1aedb9a700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8100480 0x7f1ae81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aeeb9c700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae8105ae0 0x7f1ae819c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 -- 172.21.15.5:0/1704480893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8100480 msgr2=0x7f1ae81982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8100480 0x7f1ae81982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 -- 172.21.15.5:0/1704480893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae8105ae0 msgr2=0x7f1ae819c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae8105ae0 0x7f1ae819c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:12.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 -- 172.21.15.5:0/1704480893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ae0009d00 con 0x7f1ae80ffad0 2023-07-21T19:20:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1aee39b700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9a0 tx=0x7f1ae400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4009790 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ae400cbd0 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ae819d360 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.931+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4018680 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.932+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ae81a4ba0 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.933+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ae8067300 con 0x7f1ae80ffad0 2023-07-21T19:20:12.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.935+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1ae400cd40 con 0x7f1ae80ffad0 2023-07-21T19:20:12.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.937+0000 7f1adf7fe700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 0x7f1ad40a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:12.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.937+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1ae4014070 con 0x7f1ae80ffad0 2023-07-21T19:20:12.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.937+0000 7f1aedb9a700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 0x7f1ad40a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:12.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.938+0000 7f1aedb9a700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 0x7f1ad40a37b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8199380 tx=0x7f1ad8009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:12.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:12.942+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f1ae4051790 con 0x7f1ae80ffad0 2023-07-21T19:20:13.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.233+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ae81990c0 con 0x7f1ae80ffad0 2023-07-21T19:20:13.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.234+0000 7f1adf7fe700 1 -- 172.21.15.5:0/1704480893 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 44 v44) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f1ae4096330 con 0x7f1ae80ffad0 2023-07-21T19:20:13.228 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:20:13.229 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":44,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":44,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:19:53.568652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:20:13.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 msgr2=0x7f1ad40a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 0x7f1ad40a37b0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f1ae8199380 tx=0x7f1ad8009260 comp rx=0 tx=0).stop 2023-07-21T19:20:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 msgr2=0x7f1ae8197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400e9a0 tx=0x7f1ae400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 shutdown_connections 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1ae8105ae0 0x7f1ae819c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ad40a1300 0x7f1ad40a37b0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1ae80ffad0 0x7f1ae8197d70 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 --2- 172.21.15.5:0/1704480893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1ae8100480 0x7f1ae81982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 >> 172.21.15.5:0/1704480893 conn(0x7f1ae80fb590 msgr2=0x7f1ae810aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 shutdown_connections 2023-07-21T19:20:13.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.236+0000 7f1af05ff700 1 -- 172.21.15.5:0/1704480893 wait complete. 2023-07-21T19:20:13.233 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 44 2023-07-21T19:20:13.251 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:20:13.332 INFO:teuthology.orchestra.run.smithi005.stderr:48700000 of 50000000 tuples (97%) done (elapsed 730.90 s, remaining 19.51 s) 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 -- 172.21.15.5:0/1630946774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 msgr2=0x7f98e8100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 --2- 172.21.15.5:0/1630946774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e8100e70 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f98d8009990 tx=0x7f98d8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 -- 172.21.15.5:0/1630946774 shutdown_connections 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 --2- 172.21.15.5:0/1630946774 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 0x7f98e810c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 --2- 172.21.15.5:0/1630946774 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e8100e70 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 --2- 172.21.15.5:0/1630946774 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 0x7f98e8109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 -- 172.21.15.5:0/1630946774 >> 172.21.15.5:0/1630946774 conn(0x7f98e80745d0 msgr2=0x7f98e80749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 -- 172.21.15.5:0/1630946774 shutdown_connections 2023-07-21T19:20:13.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.494+0000 7f98ed39d700 1 -- 172.21.15.5:0/1630946774 wait complete. 2023-07-21T19:20:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.495+0000 7f98ed39d700 1 Processor -- start 2023-07-21T19:20:13.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.495+0000 7f98ed39d700 1 -- start start 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 0x7f98e819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 0x7f98e81956e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8195eb0 con 0x7f98e8100aa0 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e8196020 con 0x7f98e8109fc0 2023-07-21T19:20:13.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ed39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98e81961e0 con 0x7f98e81013b0 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35828/0 (socket says 172.21.15.5:35828) 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 -- 172.21.15.5:0/2351852840 learned_addr learned my addr 172.21.15.5:0/2351852840 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 0x7f98e819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 0x7f98e81956e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 -- 172.21.15.5:0/2351852840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 msgr2=0x7f98e819bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 0x7f98e819bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 -- 172.21.15.5:0/2351852840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 msgr2=0x7f98e81956e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 0x7f98e81956e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.496+0000 7f98e7fff700 1 -- 172.21.15.5:0/2351852840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d80095f0 con 0x7f98e8100aa0 2023-07-21T19:20:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005740 tx=0x7f98d8005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98e57fa700 1 -- 172.21.15.5:0/2351852840 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d801d070 con 0x7f98e8100aa0 2023-07-21T19:20:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98e8196460 con 0x7f98e8100aa0 2023-07-21T19:20:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98e57fa700 1 -- 172.21.15.5:0/2351852840 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98d8022470 con 0x7f98e8100aa0 2023-07-21T19:20:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98e57fa700 1 -- 172.21.15.5:0/2351852840 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8017650 con 0x7f98e8100aa0 2023-07-21T19:20:13.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.497+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e81a2610 con 0x7f98e8100aa0 2023-07-21T19:20:13.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.501+0000 7f98e57fa700 1 -- 172.21.15.5:0/2351852840 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f98d80225e0 con 0x7f98e8100aa0 2023-07-21T19:20:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.503+0000 7f98e57fa700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 0x7f98d00a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.503+0000 7f9901f25b80 1 Processor -- start 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f98e57fa700 1 -- 172.21.15.5:0/2351852840 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f98d80ce0a0 con 0x7f98e8100aa0 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 -- start start 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 0x7f98d00a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad389627f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad38965110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad3885dcb0 con 0x55ad38962d30 2023-07-21T19:20:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad388b0a40 con 0x55ad389584f0 2023-07-21T19:20:13.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.504+0000 7f9901f25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad388a9ed0 con 0x55ad38958e00 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59256/0 (socket says 172.21.15.5:59256) 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 -- 172.21.15.5:0/2817469586 learned_addr learned my addr 172.21.15.5:0/2817469586 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad38965110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad389627f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 -- 172.21.15.5:0/2817469586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 msgr2=0x55ad389627f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad389627f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 -- 172.21.15.5:0/2817469586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 msgr2=0x55ad38965110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad38965110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 -- 172.21.15.5:0/2817469586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad386d7de0 con 0x55ad389584f0 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad38965110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad389627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:13.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 0x7f98d00a3810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98e8196f10 tx=0x7f98dc011720 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.505+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98d80976a0 tx=0x7f98d8005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34c0293f14a7e29e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.506+0000 7f98ce7fc700 1 -- 172.21.15.5:0/2817469586 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d809b1a0 con 0x55ad389584f0 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.506+0000 7f98ce7fc700 1 -- 172.21.15.5:0/2817469586 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98d809f1d0 con 0x55ad389584f0 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.506+0000 7f98ce7fc700 1 -- 172.21.15.5:0/2817469586 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d809a690 con 0x55ad389584f0 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.506+0000 7f9901f25b80 1 -- 172.21.15.5:0/2817469586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 msgr2=0x55ad389588c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.506+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f98d80976a0 tx=0x7f98d8005910 comp rx=0 tx=0).stop 2023-07-21T19:20:13.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 -- 172.21.15.5:0/2817469586 shutdown_connections 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad38965110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad389588c0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2817469586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad389627f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 -- 172.21.15.5:0/2817469586 >> 172.21.15.5:0/2817469586 conn(0x55ad3894dff0 msgr2=0x55ad38953930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 -- 172.21.15.5:0/2817469586 shutdown_connections 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 -- 172.21.15.5:0/2817469586 wait complete. 2023-07-21T19:20:13.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.507+0000 7f9901f25b80 1 -- start start 2023-07-21T19:20:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 Processor -- start 2023-07-21T19:20:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38963640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad38963b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad38849ac0 con 0x55ad389584f0 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad388497c0 con 0x55ad38958e00 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad3872d920 con 0x55ad38962d30 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.508+0000 7f9901f25b80 10 client.0 _ll_register_callbacks cb 0x55ad38847760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35848/0 (socket says 172.21.15.5:35848) 2023-07-21T19:20:13.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 -- 172.21.15.5:0/3802345178 learned_addr learned my addr 172.21.15.5:0/3802345178 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e77fe700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38963640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad38963b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 -- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 msgr2=0x55ad38963b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad38963b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 -- 172.21.15.5:0/3802345178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 msgr2=0x55ad38963640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38963640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 -- 172.21.15.5:0/3802345178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98e81a2610 con 0x55ad389584f0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f98d8097c10 tx=0x7f98d80a1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e22bad183a370b11 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.509+0000 7f98c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80a3560 con 0x55ad389584f0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f98d809b1a0 con 0x55ad389584f0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ad388034f0 con 0x55ad389584f0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98d809b610 con 0x55ad389584f0 2023-07-21T19:20:13.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 10 client.18186 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55ad386c0c60 con 0x55ad389584f0 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ad388034f0 con 0x55ad389584f0 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f98d80b93e0 con 0x55ad389584f0 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 10 client.18186 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 10 client.18186 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.510+0000 7f9901f25b80 20 client.18186 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:13.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.511+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad38976210 0x55ad389785f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.511+0000 7f9901f25b80 4 client.18186 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.511+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ad388a2eb0 con 0x55ad38976210 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.511+0000 7f98e77fe700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad38976210 0x55ad389785f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.512+0000 7f98e77fe700 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad38976210 0x55ad389785f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6823901f4d093906 server_cookie=3b420c51ed6645c9 in_seq=0 out_seq=0 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.512+0000 7f98c77fe700 10 client.18186 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.512+0000 7f98c77fe700 1 -- 172.21.15.5:0/3802345178 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ad388a2eb0 con 0x55ad38976210 2023-07-21T19:20:13.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.512+0000 7f98c77fe700 10 client.18186 handle_command_reply: tid=0 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.512+0000 7f9901f25b80 1 client.18186 shutdown 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 msgr2=0x55ad38963100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f98d8097c10 tx=0x7f98d80a1040 comp rx=0 tx=0).stop 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 shutdown_connections 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad38963b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38963100 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad38976210 0x55ad389785f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3802345178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38963640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 >> 172.21.15.5:0/3802345178 conn(0x55ad3894dff0 msgr2=0x55ad38953520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f98c77fe700 0 client.18186 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 shutdown_connections 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 1 -- 172.21.15.5:0/3802345178 wait complete. 2023-07-21T19:20:13.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.513+0000 7f9901f25b80 20 client.18186 trim_cache size 0 max 16384 2023-07-21T19:20:13.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.530+0000 7f9901f25b80 1 Processor -- start 2023-07-21T19:20:13.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 -- start start 2023-07-21T19:20:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38952b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad3896d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad3894db70 con 0x55ad389584f0 2023-07-21T19:20:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad3895a590 con 0x55ad38958e00 2023-07-21T19:20:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f9901f25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad38962420 con 0x55ad38962d30 2023-07-21T19:20:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35856/0 (socket says 172.21.15.5:35856) 2023-07-21T19:20:13.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f98e7fff700 1 -- 172.21.15.5:0/3738654052 learned_addr learned my addr 172.21.15.5:0/3738654052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f98e77fe700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38952b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.531+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad3896d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 -- 172.21.15.5:0/3738654052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 msgr2=0x55ad3896d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad3896d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 -- 172.21.15.5:0/3738654052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 msgr2=0x55ad38952b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38952b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 -- 172.21.15.5:0/3738654052 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ad389594a0 con 0x55ad389584f0 2023-07-21T19:20:13.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad3896d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e77fe700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38952b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98e7fff700 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f98d809ac70 tx=0x7f98d8005910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce5958dade4f2475 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98ceffd700 1 -- 172.21.15.5:0/3738654052 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d8062020 con 0x55ad389584f0 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98ceffd700 1 -- 172.21.15.5:0/3738654052 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98d809b1a0 con 0x55ad389584f0 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.532+0000 7f98ceffd700 1 -- 172.21.15.5:0/3738654052 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98d80a30d0 con 0x55ad389584f0 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 -- 172.21.15.5:0/3738654052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 msgr2=0x55ad38967130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f98d809ac70 tx=0x7f98d8005910 comp rx=0 tx=0).stop 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 -- 172.21.15.5:0/3738654052 shutdown_connections 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38962d30 0x55ad3896d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad389584f0 0x55ad38967130 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 --2- 172.21.15.5:0/3738654052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad38958e00 0x55ad38952b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 -- 172.21.15.5:0/3738654052 >> 172.21.15.5:0/3738654052 conn(0x55ad3894dff0 msgr2=0x55ad38952060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 -- 172.21.15.5:0/3738654052 shutdown_connections 2023-07-21T19:20:13.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.533+0000 7f9901f25b80 1 -- 172.21.15.5:0/3738654052 wait complete. 2023-07-21T19:20:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.534+0000 7f9901f25b80 1 -- start start 2023-07-21T19:20:13.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.534+0000 7f9901f25b80 1 Processor -- start 2023-07-21T19:20:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad38964970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad38964eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad389c90a0 con 0x55ad38962d30 2023-07-21T19:20:13.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad389c9210 con 0x55ad389584f0 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ad389c93d0 con 0x55ad38958e00 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f9901f25b80 10 client.0 _ll_register_callbacks cb 0x7f98d80974e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f98ecb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f98e7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad38964970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f98ecb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35866/0 (socket says 172.21.15.5:35866) 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.535+0000 7f98ecb9c700 1 -- 172.21.15.5:0/2680036525 learned_addr learned my addr 172.21.15.5:0/2680036525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad38964eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 -- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 msgr2=0x55ad38964eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad38964eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 -- 172.21.15.5:0/2680036525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 msgr2=0x55ad38964970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad38964970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 -- 172.21.15.5:0/2680036525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ad386c0c60 con 0x55ad38962d30 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98e77fe700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad38964eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:13.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad38964970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98ecb9c700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f98e0007b60 tx=0x7f98e0007b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab33d715040ea7df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.536+0000 7f98c77fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e000e040 con 0x55ad38962d30 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f98e00175b0 con 0x55ad38962d30 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f98d809f1d0 con 0x55ad38962d30 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f98e0004bb0 con 0x55ad38962d30 2023-07-21T19:20:13.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f9901f25b80 10 client.18192 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55ad386fe250 con 0x55ad38962d30 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f98d809f1d0 con 0x55ad38962d30 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.537+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f98e00040c0 con 0x55ad38962d30 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 10 client.18192 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 10 client.18192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 20 client.18192 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:13.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad389cab70 0x55ad389ccf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 4 client.18192 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:20:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.538+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ad38865050 con 0x55ad389cab70 2023-07-21T19:20:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.539+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad389cab70 0x55ad389ccf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.539+0000 7f98c77fe700 10 client.18192 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:20:13.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.539+0000 7f98e7fff700 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad389cab70 0x55ad389ccf80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4deac751a9dda894 server_cookie=375fc80b050e3047 in_seq=0 out_seq=0 2023-07-21T19:20:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f98c77fe700 1 -- 172.21.15.5:0/2680036525 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ad38865050 con 0x55ad389cab70 2023-07-21T19:20:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f98c77fe700 10 client.18192 handle_command_reply: tid=0 2023-07-21T19:20:13.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 client.18192 shutdown 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 msgr2=0x55ad389653f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f98e0007b60 tx=0x7f98e0007b90 comp rx=0 tx=0).stop 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 shutdown_connections 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ad38962d30 0x55ad389653f0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55ad389cab70 0x55ad389ccf80 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ad389584f0 0x55ad38964970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 --2- 172.21.15.5:0/2680036525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ad38958e00 0x55ad38964eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 >> 172.21.15.5:0/2680036525 conn(0x55ad3894dff0 msgr2=0x55ad3896d8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f98c77fe700 0 client.18192 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:20:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 shutdown_connections 2023-07-21T19:20:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.540+0000 7f9901f25b80 1 -- 172.21.15.5:0/2680036525 wait complete. 2023-07-21T19:20:13.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f9901f25b80 20 client.18192 trim_cache size 0 max 16384 2023-07-21T19:20:13.543 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:20:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 msgr2=0x7f98d00a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 0x7f98d00a3810 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f98e8196f10 tx=0x7f98dc011720 comp rx=0 tx=0).stop 2023-07-21T19:20:13.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 msgr2=0x7f98e819b690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005740 tx=0x7f98d8005770 comp rx=0 tx=0).stop 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 shutdown_connections 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f98e8109fc0 0x7f98e81956e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98d00a1360 0x7f98d00a3810 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f98e8100aa0 0x7f98e819b690 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.541+0000 7f98ed39d700 1 --2- 172.21.15.5:0/2351852840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f98e81013b0 0x7f98e819bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.542+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 >> 172.21.15.5:0/2351852840 conn(0x7f98e80745d0 msgr2=0x7f98e80fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.542+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 shutdown_connections 2023-07-21T19:20:13.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.542+0000 7f98ed39d700 1 -- 172.21.15.5:0/2351852840 wait complete. 2023-07-21T19:20:13.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:13 smithi005 ceph-mon[126460]: pgmap v394: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:13.554 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:13 smithi005 ceph-mon[126460]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T19:20:13.555 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:13 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1704480893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:20:13.557 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:20:13.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.797+0000 7f3b69fff700 1 -- 172.21.15.5:0/1443694047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b64100ca0 msgr2=0x7f3b64101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.797+0000 7f3b69fff700 1 --2- 172.21.15.5:0/1443694047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b64100ca0 0x7f3b64101070 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f3b54009990 tx=0x7f3b54009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 -- 172.21.15.5:0/1443694047 shutdown_connections 2023-07-21T19:20:13.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 --2- 172.21.15.5:0/1443694047 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b6410a070 0x7f3b6410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 --2- 172.21.15.5:0/1443694047 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b64100ca0 0x7f3b64101070 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 --2- 172.21.15.5:0/1443694047 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 0x7f3b64109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 -- 172.21.15.5:0/1443694047 >> 172.21.15.5:0/1443694047 conn(0x7f3b640fa4b0 msgr2=0x7f3b640fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 -- 172.21.15.5:0/1443694047 shutdown_connections 2023-07-21T19:20:13.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.798+0000 7f3b69fff700 1 -- 172.21.15.5:0/1443694047 wait complete. 2023-07-21T19:20:13.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.799+0000 7f3b69fff700 1 Processor -- start 2023-07-21T19:20:13.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.799+0000 7f3b69fff700 1 -- start start 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b64100ca0 0x7f3b64197d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 0x7f3b641982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419cd40 con 0x7f3b6410a070 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419ceb0 con 0x7f3b64100ca0 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b69fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6419d070 con 0x7f3b641015b0 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b63fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35896/0 (socket says 172.21.15.5:35896) 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b63fff700 1 -- 172.21.15.5:0/526668621 learned_addr learned my addr 172.21.15.5:0/526668621 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 0x7f3b641982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.800+0000 7f3b637fe700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b64100ca0 0x7f3b64197d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 -- 172.21.15.5:0/526668621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 msgr2=0x7f3b641982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 0x7f3b641982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 -- 172.21.15.5:0/526668621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b64100ca0 msgr2=0x7f3b64197d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b64100ca0 0x7f3b64197d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 -- 172.21.15.5:0/526668621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b540095f0 con 0x7f3b6410a070 2023-07-21T19:20:13.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b63fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef30 tx=0x7f3b5c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b60ff9700 1 -- 172.21.15.5:0/526668621 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c018070 con 0x7f3b6410a070 2023-07-21T19:20:13.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b6419d2f0 con 0x7f3b6410a070 2023-07-21T19:20:13.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b60ff9700 1 -- 172.21.15.5:0/526668621 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b5c00f040 con 0x7f3b6410a070 2023-07-21T19:20:13.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b60ff9700 1 -- 172.21.15.5:0/526668621 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5c00cbc0 con 0x7f3b6410a070 2023-07-21T19:20:13.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.801+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b641a4b90 con 0x7f3b6410a070 2023-07-21T19:20:13.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.806+0000 7f3b60ff9700 1 -- 172.21.15.5:0/526668621 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3b5c00cd60 con 0x7f3b6410a070 2023-07-21T19:20:13.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 Processor -- start 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 -- start start 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 0x5588abb8c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588aba87cb0 con 0x5588abb8cad0 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abadaa40 con 0x5588abb812d0 2023-07-21T19:20:13.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.807+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abad3ed0 con 0x5588abb81be0 2023-07-21T19:20:13.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35910/0 (socket says 172.21.15.5:35910) 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 -- 172.21.15.5:0/2282937476 learned_addr learned my addr 172.21.15.5:0/2282937476 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b637fe700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 0x5588abb8c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b60ff9700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 0x7f3b4c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 -- 172.21.15.5:0/2282937476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 msgr2=0x5588abb8c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 0x5588abb8c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 -- 172.21.15.5:0/2282937476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 msgr2=0x5588abb816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b62ffd700 1 -- 172.21.15.5:0/2282937476 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588ab901de0 con 0x5588abb8cad0 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b637fe700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 0x5588abb8c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b60ff9700 1 -- 172.21.15.5:0/526668621 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b5c0cc7b0 con 0x7f3b6410a070 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.808+0000 7f3b63fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 0x7f3b4c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.809+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011880 tx=0x7f3b580058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1c25cf777c49ba2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.809+0000 7f3b49ffb700 1 -- 172.21.15.5:0/2282937476 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58011ea0 con 0x5588abb8cad0 2023-07-21T19:20:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.809+0000 7f3b49ffb700 1 -- 172.21.15.5:0/2282937476 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b580044c0 con 0x5588abb8cad0 2023-07-21T19:20:13.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.809+0000 7f3b49ffb700 1 -- 172.21.15.5:0/2282937476 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5801e3a0 con 0x5588abb8cad0 2023-07-21T19:20:13.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 -- 172.21.15.5:0/2282937476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 msgr2=0x5588abb8eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b63fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 0x7f3b4c0a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef00 tx=0x7f3b5c004bf0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011880 tx=0x7f3b580058e0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 -- 172.21.15.5:0/2282937476 shutdown_connections 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb8cad0 0x5588abb8eeb0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/2282937476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb81be0 0x5588abb8c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 -- 172.21.15.5:0/2282937476 >> 172.21.15.5:0/2282937476 conn(0x5588abb77da0 msgr2=0x5588abb7d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 -- 172.21.15.5:0/2282937476 shutdown_connections 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.810+0000 7f3b7d869b80 1 -- 172.21.15.5:0/2282937476 wait complete. 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.811+0000 7f3b7d869b80 1 -- start start 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.811+0000 7f3b7d869b80 1 Processor -- start 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.811+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb8c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.811+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb89f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588aba73ac0 con 0x5588abb81be0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588aba737c0 con 0x5588abb8cad0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588ab957920 con 0x5588abb812d0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b7d869b80 10 client.0 _ll_register_callbacks cb 0x5588aba71760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b62ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb8c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35912/0 (socket says 172.21.15.5:35912) 2023-07-21T19:20:13.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 -- 172.21.15.5:0/1273211470 learned_addr learned my addr 172.21.15.5:0/1273211470 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb89f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 -- 172.21.15.5:0/1273211470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 msgr2=0x5588abb8c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb8c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 -- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 msgr2=0x5588abb89f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb89f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.812+0000 7f3b637fe700 1 -- 172.21.15.5:0/1273211470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6410d170 con 0x5588abb81be0 2023-07-21T19:20:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.813+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3b54015000 tx=0x7f3b54005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=988ac65a1b9fcc07 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.813+0000 7f3b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.813+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401d070 con 0x5588abb81be0 2023-07-21T19:20:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.813+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588aba2d4f0 con 0x5588abb81be0 2023-07-21T19:20:13.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.813+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b54004500 con 0x5588abb81be0 2023-07-21T19:20:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b540046c0 con 0x5588abb81be0 2023-07-21T19:20:13.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 10 client.18204 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f3b58019040 con 0x5588abb81be0 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5588aba2d4f0 con 0x5588abb81be0 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b5400f6c0 con 0x5588abb81be0 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 10 client.18204 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 10 client.18204 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 20 client.18204 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:13.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abb9c4b0 0x5588abb9e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 4 client.18204 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.814+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588abacceb0 con 0x5588abb9c4b0 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.815+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abb9c4b0 0x5588abb9e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.815+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abb9c4b0 0x5588abb9e8c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21626a1b82bf63fe server_cookie=6147de2e65796f53 in_seq=0 out_seq=0 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.815+0000 7f3b42ffd700 10 client.18204 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b42ffd700 1 -- 172.21.15.5:0/1273211470 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5588abacceb0 con 0x5588abb9c4b0 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b42ffd700 10 client.18204 handle_command_reply: tid=0 2023-07-21T19:20:13.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 client.18204 shutdown 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 msgr2=0x5588abb89800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f3b54015000 tx=0x7f3b54005800 comp rx=0 tx=0).stop 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 shutdown_connections 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abb9c4b0 0x5588abb9e8c0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb89f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb8c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1273211470 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb89800 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 >> 172.21.15.5:0/1273211470 conn(0x5588abb77da0 msgr2=0x5588abb7d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b42ffd700 0 client.18204 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:20:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 shutdown_connections 2023-07-21T19:20:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.816+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1273211470 wait complete. 2023-07-21T19:20:13.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.817+0000 7f3b7d869b80 20 client.18204 trim_cache size 0 max 16384 2023-07-21T19:20:13.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:13 smithi101 ceph-mon[128136]: pgmap v394: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:13.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:13 smithi101 ceph-mon[128136]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T19:20:13.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:13 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1704480893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:20:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 Processor -- start 2023-07-21T19:20:13.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 -- start start 2023-07-21T19:20:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb9e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb8cad0 0x5588abb85070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abb8a780 con 0x5588abb81be0 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abb82330 con 0x5588abb812d0 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abb7eb00 con 0x5588abb8cad0 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.833+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35928/0 (socket says 172.21.15.5:35928) 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 -- 172.21.15.5:0/1599759178 learned_addr learned my addr 172.21.15.5:0/1599759178 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb8cad0 0x5588abb85070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb9e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 -- 172.21.15.5:0/1599759178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb8cad0 msgr2=0x5588abb85070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb8cad0 0x5588abb85070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 -- 172.21.15.5:0/1599759178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 msgr2=0x5588abb9e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb9e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 -- 172.21.15.5:0/1599759178 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588abb7c0e0 con 0x5588abb81be0 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.834+0000 7f3b637fe700 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3b540098e0 tx=0x7f3b54010f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aee1eaba2873eff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b4a7fc700 1 -- 172.21.15.5:0/1599759178 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401d070 con 0x5588abb81be0 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b4a7fc700 1 -- 172.21.15.5:0/1599759178 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b5401e040 con 0x5588abb81be0 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b4a7fc700 1 -- 172.21.15.5:0/1599759178 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b54004760 con 0x5588abb81be0 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1599759178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 msgr2=0x5588abb84b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f3b540098e0 tx=0x7f3b54010f90 comp rx=0 tx=0).stop 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1599759178 shutdown_connections 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb8cad0 0x5588abb85070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb812d0 0x5588abb9e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/1599759178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb84b30 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.835+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1599759178 >> 172.21.15.5:0/1599759178 conn(0x5588abb77da0 msgr2=0x5588abb98780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.836+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1599759178 shutdown_connections 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.836+0000 7f3b7d869b80 1 -- 172.21.15.5:0/1599759178 wait complete. 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.836+0000 7f3b7d869b80 1 -- start start 2023-07-21T19:20:13.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.836+0000 7f3b7d869b80 1 Processor -- start 2023-07-21T19:20:13.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb91490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb91f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abbef330 con 0x5588abb81be0 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abbef470 con 0x5588abb8cad0 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588abbef5b0 con 0x5588abb812d0 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b7d869b80 10 client.0 _ll_register_callbacks cb 0x7f3b54005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:13.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35938/0 (socket says 172.21.15.5:35938) 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 -- 172.21.15.5:0/3946890820 learned_addr learned my addr 172.21.15.5:0/3946890820 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb91490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 -- 172.21.15.5:0/3946890820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 msgr2=0x5588abb91490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb91490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 -- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 msgr2=0x5588abb91f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb91f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.837+0000 7f3b637fe700 1 -- 172.21.15.5:0/3946890820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b58019040 con 0x5588abb81be0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b637fe700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400bfe0 tx=0x7f3b5400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c147ce5f9d5fdcda server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b42ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5401d040 con 0x5588abb81be0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b580044c0 con 0x5588abb81be0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b54005990 con 0x5588abb81be0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b5400f870 con 0x5588abb81be0 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b7d869b80 10 client.18210 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:13.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.838+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5588ab8eac60 con 0x5588abb81be0 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b5404f020 con 0x5588abb81be0 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b54040450 con 0x5588abb81be0 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 10 client.18210 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 10 client.18210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 20 client.18210 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abbf0750 0x5588abbf2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 4 client.18210 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:20:13.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.839+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5588aba8f050 con 0x5588abbf0750 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abbf0750 0x5588abbf2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b62ffd700 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abbf0750 0x5588abbf2b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e1307e3ae8a02588 server_cookie=cd60ebd7f6b71798 in_seq=0 out_seq=0 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b42ffd700 10 client.18210 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b42ffd700 1 -- 172.21.15.5:0/3946890820 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5588aba8f050 con 0x5588abbf0750 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b42ffd700 10 client.18210 handle_command_reply: tid=0 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b7d869b80 1 client.18210 shutdown 2023-07-21T19:20:13.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 msgr2=0x5588abb919d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.840+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f3b5400bfe0 tx=0x7f3b5400b260 comp rx=0 tx=0).stop 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 shutdown_connections 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5588abbf0750 0x5588abbf2b60 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5588abb8cad0 0x5588abb91f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5588abb812d0 0x5588abb91490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b42ffd700 0 client.18210 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 --2- 172.21.15.5:0/3946890820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5588abb81be0 0x5588abb919d0 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 >> 172.21.15.5:0/3946890820 conn(0x5588abb77da0 msgr2=0x5588abb8c100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 shutdown_connections 2023-07-21T19:20:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 1 -- 172.21.15.5:0/3946890820 wait complete. 2023-07-21T19:20:13.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.841+0000 7f3b7d869b80 20 client.18210 trim_cache size 0 max 16384 2023-07-21T19:20:13.842 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:20:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 msgr2=0x7f3b4c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 0x7f3b4c0a3770 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef00 tx=0x7f3b5c004bf0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 msgr2=0x7f3b6419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f3b5c00ef30 tx=0x7f3b5c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 shutdown_connections 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b4c0a12c0 0x7f3b4c0a3770 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b6410a070 0x7f3b6419c700 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b64100ca0 0x7f3b64197d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 --2- 172.21.15.5:0/526668621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b641015b0 0x7f3b641982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 >> 172.21.15.5:0/526668621 conn(0x7f3b640fa4b0 msgr2=0x7f3b6410c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 shutdown_connections 2023-07-21T19:20:13.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:13.842+0000 7f3b69fff700 1 -- 172.21.15.5:0/526668621 wait complete. 2023-07-21T19:20:13.857 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:20:13.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:13 smithi156 ceph-mon[128316]: pgmap v394: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:13.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:13 smithi156 ceph-mon[128316]: osdmap e102: 12 total, 12 up, 12 in 2023-07-21T19:20:13.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:13 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1704480893' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:20:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 -- 172.21.15.5:0/1624451814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade8067c10 msgr2=0x7fade8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 --2- 172.21.15.5:0/1624451814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade8067c10 0x7fade8067fe0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fade400fb20 tx=0x7fade400fe30 comp rx=0 tx=0).stop 2023-07-21T19:20:14.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 -- 172.21.15.5:0/1624451814 shutdown_connections 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 --2- 172.21.15.5:0/1624451814 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8104e50 0x7fade810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 --2- 172.21.15.5:0/1624451814 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade8067c10 0x7fade8067fe0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 --2- 172.21.15.5:0/1624451814 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade80685b0 0x7fade8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.055+0000 7fadef3ff700 1 -- 172.21.15.5:0/1624451814 >> 172.21.15.5:0/1624451814 conn(0x7fade8074760 msgr2=0x7fade8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.056+0000 7fadef3ff700 1 -- 172.21.15.5:0/1624451814 shutdown_connections 2023-07-21T19:20:14.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.056+0000 7fadef3ff700 1 -- 172.21.15.5:0/1624451814 wait complete. 2023-07-21T19:20:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.057+0000 7fadef3ff700 1 Processor -- start 2023-07-21T19:20:14.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.057+0000 7fadef3ff700 1 -- start start 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8067c10 0x7fade81998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade8104e50 0x7fade8193750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8193dc0 con 0x7fade8104e50 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade8193f30 con 0x7fade80685b0 2023-07-21T19:20:14.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadef3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade81940f0 con 0x7fade8067c10 2023-07-21T19:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadec99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadec99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59362/0 (socket says 172.21.15.5:59362) 2023-07-21T19:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7fadec99a700 1 -- 172.21.15.5:0/2235445055 learned_addr learned my addr 172.21.15.5:0/2235445055 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7faded19b700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8067c10 0x7fade81998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.058+0000 7faded99c700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade8104e50 0x7fade8193750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 -- 172.21.15.5:0/2235445055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8067c10 msgr2=0x7fade81998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8067c10 0x7fade81998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 -- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade8104e50 msgr2=0x7fade8193750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade8104e50 0x7fade8193750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 -- 172.21.15.5:0/2235445055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade400f750 con 0x7fade80685b0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadec99a700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e940 tx=0x7fadd800ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.059+0000 7fadde7fc700 1 -- 172.21.15.5:0/2235445055 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8009730 con 0x7fade80685b0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.060+0000 7fadde7fc700 1 -- 172.21.15.5:0/2235445055 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fadd800ce70 con 0x7fade80685b0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.060+0000 7fadde7fc700 1 -- 172.21.15.5:0/2235445055 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd8018420 con 0x7fade80685b0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.060+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade81943d0 con 0x7fade80685b0 2023-07-21T19:20:14.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.060+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade81a08e0 con 0x7fade80685b0 2023-07-21T19:20:14.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.065+0000 7fadde7fc700 1 -- 172.21.15.5:0/2235445055 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fadd800c9e0 con 0x7fade80685b0 2023-07-21T19:20:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.065+0000 7fae02d43b80 1 Processor -- start 2023-07-21T19:20:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 -- start start 2023-07-21T19:20:14.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9fe416b0 0x562a9ffeda40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff26e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9feeac50 con 0x562a9fe416b0 2023-07-21T19:20:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ff3d9e0 con 0x562a9ffedf80 2023-07-21T19:20:14.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7fae02d43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ff36e70 con 0x562a9ffee3f0 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.066+0000 7faded19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59370/0 (socket says 172.21.15.5:59370) 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 -- 172.21.15.5:0/3585531082 learned_addr learned my addr 172.21.15.5:0/3585531082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded99c700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9fe416b0 0x562a9ffeda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7fadec99a700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 -- 172.21.15.5:0/3585531082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 msgr2=0x562a9fff26e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff26e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 -- 172.21.15.5:0/3585531082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9fe416b0 msgr2=0x562a9ffeda40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9fe416b0 0x562a9ffeda40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7faded19b700 1 -- 172.21.15.5:0/3585531082 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9fd64d80 con 0x562a9ffedf80 2023-07-21T19:20:14.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.067+0000 7fadec99a700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7faded19b700 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fade400fb20 tx=0x7fade400f6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ef670e82f101d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7fadd37fe700 1 -- 172.21.15.5:0/3585531082 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4003880 con 0x562a9ffedf80 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7fadde7fc700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 0x7fadd40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7fadd37fe700 1 -- 172.21.15.5:0/3585531082 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fade40056c0 con 0x562a9ffedf80 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7fadd37fe700 1 -- 172.21.15.5:0/3585531082 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4003e90 con 0x562a9ffedf80 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.068+0000 7fadde7fc700 1 -- 172.21.15.5:0/2235445055 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fadd8014070 con 0x7fade80685b0 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7faded99c700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 0x7fadd40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 -- 172.21.15.5:0/3585531082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 msgr2=0x562a9fff21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fade400fb20 tx=0x7fade400f6a0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 -- 172.21.15.5:0/3585531082 shutdown_connections 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff26e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9fe416b0 0x562a9ffeda40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 --2- 172.21.15.5:0/3585531082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffedf80 0x562a9fff21a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.069+0000 7fae02d43b80 1 -- 172.21.15.5:0/3585531082 >> 172.21.15.5:0/3585531082 conn(0x562a9ffdabe0 msgr2=0x562a9ffdff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.070+0000 7faded99c700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 0x7fadd40a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fade819a5f0 tx=0x7fade000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.070+0000 7fae02d43b80 1 -- 172.21.15.5:0/3585531082 shutdown_connections 2023-07-21T19:20:14.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.070+0000 7fae02d43b80 1 -- 172.21.15.5:0/3585531082 wait complete. 2023-07-21T19:20:14.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.070+0000 7fae02d43b80 1 -- start start 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 Processor -- start 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 0x562a9fffce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9fffd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9fed6a60 con 0x562a9ffedf80 2023-07-21T19:20:14.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9fed6760 con 0x562a9ffee3f0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.071+0000 7fae02d43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9fdba8c0 con 0x562a9fe416b0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7fae02d43b80 10 client.0 _ll_register_callbacks cb 0x562a9fed4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7faded99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7faded99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59380/0 (socket says 172.21.15.5:59380) 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7faded99c700 1 -- 172.21.15.5:0/756271530 learned_addr learned my addr 172.21.15.5:0/756271530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7faded19b700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 0x562a9fffce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.072+0000 7fadec99a700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9fffd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 -- 172.21.15.5:0/756271530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 msgr2=0x562a9fffce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 0x562a9fffce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 -- 172.21.15.5:0/756271530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 msgr2=0x562a9fffd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9fffd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 -- 172.21.15.5:0/756271530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade810d210 con 0x562a9ffee3f0 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded19b700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 0x562a9fffce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7faded99c700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fade0004600 tx=0x7fade0004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4aa03d3691a5e866 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.073+0000 7fadd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade000e040 con 0x562a9ffee3f0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fade0016ca0 con 0x562a9ffee3f0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a9fe90490 con 0x562a9ffee3f0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fade0016ea0 con 0x562a9ffee3f0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fae02d43b80 10 client.27875 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.074+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562a9fd4dc00 con 0x562a9ffee3f0 2023-07-21T19:20:14.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a9fe90490 con 0x562a9ffee3f0 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fade00153f0 con 0x562a9ffee3f0 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 10 client.27875 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 10 client.27875 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 20 client.27875 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa00072d0 0x562aa00096e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 4 client.27875 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:14.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a9ff2fe50 con 0x562aa00072d0 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.075+0000 7faded19b700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa00072d0 0x562aa00096e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.076+0000 7faded19b700 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa00072d0 0x562aa00096e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be22d139c723f8fd server_cookie=8bcc7f138e261114 in_seq=0 out_seq=0 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.076+0000 7fadd0ff9700 10 client.27875 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.077+0000 7fadd0ff9700 1 -- 172.21.15.5:0/756271530 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562a9ff2fe50 con 0x562aa00072d0 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.077+0000 7fadd0ff9700 10 client.27875 handle_command_reply: tid=0 2023-07-21T19:20:14.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.077+0000 7fae02d43b80 1 client.27875 shutdown 2023-07-21T19:20:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.077+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 msgr2=0x562a9fff70d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.077+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fade0004600 tx=0x7fade0004b40 comp rx=0 tx=0).stop 2023-07-21T19:20:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 shutdown_connections 2023-07-21T19:20:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9ffee3f0 0x562a9fff70d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9fe416b0 0x562a9fffce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa00072d0 0x562aa00096e0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 --2- 172.21.15.5:0/756271530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9fffd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fadd0ff9700 0 client.27875 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 >> 172.21.15.5:0/756271530 conn(0x562a9ffdabe0 msgr2=0x562a9ffdf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 shutdown_connections 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 1 -- 172.21.15.5:0/756271530 wait complete. 2023-07-21T19:20:14.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.078+0000 7fae02d43b80 20 client.27875 trim_cache size 0 max 16384 2023-07-21T19:20:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.097+0000 7fae02d43b80 1 Processor -- start 2023-07-21T19:20:14.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 -- start start 2023-07-21T19:20:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffdfc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9ffeffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9fff2ef0 con 0x562a9ffedf80 2023-07-21T19:20:14.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ffe0ea0 con 0x562a9fe416b0 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fae02d43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ffee7f0 con 0x562a9ffee3f0 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fadec99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9ffeffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7faded19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffdfc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fadec99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9ffeffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35970/0 (socket says 172.21.15.5:35970) 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.098+0000 7fadec99a700 1 -- 172.21.15.5:0/2660318257 learned_addr learned my addr 172.21.15.5:0/2660318257 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded99c700 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 -- 172.21.15.5:0/2660318257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 msgr2=0x562a9fff3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff3110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 -- 172.21.15.5:0/2660318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 msgr2=0x562a9ffeffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9ffeffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 -- 172.21.15.5:0/2660318257 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9fd8b1f0 con 0x562a9fe416b0 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7faded19b700 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffdfc60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fade400fa40 tx=0x7fade40058e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a7810cf7751640f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.099+0000 7fadd3fff700 1 -- 172.21.15.5:0/2660318257 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4003900 con 0x562a9fe416b0 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fadd3fff700 1 -- 172.21.15.5:0/2660318257 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fade40360d0 con 0x562a9fe416b0 2023-07-21T19:20:14.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fadd3fff700 1 -- 172.21.15.5:0/2660318257 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fade4003e00 con 0x562a9fe416b0 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 -- 172.21.15.5:0/2660318257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 msgr2=0x562a9ffdfc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffdfc60 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fade400fa40 tx=0x7fade40058e0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 -- 172.21.15.5:0/2660318257 shutdown_connections 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562a9fff3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffdfc60 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 --2- 172.21.15.5:0/2660318257 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562a9ffeffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.100+0000 7fae02d43b80 1 -- 172.21.15.5:0/2660318257 >> 172.21.15.5:0/2660318257 conn(0x562a9ffdabe0 msgr2=0x562a9fff7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.101+0000 7fae02d43b80 1 -- 172.21.15.5:0/2660318257 shutdown_connections 2023-07-21T19:20:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.101+0000 7fae02d43b80 1 -- 172.21.15.5:0/2660318257 wait complete. 2023-07-21T19:20:14.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.101+0000 7fae02d43b80 1 -- start start 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.101+0000 7fae02d43b80 1 Processor -- start 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562aa0000630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aa005a180 con 0x562a9ffedf80 2023-07-21T19:20:14.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aa005a2c0 con 0x562a9fe416b0 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562aa005a430 con 0x562a9ffee3f0 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fae02d43b80 10 client.0 _ll_register_callbacks cb 0x7fade0003e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fadec99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fadec99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35982/0 (socket says 172.21.15.5:35982) 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7fadec99a700 1 -- 172.21.15.5:0/880802596 learned_addr learned my addr 172.21.15.5:0/880802596 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.102+0000 7faded99c700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562aa0000630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7faded19b700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 -- 172.21.15.5:0/880802596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 msgr2=0x562aa0000630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562aa0000630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 -- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 msgr2=0x562a9ffffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 -- 172.21.15.5:0/880802596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9fd64d80 con 0x562a9ffedf80 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7faded99c700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562aa0000630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadec99a700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ca5e0 tx=0x7fadd80999f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd9ef6b4c8351114 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadd0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80995a0 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.103+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fadd8010680 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fade40056c0 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fadd800cc90 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 10 client.18228 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562a9fd63250 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fade40056c0 con 0x562a9ffedf80 2023-07-21T19:20:14.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fadd80a1e70 con 0x562a9ffedf80 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 10 client.18228 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 10 client.18228 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 20 client.18228 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa005bbb0 0x562aa005dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 4 client.18228 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.104+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a9fef1ff0 con 0x562aa005bbb0 2023-07-21T19:20:14.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.105+0000 7faded19b700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa005bbb0 0x562aa005dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.105+0000 7faded19b700 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa005bbb0 0x562aa005dfc0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c36480cde8f9bcd0 server_cookie=c8ce28f00a8e03d7 in_seq=0 out_seq=0 2023-07-21T19:20:14.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.105+0000 7fadd0ff9700 10 client.18228 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.118+0000 7fadd0ff9700 1 -- 172.21.15.5:0/880802596 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a9fef1ff0 con 0x562aa005bbb0 2023-07-21T19:20:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.118+0000 7fadd0ff9700 10 client.18228 handle_command_reply: tid=0 2023-07-21T19:20:14.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.118+0000 7fae02d43b80 1 client.18228 shutdown 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 msgr2=0x562aa00000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7fadd80ca5e0 tx=0x7fadd80999f0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 shutdown_connections 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a9ffee3f0 0x562aa0000630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a9fe416b0 0x562a9ffffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562aa005bbb0 0x562aa005dfc0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 --2- 172.21.15.5:0/880802596 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a9ffedf80 0x562aa00000f0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fadd0ff9700 0 client.18228 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 >> 172.21.15.5:0/880802596 conn(0x562a9ffdabe0 msgr2=0x562a9fff3110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 shutdown_connections 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 1 -- 172.21.15.5:0/880802596 wait complete. 2023-07-21T19:20:14.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.119+0000 7fae02d43b80 20 client.18228 trim_cache size 0 max 16384 2023-07-21T19:20:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:20:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:20:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:20:14.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 msgr2=0x7fadd40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 0x7fadd40a37f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fade819a5f0 tx=0x7fade000b040 comp rx=0 tx=0).stop 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 msgr2=0x7fade8199e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fadd800e940 tx=0x7fadd800ec50 comp rx=0 tx=0).stop 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 shutdown_connections 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fadd40a1340 0x7fadd40a37f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fade8104e50 0x7fade8193750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fade8067c10 0x7fade81998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 --2- 172.21.15.5:0/2235445055 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fade80685b0 0x7fade8199e30 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 >> 172.21.15.5:0/2235445055 conn(0x7fade8074760 msgr2=0x7fade80fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 shutdown_connections 2023-07-21T19:20:14.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:14.120+0000 7fadef3ff700 1 -- 172.21.15.5:0/2235445055 wait complete. 2023-07-21T19:20:14.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 -- 172.21.15.5:0/943471665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e80fe9d0 msgr2=0x7f25e80fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/943471665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e80fe9d0 0x7f25e80fee40 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f25d0009990 tx=0x7f25d0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 -- 172.21.15.5:0/943471665 shutdown_connections 2023-07-21T19:20:14.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/943471665 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 0x7f25e810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/943471665 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e81049e0 0x7f25e8104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/943471665 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e80fe9d0 0x7f25e80fee40 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 -- 172.21.15.5:0/943471665 >> 172.21.15.5:0/943471665 conn(0x7f25e80fa4b0 msgr2=0x7f25e80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 -- 172.21.15.5:0/943471665 shutdown_connections 2023-07-21T19:20:14.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.124+0000 7f25ed9ff700 1 -- 172.21.15.5:0/943471665 wait complete. 2023-07-21T19:20:14.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.125+0000 7f25ed9ff700 1 Processor -- start 2023-07-21T19:20:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.125+0000 7f25ed9ff700 1 -- start start 2023-07-21T19:20:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 0x7f25e819b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 0x7f25e819bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8196020 con 0x7f25e81049e0 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8196190 con 0x7f25e80ff380 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25ed9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8196350 con 0x7f25e80fe9d0 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36010/0 (socket says 172.21.15.5:36010) 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25e77fe700 1 -- 172.21.15.5:0/4038509480 learned_addr learned my addr 172.21.15.5:0/4038509480 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.126+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 0x7f25e819b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25de7fc700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 0x7f25e819bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 -- 172.21.15.5:0/4038509480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 msgr2=0x7f25e819bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 0x7f25e819bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 -- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 msgr2=0x7f25e8195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 -- 172.21.15.5:0/4038509480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00095f0 con 0x7f25e80fe9d0 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e77fe700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.127+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 0x7f25e819b870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25d800eee0 tx=0x7f25d800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.128+0000 7f25ec9fd700 1 -- 172.21.15.5:0/4038509480 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8018070 con 0x7f25e80fe9d0 2023-07-21T19:20:14.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.128+0000 7f25ec9fd700 1 -- 172.21.15.5:0/4038509480 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25d800f040 con 0x7f25e80fe9d0 2023-07-21T19:20:14.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.128+0000 7f25ec9fd700 1 -- 172.21.15.5:0/4038509480 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8004280 con 0x7f25e80fe9d0 2023-07-21T19:20:14.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.128+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e81965d0 con 0x7f25e80fe9d0 2023-07-21T19:20:14.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.128+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a27f0 con 0x7f25e80fe9d0 2023-07-21T19:20:14.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.132+0000 7f25ec9fd700 1 -- 172.21.15.5:0/4038509480 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f25d8003a00 con 0x7f25e80fe9d0 2023-07-21T19:20:14.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.134+0000 7f25ec9fd700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 0x7f25d40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.134+0000 7f25ec9fd700 1 -- 172.21.15.5:0/4038509480 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25d80ccc70 con 0x7f25e80fe9d0 2023-07-21T19:20:14.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 Processor -- start 2023-07-21T19:20:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 -- start start 2023-07-21T19:20:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f25de7fc700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 0x7f25d40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9af9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef8a6cb0 con 0x55d8ef9a0be0 2023-07-21T19:20:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef8f9a40 con 0x55d8ef9a02d0 2023-07-21T19:20:14.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.135+0000 7f2601389b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef8f2ed0 con 0x55d8ef9afee0 2023-07-21T19:20:14.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57374/0 (socket says 172.21.15.5:57374) 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e77fe700 1 -- 172.21.15.5:0/1883385433 learned_addr learned my addr 172.21.15.5:0/1883385433 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25de7fc700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 0x7f25d40a3870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25d0006010 tx=0x7f25d0005960 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25de7fc700 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9af9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 -- 172.21.15.5:0/1883385433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 msgr2=0x55d8ef9b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 -- 172.21.15.5:0/1883385433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a0be0 msgr2=0x55d8ef9af9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9af9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e6ffd700 1 -- 172.21.15.5:0/1883385433 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ef720de0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.136+0000 7f25e77fe700 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.137+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9a06a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25d80ca360 tx=0x7f25d8099930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a847fd437b876d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.137+0000 7f25ddffb700 1 -- 172.21.15.5:0/1883385433 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80092e0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.137+0000 7f25ddffb700 1 -- 172.21.15.5:0/1883385433 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25d8099d70 con 0x55d8ef9a02d0 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.137+0000 7f25ddffb700 1 -- 172.21.15.5:0/1883385433 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8099fd0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 -- 172.21.15.5:0/1883385433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 msgr2=0x55d8ef9a06a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9a06a0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f25d80ca360 tx=0x7f25d8099930 comp rx=0 tx=0).stop 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 -- 172.21.15.5:0/1883385433 shutdown_connections 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9a06a0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 --2- 172.21.15.5:0/1883385433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9af9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 -- 172.21.15.5:0/1883385433 >> 172.21.15.5:0/1883385433 conn(0x55d8ef996da0 msgr2=0x55d8ef99c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 -- 172.21.15.5:0/1883385433 shutdown_connections 2023-07-21T19:20:14.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.138+0000 7f2601389b80 1 -- 172.21.15.5:0/1883385433 wait complete. 2023-07-21T19:20:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.139+0000 7f2601389b80 1 -- start start 2023-07-21T19:20:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.139+0000 7f2601389b80 1 Processor -- start 2023-07-21T19:20:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9acc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9afee0 0x55d8ef9ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef892ac0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef8927c0 con 0x55d8ef9afee0 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef776920 con 0x55d8ef9a0be0 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f2601389b80 10 client.0 _ll_register_callbacks cb 0x55d8ef890760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:14.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f25de7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f25de7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57384/0 (socket says 172.21.15.5:57384) 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f25de7fc700 1 -- 172.21.15.5:0/3452592404 learned_addr learned my addr 172.21.15.5:0/3452592404 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.140+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9acc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25e77fe700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9afee0 0x55d8ef9ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 -- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9afee0 msgr2=0x55d8ef9ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9afee0 0x55d8ef9ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 -- 172.21.15.5:0/3452592404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a02d0 msgr2=0x55d8ef9acc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9acc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 -- 172.21.15.5:0/3452592404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e81a27f0 con 0x55d8ef9a0be0 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25de7fc700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25d0015000 tx=0x7f25d0009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa66bdaf74c77637 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.141+0000 7f25c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d000f460 con 0x55d8ef9a0be0 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25d000f5c0 con 0x55d8ef9a0be0 2023-07-21T19:20:14.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8ef84c4f0 con 0x55d8ef9a0be0 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d000f780 con 0x55d8ef9a0be0 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f2601389b80 10 client.27826 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d8ef709c60 con 0x55d8ef9a0be0 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.142+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25d003c020 con 0x55d8ef9a0be0 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25d002fa00 con 0x55d8ef9a0be0 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 10 client.27826 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 10 client.27826 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:14.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 20 client.27826 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8ef9bf810 0x55d8ef9c1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 4 client.27826 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8ef8ebeb0 con 0x55d8ef9bf810 2023-07-21T19:20:14.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.143+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8ef9bf810 0x55d8ef9c1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.144+0000 7f25c6ffd700 10 client.27826 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.144+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8ef9bf810 0x55d8ef9c1c20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e074a6cd1316d6a1 server_cookie=f92318467994e991 in_seq=0 out_seq=0 2023-07-21T19:20:14.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f25c6ffd700 1 -- 172.21.15.5:0/3452592404 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8ef8ebeb0 con 0x55d8ef9bf810 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f25c6ffd700 10 client.27826 handle_command_reply: tid=0 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 client.27826 shutdown 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 msgr2=0x55d8ef9ad1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f25d0015000 tx=0x7f25d0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 shutdown_connections 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9afee0 0x55d8ef9ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8ef9bf810 0x55d8ef9c1c20 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9acc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 --2- 172.21.15.5:0/3452592404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ad1a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f25c6ffd700 0 client.27826 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 >> 172.21.15.5:0/3452592404 conn(0x55d8ef996da0 msgr2=0x55d8ef99c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 shutdown_connections 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 1 -- 172.21.15.5:0/3452592404 wait complete. 2023-07-21T19:20:14.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.145+0000 7f2601389b80 20 client.27826 trim_cache size 0 max 16384 2023-07-21T19:20:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 Processor -- start 2023-07-21T19:20:14.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 -- start start 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ac440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9a9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef9a1d90 con 0x55d8ef9afee0 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef993f80 con 0x55d8ef9a0be0 2023-07-21T19:20:14.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.161+0000 7f2601389b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef9ab1e0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9a9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57398/0 (socket says 172.21.15.5:57398) 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 -- 172.21.15.5:0/3314226713 learned_addr learned my addr 172.21.15.5:0/3314226713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25de7fc700 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ac440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 -- 172.21.15.5:0/3314226713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 msgr2=0x55d8ef9ac440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ac440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 -- 172.21.15.5:0/3314226713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 msgr2=0x55d8ef9a9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9a9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 -- 172.21.15.5:0/3314226713 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8ef747250 con 0x55d8ef9a02d0 2023-07-21T19:20:14.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.162+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25d8004b80 tx=0x7f25d8003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57476da6c7c38766 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f25deffd700 1 -- 172.21.15.5:0/3314226713 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d809dda0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f25deffd700 1 -- 172.21.15.5:0/3314226713 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25d801be10 con 0x55d8ef9a02d0 2023-07-21T19:20:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f25deffd700 1 -- 172.21.15.5:0/3314226713 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d8099260 con 0x55d8ef9a02d0 2023-07-21T19:20:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 -- 172.21.15.5:0/3314226713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 msgr2=0x55d8ef9b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f25d8004b80 tx=0x7f25d8003f90 comp rx=0 tx=0).stop 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 -- 172.21.15.5:0/3314226713 shutdown_connections 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9a9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b68b0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 --2- 172.21.15.5:0/3314226713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9ac440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.163+0000 7f2601389b80 1 -- 172.21.15.5:0/3314226713 >> 172.21.15.5:0/3314226713 conn(0x55d8ef996da0 msgr2=0x55d8ef99ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.164+0000 7f2601389b80 1 -- 172.21.15.5:0/3314226713 shutdown_connections 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.164+0000 7f2601389b80 1 -- 172.21.15.5:0/3314226713 wait complete. 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.164+0000 7f2601389b80 1 -- start start 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.164+0000 7f2601389b80 1 Processor -- start 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9b6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef9b5190 con 0x55d8ef9afee0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef9b5300 con 0x55d8ef9a0be0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8ef9b54c0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f2601389b80 10 client.0 _ll_register_callbacks cb 0x7f25d000bc10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36038/0 (socket says 172.21.15.5:36038) 2023-07-21T19:20:14.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25e77fe700 1 -- 172.21.15.5:0/2082352804 learned_addr learned my addr 172.21.15.5:0/2082352804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25e6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57408/0 (socket says 172.21.15.5:57408) 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.165+0000 7f25de7fc700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9b6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 -- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 msgr2=0x55d8ef9b6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9b6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 -- 172.21.15.5:0/2082352804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 msgr2=0x55d8ef9b4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 -- 172.21.15.5:0/2082352804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d8ef720de0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e77fe700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25e6ffd700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ca70 tx=0x7f25d80093a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e441e28531f945b7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25c6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25d80ca7f0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25d80981d0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25d8099d70 con 0x55d8ef9a02d0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f25d8099fd0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.166+0000 7f2601389b80 10 client.27832 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d8ef71f2b0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25d8099d70 con 0x55d8ef9a02d0 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25d80a1ca0 con 0x55d8ef9a02d0 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 10 client.27832 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 10 client.27832 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 20 client.27832 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:14.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8efa144e0 0x55d8efa168c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 4 client.27832 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:20:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8ef8ae050 con 0x55d8efa144e0 2023-07-21T19:20:14.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.167+0000 7f25de7fc700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8efa144e0 0x55d8efa168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f25c6ffd700 10 client.27832 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f25de7fc700 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8efa144e0 0x55d8efa168c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12e239b638ed6747 server_cookie=dda67b39c331c09d in_seq=0 out_seq=0 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f25c6ffd700 1 -- 172.21.15.5:0/2082352804 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8ef8ae050 con 0x55d8efa144e0 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f25c6ffd700 10 client.27832 handle_command_reply: tid=0 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 client.27832 shutdown 2023-07-21T19:20:14.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 msgr2=0x55d8ef9b65c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f25d800ca70 tx=0x7f25d80093a0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 shutdown_connections 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8efa144e0 0x55d8efa168c0 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8ef9afee0 0x55d8ef9b4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8ef9a02d0 0x55d8ef9b65c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f25c6ffd700 0 client.27832 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 --2- 172.21.15.5:0/2082352804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8ef9a0be0 0x55d8ef9b6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.168+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 >> 172.21.15.5:0/2082352804 conn(0x55d8ef996da0 msgr2=0x55d8ef9ad440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 shutdown_connections 2023-07-21T19:20:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f2601389b80 1 -- 172.21.15.5:0/2082352804 wait complete. 2023-07-21T19:20:14.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f2601389b80 20 client.27832 trim_cache size 0 max 16384 2023-07-21T19:20:14.172 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:20:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 msgr2=0x7f25d40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 0x7f25d40a3870 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f25d0006010 tx=0x7f25d0005960 comp rx=0 tx=0).stop 2023-07-21T19:20:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 msgr2=0x7f25e819b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:14.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 0x7f25e819b870 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f25d800eee0 tx=0x7f25d800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 shutdown_connections 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25d40a13c0 0x7f25d40a3870 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25e81049e0 0x7f25e8195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.169+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25e80fe9d0 0x7f25e819b870 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.170+0000 7f25ed9ff700 1 --2- 172.21.15.5:0/4038509480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25e80ff380 0x7f25e819bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.170+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 >> 172.21.15.5:0/4038509480 conn(0x7f25e80fa4b0 msgr2=0x7f25e80fbfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.170+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 shutdown_connections 2023-07-21T19:20:14.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:14.170+0000 7f25ed9ff700 1 -- 172.21.15.5:0/4038509480 wait complete. 2023-07-21T19:20:14.801 INFO:teuthology.orchestra.run.smithi005.stderr:48800000 of 50000000 tuples (97%) done (elapsed 732.37 s, remaining 18.01 s) 2023-07-21T19:20:15.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:20:15.187 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:20:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 -- 172.21.15.5:0/1360919916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c110af0 msgr2=0x7f708c112ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 --2- 172.21.15.5:0/1360919916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c110af0 0x7f708c112ed0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f708400b0a0 tx=0x7f708400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 -- 172.21.15.5:0/1360919916 shutdown_connections 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 --2- 172.21.15.5:0/1360919916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c110af0 0x7f708c112ed0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 --2- 172.21.15.5:0/1360919916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0728b0 0x7f708c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 --2- 172.21.15.5:0/1360919916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c0731c0 0x7f708c1105b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 -- 172.21.15.5:0/1360919916 >> 172.21.15.5:0/1360919916 conn(0x7f708c0fb5e0 msgr2=0x7f708c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 -- 172.21.15.5:0/1360919916 shutdown_connections 2023-07-21T19:20:15.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.427+0000 7f7091bff700 1 -- 172.21.15.5:0/1360919916 wait complete. 2023-07-21T19:20:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.429+0000 7f7091bff700 1 Processor -- start 2023-07-21T19:20:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.429+0000 7f7091bff700 1 -- start start 2023-07-21T19:20:15.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 0x7f708c19c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c110af0 0x7f708c1a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1a1210 con 0x7f708c0728b0 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1a1380 con 0x7f708c0731c0 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f7091bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f708c1a1540 con 0x7f708c110af0 2023-07-21T19:20:15.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36068/0 (socket says 172.21.15.5:36068) 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 -- 172.21.15.5:0/3887739246 learned_addr learned my addr 172.21.15.5:0/3887739246 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708bfff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c110af0 0x7f708c1a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708affd700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 0x7f708c19c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 -- 172.21.15.5:0/3887739246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c110af0 msgr2=0x7f708c1a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c110af0 0x7f708c1a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 -- 172.21.15.5:0/3887739246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 msgr2=0x7f708c19c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 0x7f708c19c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.430+0000 7f708b7fe700 1 -- 172.21.15.5:0/3887739246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084009d00 con 0x7f708c0728b0 2023-07-21T19:20:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f708affd700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 0x7f708c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f707c00e9a0 tx=0x7f707c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f7088ff9700 1 -- 172.21.15.5:0/3887739246 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c009790 con 0x7f708c0728b0 2023-07-21T19:20:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f708c1a1820 con 0x7f708c0728b0 2023-07-21T19:20:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f7088ff9700 1 -- 172.21.15.5:0/3887739246 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f707c00cbd0 con 0x7f708c0728b0 2023-07-21T19:20:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f7088ff9700 1 -- 172.21.15.5:0/3887739246 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c018680 con 0x7f708c0728b0 2023-07-21T19:20:15.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.431+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c1a9060 con 0x7f708c0728b0 2023-07-21T19:20:15.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.435+0000 7f7088ff9700 1 -- 172.21.15.5:0/3887739246 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f707c00cd40 con 0x7f708c0728b0 2023-07-21T19:20:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.436+0000 7f70a55d0b80 1 Processor -- start 2023-07-21T19:20:15.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.436+0000 7f70a55d0b80 1 -- start start 2023-07-21T19:20:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a64df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a546d30 con 0x55f75a640350 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a599ac0 con 0x55f75a640c60 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a592f50 con 0x55f75a64bb50 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f708bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f708bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36072/0 (socket says 172.21.15.5:36072) 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f708bfff700 1 -- 172.21.15.5:0/2849446633 learned_addr learned my addr 172.21.15.5:0/2849446633 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f708b7fe700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f708affd700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a64df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.437+0000 7f7088ff9700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 0x7f70740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 -- 172.21.15.5:0/2849446633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 msgr2=0x55f75a64df30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a64df30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 -- 172.21.15.5:0/2849446633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 msgr2=0x55f75a64b610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 -- 172.21.15.5:0/2849446633 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f75a3c0e60 con 0x55f75a640350 2023-07-21T19:20:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708affd700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a64df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:15.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f7088ff9700 1 -- 172.21.15.5:0/3887739246 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f707c014070 con 0x7f708c0728b0 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f708c069a80 tx=0x7f7084009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e858ea3929929ff1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f708bfff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 0x7f70740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.438+0000 7f7071ffb700 1 -- 172.21.15.5:0/2849446633 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f708400e040 con 0x55f75a640350 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.439+0000 7f7071ffb700 1 -- 172.21.15.5:0/2849446633 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7084007d90 con 0x55f75a640350 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.439+0000 7f7071ffb700 1 -- 172.21.15.5:0/2849446633 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7084027450 con 0x55f75a640350 2023-07-21T19:20:15.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.439+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2849446633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 msgr2=0x55f75a640720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.439+0000 7f708bfff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 0x7f70740a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7084009cd0 tx=0x7f7084003f00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.439+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f708c069a80 tx=0x7f7084009c60 comp rx=0 tx=0).stop 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2849446633 shutdown_connections 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a64df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a640720 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2849446633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2849446633 >> 172.21.15.5:0/2849446633 conn(0x55f75a636e20 msgr2=0x55f75a63c720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2849446633 shutdown_connections 2023-07-21T19:20:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2849446633 wait complete. 2023-07-21T19:20:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.440+0000 7f70a55d0b80 1 -- start start 2023-07-21T19:20:15.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 Processor -- start 2023-07-21T19:20:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6487d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a648ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a532b40 con 0x55f75a640350 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a532840 con 0x55f75a640c60 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a4169a0 con 0x55f75a64bb50 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.441+0000 7f70a55d0b80 10 client.0 _ll_register_callbacks cb 0x55f75a5307e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36080/0 (socket says 172.21.15.5:36080) 2023-07-21T19:20:15.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 -- 172.21.15.5:0/3590347138 learned_addr learned my addr 172.21.15.5:0/3590347138 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708affd700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6487d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708bfff700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a648ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 -- 172.21.15.5:0/3590347138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 msgr2=0x55f75a648ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a648ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 -- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 msgr2=0x55f75a6487d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6487d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 -- 172.21.15.5:0/3590347138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f708c113790 con 0x55f75a640350 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708bfff700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a648ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f708b7fe700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f707c095e10 tx=0x7f707c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e638278148875b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.442+0000 7f706affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707c0999f0 con 0x55f75a640350 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f75a4ec570 con 0x55f75a640350 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f707c09a090 con 0x55f75a640350 2023-07-21T19:20:15.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f707c09e0c0 con 0x55f75a640350 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 10 client.18258 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55f75a3a9ce0 con 0x55f75a640350 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f75a4ec570 con 0x55f75a640350 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f707c095f10 con 0x55f75a640350 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 10 client.18258 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 10 client.18258 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:15.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 20 client.18258 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a65b3d0 0x55f75a65d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 4 client.18258 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.443+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f75a58bf30 con 0x55f75a65b3d0 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.444+0000 7f708affd700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a65b3d0 0x55f75a65d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.444+0000 7f708affd700 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a65b3d0 0x55f75a65d7e0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6497a785ae916c84 server_cookie=c97d5cc3e4b87bd3 in_seq=0 out_seq=0 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.444+0000 7f706affd700 10 client.18258 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f706affd700 1 -- 172.21.15.5:0/3590347138 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f75a58bf30 con 0x55f75a65b3d0 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f706affd700 10 client.18258 handle_command_reply: tid=0 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 client.18258 shutdown 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 msgr2=0x55f75a64b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f707c095e10 tx=0x7f707c00eef0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 shutdown_connections 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a64bb50 0x55f75a648ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a65b3d0 0x55f75a65d7e0 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a640350 0x55f75a64b5c0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/3590347138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6487d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f706affd700 0 client.18258 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.445+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 >> 172.21.15.5:0/3590347138 conn(0x55f75a636e20 msgr2=0x55f75a63c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.446+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 shutdown_connections 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.446+0000 7f70a55d0b80 1 -- 172.21.15.5:0/3590347138 wait complete. 2023-07-21T19:20:15.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.446+0000 7f70a55d0b80 20 client.18258 trim_cache size 0 max 16384 2023-07-21T19:20:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 Processor -- start 2023-07-21T19:20:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 -- start start 2023-07-21T19:20:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a648780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a645a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a63cca0 con 0x55f75a64bb50 2023-07-21T19:20:15.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a6369f0 con 0x55f75a640c60 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.467+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a642220 con 0x55f75a640350 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a648780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a645a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59484/0 (socket says 172.21.15.5:59484) 2023-07-21T19:20:15.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a648780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57442/0 (socket says 172.21.15.5:57442) 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 -- 172.21.15.5:0/2004120992 learned_addr learned my addr 172.21.15.5:0/2004120992 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 -- 172.21.15.5:0/2004120992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 msgr2=0x55f75a648780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a648780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 -- 172.21.15.5:0/2004120992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 msgr2=0x55f75a645a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a645a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 -- 172.21.15.5:0/2004120992 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f75a3e72d0 con 0x55f75a640c60 2023-07-21T19:20:15.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708bfff700 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a645a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.468+0000 7f708affd700 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55f75a65e0f0 tx=0x7f7080009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e5b41ec791f81d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.469+0000 7f70727fc700 1 -- 172.21.15.5:0/2004120992 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080017040 con 0x55f75a640c60 2023-07-21T19:20:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.469+0000 7f70727fc700 1 -- 172.21.15.5:0/2004120992 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f708001cde0 con 0x55f75a640c60 2023-07-21T19:20:15.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.469+0000 7f70727fc700 1 -- 172.21.15.5:0/2004120992 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800116a0 con 0x55f75a640c60 2023-07-21T19:20:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2004120992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 msgr2=0x55f75a64b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55f75a65e0f0 tx=0x7f7080009c30 comp rx=0 tx=0).stop 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2004120992 shutdown_connections 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a645a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a648780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/2004120992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a64b0f0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2004120992 >> 172.21.15.5:0/2004120992 conn(0x55f75a636e20 msgr2=0x55f75a648070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2004120992 shutdown_connections 2023-07-21T19:20:15.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- 172.21.15.5:0/2004120992 wait complete. 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.470+0000 7f70a55d0b80 1 -- start start 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 Processor -- start 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a6a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a651980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a652090 con 0x55f75a64bb50 2023-07-21T19:20:15.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a652200 con 0x55f75a640c60 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f75a6523c0 con 0x55f75a640350 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f70a55d0b80 10 client.0 _ll_register_callbacks cb 0x7f707c010a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a6a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f708affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f708b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a6a9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57454/0 (socket says 172.21.15.5:57454) 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f708bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a651980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.471+0000 7f708affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59492/0 (socket says 172.21.15.5:59492) 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708b7fe700 1 -- 172.21.15.5:0/739502487 learned_addr learned my addr 172.21.15.5:0/739502487 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 -- 172.21.15.5:0/739502487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 msgr2=0x55f75a6a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a6a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 -- 172.21.15.5:0/739502487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 msgr2=0x55f75a651980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a651980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 -- 172.21.15.5:0/739502487 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f75a3c0e60 con 0x55f75a640c60 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708bfff700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a651980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f708affd700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f75a648ce0 tx=0x7f7080003e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db0d8807be88db7f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.472+0000 7f706affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080017040 con 0x55f75a640c60 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7084007d90 con 0x55f75a640c60 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7080003760 con 0x55f75a640c60 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7080004010 con 0x55f75a640c60 2023-07-21T19:20:15.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f70a55d0b80 10 client.27914 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f7080009730 con 0x55f75a640c60 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.473+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7084007d90 con 0x55f75a640c60 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7080025450 con 0x55f75a640c60 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 10 client.27914 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 10 client.27914 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:15.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 20 client.27914 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a6b19a0 0x55f75a6b3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 4 client.27914 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:20:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55f75a54e0d0 con 0x55f75a6b19a0 2023-07-21T19:20:15.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.474+0000 7f708b7fe700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a6b19a0 0x55f75a6b3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.475+0000 7f706affd700 10 client.27914 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.475+0000 7f708b7fe700 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a6b19a0 0x55f75a6b3db0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eaadedaed6f2f79 server_cookie=5188d94718bf7dc1 in_seq=0 out_seq=0 2023-07-21T19:20:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.475+0000 7f706affd700 1 -- 172.21.15.5:0/739502487 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55f75a54e0d0 con 0x55f75a6b19a0 2023-07-21T19:20:15.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.475+0000 7f706affd700 10 client.27914 handle_command_reply: tid=0 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 client.27914 shutdown 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 msgr2=0x55f75a6aa180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x55f75a648ce0 tx=0x7f7080003e20 comp rx=0 tx=0).stop 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 shutdown_connections 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f75a6b19a0 0x55f75a6b3db0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f75a64bb50 0x55f75a651980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f75a640350 0x55f75a6a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 --2- 172.21.15.5:0/739502487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f75a640c60 0x55f75a6aa180 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 >> 172.21.15.5:0/739502487 conn(0x55f75a636e20 msgr2=0x55f75a642460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f706affd700 0 client.27914 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 shutdown_connections 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 1 -- 172.21.15.5:0/739502487 wait complete. 2023-07-21T19:20:15.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.476+0000 7f70a55d0b80 20 client.27914 trim_cache size 0 max 16384 2023-07-21T19:20:15.479 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:20:15.479 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:20:15.479 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:20:15.479 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:20:15.479 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 msgr2=0x7f70740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 0x7f70740a3770 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7084009cd0 tx=0x7f7084003f00 comp rx=0 tx=0).stop 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 msgr2=0x7f708c19c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f707c00e9a0 tx=0x7f707c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 shutdown_connections 2023-07-21T19:20:15.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f708c110af0 0x7f708c1a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f70740a12c0 0x7f70740a3770 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f708c0728b0 0x7f708c19c230 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 --2- 172.21.15.5:0/3887739246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f708c0731c0 0x7f708c19c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 >> 172.21.15.5:0/3887739246 conn(0x7f708c0fb5e0 msgr2=0x7f708c0fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.477+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 shutdown_connections 2023-07-21T19:20:15.481 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.478+0000 7f7091bff700 1 -- 172.21.15.5:0/3887739246 wait complete. 2023-07-21T19:20:15.493 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:20:15.577 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:15 smithi005 ceph-mon[126460]: pgmap v396: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 29 MiB/s wr, 94 op/s 2023-07-21T19:20:15.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/4128057383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc80ffae0 msgr2=0x7fbbc80fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/4128057383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc80ffae0 0x7fbbc80fff50 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc009990 tx=0x7fbbbc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/4128057383 shutdown_connections 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/4128057383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc8100490 0x7fbbc810c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/4128057383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 0x7fbbc8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/4128057383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc80ffae0 0x7fbbc80fff50 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/4128057383 >> 172.21.15.5:0/4128057383 conn(0x7fbbc80fb5e0 msgr2=0x7fbbc80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/4128057383 shutdown_connections 2023-07-21T19:20:15.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.765+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/4128057383 wait complete. 2023-07-21T19:20:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 Processor -- start 2023-07-21T19:20:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 -- start start 2023-07-21T19:20:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 0x7fbbc8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 0x7fbbc8073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81a3f90 con 0x7fbbc80ffae0 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81a4100 con 0x7fbbc8100490 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.767+0000 7fbbcd9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc81a42c0 con 0x7fbbc8105af0 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 0x7fbbc8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 0x7fbbc8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36132/0 (socket says 172.21.15.5:36132) 2023-07-21T19:20:15.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 0x7fbbc8073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59514/0 (socket says 172.21.15.5:59514) 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/1930328794 learned_addr learned my addr 172.21.15.5:0/1930328794 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 -- 172.21.15.5:0/1930328794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 msgr2=0x7fbbc8073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 0x7fbbc8073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 -- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 msgr2=0x7fbbc8072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 0x7fbbc8072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.768+0000 7fbbc67fc700 1 -- 172.21.15.5:0/1930328794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbbc0095f0 con 0x7fbbc8100490 2023-07-21T19:20:15.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0098e0 tx=0x7fbbbc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbcc9fd700 1 -- 172.21.15.5:0/1930328794 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc01d070 con 0x7fbbc8100490 2023-07-21T19:20:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc81a4540 con 0x7fbbc8100490 2023-07-21T19:20:15.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbcc9fd700 1 -- 172.21.15.5:0/1930328794 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbbc022470 con 0x7fbbc8100490 2023-07-21T19:20:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbcc9fd700 1 -- 172.21.15.5:0/1930328794 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbbc017650 con 0x7fbbc8100490 2023-07-21T19:20:15.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.769+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc81a49c0 con 0x7fbbc8100490 2023-07-21T19:20:15.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.774+0000 7fbbcc9fd700 1 -- 172.21.15.5:0/1930328794 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbbbc0225e0 con 0x7fbbc8100490 2023-07-21T19:20:15.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.775+0000 7fbbe13dcb80 1 Processor -- start 2023-07-21T19:20:15.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.775+0000 7fbbe13dcb80 1 -- start start 2023-07-21T19:20:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.775+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.775+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d41590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.775+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d43eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343c3ccb0 con 0x565343d41ad0 2023-07-21T19:20:15.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343c8fa40 con 0x565343d36be0 2023-07-21T19:20:15.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343c88ed0 con 0x565343d362d0 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57472/0 (socket says 172.21.15.5:57472) 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc77fe700 1 -- 172.21.15.5:0/3213722301 learned_addr learned my addr 172.21.15.5:0/3213722301 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbcc9fd700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 0x7fbbb00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d41590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbcc9fd700 1 -- 172.21.15.5:0/1930328794 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbbbc0ce060 con 0x7fbbc8100490 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d43eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.776+0000 7fbbc77fe700 1 -- 172.21.15.5:0/3213722301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 msgr2=0x565343d41590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d41590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 -- 172.21.15.5:0/3213722301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 msgr2=0x565343d43eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d43eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 -- 172.21.15.5:0/3213722301 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565343ab6de0 con 0x565343d362d0 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 0x7fbbb00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0011790 tx=0x7fbbc0011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=75066e579e546c13 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbadffb700 1 -- 172.21.15.5:0/3213722301 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc000e040 con 0x565343d362d0 2023-07-21T19:20:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbadffb700 1 -- 172.21.15.5:0/3213722301 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbc000f040 con 0x565343d362d0 2023-07-21T19:20:15.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.777+0000 7fbbadffb700 1 -- 172.21.15.5:0/3213722301 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0021880 con 0x565343d362d0 2023-07-21T19:20:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/3213722301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 msgr2=0x565343d366a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fbbc0011790 tx=0x7fbbc0011aa0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 0x7fbbb00a3770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00094f0 tx=0x7fbbc001d000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/3213722301 shutdown_connections 2023-07-21T19:20:15.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d43eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d366a0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/3213722301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d41590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/3213722301 >> 172.21.15.5:0/3213722301 conn(0x565343d2cda0 msgr2=0x565343d326a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.778+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/3213722301 shutdown_connections 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.779+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/3213722301 wait complete. 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.779+0000 7fbbe13dcb80 1 -- start start 2023-07-21T19:20:15.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.779+0000 7fbbe13dcb80 1 Processor -- start 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d41510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d3e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343c28ac0 con 0x565343d41ad0 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343c287c0 con 0x565343d36be0 2023-07-21T19:20:15.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343b0c920 con 0x565343d362d0 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbe13dcb80 10 client.0 _ll_register_callbacks cb 0x565343c26760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.780+0000 7fbbc67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d3e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc67fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d3e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59528/0 (socket says 172.21.15.5:59528) 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d41510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc67fc700 1 -- 172.21.15.5:0/914081971 learned_addr learned my addr 172.21.15.5:0/914081971 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 msgr2=0x565343d3e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d3e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/914081971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 msgr2=0x565343d3eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/914081971 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc0011380 con 0x565343d362d0 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d41510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8012810 tx=0x7fbbb8012b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e1a30d2ba8e2caa8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.781+0000 7fbba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8017070 con 0x565343d362d0 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbbb8004830 con 0x565343d362d0 2023-07-21T19:20:15.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x565343be24f0 con 0x565343d362d0 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb80122e0 con 0x565343d362d0 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbbe13dcb80 10 client.27859 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x565343a9fc60 con 0x565343d362d0 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x565343be24f0 con 0x565343d362d0 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.782+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbb8025690 con 0x565343d362d0 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 10 client.27859 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 10 client.27859 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:15.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 20 client.27859 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343d51340 0x565343d53750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 4 client.27859 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565343c81eb0 con 0x565343d51340 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.783+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343d51340 0x565343d53750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.784+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343d51340 0x565343d53750 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7925af474d9805ae server_cookie=3fc7dd97362b5904 in_seq=0 out_seq=0 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.784+0000 7fbba6ffd700 10 client.27859 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.784+0000 7fbba6ffd700 1 -- 172.21.15.5:0/914081971 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565343c81eb0 con 0x565343d51340 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.784+0000 7fbba6ffd700 10 client.27859 handle_command_reply: tid=0 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.784+0000 7fbbe13dcb80 1 client.27859 shutdown 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 msgr2=0x565343d41510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d41510 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8012810 tx=0x7fbbb8012b20 comp rx=0 tx=0).stop 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 shutdown_connections 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343d51340 0x565343d53750 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d362d0 0x565343d41510 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbba6ffd700 0 client.27859 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/914081971 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d36be0 0x565343d3e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 >> 172.21.15.5:0/914081971 conn(0x565343d2cda0 msgr2=0x565343d32290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 shutdown_connections 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/914081971 wait complete. 2023-07-21T19:20:15.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.785+0000 7fbbe13dcb80 20 client.27859 trim_cache size 0 max 16384 2023-07-21T19:20:15.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.809+0000 7fbbe13dcb80 1 Processor -- start 2023-07-21T19:20:15.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.809+0000 7fbbe13dcb80 1 -- start start 2023-07-21T19:20:15.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.809+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d3fd70 con 0x565343d41ad0 2023-07-21T19:20:15.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d32be0 con 0x565343d362d0 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d381a0 con 0x565343d36be0 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59538/0 (socket says 172.21.15.5:59538) 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/976075768 learned_addr learned my addr 172.21.15.5:0/976075768 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/976075768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 msgr2=0x565343d3df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d3df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/976075768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 msgr2=0x565343d3e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.810+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/976075768 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565343add250 con 0x565343d362d0 2023-07-21T19:20:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.811+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009870 tx=0x7fbbb80122e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd2bc874163e5e69 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.811+0000 7fbbae7fc700 1 -- 172.21.15.5:0/976075768 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8017040 con 0x565343d362d0 2023-07-21T19:20:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.811+0000 7fbbae7fc700 1 -- 172.21.15.5:0/976075768 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbb8005550 con 0x565343d362d0 2023-07-21T19:20:15.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.811+0000 7fbbae7fc700 1 -- 172.21.15.5:0/976075768 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8003aa0 con 0x565343d362d0 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/976075768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 msgr2=0x565343d410b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009870 tx=0x7fbbb80122e0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/976075768 shutdown_connections 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d3e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d410b0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/976075768 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/976075768 >> 172.21.15.5:0/976075768 conn(0x565343d2cda0 msgr2=0x565343d4d7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/976075768 shutdown_connections 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/976075768 wait complete. 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.812+0000 7fbbe13dcb80 1 -- start start 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.813+0000 7fbbe13dcb80 1 Processor -- start 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.813+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.813+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d46610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.813+0000 7fbbe13dcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d46b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.813+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d47090 con 0x565343d41ad0 2023-07-21T19:20:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d47200 con 0x565343d362d0 2023-07-21T19:20:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbe13dcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565343d473c0 con 0x565343d36be0 2023-07-21T19:20:15.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbe13dcb80 10 client.0 _ll_register_callbacks cb 0x7fbbb8009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d46610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59554/0 (socket says 172.21.15.5:59554) 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/834035185 learned_addr learned my addr 172.21.15.5:0/834035185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc77fe700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d46b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/834035185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 msgr2=0x565343d46610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d46610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/834035185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 msgr2=0x565343d46b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d46b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.814+0000 7fbbc6ffd700 1 -- 172.21.15.5:0/834035185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x565343a9fc60 con 0x565343d362d0 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d46610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbbc6ffd700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8026fd0 tx=0x7fbbb8005450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76d049106d9a5263 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:15.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbba6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb8032070 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbbb8000e40 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.815+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbbc000f040 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbbb8017070 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 10 client.27932 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x565343ab52b0 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbbc000f040 con 0x565343d362d0 2023-07-21T19:20:15.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fbbb8035020 con 0x565343d362d0 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 10 client.27932 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 10 client.27932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 20 client.27932 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343dae8e0 0x565343db0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 4 client.27932 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.816+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x565343c44050 con 0x565343dae8e0 2023-07-21T19:20:15.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.817+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343dae8e0 0x565343db0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.817+0000 7fbba6ffd700 10 client.27932 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.817+0000 7fbbc67fc700 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343dae8e0 0x565343db0cc0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=caf8b0f76e752ebb server_cookie=a1823851d278aeea in_seq=0 out_seq=0 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbba6ffd700 1 -- 172.21.15.5:0/834035185 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x565343c44050 con 0x565343dae8e0 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbba6ffd700 10 client.27932 handle_command_reply: tid=0 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 client.27932 shutdown 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 msgr2=0x565343d482e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8026fd0 tx=0x7fbbb8005450 comp rx=0 tx=0).stop 2023-07-21T19:20:15.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 shutdown_connections 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565343dae8e0 0x565343db0cc0 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565343d41ad0 0x565343d46b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565343d362d0 0x565343d482e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 --2- 172.21.15.5:0/834035185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565343d36be0 0x565343d46610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 >> 172.21.15.5:0/834035185 conn(0x565343d2cda0 msgr2=0x565343d318b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbba6ffd700 0 client.27932 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:15.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 shutdown_connections 2023-07-21T19:20:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.818+0000 7fbbe13dcb80 1 -- 172.21.15.5:0/834035185 wait complete. 2023-07-21T19:20:15.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbe13dcb80 20 client.27932 trim_cache size 0 max 16384 2023-07-21T19:20:15.819 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:20:15.820 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:20:15.821 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:20:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 msgr2=0x7fbbb00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 0x7fbbb00a3770 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00094f0 tx=0x7fbbc001d000 comp rx=0 tx=0).stop 2023-07-21T19:20:15.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 msgr2=0x7fbbc8072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbbbc0098e0 tx=0x7fbbbc004730 comp rx=0 tx=0).stop 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 shutdown_connections 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbbc8105af0 0x7fbbc8073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbbb00a12c0 0x7fbbb00a3770 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbbc80ffae0 0x7fbbc8072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 --2- 172.21.15.5:0/1930328794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbbc8100490 0x7fbbc8072d50 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:15.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.819+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 >> 172.21.15.5:0/1930328794 conn(0x7fbbc80fb5e0 msgr2=0x7fbbc80fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.820+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 shutdown_connections 2023-07-21T19:20:15.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:15.820+0000 7fbbcd9ff700 1 -- 172.21.15.5:0/1930328794 wait complete. 2023-07-21T19:20:15.827 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:15 smithi101 ceph-mon[128136]: pgmap v396: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 29 MiB/s wr, 94 op/s 2023-07-21T19:20:15.828 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:20:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:20:15.496+0000 7fccf3a6f700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000019 [...16,head] /volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp/ auth{1=1,2=1} v4161 ap=2 f(v120 m2023-07-21T19:20:02.793158+0000 3=3+0) n(v12 rc2023-07-21T19:20:02.793158+0000 b3321 4=3+1) old_inodes=6 (idft sync r=1) (inest sync r=1) (ifile sync r=1) (iversion lock) caps={15339=pAsLsXsFs/-@305} | dirtyscattered=0 request=0 lock=3 dirfrag=1 caps=1 dirtyparent=0 replicated=1 dirty=0 waiter=0 authpin=1 scrubqueue=0 0x558a8c7f9180]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v38>,<0x10000000005/data v105>,<0x10000000003/postgres v25>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v41>,<0x10000000001/sv_0 v59>,<0x10000000000/_nogroup v42>,<0x1/volumes v38>]//[]","memoryvalue":"(2)0x10000000019:[<0x10000000006/pg_stat_tmp v4161>,<0x10000000005/data v2970>,<0x10000000003/postgres v791>,<0x10000000002/0727d3e5-164d-43fe-9a3c-e84d09ba34e1 v817>,<0x10000000001/sv_0 v832>,<0x10000000000/_nogroup v810>,<0x1/volumes v775>]//[]","error_str":""},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 m2023-07-21T19:20:02.777211+0000 4=4+0)","ondisk_value.rstat":"n(v0 rc2023-07-21T19:20:02.793158+0000 b3321 4=3+1)","memory_value.dirstat":"f(v120 m2023-07-21T19:20:02.793158+0000 3=3+0)","memory_value.rstat":"n(v12 rc2023-07-21T19:20:02.793158+0000 b3321 4=3+1)","error_str":"freshly-calculated rstats don't match existing ones"},"return_code":0} 2023-07-21T19:20:15.836 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:20:15.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:15 smithi156 ceph-mon[128316]: pgmap v396: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 69 KiB/s rd, 29 MiB/s wr, 94 op/s 2023-07-21T19:20:16.340 INFO:teuthology.orchestra.run.smithi005.stderr:48900000 of 50000000 tuples (97%) done (elapsed 733.91 s, remaining 16.51 s) 2023-07-21T19:20:16.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:20:16.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:20:16.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:20:16.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:20:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:20:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:20:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.826 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:16 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:20:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:20:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:20:16.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:20:16.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:20:16.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:20:16.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.905 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:16 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:20:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:20:16.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:20:16.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:20:16.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.916 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: Scrub error on inode 0x10000000019 (/volumes/_nogroup/sv_0/0727d3e5-164d-43fe-9a3c-e84d09ba34e1/postgres/data/pg_stat_tmp) see mds.b log and `damage ls` output for details 2023-07-21T19:20:16.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:20:16.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:16.917 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:16 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:20:17.680 INFO:teuthology.orchestra.run.smithi005.stderr:49000000 of 50000000 tuples (98%) done (elapsed 735.25 s, remaining 15.01 s) 2023-07-21T19:20:17.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:17 smithi101 ceph-mon[128136]: pgmap v397: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:17.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:17 smithi005 ceph-mon[126460]: pgmap v397: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:17.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:17 smithi156 ceph-mon[128316]: pgmap v397: 97 pgs: 97 active+clean; 9.4 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 24 MiB/s wr, 77 op/s 2023-07-21T19:20:18.972 INFO:teuthology.orchestra.run.smithi005.stderr:49100000 of 50000000 tuples (98%) done (elapsed 736.54 s, remaining 13.50 s) 2023-07-21T19:20:19.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:19 smithi101 ceph-mon[128136]: pgmap v398: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:19.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:19 smithi005 ceph-mon[126460]: pgmap v398: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:19.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:19 smithi156 ceph-mon[128316]: pgmap v398: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:20.302 INFO:teuthology.orchestra.run.smithi005.stderr:49200000 of 50000000 tuples (98%) done (elapsed 737.87 s, remaining 12.00 s) 2023-07-21T19:20:21.754 INFO:teuthology.orchestra.run.smithi005.stderr:49300000 of 50000000 tuples (98%) done (elapsed 739.33 s, remaining 10.50 s) 2023-07-21T19:20:21.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:21 smithi101 ceph-mon[128136]: pgmap v399: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:21.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:21 smithi005 ceph-mon[126460]: pgmap v399: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:21.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:21 smithi156 ceph-mon[128316]: pgmap v399: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 24 MiB/s wr, 185 op/s 2023-07-21T19:20:23.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:23 smithi101 ceph-mon[128136]: pgmap v400: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 27 MiB/s wr, 187 op/s 2023-07-21T19:20:23.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:23 smithi005 ceph-mon[126460]: pgmap v400: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 27 MiB/s wr, 187 op/s 2023-07-21T19:20:23.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:23 smithi156 ceph-mon[128316]: pgmap v400: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 226 KiB/s rd, 27 MiB/s wr, 187 op/s 2023-07-21T19:20:25.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:25 smithi101 ceph-mon[128136]: pgmap v401: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 22 MiB/s wr, 155 op/s 2023-07-21T19:20:25.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:25 smithi005 ceph-mon[126460]: pgmap v401: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 22 MiB/s wr, 155 op/s 2023-07-21T19:20:25.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:25 smithi156 ceph-mon[128316]: pgmap v401: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 198 KiB/s rd, 22 MiB/s wr, 155 op/s 2023-07-21T19:20:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 -- 172.21.15.5:0/855567526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9400728b0 msgr2=0x7ff940072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 --2- 172.21.15.5:0/855567526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9400728b0 0x7ff940072c80 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff930009990 tx=0x7ff930009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 -- 172.21.15.5:0/855567526 shutdown_connections 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 --2- 172.21.15.5:0/855567526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff94010c3b0 0x7ff94010e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 --2- 172.21.15.5:0/855567526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff9400728b0 0x7ff940072c80 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 --2- 172.21.15.5:0/855567526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400731c0 0x7ff94010be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 -- 172.21.15.5:0/855567526 >> 172.21.15.5:0/855567526 conn(0x7ff9400fb6f0 msgr2=0x7ff9400fdb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 -- 172.21.15.5:0/855567526 shutdown_connections 2023-07-21T19:20:27.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.292+0000 7ff9465ff700 1 -- 172.21.15.5:0/855567526 wait complete. 2023-07-21T19:20:27.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.294+0000 7ff9465ff700 1 Processor -- start 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.294+0000 7ff9465ff700 1 -- start start 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.294+0000 7ff9465ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.294+0000 7ff9465ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9400731c0 0x7ff940198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff9465ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff94010c3b0 0x7ff94019c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff9465ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019cf30 con 0x7ff9400728b0 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff9465ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019d0a0 con 0x7ff94010c3b0 2023-07-21T19:20:27.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff9465ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94019d260 con 0x7ff9400731c0 2023-07-21T19:20:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34796/0 (socket says 172.21.15.5:34796) 2023-07-21T19:20:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 -- 172.21.15.5:0/3839363485 learned_addr learned my addr 172.21.15.5:0/3839363485 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff944b9c700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff94010c3b0 0x7ff94019c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9400731c0 0x7ff940198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 -- 172.21.15.5:0/3839363485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9400731c0 msgr2=0x7ff940198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9400731c0 0x7ff940198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 -- 172.21.15.5:0/3839363485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff94010c3b0 msgr2=0x7ff94019c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff94010c3b0 0x7ff94019c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.295+0000 7ff93ffff700 1 -- 172.21.15.5:0/3839363485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300095f0 con 0x7ff9400728b0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff93ffff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff93d7fa700 1 -- 172.21.15.5:0/3839363485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93001d070 con 0x7ff9400728b0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94019d4e0 con 0x7ff9400728b0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff93d7fa700 1 -- 172.21.15.5:0/3839363485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff930022470 con 0x7ff9400728b0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff93d7fa700 1 -- 172.21.15.5:0/3839363485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930017650 con 0x7ff9400728b0 2023-07-21T19:20:27.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.296+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a4d80 con 0x7ff9400728b0 2023-07-21T19:20:27.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.300+0000 7ff93d7fa700 1 -- 172.21.15.5:0/3839363485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff9300225e0 con 0x7ff9400728b0 2023-07-21T19:20:27.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.301+0000 7ff93d7fa700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 0x7ff9280a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.302+0000 7ff93d7fa700 1 -- 172.21.15.5:0/3839363485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff9300ce110 con 0x7ff9400728b0 2023-07-21T19:20:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.302+0000 7ff959fa8b80 1 Processor -- start 2023-07-21T19:20:27.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.302+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 0x7ff9280a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.302+0000 7ff959fa8b80 1 -- start start 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.302+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d1390d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d138b580 0x5567d138b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d1290c50 con 0x5567d11e76b0 2023-07-21T19:20:27.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d12e39e0 con 0x5567d138b580 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d12dce70 con 0x5567d13912d0 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 0x7ff9280a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff940199560 tx=0x7ff934009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff944b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d138b580 0x5567d138b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:50344/0 (socket says 172.21.15.5:50344) 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d1390d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 -- 172.21.15.5:0/2948785169 learned_addr learned my addr 172.21.15.5:0/2948785169 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 -- 172.21.15.5:0/2948785169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d138b580 msgr2=0x5567d138b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d138b580 0x5567d138b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 -- 172.21.15.5:0/2948785169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 msgr2=0x5567d1390d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d1390d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93f7fe700 1 -- 172.21.15.5:0/2948785169 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d110ad80 con 0x5567d13912d0 2023-07-21T19:20:27.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.303+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d1390d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.304+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff934011d20 tx=0x7ff934014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=99ad7bda328905d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.304+0000 7ff9267fc700 1 -- 172.21.15.5:0/2948785169 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93400eb20 con 0x5567d13912d0 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.304+0000 7ff9267fc700 1 -- 172.21.15.5:0/2948785169 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff934021070 con 0x5567d13912d0 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.304+0000 7ff9267fc700 1 -- 172.21.15.5:0/2948785169 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9340253d0 con 0x5567d13912d0 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2948785169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 msgr2=0x5567d138b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7ff934011d20 tx=0x7ff934014000 comp rx=0 tx=0).stop 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2948785169 shutdown_connections 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d138b580 0x5567d138b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d1390d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2948785169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d13912d0 0x5567d138b040 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2948785169 >> 172.21.15.5:0/2948785169 conn(0x5567d1380be0 msgr2=0x5567d1385f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2948785169 shutdown_connections 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.305+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2948785169 wait complete. 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.306+0000 7ff959fa8b80 1 -- start start 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.306+0000 7ff959fa8b80 1 Processor -- start 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1394960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d13950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d127ca60 con 0x5567d13912d0 2023-07-21T19:20:27.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d127c760 con 0x5567d11e76b0 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d11608c0 con 0x5567d138b580 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff959fa8b80 10 client.0 _ll_register_callbacks cb 0x5567d127a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1394960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53318/0 (socket says 172.21.15.5:53318) 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 -- 172.21.15.5:0/2421730198 learned_addr learned my addr 172.21.15.5:0/2421730198 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:27.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff944b9c700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d13950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 -- 172.21.15.5:0/2421730198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 msgr2=0x5567d1394960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1394960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 -- 172.21.15.5:0/2421730198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 msgr2=0x5567d13950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d13950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.307+0000 7ff93ffff700 1 -- 172.21.15.5:0/2421730198 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9401a4d80 con 0x5567d11e76b0 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff93000f740 tx=0x7ff93009ae20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=864d862efe152be4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff91f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93009a900 con 0x5567d11e76b0 2023-07-21T19:20:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff93009b210 con 0x5567d11e76b0 2023-07-21T19:20:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5567d1236490 con 0x5567d11e76b0 2023-07-21T19:20:27.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.308+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93009f030 con 0x5567d11e76b0 2023-07-21T19:20:27.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 10 client.27944 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7ff934017070 con 0x5567d11e76b0 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff9300c9080 con 0x5567d11e76b0 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff93000fac0 con 0x5567d11e76b0 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 10 client.27944 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 10 client.27944 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:27.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 20 client.27944 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13a4800 0x5567d13a6be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 4 client.27944 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.309+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5567d12d5e50 con 0x5567d13a4800 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.310+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13a4800 0x5567d13a6be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.310+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13a4800 0x5567d13a6be0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8af9ee5b913a130c server_cookie=74631efc5d83ea1e in_seq=0 out_seq=0 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.310+0000 7ff91f7fe700 10 client.27944 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:27.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2421730198 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5567d12d5e50 con 0x5567d13a4800 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff91f7fe700 10 client.27944 handle_command_reply: tid=0 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 client.27944 shutdown 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 msgr2=0x5567d1397670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff93000f740 tx=0x7ff93009ae20 comp rx=0 tx=0).stop 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 shutdown_connections 2023-07-21T19:20:27.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13a4800 0x5567d13a6be0 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d13950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d1397670 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2421730198 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1394960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff91f7fe700 0 client.27944 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 >> 172.21.15.5:0/2421730198 conn(0x5567d1380be0 msgr2=0x5567d13854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 shutdown_connections 2023-07-21T19:20:27.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.311+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2421730198 wait complete. 2023-07-21T19:20:27.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.312+0000 7ff959fa8b80 20 client.27944 trim_cache size 0 max 16384 2023-07-21T19:20:27.319 INFO:teuthology.orchestra.run.smithi005.stderr:49400000 of 50000000 tuples (98%) done (elapsed 744.89 s, remaining 9.05 s) 2023-07-21T19:20:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.332+0000 7ff959fa8b80 1 Processor -- start 2023-07-21T19:20:27.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.332+0000 7ff959fa8b80 1 -- start start 2023-07-21T19:20:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d139bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d13912d0 0x5567d1385c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d13a0b20 con 0x5567d11e76b0 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d138f6e0 con 0x5567d13912d0 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d137db40 con 0x5567d138b580 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34814/0 (socket says 172.21.15.5:34814) 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 -- 172.21.15.5:0/419085265 learned_addr learned my addr 172.21.15.5:0/419085265 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:27.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff944b9c700 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d13912d0 0x5567d1385c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 -- 172.21.15.5:0/419085265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 msgr2=0x5567d139bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d139bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d139bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 -- 172.21.15.5:0/419085265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d13912d0 msgr2=0x5567d1385c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d13912d0 0x5567d1385c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.333+0000 7ff93ffff700 1 -- 172.21.15.5:0/419085265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5567d10f3c00 con 0x5567d11e76b0 2023-07-21T19:20:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.334+0000 7ff93ffff700 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9300a30a0 tx=0x7ff9300997c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fbd3eecc40f787 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.334+0000 7ff926ffd700 1 -- 172.21.15.5:0/419085265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9300a3140 con 0x5567d11e76b0 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.334+0000 7ff926ffd700 1 -- 172.21.15.5:0/419085265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5567d1236490 con 0x5567d11e76b0 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.334+0000 7ff926ffd700 1 -- 172.21.15.5:0/419085265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93009a700 con 0x5567d11e76b0 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 -- 172.21.15.5:0/419085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 msgr2=0x5567d138b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7ff9300a30a0 tx=0x7ff9300997c0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 -- 172.21.15.5:0/419085265 shutdown_connections 2023-07-21T19:20:27.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d13912d0 0x5567d1385c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d11e76b0 0x5567d138b040 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/419085265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d139bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 -- 172.21.15.5:0/419085265 >> 172.21.15.5:0/419085265 conn(0x5567d1380be0 msgr2=0x5567d138eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 -- 172.21.15.5:0/419085265 shutdown_connections 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.335+0000 7ff959fa8b80 1 -- 172.21.15.5:0/419085265 wait complete. 2023-07-21T19:20:27.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.336+0000 7ff959fa8b80 1 -- start start 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.336+0000 7ff959fa8b80 1 Processor -- start 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d139ba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1399d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d139a7b0 con 0x5567d13912d0 2023-07-21T19:20:27.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d139a920 con 0x5567d11e76b0 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5567d139aae0 con 0x5567d138b580 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff959fa8b80 10 client.0 _ll_register_callbacks cb 0x7ff93009afe0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff944b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34820/0 (socket says 172.21.15.5:34820) 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff944b9c700 1 -- 172.21.15.5:0/2954631136 learned_addr learned my addr 172.21.15.5:0/2954631136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:27.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d139ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.337+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1399d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 -- 172.21.15.5:0/2954631136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 msgr2=0x5567d1399d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1399d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 -- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 msgr2=0x5567d139ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d139ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 -- 172.21.15.5:0/2954631136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff934017070 con 0x5567d13912d0 2023-07-21T19:20:27.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff93f7fe700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1399d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff944b9c700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff938007c30 tx=0x7ff938007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8dfe809e4a12e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff91f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93801c070 con 0x5567d13912d0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff934021070 con 0x5567d13912d0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff938011c20 con 0x5567d13912d0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff93800e040 con 0x5567d13912d0 2023-07-21T19:20:27.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.338+0000 7ff959fa8b80 10 client.18300 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x5567d11311f0 con 0x5567d13912d0 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff93802d080 con 0x5567d13912d0 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff938027030 con 0x5567d13912d0 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 10 client.18300 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 10 client.18300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:27.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 20 client.18300 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13f9ad0 0x5567d13fbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 4 client.18300 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:20:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.339+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5567d1297ff0 con 0x5567d13f9ad0 2023-07-21T19:20:27.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.340+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13f9ad0 0x5567d13fbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.340+0000 7ff91f7fe700 10 client.18300 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:27.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.340+0000 7ff93ffff700 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13f9ad0 0x5567d13fbee0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c6977e82ed90181 server_cookie=e112f440c27b5526 in_seq=0 out_seq=0 2023-07-21T19:20:27.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.358+0000 7ff91f7fe700 1 -- 172.21.15.5:0/2954631136 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5567d1297ff0 con 0x5567d13f9ad0 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.358+0000 7ff91f7fe700 10 client.18300 handle_command_reply: tid=0 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.358+0000 7ff959fa8b80 1 client.18300 shutdown 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.358+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 msgr2=0x5567d139a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.358+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7ff938007c30 tx=0x7ff938007f40 comp rx=0 tx=0).stop 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 shutdown_connections 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5567d13f9ad0 0x5567d13fbee0 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5567d13912d0 0x5567d139a270 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5567d11e76b0 0x5567d139ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 --2- 172.21.15.5:0/2954631136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5567d138b580 0x5567d1399d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 >> 172.21.15.5:0/2954631136 conn(0x5567d1380be0 msgr2=0x5567d1390380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff91f7fe700 0 client.18300 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 shutdown_connections 2023-07-21T19:20:27.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 1 -- 172.21.15.5:0/2954631136 wait complete. 2023-07-21T19:20:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.359+0000 7ff959fa8b80 20 client.18300 trim_cache size 0 max 16384 2023-07-21T19:20:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:20:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:20:27.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:20:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:20:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 msgr2=0x7ff9280a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 0x7ff9280a3770 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff940199560 tx=0x7ff934009260 comp rx=0 tx=0).stop 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 msgr2=0x7ff940197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7ff930000c00 tx=0x7ff930004730 comp rx=0 tx=0).stop 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 shutdown_connections 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff94010c3b0 0x7ff94019c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff9280a12c0 0x7ff9280a3770 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff9400728b0 0x7ff940197d40 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 --2- 172.21.15.5:0/3839363485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff9400731c0 0x7ff940198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 >> 172.21.15.5:0/3839363485 conn(0x7ff9400fb6f0 msgr2=0x7ff9400fbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 shutdown_connections 2023-07-21T19:20:27.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:27.360+0000 7ff9465ff700 1 -- 172.21.15.5:0/3839363485 wait complete. 2023-07-21T19:20:27.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:27 smithi101 ceph-mon[128136]: pgmap v402: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:20:27.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:27 smithi005 ceph-mon[126460]: pgmap v402: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:20:27.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:27 smithi156 ceph-mon[128316]: pgmap v402: 97 pgs: 97 active+clean; 9.5 GiB data, 4.2 GiB used, 1.0 TiB / 1.0 TiB avail; 194 KiB/s rd, 21 MiB/s wr, 152 op/s 2023-07-21T19:20:28.697 INFO:teuthology.orchestra.run.smithi005.stderr:49500000 of 50000000 tuples (99%) done (elapsed 746.27 s, remaining 7.54 s) 2023-07-21T19:20:29.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:29 smithi101 ceph-mon[128136]: pgmap v403: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 170 op/s 2023-07-21T19:20:29.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:29 smithi005 ceph-mon[126460]: pgmap v403: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 170 op/s 2023-07-21T19:20:29.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:29 smithi156 ceph-mon[128316]: pgmap v403: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 256 KiB/s rd, 23 MiB/s wr, 170 op/s 2023-07-21T19:20:30.093 INFO:teuthology.orchestra.run.smithi005.stderr:49600000 of 50000000 tuples (99%) done (elapsed 747.67 s, remaining 6.03 s) 2023-07-21T19:20:31.436 INFO:teuthology.orchestra.run.smithi005.stderr:49700000 of 50000000 tuples (99%) done (elapsed 749.01 s, remaining 4.52 s) 2023-07-21T19:20:31.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:31 smithi101 ceph-mon[128136]: pgmap v404: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 13 MiB/s wr, 42 op/s 2023-07-21T19:20:31.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:31 smithi005 ceph-mon[126460]: pgmap v404: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 13 MiB/s wr, 42 op/s 2023-07-21T19:20:31.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:31 smithi156 ceph-mon[128316]: pgmap v404: 97 pgs: 97 active+clean; 9.5 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 94 KiB/s rd, 13 MiB/s wr, 42 op/s 2023-07-21T19:20:32.765 INFO:teuthology.orchestra.run.smithi005.stderr:49800000 of 50000000 tuples (99%) done (elapsed 750.34 s, remaining 3.01 s) 2023-07-21T19:20:33.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:33 smithi101 ceph-mon[128136]: pgmap v405: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 23 MiB/s wr, 75 op/s 2023-07-21T19:20:33.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:33 smithi005 ceph-mon[126460]: pgmap v405: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 23 MiB/s wr, 75 op/s 2023-07-21T19:20:33.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:33 smithi156 ceph-mon[128316]: pgmap v405: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 101 KiB/s rd, 23 MiB/s wr, 75 op/s 2023-07-21T19:20:34.090 INFO:teuthology.orchestra.run.smithi005.stderr:49900000 of 50000000 tuples (99%) done (elapsed 751.66 s, remaining 1.51 s) 2023-07-21T19:20:35.516 INFO:teuthology.orchestra.run.smithi005.stderr:50000000 of 50000000 tuples (100%) done (elapsed 753.09 s, remaining 0.00 s) 2023-07-21T19:20:35.608 INFO:teuthology.orchestra.run.smithi005.stderr:vacuum... 2023-07-21T19:20:35.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:35 smithi101 ceph-mon[128136]: pgmap v406: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 13 MiB/s wr, 62 op/s 2023-07-21T19:20:35.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:35 smithi005 ceph-mon[126460]: pgmap v406: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 13 MiB/s wr, 62 op/s 2023-07-21T19:20:35.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:35 smithi156 ceph-mon[128316]: pgmap v406: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 127 KiB/s rd, 13 MiB/s wr, 62 op/s 2023-07-21T19:20:37.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:37 smithi101 ceph-mon[128136]: pgmap v407: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 58 op/s 2023-07-21T19:20:37.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:37 smithi005 ceph-mon[126460]: pgmap v407: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 58 op/s 2023-07-21T19:20:37.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:37 smithi156 ceph-mon[128316]: pgmap v407: 97 pgs: 97 active+clean; 9.6 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 113 KiB/s rd, 13 MiB/s wr, 58 op/s 2023-07-21T19:20:39.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:39 smithi101 ceph-mon[128136]: pgmap v408: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 24 MiB/s wr, 89 op/s 2023-07-21T19:20:39.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:39 smithi005 ceph-mon[126460]: pgmap v408: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 24 MiB/s wr, 89 op/s 2023-07-21T19:20:39.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:39 smithi156 ceph-mon[128316]: pgmap v408: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 158 KiB/s rd, 24 MiB/s wr, 89 op/s 2023-07-21T19:20:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 -- 172.21.15.5:0/3077063521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe900105940 msgr2=0x7fe900105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 --2- 172.21.15.5:0/3077063521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe900105940 0x7fe900105d10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc009990 tx=0x7fe8fc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 -- 172.21.15.5:0/3077063521 shutdown_connections 2023-07-21T19:20:40.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 --2- 172.21.15.5:0/3077063521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9001002e0 0x7fe90010c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 --2- 172.21.15.5:0/3077063521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe900105940 0x7fe900105d10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 --2- 172.21.15.5:0/3077063521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9000ff930 0x7fe9000ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 -- 172.21.15.5:0/3077063521 >> 172.21.15.5:0/3077063521 conn(0x7fe9000fb450 msgr2=0x7fe9000fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 -- 172.21.15.5:0/3077063521 shutdown_connections 2023-07-21T19:20:40.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.534+0000 7fe90739d700 1 -- 172.21.15.5:0/3077063521 wait complete. 2023-07-21T19:20:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.535+0000 7fe90739d700 1 Processor -- start 2023-07-21T19:20:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.535+0000 7fe90739d700 1 -- start start 2023-07-21T19:20:40.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9001002e0 0x7fe900072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe900105940 0x7fe9001a3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe900073580 con 0x7fe900105940 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001a4000 con 0x7fe9000ff930 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90739d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9001a4140 con 0x7fe9001002e0 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48250/0 (socket says 172.21.15.5:48250) 2023-07-21T19:20:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90639b700 1 -- 172.21.15.5:0/684942354 learned_addr learned my addr 172.21.15.5:0/684942354 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:40.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe906b9c700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe900105940 0x7fe9001a3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90639b700 1 -- 172.21.15.5:0/684942354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9001002e0 msgr2=0x7fe900072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.536+0000 7fe90639b700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9001002e0 0x7fe900072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe90639b700 1 -- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe900105940 msgr2=0x7fe9001a3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe90639b700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe900105940 0x7fe9001a3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe90639b700 1 -- 172.21.15.5:0/684942354 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8fc0095f0 con 0x7fe9000ff930 2023-07-21T19:20:40.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe90639b700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0098e0 tx=0x7fe8fc004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe8f77fe700 1 -- 172.21.15.5:0/684942354 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc01d070 con 0x7fe9000ff930 2023-07-21T19:20:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.537+0000 7fe8f77fe700 1 -- 172.21.15.5:0/684942354 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8fc022470 con 0x7fe9000ff930 2023-07-21T19:20:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.538+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9001a4340 con 0x7fe9000ff930 2023-07-21T19:20:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.538+0000 7fe8f77fe700 1 -- 172.21.15.5:0/684942354 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc017650 con 0x7fe9000ff930 2023-07-21T19:20:40.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.538+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9001a4760 con 0x7fe9000ff930 2023-07-21T19:20:40.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.541+0000 7fe8f77fe700 1 -- 172.21.15.5:0/684942354 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe8fc0225e0 con 0x7fe9000ff930 2023-07-21T19:20:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.543+0000 7fe91be4fb80 1 Processor -- start 2023-07-21T19:20:40.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.543+0000 7fe91be4fb80 1 -- start start 2023-07-21T19:20:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.543+0000 7fe8f77fe700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 0x7fe8ec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe8f77fe700 1 -- 172.21.15.5:0/684942354 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe8fc0ce130 con 0x7fe9000ff930 2023-07-21T19:20:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2c21a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c2700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c1c7c50 con 0x56330c2c88c0 2023-07-21T19:20:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c21a9e0 con 0x56330c11e6b0 2023-07-21T19:20:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c213e70 con 0x56330c2c8470 2023-07-21T19:20:40.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe906b9c700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 0x7fe8ec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48264/0 (socket says 172.21.15.5:48264) 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe90639b700 1 -- 172.21.15.5:0/789659820 learned_addr learned my addr 172.21.15.5:0/789659820 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe906b9c700 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.544+0000 7fe905b9a700 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2c21a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 -- 172.21.15.5:0/789659820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 msgr2=0x56330c2c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2c7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 -- 172.21.15.5:0/789659820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 msgr2=0x56330c2c2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 -- 172.21.15.5:0/789659820 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330c041d80 con 0x56330c2c8470 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe906b9c700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 0x7fe8ec0a3770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009ec0 tx=0x7fe8f8016450 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe905b9a700 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2c21a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009ba0 tx=0x7fe8f0009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0f7073a901ecfb7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe8f4bfe700 1 -- 172.21.15.5:0/789659820 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004230 con 0x56330c2c8470 2023-07-21T19:20:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe8f4bfe700 1 -- 172.21.15.5:0/789659820 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8f000ed90 con 0x56330c2c8470 2023-07-21T19:20:40.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.545+0000 7fe8f4bfe700 1 -- 172.21.15.5:0/789659820 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f0004e70 con 0x56330c2c8470 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 -- 172.21.15.5:0/789659820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 msgr2=0x56330c2c21a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2c21a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe8f0009ba0 tx=0x7fe8f0009eb0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 -- 172.21.15.5:0/789659820 shutdown_connections 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/789659820 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2c21a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 -- 172.21.15.5:0/789659820 >> 172.21.15.5:0/789659820 conn(0x56330c2b7d40 msgr2=0x56330c2bd340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.546+0000 7fe91be4fb80 1 -- 172.21.15.5:0/789659820 shutdown_connections 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.547+0000 7fe91be4fb80 1 -- 172.21.15.5:0/789659820 wait complete. 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.547+0000 7fe91be4fb80 1 -- start start 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.547+0000 7fe91be4fb80 1 Processor -- start 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 0x56330c2cc0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c1b3a60 con 0x56330c11e6b0 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c1b3760 con 0x56330c2c88c0 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c0978c0 con 0x56330c2c8470 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe91be4fb80 10 client.0 _ll_register_callbacks cb 0x56330c1b1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe90639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe90639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56566/0 (socket says 172.21.15.5:56566) 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe90639b700 1 -- 172.21.15.5:0/1965213921 learned_addr learned my addr 172.21.15.5:0/1965213921 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:40.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.548+0000 7fe906b9c700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 0x56330c2cc0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe905b9a700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 -- 172.21.15.5:0/1965213921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 msgr2=0x56330c2cba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2cba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 -- 172.21.15.5:0/1965213921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 msgr2=0x56330c2ce780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 -- 172.21.15.5:0/1965213921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8f0009880 con 0x56330c2c88c0 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe90639b700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:40.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe906b9c700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 0x56330c2cc0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56330c2c8cd0 tx=0x7fe8f8004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f573e21a4036b912 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.549+0000 7fe8e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800e040 con 0x56330c2c88c0 2023-07-21T19:20:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe8f8021040 con 0x56330c2c88c0 2023-07-21T19:20:40.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56330c16d490 con 0x56330c2c88c0 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8f801aeb0 con 0x56330c2c88c0 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe91be4fb80 10 client.27962 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56330c02ac00 con 0x56330c2c88c0 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56330c16d490 con 0x56330c2c88c0 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.550+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8f8025070 con 0x56330c2c88c0 2023-07-21T19:20:40.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 10 client.27962 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 10 client.27962 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 20 client.27962 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c2dbdb0 0x56330c2de1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 4 client.27962 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56330c20ce50 con 0x56330c2dbdb0 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe90639b700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c2dbdb0 0x56330c2de1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe8e9ffb700 10 client.27962 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.551+0000 7fe90639b700 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c2dbdb0 0x56330c2de1c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5d54bc295702b9c server_cookie=8d4b478f5d637a75 in_seq=0 out_seq=0 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/1965213921 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56330c20ce50 con 0x56330c2dbdb0 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe8e9ffb700 10 client.27962 handle_command_reply: tid=0 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 client.27962 shutdown 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 msgr2=0x56330c2cc0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 0x56330c2cc0a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x56330c2c8cd0 tx=0x7fe8f8004b40 comp rx=0 tx=0).stop 2023-07-21T19:20:40.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 shutdown_connections 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c88c0 0x56330c2cc0a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.552+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c2dbdb0 0x56330c2de1c0 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c11e6b0 0x56330c2ce780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1965213921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2cba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe8e9ffb700 0 client.27962 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 >> 172.21.15.5:0/1965213921 conn(0x56330c2b7d40 msgr2=0x56330c2bc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 shutdown_connections 2023-07-21T19:20:40.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1965213921 wait complete. 2023-07-21T19:20:40.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.553+0000 7fe91be4fb80 20 client.27962 trim_cache size 0 max 16384 2023-07-21T19:20:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.571+0000 7fe91be4fb80 1 Processor -- start 2023-07-21T19:20:40.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.571+0000 7fe91be4fb80 1 -- start start 2023-07-21T19:20:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 0x56330c2ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c7240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c2c3200 con 0x56330c2c88c0 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c2c2fc0 con 0x56330c2c8470 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c2ccfe0 con 0x56330c11e6b0 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe905b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 0x56330c2ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe90639b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe906b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe905b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 0x56330c2ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48268/0 (socket says 172.21.15.5:48268) 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe905b9a700 1 -- 172.21.15.5:0/1487247551 learned_addr learned my addr 172.21.15.5:0/1487247551 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.572+0000 7fe90639b700 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39350/0 (socket says 172.21.15.5:39350) 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 -- 172.21.15.5:0/1487247551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 msgr2=0x56330c2ce320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 0x56330c2ce320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 -- 172.21.15.5:0/1487247551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 msgr2=0x56330c2c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 -- 172.21.15.5:0/1487247551 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56330c0681f0 con 0x56330c11e6b0 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe90639b700 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc022b60 tx=0x7fe8fc009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2e7ba9bab7c797 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe8f53ff700 1 -- 172.21.15.5:0/1487247551 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc09abf0 con 0x56330c11e6b0 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe8f53ff700 1 -- 172.21.15.5:0/1487247551 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8fc0b2070 con 0x56330c11e6b0 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.573+0000 7fe8f53ff700 1 -- 172.21.15.5:0/1487247551 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0a3650 con 0x56330c11e6b0 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1487247551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 msgr2=0x56330c2d2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc022b60 tx=0x7fe8fc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1487247551 shutdown_connections 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c11e6b0 0x56330c2d2d90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/1487247551 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c2c8470 0x56330c2ce320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1487247551 >> 172.21.15.5:0/1487247551 conn(0x56330c2b7d40 msgr2=0x56330c2c6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1487247551 shutdown_connections 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.574+0000 7fe91be4fb80 1 -- 172.21.15.5:0/1487247551 wait complete. 2023-07-21T19:20:40.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 -- start start 2023-07-21T19:20:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 Processor -- start 2023-07-21T19:20:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c32ece0 con 0x56330c2c88c0 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c32ee50 con 0x56330c11e6b0 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330c32f010 con 0x56330c2c8470 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.575+0000 7fe91be4fb80 10 client.0 _ll_register_callbacks cb 0x7fe8f80167f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe905b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48280/0 (socket says 172.21.15.5:48280) 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 -- 172.21.15.5:0/957602622 learned_addr learned my addr 172.21.15.5:0/957602622 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe905b9a700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2d12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39358/0 (socket says 172.21.15.5:39358) 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe906b9c700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 -- 172.21.15.5:0/957602622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 msgr2=0x56330c2d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 -- 172.21.15.5:0/957602622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 msgr2=0x56330c2d17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:40.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2d17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 -- 172.21.15.5:0/957602622 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9001a4760 con 0x56330c11e6b0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe90639b700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0a3120 tx=0x7fe8fc009300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3e731fb57681ce52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.576+0000 7fe8e9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0228c0 con 0x56330c11e6b0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe8fc0a3650 con 0x56330c11e6b0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe8f000ed90 con 0x56330c11e6b0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe8fc09f2c0 con 0x56330c11e6b0 2023-07-21T19:20:40.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 10 client.27968 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x56330c040250 con 0x56330c11e6b0 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe8f000ed90 con 0x56330c11e6b0 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fe8fc0cb050 con 0x56330c11e6b0 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 10 client.27968 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 10 client.27968 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 20 client.27968 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:40.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c3302f0 0x56330c332700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 4 client.27968 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:20:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.577+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56330c1ceff0 con 0x56330c3302f0 2023-07-21T19:20:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.578+0000 7fe905b9a700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c3302f0 0x56330c332700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.578+0000 7fe905b9a700 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c3302f0 0x56330c332700 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=649ccd155246b684 server_cookie=cdbea5b087e89a0d in_seq=0 out_seq=0 2023-07-21T19:20:40.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.578+0000 7fe8e9ffb700 10 client.27968 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe8e9ffb700 1 -- 172.21.15.5:0/957602622 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56330c1ceff0 con 0x56330c3302f0 2023-07-21T19:20:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe8e9ffb700 10 client.27968 handle_command_reply: tid=0 2023-07-21T19:20:40.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 client.27968 shutdown 2023-07-21T19:20:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 msgr2=0x56330c2d0d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0a3120 tx=0x7fe8fc009300 comp rx=0 tx=0).stop 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 shutdown_connections 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56330c3302f0 0x56330c332700 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330c2c88c0 0x56330c2d17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330c11e6b0 0x56330c2d0d70 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe8e9ffb700 0 client.27968 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 --2- 172.21.15.5:0/957602622 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330c2c8470 0x56330c2d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.596+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 >> 172.21.15.5:0/957602622 conn(0x56330c2b7d40 msgr2=0x56330c2c73c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.597+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 shutdown_connections 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.597+0000 7fe91be4fb80 1 -- 172.21.15.5:0/957602622 wait complete. 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.597+0000 7fe91be4fb80 20 client.27968 trim_cache size 0 max 16384 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:20:40.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:20:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:20:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.597+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 msgr2=0x7fe8ec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.597+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 0x7fe8ec0a3770 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fe8f8009ec0 tx=0x7fe8f8016450 comp rx=0 tx=0).stop 2023-07-21T19:20:40.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 msgr2=0x7fe900072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc0098e0 tx=0x7fe8fc004730 comp rx=0 tx=0).stop 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 shutdown_connections 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe8ec0a12c0 0x7fe8ec0a3770 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe900105940 0x7fe9001a3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9000ff930 0x7fe900072810 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 --2- 172.21.15.5:0/684942354 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9001002e0 0x7fe900072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:40.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 >> 172.21.15.5:0/684942354 conn(0x7fe9000fb450 msgr2=0x7fe9000fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 shutdown_connections 2023-07-21T19:20:40.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:40.598+0000 7fe90739d700 1 -- 172.21.15.5:0/684942354 wait complete. 2023-07-21T19:20:41.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:41 smithi101 ceph-mon[128136]: pgmap v409: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 71 op/s 2023-07-21T19:20:41.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:41 smithi005 ceph-mon[126460]: pgmap v409: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 71 op/s 2023-07-21T19:20:41.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:41 smithi156 ceph-mon[128316]: pgmap v409: 97 pgs: 97 active+clean; 9.7 GiB data, 4.3 GiB used, 1.0 TiB / 1.0 TiB avail; 96 KiB/s rd, 22 MiB/s wr, 71 op/s 2023-07-21T19:20:43.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:43 smithi101 ceph-mon[128136]: pgmap v410: 97 pgs: 97 active+clean; 9.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:20:43.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:43 smithi005 ceph-mon[126460]: pgmap v410: 97 pgs: 97 active+clean; 9.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:20:43.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:43 smithi156 ceph-mon[128316]: pgmap v410: 97 pgs: 97 active+clean; 9.8 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 32 MiB/s wr, 82 op/s 2023-07-21T19:20:45.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:45 smithi101 ceph-mon[128136]: pgmap v411: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 23 MiB/s wr, 58 op/s 2023-07-21T19:20:45.837 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:20:45.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:45 smithi005 ceph-mon[126460]: pgmap v411: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 23 MiB/s wr, 58 op/s 2023-07-21T19:20:45.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:45 smithi156 ceph-mon[128316]: pgmap v411: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 143 KiB/s rd, 23 MiB/s wr, 58 op/s 2023-07-21T19:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.086+0000 7f39e39ff700 1 -- 172.21.15.5:0/218523379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc0ffae0 msgr2=0x7f39dc0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.086+0000 7f39e39ff700 1 --2- 172.21.15.5:0/218523379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc0ffae0 0x7f39dc0fff50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f39cc009990 tx=0x7f39cc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 -- 172.21.15.5:0/218523379 shutdown_connections 2023-07-21T19:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 --2- 172.21.15.5:0/218523379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc100490 0x7f39dc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 --2- 172.21.15.5:0/218523379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 0x7f39dc105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 --2- 172.21.15.5:0/218523379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc0ffae0 0x7f39dc0fff50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 -- 172.21.15.5:0/218523379 >> 172.21.15.5:0/218523379 conn(0x7f39dc0fb5e0 msgr2=0x7f39dc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 -- 172.21.15.5:0/218523379 shutdown_connections 2023-07-21T19:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.087+0000 7f39e39ff700 1 -- 172.21.15.5:0/218523379 wait complete. 2023-07-21T19:20:46.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.089+0000 7f39e39ff700 1 Processor -- start 2023-07-21T19:20:46.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.089+0000 7f39e39ff700 1 -- start start 2023-07-21T19:20:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 0x7f39dc19dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 0x7f39dc197b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1981a0 con 0x7f39dc0ffae0 2023-07-21T19:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc198310 con 0x7f39dc105af0 2023-07-21T19:20:46.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e39ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39dc1984d0 con 0x7f39dc100490 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 0x7f39dc19dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39378/0 (socket says 172.21.15.5:39378) 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e179b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 0x7f39dc19dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56606/0 (socket says 172.21.15.5:56606) 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.090+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2949725927 learned_addr learned my addr 172.21.15.5:0/2949725927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e1f9c700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 0x7f39dc197b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2949725927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 msgr2=0x7f39dc197b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 0x7f39dc197b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 msgr2=0x7f39dc19dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 0x7f39dc19dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2949725927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0095f0 con 0x7f39dc100490 2023-07-21T19:20:46.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.091+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39cc003920 tx=0x7f39cc003950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.092+0000 7f39d27fc700 1 -- 172.21.15.5:0/2949725927 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc01d070 con 0x7f39dc100490 2023-07-21T19:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.092+0000 7f39d27fc700 1 -- 172.21.15.5:0/2949725927 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39cc00bdb0 con 0x7f39dc100490 2023-07-21T19:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.092+0000 7f39d27fc700 1 -- 172.21.15.5:0/2949725927 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc022da0 con 0x7f39dc100490 2023-07-21T19:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.092+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39dc198750 con 0x7f39dc100490 2023-07-21T19:20:46.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.092+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc1a4ab0 con 0x7f39dc100490 2023-07-21T19:20:46.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.098+0000 7f39d27fc700 1 -- 172.21.15.5:0/2949725927 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f39cc022570 con 0x7f39dc100490 2023-07-21T19:20:46.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.099+0000 7f39d27fc700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 0x7f39c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.099+0000 7f39f73ffb80 1 Processor -- start 2023-07-21T19:20:46.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39d27fc700 1 -- 172.21.15.5:0/2949725927 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f39cc0ce220 con 0x7f39dc100490 2023-07-21T19:20:46.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 -- start start 2023-07-21T19:20:46.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39e179b700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 0x7f39c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097d2d0 0x55d0f097d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 0x55d0f098aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f0883cb0 con 0x55d0f097d2d0 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f08d6a40 con 0x55d0f097dbe0 2023-07-21T19:20:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.100+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f08cfed0 con 0x55d0f0988ad0 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48300/0 (socket says 172.21.15.5:48300) 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2617432789 learned_addr learned my addr 172.21.15.5:0/2617432789 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e179b700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097d2d0 0x55d0f097d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e1f9c700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 0x55d0f098aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2617432789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 msgr2=0x55d0f098aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 0x55d0f098aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2617432789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097d2d0 msgr2=0x55d0f097d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097d2d0 0x55d0f097d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 -- 172.21.15.5:0/2617432789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f06fdde0 con 0x55d0f097dbe0 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e1f9c700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 0x55d0f098aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39cc097df0 tx=0x7f39cc095f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1d4ad5dac3114201 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.101+0000 7f39e179b700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 0x7f39c80a37f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f39d80134b0 tx=0x7f39d8013870 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.102+0000 7f39c77fe700 1 -- 172.21.15.5:0/2617432789 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc09b380 con 0x55d0f097dbe0 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.102+0000 7f39c77fe700 1 -- 172.21.15.5:0/2617432789 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39cc09f3b0 con 0x55d0f097dbe0 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.102+0000 7f39c77fe700 1 -- 172.21.15.5:0/2617432789 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39cc09b7e0 con 0x55d0f097dbe0 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2617432789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 msgr2=0x55d0f0988590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f39cc097df0 tx=0x7f39cc095f90 comp rx=0 tx=0).stop 2023-07-21T19:20:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2617432789 shutdown_connections 2023-07-21T19:20:46.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f0988ad0 0x55d0f098aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097d2d0 0x55d0f097d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2617432789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097dbe0 0x55d0f0988590 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2617432789 >> 172.21.15.5:0/2617432789 conn(0x55d0f0973da0 msgr2=0x55d0f09796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2617432789 shutdown_connections 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.103+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2617432789 wait complete. 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.104+0000 7f39f73ffb80 1 -- start start 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.104+0000 7f39f73ffb80 1 Processor -- start 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 0x55d0f0988ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 0x55d0f09893e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 0x55d0f0989920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f086fac0 con 0x55d0f097dbe0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f086f7c0 con 0x55d0f0988ad0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f0753920 con 0x55d0f097d2d0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39f73ffb80 10 client.0 _ll_register_callbacks cb 0x55d0f086d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 0x55d0f0988ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 0x55d0f09893e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e1f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 0x55d0f0989920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 0x55d0f09893e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56630/0 (socket says 172.21.15.5:56630) 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e0f9a700 1 -- 172.21.15.5:0/1646055383 learned_addr learned my addr 172.21.15.5:0/1646055383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 -- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 msgr2=0x55d0f0989920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 0x55d0f0989920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 -- 172.21.15.5:0/1646055383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 msgr2=0x55d0f09893e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 0x55d0f09893e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.105+0000 7f39e179b700 1 -- 172.21.15.5:0/1646055383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39dc1a4ab0 con 0x55d0f097d2d0 2023-07-21T19:20:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39e1f9c700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 0x55d0f0989920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39e179b700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 0x55d0f0988ea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005940 tx=0x7f39d80047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1690edfd2faddc94 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004a30 con 0x55d0f097d2d0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0f08294f0 con 0x55d0f097d2d0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f39d8012280 con 0x55d0f097d2d0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d8004b90 con 0x55d0f097d2d0 2023-07-21T19:20:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39f73ffb80 10 client.27907 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.106+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d0f06e6c60 con 0x55d0f097d2d0 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0f08294f0 con 0x55d0f097d2d0 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39d80204c0 con 0x55d0f097d2d0 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 10 client.27907 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 10 client.27907 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 20 client.27907 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f0998030 0x55d0f099a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 4 client.27907 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:46.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0f08c8eb0 con 0x55d0f0998030 2023-07-21T19:20:46.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.107+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f0998030 0x55d0f099a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.108+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f0998030 0x55d0f099a440 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e822eecffa513860 server_cookie=9e252183b6e4a9eb in_seq=0 out_seq=0 2023-07-21T19:20:46.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.108+0000 7f39c4ff9700 10 client.27907 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:46.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39c4ff9700 1 -- 172.21.15.5:0/1646055383 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0f08c8eb0 con 0x55d0f0998030 2023-07-21T19:20:46.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39c4ff9700 10 client.27907 handle_command_reply: tid=0 2023-07-21T19:20:46.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 client.27907 shutdown 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 msgr2=0x55d0f0988ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 0x55d0f0988ea0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f39d8005940 tx=0x7f39d80047c0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 shutdown_connections 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f0988ad0 0x55d0f0989920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097d2d0 0x55d0f0988ea0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f0998030 0x55d0f099a440 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/1646055383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f097dbe0 0x55d0f09893e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 >> 172.21.15.5:0/1646055383 conn(0x55d0f0973da0 msgr2=0x55d0f0979290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39c4ff9700 0 client.27907 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 shutdown_connections 2023-07-21T19:20:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 1 -- 172.21.15.5:0/1646055383 wait complete. 2023-07-21T19:20:46.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.109+0000 7f39f73ffb80 20 client.27907 trim_cache size 0 max 16384 2023-07-21T19:20:46.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.128+0000 7f39f73ffb80 1 Processor -- start 2023-07-21T19:20:46.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.128+0000 7f39f73ffb80 1 -- start start 2023-07-21T19:20:46.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f0993660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f0984d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f0983750 con 0x55d0f0988ad0 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f0973970 con 0x55d0f097d2d0 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f09880f0 con 0x55d0f097dbe0 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e1f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f0984d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f0993660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f0993660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39420/0 (socket says 172.21.15.5:39420) 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48318/0 (socket says 172.21.15.5:48318) 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e0f9a700 1 -- 172.21.15.5:0/525182616 learned_addr learned my addr 172.21.15.5:0/525182616 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e1f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f0984d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56638/0 (socket says 172.21.15.5:56638) 2023-07-21T19:20:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e179b700 1 -- 172.21.15.5:0/525182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 msgr2=0x55d0f0993660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.129+0000 7f39e179b700 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f0993660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39e179b700 1 -- 172.21.15.5:0/525182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 msgr2=0x55d0f0984d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39e179b700 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f0984d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39e179b700 1 -- 172.21.15.5:0/525182616 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0f09755e0 con 0x55d0f097d2d0 2023-07-21T19:20:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39e179b700 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f39d8020850 tx=0x7f39d8009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acc923d75f7e6138 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39c7fff700 1 -- 172.21.15.5:0/525182616 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8020630 con 0x55d0f097d2d0 2023-07-21T19:20:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39c7fff700 1 -- 172.21.15.5:0/525182616 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d80092d0 con 0x55d0f097d2d0 2023-07-21T19:20:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.130+0000 7f39c7fff700 1 -- 172.21.15.5:0/525182616 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8005280 con 0x55d0f097d2d0 2023-07-21T19:20:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- 172.21.15.5:0/525182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 msgr2=0x55d0f09943a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f39d8020850 tx=0x7f39d8009880 comp rx=0 tx=0).stop 2023-07-21T19:20:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- 172.21.15.5:0/525182616 shutdown_connections 2023-07-21T19:20:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f0984d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f09943a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/525182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f0993660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- 172.21.15.5:0/525182616 >> 172.21.15.5:0/525182616 conn(0x55d0f0973da0 msgr2=0x55d0f0982440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- 172.21.15.5:0/525182616 shutdown_connections 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- 172.21.15.5:0/525182616 wait complete. 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.131+0000 7f39f73ffb80 1 -- start start 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 Processor -- start 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f098a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f09ec670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f09ecc40 con 0x55d0f0988ad0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f09ecdb0 con 0x55d0f097d2d0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0f09ecf70 con 0x55d0f097dbe0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.132+0000 7f39f73ffb80 10 client.0 _ll_register_callbacks cb 0x7f39d8004340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48324/0 (socket says 172.21.15.5:48324) 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e0f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f098a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 -- 172.21.15.5:0/2438561591 learned_addr learned my addr 172.21.15.5:0/2438561591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e1f9c700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f09ec670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 -- 172.21.15.5:0/2438561591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 msgr2=0x55d0f098a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f098a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 -- 172.21.15.5:0/2438561591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 msgr2=0x55d0f09ec670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f09ec670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 -- 172.21.15.5:0/2438561591 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0f06e6c60 con 0x55d0f097d2d0 2023-07-21T19:20:46.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f098a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39e179b700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39d8003d40 tx=0x7f39d8009a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44a72f14656c24cc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.133+0000 7f39c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8004970 con 0x55d0f097d2d0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f39cc09f3b0 con 0x55d0f097d2d0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f39d8020640 con 0x55d0f097d2d0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39d8013c70 con 0x55d0f097d2d0 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 10 client.27986 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55d0f0724250 con 0x55d0f097d2d0 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39d804b020 con 0x55d0f097d2d0 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f39d803c480 con 0x55d0f097d2d0 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 10 client.27986 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 10 client.27986 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 20 client.27986 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.134+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f09f53a0 0x55d0f09f7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.135+0000 7f39f73ffb80 4 client.27986 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:20:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.135+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d0f088b050 con 0x55d0f09f53a0 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.135+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f09f53a0 0x55d0f09f7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.135+0000 7f39e0f9a700 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f09f53a0 0x55d0f09f7780 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3656b4df939f6224 server_cookie=bf5bbda04290c35f in_seq=0 out_seq=0 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.135+0000 7f39c4ff9700 10 client.27986 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39c4ff9700 1 -- 172.21.15.5:0/2438561591 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55d0f088b050 con 0x55d0f09f53a0 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39c4ff9700 10 client.27986 handle_command_reply: tid=0 2023-07-21T19:20:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 client.27986 shutdown 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 msgr2=0x55d0f098a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f39d8003d40 tx=0x7f39d8009a30 comp rx=0 tx=0).stop 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 shutdown_connections 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0f09f53a0 0x55d0f09f7780 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0f0988ad0 0x55d0f09ec670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0f097d2d0 0x55d0f098a050 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39c4ff9700 0 client.27986 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:46.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 --2- 172.21.15.5:0/2438561591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0f097dbe0 0x55d0f098a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 >> 172.21.15.5:0/2438561591 conn(0x55d0f0973da0 msgr2=0x55d0f0982440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 shutdown_connections 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 1 -- 172.21.15.5:0/2438561591 wait complete. 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.136+0000 7f39f73ffb80 20 client.27986 trim_cache size 0 max 16384 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:20:46.138 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:20:46.139 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:20:46.140 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:20:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 msgr2=0x7f39c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 0x7f39c80a37f0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f39d80134b0 tx=0x7f39d8013870 comp rx=0 tx=0).stop 2023-07-21T19:20:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 msgr2=0x7f39dc19e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f39cc003920 tx=0x7f39cc003950 comp rx=0 tx=0).stop 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 shutdown_connections 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39dc105af0 0x7f39dc197b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39c80a1340 0x7f39c80a37f0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39dc0ffae0 0x7f39dc19dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 --2- 172.21.15.5:0/2949725927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39dc100490 0x7f39dc19e000 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 >> 172.21.15.5:0/2949725927 conn(0x7f39dc0fb5e0 msgr2=0x7f39dc1035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 shutdown_connections 2023-07-21T19:20:46.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:20:46.137+0000 7f39e39ff700 1 -- 172.21.15.5:0/2949725927 wait complete. 2023-07-21T19:20:46.150 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:20:47.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:47 smithi101 ceph-mon[128136]: pgmap v412: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 50 op/s 2023-07-21T19:20:47.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:47 smithi005 ceph-mon[126460]: pgmap v412: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 50 op/s 2023-07-21T19:20:47.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:47 smithi156 ceph-mon[128316]: pgmap v412: 97 pgs: 97 active+clean; 9.9 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 99 KiB/s rd, 23 MiB/s wr, 50 op/s 2023-07-21T19:20:49.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:49 smithi101 ceph-mon[128136]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 35 MiB/s wr, 60 op/s 2023-07-21T19:20:49.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:49 smithi005 ceph-mon[126460]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 35 MiB/s wr, 60 op/s 2023-07-21T19:20:49.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:49 smithi156 ceph-mon[128316]: pgmap v413: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 35 MiB/s wr, 60 op/s 2023-07-21T19:20:51.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:51 smithi101 ceph-mon[128136]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 MiB/s wr, 29 op/s 2023-07-21T19:20:51.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:51 smithi005 ceph-mon[126460]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 MiB/s wr, 29 op/s 2023-07-21T19:20:51.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:51 smithi156 ceph-mon[128316]: pgmap v414: 97 pgs: 97 active+clean; 10 GiB data, 4.4 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 24 MiB/s wr, 29 op/s 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.769+0000 7f57fb59e700 1 -- 172.21.15.5:0/3954144071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc0ffd20 msgr2=0x7f57fc1000f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.769+0000 7f57fb59e700 1 --2- 172.21.15.5:0/3954144071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc0ffd20 0x7f57fc1000f0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0098f0 tx=0x7f57ec009c00 comp rx=0 tx=0).stop 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 -- 172.21.15.5:0/3954144071 shutdown_connections 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 --2- 172.21.15.5:0/3954144071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 0x7f57fc10c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 --2- 172.21.15.5:0/3954144071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc0ffd20 0x7f57fc1000f0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 --2- 172.21.15.5:0/3954144071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc1014a0 0x7f57fc101910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 -- 172.21.15.5:0/3954144071 >> 172.21.15.5:0/3954144071 conn(0x7f57fc0fb440 msgr2=0x7f57fc0fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 -- 172.21.15.5:0/3954144071 shutdown_connections 2023-07-21T19:20:53.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.770+0000 7f57fb59e700 1 -- 172.21.15.5:0/3954144071 wait complete. 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.771+0000 7f57fb59e700 1 Processor -- start 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.771+0000 7f57fb59e700 1 -- start start 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 0x7f57fc19da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 0x7f57fc19df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 0x7f57fc197aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc198270 con 0x7f57fc1014a0 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1983e0 con 0x7f57fc101e50 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fb59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc1985a0 con 0x7f57fc0ffd20 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 0x7f57fc19df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 0x7f57fc19da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 0x7f57fc19df90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33592/0 (socket says 172.21.15.5:33592) 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 0x7f57fc197aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57fa59c700 1 -- 172.21.15.5:0/943723524 learned_addr learned my addr 172.21.15.5:0/943723524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 -- 172.21.15.5:0/943723524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 msgr2=0x7f57fc197aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 0x7f57fc197aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 -- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 msgr2=0x7f57fc19df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 0x7f57fc19df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.772+0000 7f57f259c700 1 -- 172.21.15.5:0/943723524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f4009d00 con 0x7f57fc0ffd20 2023-07-21T19:20:53.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57f259c700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 0x7f57fc19da50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009810 tx=0x7f57ec00fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57f3fff700 1 -- 172.21.15.5:0/943723524 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec01c070 con 0x7f57fc0ffd20 2023-07-21T19:20:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57ec009520 con 0x7f57fc0ffd20 2023-07-21T19:20:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57f3fff700 1 -- 172.21.15.5:0/943723524 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57ec00ba70 con 0x7f57fc0ffd20 2023-07-21T19:20:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57f3fff700 1 -- 172.21.15.5:0/943723524 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec005720 con 0x7f57fc0ffd20 2023-07-21T19:20:53.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.773+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc1a47f0 con 0x7f57fc0ffd20 2023-07-21T19:20:53.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.777+0000 7f57f3fff700 1 -- 172.21.15.5:0/943723524 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f57ec020030 con 0x7f57fc0ffd20 2023-07-21T19:20:53.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.778+0000 7f58144adb80 1 Processor -- start 2023-07-21T19:20:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 -- start start 2023-07-21T19:20:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f57f3fff700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 0x7f57e80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f57f3fff700 1 -- 172.21.15.5:0/943723524 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57ec0cdb80 con 0x7f57fc0ffd20 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a962cbc50 con 0x564a963cc470 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9631e9e0 con 0x564a962226b0 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f58144adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96317e70 con 0x564a963cc8c0 2023-07-21T19:20:53.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f57fad9d700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 0x7f57e80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.779+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33598/0 (socket says 172.21.15.5:33598) 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57fa59c700 1 -- 172.21.15.5:0/4117484848 learned_addr learned my addr 172.21.15.5:0/4117484848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57fad9d700 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 -- 172.21.15.5:0/4117484848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 msgr2=0x564a963c6700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c6700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 -- 172.21.15.5:0/4117484848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 msgr2=0x564a963c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 -- 172.21.15.5:0/4117484848 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a96145d80 con 0x564a962226b0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57fa59c700 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57fad9d700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 0x7f57e80a3770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f57fc19e6e0 tx=0x7f57f4007b80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.780+0000 7f57f259c700 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cbf30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0cb270 tx=0x7f57ec0970f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=420f118ace563be4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f57f0d99700 1 -- 172.21.15.5:0/4117484848 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09ecb0 con 0x564a962226b0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f57f0d99700 1 -- 172.21.15.5:0/4117484848 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57ec0a3040 con 0x564a962226b0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f57f0d99700 1 -- 172.21.15.5:0/4117484848 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09aee0 con 0x564a962226b0 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 -- 172.21.15.5:0/4117484848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 msgr2=0x564a963cbf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cbf30 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0cb270 tx=0x7f57ec0970f0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 -- 172.21.15.5:0/4117484848 shutdown_connections 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c6700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cbf30 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 --2- 172.21.15.5:0/4117484848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963c61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.781+0000 7f58144adb80 1 -- 172.21.15.5:0/4117484848 >> 172.21.15.5:0/4117484848 conn(0x564a963bbd40 msgr2=0x564a963c1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.782+0000 7f58144adb80 1 -- 172.21.15.5:0/4117484848 shutdown_connections 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.782+0000 7f58144adb80 1 -- 172.21.15.5:0/4117484848 wait complete. 2023-07-21T19:20:53.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.782+0000 7f58144adb80 1 -- start start 2023-07-21T19:20:53.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.783+0000 7f58144adb80 1 Processor -- start 2023-07-21T19:20:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.783+0000 7f58144adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 0x564a963d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.783+0000 7f58144adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.783+0000 7f58144adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc8c0 0x564a963cf730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f58144adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a962b7a60 con 0x564a963cc8c0 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f58144adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a962b7760 con 0x564a963cc470 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f58144adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a9619b8c0 con 0x564a962226b0 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f58144adb80 10 client.0 _ll_register_callbacks cb 0x564a962b5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51694/0 (socket says 172.21.15.5:51694) 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57fa59c700 1 -- 172.21.15.5:0/770564545 learned_addr learned my addr 172.21.15.5:0/770564545 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 0x564a963d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57fad9d700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc8c0 0x564a963cf730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 -- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 msgr2=0x564a963d59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 -- 172.21.15.5:0/770564545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc8c0 msgr2=0x564a963cf730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc8c0 0x564a963cf730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57f259c700 1 -- 172.21.15.5:0/770564545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e40099f0 con 0x564a962226b0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.784+0000 7f57fa59c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57f259c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 0x564a963d54a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0cb2a0 tx=0x7f57ec0975c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8679303462b4e082 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec0cb4f0 con 0x564a962226b0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57ec0cb650 con 0x564a962226b0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a96271490 con 0x564a962226b0 2023-07-21T19:20:53.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57ec09a170 con 0x564a962226b0 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f58144adb80 10 client.27925 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f57fc1a47f0 con 0x564a962226b0 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.785+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a96271490 con 0x564a962226b0 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57ec0a2bb0 con 0x564a962226b0 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 10 client.27925 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 10 client.27925 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:53.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 20 client.27925 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a963df460 0x564a963e1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 4 client.27925 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a96310e50 con 0x564a963df460 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.786+0000 7f57fa59c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a963df460 0x564a963e1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.787+0000 7f57fa59c700 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a963df460 0x564a963e1840 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c3e52c9ae62c205 server_cookie=695d2c3aa2926cbb in_seq=0 out_seq=0 2023-07-21T19:20:53.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.787+0000 7f57e1ffb700 10 client.27925 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.788+0000 7f57e1ffb700 1 -- 172.21.15.5:0/770564545 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a96310e50 con 0x564a963df460 2023-07-21T19:20:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.788+0000 7f57e1ffb700 10 client.27925 handle_command_reply: tid=0 2023-07-21T19:20:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.788+0000 7f58144adb80 1 client.27925 shutdown 2023-07-21T19:20:53.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.788+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 msgr2=0x564a963d54a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.788+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 0x564a963d54a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0cb2a0 tx=0x7f57ec0975c0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 shutdown_connections 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a963df460 0x564a963e1840 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc8c0 0x564a963cf730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a962226b0 0x564a963d54a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f57e1ffb700 0 client.27925 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:53.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 --2- 172.21.15.5:0/770564545 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a963d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 >> 172.21.15.5:0/770564545 conn(0x564a963bbd40 msgr2=0x564a963c0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 shutdown_connections 2023-07-21T19:20:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 1 -- 172.21.15.5:0/770564545 wait complete. 2023-07-21T19:20:53.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.790+0000 7f58144adb80 20 client.27925 trim_cache size 0 max 16384 2023-07-21T19:20:53.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 Processor -- start 2023-07-21T19:20:53.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 -- start start 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cf690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96271490 con 0x564a963cc470 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a963c1f10 con 0x564a962226b0 2023-07-21T19:20:53.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f58144adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a963b8d80 con 0x564a963cc8c0 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33622/0 (socket says 172.21.15.5:33622) 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f57fa59c700 1 -- 172.21.15.5:0/1017746175 learned_addr learned my addr 172.21.15.5:0/1017746175 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.810+0000 7f57f259c700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cf690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57fad9d700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 -- 172.21.15.5:0/1017746175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 msgr2=0x564a963c0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 -- 172.21.15.5:0/1017746175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 msgr2=0x564a963ce670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 -- 172.21.15.5:0/1017746175 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a963e6440 con 0x564a962226b0 2023-07-21T19:20:53.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57fa59c700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:53.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f259c700 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cf690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0a2b10 tx=0x7f57ec0b90d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27ba4689ce26819 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f159a700 1 -- 172.21.15.5:0/1017746175 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09a7c0 con 0x564a962226b0 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f159a700 1 -- 172.21.15.5:0/1017746175 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57ec0cb4f0 con 0x564a962226b0 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.811+0000 7f57f159a700 1 -- 172.21.15.5:0/1017746175 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec09b140 con 0x564a962226b0 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 -- 172.21.15.5:0/1017746175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 msgr2=0x564a963cf690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cf690 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f57ec0a2b10 tx=0x7f57ec0b90d0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 -- 172.21.15.5:0/1017746175 shutdown_connections 2023-07-21T19:20:53.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a963c0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a962226b0 0x564a963cf690 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 --2- 172.21.15.5:0/1017746175 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a963cc470 0x564a963ce670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 -- 172.21.15.5:0/1017746175 >> 172.21.15.5:0/1017746175 conn(0x564a963bbd40 msgr2=0x564a963cdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 -- 172.21.15.5:0/1017746175 shutdown_connections 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.812+0000 7f58144adb80 1 -- 172.21.15.5:0/1017746175 wait complete. 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.813+0000 7f58144adb80 1 -- start start 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.813+0000 7f58144adb80 1 Processor -- start 2023-07-21T19:20:53.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.813+0000 7f58144adb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a96427cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96432400 con 0x564a962226b0 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96432540 con 0x564a963cc470 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a96432700 con 0x564a963cc8c0 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f58144adb80 10 client.0 _ll_register_callbacks cb 0x7f57e40095c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51718/0 (socket says 172.21.15.5:51718) 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 -- 172.21.15.5:0/4166130600 learned_addr learned my addr 172.21.15.5:0/4166130600 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57f259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33624/0 (socket says 172.21.15.5:33624) 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fad9d700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a96427cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 -- 172.21.15.5:0/4166130600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 msgr2=0x564a96427cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a96427cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 -- 172.21.15.5:0/4166130600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 msgr2=0x564a96427250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:20:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57fa59c700 1 -- 172.21.15.5:0/4166130600 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e4012040 con 0x564a963cc470 2023-07-21T19:20:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.814+0000 7f57f259c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:20:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f57fa59c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x564a963d0d90 tx=0x7f57e401fd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19354092896ffd5b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:20:53.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f57e1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:20:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e4017040 con 0x564a963cc470 2023-07-21T19:20:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f57ec0a3040 con 0x564a963cc470 2023-07-21T19:20:53.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(102..102 src has 1..102) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57e401fe40 con 0x564a963cc470 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f57e401f600 con 0x564a963cc470 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f58144adb80 10 client.28004 fetch_fsmap learned FSMap version 44 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.815+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f57fc1a47f0 con 0x564a963cc470 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57ec0a3040 con 0x564a963cc470 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f57e4010550 con 0x564a963cc470 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 10 client.28004 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:20:53.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 10 client.28004 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 20 client.28004 populate_metadata read hostname 'smithi005' 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a96433eb0 0x564a964362c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 4 client.28004 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564a962d2ff0 con 0x564a96433eb0 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.816+0000 7f57f259c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a96433eb0 0x564a964362c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.817+0000 7f57f259c700 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a96433eb0 0x564a964362c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d450612080fb959 server_cookie=b8eb8e5afdf2822d in_seq=0 out_seq=0 2023-07-21T19:20:53.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.817+0000 7f57e1ffb700 10 client.28004 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:53.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:53 smithi101 ceph-mon[128136]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 37 op/s 2023-07-21T19:20:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f57e1ffb700 1 -- 172.21.15.5:0/4166130600 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564a962d2ff0 con 0x564a96433eb0 2023-07-21T19:20:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f57e1ffb700 10 client.28004 handle_command_reply: tid=0 2023-07-21T19:20:53.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 client.28004 shutdown 2023-07-21T19:20:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 msgr2=0x564a96427790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x564a963d0d90 tx=0x7f57e401fd20 comp rx=0 tx=0).stop 2023-07-21T19:20:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 shutdown_connections 2023-07-21T19:20:53.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a963cc8c0 0x564a96427cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564a96433eb0 0x564a964362c0 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a962226b0 0x564a96427250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 --2- 172.21.15.5:0/4166130600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a963cc470 0x564a96427790 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f57e1ffb700 0 client.28004 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.833+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 >> 172.21.15.5:0/4166130600 conn(0x564a963bbd40 msgr2=0x564a963ce7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.834+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 shutdown_connections 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.834+0000 7f58144adb80 1 -- 172.21.15.5:0/4166130600 wait complete. 2023-07-21T19:20:53.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.834+0000 7f58144adb80 20 client.28004 trim_cache size 0 max 16384 2023-07-21T19:20:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:20:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:20:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:20:53.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:20:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 msgr2=0x7f57e80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 0x7f57e80a3770 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f57fc19e6e0 tx=0x7f57f4007b80 comp rx=0 tx=0).stop 2023-07-21T19:20:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 msgr2=0x7f57fc19da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:20:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 0x7f57fc19da50 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f57ec009810 tx=0x7f57ec00fa00 comp rx=0 tx=0).stop 2023-07-21T19:20:53.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 shutdown_connections 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57fc101e50 0x7f57fc197aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57fc0ffd20 0x7f57fc19da50 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57e80a12c0 0x7f57e80a3770 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 --2- 172.21.15.5:0/943723524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57fc1014a0 0x7f57fc19df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 >> 172.21.15.5:0/943723524 conn(0x7f57fc0fb440 msgr2=0x7f57fc0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 shutdown_connections 2023-07-21T19:20:53.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:20:53.835+0000 7f57fb59e700 1 -- 172.21.15.5:0/943723524 wait complete. 2023-07-21T19:20:53.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:53 smithi005 ceph-mon[126460]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 37 op/s 2023-07-21T19:20:53.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:53 smithi156 ceph-mon[128316]: pgmap v415: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 35 MiB/s wr, 37 op/s 2023-07-21T19:20:55.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:55 smithi101 ceph-mon[128136]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 28 op/s 2023-07-21T19:20:55.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:55 smithi005 ceph-mon[126460]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 28 op/s 2023-07-21T19:20:55.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:55 smithi156 ceph-mon[128316]: pgmap v416: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 26 MiB/s wr, 28 op/s 2023-07-21T19:20:57.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:57 smithi101 ceph-mon[128136]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:20:57.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:57 smithi005 ceph-mon[126460]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:20:57.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:57 smithi156 ceph-mon[128316]: pgmap v417: 97 pgs: 97 active+clean; 10 GiB data, 4.5 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:20:59.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:20:59 smithi101 ceph-mon[128136]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 31 op/s 2023-07-21T19:20:59.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:20:59 smithi005 ceph-mon[126460]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 31 op/s 2023-07-21T19:20:59.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:20:59 smithi156 ceph-mon[128316]: pgmap v418: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 31 op/s 2023-07-21T19:21:01.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:01 smithi101 ceph-mon[128136]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:21:01.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:01 smithi005 ceph-mon[126460]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:21:01.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:01 smithi156 ceph-mon[128316]: pgmap v419: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:21:03.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:03 smithi101 ceph-mon[128136]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:21:03.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:03 smithi005 ceph-mon[126460]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:21:03.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:03 smithi156 ceph-mon[128316]: pgmap v420: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:21:05.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:05 smithi101 ceph-mon[128136]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-07-21T19:21:05.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:05 smithi005 ceph-mon[126460]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-07-21T19:21:05.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:05 smithi156 ceph-mon[128316]: pgmap v421: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 25 MiB/s wr, 37 op/s 2023-07-21T19:21:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 -- 172.21.15.5:0/3336005579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ff930 msgr2=0x7f1bb40ffda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/3336005579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ff930 0x7f1bb40ffda0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1ba40098f0 tx=0x7f1ba4009c00 comp rx=0 tx=0).stop 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 -- 172.21.15.5:0/3336005579 shutdown_connections 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/3336005579 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb410c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/3336005579 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb4105940 0x7f1bb4105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/3336005579 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb40ff930 0x7f1bb40ffda0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 -- 172.21.15.5:0/3336005579 >> 172.21.15.5:0/3336005579 conn(0x7f1bb40fb450 msgr2=0x7f1bb40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 -- 172.21.15.5:0/3336005579 shutdown_connections 2023-07-21T19:21:07.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.024+0000 7f1bbb39d700 1 -- 172.21.15.5:0/3336005579 wait complete. 2023-07-21T19:21:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.026+0000 7f1bbb39d700 1 Processor -- start 2023-07-21T19:21:07.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.026+0000 7f1bbb39d700 1 -- start start 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb419dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4105940 0x7f1bb41979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4198040 con 0x7f1bb40ff930 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb41981b0 con 0x7f1bb4105940 2023-07-21T19:21:07.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbb39d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1bb4198370 con 0x7f1bb41002e0 2023-07-21T19:21:07.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bb9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb419dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40510/0 (socket says 172.21.15.5:40510) 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 -- 172.21.15.5:0/2786273082 learned_addr learned my addr 172.21.15.5:0/2786273082 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4105940 0x7f1bb41979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 -- 172.21.15.5:0/2786273082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 msgr2=0x7f1bb419dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb419dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 -- 172.21.15.5:0/2786273082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4105940 msgr2=0x7f1bb41979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.027+0000 7f1bba39b700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4105940 0x7f1bb41979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bba39b700 1 -- 172.21.15.5:0/2786273082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ba4009520 con 0x7f1bb40ff930 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bba39b700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e970 tx=0x7f1bb000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bab7fe700 1 -- 172.21.15.5:0/2786273082 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0009790 con 0x7f1bb40ff930 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bab7fe700 1 -- 172.21.15.5:0/2786273082 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bb000c7e0 con 0x7f1bb40ff930 2023-07-21T19:21:07.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1bb4198650 con 0x7f1bb40ff930 2023-07-21T19:21:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bab7fe700 1 -- 172.21.15.5:0/2786273082 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bb0003bd0 con 0x7f1bb40ff930 2023-07-21T19:21:07.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.028+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bb41a4950 con 0x7f1bb40ff930 2023-07-21T19:21:07.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.033+0000 7f1bab7fe700 1 -- 172.21.15.5:0/2786273082 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1bb000c950 con 0x7f1bb40ff930 2023-07-21T19:21:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.034+0000 7f1bd0001b80 1 Processor -- start 2023-07-21T19:21:07.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 -- start start 2023-07-21T19:21:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae7470 0x556f03ae11a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 0x556f03ae1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f039e6c50 con 0x556f03ae78c0 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03a399e0 con 0x556f0393d6b0 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03a32e70 con 0x556f03ae7470 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53058/0 (socket says 172.21.15.5:53058) 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.035+0000 7f1bbab9c700 1 -- 172.21.15.5:0/399985144 learned_addr learned my addr 172.21.15.5:0/399985144 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:07.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 0x556f03ae1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 -- 172.21.15.5:0/399985144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae7470 msgr2=0x556f03ae11a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae7470 0x556f03ae11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 -- 172.21.15.5:0/399985144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 msgr2=0x556f03ae1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 0x556f03ae1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 -- 172.21.15.5:0/399985144 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f03860d80 con 0x556f0393d6b0 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 0x556f03ae1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bab7fe700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 0x7f1ba00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bac011600 tx=0x7f1bac0099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd324b230d000416 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1bab7fe700 1 -- 172.21.15.5:0/2786273082 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f1bb0014070 con 0x7f1bb40ff930 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.036+0000 7f1ba8bfe700 1 -- 172.21.15.5:0/399985144 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac00e040 con 0x556f0393d6b0 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1bba39b700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 0x7f1ba00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1ba8bfe700 1 -- 172.21.15.5:0/399985144 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bac00f040 con 0x556f0393d6b0 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1ba8bfe700 1 -- 172.21.15.5:0/399985144 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bac004690 con 0x556f0393d6b0 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1bd0001b80 1 -- 172.21.15.5:0/399985144 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 msgr2=0x556f03ae6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f1bac011600 tx=0x7f1bac0099e0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.037+0000 7f1bba39b700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 0x7f1ba00a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0096900 tx=0x7f1bb000cf50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 -- 172.21.15.5:0/399985144 shutdown_connections 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae78c0 0x556f03ae1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03ae6f30 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/399985144 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae7470 0x556f03ae11a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 -- 172.21.15.5:0/399985144 >> 172.21.15.5:0/399985144 conn(0x556f03ad6d40 msgr2=0x556f03adc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 -- 172.21.15.5:0/399985144 shutdown_connections 2023-07-21T19:21:07.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 -- 172.21.15.5:0/399985144 wait complete. 2023-07-21T19:21:07.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.038+0000 7f1bd0001b80 1 -- start start 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 Processor -- start 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03aeaaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aeb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f039d2a60 con 0x556f03ae7470 2023-07-21T19:21:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f039d2760 con 0x556f0393d6b0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f038b68c0 con 0x556f03ae78c0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bd0001b80 10 client.0 _ll_register_callbacks cb 0x556f039d0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53060/0 (socket says 172.21.15.5:53060) 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.039+0000 7f1bbab9c700 1 -- 172.21.15.5:0/1097262162 learned_addr learned my addr 172.21.15.5:0/1097262162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03aeaaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aeb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1097262162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 msgr2=0x556f03aeb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aeb190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 msgr2=0x556f03aed800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1097262162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1bac0112b0 con 0x556f03ae7470 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03aeaaf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009810 tx=0x7f1ba400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dafeb7efa5e6c97 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.040+0000 7f1b9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:21:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba401c040 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556f0398c490 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f1ba400bb90 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ba400bec0 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1bd0001b80 10 client.18366 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556f03849c00 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.041+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ba4031020 con 0x556f03ae7470 2023-07-21T19:21:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ba4015020 con 0x556f03ae7470 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 10 client.18366 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 10 client.18366 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 20 client.18366 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03afacb0 0x556f03afd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 4 client.18366 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556f03a2be50 con 0x556f03afacb0 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.042+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03afacb0 0x556f03afd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.043+0000 7f1b9dffb700 10 client.18366 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.043+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03afacb0 0x556f03afd0c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48d4c461ee003ba2 server_cookie=a9182c253f966c55 in_seq=0 out_seq=0 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.043+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1097262162 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556f03a2be50 con 0x556f03afacb0 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.043+0000 7f1b9dffb700 10 client.18366 handle_command_reply: tid=0 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 client.18366 shutdown 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 msgr2=0x556f03aeaaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03aeaaf0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4009810 tx=0x7f1ba400b790 comp rx=0 tx=0).stop 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 shutdown_connections 2023-07-21T19:21:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aeb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03aed800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03afacb0 0x556f03afd0c0 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1097262162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03aeaaf0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 >> 172.21.15.5:0/1097262162 conn(0x556f03ad6d40 msgr2=0x556f03adb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1b9dffb700 0 client.18366 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 shutdown_connections 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1097262162 wait complete. 2023-07-21T19:21:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.044+0000 7f1bd0001b80 20 client.18366 trim_cache size 0 max 16384 2023-07-21T19:21:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 Processor -- start 2023-07-21T19:21:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 -- start start 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03af1e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03ae6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03ae57e0 con 0x556f03ae7470 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03addc30 con 0x556f0393d6b0 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.066+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03ad3d10 con 0x556f03ae78c0 2023-07-21T19:21:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03af1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03af1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53066/0 (socket says 172.21.15.5:53066) 2023-07-21T19:21:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bbab9c700 1 -- 172.21.15.5:0/2445490393 learned_addr learned my addr 172.21.15.5:0/2445490393 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03ae6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/2445490393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 msgr2=0x556f03aed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/2445490393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 msgr2=0x556f03af1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03af1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.067+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/2445490393 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556f038871f0 con 0x556f03ae7470 2023-07-21T19:21:07.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.068+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03ae6300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1ba402c610 tx=0x7f1ba4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bf7bff2fcf0659 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.069+0000 7f1ba93ff700 1 -- 172.21.15.5:0/2445490393 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba401c040 con 0x556f03ae7470 2023-07-21T19:21:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.069+0000 7f1ba93ff700 1 -- 172.21.15.5:0/2445490393 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1ba4031030 con 0x556f03ae7470 2023-07-21T19:21:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.069+0000 7f1ba93ff700 1 -- 172.21.15.5:0/2445490393 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba4005560 con 0x556f03ae7470 2023-07-21T19:21:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 -- 172.21.15.5:0/2445490393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 msgr2=0x556f03ae6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03ae6300 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f1ba402c610 tx=0x7f1ba4009c00 comp rx=0 tx=0).stop 2023-07-21T19:21:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 -- 172.21.15.5:0/2445490393 shutdown_connections 2023-07-21T19:21:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03aed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f0393d6b0 0x556f03af1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/2445490393 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f03ae7470 0x556f03ae6300 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 -- 172.21.15.5:0/2445490393 >> 172.21.15.5:0/2445490393 conn(0x556f03ad6d40 msgr2=0x556f03ae5bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 -- 172.21.15.5:0/2445490393 shutdown_connections 2023-07-21T19:21:07.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.070+0000 7f1bd0001b80 1 -- 172.21.15.5:0/2445490393 wait complete. 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.071+0000 7f1bd0001b80 1 -- start start 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.071+0000 7f1bd0001b80 1 Processor -- start 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 0x556f03ae3c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 0x556f03ae4170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03ae2990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03af23b0 con 0x556f0393d6b0 2023-07-21T19:21:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03ae2ed0 con 0x556f03ae7470 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556f03ae3090 con 0x556f03ae78c0 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bd0001b80 10 client.0 _ll_register_callbacks cb 0x7f1ba4009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 0x556f03ae3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.072+0000 7f1bb9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 0x556f03ae4170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bbab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 0x556f03ae3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40532/0 (socket says 172.21.15.5:40532) 2023-07-21T19:21:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bbab9c700 1 -- 172.21.15.5:0/1656934417 learned_addr learned my addr 172.21.15.5:0/1656934417 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03ae2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1656934417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 msgr2=0x556f03ae2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03ae2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1656934417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 msgr2=0x556f03ae3c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 0x556f03ae3c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 -- 172.21.15.5:0/1656934417 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556f03849c00 con 0x556f03ae7470 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03ae2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1bb9b9a700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 0x556f03ae4170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005e90 tx=0x7f1ba40038b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdba8bdd4d829ced server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.073+0000 7f1b9dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ba401c070 con 0x556f03ae7470 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(103..103 src has 1..103) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f1ba4000e40 con 0x556f03ae7470 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1bac00f040 con 0x556f03ae7470 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ba4005920 con 0x556f03ae7470 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1bd0001b80 10 client.28022 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.074+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x556f0385f250 con 0x556f03ae7470 2023-07-21T19:21:07.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1bac00f040 con 0x556f03ae7470 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1ba4009c70 con 0x556f03ae7470 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 10 client.28022 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 10 client.28022 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 20 client.28022 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03b4f4c0 0x556f03b518d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 4 client.28022 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556f039edff0 con 0x556f03b4f4c0 2023-07-21T19:21:07.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.075+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03b4f4c0 0x556f03b518d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.076+0000 7f1bbab9c700 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03b4f4c0 0x556f03b518d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c8b830be463c6df server_cookie=d7c740eed5d67fe9 in_seq=0 out_seq=0 2023-07-21T19:21:07.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.076+0000 7f1b9dffb700 10 client.28022 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1b9dffb700 1 -- 172.21.15.5:0/1656934417 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556f039edff0 con 0x556f03b4f4c0 2023-07-21T19:21:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1b9dffb700 10 client.28022 handle_command_reply: tid=0 2023-07-21T19:21:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 client.28022 shutdown 2023-07-21T19:21:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 msgr2=0x556f03ae4170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 0x556f03ae4170 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f1ba4005e90 tx=0x7f1ba40038b0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 shutdown_connections 2023-07-21T19:21:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556f03ae78c0 0x556f03ae2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556f03b4f4c0 0x556f03b518d0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556f0393d6b0 0x556f03ae3c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 --2- 172.21.15.5:0/1656934417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556f03ae7470 0x556f03ae4170 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 >> 172.21.15.5:0/1656934417 conn(0x556f03ad6d40 msgr2=0x556f03ae8520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1b9dffb700 0 client.28022 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 shutdown_connections 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.093+0000 7f1bd0001b80 1 -- 172.21.15.5:0/1656934417 wait complete. 2023-07-21T19:21:07.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.094+0000 7f1bd0001b80 20 client.28022 trim_cache size 0 max 16384 2023-07-21T19:21:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:21:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:21:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:21:07.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.094+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 msgr2=0x7f1ba00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.094+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 0x7f1ba00a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f1bb0096900 tx=0x7f1bb000cf50 comp rx=0 tx=0).stop 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 msgr2=0x7f1bb419d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f1bb000e970 tx=0x7f1bb000ec80 comp rx=0 tx=0).stop 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 shutdown_connections 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1bb4105940 0x7f1bb41979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1ba00a1340 0x7f1ba00a37f0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1bb40ff930 0x7f1bb419d960 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 --2- 172.21.15.5:0/2786273082 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1bb41002e0 0x7f1bb419dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 >> 172.21.15.5:0/2786273082 conn(0x7f1bb40fb450 msgr2=0x7f1bb4072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 shutdown_connections 2023-07-21T19:21:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:07.095+0000 7f1bbb39d700 1 -- 172.21.15.5:0/2786273082 wait complete. 2023-07-21T19:21:07.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:07 smithi101 ceph-mon[128136]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:21:07.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:07 smithi101 ceph-mon[128136]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T19:21:07.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:07 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:21:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:07 smithi005 ceph-mon[126460]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:21:07.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:07 smithi005 ceph-mon[126460]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T19:21:07.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:07 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:21:07.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:07 smithi156 ceph-mon[128316]: pgmap v422: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 24 MiB/s wr, 35 op/s 2023-07-21T19:21:07.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:07 smithi156 ceph-mon[128316]: osdmap e103: 12 total, 12 up, 12 in 2023-07-21T19:21:07.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:07 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:21:08.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:08 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:21:08.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:08 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:21:08.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:08 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:21:08.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:08 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:21:08.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:08 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:21:08.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:08 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:21:09.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:08 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:21:09.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:08 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:21:09.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:08 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:21:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:09 smithi005 ceph-mon[126460]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:09.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:09 smithi156 ceph-mon[128316]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:10.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:09 smithi101 ceph-mon[128136]: pgmap v424: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:11.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:11 smithi005 ceph-mon[126460]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:11.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:11 smithi156 ceph-mon[128316]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:12.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:11 smithi101 ceph-mon[128136]: pgmap v425: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 28 MiB/s wr, 48 op/s 2023-07-21T19:21:13.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:13 smithi101 ceph-mon[128136]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 28 MiB/s wr, 33 op/s 2023-07-21T19:21:13.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:13 smithi101 ceph-mon[128136]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T19:21:13.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:13 smithi005 ceph-mon[126460]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 28 MiB/s wr, 33 op/s 2023-07-21T19:21:13.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:13 smithi005 ceph-mon[126460]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T19:21:13.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:13 smithi156 ceph-mon[128316]: pgmap v426: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 28 MiB/s wr, 33 op/s 2023-07-21T19:21:13.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:13 smithi156 ceph-mon[128316]: osdmap e104: 12 total, 12 up, 12 in 2023-07-21T19:21:15.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:15 smithi101 ceph-mon[128136]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 36 MiB/s wr, 37 op/s 2023-07-21T19:21:15.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:15 smithi005 ceph-mon[126460]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 36 MiB/s wr, 37 op/s 2023-07-21T19:21:15.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:15 smithi156 ceph-mon[128316]: pgmap v428: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 36 MiB/s wr, 37 op/s 2023-07-21T19:21:16.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:21:16.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.398+0000 7f5a77dff700 1 -- 172.21.15.5:0/2270938959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 msgr2=0x7f5a70109b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.398+0000 7f5a77dff700 1 --2- 172.21.15.5:0/2270938959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70109b00 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f5a60009990 tx=0x7f5a60009ca0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 -- 172.21.15.5:0/2270938959 shutdown_connections 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 --2- 172.21.15.5:0/2270938959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a7010a040 0x7f5a7010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 --2- 172.21.15.5:0/2270938959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a70100d00 0x7f5a701010d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 --2- 172.21.15.5:0/2270938959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70109b00 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 -- 172.21.15.5:0/2270938959 >> 172.21.15.5:0/2270938959 conn(0x7f5a700fa4b0 msgr2=0x7f5a700fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 -- 172.21.15.5:0/2270938959 shutdown_connections 2023-07-21T19:21:16.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.399+0000 7f5a77dff700 1 -- 172.21.15.5:0/2270938959 wait complete. 2023-07-21T19:21:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 Processor -- start 2023-07-21T19:21:16.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 -- start start 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a7010a040 0x7f5a7019c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.400+0000 7f5a77dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7019cdf0 con 0x7f5a70101610 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a77dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7019cf60 con 0x7f5a70100d00 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a77dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7019d120 con 0x7f5a7010a040 2023-07-21T19:21:16.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57266/0 (socket says 172.21.15.5:57266) 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 -- 172.21.15.5:0/3119355187 learned_addr learned my addr 172.21.15.5:0/3119355187 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a7639c700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a7010a040 0x7f5a7019c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a7539a700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 -- 172.21.15.5:0/3119355187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a7010a040 msgr2=0x7f5a7019c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a7010a040 0x7f5a7019c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 -- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 msgr2=0x7f5a70198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.401+0000 7f5a75b9b700 1 -- 172.21.15.5:0/3119355187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a600095f0 con 0x7f5a70100d00 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00e9a0 tx=0x7f5a6c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a66ffd700 1 -- 172.21.15.5:0/3119355187 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c009790 con 0x7f5a70100d00 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a7019d400 con 0x7f5a70100d00 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a66ffd700 1 -- 172.21.15.5:0/3119355187 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a6c010430 con 0x7f5a70100d00 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a66ffd700 1 -- 172.21.15.5:0/3119355187 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c00cea0 con 0x7f5a70100d00 2023-07-21T19:21:16.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.402+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a701a4c40 con 0x7f5a70100d00 2023-07-21T19:21:16.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.406+0000 7f5a66ffd700 1 -- 172.21.15.5:0/3119355187 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5a6c0105a0 con 0x7f5a70100d00 2023-07-21T19:21:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.407+0000 7f5a66ffd700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 0x7f5a5c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.408+0000 7f5a66ffd700 1 -- 172.21.15.5:0/3119355187 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5a6c014070 con 0x7f5a70100d00 2023-07-21T19:21:16.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.408+0000 7f5a8b704b80 1 Processor -- start 2023-07-21T19:21:16.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.408+0000 7f5a7539a700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 0x7f5a5c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.408+0000 7f5a8b704b80 1 -- start start 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a7539a700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 0x7f5a5c0a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a70199420 tx=0x7f5a6000b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 0x562c1561ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15515cb0 con 0x562c1561aad0 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15568a40 con 0x562c1560fbe0 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15561ed0 con 0x562c1560f2d0 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39634/0 (socket says 172.21.15.5:39634) 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1198228997 learned_addr learned my addr 172.21.15.5:0/1198228997 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a7539a700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a7639c700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 0x562c1561ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1198228997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 msgr2=0x562c1561a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1198228997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 msgr2=0x562c1561ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 0x562c1561ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1198228997 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c1538fde0 con 0x562c1560f2d0 2023-07-21T19:21:16.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.409+0000 7f5a7639c700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 0x562c1561ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.410+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c096a50 tx=0x7f5a6c099c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6633f84ca3169cb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.410+0000 7f5a5bfff700 1 -- 172.21.15.5:0/1198228997 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c09e010 con 0x562c1560f2d0 2023-07-21T19:21:16.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.410+0000 7f5a5bfff700 1 -- 172.21.15.5:0/1198228997 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a6c0a2430 con 0x562c1560f2d0 2023-07-21T19:21:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.410+0000 7f5a5bfff700 1 -- 172.21.15.5:0/1198228997 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c0a1fa0 con 0x562c1560f2d0 2023-07-21T19:21:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1198228997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 msgr2=0x562c1560f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c096a50 tx=0x7f5a6c099c30 comp rx=0 tx=0).stop 2023-07-21T19:21:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1198228997 shutdown_connections 2023-07-21T19:21:16.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1561aad0 0x562c1561ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1560f6a0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1198228997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1198228997 >> 172.21.15.5:0/1198228997 conn(0x562c15605da0 msgr2=0x562c1560b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1198228997 shutdown_connections 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.411+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1198228997 wait complete. 2023-07-21T19:21:16.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 -- start start 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 Processor -- start 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560fbe0 0x562c15617830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c15617f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15501ac0 con 0x562c1560fbe0 2023-07-21T19:21:16.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c155017c0 con 0x562c1561aad0 2023-07-21T19:21:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c153e5920 con 0x562c1560f2d0 2023-07-21T19:21:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.412+0000 7f5a8b704b80 10 client.0 _ll_register_callbacks cb 0x562c154ff760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39636/0 (socket says 172.21.15.5:39636) 2023-07-21T19:21:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1258449432 learned_addr learned my addr 172.21.15.5:0/1258449432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a7539a700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560fbe0 0x562c15617830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a7639c700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c15617f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 msgr2=0x562c15617f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c15617f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1258449432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560fbe0 msgr2=0x562c15617830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560fbe0 0x562c15617830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 -- 172.21.15.5:0/1258449432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a701a4c40 con 0x562c1560f2d0 2023-07-21T19:21:16.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c099f70 tx=0x7f5a6c09a4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26481b27f6bdfc21 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.413+0000 7f5a597fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6c09e010 con 0x562c1560f2d0 2023-07-21T19:21:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5a6c0a21b0 con 0x562c1560f2d0 2023-07-21T19:21:16.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562c154bb4f0 con 0x562c1560f2d0 2023-07-21T19:21:16.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a6c0ca300 con 0x562c1560f2d0 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a8b704b80 10 client.27958 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562c15378c60 con 0x562c1560f2d0 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a6c063020 con 0x562c1560f2d0 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.414+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a6c0ca020 con 0x562c1560f2d0 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 10 client.27958 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 10 client.27958 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:16.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 20 client.27958 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1562a0d0 0x562c1562c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 4 client.27958 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562c1555aeb0 con 0x562c1562a0d0 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.415+0000 7f5a7539a700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1562a0d0 0x562c1562c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.416+0000 7f5a597fa700 10 client.27958 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.416+0000 7f5a7539a700 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1562a0d0 0x562c1562c4e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24f795de61429563 server_cookie=d90111c5c66ae451 in_seq=0 out_seq=0 2023-07-21T19:21:16.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a597fa700 1 -- 172.21.15.5:0/1258449432 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562c1555aeb0 con 0x562c1562a0d0 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a597fa700 10 client.27958 handle_command_reply: tid=0 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 client.27958 shutdown 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 msgr2=0x562c1561a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c099f70 tx=0x7f5a6c09a4d0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 shutdown_connections 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c15617f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560f2d0 0x562c1561a540 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1562a0d0 0x562c1562c4e0 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/1258449432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560fbe0 0x562c15617830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a597fa700 0 client.27958 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 >> 172.21.15.5:0/1258449432 conn(0x562c15605da0 msgr2=0x562c1560b290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 shutdown_connections 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 1 -- 172.21.15.5:0/1258449432 wait complete. 2023-07-21T19:21:16.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.417+0000 7f5a8b704b80 20 client.27958 trim_cache size 0 max 16384 2023-07-21T19:21:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.443+0000 7f5a8b704b80 1 Processor -- start 2023-07-21T19:21:16.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 -- start start 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c1561a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560fbe0 0x562c15617060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15618da0 con 0x562c1560f2d0 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15602f80 con 0x562c1561aad0 2023-07-21T19:21:16.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c15626e20 con 0x562c1560fbe0 2023-07-21T19:21:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a75b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c1561a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a7639c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a7639c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57292/0 (socket says 172.21.15.5:57292) 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.444+0000 7f5a7639c700 1 -- 172.21.15.5:0/4292572056 learned_addr learned my addr 172.21.15.5:0/4292572056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7539a700 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560fbe0 0x562c15617060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 -- 172.21.15.5:0/4292572056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560fbe0 msgr2=0x562c15617060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560fbe0 0x562c15617060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 -- 172.21.15.5:0/4292572056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 msgr2=0x562c1561a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c1561a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 -- 172.21.15.5:0/4292572056 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562c153b6250 con 0x562c1561aad0 2023-07-21T19:21:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.445+0000 7f5a7639c700 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a680074f0 tx=0x7f5a68009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e61c3f22f89d78b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.446+0000 7f5a64bff700 1 -- 172.21.15.5:0/4292572056 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e040 con 0x562c1561aad0 2023-07-21T19:21:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.446+0000 7f5a64bff700 1 -- 172.21.15.5:0/4292572056 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a6800f040 con 0x562c1561aad0 2023-07-21T19:21:16.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.446+0000 7f5a64bff700 1 -- 172.21.15.5:0/4292572056 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a680086b0 con 0x562c1561aad0 2023-07-21T19:21:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.446+0000 7f5a8b704b80 1 -- 172.21.15.5:0/4292572056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 msgr2=0x562c156175a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.446+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f5a680074f0 tx=0x7f5a68009920 comp rx=0 tx=0).stop 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 -- 172.21.15.5:0/4292572056 shutdown_connections 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1561aad0 0x562c156175a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c1561a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/4292572056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1560fbe0 0x562c15617060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 -- 172.21.15.5:0/4292572056 >> 172.21.15.5:0/4292572056 conn(0x562c15605da0 msgr2=0x562c15612bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 -- 172.21.15.5:0/4292572056 shutdown_connections 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 -- 172.21.15.5:0/4292572056 wait complete. 2023-07-21T19:21:16.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 -- start start 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.447+0000 7f5a8b704b80 1 Processor -- start 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c156216a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561fad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c156208d0 con 0x562c1560f2d0 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1567df60 con 0x562c1560fbe0 2023-07-21T19:21:16.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562c1567e0a0 con 0x562c1561aad0 2023-07-21T19:21:16.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a8b704b80 10 client.0 _ll_register_callbacks cb 0x7f5a6c0ca600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:16.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.448+0000 7f5a7639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39656/0 (socket says 172.21.15.5:39656) 2023-07-21T19:21:16.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 -- 172.21.15.5:0/755940760 learned_addr learned my addr 172.21.15.5:0/755940760 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:16.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7539a700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c156216a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 -- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 msgr2=0x562c1561fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 -- 172.21.15.5:0/755940760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 msgr2=0x562c156216a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:16.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c156216a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a7639c700 1 -- 172.21.15.5:0/755940760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a68009250 con 0x562c1561aad0 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.449+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c156216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.450+0000 7f5a7639c700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a68020e80 tx=0x7f5a68004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47531d1025e6e662 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.450+0000 7f5a597fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.450+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800e050 con 0x562c1561aad0 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.450+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5a68020880 con 0x562c1561aad0 2023-07-21T19:21:16.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.450+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5a6c0a2430 con 0x562c1561aad0 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5a68024040 con 0x562c1561aad0 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 10 client.27964 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x562c1538e2b0 con 0x562c1561aad0 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5a6c0a2430 con 0x562c1561aad0 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5a68005910 con 0x562c1561aad0 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 10 client.27964 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 10 client.27964 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:16.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 20 client.27964 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1567f6d0 0x562c15681ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 4 client.27964 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.451+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562c1551d050 con 0x562c1567f6d0 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.452+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1567f6d0 0x562c15681ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.452+0000 7f5a75b9b700 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1567f6d0 0x562c15681ae0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6cdf4c13bcb93bb server_cookie=bdbb010989993ab3 in_seq=0 out_seq=0 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.452+0000 7f5a597fa700 10 client.27964 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a597fa700 1 -- 172.21.15.5:0/755940760 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x562c1551d050 con 0x562c1567f6d0 2023-07-21T19:21:16.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a597fa700 10 client.27964 handle_command_reply: tid=0 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 client.27964 shutdown 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 msgr2=0x562c156201c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5a68020e80 tx=0x7f5a68004db0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 shutdown_connections 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562c1561aad0 0x562c156201c0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562c1567f6d0 0x562c15681ae0 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562c1560f2d0 0x562c156216a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 --2- 172.21.15.5:0/755940760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562c1560fbe0 0x562c1561fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 >> 172.21.15.5:0/755940760 conn(0x562c15605da0 msgr2=0x562c156113e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.453+0000 7f5a597fa700 0 client.27964 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:16.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.454+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 shutdown_connections 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.454+0000 7f5a8b704b80 1 -- 172.21.15.5:0/755940760 wait complete. 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.454+0000 7f5a8b704b80 20 client.27964 trim_cache size 0 max 16384 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:21:16.453 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:21:16.454 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:21:16.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 msgr2=0x7f5a5c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 0x7f5a5c0a3770 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a70199420 tx=0x7f5a6000b260 comp rx=0 tx=0).stop 2023-07-21T19:21:16.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 msgr2=0x7f5a70197e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:16.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5a6c00e9a0 tx=0x7f5a6c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 shutdown_connections 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5a7010a040 0x7f5a7019c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5a70100d00 0x7f5a70197e10 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5a5c0a12c0 0x7f5a5c0a3770 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 --2- 172.21.15.5:0/3119355187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5a70101610 0x7f5a70198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 >> 172.21.15.5:0/3119355187 conn(0x7f5a700fa4b0 msgr2=0x7f5a7010c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 shutdown_connections 2023-07-21T19:21:16.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:16.455+0000 7f5a77dff700 1 -- 172.21.15.5:0/3119355187 wait complete. 2023-07-21T19:21:16.469 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:21:17.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:17 smithi101 ceph-mon[128136]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-07-21T19:21:17.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:17 smithi005 ceph-mon[126460]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-07-21T19:21:17.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:17 smithi156 ceph-mon[128316]: pgmap v429: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 30 MiB/s wr, 31 op/s 2023-07-21T19:21:19.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:19 smithi101 ceph-mon[128136]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:19.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:19 smithi005 ceph-mon[126460]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:19.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:19 smithi156 ceph-mon[128316]: pgmap v430: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:20.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 -- 172.21.15.5:0/1901037921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 msgr2=0x7f7e941068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 --2- 172.21.15.5:0/1901037921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e941068d0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f7e88009920 tx=0x7f7e88009c30 comp rx=0 tx=0).stop 2023-07-21T19:21:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 -- 172.21.15.5:0/1901037921 shutdown_connections 2023-07-21T19:21:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 --2- 172.21.15.5:0/1901037921 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94106fb0 0x7f7e9410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 --2- 172.21.15.5:0/1901037921 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94101f70 0x7f7e94102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 --2- 172.21.15.5:0/1901037921 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e941068d0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 -- 172.21.15.5:0/1901037921 >> 172.21.15.5:0/1901037921 conn(0x7f7e940fd7e0 msgr2=0x7f7e940ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 -- 172.21.15.5:0/1901037921 shutdown_connections 2023-07-21T19:21:20.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.282+0000 7f7e989ff700 1 -- 172.21.15.5:0/1901037921 wait complete. 2023-07-21T19:21:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.283+0000 7f7e989ff700 1 Processor -- start 2023-07-21T19:21:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.283+0000 7f7e989ff700 1 -- start start 2023-07-21T19:21:20.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e94198380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94106fb0 0x7f7e9419c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419d030 con 0x7f7e94102910 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419d1a0 con 0x7f7e94106fb0 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e989ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e9419d360 con 0x7f7e94101f70 2023-07-21T19:21:20.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39676/0 (socket says 172.21.15.5:39676) 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e91d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e94198380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 -- 172.21.15.5:0/2151205717 learned_addr learned my addr 172.21.15.5:0/2151205717 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94106fb0 0x7f7e9419c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 -- 172.21.15.5:0/2151205717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94106fb0 msgr2=0x7f7e9419c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94106fb0 0x7f7e9419c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 -- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 msgr2=0x7f7e94198380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.284+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e94198380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e9259c700 1 -- 172.21.15.5:0/2151205717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840095f0 con 0x7f7e94101f70 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400ec10 tx=0x7f7e8400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e837fe700 1 -- 172.21.15.5:0/2151205717 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8400cc90 con 0x7f7e94101f70 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e88009520 con 0x7f7e94101f70 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e837fe700 1 -- 172.21.15.5:0/2151205717 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8400cdf0 con 0x7f7e94101f70 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e837fe700 1 -- 172.21.15.5:0/2151205717 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e84003e80 con 0x7f7e94101f70 2023-07-21T19:21:20.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.285+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e941a4ca0 con 0x7f7e94101f70 2023-07-21T19:21:20.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.289+0000 7f7e837fe700 1 -- 172.21.15.5:0/2151205717 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7e84004000 con 0x7f7e94101f70 2023-07-21T19:21:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.290+0000 7f7eac289b80 1 Processor -- start 2023-07-21T19:21:20.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 -- start start 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c10d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c0b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6b10c50 con 0x55f2a6c112d0 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6b639e0 con 0x55f2a6a676b0 2023-07-21T19:21:20.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7eac289b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6b5ce70 con 0x55f2a6c0b580 2023-07-21T19:21:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7e837fe700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 0x7f7e7c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c10d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c10d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57328/0 (socket says 172.21.15.5:57328) 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.291+0000 7f7e92d9d700 1 -- 172.21.15.5:0/1694908949 learned_addr learned my addr 172.21.15.5:0/1694908949 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e91d9b700 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c0b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e837fe700 1 -- 172.21.15.5:0/2151205717 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7e84014070 con 0x7f7e94101f70 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 0x7f7e7c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 -- 172.21.15.5:0/1694908949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 msgr2=0x55f2a6c0b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c0b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 -- 172.21.15.5:0/1694908949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 msgr2=0x55f2a6c10d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c10d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 -- 172.21.15.5:0/1694908949 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a698ad80 con 0x55f2a6c112d0 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.292+0000 7f7e9259c700 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0b040 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7e840ca800 tx=0x7f7e8400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef06d7401823bfd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.293+0000 7f7e80ff9700 1 -- 172.21.15.5:0/1694908949 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8409a180 con 0x55f2a6c112d0 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.293+0000 7f7e80ff9700 1 -- 172.21.15.5:0/1694908949 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8409e1b0 con 0x55f2a6c112d0 2023-07-21T19:21:20.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.293+0000 7f7e80ff9700 1 -- 172.21.15.5:0/1694908949 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8409a3e0 con 0x55f2a6c112d0 2023-07-21T19:21:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 -- 172.21.15.5:0/1694908949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 msgr2=0x55f2a6c0b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0b040 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f7e840ca800 tx=0x7f7e8400cf60 comp rx=0 tx=0).stop 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 0x7f7e7c0a37f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00fb80 tx=0x7f7e8c004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 -- 172.21.15.5:0/1694908949 shutdown_connections 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c0b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c10d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 --2- 172.21.15.5:0/1694908949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0b040 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 -- 172.21.15.5:0/1694908949 >> 172.21.15.5:0/1694908949 conn(0x55f2a6c00be0 msgr2=0x55f2a6c05f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 -- 172.21.15.5:0/1694908949 shutdown_connections 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.294+0000 7f7eac289b80 1 -- 172.21.15.5:0/1694908949 wait complete. 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 -- start start 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 Processor -- start 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c0b580 0x55f2a6c14910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6afca60 con 0x55f2a6c0b580 2023-07-21T19:21:20.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.295+0000 7f7eac289b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6afc760 con 0x55f2a6c112d0 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7eac289b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a69e08c0 con 0x55f2a6a676b0 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7eac289b80 10 client.0 _ll_register_callbacks cb 0x55f2a6afa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39692/0 (socket says 172.21.15.5:39692) 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e9259c700 1 -- 172.21.15.5:0/2076097455 learned_addr learned my addr 172.21.15.5:0/2076097455 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:20.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57330/0 (socket says 172.21.15.5:57330) 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e91d9b700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c0b580 0x55f2a6c14910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2076097455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 msgr2=0x55f2a6c17620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2076097455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c0b580 msgr2=0x55f2a6c14910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c0b580 0x55f2a6c14910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2076097455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e941a4ca0 con 0x55f2a6c112d0 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.296+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:20.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c018640 tx=0x7f7e8c004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32e46c8b199338ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7e79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:21:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00e040 con 0x55f2a6c112d0 2023-07-21T19:21:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2a6ab6490 con 0x55f2a6c112d0 2023-07-21T19:21:20.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7e8c003e10 con 0x55f2a6c112d0 2023-07-21T19:21:20.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c004bb0 con 0x55f2a6c112d0 2023-07-21T19:21:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7eac289b80 10 client.28052 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.297+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55f2a6973c00 con 0x55f2a6c112d0 2023-07-21T19:21:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8c031080 con 0x55f2a6c112d0 2023-07-21T19:21:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e8c004150 con 0x55f2a6c112d0 2023-07-21T19:21:20.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 10 client.28052 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 10 client.28052 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 20 client.28052 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c24c70 0x55f2a6c27080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 4 client.28052 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2a6b55e50 con 0x55f2a6c24c70 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.298+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c24c70 0x55f2a6c27080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.299+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c24c70 0x55f2a6c27080 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2cad5f97a234b35 server_cookie=4897cf37aaec9d9a in_seq=0 out_seq=0 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.299+0000 7f7e79ffb700 10 client.28052 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.299+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2076097455 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f2a6b55e50 con 0x55f2a6c24c70 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.299+0000 7f7e79ffb700 10 client.28052 handle_command_reply: tid=0 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 client.28052 shutdown 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 msgr2=0x55f2a6c15000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c018640 tx=0x7f7e8c004ad0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 shutdown_connections 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c112d0 0x55f2a6c15000 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c17620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c24c70 0x55f2a6c27080 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2076097455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c0b580 0x55f2a6c14910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7e79ffb700 0 client.28052 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 >> 172.21.15.5:0/2076097455 conn(0x55f2a6c00be0 msgr2=0x55f2a6c054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 shutdown_connections 2023-07-21T19:21:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 1 -- 172.21.15.5:0/2076097455 wait complete. 2023-07-21T19:21:20.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.300+0000 7f7eac289b80 20 client.28052 trim_cache size 0 max 16384 2023-07-21T19:21:20.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.318+0000 7f7eac289b80 1 Processor -- start 2023-07-21T19:21:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.318+0000 7f7eac289b80 1 -- start start 2023-07-21T19:21:20.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c0b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c07010 con 0x55f2a6c112d0 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c15390 con 0x55f2a6a676b0 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7eac289b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c109c0 con 0x55f2a6c0b580 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c0b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39698/0 (socket says 172.21.15.5:39698) 2023-07-21T19:21:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e9259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c0b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57334/0 (socket says 172.21.15.5:57334) 2023-07-21T19:21:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 -- 172.21.15.5:0/127740209 learned_addr learned my addr 172.21.15.5:0/127740209 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:20.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 -- 172.21.15.5:0/127740209 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 msgr2=0x55f2a6c0b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c0b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 -- 172.21.15.5:0/127740209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 msgr2=0x55f2a6c0eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e91d9b700 1 -- 172.21.15.5:0/127740209 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a6c02420 con 0x55f2a6c0b580 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.319+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.320+0000 7f7e91d9b700 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e88016000 tx=0x7f7e8800fba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af73906eda534e3f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.320+0000 7f7e817fa700 1 -- 172.21.15.5:0/127740209 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8800f720 con 0x55f2a6c0b580 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.320+0000 7f7e817fa700 1 -- 172.21.15.5:0/127740209 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8801d040 con 0x55f2a6c0b580 2023-07-21T19:21:20.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.320+0000 7f7e817fa700 1 -- 172.21.15.5:0/127740209 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e880054a0 con 0x55f2a6c0b580 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- 172.21.15.5:0/127740209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 msgr2=0x55f2a6c05c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7e88016000 tx=0x7f7e8800fba0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- 172.21.15.5:0/127740209 shutdown_connections 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c0eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6a676b0 0x55f2a6c0b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 --2- 172.21.15.5:0/127740209 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6c0b580 0x55f2a6c05c60 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- 172.21.15.5:0/127740209 >> 172.21.15.5:0/127740209 conn(0x55f2a6c00be0 msgr2=0x55f2a6c17250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- 172.21.15.5:0/127740209 shutdown_connections 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- 172.21.15.5:0/127740209 wait complete. 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.321+0000 7f7eac289b80 1 -- start start 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 Processor -- start 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c19e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 0x55f2a6c1a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c1aa40 con 0x55f2a6c112d0 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c78070 con 0x55f2a6c0b580 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a6c781b0 con 0x55f2a6a676b0 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7eac289b80 10 client.0 _ll_register_callbacks cb 0x7f7e8c018920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.322+0000 7f7e91d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 0x55f2a6c1a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53526/0 (socket says 172.21.15.5:53526) 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2577477251 learned_addr learned my addr 172.21.15.5:0/2577477251 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:20.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e91d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 0x55f2a6c1a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57346/0 (socket says 172.21.15.5:57346) 2023-07-21T19:21:20.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c19e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2577477251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 msgr2=0x55f2a6c19e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c19e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 msgr2=0x55f2a6c1a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:20.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 0x55f2a6c1a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 -- 172.21.15.5:0/2577477251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2a6973c00 con 0x55f2a6c112d0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e92d9d700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004480 tx=0x7f7e8c004950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba00095bc900d12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.323+0000 7f7e79ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e8c00e040 con 0x55f2a6c112d0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7e8409e1b0 con 0x55f2a6c112d0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7e8c007a90 con 0x55f2a6c112d0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e8c006000 con 0x55f2a6c112d0 2023-07-21T19:21:20.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 10 client.18405 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7f7e9410f3c0 con 0x55f2a6c112d0 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7e8409e1b0 con 0x55f2a6c112d0 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7e8c036030 con 0x55f2a6c112d0 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 10 client.18405 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 10 client.18405 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 20 client.18405 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.324+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c79770 0x55f2a6c7bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:20.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.325+0000 7f7eac289b80 4 client.18405 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:21:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.325+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2a6b17ff0 con 0x55f2a6c79770 2023-07-21T19:21:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.325+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c79770 0x55f2a6c7bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.325+0000 7f7e9259c700 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c79770 0x55f2a6c7bb80 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14e9efa3414046de server_cookie=70ddd03721ac10ea in_seq=0 out_seq=0 2023-07-21T19:21:20.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.325+0000 7f7e79ffb700 10 client.18405 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7e79ffb700 1 -- 172.21.15.5:0/2577477251 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2a6b17ff0 con 0x55f2a6c79770 2023-07-21T19:21:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7e79ffb700 10 client.18405 handle_command_reply: tid=0 2023-07-21T19:21:20.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 client.18405 shutdown 2023-07-21T19:21:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 msgr2=0x55f2a6c77b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c004480 tx=0x7f7e8c004950 comp rx=0 tx=0).stop 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 shutdown_connections 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a6c79770 0x55f2a6c7bb80 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a6c112d0 0x55f2a6c77b30 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a6a676b0 0x55f2a6c19e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 --2- 172.21.15.5:0/2577477251 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a6c0b580 0x55f2a6c1a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7e79ffb700 0 client.18405 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 >> 172.21.15.5:0/2577477251 conn(0x55f2a6c00be0 msgr2=0x55f2a6c0fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 shutdown_connections 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 1 -- 172.21.15.5:0/2577477251 wait complete. 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.339+0000 7f7eac289b80 20 client.18405 trim_cache size 0 max 16384 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:21:20.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:21:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:21:20.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 msgr2=0x7f7e7c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 0x7f7e7c0a37f0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7e8c00fb80 tx=0x7f7e8c004b40 comp rx=0 tx=0).stop 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 msgr2=0x7f7e94197e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7e8400ec10 tx=0x7f7e8400ef20 comp rx=0 tx=0).stop 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 shutdown_connections 2023-07-21T19:21:20.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7e94106fb0 0x7f7e9419c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7e94101f70 0x7f7e94197e40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7e7c0a1340 0x7f7e7c0a37f0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 --2- 172.21.15.5:0/2151205717 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7e94102910 0x7f7e94198380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:20.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 >> 172.21.15.5:0/2151205717 conn(0x7f7e940fd7e0 msgr2=0x7f7e940fe3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 shutdown_connections 2023-07-21T19:21:20.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:20.340+0000 7f7e989ff700 1 -- 172.21.15.5:0/2151205717 wait complete. 2023-07-21T19:21:21.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:21 smithi101 ceph-mon[128136]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:21.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:21 smithi005 ceph-mon[126460]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:21.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:21 smithi156 ceph-mon[128316]: pgmap v431: 97 pgs: 97 active+clean; 10 GiB data, 4.6 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 29 MiB/s wr, 21 op/s 2023-07-21T19:21:23.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:23 smithi101 ceph-mon[128136]: pgmap v432: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 22 op/s 2023-07-21T19:21:23.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:23 smithi005 ceph-mon[126460]: pgmap v432: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 22 op/s 2023-07-21T19:21:23.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:23 smithi156 ceph-mon[128316]: pgmap v432: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 28 MiB/s wr, 22 op/s 2023-07-21T19:21:25.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:25 smithi101 ceph-mon[128136]: pgmap v433: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:25.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:25 smithi005 ceph-mon[126460]: pgmap v433: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:25.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:25 smithi156 ceph-mon[128316]: pgmap v433: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:27.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:27 smithi101 ceph-mon[128136]: pgmap v434: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:27.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:27 smithi005 ceph-mon[126460]: pgmap v434: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:27.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:27 smithi156 ceph-mon[128316]: pgmap v434: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:29.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:29 smithi101 ceph-mon[128136]: pgmap v435: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:29.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:29 smithi005 ceph-mon[126460]: pgmap v435: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:29.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:29 smithi156 ceph-mon[128316]: pgmap v435: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:31.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:31 smithi101 ceph-mon[128136]: pgmap v436: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:31.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:31 smithi005 ceph-mon[126460]: pgmap v436: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:31.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:31 smithi156 ceph-mon[128316]: pgmap v436: 97 pgs: 97 active+clean; 11 GiB data, 4.7 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 20 op/s 2023-07-21T19:21:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.509+0000 7f27727ff700 1 -- 172.21.15.5:0/2063662810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c0685b0 msgr2=0x7f276c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.509+0000 7f27727ff700 1 --2- 172.21.15.5:0/2063662810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c0685b0 0x7f276c104770 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f27600099c0 tx=0x7f2760009cd0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 -- 172.21.15.5:0/2063662810 shutdown_connections 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 --2- 172.21.15.5:0/2063662810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c104e50 0x7f276c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 --2- 172.21.15.5:0/2063662810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c067c10 0x7f276c067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 --2- 172.21.15.5:0/2063662810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c0685b0 0x7f276c104770 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 -- 172.21.15.5:0/2063662810 >> 172.21.15.5:0/2063662810 conn(0x7f276c074760 msgr2=0x7f276c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 -- 172.21.15.5:0/2063662810 shutdown_connections 2023-07-21T19:21:33.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.510+0000 7f27727ff700 1 -- 172.21.15.5:0/2063662810 wait complete. 2023-07-21T19:21:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.511+0000 7f27727ff700 1 Processor -- start 2023-07-21T19:21:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.511+0000 7f27727ff700 1 -- start start 2023-07-21T19:21:33.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c0685b0 0x7f276c0ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c104e50 0x7f276c1a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c1a42e0 con 0x7f276c0685b0 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c1a4420 con 0x7f276c067c10 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f27727ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f276c1a4590 con 0x7f276c104e50 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f276bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c0685b0 0x7f276c0ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f276bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48282/0 (socket says 172.21.15.5:48282) 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.512+0000 7f276bfff700 1 -- 172.21.15.5:0/4093666247 learned_addr learned my addr 172.21.15.5:0/4093666247 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f2770d9c700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c104e50 0x7f276c1a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 -- 172.21.15.5:0/4093666247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c104e50 msgr2=0x7f276c1a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c104e50 0x7f276c1a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 -- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c0685b0 msgr2=0x7f276c0ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c0685b0 0x7f276c0ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 -- 172.21.15.5:0/4093666247 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27600095f0 con 0x7f276c067c10 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f276bfff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f275c00e9a0 tx=0x7f275c00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f27697fa700 1 -- 172.21.15.5:0/4093666247 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c009790 con 0x7f276c067c10 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f27697fa700 1 -- 172.21.15.5:0/4093666247 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f275c00cbd0 con 0x7f276c067c10 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f276c1a4870 con 0x7f276c067c10 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.513+0000 7f27697fa700 1 -- 172.21.15.5:0/4093666247 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c018680 con 0x7f276c067c10 2023-07-21T19:21:33.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.514+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f276c1a4d50 con 0x7f276c067c10 2023-07-21T19:21:33.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.517+0000 7f27697fa700 1 -- 172.21.15.5:0/4093666247 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f275c00cd40 con 0x7f276c067c10 2023-07-21T19:21:33.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.518+0000 7f278614ab80 1 Processor -- start 2023-07-21T19:21:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.518+0000 7f278614ab80 1 -- start start 2023-07-21T19:21:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5ca9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5ca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5ca49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5ba9c50 con 0x55dda5caa2d0 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5bfc9e0 con 0x55dda5b006b0 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f278614ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5bf5e70 con 0x55dda5ca4580 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f27697fa700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 0x7f27540a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5ca49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5ca49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35430/0 (socket says 172.21.15.5:35430) 2023-07-21T19:21:33.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f276b7fe700 1 -- 172.21.15.5:0/2931351197 learned_addr learned my addr 172.21.15.5:0/2931351197 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f2770d9c700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5ca9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f2770d9c700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 0x7f27540a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.519+0000 7f27697fa700 1 -- 172.21.15.5:0/4093666247 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f275c014070 con 0x7f276c067c10 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 -- 172.21.15.5:0/2931351197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 msgr2=0x55dda5ca49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5ca49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 -- 172.21.15.5:0/2931351197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 msgr2=0x55dda5ca4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f276bfff700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5ca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5ca4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 -- 172.21.15.5:0/2931351197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda5a23d80 con 0x55dda5b006b0 2023-07-21T19:21:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f276bfff700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5ca4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:21:33.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.520+0000 7f2770d9c700 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5ca9d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2764014b80 tx=0x7f2764014e90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a9d4e6bb09a4c2c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f27527fc700 1 -- 172.21.15.5:0/2931351197 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f276400e040 con 0x55dda5b006b0 2023-07-21T19:21:33.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f2770d9c700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 0x7f27540a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2764010000 tx=0x7f276401e000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f27527fc700 1 -- 172.21.15.5:0/2931351197 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2764003a00 con 0x55dda5b006b0 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f27527fc700 1 -- 172.21.15.5:0/2931351197 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2764021960 con 0x55dda5b006b0 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f278614ab80 1 -- 172.21.15.5:0/2931351197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 msgr2=0x55dda5ca9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.521+0000 7f278614ab80 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5ca9d90 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f2764014b80 tx=0x7f2764014e90 comp rx=0 tx=0).stop 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 -- 172.21.15.5:0/2931351197 shutdown_connections 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5ca49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5ca9d90 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 --2- 172.21.15.5:0/2931351197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5ca4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 -- 172.21.15.5:0/2931351197 >> 172.21.15.5:0/2931351197 conn(0x55dda5c99be0 msgr2=0x55dda5c9ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 -- 172.21.15.5:0/2931351197 shutdown_connections 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 -- 172.21.15.5:0/2931351197 wait complete. 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.522+0000 7f278614ab80 1 -- start start 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 Processor -- start 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5cb36c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5cad980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5b95a60 con 0x55dda5caa2d0 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5b95760 con 0x55dda5b006b0 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5a798c0 con 0x55dda5ca4580 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f278614ab80 10 client.0 _ll_register_callbacks cb 0x55dda5b93700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35432/0 (socket says 172.21.15.5:35432) 2023-07-21T19:21:33.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f276b7fe700 1 -- 172.21.15.5:0/3502843775 learned_addr learned my addr 172.21.15.5:0/3502843775 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f276bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5cb36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.523+0000 7f2770d9c700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5cad980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 -- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 msgr2=0x55dda5cb36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5cb36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 -- 172.21.15.5:0/3502843775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 msgr2=0x55dda5cad980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5cad980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 -- 172.21.15.5:0/3502843775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2764014730 con 0x55dda5ca4580 2023-07-21T19:21:33.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276bfff700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5cb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f276b7fe700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f276c068980 tx=0x7f2760005be0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d96363a532bbedd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.524+0000 7f274b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760005590 con 0x55dda5ca4580 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dda5b4f490 con 0x55dda5ca4580 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2760003760 con 0x55dda5ca4580 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f27600056f0 con 0x55dda5ca4580 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f278614ab80 10 client.27994 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55dda5a0cc00 con 0x55dda5ca4580 2023-07-21T19:21:33.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.525+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2760031020 con 0x55dda5ca4580 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f276000fad0 con 0x55dda5ca4580 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 10 client.27994 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 10 client.27994 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 20 client.27994 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5cbdae0 0x55dda5cbfef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 4 client.27994 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:33.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dda5beee50 con 0x55dda5cbdae0 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.526+0000 7f276bfff700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5cbdae0 0x55dda5cbfef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.527+0000 7f274b7fe700 10 client.27994 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.527+0000 7f276bfff700 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5cbdae0 0x55dda5cbfef0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b31c054b1ce29a7 server_cookie=bda00505b2d6556f in_seq=0 out_seq=0 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.527+0000 7f274b7fe700 1 -- 172.21.15.5:0/3502843775 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dda5beee50 con 0x55dda5cbdae0 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.527+0000 7f274b7fe700 10 client.27994 handle_command_reply: tid=0 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.527+0000 7f278614ab80 1 client.27994 shutdown 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 msgr2=0x55dda5cb3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f276c068980 tx=0x7f2760005be0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 shutdown_connections 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5cbdae0 0x55dda5cbfef0 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5cad980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5b006b0 0x55dda5cb36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 --2- 172.21.15.5:0/3502843775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5ca4580 0x55dda5cb3c00 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f274b7fe700 0 client.27994 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 >> 172.21.15.5:0/3502843775 conn(0x55dda5c99be0 msgr2=0x55dda5c9e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 shutdown_connections 2023-07-21T19:21:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 1 -- 172.21.15.5:0/3502843775 wait complete. 2023-07-21T19:21:33.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.528+0000 7f278614ab80 20 client.27994 trim_cache size 0 max 16384 2023-07-21T19:21:33.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.545+0000 7f278614ab80 1 Processor -- start 2023-07-21T19:21:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.545+0000 7f278614ab80 1 -- start start 2023-07-21T19:21:33.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5ca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5c9ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5ca8760 con 0x55dda5caa2d0 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5cae850 con 0x55dda5ca4580 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f278614ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5cba4b0 con 0x55dda5b006b0 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f276b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48316/0 (socket says 172.21.15.5:48316) 2023-07-21T19:21:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f2770d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5c9ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.546+0000 7f276b7fe700 1 -- 172.21.15.5:0/2545455732 learned_addr learned my addr 172.21.15.5:0/2545455732 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 -- 172.21.15.5:0/2545455732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 msgr2=0x55dda5ca4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5ca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 -- 172.21.15.5:0/2545455732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 msgr2=0x55dda5c9ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5c9ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 -- 172.21.15.5:0/2545455732 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dda5cadc10 con 0x55dda5ca4580 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f276000b580 tx=0x7f2760009cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=241d703a00f0c71c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f2752ffd700 1 -- 172.21.15.5:0/2545455732 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760017490 con 0x55dda5ca4580 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f2752ffd700 1 -- 172.21.15.5:0/2545455732 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2760031030 con 0x55dda5ca4580 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.547+0000 7f2752ffd700 1 -- 172.21.15.5:0/2545455732 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2760017c20 con 0x55dda5ca4580 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 -- 172.21.15.5:0/2545455732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 msgr2=0x55dda5cae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f276000b580 tx=0x7f2760009cd0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 -- 172.21.15.5:0/2545455732 shutdown_connections 2023-07-21T19:21:33.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5caa2d0 0x55dda5c9ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5ca4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 --2- 172.21.15.5:0/2545455732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5ca4580 0x55dda5cae310 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 -- 172.21.15.5:0/2545455732 >> 172.21.15.5:0/2545455732 conn(0x55dda5c99be0 msgr2=0x55dda5cc4850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 -- 172.21.15.5:0/2545455732 shutdown_connections 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.548+0000 7f278614ab80 1 -- 172.21.15.5:0/2545455732 wait complete. 2023-07-21T19:21:33.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.549+0000 7f278614ab80 1 -- start start 2023-07-21T19:21:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.549+0000 7f278614ab80 1 Processor -- start 2023-07-21T19:21:33.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5ca4580 0x55dda5cb7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5caa2d0 0x55dda5cb7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5d109a0 con 0x55dda5ca4580 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5d10ae0 con 0x55dda5caa2d0 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dda5d10c20 con 0x55dda5b006b0 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f278614ab80 10 client.0 _ll_register_callbacks cb 0x7f276000baa0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35450/0 (socket says 172.21.15.5:35450) 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 -- 172.21.15.5:0/2002699536 learned_addr learned my addr 172.21.15.5:0/2002699536 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5ca4580 0x55dda5cb7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f2770d9c700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5caa2d0 0x55dda5cb7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 -- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5caa2d0 msgr2=0x55dda5cb7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5caa2d0 0x55dda5cb7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 -- 172.21.15.5:0/2002699536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5ca4580 msgr2=0x55dda5cb7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5ca4580 0x55dda5cb7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.550+0000 7f276bfff700 1 -- 172.21.15.5:0/2002699536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dda5a0cc00 con 0x55dda5b006b0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f276bfff700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f275c09a020 tx=0x7f275c0c2620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e525d57b193163bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f274b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f275c09a4f0 con 0x55dda5b006b0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f275c09a090 con 0x55dda5b006b0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2764003a00 con 0x55dda5b006b0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7f275c09e0c0 con 0x55dda5b006b0 2023-07-21T19:21:33.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f278614ab80 10 client.27997 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55dda5a4a1f0 con 0x55dda5b006b0 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.551+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f275c0c8020 con 0x55dda5b006b0 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f275c099610 con 0x55dda5b006b0 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 10 client.27997 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 10 client.27997 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 20 client.27997 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5d122e0 0x55dda5d146f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:33.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 4 client.27997 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:21:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55dda5bb0ff0 con 0x55dda5d122e0 2023-07-21T19:21:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.552+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5d122e0 0x55dda5d146f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.553+0000 7f276b7fe700 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5d122e0 0x55dda5d146f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5dc4bbbac718e05 server_cookie=32cd5eddc6e9bd95 in_seq=0 out_seq=0 2023-07-21T19:21:33.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.553+0000 7f274b7fe700 10 client.27997 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.565+0000 7f274b7fe700 1 -- 172.21.15.5:0/2002699536 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55dda5bb0ff0 con 0x55dda5d122e0 2023-07-21T19:21:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.565+0000 7f274b7fe700 10 client.27997 handle_command_reply: tid=0 2023-07-21T19:21:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.565+0000 7f278614ab80 1 client.27997 shutdown 2023-07-21T19:21:33.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 msgr2=0x55dda5cb72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f275c09a020 tx=0x7f275c0c2620 comp rx=0 tx=0).stop 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 shutdown_connections 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dda5caa2d0 0x55dda5cb7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dda5b006b0 0x55dda5cb72c0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55dda5d122e0 0x55dda5d146f0 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 --2- 172.21.15.5:0/2002699536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dda5ca4580 0x55dda5cb7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f274b7fe700 0 client.27997 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:33.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 >> 172.21.15.5:0/2002699536 conn(0x55dda5c99be0 msgr2=0x55dda5caaa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 shutdown_connections 2023-07-21T19:21:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 1 -- 172.21.15.5:0/2002699536 wait complete. 2023-07-21T19:21:33.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.566+0000 7f278614ab80 20 client.27997 trim_cache size 0 max 16384 2023-07-21T19:21:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:21:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:21:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:21:33.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 msgr2=0x7f27540a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 0x7f27540a3770 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2764010000 tx=0x7f276401e000 comp rx=0 tx=0).stop 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 msgr2=0x7f276c0fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f275c00e9a0 tx=0x7f275c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 shutdown_connections 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f276c104e50 0x7f276c1a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f276c067c10 0x7f276c0fec40 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f27540a12c0 0x7f27540a3770 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 --2- 172.21.15.5:0/4093666247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f276c0685b0 0x7f276c0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 >> 172.21.15.5:0/4093666247 conn(0x7f276c074760 msgr2=0x7f276c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 shutdown_connections 2023-07-21T19:21:33.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:33.567+0000 7f27727ff700 1 -- 172.21.15.5:0/4093666247 wait complete. 2023-07-21T19:21:33.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:33 smithi101 ceph-mon[128136]: pgmap v437: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-07-21T19:21:33.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:33 smithi005 ceph-mon[126460]: pgmap v437: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-07-21T19:21:33.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:33 smithi156 ceph-mon[128316]: pgmap v437: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 MiB/s wr, 27 op/s 2023-07-21T19:21:35.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:35 smithi101 ceph-mon[128136]: pgmap v438: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:35.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:35 smithi005 ceph-mon[126460]: pgmap v438: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:35.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:35 smithi156 ceph-mon[128316]: pgmap v438: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:37.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:37 smithi101 ceph-mon[128136]: pgmap v439: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:21:37.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:37 smithi005 ceph-mon[126460]: pgmap v439: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:21:37.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:37 smithi156 ceph-mon[128316]: pgmap v439: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:21:39.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:39 smithi101 ceph-mon[128136]: pgmap v440: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:21:39.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:39 smithi005 ceph-mon[126460]: pgmap v440: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:21:39.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:39 smithi156 ceph-mon[128316]: pgmap v440: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:21:41.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:41 smithi101 ceph-mon[128136]: pgmap v441: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:41.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:41 smithi005 ceph-mon[126460]: pgmap v441: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:41.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:41 smithi156 ceph-mon[128316]: pgmap v441: 97 pgs: 97 active+clean; 11 GiB data, 4.8 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:43.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:43 smithi101 ceph-mon[128136]: pgmap v442: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:43.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:43 smithi005 ceph-mon[126460]: pgmap v442: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:43.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:43 smithi156 ceph-mon[128316]: pgmap v442: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:45.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:45 smithi101 ceph-mon[128136]: pgmap v443: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:21:45.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:45 smithi005 ceph-mon[126460]: pgmap v443: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:21:45.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:45 smithi156 ceph-mon[128316]: pgmap v443: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:21:46.470 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:21:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2604811758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 msgr2=0x7fdbe00fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2604811758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe00fff50 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0009990 tx=0x7fdbd0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2604811758 shutdown_connections 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2604811758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0100490 0x7fdbe010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2604811758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0105af0 0x7fdbe0105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2604811758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe00fff50 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2604811758 >> 172.21.15.5:0/2604811758 conn(0x7fdbe00fb5e0 msgr2=0x7fdbe00fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2604811758 shutdown_connections 2023-07-21T19:21:46.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.745+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2604811758 wait complete. 2023-07-21T19:21:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.746+0000 7fdbe83ff700 1 Processor -- start 2023-07-21T19:21:46.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.746+0000 7fdbe83ff700 1 -- start start 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0100490 0x7fdbe0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0105af0 0x7fdbe0073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe01a3f90 con 0x7fdbe0105af0 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe01a4100 con 0x7fdbe00ffae0 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe83ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe01a42c0 con 0x7fdbe0100490 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe619b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45898/0 (socket says 172.21.15.5:45898) 2023-07-21T19:21:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe619b700 1 -- 172.21.15.5:0/2063138020 learned_addr learned my addr 172.21.15.5:0/2063138020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0105af0 0x7fdbe0073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.747+0000 7fdbe599a700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0100490 0x7fdbe0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 -- 172.21.15.5:0/2063138020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0100490 msgr2=0x7fdbe0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0100490 0x7fdbe0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 -- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0105af0 msgr2=0x7fdbe0073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0105af0 0x7fdbe0073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 -- 172.21.15.5:0/2063138020 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbd00095f0 con 0x7fdbe00ffae0 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.748+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00e9a0 tx=0x7fdbdc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.749+0000 7fdbd77fe700 1 -- 172.21.15.5:0/2063138020 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc009790 con 0x7fdbe00ffae0 2023-07-21T19:21:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.749+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbe01a45a0 con 0x7fdbe00ffae0 2023-07-21T19:21:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.749+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe01a4b00 con 0x7fdbe00ffae0 2023-07-21T19:21:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.749+0000 7fdbd77fe700 1 -- 172.21.15.5:0/2063138020 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbdc010430 con 0x7fdbe00ffae0 2023-07-21T19:21:46.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.749+0000 7fdbd77fe700 1 -- 172.21.15.5:0/2063138020 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00cea0 con 0x7fdbe00ffae0 2023-07-21T19:21:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 -- 172.21.15.5:0/596874998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 msgr2=0x7fc7dc10ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/596874998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc10ca00 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009fc0 tx=0x7fc7d400b330 comp rx=0 tx=0).stop 2023-07-21T19:21:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 -- 172.21.15.5:0/596874998 shutdown_connections 2023-07-21T19:21:46.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/596874998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc10ca00 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/596874998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 0x7fc7dc067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/596874998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 -- 172.21.15.5:0/596874998 >> 172.21.15.5:0/596874998 conn(0x7fc7dc074760 msgr2=0x7fc7dc074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 -- 172.21.15.5:0/596874998 shutdown_connections 2023-07-21T19:21:46.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.751+0000 7fc7e2fff700 1 -- 172.21.15.5:0/596874998 wait complete. 2023-07-21T19:21:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 Processor -- start 2023-07-21T19:21:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 -- start start 2023-07-21T19:21:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 0x7fc7dc0fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1a86b0 con 0x7fc7dc067c10 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1a8820 con 0x7fc7dc104e50 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.753+0000 7fc7e2fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc1a89e0 con 0x7fc7dc0685b0 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 0x7fc7dc0fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59652/0 (socket says 172.21.15.5:59652) 2023-07-21T19:21:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45924/0 (socket says 172.21.15.5:45924) 2023-07-21T19:21:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7dbfff700 1 -- 172.21.15.5:0/3570705178 learned_addr learned my addr 172.21.15.5:0/3570705178 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 -- 172.21.15.5:0/3570705178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 msgr2=0x7fc7dc0fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 -- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 msgr2=0x7fc7dc0fea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 0x7fc7dc0fea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7e159c700 1 -- 172.21.15.5:0/3570705178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d4009c40 con 0x7fc7dc104e50 2023-07-21T19:21:46.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:46.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.754+0000 7fdbd77fe700 1 -- 172.21.15.5:0/2063138020 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdbdc0105a0 con 0x7fdbe00ffae0 2023-07-21T19:21:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7e159c700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4015000 tx=0x7fc7d40093d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/3570705178 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d400e040 con 0x7fc7dc104e50 2023-07-21T19:21:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/3570705178 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7d400bbf0 con 0x7fc7dc104e50 2023-07-21T19:21:46.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc1a8c60 con 0x7fc7dc104e50 2023-07-21T19:21:46.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fdbfbccdb80 1 Processor -- start 2023-07-21T19:21:46.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 -- start start 2023-07-21T19:21:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/3570705178 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4005040 con 0x7fc7dc104e50 2023-07-21T19:21:46.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.755+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc1a9130 con 0x7fc7dc104e50 2023-07-21T19:21:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3ccda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d3cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d2bfcb0 con 0x55b29d3c23f0 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d312a40 con 0x55b29d3cd2e0 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d30bed0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59662/0 (socket says 172.21.15.5:59662) 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.756+0000 7fdbe699c700 1 -- 172.21.15.5:0/2457748936 learned_addr learned my addr 172.21.15.5:0/2457748936 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbd77fe700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 0x7fdbcc0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 -- 172.21.15.5:0/2457748936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 msgr2=0x55b29d3cf6c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbd77fe700 1 -- 172.21.15.5:0/2063138020 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdbdc014070 con 0x7fdbe00ffae0 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d3cf6c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 -- 172.21.15.5:0/2457748936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 msgr2=0x55b29d3ccda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3ccda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3ccda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 -- 172.21.15.5:0/2457748936 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29d139de0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.757+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 0x7fdbcc0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.758+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8011790 tx=0x7fdbd8011aa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c69d2adb565fd085 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.758+0000 7fdbd4bfe700 1 -- 172.21.15.5:0/2457748936 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800e040 con 0x55b29d3c1ae0 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.758+0000 7fdbd4bfe700 1 -- 172.21.15.5:0/2457748936 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd800f040 con 0x55b29d3c1ae0 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.758+0000 7fdbd4bfe700 1 -- 172.21.15.5:0/2457748936 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80238e0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.759+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2457748936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 msgr2=0x55b29d3c1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.759+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 0x7fdbcc0a3870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8011760 tx=0x7fdbd801f000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.759+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8011790 tx=0x7fdbd8011aa0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2457748936 shutdown_connections 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d3cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3c1eb0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2457748936 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3ccda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2457748936 >> 172.21.15.5:0/2457748936 conn(0x55b29d3afda0 msgr2=0x55b29d3b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2457748936 shutdown_connections 2023-07-21T19:21:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2457748936 wait complete. 2023-07-21T19:21:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fdbfbccdb80 1 -- start start 2023-07-21T19:21:46.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.760+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/3570705178 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc7d401d470 con 0x7fc7dc104e50 2023-07-21T19:21:46.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.761+0000 7fdbfbccdb80 1 Processor -- start 2023-07-21T19:21:46.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.761+0000 7fc7f69b8b80 1 Processor -- start 2023-07-21T19:21:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3ccd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 -- start start 2023-07-21T19:21:46.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d2abac0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d2ab7c0 con 0x55b29d3c23f0 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d18f920 con 0x55b29d3cd2e0 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbfbccdb80 10 client.0 _ll_register_callbacks cb 0x55b29d2a9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59672/0 (socket says 172.21.15.5:59672) 2023-07-21T19:21:46.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbe699c700 1 -- 172.21.15.5:0/1370865114 learned_addr learned my addr 172.21.15.5:0/1370865114 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fdbe619b700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3ccd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 0x564e51745d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 0x564e51740040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517409d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51645c50 con 0x564e5159c6b0 2023-07-21T19:21:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e516989e0 con 0x564e51740580 2023-07-21T19:21:46.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.762+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51691e70 con 0x564e517462d0 2023-07-21T19:21:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 -- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 msgr2=0x55b29d3c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 -- 172.21.15.5:0/1370865114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 msgr2=0x55b29d3ccd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3ccd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 -- 172.21.15.5:0/1370865114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbd8011380 con 0x55b29d3cd2e0 2023-07-21T19:21:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 0x564e51745d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 0x564e51745d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39800/0 (socket says 172.21.15.5:39800) 2023-07-21T19:21:46.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7dbfff700 1 -- 172.21.15.5:0/4026766332 learned_addr learned my addr 172.21.15.5:0/4026766332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 0x564e51740040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbe699c700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009400 tx=0x7fdbd8009430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0943360c13c8b9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:46.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fc7d9ffb700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 0x7fc7cc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8018700 con 0x55b29d3cd2e0 2023-07-21T19:21:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.763+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdbd8023e10 con 0x55b29d3cd2e0 2023-07-21T19:21:46.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b29d2654f0 con 0x55b29d3cd2e0 2023-07-21T19:21:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/4026766332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 msgr2=0x564e517409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/4026766332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 msgr2=0x564e51745d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 0x564e51745d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/4026766332 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e514bfd80 con 0x564e517462d0 2023-07-21T19:21:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd8018860 con 0x55b29d3cd2e0 2023-07-21T19:21:46.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7d9ffb700 1 -- 172.21.15.5:0/3570705178 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc7d40cdfa0 con 0x7fc7dc104e50 2023-07-21T19:21:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbfbccdb80 10 client.28015 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:46.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x55b29d122c60 con 0x55b29d3cd2e0 2023-07-21T19:21:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 0x564e51740040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009cc0 tx=0x7fc7d0011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6091217d2e18506e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7e159c700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 0x7fc7cc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fc7c6ffd700 1 -- 172.21.15.5:0/4026766332 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0011cf0 con 0x564e517462d0 2023-07-21T19:21:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fc7c6ffd700 1 -- 172.21.15.5:0/4026766332 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7d0011e50 con 0x564e517462d0 2023-07-21T19:21:46.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fc7c6ffd700 1 -- 172.21.15.5:0/4026766332 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0005280 con 0x564e517462d0 2023-07-21T19:21:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b29d2654f0 con 0x55b29d3cd2e0 2023-07-21T19:21:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.764+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbd802ca20 con 0x55b29d3cd2e0 2023-07-21T19:21:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 10 client.28015 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 10 client.28015 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:46.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 20 client.28015 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d3dcb70 0x55b29d3def80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 4 client.28015 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b29d304eb0 con 0x55b29d3dcb70 2023-07-21T19:21:46.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.765+0000 7fdbe619b700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d3dcb70 0x55b29d3def80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/4026766332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 msgr2=0x564e51740040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7e159c700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 0x7fc7cc0a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40a2f30 tx=0x7fc7d4009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 0x564e51740040 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0009cc0 tx=0x7fc7d0011440 comp rx=0 tx=0).stop 2023-07-21T19:21:46.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fdbe619b700 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d3dcb70 0x55b29d3def80 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84c04f6dc83e51dd server_cookie=a39f2c613fbb2c10 in_seq=0 out_seq=0 2023-07-21T19:21:46.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fdbc9ffb700 10 client.28015 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/4026766332 shutdown_connections 2023-07-21T19:21:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e5159c6b0 0x564e51745d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/4026766332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e517462d0 0x564e51740040 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/4026766332 >> 172.21.15.5:0/4026766332 conn(0x564e51735be0 msgr2=0x564e5173af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/4026766332 shutdown_connections 2023-07-21T19:21:46.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.766+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/4026766332 wait complete. 2023-07-21T19:21:46.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/1370865114 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b29d304eb0 con 0x55b29d3dcb70 2023-07-21T19:21:46.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbc9ffb700 10 client.28015 handle_command_reply: tid=0 2023-07-21T19:21:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 client.28015 shutdown 2023-07-21T19:21:46.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fc7f69b8b80 1 -- start start 2023-07-21T19:21:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 msgr2=0x55b29d3ca6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009400 tx=0x7fdbd8009430 comp rx=0 tx=0).stop 2023-07-21T19:21:46.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fc7f69b8b80 1 Processor -- start 2023-07-21T19:21:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 shutdown_connections 2023-07-21T19:21:46.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3ca6c0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d3dcb70 0x55b29d3def80 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3ccd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/1370865114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 >> 172.21.15.5:0/1370865114 conn(0x55b29d3afda0 msgr2=0x55b29d3b5230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbc9ffb700 0 client.28015 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 shutdown_connections 2023-07-21T19:21:46.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.767+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/1370865114 wait complete. 2023-07-21T19:21:46.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fdbfbccdb80 20 client.28015 trim_cache size 0 max 16384 2023-07-21T19:21:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51749e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e5174a370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51631a60 con 0x564e517462d0 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e51631760 con 0x564e51740580 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e515158c0 con 0x564e5159c6b0 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7f69b8b80 10 client.0 _ll_register_callbacks cb 0x564e5162f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59690/0 (socket says 172.21.15.5:59690) 2023-07-21T19:21:46.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.768+0000 7fc7dbfff700 1 -- 172.21.15.5:0/1299843900 learned_addr learned my addr 172.21.15.5:0/1299843900 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e5174a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51749e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 -- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 msgr2=0x564e51749e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51749e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 -- 172.21.15.5:0/1299843900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 msgr2=0x564e5174a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e5174a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 -- 172.21.15.5:0/1299843900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d0009920 con 0x564e5159c6b0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80134a0 tx=0x7fc7c8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16ba7b237eeb1bba server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.769+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c8004030 con 0x564e5159c6b0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e515eb490 con 0x564e5159c6b0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc7c8004190 con 0x564e5159c6b0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7c8004350 con 0x564e5159c6b0 2023-07-21T19:21:46.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7f69b8b80 10 client.28021 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x564e514a8c00 con 0x564e5159c6b0 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564e515eb490 con 0x564e5159c6b0 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.770+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7c80223e0 con 0x564e5159c6b0 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 10 client.28021 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 10 client.28021 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 20 client.28021 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e51759c50 0x564e5175c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 4 client.28021 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e5168ae50 con 0x564e51759c50 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.771+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e51759c50 0x564e5175c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.772+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e51759c50 0x564e5175c060 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a2fa152516bca5 server_cookie=bb95a968cef680f5 in_seq=0 out_seq=0 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.772+0000 7fc7bbfff700 10 client.28021 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.772+0000 7fc7bbfff700 1 -- 172.21.15.5:0/1299843900 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e5168ae50 con 0x564e51759c50 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.772+0000 7fc7bbfff700 10 client.28021 handle_command_reply: tid=0 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 client.28021 shutdown 2023-07-21T19:21:46.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 msgr2=0x564e517498f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fc7c80134a0 tx=0x7fc7c8009260 comp rx=0 tx=0).stop 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 shutdown_connections 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e51759c50 0x564e5175c060 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e5174a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e517498f0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/1299843900 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51749e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 >> 172.21.15.5:0/1299843900 conn(0x564e51735be0 msgr2=0x564e5173a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7bbfff700 0 client.28021 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 shutdown_connections 2023-07-21T19:21:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/1299843900 wait complete. 2023-07-21T19:21:46.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.773+0000 7fc7f69b8b80 20 client.28021 trim_cache size 0 max 16384 2023-07-21T19:21:46.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 Processor -- start 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 -- start start 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3deb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d3c35a0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d3af970 con 0x55b29d3c23f0 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.787+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d3c39b0 con 0x55b29d3cd2e0 2023-07-21T19:21:46.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45966/0 (socket says 172.21.15.5:45966) 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe619b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39824/0 (socket says 172.21.15.5:39824) 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 -- 172.21.15.5:0/2897928383 learned_addr learned my addr 172.21.15.5:0/2897928383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3deb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 -- 172.21.15.5:0/2897928383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 msgr2=0x55b29d3deb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3deb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 -- 172.21.15.5:0/2897928383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 msgr2=0x55b29d3cf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe599a700 1 -- 172.21.15.5:0/2897928383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b29d160250 con 0x55b29d3c23f0 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.788+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.789+0000 7fdbe599a700 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0015000 tx=0x7fdbd0005c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39116480e575ce20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.789+0000 7fdbd53ff700 1 -- 172.21.15.5:0/2897928383 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd001d070 con 0x55b29d3c23f0 2023-07-21T19:21:46.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.789+0000 7fdbd53ff700 1 -- 172.21.15.5:0/2897928383 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd001e040 con 0x55b29d3c23f0 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.789+0000 7fdbd53ff700 1 -- 172.21.15.5:0/2897928383 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd00047d0 con 0x55b29d3c23f0 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2897928383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 msgr2=0x55b29d3e38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fdbd0015000 tx=0x7fdbd0005c30 comp rx=0 tx=0).stop 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2897928383 shutdown_connections 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3cd2e0 0x55b29d3deb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c1ae0 0x55b29d3cf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2897928383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3c23f0 0x55b29d3e38e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2897928383 >> 172.21.15.5:0/2897928383 conn(0x55b29d3afda0 msgr2=0x55b29d3b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2897928383 shutdown_connections 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.790+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2897928383 wait complete. 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 -- start start 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 Processor -- start 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3d1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3d2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d429f70 con 0x55b29d3c23f0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d42a0e0 con 0x55b29d3cd2e0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b29d42a2a0 con 0x55b29d3c1ae0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.791+0000 7fdbfbccdb80 10 client.0 _ll_register_callbacks cb 0x55b29d2a9760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe619b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3d1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45978/0 (socket says 172.21.15.5:45978) 2023-07-21T19:21:46.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 -- 172.21.15.5:0/2053793542 learned_addr learned my addr 172.21.15.5:0/2053793542 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe599a700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3d2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 -- 172.21.15.5:0/2053793542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 msgr2=0x55b29d3d1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3d1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 -- 172.21.15.5:0/2053793542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 msgr2=0x55b29d3d2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3d2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 -- 172.21.15.5:0/2053793542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbe01a4b00 con 0x55b29d3cd2e0 2023-07-21T19:21:46.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbe699c700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8023530 tx=0x7fdbd802c7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a7cb4cdd891c49d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.792+0000 7fdbc9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbd800e040 con 0x55b29d3cd2e0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdbd802d620 con 0x55b29d3cd2e0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdbd800f040 con 0x55b29d3cd2e0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdbd8005ac0 con 0x55b29d3cd2e0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbfbccdb80 10 client.28100 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x7fdbe010d220 con 0x55b29d3cd2e0 2023-07-21T19:21:46.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.793+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdbd800f040 con 0x55b29d3cd2e0 2023-07-21T19:21:46.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fdbd8018500 con 0x55b29d3cd2e0 2023-07-21T19:21:46.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 Processor -- start 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 10 client.28100 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 10 client.28100 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 20 client.28100 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d4316f0 0x55b29d433b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 4 client.28100 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b29d2c7050 con 0x55b29d4316f0 2023-07-21T19:21:46.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d4316f0 0x55b29d433b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 -- start start 2023-07-21T19:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51740040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51745340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e51750c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e517459c0 con 0x564e517462d0 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e517375f0 con 0x564e51740580 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e5173c800 con 0x564e5159c6b0 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.794+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51740040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51740040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59718/0 (socket says 172.21.15.5:59718) 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7dbfff700 1 -- 172.21.15.5:0/13365436 learned_addr learned my addr 172.21.15.5:0/13365436 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51745340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbe619b700 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d4316f0 0x55b29d433b00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=855af50a17b14739 server_cookie=1a5a45be4fc90140 in_seq=0 out_seq=0 2023-07-21T19:21:46.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbc9ffb700 10 client.28100 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e51750c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbc9ffb700 1 -- 172.21.15.5:0/2053793542 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55b29d2c7050 con 0x55b29d4316f0 2023-07-21T19:21:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbc9ffb700 10 client.28100 handle_command_reply: tid=0 2023-07-21T19:21:46.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 client.28100 shutdown 2023-07-21T19:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/13365436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 msgr2=0x564e51740040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51740040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/13365436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 msgr2=0x564e51750c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e51750c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/13365436 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e514e61f0 con 0x564e51740580 2023-07-21T19:21:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 msgr2=0x55b29d429a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8023530 tx=0x7fdbd802c7b0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e51750c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 shutdown_connections 2023-07-21T19:21:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b29d3cd2e0 0x55b29d429a00 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b29d3c1ae0 0x55b29d3d1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b29d4316f0 0x55b29d433b00 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51745340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d0009690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af8d32927fb4571c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 --2- 172.21.15.5:0/2053793542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b29d3c23f0 0x55b29d3d2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbc9ffb700 0 client.28100 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.795+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 >> 172.21.15.5:0/2053793542 conn(0x55b29d3afda0 msgr2=0x55b29d3c52a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 shutdown_connections 2023-07-21T19:21:46.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbfbccdb80 1 -- 172.21.15.5:0/2053793542 wait complete. 2023-07-21T19:21:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fc7c77fe700 1 -- 172.21.15.5:0/13365436 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d00049b0 con 0x564e51740580 2023-07-21T19:21:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fc7c77fe700 1 -- 172.21.15.5:0/13365436 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7d0011af0 con 0x564e51740580 2023-07-21T19:21:46.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fc7c77fe700 1 -- 172.21.15.5:0/13365436 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0003aa0 con 0x564e51740580 2023-07-21T19:21:46.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbfbccdb80 20 client.28100 trim_cache size 0 max 16384 2023-07-21T19:21:46.809 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:21:46.809 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:21:46.810 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:21:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 msgr2=0x7fdbcc0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 0x7fdbcc0a3870 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8011760 tx=0x7fdbd801f000 comp rx=0 tx=0).stop 2023-07-21T19:21:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 msgr2=0x7fdbe0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.796+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc00e9a0 tx=0x7fdbdc00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/13365436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 msgr2=0x564e51745340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51745340 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fc7d0006010 tx=0x7fc7d0009690 comp rx=0 tx=0).stop 2023-07-21T19:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/13365436 shutdown_connections 2023-07-21T19:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e51750c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51740040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/13365436 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e51745340 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/13365436 >> 172.21.15.5:0/13365436 conn(0x564e51735be0 msgr2=0x564e51746900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 shutdown_connections 2023-07-21T19:21:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdbcc0a13c0 0x7fdbcc0a3870 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdbe0105af0 0x7fdbe0073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdbe00ffae0 0x7fdbe0072810 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 --2- 172.21.15.5:0/2063138020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdbe0100490 0x7fdbe0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 >> 172.21.15.5:0/2063138020 conn(0x7fdbe00fb5e0 msgr2=0x7fdbe00fbc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 shutdown_connections 2023-07-21T19:21:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/13365436 shutdown_connections 2023-07-21T19:21:46.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/13365436 wait complete. 2023-07-21T19:21:46.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fdbe83ff700 1 -- 172.21.15.5:0/2063138020 wait complete. 2023-07-21T19:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.797+0000 7fc7f69b8b80 1 -- start start 2023-07-21T19:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 Processor -- start 2023-07-21T19:21:46.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51750c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e517a5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e517a9710 con 0x564e517462d0 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e517a9880 con 0x564e51740580 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e517a6500 con 0x564e5159c6b0 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.798+0000 7fc7f69b8b80 10 client.0 _ll_register_callbacks cb 0x564e5162f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:46.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45994/0 (socket says 172.21.15.5:45994) 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/3762794881 learned_addr learned my addr 172.21.15.5:0/3762794881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51750c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e517a5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/3762794881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 msgr2=0x564e51750c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51750c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/3762794881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 msgr2=0x564e517a5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:46.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e517a5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 -- 172.21.15.5:0/3762794881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d001d040 con 0x564e51740580 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e517a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7e0d9b700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d001c040 tx=0x7fc7d0004a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af5af84d3d5dee51 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.799+0000 7fc7bbfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7d0004110 con 0x564e51740580 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc7d0004270 con 0x564e51740580 2023-07-21T19:21:46.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc7d0011e50 con 0x564e51740580 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=44) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc7d0004430 con 0x564e51740580 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7f69b8b80 10 client.28106 fetch_fsmap learned FSMap version 44 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=44,monmap=4+}) v3 -- 0x564e514be250 con 0x564e51740580 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.800+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc7d003f080 con 0x564e51740580 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 44) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc7d0039030 con 0x564e51740580 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 10 client.28106 fetch_fsmap finished waiting for FSMap version 44 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 10 client.28106 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:21:46.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 20 client.28106 populate_metadata read hostname 'smithi005' 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e517bf870 0x564e517c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 4 client.28106 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564e5164cff0 con 0x564e517bf870 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.801+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e517bf870 0x564e517c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.802+0000 7fc7dbfff700 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e517bf870 0x564e517c1c50 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fa064f8b008af1c7 server_cookie=cd3f7dbad1b46c2 in_seq=0 out_seq=0 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.802+0000 7fc7bbfff700 10 client.28106 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7bbfff700 1 -- 172.21.15.5:0/3762794881 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564e5164cff0 con 0x564e517bf870 2023-07-21T19:21:46.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7bbfff700 10 client.28106 handle_command_reply: tid=0 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 client.28106 shutdown 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 msgr2=0x564e517a5a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fc7d001c040 tx=0x7fc7d0004a20 comp rx=0 tx=0).stop 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 shutdown_connections 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e517bf870 0x564e517c1c50 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e517462d0 0x564e517a5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e5159c6b0 0x564e51750c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 --2- 172.21.15.5:0/3762794881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e51740580 0x564e517a5a80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 >> 172.21.15.5:0/3762794881 conn(0x564e51735be0 msgr2=0x564e517451b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7bbfff700 0 client.28106 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 shutdown_connections 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.812+0000 7fc7f69b8b80 1 -- 172.21.15.5:0/3762794881 wait complete. 2023-07-21T19:21:46.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7f69b8b80 20 client.28106 trim_cache size 0 max 16384 2023-07-21T19:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:21:46.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 msgr2=0x7fc7cc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 0x7fc7cc0a37f0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fc7d40a2f30 tx=0x7fc7d4009920 comp rx=0 tx=0).stop 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 msgr2=0x7fc7dc0ff4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4015000 tx=0x7fc7d40093d0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 shutdown_connections 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc7dc104e50 0x7fc7dc0ff4e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc7cc0a1340 0x7fc7cc0a37f0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc7dc067c10 0x7fc7dc0fea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 --2- 172.21.15.5:0/3570705178 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc7dc0685b0 0x7fc7dc0fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.813+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 >> 172.21.15.5:0/3570705178 conn(0x7fc7dc074760 msgr2=0x7fc7dc0fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.814+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 shutdown_connections 2023-07-21T19:21:46.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:46.814+0000 7fc7e2fff700 1 -- 172.21.15.5:0/3570705178 wait complete. 2023-07-21T19:21:46.828 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:21:47.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:47 smithi101 ceph-mon[128136]: pgmap v444: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:47.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:47 smithi005 ceph-mon[126460]: pgmap v444: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:47.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:47 smithi156 ceph-mon[128316]: pgmap v444: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:49.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:49 smithi101 ceph-mon[128136]: pgmap v445: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:49.824 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:49 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:21:49.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:49 smithi005 ceph-mon[126460]: pgmap v445: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:49.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:49 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:21:49.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:49 smithi156 ceph-mon[128316]: pgmap v445: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 36 MiB/s wr, 29 op/s 2023-07-21T19:21:49.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:49 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:21:51.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:51 smithi101 ceph-mon[128136]: pgmap v446: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:51.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:51 smithi005 ceph-mon[126460]: pgmap v446: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:51.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:51 smithi156 ceph-mon[128316]: pgmap v446: 97 pgs: 97 active+clean; 11 GiB data, 4.9 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:53.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:53 smithi101 ceph-mon[128136]: pgmap v447: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:53.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:53 smithi005 ceph-mon[126460]: pgmap v447: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:53.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:53 smithi156 ceph-mon[128316]: pgmap v447: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 33 MiB/s wr, 25 op/s 2023-07-21T19:21:55.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:55 smithi101 ceph-mon[128136]: pgmap v448: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:55.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:55 smithi005 ceph-mon[126460]: pgmap v448: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:55.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:55 smithi156 ceph-mon[128316]: pgmap v448: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 26 MiB/s wr, 20 op/s 2023-07-21T19:21:57.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:57 smithi101 ceph-mon[128136]: pgmap v449: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:57.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:57 smithi005 ceph-mon[126460]: pgmap v449: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:57.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:57 smithi156 ceph-mon[128316]: pgmap v449: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:21:59.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:21:59 smithi101 ceph-mon[128136]: pgmap v450: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-07-21T19:21:59.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:21:59 smithi005 ceph-mon[126460]: pgmap v450: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-07-21T19:21:59.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:21:59 smithi156 ceph-mon[128316]: pgmap v450: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 37 MiB/s wr, 27 op/s 2023-07-21T19:21:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 -- 172.21.15.5:0/2806569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 msgr2=0x7fa77010ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:59.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 --2- 172.21.15.5:0/2806569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77010ca00 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7fa76800b0d0 tx=0x7fa76800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 -- 172.21.15.5:0/2806569351 shutdown_connections 2023-07-21T19:21:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 --2- 172.21.15.5:0/2806569351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77010ca00 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 --2- 172.21.15.5:0/2806569351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa770067c10 0x7fa770067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 --2- 172.21.15.5:0/2806569351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa7700685b0 0x7fa770104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 -- 172.21.15.5:0/2806569351 >> 172.21.15.5:0/2806569351 conn(0x7fa770074760 msgr2=0x7fa770074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 -- 172.21.15.5:0/2806569351 shutdown_connections 2023-07-21T19:21:59.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.984+0000 7fa777bff700 1 -- 172.21.15.5:0/2806569351 wait complete. 2023-07-21T19:21:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.985+0000 7fa777bff700 1 Processor -- start 2023-07-21T19:21:59.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 -- start start 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 0x7fa770197df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77019c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77019cfe0 con 0x7fa770104e50 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77019d150 con 0x7fa7700685b0 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa777bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa77019d310 con 0x7fa770067c10 2023-07-21T19:21:59.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa77519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 0x7fa770197df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa77519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34182/0 (socket says 172.21.15.5:34182) 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa77519a700 1 -- 172.21.15.5:0/314790598 learned_addr learned my addr 172.21.15.5:0/314790598 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.986+0000 7fa77619c700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77019c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 -- 172.21.15.5:0/314790598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 msgr2=0x7fa770198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 -- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 msgr2=0x7fa77019c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77019c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 -- 172.21.15.5:0/314790598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009d00 con 0x7fa770067c10 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77519a700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa77599b700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 0x7fa770197df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e9f0 tx=0x7fa76c00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.987+0000 7fa766ffd700 1 -- 172.21.15.5:0/314790598 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c00c9c0 con 0x7fa770067c10 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.988+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa77019d5f0 con 0x7fa770067c10 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.988+0000 7fa766ffd700 1 -- 172.21.15.5:0/314790598 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76c010430 con 0x7fa770067c10 2023-07-21T19:21:59.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.988+0000 7fa766ffd700 1 -- 172.21.15.5:0/314790598 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c018680 con 0x7fa770067c10 2023-07-21T19:21:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.988+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701a4e30 con 0x7fa770067c10 2023-07-21T19:21:59.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.991+0000 7fa766ffd700 1 -- 172.21.15.5:0/314790598 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa76c0105a0 con 0x7fa770067c10 2023-07-21T19:21:59.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa78b4f7b80 1 Processor -- start 2023-07-21T19:21:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa766ffd700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 0x7fa75c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa78b4f7b80 1 -- start start 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa766ffd700 1 -- 172.21.15.5:0/314790598 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa76c014070 con 0x7fa770067c10 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa77619c700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 0x7fa75c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.994+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e4731850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e4731dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e462ec50 con 0x5608e4737a80 2023-07-21T19:21:59.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e46819e0 con 0x5608e4737ed0 2023-07-21T19:21:59.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e467ae70 con 0x5608e45856b0 2023-07-21T19:21:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59546/0 (socket says 172.21.15.5:59546) 2023-07-21T19:21:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 -- 172.21.15.5:0/910255562 learned_addr learned my addr 172.21.15.5:0/910255562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77619c700 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e4731dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77519a700 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e4731850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 -- 172.21.15.5:0/910255562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 msgr2=0x5608e4731dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e4731dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 -- 172.21.15.5:0/910255562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 msgr2=0x5608e4731850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e4731850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77619c700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 0x7fa75c0a37f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7680062a0 tx=0x7fa768006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.995+0000 7fa77599b700 1 -- 172.21.15.5:0/910255562 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e44a8d80 con 0x5608e45856b0 2023-07-21T19:21:59.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.996+0000 7fa77599b700 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa76c096920 tx=0x7fa76c099af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6544730d4caa98e0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.996+0000 7fa75bfff700 1 -- 172.21.15.5:0/910255562 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c099eb0 con 0x5608e45856b0 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.996+0000 7fa75bfff700 1 -- 172.21.15.5:0/910255562 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76c09dee0 con 0x5608e45856b0 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.996+0000 7fa75bfff700 1 -- 172.21.15.5:0/910255562 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c099600 con 0x5608e45856b0 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/910255562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 msgr2=0x5608e4737540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa76c096920 tx=0x7fa76c099af0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/910255562 shutdown_connections 2023-07-21T19:21:59.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e4731dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e4737540 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/910255562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e4731850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/910255562 >> 172.21.15.5:0/910255562 conn(0x5608e471ebe0 msgr2=0x5608e4723f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:21:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/910255562 shutdown_connections 2023-07-21T19:21:59.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/910255562 wait complete. 2023-07-21T19:21:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.997+0000 7fa78b4f7b80 1 -- start start 2023-07-21T19:21:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 Processor -- start 2023-07-21T19:21:59.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e473b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e461aa60 con 0x5608e4737a80 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e461a760 con 0x5608e4737ed0 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.998+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e44fe8c0 con 0x5608e45856b0 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa78b4f7b80 10 client.0 _ll_register_callbacks cb 0x5608e4618700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:21:59.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34194/0 (socket says 172.21.15.5:34194) 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 -- 172.21.15.5:0/3064039203 learned_addr learned my addr 172.21.15.5:0/3064039203 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77519a700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e473b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 -- 172.21.15.5:0/3064039203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 msgr2=0x5608e473de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 -- 172.21.15.5:0/3064039203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 msgr2=0x5608e473b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:21:59.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e473b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77619c700 1 -- 172.21.15.5:0/3064039203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7701a4e30 con 0x5608e4737ed0 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:21:59.999+0000 7fa77599b700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa77619c700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa76800cd60 tx=0x7fa76800b3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25a1e383b838b869 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76800e040 con 0x5608e4737ed0 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa768007de0 con 0x5608e4737ed0 2023-07-21T19:21:59.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5608e45d4490 con 0x5608e4737ed0 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7680054d0 con 0x5608e4737ed0 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa78b4f7b80 10 client.28118 fetch_fsmap learned FSMap version 45 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.000+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5608e4491c00 con 0x5608e4737ed0 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5608e45d4490 con 0x5608e4737ed0 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 45) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa768030060 con 0x5608e4737ed0 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 10 client.28118 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T19:21:59.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 10 client.28118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 20 client.28118 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e474b3d0 0x5608e474d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 4 client.28118 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.001+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5608e4673e50 con 0x5608e474b3d0 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.002+0000 7fa77599b700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e474b3d0 0x5608e474d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.002+0000 7fa77599b700 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e474b3d0 0x5608e474d7e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=892668ce4cc1634 server_cookie=422a4bbdc4fd27be in_seq=0 out_seq=0 2023-07-21T19:22:00.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.002+0000 7fa7597fa700 10 client.28118 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:00.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa7597fa700 1 -- 172.21.15.5:0/3064039203 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5608e4673e50 con 0x5608e474b3d0 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa7597fa700 10 client.28118 handle_command_reply: tid=0 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 client.28118 shutdown 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 msgr2=0x5608e473b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fa76800cd60 tx=0x7fa76800b3e0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 shutdown_connections 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473b7e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e474b3d0 0x5608e474d7e0 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.003+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/3064039203 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e473b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.004+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 >> 172.21.15.5:0/3064039203 conn(0x5608e471ebe0 msgr2=0x5608e47234e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.004+0000 7fa7597fa700 0 client.28118 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.004+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 shutdown_connections 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.004+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/3064039203 wait complete. 2023-07-21T19:22:00.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.004+0000 7fa78b4f7b80 20 client.28118 trim_cache size 0 max 16384 2023-07-21T19:22:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 Processor -- start 2023-07-21T19:22:00.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 -- start start 2023-07-21T19:22:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e47423e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e4724d50 con 0x5608e4737a80 2023-07-21T19:22:00.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e4732600 con 0x5608e4737ed0 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.021+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e4732820 con 0x5608e45856b0 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59554/0 (socket says 172.21.15.5:59554) 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 -- 172.21.15.5:0/2386590210 learned_addr learned my addr 172.21.15.5:0/2386590210 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77619c700 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e47423e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 -- 172.21.15.5:0/2386590210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 msgr2=0x5608e473d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 -- 172.21.15.5:0/2386590210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 msgr2=0x5608e47423e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e47423e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 -- 172.21.15.5:0/2386590210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5608e47523a0 con 0x5608e45856b0 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.022+0000 7fa77599b700 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa76c099b60 tx=0x7fa76c01f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3241ed6143883989 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.023+0000 7fa764bff700 1 -- 172.21.15.5:0/2386590210 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0a2300 con 0x5608e45856b0 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.023+0000 7fa764bff700 1 -- 172.21.15.5:0/2386590210 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76c0b8270 con 0x5608e45856b0 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.023+0000 7fa764bff700 1 -- 172.21.15.5:0/2386590210 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76c0a20d0 con 0x5608e45856b0 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.023+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/2386590210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 msgr2=0x5608e473a8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:00.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.023+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa76c099b60 tx=0x7fa76c01f000 comp rx=0 tx=0).stop 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/2386590210 shutdown_connections 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737ed0 0x5608e473d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e473a8c0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/2386590210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737a80 0x5608e47423e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/2386590210 >> 172.21.15.5:0/2386590210 conn(0x5608e471ebe0 msgr2=0x5608e4735240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/2386590210 shutdown_connections 2023-07-21T19:22:00.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/2386590210 wait complete. 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 -- start start 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.024+0000 7fa78b4f7b80 1 Processor -- start 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 0x5608e4734590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e4734ca0 con 0x5608e4737ed0 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e479e210 con 0x5608e4737a80 2023-07-21T19:22:00.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5608e479e350 con 0x5608e45856b0 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa78b4f7b80 10 client.0 _ll_register_callbacks cb 0x7fa7680077f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77519a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34218/0 (socket says 172.21.15.5:34218) 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59560/0 (socket says 172.21.15.5:59560) 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77519a700 1 -- 172.21.15.5:0/564580909 learned_addr learned my addr 172.21.15.5:0/564580909 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:00.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.025+0000 7fa77619c700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 0x5608e4734590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 -- 172.21.15.5:0/564580909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 msgr2=0x5608e47425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 -- 172.21.15.5:0/564580909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 msgr2=0x5608e4734590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 0x5608e4734590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 -- 172.21.15.5:0/564580909 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5608e44a8d80 con 0x5608e4737a80 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77619c700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 0x5608e4734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:00.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77599b700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:00.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa77519a700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa760012cb0 tx=0x7fa760009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc24c707b47e7516 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa7597fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760017040 con 0x5608e4737a80 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa76c09dee0 con 0x5608e4737a80 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.026+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(104..104 src has 1..104) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa760004230 con 0x5608e4737a80 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=45) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa7600043f0 con 0x5608e4737a80 2023-07-21T19:22:00.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 10 client.28124 fetch_fsmap learned FSMap version 45 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=45,monmap=4+}) v3 -- 0x5608e44cf1f0 con 0x5608e4737a80 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa76002a020 con 0x5608e4737a80 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 45) v1 ==== 7466+0+0 (secure 0 0 0) 0x7fa760025070 con 0x5608e4737a80 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 10 client.28124 fetch_fsmap finished waiting for FSMap version 45 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 10 client.28124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 20 client.28124 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:00.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e479fac0 0x5608e47a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 4 client.28124 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:22:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.027+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5608e4635ff0 con 0x5608e479fac0 2023-07-21T19:22:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.028+0000 7fa77599b700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e479fac0 0x5608e47a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.028+0000 7fa77599b700 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e479fac0 0x5608e47a1ed0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f772db374f9c57 server_cookie=e8a209eda9ef27e9 in_seq=0 out_seq=0 2023-07-21T19:22:00.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.028+0000 7fa7597fa700 10 client.28124 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa7597fa700 1 -- 172.21.15.5:0/564580909 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5608e4635ff0 con 0x5608e479fac0 2023-07-21T19:22:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa7597fa700 10 client.28124 handle_command_reply: tid=0 2023-07-21T19:22:00.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 client.28124 shutdown 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 msgr2=0x5608e4733e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fa760012cb0 tx=0x7fa760009f80 comp rx=0 tx=0).stop 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 shutdown_connections 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5608e479fac0 0x5608e47a1ed0 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5608e4737ed0 0x5608e4734590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5608e45856b0 0x5608e47425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa7597fa700 0 client.28124 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 --2- 172.21.15.5:0/564580909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5608e4737a80 0x5608e4733e50 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 >> 172.21.15.5:0/564580909 conn(0x5608e471ebe0 msgr2=0x5608e4728030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 shutdown_connections 2023-07-21T19:22:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 1 -- 172.21.15.5:0/564580909 wait complete. 2023-07-21T19:22:00.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.038+0000 7fa78b4f7b80 20 client.28124 trim_cache size 0 max 16384 2023-07-21T19:22:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:22:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:22:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:22:00.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:22:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 msgr2=0x7fa75c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 0x7fa75c0a37f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa7680062a0 tx=0x7fa768006210 comp rx=0 tx=0).stop 2023-07-21T19:22:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 msgr2=0x7fa770197df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:00.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 0x7fa770197df0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fa76c00e9f0 tx=0x7fa76c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 shutdown_connections 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa75c0a1340 0x7fa75c0a37f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa770104e50 0x7fa77019c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa770067c10 0x7fa770197df0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 --2- 172.21.15.5:0/314790598 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa7700685b0 0x7fa770198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:00.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 >> 172.21.15.5:0/314790598 conn(0x7fa770074760 msgr2=0x7fa7700fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 shutdown_connections 2023-07-21T19:22:00.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:00.039+0000 7fa777bff700 1 -- 172.21.15.5:0/314790598 wait complete. 2023-07-21T19:22:01.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:01 smithi005 ceph-mon[126460]: pgmap v451: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 16 op/s 2023-07-21T19:22:01.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:01 smithi156 ceph-mon[128316]: pgmap v451: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 16 op/s 2023-07-21T19:22:02.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:01 smithi101 ceph-mon[128136]: pgmap v451: 97 pgs: 97 active+clean; 12 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 MiB/s wr, 16 op/s 2023-07-21T19:22:03.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:03 smithi005 ceph-mon[126460]: pgmap v452: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-07-21T19:22:03.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:03 smithi156 ceph-mon[128316]: pgmap v452: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-07-21T19:22:04.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:03 smithi101 ceph-mon[128136]: pgmap v452: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 35 MiB/s wr, 36 op/s 2023-07-21T19:22:05.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:05 smithi005 ceph-mon[126460]: pgmap v453: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 34 op/s 2023-07-21T19:22:05.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:05 smithi156 ceph-mon[128316]: pgmap v453: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 34 op/s 2023-07-21T19:22:06.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:05 smithi101 ceph-mon[128136]: pgmap v453: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 MiB/s wr, 34 op/s 2023-07-21T19:22:07.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:07 smithi005 ceph-mon[126460]: pgmap v454: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:22:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:07 smithi005 ceph-mon[126460]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T19:22:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:07 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:07.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:07 smithi156 ceph-mon[128316]: pgmap v454: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:22:07.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:07 smithi156 ceph-mon[128316]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T19:22:07.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:07 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:08.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:07 smithi101 ceph-mon[128136]: pgmap v454: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:22:08.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:07 smithi101 ceph-mon[128136]: osdmap e105: 12 total, 12 up, 12 in 2023-07-21T19:22:08.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:07 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:08.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:08 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:22:08.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:08 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:22:09.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:08 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:22:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:09 smithi005 ceph-mon[126460]: pgmap v456: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:09 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:09 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:22:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:09 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:22:09.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:09 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:22:09.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:09 smithi156 ceph-mon[128316]: pgmap v456: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:09.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:09 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:09.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:09 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:22:09.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:09 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:22:09.913 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:09 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:22:10.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:09 smithi101 ceph-mon[128136]: pgmap v456: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:10.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:09 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:10.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:09 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:22:10.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:09 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:22:10.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:09 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:22:11.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:11 smithi005 ceph-mon[126460]: pgmap v457: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:11.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:11 smithi156 ceph-mon[128316]: pgmap v457: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:12.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:11 smithi101 ceph-mon[128136]: pgmap v457: 97 pgs: 97 active+clean; 11 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 28 MiB/s wr, 46 op/s 2023-07-21T19:22:13.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 -- 172.21.15.5:0/2103222670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b00ffb30 msgr2=0x7f06b00fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 --2- 172.21.15.5:0/2103222670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b00ffb30 0x7f06b00fffa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f06a00098f0 tx=0x7f06a0009c00 comp rx=0 tx=0).stop 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 -- 172.21.15.5:0/2103222670 shutdown_connections 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 --2- 172.21.15.5:0/2103222670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b01004e0 0x7f06b010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 --2- 172.21.15.5:0/2103222670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b0105b60 0x7f06b0105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 --2- 172.21.15.5:0/2103222670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b00ffb30 0x7f06b00fffa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 -- 172.21.15.5:0/2103222670 >> 172.21.15.5:0/2103222670 conn(0x7f06b00fb5f0 msgr2=0x7f06b00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 -- 172.21.15.5:0/2103222670 shutdown_connections 2023-07-21T19:22:13.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.197+0000 7f06b77ff700 1 -- 172.21.15.5:0/2103222670 wait complete. 2023-07-21T19:22:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.198+0000 7f06b77ff700 1 Processor -- start 2023-07-21T19:22:13.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.198+0000 7f06b77ff700 1 -- start start 2023-07-21T19:22:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b00ffb30 0x7f06b019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b0105b60 0x7f06b0197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01981b0 con 0x7f06b01004e0 2023-07-21T19:22:13.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b0198320 con 0x7f06b0105b60 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b77ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06b01984e0 con 0x7f06b00ffb30 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b00ffb30 0x7f06b019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56374/0 (socket says 172.21.15.5:56374) 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 -- 172.21.15.5:0/3655097496 learned_addr learned my addr 172.21.15.5:0/3655097496 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:13.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b0105b60 0x7f06b0197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 -- 172.21.15.5:0/3655097496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b00ffb30 msgr2=0x7f06b019dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b00ffb30 0x7f06b019dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 -- 172.21.15.5:0/3655097496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b0105b60 msgr2=0x7f06b0197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b0105b60 0x7f06b0197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.199+0000 7f06b4d9a700 1 -- 172.21.15.5:0/3655097496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06ac0095f0 con 0x7f06b01004e0 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009810 tx=0x7f06a000fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06a67fc700 1 -- 172.21.15.5:0/3655097496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a001c070 con 0x7f06b01004e0 2023-07-21T19:22:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06a67fc700 1 -- 172.21.15.5:0/3655097496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06a000bb70 con 0x7f06b01004e0 2023-07-21T19:22:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06a0009520 con 0x7f06b01004e0 2023-07-21T19:22:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06a67fc700 1 -- 172.21.15.5:0/3655097496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a0004930 con 0x7f06b01004e0 2023-07-21T19:22:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.200+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b01a4af0 con 0x7f06b01004e0 2023-07-21T19:22:13.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.203+0000 7f06a67fc700 1 -- 172.21.15.5:0/3655097496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f06a0020070 con 0x7f06b01004e0 2023-07-21T19:22:13.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.205+0000 7f06cb156b80 1 Processor -- start 2023-07-21T19:22:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 -- start start 2023-07-21T19:22:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06a67fc700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 0x7f069c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06a67fc700 1 -- 172.21.15.5:0/3655097496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f06a00cda30 con 0x7f06b01004e0 2023-07-21T19:22:13.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382e52d0 0x556c382df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382df9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c381e4c50 con 0x556c382e52d0 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382379e0 con 0x556c3813b6b0 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06cb156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c38230e70 con 0x556c382df580 2023-07-21T19:22:13.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 0x7f069c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06b4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382e52d0 0x556c382df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58958/0 (socket says 172.21.15.5:58958) 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.206+0000 7f06b559b700 1 -- 172.21.15.5:0/2860953214 learned_addr learned my addr 172.21.15.5:0/2860953214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382df9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.202 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 -- 172.21.15.5:0/2860953214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 msgr2=0x556c382df9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382df9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 -- 172.21.15.5:0/2860953214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382e52d0 msgr2=0x556c382df040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382e52d0 0x556c382df040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 -- 172.21.15.5:0/2860953214 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c3805ed80 con 0x556c3813b6b0 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382df9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:22:13.203 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 0x7f069c0a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06b019e9e0 tx=0x7f06a8009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.204 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f06b559b700 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009fc0 tx=0x7f06ac011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcf1ab684894d4b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f069b7fe700 1 -- 172.21.15.5:0/2860953214 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00e670 con 0x556c3813b6b0 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f069b7fe700 1 -- 172.21.15.5:0/2860953214 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06ac00ec60 con 0x556c3813b6b0 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.207+0000 7f069b7fe700 1 -- 172.21.15.5:0/2860953214 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac019cc0 con 0x556c3813b6b0 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 -- 172.21.15.5:0/2860953214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 msgr2=0x556c382e4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009fc0 tx=0x7f06ac011720 comp rx=0 tx=0).stop 2023-07-21T19:22:13.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 -- 172.21.15.5:0/2860953214 shutdown_connections 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382df9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e4d90 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 --2- 172.21.15.5:0/2860953214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382e52d0 0x556c382df040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 -- 172.21.15.5:0/2860953214 >> 172.21.15.5:0/2860953214 conn(0x556c382d4be0 msgr2=0x556c382d9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 -- 172.21.15.5:0/2860953214 shutdown_connections 2023-07-21T19:22:13.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.208+0000 7f06cb156b80 1 -- 172.21.15.5:0/2860953214 wait complete. 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 -- start start 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 Processor -- start 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 0x556c382e87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c381d0a60 con 0x556c3813b6b0 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c381d0760 con 0x556c382df580 2023-07-21T19:22:13.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.209+0000 7f06cb156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c380b48c0 con 0x556c382e52d0 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06cb156b80 10 client.0 _ll_register_callbacks cb 0x556c381ce700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56386/0 (socket says 172.21.15.5:56386) 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b559b700 1 -- 172.21.15.5:0/1632406670 learned_addr learned my addr 172.21.15.5:0/1632406670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 0x556c382e87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 -- 172.21.15.5:0/1632406670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 msgr2=0x556c382e8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 -- 172.21.15.5:0/1632406670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 msgr2=0x556c382eb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 -- 172.21.15.5:0/1632406670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06ac009c20 con 0x556c382df580 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b559b700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:13.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 0x556c382e87c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f06a00cb120 tx=0x7f06a000bce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26771ad6912a3042 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.210+0000 7f0698ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a009af90 con 0x556c382df580 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556c3818a490 con 0x556c382df580 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f06a00a2ef0 con 0x556c382df580 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a0005450 con 0x556c382df580 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 10 client.28136 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:13.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556c38047c00 con 0x556c382df580 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556c3818a490 con 0x556c382df580 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f06a00a2af0 con 0x556c382df580 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 10 client.28136 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 10 client.28136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 20 client.28136 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c382f8780 0x556c382fab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 4 client.28136 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:13.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.211+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556c38229e50 con 0x556c382f8780 2023-07-21T19:22:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.212+0000 7f06b559b700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c382f8780 0x556c382fab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.212+0000 7f06b559b700 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c382f8780 0x556c382fab90 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8221dc6d7197cf88 server_cookie=7c860c3d1704ca41 in_seq=0 out_seq=0 2023-07-21T19:22:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.212+0000 7f0698ff9700 10 client.28136 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f0698ff9700 1 -- 172.21.15.5:0/1632406670 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556c38229e50 con 0x556c382f8780 2023-07-21T19:22:13.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f0698ff9700 10 client.28136 handle_command_reply: tid=0 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 client.28136 shutdown 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 msgr2=0x556c382e87c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 0x556c382e87c0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f06a00cb120 tx=0x7f06a000bce0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 shutdown_connections 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c382f8780 0x556c382fab90 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382eb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f0698ff9700 0 client.28136 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1632406670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382df580 0x556c382e87c0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 >> 172.21.15.5:0/1632406670 conn(0x556c382d4be0 msgr2=0x556c382d8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 shutdown_connections 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.213+0000 7f06cb156b80 1 -- 172.21.15.5:0/1632406670 wait complete. 2023-07-21T19:22:13.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.214+0000 7f06cb156b80 20 client.28136 trim_cache size 0 max 16384 2023-07-21T19:22:13.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 Processor -- start 2023-07-21T19:22:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 -- start start 2023-07-21T19:22:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382e52d0 0x556c382d9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382d1c00 con 0x556c3813b6b0 2023-07-21T19:22:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382a5060 con 0x556c382e52d0 2023-07-21T19:22:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06cb156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382e9da0 con 0x556c382df580 2023-07-21T19:22:13.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06b5d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382e52d0 0x556c382d9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06b4d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.229+0000 7f06b559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56402/0 (socket says 172.21.15.5:56402) 2023-07-21T19:22:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 -- 172.21.15.5:0/1811540116 learned_addr learned my addr 172.21.15.5:0/1811540116 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 -- 172.21.15.5:0/1811540116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 msgr2=0x556c382eb0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382eb0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 -- 172.21.15.5:0/1811540116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382e52d0 msgr2=0x556c382d9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382e52d0 0x556c382d9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 -- 172.21.15.5:0/1811540116 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556c380851f0 con 0x556c3813b6b0 2023-07-21T19:22:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f06b559b700 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f06ac011a70 tx=0x7f06ac009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954d651b431eb124 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.230+0000 7f069bfff700 1 -- 172.21.15.5:0/1811540116 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac0194b0 con 0x556c3813b6b0 2023-07-21T19:22:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f069bfff700 1 -- 172.21.15.5:0/1811540116 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06ac02b860 con 0x556c3813b6b0 2023-07-21T19:22:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f069bfff700 1 -- 172.21.15.5:0/1811540116 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac00e6d0 con 0x556c3813b6b0 2023-07-21T19:22:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 -- 172.21.15.5:0/1811540116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 msgr2=0x556c382efc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f06ac011a70 tx=0x7f06ac009310 comp rx=0 tx=0).stop 2023-07-21T19:22:13.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 -- 172.21.15.5:0/1811540116 shutdown_connections 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c382e52d0 0x556c382d9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c3813b6b0 0x556c382efc60 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 --2- 172.21.15.5:0/1811540116 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382df580 0x556c382eb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 -- 172.21.15.5:0/1811540116 >> 172.21.15.5:0/1811540116 conn(0x556c382d4be0 msgr2=0x556c382e3950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 -- 172.21.15.5:0/1811540116 shutdown_connections 2023-07-21T19:22:13.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.231+0000 7f06cb156b80 1 -- 172.21.15.5:0/1811540116 wait complete. 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.232+0000 7f06cb156b80 1 -- start start 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.232+0000 7f06cb156b80 1 Processor -- start 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.232+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382df580 0x556c382e2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382e0da0 con 0x556c382df580 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382e0f10 con 0x556c3813b6b0 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556c382e10d0 con 0x556c382e52d0 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06cb156b80 10 client.0 _ll_register_callbacks cb 0x7f06a009a540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58984/0 (socket says 172.21.15.5:58984) 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 -- 172.21.15.5:0/4007284265 learned_addr learned my addr 172.21.15.5:0/4007284265 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:13.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382df580 0x556c382e2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b5d9c700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 -- 172.21.15.5:0/4007284265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 msgr2=0x556c382e0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 -- 172.21.15.5:0/4007284265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382df580 msgr2=0x556c382e2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382df580 0x556c382e2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.233+0000 7f06b559b700 1 -- 172.21.15.5:0/4007284265 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556c3805ed80 con 0x556c3813b6b0 2023-07-21T19:22:13.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f06b559b700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009bf0 tx=0x7f06ac009970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2ffe53278411f472 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06ac02b530 con 0x556c3813b6b0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f06ac02b9d0 con 0x556c3813b6b0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f06ac00ec60 con 0x556c3813b6b0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06ac02bbb0 con 0x556c3813b6b0 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f06cb156b80 10 client.28142 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:13.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x556c3805d250 con 0x556c3813b6b0 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f06ac043020 con 0x556c3813b6b0 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.234+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f06ac00e700 con 0x556c3813b6b0 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 10 client.28142 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 10 client.28142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 20 client.28142 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c3834d160 0x556c3834f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:13.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 4 client.28142 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:22:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556c381ebff0 con 0x556c3834d160 2023-07-21T19:22:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c3834d160 0x556c3834f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f06b4d9a700 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c3834d160 0x556c3834f540 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8c9313b3e5c2660 server_cookie=fdb29b35bb1155fa in_seq=0 out_seq=0 2023-07-21T19:22:13.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.235+0000 7f0698ff9700 10 client.28142 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f0698ff9700 1 -- 172.21.15.5:0/4007284265 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556c381ebff0 con 0x556c3834d160 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f0698ff9700 10 client.28142 handle_command_reply: tid=0 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 client.28142 shutdown 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 msgr2=0x556c382e1b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009bf0 tx=0x7f06ac009970 comp rx=0 tx=0).stop 2023-07-21T19:22:13.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 shutdown_connections 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556c382e52d0 0x556c382e0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556c3813b6b0 0x556c382e1b00 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556c3834d160 0x556c3834f540 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 --2- 172.21.15.5:0/4007284265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556c382df580 0x556c382e2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 >> 172.21.15.5:0/4007284265 conn(0x556c382d4be0 msgr2=0x556c382e41e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f0698ff9700 0 client.28142 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 shutdown_connections 2023-07-21T19:22:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 1 -- 172.21.15.5:0/4007284265 wait complete. 2023-07-21T19:22:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.248+0000 7f06cb156b80 20 client.28142 trim_cache size 0 max 16384 2023-07-21T19:22:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:22:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:22:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:22:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 msgr2=0x7f069c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 0x7f069c0a3770 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f06b019e9e0 tx=0x7f06a8009920 comp rx=0 tx=0).stop 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 msgr2=0x7f06b019e220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f06a0009810 tx=0x7f06a000fa00 comp rx=0 tx=0).stop 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 shutdown_connections 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f06b0105b60 0x7f06b0197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f06b00ffb30 0x7f06b019dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f069c0a12c0 0x7f069c0a3770 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 --2- 172.21.15.5:0/3655097496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f06b01004e0 0x7f06b019e220 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 >> 172.21.15.5:0/3655097496 conn(0x7f06b00fb5f0 msgr2=0x7f06b00fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 shutdown_connections 2023-07-21T19:22:13.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:13.249+0000 7f06b77ff700 1 -- 172.21.15.5:0/3655097496 wait complete. 2023-07-21T19:22:13.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:13 smithi101 ceph-mon[128136]: pgmap v458: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 25 op/s 2023-07-21T19:22:13.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:13 smithi101 ceph-mon[128136]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T19:22:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:13 smithi005 ceph-mon[126460]: pgmap v458: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 25 op/s 2023-07-21T19:22:13.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:13 smithi005 ceph-mon[126460]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T19:22:13.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:13 smithi156 ceph-mon[128316]: pgmap v458: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 17 MiB/s wr, 25 op/s 2023-07-21T19:22:13.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:13 smithi156 ceph-mon[128316]: osdmap e106: 12 total, 12 up, 12 in 2023-07-21T19:22:15.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:15 smithi101 ceph-mon[128136]: pgmap v460: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 19 MiB/s wr, 26 op/s 2023-07-21T19:22:15.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:15 smithi005 ceph-mon[126460]: pgmap v460: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 19 MiB/s wr, 26 op/s 2023-07-21T19:22:15.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:15 smithi156 ceph-mon[128316]: pgmap v460: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 19 MiB/s wr, 26 op/s 2023-07-21T19:22:16.829 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:22:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1618677568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 msgr2=0x7f4fac105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1618677568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac105eb0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0009990 tx=0x7f4fa0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1618677568 shutdown_connections 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1618677568 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 0x7f4fac10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1618677568 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac105eb0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1618677568 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1618677568 >> 172.21.15.5:0/1618677568 conn(0x7f4fac0fb5f0 msgr2=0x7f4fac0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1618677568 shutdown_connections 2023-07-21T19:22:17.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.057+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1618677568 wait complete. 2023-07-21T19:22:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 Processor -- start 2023-07-21T19:22:17.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 -- start start 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 0x7f4fac198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac19bbb0 con 0x7f4fac105ae0 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac19bd20 con 0x7f4fac100480 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.058+0000 7f4fb31ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fac19bee0 con 0x7f4fac0ffad0 2023-07-21T19:22:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56802/0 (socket says 172.21.15.5:56802) 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 -- 172.21.15.5:0/1690718531 learned_addr learned my addr 172.21.15.5:0/1690718531 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fabfff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 0x7f4fac198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 -- 172.21.15.5:0/1690718531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 msgr2=0x7f4fac197cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac197cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 -- 172.21.15.5:0/1690718531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 msgr2=0x7f4fac198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 0x7f4fac198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 -- 172.21.15.5:0/1690718531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fa00095f0 con 0x7f4fac105ae0 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb179c700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400ef30 tx=0x7f4fa400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fa9ffb700 1 -- 172.21.15.5:0/1690718531 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa4018070 con 0x7f4fac105ae0 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fa9ffb700 1 -- 172.21.15.5:0/1690718531 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fa400f040 con 0x7f4fac105ae0 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fa9ffb700 1 -- 172.21.15.5:0/1690718531 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa400cc70 con 0x7f4fac105ae0 2023-07-21T19:22:17.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.059+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fac19c160 con 0x7f4fac105ae0 2023-07-21T19:22:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.060+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fac1a4b20 con 0x7f4fac105ae0 2023-07-21T19:22:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.063+0000 7f4fa9ffb700 1 -- 172.21.15.5:0/1690718531 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4fa4003a00 con 0x7f4fac105ae0 2023-07-21T19:22:17.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.064+0000 7f4fc6b27b80 1 Processor -- start 2023-07-21T19:22:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.064+0000 7f4fc6b27b80 1 -- start start 2023-07-21T19:22:17.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f5590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e5f0cb0 con 0x55716e6ea2d0 2023-07-21T19:22:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e643a40 con 0x55716e6eabe0 2023-07-21T19:22:17.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e63ced0 con 0x55716e6f5ad0 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fa9ffb700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 0x7f4f9c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fb0f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56810/0 (socket says 172.21.15.5:56810) 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 -- 172.21.15.5:0/959105111 learned_addr learned my addr 172.21.15.5:0/959105111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:17.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fa9ffb700 1 -- 172.21.15.5:0/1690718531 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4fa40cc5b0 con 0x7f4fac105ae0 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fb179c700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 0x7f4f9c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 -- 172.21.15.5:0/959105111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 msgr2=0x55716e6f7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 -- 172.21.15.5:0/959105111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 msgr2=0x55716e6f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.065+0000 7f4fabfff700 1 -- 172.21.15.5:0/959105111 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716e46ade0 con 0x55716e6ea2d0 2023-07-21T19:22:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.066+0000 7f4fabfff700 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4f98005c50 tx=0x7f4f98009830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3be0fb46ef9eb83e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.066+0000 7f4fb179c700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 0x7f4f9c0a37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40c9d00 tx=0x7f4fa4004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.066+0000 7f4f96ffd700 1 -- 172.21.15.5:0/959105111 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98013e50 con 0x55716e6ea2d0 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.066+0000 7f4f96ffd700 1 -- 172.21.15.5:0/959105111 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f980044e0 con 0x55716e6ea2d0 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.066+0000 7f4f96ffd700 1 -- 172.21.15.5:0/959105111 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f98005280 con 0x55716e6ea2d0 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/959105111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 msgr2=0x55716e6ea6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f4f98005c50 tx=0x7f4f98009830 comp rx=0 tx=0).stop 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/959105111 shutdown_connections 2023-07-21T19:22:17.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6ea6a0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/959105111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/959105111 >> 172.21.15.5:0/959105111 conn(0x55716e6e0da0 msgr2=0x55716e6e66a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/959105111 shutdown_connections 2023-07-21T19:22:17.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/959105111 wait complete. 2023-07-21T19:22:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.067+0000 7f4fc6b27b80 1 -- start start 2023-07-21T19:22:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 Processor -- start 2023-07-21T19:22:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e5dcac0 con 0x55716e6ea2d0 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e5dc7c0 con 0x55716e6eabe0 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e4c0920 con 0x55716e6f5ad0 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fc6b27b80 10 client.0 _ll_register_callbacks cb 0x55716e5da760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56824/0 (socket says 172.21.15.5:56824) 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.068+0000 7f4fabfff700 1 -- 172.21.15.5:0/2971802457 learned_addr learned my addr 172.21.15.5:0/2971802457 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:17.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 -- 172.21.15.5:0/2971802457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 msgr2=0x55716e6f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 -- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 msgr2=0x55716e6f2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 -- 172.21.15.5:0/2971802457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f9801b040 con 0x55716e6ea2d0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fabfff700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009340 tx=0x7f4f98004650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8c0c191eae03dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4f8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980051c0 con 0x55716e6ea2d0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55716e5964f0 con 0x55716e6ea2d0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.069+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4f98005320 con 0x55716e6ea2d0 2023-07-21T19:22:17.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f98013e50 con 0x55716e6ea2d0 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 10 client.18501 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x55716e453c60 con 0x55716e6ea2d0 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55716e5964f0 con 0x55716e6ea2d0 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4f98020460 con 0x55716e6ea2d0 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 10 client.18501 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 10 client.18501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 20 client.18501 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:17.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e705160 0x55716e707570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 4 client.18501 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.070+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55716e635eb0 con 0x55716e705160 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.071+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e705160 0x55716e707570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.071+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e705160 0x55716e707570 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=896d05e183887031 server_cookie=d3ec0fc70e52ce64 in_seq=0 out_seq=0 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.071+0000 7f4f8bfff700 10 client.18501 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4f8bfff700 1 -- 172.21.15.5:0/2971802457 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55716e635eb0 con 0x55716e705160 2023-07-21T19:22:17.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4f8bfff700 10 client.18501 handle_command_reply: tid=0 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 client.18501 shutdown 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 msgr2=0x55716e6f54b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f4f98009340 tx=0x7f4f98004650 comp rx=0 tx=0).stop 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 shutdown_connections 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6f5ad0 0x55716e6f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e705160 0x55716e707570 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6f54b0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/2971802457 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6eabe0 0x55716e6f2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4f8bfff700 0 client.18501 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 >> 172.21.15.5:0/2971802457 conn(0x55716e6e0da0 msgr2=0x55716e6e6290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 shutdown_connections 2023-07-21T19:22:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.072+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/2971802457 wait complete. 2023-07-21T19:22:17.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.073+0000 7f4fc6b27b80 20 client.18501 trim_cache size 0 max 16384 2023-07-21T19:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 Processor -- start 2023-07-21T19:22:17.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 -- start start 2023-07-21T19:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 0x55716e6e4e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6f5ad0 0x55716e6eda30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6eb540 con 0x55716e6f5ad0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6ebd60 con 0x55716e6ea2d0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.090+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6ec1a0 con 0x55716e6eabe0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 0x55716e6e4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6f5ad0 0x55716e6eda30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35162/0 (socket says 172.21.15.5:35162) 2023-07-21T19:22:17.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 -- 172.21.15.5:0/951339275 learned_addr learned my addr 172.21.15.5:0/951339275 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 -- 172.21.15.5:0/951339275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 msgr2=0x55716e6e4e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 0x55716e6e4e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 -- 172.21.15.5:0/951339275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6f5ad0 msgr2=0x55716e6eda30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6f5ad0 0x55716e6eda30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 -- 172.21.15.5:0/951339275 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716e491250 con 0x55716e6eabe0 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fabfff700 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 0x55716e6e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000c00 tx=0x7f4fa0005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c0850ded3c99a379 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.091+0000 7f4f977fe700 1 -- 172.21.15.5:0/951339275 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa000b910 con 0x55716e6eabe0 2023-07-21T19:22:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4f977fe700 1 -- 172.21.15.5:0/951339275 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4fa001e040 con 0x55716e6eabe0 2023-07-21T19:22:17.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4f977fe700 1 -- 172.21.15.5:0/951339275 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fa0004db0 con 0x55716e6eabe0 2023-07-21T19:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/951339275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 msgr2=0x55716e6f4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f4fa0000c00 tx=0x7f4fa0005ad0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/951339275 shutdown_connections 2023-07-21T19:22:17.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6f5ad0 0x55716e6eda30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6ea2d0 0x55716e6e4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/951339275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6f4fe0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/951339275 >> 172.21.15.5:0/951339275 conn(0x55716e6e0da0 msgr2=0x55716e701680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/951339275 shutdown_connections 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.092+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/951339275 wait complete. 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 -- start start 2023-07-21T19:22:17.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 Processor -- start 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6eeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6f5ad0 0x55716e6ef0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6ef610 con 0x55716e6ea2d0 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6ef780 con 0x55716e6f5ad0 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716e6ef940 con 0x55716e6eabe0 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fc6b27b80 10 client.0 _ll_register_callbacks cb 0x7f4f98005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:17.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.093+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56838/0 (socket says 172.21.15.5:56838) 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 -- 172.21.15.5:0/3966640890 learned_addr learned my addr 172.21.15.5:0/3966640890 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6eeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6f5ad0 0x55716e6ef0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 -- 172.21.15.5:0/3966640890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 msgr2=0x55716e6eeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6eeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 -- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6f5ad0 msgr2=0x55716e6ef0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6f5ad0 0x55716e6ef0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 -- 172.21.15.5:0/3966640890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55716e453c60 con 0x55716e6ea2d0 2023-07-21T19:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fabfff700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4f98004d40 tx=0x7f4f98004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81e4c5231d0435e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4f8bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:22:17.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f980209e0 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4f98004fc0 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.094+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f980044e0 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f98020b40 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 10 client.18507 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f4fac10d210 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f980044e0 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4f980037a0 con 0x55716e6ea2d0 2023-07-21T19:22:17.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 10 client.18507 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 10 client.18507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 20 client.18507 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e75a1b0 0x55716e75c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 4 client.18507 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.095+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55716e5f8050 con 0x55716e75a1b0 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e75a1b0 0x55716e75c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4fb0f9b700 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e75a1b0 0x55716e75c5c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d375daf46f4500a server_cookie=68ab996e111f5768 in_seq=0 out_seq=0 2023-07-21T19:22:17.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4f8bfff700 10 client.18507 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4f8bfff700 1 -- 172.21.15.5:0/3966640890 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+265 (crc 0 0 0) 0x55716e5f8050 con 0x55716e75a1b0 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4f8bfff700 10 client.18507 handle_command_reply: tid=0 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4fc6b27b80 1 client.18507 shutdown 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 msgr2=0x55716e6fc4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.096+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f4f98004d40 tx=0x7f4f98004a60 comp rx=0 tx=0).stop 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 shutdown_connections 2023-07-21T19:22:17.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716e6f5ad0 0x55716e6ef0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716e75a1b0 0x55716e75c5c0 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716e6ea2d0 0x55716e6fc4b0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4f8bfff700 0 client.18507 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 --2- 172.21.15.5:0/3966640890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716e6eabe0 0x55716e6eeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 >> 172.21.15.5:0/3966640890 conn(0x55716e6e0da0 msgr2=0x55716e700c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 shutdown_connections 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 1 -- 172.21.15.5:0/3966640890 wait complete. 2023-07-21T19:22:17.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fc6b27b80 20 client.18507 trim_cache size 0 max 16384 2023-07-21T19:22:17.098 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:22:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "scrub active (1 inodes in the stack)", 2023-07-21T19:22:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:22:17.098 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:22:17.099 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 msgr2=0x7f4f9c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 0x7f4f9c0a37f0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f4fa40c9d00 tx=0x7f4fa4004ad0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 msgr2=0x7f4fac19b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:17.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.097+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f4fa400ef30 tx=0x7f4fa400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 shutdown_connections 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f9c0a1340 0x7f4f9c0a37f0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4fac105ae0 0x7f4fac19b5b0 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4fac0ffad0 0x7f4fac197cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 --2- 172.21.15.5:0/1690718531 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4fac100480 0x7f4fac198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 >> 172.21.15.5:0/1690718531 conn(0x7f4fac0fb5f0 msgr2=0x7f4fac10acb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:17.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 shutdown_connections 2023-07-21T19:22:17.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:17.098+0000 7f4fb31ff700 1 -- 172.21.15.5:0/1690718531 wait complete. 2023-07-21T19:22:17.107 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:22:17.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:17 smithi101 ceph-mon[128136]: pgmap v461: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:22:17.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:17 smithi005 ceph-mon[126460]: pgmap v461: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:22:17.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:17 smithi156 ceph-mon[128316]: pgmap v461: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 16 MiB/s wr, 22 op/s 2023-07-21T19:22:19.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:19 smithi101 ceph-mon[128136]: pgmap v462: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:19.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:19 smithi005 ceph-mon[126460]: pgmap v462: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:19.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:19 smithi156 ceph-mon[128316]: pgmap v462: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:21.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:21 smithi101 ceph-mon[128136]: pgmap v463: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:21.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:21 smithi005 ceph-mon[126460]: pgmap v463: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:21.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:21 smithi156 ceph-mon[128316]: pgmap v463: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 2.3 MiB/s wr, 12 op/s 2023-07-21T19:22:23.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:23 smithi101 ceph-mon[128136]: pgmap v464: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2023-07-21T19:22:23.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:23 smithi005 ceph-mon[126460]: pgmap v464: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2023-07-21T19:22:23.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:23 smithi156 ceph-mon[128316]: pgmap v464: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 1.6 MiB/s wr, 12 op/s 2023-07-21T19:22:25.828 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:25 smithi101 ceph-mon[128136]: pgmap v465: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:25.903 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:25 smithi005 ceph-mon[126460]: pgmap v465: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:25.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:25 smithi156 ceph-mon[128316]: pgmap v465: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 -- 172.21.15.5:0/4129078255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc10c410 msgr2=0x7f97cc10e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 --2- 172.21.15.5:0/4129078255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc10c410 0x7f97cc10e7f0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009f90 tx=0x7f97c400b330 comp rx=0 tx=0).stop 2023-07-21T19:22:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 -- 172.21.15.5:0/4129078255 shutdown_connections 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 --2- 172.21.15.5:0/4129078255 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc10c410 0x7f97cc10e7f0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 --2- 172.21.15.5:0/4129078255 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc072830 0x7f97cc072c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 --2- 172.21.15.5:0/4129078255 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc073140 0x7f97cc10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 -- 172.21.15.5:0/4129078255 >> 172.21.15.5:0/4129078255 conn(0x7f97cc0fb650 msgr2=0x7f97cc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 -- 172.21.15.5:0/4129078255 shutdown_connections 2023-07-21T19:22:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.418+0000 7f97d1bff700 1 -- 172.21.15.5:0/4129078255 wait complete. 2023-07-21T19:22:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.419+0000 7f97d1bff700 1 Processor -- start 2023-07-21T19:22:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.419+0000 7f97d1bff700 1 -- start start 2023-07-21T19:22:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc073140 0x7f97cc1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc10c410 0x7f97cc19ca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19d050 con 0x7f97cc073140 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19d1c0 con 0x7f97cc10c410 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97d1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97cc19d380 con 0x7f97cc072830 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48254/0 (socket says 172.21.15.5:48254) 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 -- 172.21.15.5:0/1249413615 learned_addr learned my addr 172.21.15.5:0/1249413615 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97caffd700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc073140 0x7f97cc1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cbfff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc10c410 0x7f97cc19ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 -- 172.21.15.5:0/1249413615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc10c410 msgr2=0x7f97cc19ca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc10c410 0x7f97cc19ca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 -- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc073140 msgr2=0x7f97cc1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.420+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc073140 0x7f97cc1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97cb7fe700 1 -- 172.21.15.5:0/1249413615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c4009c40 con 0x7f97cc072830 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00e9c0 tx=0x7f97bc00ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97c8ff9700 1 -- 172.21.15.5:0/1249413615 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc004a30 con 0x7f97cc072830 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97cc19d660 con 0x7f97cc072830 2023-07-21T19:22:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97c8ff9700 1 -- 172.21.15.5:0/1249413615 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97bc00cc50 con 0x7f97cc072830 2023-07-21T19:22:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97c8ff9700 1 -- 172.21.15.5:0/1249413615 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc005410 con 0x7f97cc072830 2023-07-21T19:22:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.421+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc1a4ea0 con 0x7f97cc072830 2023-07-21T19:22:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.425+0000 7f97c8ff9700 1 -- 172.21.15.5:0/1249413615 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f97bc005570 con 0x7f97cc072830 2023-07-21T19:22:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.426+0000 7f97e557ab80 1 Processor -- start 2023-07-21T19:22:26.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.426+0000 7f97e557ab80 1 -- start start 2023-07-21T19:22:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eff88b6b0 0x564effa34d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 0x564effa2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff934c50 con 0x564effa352d0 2023-07-21T19:22:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff9879e0 con 0x564effa2f580 2023-07-21T19:22:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97e557ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff980e70 con 0x564eff88b6b0 2023-07-21T19:22:26.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97c8ff9700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 0x7f97b40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97c8ff9700 1 -- 172.21.15.5:0/1249413615 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f97bc014070 con 0x7f97cc072830 2023-07-21T19:22:26.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 0x564effa2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97cbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eff88b6b0 0x564effa34d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97caffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97caffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55716/0 (socket says 172.21.15.5:55716) 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.427+0000 7f97caffd700 1 -- 172.21.15.5:0/4017860562 learned_addr learned my addr 172.21.15.5:0/4017860562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97cbfff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 0x7f97b40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 -- 172.21.15.5:0/4017860562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eff88b6b0 msgr2=0x564effa34d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eff88b6b0 0x564effa34d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 -- 172.21.15.5:0/4017860562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 msgr2=0x564effa2f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 0x564effa2f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 -- 172.21.15.5:0/4017860562 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eff7aed80 con 0x564effa2f580 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 0x564effa2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97caffd700 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c0011440 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73c2ff89bf233ae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97cbfff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 0x7f97b40a37f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f97c4007900 tx=0x7f97c40096c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.428+0000 7f97b1ffb700 1 -- 172.21.15.5:0/4017860562 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0011d80 con 0x564effa2f580 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97b1ffb700 1 -- 172.21.15.5:0/4017860562 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97c0004cc0 con 0x564effa2f580 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97b1ffb700 1 -- 172.21.15.5:0/4017860562 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0019430 con 0x564effa2f580 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 -- 172.21.15.5:0/4017860562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 msgr2=0x564effa2f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f97c0009ca0 tx=0x7f97c0011440 comp rx=0 tx=0).stop 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 -- 172.21.15.5:0/4017860562 shutdown_connections 2023-07-21T19:22:26.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa2f580 0x564effa2f9d0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eff88b6b0 0x564effa34d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 --2- 172.21.15.5:0/4017860562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa352d0 0x564effa2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.429+0000 7f97e557ab80 1 -- 172.21.15.5:0/4017860562 >> 172.21.15.5:0/4017860562 conn(0x564effa24be0 msgr2=0x564effa29f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.430+0000 7f97e557ab80 1 -- 172.21.15.5:0/4017860562 shutdown_connections 2023-07-21T19:22:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.430+0000 7f97e557ab80 1 -- 172.21.15.5:0/4017860562 wait complete. 2023-07-21T19:22:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.430+0000 7f97e557ab80 1 -- start start 2023-07-21T19:22:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.430+0000 7f97e557ab80 1 Processor -- start 2023-07-21T19:22:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa38770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa38d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff920a60 con 0x564effa2f580 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff920760 con 0x564eff88b6b0 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eff8048c0 con 0x564effa352d0 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97e557ab80 10 client.0 _ll_register_callbacks cb 0x564eff91e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55726/0 (socket says 172.21.15.5:55726) 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.431+0000 7f97cb7fe700 1 -- 172.21.15.5:0/3043979494 learned_addr learned my addr 172.21.15.5:0/3043979494 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cbfff700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa38d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97caffd700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa38770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 -- 172.21.15.5:0/3043979494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 msgr2=0x564effa38d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa38d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 -- 172.21.15.5:0/3043979494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 msgr2=0x564effa38770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa38770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 -- 172.21.15.5:0/3043979494 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97cc1a4ea0 con 0x564eff88b6b0 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cbfff700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa38d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97caffd700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa38770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97bc099af0 tx=0x7f97bc099cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=574e3816c3515bcd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.432+0000 7f97aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:26.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97bc09a250 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f97bc00cdc0 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eff8da490 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97bc0ca900 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97e557ab80 10 client.28172 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x7f97c0009900 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97bc0bd080 con 0x564eff88b6b0 2023-07-21T19:22:26.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.433+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f97bc0992e0 con 0x564eff88b6b0 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 10 client.28172 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 10 client.28172 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 20 client.28172 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa48ad0 0x564effa4aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 4 client.28172 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eff979e50 con 0x564effa48ad0 2023-07-21T19:22:26.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.434+0000 7f97caffd700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa48ad0 0x564effa4aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.435+0000 7f97caffd700 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa48ad0 0x564effa4aee0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e4e0d9513ab3bf server_cookie=330f0c1d758d50b3 in_seq=0 out_seq=0 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.435+0000 7f97aaffd700 10 client.28172 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97aaffd700 1 -- 172.21.15.5:0/3043979494 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564eff979e50 con 0x564effa48ad0 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97aaffd700 10 client.28172 handle_command_reply: tid=0 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 client.28172 shutdown 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 msgr2=0x564effa3b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f97bc099af0 tx=0x7f97bc099cf0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 shutdown_connections 2023-07-21T19:22:26.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa38d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa3b560 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa48ad0 0x564effa4aee0 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3043979494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa38770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97aaffd700 0 client.28172 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.436+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 >> 172.21.15.5:0/3043979494 conn(0x564effa24be0 msgr2=0x564effa294e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.437+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 shutdown_connections 2023-07-21T19:22:26.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.437+0000 7f97e557ab80 1 -- 172.21.15.5:0/3043979494 wait complete. 2023-07-21T19:22:26.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.437+0000 7f97e557ab80 20 client.28172 trim_cache size 0 max 16384 2023-07-21T19:22:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.458+0000 7f97e557ab80 1 Processor -- start 2023-07-21T19:22:26.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.458+0000 7f97e557ab80 1 -- start start 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 0x564effa3fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa21bb0 con 0x564eff88b6b0 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa2b800 con 0x564effa352d0 2023-07-21T19:22:26.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97e557ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa2ffd0 con 0x564effa2f580 2023-07-21T19:22:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97cbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97cbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55730/0 (socket says 172.21.15.5:55730) 2023-07-21T19:22:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97cbfff700 1 -- 172.21.15.5:0/3882078291 learned_addr learned my addr 172.21.15.5:0/3882078291 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:26.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37848/0 (socket says 172.21.15.5:37848) 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.459+0000 7f97caffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 0x564effa3fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 -- 172.21.15.5:0/3882078291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 msgr2=0x564effa3fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 0x564effa3fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 -- 172.21.15.5:0/3882078291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 msgr2=0x564effa2f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 -- 172.21.15.5:0/3882078291 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eff797c00 con 0x564effa352d0 2023-07-21T19:22:26.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97caffd700 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 0x564effa3fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97cbfff700 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009f90 tx=0x7f97c4021000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4850dcf448af96f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97b27fc700 1 -- 172.21.15.5:0/3882078291 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c400e040 con 0x564effa352d0 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97b27fc700 1 -- 172.21.15.5:0/3882078291 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f97c400be80 con 0x564effa352d0 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.460+0000 7f97b27fc700 1 -- 172.21.15.5:0/3882078291 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c40049f0 con 0x564effa352d0 2023-07-21T19:22:26.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 -- 172.21.15.5:0/3882078291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 msgr2=0x564effa29c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f97c4009f90 tx=0x7f97c4021000 comp rx=0 tx=0).stop 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 -- 172.21.15.5:0/3882078291 shutdown_connections 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564effa352d0 0x564effa29c60 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eff88b6b0 0x564effa2f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 --2- 172.21.15.5:0/3882078291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa2f580 0x564effa3fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 -- 172.21.15.5:0/3882078291 >> 172.21.15.5:0/3882078291 conn(0x564effa24be0 msgr2=0x564effa32a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 -- 172.21.15.5:0/3882078291 shutdown_connections 2023-07-21T19:22:26.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.461+0000 7f97e557ab80 1 -- 172.21.15.5:0/3882078291 wait complete. 2023-07-21T19:22:26.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 -- start start 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 Processor -- start 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa32050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa30790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa30cd0 con 0x564effa2f580 2023-07-21T19:22:26.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa30e40 con 0x564eff88b6b0 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564effa31000 con 0x564effa352d0 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.462+0000 7f97e557ab80 10 client.0 _ll_register_callbacks cb 0x7f97bc099c20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa32050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97cb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55740/0 (socket says 172.21.15.5:55740) 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97cb7fe700 1 -- 172.21.15.5:0/1648652851 learned_addr learned my addr 172.21.15.5:0/1648652851 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:26.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97cbfff700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa30790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 -- 172.21.15.5:0/1648652851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 msgr2=0x564effa30790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa30790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 -- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 msgr2=0x564effa31b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97caffd700 1 -- 172.21.15.5:0/1648652851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c0009900 con 0x564effa2f580 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.463+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:26.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97caffd700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa32050 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f97c0000c00 tx=0x7f97c00058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20d2aee24c64d84b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97aaffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c0011d80 con 0x564effa2f580 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eff93bff0 con 0x564effa2f580 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f97c002f070 con 0x564effa2f580 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=47) v2 ==== 24+0+0 (secure 0 0 0) 0x7f97c0005330 con 0x564effa2f580 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97e557ab80 10 client.18525 fetch_fsmap learned FSMap version 47 2023-07-21T19:22:26.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=47,monmap=4+}) v3 -- 0x564eff7aed80 con 0x564effa2f580 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.464+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564eff93bff0 con 0x564effa2f580 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 47) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f97c0029a50 con 0x564effa2f580 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 10 client.18525 fetch_fsmap finished waiting for FSMap version 47 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 10 client.18525 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 20 client.18525 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa9d7d0 0x564effa9fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:26.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 4 client.18525 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:22:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564eff7cff10 con 0x564effa9d7d0 2023-07-21T19:22:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.465+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa9d7d0 0x564effa9fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.466+0000 7f97cb7fe700 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa9d7d0 0x564effa9fbe0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ceb1f05cd0edd9 server_cookie=2447d166bceaa3a4 in_seq=0 out_seq=0 2023-07-21T19:22:26.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.466+0000 7f97aaffd700 10 client.18525 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97aaffd700 1 -- 172.21.15.5:0/1648652851 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564eff7cff10 con 0x564effa9d7d0 2023-07-21T19:22:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97aaffd700 10 client.18525 handle_command_reply: tid=0 2023-07-21T19:22:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 client.18525 shutdown 2023-07-21T19:22:26.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 msgr2=0x564effa32050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa32050 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f97c0000c00 tx=0x7f97c00058e0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 shutdown_connections 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564effa352d0 0x564effa30790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eff88b6b0 0x564effa31b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564effa9d7d0 0x564effa9fbe0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 --2- 172.21.15.5:0/1648652851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564effa2f580 0x564effa32050 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97aaffd700 0 client.18525 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 >> 172.21.15.5:0/1648652851 conn(0x564effa24be0 msgr2=0x564effa38010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.475+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 shutdown_connections 2023-07-21T19:22:26.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97e557ab80 1 -- 172.21.15.5:0/1648652851 wait complete. 2023-07-21T19:22:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97e557ab80 20 client.18525 trim_cache size 0 max 16384 2023-07-21T19:22:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:22:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:22:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:22:26.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:22:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 msgr2=0x7f97b40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 0x7f97b40a37f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f97c4007900 tx=0x7f97c40096c0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 msgr2=0x7f97cc197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:26.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f97bc00e9c0 tx=0x7f97bc00ecd0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 shutdown_connections 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f97cc10c410 0x7f97cc19ca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f97cc072830 0x7f97cc197e60 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f97b40a1340 0x7f97b40a37f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.476+0000 7f97d1bff700 1 --2- 172.21.15.5:0/1249413615 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f97cc073140 0x7f97cc1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.477+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 >> 172.21.15.5:0/1249413615 conn(0x7f97cc0fb650 msgr2=0x7f97cc10e5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:26.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.477+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 shutdown_connections 2023-07-21T19:22:26.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:26.477+0000 7f97d1bff700 1 -- 172.21.15.5:0/1249413615 wait complete. 2023-07-21T19:22:27.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:27 smithi101 ceph-mon[128136]: pgmap v466: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:27.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:27 smithi005 ceph-mon[126460]: pgmap v466: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:27.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:27 smithi156 ceph-mon[128316]: pgmap v466: 97 pgs: 97 active+clean; 11 GiB data, 5.0 GiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 2.4 MiB/s wr, 10 op/s 2023-07-21T19:22:29.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:29 smithi101 ceph-mon[128136]: pgmap v467: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 20 op/s 2023-07-21T19:22:29.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:29 smithi005 ceph-mon[126460]: pgmap v467: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 20 op/s 2023-07-21T19:22:29.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:29 smithi156 ceph-mon[128316]: pgmap v467: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 14 MiB/s wr, 20 op/s 2023-07-21T19:22:31.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:31 smithi101 ceph-mon[128136]: pgmap v468: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:22:31.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:31 smithi005 ceph-mon[126460]: pgmap v468: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:22:31.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:31 smithi156 ceph-mon[128316]: pgmap v468: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:22:33.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:33 smithi101 ceph-mon[128136]: pgmap v469: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 22 op/s 2023-07-21T19:22:33.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:33 smithi005 ceph-mon[126460]: pgmap v469: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 22 op/s 2023-07-21T19:22:33.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:33 smithi156 ceph-mon[128316]: pgmap v469: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 25 MiB/s wr, 22 op/s 2023-07-21T19:22:35.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:35 smithi101 ceph-mon[128136]: pgmap v470: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:35.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:35 smithi005 ceph-mon[126460]: pgmap v470: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:35.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:35 smithi156 ceph-mon[128316]: pgmap v470: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:37.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:37 smithi101 ceph-mon[128136]: pgmap v471: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:37.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:37 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:37.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:37 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:22:37.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:37 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:22:37.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:37 smithi005 ceph-mon[126460]: pgmap v471: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:37.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:37 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:37.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:37 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:22:37.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:37 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:22:37.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:37 smithi156 ceph-mon[128316]: pgmap v471: 97 pgs: 97 active+clean; 12 GiB data, 5.1 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:37.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:37 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:37.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:37 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:22:37.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:37 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:22:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 -- 172.21.15.5:0/4151548645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a0105940 msgr2=0x7f25a0105d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 --2- 172.21.15.5:0/4151548645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a0105940 0x7f25a0105d10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2590009990 tx=0x7f2590009ca0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 -- 172.21.15.5:0/4151548645 shutdown_connections 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 --2- 172.21.15.5:0/4151548645 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 0x7f25a010c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 --2- 172.21.15.5:0/4151548645 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a0105940 0x7f25a0105d10 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.642+0000 7f25a519d700 1 --2- 172.21.15.5:0/4151548645 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a00ff930 0x7f25a00ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.643+0000 7f25a519d700 1 -- 172.21.15.5:0/4151548645 >> 172.21.15.5:0/4151548645 conn(0x7f25a00fb450 msgr2=0x7f25a00fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.643+0000 7f25a519d700 1 -- 172.21.15.5:0/4151548645 shutdown_connections 2023-07-21T19:22:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.643+0000 7f25a519d700 1 -- 172.21.15.5:0/4151548645 wait complete. 2023-07-21T19:22:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 Processor -- start 2023-07-21T19:22:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 -- start start 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 0x7f25a019dea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a0105940 0x7f25a01979d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0198040 con 0x7f25a01002e0 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a01981b0 con 0x7f25a0105940 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.644+0000 7f25a519d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25a0198370 con 0x7f25a00ff930 2023-07-21T19:22:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 0x7f25a019dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36980/0 (socket says 172.21.15.5:36980) 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 -- 172.21.15.5:0/3154086635 learned_addr learned my addr 172.21.15.5:0/3154086635 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f25a499c700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a0105940 0x7f25a01979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 -- 172.21.15.5:0/3154086635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a0105940 msgr2=0x7f25a01979d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a0105940 0x7f25a01979d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 -- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 msgr2=0x7f25a019dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 0x7f25a019dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 -- 172.21.15.5:0/3154086635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2598009c40 con 0x7f25a00ff930 2023-07-21T19:22:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.645+0000 7f259ffff700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25900098e0 tx=0x7f2590004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.646+0000 7f259d7fa700 1 -- 172.21.15.5:0/3154086635 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259001d070 con 0x7f25a00ff930 2023-07-21T19:22:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.646+0000 7f259d7fa700 1 -- 172.21.15.5:0/3154086635 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2590022470 con 0x7f25a00ff930 2023-07-21T19:22:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.646+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25900095f0 con 0x7f25a00ff930 2023-07-21T19:22:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.646+0000 7f259d7fa700 1 -- 172.21.15.5:0/3154086635 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2590017650 con 0x7f25a00ff930 2023-07-21T19:22:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.646+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25a01a4770 con 0x7f25a00ff930 2023-07-21T19:22:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.649+0000 7f259d7fa700 1 -- 172.21.15.5:0/3154086635 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f25900225e0 con 0x7f25a00ff930 2023-07-21T19:22:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.651+0000 7f259d7fa700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.651+0000 7f25b9d2fb80 1 Processor -- start 2023-07-21T19:22:39.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.651+0000 7f259d7fa700 1 -- 172.21.15.5:0/3154086635 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25900ce100 con 0x7f25a00ff930 2023-07-21T19:22:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 -- start start 2023-07-21T19:22:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f259f7fe700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df4a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 0x55a97df4af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97de47c50 con 0x55a97dd9e6b0 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97de9a9e0 con 0x55a97df510d0 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97de93e70 con 0x55a97df50c80 2023-07-21T19:22:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f259ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f259ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50746/0 (socket says 172.21.15.5:50746) 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f259ffff700 1 -- 172.21.15.5:0/485479914 learned_addr learned my addr 172.21.15.5:0/485479914 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.652+0000 7f25a499c700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 0x55a97df4af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259f7fe700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df4a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259f7fe700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 0x7f25880a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f25a00fed80 tx=0x7f2594009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 -- 172.21.15.5:0/485479914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 msgr2=0x55a97df4a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df4a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 -- 172.21.15.5:0/485479914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 msgr2=0x55a97df4af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 0x55a97df4af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 -- 172.21.15.5:0/485479914 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97dcc1d80 con 0x55a97dd9e6b0 2023-07-21T19:22:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f25a499c700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 0x55a97df4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:39.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f259ffff700 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f25900a3090 tx=0x7f259009afd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b4599113f73fa40 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f25867fc700 1 -- 172.21.15.5:0/485479914 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259009ab60 con 0x55a97dd9e6b0 2023-07-21T19:22:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f25867fc700 1 -- 172.21.15.5:0/485479914 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25900a10e0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.653+0000 7f25867fc700 1 -- 172.21.15.5:0/485479914 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259009b6c0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/485479914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 msgr2=0x55a97df50740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f25900a3090 tx=0x7f259009afd0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/485479914 shutdown_connections 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97df510d0 0x55a97df4af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97dd9e6b0 0x55a97df50740 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/485479914 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df4a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/485479914 >> 172.21.15.5:0/485479914 conn(0x55a97df37d40 msgr2=0x55a97df3d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.654+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/485479914 shutdown_connections 2023-07-21T19:22:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.655+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/485479914 wait complete. 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.655+0000 7f25b9d2fb80 1 -- start start 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.655+0000 7f25b9d2fb80 1 Processor -- start 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df59cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df53fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97de33a60 con 0x55a97df510d0 2023-07-21T19:22:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97de33760 con 0x55a97dd9e6b0 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97dd178c0 con 0x55a97df50c80 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25b9d2fb80 10 client.0 _ll_register_callbacks cb 0x55a97de31700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f259ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df59cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36990/0 (socket says 172.21.15.5:36990) 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f259f7fe700 1 -- 172.21.15.5:0/2799363240 learned_addr learned my addr 172.21.15.5:0/2799363240 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.656+0000 7f25a499c700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df53fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 -- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 msgr2=0x55a97df59cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df59cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 -- 172.21.15.5:0/2799363240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 msgr2=0x55a97df53fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df53fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 -- 172.21.15.5:0/2799363240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f259800e330 con 0x55a97df50c80 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259ffff700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df59cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f25a499c700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df53fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f259f7fe700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f259401d850 tx=0x7f25940047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95f20c03b41fea82 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.657+0000 7f257f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:22:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2594022070 con 0x55a97df50c80 2023-07-21T19:22:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f259401c6e0 con 0x55a97df50c80 2023-07-21T19:22:39.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a97dded490 con 0x55a97df50c80 2023-07-21T19:22:39.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259401c8a0 con 0x55a97df50c80 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f25b9d2fb80 10 client.28114 fetch_fsmap learned FSMap version 48 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x7f25a01a4770 con 0x55a97df50c80 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2594033080 con 0x55a97df50c80 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 48) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f259402d030 con 0x55a97df50c80 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f25b9d2fb80 10 client.28114 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.658+0000 7f25b9d2fb80 10 client.28114 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f25b9d2fb80 20 client.28114 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97df64250 0x55a97df66660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f25b9d2fb80 4 client.28114 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a97de8ce50 con 0x55a97df64250 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f259ffff700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97df64250 0x55a97df66660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f259ffff700 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97df64250 0x55a97df66660 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c02ecd1d809b227 server_cookie=a94d11ca60647ee3 in_seq=0 out_seq=0 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.659+0000 7f257f7fe700 10 client.28114 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f257f7fe700 1 -- 172.21.15.5:0/2799363240 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a97de8ce50 con 0x55a97df64250 2023-07-21T19:22:39.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f257f7fe700 10 client.28114 handle_command_reply: tid=0 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 client.28114 shutdown 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 msgr2=0x55a97df5a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f259401d850 tx=0x7f25940047c0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 shutdown_connections 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97df64250 0x55a97df66660 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df53fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df59cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f257f7fe700 0 client.28114 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/2799363240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df5a230 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.660+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 >> 172.21.15.5:0/2799363240 conn(0x55a97df37d40 msgr2=0x55a97df3c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.661+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 shutdown_connections 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.661+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/2799363240 wait complete. 2023-07-21T19:22:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.661+0000 7f25b9d2fb80 20 client.28114 trim_cache size 0 max 16384 2023-07-21T19:22:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.683+0000 7f25b9d2fb80 1 Processor -- start 2023-07-21T19:22:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.683+0000 7f25b9d2fb80 1 -- start start 2023-07-21T19:22:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df3cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df3d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97df39e60 con 0x55a97df510d0 2023-07-21T19:22:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97df55520 con 0x55a97dd9e6b0 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97df3d840 con 0x55a97df50c80 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25a499c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df3d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f259ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f259f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df3cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f259ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49124/0 (socket says 172.21.15.5:49124) 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f259ffff700 1 -- 172.21.15.5:0/3509667244 learned_addr learned my addr 172.21.15.5:0/3509667244 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:39.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.684+0000 7f25a499c700 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df3d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50764/0 (socket says 172.21.15.5:50764) 2023-07-21T19:22:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 -- 172.21.15.5:0/3509667244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 msgr2=0x55a97df3cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df3cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 -- 172.21.15.5:0/3509667244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 msgr2=0x55a97df3d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df3d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 -- 172.21.15.5:0/3509667244 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a97df54240 con 0x55a97dd9e6b0 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f259ffff700 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f259000b4f0 tx=0x7f25900094b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b680d15aac60ddb3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f2586ffd700 1 -- 172.21.15.5:0/3509667244 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259009ab60 con 0x55a97dd9e6b0 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f2586ffd700 1 -- 172.21.15.5:0/3509667244 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25900228c0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.685+0000 7f2586ffd700 1 -- 172.21.15.5:0/3509667244 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259009b6a0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/3509667244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 msgr2=0x55a97df53790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f259000b4f0 tx=0x7f25900094b0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/3509667244 shutdown_connections 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df510d0 0x55a97df3d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df53790 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/3509667244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df50c80 0x55a97df3cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/3509667244 >> 172.21.15.5:0/3509667244 conn(0x55a97df37d40 msgr2=0x55a97df4e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/3509667244 shutdown_connections 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.686+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/3509667244 wait complete. 2023-07-21T19:22:39.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.687+0000 7f25b9d2fb80 1 -- start start 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.687+0000 7f25b9d2fb80 1 Processor -- start 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 0x55a97df57e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df510d0 0x55a97df58340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97dfb7200 con 0x55a97df50c80 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97dfb7370 con 0x55a97dd9e6b0 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a97dfb7530 con 0x55a97df510d0 2023-07-21T19:22:39.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25b9d2fb80 10 client.0 _ll_register_callbacks cb 0x7f259401d410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f259ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f259ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49126/0 (socket says 172.21.15.5:49126) 2023-07-21T19:22:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f259ffff700 1 -- 172.21.15.5:0/1310585954 learned_addr learned my addr 172.21.15.5:0/1310585954 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f25a499c700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df510d0 0x55a97df58340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.688+0000 7f259f7fe700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 0x55a97df57e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 -- 172.21.15.5:0/1310585954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df510d0 msgr2=0x55a97df58340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df510d0 0x55a97df58340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 -- 172.21.15.5:0/1310585954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 msgr2=0x55a97df57e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 0x55a97df57e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 -- 172.21.15.5:0/1310585954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a97dcc1d80 con 0x55a97dd9e6b0 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259f7fe700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 0x55a97df57e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f259ffff700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f259000b520 tx=0x7f25900cbd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b0c30a3ee6a4864 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:39.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f257f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f259009b660 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f25900a35c0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.689+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f25900a10e0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=48) v2 ==== 24+0+0 (secure 0 0 0) 0x7f259009b200 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 10 client.28196 fetch_fsmap learned FSMap version 48 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=48,monmap=4+}) v3 -- 0x55a97dcaac00 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f25900a10e0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 48) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f25900cb2b0 con 0x55a97dd9e6b0 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 10 client.28196 fetch_fsmap finished waiting for FSMap version 48 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 10 client.28196 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 20 client.28196 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97dfb8790 0x55a97dfbab70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 4 client.28196 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a97de4eff0 con 0x55a97dfb8790 2023-07-21T19:22:39.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.690+0000 7f259f7fe700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97dfb8790 0x55a97dfbab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.691+0000 7f257f7fe700 10 client.28196 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:39.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.691+0000 7f259f7fe700 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97dfb8790 0x55a97dfbab70 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e59c31f74cc48dc server_cookie=6d96dc08e035ee42 in_seq=0 out_seq=0 2023-07-21T19:22:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f257f7fe700 1 -- 172.21.15.5:0/1310585954 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a97de4eff0 con 0x55a97dfb8790 2023-07-21T19:22:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f257f7fe700 10 client.28196 handle_command_reply: tid=0 2023-07-21T19:22:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 client.28196 shutdown 2023-07-21T19:22:39.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 msgr2=0x55a97df578c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f259000b520 tx=0x7f25900cbd50 comp rx=0 tx=0).stop 2023-07-21T19:22:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 shutdown_connections 2023-07-21T19:22:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a97df510d0 0x55a97df58340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a97dd9e6b0 0x55a97df578c0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a97dfb8790 0x55a97dfbab70 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 --2- 172.21.15.5:0/1310585954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a97df50c80 0x55a97df57e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 >> 172.21.15.5:0/1310585954 conn(0x55a97df37d40 msgr2=0x55a97df41190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f257f7fe700 0 client.28196 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 shutdown_connections 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 1 -- 172.21.15.5:0/1310585954 wait complete. 2023-07-21T19:22:39.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.699+0000 7f25b9d2fb80 20 client.28196 trim_cache size 0 max 16384 2023-07-21T19:22:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:22:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:22:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:22:39.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 msgr2=0x7f25880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 0x7f25880a3770 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f25a00fed80 tx=0x7f2594009260 comp rx=0 tx=0).stop 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 msgr2=0x7f25a019d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f25900098e0 tx=0x7f2590004730 comp rx=0 tx=0).stop 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 shutdown_connections 2023-07-21T19:22:39.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f25a0105940 0x7f25a01979d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f25a00ff930 0x7f25a019d960 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f25880a12c0 0x7f25880a3770 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 --2- 172.21.15.5:0/3154086635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f25a01002e0 0x7f25a019dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 >> 172.21.15.5:0/3154086635 conn(0x7f25a00fb450 msgr2=0x7f25a00fca70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 shutdown_connections 2023-07-21T19:22:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:39.700+0000 7f25a519d700 1 -- 172.21.15.5:0/3154086635 wait complete. 2023-07-21T19:22:39.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:39 smithi101 ceph-mon[128136]: pgmap v472: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 28 op/s 2023-07-21T19:22:39.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:39 smithi005 ceph-mon[126460]: pgmap v472: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 28 op/s 2023-07-21T19:22:39.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:39 smithi156 ceph-mon[128316]: pgmap v472: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 36 MiB/s wr, 28 op/s 2023-07-21T19:22:41.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:41 smithi101 ceph-mon[128136]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 17 op/s 2023-07-21T19:22:41.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:41 smithi005 ceph-mon[126460]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 17 op/s 2023-07-21T19:22:41.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:41 smithi156 ceph-mon[128316]: pgmap v473: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 25 MiB/s wr, 17 op/s 2023-07-21T19:22:42.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:42 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:42.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:42 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:43.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:42 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:22:43.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:43 smithi005 ceph-mon[126460]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:22:43.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:43 smithi156 ceph-mon[128316]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:22:44.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:43 smithi101 ceph-mon[128136]: pgmap v474: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:22:45.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:45 smithi005 ceph-mon[126460]: pgmap v475: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:45.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:45 smithi156 ceph-mon[128316]: pgmap v475: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:46.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:45 smithi101 ceph-mon[128136]: pgmap v475: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 25 MiB/s wr, 20 op/s 2023-07-21T19:22:47.108 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:22:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 -- 172.21.15.5:0/2489007656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 msgr2=0x7f9eac10c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 --2- 172.21.15.5:0/2489007656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac10c3e0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098f0 tx=0x7f9ea4009c00 comp rx=0 tx=0).stop 2023-07-21T19:22:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 -- 172.21.15.5:0/2489007656 shutdown_connections 2023-07-21T19:22:47.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 --2- 172.21.15.5:0/2489007656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac10c3e0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 --2- 172.21.15.5:0/2489007656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 --2- 172.21.15.5:0/2489007656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 0x7f9eac109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 -- 172.21.15.5:0/2489007656 >> 172.21.15.5:0/2489007656 conn(0x7f9eac0745d0 msgr2=0x7f9eac0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 -- 172.21.15.5:0/2489007656 shutdown_connections 2023-07-21T19:22:47.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.349+0000 7f9eabfff700 1 -- 172.21.15.5:0/2489007656 wait complete. 2023-07-21T19:22:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.350+0000 7f9eabfff700 1 Processor -- start 2023-07-21T19:22:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 -- start start 2023-07-21T19:22:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 0x7f9eac19ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac197f90 con 0x7f9eac100aa0 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac198100 con 0x7f9eac109fc0 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eabfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9eac1982c0 con 0x7f9eac1013b0 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56952/0 (socket says 172.21.15.5:56952) 2023-07-21T19:22:47.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eaaffd700 1 -- 172.21.15.5:0/981800425 learned_addr learned my addr 172.21.15.5:0/981800425 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 0x7f9eac19ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.351+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 -- 172.21.15.5:0/981800425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 msgr2=0x7f9eac19ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 0x7f9eac19ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 -- 172.21.15.5:0/981800425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 msgr2=0x7f9eac197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 -- 172.21.15.5:0/981800425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea4009520 con 0x7f9eac100aa0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00ef10 tx=0x7f9e9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/981800425 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c00c9b0 con 0x7f9eac100aa0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9eac198540 con 0x7f9eac100aa0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/981800425 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e9c00f040 con 0x7f9eac100aa0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/981800425 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c014480 con 0x7f9eac100aa0 2023-07-21T19:22:47.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.352+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac1a48a0 con 0x7f9eac100aa0 2023-07-21T19:22:47.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.356+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/981800425 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9e9c007690 con 0x7f9eac100aa0 2023-07-21T19:22:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.357+0000 7f9ea8ff9700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 0x7f9e980a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.358+0000 7f9ea8ff9700 1 -- 172.21.15.5:0/981800425 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9e9c0ccfd0 con 0x7f9eac100aa0 2023-07-21T19:22:47.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.358+0000 7f9ec4f81b80 1 Processor -- start 2023-07-21T19:22:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.358+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 0x7f9e980a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.358+0000 7f9ec4f81b80 1 -- start start 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.358+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee577f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaed52cb0 con 0x564eaee57d30 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaeda5a40 con 0x564eaee4d4f0 2023-07-21T19:22:47.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaed9eed0 con 0x564eaee4de00 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 0x7f9e980a3890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0fce40 tx=0x7f9e94009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3232948466 learned_addr learned my addr 172.21.15.5:0/3232948466 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3232948466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 msgr2=0x564eaee577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3232948466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 msgr2=0x564eaee5a110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5a110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.359+0000 7f9eaaffd700 1 -- 172.21.15.5:0/3232948466 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eaebccde0 con 0x564eaee4d4f0 2023-07-21T19:22:47.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.360+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a1f60 tx=0x7f9e9c099b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=380a79a18f4589f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.360+0000 7f9ea17fa700 1 -- 172.21.15.5:0/3232948466 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c09a040 con 0x564eaee4d4f0 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.360+0000 7f9ea17fa700 1 -- 172.21.15.5:0/3232948466 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e9c09e070 con 0x564eaee4d4f0 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.360+0000 7f9ea17fa700 1 -- 172.21.15.5:0/3232948466 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c099730 con 0x564eaee4d4f0 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/3232948466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 msgr2=0x564eaee4d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a1f60 tx=0x7f9e9c099b80 comp rx=0 tx=0).stop 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/3232948466 shutdown_connections 2023-07-21T19:22:47.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee4d8c0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/3232948466 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/3232948466 >> 172.21.15.5:0/3232948466 conn(0x564eaee42ff0 msgr2=0x564eaee48930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/3232948466 shutdown_connections 2023-07-21T19:22:47.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.361+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/3232948466 wait complete. 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 -- start start 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 Processor -- start 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee54990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee550d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaed3eac0 con 0x564eaee57d30 2023-07-21T19:22:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaed3e7c0 con 0x564eaee4d4f0 2023-07-21T19:22:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.362+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaec22920 con 0x564eaee4de00 2023-07-21T19:22:47.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9ec4f81b80 10 client.0 _ll_register_callbacks cb 0x564eaed3c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57712/0 (socket says 172.21.15.5:57712) 2023-07-21T19:22:47.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 -- 172.21.15.5:0/5650398 learned_addr learned my addr 172.21.15.5:0/5650398 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee54990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 -- 172.21.15.5:0/5650398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 msgr2=0x564eaee54990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee54990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 -- 172.21.15.5:0/5650398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 msgr2=0x564eaee550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.363+0000 7f9eaaffd700 1 -- 172.21.15.5:0/5650398 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eac1a48a0 con 0x564eaee4d4f0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eee0 tx=0x7f9e9c0ca660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4133691827a8064 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9e927fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b6070 con 0x564eaee4d4f0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9e9c09a040 con 0x564eaee4d4f0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564eaecf84f0 con 0x564eaee4d4f0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9e9c09a4b0 con 0x564eaee4d4f0 2023-07-21T19:22:47.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9ec4f81b80 10 client.28208 fetch_fsmap learned FSMap version 49 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564eaebb5c60 con 0x564eaee4d4f0 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.364+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e9c0ca020 con 0x564eaee4d4f0 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9e9c0ca190 con 0x564eaee4d4f0 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 10 client.28208 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 10 client.28208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 20 client.28208 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaee673d0 0x564eaee697e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 4 client.28208 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564eaed97eb0 con 0x564eaee673d0 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.365+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaee673d0 0x564eaee697e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.366+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaee673d0 0x564eaee697e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9699e7a24a07827f server_cookie=6fbf26709dee68e3 in_seq=0 out_seq=0 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.366+0000 7f9e927fc700 10 client.28208 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.366+0000 7f9e927fc700 1 -- 172.21.15.5:0/5650398 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564eaed97eb0 con 0x564eaee673d0 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9e927fc700 10 client.28208 handle_command_reply: tid=0 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 client.28208 shutdown 2023-07-21T19:22:47.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 msgr2=0x564eaee57780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00eee0 tx=0x7f9e9c0ca660 comp rx=0 tx=0).stop 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 shutdown_connections 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaee673d0 0x564eaee697e0 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee57780 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9e927fc700 0 client.28208 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/5650398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee54990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 >> 172.21.15.5:0/5650398 conn(0x564eaee42ff0 msgr2=0x564eaee48520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 shutdown_connections 2023-07-21T19:22:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/5650398 wait complete. 2023-07-21T19:22:47.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.367+0000 7f9ec4f81b80 20 client.28208 trim_cache size 0 max 16384 2023-07-21T19:22:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 Processor -- start 2023-07-21T19:22:47.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 -- start start 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee47b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee572b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee4e9d0 con 0x564eaee57d30 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee55e50 con 0x564eaee4d4f0 2023-07-21T19:22:47.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee49ef0 con 0x564eaee4de00 2023-07-21T19:22:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9ea3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee47b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57728/0 (socket says 172.21.15.5:57728) 2023-07-21T19:22:47.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2197506197 learned_addr learned my addr 172.21.15.5:0/2197506197 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.389+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2197506197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 msgr2=0x564eaee47b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee47b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2197506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 msgr2=0x564eaee572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 -- 172.21.15.5:0/2197506197 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564eaebf3250 con 0x564eaee4d4f0 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9ea3fff700 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee47b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:47.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c096ab0 tx=0x7f9e9c0ca5c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcb61dcfbdeaea2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9ea1ffb700 1 -- 172.21.15.5:0/2197506197 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b6030 con 0x564eaee4d4f0 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9ea1ffb700 1 -- 172.21.15.5:0/2197506197 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e9c0ca040 con 0x564eaee4d4f0 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.390+0000 7f9ea1ffb700 1 -- 172.21.15.5:0/2197506197 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c09a040 con 0x564eaee4d4f0 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/2197506197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 msgr2=0x564eaee5e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c096ab0 tx=0x7f9e9c0ca5c0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/2197506197 shutdown_connections 2023-07-21T19:22:47.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e810 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/2197506197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee47b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/2197506197 >> 172.21.15.5:0/2197506197 conn(0x564eaee42ff0 msgr2=0x564eaee47060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/2197506197 shutdown_connections 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.391+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/2197506197 wait complete. 2023-07-21T19:22:47.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.392+0000 7f9ec4f81b80 1 -- start start 2023-07-21T19:22:47.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.392+0000 7f9ec4f81b80 1 Processor -- start 2023-07-21T19:22:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee5c7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee5d2b0 con 0x564eaee57d30 2023-07-21T19:22:47.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee5d3f0 con 0x564eaee4d4f0 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564eaee5d5b0 con 0x564eaee4de00 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ec4f81b80 10 client.0 _ll_register_callbacks cb 0x7f9e940091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eaaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9ea3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee5c7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56990/0 (socket says 172.21.15.5:56990) 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 -- 172.21.15.5:0/953782815 learned_addr learned my addr 172.21.15.5:0/953782815 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:47.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 -- 172.21.15.5:0/953782815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 msgr2=0x564eaee5c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee5c7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 -- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 msgr2=0x564eaee5e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.393+0000 7f9eab7fe700 1 -- 172.21.15.5:0/953782815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e94017070 con 0x564eaee57d30 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9eab7fe700 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098c0 tx=0x7f9ea400f7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41167239ccf1ba7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:47.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9e927fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ea400b770 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9e9c09e070 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9ea400b8d0 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.394+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9ea400ba90 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 10 client.18561 fetch_fsmap learned FSMap version 49 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564eaebb5c60 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9e9c09e070 con 0x564eaee57d30 2023-07-21T19:22:47.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f9ea4015020 con 0x564eaee57d30 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 10 client.18561 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 10 client.18561 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 20 client.18561 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaeebc890 0x564eaeebeca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 4 client.18561 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.395+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564eaed5a050 con 0x564eaeebc890 2023-07-21T19:22:47.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaeebc890 0x564eaeebeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9eaaffd700 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaeebc890 0x564eaeebeca0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=966123c9ad671c30 server_cookie=9781927c996c7470 in_seq=0 out_seq=0 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9e927fc700 10 client.18561 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9e927fc700 1 -- 172.21.15.5:0/953782815 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564eaed5a050 con 0x564eaeebc890 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9e927fc700 10 client.18561 handle_command_reply: tid=0 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.396+0000 7f9ec4f81b80 1 client.18561 shutdown 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 msgr2=0x564eaee5cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f9ea40098c0 tx=0x7f9ea400f7d0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 shutdown_connections 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564eaeebc890 0x564eaeebeca0 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564eaee57d30 0x564eaee5cd30 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564eaee4d4f0 0x564eaee5e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 --2- 172.21.15.5:0/953782815 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564eaee4de00 0x564eaee5c7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 >> 172.21.15.5:0/953782815 conn(0x564eaee42ff0 msgr2=0x564eaee63a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9e927fc700 0 client.18561 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:47.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 shutdown_connections 2023-07-21T19:22:47.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 1 -- 172.21.15.5:0/953782815 wait complete. 2023-07-21T19:22:47.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.397+0000 7f9ec4f81b80 20 client.18561 trim_cache size 0 max 16384 2023-07-21T19:22:47.397 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:22:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:22:47.397 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:22:47.398 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:22:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 msgr2=0x7f9e980a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 0x7f9e980a3890 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9eac0fce40 tx=0x7f9e94009310 comp rx=0 tx=0).stop 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 msgr2=0x7f9eac19d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00ef10 tx=0x7f9e9c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 shutdown_connections 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9eac109fc0 0x7f9eac197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9e980a13e0 0x7f9e980a3890 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9eac100aa0 0x7f9eac19d8b0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 --2- 172.21.15.5:0/981800425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9eac1013b0 0x7f9eac19ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 >> 172.21.15.5:0/981800425 conn(0x7f9eac0745d0 msgr2=0x7f9eac0feda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 shutdown_connections 2023-07-21T19:22:47.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:22:47.398+0000 7f9eabfff700 1 -- 172.21.15.5:0/981800425 wait complete. 2023-07-21T19:22:47.412 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:22:47.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:47 smithi005 ceph-mon[126460]: pgmap v476: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:47.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:47 smithi156 ceph-mon[128316]: pgmap v476: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:48.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:47 smithi101 ceph-mon[128136]: pgmap v476: 97 pgs: 97 active+clean; 12 GiB data, 5.2 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:49.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:49 smithi005 ceph-mon[126460]: pgmap v477: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:22:49.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:49 smithi156 ceph-mon[128316]: pgmap v477: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:22:50.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:49 smithi101 ceph-mon[128136]: pgmap v477: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:22:51.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:51 smithi005 ceph-mon[126460]: pgmap v478: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:51.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:51 smithi156 ceph-mon[128316]: pgmap v478: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:52.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:51 smithi101 ceph-mon[128136]: pgmap v478: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 24 MiB/s wr, 19 op/s 2023-07-21T19:22:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 -- 172.21.15.5:0/34242450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 msgr2=0x7f3f9c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/34242450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c105ee0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3f98009990 tx=0x7f3f98009ca0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 -- 172.21.15.5:0/34242450 shutdown_connections 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/34242450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/34242450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c105ee0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/34242450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 -- 172.21.15.5:0/34242450 >> 172.21.15.5:0/34242450 conn(0x7f3f9c0fb600 msgr2=0x7f3f9c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 -- 172.21.15.5:0/34242450 shutdown_connections 2023-07-21T19:22:52.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.863+0000 7f3fa3bff700 1 -- 172.21.15.5:0/34242450 wait complete. 2023-07-21T19:22:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.864+0000 7f3fa3bff700 1 Processor -- start 2023-07-21T19:22:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.864+0000 7f3fa3bff700 1 -- start start 2023-07-21T19:22:52.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c1982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c19c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19cf70 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19d0e0 con 0x7f3f9c1004b0 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f9c19d2a0 con 0x7f3f9c105b10 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c1982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56998/0 (socket says 172.21.15.5:56998) 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa199b700 1 -- 172.21.15.5:0/3670891362 learned_addr learned my addr 172.21.15.5:0/3670891362 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.865+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c19c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 -- 172.21.15.5:0/3670891362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 msgr2=0x7f3f9c19c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c19c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 -- 172.21.15.5:0/3670891362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 msgr2=0x7f3f9c1982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c1982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 -- 172.21.15.5:0/3670891362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f980095f0 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa119a700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa199b700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005910 tx=0x7f3f980047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3f92ffd700 1 -- 172.21.15.5:0/3670891362 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9801d070 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f9c19d520 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3f92ffd700 1 -- 172.21.15.5:0/3670891362 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f98021410 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3f92ffd700 1 -- 172.21.15.5:0/3670891362 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f98017650 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.866+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f9c1a4dc0 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.870+0000 7f3f92ffd700 1 -- 172.21.15.5:0/3670891362 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3f98021580 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.871+0000 7f3fb745db80 1 Processor -- start 2023-07-21T19:22:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.871+0000 7f3fb745db80 1 -- start start 2023-07-21T19:22:52.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda0040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7eca5c50 con 0x564c7ebfc6b0 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ecf89e0 con 0x564c7eda0580 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fb745db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ecf1e70 con 0x564c7eda62d0 2023-07-21T19:22:52.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3f92ffd700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 0x7f3f880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57764/0 (socket says 172.21.15.5:57764) 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa119a700 1 -- 172.21.15.5:0/3981470247 learned_addr learned my addr 172.21.15.5:0/3981470247 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3f92ffd700 1 -- 172.21.15.5:0/3670891362 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f980cdf10 con 0x7f3f9c0ffb00 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.872+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 0x7f3f880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 -- 172.21.15.5:0/3981470247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 msgr2=0x564c7eda0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda0040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 -- 172.21.15.5:0/3981470247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 msgr2=0x564c7eda5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 -- 172.21.15.5:0/3981470247 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7eb1fd80 con 0x564c7eda0580 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa119a700 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0115f0 tx=0x7f3f8c011900 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a87adc67124f24c8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3f87fff700 1 -- 172.21.15.5:0/3981470247 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c004230 con 0x564c7eda0580 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 0x7f3f880a3770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f9400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3f87fff700 1 -- 172.21.15.5:0/3981470247 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f8c004390 con 0x564c7eda0580 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.873+0000 7f3f87fff700 1 -- 172.21.15.5:0/3981470247 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f8c019790 con 0x564c7eda0580 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 -- 172.21.15.5:0/3981470247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 msgr2=0x564c7eda09d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f3f8c0115f0 tx=0x7f3f8c011900 comp rx=0 tx=0).stop 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 -- 172.21.15.5:0/3981470247 shutdown_connections 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda0580 0x564c7eda09d0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3981470247 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.874+0000 7f3fb745db80 1 -- 172.21.15.5:0/3981470247 >> 172.21.15.5:0/3981470247 conn(0x564c7ed95be0 msgr2=0x564c7ed9af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.875+0000 7f3fb745db80 1 -- 172.21.15.5:0/3981470247 shutdown_connections 2023-07-21T19:22:52.872 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.875+0000 7f3fb745db80 1 -- 172.21.15.5:0/3981470247 wait complete. 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.875+0000 7f3fb745db80 1 -- start start 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 Processor -- start 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7edac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ec91a60 con 0x564c7eda0580 2023-07-21T19:22:52.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ec91760 con 0x564c7ebfc6b0 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7eb758c0 con 0x564c7eda62d0 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.876+0000 7f3fb745db80 10 client.0 _ll_register_callbacks cb 0x564c7ec8f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7edac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57016/0 (socket says 172.21.15.5:57016) 2023-07-21T19:22:52.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa119a700 1 -- 172.21.15.5:0/1657754819 learned_addr learned my addr 172.21.15.5:0/1657754819 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 -- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 msgr2=0x564c7edac590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7edac590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 -- 172.21.15.5:0/1657754819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 msgr2=0x564c7eda9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 -- 172.21.15.5:0/1657754819 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f8c009d50 con 0x564c7eda62d0 2023-07-21T19:22:52.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa119a700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3fa219c700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda9ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400ec40 tx=0x7f3f94004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a63511b1b367a97c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.877+0000 7f3f857fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9401f070 con 0x564c7eda62d0 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f9401a820 con 0x564c7eda62d0 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c7ec4b490 con 0x564c7eda62d0 2023-07-21T19:22:52.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f9401a9e0 con 0x564c7eda62d0 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3fb745db80 10 client.28150 fetch_fsmap learned FSMap version 49 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564c7eb08c00 con 0x564c7eda62d0 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f94030020 con 0x564c7eda62d0 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.878+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3f94023b30 con 0x564c7eda62d0 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 10 client.28150 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 10 client.28150 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 20 client.28150 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:52.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7edb9c10 0x564c7edbc020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 4 client.28150 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c7eceae50 con 0x564c7edb9c10 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fa199b700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7edb9c10 0x564c7edbc020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3fa199b700 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7edb9c10 0x564c7edbc020 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=514a373bf9b90553 server_cookie=273dc435f3430529 in_seq=0 out_seq=0 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.879+0000 7f3f857fa700 10 client.28150 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.880+0000 7f3f857fa700 1 -- 172.21.15.5:0/1657754819 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c7eceae50 con 0x564c7edb9c10 2023-07-21T19:22:52.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.880+0000 7f3f857fa700 10 client.28150 handle_command_reply: tid=0 2023-07-21T19:22:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.880+0000 7f3fb745db80 1 client.28150 shutdown 2023-07-21T19:22:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.880+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 msgr2=0x564c7eda9ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.880+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda9ee0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f3f9400ec40 tx=0x7f3f94004ad0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 shutdown_connections 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda62d0 0x564c7eda9ee0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7edac590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7edb9c10 0x564c7edbc020 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 --2- 172.21.15.5:0/1657754819 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda0580 0x564c7eda9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3f857fa700 0 client.28150 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:52.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 >> 172.21.15.5:0/1657754819 conn(0x564c7ed95be0 msgr2=0x564c7ed9a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 shutdown_connections 2023-07-21T19:22:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 1 -- 172.21.15.5:0/1657754819 wait complete. 2023-07-21T19:22:52.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.881+0000 7f3fb745db80 20 client.28150 trim_cache size 0 max 16384 2023-07-21T19:22:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.900+0000 7f3fb745db80 1 Processor -- start 2023-07-21T19:22:52.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.900+0000 7f3fb745db80 1 -- start start 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda7910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7edb0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 0x564c7eda5130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ed97d00 con 0x564c7ebfc6b0 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7edb5b30 con 0x564c7eda62d0 2023-07-21T19:22:52.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fb745db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ed9ba00 con 0x564c7eda0580 2023-07-21T19:22:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7edb0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fa219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 0x564c7eda5130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fa119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7edb0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48874/0 (socket says 172.21.15.5:48874) 2023-07-21T19:22:52.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.901+0000 7f3fa119a700 1 -- 172.21.15.5:0/3858564603 learned_addr learned my addr 172.21.15.5:0/3858564603 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:52.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 -- 172.21.15.5:0/3858564603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 msgr2=0x564c7edb0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7edb0c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 -- 172.21.15.5:0/3858564603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 msgr2=0x564c7eda7910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda7910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 -- 172.21.15.5:0/3858564603 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c7eb461f0 con 0x564c7eda62d0 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3fa219c700 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 0x564c7eda5130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009ec0 tx=0x7f3f9400ef50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47778364d0c9b874 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3f90bff700 1 -- 172.21.15.5:0/3858564603 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f9401f070 con 0x564c7eda62d0 2023-07-21T19:22:52.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3f90bff700 1 -- 172.21.15.5:0/3858564603 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f94030030 con 0x564c7eda62d0 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.902+0000 7f3f90bff700 1 -- 172.21.15.5:0/3858564603 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f940164f0 con 0x564c7eda62d0 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 -- 172.21.15.5:0/3858564603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 msgr2=0x564c7eda5130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 0x564c7eda5130 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009ec0 tx=0x7f3f9400ef50 comp rx=0 tx=0).stop 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 -- 172.21.15.5:0/3858564603 shutdown_connections 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7eda62d0 0x564c7eda5130 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7ebfc6b0 0x564c7eda7910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 --2- 172.21.15.5:0/3858564603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7edb0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 -- 172.21.15.5:0/3858564603 >> 172.21.15.5:0/3858564603 conn(0x564c7ed95be0 msgr2=0x564c7eda4a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 -- 172.21.15.5:0/3858564603 shutdown_connections 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.903+0000 7f3fb745db80 1 -- 172.21.15.5:0/3858564603 wait complete. 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 -- start start 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 Processor -- start 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7ee01fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda62d0 0x564c7ee0cae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ee028b0 con 0x564c7eda62d0 2023-07-21T19:22:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ee0d020 con 0x564c7ebfc6b0 2023-07-21T19:22:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c7ee0d160 con 0x564c7eda0580 2023-07-21T19:22:52.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.904+0000 7f3fb745db80 10 client.0 _ll_register_callbacks cb 0x7f3f9400e630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:22:52.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57792/0 (socket says 172.21.15.5:57792) 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 -- 172.21.15.5:0/32905681 learned_addr learned my addr 172.21.15.5:0/32905681 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa119a700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7ee01fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa219c700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda62d0 0x564c7ee0cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 -- 172.21.15.5:0/32905681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 msgr2=0x564c7ee01fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7ee01fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 -- 172.21.15.5:0/32905681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda62d0 msgr2=0x564c7ee0cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda62d0 0x564c7ee0cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 -- 172.21.15.5:0/32905681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c7eb08c00 con 0x564c7ebfc6b0 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3fa199b700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3f98097510 tx=0x7f3f9806d080 comp rx=0 tx=0).ready entity=mon.2 client_cookie=915e5f9b5036961b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.905+0000 7f3f857fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f980cba80 con 0x564c7ebfc6b0 2023-07-21T19:22:52.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f9809f040 con 0x564c7ebfc6b0 2023-07-21T19:22:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f8c004390 con 0x564c7ebfc6b0 2023-07-21T19:22:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f980a33d0 con 0x564c7ebfc6b0 2023-07-21T19:22:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 10 client.28232 fetch_fsmap learned FSMap version 49 2023-07-21T19:22:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x564c7eb1e250 con 0x564c7ebfc6b0 2023-07-21T19:22:52.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f8c004390 con 0x564c7ebfc6b0 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3f9809b390 con 0x564c7ebfc6b0 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 10 client.28232 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 10 client.28232 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 20 client.28232 populate_metadata read hostname 'smithi005' 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7ee0e550 0x564c7ee10930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 4 client.28232 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.906+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564c7ecacff0 con 0x564c7ee0e550 2023-07-21T19:22:52.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.907+0000 7f3fa119a700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7ee0e550 0x564c7ee10930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:22:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.907+0000 7f3fa119a700 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7ee0e550 0x564c7ee10930 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a4dbb5469548302c server_cookie=c427e90e1fcee1be in_seq=0 out_seq=0 2023-07-21T19:22:52.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.907+0000 7f3f857fa700 10 client.28232 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.915+0000 7f3f857fa700 1 -- 172.21.15.5:0/32905681 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564c7ecacff0 con 0x564c7ee0e550 2023-07-21T19:22:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.915+0000 7f3f857fa700 10 client.28232 handle_command_reply: tid=0 2023-07-21T19:22:52.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.915+0000 7f3fb745db80 1 client.28232 shutdown 2023-07-21T19:22:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.915+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 msgr2=0x564c7ee01a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.915+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f3f98097510 tx=0x7f3f9806d080 comp rx=0 tx=0).stop 2023-07-21T19:22:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 shutdown_connections 2023-07-21T19:22:52.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c7ee0e550 0x564c7ee10930 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c7eda62d0 0x564c7ee0cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c7ebfc6b0 0x564c7ee01a60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3f857fa700 0 client.28232 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:22:52.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 --2- 172.21.15.5:0/32905681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c7eda0580 0x564c7ee01fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 >> 172.21.15.5:0/32905681 conn(0x564c7ed95be0 msgr2=0x564c7eda52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 shutdown_connections 2023-07-21T19:22:52.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 1 -- 172.21.15.5:0/32905681 wait complete. 2023-07-21T19:22:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.916+0000 7f3fb745db80 20 client.28232 trim_cache size 0 max 16384 2023-07-21T19:22:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:22:52.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:22:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:22:52.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:22:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 msgr2=0x7f3f880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 0x7f3f880a3770 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f3f94009fd0 tx=0x7f3f9400b040 comp rx=0 tx=0).stop 2023-07-21T19:22:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 msgr2=0x7f3f9c197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:22:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f3f98005910 tx=0x7f3f980047b0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 shutdown_connections 2023-07-21T19:22:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f9c105b10 0x7f3f9c19c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f880a12c0 0x7f3f880a3770 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f9c0ffb00 0x7f3f9c197d80 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 --2- 172.21.15.5:0/3670891362 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f9c1004b0 0x7f3f9c1982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:22:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 >> 172.21.15.5:0/3670891362 conn(0x7f3f9c0fb600 msgr2=0x7f3f9c10ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:22:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 shutdown_connections 2023-07-21T19:22:52.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:22:52.917+0000 7f3fa3bff700 1 -- 172.21.15.5:0/3670891362 wait complete. 2023-07-21T19:22:53.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:53 smithi005 ceph-mon[126460]: pgmap v479: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:22:53.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:53 smithi156 ceph-mon[128316]: pgmap v479: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:22:54.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:53 smithi101 ceph-mon[128136]: pgmap v479: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:22:55.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:55 smithi005 ceph-mon[126460]: pgmap v480: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:22:55.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:55 smithi156 ceph-mon[128316]: pgmap v480: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:22:56.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:55 smithi101 ceph-mon[128136]: pgmap v480: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 19 op/s 2023-07-21T19:22:57.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:57 smithi005 ceph-mon[126460]: pgmap v481: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:22:57.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:57 smithi156 ceph-mon[128316]: pgmap v481: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:22:58.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:57 smithi101 ceph-mon[128136]: pgmap v481: 97 pgs: 97 active+clean; 12 GiB data, 5.3 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:22:59.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:22:59 smithi005 ceph-mon[126460]: pgmap v482: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:22:59.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:22:59 smithi156 ceph-mon[128316]: pgmap v482: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:23:00.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:22:59 smithi101 ceph-mon[128136]: pgmap v482: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:23:01.890 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:01 smithi005 ceph-mon[126460]: pgmap v483: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:01.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:01 smithi156 ceph-mon[128316]: pgmap v483: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:02.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:01 smithi101 ceph-mon[128136]: pgmap v483: 97 pgs: 97 active+clean; 12 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:03.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:03 smithi005 ceph-mon[126460]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:23:03.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:03 smithi156 ceph-mon[128316]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:23:04.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:03 smithi101 ceph-mon[128136]: pgmap v484: 97 pgs: 97 active+clean; 13 GiB data, 5.4 GiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:23:04.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:04 smithi156 ceph-mon[128316]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 34 op/s 2023-07-21T19:23:05.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:04 smithi101 ceph-mon[128136]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 34 op/s 2023-07-21T19:23:05.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:04 smithi005 ceph-mon[126460]: pgmap v485: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 25 MiB/s wr, 34 op/s 2023-07-21T19:23:06.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 -- 172.21.15.5:0/908453194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084067c10 msgr2=0x7f9084067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 --2- 172.21.15.5:0/908453194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084067c10 0x7f9084067fe0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f90800099c0 tx=0x7f9080009cd0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 -- 172.21.15.5:0/908453194 shutdown_connections 2023-07-21T19:23:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 --2- 172.21.15.5:0/908453194 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9084104e50 0x7f908410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 --2- 172.21.15.5:0/908453194 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084067c10 0x7f9084067fe0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 --2- 172.21.15.5:0/908453194 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f90840685b0 0x7f9084104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 -- 172.21.15.5:0/908453194 >> 172.21.15.5:0/908453194 conn(0x7f9084074760 msgr2=0x7f9084074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.096+0000 7f908b3ff700 1 -- 172.21.15.5:0/908453194 shutdown_connections 2023-07-21T19:23:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.097+0000 7f908b3ff700 1 -- 172.21.15.5:0/908453194 wait complete. 2023-07-21T19:23:06.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.098+0000 7f908b3ff700 1 Processor -- start 2023-07-21T19:23:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.098+0000 7f908b3ff700 1 -- start start 2023-07-21T19:23:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90840685b0 0x7f90840ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084104e50 0x7f90841a3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841a42d0 con 0x7f9084104e50 2023-07-21T19:23:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841a4410 con 0x7f90840685b0 2023-07-21T19:23:06.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90841a4580 con 0x7f9084067c10 2023-07-21T19:23:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908919b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47420/0 (socket says 172.21.15.5:47420) 2023-07-21T19:23:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908919b700 1 -- 172.21.15.5:0/3807493910 learned_addr learned my addr 172.21.15.5:0/3807493910 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:06.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908899a700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90840685b0 0x7f90840ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.099+0000 7f908999c700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084104e50 0x7f90841a3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 -- 172.21.15.5:0/3807493910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90840685b0 msgr2=0x7f90840ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90840685b0 0x7f90840ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 -- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084104e50 msgr2=0x7f90841a3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084104e50 0x7f90841a3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 -- 172.21.15.5:0/3807493910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f907c009c40 con 0x7f9084067c10 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f908919b700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9080005910 tx=0x7f908000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f907a7fc700 1 -- 172.21.15.5:0/3807493910 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f908001d070 con 0x7f9084067c10 2023-07-21T19:23:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f907a7fc700 1 -- 172.21.15.5:0/3807493910 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f90800054e0 con 0x7f9084067c10 2023-07-21T19:23:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.100+0000 7f907a7fc700 1 -- 172.21.15.5:0/3807493910 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9080004a90 con 0x7f9084067c10 2023-07-21T19:23:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.101+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90800095f0 con 0x7f9084067c10 2023-07-21T19:23:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.101+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90841a4a40 con 0x7f9084067c10 2023-07-21T19:23:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.106+0000 7f907a7fc700 1 -- 172.21.15.5:0/3807493910 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f908000fb80 con 0x7f9084067c10 2023-07-21T19:23:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.106+0000 7f909ed51b80 1 Processor -- start 2023-07-21T19:23:06.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 -- start start 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694d3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694d39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b693d8c50 con 0x560b694d92d0 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b6942b9e0 con 0x560b6932f6b0 2023-07-21T19:23:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.107+0000 7f909ed51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b69424e70 con 0x560b694d3580 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44148/0 (socket says 172.21.15.5:44148) 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 -- 172.21.15.5:0/2065412650 learned_addr learned my addr 172.21.15.5:0/2065412650 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908899a700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694d39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 -- 172.21.15.5:0/2065412650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 msgr2=0x560b694d39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694d39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 -- 172.21.15.5:0/2065412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 msgr2=0x560b694d3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908919b700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694d3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694d3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908999c700 1 -- 172.21.15.5:0/2065412650 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b69252d80 con 0x560b6932f6b0 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f907a7fc700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 0x7f90700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f908919b700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.108+0000 7f907a7fc700 1 -- 172.21.15.5:0/3807493910 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f90800cdb30 con 0x7f9084067c10 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.109+0000 7f908999c700 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f907c007600 tx=0x7f907c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf4b5bba5bbbc4d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.109+0000 7f908919b700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 0x7f90700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.109+0000 7f906f7fe700 1 -- 172.21.15.5:0/2065412650 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c00e070 con 0x560b6932f6b0 2023-07-21T19:23:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.109+0000 7f906f7fe700 1 -- 172.21.15.5:0/2065412650 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f907c00f040 con 0x560b6932f6b0 2023-07-21T19:23:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.109+0000 7f906f7fe700 1 -- 172.21.15.5:0/2065412650 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c015ac0 con 0x560b6932f6b0 2023-07-21T19:23:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 -- 172.21.15.5:0/2065412650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 msgr2=0x560b694d8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f907c007600 tx=0x7f907c009920 comp rx=0 tx=0).stop 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f908919b700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 0x7f90700a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x560b694cfb40 tx=0x7f9080009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 -- 172.21.15.5:0/2065412650 shutdown_connections 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694d39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8d90 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2065412650 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694d3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 -- 172.21.15.5:0/2065412650 >> 172.21.15.5:0/2065412650 conn(0x560b694c8be0 msgr2=0x560b694cdf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 -- 172.21.15.5:0/2065412650 shutdown_connections 2023-07-21T19:23:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.110+0000 7f909ed51b80 1 -- 172.21.15.5:0/2065412650 wait complete. 2023-07-21T19:23:06.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.111+0000 7f909ed51b80 1 -- start start 2023-07-21T19:23:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.111+0000 7f909ed51b80 1 Processor -- start 2023-07-21T19:23:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694df660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694dc870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694dcfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b693c4a60 con 0x560b694d92d0 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b693c4760 con 0x560b6932f6b0 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b692a88c0 con 0x560b694d3580 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f909ed51b80 10 client.0 _ll_register_callbacks cb 0x560b693c2700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694df660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694df660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44156/0 (socket says 172.21.15.5:44156) 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f908899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694dc870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f908999c700 1 -- 172.21.15.5:0/2915717478 learned_addr learned my addr 172.21.15.5:0/2915717478 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.112+0000 7f908999c700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694dcfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 -- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 msgr2=0x560b694df660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694df660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 -- 172.21.15.5:0/2915717478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 msgr2=0x560b694dcfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694dcfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 -- 172.21.15.5:0/2915717478 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f907c0092e0 con 0x560b694d3580 2023-07-21T19:23:06.111 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.113+0000 7f908899a700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694dc870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90840733d0 tx=0x7f9074011a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4dd116a47d2fcf0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f906cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074016070 con 0x560b694d3580 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9074004830 con 0x560b694d3580 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560b6937e490 con 0x560b694d3580 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f90740049f0 con 0x560b694d3580 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f909ed51b80 10 client.28168 fetch_fsmap learned FSMap version 49 2023-07-21T19:23:06.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.114+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560b6923bc00 con 0x560b694d3580 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560b6937e490 con 0x560b694d3580 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f90740227b0 con 0x560b694d3580 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 10 client.28168 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 10 client.28168 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 20 client.28168 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b694f5480 0x560b694f7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 4 client.28168 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.115+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560b6941de50 con 0x560b694f5480 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.116+0000 7f908999c700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b694f5480 0x560b694f7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.116+0000 7f906cff9700 10 client.28168 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.116+0000 7f908999c700 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b694f5480 0x560b694f7890 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d568db105874a61 server_cookie=6739bdd8e2d85c35 in_seq=0 out_seq=0 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.117+0000 7f906cff9700 1 -- 172.21.15.5:0/2915717478 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7f907c02d780 con 0x560b694f5480 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.117+0000 7f906cff9700 10 client.28168 handle_command_reply: tid=0 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.117+0000 7f909ed51b80 1 client.28168 shutdown 2023-07-21T19:23:06.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 msgr2=0x560b694dc870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694dc870 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f90840733d0 tx=0x7f9074011a10 comp rx=0 tx=0).stop 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 shutdown_connections 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b694f5480 0x560b694f7890 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d92d0 0x560b694dcfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694df660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f906cff9700 0 client.28168 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2915717478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d3580 0x560b694dc870 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 >> 172.21.15.5:0/2915717478 conn(0x560b694c8be0 msgr2=0x560b694cd4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 shutdown_connections 2023-07-21T19:23:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 1 -- 172.21.15.5:0/2915717478 wait complete. 2023-07-21T19:23:06.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.118+0000 7f909ed51b80 20 client.28168 trim_cache size 0 max 16384 2023-07-21T19:23:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 Processor -- start 2023-07-21T19:23:06.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 -- start start 2023-07-21T19:23:06.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694cdc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694c5c00 con 0x560b694d3580 2023-07-21T19:23:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694f29d0 con 0x560b6932f6b0 2023-07-21T19:23:06.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.139+0000 7f909ed51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694cad30 con 0x560b694d92d0 2023-07-21T19:23:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38854/0 (socket says 172.21.15.5:38854) 2023-07-21T19:23:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 -- 172.21.15.5:0/3946702323 learned_addr learned my addr 172.21.15.5:0/3946702323 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908999c700 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694cdc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 -- 172.21.15.5:0/3946702323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 msgr2=0x560b694cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694cdc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 -- 172.21.15.5:0/3946702323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 msgr2=0x560b694d8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.140+0000 7f908899a700 1 -- 172.21.15.5:0/3946702323 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560b692791f0 con 0x560b694d3580 2023-07-21T19:23:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.141+0000 7f908899a700 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9074001eb0 tx=0x7f9074011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23a79e2514894ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.141+0000 7f906ffff700 1 -- 172.21.15.5:0/3946702323 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9074016070 con 0x560b694d3580 2023-07-21T19:23:06.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.141+0000 7f906ffff700 1 -- 172.21.15.5:0/3946702323 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9074022890 con 0x560b694d3580 2023-07-21T19:23:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.141+0000 7f906ffff700 1 -- 172.21.15.5:0/3946702323 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90740038e0 con 0x560b694d3580 2023-07-21T19:23:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 -- 172.21.15.5:0/3946702323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 msgr2=0x560b694dc030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f9074001eb0 tx=0x7f9074011d60 comp rx=0 tx=0).stop 2023-07-21T19:23:06.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 -- 172.21.15.5:0/3946702323 shutdown_connections 2023-07-21T19:23:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694cdc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694d8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 --2- 172.21.15.5:0/3946702323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694dc030 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 -- 172.21.15.5:0/3946702323 >> 172.21.15.5:0/3946702323 conn(0x560b694c8be0 msgr2=0x560b694d7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 -- 172.21.15.5:0/3946702323 shutdown_connections 2023-07-21T19:23:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.142+0000 7f909ed51b80 1 -- 172.21.15.5:0/3946702323 wait complete. 2023-07-21T19:23:06.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 -- start start 2023-07-21T19:23:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 Processor -- start 2023-07-21T19:23:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694ec630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694ea590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694eacd0 con 0x560b694d3580 2023-07-21T19:23:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694eae40 con 0x560b6932f6b0 2023-07-21T19:23:06.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560b694eb000 con 0x560b694d92d0 2023-07-21T19:23:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.143+0000 7f909ed51b80 10 client.0 _ll_register_callbacks cb 0x7f9074009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44178/0 (socket says 172.21.15.5:44178) 2023-07-21T19:23:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908899a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694ec630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 -- 172.21.15.5:0/2967461243 learned_addr learned my addr 172.21.15.5:0/2967461243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694ea590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 -- 172.21.15.5:0/2967461243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 msgr2=0x560b694ea590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694ea590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 -- 172.21.15.5:0/2967461243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 msgr2=0x560b694ec630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:06.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694ec630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908999c700 1 -- 172.21.15.5:0/2967461243 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560b6923bc00 con 0x560b6932f6b0 2023-07-21T19:23:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.144+0000 7f908899a700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:06.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f908999c700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x560b694f8100 tx=0x7f907c023040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d533a4328df6b947 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f906cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f907c026040 con 0x560b6932f6b0 2023-07-21T19:23:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f907c00f040 con 0x560b6932f6b0 2023-07-21T19:23:06.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(106..106 src has 1..106) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f907c003a00 con 0x560b6932f6b0 2023-07-21T19:23:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=49) v2 ==== 24+0+0 (secure 0 0 0) 0x7f907c00e040 con 0x560b6932f6b0 2023-07-21T19:23:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f909ed51b80 10 client.28250 fetch_fsmap learned FSMap version 49 2023-07-21T19:23:06.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=49,monmap=4+}) v3 -- 0x560b69251250 con 0x560b6932f6b0 2023-07-21T19:23:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.145+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f907c040080 con 0x560b6932f6b0 2023-07-21T19:23:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 49) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f907c007630 con 0x560b6932f6b0 2023-07-21T19:23:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 10 client.28250 fetch_fsmap finished waiting for FSMap version 49 2023-07-21T19:23:06.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 10 client.28250 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 20 client.28250 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b6954a560 0x560b6954c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:06.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 4 client.28250 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:23:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560b693dfff0 con 0x560b6954a560 2023-07-21T19:23:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.146+0000 7f908899a700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b6954a560 0x560b6954c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.147+0000 7f906cff9700 10 client.28250 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:06.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.147+0000 7f908899a700 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b6954a560 0x560b6954c970 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0669259fa8e24be server_cookie=ebf3bb236d3b06ed in_seq=0 out_seq=0 2023-07-21T19:23:06.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.155+0000 7f906cff9700 1 -- 172.21.15.5:0/2967461243 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560b693dfff0 con 0x560b6954a560 2023-07-21T19:23:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.155+0000 7f906cff9700 10 client.28250 handle_command_reply: tid=0 2023-07-21T19:23:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.155+0000 7f909ed51b80 1 client.28250 shutdown 2023-07-21T19:23:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 msgr2=0x560b694ec0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x560b694f8100 tx=0x7f907c023040 comp rx=0 tx=0).stop 2023-07-21T19:23:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 shutdown_connections 2023-07-21T19:23:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560b694d92d0 0x560b694ea590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560b6932f6b0 0x560b694ec0f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560b6954a560 0x560b6954c970 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 --2- 172.21.15.5:0/2967461243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560b694d3580 0x560b694ec630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f906cff9700 0 client.28250 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:06.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 >> 172.21.15.5:0/2967461243 conn(0x560b694c8be0 msgr2=0x560b694d82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 shutdown_connections 2023-07-21T19:23:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 1 -- 172.21.15.5:0/2967461243 wait complete. 2023-07-21T19:23:06.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.156+0000 7f909ed51b80 20 client.28250 trim_cache size 0 max 16384 2023-07-21T19:23:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:23:06.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:23:06.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:23:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 msgr2=0x7f90700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 0x7f90700a3770 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x560b694cfb40 tx=0x7f9080009310 comp rx=0 tx=0).stop 2023-07-21T19:23:06.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 msgr2=0x7f90840fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f9080005910 tx=0x7f908000f690 comp rx=0 tx=0).stop 2023-07-21T19:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 shutdown_connections 2023-07-21T19:23:06.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f90700a12c0 0x7f90700a3770 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9084104e50 0x7f90841a3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9084067c10 0x7f90840fec40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 --2- 172.21.15.5:0/3807493910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f90840685b0 0x7f90840ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:06.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 >> 172.21.15.5:0/3807493910 conn(0x7f9084074760 msgr2=0x7f90841036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 shutdown_connections 2023-07-21T19:23:06.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:06.157+0000 7f908b3ff700 1 -- 172.21.15.5:0/3807493910 wait complete. 2023-07-21T19:23:07.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:07 smithi005 ceph-mon[126460]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 32 op/s 2023-07-21T19:23:07.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:07 smithi156 ceph-mon[128316]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 32 op/s 2023-07-21T19:23:07.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:07 smithi101 ceph-mon[128136]: pgmap v486: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 24 MiB/s wr, 32 op/s 2023-07-21T19:23:09.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:09 smithi005 ceph-mon[126460]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 36 MiB/s wr, 48 op/s 2023-07-21T19:23:09.650 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:09 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:23:09.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:09 smithi101 ceph-mon[128136]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 36 MiB/s wr, 48 op/s 2023-07-21T19:23:09.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:09 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:23:09.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:09 smithi156 ceph-mon[128316]: pgmap v487: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 36 MiB/s wr, 48 op/s 2023-07-21T19:23:09.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:09 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:23:10.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:10 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:23:10.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:10 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:23:10.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:10 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:23:10.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:10 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:23:10.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:10 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:23:10.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:10 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:23:10.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:10 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:23:10.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:10 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:23:10.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:10 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:23:11.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:11 smithi101 ceph-mon[128136]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 38 op/s 2023-07-21T19:23:11.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:11 smithi005 ceph-mon[126460]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 38 op/s 2023-07-21T19:23:11.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:11 smithi156 ceph-mon[128316]: pgmap v488: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 24 MiB/s wr, 38 op/s 2023-07-21T19:23:12.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:12 smithi101 ceph-mon[128136]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T19:23:12.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:12.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:12 smithi005 ceph-mon[126460]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T19:23:12.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:12.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:12 smithi156 ceph-mon[128316]: osdmap e107: 12 total, 12 up, 12 in 2023-07-21T19:23:12.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:13.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:13 smithi101 ceph-mon[128136]: pgmap v490: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 31 op/s 2023-07-21T19:23:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:13 smithi005 ceph-mon[126460]: pgmap v490: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 31 op/s 2023-07-21T19:23:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:13 smithi156 ceph-mon[128316]: pgmap v490: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 29 MiB/s wr, 31 op/s 2023-07-21T19:23:15.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:15 smithi101 ceph-mon[128136]: pgmap v491: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:15.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:15 smithi005 ceph-mon[126460]: pgmap v491: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:15.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:15 smithi156 ceph-mon[128316]: pgmap v491: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:17.413 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:23:17.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 -- 172.21.15.5:0/3901757125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0100b80 msgr2=0x7fb0b0100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/3901757125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0100b80 0x7fb0b0100f50 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4009990 tx=0x7fb0a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 -- 172.21.15.5:0/3901757125 shutdown_connections 2023-07-21T19:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/3901757125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b010a0a0 0x7fb0b010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/3901757125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0100b80 0x7fb0b0100f50 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/3901757125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b0101490 0x7fb0b0109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 -- 172.21.15.5:0/3901757125 >> 172.21.15.5:0/3901757125 conn(0x7fb0b00fa4b0 msgr2=0x7fb0b00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.660+0000 7fb0b6fff700 1 -- 172.21.15.5:0/3901757125 shutdown_connections 2023-07-21T19:23:17.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.661+0000 7fb0b6fff700 1 -- 172.21.15.5:0/3901757125 wait complete. 2023-07-21T19:23:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.662+0000 7fb0b6fff700 1 Processor -- start 2023-07-21T19:23:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.662+0000 7fb0b6fff700 1 -- start start 2023-07-21T19:23:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0101490 0x7fb0b019bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b010a0a0 0x7fb0b0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b01960d0 con 0x7fb0b0101490 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0196240 con 0x7fb0b0100b80 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b0196400 con 0x7fb0b010a0a0 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48280/0 (socket says 172.21.15.5:48280) 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/634743703 learned_addr learned my addr 172.21.15.5:0/634743703 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0affff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0101490 0x7fb0b019bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b559c700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b010a0a0 0x7fb0b0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/634743703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b010a0a0 msgr2=0x7fb0b0195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b010a0a0 0x7fb0b0195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0101490 msgr2=0x7fb0b019bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.663+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0101490 0x7fb0b019bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/634743703 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb09c009520 con 0x7fb0b0100b80 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4005740 tx=0x7fb0a4005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0adffb700 1 -- 172.21.15.5:0/634743703 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a401d070 con 0x7fb0b0100b80 2023-07-21T19:23:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0adffb700 1 -- 172.21.15.5:0/634743703 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0a4022470 con 0x7fb0b0100b80 2023-07-21T19:23:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0adffb700 1 -- 172.21.15.5:0/634743703 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a4017650 con 0x7fb0b0100b80 2023-07-21T19:23:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0a40095f0 con 0x7fb0b0100b80 2023-07-21T19:23:17.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.664+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b00fe8e0 con 0x7fb0b0100b80 2023-07-21T19:23:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.668+0000 7fb0adffb700 1 -- 172.21.15.5:0/634743703 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb0a40225e0 con 0x7fb0b0100b80 2023-07-21T19:23:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 Processor -- start 2023-07-21T19:23:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0adffb700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 0x7fb0a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 -- start start 2023-07-21T19:23:17.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0adffb700 1 -- 172.21.15.5:0/634743703 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb0a40cdfb0 con 0x7fb0b0100b80 2023-07-21T19:23:17.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba22578590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba22473cb0 con 0x55ba2256dbe0 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba224c6a40 con 0x55ba22578ad0 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba224bfed0 con 0x55ba2256d2d0 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.670+0000 7fb0b559c700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 0x7fb0a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37952/0 (socket says 172.21.15.5:37952) 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/949524384 learned_addr learned my addr 172.21.15.5:0/949524384 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b559c700 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/949524384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 msgr2=0x55ba2257aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/949524384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 msgr2=0x55ba22578590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba22578590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/949524384 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba222edde0 con 0x55ba2256d2d0 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097c30 tx=0x7fb0a4005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3ba632bb4a2a235 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb0b559c700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 0x7fb0a00a37f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0b019c6f0 tx=0x7fb0a8009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.671+0000 7fb09affd700 1 -- 172.21.15.5:0/949524384 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40a10d0 con 0x55ba2256d2d0 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb09affd700 1 -- 172.21.15.5:0/949524384 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0a409f1f0 con 0x55ba2256d2d0 2023-07-21T19:23:17.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb09affd700 1 -- 172.21.15.5:0/949524384 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a409b420 con 0x55ba2256d2d0 2023-07-21T19:23:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/949524384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 msgr2=0x55ba2256d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4097c30 tx=0x7fb0a4005910 comp rx=0 tx=0).stop 2023-07-21T19:23:17.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/949524384 shutdown_connections 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2256d6a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/949524384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba22578590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/949524384 >> 172.21.15.5:0/949524384 conn(0x55ba22563da0 msgr2=0x55ba225696a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.672+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/949524384 shutdown_connections 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.673+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/949524384 wait complete. 2023-07-21T19:23:17.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.673+0000 7fb0ca96fb80 1 -- start start 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 Processor -- start 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22578ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba22579940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2245fac0 con 0x55ba2256d2d0 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2245f7c0 con 0x55ba22578ad0 2023-07-21T19:23:17.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.674+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba22343920 con 0x55ba2256dbe0 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0ca96fb80 10 client.0 _ll_register_callbacks cb 0x55ba2245d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22578ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37968/0 (socket says 172.21.15.5:37968) 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 -- 172.21.15.5:0/1273779521 learned_addr learned my addr 172.21.15.5:0/1273779521 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:17.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0b559c700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba22579940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 -- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 msgr2=0x55ba22579940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba22579940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 -- 172.21.15.5:0/1273779521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 msgr2=0x55ba22578ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22578ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 -- 172.21.15.5:0/1273779521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b00fe8e0 con 0x55ba2256dbe0 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb0affff700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb09c004790 tx=0x7fb09c0119f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d95c91f13374f8c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.675+0000 7fb08ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c003680 con 0x55ba2256dbe0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ba224194f0 con 0x55ba2256dbe0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb09c0037e0 con 0x55ba2256dbe0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb09c0039a0 con 0x55ba2256dbe0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb0ca96fb80 10 client.28186 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ba222d6c60 con 0x55ba2256dbe0 2023-07-21T19:23:17.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb09c026020 con 0x55ba2256dbe0 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.676+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb09c021ac0 con 0x55ba2256dbe0 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 10 client.28186 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 10 client.28186 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 20 client.28186 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225881c0 0x55ba2258a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 4 client.28186 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ba224b8eb0 con 0x55ba225881c0 2023-07-21T19:23:17.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.677+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225881c0 0x55ba2258a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.678+0000 7fb08ffff700 10 client.28186 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.678+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225881c0 0x55ba2258a5d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0a7a620da8d9e6 server_cookie=c11d7ace6860401f in_seq=0 out_seq=0 2023-07-21T19:23:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.678+0000 7fb08ffff700 1 -- 172.21.15.5:0/1273779521 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ba224b8eb0 con 0x55ba225881c0 2023-07-21T19:23:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.678+0000 7fb08ffff700 10 client.28186 handle_command_reply: tid=0 2023-07-21T19:23:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.678+0000 7fb0ca96fb80 1 client.28186 shutdown 2023-07-21T19:23:17.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 msgr2=0x55ba225793e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb09c004790 tx=0x7fb09c0119f0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 shutdown_connections 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba22579940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225881c0 0x55ba2258a5d0 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22578ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/1273779521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256dbe0 0x55ba225793e0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb08ffff700 0 client.28186 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:17.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 >> 172.21.15.5:0/1273779521 conn(0x55ba22563da0 msgr2=0x55ba22569230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 shutdown_connections 2023-07-21T19:23:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/1273779521 wait complete. 2023-07-21T19:23:17.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.679+0000 7fb0ca96fb80 20 client.28186 trim_cache size 0 max 16384 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.701+0000 7fb0ca96fb80 1 Processor -- start 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.701+0000 7fb0ca96fb80 1 -- start start 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22572400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba22578ad0 0x55ba22572e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2256ee70 con 0x55ba2256d2d0 2023-07-21T19:23:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2256a270 con 0x55ba2256dbe0 2023-07-21T19:23:17.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2256e1c0 con 0x55ba22578ad0 2023-07-21T19:23:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22572400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0b559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba22578ad0 0x55ba22572e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48312/0 (socket says 172.21.15.5:48312) 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 -- 172.21.15.5:0/174811268 learned_addr learned my addr 172.21.15.5:0/174811268 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 -- 172.21.15.5:0/174811268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba22578ad0 msgr2=0x55ba22572e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba22578ad0 0x55ba22572e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 -- 172.21.15.5:0/174811268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 msgr2=0x55ba22572400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22572400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.702+0000 7fb0affff700 1 -- 172.21.15.5:0/174811268 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ba22314250 con 0x55ba2256dbe0 2023-07-21T19:23:17.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.703+0000 7fb0affff700 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb09c021f20 tx=0x7fb09c001e50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=960071d41662c8b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.703+0000 7fb09b7fe700 1 -- 172.21.15.5:0/174811268 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c00f560 con 0x55ba2256dbe0 2023-07-21T19:23:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.703+0000 7fb09b7fe700 1 -- 172.21.15.5:0/174811268 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb09c026030 con 0x55ba2256dbe0 2023-07-21T19:23:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.703+0000 7fb09b7fe700 1 -- 172.21.15.5:0/174811268 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb09c004ab0 con 0x55ba2256dbe0 2023-07-21T19:23:17.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/174811268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 msgr2=0x55ba22583690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fb09c021f20 tx=0x7fb09c001e50 comp rx=0 tx=0).stop 2023-07-21T19:23:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/174811268 shutdown_connections 2023-07-21T19:23:17.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba22578ad0 0x55ba22572e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256d2d0 0x55ba22572400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/174811268 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba2256dbe0 0x55ba22583690 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/174811268 >> 172.21.15.5:0/174811268 conn(0x55ba22563da0 msgr2=0x55ba22578ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/174811268 shutdown_connections 2023-07-21T19:23:17.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/174811268 wait complete. 2023-07-21T19:23:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.704+0000 7fb0ca96fb80 1 -- start start 2023-07-21T19:23:17.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 Processor -- start 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba2257c390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba2257d190 con 0x55ba2256dbe0 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba225dc090 con 0x55ba22578ad0 2023-07-21T19:23:17.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ba225dc1d0 con 0x55ba2256d2d0 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.705+0000 7fb0ca96fb80 10 client.0 _ll_register_callbacks cb 0x7fb09c009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37974/0 (socket says 172.21.15.5:37974) 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/2033302744 learned_addr learned my addr 172.21.15.5:0/2033302744 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b559c700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0affff700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba2257c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 msgr2=0x55ba2257ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/2033302744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 msgr2=0x55ba2257c390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba2257c390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b4d9b700 1 -- 172.21.15.5:0/2033302744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ba222d6c60 con 0x55ba2256d2d0 2023-07-21T19:23:17.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb0b559c700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:23:17.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.706+0000 7fb08ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:23:17.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb0b4d9b700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40976c0 tx=0x7fb0a409f120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c434d787c1b69c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:17.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0a40a3600 con 0x55ba2256d2d0 2023-07-21T19:23:17.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb0a409f1f0 con 0x55ba2256d2d0 2023-07-21T19:23:17.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb0a40228c0 con 0x55ba2256d2d0 2023-07-21T19:23:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0a409a6b0 con 0x55ba2256d2d0 2023-07-21T19:23:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb0ca96fb80 10 client.28192 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.707+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55ba222ec2b0 con 0x55ba2256d2d0 2023-07-21T19:23:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0a409f1f0 con 0x55ba2256d2d0 2023-07-21T19:23:17.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb0a40a30f0 con 0x55ba2256d2d0 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 10 client.28192 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 10 client.28192 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 20 client.28192 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225dd550 0x55ba225df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 4 client.28192 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:23:17.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.708+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ba2247b050 con 0x55ba225dd550 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.709+0000 7fb0affff700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225dd550 0x55ba225df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.709+0000 7fb0affff700 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225dd550 0x55ba225df960 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15415e3f181fd2aa server_cookie=ff4319df1fa6b114 in_seq=0 out_seq=0 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.709+0000 7fb08ffff700 10 client.28192 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb08ffff700 1 -- 172.21.15.5:0/2033302744 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ba2247b050 con 0x55ba225dd550 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb08ffff700 10 client.28192 handle_command_reply: tid=0 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 client.28192 shutdown 2023-07-21T19:23:17.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 msgr2=0x55ba2257e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb0a40976c0 tx=0x7fb0a409f120 comp rx=0 tx=0).stop 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 shutdown_connections 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ba22578ad0 0x55ba2257ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ba2256d2d0 0x55ba2257e0d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ba225dd550 0x55ba225df960 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 --2- 172.21.15.5:0/2033302744 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ba2256dbe0 0x55ba2257c390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb08ffff700 0 client.28192 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 >> 172.21.15.5:0/2033302744 conn(0x55ba22563da0 msgr2=0x55ba22572e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 shutdown_connections 2023-07-21T19:23:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.710+0000 7fb0ca96fb80 1 -- 172.21.15.5:0/2033302744 wait complete. 2023-07-21T19:23:17.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0ca96fb80 20 client.28192 trim_cache size 0 max 16384 2023-07-21T19:23:17.714 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:23:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:23:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:23:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:23:17.714 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:23:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:23:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:23:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:23:17.715 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:23:17.715 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:23:17.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 msgr2=0x7fb0a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 0x7fb0a00a37f0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb0b019c6f0 tx=0x7fb0a8009920 comp rx=0 tx=0).stop 2023-07-21T19:23:17.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 msgr2=0x7fb0b019b9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb0a4005740 tx=0x7fb0a4005770 comp rx=0 tx=0).stop 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 shutdown_connections 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0b010a0a0 0x7fb0b0195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0b0100b80 0x7fb0b019b9f0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0a00a1340 0x7fb0a00a37f0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 --2- 172.21.15.5:0/634743703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0b0101490 0x7fb0b019bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:17.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 >> 172.21.15.5:0/634743703 conn(0x7fb0b00fa4b0 msgr2=0x7fb0b010c280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:17.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.711+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 shutdown_connections 2023-07-21T19:23:17.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:17.712+0000 7fb0b6fff700 1 -- 172.21.15.5:0/634743703 wait complete. 2023-07-21T19:23:17.726 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:23:17.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:17 smithi101 ceph-mon[128136]: pgmap v492: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:17.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:17 smithi101 ceph-mon[128136]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T19:23:17.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:17 smithi005 ceph-mon[126460]: pgmap v492: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:17.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:17 smithi005 ceph-mon[126460]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T19:23:17.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:17 smithi156 ceph-mon[128316]: pgmap v492: 97 pgs: 97 active+clean; 12 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 28 MiB/s wr, 28 op/s 2023-07-21T19:23:17.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:17 smithi156 ceph-mon[128316]: osdmap e108: 12 total, 12 up, 12 in 2023-07-21T19:23:19.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 -- 172.21.15.5:0/1596630318 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560106df0 msgr2=0x7fb56010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 --2- 172.21.15.5:0/1596630318 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560106df0 0x7fb56010e9c0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fb55800b0a0 tx=0x7fb55800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 -- 172.21.15.5:0/1596630318 shutdown_connections 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 --2- 172.21.15.5:0/1596630318 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560106df0 0x7fb56010e9c0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 --2- 172.21.15.5:0/1596630318 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560101db0 0x7fb560102180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 --2- 172.21.15.5:0/1596630318 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb560106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 -- 172.21.15.5:0/1596630318 >> 172.21.15.5:0/1596630318 conn(0x7fb5600fd660 msgr2=0x7fb5600ffa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 -- 172.21.15.5:0/1596630318 shutdown_connections 2023-07-21T19:23:19.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.336+0000 7fb56599d700 1 -- 172.21.15.5:0/1596630318 wait complete. 2023-07-21T19:23:19.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.337+0000 7fb56599d700 1 Processor -- start 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 -- start start 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb56019df80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560106df0 0x7fb560197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb560198170 con 0x7fb560102750 2023-07-21T19:23:19.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601982e0 con 0x7fb560101db0 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56599d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5601984a0 con 0x7fb560106df0 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb56019df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48328/0 (socket says 172.21.15.5:48328) 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56499b700 1 -- 172.21.15.5:0/4209073488 learned_addr learned my addr 172.21.15.5:0/4209073488 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.338+0000 7fb56519c700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560106df0 0x7fb560197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 -- 172.21.15.5:0/4209073488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560106df0 msgr2=0x7fb560197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560106df0 0x7fb560197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 -- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 msgr2=0x7fb56019df80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb56019df80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 -- 172.21.15.5:0/4209073488 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb558009d00 con 0x7fb560101db0 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb55ffff700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb56019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56499b700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d950 tx=0x7fb55400dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb55dffb700 1 -- 172.21.15.5:0/4209073488 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400f890 con 0x7fb560101db0 2023-07-21T19:23:19.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb560198780 con 0x7fb560101db0 2023-07-21T19:23:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb55dffb700 1 -- 172.21.15.5:0/4209073488 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb55400fe80 con 0x7fb560101db0 2023-07-21T19:23:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.339+0000 7fb55dffb700 1 -- 172.21.15.5:0/4209073488 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55400e3b0 con 0x7fb560101db0 2023-07-21T19:23:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.340+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a4a30 con 0x7fb560101db0 2023-07-21T19:23:19.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.343+0000 7fb55dffb700 1 -- 172.21.15.5:0/4209073488 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb554010460 con 0x7fb560101db0 2023-07-21T19:23:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb55dffb700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 0x7fb5500a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 Processor -- start 2023-07-21T19:23:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb55dffb700 1 -- 172.21.15.5:0/4209073488 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5540ccb20 con 0x7fb560101db0 2023-07-21T19:23:19.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 -- start start 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb55ffff700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 0x7fb5500a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f05700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78e0ac50 con 0x563b78f0b470 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78e5d9e0 con 0x563b78d616b0 2023-07-21T19:23:19.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.345+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78e56e70 con 0x563b78f0b8c0 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48344/0 (socket says 172.21.15.5:48344) 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 -- 172.21.15.5:0/3917670735 learned_addr learned my addr 172.21.15.5:0/3917670735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56519c700 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f05700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb55ffff700 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb55ffff700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 0x7fb5500a3770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5600fdac0 tx=0x7fb54c011820 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 -- 172.21.15.5:0/3917670735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 msgr2=0x563b78f05700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f05700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 -- 172.21.15.5:0/3917670735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 msgr2=0x563b78f051a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f051a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 -- 172.21.15.5:0/3917670735 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b78c84d80 con 0x563b78d616b0 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.346+0000 7fb56499b700 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb554096600 tx=0x7fb554010760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b502eead52fe8d84 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.347+0000 7fb54affd700 1 -- 172.21.15.5:0/3917670735 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb55409dbc0 con 0x563b78d616b0 2023-07-21T19:23:19.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.347+0000 7fb54affd700 1 -- 172.21.15.5:0/3917670735 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5540a1fe0 con 0x563b78d616b0 2023-07-21T19:23:19.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.347+0000 7fb54affd700 1 -- 172.21.15.5:0/3917670735 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540a1b50 con 0x563b78d616b0 2023-07-21T19:23:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3917670735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 msgr2=0x563b78f0af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fb554096600 tx=0x7fb554010760 comp rx=0 tx=0).stop 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3917670735 shutdown_connections 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f05700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f0af30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3917670735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f051a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3917670735 >> 172.21.15.5:0/3917670735 conn(0x563b78efad40 msgr2=0x563b78f00340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3917670735 shutdown_connections 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3917670735 wait complete. 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.348+0000 7fb57a5aeb80 1 -- start start 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 Processor -- start 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 0x563b78f148d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f0eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78df6a60 con 0x563b78d616b0 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78df6760 con 0x563b78f0b470 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78cda8c0 con 0x563b78f0b8c0 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.349+0000 7fb57a5aeb80 10 client.0 _ll_register_callbacks cb 0x563b78df4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb56499b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 0x563b78f148d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48354/0 (socket says 172.21.15.5:48354) 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 -- 172.21.15.5:0/3521116852 learned_addr learned my addr 172.21.15.5:0/3521116852 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:19.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb56519c700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f0eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 -- 172.21.15.5:0/3521116852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 msgr2=0x563b78f0eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f0eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 -- 172.21.15.5:0/3521116852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 msgr2=0x563b78f148d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 0x563b78f148d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 -- 172.21.15.5:0/3521116852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5601a4a30 con 0x563b78f0b470 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb56519c700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f0eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb56499b700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 0x563b78f148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:19.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb55ffff700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb54c01da40 tx=0x7fb54c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1558a1c889e9bc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.350+0000 7fb53ffff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c004320 con 0x563b78f0b470 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b78db0490 con 0x563b78f0b470 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb54c004480 con 0x563b78f0b470 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb54c00f850 con 0x563b78f0b470 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb57a5aeb80 10 client.28280 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:19.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563b78c6dc00 con 0x563b78f0b470 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b78db0490 con 0x563b78f0b470 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.351+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb54c003b60 con 0x563b78f0b470 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 10 client.28280 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 10 client.28280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 20 client.28280 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f1ee10 0x563b78f21220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 4 client.28280 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:19.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b78e4fe50 con 0x563b78f1ee10 2023-07-21T19:23:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb56499b700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f1ee10 0x563b78f21220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb53ffff700 10 client.28280 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:19.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.352+0000 7fb56499b700 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f1ee10 0x563b78f21220 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53033d2227396d93 server_cookie=58da6d2975012c3c in_seq=0 out_seq=0 2023-07-21T19:23:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.353+0000 7fb53ffff700 1 -- 172.21.15.5:0/3521116852 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b78e4fe50 con 0x563b78f1ee10 2023-07-21T19:23:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.353+0000 7fb53ffff700 10 client.28280 handle_command_reply: tid=0 2023-07-21T19:23:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.353+0000 7fb57a5aeb80 1 client.28280 shutdown 2023-07-21T19:23:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.353+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 msgr2=0x563b78f14e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.353+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fb54c01da40 tx=0x7fb54c016000 comp rx=0 tx=0).stop 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 shutdown_connections 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78f0eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f1ee10 0x563b78f21220 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78d616b0 0x563b78f148d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/3521116852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f14e10 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb53ffff700 0 client.28280 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:19.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 >> 172.21.15.5:0/3521116852 conn(0x563b78efad40 msgr2=0x563b78eff640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 shutdown_connections 2023-07-21T19:23:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/3521116852 wait complete. 2023-07-21T19:23:19.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.354+0000 7fb57a5aeb80 20 client.28280 trim_cache size 0 max 16384 2023-07-21T19:23:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 Processor -- start 2023-07-21T19:23:19.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 -- start start 2023-07-21T19:23:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f056e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78effdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78efc750 con 0x563b78f0b470 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78f05c20 con 0x563b78d616b0 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.377+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78f0d110 con 0x563b78f0b8c0 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f056e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48356/0 (socket says 172.21.15.5:48356) 2023-07-21T19:23:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f056e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49864/0 (socket says 172.21.15.5:49864) 2023-07-21T19:23:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56519c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78effdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 -- 172.21.15.5:0/2475155027 learned_addr learned my addr 172.21.15.5:0/2475155027 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:19.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 -- 172.21.15.5:0/2475155027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 msgr2=0x563b78effdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78effdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 -- 172.21.15.5:0/2475155027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 msgr2=0x563b78f056e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f056e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 -- 172.21.15.5:0/2475155027 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b78cab1f0 con 0x563b78d616b0 2023-07-21T19:23:19.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.378+0000 7fb56499b700 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b2f0 tx=0x7fb5540af040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cd9d9383920c51e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.379+0000 7fb54b7fe700 1 -- 172.21.15.5:0/2475155027 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554099df0 con 0x563b78d616b0 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.379+0000 7fb54b7fe700 1 -- 172.21.15.5:0/2475155027 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb554099f50 con 0x563b78d616b0 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.379+0000 7fb54b7fe700 1 -- 172.21.15.5:0/2475155027 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5540992e0 con 0x563b78d616b0 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.379+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/2475155027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 msgr2=0x563b78f051a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.379+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb55400b2f0 tx=0x7fb5540af040 comp rx=0 tx=0).stop 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/2475155027 shutdown_connections 2023-07-21T19:23:19.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78f0b8c0 0x563b78effdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78d616b0 0x563b78f051a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/2475155027 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b470 0x563b78f056e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/2475155027 >> 172.21.15.5:0/2475155027 conn(0x563b78efad40 msgr2=0x563b78f0dbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/2475155027 shutdown_connections 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/2475155027 wait complete. 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 -- start start 2023-07-21T19:23:19.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.380+0000 7fb57a5aeb80 1 Processor -- start 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f67160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b8c0 0x563b78f71ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78f678d0 con 0x563b78f0b8c0 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78f67a40 con 0x563b78f0b470 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b78f721e0 con 0x563b78d616b0 2023-07-21T19:23:19.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb57a5aeb80 10 client.0 _ll_register_callbacks cb 0x7fb54c01d800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb56499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb56519c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b8c0 0x563b78f71ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb55ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f67160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb56499b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38044/0 (socket says 172.21.15.5:38044) 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.381+0000 7fb56499b700 1 -- 172.21.15.5:0/745891136 learned_addr learned my addr 172.21.15.5:0/745891136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 -- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 msgr2=0x563b78f67160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f67160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 -- 172.21.15.5:0/745891136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b8c0 msgr2=0x563b78f71ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b8c0 0x563b78f71ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 -- 172.21.15.5:0/745891136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563b78c84d80 con 0x563b78d616b0 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb56499b700 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5540a1ab0 tx=0x7fb5540ca770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5d579ffed269f6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb53ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb554099df0 con 0x563b78d616b0 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb5540b2070 con 0x563b78d616b0 2023-07-21T19:23:19.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb5540a1fe0 con 0x563b78d616b0 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb554098030 con 0x563b78d616b0 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.382+0000 7fb57a5aeb80 10 client.28210 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x563b78c83250 con 0x563b78d616b0 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb5540a1fe0 con 0x563b78d616b0 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb554095980 con 0x563b78d616b0 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 10 client.28210 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:19.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 10 client.28210 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 20 client.28210 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f73410 0x563b78f757f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 4 client.28210 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:23:19.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b78e11ff0 con 0x563b78f73410 2023-07-21T19:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.383+0000 7fb55ffff700 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f73410 0x563b78f757f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.384+0000 7fb55ffff700 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f73410 0x563b78f757f0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb0da33b6c7c64f1 server_cookie=efbb95fd49fddbd2 in_seq=0 out_seq=0 2023-07-21T19:23:19.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.384+0000 7fb53ffff700 10 client.28210 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb53ffff700 1 -- 172.21.15.5:0/745891136 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b78e11ff0 con 0x563b78f73410 2023-07-21T19:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb53ffff700 10 client.28210 handle_command_reply: tid=0 2023-07-21T19:23:19.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 client.28210 shutdown 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 msgr2=0x563b78f66c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fb5540a1ab0 tx=0x7fb5540ca770 comp rx=0 tx=0).stop 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 shutdown_connections 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b78f73410 0x563b78f757f0 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b78f0b8c0 0x563b78f71ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b78d616b0 0x563b78f66c20 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 --2- 172.21.15.5:0/745891136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b78f0b470 0x563b78f67160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb53ffff700 0 client.28210 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.392+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 >> 172.21.15.5:0/745891136 conn(0x563b78efad40 msgr2=0x563b78f0bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 shutdown_connections 2023-07-21T19:23:19.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb57a5aeb80 1 -- 172.21.15.5:0/745891136 wait complete. 2023-07-21T19:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb57a5aeb80 20 client.28210 trim_cache size 0 max 16384 2023-07-21T19:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:23:19.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 msgr2=0x7fb5500a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 0x7fb5500a3770 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb5600fdac0 tx=0x7fb54c011820 comp rx=0 tx=0).stop 2023-07-21T19:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 msgr2=0x7fb56019da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fb55400d950 tx=0x7fb55400dc60 comp rx=0 tx=0).stop 2023-07-21T19:23:19.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 shutdown_connections 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb560106df0 0x7fb560197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb560101db0 0x7fb56019da40 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb5500a12c0 0x7fb5500a3770 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 --2- 172.21.15.5:0/4209073488 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb560102750 0x7fb56019df80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.393+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 >> 172.21.15.5:0/4209073488 conn(0x7fb5600fd660 msgr2=0x7fb5600fe120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.394+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 shutdown_connections 2023-07-21T19:23:19.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:19.394+0000 7fb56599d700 1 -- 172.21.15.5:0/4209073488 wait complete. 2023-07-21T19:23:19.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:19 smithi101 ceph-mon[128136]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:19.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:19 smithi005 ceph-mon[126460]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:19.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:19 smithi156 ceph-mon[128316]: pgmap v494: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:21.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:21 smithi101 ceph-mon[128136]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 23 op/s 2023-07-21T19:23:21.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:21 smithi005 ceph-mon[126460]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 23 op/s 2023-07-21T19:23:21.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:21 smithi156 ceph-mon[128316]: pgmap v495: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 32 MiB/s wr, 23 op/s 2023-07-21T19:23:23.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:23 smithi101 ceph-mon[128136]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:23.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:23 smithi005 ceph-mon[126460]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:23.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:23 smithi156 ceph-mon[128316]: pgmap v496: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:25 smithi005 ceph-mon[126460]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:25 smithi156 ceph-mon[128316]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:25 smithi101 ceph-mon[128136]: pgmap v497: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:27.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:27 smithi005 ceph-mon[126460]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:27.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:27 smithi156 ceph-mon[128316]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:28.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:27 smithi101 ceph-mon[128136]: pgmap v498: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 29 MiB/s wr, 22 op/s 2023-07-21T19:23:29.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:29 smithi005 ceph-mon[126460]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-21T19:23:29.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:29 smithi156 ceph-mon[128316]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-21T19:23:30.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:29 smithi101 ceph-mon[128136]: pgmap v499: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 KiB/s rd, 15 MiB/s wr, 15 op/s 2023-07-21T19:23:31.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:31 smithi005 ceph-mon[126460]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:23:31.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:31 smithi156 ceph-mon[128316]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:23:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:31 smithi101 ceph-mon[128136]: pgmap v500: 97 pgs: 97 active+clean; 13 GiB data, 5.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 14 MiB/s wr, 14 op/s 2023-07-21T19:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.555+0000 7fa2a85ff700 1 -- 172.21.15.5:0/43969151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a0100dd0 msgr2=0x7fa2a01011a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.555+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/43969151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a0100dd0 0x7fa2a01011a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009990 tx=0x7fa29c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 -- 172.21.15.5:0/43969151 shutdown_connections 2023-07-21T19:23:32.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/43969151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a010a110 0x7fa2a010c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/43969151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a0100dd0 0x7fa2a01011a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/43969151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0109bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 -- 172.21.15.5:0/43969151 >> 172.21.15.5:0/43969151 conn(0x7fa2a0075b90 msgr2=0x7fa2a0075f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 -- 172.21.15.5:0/43969151 shutdown_connections 2023-07-21T19:23:32.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.556+0000 7fa2a85ff700 1 -- 172.21.15.5:0/43969151 wait complete. 2023-07-21T19:23:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.557+0000 7fa2a85ff700 1 Processor -- start 2023-07-21T19:23:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.558+0000 7fa2a85ff700 1 -- start start 2023-07-21T19:23:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.558+0000 7fa2a85ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a0100dd0 0x7fa2a0197dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.558+0000 7fa2a85ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0198310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.558+0000 7fa2a85ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a85ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019cfc0 con 0x7fa2a01016e0 2023-07-21T19:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a85ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019d130 con 0x7fa2a010a110 2023-07-21T19:23:32.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a85ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a019d2f0 con 0x7fa2a0100dd0 2023-07-21T19:23:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55212/0 (socket says 172.21.15.5:55212) 2023-07-21T19:23:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 -- 172.21.15.5:0/1397769610 learned_addr learned my addr 172.21.15.5:0/1397769610 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:32.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0198310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 -- 172.21.15.5:0/1397769610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a0100dd0 msgr2=0x7fa2a0197dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a0100dd0 0x7fa2a0197dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 -- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 msgr2=0x7fa2a0198310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0198310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a6b9c700 1 -- 172.21.15.5:0/1397769610 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa29c0095f0 con 0x7fa2a010a110 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.559+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:23:32.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ef30 tx=0x7fa29800c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2977fe700 1 -- 172.21.15.5:0/1397769610 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298018070 con 0x7fa2a010a110 2023-07-21T19:23:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2977fe700 1 -- 172.21.15.5:0/1397769610 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29800f040 con 0x7fa2a010a110 2023-07-21T19:23:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2977fe700 1 -- 172.21.15.5:0/1397769610 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa298003d00 con 0x7fa2a010a110 2023-07-21T19:23:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a019d570 con 0x7fa2a010a110 2023-07-21T19:23:32.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.560+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a01a4e10 con 0x7fa2a010a110 2023-07-21T19:23:32.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.565+0000 7fa2977fe700 1 -- 172.21.15.5:0/1397769610 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa29800caa0 con 0x7fa2a010a110 2023-07-21T19:23:32.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.567+0000 7fa2bbff5b80 1 Processor -- start 2023-07-21T19:23:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2977fe700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 0x7fa28c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2bbff5b80 1 -- start start 2023-07-21T19:23:32.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2977fe700 1 -- 172.21.15.5:0/1397769610 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa2980cc9f0 con 0x7fa2a010a110 2023-07-21T19:23:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02491c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.568+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc02453f0 0x562fc0249700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc0141c50 con 0x562fc00986b0 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc01949e0 con 0x562fc02453f0 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc018de70 con 0x562fc0244f80 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 0x7fa28c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54466/0 (socket says 172.21.15.5:54466) 2023-07-21T19:23:32.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2a639b700 1 -- 172.21.15.5:0/1366919067 learned_addr learned my addr 172.21.15.5:0/1366919067 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:32.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.569+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02491c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 -- 172.21.15.5:0/1366919067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 msgr2=0x562fc02491c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02491c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 -- 172.21.15.5:0/1366919067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc02453f0 msgr2=0x562fc0249700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:23:32.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc02453f0 0x562fc0249700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 -- 172.21.15.5:0/1366919067 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fbffbbd80 con 0x562fc00986b0 2023-07-21T19:23:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005eb0 tx=0x7fa29c005ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c6e5aed706f9908 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 0x7fa28c0a3870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2a01995f0 tx=0x7fa290009890 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa294bfe700 1 -- 172.21.15.5:0/1366919067 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01d070 con 0x562fc00986b0 2023-07-21T19:23:32.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa294bfe700 1 -- 172.21.15.5:0/1366919067 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29c004c70 con 0x562fc00986b0 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.570+0000 7fa294bfe700 1 -- 172.21.15.5:0/1366919067 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c00b970 con 0x562fc00986b0 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1366919067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 msgr2=0x562fc0244a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7fa29c005eb0 tx=0x7fa29c005ee0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1366919067 shutdown_connections 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc02453f0 0x562fc0249700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0244a40 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1366919067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02491c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.571+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1366919067 >> 172.21.15.5:0/1366919067 conn(0x562fc0231be0 msgr2=0x562fc0236f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.572+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1366919067 shutdown_connections 2023-07-21T19:23:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.572+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1366919067 wait complete. 2023-07-21T19:23:32.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.572+0000 7fa2bbff5b80 1 -- start start 2023-07-21T19:23:32.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.572+0000 7fa2bbff5b80 1 Processor -- start 2023-07-21T19:23:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02543c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 0x562fc024e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc012da60 con 0x562fc02453f0 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc012d760 con 0x562fc00986b0 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc00118c0 con 0x562fc0244f80 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.573+0000 7fa2bbff5b80 10 client.0 _ll_register_callbacks cb 0x562fc012b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55216/0 (socket says 172.21.15.5:55216) 2023-07-21T19:23:32.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 -- 172.21.15.5:0/551994789 learned_addr learned my addr 172.21.15.5:0/551994789 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 0x562fc024e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02543c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 -- 172.21.15.5:0/551994789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 msgr2=0x562fc02543c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02543c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 -- 172.21.15.5:0/551994789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 msgr2=0x562fc024e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 0x562fc024e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 -- 172.21.15.5:0/551994789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a010d240 con 0x562fc00986b0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 0x562fc024e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa2a639b700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa29c000c00 tx=0x7fa29c005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acab24497d0cd80d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.574+0000 7fa289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01d070 con 0x562fc00986b0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa29c0049f0 con 0x562fc00986b0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562fc00e7490 con 0x562fc00986b0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa29c00b710 con 0x562fc00986b0 2023-07-21T19:23:32.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa2bbff5b80 10 client.28295 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562fbffa4c00 con 0x562fc00986b0 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29c037020 con 0x562fc00986b0 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.575+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa29c00f6f0 con 0x562fc00986b0 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 10 client.28295 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 10 client.28295 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 20 client.28295 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc0260410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 4 client.28295 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562fc0186e50 con 0x562fc025e000 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.576+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc0260410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.577+0000 7fa289ffb700 10 client.28295 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.577+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc0260410 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6ab2947a39ebd7 server_cookie=9015def4d76c3129 in_seq=0 out_seq=0 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.577+0000 7fa289ffb700 1 -- 172.21.15.5:0/551994789 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562fc0186e50 con 0x562fc025e000 2023-07-21T19:23:32.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.577+0000 7fa289ffb700 10 client.28295 handle_command_reply: tid=0 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 client.28295 shutdown 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 msgr2=0x562fc0253e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fa29c000c00 tx=0x7fa29c005050 comp rx=0 tx=0).stop 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 shutdown_connections 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc0260410 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc02453f0 0x562fc024e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc00986b0 0x562fc0253e80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa289ffb700 0 client.28295 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/551994789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc0244f80 0x562fc02543c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 >> 172.21.15.5:0/551994789 conn(0x562fc0231be0 msgr2=0x562fc02364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 shutdown_connections 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/551994789 wait complete. 2023-07-21T19:23:32.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.578+0000 7fa2bbff5b80 20 client.28295 trim_cache size 0 max 16384 2023-07-21T19:23:32.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.597+0000 7fa2bbff5b80 1 Processor -- start 2023-07-21T19:23:32.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 -- start start 2023-07-21T19:23:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc02495d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0246ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc0186e50 con 0x562fc00986b0 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc0245a40 con 0x562fc0244f80 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc022ec00 con 0x562fc02453f0 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc02495d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a5b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55218/0 (socket says 172.21.15.5:55218) 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a5b9a700 1 -- 172.21.15.5:0/1271045071 learned_addr learned my addr 172.21.15.5:0/1271045071 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:32.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc02495d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54482/0 (socket says 172.21.15.5:54482) 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.598+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0246ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 -- 172.21.15.5:0/1271045071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 msgr2=0x562fc0246ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0246ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 -- 172.21.15.5:0/1271045071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 msgr2=0x562fc02495d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc02495d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 -- 172.21.15.5:0/1271045071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562fbffe21f0 con 0x562fc0244f80 2023-07-21T19:23:32.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa290009fd0 tx=0x7fa2900047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=95dc756321cdfce1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2953ff700 1 -- 172.21.15.5:0/1271045071 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290009730 con 0x562fc0244f80 2023-07-21T19:23:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2953ff700 1 -- 172.21.15.5:0/1271045071 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29002ad60 con 0x562fc0244f80 2023-07-21T19:23:32.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.599+0000 7fa2953ff700 1 -- 172.21.15.5:0/1271045071 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa290003a80 con 0x562fc0244f80 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1271045071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 msgr2=0x562fc0236c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fa290009fd0 tx=0x7fa2900047c0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1271045071 shutdown_connections 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0246ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc02495d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1271045071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0236c60 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1271045071 >> 172.21.15.5:0/1271045071 conn(0x562fc0231be0 msgr2=0x562fc024dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1271045071 shutdown_connections 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.600+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1271045071 wait complete. 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.601+0000 7fa2bbff5b80 1 -- start start 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.601+0000 7fa2bbff5b80 1 Processor -- start 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0251e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0252380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc02b0f10 con 0x562fc00986b0 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc02b1080 con 0x562fc0244f80 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562fc02b1240 con 0x562fc02453f0 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2bbff5b80 10 client.0 _ll_register_callbacks cb 0x7fa290013500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a639b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54484/0 (socket says 172.21.15.5:54484) 2023-07-21T19:23:32.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a639b700 1 -- 172.21.15.5:0/1902999593 learned_addr learned my addr 172.21.15.5:0/1902999593 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0251e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a6b9c700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0252380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.602+0000 7fa2a639b700 1 -- 172.21.15.5:0/1902999593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 msgr2=0x562fc0252380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0252380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a639b700 1 -- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 msgr2=0x562fc0251e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0251e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a639b700 1 -- 172.21.15.5:0/1902999593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa29001b040 con 0x562fc00986b0 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0251e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2a639b700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c00f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505043d2c9e9ecf9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:32.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa289ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa29c01d040 con 0x562fc00986b0 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa29c038080 con 0x562fc00986b0 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa29c004c70 con 0x562fc00986b0 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.603+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa29c00fd10 con 0x562fc00986b0 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 10 client.18645 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:32.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x562fbffba250 con 0x562fc00986b0 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa29c004c70 con 0x562fc00986b0 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa29c03a070 con 0x562fc00986b0 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 10 client.18645 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 10 client.18645 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 20 client.18645 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc02b4660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:32.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 4 client.18645 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:23:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562fc0148ff0 con 0x562fc025e000 2023-07-21T19:23:32.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.604+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc02b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.605+0000 7fa289ffb700 10 client.18645 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.605+0000 7fa2a5b9a700 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc02b4660 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=463421c3f85a5d44 server_cookie=54db9fb36a366207 in_seq=0 out_seq=0 2023-07-21T19:23:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.612+0000 7fa289ffb700 1 -- 172.21.15.5:0/1902999593 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562fc0148ff0 con 0x562fc025e000 2023-07-21T19:23:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.612+0000 7fa289ffb700 10 client.18645 handle_command_reply: tid=0 2023-07-21T19:23:32.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.612+0000 7fa2bbff5b80 1 client.18645 shutdown 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 msgr2=0x562fc0251900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7fa29c009900 tx=0x7fa29c00f960 comp rx=0 tx=0).stop 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 shutdown_connections 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562fc02453f0 0x562fc0252380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562fc025e000 0x562fc02b4660 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562fc00986b0 0x562fc0251900 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 --2- 172.21.15.5:0/1902999593 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562fc0244f80 0x562fc0251e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 >> 172.21.15.5:0/1902999593 conn(0x562fc0231be0 msgr2=0x562fc023b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa289ffb700 0 client.18645 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 shutdown_connections 2023-07-21T19:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 1 -- 172.21.15.5:0/1902999593 wait complete. 2023-07-21T19:23:32.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.613+0000 7fa2bbff5b80 20 client.18645 trim_cache size 0 max 16384 2023-07-21T19:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:23:32.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 msgr2=0x7fa28c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 0x7fa28c0a3870 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa2a01995f0 tx=0x7fa290009890 comp rx=0 tx=0).stop 2023-07-21T19:23:32.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 msgr2=0x7fa2a019c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fa29800ef30 tx=0x7fa29800c450 comp rx=0 tx=0).stop 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 shutdown_connections 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2a010a110 0x7fa2a019c980 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2a0100dd0 0x7fa2a0197dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa28c0a13c0 0x7fa28c0a3870 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 --2- 172.21.15.5:0/1397769610 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2a01016e0 0x7fa2a0198310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 >> 172.21.15.5:0/1397769610 conn(0x7fa2a0075b90 msgr2=0x7fa2a00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 shutdown_connections 2023-07-21T19:23:32.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:32.614+0000 7fa2a85ff700 1 -- 172.21.15.5:0/1397769610 wait complete. 2023-07-21T19:23:33.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:33 smithi005 ceph-mon[126460]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:23:33.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:33 smithi156 ceph-mon[128316]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:23:34.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:33 smithi101 ceph-mon[128136]: pgmap v501: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 24 MiB/s wr, 21 op/s 2023-07-21T19:23:35.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:35 smithi005 ceph-mon[126460]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-21T19:23:35.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:35 smithi156 ceph-mon[128316]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-21T19:23:36.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:35 smithi101 ceph-mon[128136]: pgmap v502: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 15 MiB/s wr, 14 op/s 2023-07-21T19:23:37.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:37 smithi005 ceph-mon[126460]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-21T19:23:37.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:37 smithi156 ceph-mon[128316]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-21T19:23:38.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:37 smithi101 ceph-mon[128136]: pgmap v503: 97 pgs: 97 active+clean; 13 GiB data, 5.6 GiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 14 MiB/s wr, 12 op/s 2023-07-21T19:23:39.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:39 smithi005 ceph-mon[126460]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 22 op/s 2023-07-21T19:23:39.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:39 smithi156 ceph-mon[128316]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 22 op/s 2023-07-21T19:23:40.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:39 smithi101 ceph-mon[128136]: pgmap v504: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 26 MiB/s wr, 22 op/s 2023-07-21T19:23:41.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:41 smithi005 ceph-mon[126460]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:41.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:41 smithi156 ceph-mon[128316]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:42.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:41 smithi101 ceph-mon[128136]: pgmap v505: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:23:43.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:43 smithi005 ceph-mon[126460]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:43.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:43 smithi156 ceph-mon[128316]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:44.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:43 smithi101 ceph-mon[128136]: pgmap v506: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:45.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 -- 172.21.15.5:0/4014733164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 msgr2=0x7f51dc106910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 --2- 172.21.15.5:0/4014733164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc106910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f51c4009990 tx=0x7f51c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 -- 172.21.15.5:0/4014733164 shutdown_connections 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 --2- 172.21.15.5:0/4014733164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc106ff0 0x7f51dc10ebc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 --2- 172.21.15.5:0/4014733164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc101fb0 0x7f51dc102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 --2- 172.21.15.5:0/4014733164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc106910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.768+0000 7f51e1bff700 1 -- 172.21.15.5:0/4014733164 >> 172.21.15.5:0/4014733164 conn(0x7f51dc0fd800 msgr2=0x7f51dc0ffc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.769+0000 7f51e1bff700 1 -- 172.21.15.5:0/4014733164 shutdown_connections 2023-07-21T19:23:45.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.769+0000 7f51e1bff700 1 -- 172.21.15.5:0/4014733164 wait complete. 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 Processor -- start 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 -- start start 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc101fb0 0x7f51dc197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 0x7f51dc19c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc19d020 con 0x7f51dc106ff0 2023-07-21T19:23:45.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc19d190 con 0x7f51dc102950 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.770+0000 7f51e1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51dc19d350 con 0x7f51dc101fb0 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37436/0 (socket says 172.21.15.5:37436) 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1239581982 learned_addr learned my addr 172.21.15.5:0/1239581982 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 0x7f51dc19c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51db7fe700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc101fb0 0x7f51dc197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1239581982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc101fb0 msgr2=0x7f51dc197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc101fb0 0x7f51dc197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 msgr2=0x7f51dc19c9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 0x7f51dc19c9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1239581982 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40095f0 con 0x7f51dc102950 2023-07-21T19:23:45.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51dbfff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 0x7f51dc19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.771+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.772+0000 7f51d97fa700 1 -- 172.21.15.5:0/1239581982 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c401d070 con 0x7f51dc102950 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.772+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51dc19d5d0 con 0x7f51dc102950 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.772+0000 7f51d97fa700 1 -- 172.21.15.5:0/1239581982 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c4005030 con 0x7f51dc102950 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.772+0000 7f51d97fa700 1 -- 172.21.15.5:0/1239581982 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4004bc0 con 0x7f51dc102950 2023-07-21T19:23:45.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.772+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51dc19d840 con 0x7f51dc102950 2023-07-21T19:23:45.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.776+0000 7f51d97fa700 1 -- 172.21.15.5:0/1239581982 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f51c40051a0 con 0x7f51dc102950 2023-07-21T19:23:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.777+0000 7f51f54c9b80 1 Processor -- start 2023-07-21T19:23:45.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.777+0000 7f51f54c9b80 1 -- start start 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2944040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29449d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c2849c50 con 0x55e1c294a2d0 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c289c9e0 con 0x55e1c2944580 2023-07-21T19:23:45.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c2895e70 con 0x55e1c27a06b0 2023-07-21T19:23:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29449d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51dbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39242/0 (socket says 172.21.15.5:39242) 2023-07-21T19:23:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51dbfff700 1 -- 172.21.15.5:0/2838638935 learned_addr learned my addr 172.21.15.5:0/2838638935 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:45.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.778+0000 7f51db7fe700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2944040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d97fa700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 0x7f51c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d97fa700 1 -- 172.21.15.5:0/1239581982 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f51c40cddf0 con 0x7f51dc102950 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 -- 172.21.15.5:0/2838638935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 msgr2=0x55e1c2949d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 -- 172.21.15.5:0/2838638935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 msgr2=0x55e1c2944040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2944040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.773 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 -- 172.21.15.5:0/2838638935 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e1c26c3d80 con 0x55e1c2944580 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51dbfff700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51db7fe700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2944040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51dbfff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 0x7f51c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.779+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29449d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f51c40cb4e0 tx=0x7f51c4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a19108ebeba0de3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.780+0000 7f51d1ffb700 1 -- 172.21.15.5:0/2838638935 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c409aa30 con 0x55e1c2944580 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.780+0000 7f51d1ffb700 1 -- 172.21.15.5:0/2838638935 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c409a850 con 0x55e1c2944580 2023-07-21T19:23:45.774 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.780+0000 7f51d1ffb700 1 -- 172.21.15.5:0/2838638935 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c40993b0 con 0x55e1c2944580 2023-07-21T19:23:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 -- 172.21.15.5:0/2838638935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 msgr2=0x55e1c29449d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29449d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f51c40cb4e0 tx=0x7f51c4009310 comp rx=0 tx=0).stop 2023-07-21T19:23:45.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51dbfff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 0x7f51c80a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51d40046e0 tx=0x7f51d4004750 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 -- 172.21.15.5:0/2838638935 shutdown_connections 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29449d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2949d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/2838638935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2944040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 -- 172.21.15.5:0/2838638935 >> 172.21.15.5:0/2838638935 conn(0x55e1c2939be0 msgr2=0x55e1c293ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 -- 172.21.15.5:0/2838638935 shutdown_connections 2023-07-21T19:23:45.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.781+0000 7f51f54c9b80 1 -- 172.21.15.5:0/2838638935 wait complete. 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.782+0000 7f51f54c9b80 1 -- start start 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.782+0000 7f51f54c9b80 1 Processor -- start 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c294d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c2835a60 con 0x55e1c294a2d0 2023-07-21T19:23:45.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c2835760 con 0x55e1c2944580 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c27198c0 con 0x55e1c27a06b0 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51f54c9b80 10 client.0 _ll_register_callbacks cb 0x55e1c2833700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39258/0 (socket says 172.21.15.5:39258) 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37462/0 (socket says 172.21.15.5:37462) 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51db7fe700 1 -- 172.21.15.5:0/1208825816 learned_addr learned my addr 172.21.15.5:0/1208825816 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:45.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51dbfff700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c294d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1208825816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 msgr2=0x55e1c29532a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1208825816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 msgr2=0x55e1c294d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c294d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.783+0000 7f51d2ffd700 1 -- 172.21.15.5:0/1208825816 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d4009da0 con 0x55e1c2944580 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51db7fe700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:45.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f51c409abf0 tx=0x7f51c4096b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2cd85973ac89d6d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51baffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c409aef0 con 0x55e1c2944580 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f51c409ef20 con 0x55e1c2944580 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e1c27ef490 con 0x55e1c2944580 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51c4005480 con 0x55e1c2944580 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51f54c9b80 10 client.28313 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.784+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7f51cc003b70 con 0x55e1c2944580 2023-07-21T19:23:45.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c40c9080 con 0x55e1c2944580 2023-07-21T19:23:45.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f51c40b9020 con 0x55e1c2944580 2023-07-21T19:23:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 10 client.28313 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:45.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 10 client.28313 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 20 client.28313 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c295d310 0x55e1c295f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 4 client.28313 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e1c288ee50 con 0x55e1c295d310 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.785+0000 7f51db7fe700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c295d310 0x55e1c295f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.786+0000 7f51db7fe700 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c295d310 0x55e1c295f720 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab4d3480063aa54 server_cookie=38554f35f03244b1 in_seq=0 out_seq=0 2023-07-21T19:23:45.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.786+0000 7f51baffd700 10 client.28313 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51baffd700 1 -- 172.21.15.5:0/1208825816 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e1c288ee50 con 0x55e1c295d310 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51baffd700 10 client.28313 handle_command_reply: tid=0 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 client.28313 shutdown 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 msgr2=0x55e1c29537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f51c409abf0 tx=0x7f51c4096b10 comp rx=0 tx=0).stop 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 shutdown_connections 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c295d310 0x55e1c295f720 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c294d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c29532a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/1208825816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c2944580 0x55e1c29537e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51baffd700 0 client.28313 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 >> 172.21.15.5:0/1208825816 conn(0x55e1c2939be0 msgr2=0x55e1c293e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 shutdown_connections 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 1 -- 172.21.15.5:0/1208825816 wait complete. 2023-07-21T19:23:45.786 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.787+0000 7f51f54c9b80 20 client.28313 trim_cache size 0 max 16384 2023-07-21T19:23:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 Processor -- start 2023-07-21T19:23:45.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 -- start start 2023-07-21T19:23:45.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2944040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c294a2d0 0x55e1c293ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c294ead0 con 0x55e1c2944580 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c2944ff0 con 0x55e1c294a2d0 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.810+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c294b4f0 con 0x55e1c27a06b0 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54196/0 (socket says 172.21.15.5:54196) 2023-07-21T19:23:45.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 -- 172.21.15.5:0/3575868038 learned_addr learned my addr 172.21.15.5:0/3575868038 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51dbfff700 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c294a2d0 0x55e1c293ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51db7fe700 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2944040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 -- 172.21.15.5:0/3575868038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 msgr2=0x55e1c2944040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2944040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 -- 172.21.15.5:0/3575868038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c294a2d0 msgr2=0x55e1c293ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c294a2d0 0x55e1c293ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 -- 172.21.15.5:0/3575868038 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc19d840 con 0x55e1c2944580 2023-07-21T19:23:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.811+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b650 tx=0x7f51c4096c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3937ceb61926a8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.812+0000 7f51d27fc700 1 -- 172.21.15.5:0/3575868038 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c409aa30 con 0x55e1c2944580 2023-07-21T19:23:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.812+0000 7f51d27fc700 1 -- 172.21.15.5:0/3575868038 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e1c27ef490 con 0x55e1c2944580 2023-07-21T19:23:45.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.812+0000 7f51d27fc700 1 -- 172.21.15.5:0/3575868038 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c4005480 con 0x55e1c2944580 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.812+0000 7f51f54c9b80 1 -- 172.21.15.5:0/3575868038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 msgr2=0x55e1c294afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.812+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f51c400b650 tx=0x7f51c4096c30 comp rx=0 tx=0).stop 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 -- 172.21.15.5:0/3575868038 shutdown_connections 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c294a2d0 0x55e1c293ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c27a06b0 0x55e1c2944040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/3575868038 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c2944580 0x55e1c294afb0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 -- 172.21.15.5:0/3575868038 >> 172.21.15.5:0/3575868038 conn(0x55e1c2939be0 msgr2=0x55e1c294a8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 -- 172.21.15.5:0/3575868038 shutdown_connections 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 -- 172.21.15.5:0/3575868038 wait complete. 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.813+0000 7f51f54c9b80 1 -- start start 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 Processor -- start 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c2944580 0x55e1c2951350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2951890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c29b01d0 con 0x55e1c294a2d0 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c29b0310 con 0x55e1c27a06b0 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e1c29b04d0 con 0x55e1c2944580 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.814+0000 7f51f54c9b80 10 client.0 _ll_register_callbacks cb 0x7f51cc003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37482/0 (socket says 172.21.15.5:37482) 2023-07-21T19:23:45.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 -- 172.21.15.5:0/4218141479 learned_addr learned my addr 172.21.15.5:0/4218141479 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51dbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2951890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 -- 172.21.15.5:0/4218141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c2944580 msgr2=0x55e1c2951350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c2944580 0x55e1c2951350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 -- 172.21.15.5:0/4218141479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 msgr2=0x55e1c2951890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2951890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 -- 172.21.15.5:0/4218141479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51cc003b70 con 0x55e1c27a06b0 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51db7fe700 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55e1c2960000 tx=0x7f51cc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15c1ee1106e1dbe2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:45.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.815+0000 7f51baffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc01c040 con 0x55e1c27a06b0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f51c409a850 con 0x55e1c27a06b0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f51cc01f8d0 con 0x55e1c27a06b0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7f51cc01fa90 con 0x55e1c27a06b0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51f54c9b80 10 client.28319 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x55e1c26acc00 con 0x55e1c27a06b0 2023-07-21T19:23:45.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f51c409a850 con 0x55e1c27a06b0 2023-07-21T19:23:45.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.816+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f51cc031cb0 con 0x55e1c27a06b0 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 10 client.28319 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 10 client.28319 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 20 client.28319 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c29b1c50 0x55e1c29b4060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 4 client.28319 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e1c2850ff0 con 0x55e1c29b1c50 2023-07-21T19:23:45.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.817+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c29b1c50 0x55e1c29b4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.818+0000 7f51d2ffd700 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c29b1c50 0x55e1c29b4060 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b849f7c11f2c6ca9 server_cookie=3c6aeb102d32258b in_seq=0 out_seq=0 2023-07-21T19:23:45.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.818+0000 7f51baffd700 10 client.28319 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51baffd700 1 -- 172.21.15.5:0/4218141479 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e1c2850ff0 con 0x55e1c29b1c50 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51baffd700 10 client.28319 handle_command_reply: tid=0 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 client.28319 shutdown 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 msgr2=0x55e1c2950e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x55e1c2960000 tx=0x7f51cc004ad0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 shutdown_connections 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e1c29b1c50 0x55e1c29b4060 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e1c294a2d0 0x55e1c2951890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e1c27a06b0 0x55e1c2950e10 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 --2- 172.21.15.5:0/4218141479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e1c2944580 0x55e1c2951350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51baffd700 0 client.28319 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:45.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 >> 172.21.15.5:0/4218141479 conn(0x55e1c2939be0 msgr2=0x55e1c2949110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 shutdown_connections 2023-07-21T19:23:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.826+0000 7f51f54c9b80 1 -- 172.21.15.5:0/4218141479 wait complete. 2023-07-21T19:23:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51f54c9b80 20 client.28319 trim_cache size 0 max 16384 2023-07-21T19:23:45.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:23:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:23:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:23:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:23:45.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 msgr2=0x7f51c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 0x7f51c80a3770 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f51d40046e0 tx=0x7f51d4004750 comp rx=0 tx=0).stop 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 msgr2=0x7f51dc198370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f51c4006010 tx=0x7f51c4004730 comp rx=0 tx=0).stop 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 shutdown_connections 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f51c80a12c0 0x7f51c80a3770 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f51dc106ff0 0x7f51dc19c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f51dc101fb0 0x7f51dc197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 --2- 172.21.15.5:0/1239581982 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f51dc102950 0x7f51dc198370 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.827+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 >> 172.21.15.5:0/1239581982 conn(0x7f51dc0fd800 msgr2=0x7f51dc0fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.828+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 shutdown_connections 2023-07-21T19:23:45.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:45.828+0000 7f51e1bff700 1 -- 172.21.15.5:0/1239581982 wait complete. 2023-07-21T19:23:45.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:45 smithi005 ceph-mon[126460]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 19 op/s 2023-07-21T19:23:45.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:45 smithi156 ceph-mon[128316]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 19 op/s 2023-07-21T19:23:46.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:45 smithi101 ceph-mon[128136]: pgmap v507: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 27 MiB/s wr, 19 op/s 2023-07-21T19:23:47.726 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:23:47.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:47 smithi005 ceph-mon[126460]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:47.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:47 smithi156 ceph-mon[128316]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:47.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 -- 172.21.15.5:0/556505739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228105ae0 msgr2=0x7fc228105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 --2- 172.21.15.5:0/556505739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228105ae0 0x7fc228105eb0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fc218009990 tx=0x7fc218009ca0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 -- 172.21.15.5:0/556505739 shutdown_connections 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 --2- 172.21.15.5:0/556505739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228100480 0x7fc22810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 --2- 172.21.15.5:0/556505739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc228105ae0 0x7fc228105eb0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 --2- 172.21.15.5:0/556505739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2280ffad0 0x7fc2280fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.980+0000 7fc22ddff700 1 -- 172.21.15.5:0/556505739 >> 172.21.15.5:0/556505739 conn(0x7fc2280fb5f0 msgr2=0x7fc2280fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.981+0000 7fc22ddff700 1 -- 172.21.15.5:0/556505739 shutdown_connections 2023-07-21T19:23:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.981+0000 7fc22ddff700 1 -- 172.21.15.5:0/556505739 wait complete. 2023-07-21T19:23:47.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.982+0000 7fc22ddff700 1 Processor -- start 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.982+0000 7fc22ddff700 1 -- start start 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc228100480 0x7fc228072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228105ae0 0x7fc2281a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228073580 con 0x7fc228100480 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a4240 con 0x7fc228105ae0 2023-07-21T19:23:47.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc22ddff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2281a4380 con 0x7fc2280ffad0 2023-07-21T19:23:47.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48028/0 (socket says 172.21.15.5:48028) 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc2277fe700 1 -- 172.21.15.5:0/1378696443 learned_addr learned my addr 172.21.15.5:0/1378696443 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc226ffd700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc228100480 0x7fc228072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.983+0000 7fc227fff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228105ae0 0x7fc2281a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 -- 172.21.15.5:0/1378696443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228105ae0 msgr2=0x7fc2281a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228105ae0 0x7fc2281a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 -- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc228100480 msgr2=0x7fc228072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc228100480 0x7fc228072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 -- 172.21.15.5:0/1378696443 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2180095f0 con 0x7fc2280ffad0 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc2277fe700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc218005770 tx=0x7fc2180057a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc224ff9700 1 -- 172.21.15.5:0/1378696443 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21801d070 con 0x7fc2280ffad0 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc224ff9700 1 -- 172.21.15.5:0/1378696443 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc218021410 con 0x7fc2280ffad0 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.984+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2281a4580 con 0x7fc2280ffad0 2023-07-21T19:23:47.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.985+0000 7fc224ff9700 1 -- 172.21.15.5:0/1378696443 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218017650 con 0x7fc2280ffad0 2023-07-21T19:23:47.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.985+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2281a4a00 con 0x7fc2280ffad0 2023-07-21T19:23:47.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.989+0000 7fc224ff9700 1 -- 172.21.15.5:0/1378696443 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc2180217f0 con 0x7fc2280ffad0 2023-07-21T19:23:47.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.990+0000 7fc24179db80 1 Processor -- start 2023-07-21T19:23:47.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.990+0000 7fc24179db80 1 -- start start 2023-07-21T19:23:47.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.990+0000 7fc224ff9700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 0x7fc2100a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc224ff9700 1 -- 172.21.15.5:0/1378696443 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc2180ce140 con 0x7fc2280ffad0 2023-07-21T19:23:47.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a19eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0912cb0 con 0x5595f0a17ad0 2023-07-21T19:23:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0965a40 con 0x5595f0a0cbe0 2023-07-21T19:23:47.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc24179db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f095eed0 con 0x5595f0a0c2d0 2023-07-21T19:23:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc227fff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 0x7fc2100a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36114/0 (socket says 172.21.15.5:36114) 2023-07-21T19:23:47.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.991+0000 7fc226ffd700 1 -- 172.21.15.5:0/1430371767 learned_addr learned my addr 172.21.15.5:0/1430371767 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc227fff700 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a19eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 -- 172.21.15.5:0/1430371767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 msgr2=0x5595f0a0c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 -- 172.21.15.5:0/1430371767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 msgr2=0x5595f0a19eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a19eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 -- 172.21.15.5:0/1430371767 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f078cde0 con 0x5595f0a0cbe0 2023-07-21T19:23:47.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc227fff700 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:23:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc226ffd700 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009d20 tx=0x7fc21c011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dd53ae95cca3ce7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc227fff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 0x7fc2100a3770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc228074920 tx=0x7fc220009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:47.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc20dffb700 1 -- 172.21.15.5:0/1430371767 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c019630 con 0x5595f0a0cbe0 2023-07-21T19:23:47.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc20dffb700 1 -- 172.21.15.5:0/1430371767 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc21c019c20 con 0x5595f0a0cbe0 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.992+0000 7fc20dffb700 1 -- 172.21.15.5:0/1430371767 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21c00eab0 con 0x5595f0a0cbe0 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 -- 172.21.15.5:0/1430371767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 msgr2=0x5595f0a17590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fc21c009d20 tx=0x7fc21c011720 comp rx=0 tx=0).stop 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 -- 172.21.15.5:0/1430371767 shutdown_connections 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a19eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 --2- 172.21.15.5:0/1430371767 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0cbe0 0x5595f0a17590 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.993+0000 7fc24179db80 1 -- 172.21.15.5:0/1430371767 >> 172.21.15.5:0/1430371767 conn(0x5595f0a02da0 msgr2=0x5595f0a086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.994+0000 7fc24179db80 1 -- 172.21.15.5:0/1430371767 shutdown_connections 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.994+0000 7fc24179db80 1 -- 172.21.15.5:0/1430371767 wait complete. 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.994+0000 7fc24179db80 1 -- start start 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.994+0000 7fc24179db80 1 Processor -- start 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a174e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a0cbe0 0x5595f0a147d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f08feac0 con 0x5595f0a0cbe0 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f08fe7c0 con 0x5595f0a17ad0 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f07e2920 con 0x5595f0a0c2d0 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.995+0000 7fc24179db80 10 client.0 _ll_register_callbacks cb 0x5595f08fc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36122/0 (socket says 172.21.15.5:36122) 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 -- 172.21.15.5:0/2281199080 learned_addr learned my addr 172.21.15.5:0/2281199080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:47.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc226ffd700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a0cbe0 0x5595f0a147d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a174e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 -- 172.21.15.5:0/2281199080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 msgr2=0x5595f0a174e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a174e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 -- 172.21.15.5:0/2281199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a0cbe0 msgr2=0x5595f0a147d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a0cbe0 0x5595f0a147d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 -- 172.21.15.5:0/2281199080 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc21c009980 con 0x5595f0a17ad0 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.996+0000 7fc227fff700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc22001aea0 tx=0x7fc220004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef0e46d96866121 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:47.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc206ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc22000e040 con 0x5595f0a17ad0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5595f08b84f0 con 0x5595f0a17ad0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc220018db0 con 0x5595f0a17ad0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc220009020 con 0x5595f0a17ad0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc24179db80 10 client.28331 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x5595f0775c60 con 0x5595f0a17ad0 2023-07-21T19:23:47.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.997+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5595f08b84f0 con 0x5595f0a17ad0 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc220008ca0 con 0x5595f0a17ad0 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 10 client.28331 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 10 client.28331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 20 client.28331 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a27480 0x5595f0a29890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 4 client.28331 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:47.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5595f0957eb0 con 0x5595f0a27480 2023-07-21T19:23:47.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.998+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a27480 0x5595f0a29890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.999+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a27480 0x5595f0a29890 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b835044c60892dc1 server_cookie=e0dea5c9ef35c18a in_seq=0 out_seq=0 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.999+0000 7fc206ffd700 10 client.28331 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.999+0000 7fc206ffd700 1 -- 172.21.15.5:0/2281199080 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5595f0957eb0 con 0x5595f0a27480 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.999+0000 7fc206ffd700 10 client.28331 handle_command_reply: tid=0 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:47.999+0000 7fc24179db80 1 client.28331 shutdown 2023-07-21T19:23:47.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 msgr2=0x5595f0a14e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fc22001aea0 tx=0x7fc220004ad0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 shutdown_connections 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a17ad0 0x5595f0a14e70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0c2d0 0x5595f0a174e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a27480 0x5595f0a29890 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 --2- 172.21.15.5:0/2281199080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a0cbe0 0x5595f0a147d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc206ffd700 0 client.28331 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:47.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 >> 172.21.15.5:0/2281199080 conn(0x5595f0a02da0 msgr2=0x5595f0a08230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 shutdown_connections 2023-07-21T19:23:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 1 -- 172.21.15.5:0/2281199080 wait complete. 2023-07-21T19:23:47.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.000+0000 7fc24179db80 20 client.28331 trim_cache size 0 max 16384 2023-07-21T19:23:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 Processor -- start 2023-07-21T19:23:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 -- start start 2023-07-21T19:23:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a1e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a07fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a0d330 con 0x5595f0a17ad0 2023-07-21T19:23:48.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a09d20 con 0x5595f0a0c2d0 2023-07-21T19:23:48.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.018+0000 7fc24179db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a24a10 con 0x5595f0a0cbe0 2023-07-21T19:23:48.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a1e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc227fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a07fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36132/0 (socket says 172.21.15.5:36132) 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 -- 172.21.15.5:0/2745257533 learned_addr learned my addr 172.21.15.5:0/2745257533 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc226ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a1e4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48042/0 (socket says 172.21.15.5:48042) 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 -- 172.21.15.5:0/2745257533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 msgr2=0x5595f0a1e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a1e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 -- 172.21.15.5:0/2745257533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 msgr2=0x5595f0a07fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a07fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 -- 172.21.15.5:0/2745257533 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5595f0a045e0 con 0x5595f0a0c2d0 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc227fff700 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a07fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.019+0000 7fc2277fe700 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2180cb890 tx=0x7fc218009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f50622b2f7904d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.020+0000 7fc20e7fc700 1 -- 172.21.15.5:0/2745257533 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc21809f270 con 0x5595f0a0c2d0 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.020+0000 7fc20e7fc700 1 -- 172.21.15.5:0/2745257533 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2180a3600 con 0x5595f0a0c2d0 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.020+0000 7fc20e7fc700 1 -- 172.21.15.5:0/2745257533 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc218021630 con 0x5595f0a0c2d0 2023-07-21T19:23:48.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.020+0000 7fc24179db80 1 -- 172.21.15.5:0/2745257533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 msgr2=0x5595f0a0f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.020+0000 7fc24179db80 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fc2180cb890 tx=0x7fc218009ca0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 -- 172.21.15.5:0/2745257533 shutdown_connections 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a07fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a0f1b0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 --2- 172.21.15.5:0/2745257533 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a1e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 -- 172.21.15.5:0/2745257533 >> 172.21.15.5:0/2745257533 conn(0x5595f0a02da0 msgr2=0x5595f0a078b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:48.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 -- 172.21.15.5:0/2745257533 shutdown_connections 2023-07-21T19:23:48.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 -- 172.21.15.5:0/2745257533 wait complete. 2023-07-21T19:23:48.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 -- start start 2023-07-21T19:23:48.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.021+0000 7fc24179db80 1 Processor -- start 2023-07-21T19:23:48.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a6fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a74390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a74900 con 0x5595f0a17ad0 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a74a70 con 0x5595f0a0c2d0 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5595f0a74c30 con 0x5595f0a0cbe0 2023-07-21T19:23:48.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc24179db80 10 client.0 _ll_register_callbacks cb 0x7fc22001a870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:48.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36136/0 (socket says 172.21.15.5:36136) 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc2277fe700 1 -- 172.21.15.5:0/471579721 learned_addr learned my addr 172.21.15.5:0/471579721 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc226ffd700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a6fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.022+0000 7fc227fff700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a74390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 -- 172.21.15.5:0/471579721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 msgr2=0x5595f0a6fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:48.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a6fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 -- 172.21.15.5:0/471579721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 msgr2=0x5595f0a74390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a74390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 -- 172.21.15.5:0/471579721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5595f0775c60 con 0x5595f0a0c2d0 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc2277fe700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc218006010 tx=0x7fc2180cbd90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ce496d1db413071d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc206ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2180995e0 con 0x5595f0a0c2d0 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc21c019c20 con 0x5595f0a0c2d0 2023-07-21T19:23:48.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc21809b4a0 con 0x5595f0a0c2d0 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=50) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc21809b6a0 con 0x5595f0a0c2d0 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc24179db80 10 client.28337 fetch_fsmap learned FSMap version 50 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.023+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=50,monmap=4+}) v3 -- 0x7fc2281a4a00 con 0x5595f0a0c2d0 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2180cb020 con 0x5595f0a0c2d0 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 50) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc2180a3170 con 0x5595f0a0c2d0 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 10 client.28337 fetch_fsmap finished waiting for FSMap version 50 2023-07-21T19:23:48.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 10 client.28337 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 20 client.28337 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a7c0b0 0x5595f0a7e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 4 client.28337 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5595f091a050 con 0x5595f0a7c0b0 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.024+0000 7fc226ffd700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a7c0b0 0x5595f0a7e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc226ffd700 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a7c0b0 0x5595f0a7e4c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f01fac499e5bdd1 server_cookie=5044acca5848b69 in_seq=0 out_seq=0 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc206ffd700 10 client.28337 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:48.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc206ffd700 1 -- 172.21.15.5:0/471579721 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5595f091a050 con 0x5595f0a7c0b0 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc206ffd700 10 client.28337 handle_command_reply: tid=0 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 client.28337 shutdown 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 msgr2=0x5595f0a6f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fc218006010 tx=0x7fc2180cbd90 comp rx=0 tx=0).stop 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 shutdown_connections 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5595f0a7c0b0 0x5595f0a7e4c0 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5595f0a17ad0 0x5595f0a74390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5595f0a0c2d0 0x5595f0a6f4c0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc206ffd700 0 client.28337 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 --2- 172.21.15.5:0/471579721 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5595f0a0cbe0 0x5595f0a6fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.025+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 >> 172.21.15.5:0/471579721 conn(0x5595f0a02da0 msgr2=0x5595f0a08140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 shutdown_connections 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc24179db80 1 -- 172.21.15.5:0/471579721 wait complete. 2023-07-21T19:23:48.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc24179db80 20 client.28337 trim_cache size 0 max 16384 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:23:48.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:23:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:23:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:23:48.028 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:23:48.028 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:23:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 msgr2=0x7fc2100a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 0x7fc2100a3770 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fc228074920 tx=0x7fc220009920 comp rx=0 tx=0).stop 2023-07-21T19:23:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 msgr2=0x7fc228072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:48.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.026+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fc218005770 tx=0x7fc2180057a0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 shutdown_connections 2023-07-21T19:23:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc228105ae0 0x7fc2281a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2280ffad0 0x7fc228072810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2100a12c0 0x7fc2100a3770 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 --2- 172.21.15.5:0/1378696443 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc228100480 0x7fc228072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 >> 172.21.15.5:0/1378696443 conn(0x7fc2280fb5f0 msgr2=0x7fc2280fbda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 shutdown_connections 2023-07-21T19:23:48.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:23:48.027+0000 7fc22ddff700 1 -- 172.21.15.5:0/1378696443 wait complete. 2023-07-21T19:23:48.041 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:23:48.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:47 smithi101 ceph-mon[128136]: pgmap v508: 97 pgs: 97 active+clean; 13 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:49.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:49 smithi005 ceph-mon[126460]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:23:49.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:49 smithi156 ceph-mon[128316]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:23:50.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:49 smithi101 ceph-mon[128136]: pgmap v509: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:23:51.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:51 smithi005 ceph-mon[126460]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:23:51.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:51 smithi156 ceph-mon[128316]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:23:52.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:51 smithi101 ceph-mon[128136]: pgmap v510: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:23:53.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:53 smithi005 ceph-mon[126460]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:53.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:53 smithi156 ceph-mon[128316]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:54.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:53 smithi101 ceph-mon[128136]: pgmap v511: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 35 MiB/s wr, 25 op/s 2023-07-21T19:23:55.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:55 smithi005 ceph-mon[126460]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:55.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:55 smithi156 ceph-mon[128316]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:56.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:55 smithi101 ceph-mon[128136]: pgmap v512: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 MiB/s wr, 18 op/s 2023-07-21T19:23:57.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:56 smithi101 ceph-mon[128136]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:23:57.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:56 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:57.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:56 smithi005 ceph-mon[126460]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:23:57.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:56 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:57.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:56 smithi156 ceph-mon[128316]: pgmap v513: 97 pgs: 97 active+clean; 13 GiB data, 5.8 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 23 MiB/s wr, 17 op/s 2023-07-21T19:23:57.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:56 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:23:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 -- 172.21.15.5:0/2854850863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a41002e0 msgr2=0x7fc8a410c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 --2- 172.21.15.5:0/2854850863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a41002e0 0x7fc8a410c770 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00b0a0 tx=0x7fc89c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 -- 172.21.15.5:0/2854850863 shutdown_connections 2023-07-21T19:23:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 --2- 172.21.15.5:0/2854850863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a41002e0 0x7fc8a410c770 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 --2- 172.21.15.5:0/2854850863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4105d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 --2- 172.21.15.5:0/2854850863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a40ff930 0x7fc8a40ffda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 -- 172.21.15.5:0/2854850863 >> 172.21.15.5:0/2854850863 conn(0x7fc8a40fb450 msgr2=0x7fc8a40fd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 -- 172.21.15.5:0/2854850863 shutdown_connections 2023-07-21T19:23:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.992+0000 7fc8a359e700 1 -- 172.21.15.5:0/2854850863 wait complete. 2023-07-21T19:23:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.993+0000 7fc8a359e700 1 Processor -- start 2023-07-21T19:23:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.993+0000 7fc8a359e700 1 -- start start 2023-07-21T19:23:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 0x7fc8a419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 0x7fc8a419df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4197a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a41980b0 con 0x7fc8a40ff930 2023-07-21T19:23:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4198220 con 0x7fc8a4105940 2023-07-21T19:23:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a359e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a41983e0 con 0x7fc8a41002e0 2023-07-21T19:23:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 0x7fc8a419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4197a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 0x7fc8a419d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43784/0 (socket says 172.21.15.5:43784) 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a1d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 0x7fc8a419df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a259c700 1 -- 172.21.15.5:0/3912293944 learned_addr learned my addr 172.21.15.5:0/3912293944 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 -- 172.21.15.5:0/3912293944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 msgr2=0x7fc8a419df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 0x7fc8a419df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 -- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 msgr2=0x7fc8a419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 0x7fc8a419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a2d9d700 1 -- 172.21.15.5:0/3912293944 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c009d00 con 0x7fc8a4105940 2023-07-21T19:23:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.994+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 0x7fc8a419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:23:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4197a40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009fd0 tx=0x7fc89c007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:58.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8937fe700 1 -- 172.21.15.5:0/3912293944 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c00e040 con 0x7fc8a4105940 2023-07-21T19:23:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8937fe700 1 -- 172.21.15.5:0/3912293944 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c00bdc0 con 0x7fc8a4105940 2023-07-21T19:23:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a4198660 con 0x7fc8a4105940 2023-07-21T19:23:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8937fe700 1 -- 172.21.15.5:0/3912293944 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c005040 con 0x7fc8a4105940 2023-07-21T19:23:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.995+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a41988b0 con 0x7fc8a4105940 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:58.999+0000 7fc8937fe700 1 -- 172.21.15.5:0/3912293944 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc89c01d480 con 0x7fc8a4105940 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.001+0000 7fc8bc252b80 1 Processor -- start 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.001+0000 7fc8937fe700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 0x7fc88c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.001+0000 7fc8bc252b80 1 -- start start 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.001+0000 7fc8937fe700 1 -- 172.21.15.5:0/3912293944 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc89c0cdd60 con 0x7fc8a4105940 2023-07-21T19:23:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 0x5593d27011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d2701700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2606c50 con 0x5593d27078c0 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d26599e0 con 0x5593d255d6b0 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2652e70 con 0x5593d2707470 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 0x7fc88c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36518/0 (socket says 172.21.15.5:36518) 2023-07-21T19:23:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 -- 172.21.15.5:0/907362485 learned_addr learned my addr 172.21.15.5:0/907362485 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d2701700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 0x5593d27011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 -- 172.21.15.5:0/907362485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 msgr2=0x5593d27011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 0x5593d27011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 -- 172.21.15.5:0/907362485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 msgr2=0x5593d2701700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d2701700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a259c700 1 -- 172.21.15.5:0/907362485 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d2480d80 con 0x5593d255d6b0 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.002+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 0x5593d27011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.003+0000 7fc8a259c700 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc894009d80 tx=0x7fc894011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5fed74563d646ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.003+0000 7fc890ff9700 1 -- 172.21.15.5:0/907362485 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894004230 con 0x5593d255d6b0 2023-07-21T19:23:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.003+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 0x7fc88c0a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40fec80 tx=0x7fc898009cb0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.003+0000 7fc890ff9700 1 -- 172.21.15.5:0/907362485 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc894004390 con 0x5593d255d6b0 2023-07-21T19:23:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.003+0000 7fc890ff9700 1 -- 172.21.15.5:0/907362485 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894019790 con 0x5593d255d6b0 2023-07-21T19:23:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 -- 172.21.15.5:0/907362485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 msgr2=0x5593d2706f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc894009d80 tx=0x7fc894011720 comp rx=0 tx=0).stop 2023-07-21T19:23:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 -- 172.21.15.5:0/907362485 shutdown_connections 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d2701700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d255d6b0 0x5593d2706f30 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/907362485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d2707470 0x5593d27011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 -- 172.21.15.5:0/907362485 >> 172.21.15.5:0/907362485 conn(0x5593d26f6d40 msgr2=0x5593d26fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 -- 172.21.15.5:0/907362485 shutdown_connections 2023-07-21T19:23:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.004+0000 7fc8bc252b80 1 -- 172.21.15.5:0/907362485 wait complete. 2023-07-21T19:23:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.005+0000 7fc8bc252b80 1 -- start start 2023-07-21T19:23:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.005+0000 7fc8bc252b80 1 Processor -- start 2023-07-21T19:23:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d2710d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d25f2a60 con 0x5593d27078c0 2023-07-21T19:23:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d25f2760 con 0x5593d2707470 2023-07-21T19:23:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d24d68c0 con 0x5593d255d6b0 2023-07-21T19:23:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8bc252b80 10 client.0 _ll_register_callbacks cb 0x5593d25f0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.006+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58430/0 (socket says 172.21.15.5:58430) 2023-07-21T19:23:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 -- 172.21.15.5:0/579944679 learned_addr learned my addr 172.21.15.5:0/579944679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d2710d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 -- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 msgr2=0x5593d2710d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d2710d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 -- 172.21.15.5:0/579944679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 msgr2=0x5593d270aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.007+0000 7fc8a259c700 1 -- 172.21.15.5:0/579944679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8940099b0 con 0x5593d255d6b0 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.008+0000 7fc8a259c700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc894005950 tx=0x7fc894003730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d688b355774c03c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.008+0000 7fc889ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.008+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894005040 con 0x5593d255d6b0 2023-07-21T19:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc8940051a0 con 0x5593d255d6b0 2023-07-21T19:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5593d25ac490 con 0x5593d255d6b0 2023-07-21T19:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc894005360 con 0x5593d255d6b0 2023-07-21T19:23:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc8bc252b80 10 client.28267 fetch_fsmap learned FSMap version 51 2023-07-21T19:23:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5593d2469c00 con 0x5593d255d6b0 2023-07-21T19:23:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.009+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5593d25ac490 con 0x5593d255d6b0 2023-07-21T19:23:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc894021040 con 0x5593d255d6b0 2023-07-21T19:23:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 10 client.28267 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 10 client.28267 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 20 client.28267 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d272bc50 0x5593d272e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 4 client.28267 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5593d264be50 con 0x5593d272bc50 2023-07-21T19:23:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.010+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d272bc50 0x5593d272e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.011+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d272bc50 0x5593d272e060 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=371361f2d6ba6509 server_cookie=3f3a8ec63dfd9d09 in_seq=0 out_seq=0 2023-07-21T19:23:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.011+0000 7fc889ffb700 10 client.28267 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.012+0000 7fc889ffb700 1 -- 172.21.15.5:0/579944679 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5593d264be50 con 0x5593d272bc50 2023-07-21T19:23:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.012+0000 7fc889ffb700 10 client.28267 handle_command_reply: tid=0 2023-07-21T19:23:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.012+0000 7fc8bc252b80 1 client.28267 shutdown 2023-07-21T19:23:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 msgr2=0x5593d2710810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc894005950 tx=0x7fc894003730 comp rx=0 tx=0).stop 2023-07-21T19:23:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 shutdown_connections 2023-07-21T19:23:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d272bc50 0x5593d272e060 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d2710810 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/579944679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d2710d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 >> 172.21.15.5:0/579944679 conn(0x5593d26f6d40 msgr2=0x5593d26fb640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc889ffb700 0 client.28267 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 shutdown_connections 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 1 -- 172.21.15.5:0/579944679 wait complete. 2023-07-21T19:23:59.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.013+0000 7fc8bc252b80 20 client.28267 trim_cache size 0 max 16384 2023-07-21T19:23:59.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.033+0000 7fc8bc252b80 1 Processor -- start 2023-07-21T19:23:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.033+0000 7fc8bc252b80 1 -- start start 2023-07-21T19:23:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.033+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d27062c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d27011a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d26f87c0 con 0x5593d27078c0 2023-07-21T19:23:59.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d27057a0 con 0x5593d2707470 2023-07-21T19:23:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d2728620 con 0x5593d255d6b0 2023-07-21T19:23:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d27062c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58436/0 (socket says 172.21.15.5:58436) 2023-07-21T19:23:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 -- 172.21.15.5:0/495300398 learned_addr learned my addr 172.21.15.5:0/495300398 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d27062c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36540/0 (socket says 172.21.15.5:36540) 2023-07-21T19:23:59.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d27011a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 -- 172.21.15.5:0/495300398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 msgr2=0x5593d27062c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d27062c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 -- 172.21.15.5:0/495300398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 msgr2=0x5593d27011a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d27011a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a259c700 1 -- 172.21.15.5:0/495300398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5593d26fda40 con 0x5593d255d6b0 2023-07-21T19:23:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.034+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d27011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:23:59.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.035+0000 7fc8a259c700 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc894018680 tx=0x7fc894011720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=12885c1ca28d8a66 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.035+0000 7fc8917fa700 1 -- 172.21.15.5:0/495300398 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894023780 con 0x5593d255d6b0 2023-07-21T19:23:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.035+0000 7fc8917fa700 1 -- 172.21.15.5:0/495300398 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc894023e70 con 0x5593d255d6b0 2023-07-21T19:23:59.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.035+0000 7fc8917fa700 1 -- 172.21.15.5:0/495300398 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc894005630 con 0x5593d255d6b0 2023-07-21T19:23:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- 172.21.15.5:0/495300398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 msgr2=0x5593d270ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc894018680 tx=0x7fc894011720 comp rx=0 tx=0).stop 2023-07-21T19:23:59.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- 172.21.15.5:0/495300398 shutdown_connections 2023-07-21T19:23:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d27011a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270ac00 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/495300398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d27062c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- 172.21.15.5:0/495300398 >> 172.21.15.5:0/495300398 conn(0x5593d26f6d40 msgr2=0x5593d2705bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:59.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- 172.21.15.5:0/495300398 shutdown_connections 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- 172.21.15.5:0/495300398 wait complete. 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 -- start start 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.036+0000 7fc8bc252b80 1 Processor -- start 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270e170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d277eaf0 con 0x5593d27078c0 2023-07-21T19:23:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d277ec30 con 0x5593d2707470 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5593d277ed70 con 0x5593d255d6b0 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8bc252b80 10 client.0 _ll_register_callbacks cb 0x7fc894009bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270e170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a1d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36544/0 (socket says 172.21.15.5:36544) 2023-07-21T19:23:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a259c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270e170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58450/0 (socket says 172.21.15.5:58450) 2023-07-21T19:23:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a1d9b700 1 -- 172.21.15.5:0/2273139079 learned_addr learned my addr 172.21.15.5:0/2273139079 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:23:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.037+0000 7fc8a2d9d700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 -- 172.21.15.5:0/2273139079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 msgr2=0x5593d270e170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270e170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 -- 172.21.15.5:0/2273139079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 msgr2=0x5593d270ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 -- 172.21.15.5:0/2273139079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5593d2469c00 con 0x5593d2707470 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8a1d9b700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5593d270c010 tx=0x7fc898009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e4d29e15ec0465 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc889ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898029040 con 0x5593d2707470 2023-07-21T19:23:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(108..108 src has 1..108) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc89800f990 con 0x5593d2707470 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.038+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc894004390 con 0x5593d2707470 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=51) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc89800fb50 con 0x5593d2707470 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 10 client.28352 fetch_fsmap learned FSMap version 51 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=51,monmap=4+}) v3 -- 0x5593d24a71f0 con 0x5593d2707470 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89803b020 con 0x5593d2707470 2023-07-21T19:23:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 51) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc89802e480 con 0x5593d2707470 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 10 client.28352 fetch_fsmap finished waiting for FSMap version 51 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 10 client.28352 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 20 client.28352 populate_metadata read hostname 'smithi005' 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d2780570 0x5593d2782980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 4 client.28352 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:23:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.039+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5593d260dff0 con 0x5593d2780570 2023-07-21T19:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.040+0000 7fc8a259c700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d2780570 0x5593d2782980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.040+0000 7fc8a259c700 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d2780570 0x5593d2782980 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=31b57ade3c56161b server_cookie=80f437473ee39f57 in_seq=0 out_seq=0 2023-07-21T19:23:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.040+0000 7fc889ffb700 10 client.28352 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc889ffb700 1 -- 172.21.15.5:0/2273139079 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5593d260dff0 con 0x5593d2780570 2023-07-21T19:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc889ffb700 10 client.28352 handle_command_reply: tid=0 2023-07-21T19:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 client.28352 shutdown 2023-07-21T19:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 msgr2=0x5593d270e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x5593d270c010 tx=0x7fc898009260 comp rx=0 tx=0).stop 2023-07-21T19:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 shutdown_connections 2023-07-21T19:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5593d2780570 0x5593d2782980 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5593d27078c0 0x5593d270ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5593d255d6b0 0x5593d270e170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 --2- 172.21.15.5:0/2273139079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5593d2707470 0x5593d270e6b0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc889ffb700 0 client.28352 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:23:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 >> 172.21.15.5:0/2273139079 conn(0x5593d26f6d40 msgr2=0x5593d2707c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 shutdown_connections 2023-07-21T19:23:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 1 -- 172.21.15.5:0/2273139079 wait complete. 2023-07-21T19:23:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.048+0000 7fc8bc252b80 20 client.28352 trim_cache size 0 max 16384 2023-07-21T19:23:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:23:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:23:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:23:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 msgr2=0x7fc88c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 0x7fc88c0a3770 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fc8a40fec80 tx=0x7fc898009cb0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 msgr2=0x7fc8a4197a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:23:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4197a40 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fc89c009fd0 tx=0x7fc89c007930 comp rx=0 tx=0).stop 2023-07-21T19:23:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 shutdown_connections 2023-07-21T19:23:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a4105940 0x7fc8a4197a40 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc88c0a12c0 0x7fc88c0a3770 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a40ff930 0x7fc8a419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 --2- 172.21.15.5:0/3912293944 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a41002e0 0x7fc8a419df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:23:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 >> 172.21.15.5:0/3912293944 conn(0x7fc8a40fb450 msgr2=0x7fc8a40fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 shutdown_connections 2023-07-21T19:23:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:23:59.049+0000 7fc8a359e700 1 -- 172.21.15.5:0/3912293944 wait complete. 2023-07-21T19:23:59.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:23:59 smithi101 ceph-mon[128136]: pgmap v514: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:23:59.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:23:59 smithi005 ceph-mon[126460]: pgmap v514: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:23:59.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:23:59 smithi156 ceph-mon[128316]: pgmap v514: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 35 MiB/s wr, 26 op/s 2023-07-21T19:24:01.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:01 smithi101 ceph-mon[128136]: pgmap v515: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:01.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:01 smithi005 ceph-mon[126460]: pgmap v515: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:01.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:01 smithi156 ceph-mon[128316]: pgmap v515: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:03 smithi101 ceph-mon[128136]: pgmap v516: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:24:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:03 smithi005 ceph-mon[126460]: pgmap v516: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:24:03.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:03 smithi156 ceph-mon[128316]: pgmap v516: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 34 MiB/s wr, 37 op/s 2023-07-21T19:24:05.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:05 smithi101 ceph-mon[128136]: pgmap v517: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 33 op/s 2023-07-21T19:24:05.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:05 smithi005 ceph-mon[126460]: pgmap v517: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 33 op/s 2023-07-21T19:24:05.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:05 smithi156 ceph-mon[128316]: pgmap v517: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 25 MiB/s wr, 33 op/s 2023-07-21T19:24:07.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:07 smithi101 ceph-mon[128136]: pgmap v518: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:24:07.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:07 smithi101 ceph-mon[128136]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:24:07.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:07 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:07.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:07 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:07.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:07 smithi005 ceph-mon[126460]: pgmap v518: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:24:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:07 smithi005 ceph-mon[126460]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:24:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:07 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:07.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:07 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:07.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:07 smithi156 ceph-mon[128316]: pgmap v518: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 24 MiB/s wr, 33 op/s 2023-07-21T19:24:07.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:07 smithi156 ceph-mon[128316]: osdmap e109: 12 total, 12 up, 12 in 2023-07-21T19:24:07.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:07 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:07.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:07 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:09.823 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:09 smithi101 ceph-mon[128136]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:09.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:09 smithi005 ceph-mon[126460]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:09.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:09 smithi156 ceph-mon[128316]: pgmap v520: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:11.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:11 smithi005 ceph-mon[126460]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:11.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:11 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:24:11.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:11 smithi156 ceph-mon[128316]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:11.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:11 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:24:12.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:11 smithi101 ceph-mon[128136]: pgmap v521: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 84 KiB/s rd, 28 MiB/s wr, 49 op/s 2023-07-21T19:24:12.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:11 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:24:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 -- 172.21.15.5:0/2459199083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 msgr2=0x7f9910106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 --2- 172.21.15.5:0/2459199083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910106930 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f99000099c0 tx=0x7f9900009cd0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 -- 172.21.15.5:0/2459199083 shutdown_connections 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 --2- 172.21.15.5:0/2459199083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 0x7f991010ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 --2- 172.21.15.5:0/2459199083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 0x7f99101023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 --2- 172.21.15.5:0/2459199083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910106930 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.221+0000 7f99151ff700 1 -- 172.21.15.5:0/2459199083 >> 172.21.15.5:0/2459199083 conn(0x7f99100fd840 msgr2=0x7f99100ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.222+0000 7f99151ff700 1 -- 172.21.15.5:0/2459199083 shutdown_connections 2023-07-21T19:24:12.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.222+0000 7f99151ff700 1 -- 172.21.15.5:0/2459199083 wait complete. 2023-07-21T19:24:12.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.223+0000 7f99151ff700 1 Processor -- start 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.223+0000 7f99151ff700 1 -- start start 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 0x7f9910197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 0x7f991019c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991019cfb0 con 0x7f9910102970 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991019d120 con 0x7f9910101fd0 2023-07-21T19:24:12.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f99151ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f991019d2e0 con 0x7f9910107010 2023-07-21T19:24:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60186/0 (socket says 172.21.15.5:60186) 2023-07-21T19:24:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 0x7f9910197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f990e59c700 1 -- 172.21.15.5:0/4119546506 learned_addr learned my addr 172.21.15.5:0/4119546506 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:12.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.224+0000 7f990f59e700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 0x7f991019c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 -- 172.21.15.5:0/4119546506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 msgr2=0x7f991019c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 0x7f991019c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 -- 172.21.15.5:0/4119546506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 msgr2=0x7f9910197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 0x7f9910197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 -- 172.21.15.5:0/4119546506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99000095f0 con 0x7f9910102970 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f990e59c700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f99000098e0 tx=0x7f990000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f98f7fff700 1 -- 172.21.15.5:0/4119546506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990001d070 con 0x7f9910102970 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f991019d560 con 0x7f9910102970 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f98f7fff700 1 -- 172.21.15.5:0/4119546506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9900005050 con 0x7f9910102970 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.225+0000 7f98f7fff700 1 -- 172.21.15.5:0/4119546506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900004970 con 0x7f9910102970 2023-07-21T19:24:12.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.226+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99101a4e00 con 0x7f9910102970 2023-07-21T19:24:12.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.230+0000 7f98f7fff700 1 -- 172.21.15.5:0/4119546506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9900005430 con 0x7f9910102970 2023-07-21T19:24:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.230+0000 7f9928bc3b80 1 Processor -- start 2023-07-21T19:24:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 -- start start 2023-07-21T19:24:12.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ec4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ebf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ebf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77dc4c50 con 0x55de77ebf580 2023-07-21T19:24:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77e179e0 con 0x55de77d1b6b0 2023-07-21T19:24:12.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.231+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77e10e70 con 0x55de77ec52d0 2023-07-21T19:24:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ebf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ec4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ec4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46002/0 (socket says 172.21.15.5:46002) 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990f59e700 1 -- 172.21.15.5:0/2677631258 learned_addr learned my addr 172.21.15.5:0/2677631258 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ebf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f98f7fff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 0x7f98f80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 -- 172.21.15.5:0/2677631258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 msgr2=0x55de77ec4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ec4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 -- 172.21.15.5:0/2677631258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 msgr2=0x55de77ebf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ebf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 -- 172.21.15.5:0/2677631258 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de77c3ed80 con 0x55de77ec52d0 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990e59c700 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ebf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f98f7fff700 1 -- 172.21.15.5:0/4119546506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f99000cda10 con 0x7f9910102970 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990f59e700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 0x7f98f80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.232+0000 7f990ed9d700 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ebf040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9904009cb0 tx=0x7f9904011440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31d70e4d747c8153 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.233+0000 7f98f57fa700 1 -- 172.21.15.5:0/2677631258 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904011d20 con 0x55de77ec52d0 2023-07-21T19:24:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.233+0000 7f98f57fa700 1 -- 172.21.15.5:0/2677631258 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9904011e80 con 0x55de77ec52d0 2023-07-21T19:24:12.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.233+0000 7f98f57fa700 1 -- 172.21.15.5:0/2677631258 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904005280 con 0x55de77ec52d0 2023-07-21T19:24:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- 172.21.15.5:0/2677631258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 msgr2=0x55de77ebf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f990f59e700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 0x7f98f80a3770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99080075c0 tx=0x7f9908009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ebf040 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f9904009cb0 tx=0x7f9904011440 comp rx=0 tx=0).stop 2023-07-21T19:24:12.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- 172.21.15.5:0/2677631258 shutdown_connections 2023-07-21T19:24:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ebf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ec4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/2677631258 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ebf040 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- 172.21.15.5:0/2677631258 >> 172.21.15.5:0/2677631258 conn(0x55de77eb4be0 msgr2=0x55de77eb9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- 172.21.15.5:0/2677631258 shutdown_connections 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- 172.21.15.5:0/2677631258 wait complete. 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.234+0000 7f9928bc3b80 1 -- start start 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.235+0000 7f9928bc3b80 1 Processor -- start 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ecb5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ec8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77ec8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77db0a60 con 0x55de77ebf580 2023-07-21T19:24:12.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77db0760 con 0x55de77ec52d0 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77c948c0 con 0x55de77d1b6b0 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f9928bc3b80 10 client.0 _ll_register_callbacks cb 0x55de77dae700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f990f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77ec8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ec8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f990f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77ec8fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46008/0 (socket says 172.21.15.5:46008) 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f990f59e700 1 -- 172.21.15.5:0/185762299 learned_addr learned my addr 172.21.15.5:0/185762299 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:12.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.236+0000 7f990ed9d700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ecb5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 -- 172.21.15.5:0/185762299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 msgr2=0x55de77ecb5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ecb5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 -- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 msgr2=0x55de77ec8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77ec8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 -- 172.21.15.5:0/185762299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99101a4e00 con 0x55de77ebf580 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f990e59c700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ec8860 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f99000cb0e0 tx=0x7f99000cb640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d65a290419105384 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f98f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:12.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f990009a1e0 con 0x55de77ebf580 2023-07-21T19:24:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f990009aaf0 con 0x55de77ebf580 2023-07-21T19:24:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de77e09e50 con 0x55de77ebf580 2023-07-21T19:24:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f990009a8c0 con 0x55de77ebf580 2023-07-21T19:24:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f9928bc3b80 10 client.18705 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:12.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.237+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x7f99040098e0 con 0x55de77ebf580 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de77e09e50 con 0x55de77ebf580 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f99000a2a20 con 0x55de77ebf580 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 10 client.18705 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 10 client.18705 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 20 client.18705 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77ed8840 0x55de77edac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 4 client.18705 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.238+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de77dcbff0 con 0x55de77ed8840 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.239+0000 7f990ed9d700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77ed8840 0x55de77edac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.239+0000 7f98f2ff5700 10 client.18705 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.239+0000 7f990ed9d700 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77ed8840 0x55de77edac50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b048541b77f52476 server_cookie=fbdaa0e12d89ba29 in_seq=0 out_seq=0 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f98f2ff5700 1 -- 172.21.15.5:0/185762299 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de77dcbff0 con 0x55de77ed8840 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f98f2ff5700 10 client.18705 handle_command_reply: tid=0 2023-07-21T19:24:12.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 client.18705 shutdown 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 msgr2=0x55de77ec8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ec8860 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f99000cb0e0 tx=0x7f99000cb640 comp rx=0 tx=0).stop 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 shutdown_connections 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77ec8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ecb5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77ed8840 0x55de77edac50 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/185762299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77ebf580 0x55de77ec8860 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 >> 172.21.15.5:0/185762299 conn(0x55de77eb4be0 msgr2=0x55de77eb8c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f98f2ff5700 0 client.18705 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.240+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 shutdown_connections 2023-07-21T19:24:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.241+0000 7f9928bc3b80 1 -- 172.21.15.5:0/185762299 wait complete. 2023-07-21T19:24:12.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.241+0000 7f9928bc3b80 20 client.18705 trim_cache size 0 max 16384 2023-07-21T19:24:12.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 Processor -- start 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 -- start start 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ec4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77eb9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77ec6e50 con 0x55de77eb4be0 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77ebb970 con 0x55de77ec52d0 2023-07-21T19:24:12.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77eb65f0 con 0x55de77d1b6b0 2023-07-21T19:24:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60230/0 (socket says 172.21.15.5:60230) 2023-07-21T19:24:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.261+0000 7f990ed9d700 1 -- 172.21.15.5:0/4240996014 learned_addr learned my addr 172.21.15.5:0/4240996014 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:12.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990e59c700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ec4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990f59e700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77eb9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 -- 172.21.15.5:0/4240996014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 msgr2=0x55de77ec4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ec4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 -- 172.21.15.5:0/4240996014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 msgr2=0x55de77eb9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77eb9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 -- 172.21.15.5:0/4240996014 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de77edf840 con 0x55de77eb4be0 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990f59e700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77eb9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:12.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f990ed9d700 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9904019000 tx=0x7f9904009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5484ec45735539e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.262+0000 7f98f5ffb700 1 -- 172.21.15.5:0/4240996014 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904005ce0 con 0x55de77eb4be0 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f98f5ffb700 1 -- 172.21.15.5:0/4240996014 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9904004870 con 0x55de77eb4be0 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f98f5ffb700 1 -- 172.21.15.5:0/4240996014 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904004500 con 0x55de77eb4be0 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 -- 172.21.15.5:0/4240996014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 msgr2=0x55de77ec6910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f9904019000 tx=0x7f9904009c30 comp rx=0 tx=0).stop 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 -- 172.21.15.5:0/4240996014 shutdown_connections 2023-07-21T19:24:12.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77ec52d0 0x55de77eb9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ec6910 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/4240996014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77d1b6b0 0x55de77ec4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 -- 172.21.15.5:0/4240996014 >> 172.21.15.5:0/4240996014 conn(0x55de77ed50e0 msgr2=0x55de77ec3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.263+0000 7f9928bc3b80 1 -- 172.21.15.5:0/4240996014 shutdown_connections 2023-07-21T19:24:12.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.264+0000 7f9928bc3b80 1 -- 172.21.15.5:0/4240996014 wait complete. 2023-07-21T19:24:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.264+0000 7f9928bc3b80 1 -- start start 2023-07-21T19:24:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.264+0000 7f9928bc3b80 1 Processor -- start 2023-07-21T19:24:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ecfa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ece270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77ece7b0 con 0x55de77eb4be0 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77ece920 con 0x55de77d1b6b0 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de77eceae0 con 0x55de77ec52d0 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f9928bc3b80 10 client.0 _ll_register_callbacks cb 0x55de77dae700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f990ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ecfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f990e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60236/0 (socket says 172.21.15.5:60236) 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f990e59c700 1 -- 172.21.15.5:0/637983887 learned_addr learned my addr 172.21.15.5:0/637983887 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.265+0000 7f990f59e700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ece270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 -- 172.21.15.5:0/637983887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 msgr2=0x55de77ece270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ece270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 -- 172.21.15.5:0/637983887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 msgr2=0x55de77ecdd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 -- 172.21.15.5:0/637983887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990401d040 con 0x55de77d1b6b0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990e59c700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f990ed9d700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ecfa40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9904009cb0 tx=0x7f9904005dd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6286fd960ecabcab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f98f2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9904013e80 con 0x55de77d1b6b0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de77d6a490 con 0x55de77d1b6b0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.266+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(109..109 src has 1..109) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f9904011020 con 0x55de77d1b6b0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9904011220 con 0x55de77d1b6b0 2023-07-21T19:24:12.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 10 client.28370 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55de77c27c00 con 0x55de77d1b6b0 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de77d6a490 con 0x55de77d1b6b0 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f9904034070 con 0x55de77d1b6b0 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 10 client.28370 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 10 client.28370 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 20 client.28370 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:12.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77f2d790 0x55de77f2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 4 client.28370 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:24:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.267+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de77c5ff10 con 0x55de77f2d790 2023-07-21T19:24:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.268+0000 7f990e59c700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77f2d790 0x55de77f2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.268+0000 7f990e59c700 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77f2d790 0x55de77f2fba0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fe01011df004abd server_cookie=9c6fce88ef043a9c in_seq=0 out_seq=0 2023-07-21T19:24:12.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.268+0000 7f98f2ff5700 10 client.28370 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.275+0000 7f98f2ff5700 1 -- 172.21.15.5:0/637983887 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de77c5ff10 con 0x55de77f2d790 2023-07-21T19:24:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.275+0000 7f98f2ff5700 10 client.28370 handle_command_reply: tid=0 2023-07-21T19:24:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.275+0000 7f9928bc3b80 1 client.28370 shutdown 2023-07-21T19:24:12.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 msgr2=0x55de77ecfa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ecfa40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f9904009cb0 tx=0x7f9904005dd0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 shutdown_connections 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de77ec52d0 0x55de77ece270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de77d1b6b0 0x55de77ecfa40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de77f2d790 0x55de77f2fba0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 --2- 172.21.15.5:0/637983887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de77eb4be0 0x55de77ecdd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f98f2ff5700 0 client.28370 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 >> 172.21.15.5:0/637983887 conn(0x55de77ed50e0 msgr2=0x55de77ecaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 shutdown_connections 2023-07-21T19:24:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 1 -- 172.21.15.5:0/637983887 wait complete. 2023-07-21T19:24:12.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.276+0000 7f9928bc3b80 20 client.28370 trim_cache size 0 max 16384 2023-07-21T19:24:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:24:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:24:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:24:12.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:24:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 msgr2=0x7f98f80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 0x7f98f80a3770 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f99080075c0 tx=0x7f9908009040 comp rx=0 tx=0).stop 2023-07-21T19:24:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 msgr2=0x7f9910198300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:12.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f99000098e0 tx=0x7f990000f690 comp rx=0 tx=0).stop 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 shutdown_connections 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9910107010 0x7f991019c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9910101fd0 0x7f9910197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f98f80a12c0 0x7f98f80a3770 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 --2- 172.21.15.5:0/4119546506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9910102970 0x7f9910198300 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 >> 172.21.15.5:0/4119546506 conn(0x7f99100fd840 msgr2=0x7f99100fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:12.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 shutdown_connections 2023-07-21T19:24:12.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:12.277+0000 7f99151ff700 1 -- 172.21.15.5:0/4119546506 wait complete. 2023-07-21T19:24:12.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:12 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:24:12.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:12 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:24:12.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:12 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:24:12.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:12 smithi005 ceph-mon[126460]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:24:12.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:12 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:24:12.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:12 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:24:12.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:12 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:24:12.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:12 smithi156 ceph-mon[128316]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:24:13.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:12 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:24:13.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:12 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:24:13.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:12 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:24:13.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:12 smithi101 ceph-mon[128136]: osdmap e110: 12 total, 12 up, 12 in 2023-07-21T19:24:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:13 smithi005 ceph-mon[126460]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:24:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:13 smithi156 ceph-mon[128316]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:24:14.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:13 smithi101 ceph-mon[128136]: pgmap v522: 97 pgs: 97 active+clean; 13 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 27 MiB/s wr, 33 op/s 2023-07-21T19:24:15.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:15 smithi005 ceph-mon[126460]: pgmap v524: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:24:15.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:15 smithi156 ceph-mon[128316]: pgmap v524: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:24:16.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:15 smithi101 ceph-mon[128136]: pgmap v524: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 35 MiB/s wr, 40 op/s 2023-07-21T19:24:17.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:17 smithi005 ceph-mon[126460]: pgmap v525: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 33 op/s 2023-07-21T19:24:17.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:17 smithi156 ceph-mon[128316]: pgmap v525: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 33 op/s 2023-07-21T19:24:18.042 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:24:18.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:17 smithi101 ceph-mon[128136]: pgmap v525: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 76 KiB/s rd, 29 MiB/s wr, 33 op/s 2023-07-21T19:24:18.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 -- 172.21.15.5:0/273420777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c100480 msgr2=0x7f871c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 --2- 172.21.15.5:0/273420777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c100480 0x7f871c10c910 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f871400b0a0 tx=0x7f871400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 -- 172.21.15.5:0/273420777 shutdown_connections 2023-07-21T19:24:18.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 --2- 172.21.15.5:0/273420777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c100480 0x7f871c10c910 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 --2- 172.21.15.5:0/273420777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c105ae0 0x7f871c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 --2- 172.21.15.5:0/273420777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 0x7f871c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 -- 172.21.15.5:0/273420777 >> 172.21.15.5:0/273420777 conn(0x7f871c0fb5f0 msgr2=0x7f871c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 -- 172.21.15.5:0/273420777 shutdown_connections 2023-07-21T19:24:18.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.284+0000 7f8720dff700 1 -- 172.21.15.5:0/273420777 wait complete. 2023-07-21T19:24:18.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.285+0000 7f8720dff700 1 Processor -- start 2023-07-21T19:24:18.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 -- start start 2023-07-21T19:24:18.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 0x7f871c19b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c100480 0x7f871c19bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c196070 con 0x7f871c105ae0 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1961e0 con 0x7f871c100480 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8720dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871c1963a0 con 0x7f871c0ffad0 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f871ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f871ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35298/0 (socket says 172.21.15.5:35298) 2023-07-21T19:24:18.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f871ad9d700 1 -- 172.21.15.5:0/1024418739 learned_addr learned my addr 172.21.15.5:0/1024418739 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.286+0000 7f8719d9b700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c100480 0x7f871c19bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871a59c700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 0x7f871c19b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 -- 172.21.15.5:0/1024418739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 msgr2=0x7f871c19b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 0x7f871c19b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 -- 172.21.15.5:0/1024418739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c100480 msgr2=0x7f871c19bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c100480 0x7f871c19bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 -- 172.21.15.5:0/1024418739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8714009d00 con 0x7f871c105ae0 2023-07-21T19:24:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f871ad9d700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f871400b070 tx=0x7f871400bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f870b7fe700 1 -- 172.21.15.5:0/1024418739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871400e040 con 0x7f871c105ae0 2023-07-21T19:24:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f870b7fe700 1 -- 172.21.15.5:0/1024418739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8714009480 con 0x7f871c105ae0 2023-07-21T19:24:18.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871c196620 con 0x7f871c105ae0 2023-07-21T19:24:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.287+0000 7f870b7fe700 1 -- 172.21.15.5:0/1024418739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8714005040 con 0x7f871c105ae0 2023-07-21T19:24:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.288+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c1a2980 con 0x7f871c105ae0 2023-07-21T19:24:18.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.291+0000 7f870b7fe700 1 -- 172.21.15.5:0/1024418739 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f871401d480 con 0x7f871c105ae0 2023-07-21T19:24:18.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.292+0000 7f873475bb80 1 Processor -- start 2023-07-21T19:24:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 -- start start 2023-07-21T19:24:18.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f870b7fe700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 0x7f87040a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f870b7fe700 1 -- 172.21.15.5:0/1024418739 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f87140cdd20 con 0x7f871c105ae0 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be32d0 0x558a55be36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f8719d9b700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 0x7f87040a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bee590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55ae9cb0 con 0x558a55beead0 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55b3ca40 con 0x558a55be3be0 2023-07-21T19:24:18.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.293+0000 7f873475bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55b35ed0 con 0x558a55be32d0 2023-07-21T19:24:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35310/0 (socket says 172.21.15.5:35310) 2023-07-21T19:24:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 -- 172.21.15.5:0/2106862508 learned_addr learned my addr 172.21.15.5:0/2106862508 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:18.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f8719d9b700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bee590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871a59c700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be32d0 0x558a55be36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 -- 172.21.15.5:0/2106862508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be32d0 msgr2=0x558a55be36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be32d0 0x558a55be36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 -- 172.21.15.5:0/2106862508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 msgr2=0x558a55bee590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bee590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 -- 172.21.15.5:0/2106862508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a55963de0 con 0x558a55beead0 2023-07-21T19:24:18.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f8719d9b700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f8719d9b700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 0x7f87040a37f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f871c1051b0 tx=0x7f8710009f90 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.294+0000 7f871ad9d700 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8714007fd0 tx=0x7f8714003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b5f20a8375b891 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.295+0000 7f8708ff9700 1 -- 172.21.15.5:0/2106862508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871409ae20 con 0x558a55beead0 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.295+0000 7f8708ff9700 1 -- 172.21.15.5:0/2106862508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871409ee50 con 0x558a55beead0 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.295+0000 7f8708ff9700 1 -- 172.21.15.5:0/2106862508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871409a310 con 0x558a55beead0 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.295+0000 7f873475bb80 1 -- 172.21.15.5:0/2106862508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 msgr2=0x558a55bf0eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.295+0000 7f873475bb80 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f8714007fd0 tx=0x7f8714003f10 comp rx=0 tx=0).stop 2023-07-21T19:24:18.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 -- 172.21.15.5:0/2106862508 shutdown_connections 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55beead0 0x558a55bf0eb0 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be32d0 0x558a55be36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 --2- 172.21.15.5:0/2106862508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bee590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 -- 172.21.15.5:0/2106862508 >> 172.21.15.5:0/2106862508 conn(0x558a55bd9da0 msgr2=0x558a55bdf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 -- 172.21.15.5:0/2106862508 shutdown_connections 2023-07-21T19:24:18.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 -- 172.21.15.5:0/2106862508 wait complete. 2023-07-21T19:24:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.296+0000 7f873475bb80 1 -- start start 2023-07-21T19:24:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 Processor -- start 2023-07-21T19:24:18.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55beb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55beead0 0x558a55bebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55ad5ac0 con 0x558a55be32d0 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55ad57c0 con 0x558a55be3be0 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.297+0000 7f873475bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a559b9920 con 0x558a55beead0 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f873475bb80 10 client.0 _ll_register_callbacks cb 0x558a55ad3760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35318/0 (socket says 172.21.15.5:35318) 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 -- 172.21.15.5:0/3531201506 learned_addr learned my addr 172.21.15.5:0/3531201506 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871ad9d700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55beead0 0x558a55bebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f8719d9b700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55beb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 -- 172.21.15.5:0/3531201506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55beead0 msgr2=0x558a55bebb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55beead0 0x558a55bebb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 -- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 msgr2=0x558a55beb620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55beb620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.298+0000 7f871a59c700 1 -- 172.21.15.5:0/3531201506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871c1a2980 con 0x558a55be32d0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f871a59c700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f870c009c80 tx=0x7f870c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=463a2fc31343e600 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f8701ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c016070 con 0x558a55be32d0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f870c003e20 con 0x558a55be32d0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558a55a8f4f0 con 0x558a55be32d0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f870c011440 con 0x558a55be32d0 2023-07-21T19:24:18.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f873475bb80 10 client.18723 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.299+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x558a5594cc60 con 0x558a55be32d0 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f870c027080 con 0x558a55be32d0 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f870c021030 con 0x558a55be32d0 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 10 client.18723 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 10 client.18723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 20 client.18723 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55c02730 0x558a55c04b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 4 client.18723 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558a55b2eeb0 con 0x558a55c02730 2023-07-21T19:24:18.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.300+0000 7f8719d9b700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55c02730 0x558a55c04b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.301+0000 7f8719d9b700 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55c02730 0x558a55c04b40 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43e0c0d521a608cb server_cookie=8630511703788ad6 in_seq=0 out_seq=0 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.301+0000 7f8701ffb700 10 client.18723 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f8701ffb700 1 -- 172.21.15.5:0/3531201506 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558a55b2eeb0 con 0x558a55c02730 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f8701ffb700 10 client.18723 handle_command_reply: tid=0 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 client.18723 shutdown 2023-07-21T19:24:18.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 msgr2=0x558a55bee3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f870c009c80 tx=0x7f870c009f90 comp rx=0 tx=0).stop 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 shutdown_connections 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55beead0 0x558a55bebb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55c02730 0x558a55c04b40 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bee3a0 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 --2- 172.21.15.5:0/3531201506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55beb620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 >> 172.21.15.5:0/3531201506 conn(0x558a55bd9da0 msgr2=0x558a55bdf290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.302+0000 7f8701ffb700 0 client.18723 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.303+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 shutdown_connections 2023-07-21T19:24:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.303+0000 7f873475bb80 1 -- 172.21.15.5:0/3531201506 wait complete. 2023-07-21T19:24:18.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.303+0000 7f873475bb80 20 client.18723 trim_cache size 0 max 16384 2023-07-21T19:24:18.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.325+0000 7f873475bb80 1 Processor -- start 2023-07-21T19:24:18.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.325+0000 7f873475bb80 1 -- start start 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 0x558a55befeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 0x558a55bf03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55bffc60 con 0x558a55bd9da0 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55becc00 con 0x558a55be32d0 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f873475bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55bd9970 con 0x558a55be3be0 2023-07-21T19:24:18.318 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f871a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f871a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35330/0 (socket says 172.21.15.5:35330) 2023-07-21T19:24:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f871a59c700 1 -- 172.21.15.5:0/4081739510 learned_addr learned my addr 172.21.15.5:0/4081739510 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f8719d9b700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 0x558a55befeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.319 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.326+0000 7f871ad9d700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 0x558a55bf03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 -- 172.21.15.5:0/4081739510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 msgr2=0x558a55bf03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 0x558a55bf03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.320 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 -- 172.21.15.5:0/4081739510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 msgr2=0x558a55bedf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 -- 172.21.15.5:0/4081739510 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558a5598a250 con 0x558a55be32d0 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f871ad9d700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 0x558a55bf03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f871a59c700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f8719d9b700 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 0x558a55befeb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x558a55c05420 tx=0x7f871001f000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=282e5b47561c6409 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f87097fa700 1 -- 172.21.15.5:0/4081739510 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710018030 con 0x558a55be32d0 2023-07-21T19:24:18.321 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f87097fa700 1 -- 172.21.15.5:0/4081739510 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8710004cc0 con 0x558a55be32d0 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.327+0000 7f87097fa700 1 -- 172.21.15.5:0/4081739510 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8710025070 con 0x558a55be32d0 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 -- 172.21.15.5:0/4081739510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 msgr2=0x558a55befeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 0x558a55befeb0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x558a55c05420 tx=0x7f871001f000 comp rx=0 tx=0).stop 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 -- 172.21.15.5:0/4081739510 shutdown_connections 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55be3be0 0x558a55bf03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55bd9da0 0x558a55bedf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 --2- 172.21.15.5:0/4081739510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be32d0 0x558a55befeb0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.322 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 -- 172.21.15.5:0/4081739510 >> 172.21.15.5:0/4081739510 conn(0x558a55bf1020 msgr2=0x558a55bfe8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 -- 172.21.15.5:0/4081739510 shutdown_connections 2023-07-21T19:24:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.328+0000 7f873475bb80 1 -- 172.21.15.5:0/4081739510 wait complete. 2023-07-21T19:24:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 -- start start 2023-07-21T19:24:18.323 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 Processor -- start 2023-07-21T19:24:18.325 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bf8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bf8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55c56690 con 0x558a55be32d0 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55c567d0 con 0x558a55be3be0 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.329+0000 7f873475bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558a55c56990 con 0x558a55bd9da0 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f873475bb80 10 client.0 _ll_register_callbacks cb 0x7f870c0097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:18.326 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f8719d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bf8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48718/0 (socket says 172.21.15.5:48718) 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 -- 172.21.15.5:0/1079074428 learned_addr learned my addr 172.21.15.5:0/1079074428 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871ad9d700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bf8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 -- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 msgr2=0x558a55bf8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bf8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 -- 172.21.15.5:0/1079074428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 msgr2=0x558a55bf8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:18.327 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bf8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.330+0000 7f871a59c700 1 -- 172.21.15.5:0/1079074428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558a5594cc60 con 0x558a55bd9da0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.331+0000 7f871a59c700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f870c0099a0 tx=0x7f870c004050 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45b3300a57624efb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.331+0000 7f8701ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.331+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870c016040 con 0x558a55bd9da0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.331+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f870c01ba40 con 0x558a55bd9da0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.331+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f871409ee50 con 0x558a55bd9da0 2023-07-21T19:24:18.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f870c011850 con 0x558a55bd9da0 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 10 client.28306 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x558a559622b0 con 0x558a55bd9da0 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f871409ee50 con 0x558a55bd9da0 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f870c02e910 con 0x558a55bd9da0 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 10 client.28306 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 10 client.28306 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 20 client.28306 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:18.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55beead0 0x558a55c5a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 4 client.28306 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.332+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558a55af1050 con 0x558a55beead0 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f8719d9b700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55beead0 0x558a55c5a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f8701ffb700 10 client.28306 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f8719d9b700 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55beead0 0x558a55c5a0b0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e324d3a7829cb3 server_cookie=4c8d89328e21a0d in_seq=0 out_seq=0 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f8701ffb700 1 -- 172.21.15.5:0/1079074428 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558a55af1050 con 0x558a55beead0 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f8701ffb700 10 client.28306 handle_command_reply: tid=0 2023-07-21T19:24:18.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.333+0000 7f873475bb80 1 client.28306 shutdown 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 msgr2=0x558a55bf8010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f870c0099a0 tx=0x7f870c004050 comp rx=0 tx=0).stop 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 shutdown_connections 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558a55be3be0 0x558a55bf8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558a55bd9da0 0x558a55bf8010 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558a55beead0 0x558a55c5a0b0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 --2- 172.21.15.5:0/1079074428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558a55be32d0 0x558a55bf8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f8701ffb700 0 client.28306 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 >> 172.21.15.5:0/1079074428 conn(0x558a55bf1020 msgr2=0x558a55be53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 shutdown_connections 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 1 -- 172.21.15.5:0/1079074428 wait complete. 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.334+0000 7f873475bb80 20 client.28306 trim_cache size 0 max 16384 2023-07-21T19:24:18.332 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "d1814ff8-ca4a-482b-8626-e839fdef3bf8": { 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "d1814ff8-ca4a-482b-8626-e839fdef3bf8", 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:24:18.333 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:24:18.334 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:24:18.334 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:24:18.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 msgr2=0x7f87040a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 0x7f87040a37f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f871c1051b0 tx=0x7f8710009f90 comp rx=0 tx=0).stop 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 msgr2=0x7f871c195a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f871400b070 tx=0x7f871400bce0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 shutdown_connections 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f87040a1340 0x7f87040a37f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f871c105ae0 0x7f871c195a00 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f871c0ffad0 0x7f871c19b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 --2- 172.21.15.5:0/1024418739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f871c100480 0x7f871c19bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 >> 172.21.15.5:0/1024418739 conn(0x7f871c0fb5f0 msgr2=0x7f871c1035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 shutdown_connections 2023-07-21T19:24:18.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:18.335+0000 7f8720dff700 1 -- 172.21.15.5:0/1024418739 wait complete. 2023-07-21T19:24:18.348 INFO:tasks.cephfs.filesystem:scrub status for tag:d1814ff8-ca4a-482b-8626-e839fdef3bf8 - {'path': '/', 'tag': 'd1814ff8-ca4a-482b-8626-e839fdef3bf8', 'options': 'recursive,force'} 2023-07-21T19:24:19.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:19 smithi005 ceph-mon[126460]: pgmap v526: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:19.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:19 smithi156 ceph-mon[128316]: pgmap v526: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:20.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:19 smithi101 ceph-mon[128136]: pgmap v526: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:21.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:21 smithi005 ceph-mon[126460]: pgmap v527: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:21.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:24:21.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:24:21.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:21 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:24:21.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:21 smithi156 ceph-mon[128316]: pgmap v527: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:21.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:24:21.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:24:21.912 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:21 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:24:22.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:21 smithi101 ceph-mon[128136]: pgmap v527: 97 pgs: 97 active+clean; 14 GiB data, 5.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 29 MiB/s wr, 23 op/s 2023-07-21T19:24:22.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:24:22.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:24:22.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:21 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:24:23.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:23 smithi005 ceph-mon[126460]: pgmap v528: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 21 op/s 2023-07-21T19:24:23.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:23 smithi156 ceph-mon[128316]: pgmap v528: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 21 op/s 2023-07-21T19:24:24.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:23 smithi101 ceph-mon[128136]: pgmap v528: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 31 MiB/s wr, 21 op/s 2023-07-21T19:24:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2313122412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 msgr2=0x7fbcc0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2313122412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc0067fe0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00099c0 tx=0x7fbcb0009cd0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2313122412 shutdown_connections 2023-07-21T19:24:25.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2313122412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2313122412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc0067fe0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2313122412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 0x7fbcc0104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2313122412 >> 172.21.15.5:0/2313122412 conn(0x7fbcc0074760 msgr2=0x7fbcc0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2313122412 shutdown_connections 2023-07-21T19:24:25.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.444+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2313122412 wait complete. 2023-07-21T19:24:25.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.445+0000 7fbcc51ff700 1 Processor -- start 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 -- start start 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 0x7fbcc00ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc01a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01a4280 con 0x7fbcc0067c10 2023-07-21T19:24:25.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01a43c0 con 0x7fbcc0104e50 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcc51ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01a4530 con 0x7fbcc00685b0 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.446+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35364/0 (socket says 172.21.15.5:35364) 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 -- 172.21.15.5:0/2280053324 learned_addr learned my addr 172.21.15.5:0/2280053324 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc01a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 0x7fbcc00ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 -- 172.21.15.5:0/2280053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 msgr2=0x7fbcc00ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 0x7fbcc00ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 -- 172.21.15.5:0/2280053324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 msgr2=0x7fbcc01a3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc01a3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 -- 172.21.15.5:0/2280053324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb00095f0 con 0x7fbcc0067c10 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc01a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:25.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.447+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00098e0 tx=0x7fbcb0005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.448+0000 7fbca7fff700 1 -- 172.21.15.5:0/2280053324 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb001d070 con 0x7fbcc0067c10 2023-07-21T19:24:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.448+0000 7fbca7fff700 1 -- 172.21.15.5:0/2280053324 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcb0003f20 con 0x7fbcc0067c10 2023-07-21T19:24:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.448+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc01a47b0 con 0x7fbcc0067c10 2023-07-21T19:24:25.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.448+0000 7fbca7fff700 1 -- 172.21.15.5:0/2280053324 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0022c80 con 0x7fbcc0067c10 2023-07-21T19:24:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.448+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc01a4bd0 con 0x7fbcc0067c10 2023-07-21T19:24:25.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.452+0000 7fbca7fff700 1 -- 172.21.15.5:0/2280053324 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbcb0004090 con 0x7fbcc0067c10 2023-07-21T19:24:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.453+0000 7fbcd8aefb80 1 Processor -- start 2023-07-21T19:24:25.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.453+0000 7fbcd8aefb80 1 -- start start 2023-07-21T19:24:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561aebd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbca7fff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 0x7fbca80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561ae6580 0x55d561ae69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5619ebc50 con 0x55d5619426b0 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561a3e9e0 con 0x55d561aec2d0 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561a37e70 con 0x55d561ae6580 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbca7fff700 1 -- 172.21.15.5:0/2280053324 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbcb00ce0c0 con 0x7fbcc0067c10 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42580/0 (socket says 172.21.15.5:42580) 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcbed9d700 1 -- 172.21.15.5:0/3739763970 learned_addr learned my addr 172.21.15.5:0/3739763970 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:25.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 0x7fbca80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.454+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561ae6580 0x55d561ae69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 -- 172.21.15.5:0/3739763970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561ae6580 msgr2=0x55d561ae69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561ae6580 0x55d561ae69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561aebd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 -- 172.21.15.5:0/3739763970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 msgr2=0x55d561aebd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561aebd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 -- 172.21.15.5:0/3739763970 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d561865d80 con 0x55d561aec2d0 2023-07-21T19:24:25.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561aebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:24:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0097c90 tx=0x7fbcb009ae70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6dacd679a5b7e45d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 0x7fbca80a3770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80077c0 tx=0x7fbcb800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbca57fa700 1 -- 172.21.15.5:0/3739763970 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb00a10e0 con 0x55d561aec2d0 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbca57fa700 1 -- 172.21.15.5:0/3739763970 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcb009f250 con 0x55d561aec2d0 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.455+0000 7fbca57fa700 1 -- 172.21.15.5:0/3739763970 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb009b480 con 0x55d561aec2d0 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3739763970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 msgr2=0x55d561ae6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fbcb0097c90 tx=0x7fbcb009ae70 comp rx=0 tx=0).stop 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3739763970 shutdown_connections 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561ae6580 0x55d561ae69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561aebd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3739763970 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561ae6040 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3739763970 >> 172.21.15.5:0/3739763970 conn(0x55d561adbbe0 msgr2=0x55d561ae0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3739763970 shutdown_connections 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.456+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3739763970 wait complete. 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.457+0000 7fbcd8aefb80 1 -- start start 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.457+0000 7fbcd8aefb80 1 Processor -- start 2023-07-21T19:24:25.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5619426b0 0x55d561af52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561aef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5619d7a60 con 0x55d561ae6580 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5619d7760 con 0x55d561aec2d0 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d5618bb8c0 con 0x55d5619426b0 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcd8aefb80 10 client.0 _ll_register_callbacks cb 0x55d5619d5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcbe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcbe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35382/0 (socket says 172.21.15.5:35382) 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcbe59c700 1 -- 172.21.15.5:0/2568367519 learned_addr learned my addr 172.21.15.5:0/2568367519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561aef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.458+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5619426b0 0x55d561af52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 -- 172.21.15.5:0/2568367519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5619426b0 msgr2=0x55d561af52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5619426b0 0x55d561af52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 -- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 msgr2=0x55d561aef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561aef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 -- 172.21.15.5:0/2568367519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc01a4bd0 con 0x55d561ae6580 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f80 tx=0x7fbcb4011540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5df4cf476f0b7a2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbca2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4015660 con 0x55d561ae6580 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbcb4011e80 con 0x55d561ae6580 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.459+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d561991490 con 0x55d561ae6580 2023-07-21T19:24:25.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb40157c0 con 0x55d561ae6580 2023-07-21T19:24:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 10 client.18738 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:25.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d56184ec00 con 0x55d561ae6580 2023-07-21T19:24:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d561991490 con 0x55d561ae6580 2023-07-21T19:24:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbcb4014ac0 con 0x55d561ae6580 2023-07-21T19:24:25.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 10 client.18738 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 10 client.18738 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 20 client.18738 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561aff7e0 0x55d561b01bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 4 client.18738 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.460+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d561a30e50 con 0x55d561aff7e0 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.461+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561aff7e0 0x55d561b01bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.461+0000 7fbca2ff5700 10 client.18738 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.461+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561aff7e0 0x55d561b01bf0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34678aea22049eb1 server_cookie=24d0af106de8b555 in_seq=0 out_seq=0 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbca2ff5700 1 -- 172.21.15.5:0/2568367519 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d561a30e50 con 0x55d561aff7e0 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbca2ff5700 10 client.18738 handle_command_reply: tid=0 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 client.18738 shutdown 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 msgr2=0x55d561af5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fbcb4009f80 tx=0x7fbcb4011540 comp rx=0 tx=0).stop 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 shutdown_connections 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561aec2d0 0x55d561aef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d5619426b0 0x55d561af52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.462+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561aff7e0 0x55d561b01bf0 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/2568367519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561af5810 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 >> 172.21.15.5:0/2568367519 conn(0x55d561adbbe0 msgr2=0x55d561adfc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbca2ff5700 0 client.18738 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:25.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 shutdown_connections 2023-07-21T19:24:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/2568367519 wait complete. 2023-07-21T19:24:25.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.463+0000 7fbcd8aefb80 20 client.18738 trim_cache size 0 max 16384 2023-07-21T19:24:25.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.479+0000 7fbcd8aefb80 1 Processor -- start 2023-07-21T19:24:25.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.479+0000 7fbcd8aefb80 1 -- start start 2023-07-21T19:24:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5619426b0 0x55d561ae6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561ae6950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561aea300 con 0x55d561ae6580 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561aea0a0 con 0x55d5619426b0 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561ad8b40 con 0x55d561aec2d0 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcbe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcbe59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35386/0 (socket says 172.21.15.5:35386) 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcbe59c700 1 -- 172.21.15.5:0/3001311508 learned_addr learned my addr 172.21.15.5:0/3001311508 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561ae6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.480+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5619426b0 0x55d561ae6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 -- 172.21.15.5:0/3001311508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 msgr2=0x55d561ae6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561ae6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 -- 172.21.15.5:0/3001311508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5619426b0 msgr2=0x55d561ae6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5619426b0 0x55d561ae6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 -- 172.21.15.5:0/3001311508 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d561aef100 con 0x55d561ae6580 2023-07-21T19:24:25.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400a010 tx=0x7fbcb4006ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12d0b6fe3df9f96 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbca5ffb700 1 -- 172.21.15.5:0/3001311508 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4005c50 con 0x55d561ae6580 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbca5ffb700 1 -- 172.21.15.5:0/3001311508 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcb4011e80 con 0x55d561ae6580 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.481+0000 7fbca5ffb700 1 -- 172.21.15.5:0/3001311508 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4014640 con 0x55d561ae6580 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3001311508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 msgr2=0x55d561aefec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400a010 tx=0x7fbcb4006ed0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3001311508 shutdown_connections 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561ae6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d5619426b0 0x55d561ae6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/3001311508 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d561ae6580 0x55d561aefec0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3001311508 >> 172.21.15.5:0/3001311508 conn(0x55d561adbbe0 msgr2=0x55d561b06550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3001311508 shutdown_connections 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.482+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/3001311508 wait complete. 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.483+0000 7fbcd8aefb80 1 -- start start 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.483+0000 7fbcd8aefb80 1 Processor -- start 2023-07-21T19:24:25.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561ae6580 0x55d561af8540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561b52690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561af8e30 con 0x55d5619426b0 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561b52bd0 con 0x55d561ae6580 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d561b52d10 con 0x55d561aec2d0 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcd8aefb80 10 client.0 _ll_register_callbacks cb 0x55d5619d5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:25.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcbed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:35392/0 (socket says 172.21.15.5:35392) 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcbed9d700 1 -- 172.21.15.5:0/1595977809 learned_addr learned my addr 172.21.15.5:0/1595977809 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561b52690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.484+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561ae6580 0x55d561af8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 -- 172.21.15.5:0/1595977809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 msgr2=0x55d561b52690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561b52690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 -- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561ae6580 msgr2=0x55d561af8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561ae6580 0x55d561af8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 -- 172.21.15.5:0/1595977809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d56184ec00 con 0x55d5619426b0 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbf59e700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561b52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcbed9d700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00cb8c0 tx=0x7fbcb0003a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5740e6f86ef54d6e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbca2ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb0022670 con 0x55d5619426b0 2023-07-21T19:24:25.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbcb009a710 con 0x55d5619426b0 2023-07-21T19:24:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbcb009f250 con 0x55d5619426b0 2023-07-21T19:24:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbcb00996b0 con 0x55d5619426b0 2023-07-21T19:24:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcd8aefb80 10 client.18744 fetch_fsmap learned FSMap version 52 2023-07-21T19:24:25.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.485+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=4+}) v3 -- 0x55d56188c1f0 con 0x55d5619426b0 2023-07-21T19:24:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbcb009f250 con 0x55d5619426b0 2023-07-21T19:24:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 52) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fbcb009b5a0 con 0x55d5619426b0 2023-07-21T19:24:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 10 client.18744 fetch_fsmap finished waiting for FSMap version 52 2023-07-21T19:24:25.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 10 client.18744 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 20 client.18744 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561b54020 0x55d561b56430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 4 client.18744 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d5619f2ff0 con 0x55d561b54020 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.486+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561b54020 0x55d561b56430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.487+0000 7fbcbe59c700 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561b54020 0x55d561b56430 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c14a77a2fad7df6 server_cookie=bbd7ff2e54075cd0 in_seq=0 out_seq=0 2023-07-21T19:24:25.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.487+0000 7fbca2ff5700 10 client.18744 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:25.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.494+0000 7fbca2ff5700 1 -- 172.21.15.5:0/1595977809 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d5619f2ff0 con 0x55d561b54020 2023-07-21T19:24:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.494+0000 7fbca2ff5700 10 client.18744 handle_command_reply: tid=0 2023-07-21T19:24:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.494+0000 7fbcd8aefb80 1 client.18744 shutdown 2023-07-21T19:24:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.494+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 msgr2=0x55d561af8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.494+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00cb8c0 tx=0x7fbcb0003a90 comp rx=0 tx=0).stop 2023-07-21T19:24:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 shutdown_connections 2023-07-21T19:24:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d561aec2d0 0x55d561b52690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d561b54020 0x55d561b56430 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d5619426b0 0x55d561af8000 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 --2- 172.21.15.5:0/1595977809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d561ae6580 0x55d561af8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbca2ff5700 0 client.18744 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 >> 172.21.15.5:0/1595977809 conn(0x55d561adbbe0 msgr2=0x55d561af50a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 shutdown_connections 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 1 -- 172.21.15.5:0/1595977809 wait complete. 2023-07-21T19:24:25.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.495+0000 7fbcd8aefb80 20 client.18744 trim_cache size 0 max 16384 2023-07-21T19:24:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:24:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:24:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:24:25.489 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 msgr2=0x7fbca80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 0x7fbca80a3770 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbcb80077c0 tx=0x7fbcb800b040 comp rx=0 tx=0).stop 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 msgr2=0x7fbcc00fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fbcb00098e0 tx=0x7fbcb0005740 comp rx=0 tx=0).stop 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 shutdown_connections 2023-07-21T19:24:25.490 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbcc0104e50 0x7fbcc01a3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbca80a12c0 0x7fbca80a3770 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbcc0067c10 0x7fbcc00fec40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 --2- 172.21.15.5:0/2280053324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbcc00685b0 0x7fbcc00ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 >> 172.21.15.5:0/2280053324 conn(0x7fbcc0074760 msgr2=0x7fbcc00fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 shutdown_connections 2023-07-21T19:24:25.491 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:25.496+0000 7fbcc51ff700 1 -- 172.21.15.5:0/2280053324 wait complete. 2023-07-21T19:24:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:25 smithi005 ceph-mon[126460]: pgmap v529: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:25 smithi156 ceph-mon[128316]: pgmap v529: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:25 smithi101 ceph-mon[128136]: pgmap v529: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:27.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:27 smithi005 ceph-mon[126460]: pgmap v530: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:24:27.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:27 smithi156 ceph-mon[128316]: pgmap v530: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:24:28.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:27 smithi101 ceph-mon[128136]: pgmap v530: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 MiB/s wr, 17 op/s 2023-07-21T19:24:29.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:29 smithi005 ceph-mon[126460]: pgmap v531: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:24:29.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:29 smithi156 ceph-mon[128316]: pgmap v531: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:24:30.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:29 smithi101 ceph-mon[128136]: pgmap v531: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 36 MiB/s wr, 27 op/s 2023-07-21T19:24:31.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:31 smithi005 ceph-mon[126460]: pgmap v532: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:31.914 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:31 smithi156 ceph-mon[128316]: pgmap v532: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:31 smithi101 ceph-mon[128136]: pgmap v532: 97 pgs: 97 active+clean; 14 GiB data, 6.0 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:32.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:32 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:32.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:32 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:33.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:32 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:33.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:33 smithi005 ceph-mon[126460]: pgmap v533: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:24:33.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:33 smithi156 ceph-mon[128316]: pgmap v533: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:24:34.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:33 smithi101 ceph-mon[128136]: pgmap v533: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 33 MiB/s wr, 24 op/s 2023-07-21T19:24:34.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:34 smithi005 ceph-mon[126460]: pgmap v534: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 21 op/s 2023-07-21T19:24:34.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:34 smithi156 ceph-mon[128316]: pgmap v534: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 21 op/s 2023-07-21T19:24:35.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:34 smithi101 ceph-mon[128136]: pgmap v534: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 25 MiB/s wr, 21 op/s 2023-07-21T19:24:37.649 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:37 smithi005 ceph-mon[126460]: pgmap v535: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:37.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:37 smithi156 ceph-mon[128316]: pgmap v535: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:37.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:37 smithi101 ceph-mon[128136]: pgmap v535: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 18 op/s 2023-07-21T19:24:38.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 -- 172.21.15.5:0/625458822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 msgr2=0x7f92641023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 --2- 172.21.15.5:0/625458822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f92641023a0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f92580099c0 tx=0x7f9258009cd0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 -- 172.21.15.5:0/625458822 shutdown_connections 2023-07-21T19:24:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 --2- 172.21.15.5:0/625458822 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 --2- 172.21.15.5:0/625458822 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f92641023a0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 --2- 172.21.15.5:0/625458822 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 -- 172.21.15.5:0/625458822 >> 172.21.15.5:0/625458822 conn(0x7f92640fd840 msgr2=0x7f92640ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 -- 172.21.15.5:0/625458822 shutdown_connections 2023-07-21T19:24:38.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.664+0000 7f926adff700 1 -- 172.21.15.5:0/625458822 wait complete. 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.665+0000 7f926adff700 1 Processor -- start 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 -- start start 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f9264197e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926419c9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419d020 con 0x7f9264107010 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419d190 con 0x7f9264102970 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926adff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926419d350 con 0x7f9264101fd0 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f9263fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f926939c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926419c9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f9263fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44714/0 (socket says 172.21.15.5:44714) 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f9263fff700 1 -- 172.21.15.5:0/918690681 learned_addr learned my addr 172.21.15.5:0/918690681 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:38.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.666+0000 7f9268b9b700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f9264197e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 -- 172.21.15.5:0/918690681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 msgr2=0x7f9264197e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f9264197e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 -- 172.21.15.5:0/918690681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 msgr2=0x7f9264198370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 -- 172.21.15.5:0/918690681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92580095f0 con 0x7f9264107010 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f9263fff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f926939c700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926419c9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f925c00efc0 tx=0x7f925c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.667+0000 7f9261ffb700 1 -- 172.21.15.5:0/918690681 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c018070 con 0x7f9264107010 2023-07-21T19:24:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.668+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f926419d5d0 con 0x7f9264107010 2023-07-21T19:24:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.668+0000 7f9261ffb700 1 -- 172.21.15.5:0/918690681 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925c00f040 con 0x7f9264107010 2023-07-21T19:24:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.668+0000 7f9261ffb700 1 -- 172.21.15.5:0/918690681 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c00cba0 con 0x7f9264107010 2023-07-21T19:24:38.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.668+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92641a4e70 con 0x7f9264107010 2023-07-21T19:24:38.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.671+0000 7f9261ffb700 1 -- 172.21.15.5:0/918690681 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f925c003a00 con 0x7f9264107010 2023-07-21T19:24:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.673+0000 7f9261ffb700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 0x7f92540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.673+0000 7f927e627b80 1 Processor -- start 2023-07-21T19:24:38.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.673+0000 7f9261ffb700 1 -- 172.21.15.5:0/918690681 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f925c0cc700 con 0x7f9264107010 2023-07-21T19:24:38.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.673+0000 7f9268b9b700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 0x7f92540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.673+0000 7f927e627b80 1 -- start start 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c420040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c4209d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c325c50 con 0x55716c27c6b0 2023-07-21T19:24:38.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c3789e0 con 0x55716c4262d0 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f927e627b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c371e70 con 0x55716c420580 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f9268b9b700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 0x7f92540a37f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258005c00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f9268b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f9268b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60856/0 (socket says 172.21.15.5:60856) 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f9268b9b700 1 -- 172.21.15.5:0/1588317796 learned_addr learned my addr 172.21.15.5:0/1588317796 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f926939c700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c4209d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.674+0000 7f9263fff700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c420040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 -- 172.21.15.5:0/1588317796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 msgr2=0x55716c420040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c420040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 -- 172.21.15.5:0/1588317796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 msgr2=0x55716c425d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 -- 172.21.15.5:0/1588317796 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716c19fd80 con 0x55716c420580 2023-07-21T19:24:38.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f9268b9b700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f926939c700 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c4209d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f925c0c9df0 tx=0x7f925c095c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74bd44a598d84854 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f924effd700 1 -- 172.21.15.5:0/1588317796 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0092e0 con 0x55716c420580 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f924effd700 1 -- 172.21.15.5:0/1588317796 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925c099800 con 0x55716c420580 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.675+0000 7f924effd700 1 -- 172.21.15.5:0/1588317796 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0b9250 con 0x55716c420580 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 -- 172.21.15.5:0/1588317796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 msgr2=0x55716c4209d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c4209d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f925c0c9df0 tx=0x7f925c095c60 comp rx=0 tx=0).stop 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 -- 172.21.15.5:0/1588317796 shutdown_connections 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c4209d0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c425d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 --2- 172.21.15.5:0/1588317796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c420040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 -- 172.21.15.5:0/1588317796 >> 172.21.15.5:0/1588317796 conn(0x55716c415be0 msgr2=0x55716c41af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 -- 172.21.15.5:0/1588317796 shutdown_connections 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.676+0000 7f927e627b80 1 -- 172.21.15.5:0/1588317796 wait complete. 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.677+0000 7f927e627b80 1 -- start start 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.677+0000 7f927e627b80 1 Processor -- start 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c27c6b0 0x55716c4298d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c42a350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c311a60 con 0x55716c420580 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c311760 con 0x55716c4262d0 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c1f58c0 con 0x55716c27c6b0 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f927e627b80 10 client.0 _ll_register_callbacks cb 0x55716c30f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f9263fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f9263fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60868/0 (socket says 172.21.15.5:60868) 2023-07-21T19:24:38.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f9263fff700 1 -- 172.21.15.5:0/1418638928 learned_addr learned my addr 172.21.15.5:0/1418638928 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f9268b9b700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c27c6b0 0x55716c4298d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.678+0000 7f926939c700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c42a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 -- 172.21.15.5:0/1418638928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c27c6b0 msgr2=0x55716c4298d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c27c6b0 0x55716c4298d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 -- 172.21.15.5:0/1418638928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 msgr2=0x55716c429e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 -- 172.21.15.5:0/1418638928 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f926410f420 con 0x55716c4262d0 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f9263fff700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f926939c700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c42a350 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f925c097b50 tx=0x7f925c096120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24bc148a5ce136de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f9243fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c09d830 con 0x55716c4262d0 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55716c2cb490 con 0x55716c4262d0 2023-07-21T19:24:38.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.679+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f925c0a1bc0 con 0x55716c4262d0 2023-07-21T19:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925c099c70 con 0x55716c4262d0 2023-07-21T19:24:38.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 10 client.28418 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f92641a4e70 con 0x55716c4262d0 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55716c2cb490 con 0x55716c4262d0 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f925c098760 con 0x55716c4262d0 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 10 client.28418 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 10 client.28418 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 20 client.28418 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:38.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c4397b0 0x55716c43bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 4 client.28418 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55716c36ae50 con 0x55716c4397b0 2023-07-21T19:24:38.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.680+0000 7f9268b9b700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c4397b0 0x55716c43bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.678 INFO:teuthology.orchestra.run.smithi005.stderr:set primary keys... 2023-07-21T19:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.681+0000 7f9268b9b700 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c4397b0 0x55716c43bbc0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e495bd16b32786e server_cookie=84964ba8bc2b6bc0 in_seq=0 out_seq=0 2023-07-21T19:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.681+0000 7f9243fff700 10 client.28418 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f9243fff700 1 -- 172.21.15.5:0/1418638928 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55716c36ae50 con 0x55716c4397b0 2023-07-21T19:24:38.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f9243fff700 10 client.28418 handle_command_reply: tid=0 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 client.28418 shutdown 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 msgr2=0x55716c42a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c42a350 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f925c097b50 tx=0x7f925c096120 comp rx=0 tx=0).stop 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 shutdown_connections 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c42a350 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c27c6b0 0x55716c4298d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c4397b0 0x55716c43bbc0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 --2- 172.21.15.5:0/1418638928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c420580 0x55716c429e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f9243fff700 0 client.28418 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 >> 172.21.15.5:0/1418638928 conn(0x55716c415be0 msgr2=0x55716c41a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 shutdown_connections 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.682+0000 7f927e627b80 1 -- 172.21.15.5:0/1418638928 wait complete. 2023-07-21T19:24:38.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.683+0000 7f927e627b80 20 client.28418 trim_cache size 0 max 16384 2023-07-21T19:24:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.699+0000 7f927e627b80 1 Processor -- start 2023-07-21T19:24:38.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.699+0000 7f927e627b80 1 -- start start 2023-07-21T19:24:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c420040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c430c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c426130 con 0x55716c27c6b0 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c412bb0 con 0x55716c4262d0 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f927e627b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c424640 con 0x55716c420580 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f926939c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f9268b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c420040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f9263fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c430c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f926939c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44736/0 (socket says 172.21.15.5:44736) 2023-07-21T19:24:38.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f926939c700 1 -- 172.21.15.5:0/4205198187 learned_addr learned my addr 172.21.15.5:0/4205198187 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.700+0000 7f926939c700 1 -- 172.21.15.5:0/4205198187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 msgr2=0x55716c430c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f926939c700 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c430c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f926939c700 1 -- 172.21.15.5:0/4205198187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 msgr2=0x55716c420040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f926939c700 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c420040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f926939c700 1 -- 172.21.15.5:0/4205198187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55716c188c00 con 0x55716c4262d0 2023-07-21T19:24:38.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f9263fff700 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c430c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f9268b9b700 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c420040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f926939c700 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f925c095d00 tx=0x7f925c096190 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13bb86b0aa219669 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f924f7fe700 1 -- 172.21.15.5:0/4205198187 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0a19d0 con 0x55716c4262d0 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f924f7fe700 1 -- 172.21.15.5:0/4205198187 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925c0a1bc0 con 0x55716c4262d0 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.701+0000 7f924f7fe700 1 -- 172.21.15.5:0/4205198187 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c098760 con 0x55716c4262d0 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- 172.21.15.5:0/4205198187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 msgr2=0x55716c425160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f925c095d00 tx=0x7f925c096190 comp rx=0 tx=0).stop 2023-07-21T19:24:38.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- 172.21.15.5:0/4205198187 shutdown_connections 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c4262d0 0x55716c425160 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c420040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 --2- 172.21.15.5:0/4205198187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c420580 0x55716c430c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- 172.21.15.5:0/4205198187 >> 172.21.15.5:0/4205198187 conn(0x55716c415be0 msgr2=0x55716c424a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- 172.21.15.5:0/4205198187 shutdown_connections 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- 172.21.15.5:0/4205198187 wait complete. 2023-07-21T19:24:38.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.702+0000 7f927e627b80 1 -- start start 2023-07-21T19:24:38.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 Processor -- start 2023-07-21T19:24:38.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c42ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 0x55716c42f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c494e80 con 0x55716c27c6b0 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c494ff0 con 0x55716c420580 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.703+0000 7f927e627b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55716c4951b0 con 0x55716c4262d0 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f927e627b80 10 client.0 _ll_register_callbacks cb 0x7f925800b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f9263fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f9263fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44752/0 (socket says 172.21.15.5:44752) 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f9263fff700 1 -- 172.21.15.5:0/2060904201 learned_addr learned my addr 172.21.15.5:0/2060904201 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f926939c700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 0x55716c42f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.704+0000 7f9268b9b700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c42ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 -- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 msgr2=0x55716c42f190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 -- 172.21.15.5:0/2060904201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 msgr2=0x55716c42ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:38.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c42ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 -- 172.21.15.5:0/2060904201 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f925800bd20 con 0x55716c4262d0 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9268b9b700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c42ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9263fff700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f926939c700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 0x55716c42f6d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ef90 tx=0x7f925c00c810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ad10d2e5a2d4a29 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9243fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f925c0a19b0 con 0x55716c4262d0 2023-07-21T19:24:38.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f925c099800 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f925c09d830 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f925c098b30 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f927e627b80 10 client.28339 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.705+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f9250012760 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f925c099800 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f925c0cc110 con 0x55716c4262d0 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 10 client.28339 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:38.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 10 client.28339 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 20 client.28339 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c496370 0x55716c498750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 4 client.28339 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.706+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55716c32cff0 con 0x55716c496370 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.707+0000 7f9268b9b700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c496370 0x55716c498750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.707+0000 7f9268b9b700 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c496370 0x55716c498750 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c235c625a7f3cf0a server_cookie=da60af7199bd7d72 in_seq=0 out_seq=0 2023-07-21T19:24:38.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.707+0000 7f9243fff700 10 client.28339 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.714+0000 7f9243fff700 1 -- 172.21.15.5:0/2060904201 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55716c32cff0 con 0x55716c496370 2023-07-21T19:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.714+0000 7f9243fff700 10 client.28339 handle_command_reply: tid=0 2023-07-21T19:24:38.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.714+0000 7f927e627b80 1 client.28339 shutdown 2023-07-21T19:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.714+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 msgr2=0x55716c42f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.714+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 0x55716c42f6d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f925c00ef90 tx=0x7f925c00c810 comp rx=0 tx=0).stop 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 shutdown_connections 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55716c4262d0 0x55716c42f6d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55716c496370 0x55716c498750 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55716c27c6b0 0x55716c42ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f9243fff700 0 client.28339 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 --2- 172.21.15.5:0/2060904201 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55716c420580 0x55716c42f190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 >> 172.21.15.5:0/2060904201 conn(0x55716c415be0 msgr2=0x55716c42a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 shutdown_connections 2023-07-21T19:24:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 1 -- 172.21.15.5:0/2060904201 wait complete. 2023-07-21T19:24:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.715+0000 7f927e627b80 20 client.28339 trim_cache size 0 max 16384 2023-07-21T19:24:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:24:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:24:38.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 msgr2=0x7f92540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 0x7f92540a37f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9258009990 tx=0x7f9258005c00 comp rx=0 tx=0).stop 2023-07-21T19:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 msgr2=0x7f926419c9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:38.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926419c9e0 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f925c00efc0 tx=0x7f925c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 shutdown_connections 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92540a1340 0x7f92540a37f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9264107010 0x7f926419c9e0 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9264101fd0 0x7f9264197e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 --2- 172.21.15.5:0/918690681 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9264102970 0x7f9264198370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 >> 172.21.15.5:0/918690681 conn(0x7f92640fd840 msgr2=0x7f9264105970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:38.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 shutdown_connections 2023-07-21T19:24:38.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:38.716+0000 7f926adff700 1 -- 172.21.15.5:0/918690681 wait complete. 2023-07-21T19:24:39.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:39 smithi101 ceph-mon[128136]: pgmap v536: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 26 op/s 2023-07-21T19:24:39.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:39 smithi005 ceph-mon[126460]: pgmap v536: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 26 op/s 2023-07-21T19:24:39.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:39 smithi156 ceph-mon[128316]: pgmap v536: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 MiB/s wr, 26 op/s 2023-07-21T19:24:41.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:41 smithi101 ceph-mon[128136]: pgmap v537: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:41.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:41 smithi005 ceph-mon[126460]: pgmap v537: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:41.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:41 smithi156 ceph-mon[128316]: pgmap v537: 97 pgs: 97 active+clean; 14 GiB data, 6.1 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 24 MiB/s wr, 16 op/s 2023-07-21T19:24:43.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:43 smithi101 ceph-mon[128136]: pgmap v538: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 26 op/s 2023-07-21T19:24:43.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:43 smithi005 ceph-mon[126460]: pgmap v538: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 26 op/s 2023-07-21T19:24:43.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:43 smithi156 ceph-mon[128316]: pgmap v538: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 32 MiB/s wr, 26 op/s 2023-07-21T19:24:45.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:45 smithi005 ceph-mon[126460]: pgmap v539: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:24:45.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:45 smithi156 ceph-mon[128316]: pgmap v539: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:24:46.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:45 smithi101 ceph-mon[128136]: pgmap v539: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 23 MiB/s wr, 25 op/s 2023-07-21T19:24:47.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:47 smithi005 ceph-mon[126460]: pgmap v540: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 23 op/s 2023-07-21T19:24:47.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:47 smithi156 ceph-mon[128316]: pgmap v540: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 23 op/s 2023-07-21T19:24:48.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:47 smithi101 ceph-mon[128136]: pgmap v540: 97 pgs: 97 active+clean; 14 GiB data, 6.2 GiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 21 MiB/s wr, 23 op/s 2023-07-21T19:24:48.348 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:24:48.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 -- 172.21.15.5:0/2557728621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd100ffae0 msgr2=0x7fdd100fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2557728621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd100ffae0 0x7fdd100fff50 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fdd00009990 tx=0x7fdd00009ca0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 -- 172.21.15.5:0/2557728621 shutdown_connections 2023-07-21T19:24:48.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2557728621 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 0x7fdd1010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2557728621 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd10105af0 0x7fdd10105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2557728621 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd100ffae0 0x7fdd100fff50 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.574+0000 7fdd187ff700 1 -- 172.21.15.5:0/2557728621 >> 172.21.15.5:0/2557728621 conn(0x7fdd100fb5e0 msgr2=0x7fdd100fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.575+0000 7fdd187ff700 1 -- 172.21.15.5:0/2557728621 shutdown_connections 2023-07-21T19:24:48.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.575+0000 7fdd187ff700 1 -- 172.21.15.5:0/2557728621 wait complete. 2023-07-21T19:24:48.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.576+0000 7fdd187ff700 1 Processor -- start 2023-07-21T19:24:48.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.576+0000 7fdd187ff700 1 -- start start 2023-07-21T19:24:48.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.576+0000 7fdd187ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 0x7fdd10197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.576+0000 7fdd187ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 0x7fdd10198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.576+0000 7fdd187ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 0x7fdd1019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd187ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019bbd0 con 0x7fdd10105af0 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd187ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019bd40 con 0x7fdd100ffae0 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd187ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd1019bf00 con 0x7fdd10100490 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd1659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 0x7fdd10197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 0x7fdd1019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd1659b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 0x7fdd10197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59368/0 (socket says 172.21.15.5:59368) 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd1659b700 1 -- 172.21.15.5:0/2608372282 learned_addr learned my addr 172.21.15.5:0/2608372282 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 0x7fdd10198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2608372282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 msgr2=0x7fdd10198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 0x7fdd10198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2608372282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 msgr2=0x7fdd10197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 0x7fdd10197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.577+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2608372282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd000095f0 con 0x7fdd10105af0 2023-07-21T19:24:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 0x7fdd1019b5d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007f10 tx=0x7fdd0800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd077fe700 1 -- 172.21.15.5:0/2608372282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08004db0 con 0x7fdd10105af0 2023-07-21T19:24:48.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd1019c1e0 con 0x7fdd10105af0 2023-07-21T19:24:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd077fe700 1 -- 172.21.15.5:0/2608372282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0800f040 con 0x7fdd10105af0 2023-07-21T19:24:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd077fe700 1 -- 172.21.15.5:0/2608372282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08005960 con 0x7fdd10105af0 2023-07-21T19:24:48.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.578+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd101a4b40 con 0x7fdd10105af0 2023-07-21T19:24:48.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.582+0000 7fdd077fe700 1 -- 172.21.15.5:0/2608372282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdd0800db20 con 0x7fdd10105af0 2023-07-21T19:24:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.584+0000 7fdd077fe700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 0x7fdcfc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.584+0000 7fdd2c0f4b80 1 Processor -- start 2023-07-21T19:24:48.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.584+0000 7fdd077fe700 1 -- 172.21.15.5:0/2608372282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdd080cc570 con 0x7fdd10105af0 2023-07-21T19:24:48.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 0x7fdcfc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 -- start start 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb809356a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb8094b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8083bcb0 con 0x55bb80935be0 2023-07-21T19:24:48.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8088ea40 con 0x55bb809352d0 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.585+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80887ed0 con 0x55bb809492e0 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 0x7fdcfc0a3770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005950 tx=0x7fdd0c00b500 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42968/0 (socket says 172.21.15.5:42968) 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 -- 172.21.15.5:0/2414112753 learned_addr learned my addr 172.21.15.5:0/2414112753 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb809356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb8094b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 -- 172.21.15.5:0/2414112753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 msgr2=0x55bb8094b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb8094b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 -- 172.21.15.5:0/2414112753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 msgr2=0x55bb809356a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb809356a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.586+0000 7fdd15d9a700 1 -- 172.21.15.5:0/2414112753 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb806b5de0 con 0x55bb80935be0 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.587+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098e0 tx=0x7fdd00009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4d92c7328891d0f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.587+0000 7fdd04ff9700 1 -- 172.21.15.5:0/2414112753 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0001d070 con 0x55bb80935be0 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.587+0000 7fdd04ff9700 1 -- 172.21.15.5:0/2414112753 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0001e040 con 0x55bb80935be0 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.587+0000 7fdd04ff9700 1 -- 172.21.15.5:0/2414112753 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd00004500 con 0x55bb80935be0 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2414112753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 msgr2=0x55bb80948da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7fdd000098e0 tx=0x7fdd00009f90 comp rx=0 tx=0).stop 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2414112753 shutdown_connections 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb8094b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb809356a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2414112753 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80948da0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2414112753 >> 172.21.15.5:0/2414112753 conn(0x55bb8092bda0 msgr2=0x55bb809316a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.588+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2414112753 shutdown_connections 2023-07-21T19:24:48.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.589+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2414112753 wait complete. 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.589+0000 7fdd2c0f4b80 1 -- start start 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.589+0000 7fdd2c0f4b80 1 Processor -- start 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb80948d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80945f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80827ac0 con 0x55bb80935be0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb808277c0 con 0x55bb809352d0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8070b920 con 0x55bb809492e0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.590+0000 7fdd2c0f4b80 10 client.0 _ll_register_callbacks cb 0x55bb80825760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.591+0000 7fdd16d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.591+0000 7fdd16d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54224/0 (socket says 172.21.15.5:54224) 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.591+0000 7fdd16d9c700 1 -- 172.21.15.5:0/25923860 learned_addr learned my addr 172.21.15.5:0/25923860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80945f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd1659b700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb80948d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd16d9c700 1 -- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 msgr2=0x55bb80948d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb80948d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd16d9c700 1 -- 172.21.15.5:0/25923860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 msgr2=0x55bb80945f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80945f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd16d9c700 1 -- 172.21.15.5:0/25923860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd1010d220 con 0x55bb809492e0 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.592+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80945f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.593+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800dfd0 tx=0x7fdd080992b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa9998b889906205 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.593+0000 7fdcf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:24:48.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.593+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd080842b0 con 0x55bb809492e0 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.593+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bb807e14f0 con 0x55bb809492e0 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.593+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdd08082e40 con 0x55bb809492e0 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.594+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd08005000 con 0x55bb809492e0 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.594+0000 7fdd2c0f4b80 10 client.28351 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.594+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55bb8069ec60 con 0x55bb809492e0 2023-07-21T19:24:48.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.594+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd080c7080 con 0x55bb809492e0 2023-07-21T19:24:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdd080998a0 con 0x55bb809492e0 2023-07-21T19:24:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 10 client.28351 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 10 client.28351 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:48.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 20 client.28351 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb80958bd0 0x55bb8095afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 4 client.28351 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.595+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bb80880eb0 con 0x55bb80958bd0 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.596+0000 7fdd1659b700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb80958bd0 0x55bb8095afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.597+0000 7fdd1659b700 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb80958bd0 0x55bb8095afe0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a68be7243bbaf46e server_cookie=28a65abbe0851b57 in_seq=0 out_seq=0 2023-07-21T19:24:48.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.597+0000 7fdcf9ffb700 10 client.28351 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.598+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/25923860 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bb80880eb0 con 0x55bb80958bd0 2023-07-21T19:24:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.598+0000 7fdcf9ffb700 10 client.28351 handle_command_reply: tid=0 2023-07-21T19:24:48.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.598+0000 7fdd2c0f4b80 1 client.28351 shutdown 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.598+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 msgr2=0x55bb809466c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.598+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800dfd0 tx=0x7fdd080992b0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 shutdown_connections 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809466c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb80948d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb80958bd0 0x55bb8095afe0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/25923860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb80935be0 0x55bb80945f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdcf9ffb700 0 client.28351 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 >> 172.21.15.5:0/25923860 conn(0x55bb8092bda0 msgr2=0x55bb80931230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 shutdown_connections 2023-07-21T19:24:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/25923860 wait complete. 2023-07-21T19:24:48.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.599+0000 7fdd2c0f4b80 20 client.28351 trim_cache size 0 max 16384 2023-07-21T19:24:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 Processor -- start 2023-07-21T19:24:48.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 -- start start 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb80935be0 0x55bb80954f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809488a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8095fe80 con 0x55bb809352d0 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8094b6c0 con 0x55bb80935be0 2023-07-21T19:24:48.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.620+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb80955490 con 0x55bb809492e0 2023-07-21T19:24:48.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42994/0 (socket says 172.21.15.5:42994) 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 -- 172.21.15.5:0/1690909377 learned_addr learned my addr 172.21.15.5:0/1690909377 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb80935be0 0x55bb80954f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809488a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 -- 172.21.15.5:0/1690909377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 msgr2=0x55bb809488a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809488a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 -- 172.21.15.5:0/1690909377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb80935be0 msgr2=0x55bb80954f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb80935be0 0x55bb80954f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.621+0000 7fdd1659b700 1 -- 172.21.15.5:0/1690909377 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bb806dc250 con 0x55bb809352d0 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.622+0000 7fdd1659b700 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c011010 tx=0x7fdd0c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f8c2d08f99153ac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.622+0000 7fdd057fa700 1 -- 172.21.15.5:0/1690909377 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c02b070 con 0x55bb809352d0 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.622+0000 7fdd057fa700 1 -- 172.21.15.5:0/1690909377 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0c015070 con 0x55bb809352d0 2023-07-21T19:24:48.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.622+0000 7fdd057fa700 1 -- 172.21.15.5:0/1690909377 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd0c025040 con 0x55bb809352d0 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/1690909377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 msgr2=0x55bb8095f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c011010 tx=0x7fdd0c016000 comp rx=0 tx=0).stop 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/1690909377 shutdown_connections 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb809492e0 0x55bb809488a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809352d0 0x55bb8095f940 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/1690909377 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb80935be0 0x55bb80954f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/1690909377 >> 172.21.15.5:0/1690909377 conn(0x55bb8092bda0 msgr2=0x55bb8092fe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/1690909377 shutdown_connections 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.623+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/1690909377 wait complete. 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 -- start start 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 Processor -- start 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb8094e8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 0x55bb8094ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb8094f4f0 con 0x55bb809492e0 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb809acfe0 con 0x55bb809352d0 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bb809ad120 con 0x55bb80935be0 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.624+0000 7fdd2c0f4b80 10 client.0 _ll_register_callbacks cb 0x7fdd080959e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42998/0 (socket says 172.21.15.5:42998) 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2837793675 learned_addr learned my addr 172.21.15.5:0/2837793675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb8094e8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 0x55bb8094ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2837793675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 msgr2=0x55bb8094ee20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 0x55bb8094ee20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 msgr2=0x55bb8094e8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb8094e8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 -- 172.21.15.5:0/2837793675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd0c010c50 con 0x55bb809492e0 2023-07-21T19:24:48.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd15d9a700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 0x55bb8094ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdd16d9c700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdd080995d0 tx=0x7fdd08000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2764256122141cbe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.625+0000 7fdcf9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd08098b80 con 0x55bb809492e0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdd0001e040 con 0x55bb809492e0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdd08082e40 con 0x55bb809492e0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdd08005ac0 con 0x55bb809492e0 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 10 client.18780 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:48.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55bb806b42b0 con 0x55bb809492e0 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdd0001e040 con 0x55bb809492e0 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fdd080652b0 con 0x55bb809492e0 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 10 client.18780 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 10 client.18780 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 20 client.18780 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.626+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb809ae330 0x55bb809b0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.627+0000 7fdd2c0f4b80 4 client.18780 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:24:48.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.627+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55bb80843050 con 0x55bb809ae330 2023-07-21T19:24:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.627+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb809ae330 0x55bb809b0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.627+0000 7fdcf9ffb700 10 client.18780 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.627+0000 7fdd1659b700 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb809ae330 0x55bb809b0710 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab6f30cf745aba7 server_cookie=95cbfef830734288 in_seq=0 out_seq=0 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdcf9ffb700 1 -- 172.21.15.5:0/2837793675 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55bb80843050 con 0x55bb809ae330 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdcf9ffb700 10 client.18780 handle_command_reply: tid=0 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 client.18780 shutdown 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 msgr2=0x55bb809acaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fdd080995d0 tx=0x7fdd08000f40 comp rx=0 tx=0).stop 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 shutdown_connections 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bb809ae330 0x55bb809b0710 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bb809492e0 0x55bb809acaa0 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bb809352d0 0x55bb8094e8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 --2- 172.21.15.5:0/2837793675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bb80935be0 0x55bb8094ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdcf9ffb700 0 client.18780 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 >> 172.21.15.5:0/2837793675 conn(0x55bb8092bda0 msgr2=0x55bb809373e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 shutdown_connections 2023-07-21T19:24:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 1 -- 172.21.15.5:0/2837793675 wait complete. 2023-07-21T19:24:48.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.628+0000 7fdd2c0f4b80 20 client.18780 trim_cache size 0 max 16384 2023-07-21T19:24:48.626 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:24:48.626 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:24:48.626 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:24:48.627 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:24:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 msgr2=0x7fdcfc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 0x7fdcfc0a3770 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c005950 tx=0x7fdd0c00b500 comp rx=0 tx=0).stop 2023-07-21T19:24:48.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 msgr2=0x7fdd1019b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 0x7fdd1019b5d0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fdd08007f10 tx=0x7fdd0800d3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 shutdown_connections 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdcfc0a12c0 0x7fdcfc0a3770 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdd10105af0 0x7fdd1019b5d0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdd100ffae0 0x7fdd10197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 --2- 172.21.15.5:0/2608372282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdd10100490 0x7fdd10198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 >> 172.21.15.5:0/2608372282 conn(0x7fdd100fb5e0 msgr2=0x7fdd1010ada0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 shutdown_connections 2023-07-21T19:24:48.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.629+0000 7fdd187ff700 1 -- 172.21.15.5:0/2608372282 wait complete. 2023-07-21T19:24:48.637 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:24:48.638 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:24:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 -- 172.21.15.5:0/647582045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0068000 msgr2=0x7f80b010c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/647582045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0068000 0x7f80b010c7a0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f80a800b0a0 tx=0x7f80a800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 -- 172.21.15.5:0/647582045 shutdown_connections 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/647582045 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0068000 0x7f80b010c7a0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/647582045 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0104a30 0x7f80b0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/647582045 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b0067ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 -- 172.21.15.5:0/647582045 >> 172.21.15.5:0/647582045 conn(0x7f80b0074a30 msgr2=0x7f80b0074e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 -- 172.21.15.5:0/647582045 shutdown_connections 2023-07-21T19:24:48.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.885+0000 7f80b4f9d700 1 -- 172.21.15.5:0/647582045 wait complete. 2023-07-21T19:24:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 Processor -- start 2023-07-21T19:24:48.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 -- start start 2023-07-21T19:24:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b019d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0068000 0x7f80b019ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 0x7f80b0197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.887+0000 7f80b4f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0197f90 con 0x7f80b0104a30 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80b4f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b0198100 con 0x7f80b0067670 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80b4f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b01982c0 con 0x7f80b0068000 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b019d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 0x7f80b0197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80af7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b019d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59404/0 (socket says 172.21.15.5:59404) 2023-07-21T19:24:48.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80af7fe700 1 -- 172.21.15.5:0/2681633411 learned_addr learned my addr 172.21.15.5:0/2681633411 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80aeffd700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0068000 0x7f80b019ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 -- 172.21.15.5:0/2681633411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0068000 msgr2=0x7f80b019ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0068000 0x7f80b019ddf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 -- 172.21.15.5:0/2681633411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 msgr2=0x7f80b019d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b019d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:48.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.888+0000 7f80affff700 1 -- 172.21.15.5:0/2681633411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a8009d00 con 0x7f80b0104a30 2023-07-21T19:24:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80affff700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 0x7f80b0197920 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009fd0 tx=0x7f80a8007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a800e040 con 0x7f80b0104a30 2023-07-21T19:24:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f80a800bdc0 con 0x7f80b0104a30 2023-07-21T19:24:48.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b0198540 con 0x7f80b0104a30 2023-07-21T19:24:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8005040 con 0x7f80b0104a30 2023-07-21T19:24:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.889+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b01a48a0 con 0x7f80b0104a30 2023-07-21T19:24:48.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.890+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b0067220 con 0x7f80b0104a30 2023-07-21T19:24:48.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.894+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f80a801d480 con 0x7f80b0104a30 2023-07-21T19:24:48.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.896+0000 7f80acff9700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 0x7f80980a38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.896+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f80a80ce180 con 0x7f80b0104a30 2023-07-21T19:24:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.896+0000 7f80af7fe700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 0x7f80980a38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:48.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.897+0000 7f80af7fe700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 0x7f80980a38e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ca0 tx=0x7f80a0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:48.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:48.898+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f80a80975c0 con 0x7f80b0104a30 2023-07-21T19:24:49.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.181+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f80b01a4a30 con 0x7f80b0104a30 2023-07-21T19:24:49.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.182+0000 7f80acff9700 1 -- 172.21.15.5:0/2681633411 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 53 v53) v1 ==== 94+0+10837 (secure 0 0 0) 0x7f80a809abe0 con 0x7f80b0104a30 2023-07-21T19:24:49.173 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:24:49.174 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":53,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":53,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:24:31.612652+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:24:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 msgr2=0x7f80980a38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 0x7f80980a38e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f80a0009ca0 tx=0x7f80a0009260 comp rx=0 tx=0).stop 2023-07-21T19:24:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 msgr2=0x7f80b0197920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 0x7f80b0197920 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009fd0 tx=0x7f80a8007930 comp rx=0 tx=0).stop 2023-07-21T19:24:49.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 shutdown_connections 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f80980a1430 0x7f80980a38e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80b0104a30 0x7f80b0197920 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f80b0067670 0x7f80b019d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 --2- 172.21.15.5:0/2681633411 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f80b0068000 0x7f80b019ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 >> 172.21.15.5:0/2681633411 conn(0x7f80b0074a30 msgr2=0x7f80b010baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 shutdown_connections 2023-07-21T19:24:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.185+0000 7f80b4f9d700 1 -- 172.21.15.5:0/2681633411 wait complete. 2023-07-21T19:24:49.177 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 53 2023-07-21T19:24:49.194 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:24:49.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 -- 172.21.15.5:0/2235356094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 msgr2=0x7f5778072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 --2- 172.21.15.5:0/2235356094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778072c80 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f5768009990 tx=0x7f5768009ca0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 -- 172.21.15.5:0/2235356094 shutdown_connections 2023-07-21T19:24:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 --2- 172.21.15.5:0/2235356094 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 0x7f577810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 --2- 172.21.15.5:0/2235356094 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778072c80 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 --2- 172.21.15.5:0/2235356094 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f577810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 -- 172.21.15.5:0/2235356094 >> 172.21.15.5:0/2235356094 conn(0x7f57780fb5e0 msgr2=0x7f57780fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 -- 172.21.15.5:0/2235356094 shutdown_connections 2023-07-21T19:24:49.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.437+0000 7f577d3ff700 1 -- 172.21.15.5:0/2235356094 wait complete. 2023-07-21T19:24:49.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.439+0000 7f577d3ff700 1 Processor -- start 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.439+0000 7f577d3ff700 1 -- start start 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.439+0000 7f577d3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f577d3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f5778198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f577d3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 0x7f577819b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f577d3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819bbd0 con 0x7f57780728b0 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f577d3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819bd40 con 0x7f577810c320 2023-07-21T19:24:49.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f577d3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577819bf00 con 0x7f57780731c0 2023-07-21T19:24:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43044/0 (socket says 172.21.15.5:43044) 2023-07-21T19:24:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 -- 172.21.15.5:0/249731602 learned_addr learned my addr 172.21.15.5:0/249731602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f57777fe700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 0x7f577819b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f57767fc700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f5778198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 -- 172.21.15.5:0/249731602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 msgr2=0x7f5778198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f5778198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 -- 172.21.15.5:0/249731602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 msgr2=0x7f577819b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 0x7f577819b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f5776ffd700 1 -- 172.21.15.5:0/249731602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57680095f0 con 0x7f57780728b0 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.440+0000 7f57767fc700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f5778198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:49.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f5776ffd700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5768005910 tx=0x7f5768009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f575ffff700 1 -- 172.21.15.5:0/249731602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576801d070 con 0x7f57780728b0 2023-07-21T19:24:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f575ffff700 1 -- 172.21.15.5:0/249731602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5768003ec0 con 0x7f57780728b0 2023-07-21T19:24:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577819c180 con 0x7f57780728b0 2023-07-21T19:24:49.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f575ffff700 1 -- 172.21.15.5:0/249731602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800fb50 con 0x7f57780728b0 2023-07-21T19:24:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.441+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a4b40 con 0x7f57780728b0 2023-07-21T19:24:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.444+0000 7f575ffff700 1 -- 172.21.15.5:0/249731602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5768004030 con 0x7f57780728b0 2023-07-21T19:24:49.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 Processor -- start 2023-07-21T19:24:49.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 -- start start 2023-07-21T19:24:49.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f575ffff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 0x7f57600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ee2ecb0 con 0x564a7ef33ad0 2023-07-21T19:24:49.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ee81a40 con 0x564a7ef28be0 2023-07-21T19:24:49.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f5790df5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ee7aed0 con 0x564a7ef282d0 2023-07-21T19:24:49.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.446+0000 7f575ffff700 1 -- 172.21.15.5:0/249731602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57680cdd30 con 0x7f57780728b0 2023-07-21T19:24:49.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f57777fe700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 0x7f57600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f57777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f57777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef35eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43054/0 (socket says 172.21.15.5:43054) 2023-07-21T19:24:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f57777fe700 1 -- 172.21.15.5:0/1558906125 learned_addr learned my addr 172.21.15.5:0/1558906125 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f57767fc700 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 -- 172.21.15.5:0/1558906125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 msgr2=0x564a7ef33590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef33590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 -- 172.21.15.5:0/1558906125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 msgr2=0x564a7ef35eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef35eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.447+0000 7f5776ffd700 1 -- 172.21.15.5:0/1558906125 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7eca8de0 con 0x564a7ef282d0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.448+0000 7f57777fe700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 0x7f57600a37f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5770007fd0 tx=0x7f5770007e60 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.448+0000 7f5776ffd700 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef286a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57680cb420 tx=0x7f576809aaa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4c9ba343c9c1b6f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.448+0000 7f575d7fa700 1 -- 172.21.15.5:0/1558906125 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576800f460 con 0x564a7ef282d0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.448+0000 7f575d7fa700 1 -- 172.21.15.5:0/1558906125 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f576809ae30 con 0x564a7ef282d0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.448+0000 7f575d7fa700 1 -- 172.21.15.5:0/1558906125 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680cb6a0 con 0x564a7ef282d0 2023-07-21T19:24:49.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 -- 172.21.15.5:0/1558906125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 msgr2=0x564a7ef286a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef286a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f57680cb420 tx=0x7f576809aaa0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 -- 172.21.15.5:0/1558906125 shutdown_connections 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef35eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef286a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 --2- 172.21.15.5:0/1558906125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 -- 172.21.15.5:0/1558906125 >> 172.21.15.5:0/1558906125 conn(0x564a7ef1eda0 msgr2=0x564a7ef246a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 -- 172.21.15.5:0/1558906125 shutdown_connections 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.449+0000 7f5790df5b80 1 -- 172.21.15.5:0/1558906125 wait complete. 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 -- start start 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 Processor -- start 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef282d0 0x564a7ef38c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef33ad0 0x564a7ef32a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ee1aac0 con 0x564a7ef28be0 2023-07-21T19:24:49.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ee1a7c0 con 0x564a7ef282d0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.450+0000 7f5790df5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ecfe920 con 0x564a7ef33ad0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f5790df5b80 10 client.0 _ll_register_callbacks cb 0x564a7ee18760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43058/0 (socket says 172.21.15.5:43058) 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 -- 172.21.15.5:0/2355425397 learned_addr learned my addr 172.21.15.5:0/2355425397 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57777fe700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef33ad0 0x564a7ef32a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef282d0 0x564a7ef38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 -- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef33ad0 msgr2=0x564a7ef32a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef33ad0 0x564a7ef32a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 -- 172.21.15.5:0/2355425397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef282d0 msgr2=0x564a7ef38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef282d0 0x564a7ef38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.451+0000 7f57767fc700 1 -- 172.21.15.5:0/2355425397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57781a4b40 con 0x564a7ef28be0 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f57767fc700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f576c00bad0 tx=0x7f576c00bde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83e7a9cc2cdccbd0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f575aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:49.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c01b440 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a7edd44f0 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f576c01b5a0 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f576c01b760 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f5790df5b80 10 client.18798 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.452+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x564a7ec91c60 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a7edd44f0 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f576c024430 con 0x564a7ef28be0 2023-07-21T19:24:49.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 10 client.18798 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 10 client.18798 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:24:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 20 client.18798 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef430b0 0x564a7ef454c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 4 client.18798 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:49.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564a7ee73eb0 con 0x564a7ef430b0 2023-07-21T19:24:49.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.453+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef430b0 0x564a7ef454c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.454+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef430b0 0x564a7ef454c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9c1ef9ccfaca87f1 server_cookie=7503ede855e27efb in_seq=0 out_seq=0 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.454+0000 7f575aff5700 10 client.18798 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f575aff5700 1 -- 172.21.15.5:0/2355425397 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564a7ee73eb0 con 0x564a7ef430b0 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f575aff5700 10 client.18798 handle_command_reply: tid=0 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 client.18798 shutdown 2023-07-21T19:24:49.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 msgr2=0x564a7ef39140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f576c00bad0 tx=0x7f576c00bde0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 shutdown_connections 2023-07-21T19:24:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef33ad0 0x564a7ef32a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef430b0 0x564a7ef454c0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef282d0 0x564a7ef38c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2355425397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef28be0 0x564a7ef39140 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 >> 172.21.15.5:0/2355425397 conn(0x564a7ef1eda0 msgr2=0x564a7ef24230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f575aff5700 0 client.18798 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 shutdown_connections 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.455+0000 7f5790df5b80 1 -- 172.21.15.5:0/2355425397 wait complete. 2023-07-21T19:24:49.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.456+0000 7f5790df5b80 20 client.18798 trim_cache size 0 max 16384 2023-07-21T19:24:49.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.469+0000 7f5790df5b80 1 Processor -- start 2023-07-21T19:24:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 -- start start 2023-07-21T19:24:49.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 0x564a7ef31950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef33ad0 0x564a7ef387b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef1e970 con 0x564a7ef282d0 2023-07-21T19:24:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef2e690 con 0x564a7ef33ad0 2023-07-21T19:24:49.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5790df5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef23490 con 0x564a7ef28be0 2023-07-21T19:24:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5776ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43066/0 (socket says 172.21.15.5:43066) 2023-07-21T19:24:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f5776ffd700 1 -- 172.21.15.5:0/3163065433 learned_addr learned my addr 172.21.15.5:0/3163065433 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f57767fc700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 0x564a7ef31950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.470+0000 7f57777fe700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef33ad0 0x564a7ef387b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 -- 172.21.15.5:0/3163065433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 msgr2=0x564a7ef31950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 0x564a7ef31950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 -- 172.21.15.5:0/3163065433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef33ad0 msgr2=0x564a7ef387b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef33ad0 0x564a7ef387b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 -- 172.21.15.5:0/3163065433 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564a7eccf250 con 0x564a7ef282d0 2023-07-21T19:24:49.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f57767fc700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 0x564a7ef31950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f5776ffd700 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f576800c010 tx=0x7f5768009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e400dacf583ae8d8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f575dffb700 1 -- 172.21.15.5:0/3163065433 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57680a3270 con 0x564a7ef282d0 2023-07-21T19:24:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f575dffb700 1 -- 172.21.15.5:0/3163065433 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5768097170 con 0x564a7ef282d0 2023-07-21T19:24:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.471+0000 7f575dffb700 1 -- 172.21.15.5:0/3163065433 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576809b2f0 con 0x564a7ef282d0 2023-07-21T19:24:49.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 -- 172.21.15.5:0/3163065433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 msgr2=0x564a7ef32bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f576800c010 tx=0x7f5768009ca0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 -- 172.21.15.5:0/3163065433 shutdown_connections 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef33ad0 0x564a7ef387b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef282d0 0x564a7ef32bc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 --2- 172.21.15.5:0/3163065433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef28be0 0x564a7ef31950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 -- 172.21.15.5:0/3163065433 >> 172.21.15.5:0/3163065433 conn(0x564a7ef1eda0 msgr2=0x564a7ef31240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 -- 172.21.15.5:0/3163065433 shutdown_connections 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.472+0000 7f5790df5b80 1 -- 172.21.15.5:0/3163065433 wait complete. 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.473+0000 7f5790df5b80 1 -- start start 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.473+0000 7f5790df5b80 1 Processor -- start 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef8f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef8f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef906d0 con 0x564a7ef33ad0 2023-07-21T19:24:49.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef90840 con 0x564a7ef28be0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564a7ef90a00 con 0x564a7ef282d0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5790df5b80 10 client.0 _ll_register_callbacks cb 0x7f576c004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f57777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f57777fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43072/0 (socket says 172.21.15.5:43072) 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f57777fe700 1 -- 172.21.15.5:0/2503245371 learned_addr learned my addr 172.21.15.5:0/2503245371 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f57767fc700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef8f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.474+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef8f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 -- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 msgr2=0x564a7ef8f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef8f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 -- 172.21.15.5:0/2503245371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 msgr2=0x564a7ef8f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef8f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 -- 172.21.15.5:0/2503245371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564a7ec91c60 con 0x564a7ef33ad0 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f57777fe700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5770003c60 tx=0x7f5770007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=749f36352ec4cbd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f575aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:49.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770018ed0 con 0x564a7ef33ad0 2023-07-21T19:24:49.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564a7ee36050 con 0x564a7ef33ad0 2023-07-21T19:24:49.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.475+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57700046d0 con 0x564a7ef33ad0 2023-07-21T19:24:49.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5770011ec0 con 0x564a7ef33ad0 2023-07-21T19:24:49.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 10 client.18804 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:49.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x564a7eca8de0 con 0x564a7ef33ad0 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564a7ee36050 con 0x564a7ef33ad0 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f57700193f0 con 0x564a7ef33ad0 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 10 client.18804 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 10 client.18804 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 20 client.18804 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef98280 0x564a7ef9a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 4 client.18804 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.476+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x564a7ecc9f70 con 0x564a7ef98280 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.477+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef98280 0x564a7ef9a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.477+0000 7f5776ffd700 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef98280 0x564a7ef9a690 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ebeb509b9c48a394 server_cookie=df966c3e9ae2fd0c in_seq=0 out_seq=0 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.477+0000 7f575aff5700 10 client.18804 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f575aff5700 1 -- 172.21.15.5:0/2503245371 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x564a7ecc9f70 con 0x564a7ef98280 2023-07-21T19:24:49.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f575aff5700 10 client.18804 handle_command_reply: tid=0 2023-07-21T19:24:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 client.18804 shutdown 2023-07-21T19:24:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 msgr2=0x564a7ef8ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5770003c60 tx=0x7f5770007950 comp rx=0 tx=0).stop 2023-07-21T19:24:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 shutdown_connections 2023-07-21T19:24:49.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564a7ef33ad0 0x564a7ef8ffc0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564a7ef282d0 0x564a7ef8f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564a7ef98280 0x564a7ef9a690 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 --2- 172.21.15.5:0/2503245371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564a7ef28be0 0x564a7ef8f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f575aff5700 0 client.18804 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 >> 172.21.15.5:0/2503245371 conn(0x564a7ef1eda0 msgr2=0x564a7ef31ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 shutdown_connections 2023-07-21T19:24:49.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 1 -- 172.21.15.5:0/2503245371 wait complete. 2023-07-21T19:24:49.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.478+0000 7f5790df5b80 20 client.18804 trim_cache size 0 max 16384 2023-07-21T19:24:49.478 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:24:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 msgr2=0x7f57600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 0x7f57600a37f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5770007fd0 tx=0x7f5770007e60 comp rx=0 tx=0).stop 2023-07-21T19:24:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 msgr2=0x7f5778197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f5768005910 tx=0x7f5768009580 comp rx=0 tx=0).stop 2023-07-21T19:24:49.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 shutdown_connections 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f577810c320 0x7f577819b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57600a1340 0x7f57600a37f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57780728b0 0x7f5778197ce0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 --2- 172.21.15.5:0/249731602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57780731c0 0x7f5778198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 >> 172.21.15.5:0/249731602 conn(0x7f57780fb5e0 msgr2=0x7f577810e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 shutdown_connections 2023-07-21T19:24:49.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.479+0000 7f577d3ff700 1 -- 172.21.15.5:0/249731602 wait complete. 2023-07-21T19:24:49.495 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:24:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.754+0000 7f5ef699d700 1 -- 172.21.15.5:0/593584866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00ff280 msgr2=0x7f5ef010b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.754+0000 7f5ef699d700 1 --2- 172.21.15.5:0/593584866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00ff280 0x7f5ef010b710 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5ee800b0a0 tx=0x7f5ee800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 -- 172.21.15.5:0/593584866 shutdown_connections 2023-07-21T19:24:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 --2- 172.21.15.5:0/593584866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00ff280 0x7f5ef010b710 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 --2- 172.21.15.5:0/593584866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef01048e0 0x7f5ef0104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 --2- 172.21.15.5:0/593584866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef00fed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 -- 172.21.15.5:0/593584866 >> 172.21.15.5:0/593584866 conn(0x7f5ef00745d0 msgr2=0x7f5ef00749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 -- 172.21.15.5:0/593584866 shutdown_connections 2023-07-21T19:24:49.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.755+0000 7f5ef699d700 1 -- 172.21.15.5:0/593584866 wait complete. 2023-07-21T19:24:49.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.756+0000 7f5ef699d700 1 Processor -- start 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.756+0000 7f5ef699d700 1 -- start start 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.756+0000 7f5ef699d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.756+0000 7f5ef699d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef00ff280 0x7f5ef010c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.756+0000 7f5ef699d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef01048e0 0x7f5ef01a3b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef699d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef010ce50 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef699d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef01a4070 con 0x7f5ef01048e0 2023-07-21T19:24:49.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef699d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef01a41b0 con 0x7f5ef00ff280 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef00ff280 0x7f5ef010c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43096/0 (socket says 172.21.15.5:43096) 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 -- 172.21.15.5:0/265381148 learned_addr learned my addr 172.21.15.5:0/265381148 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef619c700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef01048e0 0x7f5ef01a3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 -- 172.21.15.5:0/265381148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef00ff280 msgr2=0x7f5ef010c620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef00ff280 0x7f5ef010c620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 -- 172.21.15.5:0/265381148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef01048e0 msgr2=0x7f5ef01a3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef01048e0 0x7f5ef01a3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.757+0000 7f5ef599b700 1 -- 172.21.15.5:0/265381148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee8009d00 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ef599b700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e970 tx=0x7f5eec00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ee6ffd700 1 -- 172.21.15.5:0/265381148 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec009790 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ee6ffd700 1 -- 172.21.15.5:0/265381148 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5eec00c7e0 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef01a4410 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ee6ffd700 1 -- 172.21.15.5:0/265381148 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec003bd0 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.758+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef01a4940 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.762+0000 7f5ee6ffd700 1 -- 172.21.15.5:0/265381148 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5eec00c950 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.763+0000 7f5f0b498b80 1 Processor -- start 2023-07-21T19:24:49.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.763+0000 7f5f0b498b80 1 -- start start 2023-07-21T19:24:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ee6ffd700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 0x7f5edc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cd110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a4c5cb0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a518a40 con 0x55aa0a5cad30 2023-07-21T19:24:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a511ed0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ee6ffd700 1 -- 172.21.15.5:0/265381148 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5eec014070 con 0x7f5ef00fe8d0 2023-07-21T19:24:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ef619c700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 0x7f5edc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54326/0 (socket says 172.21.15.5:54326) 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ef599b700 1 -- 172.21.15.5:0/2208542334 learned_addr learned my addr 172.21.15.5:0/2208542334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.764+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef619c700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cd110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 -- 172.21.15.5:0/2208542334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 msgr2=0x55aa0a5c08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 -- 172.21.15.5:0/2208542334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 msgr2=0x55aa0a5cd110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cd110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 -- 172.21.15.5:0/2208542334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0a33fde0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5ca7f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009c80 tx=0x7f5ee0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7898a354f7ed1f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.765+0000 7f5ef619c700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 0x7f5edc0a3890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80062a0 tx=0x7f5ee8009560 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5edbfff700 1 -- 172.21.15.5:0/2208542334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011cb0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5edbfff700 1 -- 172.21.15.5:0/2208542334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee0011e10 con 0x55aa0a5c0e00 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5edbfff700 1 -- 172.21.15.5:0/2208542334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0005020 con 0x55aa0a5c0e00 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2208542334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 msgr2=0x55aa0a5ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5ca7f0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0009c80 tx=0x7f5ee0009f90 comp rx=0 tx=0).stop 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2208542334 shutdown_connections 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cd110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5c08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2208542334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5ca7f0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.766+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2208542334 >> 172.21.15.5:0/2208542334 conn(0x55aa0a5b5ff0 msgr2=0x55aa0a5bb930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.767+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2208542334 shutdown_connections 2023-07-21T19:24:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.767+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2208542334 wait complete. 2023-07-21T19:24:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.767+0000 7f5f0b498b80 1 -- start start 2023-07-21T19:24:49.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 Processor -- start 2023-07-21T19:24:49.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5ca6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5cad30 0x55aa0a5c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a4b1ac0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a4b17c0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a395920 con 0x55aa0a5cad30 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.768+0000 7f5f0b498b80 10 client.0 _ll_register_callbacks cb 0x55aa0a4af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43122/0 (socket says 172.21.15.5:43122) 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 -- 172.21.15.5:0/2903891909 learned_addr learned my addr 172.21.15.5:0/2903891909 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5ca6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef619c700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5cad30 0x55aa0a5c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 -- 172.21.15.5:0/2903891909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5cad30 msgr2=0x55aa0a5c7fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5cad30 0x55aa0a5c7fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 -- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 msgr2=0x55aa0a5ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5ca6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef519a700 1 -- 172.21.15.5:0/2903891909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef01a4940 con 0x55aa0a5c0e00 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.769+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:49.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00047f0 tx=0x7f5ee0005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ad66b8e3b05c258 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0019700 con 0x55aa0a5c0e00 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5ee0019860 con 0x55aa0a5c0e00 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aa0a46b4f0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5ee0019a20 con 0x55aa0a5c0e00 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5f0b498b80 10 client.18816 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:49.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f5ee00098e0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.770+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55aa0a46b4f0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5ee0022b90 con 0x55aa0a5c0e00 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 10 client.18816 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 10 client.18816 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 20 client.18816 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a5da4f0 0x55aa0a5dc900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 4 client.18816 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:49.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aa0a50aeb0 con 0x55aa0a5da4f0 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.771+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a5da4f0 0x55aa0a5dc900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.772+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a5da4f0 0x55aa0a5dc900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=67a32306701625ee server_cookie=8bcad11f31a5aae5 in_seq=0 out_seq=0 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.772+0000 7f5ed97fa700 10 client.18816 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5ed97fa700 1 -- 172.21.15.5:0/2903891909 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55aa0a50aeb0 con 0x55aa0a5da4f0 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5ed97fa700 10 client.18816 handle_command_reply: tid=0 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 client.18816 shutdown 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 msgr2=0x55aa0a5c79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00047f0 tx=0x7f5ee0005ce0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 shutdown_connections 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5cad30 0x55aa0a5c7fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a5da4f0 0x55aa0a5dc900 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5ca6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2903891909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c79b0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 >> 172.21.15.5:0/2903891909 conn(0x55aa0a5b5ff0 msgr2=0x55aa0a5bb520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5ed97fa700 0 client.18816 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:24:49.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 shutdown_connections 2023-07-21T19:24:49.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.773+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2903891909 wait complete. 2023-07-21T19:24:49.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.774+0000 7f5f0b498b80 20 client.18816 trim_cache size 0 max 16384 2023-07-21T19:24:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 Processor -- start 2023-07-21T19:24:49.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 -- start start 2023-07-21T19:24:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5ca2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5c2890 con 0x55aa0a5c0e00 2023-07-21T19:24:49.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5c5fb0 con 0x55aa0a5cad30 2023-07-21T19:24:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.792+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5bcef0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43136/0 (socket says 172.21.15.5:43136) 2023-07-21T19:24:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 -- 172.21.15.5:0/2117125061 learned_addr learned my addr 172.21.15.5:0/2117125061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef599b700 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 -- 172.21.15.5:0/2117125061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 msgr2=0x55aa0a5d1810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 -- 172.21.15.5:0/2117125061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 msgr2=0x55aa0a5ca2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5ca2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.793+0000 7f5ef519a700 1 -- 172.21.15.5:0/2117125061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aa0a328c60 con 0x55aa0a5c0e00 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.794+0000 7f5ef519a700 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004f20 tx=0x7f5ee0011850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed59c5678c13ed9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.794+0000 7f5ee4bff700 1 -- 172.21.15.5:0/2117125061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee00184e0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.794+0000 7f5ee4bff700 1 -- 172.21.15.5:0/2117125061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee0004b60 con 0x55aa0a5c0e00 2023-07-21T19:24:49.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.794+0000 7f5ee4bff700 1 -- 172.21.15.5:0/2117125061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0019960 con 0x55aa0a5c0e00 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2117125061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 msgr2=0x55aa0a5c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f5ee0004f20 tx=0x7f5ee0011850 comp rx=0 tx=0).stop 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2117125061 shutdown_connections 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5ca2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/2117125061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5c2d90 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2117125061 >> 172.21.15.5:0/2117125061 conn(0x55aa0a5b5ff0 msgr2=0x55aa0a5d6a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2117125061 shutdown_connections 2023-07-21T19:24:49.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.795+0000 7f5f0b498b80 1 -- 172.21.15.5:0/2117125061 wait complete. 2023-07-21T19:24:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.796+0000 7f5f0b498b80 1 -- start start 2023-07-21T19:24:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.796+0000 7f5f0b498b80 1 Processor -- start 2023-07-21T19:24:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cf5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5cfcc0 con 0x55aa0a5c0e00 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5cfe30 con 0x55aa0a5cad30 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aa0a5cfff0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.797+0000 7f5f0b498b80 10 client.0 _ll_register_callbacks cb 0x7f5eec005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54352/0 (socket says 172.21.15.5:54352) 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef519a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43140/0 (socket says 172.21.15.5:43140) 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef599b700 1 -- 172.21.15.5:0/570392854 learned_addr learned my addr 172.21.15.5:0/570392854 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef619c700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cf5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef599b700 1 -- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 msgr2=0x55aa0a5cf5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.798+0000 7f5ef599b700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cf5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ef599b700 1 -- 172.21.15.5:0/570392854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 msgr2=0x55aa0a5d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:49.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ef599b700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ef599b700 1 -- 172.21.15.5:0/570392854 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ee00098e0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ef519a700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ef599b700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0963b0 tx=0x7f5eec0092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=483045791182509f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ed97fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eec09dee0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5eec0a2300 con 0x55aa0a5c04f0 2023-07-21T19:24:49.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5ee0011e10 con 0x55aa0a5c04f0 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.799+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5eec0832c0 con 0x55aa0a5c04f0 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 10 client.28396 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55aa0a366250 con 0x55aa0a5c04f0 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5ee0011e10 con 0x55aa0a5c04f0 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5eec09a110 con 0x55aa0a5c04f0 2023-07-21T19:24:49.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 10 client.28396 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 10 client.28396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 20 client.28396 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a62f440 0x55aa0a631850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 4 client.28396 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.800+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55aa0a4cd050 con 0x55aa0a62f440 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.801+0000 7f5ef519a700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a62f440 0x55aa0a631850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:49.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.801+0000 7f5ef519a700 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a62f440 0x55aa0a631850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ffe1748155cdca8a server_cookie=9cb8a9d0cc4975b7 in_seq=0 out_seq=0 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.801+0000 7f5ed97fa700 10 client.28396 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5ed97fa700 1 -- 172.21.15.5:0/570392854 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55aa0a4cd050 con 0x55aa0a62f440 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5ed97fa700 10 client.28396 handle_command_reply: tid=0 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 client.28396 shutdown 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 msgr2=0x55aa0a5d1160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f5eec0963b0 tx=0x7f5eec0092d0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 shutdown_connections 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55aa0a62f440 0x55aa0a631850 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55aa0a5cad30 0x55aa0a5cf5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55aa0a5c04f0 0x55aa0a5d1160 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5ed97fa700 0 client.28396 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 --2- 172.21.15.5:0/570392854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55aa0a5c0e00 0x55aa0a5d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 >> 172.21.15.5:0/570392854 conn(0x55aa0a5b5ff0 msgr2=0x55aa0a5dc500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 shutdown_connections 2023-07-21T19:24:49.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.802+0000 7f5f0b498b80 1 -- 172.21.15.5:0/570392854 wait complete. 2023-07-21T19:24:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5f0b498b80 20 client.28396 trim_cache size 0 max 16384 2023-07-21T19:24:49.798 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:24:49.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 msgr2=0x7f5edc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 0x7f5edc0a3890 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f5ee80062a0 tx=0x7f5ee8009560 comp rx=0 tx=0).stop 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 msgr2=0x7f5ef010c0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7f5eec00e970 tx=0x7f5eec00ec80 comp rx=0 tx=0).stop 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 shutdown_connections 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ef01048e0 0x7f5ef01a3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5edc0a13e0 0x7f5edc0a3890 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ef00fe8d0 0x7f5ef010c0e0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 --2- 172.21.15.5:0/265381148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ef00ff280 0x7f5ef010c620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.803+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 >> 172.21.15.5:0/265381148 conn(0x7f5ef00745d0 msgr2=0x7f5ef00fce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.804+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 shutdown_connections 2023-07-21T19:24:49.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:49.804+0000 7f5ef699d700 1 -- 172.21.15.5:0/265381148 wait complete. 2023-07-21T19:24:49.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:49 smithi005 ceph-mon[126460]: pgmap v541: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 30 MiB/s wr, 32 op/s 2023-07-21T19:24:49.813 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:49 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2681633411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:24:49.815 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:24:49.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:49 smithi156 ceph-mon[128316]: pgmap v541: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 30 MiB/s wr, 32 op/s 2023-07-21T19:24:49.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:49 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2681633411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:24:50.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 -- 172.21.15.5:0/2055385790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 msgr2=0x7fb0ec10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/2055385790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec10c910 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb0e400b0a0 tx=0x7fb0e400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 -- 172.21.15.5:0/2055385790 shutdown_connections 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/2055385790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec10c910 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/2055385790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/2055385790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 0x7fb0ec0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 -- 172.21.15.5:0/2055385790 >> 172.21.15.5:0/2055385790 conn(0x7fb0ec0fb5f0 msgr2=0x7fb0ec0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 -- 172.21.15.5:0/2055385790 shutdown_connections 2023-07-21T19:24:50.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.051+0000 7fb0f27ff700 1 -- 172.21.15.5:0/2055385790 wait complete. 2023-07-21T19:24:50.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.052+0000 7fb0f27ff700 1 Processor -- start 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.052+0000 7fb0f27ff700 1 -- start start 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 0x7fb0ec197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec19c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec19cd70 con 0x7fb0ec100480 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec19cee0 con 0x7fb0ec105ae0 2023-07-21T19:24:50.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0f27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec19d0a0 con 0x7fb0ec0ffad0 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43164/0 (socket says 172.21.15.5:43164) 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/3060504213 learned_addr learned my addr 172.21.15.5:0/3060504213 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.053+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 0x7fb0ec197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec19c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/3060504213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 msgr2=0x7fb0ec197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 0x7fb0ec197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/3060504213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 msgr2=0x7fb0ec19c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec19c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/3060504213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009d00 con 0x7fb0ec100480 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9a0 tx=0x7fb0e000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0e97fa700 1 -- 172.21.15.5:0/3060504213 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0009790 con 0x7fb0ec100480 2023-07-21T19:24:50.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0e97fa700 1 -- 172.21.15.5:0/3060504213 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0e000cbd0 con 0x7fb0ec100480 2023-07-21T19:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec19d380 con 0x7fb0ec100480 2023-07-21T19:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.054+0000 7fb0e97fa700 1 -- 172.21.15.5:0/3060504213 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0018680 con 0x7fb0ec100480 2023-07-21T19:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.055+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec1a4bc0 con 0x7fb0ec100480 2023-07-21T19:24:50.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.058+0000 7fb0e97fa700 1 -- 172.21.15.5:0/3060504213 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb0e000cd40 con 0x7fb0ec100480 2023-07-21T19:24:50.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.059+0000 7fb106154b80 1 Processor -- start 2023-07-21T19:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 -- start start 2023-07-21T19:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea06a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931feab590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fda6cb0 con 0x55931feabad0 2023-07-21T19:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fdf9a40 con 0x55931fea02d0 2023-07-21T19:24:50.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb106154b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fdf2ed0 con 0x55931fea0be0 2023-07-21T19:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.060+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931feab590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea06a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59558/0 (socket says 172.21.15.5:59558) 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0f0d9c700 1 -- 172.21.15.5:0/1318270291 learned_addr learned my addr 172.21.15.5:0/1318270291 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0e97fa700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 0x7fb0d40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1318270291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 msgr2=0x55931fea06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1318270291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 msgr2=0x55931feadeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feadeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1318270291 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931fc20de0 con 0x55931fea0be0 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0e97fa700 1 -- 172.21.15.5:0/3060504213 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb0e0014070 con 0x7fb0ec100480 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 0x7fb0d40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.061+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931feab590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009cb0 tx=0x7fb0dc0058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9ab5af02b1158a1b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.062+0000 7fb0d27fc700 1 -- 172.21.15.5:0/1318270291 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011c80 con 0x55931fea0be0 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.062+0000 7fb0d27fc700 1 -- 172.21.15.5:0/1318270291 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0dc011de0 con 0x55931fea0be0 2023-07-21T19:24:50.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.062+0000 7fb0d27fc700 1 -- 172.21.15.5:0/1318270291 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc005020 con 0x55931fea0be0 2023-07-21T19:24:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 -- 172.21.15.5:0/1318270291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 msgr2=0x55931feab590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931feab590 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc009cb0 tx=0x7fb0dc0058e0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 0x7fb0d40a3770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009850 tx=0x7fb0e40099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 -- 172.21.15.5:0/1318270291 shutdown_connections 2023-07-21T19:24:50.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feadeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 --2- 172.21.15.5:0/1318270291 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931feab590 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 -- 172.21.15.5:0/1318270291 >> 172.21.15.5:0/1318270291 conn(0x55931fe96da0 msgr2=0x55931fe9c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 -- 172.21.15.5:0/1318270291 shutdown_connections 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.063+0000 7fb106154b80 1 -- 172.21.15.5:0/1318270291 wait complete. 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.064+0000 7fb106154b80 1 -- start start 2023-07-21T19:24:50.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.064+0000 7fb106154b80 1 Processor -- start 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.064+0000 7fb106154b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931fea9260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fd92ac0 con 0x55931feabad0 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fd927c0 con 0x55931fea02d0 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fc76920 con 0x55931fea0be0 2023-07-21T19:24:50.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb106154b80 10 client.0 _ll_register_callbacks cb 0x55931fd90760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931fea9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54386/0 (socket says 172.21.15.5:54386) 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/1937036666 learned_addr learned my addr 172.21.15.5:0/1937036666 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931fea9260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43182/0 (socket says 172.21.15.5:43182) 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 msgr2=0x55931fea87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/1937036666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 msgr2=0x55931fea9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.065+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931fea9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0eb7fe700 1 -- 172.21.15.5:0/1937036666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc0098e0 con 0x55931fea0be0 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e970 tx=0x7fb0e00cac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd3229571bab2d2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:24:50.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e009a090 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb0e009e0c0 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55931fd4c4f0 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e00a24e0 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb106154b80 10 client.28408 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.066+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55931fc09c60 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0e00b6020 con 0x55931fea0be0 2023-07-21T19:24:50.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb0e0096040 con 0x55931fea0be0 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 10 client.28408 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 10 client.28408 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 20 client.28408 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931febb520 0x55931febd930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 4 client.28408 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:50.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.067+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55931fdebeb0 con 0x55931febb520 2023-07-21T19:24:50.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.068+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931febb520 0x55931febd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.068+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931febb520 0x55931febd930 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cc4249a476dbd90f server_cookie=32e82e4686cc2b65 in_seq=0 out_seq=0 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.068+0000 7fb0cb7fe700 10 client.28408 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/1937036666 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55931fdebeb0 con 0x55931febb520 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb0cb7fe700 10 client.28408 handle_command_reply: tid=0 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 client.28408 shutdown 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 msgr2=0x55931fea8d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e970 tx=0x7fb0e00cac70 comp rx=0 tx=0).stop 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 shutdown_connections 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931febb520 0x55931febd930 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931fea9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea02d0 0x55931fea87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 --2- 172.21.15.5:0/1937036666 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea8d20 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 >> 172.21.15.5:0/1937036666 conn(0x55931fe96da0 msgr2=0x55931fe9c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.069+0000 7fb0cb7fe700 0 client.28408 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.070+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 shutdown_connections 2023-07-21T19:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.070+0000 7fb106154b80 1 -- 172.21.15.5:0/1937036666 wait complete. 2023-07-21T19:24:50.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.070+0000 7fb106154b80 20 client.28408 trim_cache size 0 max 16384 2023-07-21T19:24:50.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:49 smithi101 ceph-mon[128136]: pgmap v541: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 30 MiB/s wr, 32 op/s 2023-07-21T19:24:50.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:49 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2681633411' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 Processor -- start 2023-07-21T19:24:50.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 -- start start 2023-07-21T19:24:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931feabad0 0x55931fea4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fea1540 con 0x55931fea02d0 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fea21a0 con 0x55931feabad0 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.085+0000 7fb106154b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931fea11c0 con 0x55931fea0be0 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43198/0 (socket says 172.21.15.5:43198) 2023-07-21T19:24:50.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1680283832 learned_addr learned my addr 172.21.15.5:0/1680283832 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931feabad0 0x55931fea4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1680283832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 msgr2=0x55931fea3b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea3b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1680283832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931feabad0 msgr2=0x55931fea4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931feabad0 0x55931fea4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.086+0000 7fb0ebfff700 1 -- 172.21.15.5:0/1680283832 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55931fc47250 con 0x55931fea02d0 2023-07-21T19:24:50.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.087+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0097c0 tx=0x7fb0dc011820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f73653bd4291fcfe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.087+0000 7fb0d2ffd700 1 -- 172.21.15.5:0/1680283832 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc011b00 con 0x55931fea02d0 2023-07-21T19:24:50.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.087+0000 7fb0d2ffd700 1 -- 172.21.15.5:0/1680283832 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb0dc011c60 con 0x55931fea02d0 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.087+0000 7fb0d2ffd700 1 -- 172.21.15.5:0/1680283832 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc003e40 con 0x55931fea02d0 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- 172.21.15.5:0/1680283832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 msgr2=0x55931feb24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc0097c0 tx=0x7fb0dc011820 comp rx=0 tx=0).stop 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- 172.21.15.5:0/1680283832 shutdown_connections 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931feabad0 0x55931fea4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931fea02d0 0x55931feb24e0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 --2- 172.21.15.5:0/1680283832 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea0be0 0x55931fea3b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- 172.21.15.5:0/1680283832 >> 172.21.15.5:0/1680283832 conn(0x55931fe96da0 msgr2=0x55931fec2290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- 172.21.15.5:0/1680283832 shutdown_connections 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- 172.21.15.5:0/1680283832 wait complete. 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.088+0000 7fb106154b80 1 -- start start 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 Processor -- start 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea02d0 0x55931feb2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 0x55931feb08e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931feb14f0 con 0x55931feabad0 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931feb1660 con 0x55931fea0be0 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55931ff0f3f0 con 0x55931fea02d0 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb106154b80 10 client.0 _ll_register_callbacks cb 0x7fb0e0099e80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.089+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43212/0 (socket says 172.21.15.5:43212) 2023-07-21T19:24:50.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 -- 172.21.15.5:0/2127402786 learned_addr learned my addr 172.21.15.5:0/2127402786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 0x55931feb08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea02d0 0x55931feb2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 -- 172.21.15.5:0/2127402786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea02d0 msgr2=0x55931feb2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea02d0 0x55931feb2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 -- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 msgr2=0x55931feb08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 0x55931feb08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 -- 172.21.15.5:0/2127402786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0dc01d040 con 0x55931feabad0 2023-07-21T19:24:50.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0eb7fe700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 0x55931feb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:50.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0f0d9c700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4015000 tx=0x7fb0e400b3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7066a28d02dadf4c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.090+0000 7fb0cb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e402e070 con 0x55931feabad0 2023-07-21T19:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55931fdae050 con 0x55931feabad0 2023-07-21T19:24:50.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb0e4005040 con 0x55931feabad0 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0e400e040 con 0x55931feabad0 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb106154b80 10 client.18840 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55931fc1f2b0 con 0x55931feabad0 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55931fdae050 con 0x55931feabad0 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.091+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fb0e4039070 con 0x55931feabad0 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 10 client.18840 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:50.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 10 client.18840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 20 client.18840 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931ff10a80 0x55931ff12e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 4 client.18840 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55931fc41f70 con 0x55931ff10a80 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.092+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931ff10a80 0x55931ff12e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb0ebfff700 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931ff10a80 0x55931ff12e90 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69a3856eb68dd20b server_cookie=d45d4d5a2190d456 in_seq=0 out_seq=0 2023-07-21T19:24:50.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb0cb7fe700 10 client.18840 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb0cb7fe700 1 -- 172.21.15.5:0/2127402786 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55931fc41f70 con 0x55931ff10a80 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb0cb7fe700 10 client.18840 handle_command_reply: tid=0 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 client.18840 shutdown 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 msgr2=0x55931feb0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4015000 tx=0x7fb0e400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 shutdown_connections 2023-07-21T19:24:50.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55931ff10a80 0x55931ff12e90 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55931feabad0 0x55931feb0e20 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55931fea02d0 0x55931feb2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 --2- 172.21.15.5:0/2127402786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55931fea0be0 0x55931feb08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb0cb7fe700 0 client.18840 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.093+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 >> 172.21.15.5:0/2127402786 conn(0x55931fe96da0 msgr2=0x55931fea3b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 shutdown_connections 2023-07-21T19:24:50.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb106154b80 1 -- 172.21.15.5:0/2127402786 wait complete. 2023-07-21T19:24:50.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb106154b80 20 client.18840 trim_cache size 0 max 16384 2023-07-21T19:24:50.092 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 msgr2=0x7fb0d40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 0x7fb0d40a3770 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb0e4009850 tx=0x7fb0e40099e0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 msgr2=0x7fb0ec198320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.094+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb0e000e9a0 tx=0x7fb0e000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 shutdown_connections 2023-07-21T19:24:50.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb0ec105ae0 0x7fb0ec19c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb0ec0ffad0 0x7fb0ec197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb0d40a12c0 0x7fb0d40a3770 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 --2- 172.21.15.5:0/3060504213 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb0ec100480 0x7fb0ec198320 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 >> 172.21.15.5:0/3060504213 conn(0x7fb0ec0fb5f0 msgr2=0x7fb0ec0fbbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 shutdown_connections 2023-07-21T19:24:50.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:50.095+0000 7fb0f27ff700 1 -- 172.21.15.5:0/3060504213 wait complete. 2023-07-21T19:24:51.104 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:24:51.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:24:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 -- 172.21.15.5:0/1763089506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14100b60 msgr2=0x7f3e14100f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 --2- 172.21.15.5:0/1763089506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14100b60 0x7f3e14100f30 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009990 tx=0x7f3e04009ca0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 -- 172.21.15.5:0/1763089506 shutdown_connections 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 --2- 172.21.15.5:0/1763089506 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e1410a080 0x7f3e1410c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 --2- 172.21.15.5:0/1763089506 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e14100b60 0x7f3e14100f30 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 --2- 172.21.15.5:0/1763089506 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14101470 0x7f3e14109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 -- 172.21.15.5:0/1763089506 >> 172.21.15.5:0/1763089506 conn(0x7f3e140fa4b0 msgr2=0x7f3e140fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 -- 172.21.15.5:0/1763089506 shutdown_connections 2023-07-21T19:24:51.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.364+0000 7f3e199ff700 1 -- 172.21.15.5:0/1763089506 wait complete. 2023-07-21T19:24:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.365+0000 7f3e199ff700 1 Processor -- start 2023-07-21T19:24:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.365+0000 7f3e199ff700 1 -- start start 2023-07-21T19:24:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14100b60 0x7f3e140fe990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e14101470 0x7f3e140feed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e140ff700 con 0x7f3e1410a080 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141a4260 con 0x7f3e14100b60 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e199ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e141a43a0 con 0x7f3e14101470 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43232/0 (socket says 172.21.15.5:43232) 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e12ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14100b60 0x7f3e140fe990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 -- 172.21.15.5:0/2089493017 learned_addr learned my addr 172.21.15.5:0/2089493017 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e127fc700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e14101470 0x7f3e140feed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 -- 172.21.15.5:0/2089493017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e14101470 msgr2=0x7f3e140feed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e14101470 0x7f3e140feed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 -- 172.21.15.5:0/2089493017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14100b60 msgr2=0x7f3e140fe990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14100b60 0x7f3e140fe990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.366+0000 7f3e137fe700 1 -- 172.21.15.5:0/2089493017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e040095f0 con 0x7f3e1410a080 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e137fe700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c007ee0 tx=0x7f3e0c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e189fd700 1 -- 172.21.15.5:0/2089493017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c017070 con 0x7f3e1410a080 2023-07-21T19:24:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e189fd700 1 -- 172.21.15.5:0/2089493017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e0c00f040 con 0x7f3e1410a080 2023-07-21T19:24:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e141a4600 con 0x7f3e1410a080 2023-07-21T19:24:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e189fd700 1 -- 172.21.15.5:0/2089493017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c005570 con 0x7f3e1410a080 2023-07-21T19:24:51.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.367+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e141a4b30 con 0x7f3e1410a080 2023-07-21T19:24:51.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.372+0000 7f3e189fd700 1 -- 172.21.15.5:0/2089493017 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3e0c00db20 con 0x7f3e1410a080 2023-07-21T19:24:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.373+0000 7f3e2d23ab80 1 Processor -- start 2023-07-21T19:24:51.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.373+0000 7f3e2d23ab80 1 -- start start 2023-07-21T19:24:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b78c720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b799f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b692d30 con 0x56361b78c350 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b6e5ac0 con 0x56361b78cc60 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b6def50 con 0x56361b797b50 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b78c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e12ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e12ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59622/0 (socket says 172.21.15.5:59622) 2023-07-21T19:24:51.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e12ffd700 1 -- 172.21.15.5:0/1652358885 learned_addr learned my addr 172.21.15.5:0/1652358885 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b799f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.374+0000 7f3e189fd700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 0x7f3dfc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e189fd700 1 -- 172.21.15.5:0/2089493017 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3e0c0cc3b0 con 0x7f3e1410a080 2023-07-21T19:24:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e137fe700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 0x7f3dfc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 -- 172.21.15.5:0/1652358885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 msgr2=0x56361b799f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b799f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 -- 172.21.15.5:0/1652358885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 msgr2=0x56361b797610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 -- 172.21.15.5:0/1652358885 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361b50ce60 con 0x56361b78c350 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.375+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b78c720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005950 tx=0x7f3e080058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5bc30d5d57aeb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3e137fe700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 0x7f3dfc0a37f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0993b0 tx=0x7f3e0c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3df9ffb700 1 -- 172.21.15.5:0/1652358885 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e08004050 con 0x56361b78c350 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3df9ffb700 1 -- 172.21.15.5:0/1652358885 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e08004830 con 0x56361b78c350 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3df9ffb700 1 -- 172.21.15.5:0/1652358885 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e080052f0 con 0x56361b78c350 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1652358885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 msgr2=0x56361b78c720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.376+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b78c720 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005950 tx=0x7f3e080058e0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1652358885 shutdown_connections 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b799f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b78c720 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1652358885 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b797610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1652358885 >> 172.21.15.5:0/1652358885 conn(0x56361b782e20 msgr2=0x56361b788720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1652358885 shutdown_connections 2023-07-21T19:24:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1652358885 wait complete. 2023-07-21T19:24:51.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.377+0000 7f3e2d23ab80 1 -- start start 2023-07-21T19:24:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 Processor -- start 2023-07-21T19:24:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b797610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b794820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b794f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b67eb40 con 0x56361b78cc60 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b67e840 con 0x56361b78c350 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b5629a0 con 0x56361b797b50 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.378+0000 7f3e2d23ab80 10 client.0 _ll_register_callbacks cb 0x56361b67c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b797610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b797610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59630/0 (socket says 172.21.15.5:59630) 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e127fc700 1 -- 172.21.15.5:0/843990282 learned_addr learned my addr 172.21.15.5:0/843990282 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b794820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e127fc700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b794f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 -- 172.21.15.5:0/843990282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 msgr2=0x56361b794f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b794f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 -- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 msgr2=0x56361b797610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b797610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.379+0000 7f3e12ffd700 1 -- 172.21.15.5:0/843990282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e1410d1d0 con 0x56361b78cc60 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b794820 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009990 tx=0x7f3e04005fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caaaa2ba669bd9c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0401d070 con 0x56361b78cc60 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56361b638570 con 0x56361b78cc60 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3e04004500 con 0x56361b78cc60 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e040046c0 con 0x56361b78cc60 2023-07-21T19:24:51.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3e2d23ab80 10 client.18852 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x7f3e08019040 con 0x56361b78cc60 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.380+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56361b638570 con 0x56361b78cc60 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3e0400f7c0 con 0x56361b78cc60 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 10 client.18852 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 10 client.18852 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 20 client.18852 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7a7440 0x56361b7a9850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 4 client.18852 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.381+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56361b6d7f30 con 0x56361b7a7440 2023-07-21T19:24:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.382+0000 7f3e127fc700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7a7440 0x56361b7a9850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.382+0000 7f3df2ffd700 10 client.18852 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.382+0000 7f3e127fc700 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7a7440 0x56361b7a9850 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c298e36e7cbaca0 server_cookie=2721fb91b391aa3 in_seq=0 out_seq=0 2023-07-21T19:24:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3df2ffd700 1 -- 172.21.15.5:0/843990282 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56361b6d7f30 con 0x56361b7a7440 2023-07-21T19:24:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3df2ffd700 10 client.18852 handle_command_reply: tid=0 2023-07-21T19:24:51.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 client.18852 shutdown 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 msgr2=0x56361b794820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b794820 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f3e04009990 tx=0x7f3e04005fb0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 shutdown_connections 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b794f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b797610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7a7440 0x56361b7a9850 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/843990282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b794820 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 >> 172.21.15.5:0/843990282 conn(0x56361b782e20 msgr2=0x56361b7882b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.383+0000 7f3df2ffd700 0 client.18852 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.384+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 shutdown_connections 2023-07-21T19:24:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.384+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/843990282 wait complete. 2023-07-21T19:24:51.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.384+0000 7f3e2d23ab80 20 client.18852 trim_cache size 0 max 16384 2023-07-21T19:24:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 Processor -- start 2023-07-21T19:24:51.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 -- start start 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b79e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b788250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b78d5c0 con 0x56361b78cc60 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b789d80 con 0x56361b78c350 2023-07-21T19:24:51.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.405+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b7a3e80 con 0x56361b797b50 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43256/0 (socket says 172.21.15.5:43256) 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 -- 172.21.15.5:0/1886184664 learned_addr learned my addr 172.21.15.5:0/1886184664 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b79e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b788250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 -- 172.21.15.5:0/1886184664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 msgr2=0x56361b788250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b788250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 -- 172.21.15.5:0/1886184664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 msgr2=0x56361b79e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b79e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 -- 172.21.15.5:0/1886184664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56361b784660 con 0x56361b78cc60 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e127fc700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b788250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.406+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400fba0 tx=0x7f3e04009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c3669a69fe13a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.407+0000 7f3dfa7fc700 1 -- 172.21.15.5:0/1886184664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0401d070 con 0x56361b78cc60 2023-07-21T19:24:51.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.407+0000 7f3dfa7fc700 1 -- 172.21.15.5:0/1886184664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e0401e040 con 0x56361b78cc60 2023-07-21T19:24:51.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.407+0000 7f3dfa7fc700 1 -- 172.21.15.5:0/1886184664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400e9e0 con 0x56361b78cc60 2023-07-21T19:24:51.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.407+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1886184664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 msgr2=0x56361b797140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.407+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f3e0400fba0 tx=0x7f3e04009290 comp rx=0 tx=0).stop 2023-07-21T19:24:51.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1886184664 shutdown_connections 2023-07-21T19:24:51.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b788250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78c350 0x56361b79e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/1886184664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78cc60 0x56361b797140 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1886184664 >> 172.21.15.5:0/1886184664 conn(0x56361b782e20 msgr2=0x56361b78fb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1886184664 shutdown_connections 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/1886184664 wait complete. 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 -- start start 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.408+0000 7f3e2d23ab80 1 Processor -- start 2023-07-21T19:24:51.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b79d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b79d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b7fb350 con 0x56361b78c350 2023-07-21T19:24:51.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b7fb490 con 0x56361b78cc60 2023-07-21T19:24:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56361b7fb650 con 0x56361b797b50 2023-07-21T19:24:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e2d23ab80 10 client.0 _ll_register_callbacks cb 0x7f3e04005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e127fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43258/0 (socket says 172.21.15.5:43258) 2023-07-21T19:24:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e127fc700 1 -- 172.21.15.5:0/3160694864 learned_addr learned my addr 172.21.15.5:0/3160694864 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.409+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b79d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b79d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 -- 172.21.15.5:0/3160694864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 msgr2=0x56361b79d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b79d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 -- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 msgr2=0x56361b79d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b79d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 -- 172.21.15.5:0/3160694864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e08019040 con 0x56361b78c350 2023-07-21T19:24:51.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e127fc700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005950 tx=0x7f3e080058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29912e634ee90fe server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3df2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:51.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0802fa10 con 0x56361b78c350 2023-07-21T19:24:51.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3e0802fb70 con 0x56361b78c350 2023-07-21T19:24:51.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.410+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3e08004830 con 0x56361b78c350 2023-07-21T19:24:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3e0802fd30 con 0x56361b78c350 2023-07-21T19:24:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 10 client.18858 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x56361b4f5ce0 con 0x56361b78c350 2023-07-21T19:24:51.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3e0803e080 con 0x56361b78c350 2023-07-21T19:24:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3e0801d070 con 0x56361b78c350 2023-07-21T19:24:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 10 client.18858 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 10 client.18858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 20 client.18858 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7fcc10 0x56361b7feff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 4 client.18858 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:24:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.411+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56361b69a0d0 con 0x56361b7fcc10 2023-07-21T19:24:51.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.412+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7fcc10 0x56361b7feff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.412+0000 7f3e12ffd700 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7fcc10 0x56361b7feff0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65d39c52382ad665 server_cookie=3f81207deef4a2dc in_seq=0 out_seq=0 2023-07-21T19:24:51.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.412+0000 7f3df2ffd700 10 client.18858 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.413+0000 7f3df2ffd700 1 -- 172.21.15.5:0/3160694864 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56361b69a0d0 con 0x56361b7fcc10 2023-07-21T19:24:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.413+0000 7f3df2ffd700 10 client.18858 handle_command_reply: tid=0 2023-07-21T19:24:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.413+0000 7f3e2d23ab80 1 client.18858 shutdown 2023-07-21T19:24:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 msgr2=0x56361b79cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005950 tx=0x7f3e080058e0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 shutdown_connections 2023-07-21T19:24:51.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56361b797b50 0x56361b79d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56361b7fcc10 0x56361b7feff0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56361b78c350 0x56361b79cc70 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 --2- 172.21.15.5:0/3160694864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56361b78cc60 0x56361b79d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3df2ffd700 0 client.18858 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.414+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 >> 172.21.15.5:0/3160694864 conn(0x56361b782e20 msgr2=0x56361b7881e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.415+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 shutdown_connections 2023-07-21T19:24:51.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.415+0000 7f3e2d23ab80 1 -- 172.21.15.5:0/3160694864 wait complete. 2023-07-21T19:24:51.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.416+0000 7f3e2d23ab80 20 client.18858 trim_cache size 0 max 16384 2023-07-21T19:24:51.417 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:24:51.418 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:24:51.418 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "9e7de850-d1ef-45ab-a9bc-f76a37f2a9df", 2023-07-21T19:24:51.418 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:24:51.418 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:24:51.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 msgr2=0x7f3dfc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 0x7f3dfc0a37f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c0993b0 tx=0x7f3e0c00b040 comp rx=0 tx=0).stop 2023-07-21T19:24:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 msgr2=0x7f3e141a3d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c007ee0 tx=0x7f3e0c00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 shutdown_connections 2023-07-21T19:24:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3dfc0a1340 0x7f3dfc0a37f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3e1410a080 0x7f3e141a3d20 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3e14100b60 0x7f3e140fe990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 --2- 172.21.15.5:0/2089493017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3e14101470 0x7f3e140feed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.417+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 >> 172.21.15.5:0/2089493017 conn(0x7f3e140fa4b0 msgr2=0x7f3e140fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.418+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 shutdown_connections 2023-07-21T19:24:51.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.418+0000 7f3e199ff700 1 -- 172.21.15.5:0/2089493017 wait complete. 2023-07-21T19:24:51.432 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:24:51.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 -- 172.21.15.5:0/1881097597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0728b0 msgr2=0x7f351c072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 --2- 172.21.15.5:0/1881097597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0728b0 0x7f351c072c80 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f35180098f0 tx=0x7f3518009c00 comp rx=0 tx=0).stop 2023-07-21T19:24:51.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 -- 172.21.15.5:0/1881097597 shutdown_connections 2023-07-21T19:24:51.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 --2- 172.21.15.5:0/1881097597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c10c320 0x7f351c10e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 --2- 172.21.15.5:0/1881097597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0728b0 0x7f351c072c80 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 --2- 172.21.15.5:0/1881097597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c0731c0 0x7f351c10bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 -- 172.21.15.5:0/1881097597 >> 172.21.15.5:0/1881097597 conn(0x7f351c0fb5e0 msgr2=0x7f351c0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 -- 172.21.15.5:0/1881097597 shutdown_connections 2023-07-21T19:24:51.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.684+0000 7f3523bff700 1 -- 172.21.15.5:0/1881097597 wait complete. 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 Processor -- start 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 -- start start 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0731c0 0x7f351c1983a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c10c320 0x7f351c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c19ce40 con 0x7f351c0728b0 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c19cfb0 con 0x7f351c0731c0 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.685+0000 7f3523bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351c19d170 con 0x7f351c10c320 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0731c0 0x7f351c1983a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43276/0 (socket says 172.21.15.5:43276) 2023-07-21T19:24:51.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 -- 172.21.15.5:0/607766346 learned_addr learned my addr 172.21.15.5:0/607766346 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352219c700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c10c320 0x7f351c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 -- 172.21.15.5:0/607766346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c10c320 msgr2=0x7f351c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c10c320 0x7f351c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 -- 172.21.15.5:0/607766346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0731c0 msgr2=0x7f351c1983a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0731c0 0x7f351c1983a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 -- 172.21.15.5:0/607766346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518009520 con 0x7f351c0728b0 2023-07-21T19:24:51.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.686+0000 7f352199b700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3518009810 tx=0x7f351800fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.687+0000 7f3512ffd700 1 -- 172.21.15.5:0/607766346 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351801c070 con 0x7f351c0728b0 2023-07-21T19:24:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.687+0000 7f3512ffd700 1 -- 172.21.15.5:0/607766346 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f351800baf0 con 0x7f351c0728b0 2023-07-21T19:24:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.687+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351c19d3f0 con 0x7f351c0728b0 2023-07-21T19:24:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.687+0000 7f3512ffd700 1 -- 172.21.15.5:0/607766346 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3518004930 con 0x7f351c0728b0 2023-07-21T19:24:51.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.687+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351c1a4c90 con 0x7f351c0728b0 2023-07-21T19:24:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.690+0000 7f3512ffd700 1 -- 172.21.15.5:0/607766346 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3518020070 con 0x7f351c0728b0 2023-07-21T19:24:51.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 Processor -- start 2023-07-21T19:24:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f3512ffd700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 0x7f35080a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 -- start start 2023-07-21T19:24:51.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f3512ffd700 1 -- 172.21.15.5:0/607766346 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f35180cda20 con 0x7f351c0728b0 2023-07-21T19:24:51.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c5690876a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f352219c700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 0x7f35080a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569092ad0 0x55c569094eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568f8dcb0 con 0x55c569087be0 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568fe0a40 con 0x55c569092ad0 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.692+0000 7f353757ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568fd9ed0 con 0x55c5690872d0 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43290/0 (socket says 172.21.15.5:43290) 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 -- 172.21.15.5:0/2178025249 learned_addr learned my addr 172.21.15.5:0/2178025249 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352199b700 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c5690876a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352219c700 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569092ad0 0x55c569094eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 -- 172.21.15.5:0/2178025249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 msgr2=0x55c5690876a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c5690876a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 -- 172.21.15.5:0/2178025249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569092ad0 msgr2=0x55c569094eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569092ad0 0x55c569094eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 -- 172.21.15.5:0/2178025249 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c568e07de0 con 0x55c569087be0 2023-07-21T19:24:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352219c700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 0x7f35080a3770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f351400f540 tx=0x7f35140099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.693+0000 7f352119a700 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f350c009d20 tx=0x7f350c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b031fb1281efccc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.694+0000 7f3507fff700 1 -- 172.21.15.5:0/2178025249 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c019630 con 0x55c569087be0 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.694+0000 7f3507fff700 1 -- 172.21.15.5:0/2178025249 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350c019c20 con 0x55c569087be0 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.694+0000 7f3507fff700 1 -- 172.21.15.5:0/2178025249 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350c00eb60 con 0x55c569087be0 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.694+0000 7f353757ab80 1 -- 172.21.15.5:0/2178025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 msgr2=0x55c569092590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.694+0000 7f353757ab80 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f350c009d20 tx=0x7f350c011720 comp rx=0 tx=0).stop 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 -- 172.21.15.5:0/2178025249 shutdown_connections 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569092ad0 0x55c569094eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c5690876a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 --2- 172.21.15.5:0/2178025249 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569087be0 0x55c569092590 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 -- 172.21.15.5:0/2178025249 >> 172.21.15.5:0/2178025249 conn(0x55c56907dda0 msgr2=0x55c5690836a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 -- 172.21.15.5:0/2178025249 shutdown_connections 2023-07-21T19:24:51.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 -- 172.21.15.5:0/2178025249 wait complete. 2023-07-21T19:24:51.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 -- start start 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.695+0000 7f353757ab80 1 Processor -- start 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 0x55c569092500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569087be0 0x55c56908f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568f79ac0 con 0x55c569092ad0 2023-07-21T19:24:51.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568f797c0 con 0x55c5690872d0 2023-07-21T19:24:51.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c568e5d920 con 0x55c569087be0 2023-07-21T19:24:51.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f353757ab80 10 client.0 _ll_register_callbacks cb 0x55c568f77760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f352219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f352219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43292/0 (socket says 172.21.15.5:43292) 2023-07-21T19:24:51.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f352119a700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569087be0 0x55c56908f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f352199b700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 0x55c569092500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.696+0000 7f352219c700 1 -- 172.21.15.5:0/1091677240 learned_addr learned my addr 172.21.15.5:0/1091677240 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 -- 172.21.15.5:0/1091677240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569087be0 msgr2=0x55c56908f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569087be0 0x55c56908f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 -- 172.21.15.5:0/1091677240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 msgr2=0x55c56908ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 -- 172.21.15.5:0/1091677240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f350c009980 con 0x55c5690872d0 2023-07-21T19:24:51.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352219c700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:51.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f352199b700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 0x55c569092500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f35180cb110 tx=0x7f351809a500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dafdce192e0b1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f35057fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:24:51.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351809af80 con 0x55c5690872d0 2023-07-21T19:24:51.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f351809eb50 con 0x55c5690872d0 2023-07-21T19:24:51.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.697+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c568f334f0 con 0x55c5690872d0 2023-07-21T19:24:51.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35180053c0 con 0x55c5690872d0 2023-07-21T19:24:51.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 10 client.28523 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c568df0c60 con 0x55c5690872d0 2023-07-21T19:24:51.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c568f334f0 con 0x55c5690872d0 2023-07-21T19:24:51.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f35180a2ae0 con 0x55c5690872d0 2023-07-21T19:24:51.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 10 client.28523 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 10 client.28523 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 20 client.28523 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690a2010 0x55c5690a4420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 4 client.28523 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:51.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.698+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c568fd2eb0 con 0x55c5690a2010 2023-07-21T19:24:51.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.699+0000 7f352119a700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690a2010 0x55c5690a4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.699+0000 7f35057fa700 10 client.28523 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.699+0000 7f352119a700 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690a2010 0x55c5690a4420 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b5f8b0fbdc76e4b1 server_cookie=e10ed85d2705dadf in_seq=0 out_seq=0 2023-07-21T19:24:51.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.700+0000 7f35057fa700 1 -- 172.21.15.5:0/1091677240 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c568fd2eb0 con 0x55c5690a2010 2023-07-21T19:24:51.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.700+0000 7f35057fa700 10 client.28523 handle_command_reply: tid=0 2023-07-21T19:24:51.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.700+0000 7f353757ab80 1 client.28523 shutdown 2023-07-21T19:24:51.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.700+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 msgr2=0x55c569092500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.700+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 0x55c569092500 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f35180cb110 tx=0x7f351809a500 comp rx=0 tx=0).stop 2023-07-21T19:24:51.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 shutdown_connections 2023-07-21T19:24:51.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690a2010 0x55c5690a4420 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c56908ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c5690872d0 0x55c569092500 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 --2- 172.21.15.5:0/1091677240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569087be0 0x55c56908f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f35057fa700 0 client.28523 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 >> 172.21.15.5:0/1091677240 conn(0x55c56907dda0 msgr2=0x55c569083230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 shutdown_connections 2023-07-21T19:24:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 1 -- 172.21.15.5:0/1091677240 wait complete. 2023-07-21T19:24:51.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.701+0000 7f353757ab80 20 client.28523 trim_cache size 0 max 16384 2023-07-21T19:24:51.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 Processor -- start 2023-07-21T19:24:51.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 -- start start 2023-07-21T19:24:51.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c5690920a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569092ad0 0x55c56908efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c56907d970 con 0x55c5690872d0 2023-07-21T19:24:51.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c56908b3c0 con 0x55c569087be0 2023-07-21T19:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f353757ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c56908ff30 con 0x55c569092ad0 2023-07-21T19:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f352199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f352199b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43306/0 (socket says 172.21.15.5:43306) 2023-07-21T19:24:51.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f352199b700 1 -- 172.21.15.5:0/4048345998 learned_addr learned my addr 172.21.15.5:0/4048345998 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.719+0000 7f352119a700 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c5690920a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352219c700 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569092ad0 0x55c56908efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 -- 172.21.15.5:0/4048345998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569092ad0 msgr2=0x55c56908efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569092ad0 0x55c56908efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 -- 172.21.15.5:0/4048345998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 msgr2=0x55c5690920a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c5690920a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 -- 172.21.15.5:0/4048345998 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c568e2e250 con 0x55c5690872d0 2023-07-21T19:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f352199b700 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f351809b450 tx=0x7f3518009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8772341497e866f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f3510bff700 1 -- 172.21.15.5:0/4048345998 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35180b3260 con 0x55c5690872d0 2023-07-21T19:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f3510bff700 1 -- 172.21.15.5:0/4048345998 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f351809b220 con 0x55c5690872d0 2023-07-21T19:24:51.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.720+0000 7f3510bff700 1 -- 172.21.15.5:0/4048345998 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351809b4d0 con 0x55c5690872d0 2023-07-21T19:24:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 -- 172.21.15.5:0/4048345998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 msgr2=0x55c56909e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f351809b450 tx=0x7f3518009c00 comp rx=0 tx=0).stop 2023-07-21T19:24:51.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 -- 172.21.15.5:0/4048345998 shutdown_connections 2023-07-21T19:24:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c569092ad0 0x55c56908efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c5690872d0 0x55c56909e760 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 --2- 172.21.15.5:0/4048345998 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c5690920a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 -- 172.21.15.5:0/4048345998 >> 172.21.15.5:0/4048345998 conn(0x55c56907dda0 msgr2=0x55c5690a8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 -- 172.21.15.5:0/4048345998 shutdown_connections 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.721+0000 7f353757ab80 1 -- 172.21.15.5:0/4048345998 wait complete. 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 -- start start 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 Processor -- start 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c569098f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c569099490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c569097a70 con 0x55c569092ad0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c569097be0 con 0x55c569087be0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c569097da0 con 0x55c5690872d0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.722+0000 7f353757ab80 10 client.0 _ll_register_callbacks cb 0x55c568f77760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c569098f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352119a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c569099490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43318/0 (socket says 172.21.15.5:43318) 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 -- 172.21.15.5:0/233000849 learned_addr learned my addr 172.21.15.5:0/233000849 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 -- 172.21.15.5:0/233000849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 msgr2=0x55c569098f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c569098f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 -- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 msgr2=0x55c569099490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c569099490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 -- 172.21.15.5:0/233000849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c568df0c60 con 0x55c569092ad0 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352199b700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c569098f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352119a700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c569099490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f352219c700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f35140176f0 tx=0x7f3514004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd125619c95fa4c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.723+0000 7f35057fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351400e040 con 0x55c569092ad0 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f350c019c20 con 0x55c569092ad0 2023-07-21T19:24:51.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f351402b600 con 0x55c569092ad0 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f351402b7c0 con 0x55c569092ad0 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f353757ab80 10 client.18876 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55c568e062b0 con 0x55c569092ad0 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.724+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f350c019c20 con 0x55c569092ad0 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3514029020 con 0x55c569092ad0 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 10 client.18876 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 10 client.18876 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 20 client.18876 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690f7070 0x55c5690f9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 4 client.18876 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c568f95050 con 0x55c5690f7070 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f352199b700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690f7070 0x55c5690f9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f352199b700 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690f7070 0x55c5690f9480 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87a223ad2d222dad server_cookie=876f2ee80b269d87 in_seq=0 out_seq=0 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.725+0000 7f35057fa700 10 client.18876 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f35057fa700 1 -- 172.21.15.5:0/233000849 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c568f95050 con 0x55c5690f7070 2023-07-21T19:24:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f35057fa700 10 client.18876 handle_command_reply: tid=0 2023-07-21T19:24:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 client.18876 shutdown 2023-07-21T19:24:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 msgr2=0x55c569097360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f35140176f0 tx=0x7f3514004b40 comp rx=0 tx=0).stop 2023-07-21T19:24:51.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 shutdown_connections 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c5690f7070 0x55c5690f9480 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c569092ad0 0x55c569097360 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c5690872d0 0x55c569098f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 --2- 172.21.15.5:0/233000849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c569087be0 0x55c569099490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 >> 172.21.15.5:0/233000849 conn(0x55c56907dda0 msgr2=0x55c56908ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f35057fa700 0 client.18876 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 shutdown_connections 2023-07-21T19:24:51.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 1 -- 172.21.15.5:0/233000849 wait complete. 2023-07-21T19:24:51.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.726+0000 7f353757ab80 20 client.18876 trim_cache size 0 max 16384 2023-07-21T19:24:51.730 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "9e7de850-d1ef-45ab-a9bc-f76a37f2a9df": { 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "9e7de850-d1ef-45ab-a9bc-f76a37f2a9df", 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:24:51.731 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:24:51.732 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:24:51.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 msgr2=0x7f35080a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 0x7f35080a3770 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f351400f540 tx=0x7f35140099e0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 msgr2=0x7f351c197e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f3518009810 tx=0x7f351800fa00 comp rx=0 tx=0).stop 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 shutdown_connections 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f351c10c320 0x7f351c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35080a12c0 0x7f35080a3770 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f351c0728b0 0x7f351c197e60 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 --2- 172.21.15.5:0/607766346 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f351c0731c0 0x7f351c1983a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 >> 172.21.15.5:0/607766346 conn(0x7f351c0fb5e0 msgr2=0x7f351c10e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 shutdown_connections 2023-07-21T19:24:51.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:24:51.727+0000 7f3523bff700 1 -- 172.21.15.5:0/607766346 wait complete. 2023-07-21T19:24:51.734 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:51 smithi005 ceph-mon[126460]: pgmap v542: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 23 op/s 2023-07-21T19:24:51.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:51 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:24:51.735 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:51 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:24:51.737 INFO:tasks.cephfs.filesystem:scrub status for tag:9e7de850-d1ef-45ab-a9bc-f76a37f2a9df - {'path': '/', 'tag': '9e7de850-d1ef-45ab-a9bc-f76a37f2a9df', 'options': 'recursive,force'} 2023-07-21T19:24:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 -- 172.21.15.5:0/1132554495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 msgr2=0x7f6590105d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 --2- 172.21.15.5:0/1132554495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f6590105d70 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f6580009990 tx=0x7f6580009ca0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 -- 172.21.15.5:0/1132554495 shutdown_connections 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 --2- 172.21.15.5:0/1132554495 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6590068070 0x7f659010c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 --2- 172.21.15.5:0/1132554495 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f6590105d70 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 --2- 172.21.15.5:0/1132554495 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65901062b0 0x7f6590067940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 -- 172.21.15.5:0/1132554495 >> 172.21.15.5:0/1132554495 conn(0x7f6590074770 msgr2=0x7f6590074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 -- 172.21.15.5:0/1132554495 shutdown_connections 2023-07-21T19:24:51.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.906+0000 7f6594f9d700 1 -- 172.21.15.5:0/1132554495 wait complete. 2023-07-21T19:24:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.908+0000 7f6594f9d700 1 Processor -- start 2023-07-21T19:24:51.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.908+0000 7f6594f9d700 1 -- start start 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.908+0000 7f6594f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 0x7f65900fea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f6594f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f65900fefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f6594f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65901062b0 0x7f65901a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f6594f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65900ff7d0 con 0x7f65901059a0 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f6594f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901a4100 con 0x7f6590068070 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f6594f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65901a4240 con 0x7f65901062b0 2023-07-21T19:24:51.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f65900fefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 0x7f65900fea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f65900fefa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43342/0 (socket says 172.21.15.5:43342) 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658effd700 1 -- 172.21.15.5:0/2278787445 learned_addr learned my addr 172.21.15.5:0/2278787445 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658ffff700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65901062b0 0x7f65901a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 -- 172.21.15.5:0/2278787445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65901062b0 msgr2=0x7f65901a3bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65901062b0 0x7f65901a3bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 -- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 msgr2=0x7f65900fefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f65900fefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.909+0000 7f658f7fe700 1 -- 172.21.15.5:0/2278787445 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65800095f0 con 0x7f6590068070 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f658f7fe700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 0x7f65900fea60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6580005910 tx=0x7f6580005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f658cff9700 1 -- 172.21.15.5:0/2278787445 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658001d070 con 0x7f6590068070 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65901a4440 con 0x7f6590068070 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f658cff9700 1 -- 172.21.15.5:0/2278787445 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6580022470 con 0x7f6590068070 2023-07-21T19:24:51.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f658cff9700 1 -- 172.21.15.5:0/2278787445 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6580017650 con 0x7f6590068070 2023-07-21T19:24:51.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.910+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65901a4910 con 0x7f6590068070 2023-07-21T19:24:51.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.915+0000 7f658cff9700 1 -- 172.21.15.5:0/2278787445 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f65800225e0 con 0x7f6590068070 2023-07-21T19:24:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.916+0000 7f65a9b85b80 1 Processor -- start 2023-07-21T19:24:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.916+0000 7f65a9b85b80 1 -- start start 2023-07-21T19:24:51.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 0x55d1179fbf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 0x55d1179f6700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1178fbc50 con 0x55d1179fc470 2023-07-21T19:24:51.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d11794e9e0 con 0x55d1179fc8c0 2023-07-21T19:24:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117947e70 con 0x55d1178526b0 2023-07-21T19:24:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 0x55d1179fbf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 0x55d1179f6700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 0x55d1179fbf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54524/0 (socket says 172.21.15.5:54524) 2023-07-21T19:24:51.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658ffff700 1 -- 172.21.15.5:0/1466891518 learned_addr learned my addr 172.21.15.5:0/1466891518 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.917+0000 7f658effd700 1 -- 172.21.15.5:0/1466891518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 msgr2=0x55d1179fbf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658effd700 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 0x55d1179fbf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658effd700 1 -- 172.21.15.5:0/1466891518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 msgr2=0x55d1179f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658effd700 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658effd700 1 -- 172.21.15.5:0/1466891518 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d117775d80 con 0x55d1179fc8c0 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658cff9700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 0x7f65780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658cff9700 1 -- 172.21.15.5:0/2278787445 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f65800ce100 con 0x7f6590068070 2023-07-21T19:24:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658ffff700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 0x7f65780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f658effd700 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 0x55d1179f6700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6590100250 tx=0x7f6584009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1413609fb836f7d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.918+0000 7f6575ffb700 1 -- 172.21.15.5:0/1466891518 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584011ce0 con 0x55d1179fc8c0 2023-07-21T19:24:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.919+0000 7f6575ffb700 1 -- 172.21.15.5:0/1466891518 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6584011e40 con 0x55d1179fc8c0 2023-07-21T19:24:51.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.919+0000 7f6575ffb700 1 -- 172.21.15.5:0/1466891518 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6584005280 con 0x55d1179fc8c0 2023-07-21T19:24:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:51 smithi156 ceph-mon[128316]: pgmap v542: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 23 op/s 2023-07-21T19:24:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:51 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:24:51.915 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:51 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:24:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.919+0000 7f658ffff700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 0x7f65780a3770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6588009550 tx=0x7f658800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.919+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1466891518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 msgr2=0x55d1179f6700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.919+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 0x55d1179f6700 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6590100250 tx=0x7f6584009f90 comp rx=0 tx=0).stop 2023-07-21T19:24:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1466891518 shutdown_connections 2023-07-21T19:24:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc8c0 0x55d1179f6700 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1178526b0 0x55d1179fbf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1466891518 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1466891518 >> 172.21.15.5:0/1466891518 conn(0x55d1179ebd40 msgr2=0x55d1179f1340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1466891518 shutdown_connections 2023-07-21T19:24:51.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1466891518 wait complete. 2023-07-21T19:24:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.920+0000 7f65a9b85b80 1 -- start start 2023-07-21T19:24:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 Processor -- start 2023-07-21T19:24:51.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 0x55d1179ff9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc8c0 0x55d117a00000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1178e7a60 con 0x55d1179fc8c0 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1178e7760 con 0x55d1178526b0 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1177cb8c0 con 0x55d1179fc470 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f65a9b85b80 10 client.0 _ll_register_callbacks cb 0x55d1178e5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.921+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59736/0 (socket says 172.21.15.5:59736) 2023-07-21T19:24:51.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 -- 172.21.15.5:0/3376576029 learned_addr learned my addr 172.21.15.5:0/3376576029 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658ffff700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc8c0 0x55d117a00000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 -- 172.21.15.5:0/3376576029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 msgr2=0x55d1179ff9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658effd700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 0x55d1179ff9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 0x55d1179ff9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 -- 172.21.15.5:0/3376576029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc8c0 msgr2=0x55d117a00000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc8c0 0x55d117a00000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 -- 172.21.15.5:0/3376576029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6584009910 con 0x55d1178526b0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658effd700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 0x55d1179ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f65800cbe50 tx=0x7f6580005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6b1f05bf56ea5dfb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f656effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.922+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658009b030 con 0x55d1178526b0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f658009b260 con 0x55d1178526b0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d1178a1490 con 0x55d1178526b0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f658009ada0 con 0x55d1178526b0 2023-07-21T19:24:51.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 10 client.28541 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55d11775ec00 con 0x55d1178526b0 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d1178a1490 con 0x55d1178526b0 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f658009b4c0 con 0x55d1178526b0 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 10 client.28541 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 10 client.28541 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 20 client.28541 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d117a0f9a0 0x55d117a11db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 4 client.28541 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.923+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d117940e50 con 0x55d117a0f9a0 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.924+0000 7f658effd700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d117a0f9a0 0x55d117a11db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.924+0000 7f656effd700 10 client.28541 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.924+0000 7f658effd700 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d117a0f9a0 0x55d117a11db0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e758bbf2c7d72b09 server_cookie=c1733aa2e2537164 in_seq=0 out_seq=0 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f656effd700 1 -- 172.21.15.5:0/3376576029 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d117940e50 con 0x55d117a0f9a0 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f656effd700 10 client.28541 handle_command_reply: tid=0 2023-07-21T19:24:51.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 client.28541 shutdown 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 msgr2=0x55d117a02770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f65800cbe50 tx=0x7f6580005f00 comp rx=0 tx=0).stop 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 shutdown_connections 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d117a0f9a0 0x55d117a11db0 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc8c0 0x55d117a00000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d117a02770 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f656effd700 0 client.28541 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3376576029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179fc470 0x55d1179ff9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 >> 172.21.15.5:0/3376576029 conn(0x55d1179ebd40 msgr2=0x55d1179f0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.925+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 shutdown_connections 2023-07-21T19:24:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.926+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3376576029 wait complete. 2023-07-21T19:24:51.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.926+0000 7f65a9b85b80 20 client.28541 trim_cache size 0 max 16384 2023-07-21T19:24:51.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.943+0000 7f65a9b85b80 1 Processor -- start 2023-07-21T19:24:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 -- start start 2023-07-21T19:24:51.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d1179f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117a0c710 con 0x55d1179fc470 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d117940e50 con 0x55d1178526b0 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1178a1490 con 0x55d1179ebd40 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f658ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54544/0 (socket says 172.21.15.5:54544) 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f658ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179fb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43374/0 (socket says 172.21.15.5:43374) 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.944+0000 7f658f7fe700 1 -- 172.21.15.5:0/3811652319 learned_addr learned my addr 172.21.15.5:0/3811652319 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658effd700 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d1179f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 -- 172.21.15.5:0/3811652319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 msgr2=0x55d1179f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d1179f61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 -- 172.21.15.5:0/3811652319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 msgr2=0x55d1179fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 -- 172.21.15.5:0/3811652319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d11779c1f0 con 0x55d1179ebd40 2023-07-21T19:24:51.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658effd700 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d1179f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:51.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f658f7fe700 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6580097cd0 tx=0x7f6580022ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd24395855d43fff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f65767fc700 1 -- 172.21.15.5:0/3811652319 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658009ada0 con 0x55d1179ebd40 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f65767fc700 1 -- 172.21.15.5:0/3811652319 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f658009b030 con 0x55d1179ebd40 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.945+0000 7f65767fc700 1 -- 172.21.15.5:0/3811652319 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65800a3620 con 0x55d1179ebd40 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3811652319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 msgr2=0x55d1179fd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6580097cd0 tx=0x7f6580022ec0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3811652319 shutdown_connections 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1179fc470 0x55d1179fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179fd590 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/3811652319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1178526b0 0x55d1179f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3811652319 >> 172.21.15.5:0/3811652319 conn(0x55d117a0c340 msgr2=0x55d1179fab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3811652319 shutdown_connections 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.946+0000 7f65a9b85b80 1 -- 172.21.15.5:0/3811652319 wait complete. 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.947+0000 7f65a9b85b80 1 -- start start 2023-07-21T19:24:51.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.947+0000 7f65a9b85b80 1 Processor -- start 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179f9300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 0x55d1179f7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1179f7f60 con 0x55d1178526b0 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1179f80d0 con 0x55d1179fc470 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d1179f8290 con 0x55d1179ebd40 2023-07-21T19:24:51.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f65a9b85b80 10 client.0 _ll_register_callbacks cb 0x7f65800092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43386/0 (socket says 172.21.15.5:43386) 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 -- 172.21.15.5:0/1226254583 learned_addr learned my addr 172.21.15.5:0/1226254583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658effd700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179f9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658ffff700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 0x55d1179f7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 -- 172.21.15.5:0/1226254583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 msgr2=0x55d1179f9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179f9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 -- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 msgr2=0x55d1179f7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.948+0000 7f658f7fe700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 0x55d1179f7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f658f7fe700 1 -- 172.21.15.5:0/1226254583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d11775ec00 con 0x55d1178526b0 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f658effd700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f658ffff700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 0x55d1179f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f658f7fe700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6580006010 tx=0x7f6580099810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e950bf62434a39c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:24:51.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f656effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:24:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f658009ada0 con 0x55d1178526b0 2023-07-21T19:24:51.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f65800a3620 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6584011e40 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=53) v2 ==== 24+0+0 (secure 0 0 0) 0x7f65800228c0 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f65a9b85b80 10 client.18894 fetch_fsmap learned FSMap version 53 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.949+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=53,monmap=4+}) v3 -- 0x55d117774250 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65800b9080 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 53) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f658009a250 con 0x55d1178526b0 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 10 client.18894 fetch_fsmap finished waiting for FSMap version 53 2023-07-21T19:24:51.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 10 client.18894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 20 client.18894 populate_metadata read hostname 'smithi005' 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d1179fc8c0 0x55d117a66360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 4 client.18894 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d117902ff0 con 0x55d1179fc8c0 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.950+0000 7f658effd700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d1179fc8c0 0x55d117a66360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.951+0000 7f656effd700 10 client.18894 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.951+0000 7f658effd700 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d1179fc8c0 0x55d117a66360 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85987b732541dcd2 server_cookie=75fdc5ef77a68701 in_seq=0 out_seq=0 2023-07-21T19:24:51.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f656effd700 1 -- 172.21.15.5:0/1226254583 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d117902ff0 con 0x55d1179fc8c0 2023-07-21T19:24:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f656effd700 10 client.18894 handle_command_reply: tid=0 2023-07-21T19:24:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 client.18894 shutdown 2023-07-21T19:24:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 msgr2=0x55d1179f8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f6580006010 tx=0x7f6580099810 comp rx=0 tx=0).stop 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 shutdown_connections 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d1179fc470 0x55d1179f7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d1179fc8c0 0x55d117a66360 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d1178526b0 0x55d1179f8dc0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 --2- 172.21.15.5:0/1226254583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d1179ebd40 0x55d1179f9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f656effd700 0 client.18894 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:24:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 >> 172.21.15.5:0/1226254583 conn(0x55d117a0c340 msgr2=0x55d1179fb3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 shutdown_connections 2023-07-21T19:24:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 1 -- 172.21.15.5:0/1226254583 wait complete. 2023-07-21T19:24:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.959+0000 7f65a9b85b80 20 client.18894 trim_cache size 0 max 16384 2023-07-21T19:24:51.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:24:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:24:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:24:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:24:51.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 msgr2=0x7f65780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 0x7f65780a3770 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6588009550 tx=0x7f658800b040 comp rx=0 tx=0).stop 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 msgr2=0x7f65900fea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 0x7f65900fea60 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f6580005910 tx=0x7f6580005720 comp rx=0 tx=0).stop 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 shutdown_connections 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65901062b0 0x7f65901a3bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6590068070 0x7f65900fea60 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65780a12c0 0x7f65780a3770 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 --2- 172.21.15.5:0/2278787445 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65901059a0 0x7f65900fefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:24:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 >> 172.21.15.5:0/2278787445 conn(0x7f6590074770 msgr2=0x7f65900fcfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:24:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 shutdown_connections 2023-07-21T19:24:51.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:24:51.960+0000 7f6594f9d700 1 -- 172.21.15.5:0/2278787445 wait complete. 2023-07-21T19:24:52.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:51 smithi101 ceph-mon[128136]: pgmap v542: 97 pgs: 97 active+clean; 14 GiB data, 6.3 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 17 MiB/s wr, 23 op/s 2023-07-21T19:24:52.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:51 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:24:52.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:51 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:24:52.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:24:52.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:24:52.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:52 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:24:52.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:24:52.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:52.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:52 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:53.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:24:53.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:53.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:24:53.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:53.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:53.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:52 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:24:53.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:53 smithi005 ceph-mon[126460]: pgmap v543: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 110 op/s 2023-07-21T19:24:53.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:53 smithi156 ceph-mon[128316]: pgmap v543: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 110 op/s 2023-07-21T19:24:54.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:53 smithi101 ceph-mon[128136]: pgmap v543: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 168 KiB/s rd, 25 MiB/s wr, 110 op/s 2023-07-21T19:24:55.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:55 smithi005 ceph-mon[126460]: pgmap v544: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 19 MiB/s wr, 114 op/s 2023-07-21T19:24:55.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:55 smithi156 ceph-mon[128316]: pgmap v544: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 19 MiB/s wr, 114 op/s 2023-07-21T19:24:56.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:55 smithi101 ceph-mon[128136]: pgmap v544: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 167 KiB/s rd, 19 MiB/s wr, 114 op/s 2023-07-21T19:24:57.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:57 smithi005 ceph-mon[126460]: pgmap v545: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 109 op/s 2023-07-21T19:24:57.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:57 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:57.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:57 smithi156 ceph-mon[128316]: pgmap v545: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 109 op/s 2023-07-21T19:24:57.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:57 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:58.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:57 smithi101 ceph-mon[128136]: pgmap v545: 97 pgs: 97 active+clean; 15 GiB data, 6.4 GiB used, 1.0 TiB / 1.0 TiB avail; 140 KiB/s rd, 18 MiB/s wr, 109 op/s 2023-07-21T19:24:58.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:57 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:24:59.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:24:59 smithi005 ceph-mon[126460]: pgmap v546: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 26 MiB/s wr, 115 op/s 2023-07-21T19:24:59.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:24:59 smithi156 ceph-mon[128316]: pgmap v546: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 26 MiB/s wr, 115 op/s 2023-07-21T19:25:00.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:24:59 smithi101 ceph-mon[128136]: pgmap v546: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 190 KiB/s rd, 26 MiB/s wr, 115 op/s 2023-07-21T19:25:01.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:01 smithi005 ceph-mon[126460]: pgmap v547: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 17 MiB/s wr, 107 op/s 2023-07-21T19:25:01.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:01 smithi156 ceph-mon[128316]: pgmap v547: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 17 MiB/s wr, 107 op/s 2023-07-21T19:25:02.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:01 smithi101 ceph-mon[128136]: pgmap v547: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 163 KiB/s rd, 17 MiB/s wr, 107 op/s 2023-07-21T19:25:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:03 smithi005 ceph-mon[126460]: pgmap v548: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 24 MiB/s wr, 125 op/s 2023-07-21T19:25:03.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:03 smithi156 ceph-mon[128316]: pgmap v548: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 24 MiB/s wr, 125 op/s 2023-07-21T19:25:04.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:03 smithi101 ceph-mon[128136]: pgmap v548: 97 pgs: 97 active+clean; 15 GiB data, 6.6 GiB used, 1.0 TiB / 1.0 TiB avail; 177 KiB/s rd, 24 MiB/s wr, 125 op/s 2023-07-21T19:25:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 -- 172.21.15.5:0/4016319700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c10c340 msgr2=0x7fa24c10e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 --2- 172.21.15.5:0/4016319700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c10c340 0x7fa24c10e720 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b0a0 tx=0x7fa24400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 -- 172.21.15.5:0/4016319700 shutdown_connections 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 --2- 172.21.15.5:0/4016319700 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c10c340 0x7fa24c10e720 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 --2- 172.21.15.5:0/4016319700 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c0728b0 0x7fa24c072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 --2- 172.21.15.5:0/4016319700 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0731c0 0x7fa24c10be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 -- 172.21.15.5:0/4016319700 >> 172.21.15.5:0/4016319700 conn(0x7fa24c0fb600 msgr2=0x7fa24c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 -- 172.21.15.5:0/4016319700 shutdown_connections 2023-07-21T19:25:05.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.131+0000 7fa2509ff700 1 -- 172.21.15.5:0/4016319700 wait complete. 2023-07-21T19:25:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.133+0000 7fa2509ff700 1 Processor -- start 2023-07-21T19:25:05.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.133+0000 7fa2509ff700 1 -- start start 2023-07-21T19:25:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0728b0 0x7fa24c197e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 0x7fa24c19ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19d070 con 0x7fa24c0731c0 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19d1e0 con 0x7fa24c10c340 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa2509ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa24c19d3a0 con 0x7fa24c0728b0 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52776/0 (socket says 172.21.15.5:52776) 2023-07-21T19:25:05.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 -- 172.21.15.5:0/908658871 learned_addr learned my addr 172.21.15.5:0/908658871 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 0x7fa24c19ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa24a59c700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0728b0 0x7fa24c197e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 -- 172.21.15.5:0/908658871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0728b0 msgr2=0x7fa24c197e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0728b0 0x7fa24c197e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 -- 172.21.15.5:0/908658871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 msgr2=0x7fa24c19ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 0x7fa24c19ca30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa249d9b700 1 -- 172.21.15.5:0/908658871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa244009d00 con 0x7fa24c0731c0 2023-07-21T19:25:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.134+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 0x7fa24c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa249d9b700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa24000d950 tx=0x7fa24000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa23b7fe700 1 -- 172.21.15.5:0/908658871 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000f890 con 0x7fa24c0731c0 2023-07-21T19:25:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa24c19d680 con 0x7fa24c0731c0 2023-07-21T19:25:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa23b7fe700 1 -- 172.21.15.5:0/908658871 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa24000fe80 con 0x7fa24c0731c0 2023-07-21T19:25:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa23b7fe700 1 -- 172.21.15.5:0/908658871 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000e3b0 con 0x7fa24c0731c0 2023-07-21T19:25:05.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.135+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa24c1a4ec0 con 0x7fa24c0731c0 2023-07-21T19:25:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.140+0000 7fa264410b80 1 Processor -- start 2023-07-21T19:25:05.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.140+0000 7fa264410b80 1 -- start start 2023-07-21T19:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x56362990bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563629906580 0x5636299069d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362980bc50 con 0x56362990c2d0 2023-07-21T19:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362985e9e0 con 0x5636297626b0 2023-07-21T19:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa264410b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563629857e70 con 0x563629906580 2023-07-21T19:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa23b7fe700 1 -- 172.21.15.5:0/908658871 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa24000f9f0 con 0x7fa24c0731c0 2023-07-21T19:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52778/0 (socket says 172.21.15.5:52778) 2023-07-21T19:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 -- 172.21.15.5:0/1566080863 learned_addr learned my addr 172.21.15.5:0/1566080863 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:05.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x56362990bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 -- 172.21.15.5:0/1566080863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563629906580 msgr2=0x5636299069d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563629906580 0x5636299069d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.141+0000 7fa24a59c700 1 -- 172.21.15.5:0/1566080863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 msgr2=0x56362990bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x56362990bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa24a59c700 1 -- 172.21.15.5:0/1566080863 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563629685d80 con 0x56362990c2d0 2023-07-21T19:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa23c007cf0 tx=0x7fa23c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdb572b8502af654 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa238ff9700 1 -- 172.21.15.5:0/1566080863 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c010d10 con 0x56362990c2d0 2023-07-21T19:25:05.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa238ff9700 1 -- 172.21.15.5:0/1566080863 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa23c015070 con 0x56362990c2d0 2023-07-21T19:25:05.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.142+0000 7fa238ff9700 1 -- 172.21.15.5:0/1566080863 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c019630 con 0x56362990c2d0 2023-07-21T19:25:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa23b7fe700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 0x7fa2340a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa23b7fe700 1 -- 172.21.15.5:0/908658871 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa2400cc6b0 con 0x7fa24c0731c0 2023-07-21T19:25:05.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 0x7fa2340a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 -- 172.21.15.5:0/1566080863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 msgr2=0x563629906040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fa23c007cf0 tx=0x7fa23c0103d0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 -- 172.21.15.5:0/1566080863 shutdown_connections 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563629906580 0x5636299069d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x56362990bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 --2- 172.21.15.5:0/1566080863 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629906040 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.143+0000 7fa264410b80 1 -- 172.21.15.5:0/1566080863 >> 172.21.15.5:0/1566080863 conn(0x5636298fbbe0 msgr2=0x563629900f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.144+0000 7fa264410b80 1 -- 172.21.15.5:0/1566080863 shutdown_connections 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.144+0000 7fa264410b80 1 -- 172.21.15.5:0/1566080863 wait complete. 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.144+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 0x7fa2340a3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b830 tx=0x7fa244006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.144+0000 7fa264410b80 1 -- start start 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 Processor -- start 2023-07-21T19:25:05.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x56362990f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563629906580 0x56362990fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629910400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636297f7a60 con 0x56362990c2d0 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636297f7760 con 0x563629906580 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.145+0000 7fa264410b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5636296db8c0 con 0x5636297626b0 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa264410b80 10 client.0 _ll_register_callbacks cb 0x5636297f5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629910400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629910400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52792/0 (socket says 172.21.15.5:52792) 2023-07-21T19:25:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24ad9d700 1 -- 172.21.15.5:0/2891101723 learned_addr learned my addr 172.21.15.5:0/2891101723 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x56362990f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa249d9b700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563629906580 0x56362990fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 -- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563629906580 msgr2=0x56362990fec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563629906580 0x56362990fec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 -- 172.21.15.5:0/2891101723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 msgr2=0x563629910400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629910400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.146+0000 7fa24a59c700 1 -- 172.21.15.5:0/2891101723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa23c007920 con 0x5636297626b0 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x56362990f980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x563629902b40 tx=0x7fa23c011c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90f2b83d1a72d8db server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c010d80 con 0x5636297626b0 2023-07-21T19:25:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5636297b1490 con 0x5636297626b0 2023-07-21T19:25:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa23c004020 con 0x5636297626b0 2023-07-21T19:25:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.147+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa23c010ee0 con 0x5636297626b0 2023-07-21T19:25:05.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 10 client.28474 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x56362966ec00 con 0x5636297626b0 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5636297b1490 con 0x5636297626b0 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa23c005040 con 0x5636297626b0 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 10 client.28474 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 10 client.28474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 20 client.28474 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:05.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56362991fb00 0x563629921f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 4 client.28474 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.148+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563629850e50 con 0x56362991fb00 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.149+0000 7fa249d9b700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56362991fb00 0x563629921f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.149+0000 7fa249d9b700 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56362991fb00 0x563629921f10 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97e2a779e33a3e9b server_cookie=f99ee8a478ff3f3c in_seq=0 out_seq=0 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.149+0000 7fa231ffb700 10 client.28474 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa231ffb700 1 -- 172.21.15.5:0/2891101723 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563629850e50 con 0x56362991fb00 2023-07-21T19:25:05.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa231ffb700 10 client.28474 handle_command_reply: tid=0 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 client.28474 shutdown 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 msgr2=0x56362990f980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x56362990f980 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x563629902b40 tx=0x7fa23c011c30 comp rx=0 tx=0).stop 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 shutdown_connections 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56362991fb00 0x563629921f10 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56362990c2d0 0x563629910400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa231ffb700 0 client.28474 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x56362990f980 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 --2- 172.21.15.5:0/2891101723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563629906580 0x56362990fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.150+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 >> 172.21.15.5:0/2891101723 conn(0x5636298fbbe0 msgr2=0x5636299004e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.151+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 shutdown_connections 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.151+0000 7fa264410b80 1 -- 172.21.15.5:0/2891101723 wait complete. 2023-07-21T19:25:05.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.151+0000 7fa264410b80 20 client.28474 trim_cache size 0 max 16384 2023-07-21T19:25:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.166+0000 7fa264410b80 1 Processor -- start 2023-07-21T19:25:05.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.166+0000 7fa264410b80 1 -- start start 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.166+0000 7fa264410b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x563629906040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa264410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa264410b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56362990c2d0 0x56362990b210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa264410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362991c650 con 0x563629906580 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa264410b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563629910ef0 con 0x56362990c2d0 2023-07-21T19:25:05.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa264410b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56362990c010 con 0x5636297626b0 2023-07-21T19:25:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x563629906040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52800/0 (socket says 172.21.15.5:52800) 2023-07-21T19:25:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa249d9b700 1 -- 172.21.15.5:0/2323564244 learned_addr learned my addr 172.21.15.5:0/2323564244 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:05.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56362990c2d0 0x56362990b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 -- 172.21.15.5:0/2323564244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56362990c2d0 msgr2=0x56362990b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56362990c2d0 0x56362990b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 -- 172.21.15.5:0/2323564244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 msgr2=0x563629916c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa24a59c700 1 -- 172.21.15.5:0/2323564244 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5636296ac1f0 con 0x5636297626b0 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.167+0000 7fa249d9b700 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:05.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.168+0000 7fa24a59c700 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x563629906040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005450 tx=0x7fa23c005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc6bacf6d0be754 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.168+0000 7fa2397fa700 1 -- 172.21.15.5:0/2323564244 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c0056c0 con 0x5636297626b0 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.168+0000 7fa2397fa700 1 -- 172.21.15.5:0/2323564244 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa23c004490 con 0x5636297626b0 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.168+0000 7fa2397fa700 1 -- 172.21.15.5:0/2323564244 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa23c011910 con 0x5636297626b0 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- 172.21.15.5:0/2323564244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 msgr2=0x563629906040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x563629906040 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa23c005450 tx=0x7fa23c005fd0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- 172.21.15.5:0/2323564244 shutdown_connections 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56362990c2d0 0x56362990b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5636297626b0 0x563629906040 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 --2- 172.21.15.5:0/2323564244 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629916c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- 172.21.15.5:0/2323564244 >> 172.21.15.5:0/2323564244 conn(0x5636298fbbe0 msgr2=0x56362990ab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- 172.21.15.5:0/2323564244 shutdown_connections 2023-07-21T19:25:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- 172.21.15.5:0/2323564244 wait complete. 2023-07-21T19:25:05.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.169+0000 7fa264410b80 1 -- start start 2023-07-21T19:25:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 Processor -- start 2023-07-21T19:25:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x563629967b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56362990c2d0 0x563629972990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563629968730 con 0x563629906580 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563629972ed0 con 0x5636297626b0 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563629973010 con 0x56362990c2d0 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa264410b80 10 client.0 _ll_register_callbacks cb 0x7fa244009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa249d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52802/0 (socket says 172.21.15.5:52802) 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa249d9b700 1 -- 172.21.15.5:0/1968856523 learned_addr learned my addr 172.21.15.5:0/1968856523 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa24ad9d700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56362990c2d0 0x563629972990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.170+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x563629967b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 -- 172.21.15.5:0/1968856523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56362990c2d0 msgr2=0x563629972990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56362990c2d0 0x563629972990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 -- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 msgr2=0x563629967b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x563629967b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 -- 172.21.15.5:0/1968856523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2400b7780 con 0x563629906580 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x563629967b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa249d9b700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa240095c80 tx=0x7fa240010000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695f393d2fd276f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24000fca0 con 0x563629906580 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.171+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa240099780 con 0x563629906580 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa23c015070 con 0x563629906580 2023-07-21T19:25:05.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa24009d7b0 con 0x563629906580 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 10 client.18912 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x56362966ec00 con 0x563629906580 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa23c015070 con 0x563629906580 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2400b73e0 con 0x563629906580 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 10 client.18912 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 10 client.18912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 20 client.18912 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:05.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563629974750 0x563629976b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 4 client.18912 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:25:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.172+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563629812ff0 con 0x563629974750 2023-07-21T19:25:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.173+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563629974750 0x563629976b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.173+0000 7fa231ffb700 10 client.18912 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:05.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.173+0000 7fa24a59c700 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563629974750 0x563629976b60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc1eb1f6d5f448ad server_cookie=908b0c9e6fa92120 in_seq=0 out_seq=0 2023-07-21T19:25:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.181+0000 7fa231ffb700 1 -- 172.21.15.5:0/1968856523 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563629812ff0 con 0x563629974750 2023-07-21T19:25:05.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.181+0000 7fa231ffb700 10 client.18912 handle_command_reply: tid=0 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.181+0000 7fa264410b80 1 client.18912 shutdown 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 msgr2=0x563629968040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fa240095c80 tx=0x7fa240010000 comp rx=0 tx=0).stop 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 shutdown_connections 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56362990c2d0 0x563629972990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5636297626b0 0x563629967b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563629974750 0x563629976b60 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 --2- 172.21.15.5:0/1968856523 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563629906580 0x563629968040 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa231ffb700 0 client.18912 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 >> 172.21.15.5:0/1968856523 conn(0x5636298fbbe0 msgr2=0x56362990b390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 shutdown_connections 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 1 -- 172.21.15.5:0/1968856523 wait complete. 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.182+0000 7fa264410b80 20 client.18912 trim_cache size 0 max 16384 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:25:05.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:25:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 msgr2=0x7fa2340a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 0x7fa2340a3770 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa24400b830 tx=0x7fa244006210 comp rx=0 tx=0).stop 2023-07-21T19:25:05.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 msgr2=0x7fa24c1983c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fa24000d950 tx=0x7fa24000dc60 comp rx=0 tx=0).stop 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 shutdown_connections 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa24c10c340 0x7fa24c19ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa24c0728b0 0x7fa24c197e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2340a12c0 0x7fa2340a3770 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 --2- 172.21.15.5:0/908658871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa24c0731c0 0x7fa24c1983c0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 >> 172.21.15.5:0/908658871 conn(0x7fa24c0fb600 msgr2=0x7fa24c10e420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:05.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 shutdown_connections 2023-07-21T19:25:05.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:05.183+0000 7fa2509ff700 1 -- 172.21.15.5:0/908658871 wait complete. 2023-07-21T19:25:05.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:05 smithi005 ceph-mon[126460]: pgmap v549: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 18 MiB/s wr, 41 op/s 2023-07-21T19:25:05.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:05 smithi156 ceph-mon[128316]: pgmap v549: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 18 MiB/s wr, 41 op/s 2023-07-21T19:25:06.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:05 smithi101 ceph-mon[128136]: pgmap v549: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 110 KiB/s rd, 18 MiB/s wr, 41 op/s 2023-07-21T19:25:06.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:06.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:06.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:06.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:06.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:06.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:07.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:07.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:07.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:07.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:07 smithi005 ceph-mon[126460]: pgmap v550: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 27 op/s 2023-07-21T19:25:07.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:07 smithi005 ceph-mon[126460]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:25:07.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:07 smithi156 ceph-mon[128316]: pgmap v550: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 27 op/s 2023-07-21T19:25:07.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:07 smithi156 ceph-mon[128316]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:25:08.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:07 smithi101 ceph-mon[128136]: pgmap v550: 97 pgs: 97 active+clean; 15 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 85 KiB/s rd, 17 MiB/s wr, 27 op/s 2023-07-21T19:25:08.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:07 smithi101 ceph-mon[128136]: osdmap e111: 12 total, 12 up, 12 in 2023-07-21T19:25:09.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:09 smithi005 ceph-mon[126460]: pgmap v552: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:09.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:09 smithi156 ceph-mon[128316]: pgmap v552: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:10.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:09 smithi101 ceph-mon[128136]: pgmap v552: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:10.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:10 smithi005 ceph-mon[126460]: pgmap v553: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:10.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:10 smithi156 ceph-mon[128316]: pgmap v553: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:11.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:10 smithi101 ceph-mon[128136]: pgmap v553: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 20 MiB/s wr, 35 op/s 2023-07-21T19:25:12.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:12 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:25:12.410 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:12 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:25:12.571 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:12 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:25:13.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:13 smithi005 ceph-mon[126460]: pgmap v554: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 21 MiB/s wr, 22 op/s 2023-07-21T19:25:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:13 smithi005 ceph-mon[126460]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:25:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:13 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:25:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:13 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:25:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:13 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:25:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:13 smithi156 ceph-mon[128316]: pgmap v554: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 21 MiB/s wr, 22 op/s 2023-07-21T19:25:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:13 smithi156 ceph-mon[128316]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:25:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:13 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:25:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:13 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:25:13.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:13 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:25:14.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:13 smithi101 ceph-mon[128136]: pgmap v554: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 205 KiB/s rd, 21 MiB/s wr, 22 op/s 2023-07-21T19:25:14.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:13 smithi101 ceph-mon[128136]: osdmap e112: 12 total, 12 up, 12 in 2023-07-21T19:25:14.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:13 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:25:14.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:13 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:25:14.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:13 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:25:15.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:15 smithi005 ceph-mon[126460]: pgmap v556: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 26 MiB/s wr, 23 op/s 2023-07-21T19:25:15.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:15 smithi156 ceph-mon[128316]: pgmap v556: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 26 MiB/s wr, 23 op/s 2023-07-21T19:25:16.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:15 smithi101 ceph-mon[128136]: pgmap v556: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 227 KiB/s rd, 26 MiB/s wr, 23 op/s 2023-07-21T19:25:17.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:17 smithi005 ceph-mon[126460]: pgmap v557: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 21 MiB/s wr, 19 op/s 2023-07-21T19:25:17.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:17 smithi156 ceph-mon[128316]: pgmap v557: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 21 MiB/s wr, 19 op/s 2023-07-21T19:25:18.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:17 smithi101 ceph-mon[128136]: pgmap v557: 97 pgs: 97 active+clean; 14 GiB data, 6.7 GiB used, 1.0 TiB / 1.0 TiB avail; 187 KiB/s rd, 21 MiB/s wr, 19 op/s 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.351+0000 7fc1df59d700 1 -- 172.21.15.5:0/2842307333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 msgr2=0x7fc1d8103f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.351+0000 7fc1df59d700 1 --2- 172.21.15.5:0/2842307333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d8103f50 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fc1d40074e0 tx=0x7fc1d400af90 comp rx=0 tx=0).stop 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 -- 172.21.15.5:0/2842307333 shutdown_connections 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 --2- 172.21.15.5:0/2842307333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fe500 0x7fc1d810a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 --2- 172.21.15.5:0/2842307333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d8103f50 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 --2- 172.21.15.5:0/2842307333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fdb50 0x7fc1d80fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 -- 172.21.15.5:0/2842307333 >> 172.21.15.5:0/2842307333 conn(0x7fc1d80f95f0 msgr2=0x7fc1d80fba00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 -- 172.21.15.5:0/2842307333 shutdown_connections 2023-07-21T19:25:18.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.352+0000 7fc1df59d700 1 -- 172.21.15.5:0/2842307333 wait complete. 2023-07-21T19:25:18.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.353+0000 7fc1df59d700 1 Processor -- start 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.353+0000 7fc1df59d700 1 -- start start 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 0x7fc1d8197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fe500 0x7fc1d8198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d819cd00 con 0x7fc1d8103b80 2023-07-21T19:25:18.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d819ce70 con 0x7fc1d80fdb50 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1df59d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1d819d030 con 0x7fc1d80fe500 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1ded9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1ded9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57334/0 (socket says 172.21.15.5:57334) 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1ded9c700 1 -- 172.21.15.5:0/1802884131 learned_addr learned my addr 172.21.15.5:0/1802884131 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1de59b700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 0x7fc1d8197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.354+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fe500 0x7fc1d8198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 -- 172.21.15.5:0/1802884131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fe500 msgr2=0x7fc1d8198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fe500 0x7fc1d8198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 -- 172.21.15.5:0/1802884131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 msgr2=0x7fc1d8197d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 0x7fc1d8197d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 -- 172.21.15.5:0/1802884131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1d40071c0 con 0x7fc1d8103b80 2023-07-21T19:25:18.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1de59b700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 0x7fc1d8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1ded9c700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000b970 tx=0x7fc1d000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1cf7fe700 1 -- 172.21.15.5:0/1802884131 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d000d860 con 0x7fc1d8103b80 2023-07-21T19:25:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1cf7fe700 1 -- 172.21.15.5:0/1802884131 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1d000de50 con 0x7fc1d8103b80 2023-07-21T19:25:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.355+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1d819d310 con 0x7fc1d8103b80 2023-07-21T19:25:18.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.356+0000 7fc1cf7fe700 1 -- 172.21.15.5:0/1802884131 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0017400 con 0x7fc1d8103b80 2023-07-21T19:25:18.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.356+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1d81a4b50 con 0x7fc1d8103b80 2023-07-21T19:25:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.360+0000 7fc1cf7fe700 1 -- 172.21.15.5:0/1802884131 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc1d000d9c0 con 0x7fc1d8103b80 2023-07-21T19:25:18.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.361+0000 7fc1f409db80 1 Processor -- start 2023-07-21T19:25:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.361+0000 7fc1f409db80 1 -- start start 2023-07-21T19:25:18.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dddff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dcdfc50 con 0x55d98dde08c0 2023-07-21T19:25:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dd329e0 con 0x55d98dc366b0 2023-07-21T19:25:18.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1f409db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dd2be70 con 0x55d98dde0470 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1cf7fe700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 0x7fc1c40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57344/0 (socket says 172.21.15.5:57344) 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/4293213159 learned_addr learned my addr 172.21.15.5:0/4293213159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dddff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1de59b700 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1cf7fe700 1 -- 172.21.15.5:0/1802884131 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc1d0006e40 con 0x7fc1d8103b80 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.362+0000 7fc1ded9c700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 0x7fc1c40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/4293213159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 msgr2=0x55d98ddda1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/4293213159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 msgr2=0x55d98dddff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dddff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/4293213159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98db59d80 con 0x55d98dde08c0 2023-07-21T19:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006b30 tx=0x7fc1c8009590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b85849d07ae191 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.363+0000 7fc1ded9c700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 0x7fc1c40a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00963f0 tx=0x7fc1d0004be0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.364+0000 7fc1ccff9700 1 -- 172.21.15.5:0/4293213159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8018450 con 0x55d98dde08c0 2023-07-21T19:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.364+0000 7fc1ccff9700 1 -- 172.21.15.5:0/4293213159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1c8018a40 con 0x55d98dde08c0 2023-07-21T19:25:18.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.364+0000 7fc1ccff9700 1 -- 172.21.15.5:0/4293213159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8013820 con 0x55d98dde08c0 2023-07-21T19:25:18.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.364+0000 7fc1f409db80 1 -- 172.21.15.5:0/4293213159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 msgr2=0x55d98ddda700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.364+0000 7fc1f409db80 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8006b30 tx=0x7fc1c8009590 comp rx=0 tx=0).stop 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 -- 172.21.15.5:0/4293213159 shutdown_connections 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddda700 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dddff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 --2- 172.21.15.5:0/4293213159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 -- 172.21.15.5:0/4293213159 >> 172.21.15.5:0/4293213159 conn(0x55d98ddcfd40 msgr2=0x55d98ddd5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 -- 172.21.15.5:0/4293213159 shutdown_connections 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 -- 172.21.15.5:0/4293213159 wait complete. 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.365+0000 7fc1f409db80 1 -- start start 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 Processor -- start 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dde9880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde08c0 0x55d98dde3b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dccba60 con 0x55d98dde0470 2023-07-21T19:25:18.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dccb760 con 0x55d98dc366b0 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dbaf8c0 con 0x55d98dde08c0 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.366+0000 7fc1f409db80 10 client.0 _ll_register_callbacks cb 0x55d98dcc9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57348/0 (socket says 172.21.15.5:57348) 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 -- 172.21.15.5:0/1984428619 learned_addr learned my addr 172.21.15.5:0/1984428619 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde08c0 0x55d98dde3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dde9880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 -- 172.21.15.5:0/1984428619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde08c0 msgr2=0x55d98dde3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde08c0 0x55d98dde3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 -- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 msgr2=0x55d98dde9880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dde9880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.367+0000 7fc1de59b700 1 -- 172.21.15.5:0/1984428619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c8017460 con 0x55d98dde0470 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1de59b700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4015000 tx=0x7fc1d4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b9e162401078272 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:18.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d400f040 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc1d400c860 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d98dc85490 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1d400ca20 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1f409db80 10 client.18924 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.368+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55d98db42c00 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d98dc85490 con 0x55d98dde0470 2023-07-21T19:25:18.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1d4025020 con 0x55d98dde0470 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 10 client.18924 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 10 client.18924 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 20 client.18924 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98ddf3da0 0x55d98ddf61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 4 client.18924 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:18.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d98dd24e50 con 0x55d98ddf3da0 2023-07-21T19:25:18.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.369+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98ddf3da0 0x55d98ddf61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.370+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98ddf3da0 0x55d98ddf61b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37dab68890a1598c server_cookie=676040c522f93a3a in_seq=0 out_seq=0 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.370+0000 7fc1c1ffb700 10 client.18924 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.370+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/1984428619 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7fc1c802e780 con 0x55d98ddf3da0 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1c1ffb700 10 client.18924 handle_command_reply: tid=0 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 client.18924 shutdown 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 msgr2=0x55d98dde9dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fc1d4015000 tx=0x7fc1d4005270 comp rx=0 tx=0).stop 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 shutdown_connections 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde08c0 0x55d98dde3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dc366b0 0x55d98dde9880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98ddf3da0 0x55d98ddf61b0 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 --2- 172.21.15.5:0/1984428619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde0470 0x55d98dde9dc0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1c1ffb700 0 client.18924 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:18.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 >> 172.21.15.5:0/1984428619 conn(0x55d98ddcfd40 msgr2=0x55d98ddd4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 shutdown_connections 2023-07-21T19:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 1 -- 172.21.15.5:0/1984428619 wait complete. 2023-07-21T19:25:18.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.371+0000 7fc1f409db80 20 client.18924 trim_cache size 0 max 16384 2023-07-21T19:25:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.390+0000 7fc1f409db80 1 Processor -- start 2023-07-21T19:25:18.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 -- start start 2023-07-21T19:25:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dc366b0 0x55d98dddf460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dde4cd0 con 0x55d98dde08c0 2023-07-21T19:25:18.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98dddac70 con 0x55d98dde0470 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1f409db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98ddd6160 con 0x55d98dc366b0 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.391+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57356/0 (socket says 172.21.15.5:57356) 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/3127993336 learned_addr learned my addr 172.21.15.5:0/3127993336 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dc366b0 0x55d98dddf460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1de59b700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/3127993336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dc366b0 msgr2=0x55d98dddf460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dc366b0 0x55d98dddf460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/3127993336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 msgr2=0x55d98ddda1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/3127993336 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d98ddfade0 con 0x55d98dde08c0 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1de59b700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:18.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.392+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800a010 tx=0x7fc1c8006ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cfa9bb11e1d3530 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1cd7fa700 1 -- 172.21.15.5:0/3127993336 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8018650 con 0x55d98dde08c0 2023-07-21T19:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1cd7fa700 1 -- 172.21.15.5:0/3127993336 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1c802a040 con 0x55d98dde08c0 2023-07-21T19:25:18.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1cd7fa700 1 -- 172.21.15.5:0/3127993336 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8022400 con 0x55d98dde08c0 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1f409db80 1 -- 172.21.15.5:0/3127993336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 msgr2=0x55d98ddd4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1f409db80 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fc1c800a010 tx=0x7fc1c8006ac0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.393+0000 7fc1f409db80 1 -- 172.21.15.5:0/3127993336 shutdown_connections 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dde08c0 0x55d98ddd4dc0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dc366b0 0x55d98dddf460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 --2- 172.21.15.5:0/3127993336 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde0470 0x55d98ddda1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 -- 172.21.15.5:0/3127993336 >> 172.21.15.5:0/3127993336 conn(0x55d98ddcfd40 msgr2=0x55d98dde3aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 -- 172.21.15.5:0/3127993336 shutdown_connections 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 -- 172.21.15.5:0/3127993336 wait complete. 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 -- start start 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.394+0000 7fc1f409db80 1 Processor -- start 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98dde81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde08c0 0x55d98de40c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98de41170 con 0x55d98dc366b0 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98de412e0 con 0x55d98dde08c0 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d98de414a0 con 0x55d98dde0470 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1f409db80 10 client.0 _ll_register_callbacks cb 0x7fc1d4016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1ddd9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57366/0 (socket says 172.21.15.5:57366) 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/2444087635 learned_addr learned my addr 172.21.15.5:0/2444087635 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:18.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.395+0000 7fc1de59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98dde81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/2444087635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 msgr2=0x55d98dde81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98dde81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde08c0 msgr2=0x55d98de40c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde08c0 0x55d98de40c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 -- 172.21.15.5:0/2444087635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d98db42c00 con 0x55d98dc366b0 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1de59b700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98dde81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1ddd9a700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8037040 tx=0x7fc1c8006f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30e3966517bfdda4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:18.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1c8018650 con 0x55d98dc366b0 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc1c8018a40 con 0x55d98dc366b0 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.396+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc1c8000e40 con 0x55d98dc366b0 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc1c80187b0 con 0x55d98dc366b0 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 10 client.18930 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:18.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55d98db59d80 con 0x55d98dc366b0 2023-07-21T19:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1c804a020 con 0x55d98dc366b0 2023-07-21T19:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc1c803b070 con 0x55d98dc366b0 2023-07-21T19:25:18.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 10 client.18930 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 10 client.18930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 20 client.18930 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98de48850 0x55d98de4ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 4 client.18930 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.397+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d98dce6ff0 con 0x55d98de48850 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.398+0000 7fc1de59b700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98de48850 0x55d98de4ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.398+0000 7fc1de59b700 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98de48850 0x55d98de4ac60 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8feb559906d66865 server_cookie=38d5fae56d3eb470 in_seq=0 out_seq=0 2023-07-21T19:25:18.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.398+0000 7fc1c1ffb700 10 client.18930 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1c1ffb700 1 -- 172.21.15.5:0/2444087635 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d98dce6ff0 con 0x55d98de48850 2023-07-21T19:25:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1c1ffb700 10 client.18930 handle_command_reply: tid=0 2023-07-21T19:25:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 client.18930 shutdown 2023-07-21T19:25:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 msgr2=0x55d98dde7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fc1c8037040 tx=0x7fc1c8006f60 comp rx=0 tx=0).stop 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 shutdown_connections 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d98dde08c0 0x55d98de40c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d98de48850 0x55d98de4ac60 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d98dc366b0 0x55d98dde7c60 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 --2- 172.21.15.5:0/2444087635 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d98dde0470 0x55d98dde81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 >> 172.21.15.5:0/2444087635 conn(0x55d98ddcfd40 msgr2=0x55d98dde2350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1c1ffb700 0 client.18930 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 shutdown_connections 2023-07-21T19:25:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 1 -- 172.21.15.5:0/2444087635 wait complete. 2023-07-21T19:25:18.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.409+0000 7fc1f409db80 20 client.18930 trim_cache size 0 max 16384 2023-07-21T19:25:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:25:18.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:25:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:25:18.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:25:18.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 msgr2=0x7fc1c40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 0x7fc1c40a3770 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fc1d00963f0 tx=0x7fc1d0004be0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 msgr2=0x7fc1d819c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:18.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000b970 tx=0x7fc1d000bc80 comp rx=0 tx=0).stop 2023-07-21T19:25:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 shutdown_connections 2023-07-21T19:25:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc1c40a12c0 0x7fc1c40a3770 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc1d8103b80 0x7fc1d819c6c0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc1d80fdb50 0x7fc1d8197d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 --2- 172.21.15.5:0/1802884131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc1d80fe500 0x7fc1d8198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 >> 172.21.15.5:0/1802884131 conn(0x7fc1d80f95f0 msgr2=0x7fc1d80fad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:18.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 shutdown_connections 2023-07-21T19:25:18.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:18.410+0000 7fc1df59d700 1 -- 172.21.15.5:0/1802884131 wait complete. 2023-07-21T19:25:19.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:19 smithi005 ceph-mon[126460]: pgmap v558: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:19.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:19 smithi156 ceph-mon[128316]: pgmap v558: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:20.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:19 smithi101 ceph-mon[128136]: pgmap v558: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:21.738 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:25:21.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:21 smithi005 ceph-mon[126460]: pgmap v559: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:21.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:21 smithi156 ceph-mon[128316]: pgmap v559: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 -- 172.21.15.5:0/838521379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd8101490 msgr2=0x7f8bd8109b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:21.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/838521379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd8101490 0x7f8bd8109b60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009990 tx=0x7f8bcc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 -- 172.21.15.5:0/838521379 shutdown_connections 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/838521379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd810a0a0 0x7f8bd810c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/838521379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8100b80 0x7f8bd8100f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/838521379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd8101490 0x7f8bd8109b60 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 -- 172.21.15.5:0/838521379 >> 172.21.15.5:0/838521379 conn(0x7f8bd80fa4b0 msgr2=0x7f8bd80fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.983+0000 7f8bdcfff700 1 -- 172.21.15.5:0/838521379 shutdown_connections 2023-07-21T19:25:21.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.984+0000 7f8bdcfff700 1 -- 172.21.15.5:0/838521379 wait complete. 2023-07-21T19:25:21.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.985+0000 7f8bdcfff700 1 Processor -- start 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.985+0000 7f8bdcfff700 1 -- start start 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.985+0000 7f8bdcfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd8100b80 0x7f8bd80fe9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.985+0000 7f8bdcfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 0x7f8bd80feef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.985+0000 7f8bdcfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bdcfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd80ff720 con 0x7f8bd810a0a0 2023-07-21T19:25:21.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bdcfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81a4280 con 0x7f8bd8100b80 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bdcfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd81a43c0 con 0x7f8bd8101490 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57382/0 (socket says 172.21.15.5:57382) 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 -- 172.21.15.5:0/2128399499 learned_addr learned my addr 172.21.15.5:0/2128399499 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd659c700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd8100b80 0x7f8bd80fe9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 0x7f8bd80feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 -- 172.21.15.5:0/2128399499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 msgr2=0x7f8bd80feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 0x7f8bd80feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 -- 172.21.15.5:0/2128399499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd8100b80 msgr2=0x7f8bd80fe9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:21.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd8100b80 0x7f8bd80fe9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd6d9d700 1 -- 172.21.15.5:0/2128399499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bcc0095f0 con 0x7f8bd810a0a0 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.986+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 0x7f8bd80feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bd6d9d700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ef90 tx=0x7f8bd000c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2128399499 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0018070 con 0x7f8bd810a0a0 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bd81a45c0 con 0x7f8bd810a0a0 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2128399499 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bd000f040 con 0x7f8bd810a0a0 2023-07-21T19:25:21.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2128399499 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd0003d70 con 0x7f8bd810a0a0 2023-07-21T19:25:21.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.987+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd81a4aa0 con 0x7f8bd810a0a0 2023-07-21T19:25:21.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.991+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2128399499 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8bd000ca50 con 0x7f8bd810a0a0 2023-07-21T19:25:21.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.992+0000 7f8bf0971b80 1 Processor -- start 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 -- start start 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbf590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefbfad0 0x558faefc1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faeebacb0 con 0x558faefb4be0 2023-07-21T19:25:21.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faef0da40 con 0x558faefb42d0 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.993+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faef06ed0 con 0x558faefbfad0 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54528/0 (socket says 172.21.15.5:54528) 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/3644289077 learned_addr learned my addr 172.21.15.5:0/3644289077 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:21.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bc77fe700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 0x7f8bc00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefbfad0 0x558faefc1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bc77fe700 1 -- 172.21.15.5:0/2128399499 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bd00cc930 con 0x7f8bd810a0a0 2023-07-21T19:25:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd6d9d700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 0x7f8bc00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/3644289077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefbfad0 msgr2=0x558faefc1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:21.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefbfad0 0x558faefc1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/3644289077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 msgr2=0x558faefbf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/3644289077 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558faed34de0 con 0x558faefb42d0 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.994+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009990 tx=0x7f8bcc01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=935a5a7b546dbce4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.995+0000 7f8bd6d9d700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 0x7f8bc00a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0004fd0 tx=0x7f8bd0004c60 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.995+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3644289077 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc00bb70 con 0x558faefb42d0 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.995+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3644289077 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bcc01e070 con 0x558faefb42d0 2023-07-21T19:25:21.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.995+0000 7f8bc4ff9700 1 -- 172.21.15.5:0/3644289077 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc0049a0 con 0x558faefb42d0 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3644289077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 msgr2=0x558faefb46a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc009990 tx=0x7f8bcc01a040 comp rx=0 tx=0).stop 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3644289077 shutdown_connections 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefbfad0 0x558faefc1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb42d0 0x558faefb46a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3644289077 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3644289077 >> 172.21.15.5:0/3644289077 conn(0x558faefaada0 msgr2=0x558faefb06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3644289077 shutdown_connections 2023-07-21T19:25:21.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.996+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3644289077 wait complete. 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 -- start start 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 Processor -- start 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefbf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefbcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:21.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faeea6ac0 con 0x558faefb4be0 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faeea67c0 con 0x558faefbfad0 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.997+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faed8a920 con 0x558faefb42d0 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bf0971b80 10 client.0 _ll_register_callbacks cb 0x558faeea4760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57392/0 (socket says 172.21.15.5:57392) 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 -- 172.21.15.5:0/3575116438 learned_addr learned my addr 172.21.15.5:0/3575116438 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:21.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefbf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefbcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 -- 172.21.15.5:0/3575116438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 msgr2=0x558faefbf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefbf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 -- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 msgr2=0x558faefbcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefbcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd659c700 1 -- 172.21.15.5:0/3575116438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bd810d180 con 0x558faefb4be0 2023-07-21T19:25:21.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.998+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefbcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bd659c700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b960 tx=0x7f8bc800bc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b59477f17654710 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8016070 con 0x558faefb4be0 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558faee604f0 con 0x558faefb4be0 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bc8003d00 con 0x558faefb4be0 2023-07-21T19:25:21.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bc8011440 con 0x558faefb4be0 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bf0971b80 10 client.18942 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:21.999+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x558faed1dc60 con 0x558faefb4be0 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bc8027080 con 0x558faefb4be0 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bc8021030 con 0x558faefb4be0 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 10 client.18942 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 10 client.18942 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 20 client.18942 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:21.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faefcf480 0x558faefd1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 4 client.18942 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558faeeffeb0 con 0x558faefcf480 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.000+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faefcf480 0x558faefd1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.001+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faefcf480 0x558faefd1890 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e4811b241eda4e server_cookie=e7ac1635071f4a18 in_seq=0 out_seq=0 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.001+0000 7f8bbdffb700 10 client.18942 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bbdffb700 1 -- 172.21.15.5:0/3575116438 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558faeeffeb0 con 0x558faefcf480 2023-07-21T19:25:22.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bbdffb700 10 client.18942 handle_command_reply: tid=0 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 client.18942 shutdown 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 msgr2=0x558faefbc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b960 tx=0x7f8bc800bc70 comp rx=0 tx=0).stop 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 shutdown_connections 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefbcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefbf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faefcf480 0x558faefd1890 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/3575116438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb4be0 0x558faefbc7f0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bbdffb700 0 client.18942 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.002+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 >> 172.21.15.5:0/3575116438 conn(0x558faefaada0 msgr2=0x558faefb0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.003+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 shutdown_connections 2023-07-21T19:25:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.003+0000 7f8bf0971b80 1 -- 172.21.15.5:0/3575116438 wait complete. 2023-07-21T19:25:22.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.003+0000 7f8bf0971b80 20 client.18942 trim_cache size 0 max 16384 2023-07-21T19:25:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.022+0000 7f8bf0971b80 1 Processor -- start 2023-07-21T19:25:22.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.022+0000 7f8bf0971b80 1 -- start start 2023-07-21T19:25:22.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.022+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 0x558faefb7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.022+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 0x558faefb7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefb80b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faefacec0 con 0x558faefb42d0 2023-07-21T19:25:22.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faefb1d20 con 0x558faefbfad0 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faefb5540 con 0x558faefb4be0 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 0x558faefb7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 0x558faefb7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57406/0 (socket says 172.21.15.5:57406) 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 0x558faefb7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/2943607603 learned_addr learned my addr 172.21.15.5:0/2943607603 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefb80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 -- 172.21.15.5:0/2943607603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 msgr2=0x558faefb80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefb80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 -- 172.21.15.5:0/2943607603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 msgr2=0x558faefb7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 0x558faefb7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.023+0000 7f8bd659c700 1 -- 172.21.15.5:0/2943607603 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558faed5b250 con 0x558faefb4be0 2023-07-21T19:25:22.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.024+0000 7f8bd659c700 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 0x558faefb7b70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b930 tx=0x7f8bc8005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f19b51b7d1ba9127 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.024+0000 7f8bc57fa700 1 -- 172.21.15.5:0/2943607603 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc802a070 con 0x558faefb4be0 2023-07-21T19:25:22.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.024+0000 7f8bc57fa700 1 -- 172.21.15.5:0/2943607603 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558faee604f0 con 0x558faefb4be0 2023-07-21T19:25:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.024+0000 7f8bc57fa700 1 -- 172.21.15.5:0/2943607603 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bc8016070 con 0x558faefb4be0 2023-07-21T19:25:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 -- 172.21.15.5:0/2943607603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 msgr2=0x558faefb7b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 0x558faefb7b70 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800b930 tx=0x7f8bc8005ce0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 -- 172.21.15.5:0/2943607603 shutdown_connections 2023-07-21T19:25:22.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefbfad0 0x558faefb80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefb42d0 0x558faefb7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/2943607603 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb4be0 0x558faefb7b70 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 -- 172.21.15.5:0/2943607603 >> 172.21.15.5:0/2943607603 conn(0x558faefaada0 msgr2=0x558faefcb760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 -- 172.21.15.5:0/2943607603 shutdown_connections 2023-07-21T19:25:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.025+0000 7f8bf0971b80 1 -- 172.21.15.5:0/2943607603 wait complete. 2023-07-21T19:25:22.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.026+0000 7f8bf0971b80 1 -- start start 2023-07-21T19:25:22.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.026+0000 7f8bf0971b80 1 Processor -- start 2023-07-21T19:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefc66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb4be0 0x558faefc4a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faefc58c0 con 0x558faefbfad0 2023-07-21T19:25:22.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf023310 con 0x558faefb4be0 2023-07-21T19:25:22.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558faf023450 con 0x558faefb42d0 2023-07-21T19:25:22.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bf0971b80 10 client.0 _ll_register_callbacks cb 0x7f8bc8004280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bd5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57408/0 (socket says 172.21.15.5:57408) 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/1507726692 learned_addr learned my addr 172.21.15.5:0/1507726692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefc66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.027+0000 7f8bd659c700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb4be0 0x558faefc4a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/1507726692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 msgr2=0x558faefc66d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefc66d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb4be0 msgr2=0x558faefc4a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb4be0 0x558faefc4a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 -- 172.21.15.5:0/1507726692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558faed1dc60 con 0x558faefbfad0 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bd5d9b700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b2e0 tx=0x7f8bcc005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d020a631de7f066e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bbdffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc037a10 con 0x558faefbfad0 2023-07-21T19:25:22.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8bcc01e070 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.028+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bcc037b70 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bcc037d30 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 10 client.18948 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x558faed332b0 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bcc01e070 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8bcc00f460 con 0x558faefbfad0 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 10 client.18948 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:22.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 10 client.18948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 20 client.18948 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faf0246c0 0x558faf026aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 4 client.18948 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.029+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558faeec2050 con 0x558faf0246c0 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.030+0000 7f8bd659c700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faf0246c0 0x558faf026aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.030+0000 7f8bd659c700 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faf0246c0 0x558faf026aa0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5982f12b61445b7d server_cookie=c2ad271d61147d8f in_seq=0 out_seq=0 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.030+0000 7f8bbdffb700 10 client.18948 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bbdffb700 1 -- 172.21.15.5:0/1507726692 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558faeec2050 con 0x558faf0246c0 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bbdffb700 10 client.18948 handle_command_reply: tid=0 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 client.18948 shutdown 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 msgr2=0x558faefc5180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f8bcc00b2e0 tx=0x7f8bcc005740 comp rx=0 tx=0).stop 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 shutdown_connections 2023-07-21T19:25:22.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558faf0246c0 0x558faf026aa0 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558faefbfad0 0x558faefc5180 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bbdffb700 0 client.18948 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558faefb42d0 0x558faefc66d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 --2- 172.21.15.5:0/1507726692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558faefb4be0 0x558faefc4a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 >> 172.21.15.5:0/1507726692 conn(0x558faefaada0 msgr2=0x558faefb7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 shutdown_connections 2023-07-21T19:25:22.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 1 -- 172.21.15.5:0/1507726692 wait complete. 2023-07-21T19:25:22.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.031+0000 7f8bf0971b80 20 client.18948 trim_cache size 0 max 16384 2023-07-21T19:25:22.037 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:25:22.037 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:25:22.037 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:25:22.038 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:25:22.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 msgr2=0x7f8bc00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 0x7f8bc00a3770 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0004fd0 tx=0x7f8bd0004c60 comp rx=0 tx=0).stop 2023-07-21T19:25:22.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 msgr2=0x7f8bd81a3d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f8bd000ef90 tx=0x7f8bd000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 shutdown_connections 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bc00a12c0 0x7f8bc00a3770 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8bd810a0a0 0x7f8bd81a3d40 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8bd8100b80 0x7f8bd80fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 --2- 172.21.15.5:0/2128399499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8bd8101490 0x7f8bd80feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 >> 172.21.15.5:0/2128399499 conn(0x7f8bd80fa4b0 msgr2=0x7f8bd80fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 shutdown_connections 2023-07-21T19:25:22.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.032+0000 7f8bdcfff700 1 -- 172.21.15.5:0/2128399499 wait complete. 2023-07-21T19:25:22.042 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:25:22.043 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:25:22.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:21 smithi101 ceph-mon[128136]: pgmap v559: 97 pgs: 97 active+clean; 15 GiB data, 6.8 GiB used, 1.0 TiB / 1.0 TiB avail; 199 KiB/s rd, 21 MiB/s wr, 17 op/s 2023-07-21T19:25:22.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 -- 172.21.15.5:0/1059343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c40731c0 msgr2=0x7f35c410bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1059343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c40731c0 0x7f35c410bdd0 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f35b8009990 tx=0x7f35b8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 -- 172.21.15.5:0/1059343296 shutdown_connections 2023-07-21T19:25:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1059343296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c410c310 0x7f35c410e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1059343296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40728b0 0x7f35c4072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1059343296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c40731c0 0x7f35c410bdd0 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 -- 172.21.15.5:0/1059343296 >> 172.21.15.5:0/1059343296 conn(0x7f35c40fb590 msgr2=0x7f35c40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 -- 172.21.15.5:0/1059343296 shutdown_connections 2023-07-21T19:25:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.279+0000 7f35c99ff700 1 -- 172.21.15.5:0/1059343296 wait complete. 2023-07-21T19:25:22.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.280+0000 7f35c99ff700 1 Processor -- start 2023-07-21T19:25:22.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.280+0000 7f35c99ff700 1 -- start start 2023-07-21T19:25:22.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c40728b0 0x7f35c4197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40731c0 0x7f35c4198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c419bbc0 con 0x7f35c410c310 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c419bd30 con 0x7f35c40731c0 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c99ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35c419bef0 con 0x7f35c40728b0 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57430/0 (socket says 172.21.15.5:57430) 2023-07-21T19:25:22.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c37fe700 1 -- 172.21.15.5:0/1466130351 learned_addr learned my addr 172.21.15.5:0/1466130351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c27fc700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40731c0 0x7f35c4198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.281+0000 7f35c2ffd700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c40728b0 0x7f35c4197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 -- 172.21.15.5:0/1466130351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c40728b0 msgr2=0x7f35c4197cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c40728b0 0x7f35c4197cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 -- 172.21.15.5:0/1466130351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40731c0 msgr2=0x7f35c4198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40731c0 0x7f35c4198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 -- 172.21.15.5:0/1466130351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b80095f0 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c37fe700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00ef30 tx=0x7f35bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc018070 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35c419c170 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35bc00f040 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35bc00cb80 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.282+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35c41a4b30 con 0x7f35c410c310 2023-07-21T19:25:22.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.283+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35c4067300 con 0x7f35c410c310 2023-07-21T19:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.287+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f35bc00cd40 con 0x7f35c410c310 2023-07-21T19:25:22.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.289+0000 7f35c89fd700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 0x7f35ac0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.289+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f35bc0ccdb0 con 0x7f35c410c310 2023-07-21T19:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.289+0000 7f35c2ffd700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 0x7f35ac0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.289+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f35bc099c80 con 0x7f35c410c310 2023-07-21T19:25:22.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.290+0000 7f35c2ffd700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 0x7f35ac0a37b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009c60 tx=0x7f35b4009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.580+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35c41a4cc0 con 0x7f35c410c310 2023-07-21T19:25:22.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.581+0000 7f35c89fd700 1 -- 172.21.15.5:0/1466130351 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+10836 (secure 0 0 0) 0x7f35bc099810 con 0x7f35c410c310 2023-07-21T19:25:22.572 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:25:22.573 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":54,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":54,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:24:56.603176+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:25:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.584+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 msgr2=0x7f35ac0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.584+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 0x7f35ac0a37b0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f35b4009c60 tx=0x7f35b4009260 comp rx=0 tx=0).stop 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.584+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 msgr2=0x7f35c419b5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.584+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f35bc00ef30 tx=0x7f35bc00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 shutdown_connections 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35ac0a1300 0x7f35ac0a37b0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35c410c310 0x7f35c419b5c0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35c40728b0 0x7f35c4197cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 --2- 172.21.15.5:0/1466130351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35c40731c0 0x7f35c4198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 >> 172.21.15.5:0/1466130351 conn(0x7f35c40fb590 msgr2=0x7f35c410e3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 shutdown_connections 2023-07-21T19:25:22.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.585+0000 7f35c99ff700 1 -- 172.21.15.5:0/1466130351 wait complete. 2023-07-21T19:25:22.580 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 54 2023-07-21T19:25:22.597 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:25:22.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/2582876275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e38100380 msgr2=0x7f5e3810c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/2582876275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e38100380 0x7f5e3810c810 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f5e3000b0a0 tx=0x7f5e3000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/2582876275 shutdown_connections 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/2582876275 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e38100380 0x7f5e3810c810 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/2582876275 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e381059e0 0x7f5e38105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/2582876275 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e380ff9d0 0x7f5e380ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.860+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/2582876275 >> 172.21.15.5:0/2582876275 conn(0x7f5e380fb4b0 msgr2=0x7f5e380fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.861+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/2582876275 shutdown_connections 2023-07-21T19:25:22.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.861+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/2582876275 wait complete. 2023-07-21T19:25:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.862+0000 7f5e3ed9d700 1 Processor -- start 2023-07-21T19:25:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.862+0000 7f5e3ed9d700 1 -- start start 2023-07-21T19:25:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38100380 0x7f5e3819de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e381059e0 0x7f5e38197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38198000 con 0x7f5e380ff9d0 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38198170 con 0x7f5e381059e0 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3ed9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e38198330 con 0x7f5e38100380 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57448/0 (socket says 172.21.15.5:57448) 2023-07-21T19:25:22.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/3370230239 learned_addr learned my addr 172.21.15.5:0/3370230239 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38100380 0x7f5e3819de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e381059e0 0x7f5e38197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/3370230239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38100380 msgr2=0x7f5e3819de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38100380 0x7f5e3819de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/3370230239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e381059e0 msgr2=0x7f5e38197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e381059e0 0x7f5e38197990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.863+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/3370230239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e30009d00 con 0x7f5e380ff9d0 2023-07-21T19:25:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e970 tx=0x7f5e3400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e2effd700 1 -- 172.21.15.5:0/3370230239 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34009790 con 0x7f5e380ff9d0 2023-07-21T19:25:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e2effd700 1 -- 172.21.15.5:0/3370230239 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e3400eec0 con 0x7f5e380ff9d0 2023-07-21T19:25:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e38198610 con 0x7f5e380ff9d0 2023-07-21T19:25:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e2effd700 1 -- 172.21.15.5:0/3370230239 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34003c10 con 0x7f5e380ff9d0 2023-07-21T19:25:22.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.864+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e381a4910 con 0x7f5e380ff9d0 2023-07-21T19:25:22.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.868+0000 7f5e2effd700 1 -- 172.21.15.5:0/3370230239 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5e34003db0 con 0x7f5e380ff9d0 2023-07-21T19:25:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.869+0000 7f5e5393bb80 1 Processor -- start 2023-07-21T19:25:22.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.869+0000 7f5e5393bb80 1 -- start start 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619edb8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619de0cb0 con 0x561619ee5d30 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619e33a40 con 0x561619edb4f0 2023-07-21T19:25:22.861 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619e2ced0 con 0x561619edbe00 2023-07-21T19:25:22.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3d59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619edb8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619edb8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54594/0 (socket says 172.21.15.5:54594) 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee57f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36760/0 (socket says 172.21.15.5:36760) 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.870+0000 7f5e3e59c700 1 -- 172.21.15.5:0/17872555 learned_addr learned my addr 172.21.15.5:0/17872555 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e2effd700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 0x7f5e240a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 -- 172.21.15.5:0/17872555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 msgr2=0x561619ee57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e2effd700 1 -- 172.21.15.5:0/3370230239 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e34014070 con 0x7f5e380ff9d0 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 -- 172.21.15.5:0/17872555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 msgr2=0x561619edb8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619edb8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 -- 172.21.15.5:0/17872555 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561619c5ade0 con 0x561619ee5d30 2023-07-21T19:25:22.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 0x7f5e240a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.871+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee8110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009c80 tx=0x7f5e28009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76e5489e3271db85 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e23fff700 1 -- 172.21.15.5:0/17872555 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28011cb0 con 0x561619ee5d30 2023-07-21T19:25:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e23fff700 1 -- 172.21.15.5:0/17872555 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e28011e10 con 0x561619ee5d30 2023-07-21T19:25:22.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e23fff700 1 -- 172.21.15.5:0/17872555 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28005280 con 0x561619ee5d30 2023-07-21T19:25:22.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e5393bb80 1 -- 172.21.15.5:0/17872555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 msgr2=0x561619ee8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 0x7f5e240a3890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009850 tx=0x7f5e300099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.872+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee8110 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f5e28009c80 tx=0x7f5e28009f90 comp rx=0 tx=0).stop 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 -- 172.21.15.5:0/17872555 shutdown_connections 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee8110 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619edb8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/17872555 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 -- 172.21.15.5:0/17872555 >> 172.21.15.5:0/17872555 conn(0x561619ed0ff0 msgr2=0x561619ed6930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 -- 172.21.15.5:0/17872555 shutdown_connections 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 -- 172.21.15.5:0/17872555 wait complete. 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.873+0000 7f5e5393bb80 1 -- start start 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 Processor -- start 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee2930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.869 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee2f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619dccac0 con 0x561619ee5d30 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619dcc7c0 con 0x561619edb4f0 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619cb0920 con 0x561619edbe00 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e5393bb80 10 client.0 _ll_register_callbacks cb 0x561619dca760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.874+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57470/0 (socket says 172.21.15.5:57470) 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3e59c700 1 -- 172.21.15.5:0/360804758 learned_addr learned my addr 172.21.15.5:0/360804758 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee2930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 msgr2=0x561619ee2930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee2930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/360804758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 msgr2=0x561619ee2f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee2f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/360804758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e280098e0 con 0x561619edb4f0 2023-07-21T19:25:22.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e340ca600 tx=0x7f5e34099e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd773eefc9b0d781 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.875+0000 7f5e217fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3400ccd0 con 0x561619edb4f0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e34099f80 con 0x561619edb4f0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561619d864f0 con 0x561619edb4f0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3409dfb0 con 0x561619edb4f0 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e5393bb80 10 client.28607 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:22.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x561619c43c60 con 0x561619edb4f0 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561619d864f0 con 0x561619edb4f0 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.876+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e340a1f40 con 0x561619edb4f0 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 10 client.28607 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 10 client.28607 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 20 client.28607 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619ef5730 0x561619ef7b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 4 client.28607 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561619e25eb0 con 0x561619ef5730 2023-07-21T19:25:22.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.877+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619ef5730 0x561619ef7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.878+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619ef5730 0x561619ef7b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=655edfb459e77641 server_cookie=e4cd1dcd198e4116 in_seq=0 out_seq=0 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.878+0000 7f5e217fa700 10 client.28607 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.878+0000 7f5e217fa700 1 -- 172.21.15.5:0/360804758 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561619e25eb0 con 0x561619ef5730 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.878+0000 7f5e217fa700 10 client.28607 handle_command_reply: tid=0 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.878+0000 7f5e5393bb80 1 client.28607 shutdown 2023-07-21T19:25:22.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 msgr2=0x561619ee5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5720 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5e340ca600 tx=0x7f5e34099e40 comp rx=0 tx=0).stop 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 shutdown_connections 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619ee2f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619ef5730 0x561619ef7b40 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5720 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/360804758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edbe00 0x561619ee2930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e217fa700 0 client.28607 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 >> 172.21.15.5:0/360804758 conn(0x561619ed0ff0 msgr2=0x561619ed6520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 shutdown_connections 2023-07-21T19:25:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 1 -- 172.21.15.5:0/360804758 wait complete. 2023-07-21T19:25:22.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.879+0000 7f5e5393bb80 20 client.28607 trim_cache size 0 max 16384 2023-07-21T19:25:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.900+0000 7f5e5393bb80 1 Processor -- start 2023-07-21T19:25:22.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.900+0000 7f5e5393bb80 1 -- start start 2023-07-21T19:25:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 0x561619eddc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619ee5d30 0x561619ede1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619ed0e50 con 0x561619edbe00 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619edc9b0 con 0x561619edb4f0 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619ef2d60 con 0x561619ee5d30 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e3d59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 0x561619eddc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e3dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e3d59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 0x561619eddc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57474/0 (socket says 172.21.15.5:57474) 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.901+0000 7f5e3d59a700 1 -- 172.21.15.5:0/1422677148 learned_addr learned my addr 172.21.15.5:0/1422677148 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619ee5d30 0x561619ede1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/1422677148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619ee5d30 msgr2=0x561619ede1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619ee5d30 0x561619ede1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/1422677148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 msgr2=0x561619eddc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 0x561619eddc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 -- 172.21.15.5:0/1422677148 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561619c81250 con 0x561619edb4f0 2023-07-21T19:25:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e340969f0 tx=0x7f5e340d03e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2434428235246803 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e2cff9700 1 -- 172.21.15.5:0/1422677148 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34099ac0 con 0x561619edb4f0 2023-07-21T19:25:22.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e2cff9700 1 -- 172.21.15.5:0/1422677148 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e3400ccd0 con 0x561619edb4f0 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.902+0000 7f5e2cff9700 1 -- 172.21.15.5:0/1422677148 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34094e50 con 0x561619edb4f0 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 -- 172.21.15.5:0/1422677148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 msgr2=0x561619ee5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f5e340969f0 tx=0x7f5e340d03e0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 -- 172.21.15.5:0/1422677148 shutdown_connections 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619ee5d30 0x561619ede1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edb4f0 0x561619ee5250 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/1422677148 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619edbe00 0x561619eddc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 -- 172.21.15.5:0/1422677148 >> 172.21.15.5:0/1422677148 conn(0x561619ed0ff0 msgr2=0x561619ef0fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 -- 172.21.15.5:0/1422677148 shutdown_connections 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.903+0000 7f5e5393bb80 1 -- 172.21.15.5:0/1422677148 wait complete. 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.904+0000 7f5e5393bb80 1 -- start start 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.904+0000 7f5e5393bb80 1 Processor -- start 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edb4f0 0x561619eec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edbe00 0x561619eec6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619eeac50 con 0x561619ee5d30 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619eeadc0 con 0x561619edbe00 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561619eeaf80 con 0x561619edb4f0 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e5393bb80 10 client.0 _ll_register_callbacks cb 0x7f5e340962c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57482/0 (socket says 172.21.15.5:57482) 2023-07-21T19:25:22.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 -- 172.21.15.5:0/2181895705 learned_addr learned my addr 172.21.15.5:0/2181895705 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edb4f0 0x561619eec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3d59a700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edbe00 0x561619eec6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 -- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edb4f0 msgr2=0x561619eec190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edb4f0 0x561619eec190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 -- 172.21.15.5:0/2181895705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edbe00 msgr2=0x561619eec6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edbe00 0x561619eec6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.905+0000 7f5e3e59c700 1 -- 172.21.15.5:0/2181895705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561619c43c60 con 0x561619ee5d30 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e3e59c700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e300062a0 tx=0x7f5e3001f070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ee4f56d77c85845 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e217fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3002c040 con 0x561619ee5d30 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5e28011e10 con 0x561619ee5d30 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5e30007e20 con 0x561619ee5d30 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e3000e040 con 0x561619ee5d30 2023-07-21T19:25:22.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e5393bb80 10 client.18972 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.906+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x561619c592b0 con 0x561619ee5d30 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5e3003e020 con 0x561619ee5d30 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5e300050d0 con 0x561619ee5d30 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 10 client.18972 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 10 client.18972 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 20 client.18972 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619f4a6e0 0x561619f4caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:22.903 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 4 client.18972 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:25:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x561619de8050 con 0x561619f4a6e0 2023-07-21T19:25:22.904 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.907+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619f4a6e0 0x561619f4caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:22.904 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:22 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1466130351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e217fa700 10 client.18972 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e3dd9b700 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619f4a6e0 0x561619f4caf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dbf37df5addabd37 server_cookie=70435e026c962f9f in_seq=0 out_seq=0 2023-07-21T19:25:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e217fa700 1 -- 172.21.15.5:0/2181895705 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x561619de8050 con 0x561619f4a6e0 2023-07-21T19:25:22.905 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e217fa700 10 client.18972 handle_command_reply: tid=0 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e5393bb80 1 client.18972 shutdown 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 msgr2=0x561619eea5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.908+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f5e300062a0 tx=0x7f5e3001f070 comp rx=0 tx=0).stop 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 shutdown_connections 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561619ee5d30 0x561619eea5c0 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561619edb4f0 0x561619eec190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561619f4a6e0 0x561619f4caf0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.906 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 --2- 172.21.15.5:0/2181895705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561619edbe00 0x561619eec6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 >> 172.21.15.5:0/2181895705 conn(0x561619ed0ff0 msgr2=0x561619eddc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e217fa700 0 client.18972 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 shutdown_connections 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 1 -- 172.21.15.5:0/2181895705 wait complete. 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e5393bb80 20 client.18972 trim_cache size 0 max 16384 2023-07-21T19:25:22.907 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:25:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 msgr2=0x7f5e240a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.909+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 0x7f5e240a3890 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f5e30009850 tx=0x7f5e300099e0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.908 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 msgr2=0x7f5e3819d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f5e3400e970 tx=0x7f5e3400ec80 comp rx=0 tx=0).stop 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 shutdown_connections 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5e381059e0 0x7f5e38197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5e240a13e0 0x7f5e240a3890 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5e380ff9d0 0x7f5e3819d920 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 --2- 172.21.15.5:0/3370230239 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5e38100380 0x7f5e3819de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:22.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 >> 172.21.15.5:0/3370230239 conn(0x7f5e380fb4b0 msgr2=0x7f5e38072740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 shutdown_connections 2023-07-21T19:25:22.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:22.910+0000 7f5e3ed9d700 1 -- 172.21.15.5:0/3370230239 wait complete. 2023-07-21T19:25:22.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:22 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1466130351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:22.921 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:25:23.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:22 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1466130351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:23.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.165+0000 7f41db59e700 1 -- 172.21.15.5:0/3621766602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 msgr2=0x7f41dc100e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.165+0000 7f41db59e700 1 --2- 172.21.15.5:0/3621766602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc100e70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f41cc009990 tx=0x7f41cc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 -- 172.21.15.5:0/3621766602 shutdown_connections 2023-07-21T19:25:23.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 --2- 172.21.15.5:0/3621766602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 0x7f41dc10c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 --2- 172.21.15.5:0/3621766602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc100e70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 --2- 172.21.15.5:0/3621766602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc109a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 -- 172.21.15.5:0/3621766602 >> 172.21.15.5:0/3621766602 conn(0x7f41dc0745d0 msgr2=0x7f41dc0749d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 -- 172.21.15.5:0/3621766602 shutdown_connections 2023-07-21T19:25:23.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.166+0000 7f41db59e700 1 -- 172.21.15.5:0/3621766602 wait complete. 2023-07-21T19:25:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.167+0000 7f41db59e700 1 Processor -- start 2023-07-21T19:25:23.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.167+0000 7f41db59e700 1 -- start start 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc0fee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 0x7f41dc1a3ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc0ff640 con 0x7f41dc100aa0 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc1a4020 con 0x7f41dc1013b0 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41db59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41dc1a4160 con 0x7f41dc109fc0 2023-07-21T19:25:23.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41da59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41da59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57500/0 (socket says 172.21.15.5:57500) 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41da59c700 1 -- 172.21.15.5:0/4048738629 learned_addr learned my addr 172.21.15.5:0/4048738629 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc0fee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.168+0000 7f41dad9d700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 0x7f41dc1a3ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 -- 172.21.15.5:0/4048738629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 msgr2=0x7f41dc1a3ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 0x7f41dc1a3ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 -- 172.21.15.5:0/4048738629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 msgr2=0x7f41dc0fee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc0fee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 -- 172.21.15.5:0/4048738629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41cc0095f0 con 0x7f41dc100aa0 2023-07-21T19:25:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41da59c700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f41cc005740 tx=0x7f41cc005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41cb7fe700 1 -- 172.21.15.5:0/4048738629 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc01d070 con 0x7f41dc100aa0 2023-07-21T19:25:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41cb7fe700 1 -- 172.21.15.5:0/4048738629 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41cc022470 con 0x7f41dc100aa0 2023-07-21T19:25:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41cb7fe700 1 -- 172.21.15.5:0/4048738629 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc017650 con 0x7f41dc100aa0 2023-07-21T19:25:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41dc1a4360 con 0x7f41dc100aa0 2023-07-21T19:25:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.169+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc1a4780 con 0x7f41dc100aa0 2023-07-21T19:25:23.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.173+0000 7f41cb7fe700 1 -- 172.21.15.5:0/4048738629 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f41cc00fb00 con 0x7f41dc100aa0 2023-07-21T19:25:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.174+0000 7f41f46a7b80 1 Processor -- start 2023-07-21T19:25:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 -- start start 2023-07-21T19:25:23.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41cb7fe700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 0x7f41c40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41cb7fe700 1 -- 172.21.15.5:0/4048738629 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f41cc0ce120 con 0x7f41dc100aa0 2023-07-21T19:25:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c2000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7c2540 0x56332a7c4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a6b4cb0 con 0x56332a7afe00 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a707a40 con 0x56332a7c2540 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a700ed0 con 0x56332a7af4f0 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.175+0000 7f41dad9d700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 0x7f41c40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36820/0 (socket says 172.21.15.5:36820) 2023-07-21T19:25:23.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 -- 172.21.15.5:0/1443469227 learned_addr learned my addr 172.21.15.5:0/1443469227 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41dad9d700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7c2540 0x56332a7c4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c2000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 -- 172.21.15.5:0/1443469227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7c2540 msgr2=0x56332a7c4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7c2540 0x56332a7c4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 -- 172.21.15.5:0/1443469227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 msgr2=0x56332a7c2000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c2000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 -- 172.21.15.5:0/1443469227 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332a52ede0 con 0x56332a7af4f0 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41dad9d700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 0x7f41c40a3890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41dc1009e0 tx=0x7f41d4009ba0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.176+0000 7f41da59c700 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0a30b0 tx=0x7f41cc09ade0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6fbf492f2ff4817 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.177+0000 7f41c8ff9700 1 -- 172.21.15.5:0/1443469227 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0a10e0 con 0x56332a7af4f0 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.177+0000 7f41c8ff9700 1 -- 172.21.15.5:0/1443469227 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41cc09f250 con 0x56332a7af4f0 2023-07-21T19:25:23.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.177+0000 7f41c8ff9700 1 -- 172.21.15.5:0/1443469227 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0996e0 con 0x56332a7af4f0 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1443469227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 msgr2=0x56332a7af8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0a30b0 tx=0x7f41cc09ade0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1443469227 shutdown_connections 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7c2540 0x56332a7c4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7af8c0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1443469227 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c2000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1443469227 >> 172.21.15.5:0/1443469227 conn(0x56332a7a4ff0 msgr2=0x56332a7aa930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1443469227 shutdown_connections 2023-07-21T19:25:23.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.178+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1443469227 wait complete. 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.179+0000 7f41f46a7b80 1 -- start start 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.179+0000 7f41f46a7b80 1 Processor -- start 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a6a0ac0 con 0x56332a7af4f0 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a6a07c0 con 0x56332a7afe00 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a584920 con 0x56332a7c2540 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41f46a7b80 10 client.0 _ll_register_callbacks cb 0x56332a69e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41da59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41d9d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41da59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57518/0 (socket says 172.21.15.5:57518) 2023-07-21T19:25:23.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41da59c700 1 -- 172.21.15.5:0/2269274547 learned_addr learned my addr 172.21.15.5:0/2269274547 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.180+0000 7f41dad9d700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 -- 172.21.15.5:0/2269274547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 msgr2=0x56332a7cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 -- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 msgr2=0x56332a7c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 -- 172.21.15.5:0/2269274547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41dc1a4780 con 0x56332a7af4f0 2023-07-21T19:25:23.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41da59c700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000c00 tx=0x7f41cc005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ca35d72000b0a0a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0a35e0 con 0x56332a7af4f0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56332a65a4f0 con 0x56332a7af4f0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f41cc0a10b0 con 0x56332a7af4f0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41cc022870 con 0x56332a7af4f0 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41f46a7b80 10 client.18981 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:23.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.181+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x56332a517c60 con 0x56332a7af4f0 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56332a65a4f0 con 0x56332a7af4f0 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41cc0b9020 con 0x56332a7af4f0 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 10 client.18981 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 10 client.18981 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 20 client.18981 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a7d16c0 0x56332a7d3ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 4 client.18981 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:23.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.182+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56332a6f9eb0 con 0x56332a7d16c0 2023-07-21T19:25:23.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.183+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a7d16c0 0x56332a7d3ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.183+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a7d16c0 0x56332a7d3ad0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=54572535793c7f13 server_cookie=9e589efd9efc5d64 in_seq=0 out_seq=0 2023-07-21T19:25:23.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.183+0000 7f41c1ffb700 10 client.18981 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:23.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41c1ffb700 1 -- 172.21.15.5:0/2269274547 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56332a6f9eb0 con 0x56332a7d16c0 2023-07-21T19:25:23.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41c1ffb700 10 client.18981 handle_command_reply: tid=0 2023-07-21T19:25:23.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 client.18981 shutdown 2023-07-21T19:25:23.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 msgr2=0x56332a7c2910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f41cc000c00 tx=0x7f41cc005c70 comp rx=0 tx=0).stop 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 shutdown_connections 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7af4f0 0x56332a7c2910 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a7d16c0 0x56332a7d3ad0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/2269274547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41c1ffb700 0 client.18981 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 >> 172.21.15.5:0/2269274547 conn(0x56332a7a4ff0 msgr2=0x56332a7aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 shutdown_connections 2023-07-21T19:25:23.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.184+0000 7f41f46a7b80 1 -- 172.21.15.5:0/2269274547 wait complete. 2023-07-21T19:25:23.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.185+0000 7f41f46a7b80 20 client.18981 trim_cache size 0 max 16384 2023-07-21T19:25:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.200+0000 7f41f46a7b80 1 Processor -- start 2023-07-21T19:25:23.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 -- start start 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7c7900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7c1750 con 0x56332a7c2540 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7a4b70 con 0x56332a7afe00 2023-07-21T19:25:23.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7ab6a0 con 0x56332a7af4f0 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41dad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41dad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57532/0 (socket says 172.21.15.5:57532) 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41dad9d700 1 -- 172.21.15.5:0/1934546139 learned_addr learned my addr 172.21.15.5:0/1934546139 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41da59c700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.201+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 -- 172.21.15.5:0/1934546139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 msgr2=0x56332a7c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 -- 172.21.15.5:0/1934546139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 msgr2=0x56332a7c2910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 -- 172.21.15.5:0/1934546139 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56332a555250 con 0x56332a7c2540 2023-07-21T19:25:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:23.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41dad9d700 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f41d40047b0 tx=0x7f41d4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5818ee0239017530 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.202+0000 7f41c97fa700 1 -- 172.21.15.5:0/1934546139 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4008e80 con 0x56332a7c2540 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41c97fa700 1 -- 172.21.15.5:0/1934546139 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41d400f040 con 0x56332a7c2540 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41c97fa700 1 -- 172.21.15.5:0/1934546139 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4024d40 con 0x56332a7c2540 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1934546139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 msgr2=0x56332a7c2e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f41d40047b0 tx=0x7f41d4015000 comp rx=0 tx=0).stop 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1934546139 shutdown_connections 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7c2540 0x56332a7c2e50 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7af4f0 0x56332a7c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1934546139 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7afe00 0x56332a7c2910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.203+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1934546139 >> 172.21.15.5:0/1934546139 conn(0x56332a7a4ff0 msgr2=0x56332a7cdf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.204+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1934546139 shutdown_connections 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.204+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1934546139 wait complete. 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.204+0000 7f41f46a7b80 1 -- start start 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.204+0000 7f41f46a7b80 1 Processor -- start 2023-07-21T19:25:23.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 0x56332a7c75e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c7b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7c5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7c6200 con 0x56332a7afe00 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7c6370 con 0x56332a7af4f0 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56332a7c6530 con 0x56332a7c2540 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41f46a7b80 10 client.0 _ll_register_callbacks cb 0x7f41cc09b080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:23.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41d9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c7b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41d9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c7b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57538/0 (socket says 172.21.15.5:57538) 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41d9d9b700 1 -- 172.21.15.5:0/1423066542 learned_addr learned my addr 172.21.15.5:0/1423066542 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41da59c700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 0x56332a7c75e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.205+0000 7f41dad9d700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7c5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 -- 172.21.15.5:0/1423066542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 msgr2=0x56332a7c5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7c5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 -- 172.21.15.5:0/1423066542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 msgr2=0x56332a7c7b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c7b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 -- 172.21.15.5:0/1423066542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d0013040 con 0x56332a7af4f0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41da59c700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 0x56332a7c75e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0b2040 tx=0x7f41cc005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bc7ef6c57522633 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc0b94c0 con 0x56332a7af4f0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f41cc0a10b0 con 0x56332a7af4f0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f41cc09f250 con 0x56332a7af4f0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41cc0a3150 con 0x56332a7af4f0 2023-07-21T19:25:23.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41f46a7b80 10 client.28631 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.206+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f41d4004400 con 0x56332a7af4f0 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f41cc09f250 con 0x56332a7af4f0 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f41cc097010 con 0x56332a7af4f0 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 10 client.28631 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 10 client.28631 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 20 client.28631 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:23.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a826460 0x56332a828870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 4 client.28631 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:25:23.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56332a6bc050 con 0x56332a826460 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.207+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a826460 0x56332a828870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41c1ffb700 10 client.28631 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41d9d9b700 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a826460 0x56332a828870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c49d3796c8e76f2b server_cookie=ec50e6a5b56d20e0 in_seq=0 out_seq=0 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41c1ffb700 1 -- 172.21.15.5:0/1423066542 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56332a6bc050 con 0x56332a826460 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41c1ffb700 10 client.28631 handle_command_reply: tid=0 2023-07-21T19:25:23.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 client.28631 shutdown 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 msgr2=0x56332a7c75e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 0x56332a7c75e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f41cc0b2040 tx=0x7f41cc005c00 comp rx=0 tx=0).stop 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 shutdown_connections 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56332a7c2540 0x56332a7c5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56332a826460 0x56332a828870 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56332a7af4f0 0x56332a7c75e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 --2- 172.21.15.5:0/1423066542 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56332a7afe00 0x56332a7c7b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41c1ffb700 0 client.28631 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.208+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 >> 172.21.15.5:0/1423066542 conn(0x56332a7a4ff0 msgr2=0x56332a7bc8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 shutdown_connections 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41f46a7b80 1 -- 172.21.15.5:0/1423066542 wait complete. 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41f46a7b80 20 client.28631 trim_cache size 0 max 16384 2023-07-21T19:25:23.207 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:25:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 msgr2=0x7f41c40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 0x7f41c40a3890 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f41dc1009e0 tx=0x7f41d4009ba0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 msgr2=0x7f41dc0fe8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.209+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f41cc005740 tx=0x7f41cc005770 comp rx=0 tx=0).stop 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 shutdown_connections 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f41dc109fc0 0x7f41dc1a3ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f41c40a13e0 0x7f41c40a3890 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f41dc100aa0 0x7f41dc0fe8d0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 --2- 172.21.15.5:0/4048738629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f41dc1013b0 0x7f41dc0fee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 >> 172.21.15.5:0/4048738629 conn(0x7f41dc0745d0 msgr2=0x7f41dc101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 shutdown_connections 2023-07-21T19:25:23.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.210+0000 7f41db59e700 1 -- 172.21.15.5:0/4048738629 wait complete. 2023-07-21T19:25:23.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:25:23.444 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.453+0000 7f0e41dff700 1 -- 172.21.15.5:0/1830203347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 msgr2=0x7f0e3c101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.453+0000 7f0e41dff700 1 --2- 172.21.15.5:0/1830203347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c101070 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009990 tx=0x7f0e2c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 -- 172.21.15.5:0/1830203347 shutdown_connections 2023-07-21T19:25:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 --2- 172.21.15.5:0/1830203347 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 0x7f0e3c10c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 --2- 172.21.15.5:0/1830203347 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c101070 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 --2- 172.21.15.5:0/1830203347 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 0x7f0e3c109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 -- 172.21.15.5:0/1830203347 >> 172.21.15.5:0/1830203347 conn(0x7f0e3c0fa4b0 msgr2=0x7f0e3c0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 -- 172.21.15.5:0/1830203347 shutdown_connections 2023-07-21T19:25:23.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.454+0000 7f0e41dff700 1 -- 172.21.15.5:0/1830203347 wait complete. 2023-07-21T19:25:23.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.455+0000 7f0e41dff700 1 Processor -- start 2023-07-21T19:25:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.455+0000 7f0e41dff700 1 -- start start 2023-07-21T19:25:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 0x7f0e3c0fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 0x7f0e3c1a3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c0ff660 con 0x7f0e3c100ca0 2023-07-21T19:25:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1a4260 con 0x7f0e3c1015b0 2023-07-21T19:25:23.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e41dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e3c1a43a0 con 0x7f0e3c10a070 2023-07-21T19:25:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57558/0 (socket says 172.21.15.5:57558) 2023-07-21T19:25:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.456+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2965040679 learned_addr learned my addr 172.21.15.5:0/2965040679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 0x7f0e3c1a3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 0x7f0e3c0fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2965040679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 msgr2=0x7f0e3c1a3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 0x7f0e3c1a3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2965040679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 msgr2=0x7f0e3c0fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 0x7f0e3c0fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2965040679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0095f0 con 0x7f0e3c100ca0 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005740 tx=0x7f0e2c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e38ff9700 1 -- 172.21.15.5:0/2965040679 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c01d070 con 0x7f0e3c100ca0 2023-07-21T19:25:23.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e3c1a45a0 con 0x7f0e3c100ca0 2023-07-21T19:25:23.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.457+0000 7f0e38ff9700 1 -- 172.21.15.5:0/2965040679 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e2c022470 con 0x7f0e3c100ca0 2023-07-21T19:25:23.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.458+0000 7f0e38ff9700 1 -- 172.21.15.5:0/2965040679 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c017650 con 0x7f0e3c100ca0 2023-07-21T19:25:23.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.458+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3c1a4a70 con 0x7f0e3c100ca0 2023-07-21T19:25:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.461+0000 7f0e38ff9700 1 -- 172.21.15.5:0/2965040679 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0e2c00fb00 con 0x7f0e3c100ca0 2023-07-21T19:25:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.462+0000 7f0e557acb80 1 Processor -- start 2023-07-21T19:25:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 -- start start 2023-07-21T19:25:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e38ff9700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 0x7f0e240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e38ff9700 1 -- 172.21.15.5:0/2965040679 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0e2c0ce2a0 con 0x7f0e3c100ca0 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ecd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 0x7f0e240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71ed8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71dd3cb0 con 0x563b71ed8ad0 2023-07-21T19:25:23.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71e26a40 con 0x563b71ecdbe0 2023-07-21T19:25:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e557acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71e1fed0 con 0x563b71ecd2d0 2023-07-21T19:25:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.463+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57568/0 (socket says 172.21.15.5:57568) 2023-07-21T19:25:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 -- 172.21.15.5:0/252887031 learned_addr learned my addr 172.21.15.5:0/252887031 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3affd700 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71ed8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ecd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 0x7f0e240a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009c90 tx=0x7f0e34004320 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 -- 172.21.15.5:0/252887031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 msgr2=0x563b71ecd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ecd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 -- 172.21.15.5:0/252887031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 msgr2=0x563b71ed8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71ed8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 -- 172.21.15.5:0/252887031 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b71c4dde0 con 0x563b71ed8ad0 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.464+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0e340094e0 tx=0x7f0e3401c030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5d7478059d4866c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.465+0000 7f0e21ffb700 1 -- 172.21.15.5:0/252887031 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34004db0 con 0x563b71ed8ad0 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.465+0000 7f0e21ffb700 1 -- 172.21.15.5:0/252887031 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e3400f040 con 0x563b71ed8ad0 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.465+0000 7f0e21ffb700 1 -- 172.21.15.5:0/252887031 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e34024790 con 0x563b71ed8ad0 2023-07-21T19:25:23.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.465+0000 7f0e557acb80 1 -- 172.21.15.5:0/252887031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 msgr2=0x563b71edaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.465+0000 7f0e557acb80 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f0e340094e0 tx=0x7f0e3401c030 comp rx=0 tx=0).stop 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 -- 172.21.15.5:0/252887031 shutdown_connections 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71edaeb0 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ecd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 --2- 172.21.15.5:0/252887031 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71ed8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 -- 172.21.15.5:0/252887031 >> 172.21.15.5:0/252887031 conn(0x563b71ec3da0 msgr2=0x563b71ec96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 -- 172.21.15.5:0/252887031 shutdown_connections 2023-07-21T19:25:23.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 -- 172.21.15.5:0/252887031 wait complete. 2023-07-21T19:25:23.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.466+0000 7f0e557acb80 1 -- start start 2023-07-21T19:25:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.467+0000 7f0e557acb80 1 Processor -- start 2023-07-21T19:25:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.467+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 0x563b71ed8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.467+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 0x563b71ed57a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.467+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71ed5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e557acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71dbfac0 con 0x563b71ed8ad0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e557acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71dbf7c0 con 0x563b71ecd2d0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e557acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ca3920 con 0x563b71ecdbe0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e557acb80 10 client.0 _ll_register_callbacks cb 0x563b71dbd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71ed5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71ed5ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57582/0 (socket says 172.21.15.5:57582) 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3bfff700 1 -- 172.21.15.5:0/2337171873 learned_addr learned my addr 172.21.15.5:0/2337171873 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 0x563b71ed8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 0x563b71ed57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2337171873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 msgr2=0x563b71ed57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 0x563b71ed57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2337171873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 msgr2=0x563b71ed5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71ed5ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2337171873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e34009d70 con 0x563b71ecd2d0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.468+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 0x563b71ed57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 0x563b71ed8520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cb990 tx=0x7f0e2c005910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85eb26988abb867e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e1affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0a3760 con 0x563b71ecd2d0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0e2c022850 con 0x563b71ecd2d0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b71d794f0 con 0x563b71ecd2d0 2023-07-21T19:25:23.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2c09f020 con 0x563b71ecd2d0 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e557acb80 10 client.28643 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x563b71c36c60 con 0x563b71ecd2d0 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.469+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e2c0c9080 con 0x563b71ecd2d0 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e2c09a890 con 0x563b71ecd2d0 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 10 client.28643 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 10 client.28643 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:23.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 20 client.28643 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71ee8160 0x563b71eea570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 4 client.28643 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b71e18eb0 con 0x563b71ee8160 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.470+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71ee8160 0x563b71eea570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.471+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71ee8160 0x563b71eea570 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df0c4b77c42fadf8 server_cookie=e7783b95ef42baed in_seq=0 out_seq=0 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.471+0000 7f0e1affd700 10 client.28643 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:23.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.471+0000 7f0e1affd700 1 -- 172.21.15.5:0/2337171873 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b71e18eb0 con 0x563b71ee8160 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.471+0000 7f0e1affd700 10 client.28643 handle_command_reply: tid=0 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 client.28643 shutdown 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 msgr2=0x563b71ed8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 0x563b71ed8520 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c0cb990 tx=0x7f0e2c005910 comp rx=0 tx=0).stop 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 shutdown_connections 2023-07-21T19:25:23.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71ee8160 0x563b71eea570 crc :-1 s=READY pgs=355 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ed8ad0 0x563b71ed5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecd2d0 0x563b71ed8520 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2337171873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecdbe0 0x563b71ed57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e1affd700 0 client.28643 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 >> 172.21.15.5:0/2337171873 conn(0x563b71ec3da0 msgr2=0x563b71ec9290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 shutdown_connections 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 1 -- 172.21.15.5:0/2337171873 wait complete. 2023-07-21T19:25:23.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.472+0000 7f0e557acb80 20 client.28643 trim_cache size 0 max 16384 2023-07-21T19:25:23.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 Processor -- start 2023-07-21T19:25:23.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 -- start start 2023-07-21T19:25:23.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ed80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.477 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 0x563b71ed5500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ed6750 con 0x563b71ecdbe0 2023-07-21T19:25:23.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ece330 con 0x563b71ed8ad0 2023-07-21T19:25:23.478 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.486+0000 7f0e557acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ec3970 con 0x563b71ecd2d0 2023-07-21T19:25:23.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 0x563b71ed5500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57590/0 (socket says 172.21.15.5:57590) 2023-07-21T19:25:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 -- 172.21.15.5:0/1266376155 learned_addr learned my addr 172.21.15.5:0/1266376155 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ed80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 -- 172.21.15.5:0/1266376155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 msgr2=0x563b71ed80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ed80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 -- 172.21.15.5:0/1266376155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 msgr2=0x563b71ed5500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 0x563b71ed5500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 -- 172.21.15.5:0/1266376155 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b71c74250 con 0x563b71ecdbe0 2023-07-21T19:25:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 0x563b71ed5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.487+0000 7f0e3affd700 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c1000e0 tx=0x7f0e30009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba80df5fdbc5ce9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.488+0000 7f0e227fc700 1 -- 172.21.15.5:0/1266376155 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3001f110 con 0x563b71ecdbe0 2023-07-21T19:25:23.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.488+0000 7f0e227fc700 1 -- 172.21.15.5:0/1266376155 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e30018040 con 0x563b71ecdbe0 2023-07-21T19:25:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.488+0000 7f0e227fc700 1 -- 172.21.15.5:0/1266376155 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e3001e680 con 0x563b71ecdbe0 2023-07-21T19:25:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.488+0000 7f0e557acb80 1 -- 172.21.15.5:0/1266376155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 msgr2=0x563b71ed4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.488+0000 7f0e557acb80 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0e3c1000e0 tx=0x7f0e30009260 comp rx=0 tx=0).stop 2023-07-21T19:25:23.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 -- 172.21.15.5:0/1266376155 shutdown_connections 2023-07-21T19:25:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ed8ad0 0x563b71ed5500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ecd2d0 0x563b71ed80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 --2- 172.21.15.5:0/1266376155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecdbe0 0x563b71ed4fc0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 -- 172.21.15.5:0/1266376155 >> 172.21.15.5:0/1266376155 conn(0x563b71ec3da0 msgr2=0x563b71ed0b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 -- 172.21.15.5:0/1266376155 shutdown_connections 2023-07-21T19:25:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 -- 172.21.15.5:0/1266376155 wait complete. 2023-07-21T19:25:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 -- start start 2023-07-21T19:25:23.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.489+0000 7f0e557acb80 1 Processor -- start 2023-07-21T19:25:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71edd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ed8ad0 0x563b71eddb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ede0b0 con 0x563b71ecd2d0 2023-07-21T19:25:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ede220 con 0x563b71ecdbe0 2023-07-21T19:25:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b71ede390 con 0x563b71ed8ad0 2023-07-21T19:25:23.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e557acb80 10 client.0 _ll_register_callbacks cb 0x7f0e2c09b240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57604/0 (socket says 172.21.15.5:57604) 2023-07-21T19:25:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2807241728 learned_addr learned my addr 172.21.15.5:0/2807241728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:23.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3bfff700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ed8ad0 0x563b71eddb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71edd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2807241728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ed8ad0 msgr2=0x563b71eddb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ed8ad0 0x563b71eddb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 msgr2=0x563b71edd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.490+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71edd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e3b7fe700 1 -- 172.21.15.5:0/2807241728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e3402b040 con 0x563b71ecd2d0 2023-07-21T19:25:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e3b7fe700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c09b330 tx=0x7f0e2c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34f35727d42636d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:23.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e1affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c0a3760 con 0x563b71ecd2d0 2023-07-21T19:25:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0e3400f040 con 0x563b71ecd2d0 2023-07-21T19:25:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0e2c09f040 con 0x563b71ecd2d0 2023-07-21T19:25:23.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0e2c09f200 con 0x563b71ecd2d0 2023-07-21T19:25:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e557acb80 10 client.19005 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f0e3001fa40 con 0x563b71ecd2d0 2023-07-21T19:25:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.491+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0e3400f040 con 0x563b71ecd2d0 2023-07-21T19:25:23.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0e2c0a3310 con 0x563b71ecd2d0 2023-07-21T19:25:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 10 client.19005 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 10 client.19005 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 20 client.19005 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:23.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71f3d0c0 0x563b71f3f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 4 client.19005 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:25:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563b71ddb050 con 0x563b71f3d0c0 2023-07-21T19:25:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.492+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71f3d0c0 0x563b71f3f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:23.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e3affd700 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71f3d0c0 0x563b71f3f4d0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f3ebb1fa1e4df8c server_cookie=5805d3f9ed299527 in_seq=0 out_seq=0 2023-07-21T19:25:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e1affd700 10 client.19005 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e1affd700 1 -- 172.21.15.5:0/2807241728 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563b71ddb050 con 0x563b71f3d0c0 2023-07-21T19:25:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e1affd700 10 client.19005 handle_command_reply: tid=0 2023-07-21T19:25:23.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 client.19005 shutdown 2023-07-21T19:25:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 msgr2=0x563b71edf2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c09b330 tx=0x7f0e2c005fd0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 shutdown_connections 2023-07-21T19:25:23.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b71ed8ad0 0x563b71eddb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b71f3d0c0 0x563b71f3f4d0 crc :-1 s=READY pgs=356 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b71ecd2d0 0x563b71edf2d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 --2- 172.21.15.5:0/2807241728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b71ecdbe0 0x563b71edd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e1affd700 0 client.19005 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.493+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 >> 172.21.15.5:0/2807241728 conn(0x563b71ec3da0 msgr2=0x563b71ecf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 shutdown_connections 2023-07-21T19:25:23.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e557acb80 1 -- 172.21.15.5:0/2807241728 wait complete. 2023-07-21T19:25:23.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e557acb80 20 client.19005 trim_cache size 0 max 16384 2023-07-21T19:25:23.506 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:25:23.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 msgr2=0x7f0e240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 0x7f0e240a3770 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009c90 tx=0x7f0e34004320 comp rx=0 tx=0).stop 2023-07-21T19:25:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 msgr2=0x7f0e3c0fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:23.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.494+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c005740 tx=0x7f0e2c005770 comp rx=0 tx=0).stop 2023-07-21T19:25:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 shutdown_connections 2023-07-21T19:25:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0e3c10a070 0x7f0e3c1a3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0e240a12c0 0x7f0e240a3770 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0e3c100ca0 0x7f0e3c0fe8f0 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 --2- 172.21.15.5:0/2965040679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0e3c1015b0 0x7f0e3c0fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 >> 172.21.15.5:0/2965040679 conn(0x7f0e3c0fa4b0 msgr2=0x7f0e3c0fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 shutdown_connections 2023-07-21T19:25:23.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:23.495+0000 7f0e41dff700 1 -- 172.21.15.5:0/2965040679 wait complete. 2023-07-21T19:25:23.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:23 smithi005 ceph-mon[126460]: pgmap v560: 97 pgs: 97 active+clean; 15 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:25:23.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:23 smithi156 ceph-mon[128316]: pgmap v560: 97 pgs: 97 active+clean; 15 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:25:24.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:23 smithi101 ceph-mon[128136]: pgmap v560: 97 pgs: 97 active+clean; 15 GiB data, 6.9 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 22 MiB/s wr, 18 op/s 2023-07-21T19:25:24.511 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:25:24.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:25:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 -- 172.21.15.5:0/3426365994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c8100490 msgr2=0x7f63c810c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 --2- 172.21.15.5:0/3426365994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c8100490 0x7f63c810c920 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b0a0 tx=0x7f63c000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 -- 172.21.15.5:0/3426365994 shutdown_connections 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 --2- 172.21.15.5:0/3426365994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c8100490 0x7f63c810c920 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 --2- 172.21.15.5:0/3426365994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 0x7f63c8105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 --2- 172.21.15.5:0/3426365994 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c80ffae0 0x7f63c80fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 -- 172.21.15.5:0/3426365994 >> 172.21.15.5:0/3426365994 conn(0x7f63c80fb5e0 msgr2=0x7f63c80fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 -- 172.21.15.5:0/3426365994 shutdown_connections 2023-07-21T19:25:24.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.744+0000 7f63d05ff700 1 -- 172.21.15.5:0/3426365994 wait complete. 2023-07-21T19:25:24.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.745+0000 7f63d05ff700 1 Processor -- start 2023-07-21T19:25:24.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.745+0000 7f63d05ff700 1 -- start start 2023-07-21T19:25:24.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c8100490 0x7f63c81982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 0x7f63c819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c819cd90 con 0x7f63c80ffae0 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c819cf00 con 0x7f63c8100490 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63d05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c819d0c0 con 0x7f63c8105af0 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57622/0 (socket says 172.21.15.5:57622) 2023-07-21T19:25:24.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ce39b700 1 -- 172.21.15.5:0/689005859 learned_addr learned my addr 172.21.15.5:0/689005859 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c8100490 0x7f63c81982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 0x7f63c819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ce39b700 1 -- 172.21.15.5:0/689005859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 msgr2=0x7f63c819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.746+0000 7f63ce39b700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 0x7f63c819c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63ce39b700 1 -- 172.21.15.5:0/689005859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c8100490 msgr2=0x7f63c81982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63ce39b700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c8100490 0x7f63c81982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63ce39b700 1 -- 172.21.15.5:0/689005859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c0009d00 con 0x7f63c80ffae0 2023-07-21T19:25:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63ce39b700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f63c400e9a0 tx=0x7f63c400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63bf7fe700 1 -- 172.21.15.5:0/689005859 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4009790 con 0x7f63c80ffae0 2023-07-21T19:25:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63bf7fe700 1 -- 172.21.15.5:0/689005859 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c400cbd0 con 0x7f63c80ffae0 2023-07-21T19:25:24.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c819d3a0 con 0x7f63c80ffae0 2023-07-21T19:25:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63bf7fe700 1 -- 172.21.15.5:0/689005859 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c4018680 con 0x7f63c80ffae0 2023-07-21T19:25:24.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.747+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c81a4be0 con 0x7f63c80ffae0 2023-07-21T19:25:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.751+0000 7f63bf7fe700 1 -- 172.21.15.5:0/689005859 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f63c400cd40 con 0x7f63c80ffae0 2023-07-21T19:25:24.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.753+0000 7f63e3f76b80 1 Processor -- start 2023-07-21T19:25:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.753+0000 7f63bf7fe700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 0x7f63b40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.753+0000 7f63e3f76b80 1 -- start start 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.753+0000 7f63bf7fe700 1 -- 172.21.15.5:0/689005859 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63c4014070 con 0x7f63c80ffae0 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.753+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 0x7f63b40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193ac60 0x564051945610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051947f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564051840d30 con 0x56405193ac60 2023-07-21T19:25:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564051893ac0 con 0x564051945b50 2023-07-21T19:25:24.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405188cf50 con 0x56405193a350 2023-07-21T19:25:24.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36938/0 (socket says 172.21.15.5:36938) 2023-07-21T19:25:24.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63ce39b700 1 -- 172.21.15.5:0/526239543 learned_addr learned my addr 172.21.15.5:0/526239543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:24.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193ac60 0x564051945610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051947f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.754+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 0x7f63b40a3770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b070 tx=0x7f63c000b700 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 -- 172.21.15.5:0/526239543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 msgr2=0x564051947f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051947f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 -- 172.21.15.5:0/526239543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193ac60 msgr2=0x564051945610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193ac60 0x564051945610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 -- 172.21.15.5:0/526239543 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640516bae60 con 0x56405193a350 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63ce39b700 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f63c4096b10 tx=0x7f63c4099af0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d1f09e18d0949eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63bcbfe700 1 -- 172.21.15.5:0/526239543 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40a24f0 con 0x56405193a350 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63bcbfe700 1 -- 172.21.15.5:0/526239543 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c409a300 con 0x56405193a350 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.755+0000 7f63bcbfe700 1 -- 172.21.15.5:0/526239543 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40a2060 con 0x56405193a350 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 -- 172.21.15.5:0/526239543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 msgr2=0x56405193a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f63c4096b10 tx=0x7f63c4099af0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 -- 172.21.15.5:0/526239543 shutdown_connections 2023-07-21T19:25:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051947f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193a350 0x56405193a720 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/526239543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193ac60 0x564051945610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 -- 172.21.15.5:0/526239543 >> 172.21.15.5:0/526239543 conn(0x564051930e20 msgr2=0x564051936720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 -- 172.21.15.5:0/526239543 shutdown_connections 2023-07-21T19:25:24.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.756+0000 7f63e3f76b80 1 -- 172.21.15.5:0/526239543 wait complete. 2023-07-21T19:25:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 -- start start 2023-07-21T19:25:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 Processor -- start 2023-07-21T19:25:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051942f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.757+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405182cb40 con 0x56405193a350 2023-07-21T19:25:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56405182c840 con 0x564051945b50 2023-07-21T19:25:24.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640517109a0 con 0x56405193ac60 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63e3f76b80 10 client.0 _ll_register_callbacks cb 0x56405182a7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57654/0 (socket says 172.21.15.5:57654) 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63ce39b700 1 -- 172.21.15.5:0/4046350582 learned_addr learned my addr 172.21.15.5:0/4046350582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:24.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051942f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 -- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 msgr2=0x564051942f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051942f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 -- 172.21.15.5:0/4046350582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 msgr2=0x5640519455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63cdb9a700 1 -- 172.21.15.5:0/4046350582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c81a4be0 con 0x56405193ac60 2023-07-21T19:25:24.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.758+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63b80126d0 tx=0x7f63b80129e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=585016dd82db55b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b8017070 con 0x56405193ac60 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5640517e6570 con 0x56405193ac60 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63b8004260 con 0x56405193ac60 2023-07-21T19:25:24.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63b8004420 con 0x56405193ac60 2023-07-21T19:25:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63e3f76b80 10 client.28582 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.759+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5640516a3ce0 con 0x56405193ac60 2023-07-21T19:25:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5640517e6570 con 0x56405193ac60 2023-07-21T19:25:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63b800f4d0 con 0x56405193ac60 2023-07-21T19:25:24.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 10 client.28582 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 10 client.28582 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 20 client.28582 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519555c0 0x5640519579d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 4 client.28582 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564051885f30 con 0x5640519555c0 2023-07-21T19:25:24.760 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.760+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519555c0 0x5640519579d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.761+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519555c0 0x5640519579d0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=436cccfcaa59c23f server_cookie=d304aec4e4ccbce3 in_seq=0 out_seq=0 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.761+0000 7f63b1ffb700 10 client.28582 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63b1ffb700 1 -- 172.21.15.5:0/4046350582 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564051885f30 con 0x5640519555c0 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63b1ffb700 10 client.28582 handle_command_reply: tid=0 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 client.28582 shutdown 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 msgr2=0x564051942850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942850 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f63b80126d0 tx=0x7f63b80129e0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 shutdown_connections 2023-07-21T19:25:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x564051942f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519555c0 0x5640519579d0 crc :-1 s=READY pgs=357 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x5640519455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4046350582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942850 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63b1ffb700 0 client.28582 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 >> 172.21.15.5:0/4046350582 conn(0x564051930e20 msgr2=0x5640519362b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 shutdown_connections 2023-07-21T19:25:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.762+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4046350582 wait complete. 2023-07-21T19:25:24.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.763+0000 7f63e3f76b80 20 client.28582 trim_cache size 0 max 16384 2023-07-21T19:25:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 Processor -- start 2023-07-21T19:25:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 -- start start 2023-07-21T19:25:24.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405193fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640519309f0 con 0x56405193a350 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564051937550 con 0x564051945b50 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564051940dd0 con 0x56405193ac60 2023-07-21T19:25:24.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.785+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57660/0 (socket says 172.21.15.5:57660) 2023-07-21T19:25:24.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 -- 172.21.15.5:0/4207681221 learned_addr learned my addr 172.21.15.5:0/4207681221 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 -- 172.21.15.5:0/4207681221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 msgr2=0x564051942570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405193fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 -- 172.21.15.5:0/4207681221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 msgr2=0x56405193fb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405193fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 -- 172.21.15.5:0/4207681221 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5640516e12d0 con 0x56405193a350 2023-07-21T19:25:24.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63ce39b700 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f63c40a1fc0 tx=0x7f63c409a220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a7d2ec73722062e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.786+0000 7f63bd3ff700 1 -- 172.21.15.5:0/4207681221 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40963e0 con 0x56405193a350 2023-07-21T19:25:24.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63bd3ff700 1 -- 172.21.15.5:0/4207681221 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c4098440 con 0x56405193a350 2023-07-21T19:25:24.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63bd3ff700 1 -- 172.21.15.5:0/4207681221 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40a2060 con 0x56405193a350 2023-07-21T19:25:24.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4207681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 msgr2=0x564051942030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f63c40a1fc0 tx=0x7f63c409a220 comp rx=0 tx=0).stop 2023-07-21T19:25:24.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4207681221 shutdown_connections 2023-07-21T19:25:24.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405193fb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x564051942030 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/4207681221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x564051942570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.787+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4207681221 >> 172.21.15.5:0/4207681221 conn(0x564051930e20 msgr2=0x56405193db70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.788+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4207681221 shutdown_connections 2023-07-21T19:25:24.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.788+0000 7f63e3f76b80 1 -- 172.21.15.5:0/4207681221 wait complete. 2023-07-21T19:25:24.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.788+0000 7f63e3f76b80 1 -- start start 2023-07-21T19:25:24.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.788+0000 7f63e3f76b80 1 Processor -- start 2023-07-21T19:25:24.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x56405194b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405194b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640519a94a0 con 0x56405193a350 2023-07-21T19:25:24.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640519a9610 con 0x564051945b50 2023-07-21T19:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5640519a97d0 con 0x56405193ac60 2023-07-21T19:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63e3f76b80 10 client.0 _ll_register_callbacks cb 0x7f63b8009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63ce39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57672/0 (socket says 172.21.15.5:57672) 2023-07-21T19:25:24.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63ce39b700 1 -- 172.21.15.5:0/2497079931 learned_addr learned my addr 172.21.15.5:0/2497079931 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63ceb9c700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405194b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.789+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x56405194b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 -- 172.21.15.5:0/2497079931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 msgr2=0x56405194b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x56405194b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 -- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 msgr2=0x56405194b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:24.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405194b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 -- 172.21.15.5:0/2497079931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5640516bae60 con 0x56405193a350 2023-07-21T19:25:24.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63ce39b700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f63c4096b10 tx=0x7f63c40a2590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba3b14bc10b2fe8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:24.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63b1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63c40b84d0 con 0x56405193a350 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63c409a530 con 0x56405193a350 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63c409a300 con 0x56405193a350 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63c40a2060 con 0x56405193a350 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63e3f76b80 10 client.19023 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:24.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.790+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x5640516b9330 con 0x56405193a350 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63c407c020 con 0x56405193a350 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f63c4099000 con 0x56405193a350 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 10 client.19023 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 10 client.19023 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 20 client.19023 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519aaab0 0x5640519ace90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 4 client.19023 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:25:24.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5640518480d0 con 0x5640519aaab0 2023-07-21T19:25:24.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.791+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519aaab0 0x5640519ace90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:24.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.792+0000 7f63b1ffb700 10 client.19023 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:24.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.792+0000 7f63cdb9a700 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519aaab0 0x5640519ace90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f71504512f36a server_cookie=564031f5c820b22e in_seq=0 out_seq=0 2023-07-21T19:25:24.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63b1ffb700 1 -- 172.21.15.5:0/2497079931 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5640518480d0 con 0x5640519aaab0 2023-07-21T19:25:24.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63b1ffb700 10 client.19023 handle_command_reply: tid=0 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63e3f76b80 1 client.19023 shutdown 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 msgr2=0x56405194ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f63c4096b10 tx=0x7f63c40a2590 comp rx=0 tx=0).stop 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.793+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 shutdown_connections 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564051945b50 0x56405194b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5640519aaab0 0x5640519ace90 crc :-1 s=READY pgs=358 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56405193a350 0x56405194ac80 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 --2- 172.21.15.5:0/2497079931 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56405193ac60 0x56405194b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 >> 172.21.15.5:0/2497079931 conn(0x564051930e20 msgr2=0x56405193c460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63b1ffb700 0 client.19023 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 shutdown_connections 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 1 -- 172.21.15.5:0/2497079931 wait complete. 2023-07-21T19:25:24.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.794+0000 7f63e3f76b80 20 client.19023 trim_cache size 0 max 16384 2023-07-21T19:25:24.798 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:25:24.798 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:25:24.798 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "21c9ad14-98e0-495d-9702-9f9253636b2c", 2023-07-21T19:25:24.798 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:25:24.798 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:25:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 msgr2=0x7f63b40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 0x7f63b40a3770 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f63c000b070 tx=0x7f63c000b700 comp rx=0 tx=0).stop 2023-07-21T19:25:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 msgr2=0x7f63c8197db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:24.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f63c400e9a0 tx=0x7f63c400ed60 comp rx=0 tx=0).stop 2023-07-21T19:25:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 shutdown_connections 2023-07-21T19:25:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63c8105af0 0x7f63c819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63b40a12c0 0x7f63b40a3770 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63c80ffae0 0x7f63c8197db0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 --2- 172.21.15.5:0/689005859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63c8100490 0x7f63c81982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:24.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 >> 172.21.15.5:0/689005859 conn(0x7f63c80fb5e0 msgr2=0x7f63c810ade0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:24.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 shutdown_connections 2023-07-21T19:25:24.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:24.795+0000 7f63d05ff700 1 -- 172.21.15.5:0/689005859 wait complete. 2023-07-21T19:25:24.809 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:25:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 -- 172.21.15.5:0/2661067022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec74100380 msgr2=0x7fec7410c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2661067022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec74100380 0x7fec7410c810 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00b0a0 tx=0x7fec6c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 -- 172.21.15.5:0/2661067022 shutdown_connections 2023-07-21T19:25:25.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2661067022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec74100380 0x7fec7410c810 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2661067022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec741059e0 0x7fec74105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2661067022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec740ff9d0 0x7fec740ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 -- 172.21.15.5:0/2661067022 >> 172.21.15.5:0/2661067022 conn(0x7fec740fb4b0 msgr2=0x7fec740fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 -- 172.21.15.5:0/2661067022 shutdown_connections 2023-07-21T19:25:25.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.070+0000 7fec7b19d700 1 -- 172.21.15.5:0/2661067022 wait complete. 2023-07-21T19:25:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 Processor -- start 2023-07-21T19:25:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 -- start start 2023-07-21T19:25:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec740ff9d0 0x7fec74072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 0x7fec741a3aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec74073580 con 0x7fec741059e0 2023-07-21T19:25:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741a3fe0 con 0x7fec740ff9d0 2023-07-21T19:25:25.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.072+0000 7fec7b19d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec741a4120 con 0x7fec74100380 2023-07-21T19:25:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 0x7fec741a3aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36990/0 (socket says 172.21.15.5:36990) 2023-07-21T19:25:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 -- 172.21.15.5:0/2114915674 learned_addr learned my addr 172.21.15.5:0/2114915674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:25.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec740ff9d0 0x7fec74072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 -- 172.21.15.5:0/2114915674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec740ff9d0 msgr2=0x7fec74072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec740ff9d0 0x7fec74072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 -- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 msgr2=0x7fec741a3aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 0x7fec741a3aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 -- 172.21.15.5:0/2114915674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec6c009d00 con 0x7fec74100380 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7a99c700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 0x7fec741a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.073+0000 7fec7999a700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9a0 tx=0x7fec6400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.074+0000 7fec6b7fe700 1 -- 172.21.15.5:0/2114915674 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec64009790 con 0x7fec74100380 2023-07-21T19:25:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.074+0000 7fec6b7fe700 1 -- 172.21.15.5:0/2114915674 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec64010430 con 0x7fec74100380 2023-07-21T19:25:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.074+0000 7fec6b7fe700 1 -- 172.21.15.5:0/2114915674 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6400cea0 con 0x7fec74100380 2023-07-21T19:25:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.074+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec741a4380 con 0x7fec74100380 2023-07-21T19:25:25.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.074+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec741a4810 con 0x7fec74100380 2023-07-21T19:25:25.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.078+0000 7fec6b7fe700 1 -- 172.21.15.5:0/2114915674 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fec640105a0 con 0x7fec74100380 2023-07-21T19:25:25.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 Processor -- start 2023-07-21T19:25:25.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 -- start start 2023-07-21T19:25:25.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1aed30 0x562f5f1b1110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f0a9cb0 con 0x562f5f1aed30 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f0fca40 con 0x562f5f1a44f0 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.079+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f0f5ed0 con 0x562f5f1a4e00 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1aed30 0x562f5f1b1110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54788/0 (socket says 172.21.15.5:54788) 2023-07-21T19:25:25.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 -- 172.21.15.5:0/3790657714 learned_addr learned my addr 172.21.15.5:0/3790657714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a19b700 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 -- 172.21.15.5:0/3790657714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a4e00 msgr2=0x562f5f1ae7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 -- 172.21.15.5:0/3790657714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1aed30 msgr2=0x562f5f1b1110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1aed30 0x562f5f1b1110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 -- 172.21.15.5:0/3790657714 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5ef23de0 con 0x562f5f1a44f0 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec6b7fe700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 0x7fec600a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec7a99c700 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00b0a0 tx=0x7fec6c009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c8ec4519fbaba61a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.080+0000 7fec6b7fe700 1 -- 172.21.15.5:0/2114915674 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fec64014070 con 0x7fec74100380 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 0x7fec600a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec68bfe700 1 -- 172.21.15.5:0/3790657714 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00e040 con 0x562f5f1a44f0 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec68bfe700 1 -- 172.21.15.5:0/3790657714 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec6c0074e0 con 0x562f5f1a44f0 2023-07-21T19:25:25.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec68bfe700 1 -- 172.21.15.5:0/3790657714 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c005040 con 0x562f5f1a44f0 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3790657714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 msgr2=0x562f5f1a48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 0x7fec600a3810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec74100b10 tx=0x7fec70009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.081+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00b0a0 tx=0x7fec6c009ba0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3790657714 shutdown_connections 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1aed30 0x562f5f1b1110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1a48c0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3790657714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3790657714 >> 172.21.15.5:0/3790657714 conn(0x562f5f199ff0 msgr2=0x562f5f19f930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3790657714 shutdown_connections 2023-07-21T19:25:25.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.082+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3790657714 wait complete. 2023-07-21T19:25:25.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.083+0000 7fec8fdc3b80 1 -- start start 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.083+0000 7fec8fdc3b80 1 Processor -- start 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1ae750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ab960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f095ac0 con 0x562f5f1a4e00 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f0957c0 con 0x562f5f1a44f0 2023-07-21T19:25:25.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5ef79920 con 0x562f5f1aed30 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec8fdc3b80 10 client.0 _ll_register_callbacks cb 0x562f5f093760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37010/0 (socket says 172.21.15.5:37010) 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec7a99c700 1 -- 172.21.15.5:0/2209375978 learned_addr learned my addr 172.21.15.5:0/2209375978 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec7999a700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ab960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.084+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 -- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 msgr2=0x562f5f1ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 -- 172.21.15.5:0/2209375978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 msgr2=0x562f5f1ab960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ab960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 -- 172.21.15.5:0/2209375978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7410d110 con 0x562f5f1aed30 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7999a700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec7a99c700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0062a0 tx=0x7fec6c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a2ad7d3fcd5425e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:25.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00e040 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f5f04f4f0 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.085+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fec6c00bcb0 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.086+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec6c003a00 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.086+0000 7fec8fdc3b80 10 client.28600 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.086+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x562f5ef0cc60 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.086+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562f5f04f4f0 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.086+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec6c00b0d0 con 0x562f5f1aed30 2023-07-21T19:25:25.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 10 client.28600 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:25.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 10 client.28600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:25.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 20 client.28600 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:25.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f1be280 0x562f5f1c0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 4 client.28600 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:25.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f5f0eeeb0 con 0x562f5f1be280 2023-07-21T19:25:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.087+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f1be280 0x562f5f1c0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.088+0000 7fec5dffb700 10 client.28600 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:25.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.088+0000 7fec7a19b700 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f1be280 0x562f5f1c0660 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2fdf640126ea2e1 server_cookie=7770e4eecbca3cd9 in_seq=0 out_seq=0 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec5dffb700 1 -- 172.21.15.5:0/2209375978 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f5f0eeeb0 con 0x562f5f1be280 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec5dffb700 10 client.28600 handle_command_reply: tid=0 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 client.28600 shutdown 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 msgr2=0x562f5f1abf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fec6c0062a0 tx=0x7fec6c0099e0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 shutdown_connections 2023-07-21T19:25:25.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1abf90 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f1be280 0x562f5f1c0660 crc :-1 s=READY pgs=359 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/2209375978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ab960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 >> 172.21.15.5:0/2209375978 conn(0x562f5f199ff0 msgr2=0x562f5f19f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec5dffb700 0 client.28600 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 shutdown_connections 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/2209375978 wait complete. 2023-07-21T19:25:25.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.089+0000 7fec8fdc3b80 20 client.28600 trim_cache size 0 max 16384 2023-07-21T19:25:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.110+0000 7fec8fdc3b80 1 Processor -- start 2023-07-21T19:25:25.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.110+0000 7fec8fdc3b80 1 -- start start 2023-07-21T19:25:25.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.110+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1b57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.110+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1a6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1a5440 con 0x562f5f1a4e00 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1a51d0 con 0x562f5f1a44f0 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1c0cc0 con 0x562f5f1aed30 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1a6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57720/0 (socket says 172.21.15.5:57720) 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7999a700 1 -- 172.21.15.5:0/15603910 learned_addr learned my addr 172.21.15.5:0/15603910 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:25.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a19b700 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1b57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 -- 172.21.15.5:0/15603910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 msgr2=0x562f5f1b57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1b57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 -- 172.21.15.5:0/15603910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 msgr2=0x562f5f1ae280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.111+0000 7fec7a99c700 1 -- 172.21.15.5:0/15603910 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f5ef4a250 con 0x562f5f1aed30 2023-07-21T19:25:25.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.112+0000 7fec7a99c700 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1a6cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00d010 tx=0x7fec6c0021d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38070e854e793fdc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.112+0000 7fec693ff700 1 -- 172.21.15.5:0/15603910 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c00e030 con 0x562f5f1aed30 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.112+0000 7fec693ff700 1 -- 172.21.15.5:0/15603910 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec6c004920 con 0x562f5f1aed30 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.112+0000 7fec693ff700 1 -- 172.21.15.5:0/15603910 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec6c025ae0 con 0x562f5f1aed30 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/15603910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 msgr2=0x562f5f1a6cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1a6cb0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fec6c00d010 tx=0x7fec6c0021d0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/15603910 shutdown_connections 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1aed30 0x562f5f1a6cb0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1a44f0 0x562f5f1b57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/15603910 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1ae280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/15603910 >> 172.21.15.5:0/15603910 conn(0x562f5f199ff0 msgr2=0x562f5f19eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/15603910 shutdown_connections 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/15603910 wait complete. 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.113+0000 7fec8fdc3b80 1 -- start start 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 Processor -- start 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 0x562f5f1b56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1aed30 0x562f5f1b3f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1b44c0 con 0x562f5f1a4e00 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1b4600 con 0x562f5f1aed30 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f5f1b47c0 con 0x562f5f1a44f0 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec8fdc3b80 10 client.0 _ll_register_callbacks cb 0x7fec640d0750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec7999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec7a19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 0x562f5f1b56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec7999a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57730/0 (socket says 172.21.15.5:57730) 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.114+0000 7fec7999a700 1 -- 172.21.15.5:0/3201384634 learned_addr learned my addr 172.21.15.5:0/3201384634 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a99c700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1aed30 0x562f5f1b3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 -- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1aed30 msgr2=0x562f5f1b3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1aed30 0x562f5f1b3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 -- 172.21.15.5:0/3201384634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 msgr2=0x562f5f1b3a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 -- 172.21.15.5:0/3201384634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562f5ef0cc60 con 0x562f5f1a44f0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7999a700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec7a19b700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 0x562f5f1b56d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562f5f0f6330 tx=0x7fec70004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=245849432f031ef2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.115+0000 7fec5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec70017070 con 0x562f5f1a44f0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fec6c0074e0 con 0x562f5f1a44f0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fec70029960 con 0x562f5f1a44f0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec70029b20 con 0x562f5f1a44f0 2023-07-21T19:25:25.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec8fdc3b80 10 client.28606 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:25.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x562f5ef222b0 con 0x562f5f1a44f0 2023-07-21T19:25:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.116+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fec6c0074e0 con 0x562f5f1a44f0 2023-07-21T19:25:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fec70005140 con 0x562f5f1a44f0 2023-07-21T19:25:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 10 client.28606 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:25.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 10 client.28606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 20 client.28606 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f213550 0x562f5f215960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 4 client.28606 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x562f5f0b1050 con 0x562f5f213550 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.117+0000 7fec7999a700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f213550 0x562f5f215960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec7999a700 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f213550 0x562f5f215960 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74895ac9792b1a87 server_cookie=6b66e319b1f56848 in_seq=0 out_seq=0 2023-07-21T19:25:25.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec5dffb700 10 client.28606 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec5dffb700 1 -- 172.21.15.5:0/3201384634 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x562f5f0b1050 con 0x562f5f213550 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec5dffb700 10 client.28606 handle_command_reply: tid=0 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 client.28606 shutdown 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 msgr2=0x562f5f1b56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 0x562f5f1b56d0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x562f5f0f6330 tx=0x7fec70004750 comp rx=0 tx=0).stop 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 shutdown_connections 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f5f1aed30 0x562f5f1b3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f5f1a44f0 0x562f5f1b56d0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f5f213550 0x562f5f215960 crc :-1 s=READY pgs=360 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 --2- 172.21.15.5:0/3201384634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f5f1a4e00 0x562f5f1b3a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec5dffb700 0 client.28606 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 >> 172.21.15.5:0/3201384634 conn(0x562f5f199ff0 msgr2=0x562f5f1a6cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 shutdown_connections 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.118+0000 7fec8fdc3b80 1 -- 172.21.15.5:0/3201384634 wait complete. 2023-07-21T19:25:25.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec8fdc3b80 20 client.28606 trim_cache size 0 max 16384 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "21c9ad14-98e0-495d-9702-9f9253636b2c": { 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:25:25.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "21c9ad14-98e0-495d-9702-9f9253636b2c", 2023-07-21T19:25:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:25:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:25:25.118 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:25:25.118 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 msgr2=0x7fec600a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 0x7fec600a3810 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fec74100b10 tx=0x7fec70009260 comp rx=0 tx=0).stop 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 msgr2=0x7fec74072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fec6400e9a0 tx=0x7fec6400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 shutdown_connections 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fec600a1360 0x7fec600a3810 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fec741059e0 0x7fec741a3aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fec740ff9d0 0x7fec74072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 --2- 172.21.15.5:0/2114915674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fec74100380 0x7fec74072d50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.119+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 >> 172.21.15.5:0/2114915674 conn(0x7fec740fb4b0 msgr2=0x7fec740fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.120+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 shutdown_connections 2023-07-21T19:25:25.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:25.120+0000 7fec7b19d700 1 -- 172.21.15.5:0/2114915674 wait complete. 2023-07-21T19:25:25.129 INFO:tasks.cephfs.filesystem:scrub status for tag:21c9ad14-98e0-495d-9702-9f9253636b2c - {'path': '/', 'tag': '21c9ad14-98e0-495d-9702-9f9253636b2c', 'options': 'recursive,force'} 2023-07-21T19:25:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: pgmap v561: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 16 op/s 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:25:25.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:25 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: pgmap v561: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 16 op/s 2023-07-21T19:25:25.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:25:25.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:25.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:25 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:26.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: pgmap v561: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 18 MiB/s wr, 16 op/s 2023-07-21T19:25:26.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:26.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:26.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:25 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:25:27.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:27 smithi005 ceph-mon[126460]: pgmap v562: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-21T19:25:27.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:27 smithi156 ceph-mon[128316]: pgmap v562: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-21T19:25:28.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:27 smithi101 ceph-mon[128136]: pgmap v562: 97 pgs: 97 active+clean; 15 GiB data, 7.0 GiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 17 MiB/s wr, 15 op/s 2023-07-21T19:25:29.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:29 smithi005 ceph-mon[126460]: pgmap v563: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 24 MiB/s wr, 161 op/s 2023-07-21T19:25:29.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:29 smithi156 ceph-mon[128316]: pgmap v563: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 24 MiB/s wr, 161 op/s 2023-07-21T19:25:30.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:29 smithi101 ceph-mon[128136]: pgmap v563: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 129 KiB/s rd, 24 MiB/s wr, 161 op/s 2023-07-21T19:25:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 -- 172.21.15.5:0/2466927152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c0685b0 msgr2=0x7f3a8c104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2466927152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c104770 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3a800099c0 tx=0x7f3a80009cd0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 -- 172.21.15.5:0/2466927152 shutdown_connections 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2466927152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c104e50 0x7f3a8c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2466927152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c067c10 0x7f3a8c067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2466927152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c104770 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 -- 172.21.15.5:0/2466927152 >> 172.21.15.5:0/2466927152 conn(0x7f3a8c074760 msgr2=0x7f3a8c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 -- 172.21.15.5:0/2466927152 shutdown_connections 2023-07-21T19:25:31.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.564+0000 7f3a925ff700 1 -- 172.21.15.5:0/2466927152 wait complete. 2023-07-21T19:25:31.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.565+0000 7f3a925ff700 1 Processor -- start 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 -- start start 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c19c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c104e50 0x7f3a8c1959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c196020 con 0x7f3a8c067c10 2023-07-21T19:25:31.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c196190 con 0x7f3a8c104e50 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a925ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a8c196350 con 0x7f3a8c0685b0 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48520/0 (socket says 172.21.15.5:48520) 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2591963732 learned_addr learned my addr 172.21.15.5:0/2591963732 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.566+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c104e50 0x7f3a8c1959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c19c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2591963732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c0685b0 msgr2=0x7f3a8c19c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c19c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2591963732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c104e50 msgr2=0x7f3a8c1959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c104e50 0x7f3a8c1959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2591963732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800095f0 con 0x7f3a8c067c10 2023-07-21T19:25:31.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9a0 tx=0x7f3a7c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a897fa700 1 -- 172.21.15.5:0/2591963732 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c009790 con 0x7f3a8c067c10 2023-07-21T19:25:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a897fa700 1 -- 172.21.15.5:0/2591963732 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a7c00cbd0 con 0x7f3a8c067c10 2023-07-21T19:25:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.567+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a8c196630 con 0x7f3a8c067c10 2023-07-21T19:25:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.568+0000 7f3a897fa700 1 -- 172.21.15.5:0/2591963732 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c018680 con 0x7f3a8c067c10 2023-07-21T19:25:31.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.568+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8c1a2b40 con 0x7f3a8c067c10 2023-07-21T19:25:31.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.571+0000 7f3a897fa700 1 -- 172.21.15.5:0/2591963732 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3a7c00cd40 con 0x7f3a8c067c10 2023-07-21T19:25:31.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3aa5e2bb80 1 Processor -- start 2023-07-21T19:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3aa5e2bb80 1 -- start start 2023-07-21T19:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3a897fa700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 0x7f3a740a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3a897fa700 1 -- 172.21.15.5:0/2591963732 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a7c014070 con 0x7f3a8c067c10 2023-07-21T19:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb1328a450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 0x7f3a740a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.573+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb1328ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1318bc50 con 0x55fb130e26b0 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb131de9e0 con 0x55fb1328a990 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb131d7e70 con 0x55fb132906e0 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48526/0 (socket says 172.21.15.5:48526) 2023-07-21T19:25:31.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 -- 172.21.15.5:0/715168736 learned_addr learned my addr 172.21.15.5:0/715168736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb1328ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 -- 172.21.15.5:0/715168736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 msgr2=0x55fb1328a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb1328a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb1328a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 -- 172.21.15.5:0/715168736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 msgr2=0x55fb1328ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.574+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb1328ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a8bfff700 1 -- 172.21.15.5:0/715168736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb13005d80 con 0x55fb130e26b0 2023-07-21T19:25:31.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb1328a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb1328ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 0x7f3a740a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009ec0 tx=0x7f3a8400fc40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0a1fb0 tx=0x7f3a7c099f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7c96a2491aca6c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a727fc700 1 -- 172.21.15.5:0/715168736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0a24e0 con 0x55fb130e26b0 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a727fc700 1 -- 172.21.15.5:0/715168736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a7c09a2f0 con 0x55fb130e26b0 2023-07-21T19:25:31.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.575+0000 7f3a727fc700 1 -- 172.21.15.5:0/715168736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0a2050 con 0x55fb130e26b0 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/715168736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 msgr2=0x55fb132901a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c0a1fb0 tx=0x7f3a7c099f70 comp rx=0 tx=0).stop 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/715168736 shutdown_connections 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb1328ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb130e26b0 0x55fb132901a0 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/715168736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb1328a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/715168736 >> 172.21.15.5:0/715168736 conn(0x55fb1327bbe0 msgr2=0x55fb13280f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/715168736 shutdown_connections 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.576+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/715168736 wait complete. 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.577+0000 7f3aa5e2bb80 1 -- start start 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.577+0000 7f3aa5e2bb80 1 Processor -- start 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13296a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132943b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb13177a60 con 0x55fb1328a990 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb13177760 con 0x55fb130e26b0 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb1305b8c0 con 0x55fb132906e0 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3aa5e2bb80 10 client.0 _ll_register_callbacks cb 0x55fb13175700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48540/0 (socket says 172.21.15.5:48540) 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/882761919 learned_addr learned my addr 172.21.15.5:0/882761919 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:31.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132943b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13296a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/882761919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 msgr2=0x55fb132943b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.578+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132943b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 msgr2=0x55fb13296a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13296a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/882761919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a8400fdc0 con 0x55fb1328a990 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b580 tx=0x7f3a8000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9758283bdc14bc21 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800054d0 con 0x55fb1328a990 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fb13131490 con 0x55fb1328a990 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a80005630 con 0x55fb1328a990 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.579+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a80003760 con 0x55fb1328a990 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 10 client.19053 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f3a8c1a2b40 con 0x55fb1328a990 2023-07-21T19:25:31.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a8002f020 con 0x55fb1328a990 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a8000f770 con 0x55fb1328a990 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 10 client.19053 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 10 client.19053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 20 client.19053 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132a3ea0 0x55fb132a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 4 client.19053 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:31.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.580+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fb131d0e50 con 0x55fb132a3ea0 2023-07-21T19:25:31.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.581+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132a3ea0 0x55fb132a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.581+0000 7f3a6b7fe700 10 client.19053 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.581+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132a3ea0 0x55fb132a62b0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41f84c9fa0fd0edf server_cookie=54f89b2fc747c9a2 in_seq=0 out_seq=0 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/882761919 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fb131d0e50 con 0x55fb132a3ea0 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3a6b7fe700 10 client.19053 handle_command_reply: tid=0 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 client.19053 shutdown 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 msgr2=0x55fb13293cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000b580 tx=0x7f3a8000be10 comp rx=0 tx=0).stop 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 shutdown_connections 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132943b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13296a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132a3ea0 0x55fb132a62b0 crc :-1 s=READY pgs=361 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/882761919 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb13293cc0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3a6b7fe700 0 client.19053 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.582+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 >> 172.21.15.5:0/882761919 conn(0x55fb1327bbe0 msgr2=0x55fb132804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.583+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 shutdown_connections 2023-07-21T19:25:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.583+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/882761919 wait complete. 2023-07-21T19:25:31.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.583+0000 7f3aa5e2bb80 20 client.19053 trim_cache size 0 max 16384 2023-07-21T19:25:31.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 Processor -- start 2023-07-21T19:25:31.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 -- start start 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb13280c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb13278bd0 con 0x55fb132906e0 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb132952a0 con 0x55fb1328a990 2023-07-21T19:25:31.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.603+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb132903f0 con 0x55fb130e26b0 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a90b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a90b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:48546/0 (socket says 172.21.15.5:48546) 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a90b9c700 1 -- 172.21.15.5:0/917089185 learned_addr learned my addr 172.21.15.5:0/917089185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38734/0 (socket says 172.21.15.5:38734) 2023-07-21T19:25:31.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb13280c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/917089185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 msgr2=0x55fb13293480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/917089185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 msgr2=0x55fb132965e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8b7fe700 1 -- 172.21.15.5:0/917089185 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fb12feec00 con 0x55fb1328a990 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.604+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.605+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb13280c60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000fb00 tx=0x7f3a80005050 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13c5e839fd0b24b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.605+0000 7f3a72ffd700 1 -- 172.21.15.5:0/917089185 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800054d0 con 0x55fb1328a990 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.605+0000 7f3a72ffd700 1 -- 172.21.15.5:0/917089185 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a8002f030 con 0x55fb1328a990 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.605+0000 7f3a72ffd700 1 -- 172.21.15.5:0/917089185 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a800039a0 con 0x55fb1328a990 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/917089185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 msgr2=0x55fb13280c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb13280c60 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f3a8000fb00 tx=0x7f3a80005050 comp rx=0 tx=0).stop 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/917089185 shutdown_connections 2023-07-21T19:25:31.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb132906e0 0x55fb132965e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb130e26b0 0x55fb13293480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/917089185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb1328a990 0x55fb13280c60 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/917089185 >> 172.21.15.5:0/917089185 conn(0x55fb1327bbe0 msgr2=0x55fb1328dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/917089185 shutdown_connections 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.606+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/917089185 wait complete. 2023-07-21T19:25:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.607+0000 7f3aa5e2bb80 1 -- start start 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.607+0000 7f3aa5e2bb80 1 Processor -- start 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb132995a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb13299ee0 con 0x55fb1328a990 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb132f72a0 con 0x55fb130e26b0 2023-07-21T19:25:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fb132f7460 con 0x55fb132906e0 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.608+0000 7f3aa5e2bb80 10 client.0 _ll_register_callbacks cb 0x7f3a80009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41442/0 (socket says 172.21.15.5:41442) 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2349962272 learned_addr learned my addr 172.21.15.5:0/2349962272 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb132995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a90b9c700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2349962272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 msgr2=0x55fb132f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132f6d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2349962272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 msgr2=0x55fb132995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.609+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb132995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.610+0000 7f3a8bfff700 1 -- 172.21.15.5:0/2349962272 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55fb13005d80 con 0x55fb130e26b0 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.610+0000 7f3a8bfff700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c096590 tx=0x7f3a7c096500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c52a5df82c586c6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.610+0000 7f3a6b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.611+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7c0b8040 con 0x55fb130e26b0 2023-07-21T19:25:31.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.611+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a7c098430 con 0x55fb130e26b0 2023-07-21T19:25:31.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.611+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a7c09a2f0 con 0x55fb130e26b0 2023-07-21T19:25:31.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.617+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a7c0ca2d0 con 0x55fb130e26b0 2023-07-21T19:25:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.617+0000 7f3aa5e2bb80 10 client.28700 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:31.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.617+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f3a84009040 con 0x55fb130e26b0 2023-07-21T19:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a7c07c020 con 0x55fb130e26b0 2023-07-21T19:25:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3a7c099690 con 0x55fb130e26b0 2023-07-21T19:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 10 client.28700 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 10 client.28700 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 20 client.28700 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132f8840 0x55fb132fac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 4 client.28700 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:25:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.618+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fb13192ff0 con 0x55fb132f8840 2023-07-21T19:25:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.619+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132f8840 0x55fb132fac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.620+0000 7f3a8b7fe700 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132f8840 0x55fb132fac50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8447522abbabd303 server_cookie=4706a0d16c370083 in_seq=0 out_seq=0 2023-07-21T19:25:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.620+0000 7f3a6b7fe700 10 client.28700 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3a6b7fe700 1 -- 172.21.15.5:0/2349962272 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fb13192ff0 con 0x55fb132f8840 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3a6b7fe700 10 client.28700 handle_command_reply: tid=0 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 client.28700 shutdown 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 msgr2=0x55fb13299060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c096590 tx=0x7f3a7c096500 comp rx=0 tx=0).stop 2023-07-21T19:25:31.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 shutdown_connections 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fb132906e0 0x55fb132f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fb130e26b0 0x55fb13299060 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fb132f8840 0x55fb132fac50 crc :-1 s=READY pgs=362 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 --2- 172.21.15.5:0/2349962272 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fb1328a990 0x55fb132995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3a6b7fe700 0 client.28700 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 >> 172.21.15.5:0/2349962272 conn(0x55fb1327bbe0 msgr2=0x55fb1328eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 shutdown_connections 2023-07-21T19:25:31.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 1 -- 172.21.15.5:0/2349962272 wait complete. 2023-07-21T19:25:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.631+0000 7f3aa5e2bb80 20 client.28700 trim_cache size 0 max 16384 2023-07-21T19:25:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:25:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:25:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:25:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:25:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 msgr2=0x7f3a740a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 0x7f3a740a3770 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3a84009ec0 tx=0x7f3a8400fc40 comp rx=0 tx=0).stop 2023-07-21T19:25:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 msgr2=0x7f3a8c19bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f3a7c00e9a0 tx=0x7f3a7c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 shutdown_connections 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a8c104e50 0x7f3a8c1959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a740a12c0 0x7f3a740a3770 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a8c067c10 0x7f3a8c19bb50 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 --2- 172.21.15.5:0/2591963732 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a8c0685b0 0x7f3a8c19c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 >> 172.21.15.5:0/2591963732 conn(0x7f3a8c074760 msgr2=0x7f3a8c103f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 shutdown_connections 2023-07-21T19:25:31.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:31.632+0000 7f3a925ff700 1 -- 172.21.15.5:0/2591963732 wait complete. 2023-07-21T19:25:31.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:31 smithi005 ceph-mon[126460]: pgmap v564: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 154 op/s 2023-07-21T19:25:31.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:31 smithi156 ceph-mon[128316]: pgmap v564: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 154 op/s 2023-07-21T19:25:32.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:31 smithi101 ceph-mon[128136]: pgmap v564: 97 pgs: 97 active+clean; 15 GiB data, 7.1 GiB used, 1.0 TiB / 1.0 TiB avail; 102 KiB/s rd, 15 MiB/s wr, 154 op/s 2023-07-21T19:25:33.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:33 smithi005 ceph-mon[126460]: pgmap v565: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 27 MiB/s wr, 191 op/s 2023-07-21T19:25:33.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:33 smithi156 ceph-mon[128316]: pgmap v565: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 27 MiB/s wr, 191 op/s 2023-07-21T19:25:34.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:33 smithi101 ceph-mon[128136]: pgmap v565: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 27 MiB/s wr, 191 op/s 2023-07-21T19:25:35.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:35 smithi005 ceph-mon[126460]: pgmap v566: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 19 MiB/s wr, 183 op/s 2023-07-21T19:25:35.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:35 smithi156 ceph-mon[128316]: pgmap v566: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 19 MiB/s wr, 183 op/s 2023-07-21T19:25:36.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:35 smithi101 ceph-mon[128136]: pgmap v566: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 19 MiB/s wr, 183 op/s 2023-07-21T19:25:36.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:36 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:36.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:36 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:36.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:36 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:25:36.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:36 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:36.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:36 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:36.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:36 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:25:37.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:36 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:37.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:36 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:37.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:36 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:25:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:37 smithi005 ceph-mon[126460]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 182 op/s 2023-07-21T19:25:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:37 smithi156 ceph-mon[128316]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 182 op/s 2023-07-21T19:25:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:37 smithi101 ceph-mon[128136]: pgmap v567: 97 pgs: 97 active+clean; 15 GiB data, 7.2 GiB used, 1.0 TiB / 1.0 TiB avail; 114 KiB/s rd, 18 MiB/s wr, 182 op/s 2023-07-21T19:25:39.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:39 smithi005 ceph-mon[126460]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 219 op/s 2023-07-21T19:25:39.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:39 smithi156 ceph-mon[128316]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 219 op/s 2023-07-21T19:25:40.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:39 smithi101 ceph-mon[128136]: pgmap v568: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 136 KiB/s rd, 33 MiB/s wr, 219 op/s 2023-07-21T19:25:41.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:41 smithi005 ceph-mon[126460]: pgmap v569: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 73 op/s 2023-07-21T19:25:41.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:41 smithi156 ceph-mon[128316]: pgmap v569: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 73 op/s 2023-07-21T19:25:42.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:41 smithi101 ceph-mon[128136]: pgmap v569: 97 pgs: 97 active+clean; 15 GiB data, 7.4 GiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 26 MiB/s wr, 73 op/s 2023-07-21T19:25:43.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:43 smithi005 ceph-mon[126460]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 39 MiB/s wr, 136 op/s 2023-07-21T19:25:43.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:43 smithi156 ceph-mon[128316]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 39 MiB/s wr, 136 op/s 2023-07-21T19:25:44.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:43 smithi101 ceph-mon[128136]: pgmap v570: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 39 MiB/s wr, 136 op/s 2023-07-21T19:25:44.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 -- 172.21.15.5:0/364788864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c105b10 msgr2=0x7f4f7c105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 --2- 172.21.15.5:0/364788864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c105b10 0x7f4f7c105ee0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c009990 tx=0x7f4f6c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 -- 172.21.15.5:0/364788864 shutdown_connections 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 --2- 172.21.15.5:0/364788864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 0x7f4f7c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 --2- 172.21.15.5:0/364788864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c105b10 0x7f4f7c105ee0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 --2- 172.21.15.5:0/364788864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c0ffb00 0x7f4f7c0fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 -- 172.21.15.5:0/364788864 >> 172.21.15.5:0/364788864 conn(0x7f4f7c0fb600 msgr2=0x7f4f7c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 -- 172.21.15.5:0/364788864 shutdown_connections 2023-07-21T19:25:44.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.805+0000 7f4f81bff700 1 -- 172.21.15.5:0/364788864 wait complete. 2023-07-21T19:25:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.806+0000 7f4f81bff700 1 Processor -- start 2023-07-21T19:25:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.806+0000 7f4f81bff700 1 -- start start 2023-07-21T19:25:44.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.806+0000 7f4f81bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c0ffb00 0x7f4f7c0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f81bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 0x7f4f7c072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f81bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f81bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1a4300 con 0x7f4f7c105b10 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f81bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1a4440 con 0x7f4f7c0ffb00 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f81bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f7c1a45b0 con 0x7f4f7c1004b0 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c0ffb00 0x7f4f7c0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41534/0 (socket says 172.21.15.5:41534) 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 -- 172.21.15.5:0/2921167974 learned_addr learned my addr 172.21.15.5:0/2921167974 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7affd700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 0x7f4f7c072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 -- 172.21.15.5:0/2921167974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 msgr2=0x7f4f7c072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 0x7f4f7c072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 -- 172.21.15.5:0/2921167974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c0ffb00 msgr2=0x7f4f7c0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c0ffb00 0x7f4f7c0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.807+0000 7f4f7bfff700 1 -- 172.21.15.5:0/2921167974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f6c0095f0 con 0x7f4f7c105b10 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f7bfff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400ef30 tx=0x7f4f7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f78ff9700 1 -- 172.21.15.5:0/2921167974 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f74018070 con 0x7f4f7c105b10 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f78ff9700 1 -- 172.21.15.5:0/2921167974 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f7400f040 con 0x7f4f7c105b10 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f7c1a4830 con 0x7f4f7c105b10 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f78ff9700 1 -- 172.21.15.5:0/2921167974 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f7400cbc0 con 0x7f4f7c105b10 2023-07-21T19:25:44.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.808+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f7c1a4cf0 con 0x7f4f7c105b10 2023-07-21T19:25:44.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.812+0000 7f4f78ff9700 1 -- 172.21.15.5:0/2921167974 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4f74007500 con 0x7f4f7c105b10 2023-07-21T19:25:44.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 Processor -- start 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 -- start start 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed600c50 con 0x55f4ed5576b0 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6539e0 con 0x55f4ed7012d0 2023-07-21T19:25:44.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed64ce70 con 0x55f4ed6fb580 2023-07-21T19:25:44.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.813+0000 7f4f78ff9700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 0x7f4f640a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f78ff9700 1 -- 172.21.15.5:0/2921167974 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4f740cc7d0 con 0x7f4f7c105b10 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7bfff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 0x7f4f640a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36342/0 (socket says 172.21.15.5:36342) 2023-07-21T19:25:44.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:32794/0 (socket says 172.21.15.5:32794) 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/4067433993 learned_addr learned my addr 172.21.15.5:0/4067433993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7affd700 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7bfff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 0x7f4f640a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4f740090a0 tx=0x7f4f74004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/4067433993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 msgr2=0x55f4ed6fb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.814+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/4067433993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 msgr2=0x55f4ed700d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/4067433993 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ed47ad80 con 0x55f4ed7012d0 2023-07-21T19:25:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f7affd700 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000c00 tx=0x7f4f6c005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4a287c122766b63 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f61ffb700 1 -- 172.21.15.5:0/4067433993 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01d070 con 0x55f4ed7012d0 2023-07-21T19:25:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f61ffb700 1 -- 172.21.15.5:0/4067433993 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f6c01e040 con 0x55f4ed7012d0 2023-07-21T19:25:44.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.815+0000 7f4f61ffb700 1 -- 172.21.15.5:0/4067433993 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c004740 con 0x55f4ed7012d0 2023-07-21T19:25:44.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 -- 172.21.15.5:0/4067433993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 msgr2=0x55f4ed6fb040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c000c00 tx=0x7f4f6c005bc0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 -- 172.21.15.5:0/4067433993 shutdown_connections 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/4067433993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fb040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 -- 172.21.15.5:0/4067433993 >> 172.21.15.5:0/4067433993 conn(0x55f4ed6f0be0 msgr2=0x55f4ed6f5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 -- 172.21.15.5:0/4067433993 shutdown_connections 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.816+0000 7f4f9546ab80 1 -- 172.21.15.5:0/4067433993 wait complete. 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 -- start start 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 Processor -- start 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed7048c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed704ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed5eca60 con 0x55f4ed5576b0 2023-07-21T19:25:44.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed5ec760 con 0x55f4ed7012d0 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.817+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed4d08c0 con 0x55f4ed6fb580 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f9546ab80 10 client.0 _ll_register_callbacks cb 0x55f4ed5ea700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41548/0 (socket says 172.21.15.5:41548) 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 -- 172.21.15.5:0/3169980041 learned_addr learned my addr 172.21.15.5:0/3169980041 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed7048c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed704ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 -- 172.21.15.5:0/3169980041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 msgr2=0x55f4ed7048c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed7048c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 -- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 msgr2=0x55f4ed704ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed704ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 -- 172.21.15.5:0/3169980041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f70019040 con 0x55f4ed5576b0 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.818+0000 7f4f7affd700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4f70004610 tx=0x7f4f700047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9a382f5954dbd6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:25:44.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f70016040 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4f70003e20 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4ed5a6490 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f70004830 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f9546ab80 10 client.19071 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x7f4f7c10d240 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.819+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f4ed5a6490 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f70020880 con 0x55f4ed5576b0 2023-07-21T19:25:44.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 10 client.19071 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 10 client.19071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 20 client.19071 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed714af0 0x55f4ed716f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 4 client.19071 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4ed645e50 con 0x55f4ed714af0 2023-07-21T19:25:44.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.820+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed714af0 0x55f4ed716f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.821+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed714af0 0x55f4ed716f00 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca9980261cf5dd4a server_cookie=6587d146d95b4244 in_seq=0 out_seq=0 2023-07-21T19:25:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.821+0000 7f4f5affd700 10 client.19071 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:44.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.821+0000 7f4f5affd700 1 -- 172.21.15.5:0/3169980041 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4ed645e50 con 0x55f4ed714af0 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.821+0000 7f4f5affd700 10 client.19071 handle_command_reply: tid=0 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 client.19071 shutdown 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 msgr2=0x55f4ed7075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f4f70004610 tx=0x7f4f700047c0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 shutdown_connections 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed704ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed714af0 0x55f4ed716f00 crc :-1 s=READY pgs=363 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed7075a0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/3169980041 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed7048c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 >> 172.21.15.5:0/3169980041 conn(0x55f4ed6f0be0 msgr2=0x55f4ed6f4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f5affd700 0 client.19071 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 shutdown_connections 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 1 -- 172.21.15.5:0/3169980041 wait complete. 2023-07-21T19:25:44.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.822+0000 7f4f9546ab80 20 client.19071 trim_cache size 0 max 16384 2023-07-21T19:25:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 Processor -- start 2023-07-21T19:25:44.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 -- start start 2023-07-21T19:25:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed707060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6f7bb0 con 0x55f4ed5576b0 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6c1090 con 0x55f4ed7012d0 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.839+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed7016a0 con 0x55f4ed6fb580 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:32812/0 (socket says 172.21.15.5:32812) 2023-07-21T19:25:44.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/304823465 learned_addr learned my addr 172.21.15.5:0/304823465 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/304823465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 msgr2=0x55f4ed707060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed707060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/304823465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 msgr2=0x55f4ed700160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/304823465 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4ed463c00 con 0x55f4ed6fb580 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.840+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c005930 tx=0x7f4f6c005960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bcf5633830da057 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f627fc700 1 -- 172.21.15.5:0/304823465 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01d040 con 0x55f4ed6fb580 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f627fc700 1 -- 172.21.15.5:0/304823465 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f6c00b710 con 0x55f4ed6fb580 2023-07-21T19:25:44.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f627fc700 1 -- 172.21.15.5:0/304823465 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c035a30 con 0x55f4ed6fb580 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 -- 172.21.15.5:0/304823465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 msgr2=0x55f4ed6f5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c005930 tx=0x7f4f6c005960 comp rx=0 tx=0).stop 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 -- 172.21.15.5:0/304823465 shutdown_connections 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed7012d0 0x55f4ed707060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed5576b0 0x55f4ed700160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/304823465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed6fb580 0x55f4ed6f5c60 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 -- 172.21.15.5:0/304823465 >> 172.21.15.5:0/304823465 conn(0x55f4ed6f0be0 msgr2=0x55f4ed6ffa50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 -- 172.21.15.5:0/304823465 shutdown_connections 2023-07-21T19:25:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.841+0000 7f4f9546ab80 1 -- 172.21.15.5:0/304823465 wait complete. 2023-07-21T19:25:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 -- start start 2023-07-21T19:25:44.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 Processor -- start 2023-07-21T19:25:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed5576b0 0x55f4ed6fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fcda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6fe4c0 con 0x55f4ed7012d0 2023-07-21T19:25:44.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6fe630 con 0x55f4ed6fb580 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.842+0000 7f4f9546ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4ed6fd2e0 con 0x55f4ed5576b0 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f9546ab80 10 client.0 _ll_register_callbacks cb 0x7f4f70004300 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed5576b0 0x55f4ed6fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36368/0 (socket says 172.21.15.5:36368) 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/321803939 learned_addr learned my addr 172.21.15.5:0/321803939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:44.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7affd700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fcda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/321803939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed5576b0 msgr2=0x55f4ed6fddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed5576b0 0x55f4ed6fddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/321803939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 msgr2=0x55f4ed6fcda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fcda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7b7fe700 1 -- 172.21.15.5:0/321803939 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4ed47ad80 con 0x55f4ed6fb580 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.843+0000 7f4f7affd700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f7b7fe700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00bee0 tx=0x7f4f6c0053a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b14d087eab9ff31f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:44.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f5affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f6c01d040 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4f6c01e040 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4f6c00bb80 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4f6c004500 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f9546ab80 10 client.28715 fetch_fsmap learned FSMap version 54 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.844+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=4+}) v3 -- 0x55f4ed4a11f0 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4f6c01e040 con 0x55f4ed6fb580 2023-07-21T19:25:44.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 54) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4f6c00f6c0 con 0x55f4ed6fb580 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 10 client.28715 fetch_fsmap finished waiting for FSMap version 54 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 10 client.28715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 20 client.28715 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed769640 0x55f4ed76ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 4 client.28715 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.845+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f4ed607ff0 con 0x55f4ed769640 2023-07-21T19:25:44.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.846+0000 7f4f7affd700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed769640 0x55f4ed76ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.846+0000 7f4f7affd700 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed769640 0x55f4ed76ba50 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d07b4cc55bf480b server_cookie=d8a401198bfbcc94 in_seq=0 out_seq=0 2023-07-21T19:25:44.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.846+0000 7f4f5affd700 10 client.28715 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f5affd700 1 -- 172.21.15.5:0/321803939 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f4ed607ff0 con 0x55f4ed769640 2023-07-21T19:25:44.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f5affd700 10 client.28715 handle_command_reply: tid=0 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 client.28715 shutdown 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 msgr2=0x55f4ed6fc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4f6c00bee0 tx=0x7f4f6c0053a0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 shutdown_connections 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4ed769640 0x55f4ed76ba50 crc :-1 s=READY pgs=364 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4ed7012d0 0x55f4ed6fcda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4ed5576b0 0x55f4ed6fddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f5affd700 0 client.28715 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 --2- 172.21.15.5:0/321803939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4ed6fb580 0x55f4ed6fc860 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 >> 172.21.15.5:0/321803939 conn(0x55f4ed6f0be0 msgr2=0x55f4ed6fa030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 shutdown_connections 2023-07-21T19:25:44.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 1 -- 172.21.15.5:0/321803939 wait complete. 2023-07-21T19:25:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.855+0000 7f4f9546ab80 20 client.28715 trim_cache size 0 max 16384 2023-07-21T19:25:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:25:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:25:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:25:44.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:25:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 msgr2=0x7f4f640a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 0x7f4f640a3770 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f4f740090a0 tx=0x7f4f74004ad0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 msgr2=0x7f4f7c1a3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:44.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f4f7400ef30 tx=0x7f4f7400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 shutdown_connections 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4f640a12c0 0x7f4f640a3770 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4f7c105b10 0x7f4f7c1a3dc0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4f7c0ffb00 0x7f4f7c0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 --2- 172.21.15.5:0/2921167974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4f7c1004b0 0x7f4f7c072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 >> 172.21.15.5:0/2921167974 conn(0x7f4f7c0fb600 msgr2=0x7f4f7c0fbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:44.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 shutdown_connections 2023-07-21T19:25:44.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:25:44.856+0000 7f4f81bff700 1 -- 172.21.15.5:0/2921167974 wait complete. 2023-07-21T19:25:45.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:45 smithi005 ceph-mon[126460]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:45.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:45 smithi156 ceph-mon[128316]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:46.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:45 smithi101 ceph-mon[128136]: pgmap v571: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:47.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:47 smithi005 ceph-mon[126460]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:47.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:47 smithi156 ceph-mon[128316]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:48.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:47 smithi101 ceph-mon[128136]: pgmap v572: 97 pgs: 97 active+clean; 15 GiB data, 7.6 GiB used, 1.0 TiB / 1.0 TiB avail; 228 KiB/s rd, 28 MiB/s wr, 99 op/s 2023-07-21T19:25:49.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:49 smithi005 ceph-mon[126460]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 42 MiB/s wr, 135 op/s 2023-07-21T19:25:49.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:49 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:49.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:49 smithi156 ceph-mon[128316]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 42 MiB/s wr, 135 op/s 2023-07-21T19:25:49.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:49 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:50.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:49 smithi101 ceph-mon[128136]: pgmap v573: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 295 KiB/s rd, 42 MiB/s wr, 135 op/s 2023-07-21T19:25:50.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:49 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:51.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:51 smithi005 ceph-mon[126460]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 28 MiB/s wr, 98 op/s 2023-07-21T19:25:51.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:51 smithi156 ceph-mon[128316]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 28 MiB/s wr, 98 op/s 2023-07-21T19:25:52.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:51 smithi101 ceph-mon[128136]: pgmap v574: 97 pgs: 97 active+clean; 15 GiB data, 7.8 GiB used, 1.0 TiB / 1.0 TiB avail; 273 KiB/s rd, 28 MiB/s wr, 98 op/s 2023-07-21T19:25:53.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:53 smithi005 ceph-mon[126460]: pgmap v575: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 41 MiB/s wr, 105 op/s 2023-07-21T19:25:53.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:53 smithi156 ceph-mon[128316]: pgmap v575: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 41 MiB/s wr, 105 op/s 2023-07-21T19:25:54.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:53 smithi101 ceph-mon[128136]: pgmap v575: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 291 KiB/s rd, 41 MiB/s wr, 105 op/s 2023-07-21T19:25:54.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:54 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:54.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:54 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:55.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:54 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:25:55.131 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:25:55.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 -- 172.21.15.5:0/768379634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 msgr2=0x7f1394101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 --2- 172.21.15.5:0/768379634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f1394101070 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f138400b0a0 tx=0x7f138400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 -- 172.21.15.5:0/768379634 shutdown_connections 2023-07-21T19:25:55.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 --2- 172.21.15.5:0/768379634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f139410a070 0x7f139410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 --2- 172.21.15.5:0/768379634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f1394101070 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 --2- 172.21.15.5:0/768379634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f13941015b0 0x7f1394109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 -- 172.21.15.5:0/768379634 >> 172.21.15.5:0/768379634 conn(0x7f13940fa4b0 msgr2=0x7f13940fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 -- 172.21.15.5:0/768379634 shutdown_connections 2023-07-21T19:25:55.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.642+0000 7f13993ff700 1 -- 172.21.15.5:0/768379634 wait complete. 2023-07-21T19:25:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.643+0000 7f13993ff700 1 Processor -- start 2023-07-21T19:25:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.643+0000 7f13993ff700 1 -- start start 2023-07-21T19:25:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f13940fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 0x7f13940fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f139410a070 0x7f13941a3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13940ff660 con 0x7f13941015b0 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a4380 con 0x7f139410a070 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13993ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13941a44c0 con 0x7f1394100ca0 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f138a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 0x7f13940fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f13940fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f13940fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52546/0 (socket says 172.21.15.5:52546) 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f1392ffd700 1 -- 172.21.15.5:0/297738088 learned_addr learned my addr 172.21.15.5:0/297738088 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.644+0000 7f13937fe700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f139410a070 0x7f13941a3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 -- 172.21.15.5:0/297738088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 msgr2=0x7f13940fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f13940fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 -- 172.21.15.5:0/297738088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f139410a070 msgr2=0x7f13941a3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f139410a070 0x7f13941a3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 -- 172.21.15.5:0/297738088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1384009d00 con 0x7f13941015b0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f138a7fc700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 0x7f13940fee30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e9f0 tx=0x7f137c00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f1390ff9700 1 -- 172.21.15.5:0/297738088 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00c9c0 con 0x7f13941015b0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13941a4720 con 0x7f13941015b0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.645+0000 7f1390ff9700 1 -- 172.21.15.5:0/297738088 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f137c010430 con 0x7f13941015b0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.646+0000 7f1390ff9700 1 -- 172.21.15.5:0/297738088 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c018680 con 0x7f13941015b0 2023-07-21T19:25:55.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.646+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a4c00 con 0x7f13941015b0 2023-07-21T19:25:55.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.649+0000 7f1390ff9700 1 -- 172.21.15.5:0/297738088 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f137c0105a0 con 0x7f13941015b0 2023-07-21T19:25:55.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.651+0000 7f1390ff9700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 0x7f13800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.651+0000 7f1390ff9700 1 -- 172.21.15.5:0/297738088 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f137c014070 con 0x7f13941015b0 2023-07-21T19:25:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.651+0000 7f1392ffd700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 0x7f13800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.652+0000 7f1392ffd700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 0x7f13800a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f138400b780 tx=0x7f1384006210 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 Processor -- start 2023-07-21T19:25:55.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 -- start start 2023-07-21T19:25:55.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e242d0 0x55d212e246a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 0x55d212e31eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.666+0000 7f13acd82b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212d2acb0 con 0x55d212e242d0 2023-07-21T19:25:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f13acd82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212d7da40 con 0x55d212e24be0 2023-07-21T19:25:55.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f13acd82b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212d76ed0 con 0x55d212e2fad0 2023-07-21T19:25:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e242d0 0x55d212e246a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f13937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 0x55d212e31eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f13937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 0x55d212e31eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52554/0 (socket says 172.21.15.5:52554) 2023-07-21T19:25:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f13937fe700 1 -- 172.21.15.5:0/1136244809 learned_addr learned my addr 172.21.15.5:0/1136244809 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:55.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 -- 172.21.15.5:0/1136244809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 msgr2=0x55d212e31eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 0x55d212e31eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 -- 172.21.15.5:0/1136244809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e242d0 msgr2=0x55d212e246a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e242d0 0x55d212e246a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.667+0000 7f138a7fc700 1 -- 172.21.15.5:0/1136244809 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d212ba4de0 con 0x55d212e24be0 2023-07-21T19:25:55.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.668+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ca5f0 tx=0x7f137c0998d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=916ba5f79f0e00c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.668+0000 7f13897fa700 1 -- 172.21.15.5:0/1136244809 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c09dfa0 con 0x55d212e24be0 2023-07-21T19:25:55.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.668+0000 7f13897fa700 1 -- 172.21.15.5:0/1136244809 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f137c0a23c0 con 0x55d212e24be0 2023-07-21T19:25:55.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.668+0000 7f13897fa700 1 -- 172.21.15.5:0/1136244809 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c0996c0 con 0x55d212e24be0 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 -- 172.21.15.5:0/1136244809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 msgr2=0x55d212e2f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2f590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f137c0ca5f0 tx=0x7f137c0998d0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 -- 172.21.15.5:0/1136244809 shutdown_connections 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e2fad0 0x55d212e31eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e242d0 0x55d212e246a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1136244809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2f590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 -- 172.21.15.5:0/1136244809 >> 172.21.15.5:0/1136244809 conn(0x55d212e1ada0 msgr2=0x55d212e206a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 -- 172.21.15.5:0/1136244809 shutdown_connections 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.669+0000 7f13acd82b80 1 -- 172.21.15.5:0/1136244809 wait complete. 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.670+0000 7f13acd82b80 1 -- start start 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.670+0000 7f13acd82b80 1 Processor -- start 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 0x55d212e2f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 0x55d212e2c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2cf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212d16ac0 con 0x55d212e2fad0 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212d167c0 con 0x55d212e242d0 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212bfa920 con 0x55d212e24be0 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13acd82b80 10 client.0 _ll_register_callbacks cb 0x55d212d14760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 0x55d212e2f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f138a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 0x55d212e2c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 0x55d212e2f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:40660/0 (socket says 172.21.15.5:40660) 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f1392ffd700 1 -- 172.21.15.5:0/1823886642 learned_addr learned my addr 172.21.15.5:0/1823886642 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.671+0000 7f13937fe700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 -- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 msgr2=0x55d212e2f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 0x55d212e2f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 -- 172.21.15.5:0/1823886642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 msgr2=0x55d212e2cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 -- 172.21.15.5:0/1823886642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13941a4c00 con 0x55d212e24be0 2023-07-21T19:25:55.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f13937fe700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:25:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f138a7fc700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 0x55d212e2c7e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f137c0982c0 tx=0x7f137c09a4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f2abfffa29c4418 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f137a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c09dfa0 con 0x55d212e24be0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f137c0a2140 con 0x55d212e24be0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.672+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d212cd04f0 con 0x55d212e24be0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f137c0b80d0 con 0x55d212e24be0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 10 client.28651 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d212b8dc60 con 0x55d212e24be0 2023-07-21T19:25:55.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d212cd04f0 con 0x55d212e24be0 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f137c094d10 con 0x55d212e24be0 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 10 client.28651 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 10 client.28651 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 20 client.28651 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e3f030 0x55d212e41410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 4 client.28651 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.673+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d212d6feb0 con 0x55d212e3f030 2023-07-21T19:25:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.674+0000 7f1392ffd700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e3f030 0x55d212e41410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.674+0000 7f1392ffd700 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e3f030 0x55d212e41410 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53f0f17b0f391b41 server_cookie=7444d0b452b2712f in_seq=0 out_seq=0 2023-07-21T19:25:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.674+0000 7f137a7fc700 10 client.28651 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:55.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.675+0000 7f137a7fc700 1 -- 172.21.15.5:0/1823886642 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d212d6feb0 con 0x55d212e3f030 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.675+0000 7f137a7fc700 10 client.28651 handle_command_reply: tid=0 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.675+0000 7f13acd82b80 1 client.28651 shutdown 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.675+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 msgr2=0x55d212e2c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.675+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 0x55d212e2c7e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f137c0982c0 tx=0x7f137c09a4b0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 shutdown_connections 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e3f030 0x55d212e41410 crc :-1 s=READY pgs=365 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e242d0 0x55d212e2f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 --2- 172.21.15.5:0/1823886642 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e24be0 0x55d212e2c7e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f137a7fc700 0 client.28651 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 >> 172.21.15.5:0/1823886642 conn(0x55d212e1ada0 msgr2=0x55d212e20230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 shutdown_connections 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 1 -- 172.21.15.5:0/1823886642 wait complete. 2023-07-21T19:25:55.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.676+0000 7f13acd82b80 20 client.28651 trim_cache size 0 max 16384 2023-07-21T19:25:55.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 Processor -- start 2023-07-21T19:25:55.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 -- start start 2023-07-21T19:25:55.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2c500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e21d20 con 0x55d212e2fad0 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e25540 con 0x55d212e24be0 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f13acd82b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e2ddc0 con 0x55d212e242d0 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.699+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52570/0 (socket says 172.21.15.5:52570) 2023-07-21T19:25:55.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 -- 172.21.15.5:0/3077329532 learned_addr learned my addr 172.21.15.5:0/3077329532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:55.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f138a7fc700 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f13937fe700 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2c500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 -- 172.21.15.5:0/3077329532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 msgr2=0x55d212e2bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 -- 172.21.15.5:0/3077329532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 msgr2=0x55d212e2c500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2c500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 -- 172.21.15.5:0/3077329532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d212e1c5e0 con 0x55d212e242d0 2023-07-21T19:25:55.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1392ffd700 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fd0 tx=0x7f1384005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd5a14de54bab33c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.700+0000 7f1389ffb700 1 -- 172.21.15.5:0/3077329532 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400e040 con 0x55d212e242d0 2023-07-21T19:25:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f1389ffb700 1 -- 172.21.15.5:0/3077329532 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f138400be10 con 0x55d212e242d0 2023-07-21T19:25:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f1389ffb700 1 -- 172.21.15.5:0/3077329532 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13840050b0 con 0x55d212e242d0 2023-07-21T19:25:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 -- 172.21.15.5:0/3077329532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 msgr2=0x55d212e2f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1384009fd0 tx=0x7f1384005040 comp rx=0 tx=0).stop 2023-07-21T19:25:55.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 -- 172.21.15.5:0/3077329532 shutdown_connections 2023-07-21T19:25:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e2fad0 0x55d212e2c500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e2f100 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 --2- 172.21.15.5:0/3077329532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e24be0 0x55d212e2bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.701+0000 7f13acd82b80 1 -- 172.21.15.5:0/3077329532 >> 172.21.15.5:0/3077329532 conn(0x55d212e1ada0 msgr2=0x55d212e27b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.702+0000 7f13acd82b80 1 -- 172.21.15.5:0/3077329532 shutdown_connections 2023-07-21T19:25:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.702+0000 7f13acd82b80 1 -- 172.21.15.5:0/3077329532 wait complete. 2023-07-21T19:25:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.702+0000 7f13acd82b80 1 -- start start 2023-07-21T19:25:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.702+0000 7f13acd82b80 1 Processor -- start 2023-07-21T19:25:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e24be0 0x55d212e34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e2fad0 0x55d212e349d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e36ae0 con 0x55d212e24be0 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e34f10 con 0x55d212e2fad0 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d212e35080 con 0x55d212e242d0 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13acd82b80 10 client.0 _ll_register_callbacks cb 0x7f1384009460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:55.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f138a7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e24be0 0x55d212e34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52578/0 (socket says 172.21.15.5:52578) 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 -- 172.21.15.5:0/2562100428 learned_addr learned my addr 172.21.15.5:0/2562100428 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f13937fe700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e2fad0 0x55d212e349d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 -- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e2fad0 msgr2=0x55d212e349d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e2fad0 0x55d212e349d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 -- 172.21.15.5:0/2562100428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e24be0 msgr2=0x55d212e34490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.703+0000 7f1392ffd700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e24be0 0x55d212e34490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f1392ffd700 1 -- 172.21.15.5:0/2562100428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f139410d170 con 0x55d212e242d0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f1392ffd700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13840062a0 tx=0x7f1384007830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=90ee0a5fb4203e4f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f137a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400e040 con 0x55d212e242d0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1384024040 con 0x55d212e242d0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f137c0a23c0 con 0x55d212e242d0 2023-07-21T19:25:55.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.704+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13840054c0 con 0x55d212e242d0 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 10 client.28657 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d212bcb250 con 0x55d212e242d0 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f137c0a23c0 con 0x55d212e242d0 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f1384039070 con 0x55d212e242d0 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 10 client.28657 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 10 client.28657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:25:55.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 20 client.28657 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:55.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e93e80 0x55d212e96290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:55.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 4 client.28657 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:25:55.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55d212d32050 con 0x55d212e93e80 2023-07-21T19:25:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.705+0000 7f138a7fc700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e93e80 0x55d212e96290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f138a7fc700 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e93e80 0x55d212e96290 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abbb67be8a4535e9 server_cookie=353b530b3ea59dba in_seq=0 out_seq=0 2023-07-21T19:25:55.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f137a7fc700 10 client.28657 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f137a7fc700 1 -- 172.21.15.5:0/2562100428 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55d212d32050 con 0x55d212e93e80 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f137a7fc700 10 client.28657 handle_command_reply: tid=0 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f13acd82b80 1 client.28657 shutdown 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 msgr2=0x55d212e361d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f13840062a0 tx=0x7f1384007830 comp rx=0 tx=0).stop 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 shutdown_connections 2023-07-21T19:25:55.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.706+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d212e2fad0 0x55d212e349d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d212e242d0 0x55d212e361d0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d212e93e80 0x55d212e96290 crc :-1 s=READY pgs=366 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 --2- 172.21.15.5:0/2562100428 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d212e24be0 0x55d212e34490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 >> 172.21.15.5:0/2562100428 conn(0x55d212e1ada0 msgr2=0x55d212e261a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f137a7fc700 0 client.28657 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 shutdown_connections 2023-07-21T19:25:55.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 1 -- 172.21.15.5:0/2562100428 wait complete. 2023-07-21T19:25:55.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.707+0000 7f13acd82b80 20 client.28657 trim_cache size 0 max 16384 2023-07-21T19:25:55.707 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:25:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:25:55.707 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:25:55.708 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:25:55.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 msgr2=0x7f13800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 0x7f13800a3770 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f138400b780 tx=0x7f1384006210 comp rx=0 tx=0).stop 2023-07-21T19:25:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 msgr2=0x7f13940fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 0x7f13940fee30 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f137c00e9f0 tx=0x7f137c00ed00 comp rx=0 tx=0).stop 2023-07-21T19:25:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 shutdown_connections 2023-07-21T19:25:55.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f139410a070 0x7f13941a3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1394100ca0 0x7f13940fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f13800a12c0 0x7f13800a3770 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 --2- 172.21.15.5:0/297738088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f13941015b0 0x7f13940fee30 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 >> 172.21.15.5:0/297738088 conn(0x7f13940fa4b0 msgr2=0x7f139410c270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:55.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 shutdown_connections 2023-07-21T19:25:55.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:55.708+0000 7f13993ff700 1 -- 172.21.15.5:0/297738088 wait complete. 2023-07-21T19:25:55.721 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:25:55.721 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:25:55.902 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:55 smithi005 ceph-mon[126460]: pgmap v576: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:55.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:55 smithi156 ceph-mon[128316]: pgmap v576: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:56.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:55 smithi101 ceph-mon[128136]: pgmap v576: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:56.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 -- 172.21.15.5:0/1026272812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a0100ca0 msgr2=0x7fd3a0101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/1026272812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a0100ca0 0x7fd3a0101070 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fd3900098f0 tx=0x7fd390009c00 comp rx=0 tx=0).stop 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 -- 172.21.15.5:0/1026272812 shutdown_connections 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/1026272812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 0x7fd3a010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/1026272812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a0100ca0 0x7fd3a0101070 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/1026272812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a01015b0 0x7fd3a0109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 -- 172.21.15.5:0/1026272812 >> 172.21.15.5:0/1026272812 conn(0x7fd3a00fa490 msgr2=0x7fd3a00fc8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 -- 172.21.15.5:0/1026272812 shutdown_connections 2023-07-21T19:25:56.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.093+0000 7fd3a63ff700 1 -- 172.21.15.5:0/1026272812 wait complete. 2023-07-21T19:25:56.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.095+0000 7fd3a63ff700 1 Processor -- start 2023-07-21T19:25:56.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.095+0000 7fd3a63ff700 1 -- start start 2023-07-21T19:25:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.095+0000 7fd3a63ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.095+0000 7fd3a63ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a01015b0 0x7fd3a019be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.095+0000 7fd3a63ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 0x7fd3a0195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd3a63ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0195fe0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd3a63ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0196150 con 0x7fd3a01015b0 2023-07-21T19:25:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd3a63ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a0196310 con 0x7fd3a010a070 2023-07-21T19:25:56.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49942/0 (socket says 172.21.15.5:49942) 2023-07-21T19:25:56.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 -- 172.21.15.5:0/2036659705 learned_addr learned my addr 172.21.15.5:0/2036659705 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd3a499c700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 0x7fd3a0195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39f7fe700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a01015b0 0x7fd3a019be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 -- 172.21.15.5:0/2036659705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 msgr2=0x7fd3a0195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 0x7fd3a0195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 -- 172.21.15.5:0/2036659705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a01015b0 msgr2=0x7fd3a019be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a01015b0 0x7fd3a019be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.096+0000 7fd39ffff700 1 -- 172.21.15.5:0/2036659705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3940095f0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd39ffff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd390009230 tx=0x7fd39000b560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39001c070 con 0x7fd3a0100ca0 2023-07-21T19:25:56.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd39000bbe0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd390009520 con 0x7fd3a0100ca0 2023-07-21T19:25:56.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3900038e0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.097+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a01a2710 con 0x7fd3a0100ca0 2023-07-21T19:25:56.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.099+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a0067300 con 0x7fd3a0100ca0 2023-07-21T19:25:56.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.101+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd390020030 con 0x7fd3a0100ca0 2023-07-21T19:25:56.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.102+0000 7fd39d7fa700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 0x7fd3880a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.102+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd3900ce010 con 0x7fd3a0100ca0 2023-07-21T19:25:56.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.103+0000 7fd39f7fe700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 0x7fd3880a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.103+0000 7fd39f7fe700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 0x7fd3880a3830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd3a010b650 tx=0x7fd394009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.105+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fd3900974b0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.421+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd3a0196e80 con 0x7fd3a0100ca0 2023-07-21T19:25:56.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.421+0000 7fd39d7fa700 1 -- 172.21.15.5:0/2036659705 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+10839 (secure 0 0 0) 0x7fd39009aad0 con 0x7fd3a0100ca0 2023-07-21T19:25:56.412 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:25:56.412 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:25:53.614690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 msgr2=0x7fd3880a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 0x7fd3880a3830 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fd3a010b650 tx=0x7fd394009310 comp rx=0 tx=0).stop 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 msgr2=0x7fd3a019b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7fd390009230 tx=0x7fd39000b560 comp rx=0 tx=0).stop 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 shutdown_connections 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd3a010a070 0x7fd3a0195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd3880a1380 0x7fd3880a3830 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd3a0100ca0 0x7fd3a019b900 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 --2- 172.21.15.5:0/2036659705 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd3a01015b0 0x7fd3a019be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.424+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 >> 172.21.15.5:0/2036659705 conn(0x7fd3a00fa490 msgr2=0x7fd3a010c2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.425+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 shutdown_connections 2023-07-21T19:25:56.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.425+0000 7fd3a63ff700 1 -- 172.21.15.5:0/2036659705 wait complete. 2023-07-21T19:25:56.417 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 56 2023-07-21T19:25:56.438 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:25:56.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 -- 172.21.15.5:0/830333339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 msgr2=0x7f8da0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 --2- 172.21.15.5:0/830333339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da0105eb0 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c009990 tx=0x7f8d9c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 -- 172.21.15.5:0/830333339 shutdown_connections 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 --2- 172.21.15.5:0/830333339 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 0x7f8da010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 --2- 172.21.15.5:0/830333339 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da0105eb0 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 --2- 172.21.15.5:0/830333339 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 -- 172.21.15.5:0/830333339 >> 172.21.15.5:0/830333339 conn(0x7f8da00fb5f0 msgr2=0x7f8da00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 -- 172.21.15.5:0/830333339 shutdown_connections 2023-07-21T19:25:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.732+0000 7f8da77ff700 1 -- 172.21.15.5:0/830333339 wait complete. 2023-07-21T19:25:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.734+0000 7f8da77ff700 1 Processor -- start 2023-07-21T19:25:56.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.734+0000 7f8da77ff700 1 -- start start 2023-07-21T19:25:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 0x7f8da0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da019bbb0 con 0x7f8da0105ae0 2023-07-21T19:25:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da019bd20 con 0x7f8da00ffad0 2023-07-21T19:25:56.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da77ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da019bee0 con 0x7f8da0100480 2023-07-21T19:25:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da5d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2023-07-21T19:25:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da559b700 1 -- 172.21.15.5:0/1803770858 learned_addr learned my addr 172.21.15.5:0/1803770858 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.735+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 0x7f8da0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 -- 172.21.15.5:0/1803770858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 msgr2=0x7f8da0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 0x7f8da0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 -- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 msgr2=0x7f8da019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 -- 172.21.15.5:0/1803770858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d9c0095f0 con 0x7f8da00ffad0 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da5d9c700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:56.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da559b700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0036c0 tx=0x7f8d9c005740 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8d967fc700 1 -- 172.21.15.5:0/1803770858 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c01d070 con 0x7f8da00ffad0 2023-07-21T19:25:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8d967fc700 1 -- 172.21.15.5:0/1803770858 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d9c003ec0 con 0x7f8da00ffad0 2023-07-21T19:25:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da019c160 con 0x7f8da00ffad0 2023-07-21T19:25:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.736+0000 7f8d967fc700 1 -- 172.21.15.5:0/1803770858 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c022b00 con 0x7f8da00ffad0 2023-07-21T19:25:56.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.737+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da01a4b20 con 0x7f8da00ffad0 2023-07-21T19:25:56.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.741+0000 7f8d967fc700 1 -- 172.21.15.5:0/1803770858 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8d9c004030 con 0x7f8da00ffad0 2023-07-21T19:25:56.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.743+0000 7f8d967fc700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 0x7f8d8c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.743+0000 7f8d967fc700 1 -- 172.21.15.5:0/1803770858 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d9c0ce210 con 0x7f8da00ffad0 2023-07-21T19:25:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.743+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 0x7f8d8c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.744+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 0x7f8d8c0a37f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8da0199290 tx=0x7f8d9000b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.742 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 Processor -- start 2023-07-21T19:25:56.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 -- start start 2023-07-21T19:25:56.743 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb12d0 0x55dd32eb16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32ebcad0 0x55dd32ebeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32db7cb0 con 0x55dd32eb1be0 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32e0aa40 con 0x55dd32eb12d0 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.751+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32e03ed0 con 0x55dd32ebcad0 2023-07-21T19:25:56.744 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36488/0 (socket says 172.21.15.5:36488) 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 -- 172.21.15.5:0/1550160713 learned_addr learned my addr 172.21.15.5:0/1550160713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 -- 172.21.15.5:0/1550160713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32ebcad0 msgr2=0x55dd32ebeeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32ebcad0 0x55dd32ebeeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 -- 172.21.15.5:0/1550160713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb12d0 msgr2=0x55dd32eb16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb12d0 0x55dd32eb16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.745 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.752+0000 7f8da4d9a700 1 -- 172.21.15.5:0/1550160713 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd32c31de0 con 0x55dd32eb1be0 2023-07-21T19:25:56.746 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.753+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8d90010f40 tx=0x7f8d90013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2458a15f5c32eb0a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.753+0000 7f8d8b7fe700 1 -- 172.21.15.5:0/1550160713 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9000b950 con 0x55dd32eb1be0 2023-07-21T19:25:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.753+0000 7f8d8b7fe700 1 -- 172.21.15.5:0/1550160713 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d90015070 con 0x55dd32eb1be0 2023-07-21T19:25:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.753+0000 7f8d8b7fe700 1 -- 172.21.15.5:0/1550160713 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9001c8d0 con 0x55dd32eb1be0 2023-07-21T19:25:56.747 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.754+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1550160713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 msgr2=0x55dd32ebc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.754+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f8d90010f40 tx=0x7f8d90013000 comp rx=0 tx=0).stop 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1550160713 shutdown_connections 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32ebcad0 0x55dd32ebeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb12d0 0x55dd32eb16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1550160713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32ebc590 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1550160713 >> 172.21.15.5:0/1550160713 conn(0x55dd32ea7da0 msgr2=0x55dd32ead6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.748 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1550160713 shutdown_connections 2023-07-21T19:25:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.755+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1550160713 wait complete. 2023-07-21T19:25:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.756+0000 7f8dbb086b80 1 -- start start 2023-07-21T19:25:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.756+0000 7f8dbb086b80 1 Processor -- start 2023-07-21T19:25:56.749 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.750 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32eba2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32da3ac0 con 0x55dd32eb1be0 2023-07-21T19:25:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32da37c0 con 0x55dd32ebcad0 2023-07-21T19:25:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32c87920 con 0x55dd32eb12d0 2023-07-21T19:25:56.751 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8dbb086b80 10 client.0 _ll_register_callbacks cb 0x55dd32da1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8da559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8da5d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32eba2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8da559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35006/0 (socket says 172.21.15.5:35006) 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.757+0000 7f8da559b700 1 -- 172.21.15.5:0/1468695052 learned_addr learned my addr 172.21.15.5:0/1468695052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 -- 172.21.15.5:0/1468695052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 msgr2=0x55dd32eba2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.752 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32eba2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 -- 172.21.15.5:0/1468695052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 msgr2=0x55dd32eb9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 -- 172.21.15.5:0/1468695052 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d90010c20 con 0x55dd32eb12d0 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8da559b700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c097f00 tx=0x7f8d9c09adf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=899b0f1f9f91d64c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a3850 con 0x55dd32eb12d0 2023-07-21T19:25:56.753 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d9c09b6f0 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.758+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55dd32d5d4f0 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d9c09b8b0 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 10 client.28669 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55dd32c1ac60 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55dd32d5d4f0 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8d9c0a10e0 con 0x55dd32eb12d0 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 10 client.28669 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:56.754 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 10 client.28669 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:25:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 20 client.28669 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32ecc070 0x55dd32ece480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.759+0000 7f8dbb086b80 4 client.28669 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.760+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55dd32dfceb0 con 0x55dd32ecc070 2023-07-21T19:25:56.755 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.760+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32ecc070 0x55dd32ece480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.761+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32ecc070 0x55dd32ece480 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=71ac68441533718 server_cookie=d6acaca3ea454019 in_seq=0 out_seq=0 2023-07-21T19:25:56.756 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.761+0000 7f8d88ff9700 10 client.28669 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8d88ff9700 1 -- 172.21.15.5:0/1468695052 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55dd32dfceb0 con 0x55dd32ecc070 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8d88ff9700 10 client.28669 handle_command_reply: tid=0 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 client.28669 shutdown 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 msgr2=0x55dd32eb9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c097f00 tx=0x7f8d9c09adf0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.757 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 shutdown_connections 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32ecc070 0x55dd32ece480 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32eba2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32eb9870 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/1468695052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 >> 172.21.15.5:0/1468695052 conn(0x55dd32ea7da0 msgr2=0x55dd32ead230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8d88ff9700 0 client.28669 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:56.758 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 shutdown_connections 2023-07-21T19:25:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.762+0000 7f8dbb086b80 1 -- 172.21.15.5:0/1468695052 wait complete. 2023-07-21T19:25:56.759 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.763+0000 7f8dbb086b80 20 client.28669 trim_cache size 0 max 16384 2023-07-21T19:25:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 Processor -- start 2023-07-21T19:25:56.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 -- start start 2023-07-21T19:25:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 0x55dd32eba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32eb2540 con 0x55dd32ebcad0 2023-07-21T19:25:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32ebae50 con 0x55dd32eb1be0 2023-07-21T19:25:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.778+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32ec8f00 con 0x55dd32eb12d0 2023-07-21T19:25:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 0x55dd32eba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35018/0 (socket says 172.21.15.5:35018) 2023-07-21T19:25:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da559b700 1 -- 172.21.15.5:0/3663381875 learned_addr learned my addr 172.21.15.5:0/3663381875 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 -- 172.21.15.5:0/3663381875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 msgr2=0x55dd32ec34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 -- 172.21.15.5:0/3663381875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb1be0 msgr2=0x55dd32eb9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 -- 172.21.15.5:0/3663381875 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55dd32c58250 con 0x55dd32ebcad0 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da559b700 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.779+0000 7f8da5d9c700 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 0x55dd32eba320 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003f10 tx=0x7f8d98004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0330ced0e927eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.780+0000 7f8d8bfff700 1 -- 172.21.15.5:0/3663381875 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980048c0 con 0x55dd32ebcad0 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.780+0000 7f8d8bfff700 1 -- 172.21.15.5:0/3663381875 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d98015ae0 con 0x55dd32ebcad0 2023-07-21T19:25:56.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.780+0000 7f8d8bfff700 1 -- 172.21.15.5:0/3663381875 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980149d0 con 0x55dd32ebcad0 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- 172.21.15.5:0/3663381875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 msgr2=0x55dd32eba320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 0x55dd32eba320 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f8d98003f10 tx=0x7f8d98004320 comp rx=0 tx=0).stop 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- 172.21.15.5:0/3663381875 shutdown_connections 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32ebcad0 0x55dd32eba320 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32ec34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/3663381875 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32eb1be0 0x55dd32eb9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- 172.21.15.5:0/3663381875 >> 172.21.15.5:0/3663381875 conn(0x55dd32ea7da0 msgr2=0x55dd32eb4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- 172.21.15.5:0/3663381875 shutdown_connections 2023-07-21T19:25:56.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- 172.21.15.5:0/3663381875 wait complete. 2023-07-21T19:25:56.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.781+0000 7f8dbb086b80 1 -- start start 2023-07-21T19:25:56.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 Processor -- start 2023-07-21T19:25:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32f140b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32f1ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32f1f480 con 0x55dd32eb1be0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32f1f5c0 con 0x55dd32ebcad0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dd32f1f780 con 0x55dd32eb12d0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.782+0000 7f8dbb086b80 10 client.0 _ll_register_callbacks cb 0x7f8d9c09b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da4d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36522/0 (socket says 172.21.15.5:36522) 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32f140b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da4d9a700 1 -- 172.21.15.5:0/4065476003 learned_addr learned my addr 172.21.15.5:0/4065476003 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da5d9c700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32f1ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 -- 172.21.15.5:0/4065476003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 msgr2=0x55dd32f1ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32f1ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 -- 172.21.15.5:0/4065476003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 msgr2=0x55dd32f145f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 -- 172.21.15.5:0/4065476003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dd32c1ac60 con 0x55dd32eb12d0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8da559b700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32f140b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cbf60 tx=0x7f8d9c009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39d56b446d95cdec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.783+0000 7f8d88ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c0a33c0 con 0x55dd32eb12d0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d9c0a3850 con 0x55dd32eb12d0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d90015070 con 0x55dd32eb12d0 2023-07-21T19:25:56.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d9c09f170 con 0x55dd32eb12d0 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8dbb086b80 10 client.28675 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f8d98003bf0 con 0x55dd32eb12d0 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d90015070 con 0x55dd32eb12d0 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.784+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8d9c09b760 con 0x55dd32eb12d0 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 10 client.28675 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 10 client.28675 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 20 client.28675 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:56.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32f20b70 0x55dd32f22f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 4 client.28675 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55dd32dbf050 con 0x55dd32f20b70 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.785+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32f20b70 0x55dd32f22f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8da4d9a700 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32f20b70 0x55dd32f22f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=4c610bbabd0f285a server_cookie=55d49a691a79bbfd in_seq=0 out_seq=0 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8d88ff9700 10 client.28675 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8d88ff9700 1 -- 172.21.15.5:0/4065476003 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55dd32dbf050 con 0x55dd32f20b70 2023-07-21T19:25:56.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8d88ff9700 10 client.28675 handle_command_reply: tid=0 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 client.28675 shutdown 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 msgr2=0x55dd32f140b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32f140b0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0cbf60 tx=0x7f8d9c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 shutdown_connections 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55dd32f20b70 0x55dd32f22f80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55dd32ebcad0 0x55dd32f1ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55dd32eb12d0 0x55dd32f140b0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8d88ff9700 0 client.28675 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 --2- 172.21.15.5:0/4065476003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55dd32eb1be0 0x55dd32f145f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.786+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 >> 172.21.15.5:0/4065476003 conn(0x55dd32ea7da0 msgr2=0x55dd32eb9de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 shutdown_connections 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8dbb086b80 1 -- 172.21.15.5:0/4065476003 wait complete. 2023-07-21T19:25:56.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8dbb086b80 20 client.28675 trim_cache size 0 max 16384 2023-07-21T19:25:56.786 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:25:56.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 msgr2=0x7f8d8c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 0x7f8d8c0a37f0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8da0199290 tx=0x7f8d9000b410 comp rx=0 tx=0).stop 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 msgr2=0x7f8da0197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.787+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c0036c0 tx=0x7f8d9c005740 comp rx=0 tx=0).stop 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 shutdown_connections 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d8c0a1340 0x7f8d8c0a37f0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8da0105ae0 0x7f8da019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8da00ffad0 0x7f8da0197cc0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 --2- 172.21.15.5:0/1803770858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8da0100480 0x7f8da0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 >> 172.21.15.5:0/1803770858 conn(0x7f8da00fb5f0 msgr2=0x7f8da010ae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 shutdown_connections 2023-07-21T19:25:56.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:56.788+0000 7f8da77ff700 1 -- 172.21.15.5:0/1803770858 wait complete. 2023-07-21T19:25:56.795 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:56 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2036659705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:56.798 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:25:56.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:56 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2036659705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:57.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.040+0000 7fe2575ff700 1 -- 172.21.15.5:0/1361015922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25010a060 msgr2=0x7fe25010c4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.040+0000 7fe2575ff700 1 --2- 172.21.15.5:0/1361015922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25010a060 0x7fe25010c4c0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fe24800b0a0 tx=0x7fe24800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 -- 172.21.15.5:0/1361015922 shutdown_connections 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 --2- 172.21.15.5:0/1361015922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe25010a060 0x7fe25010c4c0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 --2- 172.21.15.5:0/1361015922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe250100d20 0x7fe2501010f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 --2- 172.21.15.5:0/1361015922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250101630 0x7fe250109b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 -- 172.21.15.5:0/1361015922 >> 172.21.15.5:0/1361015922 conn(0x7fe2500fa4b0 msgr2=0x7fe2500fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 -- 172.21.15.5:0/1361015922 shutdown_connections 2023-07-21T19:25:57.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.041+0000 7fe2575ff700 1 -- 172.21.15.5:0/1361015922 wait complete. 2023-07-21T19:25:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.042+0000 7fe2575ff700 1 Processor -- start 2023-07-21T19:25:57.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.042+0000 7fe2575ff700 1 -- start start 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.042+0000 7fe2575ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250100d20 0x7fe250197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe2575ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe250101630 0x7fe2501982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe2575ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe2575ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25019cd90 con 0x7fe250101630 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe2575ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25019cf00 con 0x7fe250100d20 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe2575ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe25019d0c0 con 0x7fe25010a060 2023-07-21T19:25:57.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 -- 172.21.15.5:0/4174110507 learned_addr learned my addr 172.21.15.5:0/4174110507 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe250101630 0x7fe2501982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe25539b700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250100d20 0x7fe250197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 -- 172.21.15.5:0/4174110507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250100d20 msgr2=0x7fe250197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250100d20 0x7fe250197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 -- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe250101630 msgr2=0x7fe2501982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe250101630 0x7fe2501982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.043+0000 7fe255b9c700 1 -- 172.21.15.5:0/4174110507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe248009d00 con 0x7fe25010a060 2023-07-21T19:25:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.044+0000 7fe255b9c700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe24800b070 tx=0x7fe24800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.044+0000 7fe2467fc700 1 -- 172.21.15.5:0/4174110507 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24800e040 con 0x7fe25010a060 2023-07-21T19:25:57.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.044+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe25019d340 con 0x7fe25010a060 2023-07-21T19:25:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.044+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2501a4be0 con 0x7fe25010a060 2023-07-21T19:25:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.045+0000 7fe2467fc700 1 -- 172.21.15.5:0/4174110507 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe248009b80 con 0x7fe25010a060 2023-07-21T19:25:57.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.045+0000 7fe2467fc700 1 -- 172.21.15.5:0/4174110507 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24801dde0 con 0x7fe25010a060 2023-07-21T19:25:57.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.049+0000 7fe2467fc700 1 -- 172.21.15.5:0/4174110507 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe24801d480 con 0x7fe25010a060 2023-07-21T19:25:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.050+0000 7fe26ae9ab80 1 Processor -- start 2023-07-21T19:25:57.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.050+0000 7fe26ae9ab80 1 -- start start 2023-07-21T19:25:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.050+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.050+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341b0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340a9cb0 con 0x562e341aead0 2023-07-21T19:25:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340fca40 con 0x562e341a3be0 2023-07-21T19:25:57.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340f5ed0 con 0x562e341a32d0 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe25539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35060/0 (socket says 172.21.15.5:35060) 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe254b9a700 1 -- 172.21.15.5:0/471951409 learned_addr learned my addr 172.21.15.5:0/471951409 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe25539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2023-07-21T19:25:57.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe2467fc700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 0x7fe23c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe2467fc700 1 -- 172.21.15.5:0/4174110507 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe2480cdec0 con 0x7fe25010a060 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.051+0000 7fe254b9a700 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341b0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 -- 172.21.15.5:0/471951409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 msgr2=0x562e341ae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 -- 172.21.15.5:0/471951409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 msgr2=0x562e341b0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341b0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 -- 172.21.15.5:0/471951409 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e33f23de0 con 0x562e341a32d0 2023-07-21T19:25:57.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe25539b700 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe255b9c700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 0x7fe23c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe254b9a700 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2400115e0 tx=0x7fe24000a800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=545cf313f342dcea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe23b7fe700 1 -- 172.21.15.5:0/471951409 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240004220 con 0x562e341a32d0 2023-07-21T19:25:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe23b7fe700 1 -- 172.21.15.5:0/471951409 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe240004380 con 0x562e341a32d0 2023-07-21T19:25:57.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.052+0000 7fe23b7fe700 1 -- 172.21.15.5:0/471951409 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000f930 con 0x562e341a32d0 2023-07-21T19:25:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/471951409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 msgr2=0x562e341a36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe255b9c700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 0x7fe23c0a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe248097a30 tx=0x7fe248009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fe2400115e0 tx=0x7fe24000a800 comp rx=0 tx=0).stop 2023-07-21T19:25:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/471951409 shutdown_connections 2023-07-21T19:25:57.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341b0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341a36a0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/471951409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a3be0 0x562e341ae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/471951409 >> 172.21.15.5:0/471951409 conn(0x562e34199da0 msgr2=0x562e3419f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/471951409 shutdown_connections 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.053+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/471951409 wait complete. 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.054+0000 7fe26ae9ab80 1 -- start start 2023-07-21T19:25:57.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.054+0000 7fe26ae9ab80 1 Processor -- start 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e341ab720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34095ac0 con 0x562e341a3be0 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e340957c0 con 0x562e341aead0 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e33f79920 con 0x562e341a32d0 2023-07-21T19:25:57.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.055+0000 7fe26ae9ab80 10 client.0 _ll_register_callbacks cb 0x562e34093760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35706/0 (socket says 172.21.15.5:35706) 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 -- 172.21.15.5:0/1772394973 learned_addr learned my addr 172.21.15.5:0/1772394973 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 -- 172.21.15.5:0/1772394973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 msgr2=0x562e341ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 -- 172.21.15.5:0/1772394973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 msgr2=0x562e341ab720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e341ab720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.056+0000 7fe254b9a700 1 -- 172.21.15.5:0/1772394973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c0098e0 con 0x562e341aead0 2023-07-21T19:25:57.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe254b9a700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe240003fd0 tx=0x7fe240003eb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2563b0acd26452 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe238ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:25:57.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe24000f860 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562e3404f4f0 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe240003a00 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe24000a4b0 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.057+0000 7fe26ae9ab80 10 client.28763 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7fe24000ad00 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562e3404f4f0 con 0x562e341aead0 2023-07-21T19:25:57.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe24001b070 con 0x562e341aead0 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 10 client.28763 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 10 client.28763 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 20 client.28763 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e341be3b0 0x562e341c07c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 4 client.28763 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.058+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562e340eeeb0 con 0x562e341be3b0 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.059+0000 7fe25539b700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e341be3b0 0x562e341c07c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.059+0000 7fe25539b700 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e341be3b0 0x562e341c07c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=32f7aa4577b9802b server_cookie=e25b32b49952bfff in_seq=0 out_seq=0 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.059+0000 7fe238ff9700 10 client.28763 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe238ff9700 1 -- 172.21.15.5:0/1772394973 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562e340eeeb0 con 0x562e341be3b0 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe238ff9700 10 client.28763 handle_command_reply: tid=0 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 client.28763 shutdown 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 msgr2=0x562e341abda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe240003fd0 tx=0x7fe240003eb0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 shutdown_connections 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e341be3b0 0x562e341c07c0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341aead0 0x562e341abda0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a32d0 0x562e341ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe238ff9700 0 client.28763 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/1772394973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e341ab720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.060+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 >> 172.21.15.5:0/1772394973 conn(0x562e34199da0 msgr2=0x562e3419f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.061+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 shutdown_connections 2023-07-21T19:25:57.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.061+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/1772394973 wait complete. 2023-07-21T19:25:57.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.061+0000 7fe26ae9ab80 20 client.28763 trim_cache size 0 max 16384 2023-07-21T19:25:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.077+0000 7fe26ae9ab80 1 Processor -- start 2023-07-21T19:25:57.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 -- start start 2023-07-21T19:25:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e341b5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 0x562e341ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e341ac130 con 0x562e341aead0 2023-07-21T19:25:57.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e341ac3a0 con 0x562e341a32d0 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.078+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e341acb10 con 0x562e341a3be0 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36566/0 (socket says 172.21.15.5:36566) 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 -- 172.21.15.5:0/2355553962 learned_addr learned my addr 172.21.15.5:0/2355553962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe25539b700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 0x562e341ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e341b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 -- 172.21.15.5:0/2355553962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 msgr2=0x562e341ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 0x562e341ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 -- 172.21.15.5:0/2355553962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 msgr2=0x562e341b5580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e341b5580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 -- 172.21.15.5:0/2355553962 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562e33f0cc60 con 0x562e341aead0 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe25539b700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 0x562e341ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.079+0000 7fe254b9a700 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe240038910 tx=0x7fe24000af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e32fc5311dd7455 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.080+0000 7fe23bfff700 1 -- 172.21.15.5:0/2355553962 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240038c00 con 0x562e341aead0 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.080+0000 7fe23bfff700 1 -- 172.21.15.5:0/2355553962 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe24000a380 con 0x562e341aead0 2023-07-21T19:25:57.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.080+0000 7fe23bfff700 1 -- 172.21.15.5:0/2355553962 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240005280 con 0x562e341aead0 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/2355553962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 msgr2=0x562e341a89b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fe240038910 tx=0x7fe24000af20 comp rx=0 tx=0).stop 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/2355553962 shutdown_connections 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341aead0 0x562e341a89b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e341b5580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/2355553962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341a3be0 0x562e341ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/2355553962 >> 172.21.15.5:0/2355553962 conn(0x562e34199da0 msgr2=0x562e341ba810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/2355553962 shutdown_connections 2023-07-21T19:25:57.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/2355553962 wait complete. 2023-07-21T19:25:57.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:56 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2036659705' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:25:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.081+0000 7fe26ae9ab80 1 -- start start 2023-07-21T19:25:57.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 Processor -- start 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e3420b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e3420bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 0x562e34206200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e342067d0 con 0x562e341a3be0 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34206940 con 0x562e341a32d0 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562e34206b00 con 0x562e341aead0 2023-07-21T19:25:57.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.082+0000 7fe26ae9ab80 10 client.0 _ll_register_callbacks cb 0x7fe240005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe25539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e3420bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe25539b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e3420bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36572/0 (socket says 172.21.15.5:36572) 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe25539b700 1 -- 172.21.15.5:0/969134698 learned_addr learned my addr 172.21.15.5:0/969134698 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 0x562e34206200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e3420b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 -- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 msgr2=0x562e3420b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e3420b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 -- 172.21.15.5:0/969134698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 msgr2=0x562e3420bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e3420bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.083+0000 7fe254b9a700 1 -- 172.21.15.5:0/969134698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24c01c950 con 0x562e341aead0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe254b9a700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 0x562e34206200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe240005fa0 tx=0x7fe240014040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bc0754081a65eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe238ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe240037dc0 con 0x562e341aead0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe240000e40 con 0x562e341aead0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe240004380 con 0x562e341aead0 2023-07-21T19:25:57.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe240038c00 con 0x562e341aead0 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe26ae9ab80 10 client.28693 fetch_fsmap learned FSMap version 56 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7fe24000ad00 con 0x562e341aead0 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.084+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe24004e020 con 0x562e341aead0 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe240049070 con 0x562e341aead0 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 10 client.28693 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 10 client.28693 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:25:57.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 20 client.28693 populate_metadata read hostname 'smithi005' 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e34212f70 0x562e34215350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 4 client.28693 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x562e340b1050 con 0x562e34212f70 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.085+0000 7fe25539b700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e34212f70 0x562e34215350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe238ff9700 10 client.28693 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe25539b700 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e34212f70 0x562e34215350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c19f3d53f46ccbbb server_cookie=d3622dbcd7cc58f3 in_seq=0 out_seq=0 2023-07-21T19:25:57.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe238ff9700 1 -- 172.21.15.5:0/969134698 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x562e340b1050 con 0x562e34212f70 2023-07-21T19:25:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe238ff9700 10 client.28693 handle_command_reply: tid=0 2023-07-21T19:25:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe26ae9ab80 1 client.28693 shutdown 2023-07-21T19:25:57.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 msgr2=0x562e34206200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 0x562e34206200 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fe240005fa0 tx=0x7fe240014040 comp rx=0 tx=0).stop 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.086+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 shutdown_connections 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562e341aead0 0x562e34206200 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x562e34212f70 0x562e34215350 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562e341a32d0 0x562e3420b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe238ff9700 0 client.28693 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 --2- 172.21.15.5:0/969134698 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562e341a3be0 0x562e3420bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 >> 172.21.15.5:0/969134698 conn(0x562e34199da0 msgr2=0x562e341a53e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 shutdown_connections 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 1 -- 172.21.15.5:0/969134698 wait complete. 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe26ae9ab80 20 client.28693 trim_cache size 0 max 16384 2023-07-21T19:25:57.084 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:25:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 msgr2=0x7fe23c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.087+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 0x7fe23c0a3770 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe248097a30 tx=0x7fe248009040 comp rx=0 tx=0).stop 2023-07-21T19:25:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 msgr2=0x7fe25019c750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:25:57.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fe24800b070 tx=0x7fe24800bce0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 shutdown_connections 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe25010a060 0x7fe25019c750 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe250100d20 0x7fe250197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe23c0a12c0 0x7fe23c0a3770 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 --2- 172.21.15.5:0/4174110507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe250101630 0x7fe2501982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 >> 172.21.15.5:0/4174110507 conn(0x7fe2500fa4b0 msgr2=0x7fe2500fa880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:25:57.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 shutdown_connections 2023-07-21T19:25:57.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:25:57.088+0000 7fe2575ff700 1 -- 172.21.15.5:0/4174110507 wait complete. 2023-07-21T19:25:57.097 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:25:57.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:57 smithi005 ceph-mon[126460]: pgmap v577: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:57.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:57 smithi156 ceph-mon[128316]: pgmap v577: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:58.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:57 smithi101 ceph-mon[128136]: pgmap v577: 97 pgs: 97 active+clean; 16 GiB data, 8.0 GiB used, 1.0 TiB / 1.0 TiB avail; 119 KiB/s rd, 29 MiB/s wr, 45 op/s 2023-07-21T19:25:59.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:25:58 smithi101 ceph-mon[128136]: pgmap v578: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 42 MiB/s wr, 90 op/s 2023-07-21T19:25:59.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:25:58 smithi156 ceph-mon[128316]: pgmap v578: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 42 MiB/s wr, 90 op/s 2023-07-21T19:26:00.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:25:58 smithi005 ceph-mon[126460]: pgmap v578: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 376 KiB/s rd, 42 MiB/s wr, 90 op/s 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 -- 172.21.15.5:0/1265354221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8102910 msgr2=0x7f2fd81068d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/1265354221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8102910 0x7f2fd81068d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0099c0 tx=0x7f2fcc009cd0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 -- 172.21.15.5:0/1265354221 shutdown_connections 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/1265354221 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 0x7f2fd810eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/1265354221 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8101f70 0x7f2fd8102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/1265354221 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8102910 0x7f2fd81068d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 -- 172.21.15.5:0/1265354221 >> 172.21.15.5:0/1265354221 conn(0x7f2fd80fd7e0 msgr2=0x7f2fd80ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.860+0000 7f2fde9ff700 1 -- 172.21.15.5:0/1265354221 shutdown_connections 2023-07-21T19:26:00.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.861+0000 7f2fde9ff700 1 -- 172.21.15.5:0/1265354221 wait complete. 2023-07-21T19:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 Processor -- start 2023-07-21T19:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 -- start start 2023-07-21T19:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 0x7f2fd81982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 0x7f2fd819c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819cd80 con 0x7f2fd8101f70 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819cef0 con 0x7f2fd8106fb0 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.862+0000 7f2fde9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fd819d0b0 con 0x7f2fd8102910 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 0x7f2fd81982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36584/0 (socket says 172.21.15.5:36584) 2023-07-21T19:26:00.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd7fff700 1 -- 172.21.15.5:0/3350746714 learned_addr learned my addr 172.21.15.5:0/3350746714 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 0x7f2fd819c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3350746714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 msgr2=0x7f2fd819c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 0x7f2fd819c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 msgr2=0x7f2fd8197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3350746714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc0095f0 con 0x7f2fd8102910 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.863+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:00.856 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.864+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 0x7f2fd81982e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0098e0 tx=0x7f2fcc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.864+0000 7f2fd57fa700 1 -- 172.21.15.5:0/3350746714 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc01d070 con 0x7f2fd8102910 2023-07-21T19:26:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.864+0000 7f2fd57fa700 1 -- 172.21.15.5:0/3350746714 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc005050 con 0x7f2fd8102910 2023-07-21T19:26:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.864+0000 7f2fd57fa700 1 -- 172.21.15.5:0/3350746714 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc004950 con 0x7f2fd8102910 2023-07-21T19:26:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.864+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fd819d330 con 0x7f2fd8102910 2023-07-21T19:26:00.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.865+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd819d580 con 0x7f2fd8102910 2023-07-21T19:26:00.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 -- 172.21.15.5:0/3563874708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768067ee0 msgr2=0x7fe768068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3563874708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768067ee0 0x7fe768068350 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098f0 tx=0x7fe758009c00 comp rx=0 tx=0).stop 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 -- 172.21.15.5:0/3563874708 shutdown_connections 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3563874708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 0x7fe768102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3563874708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768104a60 0x7fe7680679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3563874708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768067ee0 0x7fe768068350 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 -- 172.21.15.5:0/3563874708 >> 172.21.15.5:0/3563874708 conn(0x7fe768074760 msgr2=0x7fe768074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 -- 172.21.15.5:0/3563874708 shutdown_connections 2023-07-21T19:26:00.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.867+0000 7fe76fdff700 1 -- 172.21.15.5:0/3563874708 wait complete. 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.868+0000 7fe76fdff700 1 Processor -- start 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 -- start start 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 0x7fe76819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 0x7fe76819c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 0x7fe7681a0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681a1530 con 0x7fe768104a60 2023-07-21T19:26:00.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681a16a0 con 0x7fe768067ee0 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76fdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7681a1860 con 0x7fe7681024f0 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.869+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 0x7fe76819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 0x7fe76819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35734/0 (socket says 172.21.15.5:35734) 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76db9b700 1 -- 172.21.15.5:0/3627983407 learned_addr learned my addr 172.21.15.5:0/3627983407 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 0x7fe7681a0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76d39a700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 0x7fe76819c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.869+0000 7f2fd57fa700 1 -- 172.21.15.5:0/3350746714 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2fcc0051c0 con 0x7f2fd8102910 2023-07-21T19:26:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 -- 172.21.15.5:0/3627983407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 msgr2=0x7fe76819c880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 0x7fe76819c880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 -- 172.21.15.5:0/3627983407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 msgr2=0x7fe76819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 0x7fe76819c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 -- 172.21.15.5:0/3627983407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe760009d00 con 0x7fe768104a60 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.870+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 0x7fe7681a0ef0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe760009cd0 tx=0x7fe76000c620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.871+0000 7fe75effd700 1 -- 172.21.15.5:0/3627983407 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe760017070 con 0x7fe768104a60 2023-07-21T19:26:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.871+0000 7fe75effd700 1 -- 172.21.15.5:0/3627983407 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76000f040 con 0x7fe768104a60 2023-07-21T19:26:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.871+0000 7fe75effd700 1 -- 172.21.15.5:0/3627983407 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe76000e050 con 0x7fe768104a60 2023-07-21T19:26:00.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.871+0000 7f2ff228fb80 1 Processor -- start 2023-07-21T19:26:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.871+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe758009520 con 0x7fe768104a60 2023-07-21T19:26:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.871+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7681a91a0 con 0x7fe768104a60 2023-07-21T19:26:00.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.871+0000 7f2ff228fb80 1 -- start start 2023-07-21T19:26:00.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da282d0 0x56134da286a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da33590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.870 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d92ecb0 con 0x56134da28be0 2023-07-21T19:26:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d981a40 con 0x56134da33ad0 2023-07-21T19:26:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d97aed0 con 0x56134da282d0 2023-07-21T19:26:00.871 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2fd57fa700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 0x7f2fc00a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.872+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35744/0 (socket says 172.21.15.5:35744) 2023-07-21T19:26:00.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3447010549 learned_addr learned my addr 172.21.15.5:0/3447010549 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da282d0 0x56134da286a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd57fa700 1 -- 172.21.15.5:0/3350746714 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2fcc0cd810 con 0x7f2fd8102910 2023-07-21T19:26:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 0x7f2fc00a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da33590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3447010549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da282d0 msgr2=0x56134da286a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da282d0 0x56134da286a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3447010549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 msgr2=0x56134da33590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da33590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.873+0000 7f2fd77fe700 1 -- 172.21.15.5:0/3447010549 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134d7a8de0 con 0x56134da33ad0 2023-07-21T19:26:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.874+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0a2800 tx=0x7f2fcc096db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=caa1258208c9bdea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.874+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 0x7f2fc00a38c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009ec0 tx=0x7f2fd000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.874+0000 7f2fbe7fc700 1 -- 172.21.15.5:0/3447010549 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09a970 con 0x56134da33ad0 2023-07-21T19:26:00.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.874+0000 7f2fbe7fc700 1 -- 172.21.15.5:0/3447010549 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc09e9a0 con 0x56134da33ad0 2023-07-21T19:26:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.874+0000 7f2fbe7fc700 1 -- 172.21.15.5:0/3447010549 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09abd0 con 0x56134da33ad0 2023-07-21T19:26:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 -- 172.21.15.5:0/3447010549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 msgr2=0x56134da35eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0a2800 tx=0x7f2fcc096db0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 -- 172.21.15.5:0/3447010549 shutdown_connections 2023-07-21T19:26:00.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da33ad0 0x56134da35eb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da282d0 0x56134da286a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/3447010549 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da33590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 -- 172.21.15.5:0/3447010549 >> 172.21.15.5:0/3447010549 conn(0x56134da1eda0 msgr2=0x56134da246a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 -- 172.21.15.5:0/3447010549 shutdown_connections 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.875+0000 7f2ff228fb80 1 -- 172.21.15.5:0/3447010549 wait complete. 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.876+0000 7f2ff228fb80 1 -- start start 2023-07-21T19:26:00.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.876+0000 7f2ff228fb80 1 Processor -- start 2023-07-21T19:26:00.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.876+0000 7fe75effd700 1 -- 172.21.15.5:0/3627983407 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe76000cd10 con 0x7fe768104a60 2023-07-21T19:26:00.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da38ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 0x56134da39020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 0x56134da32a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d91aac0 con 0x56134da33ad0 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d91a7c0 con 0x56134da282d0 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134d7fe920 con 0x56134da28be0 2023-07-21T19:26:00.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.877+0000 7f2ff228fb80 10 client.0 _ll_register_callbacks cb 0x56134d918760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:00.880 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.877+0000 7fe78364bb80 1 Processor -- start 2023-07-21T19:26:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da38ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 0x56134da39020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da38ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35750/0 (socket says 172.21.15.5:35750) 2023-07-21T19:26:00.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd7fff700 1 -- 172.21.15.5:0/2046963028 learned_addr learned my addr 172.21.15.5:0/2046963028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 0x56134da32a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.878+0000 7fe78364bb80 1 -- start start 2023-07-21T19:26:00.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 -- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 msgr2=0x56134da38ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da38ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 -- 172.21.15.5:0/2046963028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 msgr2=0x56134da32a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 0x56134da32a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 -- 172.21.15.5:0/2046963028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fd819d580 con 0x56134da28be0 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 0x56134da32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 0x56134da39020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc098cc0 tx=0x7f2fcc00c500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5962618fae5e45e9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.878+0000 7f2fb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab7fe56b0 0x556ab818ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81899d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab808ec50 con 0x556ab8189580 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab80e19e0 con 0x556ab7fe56b0 2023-07-21T19:26:00.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe78364bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab80dae70 con 0x556ab818f2d0 2023-07-21T19:26:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.879+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc09a970 con 0x56134da28be0 2023-07-21T19:26:00.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.879+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2fcc09a070 con 0x56134da28be0 2023-07-21T19:26:00.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76d39a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab7fe56b0 0x556ab818ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35154/0 (socket says 172.21.15.5:35154) 2023-07-21T19:26:00.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 -- 172.21.15.5:0/574516524 learned_addr learned my addr 172.21.15.5:0/574516524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.879+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56134d8d44f0 con 0x56134da28be0 2023-07-21T19:26:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76d39a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81899d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe75effd700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 0x7fe7540a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.879+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fcc09abd0 con 0x56134da28be0 2023-07-21T19:26:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 -- 172.21.15.5:0/574516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab7fe56b0 msgr2=0x556ab818ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab7fe56b0 0x556ab818ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.879+0000 7fe76db9b700 1 -- 172.21.15.5:0/574516524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 msgr2=0x556ab81899d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe76db9b700 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81899d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe76db9b700 1 -- 172.21.15.5:0/574516524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab7f08d80 con 0x556ab818f2d0 2023-07-21T19:26:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe76d39a700 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe75effd700 1 -- 172.21.15.5:0/3627983407 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe7600cc580 con 0x7fe768104a60 2023-07-21T19:26:00.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 10 client.28711 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56134d791c60 con 0x56134da28be0 2023-07-21T19:26:00.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc0ca020 con 0x56134da28be0 2023-07-21T19:26:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 0x7fe7540a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe76db9b700 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe764009bb0 tx=0x7fe764009ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f05653df25e3241f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe753fff700 1 -- 172.21.15.5:0/574516524 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640096b0 con 0x556ab818f2d0 2023-07-21T19:26:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe753fff700 1 -- 172.21.15.5:0/574516524 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76400ee80 con 0x556ab818f2d0 2023-07-21T19:26:00.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.880+0000 7fe753fff700 1 -- 172.21.15.5:0/574516524 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640123d0 con 0x556ab818f2d0 2023-07-21T19:26:00.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2fcc0ba280 con 0x56134da28be0 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 10 client.28711 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 10 client.28711 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 20 client.28711 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da53b50 0x56134da55f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 4 client.28711 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.880+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56134d973eb0 con 0x56134da53b50 2023-07-21T19:26:00.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.881+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da53b50 0x56134da55f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe76e39c700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 0x7fe7540a38c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe760099580 tx=0x7fe760004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 -- 172.21.15.5:0/574516524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 msgr2=0x556ab8189040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fe764009bb0 tx=0x7fe764009ec0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 -- 172.21.15.5:0/574516524 shutdown_connections 2023-07-21T19:26:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81899d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab7fe56b0 0x556ab818ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 --2- 172.21.15.5:0/574516524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab818f2d0 0x556ab8189040 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.881+0000 7fe78364bb80 1 -- 172.21.15.5:0/574516524 >> 172.21.15.5:0/574516524 conn(0x556ab817ebe0 msgr2=0x556ab8183f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.881+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da53b50 0x56134da55f60 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c6c27b84f97a61c server_cookie=313e2c938cc912f2 in_seq=0 out_seq=0 2023-07-21T19:26:00.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.881+0000 7f2fb77fe700 10 client.28711 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.882+0000 7fe78364bb80 1 -- 172.21.15.5:0/574516524 shutdown_connections 2023-07-21T19:26:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.882+0000 7f2fb77fe700 1 -- 172.21.15.5:0/2046963028 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56134d973eb0 con 0x56134da53b50 2023-07-21T19:26:00.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.882+0000 7f2fb77fe700 10 client.28711 handle_command_reply: tid=0 2023-07-21T19:26:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.882+0000 7fe78364bb80 1 -- 172.21.15.5:0/574516524 wait complete. 2023-07-21T19:26:00.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.882+0000 7fe78364bb80 1 -- start start 2023-07-21T19:26:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.882+0000 7f2ff228fb80 1 client.28711 shutdown 2023-07-21T19:26:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 msgr2=0x56134da39020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 0x56134da39020 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc098cc0 tx=0x7f2fcc00c500 comp rx=0 tx=0).stop 2023-07-21T19:26:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 Processor -- start 2023-07-21T19:26:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 shutdown_connections 2023-07-21T19:26:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da53b50 0x56134da55f60 crc :-1 s=READY pgs=367 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da33ad0 0x56134da32a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da38ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/2046963028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da28be0 0x56134da39020 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2fb77fe700 0 client.28711 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 >> 172.21.15.5:0/2046963028 conn(0x56134da1eda0 msgr2=0x56134da24230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 shutdown_connections 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 1 -- 172.21.15.5:0/2046963028 wait complete. 2023-07-21T19:26:00.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.883+0000 7f2ff228fb80 20 client.28711 trim_cache size 0 max 16384 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab8192930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab8192e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81933b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab807aa60 con 0x556ab8189580 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab807a760 con 0x556ab818f2d0 2023-07-21T19:26:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab7f5e8c0 con 0x556ab7fe56b0 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.883+0000 7fe78364bb80 10 client.0 _ll_register_callbacks cb 0x556ab8078700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab8192930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab8192930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35162/0 (socket says 172.21.15.5:35162) 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 -- 172.21.15.5:0/3532430280 learned_addr learned my addr 172.21.15.5:0/3532430280 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81933b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76db9b700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab8192e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 -- 172.21.15.5:0/3532430280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 msgr2=0x556ab8192930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.884+0000 7fe76d39a700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab8192930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe76d39a700 1 -- 172.21.15.5:0/3532430280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 msgr2=0x556ab8192e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe76d39a700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab8192e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe76d39a700 1 -- 172.21.15.5:0/3532430280 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe764009810 con 0x556ab818f2d0 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe76db9b700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab8192e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe76d39a700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81933b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098f0 tx=0x7fe758005dc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=514eb865491e97d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe7517fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe75800b8d0 con 0x556ab818f2d0 2023-07-21T19:26:00.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe75800ba30 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ab8034490 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe75800bbf0 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe78364bb80 10 client.28790 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.885+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556ab7ef1c00 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ab8034490 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe758015020 con 0x556ab818f2d0 2023-07-21T19:26:00.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 10 client.28790 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 10 client.28790 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 20 client.28790 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81a2bf0 0x556ab81a5000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 4 client.28790 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.886+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556ab80d3e50 con 0x556ab81a2bf0 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.887+0000 7fe76db9b700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81a2bf0 0x556ab81a5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.887+0000 7fe7517fa700 10 client.28790 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.887+0000 7fe76db9b700 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81a2bf0 0x556ab81a5000 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30dce0b3a6d583ce server_cookie=37045ab1a1abc78a in_seq=0 out_seq=0 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe7517fa700 1 -- 172.21.15.5:0/3532430280 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556ab80d3e50 con 0x556ab81a2bf0 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe7517fa700 10 client.28790 handle_command_reply: tid=0 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 client.28790 shutdown 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 msgr2=0x556ab81933b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81933b0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe7580098f0 tx=0x7fe758005dc0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 shutdown_connections 2023-07-21T19:26:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81933b0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab8192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81a2bf0 0x556ab81a5000 crc :-1 s=READY pgs=368 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 --2- 172.21.15.5:0/3532430280 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab8192e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe7517fa700 0 client.28790 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.888+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 >> 172.21.15.5:0/3532430280 conn(0x556ab817ebe0 msgr2=0x556ab81834e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.889+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 shutdown_connections 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.889+0000 7fe78364bb80 1 -- 172.21.15.5:0/3532430280 wait complete. 2023-07-21T19:26:00.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.889+0000 7fe78364bb80 20 client.28790 trim_cache size 0 max 16384 2023-07-21T19:26:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.903+0000 7f2ff228fb80 1 Processor -- start 2023-07-21T19:26:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 -- start start 2023-07-21T19:26:00.909 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134da2d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da318d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da386b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da29540 con 0x56134da28be0 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da1e970 con 0x56134da282d0 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da254b0 con 0x56134da1eda0 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.910 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da318d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35782/0 (socket says 172.21.15.5:35782) 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.904+0000 7f2fd77fe700 1 -- 172.21.15.5:0/1631290617 learned_addr learned my addr 172.21.15.5:0/1631290617 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134da2d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da386b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1631290617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 msgr2=0x56134da318d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da318d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1631290617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 msgr2=0x56134da386b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da386b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.911 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.905+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1631290617 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56134d7cf250 con 0x56134da1eda0 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.906+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134da2d4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56134da53f20 tx=0x7f2fc8009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=82f6d250a8fa4ea6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.906+0000 7f2fbeffd700 1 -- 172.21.15.5:0/1631290617 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc801d4f0 con 0x56134da1eda0 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.906+0000 7f2fbeffd700 1 -- 172.21.15.5:0/1631290617 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fc8017040 con 0x56134da1eda0 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.906+0000 7f2fbeffd700 1 -- 172.21.15.5:0/1631290617 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8004a90 con 0x56134da1eda0 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1631290617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 msgr2=0x56134da2d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134da2d4a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x56134da53f20 tx=0x7f2fc8009260 comp rx=0 tx=0).stop 2023-07-21T19:26:00.912 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1631290617 shutdown_connections 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da386b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134da2d4a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1631290617 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134da318d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1631290617 >> 172.21.15.5:0/1631290617 conn(0x56134da33700 msgr2=0x56134da311c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.907+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1631290617 shutdown_connections 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.908+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1631290617 wait complete. 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.908+0000 7f2ff228fb80 1 -- start start 2023-07-21T19:26:00.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.908+0000 7f2ff228fb80 1 Processor -- start 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134daa1620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da9bf80 con 0x56134da28be0 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da9c0f0 con 0x56134da282d0 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56134da9c2b0 con 0x56134da1eda0 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.909+0000 7f2ff228fb80 10 client.0 _ll_register_callbacks cb 0x7f2fd0009360 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:00.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fdcf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fd7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134daa1620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fdcf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36666/0 (socket says 172.21.15.5:36666) 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fd77fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35784/0 (socket says 172.21.15.5:35784) 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.910+0000 7f2fdcf9c700 1 -- 172.21.15.5:0/1590875007 learned_addr learned my addr 172.21.15.5:0/1590875007 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 msgr2=0x56134daa1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 Processor -- start 2023-07-21T19:26:00.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1590875007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 msgr2=0x56134da9b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 -- 172.21.15.5:0/1590875007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fc8009db0 con 0x56134da1eda0 2023-07-21T19:26:00.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fdcf9c700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:00.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:00.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 -- start start 2023-07-21T19:26:00.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fd7fff700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134daa1620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56134da53f20 tx=0x7f2fc801d610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3a94db0b9b1f34dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.911+0000 7f2fb77fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab7fe56b0 0x556ab8189040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab818e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab8185970 con 0x556ab7fe56b0 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab8184ea0 con 0x556ab818f2d0 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.911+0000 7fe78364bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab81805f0 con 0x556ab8189580 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35182/0 (socket says 172.21.15.5:35182) 2023-07-21T19:26:00.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 -- 172.21.15.5:0/1206143177 learned_addr learned my addr 172.21.15.5:0/1206143177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fc8010b20 con 0x56134da1eda0 2023-07-21T19:26:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2fc801d9d0 con 0x56134da1eda0 2023-07-21T19:26:00.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2fcc09e9a0 con 0x56134da1eda0 2023-07-21T19:26:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76d39a700 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab7fe56b0 0x556ab8189040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76d39a700 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab818e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2fc801db90 con 0x56134da1eda0 2023-07-21T19:26:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2ff228fb80 10 client.28723 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:00.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56134d7a72b0 con 0x56134da1eda0 2023-07-21T19:26:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 -- 172.21.15.5:0/1206143177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 msgr2=0x556ab818e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab818e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 -- 172.21.15.5:0/1206143177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab7fe56b0 msgr2=0x556ab8189040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab7fe56b0 0x556ab8189040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.912+0000 7fe76db9b700 1 -- 172.21.15.5:0/1206143177 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556ab7f2f1f0 con 0x556ab8189580 2023-07-21T19:26:00.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.912+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2fcc09e9a0 con 0x56134da1eda0 2023-07-21T19:26:00.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.913+0000 7fe76db9b700 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe764013c60 tx=0x7fe764008040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10d73109fc4c9dd1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f2fc8005150 con 0x56134da1eda0 2023-07-21T19:26:00.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 10 client.28723 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 10 client.28723 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 20 client.28723 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da33ad0 0x56134daab3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 4 client.28723 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:26:00.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56134d936050 con 0x56134da33ad0 2023-07-21T19:26:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.913+0000 7fe75cbff700 1 -- 172.21.15.5:0/1206143177 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764021040 con 0x556ab8189580 2023-07-21T19:26:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.913+0000 7fe75cbff700 1 -- 172.21.15.5:0/1206143177 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe76400ed00 con 0x556ab8189580 2023-07-21T19:26:00.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.913+0000 7fe75cbff700 1 -- 172.21.15.5:0/1206143177 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7640292d0 con 0x556ab8189580 2023-07-21T19:26:00.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.913+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da33ad0 0x56134daab3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2fd77fe700 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da33ad0 0x56134daab3c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b32c0f7c56abf477 server_cookie=3ecc85d7cf9cfc27 in_seq=0 out_seq=0 2023-07-21T19:26:00.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2fb77fe700 10 client.28723 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2fb77fe700 1 -- 172.21.15.5:0/1590875007 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56134d936050 con 0x56134da33ad0 2023-07-21T19:26:00.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2fb77fe700 10 client.28723 handle_command_reply: tid=0 2023-07-21T19:26:00.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 client.28723 shutdown 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 -- 172.21.15.5:0/1206143177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 msgr2=0x556ab8199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe764013c60 tx=0x7fe764008040 comp rx=0 tx=0).stop 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 -- 172.21.15.5:0/1206143177 shutdown_connections 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab818e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab7fe56b0 0x556ab8189040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 --2- 172.21.15.5:0/1206143177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab8189580 0x556ab8199c70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.914+0000 7fe78364bb80 1 -- 172.21.15.5:0/1206143177 >> 172.21.15.5:0/1206143177 conn(0x556ab817ebe0 msgr2=0x556ab818dab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 msgr2=0x56134daa1620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134daa1620 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x56134da53f20 tx=0x7f2fc801d610 comp rx=0 tx=0).stop 2023-07-21T19:26:00.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 shutdown_connections 2023-07-21T19:26:00.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56134da33ad0 0x56134daab3c0 crc :-1 s=READY pgs=369 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56134da28be0 0x56134da9b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56134da1eda0 0x56134daa1620 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2fb77fe700 0 client.28723 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 --2- 172.21.15.5:0/1590875007 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56134da282d0 0x56134daa1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.914+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 >> 172.21.15.5:0/1590875007 conn(0x56134da33700 msgr2=0x56134da32980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.915+0000 7fe78364bb80 1 -- 172.21.15.5:0/1206143177 shutdown_connections 2023-07-21T19:26:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.915+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 shutdown_connections 2023-07-21T19:26:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.915+0000 7f2ff228fb80 1 -- 172.21.15.5:0/1590875007 wait complete. 2023-07-21T19:26:00.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.915+0000 7f2ff228fb80 20 client.28723 trim_cache size 0 max 16384 2023-07-21T19:26:00.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.915+0000 7fe78364bb80 1 -- 172.21.15.5:0/1206143177 wait complete. 2023-07-21T19:26:00.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.915+0000 7fe78364bb80 1 -- start start 2023-07-21T19:26:00.931 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:26:00.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 Processor -- start 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab81eaa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81f5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab81eb710 con 0x556ab8189580 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab81eb850 con 0x556ab818f2d0 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556ab81f5fc0 con 0x556ab7fe56b0 2023-07-21T19:26:00.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.916+0000 7fe78364bb80 10 client.0 _ll_register_callbacks cb 0x7fe75800f930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.917+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.917+0000 7fe76db9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36678/0 (socket says 172.21.15.5:36678) 2023-07-21T19:26:00.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.917+0000 7fe76db9b700 1 -- 172.21.15.5:0/2870743383 learned_addr learned my addr 172.21.15.5:0/2870743383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.917+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 msgr2=0x7f2fc00a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.917+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 0x7f2fc00a38c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f2fd0009ec0 tx=0x7f2fd000b040 comp rx=0 tx=0).stop 2023-07-21T19:26:00.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.917+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 msgr2=0x7f2fd81982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.917+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 0x7f2fd81982e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0098e0 tx=0x7f2fcc00f690 comp rx=0 tx=0).stop 2023-07-21T19:26:00.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 -- 172.21.15.5:0/2870743383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 msgr2=0x556ab81eaa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 shutdown_connections 2023-07-21T19:26:00.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2fd8106fb0 0x7f2fd819c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2fc00a1410 0x7f2fc00a38c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2fd8101f70 0x7f2fd8197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 --2- 172.21.15.5:0/3350746714 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2fd8102910 0x7f2fd81982e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 >> 172.21.15.5:0/3350746714 conn(0x7f2fd80fd7e0 msgr2=0x7f2fd80fe360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.917+0000 7fe76d39a700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab81eaa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab81eaa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 -- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 msgr2=0x556ab81f5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:00.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81f5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 -- 172.21.15.5:0/2870743383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x556ab7f08d80 con 0x556ab8189580 2023-07-21T19:26:00.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76d39a700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab81eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:26:00.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 shutdown_connections 2023-07-21T19:26:00.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:00.918+0000 7f2fde9ff700 1 -- 172.21.15.5:0/3350746714 wait complete. 2023-07-21T19:26:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe76db9b700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe7640048a0 tx=0x7fe76400edb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77d03ac9b0481f89 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.918+0000 7fe7517fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe764021040 con 0x556ab8189580 2023-07-21T19:26:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556ab8095ff0 con 0x556ab8189580 2023-07-21T19:26:00.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe764005c50 con 0x556ab8189580 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe7640138b0 con 0x556ab8189580 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe78364bb80 10 client.19167 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.919+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556ab7f07250 con 0x556ab8189580 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x556ab8095ff0 con 0x556ab8189580 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe764011070 con 0x556ab8189580 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 10 client.19167 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:00.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 10 client.19167 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 20 client.19167 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81f7220 0x556ab81f9630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 4 client.19167 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.920+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556ab7f29f10 con 0x556ab81f7220 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.921+0000 7fe76d39a700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81f7220 0x556ab81f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.922+0000 7fe76d39a700 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81f7220 0x556ab81f9630 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c2c6dc0c105621 server_cookie=e2ddc2818ea638c7 in_seq=0 out_seq=0 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.922+0000 7fe7517fa700 10 client.19167 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe7517fa700 1 -- 172.21.15.5:0/2870743383 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556ab7f29f10 con 0x556ab81f7220 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe7517fa700 10 client.19167 handle_command_reply: tid=0 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 client.19167 shutdown 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 msgr2=0x556ab81eaf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fe7640048a0 tx=0x7fe76400edb0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 shutdown_connections 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556ab818f2d0 0x556ab81f5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556ab7fe56b0 0x556ab81eaa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556ab81f7220 0x556ab81f9630 crc :-1 s=READY pgs=370 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 --2- 172.21.15.5:0/2870743383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556ab8189580 0x556ab81eaf40 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe7517fa700 0 client.19167 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 >> 172.21.15.5:0/2870743383 conn(0x556ab817ebe0 msgr2=0x556ab818e340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 shutdown_connections 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.945+0000 7fe78364bb80 1 -- 172.21.15.5:0/2870743383 wait complete. 2023-07-21T19:26:00.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe78364bb80 20 client.19167 trim_cache size 0 max 16384 2023-07-21T19:26:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:26:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:26:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:26:00.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:26:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 msgr2=0x7fe7540a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 0x7fe7540a38c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe760099580 tx=0x7fe760004ad0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 msgr2=0x7fe7681a0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 0x7fe7681a0ef0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7fe760009cd0 tx=0x7fe76000c620 comp rx=0 tx=0).stop 2023-07-21T19:26:00.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 shutdown_connections 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe7540a1410 0x7fe7540a38c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe768104a60 0x7fe7681a0ef0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe768067ee0 0x7fe76819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 --2- 172.21.15.5:0/3627983407 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe7681024f0 0x7fe76819c880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.946+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 >> 172.21.15.5:0/3627983407 conn(0x7fe768074760 msgr2=0x7fe7681014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.947+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 shutdown_connections 2023-07-21T19:26:00.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:00.947+0000 7fe76fdff700 1 -- 172.21.15.5:0/3627983407 wait complete. 2023-07-21T19:26:01.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:01 smithi156 ceph-mon[128316]: pgmap v579: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:26:01.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:01 smithi101 ceph-mon[128136]: pgmap v579: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:26:01.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:01 smithi005 ceph-mon[126460]: pgmap v579: 97 pgs: 97 active+clean; 16 GiB data, 8.2 GiB used, 1.0 TiB / 1.0 TiB avail; 310 KiB/s rd, 27 MiB/s wr, 54 op/s 2023-07-21T19:26:01.956 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:26:01.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:26:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 -- 172.21.15.5:0/3870735816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00073160 msgr2=0x7f4b0010be00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.240 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 --2- 172.21.15.5:0/3870735816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00073160 0x7f4b0010be00 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0098f0 tx=0x7f4aec009c00 comp rx=0 tx=0).stop 2023-07-21T19:26:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 -- 172.21.15.5:0/3870735816 shutdown_connections 2023-07-21T19:26:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 --2- 172.21.15.5:0/3870735816 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b0010c340 0x7f4b0010e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 --2- 172.21.15.5:0/3870735816 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b00072850 0x7f4b00072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 --2- 172.21.15.5:0/3870735816 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00073160 0x7f4b0010be00 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 -- 172.21.15.5:0/3870735816 >> 172.21.15.5:0/3870735816 conn(0x7f4b000fb5a0 msgr2=0x7f4b000fd9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 -- 172.21.15.5:0/3870735816 shutdown_connections 2023-07-21T19:26:02.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.249+0000 7f4b071ff700 1 -- 172.21.15.5:0/3870735816 wait complete. 2023-07-21T19:26:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.251+0000 7f4b071ff700 1 Processor -- start 2023-07-21T19:26:02.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.251+0000 7f4b071ff700 1 -- start start 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b00073160 0x7f4b00198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b0010c340 0x7f4b0019b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0019bbd0 con 0x7f4b0010c340 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0019bd40 con 0x7f4b00073160 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b071ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b0019bf00 con 0x7f4b00072850 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35220/0 (socket says 172.21.15.5:35220) 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 -- 172.21.15.5:0/2295394397 learned_addr learned my addr 172.21.15.5:0/2295394397 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4afffff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b00073160 0x7f4b00198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b0579c700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b0010c340 0x7f4b0019b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 -- 172.21.15.5:0/2295394397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b00073160 msgr2=0x7f4b00198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b00073160 0x7f4b00198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 -- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b0010c340 msgr2=0x7f4b0019b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b0010c340 0x7f4b0019b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.252+0000 7f4b04f9b700 1 -- 172.21.15.5:0/2295394397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af40095f0 con 0x7f4b00072850 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4af400ec10 tx=0x7f4af400ef20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4afdffb700 1 -- 172.21.15.5:0/2295394397 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af400cc90 con 0x7f4b00072850 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4afdffb700 1 -- 172.21.15.5:0/2295394397 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af400cdf0 con 0x7f4b00072850 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4aec009520 con 0x7f4b00072850 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4afdffb700 1 -- 172.21.15.5:0/2295394397 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4003e80 con 0x7f4b00072850 2023-07-21T19:26:02.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.253+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b001a4960 con 0x7f4b00072850 2023-07-21T19:26:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.257+0000 7f4afdffb700 1 -- 172.21.15.5:0/2295394397 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4af4004000 con 0x7f4b00072850 2023-07-21T19:26:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.258+0000 7f4b1abfcb80 1 Processor -- start 2023-07-21T19:26:02.248 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.258+0000 7f4b1abfcb80 1 -- start start 2023-07-21T19:26:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.258+0000 7f4afdffb700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 0x7f4af00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.249 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7bb720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afdffb700 1 -- 172.21.15.5:0/2295394397 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4af4014070 con 0x7f4b00072850 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7c8f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f6c1d30 con 0x56284f7bbc60 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f714ac0 con 0x56284f7bb350 2023-07-21T19:26:02.250 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f70df50 con 0x56284f7c6b50 2023-07-21T19:26:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b0579c700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 0x7f4af00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b0579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7c8f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36706/0 (socket says 172.21.15.5:36706) 2023-07-21T19:26:02.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 -- 172.21.15.5:0/3640750448 learned_addr learned my addr 172.21.15.5:0/3640750448 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7bb720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 -- 172.21.15.5:0/3640750448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 msgr2=0x56284f7c8f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7c8f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 -- 172.21.15.5:0/3640750448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 msgr2=0x56284f7bb720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.259+0000 7f4afffff700 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7bb720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4afffff700 1 -- 172.21.15.5:0/3640750448 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284f53be60 con 0x56284f7bbc60 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4b0579c700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 0x7f4af00a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009550 tx=0x7f4af8003e30 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4afffff700 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0098f0 tx=0x7f4aec00bec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c79187132988b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4aeaffd700 1 -- 172.21.15.5:0/3640750448 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec00fea0 con 0x56284f7bbc60 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4aeaffd700 1 -- 172.21.15.5:0/3640750448 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4aec01d040 con 0x56284f7bbc60 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.260+0000 7f4aeaffd700 1 -- 172.21.15.5:0/3640750448 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec0052a0 con 0x56284f7bbc60 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3640750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 msgr2=0x56284f7c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f4aec0098f0 tx=0x7f4aec00bec0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3640750448 shutdown_connections 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7c8f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7bb720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3640750448 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bbc60 0x56284f7c6610 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3640750448 >> 172.21.15.5:0/3640750448 conn(0x56284f7b1e20 msgr2=0x56284f7b7720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3640750448 shutdown_connections 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.261+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3640750448 wait complete. 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.262+0000 7f4b1abfcb80 1 -- start start 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.262+0000 7f4b1abfcb80 1 Processor -- start 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f6adb40 con 0x56284f7c6b50 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f6ad840 con 0x56284f7bbc60 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f5919a0 con 0x56284f7bb350 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b1abfcb80 10 client.0 _ll_register_callbacks cb 0x56284f6ab7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4afffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35236/0 (socket says 172.21.15.5:35236) 2023-07-21T19:26:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b0579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4b04f9b700 1 -- 172.21.15.5:0/4206215526 learned_addr learned my addr 172.21.15.5:0/4206215526 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.263+0000 7f4afffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35858/0 (socket says 172.21.15.5:35858) 2023-07-21T19:26:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 -- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 msgr2=0x56284f7c37a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 -- 172.21.15.5:0/4206215526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 msgr2=0x56284f7c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 -- 172.21.15.5:0/4206215526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b0010f360 con 0x56284f7bb350 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4afffff700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4af40cae80 tx=0x7f4af409a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de250e3d2d05dcd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4adffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:02.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af40044b0 con 0x56284f7bb350 2023-07-21T19:26:02.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4af40b2be0 con 0x56284f7bb350 2023-07-21T19:26:02.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.264+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56284f667570 con 0x56284f7bb350 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4af409a460 con 0x56284f7bb350 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 10 client.28738 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56284f524ce0 con 0x56284f7bb350 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af40c8020 con 0x56284f7bb350 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4af40a21c0 con 0x56284f7bb350 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 10 client.28738 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:02.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 10 client.28738 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 20 client.28738 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f7d6410 0x56284f7d87f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 4 client.28738 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.265+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56284f706f30 con 0x56284f7d6410 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.266+0000 7f4afffff700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f7d6410 0x56284f7d87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.266+0000 7f4afffff700 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f7d6410 0x56284f7d87f0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ceb6d3a64f2ac628 server_cookie=8b60370605fa543e in_seq=0 out_seq=0 2023-07-21T19:26:02.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.266+0000 7f4adffff700 10 client.28738 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.267+0000 7f4adffff700 1 -- 172.21.15.5:0/4206215526 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56284f706f30 con 0x56284f7d6410 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.267+0000 7f4adffff700 10 client.28738 handle_command_reply: tid=0 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.267+0000 7f4b1abfcb80 1 client.28738 shutdown 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.267+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 msgr2=0x56284f7c6590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.267+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f4af40cae80 tx=0x7f4af409a340 comp rx=0 tx=0).stop 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 shutdown_connections 2023-07-21T19:26:02.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f7d6410 0x56284f7d87f0 crc :-1 s=READY pgs=371 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bb350 0x56284f7c6590 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/4206215526 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7c37a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4adffff700 0 client.28738 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 >> 172.21.15.5:0/4206215526 conn(0x56284f7b1e20 msgr2=0x56284f7b7310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 shutdown_connections 2023-07-21T19:26:02.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/4206215526 wait complete. 2023-07-21T19:26:02.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.268+0000 7f4b1abfcb80 20 client.28738 trim_cache size 0 max 16384 2023-07-21T19:26:02.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.288+0000 7f4b1abfcb80 1 Processor -- start 2023-07-21T19:26:02.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.288+0000 7f4b1abfcb80 1 -- start start 2023-07-21T19:26:02.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.288+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7cd530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 0x56284f7bdb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7be0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f7bc5c0 con 0x56284f7c6b50 2023-07-21T19:26:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f7bcde0 con 0x56284f7bb350 2023-07-21T19:26:02.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f7be5f0 con 0x56284f7bbc60 2023-07-21T19:26:02.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b0579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7be0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4afffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 0x56284f7bdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7cd530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4afffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 0x56284f7bdb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35252/0 (socket says 172.21.15.5:35252) 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4afffff700 1 -- 172.21.15.5:0/1904291071 learned_addr learned my addr 172.21.15.5:0/1904291071 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 -- 172.21.15.5:0/1904291071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 msgr2=0x56284f7bdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 0x56284f7bdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 -- 172.21.15.5:0/1904291071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 msgr2=0x56284f7be0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7be0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.289+0000 7f4b04f9b700 1 -- 172.21.15.5:0/1904291071 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56284f7b3660 con 0x56284f7bb350 2023-07-21T19:26:02.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.290+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7cd530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4af4016ab0 tx=0x7f4af40987b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3817078b7f854da8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.290+0000 7f4aeb7fe700 1 -- 172.21.15.5:0/1904291071 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af4099fd0 con 0x56284f7bb350 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.290+0000 7f4aeb7fe700 1 -- 172.21.15.5:0/1904291071 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4af40c8070 con 0x56284f7bb350 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.290+0000 7f4aeb7fe700 1 -- 172.21.15.5:0/1904291071 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af40042b0 con 0x56284f7bb350 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/1904291071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 msgr2=0x56284f7cd530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7cd530 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4af4016ab0 tx=0x7f4af40987b0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/1904291071 shutdown_connections 2023-07-21T19:26:02.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7c6b50 0x56284f7be0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bb350 0x56284f7cd530 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/1904291071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7bbc60 0x56284f7bdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/1904291071 >> 172.21.15.5:0/1904291071 conn(0x56284f7b1e20 msgr2=0x56284f7bd460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/1904291071 shutdown_connections 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/1904291071 wait complete. 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.291+0000 7f4b1abfcb80 1 -- start start 2023-07-21T19:26:02.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 Processor -- start 2023-07-21T19:26:02.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bb350 0x56284f7cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7cb9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f7cc800 con 0x56284f7bb350 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f82a2b0 con 0x56284f7bbc60 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56284f82a3f0 con 0x56284f7c6b50 2023-07-21T19:26:02.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.292+0000 7f4b1abfcb80 10 client.0 _ll_register_callbacks cb 0x7f4af4099b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4b04f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bb350 0x56284f7cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4b0579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4b0579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35258/0 (socket says 172.21.15.5:35258) 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4b0579c700 1 -- 172.21.15.5:0/3602630136 learned_addr learned my addr 172.21.15.5:0/3602630136 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7cb9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 -- 172.21.15.5:0/3602630136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 msgr2=0x56284f7cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 -- 172.21.15.5:0/3602630136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bb350 msgr2=0x56284f7cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bb350 0x56284f7cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4afffff700 1 -- 172.21.15.5:0/3602630136 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56284f524ce0 con 0x56284f7bbc60 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.293+0000 7f4b0579c700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4afffff700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7cb9f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009c40 tx=0x7f4aec00b980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3febf40238129517 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4adffff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4aec032040 con 0x56284f7bbc60 2023-07-21T19:26:02.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4aec01d040 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4aec0321a0 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4aec032380 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4b1abfcb80 10 client.28817 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.294+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56284f5622d0 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4aec01d040 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4aec015020 con 0x56284f7bbc60 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 10 client.28817 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:02.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 10 client.28817 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 20 client.28817 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f82b7a0 0x56284f82dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 4 client.28817 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.295+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x56284f6c90d0 con 0x56284f82b7a0 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.296+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f82b7a0 0x56284f82dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.296+0000 7f4b04f9b700 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f82b7a0 0x56284f82dbb0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45fe955b82064d58 server_cookie=daa3b212f774a340 in_seq=0 out_seq=0 2023-07-21T19:26:02.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.296+0000 7f4adffff700 10 client.28817 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4adffff700 1 -- 172.21.15.5:0/3602630136 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x56284f6c90d0 con 0x56284f82b7a0 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4adffff700 10 client.28817 handle_command_reply: tid=0 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 client.28817 shutdown 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 msgr2=0x56284f7cb9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7cb9f0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f4aec009c40 tx=0x7f4aec00b980 comp rx=0 tx=0).stop 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 shutdown_connections 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56284f7c6b50 0x56284f7cc130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56284f82b7a0 0x56284f82dbb0 crc :-1 s=READY pgs=372 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56284f7bb350 0x56284f7cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 --2- 172.21.15.5:0/3602630136 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56284f7bbc60 0x56284f7cb9f0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.297+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 >> 172.21.15.5:0/3602630136 conn(0x56284f7b1e20 msgr2=0x56284f7bdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.298+0000 7f4adffff700 0 client.28817 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.298+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 shutdown_connections 2023-07-21T19:26:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.298+0000 7f4b1abfcb80 1 -- 172.21.15.5:0/3602630136 wait complete. 2023-07-21T19:26:02.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.298+0000 7f4b1abfcb80 20 client.28817 trim_cache size 0 max 16384 2023-07-21T19:26:02.301 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:26:02.301 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:26:02.302 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "316f7ad1-ba85-41e7-aae4-565376e84e00", 2023-07-21T19:26:02.302 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:26:02.302 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:26:02.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 msgr2=0x7f4af00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 0x7f4af00a3770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4af8009550 tx=0x7f4af8003e30 comp rx=0 tx=0).stop 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 msgr2=0x7f4b00197ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f4af400ec10 tx=0x7f4af400ef20 comp rx=0 tx=0).stop 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 shutdown_connections 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4af00a12c0 0x7f4af00a3770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b0010c340 0x7f4b0019b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b00072850 0x7f4b00197ce0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 --2- 172.21.15.5:0/2295394397 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b00073160 0x7f4b00198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 >> 172.21.15.5:0/2295394397 conn(0x7f4b000fb5a0 msgr2=0x7f4b0010e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 shutdown_connections 2023-07-21T19:26:02.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.299+0000 7f4b071ff700 1 -- 172.21.15.5:0/2295394397 wait complete. 2023-07-21T19:26:02.310 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:26:02.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.574+0000 7f63fcfff700 1 -- 172.21.15.5:0/3396262333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f81015b0 msgr2=0x7f63f8109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.574+0000 7f63fcfff700 1 --2- 172.21.15.5:0/3396262333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f81015b0 0x7f63f8109aa0 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f63ec009990 tx=0x7f63ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 -- 172.21.15.5:0/3396262333 shutdown_connections 2023-07-21T19:26:02.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 --2- 172.21.15.5:0/3396262333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f810c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 --2- 172.21.15.5:0/3396262333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f8100ca0 0x7f63f8101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 --2- 172.21.15.5:0/3396262333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f81015b0 0x7f63f8109aa0 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 -- 172.21.15.5:0/3396262333 >> 172.21.15.5:0/3396262333 conn(0x7f63f80fa510 msgr2=0x7f63f80fc920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 -- 172.21.15.5:0/3396262333 shutdown_connections 2023-07-21T19:26:02.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.575+0000 7f63fcfff700 1 -- 172.21.15.5:0/3396262333 wait complete. 2023-07-21T19:26:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.576+0000 7f63fcfff700 1 Processor -- start 2023-07-21T19:26:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.576+0000 7f63fcfff700 1 -- start start 2023-07-21T19:26:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f81015b0 0x7f63f8102050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f8102590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a8440 con 0x7f63f8100ca0 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a85b0 con 0x7f63f810a070 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63fcfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81a8770 con 0x7f63f81015b0 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63f6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f8102590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36736/0 (socket says 172.21.15.5:36736) 2023-07-21T19:26:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63f659c700 1 -- 172.21.15.5:0/4024619694 learned_addr learned my addr 172.21.15.5:0/4024619694 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.577+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f81015b0 0x7f63f8102050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 -- 172.21.15.5:0/4024619694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f81015b0 msgr2=0x7f63f8102050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f81015b0 0x7f63f8102050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 -- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 msgr2=0x7f63f8101b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 -- 172.21.15.5:0/4024619694 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec0095f0 con 0x7f63f810a070 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f659c700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f8102590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f63f000efc0 tx=0x7f63f000c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.578+0000 7f63e77fe700 1 -- 172.21.15.5:0/4024619694 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0018070 con 0x7f63f810a070 2023-07-21T19:26:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.579+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f81a89f0 con 0x7f63f810a070 2023-07-21T19:26:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.579+0000 7f63e77fe700 1 -- 172.21.15.5:0/4024619694 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63f000f040 con 0x7f63f810a070 2023-07-21T19:26:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.579+0000 7f63e77fe700 1 -- 172.21.15.5:0/4024619694 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f000cba0 con 0x7f63f810a070 2023-07-21T19:26:02.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.579+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a8e80 con 0x7f63f810a070 2023-07-21T19:26:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.584+0000 7f63e77fe700 1 -- 172.21.15.5:0/4024619694 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f63f0003a00 con 0x7f63f810a070 2023-07-21T19:26:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.585+0000 7f6410a11b80 1 Processor -- start 2023-07-21T19:26:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.585+0000 7f6410a11b80 1 -- start start 2023-07-21T19:26:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f63e77fe700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 0x7f63e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de91eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dd8acb0 con 0x56048de84be0 2023-07-21T19:26:02.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dddda40 con 0x56048de842d0 2023-07-21T19:26:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f6410a11b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048ddd6ed0 con 0x56048de8fad0 2023-07-21T19:26:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f63e77fe700 1 -- 172.21.15.5:0/4024619694 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63f00cc7e0 con 0x7f63f810a070 2023-07-21T19:26:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f63f659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f63f659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35928/0 (socket says 172.21.15.5:35928) 2023-07-21T19:26:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.586+0000 7f63f659c700 1 -- 172.21.15.5:0/1649911285 learned_addr learned my addr 172.21.15.5:0/1649911285 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 0x7f63e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de91eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 -- 172.21.15.5:0/1649911285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 msgr2=0x56048de91eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de91eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 -- 172.21.15.5:0/1649911285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 msgr2=0x56048de8f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 -- 172.21.15.5:0/1649911285 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56048dc04de0 con 0x56048de842d0 2023-07-21T19:26:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f659c700 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009f70 tx=0x7f63e800e4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a386d9db47e568 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.587+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 0x7f63e00a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63f810b3b0 tx=0x7f63ec01a040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.588+0000 7f63e4ff9700 1 -- 172.21.15.5:0/1649911285 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8009a40 con 0x56048de842d0 2023-07-21T19:26:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.588+0000 7f63e4ff9700 1 -- 172.21.15.5:0/1649911285 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e800ed90 con 0x56048de842d0 2023-07-21T19:26:02.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.588+0000 7f63e4ff9700 1 -- 172.21.15.5:0/1649911285 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8012750 con 0x56048de842d0 2023-07-21T19:26:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- 172.21.15.5:0/1649911285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 msgr2=0x56048de846a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009f70 tx=0x7f63e800e4b0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- 172.21.15.5:0/1649911285 shutdown_connections 2023-07-21T19:26:02.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de91eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de846a0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1649911285 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- 172.21.15.5:0/1649911285 >> 172.21.15.5:0/1649911285 conn(0x56048de7ada0 msgr2=0x56048de806a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- 172.21.15.5:0/1649911285 shutdown_connections 2023-07-21T19:26:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- 172.21.15.5:0/1649911285 wait complete. 2023-07-21T19:26:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.589+0000 7f6410a11b80 1 -- start start 2023-07-21T19:26:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.590+0000 7f6410a11b80 1 Processor -- start 2023-07-21T19:26:02.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 0x56048de8f480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 0x56048de8cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dd76ac0 con 0x56048de84be0 2023-07-21T19:26:02.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dd767c0 con 0x56048de8fad0 2023-07-21T19:26:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dc5a920 con 0x56048de842d0 2023-07-21T19:26:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f6410a11b80 10 client.0 _ll_register_callbacks cb 0x56048dd74760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36764/0 (socket says 172.21.15.5:36764) 2023-07-21T19:26:02.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f5d9b700 1 -- 172.21.15.5:0/3306250949 learned_addr learned my addr 172.21.15.5:0/3306250949 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 0x56048de8f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 0x56048de8cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 -- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 msgr2=0x56048de8cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 0x56048de8cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 -- 172.21.15.5:0/3306250949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 msgr2=0x56048de8c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.591+0000 7f63f659c700 1 -- 172.21.15.5:0/3306250949 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e8009ba0 con 0x56048de842d0 2023-07-21T19:26:02.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.592+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 0x56048de8cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.592+0000 7f63f659c700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 0x56048de8f480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f80 tx=0x7f63e8013c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2410f3fe99c26b5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.592+0000 7f63ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:02.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.594+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8006e60 con 0x56048de842d0 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.594+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63e801c860 con 0x56048de842d0 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.594+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56048dd304f0 con 0x56048de842d0 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e801ca20 con 0x56048de842d0 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 10 client.28753 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56048dbedc60 con 0x56048de842d0 2023-07-21T19:26:02.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e8031080 con 0x56048de842d0 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63e802b030 con 0x56048de842d0 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 10 client.28753 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 10 client.28753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 20 client.28753 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048de9f350 0x56048dea1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 4 client.28753 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.595+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56048ddcfeb0 con 0x56048de9f350 2023-07-21T19:26:02.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.596+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048de9f350 0x56048dea1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.596+0000 7f63ddffb700 10 client.28753 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.596+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048de9f350 0x56048dea1760 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8bc0e5b21b7c3a2 server_cookie=c4c6992c01cd250d in_seq=0 out_seq=0 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.597+0000 7f63ddffb700 1 -- 172.21.15.5:0/3306250949 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56048ddcfeb0 con 0x56048de9f350 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.597+0000 7f63ddffb700 10 client.28753 handle_command_reply: tid=0 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.597+0000 7f6410a11b80 1 client.28753 shutdown 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 msgr2=0x56048de8f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 0x56048de8f480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f80 tx=0x7f63e8013c00 comp rx=0 tx=0).stop 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 shutdown_connections 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de8fad0 0x56048de8cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de842d0 0x56048de8f480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048de9f350 0x56048dea1760 crc :-1 s=READY pgs=373 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3306250949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de8c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f63ddffb700 0 client.28753 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 >> 172.21.15.5:0/3306250949 conn(0x56048de7ada0 msgr2=0x56048de80230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 shutdown_connections 2023-07-21T19:26:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 1 -- 172.21.15.5:0/3306250949 wait complete. 2023-07-21T19:26:02.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.598+0000 7f6410a11b80 20 client.28753 trim_cache size 0 max 16384 2023-07-21T19:26:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 Processor -- start 2023-07-21T19:26:02.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 -- start start 2023-07-21T19:26:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de84be0 0x56048de8bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de8c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048de84fb0 con 0x56048de842d0 2023-07-21T19:26:02.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048dea1dc0 con 0x56048de84be0 2023-07-21T19:26:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.621+0000 7f6410a11b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048de8abf0 con 0x56048de8fad0 2023-07-21T19:26:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36778/0 (socket says 172.21.15.5:36778) 2023-07-21T19:26:02.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 -- 172.21.15.5:0/1204141601 learned_addr learned my addr 172.21.15.5:0/1204141601 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de8c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de84be0 0x56048de8bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 -- 172.21.15.5:0/1204141601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 msgr2=0x56048de8c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de8c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 -- 172.21.15.5:0/1204141601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de84be0 msgr2=0x56048de8bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de84be0 0x56048de8bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 -- 172.21.15.5:0/1204141601 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56048dc2b250 con 0x56048de842d0 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de8c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.622+0000 7f63f659c700 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f10 tx=0x7f63e8009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=526f801362c65b4b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.623+0000 7f63e57fa700 1 -- 172.21.15.5:0/1204141601 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8025ea0 con 0x56048de842d0 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.623+0000 7f63e57fa700 1 -- 172.21.15.5:0/1204141601 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56048dd304f0 con 0x56048de842d0 2023-07-21T19:26:02.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.623+0000 7f63e57fa700 1 -- 172.21.15.5:0/1204141601 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80256a0 con 0x56048de842d0 2023-07-21T19:26:02.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.623+0000 7f6410a11b80 1 -- 172.21.15.5:0/1204141601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 msgr2=0x56048de8efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.623+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f10 tx=0x7f63e8009210 comp rx=0 tx=0).stop 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 -- 172.21.15.5:0/1204141601 shutdown_connections 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de8c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de842d0 0x56048de8efb0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 --2- 172.21.15.5:0/1204141601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de84be0 0x56048de8bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 -- 172.21.15.5:0/1204141601 >> 172.21.15.5:0/1204141601 conn(0x56048de7ada0 msgr2=0x56048de879f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 -- 172.21.15.5:0/1204141601 shutdown_connections 2023-07-21T19:26:02.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 -- 172.21.15.5:0/1204141601 wait complete. 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 -- start start 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.624+0000 7f6410a11b80 1 Processor -- start 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de961a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de94680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048de94d90 con 0x56048de84be0 2023-07-21T19:26:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048de94f00 con 0x56048de842d0 2023-07-21T19:26:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56048de950c0 con 0x56048de8fad0 2023-07-21T19:26:02.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f6410a11b80 10 client.0 _ll_register_callbacks cb 0x7f63e8009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f63f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f63f5d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36780/0 (socket says 172.21.15.5:36780) 2023-07-21T19:26:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f63f659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de961a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.625+0000 7f63f5d9b700 1 -- 172.21.15.5:0/3514234211 learned_addr learned my addr 172.21.15.5:0/3514234211 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f6d9d700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de94680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 -- 172.21.15.5:0/3514234211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 msgr2=0x56048de94680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de94680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 -- 172.21.15.5:0/3514234211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 msgr2=0x56048de966e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 -- 172.21.15.5:0/3514234211 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a8e80 con 0x56048de842d0 2023-07-21T19:26:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63f659c700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de961a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f80 tx=0x7f63e8005d90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa53c2c6b8e5f56f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:02.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8004450 con 0x56048de842d0 2023-07-21T19:26:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(112..112 src has 1..112) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f63e802d400 con 0x56048de842d0 2023-07-21T19:26:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.626+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f63e800ed90 con 0x56048de842d0 2023-07-21T19:26:02.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f63e8025440 con 0x56048de842d0 2023-07-21T19:26:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 10 client.28835 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56048dbedc60 con 0x56048de842d0 2023-07-21T19:26:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f63e8045020 con 0x56048de842d0 2023-07-21T19:26:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f63e8011070 con 0x56048de842d0 2023-07-21T19:26:02.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 10 client.28835 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 10 client.28835 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 20 client.28835 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048def3f10 0x56048def62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:02.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 4 client.28835 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:26:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.627+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56048dd92050 con 0x56048def3f10 2023-07-21T19:26:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048def3f10 0x56048def62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f63f5d9b700 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048def3f10 0x56048def62f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0e228421c23472 server_cookie=a392c1626163c14d in_seq=0 out_seq=0 2023-07-21T19:26:02.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f63ddffb700 10 client.28835 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f63ddffb700 1 -- 172.21.15.5:0/3514234211 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56048dd92050 con 0x56048def3f10 2023-07-21T19:26:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f63ddffb700 10 client.28835 handle_command_reply: tid=0 2023-07-21T19:26:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.628+0000 7f6410a11b80 1 client.28835 shutdown 2023-07-21T19:26:02.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 msgr2=0x56048de961a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de961a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f63e8013f80 tx=0x7f63e8005d90 comp rx=0 tx=0).stop 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 shutdown_connections 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56048de8fad0 0x56048de94680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56048de842d0 0x56048de961a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56048def3f10 0x56048def62f0 crc :-1 s=READY pgs=374 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 --2- 172.21.15.5:0/3514234211 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56048de84be0 0x56048de966e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f63ddffb700 0 client.28835 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 >> 172.21.15.5:0/3514234211 conn(0x56048de7ada0 msgr2=0x56048de8bf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 shutdown_connections 2023-07-21T19:26:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 1 -- 172.21.15.5:0/3514234211 wait complete. 2023-07-21T19:26:02.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.629+0000 7f6410a11b80 20 client.28835 trim_cache size 0 max 16384 2023-07-21T19:26:02.634 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:26:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:26:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:26:02.635 INFO:teuthology.orchestra.run.smithi005.stdout: "316f7ad1-ba85-41e7-aae4-565376e84e00": { 2023-07-21T19:26:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:26:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "316f7ad1-ba85-41e7-aae4-565376e84e00", 2023-07-21T19:26:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:26:02.636 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:26:02.637 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:26:02.637 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:26:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 msgr2=0x7f63e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 0x7f63e00a3770 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f63f810b3b0 tx=0x7f63ec01a040 comp rx=0 tx=0).stop 2023-07-21T19:26:02.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 msgr2=0x7f63f8102590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:02.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f8102590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f63f000efc0 tx=0x7f63f000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 shutdown_connections 2023-07-21T19:26:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f63f810a070 0x7f63f8102590 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f63e00a12c0 0x7f63e00a3770 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f63f8100ca0 0x7f63f8101b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 --2- 172.21.15.5:0/4024619694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f63f81015b0 0x7f63f8102050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:02.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 >> 172.21.15.5:0/4024619694 conn(0x7f63f80fa510 msgr2=0x7f63f810c200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 shutdown_connections 2023-07-21T19:26:02.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:02.630+0000 7f63fcfff700 1 -- 172.21.15.5:0/4024619694 wait complete. 2023-07-21T19:26:02.642 INFO:tasks.cephfs.filesystem:scrub status for tag:316f7ad1-ba85-41e7-aae4-565376e84e00 - {'path': '/', 'tag': '316f7ad1-ba85-41e7-aae4-565376e84e00', 'options': 'recursive,force'} 2023-07-21T19:26:03.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:26:03.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:03.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:03.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: pgmap v580: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 41 MiB/s wr, 120 op/s 2023-07-21T19:26:03.661 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:03 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:26:03.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:03.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:03.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: pgmap v580: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 41 MiB/s wr, 120 op/s 2023-07-21T19:26:03.822 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:03 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:26:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: pgmap v580: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 334 KiB/s rd, 41 MiB/s wr, 120 op/s 2023-07-21T19:26:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:03 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:05.660 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:05 smithi156 ceph-mon[128316]: pgmap v581: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 27 MiB/s wr, 122 op/s 2023-07-21T19:26:05.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:05 smithi101 ceph-mon[128136]: pgmap v581: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 27 MiB/s wr, 122 op/s 2023-07-21T19:26:05.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:05 smithi005 ceph-mon[126460]: pgmap v581: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 336 KiB/s rd, 27 MiB/s wr, 122 op/s 2023-07-21T19:26:07.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:07 smithi101 ceph-mon[128136]: pgmap v582: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 26 MiB/s wr, 119 op/s 2023-07-21T19:26:07.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:07 smithi101 ceph-mon[128136]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:26:07.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:07 smithi005 ceph-mon[126460]: pgmap v582: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 26 MiB/s wr, 119 op/s 2023-07-21T19:26:07.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:07 smithi005 ceph-mon[126460]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:26:07.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:07 smithi156 ceph-mon[128316]: pgmap v582: 97 pgs: 97 active+clean; 16 GiB data, 8.3 GiB used, 1.0 TiB / 1.0 TiB avail; 302 KiB/s rd, 26 MiB/s wr, 119 op/s 2023-07-21T19:26:07.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:07 smithi156 ceph-mon[128316]: osdmap e113: 12 total, 12 up, 12 in 2023-07-21T19:26:09.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:09 smithi005 ceph-mon[126460]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:09.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:09 smithi156 ceph-mon[128316]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:10.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:09 smithi101 ceph-mon[128136]: pgmap v584: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:11.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:11 smithi005 ceph-mon[126460]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:11.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:11 smithi156 ceph-mon[128316]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:12.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:11 smithi101 ceph-mon[128136]: pgmap v585: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 211 KiB/s rd, 31 MiB/s wr, 256 op/s 2023-07-21T19:26:13.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:13 smithi005 ceph-mon[126460]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 242 op/s 2023-07-21T19:26:13.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:13 smithi005 ceph-mon[126460]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:26:13.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:13 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:26:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:13 smithi156 ceph-mon[128316]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 242 op/s 2023-07-21T19:26:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:13 smithi156 ceph-mon[128316]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:26:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:13 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:26:14.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:13 smithi101 ceph-mon[128136]: pgmap v586: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 242 op/s 2023-07-21T19:26:14.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:13 smithi101 ceph-mon[128136]: osdmap e114: 12 total, 12 up, 12 in 2023-07-21T19:26:14.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:13 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.437+0000 7f547f59e700 1 -- 172.21.15.5:0/2173851360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5480072850 msgr2=0x7f5480072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.437+0000 7f547f59e700 1 --2- 172.21.15.5:0/2173851360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5480072850 0x7f5480072c20 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f5470009990 tx=0x7f5470009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 -- 172.21.15.5:0/2173851360 shutdown_connections 2023-07-21T19:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 --2- 172.21.15.5:0/2173851360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f548010c3d0 0x7f548010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 --2- 172.21.15.5:0/2173851360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5480072850 0x7f5480072c20 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 --2- 172.21.15.5:0/2173851360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5480073160 0x7f548010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 -- 172.21.15.5:0/2173851360 >> 172.21.15.5:0/2173851360 conn(0x7f54800fb5f0 msgr2=0x7f54800fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 -- 172.21.15.5:0/2173851360 shutdown_connections 2023-07-21T19:26:15.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.438+0000 7f547f59e700 1 -- 172.21.15.5:0/2173851360 wait complete. 2023-07-21T19:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.441+0000 7f547f59e700 1 Processor -- start 2023-07-21T19:26:15.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.442+0000 7f547f59e700 1 -- start start 2023-07-21T19:26:15.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.442+0000 7f547f59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5480072850 0x7f548010b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.442+0000 7f547f59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547f59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548010c3d0 0x7f5480108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547f59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480109400 con 0x7f548010c3d0 2023-07-21T19:26:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547f59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480109570 con 0x7f5480073160 2023-07-21T19:26:15.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547f59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5480109730 con 0x7f5480072850 2023-07-21T19:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548010c3d0 0x7f5480108ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34816/0 (socket says 172.21.15.5:34816) 2023-07-21T19:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.443+0000 7f547dd9b700 1 -- 172.21.15.5:0/2072864235 learned_addr learned my addr 172.21.15.5:0/2072864235 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:15.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 -- 172.21.15.5:0/2072864235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5480072850 msgr2=0x7f548010b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5480072850 0x7f548010b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 -- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548010c3d0 msgr2=0x7f5480108ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548010c3d0 0x7f5480108ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 -- 172.21.15.5:0/2072864235 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54700095f0 con 0x7f5480073160 2023-07-21T19:26:15.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547dd9b700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f547400e9a0 tx=0x7f547400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f546f7fe700 1 -- 172.21.15.5:0/2072864235 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474009790 con 0x7f5480073160 2023-07-21T19:26:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.444+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54801a8aa0 con 0x7f5480073160 2023-07-21T19:26:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.445+0000 7f546f7fe700 1 -- 172.21.15.5:0/2072864235 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f547400cbd0 con 0x7f5480073160 2023-07-21T19:26:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.445+0000 7f546f7fe700 1 -- 172.21.15.5:0/2072864235 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5474018680 con 0x7f5480073160 2023-07-21T19:26:15.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.445+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54801a8fd0 con 0x7f5480073160 2023-07-21T19:26:15.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.449+0000 7f546f7fe700 1 -- 172.21.15.5:0/2072864235 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f547400cd40 con 0x7f5480073160 2023-07-21T19:26:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.450+0000 7f54987dfb80 1 Processor -- start 2023-07-21T19:26:15.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.450+0000 7f54987dfb80 1 -- start start 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.450+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 0x55d656939f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.450+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 0x55d6569341a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656934700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f54987dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d656839c50 con 0x55d65693a470 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f54987dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65688c9e0 con 0x55d65693a8c0 2023-07-21T19:26:15.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f54987dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d656885e70 con 0x55d6567906b0 2023-07-21T19:26:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f547ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 0x55d656939f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f547e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 0x55d6569341a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f547ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 0x55d656939f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58832/0 (socket says 172.21.15.5:58832) 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f547ed9d700 1 -- 172.21.15.5:0/4255861988 learned_addr learned my addr 172.21.15.5:0/4255861988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.451+0000 7f547dd9b700 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656934700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 -- 172.21.15.5:0/4255861988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 msgr2=0x55d656939f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 0x55d656939f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 -- 172.21.15.5:0/4255861988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 msgr2=0x55d656934700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656934700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 -- 172.21.15.5:0/4255861988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6566b3d80 con 0x55d65693a470 2023-07-21T19:26:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f547e59c700 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 0x55d6569341a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5470005910 tx=0x7f5470005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e73de26182ccee server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f546f7fe700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 0x7f54680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f546f7fe700 1 -- 172.21.15.5:0/2072864235 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5474014070 con 0x7f5480073160 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f546cff9700 1 -- 172.21.15.5:0/4255861988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547001d070 con 0x55d65693a470 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.452+0000 7f546cff9700 1 -- 172.21.15.5:0/4255861988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5470004cc0 con 0x55d65693a470 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.453+0000 7f546cff9700 1 -- 172.21.15.5:0/4255861988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547000b710 con 0x55d65693a470 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.453+0000 7f547ed9d700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 0x7f54680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.453+0000 7f54987dfb80 1 -- 172.21.15.5:0/4255861988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 msgr2=0x55d6569341a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.453+0000 7f547ed9d700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 0x7f54680a37f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5478009790 tx=0x7f5478009130 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.453+0000 7f54987dfb80 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 0x55d6569341a0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5470005910 tx=0x7f5470005c00 comp rx=0 tx=0).stop 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 -- 172.21.15.5:0/4255861988 shutdown_connections 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656934700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d6567906b0 0x55d656939f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 --2- 172.21.15.5:0/4255861988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d65693a470 0x55d6569341a0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 -- 172.21.15.5:0/4255861988 >> 172.21.15.5:0/4255861988 conn(0x55d656929d40 msgr2=0x55d65692f340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 -- 172.21.15.5:0/4255861988 shutdown_connections 2023-07-21T19:26:15.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 -- 172.21.15.5:0/4255861988 wait complete. 2023-07-21T19:26:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.454+0000 7f54987dfb80 1 -- start start 2023-07-21T19:26:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.455+0000 7f54987dfb80 1 Processor -- start 2023-07-21T19:26:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d656943800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 0x55d65693da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d656825a60 con 0x55d6567906b0 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d656825760 con 0x55d65693a470 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6567098c0 con 0x55d65693a8c0 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f54987dfb80 10 client.0 _ll_register_callbacks cb 0x55d656823700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f547dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f547dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34834/0 (socket says 172.21.15.5:34834) 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f547dd9b700 1 -- 172.21.15.5:0/3469242789 learned_addr learned my addr 172.21.15.5:0/3469242789 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.456+0000 7f547ed9d700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 0x55d65693da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547e59c700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d656943800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 -- 172.21.15.5:0/3469242789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 msgr2=0x55d65693da90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 0x55d65693da90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 -- 172.21.15.5:0/3469242789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 msgr2=0x55d656943800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d656943800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 -- 172.21.15.5:0/3469242789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548010f3f0 con 0x55d65693a470 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547ed9d700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 0x55d65693da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f547dd9b700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f547400c5b0 tx=0x7f5474099bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=db72583a7236e1ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f5465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547409a090 con 0x55d65693a470 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6567df490 con 0x55d65693a470 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.457+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f547409e0c0 con 0x55d65693a470 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f54740a24e0 con 0x55d65693a470 2023-07-21T19:26:15.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 10 client.28847 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d65669cc00 con 0x55d65693a470 2023-07-21T19:26:15.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d6567df490 con 0x55d65693a470 2023-07-21T19:26:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f5474095e80 con 0x55d65693a470 2023-07-21T19:26:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 10 client.28847 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 10 client.28847 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:15.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 20 client.28847 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.458+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d65694dc70 0x55d656950080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.459+0000 7f54987dfb80 4 client.28847 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.459+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d65687ee50 con 0x55d65694dc70 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.459+0000 7f547e59c700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d65694dc70 0x55d656950080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.460+0000 7f5465ffb700 10 client.28847 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.460+0000 7f547e59c700 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d65694dc70 0x55d656950080 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68f14c91cff04a72 server_cookie=bcd94130c32bba11 in_seq=0 out_seq=0 2023-07-21T19:26:15.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.460+0000 7f5465ffb700 1 -- 172.21.15.5:0/3469242789 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d65687ee50 con 0x55d65694dc70 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.460+0000 7f5465ffb700 10 client.28847 handle_command_reply: tid=0 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 client.28847 shutdown 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 msgr2=0x55d656943d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f547400c5b0 tx=0x7f5474099bd0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 shutdown_connections 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a8c0 0x55d65693da90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d65694dc70 0x55d656950080 crc :-1 s=READY pgs=375 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d656943800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3469242789 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a470 0x55d656943d40 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 >> 172.21.15.5:0/3469242789 conn(0x55d656929d40 msgr2=0x55d65692e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f5465ffb700 0 client.28847 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 shutdown_connections 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 1 -- 172.21.15.5:0/3469242789 wait complete. 2023-07-21T19:26:15.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.461+0000 7f54987dfb80 20 client.28847 trim_cache size 0 max 16384 2023-07-21T19:26:15.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.479+0000 7f54987dfb80 1 Processor -- start 2023-07-21T19:26:15.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 -- start start 2023-07-21T19:26:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569392a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d65692edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d656934ea0 con 0x55d6567906b0 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65692b750 con 0x55d65693a8c0 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f54987dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d65693bff0 con 0x55d65693a470 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f547e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f547e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37868/0 (socket says 172.21.15.5:37868) 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f547e59c700 1 -- 172.21.15.5:0/3888372652 learned_addr learned my addr 172.21.15.5:0/3888372652 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f547dd9b700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569392a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.480+0000 7f547ed9d700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d65692edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 -- 172.21.15.5:0/3888372652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 msgr2=0x55d6569392a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569392a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 -- 172.21.15.5:0/3888372652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 msgr2=0x55d65692edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d65692edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 -- 172.21.15.5:0/3888372652 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6566da1f0 con 0x55d6567906b0 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547dd9b700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:15.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f547e59c700 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f547000b2c0 tx=0x7f5470005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9abd162b7d031557 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f546d7fa700 1 -- 172.21.15.5:0/3888372652 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547001d040 con 0x55d6567906b0 2023-07-21T19:26:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f546d7fa700 1 -- 172.21.15.5:0/3888372652 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f547001e040 con 0x55d6567906b0 2023-07-21T19:26:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.481+0000 7f546d7fa700 1 -- 172.21.15.5:0/3888372652 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547002c050 con 0x55d6567906b0 2023-07-21T19:26:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 -- 172.21.15.5:0/3888372652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 msgr2=0x55d65693dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f547000b2c0 tx=0x7f5470005c70 comp rx=0 tx=0).stop 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 -- 172.21.15.5:0/3888372652 shutdown_connections 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d65692edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d65693dbc0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 --2- 172.21.15.5:0/3888372652 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569392a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 -- 172.21.15.5:0/3888372652 >> 172.21.15.5:0/3888372652 conn(0x55d656929d40 msgr2=0x55d656938b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 -- 172.21.15.5:0/3888372652 shutdown_connections 2023-07-21T19:26:15.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.482+0000 7f54987dfb80 1 -- 172.21.15.5:0/3888372652 wait complete. 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.483+0000 7f54987dfb80 1 -- start start 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.483+0000 7f54987dfb80 1 Processor -- start 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569418f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656941e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6569a0b30 con 0x55d6567906b0 2023-07-21T19:26:15.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6569a0c70 con 0x55d65693a8c0 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6569a0e00 con 0x55d65693a470 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f54987dfb80 10 client.0 _ll_register_callbacks cb 0x55d656823700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f547e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f547e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:37876/0 (socket says 172.21.15.5:37876) 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f547e59c700 1 -- 172.21.15.5:0/466341692 learned_addr learned my addr 172.21.15.5:0/466341692 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f547ed9d700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656941e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.484+0000 7f547dd9b700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569418f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 -- 172.21.15.5:0/466341692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 msgr2=0x55d6569418f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569418f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 -- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 msgr2=0x55d656941e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656941e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 -- 172.21.15.5:0/466341692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d65669cc00 con 0x55d6567906b0 2023-07-21T19:26:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f547e59c700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5470004f50 tx=0x7f547002cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=339900e9b5b2b9de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:15.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.485+0000 7f5465ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f547001d050 con 0x55d6567906b0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5470004cc0 con 0x55d6567906b0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f54700049e0 con 0x55d6567906b0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5470005340 con 0x55d6567906b0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 10 client.19221 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d6566b3d80 con 0x55d6567906b0 2023-07-21T19:26:15.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5470004cc0 con 0x55d6567906b0 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f54700099c0 con 0x55d6567906b0 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 10 client.19221 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 10 client.19221 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 20 client.19221 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d6569a20e0 0x55d6569a44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 4 client.19221 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:26:15.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.486+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d656840ff0 con 0x55d6569a20e0 2023-07-21T19:26:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.487+0000 7f547dd9b700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d6569a20e0 0x55d6569a44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.487+0000 7f547dd9b700 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d6569a20e0 0x55d6569a44c0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b79007fdc3e1005 server_cookie=d78cd06e5d603908 in_seq=0 out_seq=0 2023-07-21T19:26:15.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.487+0000 7f5465ffb700 10 client.19221 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f5465ffb700 1 -- 172.21.15.5:0/466341692 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d656840ff0 con 0x55d6569a20e0 2023-07-21T19:26:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f5465ffb700 10 client.19221 handle_command_reply: tid=0 2023-07-21T19:26:15.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 client.19221 shutdown 2023-07-21T19:26:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 msgr2=0x55d6569413b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f5470004f50 tx=0x7f547002cae0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 shutdown_connections 2023-07-21T19:26:15.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d65693a8c0 0x55d656941e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d6569a20e0 0x55d6569a44c0 crc :-1 s=READY pgs=376 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d6567906b0 0x55d6569413b0 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 --2- 172.21.15.5:0/466341692 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d65693a470 0x55d6569418f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 >> 172.21.15.5:0/466341692 conn(0x55d656929d40 msgr2=0x55d65693cad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f5465ffb700 0 client.19221 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 shutdown_connections 2023-07-21T19:26:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.502+0000 7f54987dfb80 1 -- 172.21.15.5:0/466341692 wait complete. 2023-07-21T19:26:15.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.503+0000 7f54987dfb80 20 client.19221 trim_cache size 0 max 16384 2023-07-21T19:26:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:26:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:26:15.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:26:15.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:26:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.503+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 msgr2=0x7f54680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.503+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 0x7f54680a37f0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5478009790 tx=0x7f5478009130 comp rx=0 tx=0).stop 2023-07-21T19:26:15.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.503+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 msgr2=0x7f5480108980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.503+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f547400e9a0 tx=0x7f547400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 shutdown_connections 2023-07-21T19:26:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f54680a1340 0x7f54680a37f0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f548010c3d0 0x7f5480108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5480072850 0x7f548010b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 --2- 172.21.15.5:0/2072864235 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5480073160 0x7f5480108980 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 >> 172.21.15.5:0/2072864235 conn(0x7f54800fb5f0 msgr2=0x7f548010e530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:15.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 shutdown_connections 2023-07-21T19:26:15.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:15.504+0000 7f547f59e700 1 -- 172.21.15.5:0/2072864235 wait complete. 2023-07-21T19:26:15.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:15 smithi005 ceph-mon[126460]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 38 MiB/s wr, 292 op/s 2023-07-21T19:26:15.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:15 smithi156 ceph-mon[128316]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 38 MiB/s wr, 292 op/s 2023-07-21T19:26:16.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:15 smithi101 ceph-mon[128136]: pgmap v588: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 260 KiB/s rd, 38 MiB/s wr, 292 op/s 2023-07-21T19:26:16.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:26:16.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:26:16.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:26:16.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:26:16.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:26:16.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:16 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:26:16.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:26:16.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:26:16.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:26:16.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:26:16.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:26:16.911 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:16 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:26:17.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:26:17.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:26:17.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:26:17.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:26:17.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:26:17.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:16 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:26:17.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:17 smithi005 ceph-mon[126460]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 235 op/s 2023-07-21T19:26:17.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:17 smithi156 ceph-mon[128316]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 235 op/s 2023-07-21T19:26:18.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:17 smithi101 ceph-mon[128136]: pgmap v589: 97 pgs: 97 active+clean; 15 GiB data, 8.4 GiB used, 1.0 TiB / 1.0 TiB avail; 209 KiB/s rd, 30 MiB/s wr, 235 op/s 2023-07-21T19:26:19.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:19 smithi005 ceph-mon[126460]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:19.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:19 smithi156 ceph-mon[128316]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:20.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:19 smithi101 ceph-mon[128136]: pgmap v590: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:21.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:21 smithi005 ceph-mon[126460]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:21.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:21 smithi156 ceph-mon[128316]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:21 smithi101 ceph-mon[128136]: pgmap v591: 97 pgs: 97 active+clean; 15 GiB data, 8.6 GiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 33 MiB/s wr, 87 op/s 2023-07-21T19:26:23.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:23 smithi005 ceph-mon[126460]: pgmap v592: 97 pgs: 97 active+clean; 16 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 34 MiB/s wr, 76 op/s 2023-07-21T19:26:23.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:23 smithi156 ceph-mon[128316]: pgmap v592: 97 pgs: 97 active+clean; 16 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 34 MiB/s wr, 76 op/s 2023-07-21T19:26:24.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:23 smithi101 ceph-mon[128136]: pgmap v592: 97 pgs: 97 active+clean; 16 GiB data, 8.7 GiB used, 1.0 TiB / 1.0 TiB avail; 153 KiB/s rd, 34 MiB/s wr, 76 op/s 2023-07-21T19:26:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:25 smithi005 ceph-mon[126460]: pgmap v593: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-07-21T19:26:25.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:25 smithi156 ceph-mon[128316]: pgmap v593: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-07-21T19:26:26.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:25 smithi101 ceph-mon[128136]: pgmap v593: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 29 MiB/s wr, 63 op/s 2023-07-21T19:26:27.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:27 smithi005 ceph-mon[126460]: pgmap v594: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-07-21T19:26:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:27 smithi156 ceph-mon[128316]: pgmap v594: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-07-21T19:26:28.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:27 smithi101 ceph-mon[128136]: pgmap v594: 97 pgs: 97 active+clean; 16 GiB data, 8.8 GiB used, 1.0 TiB / 1.0 TiB avail; 107 KiB/s rd, 28 MiB/s wr, 62 op/s 2023-07-21T19:26:28.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 -- 172.21.15.5:0/4255896070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c100500 msgr2=0x7fd30c10c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 --2- 172.21.15.5:0/4255896070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c100500 0x7fd30c10c940 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fd304009f90 tx=0x7fd30400b330 comp rx=0 tx=0).stop 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 -- 172.21.15.5:0/4255896070 shutdown_connections 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 --2- 172.21.15.5:0/4255896070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c100500 0x7fd30c10c940 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 --2- 172.21.15.5:0/4255896070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c105f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 --2- 172.21.15.5:0/4255896070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c0ffb50 0x7fd30c0fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 -- 172.21.15.5:0/4255896070 >> 172.21.15.5:0/4255896070 conn(0x7fd30c0fb650 msgr2=0x7fd30c0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 -- 172.21.15.5:0/4255896070 shutdown_connections 2023-07-21T19:26:28.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.709+0000 7fd311dff700 1 -- 172.21.15.5:0/4255896070 wait complete. 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.710+0000 7fd311dff700 1 Processor -- start 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 -- start start 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 0x7fd30c1983f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c19ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c19d0a0 con 0x7fd30c105b80 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c19d210 con 0x7fd30c0ffb50 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.711+0000 7fd311dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c19d3d0 con 0x7fd30c100500 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38452/0 (socket says 172.21.15.5:38452) 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30b7fe700 1 -- 172.21.15.5:0/2386370869 learned_addr learned my addr 172.21.15.5:0/2386370869 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 0x7fd30c1983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30bfff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c19ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 -- 172.21.15.5:0/2386370869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 msgr2=0x7fd30c197eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 -- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 msgr2=0x7fd30c19ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c19ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30affd700 1 -- 172.21.15.5:0/2386370869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2fc0095f0 con 0x7fd30c100500 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.712+0000 7fd30bfff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd30affd700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 0x7fd30c1983f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9f0 tx=0x7fd30000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd308ff9700 1 -- 172.21.15.5:0/2386370869 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30000c9c0 con 0x7fd30c100500 2023-07-21T19:26:28.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd308ff9700 1 -- 172.21.15.5:0/2386370869 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd300010430 con 0x7fd30c100500 2023-07-21T19:26:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd304009c40 con 0x7fd30c100500 2023-07-21T19:26:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd308ff9700 1 -- 172.21.15.5:0/2386370869 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300018680 con 0x7fd30c100500 2023-07-21T19:26:28.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.713+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c19d8a0 con 0x7fd30c100500 2023-07-21T19:26:28.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.717+0000 7fd308ff9700 1 -- 172.21.15.5:0/2386370869 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd3000105a0 con 0x7fd30c100500 2023-07-21T19:26:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.719+0000 7fd3257c2b80 1 Processor -- start 2023-07-21T19:26:28.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.719+0000 7fd3257c2b80 1 -- start start 2023-07-21T19:26:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.719+0000 7fd308ff9700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 0x7fd2f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd308ff9700 1 -- 172.21.15.5:0/2386370869 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd300014070 con 0x7fd30c100500 2023-07-21T19:26:28.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb1bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb169d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87ca1bc50 con 0x55e87c9726b0 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87ca6e9e0 con 0x55e87cb1c2d0 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87ca67e70 con 0x55e87cb16580 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd30bfff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 0x7fd2f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd30affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb1bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd30affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38460/0 (socket says 172.21.15.5:38460) 2023-07-21T19:26:28.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.720+0000 7fd30affd700 1 -- 172.21.15.5:0/509933552 learned_addr learned my addr 172.21.15.5:0/509933552 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30bfff700 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb169d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 -- 172.21.15.5:0/509933552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 msgr2=0x55e87cb169d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb169d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 -- 172.21.15.5:0/509933552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 msgr2=0x55e87cb1bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb1bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 -- 172.21.15.5:0/509933552 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e87c895d80 con 0x55e87cb1c2d0 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30bfff700 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30affd700 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd3000a1da0 tx=0x7fd300099d50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc71315d84fe7fb9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.721+0000 7fd30bfff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 0x7fd2f40a37f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd304009790 tx=0x7fd3040094e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.722+0000 7fd2f1ffb700 1 -- 172.21.15.5:0/509933552 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300010880 con 0x55e87cb1c2d0 2023-07-21T19:26:28.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.722+0000 7fd2f1ffb700 1 -- 172.21.15.5:0/509933552 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd300099e80 con 0x55e87cb1c2d0 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.722+0000 7fd2f1ffb700 1 -- 172.21.15.5:0/509933552 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000995d0 con 0x55e87cb1c2d0 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.722+0000 7fd3257c2b80 1 -- 172.21.15.5:0/509933552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 msgr2=0x55e87cb16040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.722+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fd3000a1da0 tx=0x7fd300099d50 comp rx=0 tx=0).stop 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 -- 172.21.15.5:0/509933552 shutdown_connections 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb169d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb1bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/509933552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb16040 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 -- 172.21.15.5:0/509933552 >> 172.21.15.5:0/509933552 conn(0x55e87cb0bbe0 msgr2=0x55e87cb10f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 -- 172.21.15.5:0/509933552 shutdown_connections 2023-07-21T19:26:28.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.723+0000 7fd3257c2b80 1 -- 172.21.15.5:0/509933552 wait complete. 2023-07-21T19:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.724+0000 7fd3257c2b80 1 -- start start 2023-07-21T19:26:28.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.724+0000 7fd3257c2b80 1 Processor -- start 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb22650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb1f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87ca07a60 con 0x55e87c9726b0 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87ca07760 con 0x55e87cb1c2d0 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87c8eb8c0 con 0x55e87cb16580 2023-07-21T19:26:28.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd3257c2b80 10 client.0 _ll_register_callbacks cb 0x55e87ca05700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd30bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd30bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38470/0 (socket says 172.21.15.5:38470) 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd30bfff700 1 -- 172.21.15.5:0/73333437 learned_addr learned my addr 172.21.15.5:0/73333437 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb22650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.725+0000 7fd30affd700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb1f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 -- 172.21.15.5:0/73333437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 msgr2=0x55e87cb1f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb1f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 -- 172.21.15.5:0/73333437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 msgr2=0x55e87cb22650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb22650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 -- 172.21.15.5:0/73333437 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c19d8a0 con 0x55e87cb1c2d0 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd30bfff700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd304003d80 tx=0x7fd30400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7c3c5a98ce7064 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd2eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400e040 con 0x55e87cb1c2d0 2023-07-21T19:26:28.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e87c9c1490 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.726+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd304024070 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd30400be80 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 10 client.28865 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e87c87ec00 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e87c9c1490 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd30401e760 con 0x55e87cb1c2d0 2023-07-21T19:26:28.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 10 client.28865 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 10 client.28865 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 20 client.28865 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.727+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb2fc10 0x55e87cb32020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.728+0000 7fd3257c2b80 4 client.28865 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:28.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.728+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e87ca60e50 con 0x55e87cb2fc10 2023-07-21T19:26:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.728+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb2fc10 0x55e87cb32020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.729+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb2fc10 0x55e87cb32020 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f40497cee3096d7 server_cookie=c7d9b69dad64bf83 in_seq=0 out_seq=0 2023-07-21T19:26:28.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.729+0000 7fd2eaffd700 10 client.28865 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.729+0000 7fd2eaffd700 1 -- 172.21.15.5:0/73333437 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e87ca60e50 con 0x55e87cb2fc10 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.729+0000 7fd2eaffd700 10 client.28865 handle_command_reply: tid=0 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 client.28865 shutdown 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 msgr2=0x55e87cb1ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fd304003d80 tx=0x7fd30400b330 comp rx=0 tx=0).stop 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 shutdown_connections 2023-07-21T19:26:28.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1ffa0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb2fc10 0x55e87cb32020 crc :-1 s=READY pgs=377 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb22650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/73333437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb16580 0x55e87cb1f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 >> 172.21.15.5:0/73333437 conn(0x55e87cb0bbe0 msgr2=0x55e87cb104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd2eaffd700 0 client.28865 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 shutdown_connections 2023-07-21T19:26:28.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.730+0000 7fd3257c2b80 1 -- 172.21.15.5:0/73333437 wait complete. 2023-07-21T19:26:28.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.731+0000 7fd3257c2b80 20 client.28865 trim_cache size 0 max 16384 2023-07-21T19:26:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 Processor -- start 2023-07-21T19:26:28.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 -- start start 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87c9726b0 0x55e87cb16040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb20970 con 0x55e87cb1c2d0 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb0d5f0 con 0x55e87cb16580 2023-07-21T19:26:28.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.749+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb20300 con 0x55e87c9726b0 2023-07-21T19:26:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38476/0 (socket says 172.21.15.5:38476) 2023-07-21T19:26:28.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 -- 172.21.15.5:0/1676848718 learned_addr learned my addr 172.21.15.5:0/1676848718 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87c9726b0 0x55e87cb16040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30bfff700 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 -- 172.21.15.5:0/1676848718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87c9726b0 msgr2=0x55e87cb16040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87c9726b0 0x55e87cb16040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 -- 172.21.15.5:0/1676848718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87cb1c2d0 msgr2=0x55e87cb1b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1b0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.750+0000 7fd30affd700 1 -- 172.21.15.5:0/1676848718 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e87c8bc1f0 con 0x55e87cb16580 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.751+0000 7fd30affd700 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd3000ca5b0 tx=0x7fd300010b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=378f2b7ed2a54c00 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.751+0000 7fd2f27fc700 1 -- 172.21.15.5:0/1676848718 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b8070 con 0x55e87cb16580 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.751+0000 7fd2f27fc700 1 -- 172.21.15.5:0/1676848718 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd30009deb0 con 0x55e87cb16580 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.751+0000 7fd2f27fc700 1 -- 172.21.15.5:0/1676848718 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd300099640 con 0x55e87cb16580 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 -- 172.21.15.5:0/1676848718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 msgr2=0x55e87cb26c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd3000ca5b0 tx=0x7fd300010b80 comp rx=0 tx=0).stop 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 -- 172.21.15.5:0/1676848718 shutdown_connections 2023-07-21T19:26:28.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87cb1c2d0 0x55e87cb1b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87c9726b0 0x55e87cb16040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/1676848718 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb26c60 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 -- 172.21.15.5:0/1676848718 >> 172.21.15.5:0/1676848718 conn(0x55e87cb0bbe0 msgr2=0x55e87cb1a9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 -- 172.21.15.5:0/1676848718 shutdown_connections 2023-07-21T19:26:28.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.752+0000 7fd3257c2b80 1 -- 172.21.15.5:0/1676848718 wait complete. 2023-07-21T19:26:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.753+0000 7fd3257c2b80 1 -- start start 2023-07-21T19:26:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.753+0000 7fd3257c2b80 1 Processor -- start 2023-07-21T19:26:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.753+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb19140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb1c2d0 0x55e87cb17790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb17cd0 con 0x55e87c9726b0 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb17e10 con 0x55e87cb16580 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e87cb17fd0 con 0x55e87cb1c2d0 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd3257c2b80 10 client.0 _ll_register_callbacks cb 0x7fd304008f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb19140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49494/0 (socket says 172.21.15.5:49494) 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30b7fe700 1 -- 172.21.15.5:0/554574570 learned_addr learned my addr 172.21.15.5:0/554574570 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30affd700 1 -- 172.21.15.5:0/554574570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb1c2d0 msgr2=0x55e87cb17790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.754+0000 7fd30affd700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb1c2d0 0x55e87cb17790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd30affd700 1 -- 172.21.15.5:0/554574570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 msgr2=0x55e87cb18c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd30affd700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd30affd700 1 -- 172.21.15.5:0/554574570 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e87c895d80 con 0x55e87cb16580 2023-07-21T19:26:28.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd30affd700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb19140 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e87cb0d730 tx=0x7fd300099d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7cc44aca8343153 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd2eaffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3000b8040 con 0x55e87cb16580 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd300099370 con 0x55e87cb16580 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd300099e80 con 0x55e87cb16580 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd300099530 con 0x55e87cb16580 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd3257c2b80 10 client.28871 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:28.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.755+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e87c894250 con 0x55e87cb16580 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd300099e80 con 0x55e87cb16580 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd3000adb80 con 0x55e87cb16580 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 10 client.28871 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 10 client.28871 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 20 client.28871 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb84500 0x55e87cb868e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 4 client.28871 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:26:28.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e87ca22ff0 con 0x55e87cb84500 2023-07-21T19:26:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.756+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb84500 0x55e87cb868e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.757+0000 7fd30b7fe700 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb84500 0x55e87cb868e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2297058c477ee53f server_cookie=836c8dd789f89925 in_seq=0 out_seq=0 2023-07-21T19:26:28.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.757+0000 7fd2eaffd700 10 client.28871 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.773+0000 7fd2eaffd700 1 -- 172.21.15.5:0/554574570 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e87ca22ff0 con 0x55e87cb84500 2023-07-21T19:26:28.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.773+0000 7fd2eaffd700 10 client.28871 handle_command_reply: tid=0 2023-07-21T19:26:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.773+0000 7fd3257c2b80 1 client.28871 shutdown 2023-07-21T19:26:28.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 msgr2=0x55e87cb19140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb19140 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x55e87cb0d730 tx=0x7fd300099d20 comp rx=0 tx=0).stop 2023-07-21T19:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 shutdown_connections 2023-07-21T19:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e87cb1c2d0 0x55e87cb17790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55e87cb84500 0x55e87cb868e0 crc :-1 s=READY pgs=378 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e87c9726b0 0x55e87cb18c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 --2- 172.21.15.5:0/554574570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e87cb16580 0x55e87cb19140 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd2eaffd700 0 client.28871 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:28.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 >> 172.21.15.5:0/554574570 conn(0x55e87cb0bbe0 msgr2=0x55e87cb1c900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 shutdown_connections 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 1 -- 172.21.15.5:0/554574570 wait complete. 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.774+0000 7fd3257c2b80 20 client.28871 trim_cache size 0 max 16384 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:26:28.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:26:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:26:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 msgr2=0x7fd2f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 0x7fd2f40a37f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fd304009790 tx=0x7fd3040094e0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 msgr2=0x7fd30c1983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 0x7fd30c1983f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fd30000e9f0 tx=0x7fd30000ed00 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 shutdown_connections 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd2f40a1340 0x7fd2f40a37f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd30c105b80 0x7fd30c19ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd30c0ffb50 0x7fd30c197eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 --2- 172.21.15.5:0/2386370869 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd30c100500 0x7fd30c1983f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:28.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 >> 172.21.15.5:0/2386370869 conn(0x7fd30c0fb650 msgr2=0x7fd30c10ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 shutdown_connections 2023-07-21T19:26:28.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:28.775+0000 7fd311dff700 1 -- 172.21.15.5:0/2386370869 wait complete. 2023-07-21T19:26:29.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:29 smithi005 ceph-mon[126460]: pgmap v595: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 42 MiB/s wr, 127 op/s 2023-07-21T19:26:29.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:29 smithi156 ceph-mon[128316]: pgmap v595: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 42 MiB/s wr, 127 op/s 2023-07-21T19:26:30.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:29 smithi101 ceph-mon[128136]: pgmap v595: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 186 KiB/s rd, 42 MiB/s wr, 127 op/s 2023-07-21T19:26:31.516 INFO:teuthology.orchestra.run.smithi005.stderr:done. 2023-07-21T19:26:31.525 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo -u postgres -- pgbench -c 100 -j 4 --progress=5 --time=900' 2023-07-21T19:26:31.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:31 smithi005 ceph-mon[126460]: pgmap v596: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 27 MiB/s wr, 110 op/s 2023-07-21T19:26:31.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:31 smithi156 ceph-mon[128316]: pgmap v596: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 27 MiB/s wr, 110 op/s 2023-07-21T19:26:31.936 INFO:teuthology.orchestra.run.smithi005.stderr:starting vacuum...end. 2023-07-21T19:26:32.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:31 smithi101 ceph-mon[128136]: pgmap v596: 97 pgs: 97 active+clean; 16 GiB data, 8.9 GiB used, 1.0 TiB / 1.0 TiB avail; 173 KiB/s rd, 27 MiB/s wr, 110 op/s 2023-07-21T19:26:32.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:26:33.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 -- 172.21.15.5:0/1286158063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c100480 msgr2=0x7f7d7c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 --2- 172.21.15.5:0/1286158063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c100480 0x7f7d7c10c910 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7d7400b0a0 tx=0x7f7d7400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 -- 172.21.15.5:0/1286158063 shutdown_connections 2023-07-21T19:26:33.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 --2- 172.21.15.5:0/1286158063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c100480 0x7f7d7c10c910 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 --2- 172.21.15.5:0/1286158063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 --2- 172.21.15.5:0/1286158063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.021+0000 7f7d82fff700 1 -- 172.21.15.5:0/1286158063 >> 172.21.15.5:0/1286158063 conn(0x7f7d7c0fb610 msgr2=0x7f7d7c0fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.022+0000 7f7d82fff700 1 -- 172.21.15.5:0/1286158063 shutdown_connections 2023-07-21T19:26:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.022+0000 7f7d82fff700 1 -- 172.21.15.5:0/1286158063 wait complete. 2023-07-21T19:26:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.024+0000 7f7d82fff700 1 Processor -- start 2023-07-21T19:26:33.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.024+0000 7f7d82fff700 1 -- start start 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.024+0000 7f7d82fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c195cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d82fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d82fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d82fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c199bb0 con 0x7f7d7c0ffad0 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d82fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c199d20 con 0x7f7d7c100480 2023-07-21T19:26:33.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d82fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7c199ee0 con 0x7f7d7c105ae0 2023-07-21T19:26:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c195cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c195cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49520/0 (socket says 172.21.15.5:49520) 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d80d9b700 1 -- 172.21.15.5:0/3778138048 learned_addr learned my addr 172.21.15.5:0/3778138048 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c1995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33814/0 (socket says 172.21.15.5:33814) 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.025+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38512/0 (socket says 172.21.15.5:38512) 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 -- 172.21.15.5:0/3778138048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 msgr2=0x7f7d7c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 -- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 msgr2=0x7f7d7c195cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c195cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 -- 172.21.15.5:0/3778138048 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d74009d00 con 0x7f7d7c100480 2023-07-21T19:26:33.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d68013f40 tx=0x7f7d68013f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.026+0000 7f7d79ffb700 1 -- 172.21.15.5:0/3778138048 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d680114e0 con 0x7f7d7c100480 2023-07-21T19:26:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.027+0000 7f7d79ffb700 1 -- 172.21.15.5:0/3778138048 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d68011b40 con 0x7f7d7c100480 2023-07-21T19:26:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.027+0000 7f7d79ffb700 1 -- 172.21.15.5:0/3778138048 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d68014df0 con 0x7f7d7c100480 2023-07-21T19:26:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.027+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d7c19a160 con 0x7f7d7c100480 2023-07-21T19:26:33.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.027+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d7c072920 con 0x7f7d7c100480 2023-07-21T19:26:33.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.032+0000 7f7d79ffb700 1 -- 172.21.15.5:0/3778138048 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7d6800f850 con 0x7f7d7c100480 2023-07-21T19:26:33.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.033+0000 7f7d969c9b80 1 Processor -- start 2023-07-21T19:26:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.033+0000 7f7d969c9b80 1 -- start start 2023-07-21T19:26:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 0x557b969856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96990ad0 0x557b96992eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9688bcb0 con 0x557b96985be0 2023-07-21T19:26:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b968dea40 con 0x557b969852d0 2023-07-21T19:26:33.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.034+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b968d7ed0 con 0x557b96990ad0 2023-07-21T19:26:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 0x557b969856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49536/0 (socket says 172.21.15.5:49536) 2023-07-21T19:26:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d80d9b700 1 -- 172.21.15.5:0/1557718246 learned_addr learned my addr 172.21.15.5:0/1557718246 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96990ad0 0x557b96992eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 -- 172.21.15.5:0/1557718246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96990ad0 msgr2=0x557b96992eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96990ad0 0x557b96992eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 -- 172.21.15.5:0/1557718246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 msgr2=0x557b96990590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d8159c700 1 -- 172.21.15.5:0/1557718246 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b96705de0 con 0x557b969852d0 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.035+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:33.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d8159c700 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 0x557b969856a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009fd0 tx=0x7f7d74009410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29ab113c95459f07 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d79ffb700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 0x7f7d6c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d66ffd700 1 -- 172.21.15.5:0/1557718246 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d7400e040 con 0x557b969852d0 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d79ffb700 1 -- 172.21.15.5:0/3778138048 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7d680d3fe0 con 0x7f7d7c100480 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d66ffd700 1 -- 172.21.15.5:0/1557718246 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d74007d90 con 0x557b969852d0 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.036+0000 7f7d66ffd700 1 -- 172.21.15.5:0/1557718246 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d74003d70 con 0x557b969852d0 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.037+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 0x7f7d6c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.037+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1557718246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 msgr2=0x557b969856a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.037+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 0x557b969856a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f7d74009fd0 tx=0x7f7d74009410 comp rx=0 tx=0).stop 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1557718246 shutdown_connections 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96990ad0 0x557b96992eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969852d0 0x557b969856a0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1557718246 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96985be0 0x557b96990590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1557718246 >> 172.21.15.5:0/1557718246 conn(0x557b9697bda0 msgr2=0x557b969816a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 0x7f7d6c0a37f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7d70010f10 tx=0x7f7d70012040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1557718246 shutdown_connections 2023-07-21T19:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.038+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1557718246 wait complete. 2023-07-21T19:26:33.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.039+0000 7f7d969c9b80 1 -- start start 2023-07-21T19:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.039+0000 7f7d969c9b80 1 Processor -- start 2023-07-21T19:26:33.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96990580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b9698df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b96877ac0 con 0x557b969852d0 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b968777c0 con 0x557b96990ad0 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b9675b920 con 0x557b96985be0 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d969c9b80 10 client.0 _ll_register_callbacks cb 0x557b96875760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:33.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b9698df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96990580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d80d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96990580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49540/0 (socket says 172.21.15.5:49540) 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d80d9b700 1 -- 172.21.15.5:0/2866982682 learned_addr learned my addr 172.21.15.5:0/2866982682 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33828/0 (socket says 172.21.15.5:33828) 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.040+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b9698df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38528/0 (socket says 172.21.15.5:38528) 2023-07-21T19:26:33.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 -- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 msgr2=0x557b9698df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b9698df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 -- 172.21.15.5:0/2866982682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 msgr2=0x557b96990580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96990580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 -- 172.21.15.5:0/2866982682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d70010530 con 0x557b96985be0 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d680d1820 tx=0x7f7d680d1c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c754ae3df3c0967 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.041+0000 7f7d5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d680a0ce0 con 0x557b96985be0 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7d68011640 con 0x557b96985be0 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b968314f0 con 0x557b96985be0 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d680a0b00 con 0x557b96985be0 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d969c9b80 10 client.28795 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:33.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.042+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f7d7c10d210 con 0x557b96985be0 2023-07-21T19:26:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557b968314f0 con 0x557b96985be0 2023-07-21T19:26:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7d680a1400 con 0x557b96985be0 2023-07-21T19:26:33.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 10 client.28795 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 10 client.28795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 20 client.28795 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:33.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969a0390 0x557b969a27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 4 client.28795 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.043+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b968d0eb0 con 0x557b969a0390 2023-07-21T19:26:33.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.044+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969a0390 0x557b969a27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.045+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969a0390 0x557b969a27a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8e3a39f6e8bbc0a server_cookie=c12e63ad146dada7 in_seq=0 out_seq=0 2023-07-21T19:26:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.045+0000 7f7d5bfff700 10 client.28795 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:33.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.045+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2866982682 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557b968d0eb0 con 0x557b969a0390 2023-07-21T19:26:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.045+0000 7f7d5bfff700 10 client.28795 handle_command_reply: tid=0 2023-07-21T19:26:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 client.28795 shutdown 2023-07-21T19:26:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 msgr2=0x557b9698d800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7d680d1820 tx=0x7f7d680d1c70 comp rx=0 tx=0).stop 2023-07-21T19:26:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 shutdown_connections 2023-07-21T19:26:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b9698df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969a0390 0x557b969a27a0 crc :-1 s=READY pgs=379 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96990580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2866982682 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b96985be0 0x557b9698d800 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 >> 172.21.15.5:0/2866982682 conn(0x557b9697bda0 msgr2=0x557b96981230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d5bfff700 0 client.28795 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:33.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 shutdown_connections 2023-07-21T19:26:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.046+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2866982682 wait complete. 2023-07-21T19:26:33.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.047+0000 7f7d969c9b80 20 client.28795 trim_cache size 0 max 16384 2023-07-21T19:26:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 Processor -- start 2023-07-21T19:26:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 -- start start 2023-07-21T19:26:33.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96988b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b96989050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 0x557b9698cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b969a01c0 con 0x557b969852d0 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b96986540 con 0x557b96990ad0 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.075+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b96986d60 con 0x557b969818c0 2023-07-21T19:26:33.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 0x557b9698cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d8159c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 0x557b9698cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33840/0 (socket says 172.21.15.5:33840) 2023-07-21T19:26:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d8159c700 1 -- 172.21.15.5:0/1774059130 learned_addr learned my addr 172.21.15.5:0/1774059130 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96988b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b96989050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 -- 172.21.15.5:0/1774059130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 msgr2=0x557b9698cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 0x557b9698cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 -- 172.21.15.5:0/1774059130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 msgr2=0x557b96989050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b96989050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.076+0000 7f7d80d9b700 1 -- 172.21.15.5:0/1774059130 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b9697d5e0 con 0x557b969852d0 2023-07-21T19:26:33.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.077+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96988b10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x557b969a3080 tx=0x7f7d70004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=238f7ac7b34e3bf1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.078+0000 7f7d677fe700 1 -- 172.21.15.5:0/1774059130 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70029d00 con 0x557b969852d0 2023-07-21T19:26:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.078+0000 7f7d677fe700 1 -- 172.21.15.5:0/1774059130 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d70022040 con 0x557b969852d0 2023-07-21T19:26:33.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.078+0000 7f7d677fe700 1 -- 172.21.15.5:0/1774059130 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70003c50 con 0x557b969852d0 2023-07-21T19:26:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1774059130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 msgr2=0x557b96988b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96988b10 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x557b969a3080 tx=0x7f7d70004750 comp rx=0 tx=0).stop 2023-07-21T19:26:33.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1774059130 shutdown_connections 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969818c0 0x557b9698cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b969852d0 0x557b96988b10 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/1774059130 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b96990ad0 0x557b96989050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.079+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1774059130 >> 172.21.15.5:0/1774059130 conn(0x557b9697bda0 msgr2=0x557b969a23a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.080+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1774059130 shutdown_connections 2023-07-21T19:26:33.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.080+0000 7f7d969c9b80 1 -- 172.21.15.5:0/1774059130 wait complete. 2023-07-21T19:26:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.080+0000 7f7d969c9b80 1 -- start start 2023-07-21T19:26:33.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 Processor -- start 2023-07-21T19:26:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 0x557b96997660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96990ad0 0x557b96996060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b96996770 con 0x557b96990ad0 2023-07-21T19:26:33.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b969f4260 con 0x557b969818c0 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b969f43a0 con 0x557b969852d0 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.081+0000 7f7d969c9b80 10 client.0 _ll_register_callbacks cb 0x7f7d680d7780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d7bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33852/0 (socket says 172.21.15.5:33852) 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d7bfff700 1 -- 172.21.15.5:0/2111101854 learned_addr learned my addr 172.21.15.5:0/2111101854 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 0x557b96997660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d8159c700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96990ad0 0x557b96996060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 -- 172.21.15.5:0/2111101854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 msgr2=0x557b96995920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 -- 172.21.15.5:0/2111101854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96990ad0 msgr2=0x557b96996060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96990ad0 0x557b96996060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.082+0000 7f7d80d9b700 1 -- 172.21.15.5:0/2111101854 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d70010c40 con 0x557b969818c0 2023-07-21T19:26:33.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.083+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.083+0000 7f7d80d9b700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 0x557b96997660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x557b969a3080 tx=0x7f7d70029ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc2b675f5a75326f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.083+0000 7f7d5bfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d70029450 con 0x557b969818c0 2023-07-21T19:26:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7d70015040 con 0x557b969818c0 2023-07-21T19:26:33.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7d74007d90 con 0x557b969818c0 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7d70029e50 con 0x557b969818c0 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d969c9b80 10 client.28889 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.084+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x557b966eec60 con 0x557b969818c0 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7d70046080 con 0x557b969818c0 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7d70040030 con 0x557b969818c0 2023-07-21T19:26:33.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 10 client.28889 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 10 client.28889 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 20 client.28889 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969f5930 0x557b969f7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 4 client.28889 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.085+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b96893050 con 0x557b969f5930 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969f5930 0x557b969f7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d7bfff700 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969f5930 0x557b969f7d40 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d7ba3e5ea9cc470 server_cookie=c18419d289698556 in_seq=0 out_seq=0 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d5bfff700 10 client.28889 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d5bfff700 1 -- 172.21.15.5:0/2111101854 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x557b96893050 con 0x557b969f5930 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d5bfff700 10 client.28889 handle_command_reply: tid=0 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.086+0000 7f7d969c9b80 1 client.28889 shutdown 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 msgr2=0x557b96997660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 0x557b96997660 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x557b969a3080 tx=0x7f7d70029ba0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 shutdown_connections 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557b969f5930 0x557b969f7d40 crc :-1 s=READY pgs=380 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557b96990ad0 0x557b96996060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557b969818c0 0x557b96997660 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 --2- 172.21.15.5:0/2111101854 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557b969852d0 0x557b96995920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 >> 172.21.15.5:0/2111101854 conn(0x557b9697bda0 msgr2=0x557b96988b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d5bfff700 0 client.28889 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:33.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 shutdown_connections 2023-07-21T19:26:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.087+0000 7f7d969c9b80 1 -- 172.21.15.5:0/2111101854 wait complete. 2023-07-21T19:26:33.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.088+0000 7f7d969c9b80 20 client.28889 trim_cache size 0 max 16384 2023-07-21T19:26:33.081 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:26:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:26:33.082 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:26:33.082 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:26:33.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 msgr2=0x7f7d6c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 0x7f7d6c0a37f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f7d70010f10 tx=0x7f7d70012040 comp rx=0 tx=0).stop 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 msgr2=0x7f7d7c196200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f7d68013f40 tx=0x7f7d68013f70 comp rx=0 tx=0).stop 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 shutdown_connections 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7d7c105ae0 0x7f7d7c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7d6c0a1340 0x7f7d6c0a37f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7d7c0ffad0 0x7f7d7c195cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 --2- 172.21.15.5:0/3778138048 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7d7c100480 0x7f7d7c196200 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 >> 172.21.15.5:0/3778138048 conn(0x7f7d7c0fb610 msgr2=0x7f7d7c0fbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 shutdown_connections 2023-07-21T19:26:33.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.089+0000 7f7d82fff700 1 -- 172.21.15.5:0/3778138048 wait complete. 2023-07-21T19:26:33.105 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:26:33.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:26:33.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.434+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/4132774852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 msgr2=0x7fbf24110600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.434+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/4132774852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf24110600 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c0098f0 tx=0x7fbf1c009c00 comp rx=0 tx=0).stop 2023-07-21T19:26:33.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/4132774852 shutdown_connections 2023-07-21T19:26:33.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/4132774852 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 0x7fbf24112f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/4132774852 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 0x7fbf24072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/4132774852 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf24110600 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/4132774852 >> 172.21.15.5:0/4132774852 conn(0x7fbf240fb650 msgr2=0x7fbf240fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/4132774852 shutdown_connections 2023-07-21T19:26:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.435+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/4132774852 wait complete. 2023-07-21T19:26:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.436+0000 7fbf2c3ff700 1 Processor -- start 2023-07-21T19:26:33.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.437+0000 7fbf2c3ff700 1 -- start start 2023-07-21T19:26:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 0x7fbf2410be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 0x7fbf2419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2419fcc0 con 0x7fbf24110b40 2023-07-21T19:26:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2419fe30 con 0x7fbf240731c0 2023-07-21T19:26:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.438+0000 7fbf2c3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf2419fff0 con 0x7fbf240728b0 2023-07-21T19:26:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38562/0 (socket says 172.21.15.5:38562) 2023-07-21T19:26:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 -- 172.21.15.5:0/3350392712 learned_addr learned my addr 172.21.15.5:0/3350392712 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2a19b700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 0x7fbf2410be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2a99c700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 0x7fbf2419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 -- 172.21.15.5:0/3350392712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 msgr2=0x7fbf2410be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 0x7fbf2410be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 -- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 msgr2=0x7fbf2419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.430 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 0x7fbf2419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.431 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.439+0000 7fbf2999a700 1 -- 172.21.15.5:0/3350392712 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c009520 con 0x7fbf240731c0 2023-07-21T19:26:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf2999a700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009810 tx=0x7fbf1c00fa00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c01c070 con 0x7fbf240731c0 2023-07-21T19:26:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbf1c00bba0 con 0x7fbf240731c0 2023-07-21T19:26:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1c0049c0 con 0x7fbf240731c0 2023-07-21T19:26:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf241a0270 con 0x7fbf240731c0 2023-07-21T19:26:33.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.440+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf241a06d0 con 0x7fbf240731c0 2023-07-21T19:26:33.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.442+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf24103040 con 0x7fbf240731c0 2023-07-21T19:26:33.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.447+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbf1c020070 con 0x7fbf240731c0 2023-07-21T19:26:33.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.449+0000 7fbf1b7fe700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 0x7fbf0c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:33.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.449+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbf1c0cda80 con 0x7fbf240731c0 2023-07-21T19:26:33.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.450+0000 7fbf2a19b700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 0x7fbf0c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:33.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.451+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fbf1c096ec0 con 0x7fbf240731c0 2023-07-21T19:26:33.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.452+0000 7fbf2a19b700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 0x7fbf0c0a38b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf20020540 tx=0x7fbf2001c040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:33.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.889+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf2410d270 con 0x7fbf240731c0 2023-07-21T19:26:33.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.890+0000 7fbf1b7fe700 1 -- 172.21.15.5:0/3350392712 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+10839 (secure 0 0 0) 0x7fbf1c09a4e0 con 0x7fbf240731c0 2023-07-21T19:26:33.879 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:26:33.879 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:25:53.614690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:26:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 msgr2=0x7fbf0c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 0x7fbf0c0a38b0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fbf20020540 tx=0x7fbf2001c040 comp rx=0 tx=0).stop 2023-07-21T19:26:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 msgr2=0x7fbf2410c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:33.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fbf1c009810 tx=0x7fbf1c00fa00 comp rx=0 tx=0).stop 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 shutdown_connections 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbf0c0a1400 0x7fbf0c0a38b0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbf24110b40 0x7fbf2419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbf240728b0 0x7fbf2410be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 --2- 172.21.15.5:0/3350392712 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbf240731c0 0x7fbf2410c390 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 >> 172.21.15.5:0/3350392712 conn(0x7fbf240fb650 msgr2=0x7fbf240fc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 shutdown_connections 2023-07-21T19:26:33.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:33.893+0000 7fbf2c3ff700 1 -- 172.21.15.5:0/3350392712 wait complete. 2023-07-21T19:26:33.884 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 56 2023-07-21T19:26:33.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:33 smithi005 ceph-mon[126460]: pgmap v597: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 38 MiB/s wr, 179 op/s 2023-07-21T19:26:33.902 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:26:33.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:33 smithi156 ceph-mon[128316]: pgmap v597: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 38 MiB/s wr, 179 op/s 2023-07-21T19:26:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:33 smithi101 ceph-mon[128136]: pgmap v597: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 308 KiB/s rd, 38 MiB/s wr, 179 op/s 2023-07-21T19:26:34.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 -- 172.21.15.5:0/138818999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fdde0 msgr2=0x7f3ba80fe1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/138818999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80fe1b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3b98009990 tx=0x7f3b98009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 -- 172.21.15.5:0/138818999 shutdown_connections 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/138818999 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 0x7f3ba810c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/138818999 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80fe1b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/138818999 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fe780 0x7f3ba8104750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 -- 172.21.15.5:0/138818999 >> 172.21.15.5:0/138818999 conn(0x7f3ba8075b70 msgr2=0x7f3ba8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 -- 172.21.15.5:0/138818999 shutdown_connections 2023-07-21T19:26:34.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.272+0000 7f3bae5ff700 1 -- 172.21.15.5:0/138818999 wait complete. 2023-07-21T19:26:34.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.273+0000 7f3bae5ff700 1 Processor -- start 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.273+0000 7f3bae5ff700 1 -- start start 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 0x7f3ba81a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba8100350 con 0x7f3ba8104e80 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81a42e0 con 0x7f3ba80fdde0 2023-07-21T19:26:34.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bae5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba81a4420 con 0x7f3ba80fe780 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33896/0 (socket says 172.21.15.5:33896) 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 -- 172.21.15.5:0/1906348051 learned_addr learned my addr 172.21.15.5:0/1906348051 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 0x7f3ba81a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 -- 172.21.15.5:0/1906348051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 msgr2=0x7f3ba80ff5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80ff5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 -- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 msgr2=0x7f3ba81a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 0x7f3ba81a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba77fe700 1 -- 172.21.15.5:0/1906348051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b980095f0 con 0x7f3ba80fe780 2023-07-21T19:26:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.274+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:34.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d980 tx=0x7f3b9c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3ba57fa700 1 -- 172.21.15.5:0/1906348051 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00f890 con 0x7f3ba80fe780 2023-07-21T19:26:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3ba57fa700 1 -- 172.21.15.5:0/1906348051 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b9c00fe80 con 0x7f3ba80fe780 2023-07-21T19:26:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3ba57fa700 1 -- 172.21.15.5:0/1906348051 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00e3b0 con 0x7f3ba80fe780 2023-07-21T19:26:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba81a4680 con 0x7f3ba80fe780 2023-07-21T19:26:34.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.275+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba81a4b10 con 0x7f3ba80fe780 2023-07-21T19:26:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.279+0000 7f3ba57fa700 1 -- 172.21.15.5:0/1906348051 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3b9c00f9f0 con 0x7f3ba80fe780 2023-07-21T19:26:34.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 Processor -- start 2023-07-21T19:26:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 -- start start 2023-07-21T19:26:34.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3ba57fa700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 0x7f3b900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x56486328c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863299eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863192cb0 con 0x56486328cbe0 2023-07-21T19:26:34.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648631e5a40 con 0x56486328c2d0 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648631deed0 con 0x564863297ad0 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.280+0000 7f3ba57fa700 1 -- 172.21.15.5:0/1906348051 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b9c0cc800 con 0x7f3ba80fe780 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 0x7f3b900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49612/0 (socket says 172.21.15.5:49612) 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 -- 172.21.15.5:0/3454464948 learned_addr learned my addr 172.21.15.5:0/3454464948 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x56486328c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863299eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 -- 172.21.15.5:0/3454464948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 msgr2=0x564863299eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863299eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 -- 172.21.15.5:0/3454464948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 msgr2=0x56486328c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x56486328c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 -- 172.21.15.5:0/3454464948 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56486300cde0 con 0x56486328cbe0 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x56486328c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.281+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0962e0 tx=0x7f3b9c04fc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74e442a79818bb10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 0x7f3b900a3770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ba81016f0 tx=0x7f3ba0009bf0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3b8e7fc700 1 -- 172.21.15.5:0/3454464948 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b7ea0 con 0x56486328cbe0 2023-07-21T19:26:34.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3b8e7fc700 1 -- 172.21.15.5:0/3454464948 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b9c00fca0 con 0x56486328cbe0 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3b8e7fc700 1 -- 172.21.15.5:0/3454464948 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0ca170 con 0x56486328cbe0 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3454464948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 msgr2=0x564863297590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.282+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c0962e0 tx=0x7f3b9c04fc50 comp rx=0 tx=0).stop 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3454464948 shutdown_connections 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863299eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x56486328c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3454464948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863297590 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3454464948 >> 172.21.15.5:0/3454464948 conn(0x564863282da0 msgr2=0x5648632886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3454464948 shutdown_connections 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3454464948 wait complete. 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 -- start start 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.283+0000 7f3bc1ec9b80 1 Processor -- start 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x564863297510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863294ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486317eac0 con 0x56486328cbe0 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56486317e7c0 con 0x56486328c2d0 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863062920 con 0x564863297ad0 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3bc1ec9b80 10 client.0 _ll_register_callbacks cb 0x56486317c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3ba77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49628/0 (socket says 172.21.15.5:49628) 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3ba77fe700 1 -- 172.21.15.5:0/2195946802 learned_addr learned my addr 172.21.15.5:0/2195946802 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.284+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x564863297510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863294ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 -- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 msgr2=0x564863294ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863294ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 -- 172.21.15.5:0/2195946802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 msgr2=0x564863297510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x564863297510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 -- 172.21.15.5:0/2195946802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba810d1b0 con 0x56486328cbe0 2023-07-21T19:26:34.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x564863297510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c099800 tx=0x7f3b9c0994e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4ad09dc46147c5a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c0b7ea0 con 0x56486328cbe0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b9c0a1cc0 con 0x56486328cbe0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648631384f0 con 0x56486328cbe0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9c0ca370 con 0x56486328cbe0 2023-07-21T19:26:34.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3bc1ec9b80 10 client.19272 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.285+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f3ba81a4b10 con 0x56486328cbe0 2023-07-21T19:26:34.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648631384f0 con 0x56486328cbe0 2023-07-21T19:26:34.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3b9c098d60 con 0x56486328cbe0 2023-07-21T19:26:34.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 10 client.19272 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 10 client.19272 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:26:34.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 20 client.19272 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5648632af760 0x5648632b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 4 client.19272 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.286+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648631d7eb0 con 0x5648632af760 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.287+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5648632af760 0x5648632b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.287+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5648632af760 0x5648632b1b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2883d76bad58f46e server_cookie=1c4e665ecd0c9326 in_seq=0 out_seq=0 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.287+0000 7f3b877fe700 10 client.19272 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:26:34.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3b877fe700 1 -- 172.21.15.5:0/2195946802 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5648631d7eb0 con 0x5648632af760 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3b877fe700 10 client.19272 handle_command_reply: tid=0 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 client.19272 shutdown 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 msgr2=0x564863294790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c099800 tx=0x7f3b9c0994e0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 shutdown_connections 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863294ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x5648632af760 0x5648632b1b70 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328c2d0 0x564863297510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.289+0000 7f3b877fe700 0 client.19272 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.288+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/2195946802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328cbe0 0x564863294790 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.289+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 >> 172.21.15.5:0/2195946802 conn(0x564863282da0 msgr2=0x564863288290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.289+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 shutdown_connections 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.289+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/2195946802 wait complete. 2023-07-21T19:26:34.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.289+0000 7f3bc1ec9b80 20 client.19272 trim_cache size 0 max 16384 2023-07-21T19:26:34.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 Processor -- start 2023-07-21T19:26:34.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 -- start start 2023-07-21T19:26:34.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x56486328ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863293f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648631384f0 con 0x56486328c2d0 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648632ac640 con 0x56486328cbe0 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.303+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564863284ec0 con 0x564863297ad0 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49644/0 (socket says 172.21.15.5:49644) 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 -- 172.21.15.5:0/724028831 learned_addr learned my addr 172.21.15.5:0/724028831 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x56486328ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 -- 172.21.15.5:0/724028831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 msgr2=0x564863293f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863293f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 -- 172.21.15.5:0/724028831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 msgr2=0x56486328ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x56486328ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 -- 172.21.15.5:0/724028831 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564862ff5c60 con 0x56486328c2d0 2023-07-21T19:26:34.295 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.304+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3b980095c0 tx=0x7f3b9800b540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f221ea535a2e18 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3b8effd700 1 -- 172.21.15.5:0/724028831 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9801d040 con 0x56486328c2d0 2023-07-21T19:26:34.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3b8effd700 1 -- 172.21.15.5:0/724028831 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b98028070 con 0x56486328c2d0 2023-07-21T19:26:34.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3b8effd700 1 -- 172.21.15.5:0/724028831 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b98003b60 con 0x56486328c2d0 2023-07-21T19:26:34.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/724028831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 msgr2=0x56486328fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f3b980095c0 tx=0x7f3b9800b540 comp rx=0 tx=0).stop 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/724028831 shutdown_connections 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564863297ad0 0x564863293f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56486328c2d0 0x56486328fa50 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/724028831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x56486328ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/724028831 >> 172.21.15.5:0/724028831 conn(0x564863282da0 msgr2=0x5648632abf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/724028831 shutdown_connections 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.305+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/724028831 wait complete. 2023-07-21T19:26:34.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 -- start start 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 Processor -- start 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 0x5648632a6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x5648632a4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648632a5810 con 0x564863297ad0 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648632a5980 con 0x56486328cbe0 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.306+0000 7f3bc1ec9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648632a5b40 con 0x56486328c2d0 2023-07-21T19:26:34.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bc1ec9b80 10 client.0 _ll_register_callbacks cb 0x7f3b98009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49660/0 (socket says 172.21.15.5:49660) 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 -- 172.21.15.5:0/3521639164 learned_addr learned my addr 172.21.15.5:0/3521639164 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 0x5648632a6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3ba77fe700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x5648632a4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 -- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 msgr2=0x5648632a6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 0x5648632a6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 -- 172.21.15.5:0/3521639164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 msgr2=0x5648632a4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x5648632a4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3bacb9c700 1 -- 172.21.15.5:0/3521639164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564863033250 con 0x564863297ad0 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.307+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 0x5648632a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3bacb9c700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0019f40 tx=0x7f3ba0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975c4a88fad40d67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba00095e0 con 0x564863297ad0 2023-07-21T19:26:34.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b9c00fca0 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3ba001bd50 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba0004bb0 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3bc1ec9b80 10 client.19278 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x56486300b2b0 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b9c00fca0 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.308+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3ba0023070 con 0x564863297ad0 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 10 client.19278 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 10 client.19278 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:26:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 20 client.19278 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564863304ac0 0x564863306ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 4 client.19278 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:26:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x56486319a050 con 0x564863304ac0 2023-07-21T19:26:34.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564863304ac0 0x564863306ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3ba7fff700 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564863304ac0 0x564863306ed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b991b2d732a4c455 server_cookie=b05a65f4bc505842 in_seq=0 out_seq=0 2023-07-21T19:26:34.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.309+0000 7f3b877fe700 10 client.19278 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:26:34.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3b877fe700 1 -- 172.21.15.5:0/3521639164 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x56486319a050 con 0x564863304ac0 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3b877fe700 10 client.19278 handle_command_reply: tid=0 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 client.19278 shutdown 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 msgr2=0x5648632a52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0019f40 tx=0x7f3ba0004ad0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 shutdown_connections 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564863297ad0 0x5648632a52d0 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56486328c2d0 0x5648632a6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x564863304ac0 0x564863306ed0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 --2- 172.21.15.5:0/3521639164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56486328cbe0 0x5648632a4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 >> 172.21.15.5:0/3521639164 conn(0x564863282da0 msgr2=0x56486328e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3b877fe700 0 client.19278 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 shutdown_connections 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 1 -- 172.21.15.5:0/3521639164 wait complete. 2023-07-21T19:26:34.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.310+0000 7f3bc1ec9b80 20 client.19278 trim_cache size 0 max 16384 2023-07-21T19:26:34.308 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:26:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 msgr2=0x7f3b900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 0x7f3b900a3770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3ba81016f0 tx=0x7f3ba0009bf0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 msgr2=0x7f3ba80ffb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c00d980 tx=0x7f3b9c00dc90 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 shutdown_connections 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b900a12c0 0x7f3b900a3770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3ba8104e80 0x7f3ba81a3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3ba80fdde0 0x7f3ba80ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 --2- 172.21.15.5:0/1906348051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3ba80fe780 0x7f3ba80ffb20 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 >> 172.21.15.5:0/1906348051 conn(0x7f3ba8075b70 msgr2=0x7f3ba80fb9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 shutdown_connections 2023-07-21T19:26:34.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.311+0000 7f3bae5ff700 1 -- 172.21.15.5:0/1906348051 wait complete. 2023-07-21T19:26:34.317 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:26:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.535+0000 7fdde2fff700 1 -- 172.21.15.5:0/3981818046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc0731c0 msgr2=0x7fdddc10bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.535+0000 7fdde2fff700 1 --2- 172.21.15.5:0/3981818046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc0731c0 0x7fdddc10bdd0 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009990 tx=0x7fddc8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 -- 172.21.15.5:0/3981818046 shutdown_connections 2023-07-21T19:26:34.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 --2- 172.21.15.5:0/3981818046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc10c310 0x7fdddc10e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 --2- 172.21.15.5:0/3981818046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0728b0 0x7fdddc072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 --2- 172.21.15.5:0/3981818046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc0731c0 0x7fdddc10bdd0 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 -- 172.21.15.5:0/3981818046 >> 172.21.15.5:0/3981818046 conn(0x7fdddc0fb5f0 msgr2=0x7fdddc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 -- 172.21.15.5:0/3981818046 shutdown_connections 2023-07-21T19:26:34.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.536+0000 7fdde2fff700 1 -- 172.21.15.5:0/3981818046 wait complete. 2023-07-21T19:26:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 Processor -- start 2023-07-21T19:26:34.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 -- start start 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc0728b0 0x7fdddc19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 0x7fdddc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc198100 con 0x7fdddc10c310 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc198270 con 0x7fdddc0731c0 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde2fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdddc198430 con 0x7fdddc0728b0 2023-07-21T19:26:34.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.537+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49686/0 (socket says 172.21.15.5:49686) 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 -- 172.21.15.5:0/1079247487 learned_addr learned my addr 172.21.15.5:0/1079247487 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc0728b0 0x7fdddc19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 0x7fdddc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 -- 172.21.15.5:0/1079247487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc0728b0 msgr2=0x7fdddc19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc0728b0 0x7fdddc19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 -- 172.21.15.5:0/1079247487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 msgr2=0x7fdddc19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 0x7fdddc19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 -- 172.21.15.5:0/1079247487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc80095f0 con 0x7fdddc10c310 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 0x7fdddc19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde159c700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007d70 tx=0x7fddd400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fddd9ffb700 1 -- 172.21.15.5:0/1079247487 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd400ad40 con 0x7fdddc10c310 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fddd9ffb700 1 -- 172.21.15.5:0/1079247487 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddd40128b0 con 0x7fdddc10c310 2023-07-21T19:26:34.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fddd9ffb700 1 -- 172.21.15.5:0/1079247487 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40116b0 con 0x7fdddc10c310 2023-07-21T19:26:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdddc198710 con 0x7fdddc10c310 2023-07-21T19:26:34.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.538+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc1a4a10 con 0x7fdddc10c310 2023-07-21T19:26:34.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.542+0000 7fddd9ffb700 1 -- 172.21.15.5:0/1079247487 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fddd4012a20 con 0x7fdddc10c310 2023-07-21T19:26:34.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.543+0000 7fddf68e8b80 1 Processor -- start 2023-07-21T19:26:34.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.543+0000 7fddf68e8b80 1 -- start start 2023-07-21T19:26:34.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddd9ffb700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 0x7fddcc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946d6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637945d1cb0 con 0x5637946d6ad0 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563794624a40 con 0x5637946cb2d0 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379461ded0 con 0x5637946cbbe0 2023-07-21T19:26:34.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fddd9ffb700 1 -- 172.21.15.5:0/1079247487 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fddd40cce30 con 0x7fdddc10c310 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 0x7fddcc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde0d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38642/0 (socket says 172.21.15.5:38642) 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde0d9b700 1 -- 172.21.15.5:0/2609410631 learned_addr learned my addr 172.21.15.5:0/2609410631 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdddbfff700 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946d6590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde0d9b700 1 -- 172.21.15.5:0/2609410631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 msgr2=0x5637946d6590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.544+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946d6590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdde0d9b700 1 -- 172.21.15.5:0/2609410631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 msgr2=0x5637946d8eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d8eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdde0d9b700 1 -- 172.21.15.5:0/2609410631 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56379444bde0 con 0x5637946cb2d0 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdde159c700 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009d10 tx=0x7fddd00116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e228112a60612e41 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 0x7fddcc0a37f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005f50 tx=0x7fddc8005e80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fddc6ffd700 1 -- 172.21.15.5:0/2609410631 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd00097e0 con 0x5637946cb2d0 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fddc6ffd700 1 -- 172.21.15.5:0/2609410631 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddd0009e60 con 0x5637946cb2d0 2023-07-21T19:26:34.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.545+0000 7fddc6ffd700 1 -- 172.21.15.5:0/2609410631 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd000f710 con 0x5637946cb2d0 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2609410631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 msgr2=0x5637946cb6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009d10 tx=0x7fddd00116c0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2609410631 shutdown_connections 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d8eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cb2d0 0x5637946cb6a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2609410631 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946d6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2609410631 >> 172.21.15.5:0/2609410631 conn(0x5637946c1da0 msgr2=0x5637946c76a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2609410631 shutdown_connections 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2609410631 wait complete. 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.546+0000 7fddf68e8b80 1 -- start start 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 Processor -- start 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x5637946d6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x5637946d36b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637945bdac0 con 0x5637946d6ad0 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637945bd7c0 con 0x5637946cbbe0 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637944a1920 con 0x5637946cb2d0 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fddf68e8b80 10 client.0 _ll_register_callbacks cb 0x5637945bb760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.547+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49700/0 (socket says 172.21.15.5:49700) 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 -- 172.21.15.5:0/2666788419 learned_addr learned my addr 172.21.15.5:0/2666788419 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdddbfff700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x5637946d36b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x5637946d6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 -- 172.21.15.5:0/2666788419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 msgr2=0x5637946d6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x5637946d6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 -- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 msgr2=0x5637946d36b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x5637946d36b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 -- 172.21.15.5:0/2666788419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdddc1a4a10 con 0x5637946d6ad0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fdde159c700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ca690 tx=0x7fddd40861f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1ab08fc811181b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fddbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd40a2400 con 0x5637946d6ad0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fddd4098350 con 0x5637946d6ad0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.548+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5637945774f0 con 0x5637946d6ad0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd40ca7a0 con 0x5637946d6ad0 2023-07-21T19:26:34.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 10 client.19290 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7fddd0009940 con 0x5637946d6ad0 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5637945774f0 con 0x5637946d6ad0 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fddd409a210 con 0x5637946d6ad0 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 10 client.19290 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 10 client.19290 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 20 client.19290 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5637946e6300 0x5637946e8710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 4 client.19290 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.549+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563794616eb0 con 0x5637946e6300 2023-07-21T19:26:34.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.550+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5637946e6300 0x5637946e8710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.550+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5637946e6300 0x5637946e8710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9891c0482f891680 server_cookie=43d628f2b2dd39c7 in_seq=0 out_seq=0 2023-07-21T19:26:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.550+0000 7fddbbfff700 10 client.19290 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:26:34.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddbbfff700 1 -- 172.21.15.5:0/2666788419 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563794616eb0 con 0x5637946e6300 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddbbfff700 10 client.19290 handle_command_reply: tid=0 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 client.19290 shutdown 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 msgr2=0x5637946d3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fddd40ca690 tx=0x7fddd40861f0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 shutdown_connections 2023-07-21T19:26:34.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x5637946d3c90 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x5637946d6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x5637946e6300 0x5637946e8710 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/2666788419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x5637946d36b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 >> 172.21.15.5:0/2666788419 conn(0x5637946c1da0 msgr2=0x5637946c7290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddbbfff700 0 client.19290 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 shutdown_connections 2023-07-21T19:26:34.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.551+0000 7fddf68e8b80 1 -- 172.21.15.5:0/2666788419 wait complete. 2023-07-21T19:26:34.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.552+0000 7fddf68e8b80 20 client.19290 trim_cache size 0 max 16384 2023-07-21T19:26:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.565+0000 7fddf68e8b80 1 Processor -- start 2023-07-21T19:26:34.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.565+0000 7fddf68e8b80 1 -- start start 2023-07-21T19:26:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946dd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946d6ad0 0x5637946cea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637946cc540 con 0x5637946cb2d0 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637946ccd60 con 0x5637946d6ad0 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5637946d1c10 con 0x5637946cbbe0 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49706/0 (socket says 172.21.15.5:49706) 2023-07-21T19:26:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdddbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946dd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 -- 172.21.15.5:0/591075578 learned_addr learned my addr 172.21.15.5:0/591075578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde159c700 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946d6ad0 0x5637946cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 -- 172.21.15.5:0/591075578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 msgr2=0x5637946dd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946dd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 -- 172.21.15.5:0/591075578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946d6ad0 msgr2=0x5637946cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946d6ad0 0x5637946cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.566+0000 7fdde0d9b700 1 -- 172.21.15.5:0/591075578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563794434c60 con 0x5637946cb2d0 2023-07-21T19:26:34.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009ce0 tx=0x7fddd0005920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3ee654180bc8f30 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fddc77fe700 1 -- 172.21.15.5:0/591075578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0011e40 con 0x5637946cb2d0 2023-07-21T19:26:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fddc77fe700 1 -- 172.21.15.5:0/591075578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddd0005540 con 0x5637946cb2d0 2023-07-21T19:26:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fddc77fe700 1 -- 172.21.15.5:0/591075578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd0012450 con 0x5637946cb2d0 2023-07-21T19:26:34.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fddf68e8b80 1 -- 172.21.15.5:0/591075578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 msgr2=0x5637946ce1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.567+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7fddd0009ce0 tx=0x7fddd0005920 comp rx=0 tx=0).stop 2023-07-21T19:26:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 -- 172.21.15.5:0/591075578 shutdown_connections 2023-07-21T19:26:34.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946d6ad0 0x5637946cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946cb2d0 0x5637946ce1b0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/591075578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cbbe0 0x5637946dd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 -- 172.21.15.5:0/591075578 >> 172.21.15.5:0/591075578 conn(0x5637946c1da0 msgr2=0x5637946e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 -- 172.21.15.5:0/591075578 shutdown_connections 2023-07-21T19:26:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 -- 172.21.15.5:0/591075578 wait complete. 2023-07-21T19:26:34.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 -- start start 2023-07-21T19:26:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.568+0000 7fddf68e8b80 1 Processor -- start 2023-07-21T19:26:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x563794732550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x563794732a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379472e7f0 con 0x5637946d6ad0 2023-07-21T19:26:34.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379472e960 con 0x5637946cbbe0 2023-07-21T19:26:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379472eb20 con 0x5637946cb2d0 2023-07-21T19:26:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fddf68e8b80 10 client.0 _ll_register_callbacks cb 0x7fddd4012d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49722/0 (socket says 172.21.15.5:49722) 2023-07-21T19:26:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 -- 172.21.15.5:0/1719490538 learned_addr learned my addr 172.21.15.5:0/1719490538 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x563794732550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x563794732a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 -- 172.21.15.5:0/1719490538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 msgr2=0x563794732550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x563794732550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 -- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 msgr2=0x563794732a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x563794732a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdde159c700 1 -- 172.21.15.5:0/1719490538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56379444bde0 con 0x5637946d6ad0 2023-07-21T19:26:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.569+0000 7fdddbfff700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x563794732a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:26:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fdde159c700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fddd4086010 tx=0x7fddd4007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a790f89fa21d94 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddd4007aa0 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fddd40a2400 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fddd0009e60 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fddd4099570 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddf68e8b80 10 client.19296 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563794472250 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fddd0009e60 con 0x5637946d6ad0 2023-07-21T19:26:34.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fddd40ca260 con 0x5637946d6ad0 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddf68e8b80 10 client.19296 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.570+0000 7fddf68e8b80 10 client.19296 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fddf68e8b80 20 client.19296 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56379473b1c0 0x56379473d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fddf68e8b80 4 client.19296 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5637945d9050 con 0x56379473b1c0 2023-07-21T19:26:34.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56379473b1c0 0x56379473d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fdde0d9b700 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56379473b1c0 0x56379473d5d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2698f06d6b75558 server_cookie=ce1673bb0fb988f0 in_seq=0 out_seq=0 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.571+0000 7fddbbfff700 10 client.19296 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddbbfff700 1 -- 172.21.15.5:0/1719490538 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5637945d9050 con 0x56379473b1c0 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddbbfff700 10 client.19296 handle_command_reply: tid=0 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 client.19296 shutdown 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 msgr2=0x56379472e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7fddd4086010 tx=0x7fddd4007f70 comp rx=0 tx=0).stop 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 shutdown_connections 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5637946d6ad0 0x56379472e150 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5637946cb2d0 0x563794732550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x56379473b1c0 0x56379473d5d0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 --2- 172.21.15.5:0/1719490538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5637946cbbe0 0x563794732a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 >> 172.21.15.5:0/1719490538 conn(0x5637946c1da0 msgr2=0x5637946cea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddbbfff700 0 client.19296 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:26:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 shutdown_connections 2023-07-21T19:26:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 1 -- 172.21.15.5:0/1719490538 wait complete. 2023-07-21T19:26:34.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.572+0000 7fddf68e8b80 20 client.19296 trim_cache size 0 max 16384 2023-07-21T19:26:34.577 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:26:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 msgr2=0x7fddcc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 0x7fddcc0a37f0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fddc8005f50 tx=0x7fddc8005e80 comp rx=0 tx=0).stop 2023-07-21T19:26:34.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 msgr2=0x7fdddc197a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fddd4007d70 tx=0x7fddd400a340 comp rx=0 tx=0).stop 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 shutdown_connections 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fddcc0a1340 0x7fddcc0a37f0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdddc10c310 0x7fdddc197a90 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdddc0728b0 0x7fdddc19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 --2- 172.21.15.5:0/1079247487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdddc0731c0 0x7fdddc19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 >> 172.21.15.5:0/1079247487 conn(0x7fdddc0fb5f0 msgr2=0x7fdddc10e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 shutdown_connections 2023-07-21T19:26:34.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.573+0000 7fdde2fff700 1 -- 172.21.15.5:0/1079247487 wait complete. 2023-07-21T19:26:34.581 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:26:34.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 -- 172.21.15.5:0/3136194597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 msgr2=0x7fa3b4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/3136194597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4105eb0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4009990 tx=0x7fa3a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 -- 172.21.15.5:0/3136194597 shutdown_connections 2023-07-21T19:26:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/3136194597 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 0x7fa3b410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/3136194597 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4105eb0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.800+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/3136194597 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.801+0000 7fa3b9fff700 1 -- 172.21.15.5:0/3136194597 >> 172.21.15.5:0/3136194597 conn(0x7fa3b40fb5f0 msgr2=0x7fa3b40fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.801+0000 7fa3b9fff700 1 -- 172.21.15.5:0/3136194597 shutdown_connections 2023-07-21T19:26:34.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.801+0000 7fa3b9fff700 1 -- 172.21.15.5:0/3136194597 wait complete. 2023-07-21T19:26:34.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 Processor -- start 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 -- start start 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 0x7fa3b419be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4195960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4195fd0 con 0x7fa3b40ffad0 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4196140 con 0x7fa3b4100480 2023-07-21T19:26:34.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b9fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3b4196300 con 0x7fa3b4105ae0 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49740/0 (socket says 172.21.15.5:49740) 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.802+0000 7fa3b37fe700 1 -- 172.21.15.5:0/943073490 learned_addr learned my addr 172.21.15.5:0/943073490 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 0x7fa3b419be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4195960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 -- 172.21.15.5:0/943073490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 msgr2=0x7fa3b4195960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4195960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 -- 172.21.15.5:0/943073490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 msgr2=0x7fa3b419be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 0x7fa3b419be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 -- 172.21.15.5:0/943073490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a40095f0 con 0x7fa3b40ffad0 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005740 tx=0x7fa3a400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b0ff9700 1 -- 172.21.15.5:0/943073490 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a401d070 con 0x7fa3b40ffad0 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b0ff9700 1 -- 172.21.15.5:0/943073490 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3a4022470 con 0x7fa3b40ffad0 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b4196580 con 0x7fa3b40ffad0 2023-07-21T19:26:34.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b0ff9700 1 -- 172.21.15.5:0/943073490 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a4017650 con 0x7fa3b40ffad0 2023-07-21T19:26:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.803+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3b41a28e0 con 0x7fa3b40ffad0 2023-07-21T19:26:34.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.807+0000 7fa3b0ff9700 1 -- 172.21.15.5:0/943073490 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa3a40225e0 con 0x7fa3b40ffad0 2023-07-21T19:26:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.808+0000 7fa3b0ff9700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 0x7fa39c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.808+0000 7fa3cd83db80 1 Processor -- start 2023-07-21T19:26:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.808+0000 7fa3b0ff9700 1 -- 172.21.15.5:0/943073490 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a40ce170 con 0x7fa3b40ffad0 2023-07-21T19:26:34.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.808+0000 7fa3cd83db80 1 -- start start 2023-07-21T19:26:34.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.808+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 0x7fa39c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a3196beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a31864cb0 con 0x560a31969ad0 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a318b7a40 con 0x560a3195e2d0 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a318b0ed0 con 0x560a3195ebe0 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34006/0 (socket says 172.21.15.5:34006) 2023-07-21T19:26:34.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a3196beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38690/0 (socket says 172.21.15.5:38690) 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b2ffd700 1 -- 172.21.15.5:0/902311009 learned_addr learned my addr 172.21.15.5:0/902311009 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 0x7fa39c0a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40fdf50 tx=0x7fa3a8009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 -- 172.21.15.5:0/902311009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 msgr2=0x560a31969590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 -- 172.21.15.5:0/902311009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 msgr2=0x560a3196beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a3196beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b37fe700 1 -- 172.21.15.5:0/902311009 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a316dede0 con 0x560a3195e2d0 2023-07-21T19:26:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.809+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.810+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cb9d0 tx=0x7fa3a4097850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d074e53216507d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.810+0000 7fa399ffb700 1 -- 172.21.15.5:0/902311009 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a409b3e0 con 0x560a3195e2d0 2023-07-21T19:26:34.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.810+0000 7fa399ffb700 1 -- 172.21.15.5:0/902311009 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3a409f410 con 0x560a3195e2d0 2023-07-21T19:26:34.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.810+0000 7fa399ffb700 1 -- 172.21.15.5:0/902311009 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40a10f0 con 0x560a3195e2d0 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- 172.21.15.5:0/902311009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 msgr2=0x560a3195e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cb9d0 tx=0x7fa3a4097850 comp rx=0 tx=0).stop 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- 172.21.15.5:0/902311009 shutdown_connections 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a3196beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195e2d0 0x560a3195e6a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/902311009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a31969590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.806 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- 172.21.15.5:0/902311009 >> 172.21.15.5:0/902311009 conn(0x560a31954da0 msgr2=0x560a3195a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- 172.21.15.5:0/902311009 shutdown_connections 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- 172.21.15.5:0/902311009 wait complete. 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.811+0000 7fa3cd83db80 1 -- start start 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 Processor -- start 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a3196dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.807 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a31969ad0 0x560a31968a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a31850ac0 con 0x560a3195e2d0 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a318507c0 con 0x560a31969ad0 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a31734920 con 0x560a3195ebe0 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3cd83db80 10 client.0 _ll_register_callbacks cb 0x560a3184e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49752/0 (socket says 172.21.15.5:49752) 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3b37fe700 1 -- 172.21.15.5:0/4123093889 learned_addr learned my addr 172.21.15.5:0/4123093889 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.808 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a3196dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.812+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a31969ad0 0x560a31968a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 -- 172.21.15.5:0/4123093889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 msgr2=0x560a3196dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a3196dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 -- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a31969ad0 msgr2=0x560a31968a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a31969ad0 0x560a31968a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 -- 172.21.15.5:0/4123093889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a8011ab0 con 0x560a3195e2d0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cba30 tx=0x7fa3a4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae569569b37eed03 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a40a37a0 con 0x560a3195e2d0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a40b2070 con 0x560a3195e2d0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.813+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560a3180a4f0 con 0x560a3195e2d0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3a409b3e0 con 0x560a3195e2d0 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 10 client.19308 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7fa3b41a28e0 con 0x560a3195e2d0 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560a3180a4f0 con 0x560a3195e2d0 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa3a409a8d0 con 0x560a3195e2d0 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 10 client.19308 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 10 client.19308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 20 client.19308 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a31978b70 0x560a3197af50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 4 client.19308 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560a318a9eb0 con 0x560a31978b70 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.814+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a31978b70 0x560a3197af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.815+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a31978b70 0x560a3197af50 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a11aa69634f59bcd server_cookie=259bf53bad92ff93 in_seq=0 out_seq=0 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.815+0000 7fa392ffd700 10 client.19308 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa392ffd700 1 -- 172.21.15.5:0/4123093889 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560a318a9eb0 con 0x560a31978b70 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa392ffd700 10 client.19308 handle_command_reply: tid=0 2023-07-21T19:26:34.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 client.19308 shutdown 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 msgr2=0x560a3196dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fa3a40cba30 tx=0x7fa3a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 shutdown_connections 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a31969ad0 0x560a31968a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a31978b70 0x560a3197af50 crc :-1 s=READY pgs=381 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a3196dab0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/4123093889 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195ebe0 0x560a3196dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 >> 172.21.15.5:0/4123093889 conn(0x560a31954da0 msgr2=0x560a3195a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa392ffd700 0 client.19308 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 shutdown_connections 2023-07-21T19:26:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 1 -- 172.21.15.5:0/4123093889 wait complete. 2023-07-21T19:26:34.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.816+0000 7fa3cd83db80 20 client.19308 trim_cache size 0 max 16384 2023-07-21T19:26:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.830+0000 7fa3cd83db80 1 Processor -- start 2023-07-21T19:26:34.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.830+0000 7fa3cd83db80 1 -- start start 2023-07-21T19:26:34.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.830+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195e2d0 0x560a3196eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a31968210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3196a320 con 0x560a31969ad0 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a319549a0 con 0x560a3195ebe0 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a3196a0d0 con 0x560a3195e2d0 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49768/0 (socket says 172.21.15.5:49768) 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 -- 172.21.15.5:0/271783442 learned_addr learned my addr 172.21.15.5:0/271783442 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a31968210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195e2d0 0x560a3196eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 -- 172.21.15.5:0/271783442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195e2d0 msgr2=0x560a3196eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195e2d0 0x560a3196eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 -- 172.21.15.5:0/271783442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 msgr2=0x560a31968210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a31968210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 -- 172.21.15.5:0/271783442 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560a316c7c60 con 0x560a31969ad0 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.831+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007630 tx=0x7fa3ac009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59061f7fdde4fcea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa39a7fc700 1 -- 172.21.15.5:0/271783442 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac003e10 con 0x560a31969ad0 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa39a7fc700 1 -- 172.21.15.5:0/271783442 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3ac011040 con 0x560a31969ad0 2023-07-21T19:26:34.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa39a7fc700 1 -- 172.21.15.5:0/271783442 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3ac018750 con 0x560a31969ad0 2023-07-21T19:26:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 -- 172.21.15.5:0/271783442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 msgr2=0x560a31968750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7fa3ac007630 tx=0x7fa3ac009040 comp rx=0 tx=0).stop 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 -- 172.21.15.5:0/271783442 shutdown_connections 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a31969ad0 0x560a31968750 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a3195e2d0 0x560a3196eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/271783442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a31968210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 -- 172.21.15.5:0/271783442 >> 172.21.15.5:0/271783442 conn(0x560a31954da0 msgr2=0x560a3196e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 -- 172.21.15.5:0/271783442 shutdown_connections 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.832+0000 7fa3cd83db80 1 -- 172.21.15.5:0/271783442 wait complete. 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 -- start start 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 Processor -- start 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a319717f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 0x560a31971d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.833+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a31972350 con 0x560a3195e2d0 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a319724c0 con 0x560a3195ebe0 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3cd83db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560a31972680 con 0x560a31969ad0 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3cd83db80 10 client.0 _ll_register_callbacks cb 0x7fa3ac009de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49770/0 (socket says 172.21.15.5:49770) 2023-07-21T19:26:34.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 -- 172.21.15.5:0/1462888670 learned_addr learned my addr 172.21.15.5:0/1462888670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a319717f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 0x560a31971d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 -- 172.21.15.5:0/1462888670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 msgr2=0x560a31971d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 0x560a31971d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 -- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 msgr2=0x560a319717f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a319717f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 -- 172.21.15.5:0/1462888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3ac009300 con 0x560a3195e2d0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b3fff700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 0x560a31971d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa3b37fe700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4099730 tx=0x7fa3a409af20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af76bd097b11426b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.834+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a409b840 con 0x560a3195e2d0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa3a40b2070 con 0x560a3195e2d0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa3a409f410 con 0x560a3195e2d0 2023-07-21T19:26:34.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa3a409b3e0 con 0x560a3195e2d0 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 10 client.19314 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x560a31705250 con 0x560a3195e2d0 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa3a409f410 con 0x560a3195e2d0 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa3a4084cc0 con 0x560a3195e2d0 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 10 client.19314 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:34.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 10 client.19314 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:34.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 20 client.19314 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a319cda80 0x560a319cfe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 4 client.19314 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.835+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x560a3186c050 con 0x560a319cda80 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a319cda80 0x560a319cfe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa3b2ffd700 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a319cda80 0x560a319cfe60 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b96ce4f852a824b9 server_cookie=4a1f5f7b7af0672 in_seq=0 out_seq=0 2023-07-21T19:26:34.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa392ffd700 10 client.19314 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa392ffd700 1 -- 172.21.15.5:0/1462888670 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x560a3186c050 con 0x560a319cda80 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa392ffd700 10 client.19314 handle_command_reply: tid=0 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa3cd83db80 1 client.19314 shutdown 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 msgr2=0x560a319754e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.836+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4099730 tx=0x7fa3a409af20 comp rx=0 tx=0).stop 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 shutdown_connections 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560a31969ad0 0x560a31971d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560a319cda80 0x560a319cfe60 crc :-1 s=READY pgs=382 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560a3195e2d0 0x560a319754e0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 --2- 172.21.15.5:0/1462888670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560a3195ebe0 0x560a319717f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 >> 172.21.15.5:0/1462888670 conn(0x560a31954da0 msgr2=0x560a31960b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa392ffd700 0 client.19314 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 shutdown_connections 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 1 -- 172.21.15.5:0/1462888670 wait complete. 2023-07-21T19:26:34.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3cd83db80 20 client.19314 trim_cache size 0 max 16384 2023-07-21T19:26:34.839 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 msgr2=0x7fa39c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 0x7fa39c0a3770 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fa3b40fdf50 tx=0x7fa3a8009260 comp rx=0 tx=0).stop 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 msgr2=0x7fa3b419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7fa3a4005740 tx=0x7fa3a400bf50 comp rx=0 tx=0).stop 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 shutdown_connections 2023-07-21T19:26:34.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa3b4105ae0 0x7fa3b4195960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa39c0a12c0 0x7fa39c0a3770 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.837+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa3b40ffad0 0x7fa3b419b8f0 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.838+0000 7fa3b9fff700 1 --2- 172.21.15.5:0/943073490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa3b4100480 0x7fa3b419be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.838+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 >> 172.21.15.5:0/943073490 conn(0x7fa3b40fb5f0 msgr2=0x7fa3b40fbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.838+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 shutdown_connections 2023-07-21T19:26:34.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:34.838+0000 7fa3b9fff700 1 -- 172.21.15.5:0/943073490 wait complete. 2023-07-21T19:26:34.842 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:34 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3350392712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:26:34.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:34 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3350392712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:26:35.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:34 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3350392712' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:26:35.845 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:26:35.845 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:26:35.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:35 smithi005 ceph-mon[126460]: pgmap v598: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:35.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:35 smithi156 ceph-mon[128316]: pgmap v598: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:36.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:35 smithi101 ceph-mon[128136]: pgmap v598: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.198+0000 7f3a265ff700 1 -- 172.21.15.5:0/2201399478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a20107da0 msgr2=0x7f3a2010e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.198+0000 7f3a265ff700 1 --2- 172.21.15.5:0/2201399478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a20107da0 0x7f3a2010e570 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b0a0 tx=0x7f3a1000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 -- 172.21.15.5:0/2201399478 shutdown_connections 2023-07-21T19:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 --2- 172.21.15.5:0/2201399478 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a201113d0 0x7f3a201137b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 --2- 172.21.15.5:0/2201399478 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a20107da0 0x7f3a2010e570 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 --2- 172.21.15.5:0/2201399478 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a2010eab0 0x7f3a20110e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 -- 172.21.15.5:0/2201399478 >> 172.21.15.5:0/2201399478 conn(0x7f3a2006c200 msgr2=0x7f3a2006c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 -- 172.21.15.5:0/2201399478 shutdown_connections 2023-07-21T19:26:36.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.199+0000 7f3a265ff700 1 -- 172.21.15.5:0/2201399478 wait complete. 2023-07-21T19:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.201+0000 7f3a265ff700 1 Processor -- start 2023-07-21T19:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.201+0000 7f3a265ff700 1 -- start start 2023-07-21T19:26:36.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.201+0000 7f3a265ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a265ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2010eab0 0x7f3a20111db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a265ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201113d0 0x7f3a201122f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a265ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201ac6c0 con 0x7f3a2010eab0 2023-07-21T19:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a265ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201ac830 con 0x7f3a20107da0 2023-07-21T19:26:36.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a265ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a201ac9f0 con 0x7f3a201113d0 2023-07-21T19:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38730/0 (socket says 172.21.15.5:38730) 2023-07-21T19:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.202+0000 7f3a1ffff700 1 -- 172.21.15.5:0/128396586 learned_addr learned my addr 172.21.15.5:0/128396586 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a17fff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2010eab0 0x7f3a20111db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201113d0 0x7f3a201122f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a1ffff700 1 -- 172.21.15.5:0/128396586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201113d0 msgr2=0x7f3a201122f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201113d0 0x7f3a201122f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a1ffff700 1 -- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2010eab0 msgr2=0x7f3a20111db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2010eab0 0x7f3a20111db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.203+0000 7f3a1ffff700 1 -- 172.21.15.5:0/128396586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a10009d00 con 0x7f3a20107da0 2023-07-21T19:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.204+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004d00 tx=0x7f3a10004de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.204+0000 7f3a1dffb700 1 -- 172.21.15.5:0/128396586 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1000e040 con 0x7f3a20107da0 2023-07-21T19:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.204+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a201acc70 con 0x7f3a20107da0 2023-07-21T19:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.204+0000 7f3a1dffb700 1 -- 172.21.15.5:0/128396586 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a100078c0 con 0x7f3a20107da0 2023-07-21T19:26:36.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.204+0000 7f3a1dffb700 1 -- 172.21.15.5:0/128396586 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10005440 con 0x7f3a20107da0 2023-07-21T19:26:36.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.205+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201ad0f0 con 0x7f3a20107da0 2023-07-21T19:26:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.209+0000 7f3a1dffb700 1 -- 172.21.15.5:0/128396586 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3a1001d880 con 0x7f3a20107da0 2023-07-21T19:26:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.211+0000 7f3a39f60b80 1 Processor -- start 2023-07-21T19:26:36.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.211+0000 7f3a1dffb700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 0x7f3a0c0a3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 -- start start 2023-07-21T19:26:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a1dffb700 1 -- 172.21.15.5:0/128396586 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a100ce810 con 0x7f3a20107da0 2023-07-21T19:26:36.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x56247499d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247499db50 0x56247499ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562474898d30 con 0x562474992c60 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624748ebac0 con 0x562474992350 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.212+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624748e4f50 con 0x56247499db50 2023-07-21T19:26:36.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a24b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247499db50 0x56247499ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38736/0 (socket says 172.21.15.5:38736) 2023-07-21T19:26:36.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 -- 172.21.15.5:0/4204958576 learned_addr learned my addr 172.21.15.5:0/4204958576 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a17fff700 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x56247499d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 0x7f3a0c0a3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 -- 172.21.15.5:0/4204958576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247499db50 msgr2=0x56247499ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247499db50 0x56247499ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 -- 172.21.15.5:0/4204958576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 msgr2=0x56247499d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x56247499d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.213+0000 7f3a1ffff700 1 -- 172.21.15.5:0/4204958576 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562474712e60 con 0x562474992350 2023-07-21T19:26:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.214+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a10098380 tx=0x7f3a1000b3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c66bfae81c9e9b15 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.214+0000 7f3a167fc700 1 -- 172.21.15.5:0/4204958576 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1009f940 con 0x562474992350 2023-07-21T19:26:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.214+0000 7f3a167fc700 1 -- 172.21.15.5:0/4204958576 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a100a3cd0 con 0x562474992350 2023-07-21T19:26:36.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.214+0000 7f3a167fc700 1 -- 172.21.15.5:0/4204958576 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1009bb70 con 0x562474992350 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.215+0000 7f3a39f60b80 1 -- 172.21.15.5:0/4204958576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 msgr2=0x562474992720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.215+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f3a10098380 tx=0x7f3a1000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 -- 172.21.15.5:0/4204958576 shutdown_connections 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56247499db50 0x56247499ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x562474992720 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/4204958576 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x56247499d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 -- 172.21.15.5:0/4204958576 >> 172.21.15.5:0/4204958576 conn(0x562474988e20 msgr2=0x56247498e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 0x7f3a0c0a3820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a2006dd20 tx=0x7f3a18009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 -- 172.21.15.5:0/4204958576 shutdown_connections 2023-07-21T19:26:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.216+0000 7f3a39f60b80 1 -- 172.21.15.5:0/4204958576 wait complete. 2023-07-21T19:26:36.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.217+0000 7f3a39f60b80 1 -- start start 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.217+0000 7f3a39f60b80 1 Processor -- start 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992350 0x56247499df20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x56247499e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562474884b40 con 0x562474992350 2023-07-21T19:26:36.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562474884840 con 0x56247499db50 2023-07-21T19:26:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624747689a0 con 0x562474992c60 2023-07-21T19:26:36.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.218+0000 7f3a39f60b80 10 client.0 _ll_register_callbacks cb 0x5624748827e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.219+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.219+0000 7f3a17fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34066/0 (socket says 172.21.15.5:34066) 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.219+0000 7f3a17fff700 1 -- 172.21.15.5:0/694903314 learned_addr learned my addr 172.21.15.5:0/694903314 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.219+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992350 0x56247499df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.219+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x56247499e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 -- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 msgr2=0x56247499e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x56247499e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 -- 172.21.15.5:0/694903314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992350 msgr2=0x56247499df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992350 0x56247499df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 -- 172.21.15.5:0/694903314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a201ad0f0 con 0x562474992c60 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f3a17fff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800de50 tx=0x7f3a0800f4c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32ef28b49af6d938 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.220+0000 7f39ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a0800fee0 con 0x562474992c60 2023-07-21T19:26:36.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a08014460 con 0x562474992c60 2023-07-21T19:26:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56247483e570 con 0x562474992c60 2023-07-21T19:26:36.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a08014620 con 0x562474992c60 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f3a39f60b80 10 client.28861 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.221+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5624746fbce0 con 0x562474992c60 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.222+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56247483e570 con 0x562474992c60 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a08014780 con 0x562474992c60 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 10 client.28861 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 10 client.28861 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:36.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 20 client.28861 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624749ad1c0 0x5624749af5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 4 client.28861 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5624748ddf30 con 0x5624749ad1c0 2023-07-21T19:26:36.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.223+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624749ad1c0 0x5624749af5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.224+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624749ad1c0 0x5624749af5d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=710a1a3e585b4dc3 server_cookie=d6543d7a8fd2e13f in_seq=0 out_seq=0 2023-07-21T19:26:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.224+0000 7f39ff7fe700 10 client.28861 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.225+0000 7f39ff7fe700 1 -- 172.21.15.5:0/694903314 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5624748ddf30 con 0x5624749ad1c0 2023-07-21T19:26:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.225+0000 7f39ff7fe700 10 client.28861 handle_command_reply: tid=0 2023-07-21T19:26:36.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.225+0000 7f3a39f60b80 1 client.28861 shutdown 2023-07-21T19:26:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.226+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 msgr2=0x56247499e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.226+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3a0800de50 tx=0x7f3a0800f4c0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 shutdown_connections 2023-07-21T19:26:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x56247499e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5624749ad1c0 0x5624749af5d0 crc :-1 s=READY pgs=383 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992350 0x56247499df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/694903314 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499e460 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 >> 172.21.15.5:0/694903314 conn(0x562474988e20 msgr2=0x56247498e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f39ff7fe700 0 client.28861 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.227+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 shutdown_connections 2023-07-21T19:26:36.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.228+0000 7f3a39f60b80 1 -- 172.21.15.5:0/694903314 wait complete. 2023-07-21T19:26:36.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.228+0000 7f3a39f60b80 20 client.28861 trim_cache size 0 max 16384 2023-07-21T19:26:36.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.262+0000 7f3a39f60b80 1 Processor -- start 2023-07-21T19:26:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 -- start start 2023-07-21T19:26:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x5624749a1f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499ba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 0x562474999df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562474993ef0 con 0x56247499db50 2023-07-21T19:26:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56247498fd80 con 0x562474992350 2023-07-21T19:26:36.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.263+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562474988a20 con 0x562474992c60 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a24b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 0x562474999df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x5624749a1f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a24b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 0x562474999df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49820/0 (socket says 172.21.15.5:49820) 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a24b9c700 1 -- 172.21.15.5:0/3590029484 learned_addr learned my addr 172.21.15.5:0/3590029484 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a17fff700 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 -- 172.21.15.5:0/3590029484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 msgr2=0x56247499ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 -- 172.21.15.5:0/3590029484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 msgr2=0x562474999df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 0x562474999df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a1ffff700 1 -- 172.21.15.5:0/3590029484 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56247498d160 con 0x562474992350 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.264+0000 7f3a17fff700 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.265+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x5624749a1f50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b070 tx=0x7f3a100bb040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc86202da140ac76 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.266+0000 7f3a16ffd700 1 -- 172.21.15.5:0/3590029484 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a10099cb0 con 0x562474992350 2023-07-21T19:26:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.266+0000 7f3a16ffd700 1 -- 172.21.15.5:0/3590029484 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a1001db30 con 0x562474992350 2023-07-21T19:26:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.266+0000 7f3a16ffd700 1 -- 172.21.15.5:0/3590029484 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a100b9a10 con 0x562474992350 2023-07-21T19:26:36.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 -- 172.21.15.5:0/3590029484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 msgr2=0x5624749a1f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x5624749a1f50 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f3a1000b070 tx=0x7f3a100bb040 comp rx=0 tx=0).stop 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 -- 172.21.15.5:0/3590029484 shutdown_connections 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56247499db50 0x562474999df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562474992350 0x5624749a1f50 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/3590029484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992c60 0x56247499ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.267+0000 7f3a39f60b80 1 -- 172.21.15.5:0/3590029484 >> 172.21.15.5:0/3590029484 conn(0x562474988e20 msgr2=0x56247499b310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.268+0000 7f3a39f60b80 1 -- 172.21.15.5:0/3590029484 shutdown_connections 2023-07-21T19:26:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.268+0000 7f3a39f60b80 1 -- 172.21.15.5:0/3590029484 wait complete. 2023-07-21T19:26:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.269+0000 7f3a39f60b80 1 -- start start 2023-07-21T19:26:36.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.269+0000 7f3a39f60b80 1 Processor -- start 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992350 0x562474a035c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x562474a03b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624749fdf40 con 0x562474992c60 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624749fe0b0 con 0x56247499db50 2023-07-21T19:26:36.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5624749fe270 con 0x562474992350 2023-07-21T19:26:36.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a39f60b80 10 client.0 _ll_register_callbacks cb 0x7f3a0800d8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:36.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a24b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a24b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38760/0 (socket says 172.21.15.5:38760) 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a24b9c700 1 -- 172.21.15.5:0/936900395 learned_addr learned my addr 172.21.15.5:0/936900395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992350 0x562474a035c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.270+0000 7f3a17fff700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x562474a03b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 -- 172.21.15.5:0/936900395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992350 msgr2=0x562474a035c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992350 0x562474a035c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 -- 172.21.15.5:0/936900395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 msgr2=0x562474a03b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x562474a03b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 -- 172.21.15.5:0/936900395 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562474712e60 con 0x56247499db50 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f3a24b9c700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a18022bb0 tx=0x7f3a180047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=919099b7127774c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.271+0000 7f39ff7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a1800fdb0 con 0x56247499db50 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3a18004020 con 0x56247499db50 2023-07-21T19:26:36.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3a100a3cd0 con 0x56247499db50 2023-07-21T19:26:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3a180041e0 con 0x56247499db50 2023-07-21T19:26:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f3a39f60b80 10 client.28955 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.272+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5624747392d0 con 0x56247499db50 2023-07-21T19:26:36.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3a18036080 con 0x56247499db50 2023-07-21T19:26:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3a18030030 con 0x56247499db50 2023-07-21T19:26:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 10 client.28955 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 10 client.28955 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:36.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 20 client.28955 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562474a0b560 0x562474a0d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 4 client.28955 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.273+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x5624748a00d0 con 0x562474a0b560 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.274+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562474a0b560 0x562474a0d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.274+0000 7f3a1ffff700 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562474a0b560 0x562474a0d970 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ab53be6560b975c server_cookie=6ed1070d7f09ac51 in_seq=0 out_seq=0 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.274+0000 7f39ff7fe700 10 client.28955 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.275+0000 7f39ff7fe700 1 -- 172.21.15.5:0/936900395 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x5624748a00d0 con 0x562474a0b560 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.275+0000 7f39ff7fe700 10 client.28955 handle_command_reply: tid=0 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.275+0000 7f3a39f60b80 1 client.28955 shutdown 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 msgr2=0x5624749fd960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f3a18022bb0 tx=0x7f3a180047c0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 shutdown_connections 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56247499db50 0x5624749fd960 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562474992350 0x562474a035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562474a0b560 0x562474a0d970 crc :-1 s=READY pgs=384 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 --2- 172.21.15.5:0/936900395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562474992c60 0x562474a03b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 >> 172.21.15.5:0/936900395 conn(0x562474988e20 msgr2=0x56247499bba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f39ff7fe700 0 client.28955 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 shutdown_connections 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.276+0000 7f3a39f60b80 1 -- 172.21.15.5:0/936900395 wait complete. 2023-07-21T19:26:36.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.277+0000 7f3a39f60b80 20 client.28955 trim_cache size 0 max 16384 2023-07-21T19:26:36.271 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:26:36.271 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:26:36.271 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "a5497d32-e0fb-40df-889a-7e6b0add4a6b", 2023-07-21T19:26:36.271 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:26:36.271 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 msgr2=0x7f3a0c0a3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 0x7f3a0c0a3820 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3a2006dd20 tx=0x7f3a18009260 comp rx=0 tx=0).stop 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 msgr2=0x7f3a20111870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f3a10004d00 tx=0x7f3a10004de0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 shutdown_connections 2023-07-21T19:26:36.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3a201113d0 0x7f3a201122f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3a20107da0 0x7f3a20111870 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3a0c0a1370 0x7f3a0c0a3820 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 --2- 172.21.15.5:0/128396586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3a2010eab0 0x7f3a20111db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 >> 172.21.15.5:0/128396586 conn(0x7f3a2006c200 msgr2=0x7f3a20113a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 shutdown_connections 2023-07-21T19:26:36.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.278+0000 7f3a265ff700 1 -- 172.21.15.5:0/128396586 wait complete. 2023-07-21T19:26:36.296 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:26:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 -- 172.21.15.5:0/2577180348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150110900 msgr2=0x7fc150112ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 --2- 172.21.15.5:0/2577180348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150110900 0x7fc150112ce0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fc148009f90 tx=0x7fc14800b330 comp rx=0 tx=0).stop 2023-07-21T19:26:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 -- 172.21.15.5:0/2577180348 shutdown_connections 2023-07-21T19:26:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 --2- 172.21.15.5:0/2577180348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150110900 0x7fc150112ce0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 --2- 172.21.15.5:0/2577180348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150107350 0x7fc150107720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 --2- 172.21.15.5:0/2577180348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107c60 0x7fc1501103c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.647+0000 7fc1577ff700 1 -- 172.21.15.5:0/2577180348 >> 172.21.15.5:0/2577180348 conn(0x7fc150075a00 msgr2=0x7fc150077e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.648+0000 7fc1577ff700 1 -- 172.21.15.5:0/2577180348 shutdown_connections 2023-07-21T19:26:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.648+0000 7fc1577ff700 1 -- 172.21.15.5:0/2577180348 wait complete. 2023-07-21T19:26:36.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.649+0000 7fc1577ff700 1 Processor -- start 2023-07-21T19:26:36.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 -- start start 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150107c60 0x7fc15019c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150110900 0x7fc1501a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a10a0 con 0x7fc150110900 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a1210 con 0x7fc150107c60 2023-07-21T19:26:36.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.650+0000 7fc1577ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1501a13d0 con 0x7fc150107350 2023-07-21T19:26:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.651+0000 7fc15559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.651+0000 7fc155d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150110900 0x7fc1501a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.651+0000 7fc15559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47248/0 (socket says 172.21.15.5:47248) 2023-07-21T19:26:36.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.651+0000 7fc15559b700 1 -- 172.21.15.5:0/3589767871 learned_addr learned my addr 172.21.15.5:0/3589767871 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.651+0000 7fc15559b700 1 -- 172.21.15.5:0/3589767871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150107c60 msgr2=0x7fc15019c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc15559b700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150107c60 0x7fc15019c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc15559b700 1 -- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150110900 msgr2=0x7fc1501a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc15559b700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150110900 0x7fc1501a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc15559b700 1 -- 172.21.15.5:0/3589767871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc148009c40 con 0x7fc150107350 2023-07-21T19:26:36.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc15559b700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc14000d950 tx=0x7fc14000dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc1467fc700 1 -- 172.21.15.5:0/3589767871 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140009b40 con 0x7fc150107350 2023-07-21T19:26:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc1467fc700 1 -- 172.21.15.5:0/3589767871 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc140009ca0 con 0x7fc150107350 2023-07-21T19:26:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.652+0000 7fc1467fc700 1 -- 172.21.15.5:0/3589767871 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14000f620 con 0x7fc150107350 2023-07-21T19:26:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.653+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1501a16b0 con 0x7fc150107350 2023-07-21T19:26:36.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.654+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15007af60 con 0x7fc150107350 2023-07-21T19:26:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.658+0000 7fc1467fc700 1 -- 172.21.15.5:0/3589767871 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc140010460 con 0x7fc150107350 2023-07-21T19:26:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.660+0000 7fc16b1b7b80 1 Processor -- start 2023-07-21T19:26:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 -- start start 2023-07-21T19:26:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c40d66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40e1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c40e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c3fdccb0 con 0x5629c40d62d0 2023-07-21T19:26:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c402fa40 con 0x5629c40d6be0 2023-07-21T19:26:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.661+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c4028ed0 con 0x5629c40e1ad0 2023-07-21T19:26:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc1467fc700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 0x7fc13c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc15559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40e1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc155d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c40d66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc1467fc700 1 -- 172.21.15.5:0/3589767871 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc1400cc140 con 0x7fc150107350 2023-07-21T19:26:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc15559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40e1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46218/0 (socket says 172.21.15.5:46218) 2023-07-21T19:26:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc15559b700 1 -- 172.21.15.5:0/1360400773 learned_addr learned my addr 172.21.15.5:0/1360400773 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc154d9a700 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c40e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.662+0000 7fc155d9c700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 0x7fc13c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 -- 172.21.15.5:0/1360400773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 msgr2=0x5629c40e3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c40e3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 -- 172.21.15.5:0/1360400773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 msgr2=0x5629c40e1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40e1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 -- 172.21.15.5:0/1360400773 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c3e56de0 con 0x5629c40d62d0 2023-07-21T19:26:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.663+0000 7fc155d9c700 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c40d66a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc150069a80 tx=0x7fc14800bb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=667c5daac3e30427 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.664+0000 7fc155d9c700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 0x7fc13c0a3830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e000 tx=0x7fc148025000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.664+0000 7fc13b7fe700 1 -- 172.21.15.5:0/1360400773 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14800e030 con 0x5629c40d62d0 2023-07-21T19:26:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.664+0000 7fc13b7fe700 1 -- 172.21.15.5:0/1360400773 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc14800be30 con 0x5629c40d62d0 2023-07-21T19:26:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.664+0000 7fc13b7fe700 1 -- 172.21.15.5:0/1360400773 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc14801f440 con 0x5629c40d62d0 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1360400773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 msgr2=0x5629c40d66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c40d66a0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc150069a80 tx=0x7fc14800bb20 comp rx=0 tx=0).stop 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1360400773 shutdown_connections 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c40e3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c40d66a0 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1360400773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40e1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.665+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1360400773 >> 172.21.15.5:0/1360400773 conn(0x5629c40ccda0 msgr2=0x5629c40d26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.666+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1360400773 shutdown_connections 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.666+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1360400773 wait complete. 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.667+0000 7fc16b1b7b80 1 -- start start 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.667+0000 7fc16b1b7b80 1 Processor -- start 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40de860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40deda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c3fc8ac0 con 0x5629c40e1ad0 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c3fc87c0 con 0x5629c40d6be0 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c3eac920 con 0x5629c40d62d0 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc16b1b7b80 10 client.0 _ll_register_callbacks cb 0x5629c3fc6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc15559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40de860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc155d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc155d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39706/0 (socket says 172.21.15.5:39706) 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc155d9c700 1 -- 172.21.15.5:0/4001632472 learned_addr learned my addr 172.21.15.5:0/4001632472 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.668+0000 7fc154d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40deda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 -- 172.21.15.5:0/4001632472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 msgr2=0x5629c40de860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40de860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 -- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 msgr2=0x5629c40deda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40deda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 -- 172.21.15.5:0/4001632472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1501137d0 con 0x5629c40e1ad0 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc15559b700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40de860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc155d9c700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b850 tx=0x7fc14800b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d6d32a5027bc497 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.669+0000 7fc138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc148034070 con 0x5629c40e1ad0 2023-07-21T19:26:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5629c3f824f0 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc1480078c0 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc148007be0 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc16b1b7b80 10 client.19344 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.670+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5629c3e3fc60 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5629c3f824f0 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc148037020 con 0x5629c40e1ad0 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 10 client.19344 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 10 client.19344 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 20 client.19344 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c40f1210 0x5629c40f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 4 client.19344 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.671+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5629c4021eb0 con 0x5629c40f1210 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.672+0000 7fc15559b700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c40f1210 0x5629c40f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.672+0000 7fc15559b700 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c40f1210 0x5629c40f3620 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc5c4734eb34aac server_cookie=801986551720a283 in_seq=0 out_seq=0 2023-07-21T19:26:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.672+0000 7fc138ff9700 10 client.19344 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc138ff9700 1 -- 172.21.15.5:0/4001632472 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5629c4021eb0 con 0x5629c40f1210 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc138ff9700 10 client.19344 handle_command_reply: tid=0 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 client.19344 shutdown 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 msgr2=0x5629c40df2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7fc14800b850 tx=0x7fc14800b880 comp rx=0 tx=0).stop 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 shutdown_connections 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c40f1210 0x5629c40f3620 crc :-1 s=READY pgs=385 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40e1ad0 0x5629c40df2e0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40de860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/4001632472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c40deda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.673+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 >> 172.21.15.5:0/4001632472 conn(0x5629c40ccda0 msgr2=0x5629c40d2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.674+0000 7fc138ff9700 0 client.19344 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.674+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 shutdown_connections 2023-07-21T19:26:36.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.674+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/4001632472 wait complete. 2023-07-21T19:26:36.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.674+0000 7fc16b1b7b80 20 client.19344 trim_cache size 0 max 16384 2023-07-21T19:26:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 Processor -- start 2023-07-21T19:26:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 -- start start 2023-07-21T19:26:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d6be0 0x5629c40d9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40e1ad0 0x5629c40da110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c40d7320 con 0x5629c40d6be0 2023-07-21T19:26:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.703+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c40d81a0 con 0x5629c40e1ad0 2023-07-21T19:26:36.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c40dce00 con 0x5629c40d62d0 2023-07-21T19:26:36.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47272/0 (socket says 172.21.15.5:47272) 2023-07-21T19:26:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 -- 172.21.15.5:0/48168114 learned_addr learned my addr 172.21.15.5:0/48168114 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc155d9c700 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40e1ad0 0x5629c40da110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc154d9a700 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d6be0 0x5629c40d9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 -- 172.21.15.5:0/48168114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40e1ad0 msgr2=0x5629c40da110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40e1ad0 0x5629c40da110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.704+0000 7fc15559b700 1 -- 172.21.15.5:0/48168114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d6be0 msgr2=0x5629c40d9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc15559b700 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d6be0 0x5629c40d9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc15559b700 1 -- 172.21.15.5:0/48168114 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5629c40ce5e0 con 0x5629c40d62d0 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc15559b700 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc140095710 tx=0x7fc140064660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f3e539304c8a25d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc13bfff700 1 -- 172.21.15.5:0/48168114 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140099210 con 0x5629c40d62d0 2023-07-21T19:26:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc13bfff700 1 -- 172.21.15.5:0/48168114 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc14007a070 con 0x5629c40d62d0 2023-07-21T19:26:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.705+0000 7fc13bfff700 1 -- 172.21.15.5:0/48168114 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc140095920 con 0x5629c40d62d0 2023-07-21T19:26:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/48168114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 msgr2=0x5629c40e84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc140095710 tx=0x7fc140064660 comp rx=0 tx=0).stop 2023-07-21T19:26:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/48168114 shutdown_connections 2023-07-21T19:26:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40e1ad0 0x5629c40da110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40d62d0 0x5629c40e84b0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/48168114 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d6be0 0x5629c40d9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.706+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/48168114 >> 172.21.15.5:0/48168114 conn(0x5629c40ccda0 msgr2=0x5629c40f3220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.707+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/48168114 shutdown_connections 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.707+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/48168114 wait complete. 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.707+0000 7fc16b1b7b80 1 -- start start 2023-07-21T19:26:36.700 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.707+0000 7fc16b1b7b80 1 Processor -- start 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c413e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c4139060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c4139740 con 0x5629c40d62d0 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c41398b0 con 0x5629c40d6be0 2023-07-21T19:26:36.701 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5629c4139a70 con 0x5629c40e1ad0 2023-07-21T19:26:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.708+0000 7fc16b1b7b80 10 client.0 _ll_register_callbacks cb 0x7fc14c00b260 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39712/0 (socket says 172.21.15.5:39712) 2023-07-21T19:26:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 -- 172.21.15.5:0/1117701847 learned_addr learned my addr 172.21.15.5:0/1117701847 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:36.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc155d9c700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c4139060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 -- 172.21.15.5:0/1117701847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 msgr2=0x5629c4139060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c4139060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 -- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 msgr2=0x5629c413e8c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:26:36.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c413e8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.709+0000 7fc15559b700 1 -- 172.21.15.5:0/1117701847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc15007af60 con 0x5629c40d62d0 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc15559b700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc140009520 tx=0x7fc140098dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3277e250ef77957b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc138ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1400a1660 con 0x5629c40d62d0 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc140009e10 con 0x5629c40d62d0 2023-07-21T19:26:36.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc14800be30 con 0x5629c40d62d0 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc140095920 con 0x5629c40d62d0 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc16b1b7b80 10 client.19350 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.710+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5629c3e7d250 con 0x5629c40d62d0 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc1400c7080 con 0x5629c40d62d0 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc140098700 con 0x5629c40d62d0 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 10 client.19350 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:36.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 10 client.19350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 20 client.19350 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c4156ca0 0x5629c4159080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 4 client.19350 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5629c3fe4050 con 0x5629c4156ca0 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.711+0000 7fc154d9a700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c4156ca0 0x5629c4159080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:36.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.712+0000 7fc154d9a700 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c4156ca0 0x5629c4159080 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23dccd435add4673 server_cookie=15f4cd409dcbae51 in_seq=0 out_seq=0 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.712+0000 7fc138ff9700 10 client.19350 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.712+0000 7fc138ff9700 1 -- 172.21.15.5:0/1117701847 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5629c3fe4050 con 0x5629c4156ca0 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.712+0000 7fc138ff9700 10 client.19350 handle_command_reply: tid=0 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.712+0000 7fc16b1b7b80 1 client.19350 shutdown 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 msgr2=0x5629c413e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7fc140009520 tx=0x7fc140098dd0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 shutdown_connections 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5629c40e1ad0 0x5629c4139060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5629c4156ca0 0x5629c4159080 crc :-1 s=READY pgs=386 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5629c40d62d0 0x5629c413e380 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 --2- 172.21.15.5:0/1117701847 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5629c40d6be0 0x5629c413e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 >> 172.21.15.5:0/1117701847 conn(0x5629c40ccda0 msgr2=0x5629c40d9bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc138ff9700 0 client.19350 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:36.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 shutdown_connections 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.713+0000 7fc16b1b7b80 1 -- 172.21.15.5:0/1117701847 wait complete. 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.714+0000 7fc16b1b7b80 20 client.19350 trim_cache size 0 max 16384 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:26:36.710 INFO:teuthology.orchestra.run.smithi005.stdout: "a5497d32-e0fb-40df-889a-7e6b0add4a6b": { 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "a5497d32-e0fb-40df-889a-7e6b0add4a6b", 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:26:36.711 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:26:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.714+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 msgr2=0x7fc13c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.714+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 0x7fc13c0a3830 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc14800e000 tx=0x7fc148025000 comp rx=0 tx=0).stop 2023-07-21T19:26:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.714+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 msgr2=0x7fc15019c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:36.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.714+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc14000d950 tx=0x7fc14000dd10 comp rx=0 tx=0).stop 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 shutdown_connections 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc13c0a1380 0x7fc13c0a3830 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc150110900 0x7fc1501a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc150107350 0x7fc15019c0c0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 --2- 172.21.15.5:0/3589767871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc150107c60 0x7fc15019c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 >> 172.21.15.5:0/3589767871 conn(0x7fc150075a00 msgr2=0x7fc150075f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.715+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 shutdown_connections 2023-07-21T19:26:36.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:26:36.716+0000 7fc1577ff700 1 -- 172.21.15.5:0/3589767871 wait complete. 2023-07-21T19:26:36.734 INFO:tasks.cephfs.filesystem:scrub status for tag:a5497d32-e0fb-40df-889a-7e6b0add4a6b - {'path': '/', 'tag': 'a5497d32-e0fb-40df-889a-7e6b0add4a6b', 'options': 'recursive,force'} 2023-07-21T19:26:36.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:36 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:36.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:36 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:36.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:36 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:36.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:36 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:36.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:36 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:36.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:36 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:36.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 5.0 s, 352.6 tps, lat 252.430 ms stddev 557.629 2023-07-21T19:26:37.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:36 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:37.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:36 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:37.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:36 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:26:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:37.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:37 smithi005 ceph-mon[126460]: pgmap v599: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:37.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:37.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:37 smithi156 ceph-mon[128316]: pgmap v599: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:38.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:38.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:37 smithi101 ceph-mon[128136]: pgmap v599: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 215 KiB/s rd, 24 MiB/s wr, 137 op/s 2023-07-21T19:26:38.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:38 smithi005 ceph-mon[126460]: pgmap v600: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 27 MiB/s wr, 287 op/s 2023-07-21T19:26:38.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:38 smithi156 ceph-mon[128316]: pgmap v600: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 27 MiB/s wr, 287 op/s 2023-07-21T19:26:39.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:38 smithi101 ceph-mon[128136]: pgmap v600: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 27 MiB/s wr, 287 op/s 2023-07-21T19:26:41.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:41 smithi101 ceph-mon[128136]: pgmap v601: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 13 MiB/s wr, 222 op/s 2023-07-21T19:26:41.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:41 smithi005 ceph-mon[126460]: pgmap v601: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 13 MiB/s wr, 222 op/s 2023-07-21T19:26:41.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:41 smithi156 ceph-mon[128316]: pgmap v601: 97 pgs: 97 active+clean; 16 GiB data, 9.1 GiB used, 1.0 TiB / 1.0 TiB avail; 367 KiB/s rd, 13 MiB/s wr, 222 op/s 2023-07-21T19:26:41.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 10.0 s, 1190.8 tps, lat 83.454 ms stddev 19.510 2023-07-21T19:26:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 -- 172.21.15.5:0/1646251963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e80fdf10 msgr2=0x7f56e80fe380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/1646251963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e80fdf10 0x7f56e80fe380 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009990 tx=0x7f56dc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 -- 172.21.15.5:0/1646251963 shutdown_connections 2023-07-21T19:26:41.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/1646251963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e80fed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/1646251963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e81043f0 0x7f56e81047c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/1646251963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e80fdf10 0x7f56e80fe380 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 -- 172.21.15.5:0/1646251963 >> 172.21.15.5:0/1646251963 conn(0x7f56e8073d50 msgr2=0x7f56e8074150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 -- 172.21.15.5:0/1646251963 shutdown_connections 2023-07-21T19:26:41.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.003+0000 7f56ed1ff700 1 -- 172.21.15.5:0/1646251963 wait complete. 2023-07-21T19:26:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.004+0000 7f56ed1ff700 1 Processor -- start 2023-07-21T19:26:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 -- start start 2023-07-21T19:26:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e80fdf10 0x7f56e819c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:41.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e81043f0 0x7f56e81a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81a1550 con 0x7f56e80fdf10 2023-07-21T19:26:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81a16c0 con 0x7f56e81043f0 2023-07-21T19:26:41.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.005+0000 7f56ed1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56e81a1880 con 0x7f56e80fe8c0 2023-07-21T19:26:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.006+0000 7f56e659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.006+0000 7f56e659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47298/0 (socket says 172.21.15.5:47298) 2023-07-21T19:26:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.006+0000 7f56e659c700 1 -- 172.21.15.5:0/3050204296 learned_addr learned my addr 172.21.15.5:0/3050204296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:41.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.006+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e80fdf10 0x7f56e819c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 -- 172.21.15.5:0/3050204296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e81043f0 msgr2=0x7f56e81a0f10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:26:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e81043f0 0x7f56e81a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 -- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e80fdf10 msgr2=0x7f56e819c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e80fdf10 0x7f56e819c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 -- 172.21.15.5:0/3050204296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56dc0095f0 con 0x7f56e80fe8c0 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.007+0000 7f56e659c700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0036c0 tx=0x7f56dc005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.008+0000 7f56cffff700 1 -- 172.21.15.5:0/3050204296 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc01d070 con 0x7f56e80fe8c0 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.008+0000 7f56cffff700 1 -- 172.21.15.5:0/3050204296 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56dc003f40 con 0x7f56e80fe8c0 2023-07-21T19:26:41.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.008+0000 7f56cffff700 1 -- 172.21.15.5:0/3050204296 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56dc022c30 con 0x7f56e80fe8c0 2023-07-21T19:26:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.008+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56e81a1b00 con 0x7f56e80fe8c0 2023-07-21T19:26:42.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.008+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e81a1d80 con 0x7f56e80fe8c0 2023-07-21T19:26:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.014+0000 7f56cffff700 1 -- 172.21.15.5:0/3050204296 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f56dc022470 con 0x7f56e80fe8c0 2023-07-21T19:26:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.015+0000 7f5700be4b80 1 Processor -- start 2023-07-21T19:26:42.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.015+0000 7f5700be4b80 1 -- start start 2023-07-21T19:26:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.015+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f846a6b0 0x5577f8613d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.015+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f860e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f5700be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f8513c50 con 0x5577f846a6b0 2023-07-21T19:26:42.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f5700be4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f85669e0 con 0x5577f860e580 2023-07-21T19:26:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f5700be4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f855fe70 con 0x5577f86142d0 2023-07-21T19:26:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47308/0 (socket says 172.21.15.5:47308) 2023-07-21T19:26:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f56e6d9d700 1 -- 172.21.15.5:0/4010690882 learned_addr learned my addr 172.21.15.5:0/4010690882 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:42.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.016+0000 7f56e759e700 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f846a6b0 0x5577f8613d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 -- 172.21.15.5:0/4010690882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 msgr2=0x5577f860e9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:26:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f860e9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 -- 172.21.15.5:0/4010690882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f846a6b0 msgr2=0x5577f8613d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f846a6b0 0x5577f8613d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 -- 172.21.15.5:0/4010690882 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577f838dd80 con 0x5577f86142d0 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009940 tx=0x7f56e0003f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3609591e674f5ac server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56cd7fa700 1 -- 172.21.15.5:0/4010690882 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004780 con 0x5577f86142d0 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.017+0000 7f56cffff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 0x7f56d00a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.018+0000 7f56cd7fa700 1 -- 172.21.15.5:0/4010690882 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e000f040 con 0x5577f86142d0 2023-07-21T19:26:42.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.018+0000 7f56cffff700 1 -- 172.21.15.5:0/3050204296 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f56dc0ce0c0 con 0x7f56e80fe8c0 2023-07-21T19:26:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.018+0000 7f56cd7fa700 1 -- 172.21.15.5:0/4010690882 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0015800 con 0x5577f86142d0 2023-07-21T19:26:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.018+0000 7f56e759e700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 0x7f56d00a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.019+0000 7f5700be4b80 1 -- 172.21.15.5:0/4010690882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 msgr2=0x5577f860e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.019+0000 7f5700be4b80 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f56e0009940 tx=0x7f56e0003f90 comp rx=0 tx=0).stop 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.019+0000 7f56e759e700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 0x7f56d00a37d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005d90 tx=0x7f56d800b2c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 -- 172.21.15.5:0/4010690882 shutdown_connections 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f860e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f846a6b0 0x5577f8613d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 --2- 172.21.15.5:0/4010690882 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f86142d0 0x5577f860e040 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 -- 172.21.15.5:0/4010690882 >> 172.21.15.5:0/4010690882 conn(0x5577f8603be0 msgr2=0x5577f8608f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 -- 172.21.15.5:0/4010690882 shutdown_connections 2023-07-21T19:26:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.020+0000 7f5700be4b80 1 -- 172.21.15.5:0/4010690882 wait complete. 2023-07-21T19:26:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.021+0000 7f5700be4b80 1 -- start start 2023-07-21T19:26:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.021+0000 7f5700be4b80 1 Processor -- start 2023-07-21T19:26:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f860e580 0x5577f8617de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f86142d0 0x5577f8618320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f84ffa60 con 0x5577f86142d0 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f84ff760 con 0x5577f846a6b0 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f83e38c0 con 0x5577f860e580 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f5700be4b80 10 client.0 _ll_register_callbacks cb 0x5577f84fd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46266/0 (socket says 172.21.15.5:46266) 2023-07-21T19:26:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.022+0000 7f56e6d9d700 1 -- 172.21.15.5:0/2353364420 learned_addr learned my addr 172.21.15.5:0/2353364420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e659c700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f860e580 0x5577f8617de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 -- 172.21.15.5:0/2353364420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f860e580 msgr2=0x5577f8617de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f860e580 0x5577f8617de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 -- 172.21.15.5:0/2353364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f86142d0 msgr2=0x5577f8618320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f86142d0 0x5577f8618320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 -- 172.21.15.5:0/2353364420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56e0003c70 con 0x5577f846a6b0 2023-07-21T19:26:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f80 tx=0x7f56e0007e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18fb7868674ea2fc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56caff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.023+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e00076b0 con 0x5577f846a6b0 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f56e0007810 con 0x5577f846a6b0 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5577f84b9490 con 0x5577f846a6b0 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e00079d0 con 0x5577f846a6b0 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f5700be4b80 10 client.28973 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5577f8376c00 con 0x5577f846a6b0 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.024+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5577f84b9490 con 0x5577f846a6b0 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.025+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f56e0027440 con 0x5577f846a6b0 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 10 client.28973 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 10 client.28973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 20 client.28973 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f8638af0 0x5577f863aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 4 client.28973 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5577f8558e50 con 0x5577f8638af0 2023-07-21T19:26:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.026+0000 7f56e659c700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f8638af0 0x5577f863aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.027+0000 7f56caff5700 10 client.28973 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.027+0000 7f56e659c700 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f8638af0 0x5577f863aed0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9dc5ec37e3195d91 server_cookie=77c3ad6bc05923 in_seq=0 out_seq=0 2023-07-21T19:26:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.027+0000 7f56caff5700 1 -- 172.21.15.5:0/2353364420 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5577f8558e50 con 0x5577f8638af0 2023-07-21T19:26:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.027+0000 7f56caff5700 10 client.28973 handle_command_reply: tid=0 2023-07-21T19:26:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.028+0000 7f5700be4b80 1 client.28973 shutdown 2023-07-21T19:26:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.028+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 msgr2=0x5577f86178a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.028+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f56e0000f80 tx=0x7f56e0007e40 comp rx=0 tx=0).stop 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 shutdown_connections 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f8638af0 0x5577f863aed0 crc :-1 s=READY pgs=387 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f86142d0 0x5577f8618320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f846a6b0 0x5577f86178a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 --2- 172.21.15.5:0/2353364420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f860e580 0x5577f8617de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 >> 172.21.15.5:0/2353364420 conn(0x5577f8603be0 msgr2=0x5577f86084e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f56caff5700 0 client.28973 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.031+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 shutdown_connections 2023-07-21T19:26:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.032+0000 7f5700be4b80 1 -- 172.21.15.5:0/2353364420 wait complete. 2023-07-21T19:26:42.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.032+0000 7f5700be4b80 20 client.28973 trim_cache size 0 max 16384 2023-07-21T19:26:42.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 Processor -- start 2023-07-21T19:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 -- start start 2023-07-21T19:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f861ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f8608c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f860f040 con 0x5577f8603be0 2023-07-21T19:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f8600c00 con 0x5577f860e580 2023-07-21T19:26:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.055+0000 7f5700be4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f8605d00 con 0x5577f846a6b0 2023-07-21T19:26:42.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39754/0 (socket says 172.21.15.5:39754) 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 -- 172.21.15.5:0/729756916 learned_addr learned my addr 172.21.15.5:0/729756916 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e759e700 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f8608c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e659c700 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f861ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 -- 172.21.15.5:0/729756916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 msgr2=0x5577f861ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f861ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 -- 172.21.15.5:0/729756916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 msgr2=0x5577f8608c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f8608c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.056+0000 7f56e6d9d700 1 -- 172.21.15.5:0/729756916 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5577f860ec60 con 0x5577f8603be0 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.057+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f56e0003f40 tx=0x7f56e0003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad9fd1e7f89f2459 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.057+0000 7f56cdffb700 1 -- 172.21.15.5:0/729756916 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0027440 con 0x5577f8603be0 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.057+0000 7f56cdffb700 1 -- 172.21.15.5:0/729756916 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e00275a0 con 0x5577f8603be0 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.057+0000 7f56cdffb700 1 -- 172.21.15.5:0/729756916 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0030ab0 con 0x5577f8603be0 2023-07-21T19:26:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 -- 172.21.15.5:0/729756916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 msgr2=0x5577f86132e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f56e0003f40 tx=0x7f56e0003f70 comp rx=0 tx=0).stop 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 -- 172.21.15.5:0/729756916 shutdown_connections 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f8608c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86132e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 --2- 172.21.15.5:0/729756916 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f861ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.058+0000 7f5700be4b80 1 -- 172.21.15.5:0/729756916 >> 172.21.15.5:0/729756916 conn(0x5577f86352b0 msgr2=0x5577f8614900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.059+0000 7f5700be4b80 1 -- 172.21.15.5:0/729756916 shutdown_connections 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.059+0000 7f5700be4b80 1 -- 172.21.15.5:0/729756916 wait complete. 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.059+0000 7f5700be4b80 1 -- start start 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.059+0000 7f5700be4b80 1 Processor -- start 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86971f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f861cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f861d220 con 0x5577f8603be0 2023-07-21T19:26:42.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f861d390 con 0x5577f860e580 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5577f861d550 con 0x5577f846a6b0 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f5700be4b80 10 client.0 _ll_register_callbacks cb 0x7f56e0003e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f56e6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47330/0 (socket says 172.21.15.5:47330) 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f56e6d9d700 1 -- 172.21.15.5:0/3408007575 learned_addr learned my addr 172.21.15.5:0/3408007575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:42.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.060+0000 7f56e759e700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f861cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e659c700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 -- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 msgr2=0x5577f861cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f861cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 -- 172.21.15.5:0/3408007575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 msgr2=0x5577f86971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 -- 172.21.15.5:0/3408007575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5577f8376c00 con 0x5577f846a6b0 2023-07-21T19:26:42.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e759e700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f861cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56e6d9d700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007e80 tx=0x7f56e0015760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=68eda2ab6ef07bf3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.061+0000 7f56caff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.062+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56e0027440 con 0x5577f846a6b0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.062+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f56e0031440 con 0x5577f846a6b0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.062+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56e000f040 con 0x5577f846a6b0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.062+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56e0031640 con 0x5577f846a6b0 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 10 client.28903 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:42.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5577f83b41f0 con 0x5577f846a6b0 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f56e000f040 con 0x5577f846a6b0 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f56e0043070 con 0x5577f846a6b0 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 10 client.28903 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 10 client.28903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 20 client.28903 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f86142d0 0x5577f86a0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 4 client.28903 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.063+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5577f851aff0 con 0x5577f86142d0 2023-07-21T19:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.064+0000 7f56e659c700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f86142d0 0x5577f86a0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.064+0000 7f56e659c700 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f86142d0 0x5577f86a0580 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d3c8f9d710188db server_cookie=a427b30ec41ce343 in_seq=0 out_seq=0 2023-07-21T19:26:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.064+0000 7f56caff5700 10 client.28903 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f56caff5700 1 -- 172.21.15.5:0/3408007575 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5577f851aff0 con 0x5577f86142d0 2023-07-21T19:26:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f56caff5700 10 client.28903 handle_command_reply: tid=0 2023-07-21T19:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 client.28903 shutdown 2023-07-21T19:26:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 msgr2=0x5577f8696cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f56e0007e80 tx=0x7f56e0015760 comp rx=0 tx=0).stop 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 shutdown_connections 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5577f860e580 0x5577f861cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5577f846a6b0 0x5577f8696cb0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5577f86142d0 0x5577f86a0580 crc :-1 s=READY pgs=388 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 --2- 172.21.15.5:0/3408007575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5577f8603be0 0x5577f86971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 >> 172.21.15.5:0/3408007575 conn(0x5577f86352b0 msgr2=0x5577f86191b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f56caff5700 0 client.28903 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.092+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 shutdown_connections 2023-07-21T19:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.093+0000 7f5700be4b80 1 -- 172.21.15.5:0/3408007575 wait complete. 2023-07-21T19:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.093+0000 7f5700be4b80 20 client.28903 trim_cache size 0 max 16384 2023-07-21T19:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:26:42.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 msgr2=0x7f56d00a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 0x7f56d00a37d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f56d8005d90 tx=0x7f56d800b2c0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 msgr2=0x7f56e819c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f56dc0036c0 tx=0x7f56dc005740 comp rx=0 tx=0).stop 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 shutdown_connections 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f56e81043f0 0x7f56e81a0f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f56d00a1320 0x7f56d00a37d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f56e80fdf10 0x7f56e819c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 --2- 172.21.15.5:0/3050204296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f56e80fe8c0 0x7f56e819c8a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.094+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 >> 172.21.15.5:0/3050204296 conn(0x7f56e8073d50 msgr2=0x7f56e8101f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.095+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 shutdown_connections 2023-07-21T19:26:42.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:42.095+0000 7f56ed1ff700 1 -- 172.21.15.5:0/3050204296 wait complete. 2023-07-21T19:26:43.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:43 smithi005 ceph-mon[126460]: pgmap v602: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-07-21T19:26:43.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:43 smithi156 ceph-mon[128316]: pgmap v602: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-07-21T19:26:44.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:43 smithi101 ceph-mon[128136]: pgmap v602: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 534 KiB/s rd, 21 MiB/s wr, 318 op/s 2023-07-21T19:26:45.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:45 smithi005 ceph-mon[126460]: pgmap v603: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 11 MiB/s wr, 251 op/s 2023-07-21T19:26:45.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:45 smithi156 ceph-mon[128316]: pgmap v603: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 11 MiB/s wr, 251 op/s 2023-07-21T19:26:46.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:45 smithi101 ceph-mon[128136]: pgmap v603: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 11 MiB/s wr, 251 op/s 2023-07-21T19:26:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 15.0 s, 527.8 tps, lat 188.929 ms stddev 76.011 2023-07-21T19:26:47.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:47 smithi005 ceph-mon[126460]: pgmap v604: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 10 MiB/s wr, 247 op/s 2023-07-21T19:26:47.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:47 smithi156 ceph-mon[128316]: pgmap v604: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 10 MiB/s wr, 247 op/s 2023-07-21T19:26:48.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:47 smithi101 ceph-mon[128136]: pgmap v604: 97 pgs: 97 active+clean; 16 GiB data, 9.2 GiB used, 1.0 TiB / 1.0 TiB avail; 399 KiB/s rd, 10 MiB/s wr, 247 op/s 2023-07-21T19:26:49.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:49 smithi005 ceph-mon[126460]: pgmap v605: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 18 MiB/s wr, 781 op/s 2023-07-21T19:26:49.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:49 smithi156 ceph-mon[128316]: pgmap v605: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 18 MiB/s wr, 781 op/s 2023-07-21T19:26:50.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:49 smithi101 ceph-mon[128136]: pgmap v605: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 487 KiB/s rd, 18 MiB/s wr, 781 op/s 2023-07-21T19:26:51.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:51 smithi005 ceph-mon[126460]: pgmap v606: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 15 MiB/s wr, 631 op/s 2023-07-21T19:26:51.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:51 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:26:51.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:51 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:51.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:51 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:51.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:51 smithi156 ceph-mon[128316]: pgmap v606: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 15 MiB/s wr, 631 op/s 2023-07-21T19:26:51.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:51 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:26:51.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:51 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:51.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:51 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:51.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 20.0 s, 713.8 tps, lat 140.078 ms stddev 52.536 2023-07-21T19:26:52.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:51 smithi101 ceph-mon[128136]: pgmap v606: 97 pgs: 97 active+clean; 16 GiB data, 9.3 GiB used, 1.0 TiB / 1.0 TiB avail; 255 KiB/s rd, 15 MiB/s wr, 631 op/s 2023-07-21T19:26:52.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:51 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:26:52.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:51 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:52.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:51 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:26:53.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:53 smithi005 ceph-mon[126460]: pgmap v607: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2023-07-21T19:26:53.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:53 smithi156 ceph-mon[128316]: pgmap v607: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2023-07-21T19:26:54.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:53 smithi101 ceph-mon[128136]: pgmap v607: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 23 MiB/s wr, 1.20k op/s 2023-07-21T19:26:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/699839321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54110b50 msgr2=0x7f3f54112f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/699839321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54110b50 0x7f3f54112f50 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c00d290 tx=0x7f3f4c00d5a0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/699839321 shutdown_connections 2023-07-21T19:26:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/699839321 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54110b50 0x7f3f54112f50 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/699839321 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 0x7f3f54107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/699839321 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54108090 0x7f3f54110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.656+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/699839321 >> 172.21.15.5:0/699839321 conn(0x7f3f5406c270 msgr2=0x7f3f5406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.657+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/699839321 shutdown_connections 2023-07-21T19:26:55.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.657+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/699839321 wait complete. 2023-07-21T19:26:55.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.659+0000 7f3f5c7ff700 1 Processor -- start 2023-07-21T19:26:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 -- start start 2023-07-21T19:26:55.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 0x7f3f541a32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54108090 0x7f3f541a3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5419cac0 con 0x7f3f54108090 2023-07-21T19:26:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5419cc30 con 0x7f3f54110b50 2023-07-21T19:26:55.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.663+0000 7f3f5c7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f5419cdf0 con 0x7f3f54107780 2023-07-21T19:26:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.664+0000 7f3f5ad9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.664+0000 7f3f5ad9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44354/0 (socket says 172.21.15.5:44354) 2023-07-21T19:26:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.664+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3974845513 learned_addr learned my addr 172.21.15.5:0/3974845513 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.664+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 0x7f3f541a32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.664+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54108090 0x7f3f541a3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.665+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3974845513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 msgr2=0x7f3f541a32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.665+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 0x7f3f541a32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.665+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54108090 msgr2=0x7f3f541a3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.665+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54108090 0x7f3f541a3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.666+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3974845513 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009c40 con 0x7f3f54110b50 2023-07-21T19:26:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.666+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006050 tx=0x7f3f4c00d1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.666+0000 7f3f4b7fe700 1 -- 172.21.15.5:0/3974845513 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c007b80 con 0x7f3f54110b50 2023-07-21T19:26:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.666+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f541a0e70 con 0x7f3f54110b50 2023-07-21T19:26:55.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.667+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541a1370 con 0x7f3f54110b50 2023-07-21T19:26:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.667+0000 7f3f4b7fe700 1 -- 172.21.15.5:0/3974845513 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f4c009ac0 con 0x7f3f54110b50 2023-07-21T19:26:55.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.667+0000 7f3f4b7fe700 1 -- 172.21.15.5:0/3974845513 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c005300 con 0x7f3f54110b50 2023-07-21T19:26:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.675+0000 7f3f70117b80 1 Processor -- start 2023-07-21T19:26:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.675+0000 7f3f70117b80 1 -- start start 2023-07-21T19:26:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134186040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x5621341869d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213408bc50 con 0x56213418c2d0 2023-07-21T19:26:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621340de9e0 con 0x562134186580 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.676+0000 7f3f70117b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621340d7e70 con 0x562133fe26b0 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.677+0000 7f3f4b7fe700 1 -- 172.21.15.5:0/3974845513 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3f4c005460 con 0x7f3f54110b50 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.678+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x5621341869d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.678+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.678+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58426/0 (socket says 172.21.15.5:58426) 2023-07-21T19:26:55.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.678+0000 7f3f59d9a700 1 -- 172.21.15.5:0/1345963306 learned_addr learned my addr 172.21.15.5:0/1345963306 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.679+0000 7f3f59d9a700 1 -- 172.21.15.5:0/1345963306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 msgr2=0x5621341869d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.679+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x5621341869d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.679+0000 7f3f59d9a700 1 -- 172.21.15.5:0/1345963306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 msgr2=0x562134186040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.679+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134186040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.679+0000 7f3f59d9a700 1 -- 172.21.15.5:0/1345963306 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562133f05d80 con 0x562133fe26b0 2023-07-21T19:26:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.680+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f44014a30 tx=0x7f3f44005d60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10bef7dd100a0fb6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.681+0000 7f3f4b7fe700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 0x7f3f400a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.681+0000 7f3f4b7fe700 1 -- 172.21.15.5:0/3974845513 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f4c0ce370 con 0x7f3f54110b50 2023-07-21T19:26:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.682+0000 7f3f48ff9700 1 -- 172.21.15.5:0/1345963306 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44014110 con 0x562133fe26b0 2023-07-21T19:26:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.682+0000 7f3f48ff9700 1 -- 172.21.15.5:0/1345963306 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f4401a030 con 0x562133fe26b0 2023-07-21T19:26:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.682+0000 7f3f48ff9700 1 -- 172.21.15.5:0/1345963306 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44012740 con 0x562133fe26b0 2023-07-21T19:26:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.683+0000 7f3f70117b80 1 -- 172.21.15.5:0/1345963306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 msgr2=0x56213418bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.683+0000 7f3f70117b80 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f3f44014a30 tx=0x7f3f44005d60 comp rx=0 tx=0).stop 2023-07-21T19:26:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 0x7f3f400a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f70117b80 1 -- 172.21.15.5:0/1345963306 shutdown_connections 2023-07-21T19:26:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f70117b80 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x5621341869d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f70117b80 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562133fe26b0 0x56213418bd90 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f70117b80 1 --2- 172.21.15.5:0/1345963306 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134186040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.684+0000 7f3f70117b80 1 -- 172.21.15.5:0/1345963306 >> 172.21.15.5:0/1345963306 conn(0x56213417bbe0 msgr2=0x562134180f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.687+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 0x7f3f400a3870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f50005b40 tx=0x7f3f50005ab0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.687+0000 7f3f70117b80 1 -- 172.21.15.5:0/1345963306 shutdown_connections 2023-07-21T19:26:55.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.687+0000 7f3f70117b80 1 -- 172.21.15.5:0/1345963306 wait complete. 2023-07-21T19:26:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.688+0000 7f3f70117b80 1 -- start start 2023-07-21T19:26:55.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.688+0000 7f3f70117b80 1 Processor -- start 2023-07-21T19:26:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562133fe26b0 0x56213418f980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x56213418fec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562134077a60 con 0x56213418c2d0 2023-07-21T19:26:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562134077760 con 0x562133fe26b0 2023-07-21T19:26:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562133f5b8c0 con 0x562134186580 2023-07-21T19:26:55.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.689+0000 7f3f70117b80 10 client.0 _ll_register_callbacks cb 0x562134075700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.692+0000 7f3f5ad9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.692+0000 7f3f5ad9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57666/0 (socket says 172.21.15.5:57666) 2023-07-21T19:26:55.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.692+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3586196583 learned_addr learned my addr 172.21.15.5:0/3586196583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.693+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3586196583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 msgr2=0x56213418fec0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:26:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.693+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x56213418fec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.693+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562133fe26b0 msgr2=0x56213418f980 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:26:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.693+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562133fe26b0 0x56213418f980 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.693+0000 7f3f5ad9c700 1 -- 172.21.15.5:0/3586196583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f44014710 con 0x56213418c2d0 2023-07-21T19:26:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.695+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0cbac0 tx=0x7f3f4c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4bab8464a481e74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.695+0000 7f3f3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:26:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.696+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c09f4a0 con 0x56213418c2d0 2023-07-21T19:26:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.696+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f4c0a3830 con 0x56213418c2d0 2023-07-21T19:26:55.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.696+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562134031490 con 0x56213418c2d0 2023-07-21T19:26:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.697+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f4c0a10e0 con 0x56213418c2d0 2023-07-21T19:26:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.697+0000 7f3f70117b80 10 client.19374 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.697+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x562133eeec00 con 0x56213418c2d0 2023-07-21T19:26:55.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f4c0b7020 con 0x56213418c2d0 2023-07-21T19:26:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f4c09a960 con 0x56213418c2d0 2023-07-21T19:26:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 10 client.19374 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 10 client.19374 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:55.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 20 client.19374 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56213419f8e0 0x5621341a1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 4 client.19374 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:26:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.698+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621340d0e50 con 0x56213419f8e0 2023-07-21T19:26:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.699+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56213419f8e0 0x5621341a1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.700+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56213419f8e0 0x5621341a1cf0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=596e7519dfdb89d9 server_cookie=852a828d3f1a37e5 in_seq=0 out_seq=0 2023-07-21T19:26:55.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.700+0000 7f3f3dffb700 10 client.19374 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.701+0000 7f3f3dffb700 1 -- 172.21.15.5:0/3586196583 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5621340d0e50 con 0x56213419f8e0 2023-07-21T19:26:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.701+0000 7f3f3dffb700 10 client.19374 handle_command_reply: tid=0 2023-07-21T19:26:55.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.702+0000 7f3f70117b80 1 client.19374 shutdown 2023-07-21T19:26:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.702+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 msgr2=0x562134190400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.702+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c0cbac0 tx=0x7f3f4c013000 comp rx=0 tx=0).stop 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 shutdown_connections 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56213419f8e0 0x5621341a1cf0 crc :-1 s=READY pgs=389 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56213418c2d0 0x562134190400 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562133fe26b0 0x56213418f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3586196583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x56213418fec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 >> 172.21.15.5:0/3586196583 conn(0x56213417bbe0 msgr2=0x5621341804e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f3dffb700 0 client.19374 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 shutdown_connections 2023-07-21T19:26:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.703+0000 7f3f70117b80 1 -- 172.21.15.5:0/3586196583 wait complete. 2023-07-21T19:26:55.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.704+0000 7f3f70117b80 20 client.19374 trim_cache size 0 max 16384 2023-07-21T19:26:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.739+0000 7f3f70117b80 1 Processor -- start 2023-07-21T19:26:55.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.739+0000 7f3f70117b80 1 -- start start 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.739+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134186040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213418c2d0 0x562134196c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f70117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213417dd00 con 0x562133fe26b0 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f70117b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213417d5f0 con 0x562134186580 2023-07-21T19:26:55.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f70117b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56213418dee0 con 0x56213418c2d0 2023-07-21T19:26:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:44368/0 (socket says 172.21.15.5:44368) 2023-07-21T19:26:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.740+0000 7f3f59d9a700 1 -- 172.21.15.5:0/3141246881 learned_addr learned my addr 172.21.15.5:0/3141246881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213418c2d0 0x562134196c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f59d9a700 1 -- 172.21.15.5:0/3141246881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213418c2d0 msgr2=0x562134196c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213418c2d0 0x562134196c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f59d9a700 1 -- 172.21.15.5:0/3141246881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 msgr2=0x562134186040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134186040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.741+0000 7f3f59d9a700 1 -- 172.21.15.5:0/3141246881 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56213419c260 con 0x562134186580 2023-07-21T19:26:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.742+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400ebf0 tx=0x7f3f4400ea90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=411fefd52af69bf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.743+0000 7f3f497fa700 1 -- 172.21.15.5:0/3141246881 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4400e500 con 0x562134186580 2023-07-21T19:26:55.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.744+0000 7f3f497fa700 1 -- 172.21.15.5:0/3141246881 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f440044c0 con 0x562134186580 2023-07-21T19:26:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.744+0000 7f3f497fa700 1 -- 172.21.15.5:0/3141246881 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f44004710 con 0x562134186580 2023-07-21T19:26:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.744+0000 7f3f70117b80 1 -- 172.21.15.5:0/3141246881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 msgr2=0x56213418b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.744+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3f4400ebf0 tx=0x7f3f4400ea90 comp rx=0 tx=0).stop 2023-07-21T19:26:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.745+0000 7f3f70117b80 1 -- 172.21.15.5:0/3141246881 shutdown_connections 2023-07-21T19:26:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.745+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56213418c2d0 0x562134196c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.745+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134186040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.745+0000 7f3f70117b80 1 --2- 172.21.15.5:0/3141246881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562134186580 0x56213418b240 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.745+0000 7f3f70117b80 1 -- 172.21.15.5:0/3141246881 >> 172.21.15.5:0/3141246881 conn(0x56213417bbe0 msgr2=0x56213418ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.747+0000 7f3f70117b80 1 -- 172.21.15.5:0/3141246881 shutdown_connections 2023-07-21T19:26:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.748+0000 7f3f70117b80 1 -- 172.21.15.5:0/3141246881 wait complete. 2023-07-21T19:26:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.748+0000 7f3f70117b80 1 -- start start 2023-07-21T19:26:55.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.749+0000 7f3f70117b80 1 Processor -- start 2023-07-21T19:26:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134194c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56213418c2d0 0x5621341956d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621341f28b0 con 0x562133fe26b0 2023-07-21T19:26:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621341f2a20 con 0x56213418c2d0 2023-07-21T19:26:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.750+0000 7f3f70117b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621341f2be0 con 0x562134186580 2023-07-21T19:26:55.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.751+0000 7f3f70117b80 10 client.0 _ll_register_callbacks cb 0x7f3f4c0977b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:26:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.751+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.751+0000 7f3f59d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58438/0 (socket says 172.21.15.5:58438) 2023-07-21T19:26:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.751+0000 7f3f59d9a700 1 -- 172.21.15.5:0/2840929623 learned_addr learned my addr 172.21.15.5:0/2840929623 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:26:55.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f5ad9c700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56213418c2d0 0x5621341956d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f59d9a700 1 -- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56213418c2d0 msgr2=0x5621341956d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56213418c2d0 0x5621341956d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f59d9a700 1 -- 172.21.15.5:0/2840929623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 msgr2=0x562134194c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134194c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.752+0000 7f3f59d9a700 1 -- 172.21.15.5:0/2840929623 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562133f05d80 con 0x562134186580 2023-07-21T19:26:55.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.753+0000 7f3f59d9a700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f44005720 tx=0x7f3f44005750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=447cef3c3a8a2831 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:26:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.753+0000 7f3f3dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:26:55.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.753+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4402e070 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.753+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(114..114 src has 1..114) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3f4400e500 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.754+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3f4401a030 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.754+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3f4400e7d0 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.754+0000 7f3f70117b80 10 client.28918 fetch_fsmap learned FSMap version 56 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.754+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x562133f2c1f0 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.754+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3f4403a030 con 0x562134186580 2023-07-21T19:26:55.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f3f440116c0 con 0x562134186580 2023-07-21T19:26:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 10 client.28918 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:26:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 10 client.28918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:26:55.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 20 client.28918 populate_metadata read hostname 'smithi005' 2023-07-21T19:26:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621341f3e70 0x5621341f6250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:26:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 4 client.28918 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:26:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.755+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562134092ff0 con 0x5621341f3e70 2023-07-21T19:26:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.756+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621341f3e70 0x5621341f6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:26:55.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.756+0000 7f3f5a59b700 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621341f3e70 0x5621341f6250 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffe741394f3d1fc0 server_cookie=a986676947c75a6e in_seq=0 out_seq=0 2023-07-21T19:26:55.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.758+0000 7f3f3dffb700 10 client.28918 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.773+0000 7f3f3dffb700 1 -- 172.21.15.5:0/2840929623 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562134092ff0 con 0x5621341f3e70 2023-07-21T19:26:55.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.773+0000 7f3f3dffb700 10 client.28918 handle_command_reply: tid=0 2023-07-21T19:26:55.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.773+0000 7f3f70117b80 1 client.28918 shutdown 2023-07-21T19:26:55.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 msgr2=0x562134195190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f3f44005720 tx=0x7f3f44005750 comp rx=0 tx=0).stop 2023-07-21T19:26:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 shutdown_connections 2023-07-21T19:26:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56213418c2d0 0x5621341956d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621341f3e70 0x5621341f6250 crc :-1 s=READY pgs=390 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562133fe26b0 0x562134194c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 --2- 172.21.15.5:0/2840929623 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562134186580 0x562134195190 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 >> 172.21.15.5:0/2840929623 conn(0x56213417bbe0 msgr2=0x56213418e130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.777+0000 7f3f3dffb700 0 client.28918 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:26:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.778+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 shutdown_connections 2023-07-21T19:26:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.778+0000 7f3f70117b80 1 -- 172.21.15.5:0/2840929623 wait complete. 2023-07-21T19:26:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.779+0000 7f3f70117b80 20 client.28918 trim_cache size 0 max 16384 2023-07-21T19:26:55.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:26:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:26:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:26:55.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:26:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 msgr2=0x7f3f400a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 0x7f3f400a3870 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f3f50005b40 tx=0x7f3f50005ab0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 msgr2=0x7f3f5419c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:26:55.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c006050 tx=0x7f3f4c00d1f0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 shutdown_connections 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3f54110b50 0x7f3f5419c2f0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3f54107780 0x7f3f541a32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3f400a13c0 0x7f3f400a3870 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 --2- 172.21.15.5:0/3974845513 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3f54108090 0x7f3f541a3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:26:55.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.781+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 >> 172.21.15.5:0/3974845513 conn(0x7f3f5406c270 msgr2=0x7f3f5406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:26:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.786+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 shutdown_connections 2023-07-21T19:26:55.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:26:55.786+0000 7f3f5c7ff700 1 -- 172.21.15.5:0/3974845513 wait complete. 2023-07-21T19:26:55.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:55 smithi005 ceph-mon[126460]: pgmap v608: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:55.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:55 smithi156 ceph-mon[128316]: pgmap v608: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:56.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:55 smithi101 ceph-mon[128136]: pgmap v608: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 25.0 s, 620.9 tps, lat 160.708 ms stddev 51.135 2023-07-21T19:26:57.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:56 smithi101 ceph-mon[128136]: pgmap v609: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:57.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:56 smithi005 ceph-mon[126460]: pgmap v609: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:57.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:56 smithi156 ceph-mon[128316]: pgmap v609: 97 pgs: 97 active+clean; 16 GiB data, 9.5 GiB used, 1.0 TiB / 1.0 TiB avail; 325 KiB/s rd, 16 MiB/s wr, 1.11k op/s 2023-07-21T19:26:59.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:26:59 smithi005 ceph-mon[126460]: pgmap v610: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-07-21T19:26:59.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:26:59 smithi156 ceph-mon[128316]: pgmap v610: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-07-21T19:27:00.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:26:59 smithi101 ceph-mon[128136]: pgmap v610: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 528 KiB/s rd, 25 MiB/s wr, 1.75k op/s 2023-07-21T19:27:01.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:01 smithi005 ceph-mon[126460]: pgmap v611: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2023-07-21T19:27:01.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:01 smithi156 ceph-mon[128316]: pgmap v611: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2023-07-21T19:27:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 30.0 s, 678.2 tps, lat 146.931 ms stddev 44.009 2023-07-21T19:27:02.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:01 smithi101 ceph-mon[128136]: pgmap v611: 97 pgs: 97 active+clean; 16 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 439 KiB/s rd, 18 MiB/s wr, 1.22k op/s 2023-07-21T19:27:03.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:03 smithi005 ceph-mon[126460]: pgmap v612: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2023-07-21T19:27:03.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:03 smithi156 ceph-mon[128316]: pgmap v612: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2023-07-21T19:27:04.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:03 smithi101 ceph-mon[128136]: pgmap v612: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 647 KiB/s rd, 27 MiB/s wr, 1.85k op/s 2023-07-21T19:27:05.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:05 smithi005 ceph-mon[126460]: pgmap v613: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:05.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:05 smithi156 ceph-mon[128316]: pgmap v613: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:06.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:05 smithi101 ceph-mon[128136]: pgmap v613: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:06.735 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:27:06.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:06.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:06.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:06 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:06.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:06 smithi005 ceph-mon[126460]: pgmap v614: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:06.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:06.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:06.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:06 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:06.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:06 smithi156 ceph-mon[128316]: pgmap v614: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 35.0 s, 623.1 tps, lat 154.241 ms stddev 56.505 2023-07-21T19:27:07.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:07.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:07.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:06 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:07.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:06 smithi101 ceph-mon[128136]: pgmap v614: 97 pgs: 97 active+clean; 16 GiB data, 9.8 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 18 MiB/s wr, 1.28k op/s 2023-07-21T19:27:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 -- 172.21.15.5:0/2730944276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b810e280 msgr2=0x7f02b810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 --2- 172.21.15.5:0/2730944276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b810e280 0x7f02b810e650 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f02b4009f90 tx=0x7f02b400b330 comp rx=0 tx=0).stop 2023-07-21T19:27:07.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 -- 172.21.15.5:0/2730944276 shutdown_connections 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 --2- 172.21.15.5:0/2730944276 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b806d3a0 0x7f02b811b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 --2- 172.21.15.5:0/2730944276 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b810e280 0x7f02b810e650 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 --2- 172.21.15.5:0/2730944276 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 -- 172.21.15.5:0/2730944276 >> 172.21.15.5:0/2730944276 conn(0x7f02b806bf70 msgr2=0x7f02b8070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.380+0000 7f02bebff700 1 -- 172.21.15.5:0/2730944276 shutdown_connections 2023-07-21T19:27:07.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.381+0000 7f02bebff700 1 -- 172.21.15.5:0/2730944276 wait complete. 2023-07-21T19:27:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.382+0000 7f02bebff700 1 Processor -- start 2023-07-21T19:27:07.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 -- start start 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b806d3a0 0x7f02b8117210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 0x7f02b81a7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81a8480 con 0x7f02b806d3a0 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81a85f0 con 0x7f02b810e280 2023-07-21T19:27:07.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.383+0000 7f02bebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02b81a87b0 con 0x7f02b806c9a0 2023-07-21T19:27:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.384+0000 7f02bd19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 0x7f02b81a7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.384+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.384+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44450/0 (socket says 172.21.15.5:44450) 2023-07-21T19:27:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.384+0000 7f02bc99b700 1 -- 172.21.15.5:0/2330663962 learned_addr learned my addr 172.21.15.5:0/2330663962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:07.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 -- 172.21.15.5:0/2330663962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 msgr2=0x7f02b8116cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 -- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b806d3a0 msgr2=0x7f02b8117210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b806d3a0 0x7f02b8117210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 -- 172.21.15.5:0/2330663962 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02b4009c40 con 0x7f02b810e280 2023-07-21T19:27:07.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bc99b700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.385+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 0x7f02b81a7f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b3c0 tx=0x7f02b000b6d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.386+0000 7f02adffb700 1 -- 172.21.15.5:0/2330663962 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0011830 con 0x7f02b810e280 2023-07-21T19:27:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.386+0000 7f02adffb700 1 -- 172.21.15.5:0/2330663962 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b000be80 con 0x7f02b810e280 2023-07-21T19:27:07.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.386+0000 7f02adffb700 1 -- 172.21.15.5:0/2330663962 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0010c60 con 0x7f02b810e280 2023-07-21T19:27:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.386+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02b81a8a90 con 0x7f02b810e280 2023-07-21T19:27:07.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.386+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02b81a8f50 con 0x7f02b810e280 2023-07-21T19:27:07.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.393+0000 7f02d25ccb80 1 Processor -- start 2023-07-21T19:27:07.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.395+0000 7f02adffb700 1 -- 172.21.15.5:0/2330663962 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f02b001a400 con 0x7f02b810e280 2023-07-21T19:27:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.397+0000 7f02adffb700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 0x7f02a00a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.398+0000 7f02adffb700 1 -- 172.21.15.5:0/2330663962 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f02b00cc4d0 con 0x7f02b810e280 2023-07-21T19:27:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.398+0000 7f02d25ccb80 1 -- start start 2023-07-21T19:27:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02affff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 0x7f02a00a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d46a7cb0 con 0x55a6d47acad0 2023-07-21T19:27:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d46faa40 con 0x55a6d47a1be0 2023-07-21T19:27:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.399+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d46f3ed0 con 0x55a6d47a12d0 2023-07-21T19:27:07.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.400+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.400+0000 7f02affff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.400+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44466/0 (socket says 172.21.15.5:44466) 2023-07-21T19:27:07.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.400+0000 7f02bc99b700 1 -- 172.21.15.5:0/4214281410 learned_addr learned my addr 172.21.15.5:0/4214281410 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.401+0000 7f02affff700 1 -- 172.21.15.5:0/4214281410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 msgr2=0x55a6d47a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.401+0000 7f02affff700 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47a16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.401+0000 7f02affff700 1 -- 172.21.15.5:0/4214281410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 msgr2=0x55a6d47aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.401+0000 7f02affff700 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.401+0000 7f02affff700 1 -- 172.21.15.5:0/4214281410 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6d4521de0 con 0x55a6d47a1be0 2023-07-21T19:27:07.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.402+0000 7f02affff700 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47ac590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005e50 tx=0x7f02a8028000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f3d49d1a47e90df server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.402+0000 7f02affff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 0x7f02a00a3980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005950 tx=0x7f02a80257b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.402+0000 7f029effd700 1 -- 172.21.15.5:0/4214281410 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8025110 con 0x55a6d47a1be0 2023-07-21T19:27:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.402+0000 7f029effd700 1 -- 172.21.15.5:0/4214281410 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02a8025270 con 0x55a6d47a1be0 2023-07-21T19:27:07.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.402+0000 7f029effd700 1 -- 172.21.15.5:0/4214281410 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02a8005020 con 0x55a6d47a1be0 2023-07-21T19:27:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.406+0000 7f02d25ccb80 1 -- 172.21.15.5:0/4214281410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 msgr2=0x55a6d47ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.406+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47ac590 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005e50 tx=0x7f02a8028000 comp rx=0 tx=0).stop 2023-07-21T19:27:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.407+0000 7f02d25ccb80 1 -- 172.21.15.5:0/4214281410 shutdown_connections 2023-07-21T19:27:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.407+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.407+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.407+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/4214281410 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47ac590 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.407+0000 7f02d25ccb80 1 -- 172.21.15.5:0/4214281410 >> 172.21.15.5:0/4214281410 conn(0x55a6d4797da0 msgr2=0x55a6d479d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.411+0000 7f02d25ccb80 1 -- 172.21.15.5:0/4214281410 shutdown_connections 2023-07-21T19:27:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.412+0000 7f02d25ccb80 1 -- 172.21.15.5:0/4214281410 wait complete. 2023-07-21T19:27:07.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.413+0000 7f02d25ccb80 1 -- start start 2023-07-21T19:27:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.413+0000 7f02d25ccb80 1 Processor -- start 2023-07-21T19:27:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 0x55a6d47a9930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a1be0 0x55a6d47a9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47aa3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d4693ac0 con 0x55a6d47a1be0 2023-07-21T19:27:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d46937c0 con 0x55a6d47a12d0 2023-07-21T19:27:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d4577920 con 0x55a6d47acad0 2023-07-21T19:27:07.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.414+0000 7f02d25ccb80 10 client.0 _ll_register_callbacks cb 0x55a6d4691760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.415+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 0x55a6d47a9930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.415+0000 7f02bc99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 0x55a6d47a9930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55796/0 (socket says 172.21.15.5:55796) 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.415+0000 7f02bc99b700 1 -- 172.21.15.5:0/2244366121 learned_addr learned my addr 172.21.15.5:0/2244366121 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.417+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47aa3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 -- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 msgr2=0x55a6d47a9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 0x55a6d47a9930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 -- 172.21.15.5:0/2244366121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a1be0 msgr2=0x55a6d47a9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a1be0 0x55a6d47a9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 -- 172.21.15.5:0/2244366121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a802b070 con 0x55a6d47acad0 2023-07-21T19:27:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.418+0000 7f02bd19c700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47aa3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02b00c9cb0 tx=0x7f02b0099010 comp rx=0 tx=0).ready entity=mon.1 client_cookie=376800f43c8e204f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.421+0000 7f0297fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:07.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.421+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0099a90 con 0x55a6d47acad0 2023-07-21T19:27:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.421+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f02b009d660 con 0x55a6d47acad0 2023-07-21T19:27:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.422+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a6d464d4f0 con 0x55a6d47acad0 2023-07-21T19:27:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.426+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b00a1a80 con 0x55a6d47acad0 2023-07-21T19:27:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.426+0000 7f02d25ccb80 10 client.28930 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:07.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.426+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55a6d450ac60 con 0x55a6d47acad0 2023-07-21T19:27:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a6d464d4f0 con 0x55a6d47acad0 2023-07-21T19:27:07.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f02b00a15f0 con 0x55a6d47acad0 2023-07-21T19:27:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 10 client.28930 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 10 client.28930 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 20 client.28930 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d47bc5e0 0x55a6d47be9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 4 client.28930 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.435+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a6d46eceb0 con 0x55a6d47bc5e0 2023-07-21T19:27:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.436+0000 7f02bc99b700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d47bc5e0 0x55a6d47be9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.437+0000 7f02bc99b700 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d47bc5e0 0x55a6d47be9f0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8439a246b6b54f0d server_cookie=8f6df8073bb7a5b0 in_seq=0 out_seq=0 2023-07-21T19:27:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.437+0000 7f0297fff700 10 client.28930 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.439+0000 7f0297fff700 1 -- 172.21.15.5:0/2244366121 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a6d46eceb0 con 0x55a6d47bc5e0 2023-07-21T19:27:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.439+0000 7f0297fff700 10 client.28930 handle_command_reply: tid=0 2023-07-21T19:27:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.439+0000 7f02d25ccb80 1 client.28930 shutdown 2023-07-21T19:27:07.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.457+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 msgr2=0x55a6d47aa3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.457+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47aa3b0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f02b00c9cb0 tx=0x7f02b0099010 comp rx=0 tx=0).stop 2023-07-21T19:27:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 shutdown_connections 2023-07-21T19:27:07.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47aa3b0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a12d0 0x55a6d47a9930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d47bc5e0 0x55a6d47be9f0 crc :-1 s=READY pgs=391 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/2244366121 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a1be0 0x55a6d47a9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 >> 172.21.15.5:0/2244366121 conn(0x55a6d4797da0 msgr2=0x55a6d479d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.458+0000 7f0297fff700 0 client.28930 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.462+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 shutdown_connections 2023-07-21T19:27:07.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.462+0000 7f02d25ccb80 1 -- 172.21.15.5:0/2244366121 wait complete. 2023-07-21T19:27:07.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.463+0000 7f02d25ccb80 20 client.28930 trim_cache size 0 max 16384 2023-07-21T19:27:07.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.504+0000 7f02d25ccb80 1 Processor -- start 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 -- start start 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47b34a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47a41b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47a2450 con 0x55a6d47acad0 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47a7ed0 con 0x55a6d47a1be0 2023-07-21T19:27:07.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.510+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47b39e0 con 0x55a6d47a12d0 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.511+0000 7f02bd19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.511+0000 7f02bd19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39304/0 (socket says 172.21.15.5:39304) 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.511+0000 7f02bd19c700 1 -- 172.21.15.5:0/914331338 learned_addr learned my addr 172.21.15.5:0/914331338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bc99b700 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47b34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 -- 172.21.15.5:0/914331338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 msgr2=0x55a6d47b34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47b34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 -- 172.21.15.5:0/914331338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 msgr2=0x55a6d47a41b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:27:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47a41b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 -- 172.21.15.5:0/914331338 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a6d4548250 con 0x55a6d47acad0 2023-07-21T19:27:07.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.512+0000 7f02bd19c700 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f02b00960a0 tx=0x7f02b0009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dbcfd434e69cc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.518+0000 7f029f7fe700 1 -- 172.21.15.5:0/914331338 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b00b5030 con 0x55a6d47acad0 2023-07-21T19:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.518+0000 7f029f7fe700 1 -- 172.21.15.5:0/914331338 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b009d660 con 0x55a6d47acad0 2023-07-21T19:27:07.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.518+0000 7f029f7fe700 1 -- 172.21.15.5:0/914331338 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0098b20 con 0x55a6d47acad0 2023-07-21T19:27:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.519+0000 7f02d25ccb80 1 -- 172.21.15.5:0/914331338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 msgr2=0x55a6d47a4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.519+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f02b00960a0 tx=0x7f02b0009210 comp rx=0 tx=0).stop 2023-07-21T19:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.521+0000 7f02d25ccb80 1 -- 172.21.15.5:0/914331338 shutdown_connections 2023-07-21T19:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.521+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47acad0 0x55a6d47a4c70 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.521+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47a12d0 0x55a6d47b34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.521+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/914331338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47a41b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.521+0000 7f02d25ccb80 1 -- 172.21.15.5:0/914331338 >> 172.21.15.5:0/914331338 conn(0x55a6d4797da0 msgr2=0x55a6d47b87f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.523+0000 7f02d25ccb80 1 -- 172.21.15.5:0/914331338 shutdown_connections 2023-07-21T19:27:07.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.523+0000 7f02d25ccb80 1 -- 172.21.15.5:0/914331338 wait complete. 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.524+0000 7f02d25ccb80 1 -- start start 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.524+0000 7f02d25ccb80 1 Processor -- start 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a12d0 0x55a6d47b3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47b1490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47b1fb0 con 0x55a6d47a12d0 2023-07-21T19:27:07.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47b2120 con 0x55a6d47a1be0 2023-07-21T19:27:07.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a6d47b22e0 con 0x55a6d47acad0 2023-07-21T19:27:07.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.525+0000 7f02d25ccb80 10 client.0 _ll_register_callbacks cb 0x7f02b0099440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:07.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.526+0000 7f02bd19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.526+0000 7f02bd19c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44490/0 (socket says 172.21.15.5:44490) 2023-07-21T19:27:07.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.526+0000 7f02bd19c700 1 -- 172.21.15.5:0/3205903408 learned_addr learned my addr 172.21.15.5:0/3205903408 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.526+0000 7f02bc99b700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a12d0 0x55a6d47b3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.527+0000 7f02bd19c700 1 -- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 msgr2=0x55a6d47b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.527+0000 7f02bd19c700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.527+0000 7f02bd19c700 1 -- 172.21.15.5:0/3205903408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a12d0 msgr2=0x55a6d47b3170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.527+0000 7f02bd19c700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a12d0 0x55a6d47b3170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.527+0000 7f02bd19c700 1 -- 172.21.15.5:0/3205903408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a6d450ac60 con 0x55a6d47acad0 2023-07-21T19:27:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.530+0000 7f02bd19c700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02b0004820 tx=0x7f02b0095970 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d87dc78f717d8c89 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.530+0000 7f0297fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.531+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02b0099a90 con 0x55a6d47acad0 2023-07-21T19:27:07.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.531+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f02b00b6070 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.532+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02a8025270 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.532+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02b0010380 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.532+0000 7f02d25ccb80 10 client.28936 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.532+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55a6d45202b0 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.533+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02b00a5020 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.533+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f02b009aa90 con 0x55a6d47acad0 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 10 client.28936 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:07.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 10 client.28936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 20 client.28936 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d4811510 0x55a6d48138f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 4 client.28936 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:27:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.534+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a6d46af050 con 0x55a6d4811510 2023-07-21T19:27:07.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.535+0000 7f02bc99b700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d4811510 0x55a6d48138f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.537+0000 7f02bc99b700 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d4811510 0x55a6d48138f0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=63e97e01dcfadfa9 server_cookie=22c019d5d8c05f28 in_seq=0 out_seq=0 2023-07-21T19:27:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.537+0000 7f0297fff700 10 client.28936 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.538+0000 7f0297fff700 1 -- 172.21.15.5:0/3205903408 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a6d46af050 con 0x55a6d4811510 2023-07-21T19:27:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.538+0000 7f0297fff700 10 client.28936 handle_command_reply: tid=0 2023-07-21T19:27:07.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.543+0000 7f02d25ccb80 1 client.28936 shutdown 2023-07-21T19:27:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.558+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 msgr2=0x55a6d47b19d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.558+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f02b0004820 tx=0x7f02b0095970 comp rx=0 tx=0).stop 2023-07-21T19:27:07.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 shutdown_connections 2023-07-21T19:27:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a6d47acad0 0x55a6d47b19d0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a6d4811510 0x55a6d48138f0 crc :-1 s=READY pgs=392 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a6d47a12d0 0x55a6d47b3170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 --2- 172.21.15.5:0/3205903408 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a6d47a1be0 0x55a6d47b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 >> 172.21.15.5:0/3205903408 conn(0x55a6d4797da0 msgr2=0x55a6d47a4c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.559+0000 7f0297fff700 0 client.28936 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.561+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 shutdown_connections 2023-07-21T19:27:07.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.561+0000 7f02d25ccb80 1 -- 172.21.15.5:0/3205903408 wait complete. 2023-07-21T19:27:07.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.562+0000 7f02d25ccb80 20 client.28936 trim_cache size 0 max 16384 2023-07-21T19:27:07.555 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:07.555 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:27:07.555 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:27:07.556 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.565+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 msgr2=0x7f02a00a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.565+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 0x7f02a00a3980 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005950 tx=0x7f02a80257b0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.565+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 msgr2=0x7f02b81a7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.565+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 0x7f02b81a7f40 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f02b000b3c0 tx=0x7f02b000b6d0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 shutdown_connections 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02b810e280 0x7f02b81a7f40 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02b806c9a0 0x7f02b8116cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02a00a14d0 0x7f02a00a3980 secure :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f02a8005950 tx=0x7f02a80257b0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 --2- 172.21.15.5:0/2330663962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02b806d3a0 0x7f02b8117210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:07.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.566+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 >> 172.21.15.5:0/2330663962 conn(0x7f02b806bf70 msgr2=0x7f02b80708c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:07.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.567+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 shutdown_connections 2023-07-21T19:27:07.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:07.568+0000 7f02bebff700 1 -- 172.21.15.5:0/2330663962 wait complete. 2023-07-21T19:27:07.607 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:27:07.607 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:27:07.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:07 smithi005 ceph-mon[126460]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:27:08.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.078+0000 7f2b227ff700 1 -- 172.21.15.5:0/1300704750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c0ffb00 msgr2=0x7f2b1c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:08.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.078+0000 7f2b227ff700 1 --2- 172.21.15.5:0/1300704750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c0ffb00 0x7f2b1c0fff70 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2b1401c0d0 tx=0x7f2b1401c3e0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:07 smithi101 ceph-mon[128136]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:27:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 -- 172.21.15.5:0/1300704750 shutdown_connections 2023-07-21T19:27:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 --2- 172.21.15.5:0/1300704750 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c1004b0 0x7f2b1c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 --2- 172.21.15.5:0/1300704750 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2b1c105b10 0x7f2b1c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 --2- 172.21.15.5:0/1300704750 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c0ffb00 0x7f2b1c0fff70 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 -- 172.21.15.5:0/1300704750 >> 172.21.15.5:0/1300704750 conn(0x7f2b1c0fb600 msgr2=0x7f2b1c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 -- 172.21.15.5:0/1300704750 shutdown_connections 2023-07-21T19:27:08.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.081+0000 7f2b227ff700 1 -- 172.21.15.5:0/1300704750 wait complete. 2023-07-21T19:27:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.083+0000 7f2b227ff700 1 Processor -- start 2023-07-21T19:27:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 -- start start 2023-07-21T19:27:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2b1c0ffb00 0x7f2b1c06c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c1004b0 0x7f2b1c06c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:08.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c070f10 con 0x7f2b1c105b10 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c071080 con 0x7f2b1c0ffb00 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.084+0000 7f2b227ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b1c071240 con 0x7f2b1c1004b0 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.085+0000 7f2b20d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.085+0000 7f2b20d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39338/0 (socket says 172.21.15.5:39338) 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.085+0000 7f2b20d9c700 1 -- 172.21.15.5:0/321656104 learned_addr learned my addr 172.21.15.5:0/321656104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:08.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 -- 172.21.15.5:0/321656104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c1004b0 msgr2=0x7f2b1c06c690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c1004b0 0x7f2b1c06c690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 -- 172.21.15.5:0/321656104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2b1c0ffb00 msgr2=0x7f2b1c06c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2b1c0ffb00 0x7f2b1c06c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 -- 172.21.15.5:0/321656104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1401ad10 con 0x7f2b1c105b10 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.086+0000 7f2b20d9c700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b550 tx=0x7f2b1000b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.087+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10011830 con 0x7f2b1c105b10 2023-07-21T19:27:08.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.087+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2b10011e20 con 0x7f2b1c105b10 2023-07-21T19:27:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.087+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b1c071520 con 0x7f2b1c105b10 2023-07-21T19:27:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.087+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b1c071a30 con 0x7f2b1c105b10 2023-07-21T19:27:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.089+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b10010da0 con 0x7f2b1c105b10 2023-07-21T19:27:08.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.090+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b08005320 con 0x7f2b1c105b10 2023-07-21T19:27:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.094+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2b10011990 con 0x7f2b1c105b10 2023-07-21T19:27:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.096+0000 7f2b197fa700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 0x7f2b040c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:08.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.097+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f2b100cc620 con 0x7f2b1c105b10 2023-07-21T19:27:08.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.101+0000 7f2b1bfff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 0x7f2b040c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:08.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.103+0000 7f2b1bfff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 0x7f2b040c5900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005950 tx=0x7f2b0c0058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:08.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.104+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2b10095ac0 con 0x7f2b1c105b10 2023-07-21T19:27:08.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:07 smithi156 ceph-mon[128316]: osdmap e115: 12 total, 12 up, 12 in 2023-07-21T19:27:08.607 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:27:08.607 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:25:53.614690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:27:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.615+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2b08005190 con 0x7f2b1c105b10 2023-07-21T19:27:08.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.616+0000 7f2b197fa700 1 -- 172.21.15.5:0/321656104 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f2b100990e0 con 0x7f2b1c105b10 2023-07-21T19:27:08.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 msgr2=0x7f2b040c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 0x7f2b040c5900 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005950 tx=0x7f2b0c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 msgr2=0x7f2b1c06cbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:08.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f2b1000b550 tx=0x7f2b1000b910 comp rx=0 tx=0).stop 2023-07-21T19:27:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 shutdown_connections 2023-07-21T19:27:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2b040c3450 0x7f2b040c5900 secure :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005950 tx=0x7f2b0c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2b1c105b10 0x7f2b1c06cbd0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2b1c0ffb00 0x7f2b1c06c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 --2- 172.21.15.5:0/321656104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2b1c1004b0 0x7f2b1c06c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:08.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 >> 172.21.15.5:0/321656104 conn(0x7f2b1c0fb600 msgr2=0x7f2b1c0fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:08.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.621+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 shutdown_connections 2023-07-21T19:27:08.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:08.622+0000 7f2b227ff700 1 -- 172.21.15.5:0/321656104 wait complete. 2023-07-21T19:27:08.615 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 56 2023-07-21T19:27:08.645 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:27:08.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:08 smithi005 ceph-mon[126460]: pgmap v616: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:08.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/321656104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:09.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.038+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3611128465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 msgr2=0x7fe3d0118d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.038+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3611128465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 0x7fe3d0118d80 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe3c001a9a0 tx=0x7fe3c001acb0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3611128465 shutdown_connections 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3611128465 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d011b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3611128465 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d006bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3611128465 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 0x7fe3d0118d80 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3611128465 >> 172.21.15.5:0/3611128465 conn(0x7fe3d006a980 msgr2=0x7fe3d006ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3611128465 shutdown_connections 2023-07-21T19:27:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.039+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3611128465 wait complete. 2023-07-21T19:27:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.041+0000 7fe3d4fff700 1 Processor -- start 2023-07-21T19:27:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.041+0000 7fe3d4fff700 1 -- start start 2023-07-21T19:27:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 0x7fe3d0114af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d01a7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a8470 con 0x7fe3d006bbc0 2023-07-21T19:27:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a85e0 con 0x7fe3d01192c0 2023-07-21T19:27:09.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.042+0000 7fe3d4fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3d01a87a0 con 0x7fe3d006c4d0 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.043+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.043+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39372/0 (socket says 172.21.15.5:39372) 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.043+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3662546813 learned_addr learned my addr 172.21.15.5:0/3662546813 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.043+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d01a7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3662546813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 msgr2=0x7fe3d0114af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 0x7fe3d0114af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3662546813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 msgr2=0x7fe3d01a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d01a7f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3662546813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3c001a600 con 0x7fe3d006bbc0 2023-07-21T19:27:09.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.044+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d01a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:09.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.045+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2484294313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00fea30 msgr2=0x7fc5b00feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.045+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2484294313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00fea30 0x7fc5b00feea0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001a9a0 tx=0x7fc5a001acb0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.046+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ba50 tx=0x7fe3c800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fe3bf7fe700 1 -- 172.21.15.5:0/3662546813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800c9b0 con 0x7fe3d006bbc0 2023-07-21T19:27:09.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fe3bf7fe700 1 -- 172.21.15.5:0/3662546813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c8014440 con 0x7fe3d006bbc0 2023-07-21T19:27:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fe3bf7fe700 1 -- 172.21.15.5:0/3662546813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80075d0 con 0x7fe3d006bbc0 2023-07-21T19:27:09.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3d01a8a80 con 0x7fe3d006bbc0 2023-07-21T19:27:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2484294313 shutdown_connections 2023-07-21T19:27:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2484294313 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2484294313 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b0104a30 0x7fc5b0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2484294313 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00fea30 0x7fc5b00feea0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2484294313 >> 172.21.15.5:0/2484294313 conn(0x7fc5b0074760 msgr2=0x7fc5b0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2484294313 shutdown_connections 2023-07-21T19:27:09.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2484294313 wait complete. 2023-07-21T19:27:09.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.049+0000 7fc5b4dff700 1 Processor -- start 2023-07-21T19:27:09.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.047+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d01a8f40 con 0x7fe3d006bbc0 2023-07-21T19:27:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.049+0000 7fc5b4dff700 1 -- start start 2023-07-21T19:27:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.049+0000 7fc5b4dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b00fea30 0x7fc5b006c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.049+0000 7fc5b4dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.050+0000 7fc5b4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b0104a30 0x7fc5b0112380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.050+0000 7fc5b4dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b01128c0 con 0x7fc5b00fea30 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.050+0000 7fc5b4dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b0112a30 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.050+0000 7fc5b4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b0112bf0 con 0x7fc5b0104a30 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.051+0000 7fc5add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.051+0000 7fc5add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55846/0 (socket says 172.21.15.5:55846) 2023-07-21T19:27:09.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.051+0000 7fc5add9b700 1 -- 172.21.15.5:0/2626757358 learned_addr learned my addr 172.21.15.5:0/2626757358 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 -- 172.21.15.5:0/2626757358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b0104a30 msgr2=0x7fc5b0112380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b0104a30 0x7fc5b0112380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 -- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b00fea30 msgr2=0x7fc5b006c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b00fea30 0x7fc5b006c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 -- 172.21.15.5:0/2626757358 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a001a600 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.052+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001a8f0 tx=0x7fc5a0005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.053+0000 7fc59f7fe700 1 -- 172.21.15.5:0/2626757358 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a002d070 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.053+0000 7fc59f7fe700 1 -- 172.21.15.5:0/2626757358 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5a0031410 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.053+0000 7fc59f7fe700 1 -- 172.21.15.5:0/2626757358 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0031ea0 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.053+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b0112e10 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.054+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b0113270 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.055+0000 7fe3e89f6b80 1 Processor -- start 2023-07-21T19:27:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 -- start start 2023-07-21T19:27:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a6590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f0a1cb0 con 0x563f5f19bbe0 2023-07-21T19:27:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f0f4a40 con 0x563f5f1a6ad0 2023-07-21T19:27:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f0eded0 con 0x563f5f19b2d0 2023-07-21T19:27:09.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.056+0000 7fe3cdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55858/0 (socket says 172.21.15.5:55858) 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 -- 172.21.15.5:0/4235261095 learned_addr learned my addr 172.21.15.5:0/4235261095 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 -- 172.21.15.5:0/4235261095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 msgr2=0x563f5f19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 -- 172.21.15.5:0/4235261095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 msgr2=0x563f5f1a6590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a6590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.057+0000 7fe3cdd9b700 1 -- 172.21.15.5:0/4235261095 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f5ef1bde0 con 0x563f5f1a6ad0 2023-07-21T19:27:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.058+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3c001a8f0 tx=0x7fe3c0003680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ff0953ecfea8613f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.059+0000 7fe3bf7fe700 1 -- 172.21.15.5:0/3662546813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe3c80145b0 con 0x7fe3d006bbc0 2023-07-21T19:27:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.059+0000 7fe3bcff9700 1 -- 172.21.15.5:0/4235261095 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c001ca60 con 0x563f5f1a6ad0 2023-07-21T19:27:09.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.059+0000 7fe3bcff9700 1 -- 172.21.15.5:0/4235261095 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c0005560 con 0x563f5f1a6ad0 2023-07-21T19:27:09.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.059+0000 7fe3bcff9700 1 -- 172.21.15.5:0/4235261095 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c0003d70 con 0x563f5f1a6ad0 2023-07-21T19:27:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.060+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4235261095 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 msgr2=0x563f5f1a8eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.060+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fe3c001a8f0 tx=0x7fe3c0003680 comp rx=0 tx=0).stop 2023-07-21T19:27:09.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4235261095 shutdown_connections 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a8eb0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4235261095 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a6590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4235261095 >> 172.21.15.5:0/4235261095 conn(0x563f5f191da0 msgr2=0x563f5f1976a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4235261095 shutdown_connections 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3bf7fe700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 0x7fe3b80a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.061+0000 7fe3bf7fe700 1 -- 172.21.15.5:0/3662546813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe3c8006e40 con 0x7fe3d006bbc0 2023-07-21T19:27:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.063+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 0x7fe3b80a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.063+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4235261095 wait complete. 2023-07-21T19:27:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.064+0000 7fc5c8653b80 1 Processor -- start 2023-07-21T19:27:09.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.064+0000 7fc59f7fe700 1 -- 172.21.15.5:0/2626757358 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc5a0003b20 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.064+0000 7fe3e89f6b80 1 -- start start 2023-07-21T19:27:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.064+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 0x7fe3b80a3810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005850 tx=0x7fe3c001a320 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.064+0000 7fe3e89f6b80 1 Processor -- start 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f08dac0 con 0x563f5f19bbe0 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f08d7c0 con 0x563f5f1a6ad0 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5ef71920 con 0x563f5f19b2d0 2023-07-21T19:27:09.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.065+0000 7fe3e89f6b80 10 client.0 _ll_register_callbacks cb 0x563f5f08b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44564/0 (socket says 172.21.15.5:44564) 2023-07-21T19:27:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4152126134 learned_addr learned my addr 172.21.15.5:0/4152126134 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4152126134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 msgr2=0x563f5f1a42f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a42f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4152126134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 msgr2=0x563f5f1a3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4152126134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3d011bf90 con 0x563f5f19b2d0 2023-07-21T19:27:09.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:27:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80960a0 tx=0x7fe3c8099630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1cb495dbb4e344b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fe3b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c8098d20 con 0x563f5f19b2d0 2023-07-21T19:27:09.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe3c80a1a80 con 0x563f5f19b2d0 2023-07-21T19:27:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.066+0000 7fc59f7fe700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 0x7fc5980a3970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fc59f7fe700 1 -- 172.21.15.5:0/2626757358 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc5a00de4f0 con 0x7fc5b00ff3e0 2023-07-21T19:27:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f5f0474f0 con 0x563f5f19b2d0 2023-07-21T19:27:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c80b90d0 con 0x563f5f19b2d0 2023-07-21T19:27:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3e89f6b80 10 client.28948 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.067+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563f5ef04c60 con 0x563f5f19b2d0 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 -- start start 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x557323091d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732308c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322f91c50 con 0x55732308c580 2023-07-21T19:27:09.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322fe49e0 con 0x5573230922d0 2023-07-21T19:27:09.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322fdde70 con 0x557322ee86b0 2023-07-21T19:27:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c80c7080 con 0x563f5f19b2d0 2023-07-21T19:27:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3c8095420 con 0x563f5f19b2d0 2023-07-21T19:27:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 10 client.28948 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 10 client.28948 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:27:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 20 client.28948 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f1b6460 0x563f5f1b8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 4 client.28948 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f5f0e6eb0 con 0x563f5f1b6460 2023-07-21T19:27:09.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.068+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f1b6460 0x563f5f1b8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55872/0 (socket says 172.21.15.5:55872) 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 -- 172.21.15.5:0/404228833 learned_addr learned my addr 172.21.15.5:0/404228833 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 -- 172.21.15.5:0/404228833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 msgr2=0x557323091d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x557323091d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 -- 172.21.15.5:0/404228833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 msgr2=0x55732308c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732308c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 -- 172.21.15.5:0/404228833 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557322e0bd80 con 0x5573230922d0 2023-07-21T19:27:09.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.069+0000 7fc5add9b700 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001c5d0 tx=0x7fc5a0005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5c2af9c8223f3299 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f1b6460 0x563f5f1b8870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e65d8d397652fd8e server_cookie=9307c17d78438923 in_seq=0 out_seq=0 2023-07-21T19:27:09.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fe3b5ffb700 10 client.28948 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 0x7fc5980a3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fc59cff9700 1 -- 172.21.15.5:0/404228833 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00ab5d0 con 0x5573230922d0 2023-07-21T19:27:09.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fc59cff9700 1 -- 172.21.15.5:0/404228833 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5a00af600 con 0x5573230922d0 2023-07-21T19:27:09.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fc59cff9700 1 -- 172.21.15.5:0/404228833 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0031810 con 0x5573230922d0 2023-07-21T19:27:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/4152126134 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f5f0e6eb0 con 0x563f5f1b6460 2023-07-21T19:27:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.070+0000 7fe3b5ffb700 10 client.28948 handle_command_reply: tid=0 2023-07-21T19:27:09.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 client.28948 shutdown 2023-07-21T19:27:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 -- 172.21.15.5:0/404228833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 msgr2=0x55732308c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001c5d0 tx=0x7fc5a0005b00 comp rx=0 tx=0).stop 2023-07-21T19:27:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 msgr2=0x563f5f1a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80960a0 tx=0x7fe3c8099630 comp rx=0 tx=0).stop 2023-07-21T19:27:09.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 0x7fc5980a3970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005fd0 tx=0x7fc5a4005f00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 shutdown_connections 2023-07-21T19:27:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f1b6460 0x563f5f1b8870 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a42f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1a3870 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4152126134 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19bbe0 0x563f5f1a3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 >> 172.21.15.5:0/4152126134 conn(0x563f5f191da0 msgr2=0x563f5f197230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3b5ffb700 0 client.28948 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 -- 172.21.15.5:0/404228833 shutdown_connections 2023-07-21T19:27:09.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732308c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x557323091d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/404228833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x55732308c040 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fc5c8653b80 1 -- 172.21.15.5:0/404228833 >> 172.21.15.5:0/404228833 conn(0x557323081be0 msgr2=0x557323086f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 shutdown_connections 2023-07-21T19:27:09.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.071+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4152126134 wait complete. 2023-07-21T19:27:09.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.072+0000 7fe3e89f6b80 20 client.28948 trim_cache size 0 max 16384 2023-07-21T19:27:09.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:08 smithi101 ceph-mon[128136]: pgmap v616: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:09.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/321656104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.072+0000 7fc5c8653b80 1 -- 172.21.15.5:0/404228833 shutdown_connections 2023-07-21T19:27:09.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.072+0000 7fc5c8653b80 1 -- 172.21.15.5:0/404228833 wait complete. 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.073+0000 7fc5c8653b80 1 -- start start 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.073+0000 7fc5c8653b80 1 Processor -- start 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 0x55732309b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573230922d0 0x5573230955b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322f7da60 con 0x557322ee86b0 2023-07-21T19:27:09.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322f7d760 con 0x55732308c580 2023-07-21T19:27:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557322e618c0 con 0x5573230922d0 2023-07-21T19:27:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.074+0000 7fc5c8653b80 10 client.0 _ll_register_callbacks cb 0x557322f7b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.075+0000 7fc5ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.075+0000 7fc5aed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573230922d0 0x5573230955b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.075+0000 7fc5ae59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39408/0 (socket says 172.21.15.5:39408) 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.075+0000 7fc5ae59c700 1 -- 172.21.15.5:0/2536207790 learned_addr learned my addr 172.21.15.5:0/2536207790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.075+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 0x55732309b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 -- 172.21.15.5:0/2536207790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573230922d0 msgr2=0x5573230955b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573230922d0 0x5573230955b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 -- 172.21.15.5:0/2536207790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 msgr2=0x55732309b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 -- 172.21.15.5:0/2536207790 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b0113270 con 0x55732308c580 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc5add9b700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 0x55732309b8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x557323088b40 tx=0x7fc5a001a320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f36ed62415b7ea6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00ab5d0 con 0x55732308c580 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.076+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc5a00b3990 con 0x55732308c580 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557322f37490 con 0x55732308c580 2023-07-21T19:27:09.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a0031810 con 0x55732308c580 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc5c8653b80 10 client.29030 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x557322df4c00 con 0x55732308c580 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x557322f37490 con 0x55732308c580 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.077+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc5a00b3500 con 0x55732308c580 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 10 client.29030 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 10 client.29030 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 20 client.29030 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230a5380 0x5573230a7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 4 client.29030 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.078+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557322fd6e50 con 0x5573230a5380 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.079+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230a5380 0x5573230a7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.080+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230a5380 0x5573230a7760 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d6f6f2d081c970a server_cookie=8c975c52180249c in_seq=0 out_seq=0 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.080+0000 7fc595ffb700 10 client.29030 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.081+0000 7fc595ffb700 1 -- 172.21.15.5:0/2536207790 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557322fd6e50 con 0x5573230a5380 2023-07-21T19:27:09.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.081+0000 7fc595ffb700 10 client.29030 handle_command_reply: tid=0 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.081+0000 7fc5c8653b80 1 client.29030 shutdown 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 msgr2=0x55732309b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 0x55732309b8a0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x557323088b40 tx=0x7fc5a001a320 comp rx=0 tx=0).stop 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 shutdown_connections 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5573230922d0 0x5573230955b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230a5380 0x5573230a7760 crc :-1 s=READY pgs=393 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732309b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/2536207790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55732308c580 0x55732309b8a0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 >> 172.21.15.5:0/2536207790 conn(0x557323081be0 msgr2=0x557323085c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc595ffb700 0 client.29030 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 shutdown_connections 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.082+0000 7fc5c8653b80 1 -- 172.21.15.5:0/2536207790 wait complete. 2023-07-21T19:27:09.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.083+0000 7fc5c8653b80 20 client.29030 trim_cache size 0 max 16384 2023-07-21T19:27:09.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 Processor -- start 2023-07-21T19:27:09.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 -- start start 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 0x563f5f1ad4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 0x563f5f1a4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f193ec0 con 0x563f5f19b2d0 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f19e730 con 0x563f5f1a6ad0 2023-07-21T19:27:09.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.096+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f198d20 con 0x563f5f19bbe0 2023-07-21T19:27:09.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.097+0000 7fe3cdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 0x563f5f1a4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.098+0000 7fe3cdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 0x563f5f1a4120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44582/0 (socket says 172.21.15.5:44582) 2023-07-21T19:27:09.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.098+0000 7fe3cdd9b700 1 -- 172.21.15.5:0/4217291017 learned_addr learned my addr 172.21.15.5:0/4217291017 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.098+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.098+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 0x563f5f1ad4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4217291017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 msgr2=0x563f5f1a4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 0x563f5f1a4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4217291017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 msgr2=0x563f5f1a0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 -- 172.21.15.5:0/4217291017 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f5f1935e0 con 0x563f5f19b2d0 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.099+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 0x563f5f1ad4a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80997f0 tx=0x7fe3c8004be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=503f30fa08556f2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.100+0000 7fe3bd7fa700 1 -- 172.21.15.5:0/4217291017 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a1a80 con 0x563f5f19b2d0 2023-07-21T19:27:09.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.100+0000 7fe3bd7fa700 1 -- 172.21.15.5:0/4217291017 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563f5f0474f0 con 0x563f5f19b2d0 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.100+0000 7fe3bd7fa700 1 -- 172.21.15.5:0/4217291017 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c800cd70 con 0x563f5f19b2d0 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.100+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4217291017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 msgr2=0x563f5f1ad4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.100+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 0x563f5f1ad4a0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80997f0 tx=0x7fe3c8004be0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.103+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4217291017 shutdown_connections 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.103+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1a0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.103+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f19b2d0 0x563f5f1ad4a0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.103+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/4217291017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19bbe0 0x563f5f1a4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.103+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4217291017 >> 172.21.15.5:0/4217291017 conn(0x563f5f191da0 msgr2=0x563f5f19ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.104+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4217291017 shutdown_connections 2023-07-21T19:27:09.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.105+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/4217291017 wait complete. 2023-07-21T19:27:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.105+0000 7fe3e89f6b80 1 -- start start 2023-07-21T19:27:09.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.106+0000 7fe3e89f6b80 1 Processor -- start 2023-07-21T19:27:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 0x563f5f2096f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f209c30 con 0x563f5f207310 2023-07-21T19:27:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f209d70 con 0x563f5f1a6ad0 2023-07-21T19:27:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f5f209eb0 con 0x563f5f19b2d0 2023-07-21T19:27:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.107+0000 7fe3e89f6b80 10 client.0 _ll_register_callbacks cb 0x7fe3c80994d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.108+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.108+0000 7fe3ce59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2023-07-21T19:27:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.108+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3057347536 learned_addr learned my addr 172.21.15.5:0/3057347536 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.108+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.109+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 0x563f5f2096f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3057347536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 msgr2=0x563f5f1ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3057347536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 msgr2=0x563f5f2096f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 0x563f5f2096f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 -- 172.21.15.5:0/3057347536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f5ef04c60 con 0x563f5f19b2d0 2023-07-21T19:27:09.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ced9d700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 0x563f5f2096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3ce59c700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8097980 tx=0x7fe3c8004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c88efbb024232f16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3b5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.110+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3c80a1a80 con 0x563f5f19b2d0 2023-07-21T19:27:09.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.111+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fe3c80b5030 con 0x563f5f19b2d0 2023-07-21T19:27:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.111+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe3c0005560 con 0x563f5f19b2d0 2023-07-21T19:27:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.112+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe3c80b90d0 con 0x563f5f19b2d0 2023-07-21T19:27:09.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.112+0000 7fe3e89f6b80 10 client.28957 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.112+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563f5ef42250 con 0x563f5f19b2d0 2023-07-21T19:27:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.113+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe3c0005560 con 0x563f5f19b2d0 2023-07-21T19:27:09.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.113+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe3c8098590 con 0x563f5f19b2d0 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 10 client.28957 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 10 client.28957 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 20 client.28957 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f20b0b0 0x563f5f20d490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 4 client.28957 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:09.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.114+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563f5f0a9050 con 0x563f5f20b0b0 2023-07-21T19:27:09.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.115+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f20b0b0 0x563f5f20d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.117+0000 7fc5c8653b80 1 Processor -- start 2023-07-21T19:27:09.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.117+0000 7fc5c8653b80 1 -- start start 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.117+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732308c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55732308c580 0x557323093130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323092190 con 0x557322ee86b0 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323088b70 con 0x5573230922d0 2023-07-21T19:27:09.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323091a10 con 0x55732308c580 2023-07-21T19:27:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5aed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55890/0 (socket says 172.21.15.5:55890) 2023-07-21T19:27:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fc5aed9d700 1 -- 172.21.15.5:0/3122120487 learned_addr learned my addr 172.21.15.5:0/3122120487 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.119+0000 7fc5add9b700 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55732308c580 0x557323093130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.117+0000 7fe3cdd9b700 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f20b0b0 0x563f5f20d490 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=8948cc3b6f8fd410 server_cookie=79e57f192c628f27 in_seq=0 out_seq=0 2023-07-21T19:27:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.117+0000 7fe3b5ffb700 10 client.28957 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fe3b5ffb700 1 -- 172.21.15.5:0/3057347536 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563f5f0a9050 con 0x563f5f20b0b0 2023-07-21T19:27:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fe3b5ffb700 10 client.28957 handle_command_reply: tid=0 2023-07-21T19:27:09.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.118+0000 7fe3e89f6b80 1 client.28957 shutdown 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.119+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 msgr2=0x563f5f1ab160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.119+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe3c8097980 tx=0x7fe3c8004ad0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 shutdown_connections 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563f5f207310 0x563f5f2096f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563f5f19b2d0 0x563f5f1ab160 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x563f5f20b0b0 0x563f5f20d490 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 --2- 172.21.15.5:0/3057347536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563f5f1a6ad0 0x563f5f1ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 >> 172.21.15.5:0/3057347536 conn(0x563f5f191da0 msgr2=0x563f5f1a4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3b5ffb700 0 client.28957 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 -- 172.21.15.5:0/3122120487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55732308c580 msgr2=0x557323093130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55732308c580 0x557323093130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 -- 172.21.15.5:0/3122120487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 msgr2=0x55732308c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732308c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 -- 172.21.15.5:0/3122120487 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557323095120 con 0x5573230922d0 2023-07-21T19:27:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 shutdown_connections 2023-07-21T19:27:09.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 1 -- 172.21.15.5:0/3057347536 wait complete. 2023-07-21T19:27:09.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fe3e89f6b80 20 client.28957 trim_cache size 0 max 16384 2023-07-21T19:27:09.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.120+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400dc10 tx=0x7fc5a4010000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=19f792753182d2c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.114 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fc59d7fa700 1 -- 172.21.15.5:0/3122120487 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a4010a70 con 0x5573230922d0 2023-07-21T19:27:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fc59d7fa700 1 -- 172.21.15.5:0/3122120487 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5a4010bd0 con 0x5573230922d0 2023-07-21T19:27:09.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fc59d7fa700 1 -- 172.21.15.5:0/3122120487 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a401e3f0 con 0x5573230922d0 2023-07-21T19:27:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 msgr2=0x7fe3b80a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 0x7fe3b80a3810 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fe3c0005850 tx=0x7fe3c001a320 comp rx=0 tx=0).stop 2023-07-21T19:27:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 msgr2=0x7fe3d01145b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.121+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fe3c800ba50 tx=0x7fe3c800bd60 comp rx=0 tx=0).stop 2023-07-21T19:27:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 -- 172.21.15.5:0/3122120487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 msgr2=0x557323086c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc5a400dc10 tx=0x7fc5a4010000 comp rx=0 tx=0).stop 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 shutdown_connections 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe3d01192c0 0x7fe3d01a7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe3b80a1360 0x7fe3b80a3810 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe3d006bbc0 0x7fe3d01145b0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 --2- 172.21.15.5:0/3662546813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe3d006c4d0 0x7fe3d0114af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 >> 172.21.15.5:0/3662546813 conn(0x7fe3d006a980 msgr2=0x7fe3d0109520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 -- 172.21.15.5:0/3122120487 shutdown_connections 2023-07-21T19:27:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x557323086c60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557322ee86b0 0x55732308c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/3122120487 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55732308c580 0x557323093130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.122+0000 7fc5c8653b80 1 -- 172.21.15.5:0/3122120487 >> 172.21.15.5:0/3122120487 conn(0x557323081be0 msgr2=0x55732308c990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.123+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 shutdown_connections 2023-07-21T19:27:09.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.123+0000 7fe3d4fff700 1 -- 172.21.15.5:0/3662546813 wait complete. 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.125+0000 7fc5c8653b80 1 -- 172.21.15.5:0/3122120487 shutdown_connections 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.126+0000 7fc5c8653b80 1 -- 172.21.15.5:0/3122120487 wait complete. 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.126+0000 7fc5c8653b80 1 -- start start 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.127+0000 7fc5c8653b80 1 Processor -- start 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.127+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.127+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732309b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.127+0000 7fc5c8653b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x5573230994f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.128+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323099b70 con 0x55732308c580 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.128+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323099ce0 con 0x5573230922d0 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.128+0000 7fc5c8653b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557323099ea0 con 0x557322ee86b0 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.128+0000 7fc5c8653b80 10 client.0 _ll_register_callbacks cb 0x7fc5a00a79e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2023-07-21T19:27:09.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 -- 172.21.15.5:0/1049964378 learned_addr learned my addr 172.21.15.5:0/1049964378 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5add9b700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732309b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5aed9d700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x5573230994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 -- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 msgr2=0x5573230994f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x5573230994f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 -- 172.21.15.5:0/1049964378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 msgr2=0x55732309b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732309b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.129+0000 7fc5ae59c700 1 -- 172.21.15.5:0/1049964378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5a400d890 con 0x557322ee86b0 2023-07-21T19:27:09.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.130+0000 7fc5ae59c700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5573230a8070 tx=0x7fc5a80079e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f59f984bab2a843 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.130+0000 7fc595ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.130+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8019430 con 0x557322ee86b0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.130+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc5a8019590 con 0x557322ee86b0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.131+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc5a00af600 con 0x557322ee86b0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.131+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5a8022420 con 0x557322ee86b0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.131+0000 7fc5c8653b80 10 client.28963 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.131+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x557322e321f0 con 0x557322ee86b0 2023-07-21T19:27:09.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.131+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5a00af600 con 0x557322ee86b0 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc5a802c6b0 con 0x557322ee86b0 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 10 client.28963 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 10 client.28963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 20 client.28963 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230fa2a0 0x5573230fc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 4 client.28963 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557322f98ff0 con 0x5573230fa2a0 2023-07-21T19:27:09.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.132+0000 7fc5add9b700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230fa2a0 0x5573230fc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.133+0000 7fc5add9b700 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230fa2a0 0x5573230fc6b0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fc60417e357d5f5 server_cookie=fef9ed0c188fc2cb in_seq=0 out_seq=0 2023-07-21T19:27:09.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.133+0000 7fc595ffb700 10 client.28963 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:09.145 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.160+0000 7fc595ffb700 1 -- 172.21.15.5:0/1049964378 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557322f98ff0 con 0x5573230fa2a0 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.160+0000 7fc595ffb700 10 client.28963 handle_command_reply: tid=0 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.162+0000 7fc5c8653b80 1 client.28963 shutdown 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.162+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 msgr2=0x55732309afd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.162+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5573230a8070 tx=0x7fc5a80079e0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 shutdown_connections 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5573230922d0 0x5573230994f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557322ee86b0 0x55732309afd0 secure :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0x5573230a8070 tx=0x7fc5a80079e0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5573230fa2a0 0x5573230fc6b0 crc :-1 s=READY pgs=394 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 --2- 172.21.15.5:0/1049964378 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55732308c580 0x55732309b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 >> 172.21.15.5:0/1049964378 conn(0x557323081be0 msgr2=0x557323093d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.164+0000 7fc595ffb700 0 client.28963 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:09.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.165+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 shutdown_connections 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.165+0000 7fc5c8653b80 1 -- 172.21.15.5:0/1049964378 wait complete. 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.165+0000 7fc5c8653b80 20 client.28963 trim_cache size 0 max 16384 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:27:09.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:27:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.166+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 msgr2=0x7fc5980a3970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.166+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 0x7fc5980a3970 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fc5a4005fd0 tx=0x7fc5a4005f00 comp rx=0 tx=0).stop 2023-07-21T19:27:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 msgr2=0x7fc5b006cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fc5a001a8f0 tx=0x7fc5a0005750 comp rx=0 tx=0).stop 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 shutdown_connections 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc5b0104a30 0x7fc5b0112380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc5980a14c0 0x7fc5980a3970 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5b00fea30 0x7fc5b006c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 --2- 172.21.15.5:0/2626757358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5b00ff3e0 0x7fc5b006cd60 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 >> 172.21.15.5:0/2626757358 conn(0x7fc5b0074760 msgr2=0x7fc5b00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.167+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 shutdown_connections 2023-07-21T19:27:09.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:09.168+0000 7fc5b4dff700 1 -- 172.21.15.5:0/2626757358 wait complete. 2023-07-21T19:27:09.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:08 smithi156 ceph-mon[128316]: pgmap v616: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:09.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/321656104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 -- 172.21.15.5:0/1981052300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110ae0 msgr2=0x7fc454112ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 --2- 172.21.15.5:0/1981052300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110ae0 0x7fc454112ec0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc44c00b0a0 tx=0x7fc44c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 -- 172.21.15.5:0/1981052300 shutdown_connections 2023-07-21T19:27:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 --2- 172.21.15.5:0/1981052300 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc454110ae0 0x7fc454112ec0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 --2- 172.21.15.5:0/1981052300 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540728b0 0x7fc454072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 --2- 172.21.15.5:0/1981052300 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4540731c0 0x7fc4541105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 -- 172.21.15.5:0/1981052300 >> 172.21.15.5:0/1981052300 conn(0x7fc4540fb5f0 msgr2=0x7fc4540fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 -- 172.21.15.5:0/1981052300 shutdown_connections 2023-07-21T19:27:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.538+0000 7fc4593ff700 1 -- 172.21.15.5:0/1981052300 wait complete. 2023-07-21T19:27:09.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 Processor -- start 2023-07-21T19:27:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 -- start start 2023-07-21T19:27:09.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4540728b0 0x7fc4540719a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 0x7fc454071f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 0x7fc45406a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.540+0000 7fc4593ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406b0e0 con 0x7fc454110ae0 2023-07-21T19:27:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4593ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406b250 con 0x7fc4540728b0 2023-07-21T19:27:09.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4593ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45406b410 con 0x7fc4540731c0 2023-07-21T19:27:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 0x7fc45406a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 0x7fc45406a9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39460/0 (socket says 172.21.15.5:39460) 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 0x7fc454071f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc4537fe700 1 -- 172.21.15.5:0/2533137935 learned_addr learned my addr 172.21.15.5:0/2533137935 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.541+0000 7fc452ffd700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4540728b0 0x7fc4540719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 -- 172.21.15.5:0/2533137935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4540728b0 msgr2=0x7fc4540719a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4540728b0 0x7fc4540719a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 -- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 msgr2=0x7fc45406a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 0x7fc45406a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 -- 172.21.15.5:0/2533137935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c009d00 con 0x7fc4540731c0 2023-07-21T19:27:09.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.542+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 0x7fc454071f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b450 tx=0x7fc44800b760 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.544+0000 7fc43bfff700 1 -- 172.21.15.5:0/2533137935 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448011830 con 0x7fc4540731c0 2023-07-21T19:27:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.544+0000 7fc43bfff700 1 -- 172.21.15.5:0/2533137935 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc448011e20 con 0x7fc4540731c0 2023-07-21T19:27:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.544+0000 7fc43bfff700 1 -- 172.21.15.5:0/2533137935 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448010d50 con 0x7fc4540731c0 2023-07-21T19:27:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.544+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc45406b6f0 con 0x7fc4540731c0 2023-07-21T19:27:09.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.545+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45406f770 con 0x7fc4540731c0 2023-07-21T19:27:09.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.552+0000 7fc43bfff700 1 -- 172.21.15.5:0/2533137935 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc448011990 con 0x7fc4540731c0 2023-07-21T19:27:09.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.554+0000 7fc46ccc1b80 1 Processor -- start 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 -- start start 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6a9d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6a9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6998cb0 con 0x55acf6a922d0 2023-07-21T19:27:09.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf69eba40 con 0x55acf6a9dad0 2023-07-21T19:27:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.555+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf69e4ed0 con 0x55acf6a92be0 2023-07-21T19:27:09.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.556+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.556+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39476/0 (socket says 172.21.15.5:39476) 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.556+0000 7fc4537fe700 1 -- 172.21.15.5:0/3338389860 learned_addr learned my addr 172.21.15.5:0/3338389860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.557+0000 7fc43bfff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 0x7fc43c0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.557+0000 7fc43bfff700 1 -- 172.21.15.5:0/2533137935 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc4480cc2f0 con 0x7fc4540731c0 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.557+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6a9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.557+0000 7fc452ffd700 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6a9d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 0x7fc43c0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 -- 172.21.15.5:0/3338389860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 msgr2=0x55acf6a9d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6a9d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 -- 172.21.15.5:0/3338389860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 msgr2=0x55acf6a9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6a9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.558+0000 7fc4537fe700 1 -- 172.21.15.5:0/3338389860 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf6812de0 con 0x55acf6a922d0 2023-07-21T19:27:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.559+0000 7fc4537fe700 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc44c004c30 tx=0x7fc44c004c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32c0ac63c3e9b844 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.560+0000 7fc4537fe700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 0x7fc43c0a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc44c024000 tx=0x7fc44c027000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.560+0000 7fc4397fa700 1 -- 172.21.15.5:0/3338389860 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00e070 con 0x55acf6a922d0 2023-07-21T19:27:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.560+0000 7fc4397fa700 1 -- 172.21.15.5:0/3338389860 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc44c0094c0 con 0x55acf6a922d0 2023-07-21T19:27:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.560+0000 7fc4397fa700 1 -- 172.21.15.5:0/3338389860 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c02aa50 con 0x55acf6a922d0 2023-07-21T19:27:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.561+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3338389860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 msgr2=0x55acf6a926a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.561+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc44c004c30 tx=0x7fc44c004c60 comp rx=0 tx=0).stop 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.561+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3338389860 shutdown_connections 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6a9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6a926a0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3338389860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6a9d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3338389860 >> 172.21.15.5:0/3338389860 conn(0x55acf6a88da0 msgr2=0x55acf6a8e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3338389860 shutdown_connections 2023-07-21T19:27:09.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.562+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3338389860 wait complete. 2023-07-21T19:27:09.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.563+0000 7fc46ccc1b80 1 -- start start 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.563+0000 7fc46ccc1b80 1 Processor -- start 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a9af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6984ac0 con 0x55acf6a92be0 2023-07-21T19:27:09.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf69847c0 con 0x55acf6a922d0 2023-07-21T19:27:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6868920 con 0x55acf6a9dad0 2023-07-21T19:27:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.564+0000 7fc46ccc1b80 10 client.0 _ll_register_callbacks cb 0x55acf6982760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.565+0000 7fc452ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.565+0000 7fc452ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55930/0 (socket says 172.21.15.5:55930) 2023-07-21T19:27:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.565+0000 7fc452ffd700 1 -- 172.21.15.5:0/3196947351 learned_addr learned my addr 172.21.15.5:0/3196947351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 -- 172.21.15.5:0/3196947351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 msgr2=0x55acf6a9af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a9af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 -- 172.21.15.5:0/3196947351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 msgr2=0x55acf6a9a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 -- 172.21.15.5:0/3196947351 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc454113780 con 0x55acf6a922d0 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc452ffd700 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc444011290 tx=0x7fc444011650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cea6138cb805f092 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.566+0000 7fc436ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444015660 con 0x55acf6a922d0 2023-07-21T19:27:09.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc4440157c0 con 0x55acf6a922d0 2023-07-21T19:27:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf69ddeb0 con 0x55acf6a922d0 2023-07-21T19:27:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc444015980 con 0x55acf6a922d0 2023-07-21T19:27:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc46ccc1b80 10 client.29054 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.567+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55acf67fbc60 con 0x55acf6a922d0 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.568+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55acf69ddeb0 con 0x55acf6a922d0 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.568+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc44401ec00 con 0x55acf6a922d0 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 10 client.29054 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 10 client.29054 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 20 client.29054 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6aad500 0x55acf6aaf950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 4 client.29054 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55acf69a0050 con 0x55acf6aad500 2023-07-21T19:27:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.569+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6aad500 0x55acf6aaf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.570+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6aad500 0x55acf6aaf950 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=383a9893bd0493c2 server_cookie=49b10109e0b3bd14 in_seq=0 out_seq=0 2023-07-21T19:27:09.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.570+0000 7fc436ff5700 10 client.29054 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.571+0000 7fc436ff5700 1 -- 172.21.15.5:0/3196947351 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55acf69a0050 con 0x55acf6aad500 2023-07-21T19:27:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.571+0000 7fc436ff5700 10 client.29054 handle_command_reply: tid=0 2023-07-21T19:27:09.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.571+0000 7fc46ccc1b80 1 client.29054 shutdown 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.571+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 msgr2=0x55acf6a9d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.571+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fc444011290 tx=0x7fc444011650 comp rx=0 tx=0).stop 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 shutdown_connections 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a9af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6aad500 0x55acf6aaf950 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9d560 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc436ff5700 0 client.29054 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:09.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3196947351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 >> 172.21.15.5:0/3196947351 conn(0x55acf6a88da0 msgr2=0x55acf6a8e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 shutdown_connections 2023-07-21T19:27:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3196947351 wait complete. 2023-07-21T19:27:09.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.572+0000 7fc46ccc1b80 20 client.29054 trim_cache size 0 max 16384 2023-07-21T19:27:09.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.599+0000 7fc46ccc1b80 1 Processor -- start 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.599+0000 7fc46ccc1b80 1 -- start start 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a95b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6a941a0 con 0x55acf6a92be0 2023-07-21T19:27:09.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6aaab20 con 0x55acf6a922d0 2023-07-21T19:27:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6a8a6d0 con 0x55acf6a9dad0 2023-07-21T19:27:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a95b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc4537fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a95b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44638/0 (socket says 172.21.15.5:44638) 2023-07-21T19:27:09.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc4537fe700 1 -- 172.21.15.5:0/3178369927 learned_addr learned my addr 172.21.15.5:0/3178369927 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.600+0000 7fc452ffd700 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.601+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 -- 172.21.15.5:0/3178369927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 msgr2=0x55acf6a95b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a95b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 -- 172.21.15.5:0/3178369927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 msgr2=0x55acf6a9feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 -- 172.21.15.5:0/3178369927 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55acf6839250 con 0x55acf6a92be0 2023-07-21T19:27:09.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.602+0000 7fc4527fc700 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9d100 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc4480c9ad0 tx=0x7fc4480ce680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f83ea9fdd337d4b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.603+0000 7fc439ffb700 1 -- 172.21.15.5:0/3178369927 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448010eb0 con 0x55acf6a92be0 2023-07-21T19:27:09.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.603+0000 7fc439ffb700 1 -- 172.21.15.5:0/3178369927 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc448011c40 con 0x55acf6a92be0 2023-07-21T19:27:09.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.603+0000 7fc439ffb700 1 -- 172.21.15.5:0/3178369927 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448010480 con 0x55acf6a92be0 2023-07-21T19:27:09.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.604+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3178369927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 msgr2=0x55acf6a9d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.604+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9d100 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc4480c9ad0 tx=0x7fc4480ce680 comp rx=0 tx=0).stop 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3178369927 shutdown_connections 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a9dad0 0x55acf6a95b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a922d0 0x55acf6a9feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/3178369927 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a92be0 0x55acf6a9d100 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3178369927 >> 172.21.15.5:0/3178369927 conn(0x55acf6a88da0 msgr2=0x55acf6a8ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3178369927 shutdown_connections 2023-07-21T19:27:09.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.605+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/3178369927 wait complete. 2023-07-21T19:27:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.606+0000 7fc46ccc1b80 1 -- start start 2023-07-21T19:27:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.606+0000 7fc46ccc1b80 1 Processor -- start 2023-07-21T19:27:09.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6aa3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6aa2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6aa2a40 con 0x55acf6a922d0 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6aa2bb0 con 0x55acf6a9dad0 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55acf6aa2d70 con 0x55acf6a92be0 2023-07-21T19:27:09.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.607+0000 7fc46ccc1b80 10 client.0 _ll_register_callbacks cb 0x7fc444009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc4527fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44650/0 (socket says 172.21.15.5:44650) 2023-07-21T19:27:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc4527fc700 1 -- 172.21.15.5:0/2056069909 learned_addr learned my addr 172.21.15.5:0/2056069909 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:09.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc4537fe700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6aa2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc452ffd700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6aa3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.608+0000 7fc4527fc700 1 -- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 msgr2=0x55acf6aa2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6aa2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc4527fc700 1 -- 172.21.15.5:0/2056069909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 msgr2=0x55acf6aa3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6aa3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc4527fc700 1 -- 172.21.15.5:0/2056069909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45406f770 con 0x55acf6a92be0 2023-07-21T19:27:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc4527fc700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc448095ec0 tx=0x7fc448010380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96985f6600df2b80 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc436ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:09.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448099450 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55acf693e4f0 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.609+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc44809d480 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4480a18a0 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 10 client.28975 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55acf68112b0 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55acf693e4f0 con 0x55acf6a92be0 2023-07-21T19:27:09.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc448098940 con 0x55acf6a92be0 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 10 client.28975 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 10 client.28975 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 20 client.28975 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6b02180 0x55acf6b04590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 4 client.28975 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:09.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.610+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55acf6833f70 con 0x55acf6b02180 2023-07-21T19:27:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.611+0000 7fc452ffd700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6b02180 0x55acf6b04590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.612+0000 7fc452ffd700 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6b02180 0x55acf6b04590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4de86241ef758397 server_cookie=9b00728308bc8d67 in_seq=0 out_seq=0 2023-07-21T19:27:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.612+0000 7fc436ff5700 10 client.28975 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:09.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.612+0000 7fc436ff5700 1 -- 172.21.15.5:0/2056069909 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55acf6833f70 con 0x55acf6b02180 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.612+0000 7fc436ff5700 10 client.28975 handle_command_reply: tid=0 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.612+0000 7fc46ccc1b80 1 client.28975 shutdown 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 msgr2=0x55acf6aa4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fc448095ec0 tx=0x7fc448010380 comp rx=0 tx=0).stop 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 shutdown_connections 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55acf6b02180 0x55acf6b04590 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55acf6a9dad0 0x55acf6aa2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55acf6a922d0 0x55acf6aa3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 --2- 172.21.15.5:0/2056069909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55acf6a92be0 0x55acf6aa4350 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 >> 172.21.15.5:0/2056069909 conn(0x55acf6a88da0 msgr2=0x55acf6a95b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.613+0000 7fc436ff5700 0 client.28975 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:09.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.614+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 shutdown_connections 2023-07-21T19:27:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.614+0000 7fc46ccc1b80 1 -- 172.21.15.5:0/2056069909 wait complete. 2023-07-21T19:27:09.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.614+0000 7fc46ccc1b80 20 client.28975 trim_cache size 0 max 16384 2023-07-21T19:27:09.606 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 msgr2=0x7fc43c0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 0x7fc43c0a3940 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fc44c024000 tx=0x7fc44c027000 comp rx=0 tx=0).stop 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 msgr2=0x7fc454071f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 0x7fc454071f20 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fc44800b450 tx=0x7fc44800b760 comp rx=0 tx=0).stop 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 shutdown_connections 2023-07-21T19:27:09.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc43c0a1490 0x7fc43c0a3940 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc454110ae0 0x7fc45406a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4540728b0 0x7fc4540719a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 --2- 172.21.15.5:0/2533137935 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4540731c0 0x7fc454071f20 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.615+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 >> 172.21.15.5:0/2533137935 conn(0x7fc4540fb5f0 msgr2=0x7fc4540fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.616+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 shutdown_connections 2023-07-21T19:27:09.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:09.616+0000 7fc4593ff700 1 -- 172.21.15.5:0/2533137935 wait complete. 2023-07-21T19:27:09.642 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:27:10.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.075+0000 7f896d7ff700 1 -- 172.21.15.5:0/3485180959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896810e280 msgr2=0x7f896810e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.075+0000 7f896d7ff700 1 --2- 172.21.15.5:0/3485180959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896810e280 0x7f896810e650 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f895c009990 tx=0x7f895c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 -- 172.21.15.5:0/3485180959 shutdown_connections 2023-07-21T19:27:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 --2- 172.21.15.5:0/3485180959 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806d3a0 0x7f896811b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 --2- 172.21.15.5:0/3485180959 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896810e280 0x7f896810e650 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 --2- 172.21.15.5:0/3485180959 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896806c9a0 0x7f896806ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 -- 172.21.15.5:0/3485180959 >> 172.21.15.5:0/3485180959 conn(0x7f896806bf70 msgr2=0x7f8968070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 -- 172.21.15.5:0/3485180959 shutdown_connections 2023-07-21T19:27:10.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.076+0000 7f896d7ff700 1 -- 172.21.15.5:0/3485180959 wait complete. 2023-07-21T19:27:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 Processor -- start 2023-07-21T19:27:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 -- start start 2023-07-21T19:27:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806c9a0 0x7f89681aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896810e280 0x7f89681a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681a4d90 con 0x7f896806c9a0 2023-07-21T19:27:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681a4f00 con 0x7f896806d3a0 2023-07-21T19:27:10.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.078+0000 7f896d7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89681a50c0 con 0x7f896810e280 2023-07-21T19:27:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.079+0000 7f89667fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.079+0000 7f89667fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55956/0 (socket says 172.21.15.5:55956) 2023-07-21T19:27:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.079+0000 7f89667fc700 1 -- 172.21.15.5:0/2456619573 learned_addr learned my addr 172.21.15.5:0/2456619573 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.079+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806c9a0 0x7f89681aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.079+0000 7f89677fe700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896810e280 0x7f89681a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 -- 172.21.15.5:0/2456619573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896810e280 msgr2=0x7f89681a4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896810e280 0x7f89681a4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 -- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806c9a0 msgr2=0x7f89681aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806c9a0 0x7f89681aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 -- 172.21.15.5:0/2456619573 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f895c0095f0 con 0x7f896806d3a0 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.080+0000 7f89667fc700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f895800c9b0 tx=0x7f895800ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.081+0000 7f894ffff700 1 -- 172.21.15.5:0/2456619573 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958007b00 con 0x7f896806d3a0 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.081+0000 7f894ffff700 1 -- 172.21.15.5:0/2456619573 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f895800ce50 con 0x7f896806d3a0 2023-07-21T19:27:10.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.081+0000 7f894ffff700 1 -- 172.21.15.5:0/2456619573 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958018400 con 0x7f896806d3a0 2023-07-21T19:27:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.081+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89681a53a0 con 0x7f896806d3a0 2023-07-21T19:27:10.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.081+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89681b16a0 con 0x7f896806d3a0 2023-07-21T19:27:10.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.087+0000 7f894ffff700 1 -- 172.21.15.5:0/2456619573 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f895800f450 con 0x7f896806d3a0 2023-07-21T19:27:10.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.089+0000 7f8981131b80 1 Processor -- start 2023-07-21T19:27:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.091+0000 7f894ffff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 0x7f89500a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.091+0000 7f894ffff700 1 -- 172.21.15.5:0/2456619573 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f89580ccb80 con 0x7f896806d3a0 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.092+0000 7f8981131b80 1 -- start start 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6ac590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aeeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c5a7cb0 con 0x55ac1c6acad0 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c5faa40 con 0x55ac1c6a12d0 2023-07-21T19:27:10.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.093+0000 7f8981131b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c5f3ed0 con 0x55ac1c6a1be0 2023-07-21T19:27:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.094+0000 7f89677fe700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 0x7f89500a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.095+0000 7f89677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aeeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.095+0000 7f89677fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aeeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39518/0 (socket says 172.21.15.5:39518) 2023-07-21T19:27:10.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.095+0000 7f89677fe700 1 -- 172.21.15.5:0/556359087 learned_addr learned my addr 172.21.15.5:0/556359087 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.095+0000 7f8966ffd700 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6ac590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 -- 172.21.15.5:0/556359087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 msgr2=0x55ac1c6a16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 -- 172.21.15.5:0/556359087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 msgr2=0x55ac1c6aeeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aeeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.096+0000 7f89667fc700 1 -- 172.21.15.5:0/556359087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac1c421de0 con 0x55ac1c6a1be0 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.097+0000 7f89667fc700 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6ac590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89580ca2b0 tx=0x7f89580995e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c7f8c2be1995731 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.097+0000 7f894d7fa700 1 -- 172.21.15.5:0/556359087 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895809dc60 con 0x55ac1c6a1be0 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.097+0000 7f894d7fa700 1 -- 172.21.15.5:0/556359087 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89580a2080 con 0x55ac1c6a1be0 2023-07-21T19:27:10.088 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.097+0000 7f894d7fa700 1 -- 172.21.15.5:0/556359087 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89580b2260 con 0x55ac1c6a1be0 2023-07-21T19:27:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 -- 172.21.15.5:0/556359087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 msgr2=0x55ac1c6ac590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f89677fe700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 0x7f89500a3900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8960004230 tx=0x7f8960009c50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6ac590 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f89580ca2b0 tx=0x7f89580995e0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 -- 172.21.15.5:0/556359087 shutdown_connections 2023-07-21T19:27:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aeeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 --2- 172.21.15.5:0/556359087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6ac590 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.098+0000 7f8981131b80 1 -- 172.21.15.5:0/556359087 >> 172.21.15.5:0/556359087 conn(0x55ac1c697da0 msgr2=0x55ac1c69d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.100+0000 7f8981131b80 1 -- 172.21.15.5:0/556359087 shutdown_connections 2023-07-21T19:27:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.100+0000 7f8981131b80 1 -- 172.21.15.5:0/556359087 wait complete. 2023-07-21T19:27:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.101+0000 7f8981131b80 1 -- start start 2023-07-21T19:27:10.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.101+0000 7f8981131b80 1 Processor -- start 2023-07-21T19:27:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6a9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aa2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c593ac0 con 0x55ac1c6a12d0 2023-07-21T19:27:10.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c5937c0 con 0x55ac1c6a1be0 2023-07-21T19:27:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c477920 con 0x55ac1c6acad0 2023-07-21T19:27:10.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.102+0000 7f8981131b80 10 client.0 _ll_register_callbacks cb 0x55ac1c591760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f8966ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a9820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39524/0 (socket says 172.21.15.5:39524) 2023-07-21T19:27:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f8966ffd700 1 -- 172.21.15.5:0/1406897261 learned_addr learned my addr 172.21.15.5:0/1406897261 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89677fe700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aa2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.094 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89667fc700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6a9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89677fe700 1 -- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 msgr2=0x55ac1c6a9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89677fe700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6a9d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89677fe700 1 -- 172.21.15.5:0/1406897261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 msgr2=0x55ac1c6a9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.104+0000 7f89677fe700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.105+0000 7f89677fe700 1 -- 172.21.15.5:0/1406897261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89681b16a0 con 0x55ac1c6acad0 2023-07-21T19:27:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.105+0000 7f89677fe700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aa2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8960007c90 tx=0x7f8960004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6458d4e9f6a2f8d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.105+0000 7f894aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.105+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89600095e0 con 0x55ac1c6acad0 2023-07-21T19:27:10.096 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.106+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8960021040 con 0x55ac1c6acad0 2023-07-21T19:27:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.106+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ac1c54d4f0 con 0x55ac1c6acad0 2023-07-21T19:27:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.106+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f89600099a0 con 0x55ac1c6acad0 2023-07-21T19:27:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.106+0000 7f8981131b80 10 client.28984 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:10.097 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.106+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55ac1c40ac60 con 0x55ac1c6acad0 2023-07-21T19:27:10.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.107+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ac1c54d4f0 con 0x55ac1c6acad0 2023-07-21T19:27:10.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.108+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f89600257f0 con 0x55ac1c6acad0 2023-07-21T19:27:10.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.108+0000 7f8981131b80 10 client.28984 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:10.098 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.108+0000 7f8981131b80 10 client.28984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:10.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.109+0000 7f8981131b80 20 client.28984 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:10.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.109+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c6bc3f0 0x55ac1c6be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.109+0000 7f8981131b80 4 client.28984 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:10.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.109+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ac1c5eceb0 con 0x55ac1c6bc3f0 2023-07-21T19:27:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.109+0000 7f8966ffd700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c6bc3f0 0x55ac1c6be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.110+0000 7f894aff5700 10 client.28984 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:10.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.110+0000 7f8966ffd700 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c6bc3f0 0x55ac1c6be7d0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91ebd80bdde955e server_cookie=45a95fafceedc548 in_seq=0 out_seq=0 2023-07-21T19:27:10.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.111+0000 7f894aff5700 1 -- 172.21.15.5:0/1406897261 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ac1c5eceb0 con 0x55ac1c6bc3f0 2023-07-21T19:27:10.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.111+0000 7f894aff5700 10 client.28984 handle_command_reply: tid=0 2023-07-21T19:27:10.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.111+0000 7f8981131b80 1 client.28984 shutdown 2023-07-21T19:27:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 msgr2=0x55ac1c6aa2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aa2a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f8960007c90 tx=0x7f8960004b40 comp rx=0 tx=0).stop 2023-07-21T19:27:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 shutdown_connections 2023-07-21T19:27:10.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6aa2a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c6bc3f0 0x55ac1c6be7d0 crc :-1 s=READY pgs=395 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6a9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 --2- 172.21.15.5:0/1406897261 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6a9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 >> 172.21.15.5:0/1406897261 conn(0x55ac1c697da0 msgr2=0x55ac1c69d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.112+0000 7f894aff5700 0 client.28984 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.114+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 shutdown_connections 2023-07-21T19:27:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.114+0000 7f8981131b80 1 -- 172.21.15.5:0/1406897261 wait complete. 2023-07-21T19:27:10.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.115+0000 7f8981131b80 20 client.28984 trim_cache size 0 max 16384 2023-07-21T19:27:10.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.140+0000 7f8981131b80 1 Processor -- start 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.140+0000 7f8981131b80 1 -- start start 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6b7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6a6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c6a2e30 con 0x55ac1c6a12d0 2023-07-21T19:27:10.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c6ac420 con 0x55ac1c6a1be0 2023-07-21T19:27:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.141+0000 7f8981131b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c694fa0 con 0x55ac1c6acad0 2023-07-21T19:27:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f89677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6a6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f89677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6a6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44676/0 (socket says 172.21.15.5:44676) 2023-07-21T19:27:10.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f89677fe700 1 -- 172.21.15.5:0/3612076333 learned_addr learned my addr 172.21.15.5:0/3612076333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f8966ffd700 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6b7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f89667fc700 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f8966ffd700 1 -- 172.21.15.5:0/3612076333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 msgr2=0x55ac1c6a6a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.142+0000 7f8966ffd700 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6a6a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.143+0000 7f8966ffd700 1 -- 172.21.15.5:0/3612076333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 msgr2=0x55ac1c6b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.143+0000 7f8966ffd700 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.143+0000 7f8966ffd700 1 -- 172.21.15.5:0/3612076333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ac1c448250 con 0x55ac1c6a12d0 2023-07-21T19:27:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.145+0000 7f8966ffd700 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6b7d90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x55ac1c6bf0b0 tx=0x7f895c0058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74ce9dbb7e724ac2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.145+0000 7f894dffb700 1 -- 172.21.15.5:0/3612076333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c028b90 con 0x55ac1c6a12d0 2023-07-21T19:27:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.146+0000 7f894dffb700 1 -- 172.21.15.5:0/3612076333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f895c01e030 con 0x55ac1c6a12d0 2023-07-21T19:27:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.146+0000 7f894dffb700 1 -- 172.21.15.5:0/3612076333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c00b770 con 0x55ac1c6a12d0 2023-07-21T19:27:10.137 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.147+0000 7f8981131b80 1 -- 172.21.15.5:0/3612076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 msgr2=0x55ac1c6b7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.147+0000 7f8981131b80 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6b7d90 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x55ac1c6bf0b0 tx=0x7f895c0058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 -- 172.21.15.5:0/3612076333 shutdown_connections 2023-07-21T19:27:10.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c6a6a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a12d0 0x55ac1c6b7d90 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 --2- 172.21.15.5:0/3612076333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a1be0 0x55ac1c6b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 -- 172.21.15.5:0/3612076333 >> 172.21.15.5:0/3612076333 conn(0x55ac1c697da0 msgr2=0x55ac1c6b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 -- 172.21.15.5:0/3612076333 shutdown_connections 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.148+0000 7f8981131b80 1 -- 172.21.15.5:0/3612076333 wait complete. 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.149+0000 7f8981131b80 1 -- start start 2023-07-21T19:27:10.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.149+0000 7f8981131b80 1 Processor -- start 2023-07-21T19:27:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c708740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a1be0 0x55ac1c708c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c704920 con 0x55ac1c6a1be0 2023-07-21T19:27:10.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c704a90 con 0x55ac1c6a12d0 2023-07-21T19:27:10.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ac1c704bd0 con 0x55ac1c6acad0 2023-07-21T19:27:10.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.150+0000 7f8981131b80 10 client.0 _ll_register_callbacks cb 0x7f8960007870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:10.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.151+0000 7f89677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.151+0000 7f89677fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44684/0 (socket says 172.21.15.5:44684) 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.151+0000 7f89677fe700 1 -- 172.21.15.5:0/2314687984 learned_addr learned my addr 172.21.15.5:0/2314687984 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.151+0000 7f89667fc700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a1be0 0x55ac1c708c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 -- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 msgr2=0x55ac1c708740 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c708740 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 -- 172.21.15.5:0/2314687984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a1be0 msgr2=0x55ac1c708c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:10.142 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a1be0 0x55ac1c708c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 -- 172.21.15.5:0/2314687984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896811bfa0 con 0x55ac1c6acad0 2023-07-21T19:27:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f89677fe700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8960007c90 tx=0x7f8960009100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7990d3bc406408 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:10.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.152+0000 7f894aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.153+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896001be80 con 0x55ac1c6acad0 2023-07-21T19:27:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.153+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8960021050 con 0x55ac1c6acad0 2023-07-21T19:27:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.153+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f89580a2080 con 0x55ac1c6acad0 2023-07-21T19:27:10.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.154+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8960003e70 con 0x55ac1c6acad0 2023-07-21T19:27:10.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.154+0000 7f8981131b80 10 client.28990 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:10.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.154+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55ac1c4202b0 con 0x55ac1c6acad0 2023-07-21T19:27:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.156+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f89580a2080 con 0x55ac1c6acad0 2023-07-21T19:27:10.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.156+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f896003f070 con 0x55ac1c6acad0 2023-07-21T19:27:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 10 client.28990 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 10 client.28990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 20 client.28990 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c722540 0x55ac1c724950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:10.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 4 client.28990 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ac1c5af050 con 0x55ac1c722540 2023-07-21T19:27:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.157+0000 7f8966ffd700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c722540 0x55ac1c724950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:10.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.158+0000 7f8966ffd700 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c722540 0x55ac1c724950 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7033f17b039779e4 server_cookie=4562dd7125e6236f in_seq=0 out_seq=0 2023-07-21T19:27:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.158+0000 7f894aff5700 10 client.28990 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.158+0000 7f894aff5700 1 -- 172.21.15.5:0/2314687984 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ac1c5af050 con 0x55ac1c722540 2023-07-21T19:27:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.158+0000 7f894aff5700 10 client.28990 handle_command_reply: tid=0 2023-07-21T19:27:10.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 client.28990 shutdown 2023-07-21T19:27:10.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 msgr2=0x55ac1c704210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8960007c90 tx=0x7f8960009100 comp rx=0 tx=0).stop 2023-07-21T19:27:10.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 shutdown_connections 2023-07-21T19:27:10.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ac1c6acad0 0x55ac1c704210 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ac1c6a12d0 0x55ac1c708740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ac1c722540 0x55ac1c724950 crc :-1 s=READY pgs=396 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 --2- 172.21.15.5:0/2314687984 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ac1c6a1be0 0x55ac1c708c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f894aff5700 0 client.28990 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.159+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 >> 172.21.15.5:0/2314687984 conn(0x55ac1c697da0 msgr2=0x55ac1c6ab130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.160+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 shutdown_connections 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.160+0000 7f8981131b80 1 -- 172.21.15.5:0/2314687984 wait complete. 2023-07-21T19:27:10.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.160+0000 7f8981131b80 20 client.28990 trim_cache size 0 max 16384 2023-07-21T19:27:10.152 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 msgr2=0x7f89500a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 0x7f89500a3900 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8960004230 tx=0x7f8960009c50 comp rx=0 tx=0).stop 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 msgr2=0x7f89681aabf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f895800c9b0 tx=0x7f895800ccc0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 shutdown_connections 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f896810e280 0x7f89681a4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f89500a1450 0x7f89500a3900 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f896806c9a0 0x7f89681aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 --2- 172.21.15.5:0/2456619573 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f896806d3a0 0x7f89681aabf0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.162+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 >> 172.21.15.5:0/2456619573 conn(0x7f896806bf70 msgr2=0x7f8968070830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.164+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 shutdown_connections 2023-07-21T19:27:10.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:10.164+0000 7f896d7ff700 1 -- 172.21.15.5:0/2456619573 wait complete. 2023-07-21T19:27:11.186 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:27:11.186 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:27:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 -- 172.21.15.5:0/46560654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 msgr2=0x7f370410e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 --2- 172.21.15.5:0/46560654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370410e5d0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009f90 tx=0x7f36fc00b330 comp rx=0 tx=0).stop 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 -- 172.21.15.5:0/46560654 shutdown_connections 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 --2- 172.21.15.5:0/46560654 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 0x7f370411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 --2- 172.21.15.5:0/46560654 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370410e5d0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 --2- 172.21.15.5:0/46560654 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f370406cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 -- 172.21.15.5:0/46560654 >> 172.21.15.5:0/46560654 conn(0x7f370406bf00 msgr2=0x7f3704070e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.574+0000 7f370b1ff700 1 -- 172.21.15.5:0/46560654 shutdown_connections 2023-07-21T19:27:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.575+0000 7f370b1ff700 1 -- 172.21.15.5:0/46560654 wait complete. 2023-07-21T19:27:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.576+0000 7f370b1ff700 1 Processor -- start 2023-07-21T19:27:11.565 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 -- start start 2023-07-21T19:27:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 0x7f3704115590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370411a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370411b160 con 0x7f370406d2e0 2023-07-21T19:27:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370411b2d0 con 0x7f370406c930 2023-07-21T19:27:11.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.577+0000 7f370b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f370411b490 con 0x7f370410e200 2023-07-21T19:27:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.578+0000 7f3708f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.578+0000 7f3708f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56002/0 (socket says 172.21.15.5:56002) 2023-07-21T19:27:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.578+0000 7f3708f9b700 1 -- 172.21.15.5:0/166880484 learned_addr learned my addr 172.21.15.5:0/166880484 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:11.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.578+0000 7f370979c700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370411a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3703fff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 0x7f3704115590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3708f9b700 1 -- 172.21.15.5:0/166880484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 msgr2=0x7f370411a990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3708f9b700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370411a990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3708f9b700 1 -- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 msgr2=0x7f3704115590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3708f9b700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 0x7f3704115590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.579+0000 7f3708f9b700 1 -- 172.21.15.5:0/166880484 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36fc009c40 con 0x7f370406c930 2023-07-21T19:27:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.580+0000 7f3708f9b700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d010 tx=0x7f36fc0077b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.580+0000 7f3701ffb700 1 -- 172.21.15.5:0/166880484 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc00e040 con 0x7f370406c930 2023-07-21T19:27:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.580+0000 7f3701ffb700 1 -- 172.21.15.5:0/166880484 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36fc007e90 con 0x7f370406c930 2023-07-21T19:27:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.581+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f370411b710 con 0x7f370406c930 2023-07-21T19:27:11.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.581+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37041ba030 con 0x7f370406c930 2023-07-21T19:27:11.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.583+0000 7f3701ffb700 1 -- 172.21.15.5:0/166880484 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc01b6b0 con 0x7f370406c930 2023-07-21T19:27:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.589+0000 7f371ea5ab80 1 Processor -- start 2023-07-21T19:27:11.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.589+0000 7f371ea5ab80 1 -- start start 2023-07-21T19:27:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda39d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda298d30 con 0x559fda392c60 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda2ebac0 con 0x559fda39db50 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda2e4f50 con 0x559fda392350 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f370979c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f370979c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44712/0 (socket says 172.21.15.5:44712) 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.591+0000 7f370979c700 1 -- 172.21.15.5:0/580832233 learned_addr learned my addr 172.21.15.5:0/580832233 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:11.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f3703fff700 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f3701ffb700 1 -- 172.21.15.5:0/166880484 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f36fc01b8d0 con 0x7f370406c930 2023-07-21T19:27:11.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f370979c700 1 -- 172.21.15.5:0/580832233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 msgr2=0x559fda39ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f370979c700 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f370979c700 1 -- 172.21.15.5:0/580832233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 msgr2=0x559fda39d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f370979c700 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda39d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.592+0000 7f370979c700 1 -- 172.21.15.5:0/580832233 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fda112e60 con 0x559fda392350 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.593+0000 7f370979c700 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009a70 tx=0x7f36f4009d80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d2ddbfda6d49ad72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.593+0000 7f36faffd700 1 -- 172.21.15.5:0/580832233 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4009570 con 0x559fda392350 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.594+0000 7f36faffd700 1 -- 172.21.15.5:0/580832233 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36f400ed00 con 0x559fda392350 2023-07-21T19:27:11.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.594+0000 7f36faffd700 1 -- 172.21.15.5:0/580832233 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36f4013dd0 con 0x559fda392350 2023-07-21T19:27:11.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 -- 172.21.15.5:0/580832233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 msgr2=0x559fda392720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009a70 tx=0x7f36f4009d80 comp rx=0 tx=0).stop 2023-07-21T19:27:11.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 -- 172.21.15.5:0/580832233 shutdown_connections 2023-07-21T19:27:11.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda392720 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/580832233 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda39d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f371ea5ab80 1 -- 172.21.15.5:0/580832233 >> 172.21.15.5:0/580832233 conn(0x559fda388e20 msgr2=0x559fda38e720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.595+0000 7f3701ffb700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 0x7f36f00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.596+0000 7f3701ffb700 1 -- 172.21.15.5:0/166880484 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f36fc0ce8d0 con 0x7f370406c930 2023-07-21T19:27:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.596+0000 7f3703fff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 0x7f36f00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.597+0000 7f371ea5ab80 1 -- 172.21.15.5:0/580832233 shutdown_connections 2023-07-21T19:27:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.597+0000 7f371ea5ab80 1 -- 172.21.15.5:0/580832233 wait complete. 2023-07-21T19:27:11.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.597+0000 7f3703fff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 0x7f36f00a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f37041164d0 tx=0x7f36ec009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.597+0000 7f371ea5ab80 1 -- start start 2023-07-21T19:27:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.598+0000 7f371ea5ab80 1 Processor -- start 2023-07-21T19:27:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.598+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 0x559fda39d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.598+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.598+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.598+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda284b40 con 0x559fda392350 2023-07-21T19:27:11.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda284840 con 0x559fda39db50 2023-07-21T19:27:11.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda1689a0 con 0x559fda392c60 2023-07-21T19:27:11.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f371ea5ab80 10 client.0 _ll_register_callbacks cb 0x559fda2827e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f3703fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f3703fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44726/0 (socket says 172.21.15.5:44726) 2023-07-21T19:27:11.589 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f3703fff700 1 -- 172.21.15.5:0/2517687 learned_addr learned my addr 172.21.15.5:0/2517687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f370979c700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.599+0000 7f3708f9b700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 0x559fda39d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 -- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 msgr2=0x559fda39ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 -- 172.21.15.5:0/2517687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 msgr2=0x559fda39d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 0x559fda39d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 -- 172.21.15.5:0/2517687 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36f40096d0 con 0x559fda392c60 2023-07-21T19:27:11.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3708f9b700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 0x559fda39d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:27:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f3703fff700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f36ec004280 tx=0x7f36ec0047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fefe137b396dabe5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.600+0000 7f36e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:11.591 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec017070 con 0x559fda392c60 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f36ec003d20 con 0x559fda392c60 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fda23e570 con 0x559fda392c60 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36ec029440 con 0x559fda392c60 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f371ea5ab80 10 client.29002 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559fda0fbce0 con 0x559fda392c60 2023-07-21T19:27:11.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.601+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36ec035080 con 0x559fda392c60 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f36ec01f4e0 con 0x559fda392c60 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 10 client.29002 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 10 client.29002 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 20 client.29002 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda3ad640 0x559fda3afa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 4 client.29002 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:11.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.602+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fda2ddf30 con 0x559fda3ad640 2023-07-21T19:27:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.603+0000 7f3708f9b700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda3ad640 0x559fda3afa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.605+0000 7f3708f9b700 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda3ad640 0x559fda3afa50 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ddd3401b38e20e7a server_cookie=5f39b1998c908ccb in_seq=0 out_seq=0 2023-07-21T19:27:11.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.605+0000 7f36e3fff700 10 client.29002 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.607+0000 7f36e3fff700 1 -- 172.21.15.5:0/2517687 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559fda2ddf30 con 0x559fda3ad640 2023-07-21T19:27:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.607+0000 7f36e3fff700 10 client.29002 handle_command_reply: tid=0 2023-07-21T19:27:11.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.607+0000 7f371ea5ab80 1 client.29002 shutdown 2023-07-21T19:27:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 msgr2=0x559fda39a7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f36ec004280 tx=0x7f36ec0047c0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 shutdown_connections 2023-07-21T19:27:11.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda39ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda3ad640 0x559fda3afa50 crc :-1 s=READY pgs=397 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392350 0x559fda39d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2517687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392c60 0x559fda39a7b0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 >> 172.21.15.5:0/2517687 conn(0x559fda388e20 msgr2=0x559fda38e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f36e3fff700 0 client.29002 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 shutdown_connections 2023-07-21T19:27:11.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.608+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2517687 wait complete. 2023-07-21T19:27:11.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.609+0000 7f371ea5ab80 20 client.29002 trim_cache size 0 max 16384 2023-07-21T19:27:11.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.646+0000 7f371ea5ab80 1 Processor -- start 2023-07-21T19:27:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 -- start start 2023-07-21T19:27:11.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda38e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 0x559fda38e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda39db50 0x559fda399f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda38af40 con 0x559fda39db50 2023-07-21T19:27:11.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda393eb0 con 0x559fda392c60 2023-07-21T19:27:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.647+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda394220 con 0x559fda392350 2023-07-21T19:27:11.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.648+0000 7f3703fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 0x559fda38e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.648+0000 7f3708f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda38e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.648+0000 7f370979c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda39db50 0x559fda399f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.648+0000 7f3708f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda38e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44728/0 (socket says 172.21.15.5:44728) 2023-07-21T19:27:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.648+0000 7f3708f9b700 1 -- 172.21.15.5:0/2021245817 learned_addr learned my addr 172.21.15.5:0/2021245817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:11.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.649+0000 7f3703fff700 1 -- 172.21.15.5:0/2021245817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 msgr2=0x559fda38e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.649+0000 7f3703fff700 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda38e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.649+0000 7f3703fff700 1 -- 172.21.15.5:0/2021245817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda39db50 msgr2=0x559fda399f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.649+0000 7f3703fff700 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda39db50 0x559fda399f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.649+0000 7f3703fff700 1 -- 172.21.15.5:0/2021245817 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fda38a660 con 0x559fda392c60 2023-07-21T19:27:11.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.650+0000 7f3703fff700 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 0x559fda38e790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01f910 tx=0x7f36ec009400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bf652c7a192b156 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.650+0000 7f36fb7fe700 1 -- 172.21.15.5:0/2021245817 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec009af0 con 0x559fda392c60 2023-07-21T19:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.650+0000 7f36fb7fe700 1 -- 172.21.15.5:0/2021245817 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559fda23e570 con 0x559fda392c60 2023-07-21T19:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.650+0000 7f36fb7fe700 1 -- 172.21.15.5:0/2021245817 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36ec01f4c0 con 0x559fda392c60 2023-07-21T19:27:11.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2021245817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 msgr2=0x559fda38e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 0x559fda38e790 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f36ec01f910 tx=0x7f36ec009400 comp rx=0 tx=0).stop 2023-07-21T19:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2021245817 shutdown_connections 2023-07-21T19:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda39db50 0x559fda399f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda38e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/2021245817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda392c60 0x559fda38e790 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.651+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2021245817 >> 172.21.15.5:0/2021245817 conn(0x559fda388e20 msgr2=0x559fda395ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.652+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2021245817 shutdown_connections 2023-07-21T19:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.653+0000 7f371ea5ab80 1 -- 172.21.15.5:0/2021245817 wait complete. 2023-07-21T19:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.653+0000 7f371ea5ab80 1 -- start start 2023-07-21T19:27:11.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.654+0000 7f371ea5ab80 1 Processor -- start 2023-07-21T19:27:11.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda3a40e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda3a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda3a15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda3a1b90 con 0x559fda392c60 2023-07-21T19:27:11.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda3a1d00 con 0x559fda39db50 2023-07-21T19:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fda3a1ec0 con 0x559fda392350 2023-07-21T19:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.656+0000 7f371ea5ab80 10 client.0 _ll_register_callbacks cb 0x7f36ec004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.657+0000 7f3703fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda3a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.657+0000 7f3703fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda3a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39594/0 (socket says 172.21.15.5:39594) 2023-07-21T19:27:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.657+0000 7f3703fff700 1 -- 172.21.15.5:0/3173958560 learned_addr learned my addr 172.21.15.5:0/3173958560 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.657+0000 7f3708f9b700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda3a40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.658+0000 7f370979c700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda3a15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 -- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 msgr2=0x559fda3a15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda3a15c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 -- 172.21.15.5:0/3173958560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 msgr2=0x559fda3a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:11.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:11 smithi005 ceph-mon[126460]: pgmap v617: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda3a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 -- 172.21.15.5:0/3173958560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fda0fbce0 con 0x559fda392350 2023-07-21T19:27:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f3708f9b700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda3a40e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f80 tx=0x7f36fc009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30fadd7787d8e095 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:11.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.659+0000 7f36e3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.660+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36fc0cc420 con 0x559fda392350 2023-07-21T19:27:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.660+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f36fc011390 con 0x559fda392350 2023-07-21T19:27:11.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.660+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f36f400ed00 con 0x559fda392350 2023-07-21T19:27:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.661+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36fc09b9b0 con 0x559fda392350 2023-07-21T19:27:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.662+0000 7f371ea5ab80 10 client.29008 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:11.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.662+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559fda1392d0 con 0x559fda392350 2023-07-21T19:27:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36f400ed00 con 0x559fda392350 2023-07-21T19:27:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f36fc0b9b40 con 0x559fda392350 2023-07-21T19:27:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 10 client.29008 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:11.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 10 client.29008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 20 client.29008 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda4034b0 0x559fda4058c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 4 client.29008 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:27:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.664+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559fda2a00d0 con 0x559fda4034b0 2023-07-21T19:27:11.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.665+0000 7f3703fff700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda4034b0 0x559fda4058c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.666+0000 7f3703fff700 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda4034b0 0x559fda4058c0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2fdd363688b043a6 server_cookie=fada5826be3d43b9 in_seq=0 out_seq=0 2023-07-21T19:27:11.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.666+0000 7f36e3fff700 10 client.29008 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.667+0000 7f36e3fff700 1 -- 172.21.15.5:0/3173958560 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559fda2a00d0 con 0x559fda4034b0 2023-07-21T19:27:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.667+0000 7f36e3fff700 10 client.29008 handle_command_reply: tid=0 2023-07-21T19:27:11.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.667+0000 7f371ea5ab80 1 client.29008 shutdown 2023-07-21T19:27:11.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 msgr2=0x559fda3a40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda3a40e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f36fc000f80 tx=0x7f36fc009040 comp rx=0 tx=0).stop 2023-07-21T19:27:11.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 shutdown_connections 2023-07-21T19:27:11.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559fda39db50 0x559fda3a15c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559fda392350 0x559fda3a40e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559fda4034b0 0x559fda4058c0 crc :-1 s=READY pgs=398 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 --2- 172.21.15.5:0/3173958560 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559fda392c60 0x559fda3a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 >> 172.21.15.5:0/3173958560 conn(0x559fda388e20 msgr2=0x559fda38e250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f36e3fff700 0 client.29008 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:11.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 shutdown_connections 2023-07-21T19:27:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.668+0000 7f371ea5ab80 1 -- 172.21.15.5:0/3173958560 wait complete. 2023-07-21T19:27:11.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.669+0000 7f371ea5ab80 20 client.29008 trim_cache size 0 max 16384 2023-07-21T19:27:11.660 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:27:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "1fcedf10-7ada-42e7-983c-3bbe3402e248", 2023-07-21T19:27:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:27:11.661 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.670+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 msgr2=0x7f36f00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.670+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 0x7f36f00a3940 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f37041164d0 tx=0x7f36ec009310 comp rx=0 tx=0).stop 2023-07-21T19:27:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.670+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 msgr2=0x7f3704115050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.670+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f36fc00d010 tx=0x7f36fc0077b0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 shutdown_connections 2023-07-21T19:27:11.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f370410e200 0x7f370411a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f370406c930 0x7f3704115050 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36f00a1490 0x7f36f00a3940 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 --2- 172.21.15.5:0/166880484 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f370406d2e0 0x7f3704115590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:11.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 >> 172.21.15.5:0/166880484 conn(0x7f370406bf00 msgr2=0x7f370406f030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:11.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 shutdown_connections 2023-07-21T19:27:11.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:11.672+0000 7f370b1ff700 1 -- 172.21.15.5:0/166880484 wait complete. 2023-07-21T19:27:11.693 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:27:11.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:11 smithi101 ceph-mon[128136]: pgmap v617: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:11.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:11 smithi156 ceph-mon[128316]: pgmap v617: 97 pgs: 37 active+clean+snaptrim_wait, 40 active+clean, 20 active+clean+snaptrim; 16 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 678 KiB/s rd, 20 MiB/s wr, 1.31k op/s 2023-07-21T19:27:11.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 40.0 s, 502.0 tps, lat 207.561 ms stddev 91.727 2023-07-21T19:27:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.092+0000 7f829a7ff700 1 -- 172.21.15.5:0/3872165480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f829407a6e0 msgr2=0x7f829410ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.092+0000 7f829a7ff700 1 --2- 172.21.15.5:0/3872165480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f829407a6e0 0x7f829410ac90 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f828400b0d0 tx=0x7f828400b3e0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.094+0000 7f829a7ff700 1 -- 172.21.15.5:0/3872165480 shutdown_connections 2023-07-21T19:27:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.094+0000 7f829a7ff700 1 --2- 172.21.15.5:0/3872165480 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829410b370 0x7f8294112f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.094+0000 7f829a7ff700 1 --2- 172.21.15.5:0/3872165480 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8294079d40 0x7f829407a110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.094+0000 7f829a7ff700 1 --2- 172.21.15.5:0/3872165480 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f829407a6e0 0x7f829410ac90 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.094+0000 7f829a7ff700 1 -- 172.21.15.5:0/3872165480 >> 172.21.15.5:0/3872165480 conn(0x7f8294075590 msgr2=0x7f82940779a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.096+0000 7f829a7ff700 1 -- 172.21.15.5:0/3872165480 shutdown_connections 2023-07-21T19:27:12.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.096+0000 7f829a7ff700 1 -- 172.21.15.5:0/3872165480 wait complete. 2023-07-21T19:27:12.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.097+0000 7f829a7ff700 1 Processor -- start 2023-07-21T19:27:12.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.098+0000 7f829a7ff700 1 -- start start 2023-07-21T19:27:12.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.098+0000 7f829a7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 0x7f8294080560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.098+0000 7f829a7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829407a6e0 0x7f8294080aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f829a7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 0x7f829407b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f829a7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829407bd90 con 0x7f829407a6e0 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f829a7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829407bf00 con 0x7f829410b370 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f829a7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829407c0c0 con 0x7f8294079d40 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f8293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 0x7f8294080560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f8293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 0x7f8294080560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44746/0 (socket says 172.21.15.5:44746) 2023-07-21T19:27:12.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.099+0000 7f8293fff700 1 -- 172.21.15.5:0/1709884360 learned_addr learned my addr 172.21.15.5:0/1709884360 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:12.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.100+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 0x7f829407b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.101+0000 7f8298d9c700 1 -- 172.21.15.5:0/1709884360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 msgr2=0x7f8294080560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.101+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 0x7f8294080560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.101+0000 7f8298d9c700 1 -- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829407a6e0 msgr2=0x7f8294080aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.101+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829407a6e0 0x7f8294080aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.101+0000 7f8298d9c700 1 -- 172.21.15.5:0/1709884360 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8284009d00 con 0x7f829410b370 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.102+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 0x7f829407b750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f828800d720 tx=0x7f828800dae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.102+0000 7f82917fa700 1 -- 172.21.15.5:0/1709884360 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288004220 con 0x7f829410b370 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.102+0000 7f82917fa700 1 -- 172.21.15.5:0/1709884360 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8288004380 con 0x7f829410b370 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.102+0000 7f82917fa700 1 -- 172.21.15.5:0/1709884360 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288003b20 con 0x7f829410b370 2023-07-21T19:27:12.092 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.102+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829407c3a0 con 0x7f829410b370 2023-07-21T19:27:12.093 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.103+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829407f4a0 con 0x7f829410b370 2023-07-21T19:27:12.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.110+0000 7f82ae137b80 1 Processor -- start 2023-07-21T19:27:12.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.112+0000 7f82917fa700 1 -- 172.21.15.5:0/1709884360 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8288003c80 con 0x7f829410b370 2023-07-21T19:27:12.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.113+0000 7f82ae137b80 1 -- start start 2023-07-21T19:27:12.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efc986a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efb9ecb0 con 0x5632efc98be0 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efbf1a40 con 0x5632efca3ad0 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.114+0000 7f82ae137b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efbeaed0 con 0x5632efc982d0 2023-07-21T19:27:12.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.115+0000 7f8293fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.115+0000 7f8293fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39620/0 (socket says 172.21.15.5:39620) 2023-07-21T19:27:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.115+0000 7f8293fff700 1 -- 172.21.15.5:0/2538564123 learned_addr learned my addr 172.21.15.5:0/2538564123 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.115+0000 7f82937fe700 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.115+0000 7f82937fe700 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efc986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82917fa700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 0x7f827c0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82937fe700 1 -- 172.21.15.5:0/2538564123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 msgr2=0x5632efc986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82937fe700 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efc986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82937fe700 1 -- 172.21.15.5:0/2538564123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 msgr2=0x5632efca3590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82937fe700 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca3590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.116+0000 7f82937fe700 1 -- 172.21.15.5:0/2538564123 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632efa18de0 con 0x5632efca3ad0 2023-07-21T19:27:12.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.117+0000 7f82917fa700 1 -- 172.21.15.5:0/1709884360 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f82880cc1d0 con 0x7f829410b370 2023-07-21T19:27:12.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.117+0000 7f82937fe700 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca5eb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8294075db0 tx=0x7f8284006210 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81cb6f5f1ece1f24 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.117+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 0x7f827c0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.118+0000 7f827a7fc700 1 -- 172.21.15.5:0/2538564123 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400e070 con 0x5632efca3ad0 2023-07-21T19:27:12.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.118+0000 7f827a7fc700 1 -- 172.21.15.5:0/2538564123 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8284023070 con 0x5632efca3ad0 2023-07-21T19:27:12.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.118+0000 7f827a7fc700 1 -- 172.21.15.5:0/2538564123 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8284005050 con 0x5632efca3ad0 2023-07-21T19:27:12.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.120+0000 7f82ae137b80 1 -- 172.21.15.5:0/2538564123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 msgr2=0x5632efca5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.120+0000 7f82ae137b80 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca5eb0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8294075db0 tx=0x7f8284006210 comp rx=0 tx=0).stop 2023-07-21T19:27:12.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.120+0000 7f8298d9c700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 0x7f827c0a3900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f82880a1140 tx=0x7f828800df90 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.125+0000 7f82ae137b80 1 -- 172.21.15.5:0/2538564123 shutdown_connections 2023-07-21T19:27:12.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.125+0000 7f82ae137b80 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca5eb0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.125+0000 7f82ae137b80 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efc986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.125+0000 7f82ae137b80 1 --2- 172.21.15.5:0/2538564123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca3590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.125+0000 7f82ae137b80 1 -- 172.21.15.5:0/2538564123 >> 172.21.15.5:0/2538564123 conn(0x5632efc8eda0 msgr2=0x5632efc946a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.127+0000 7f82ae137b80 1 -- 172.21.15.5:0/2538564123 shutdown_connections 2023-07-21T19:27:12.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.128+0000 7f82ae137b80 1 -- 172.21.15.5:0/2538564123 wait complete. 2023-07-21T19:27:12.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.128+0000 7f82ae137b80 1 -- start start 2023-07-21T19:27:12.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.129+0000 7f82ae137b80 1 Processor -- start 2023-07-21T19:27:12.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca0810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efca3ad0 0x5632efca0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efb8aac0 con 0x5632efc98be0 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efb8a7c0 con 0x5632efc982d0 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efa6e920 con 0x5632efca3ad0 2023-07-21T19:27:12.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.130+0000 7f82ae137b80 10 client.0 _ll_register_callbacks cb 0x5632efb88760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56078/0 (socket says 172.21.15.5:56078) 2023-07-21T19:27:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 -- 172.21.15.5:0/1633179387 learned_addr learned my addr 172.21.15.5:0/1633179387 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:12.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efca3ad0 0x5632efca0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 -- 172.21.15.5:0/1633179387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efca3ad0 msgr2=0x5632efca0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efca3ad0 0x5632efca0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.131+0000 7f82937fe700 1 -- 172.21.15.5:0/1633179387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 msgr2=0x5632efca0810 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:27:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.132+0000 7f82937fe700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.132+0000 7f82937fe700 1 -- 172.21.15.5:0/1633179387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8294113760 con 0x5632efc982d0 2023-07-21T19:27:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.132+0000 7f82937fe700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82840062a0 tx=0x7f82840096a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b5c033ebe42ed66 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.132+0000 7f82737fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.133+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400e070 con 0x5632efc982d0 2023-07-21T19:27:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.133+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f828401e430 con 0x5632efc982d0 2023-07-21T19:27:12.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.133+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5632efb444f0 con 0x5632efc982d0 2023-07-21T19:27:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828402b650 con 0x5632efc982d0 2023-07-21T19:27:12.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82ae137b80 10 client.29102 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5632efa01c60 con 0x5632efc982d0 2023-07-21T19:27:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5632efb444f0 con 0x5632efc982d0 2023-07-21T19:27:12.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f828400b100 con 0x5632efc982d0 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82ae137b80 10 client.29102 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82ae137b80 10 client.29102 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.134+0000 7f82ae137b80 20 client.29102 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.135+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efcc4000 0x5632efcc6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.135+0000 7f82ae137b80 4 client.29102 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.135+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5632efbe3eb0 con 0x5632efcc4000 2023-07-21T19:27:12.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.135+0000 7f8293fff700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efcc4000 0x5632efcc6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.137+0000 7f8293fff700 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efcc4000 0x5632efcc6410 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2779443d76f9de11 server_cookie=30128428defbe376 in_seq=0 out_seq=0 2023-07-21T19:27:12.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.137+0000 7f82737fe700 10 client.29102 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.138+0000 7f82737fe700 1 -- 172.21.15.5:0/1633179387 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5632efbe3eb0 con 0x5632efcc4000 2023-07-21T19:27:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.138+0000 7f82737fe700 10 client.29102 handle_command_reply: tid=0 2023-07-21T19:27:12.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.138+0000 7f82ae137b80 1 client.29102 shutdown 2023-07-21T19:27:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.139+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 msgr2=0x5632efca3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.139+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f82840062a0 tx=0x7f82840096a0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 shutdown_connections 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efca3ad0 0x5632efca0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca3590 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efcc4000 0x5632efcc6410 crc :-1 s=READY pgs=399 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 --2- 172.21.15.5:0/1633179387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efca0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 >> 172.21.15.5:0/1633179387 conn(0x5632efc8eda0 msgr2=0x5632efc94230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.140+0000 7f82737fe700 0 client.29102 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.142+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 shutdown_connections 2023-07-21T19:27:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.142+0000 7f82ae137b80 1 -- 172.21.15.5:0/1633179387 wait complete. 2023-07-21T19:27:12.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.143+0000 7f82ae137b80 20 client.29102 trim_cache size 0 max 16384 2023-07-21T19:27:12.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.179+0000 7f82ae137b80 1 Processor -- start 2023-07-21T19:27:12.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.179+0000 7f82ae137b80 1 -- start start 2023-07-21T19:27:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.179+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.179+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efc9ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.180+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.180+0000 7f82ae137b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efc8bf80 con 0x5632efc98be0 2023-07-21T19:27:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.180+0000 7f82ae137b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efca1790 con 0x5632efca3ad0 2023-07-21T19:27:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.180+0000 7f82ae137b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efc9c480 con 0x5632efc982d0 2023-07-21T19:27:12.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44764/0 (socket says 172.21.15.5:44764) 2023-07-21T19:27:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 -- 172.21.15.5:0/870184474 learned_addr learned my addr 172.21.15.5:0/870184474 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:12.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 -- 172.21.15.5:0/870184474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 msgr2=0x5632efca0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 -- 172.21.15.5:0/870184474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 msgr2=0x5632efc9ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efc9ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.181+0000 7f82937fe700 1 -- 172.21.15.5:0/870184474 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5632efa3f250 con 0x5632efc982d0 2023-07-21T19:27:12.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.182+0000 7f82937fe700 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f828401eea0 tx=0x7f828400bae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=299e119a6ab3d426 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.182+0000 7f827affd700 1 -- 172.21.15.5:0/870184474 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828402f070 con 0x5632efc982d0 2023-07-21T19:27:12.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.182+0000 7f827affd700 1 -- 172.21.15.5:0/870184474 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8284009490 con 0x5632efc982d0 2023-07-21T19:27:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.182+0000 7f827affd700 1 -- 172.21.15.5:0/870184474 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828400e040 con 0x5632efc982d0 2023-07-21T19:27:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.183+0000 7f82ae137b80 1 -- 172.21.15.5:0/870184474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 msgr2=0x5632efca3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.183+0000 7f82ae137b80 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f828401eea0 tx=0x7f828400bae0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.184+0000 7f82ae137b80 1 -- 172.21.15.5:0/870184474 shutdown_connections 2023-07-21T19:27:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.184+0000 7f82ae137b80 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efca3ad0 0x5632efca0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.184+0000 7f82ae137b80 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc982d0 0x5632efca3130 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.184+0000 7f82ae137b80 1 --2- 172.21.15.5:0/870184474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efc98be0 0x5632efc9ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.184+0000 7f82ae137b80 1 -- 172.21.15.5:0/870184474 >> 172.21.15.5:0/870184474 conn(0x5632efc8eda0 msgr2=0x5632efcc0710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.186+0000 7f82ae137b80 1 -- 172.21.15.5:0/870184474 shutdown_connections 2023-07-21T19:27:12.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.187+0000 7f82ae137b80 1 -- 172.21.15.5:0/870184474 wait complete. 2023-07-21T19:27:12.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.187+0000 7f82ae137b80 1 -- start start 2023-07-21T19:27:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.188+0000 7f82ae137b80 1 Processor -- start 2023-07-21T19:27:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca9f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efca3ad0 0x5632efca8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efca8a70 con 0x5632efca3ad0 2023-07-21T19:27:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efca8be0 con 0x5632efc982d0 2023-07-21T19:27:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5632efca8da0 con 0x5632efc98be0 2023-07-21T19:27:12.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82ae137b80 10 client.0 _ll_register_callbacks cb 0x7f828400bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.189+0000 7f82937fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca9f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.190+0000 7f8293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.190+0000 7f8293fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44778/0 (socket says 172.21.15.5:44778) 2023-07-21T19:27:12.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.190+0000 7f8293fff700 1 -- 172.21.15.5:0/574260771 learned_addr learned my addr 172.21.15.5:0/574260771 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:12.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 -- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 msgr2=0x5632efca9f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca9f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 -- 172.21.15.5:0/574260771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efca3ad0 msgr2=0x5632efca8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:12.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efca3ad0 0x5632efca8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 -- 172.21.15.5:0/574260771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5632efa01c60 con 0x5632efc98be0 2023-07-21T19:27:12.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f8293fff700 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c01dca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=936c8461a51e8b72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:12.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.191+0000 7f82737fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:12.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828c017040 con 0x5632efc98be0 2023-07-21T19:27:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(115..115 src has 1..115) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f828c0147d0 con 0x5632efc98be0 2023-07-21T19:27:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8284023070 con 0x5632efc98be0 2023-07-21T19:27:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f828c01d620 con 0x5632efc98be0 2023-07-21T19:27:12.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82ae137b80 10 client.29026 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:12.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.192+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5632efa172b0 con 0x5632efc98be0 2023-07-21T19:27:12.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8284023070 con 0x5632efc98be0 2023-07-21T19:27:12.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f828c0127a0 con 0x5632efc98be0 2023-07-21T19:27:12.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 10 client.29026 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 10 client.29026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 20 client.29026 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:12.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efd191a0 0x5632efd1b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 4 client.29026 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:27:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.193+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5632efba6050 con 0x5632efd191a0 2023-07-21T19:27:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.194+0000 7f82937fe700 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efd191a0 0x5632efd1b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:12.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.195+0000 7f82937fe700 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efd191a0 0x5632efd1b5b0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4a21d3a33d14e8c1 server_cookie=4ed71e1a5e979aa1 in_seq=0 out_seq=0 2023-07-21T19:27:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.195+0000 7f82737fe700 10 client.29026 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.195+0000 7f82737fe700 1 -- 172.21.15.5:0/574260771 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5632efba6050 con 0x5632efd191a0 2023-07-21T19:27:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.195+0000 7f82737fe700 10 client.29026 handle_command_reply: tid=0 2023-07-21T19:27:12.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.195+0000 7f82ae137b80 1 client.29026 shutdown 2023-07-21T19:27:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.196+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 msgr2=0x5632efcaa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.196+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f828c000c00 tx=0x7f828c01dca0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 shutdown_connections 2023-07-21T19:27:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5632efd191a0 0x5632efd1b5b0 crc :-1 s=READY pgs=400 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5632efca3ad0 0x5632efca8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5632efc982d0 0x5632efca9f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 --2- 172.21.15.5:0/574260771 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5632efc98be0 0x5632efcaa470 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 >> 172.21.15.5:0/574260771 conn(0x5632efc8eda0 msgr2=0x5632efc938b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82737fe700 0 client.29026 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 shutdown_connections 2023-07-21T19:27:12.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.197+0000 7f82ae137b80 1 -- 172.21.15.5:0/574260771 wait complete. 2023-07-21T19:27:12.196 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.198+0000 7f82ae137b80 20 client.29026 trim_cache size 0 max 16384 2023-07-21T19:27:12.196 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:12.196 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:27:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:27:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "1fcedf10-7ada-42e7-983c-3bbe3402e248": { 2023-07-21T19:27:12.197 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:27:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "1fcedf10-7ada-42e7-983c-3bbe3402e248", 2023-07-21T19:27:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:27:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:27:12.198 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:27:12.199 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:12.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 msgr2=0x7f827c0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 0x7f827c0a3900 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f82880a1140 tx=0x7f828800df90 comp rx=0 tx=0).stop 2023-07-21T19:27:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 msgr2=0x7f829407b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:12.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 0x7f829407b750 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f828800d720 tx=0x7f828800dae0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 shutdown_connections 2023-07-21T19:27:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f829410b370 0x7f829407b750 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8294079d40 0x7f8294080560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f827c0a1450 0x7f827c0a3900 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 --2- 172.21.15.5:0/1709884360 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f829407a6e0 0x7f8294080aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.199+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 >> 172.21.15.5:0/1709884360 conn(0x7f8294075590 msgr2=0x7f8294076160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.200+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 shutdown_connections 2023-07-21T19:27:12.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:12.200+0000 7f829a7ff700 1 -- 172.21.15.5:0/1709884360 wait complete. 2023-07-21T19:27:12.220 INFO:tasks.cephfs.filesystem:scrub status for tag:1fcedf10-7ada-42e7-983c-3bbe3402e248 - {'path': '/', 'tag': '1fcedf10-7ada-42e7-983c-3bbe3402e248', 'options': 'recursive,force'} 2023-07-21T19:27:12.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:12 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:27:12.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:12 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:12.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:12 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:12.648 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:12 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:12.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:12 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:27:12.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:12 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:12.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:12 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:12.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:12 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:12.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:12 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:27:12.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:12 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:12.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:12 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:12.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:12 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:13.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:13.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: pgmap v618: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-21T19:27:13.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:13 smithi101 ceph-mon[128136]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:27:13.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: pgmap v618: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-21T19:27:13.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:13 smithi005 ceph-mon[126460]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:27:13.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: pgmap v618: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 615 KiB/s rd, 18 MiB/s wr, 1.26k op/s 2023-07-21T19:27:13.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:13 smithi156 ceph-mon[128316]: osdmap e116: 12 total, 12 up, 12 in 2023-07-21T19:27:15.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:15 smithi101 ceph-mon[128136]: pgmap v620: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-21T19:27:15.901 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:15 smithi005 ceph-mon[126460]: pgmap v620: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-21T19:27:15.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:15 smithi156 ceph-mon[128316]: pgmap v620: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 768 KiB/s rd, 22 MiB/s wr, 1.58k op/s 2023-07-21T19:27:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 45.0 s, 630.2 tps, lat 156.039 ms stddev 79.566 2023-07-21T19:27:17.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:17 smithi101 ceph-mon[128136]: pgmap v621: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-21T19:27:17.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:17 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:27:17.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:17 smithi005 ceph-mon[126460]: pgmap v621: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-21T19:27:17.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:17 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:27:17.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:17 smithi156 ceph-mon[128316]: pgmap v621: 97 pgs: 97 active+clean; 15 GiB data, 9.6 GiB used, 1.0 TiB / 1.0 TiB avail; 632 KiB/s rd, 18 MiB/s wr, 1.30k op/s 2023-07-21T19:27:17.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:17 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:27:19.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:19 smithi005 ceph-mon[126460]: pgmap v622: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:19.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:19 smithi156 ceph-mon[128316]: pgmap v622: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:20.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:19 smithi101 ceph-mon[128136]: pgmap v622: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:21.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:21 smithi005 ceph-mon[126460]: pgmap v623: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:21.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:21 smithi156 ceph-mon[128316]: pgmap v623: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 50.0 s, 633.0 tps, lat 73.916 ms stddev 60.435 2023-07-21T19:27:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:21 smithi101 ceph-mon[128136]: pgmap v623: 97 pgs: 97 active+clean; 15 GiB data, 9.7 GiB used, 1.0 TiB / 1.0 TiB avail; 384 KiB/s rd, 19 MiB/s wr, 1.41k op/s 2023-07-21T19:27:23.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:23 smithi005 ceph-mon[126460]: pgmap v624: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-21T19:27:23.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:23 smithi156 ceph-mon[128316]: pgmap v624: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-21T19:27:24.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:23 smithi101 ceph-mon[128136]: pgmap v624: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 483 KiB/s rd, 20 MiB/s wr, 1.39k op/s 2023-07-21T19:27:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.671+0000 7f4ef31ff700 1 -- 172.21.15.5:0/894947179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec103260 msgr2=0x7f4eec10ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.671+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/894947179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec103260 0x7f4eec10ae30 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b280 tx=0x7f4ee4008910 comp rx=0 tx=0).stop 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 -- 172.21.15.5:0/894947179 shutdown_connections 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/894947179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec103260 0x7f4eec10ae30 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/894947179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec0fe220 0x7f4eec0fe5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/894947179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec102b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 -- 172.21.15.5:0/894947179 >> 172.21.15.5:0/894947179 conn(0x7f4eec073d50 msgr2=0x7f4eec074150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.672+0000 7f4ef31ff700 1 -- 172.21.15.5:0/894947179 shutdown_connections 2023-07-21T19:27:25.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.673+0000 7f4ef31ff700 1 -- 172.21.15.5:0/894947179 wait complete. 2023-07-21T19:27:25.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.674+0000 7f4ef31ff700 1 Processor -- start 2023-07-21T19:27:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 -- start start 2023-07-21T19:27:25.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec0fe220 0x7f4eec19c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec103260 0x7f4eec1a0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1a1570 con 0x7f4eec0febc0 2023-07-21T19:27:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.675+0000 7f4ef31ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1a16e0 con 0x7f4eec0fe220 2023-07-21T19:27:25.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4ef31ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec1a18a0 con 0x7f4eec103260 2023-07-21T19:27:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4ef179c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec103260 0x7f4eec1a0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41428/0 (socket says 172.21.15.5:41428) 2023-07-21T19:27:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4eebfff700 1 -- 172.21.15.5:0/4251540578 learned_addr learned my addr 172.21.15.5:0/4251540578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.676+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec0fe220 0x7f4eec19c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 -- 172.21.15.5:0/4251540578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec103260 msgr2=0x7f4eec1a0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec103260 0x7f4eec1a0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 -- 172.21.15.5:0/4251540578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec0fe220 msgr2=0x7f4eec19c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec0fe220 0x7f4eec19c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 -- 172.21.15.5:0/4251540578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ee000b800 con 0x7f4eec0febc0 2023-07-21T19:27:25.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.677+0000 7f4eebfff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b4d0 tx=0x7f4ed800b7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.678+0000 7f4ee9ffb700 1 -- 172.21.15.5:0/4251540578 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8011830 con 0x7f4eec0febc0 2023-07-21T19:27:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.678+0000 7f4ee9ffb700 1 -- 172.21.15.5:0/4251540578 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ed8011e20 con 0x7f4eec0febc0 2023-07-21T19:27:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.678+0000 7f4ee9ffb700 1 -- 172.21.15.5:0/4251540578 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8010cd0 con 0x7f4eec0febc0 2023-07-21T19:27:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.678+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ee4008c30 con 0x7f4eec0febc0 2023-07-21T19:27:25.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.678+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec10d020 con 0x7f4eec0febc0 2023-07-21T19:27:25.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.685+0000 7f4f06a88b80 1 Processor -- start 2023-07-21T19:27:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.685+0000 7f4ee9ffb700 1 -- 172.21.15.5:0/4251540578 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4ed8011990 con 0x7f4eec0febc0 2023-07-21T19:27:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.686+0000 7f4f06a88b80 1 -- start start 2023-07-21T19:27:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.686+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610a726b0 0x558610c27240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.686+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27780 0x558610c2b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.687+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.687+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610b1bc50 con 0x558610c27bf0 2023-07-21T19:27:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.687+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610b6e9e0 con 0x558610c27780 2023-07-21T19:27:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.687+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610b67e70 con 0x558610a726b0 2023-07-21T19:27:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41444/0 (socket says 172.21.15.5:41444) 2023-07-21T19:27:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 -- 172.21.15.5:0/2815733374 learned_addr learned my addr 172.21.15.5:0/2815733374 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 -- 172.21.15.5:0/2815733374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610a726b0 msgr2=0x558610c27240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610a726b0 0x558610c27240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 -- 172.21.15.5:0/2815733374 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27780 msgr2=0x558610c2b9a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27780 0x558610c2b9a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4eebfff700 1 -- 172.21.15.5:0/2815733374 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558610995d80 con 0x558610c27bf0 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.688+0000 7f4ee9ffb700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 0x7f4edc0a38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4eebfff700 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80c9c40 tx=0x7f4ed80d2760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a720dc68a04be854 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4ef179c700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 0x7f4edc0a38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4ee9ffb700 1 -- 172.21.15.5:0/4251540578 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4ed80cc470 con 0x7f4eec0febc0 2023-07-21T19:27:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4ed6ffd700 1 -- 172.21.15.5:0/2815733374 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8095990 con 0x558610c27bf0 2023-07-21T19:27:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4ed6ffd700 1 -- 172.21.15.5:0/2815733374 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ed8099420 con 0x558610c27bf0 2023-07-21T19:27:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.689+0000 7f4ed6ffd700 1 -- 172.21.15.5:0/2815733374 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8099190 con 0x558610c27bf0 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.690+0000 7f4f06a88b80 1 -- 172.21.15.5:0/2815733374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 msgr2=0x558610c2bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.690+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80c9c40 tx=0x7f4ed80d2760 comp rx=0 tx=0).stop 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.690+0000 7f4ef179c700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 0x7f4edc0a38a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b250 tx=0x7f4ee4007680 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 -- 172.21.15.5:0/2815733374 shutdown_connections 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c2bee0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610a726b0 0x558610c27240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/2815733374 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27780 0x558610c2b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 -- 172.21.15.5:0/2815733374 >> 172.21.15.5:0/2815733374 conn(0x558610c0bbe0 msgr2=0x558610c10f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 -- 172.21.15.5:0/2815733374 shutdown_connections 2023-07-21T19:27:25.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.691+0000 7f4f06a88b80 1 -- 172.21.15.5:0/2815733374 wait complete. 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.692+0000 7f4f06a88b80 1 -- start start 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.692+0000 7f4f06a88b80 1 Processor -- start 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c335e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c30e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610b07a60 con 0x558610c27bf0 2023-07-21T19:27:25.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610b07760 con 0x558610a726b0 2023-07-21T19:27:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5586109eb8c0 con 0x558610c27780 2023-07-21T19:27:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.693+0000 7f4f06a88b80 10 client.0 _ll_register_callbacks cb 0x558610b05700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.694+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.694+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:32934/0 (socket says 172.21.15.5:32934) 2023-07-21T19:27:25.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.694+0000 7f4eebfff700 1 -- 172.21.15.5:0/20963798 learned_addr learned my addr 172.21.15.5:0/20963798 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.694+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c335e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 -- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 msgr2=0x558610c335e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c335e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 -- 172.21.15.5:0/20963798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 msgr2=0x558610c30e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4ef179c700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c30e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c30e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 -- 172.21.15.5:0/20963798 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec10d020 con 0x558610c27780 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4ef179c700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c30e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.695+0000 7f4eebfff700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80a1570 tx=0x7f4ed80cff00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bde26129bb4d3dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.696+0000 7f4ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:25.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.696+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80cfa90 con 0x558610c27780 2023-07-21T19:27:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.696+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4ed8010410 con 0x558610c27780 2023-07-21T19:27:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.696+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558610ac1490 con 0x558610c27780 2023-07-21T19:27:25.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.698+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed8098fb0 con 0x558610c27780 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.698+0000 7f4f06a88b80 10 client.29035 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.698+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55861097ec00 con 0x558610c27780 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.698+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558610ac1490 con 0x558610c27780 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4ed80c9e40 con 0x558610c27780 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 10 client.29035 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 10 client.29035 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:25.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 20 client.29035 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c40680 0x558610c42a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 4 client.29035 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558610b60e50 con 0x558610c40680 2023-07-21T19:27:25.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.699+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c40680 0x558610c42a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.701+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c40680 0x558610c42a60 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d74d1fd5b3992332 server_cookie=cf3911531a5fe00a in_seq=0 out_seq=0 2023-07-21T19:27:25.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.701+0000 7f4ecbfff700 10 client.29035 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.704+0000 7f4ecbfff700 1 -- 172.21.15.5:0/20963798 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558610b60e50 con 0x558610c40680 2023-07-21T19:27:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.705+0000 7f4ecbfff700 10 client.29035 handle_command_reply: tid=0 2023-07-21T19:27:25.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.705+0000 7f4f06a88b80 1 client.29035 shutdown 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 msgr2=0x558610c30860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f4ed80a1570 tx=0x7f4ed80cff00 comp rx=0 tx=0).stop 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 shutdown_connections 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c40680 0x558610c42a60 crc :-1 s=READY pgs=401 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c30e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c335e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/20963798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30860 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 >> 172.21.15.5:0/20963798 conn(0x558610c0bbe0 msgr2=0x558610c104e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4ecbfff700 0 client.29035 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 shutdown_connections 2023-07-21T19:27:25.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.706+0000 7f4f06a88b80 1 -- 172.21.15.5:0/20963798 wait complete. 2023-07-21T19:27:25.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.707+0000 7f4f06a88b80 20 client.29035 trim_cache size 0 max 16384 2023-07-21T19:27:25.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 Processor -- start 2023-07-21T19:27:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 -- start start 2023-07-21T19:27:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c37c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c10c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610c2c750 con 0x558610c27bf0 2023-07-21T19:27:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.738+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610c31840 con 0x558610a726b0 2023-07-21T19:27:25.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610ac1490 con 0x558610c27780 2023-07-21T19:27:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4ef179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4ef179c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41458/0 (socket says 172.21.15.5:41458) 2023-07-21T19:27:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4ef179c700 1 -- 172.21.15.5:0/1784582433 learned_addr learned my addr 172.21.15.5:0/1784582433 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:25.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c37c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.739+0000 7f4eebfff700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c10c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef0f9b700 1 -- 172.21.15.5:0/1784582433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 msgr2=0x558610c10c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c10c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef0f9b700 1 -- 172.21.15.5:0/1784582433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 msgr2=0x558610c297d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef0f9b700 1 -- 172.21.15.5:0/1784582433 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5586109bc1f0 con 0x558610a726b0 2023-07-21T19:27:25.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.740+0000 7f4ef179c700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.742+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c37c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x558610c432e0 tx=0x7f4ee001bf10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3a7c7a769ad8474 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.743+0000 7f4ed77fe700 1 -- 172.21.15.5:0/1784582433 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee0010040 con 0x558610a726b0 2023-07-21T19:27:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.743+0000 7f4ed77fe700 1 -- 172.21.15.5:0/1784582433 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ee001b5e0 con 0x558610a726b0 2023-07-21T19:27:25.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.743+0000 7f4ed77fe700 1 -- 172.21.15.5:0/1784582433 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee00173f0 con 0x558610a726b0 2023-07-21T19:27:25.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1784582433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 msgr2=0x558610c37c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c37c70 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x558610c432e0 tx=0x7f4ee001bf10 comp rx=0 tx=0).stop 2023-07-21T19:27:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1784582433 shutdown_connections 2023-07-21T19:27:25.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610c27bf0 0x558610c297d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610a726b0 0x558610c37c70 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1784582433 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c10c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.744+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1784582433 >> 172.21.15.5:0/1784582433 conn(0x558610c0bbe0 msgr2=0x558610c2a500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.745+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1784582433 shutdown_connections 2023-07-21T19:27:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.745+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1784582433 wait complete. 2023-07-21T19:27:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.746+0000 7f4f06a88b80 1 -- start start 2023-07-21T19:27:25.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.746+0000 7f4f06a88b80 1 Processor -- start 2023-07-21T19:27:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610a726b0 0x558610c302c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610c2f5c0 con 0x558610a726b0 2023-07-21T19:27:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610c2f730 con 0x558610c27bf0 2023-07-21T19:27:25.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558610c2f8f0 con 0x558610c27780 2023-07-21T19:27:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.747+0000 7f4f06a88b80 10 client.0 _ll_register_callbacks cb 0x558610b05700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42654/0 (socket says 172.21.15.5:42654) 2023-07-21T19:27:25.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 -- 172.21.15.5:0/1534394413 learned_addr learned my addr 172.21.15.5:0/1534394413 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 -- 172.21.15.5:0/1534394413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 msgr2=0x558610c30800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 -- 172.21.15.5:0/1534394413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610a726b0 msgr2=0x558610c302c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:25.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610a726b0 0x558610c302c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.748+0000 7f4ef179c700 1 -- 172.21.15.5:0/1534394413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec10b670 con 0x558610c27bf0 2023-07-21T19:27:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.749+0000 7f4ef179c700 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4016040 tx=0x7f4ee4008910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bee2124fdcbb0c1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:25.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.749+0000 7f4ecbfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.750+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee401eb10 con 0x558610c27bf0 2023-07-21T19:27:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.750+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4ee4025040 con 0x558610c27bf0 2023-07-21T19:27:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.750+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4ed8099420 con 0x558610c27bf0 2023-07-21T19:27:25.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.751+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ee4012630 con 0x558610c27bf0 2023-07-21T19:27:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.751+0000 7f4f06a88b80 10 client.29123 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:25.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.751+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f4ee000bcc0 con 0x558610c27bf0 2023-07-21T19:27:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ee4039030 con 0x558610c27bf0 2023-07-21T19:27:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4ee4030020 con 0x558610c27bf0 2023-07-21T19:27:25.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 10 client.29123 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 10 client.29123 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 20 client.29123 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c95300 0x558610c97730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:25.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 4 client.29123 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:27:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558610b22ff0 con 0x558610c95300 2023-07-21T19:27:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.754+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c95300 0x558610c97730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.756+0000 7f4ecbfff700 10 client.29123 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:25.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.755+0000 7f4ef0f9b700 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c95300 0x558610c97730 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9de23d5d68cf0305 server_cookie=8182fc01f6d3ed52 in_seq=0 out_seq=0 2023-07-21T19:27:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.771+0000 7f4ecbfff700 1 -- 172.21.15.5:0/1534394413 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558610b22ff0 con 0x558610c95300 2023-07-21T19:27:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.771+0000 7f4ecbfff700 10 client.29123 handle_command_reply: tid=0 2023-07-21T19:27:25.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.771+0000 7f4f06a88b80 1 client.29123 shutdown 2023-07-21T19:27:25.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 msgr2=0x558610c2f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4016040 tx=0x7f4ee4008910 comp rx=0 tx=0).stop 2023-07-21T19:27:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 shutdown_connections 2023-07-21T19:27:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558610c27bf0 0x558610c2f080 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558610c95300 0x558610c97730 crc :-1 s=READY pgs=402 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558610a726b0 0x558610c302c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 --2- 172.21.15.5:0/1534394413 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558610c27780 0x558610c30800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 >> 172.21.15.5:0/1534394413 conn(0x558610c0bbe0 msgr2=0x558610c2c940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4ecbfff700 0 client.29123 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.772+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 shutdown_connections 2023-07-21T19:27:25.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.773+0000 7f4f06a88b80 1 -- 172.21.15.5:0/1534394413 wait complete. 2023-07-21T19:27:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.773+0000 7f4f06a88b80 20 client.29123 trim_cache size 0 max 16384 2023-07-21T19:27:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:27:25.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:27:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:27:25.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:27:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.774+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 msgr2=0x7f4edc0a38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.774+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 0x7f4edc0a38a0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b250 tx=0x7f4ee4007680 comp rx=0 tx=0).stop 2023-07-21T19:27:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.774+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 msgr2=0x7f4eec19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.774+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b4d0 tx=0x7f4ed800b7e0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 shutdown_connections 2023-07-21T19:27:25.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4eec103260 0x7f4eec1a0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4eec0fe220 0x7f4eec19c380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4edc0a13f0 0x7f4edc0a38a0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 --2- 172.21.15.5:0/4251540578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4eec0febc0 0x7f4eec19c8c0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:25.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 >> 172.21.15.5:0/4251540578 conn(0x7f4eec073d50 msgr2=0x7f4eec101bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.775+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 shutdown_connections 2023-07-21T19:27:25.771 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:25.776+0000 7f4ef31ff700 1 -- 172.21.15.5:0/4251540578 wait complete. 2023-07-21T19:27:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:25 smithi005 ceph-mon[126460]: pgmap v625: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-07-21T19:27:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:25 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:27:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:25 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:27:25.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:25 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:27:25.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:25 smithi156 ceph-mon[128316]: pgmap v625: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-07-21T19:27:25.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:25 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:27:25.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:25 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:27:25.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:25 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:27:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:25 smithi101 ceph-mon[128136]: pgmap v625: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 411 KiB/s rd, 17 MiB/s wr, 1.18k op/s 2023-07-21T19:27:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:25 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:27:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:25 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:27:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:25 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:27:26.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:26 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:26.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:26 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:26.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:26 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:26.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:26 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:26.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:26 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:26.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:26 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:26.937 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 55.0 s, 527.9 tps, lat 261.010 ms stddev 808.291 2023-07-21T19:27:27.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:26 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:27.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:26 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:27.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:26 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:27 smithi005 ceph-mon[126460]: pgmap v626: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2023-07-21T19:27:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:27 smithi156 ceph-mon[128316]: pgmap v626: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2023-07-21T19:27:28.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:27 smithi101 ceph-mon[128136]: pgmap v626: 97 pgs: 97 active+clean; 15 GiB data, 9.9 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 17 MiB/s wr, 1.16k op/s 2023-07-21T19:27:29.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:29 smithi156 ceph-mon[128316]: pgmap v627: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-07-21T19:27:30.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:29 smithi101 ceph-mon[128136]: pgmap v627: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-07-21T19:27:30.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:29 smithi005 ceph-mon[126460]: pgmap v627: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 575 KiB/s rd, 24 MiB/s wr, 1.62k op/s 2023-07-21T19:27:30.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:30 smithi156 ceph-mon[128316]: pgmap v628: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-21T19:27:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:30 smithi101 ceph-mon[128136]: pgmap v628: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-21T19:27:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:30 smithi005 ceph-mon[126460]: pgmap v628: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 410 KiB/s rd, 15 MiB/s wr, 1.04k op/s 2023-07-21T19:27:31.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 60.0 s, 511.1 tps, lat 224.686 ms stddev 433.005 2023-07-21T19:27:33.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:33 smithi005 ceph-mon[126460]: pgmap v629: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2023-07-21T19:27:33.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:33 smithi156 ceph-mon[128316]: pgmap v629: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2023-07-21T19:27:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:33 smithi101 ceph-mon[128136]: pgmap v629: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 562 KiB/s rd, 22 MiB/s wr, 1.49k op/s 2023-07-21T19:27:35.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:35 smithi005 ceph-mon[126460]: pgmap v630: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:35.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:35 smithi156 ceph-mon[128316]: pgmap v630: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:36.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:35 smithi101 ceph-mon[128136]: pgmap v630: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:36.937 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 65.0 s, 640.9 tps, lat 160.173 ms stddev 183.717 2023-07-21T19:27:37.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:37 smithi005 ceph-mon[126460]: pgmap v631: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:37.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:37 smithi156 ceph-mon[128316]: pgmap v631: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:38.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:37 smithi101 ceph-mon[128136]: pgmap v631: 97 pgs: 97 active+clean; 15 GiB data, 10 GiB used, 1.0 TiB / 1.0 TiB avail; 324 KiB/s rd, 14 MiB/s wr, 918 op/s 2023-07-21T19:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 -- 172.21.15.5:0/1023864985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c107780 msgr2=0x7f746c107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 --2- 172.21.15.5:0/1023864985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c107780 0x7f746c107b50 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f746400c310 tx=0x7f746400c620 comp rx=0 tx=0).stop 2023-07-21T19:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 -- 172.21.15.5:0/1023864985 shutdown_connections 2023-07-21T19:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 --2- 172.21.15.5:0/1023864985 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f746c110b50 0x7f746c112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 --2- 172.21.15.5:0/1023864985 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c107780 0x7f746c107b50 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 --2- 172.21.15.5:0/1023864985 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c108090 0x7f746c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 -- 172.21.15.5:0/1023864985 >> 172.21.15.5:0/1023864985 conn(0x7f746c06c270 msgr2=0x7f746c06c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.042+0000 7f74719ff700 1 -- 172.21.15.5:0/1023864985 shutdown_connections 2023-07-21T19:27:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.043+0000 7f74719ff700 1 -- 172.21.15.5:0/1023864985 wait complete. 2023-07-21T19:27:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 Processor -- start 2023-07-21T19:27:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 -- start start 2023-07-21T19:27:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c108090 0x7f746c11c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f746c110b50 0x7f746c117260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c117a30 con 0x7f746c110b50 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c117ba0 con 0x7f746c108090 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.045+0000 7f74719ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746c117d60 con 0x7f746c107780 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51360/0 (socket says 172.21.15.5:51360) 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 -- 172.21.15.5:0/2832712704 learned_addr learned my addr 172.21.15.5:0/2832712704 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746a7fc700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c108090 0x7f746c11c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 -- 172.21.15.5:0/2832712704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c108090 msgr2=0x7f746c11c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c108090 0x7f746c11c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 -- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f746c110b50 msgr2=0x7f746c117260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f746c110b50 0x7f746c117260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.046+0000 7f746affd700 1 -- 172.21.15.5:0/2832712704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f746400c050 con 0x7f746c107780 2023-07-21T19:27:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.047+0000 7f746affd700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7464014ef0 tx=0x7f7464012500 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.049+0000 7f74709fd700 1 -- 172.21.15.5:0/2832712704 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746400f040 con 0x7f746c107780 2023-07-21T19:27:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.050+0000 7f74709fd700 1 -- 172.21.15.5:0/2832712704 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7464003e10 con 0x7f746c107780 2023-07-21T19:27:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.050+0000 7f74709fd700 1 -- 172.21.15.5:0/2832712704 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74640057c0 con 0x7f746c107780 2023-07-21T19:27:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.050+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746c117fe0 con 0x7f746c107780 2023-07-21T19:27:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.050+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746c122310 con 0x7f746c107780 2023-07-21T19:27:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.057+0000 7f74853cfb80 1 Processor -- start 2023-07-21T19:27:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.059+0000 7f74853cfb80 1 -- start start 2023-07-21T19:27:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.059+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a92d0 0x557ee90a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f74853cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8fa8c50 con 0x557ee8eff6b0 2023-07-21T19:27:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f74853cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8ffb9e0 con 0x557ee90a92d0 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f74853cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8ff4e70 con 0x557ee90a3580 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f746a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59230/0 (socket says 172.21.15.5:59230) 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f746a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51376/0 (socket says 172.21.15.5:51376) 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.060+0000 7f746b7fe700 1 -- 172.21.15.5:0/2545606661 learned_addr learned my addr 172.21.15.5:0/2545606661 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746b7fe700 1 -- 172.21.15.5:0/2545606661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 msgr2=0x557ee90a39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746b7fe700 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746b7fe700 1 -- 172.21.15.5:0/2545606661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a92d0 msgr2=0x557ee90a3040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:27:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746b7fe700 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a92d0 0x557ee90a3040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746b7fe700 1 -- 172.21.15.5:0/2545606661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ee8e22d80 con 0x557ee8eff6b0 2023-07-21T19:27:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f746a7fc700 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.061+0000 7f74709fd700 1 -- 172.21.15.5:0/2832712704 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7464005920 con 0x7f746c107780 2023-07-21T19:27:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.062+0000 7f746b7fe700 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f745c009bc0 tx=0x7f745c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4852090ce259958a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.062+0000 7f7451ffb700 1 -- 172.21.15.5:0/2545606661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c00ea80 con 0x557ee8eff6b0 2023-07-21T19:27:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.062+0000 7f7451ffb700 1 -- 172.21.15.5:0/2545606661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f745c004cc0 con 0x557ee8eff6b0 2023-07-21T19:27:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.062+0000 7f7451ffb700 1 -- 172.21.15.5:0/2545606661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f745c005520 con 0x557ee8eff6b0 2023-07-21T19:27:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.063+0000 7f74853cfb80 1 -- 172.21.15.5:0/2545606661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 msgr2=0x557ee90a8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.063+0000 7f74853cfb80 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f745c009bc0 tx=0x7f745c009ed0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.063+0000 7f74709fd700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 0x7f74540a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74709fd700 1 -- 172.21.15.5:0/2832712704 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f74640ce6d0 con 0x7f746c107780 2023-07-21T19:27:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74853cfb80 1 -- 172.21.15.5:0/2545606661 shutdown_connections 2023-07-21T19:27:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74853cfb80 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a3580 0x557ee90a39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74853cfb80 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a8d90 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74853cfb80 1 --2- 172.21.15.5:0/2545606661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a92d0 0x557ee90a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f74853cfb80 1 -- 172.21.15.5:0/2545606661 >> 172.21.15.5:0/2545606661 conn(0x557ee9098be0 msgr2=0x557ee909df10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.064+0000 7f746b7fe700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 0x7f74540a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.066+0000 7f746b7fe700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 0x7f74540a3980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f745c0046e0 tx=0x7f745c000bc0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.066+0000 7f74853cfb80 1 -- 172.21.15.5:0/2545606661 shutdown_connections 2023-07-21T19:27:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.067+0000 7f74853cfb80 1 -- 172.21.15.5:0/2545606661 wait complete. 2023-07-21T19:27:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.067+0000 7f74853cfb80 1 -- start start 2023-07-21T19:27:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.068+0000 7f74853cfb80 1 Processor -- start 2023-07-21T19:27:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 0x557ee90af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a3580 0x557ee90ac880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8f94a60 con 0x557ee90a3580 2023-07-21T19:27:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8f94760 con 0x557ee8eff6b0 2023-07-21T19:27:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee8e788c0 con 0x557ee90a92d0 2023-07-21T19:27:39.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f74853cfb80 10 client.0 _ll_register_callbacks cb 0x557ee8f92700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51380/0 (socket says 172.21.15.5:51380) 2023-07-21T19:27:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f746affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 0x557ee90af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.069+0000 7f746b7fe700 1 -- 172.21.15.5:0/380196024 learned_addr learned my addr 172.21.15.5:0/380196024 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:39.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746a7fc700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a3580 0x557ee90ac880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746affd700 1 -- 172.21.15.5:0/380196024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 msgr2=0x557ee90acfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746affd700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746affd700 1 -- 172.21.15.5:0/380196024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a3580 msgr2=0x557ee90ac880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746affd700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a3580 0x557ee90ac880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746affd700 1 -- 172.21.15.5:0/380196024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746000b8a0 con 0x557ee8eff6b0 2023-07-21T19:27:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.070+0000 7f746b7fe700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f746affd700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 0x557ee90af670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f74640cbdc0 tx=0x7f746409b1b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3df43b70630482de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f744affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f746409bc30 con 0x557ee8eff6b0 2023-07-21T19:27:39.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557ee8f4e490 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f746409b7d0 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f746409f800 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f74853cfb80 10 client.29132 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.071+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f745c0098a0 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.072+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f74640b3080 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.072+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f74640a3700 con 0x557ee8eff6b0 2023-07-21T19:27:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 10 client.29132 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 10 client.29132 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 20 client.29132 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee90bc990 0x557ee90beda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 4 client.29132 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557ee8fede50 con 0x557ee90bc990 2023-07-21T19:27:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.073+0000 7f746a7fc700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee90bc990 0x557ee90beda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.074+0000 7f746a7fc700 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee90bc990 0x557ee90beda0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a573a53e80ff7678 server_cookie=f2ca11621551f781 in_seq=0 out_seq=0 2023-07-21T19:27:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.074+0000 7f744affd700 10 client.29132 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.074+0000 7f744affd700 1 -- 172.21.15.5:0/380196024 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557ee8fede50 con 0x557ee90bc990 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.074+0000 7f744affd700 10 client.29132 handle_command_reply: tid=0 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.075+0000 7f74853cfb80 1 client.29132 shutdown 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.075+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 msgr2=0x557ee90af670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.075+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 0x557ee90af670 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f74640cbdc0 tx=0x7f746409b1b0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 shutdown_connections 2023-07-21T19:27:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90acfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee8eff6b0 0x557ee90af670 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee90bc990 0x557ee90beda0 crc :-1 s=READY pgs=403 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 --2- 172.21.15.5:0/380196024 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a3580 0x557ee90ac880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 >> 172.21.15.5:0/380196024 conn(0x557ee9098be0 msgr2=0x557ee909d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f744affd700 0 client.29132 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 shutdown_connections 2023-07-21T19:27:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 1 -- 172.21.15.5:0/380196024 wait complete. 2023-07-21T19:27:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.076+0000 7f74853cfb80 20 client.29132 trim_cache size 0 max 16384 2023-07-21T19:27:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.103+0000 7f74853cfb80 1 Processor -- start 2023-07-21T19:27:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.103+0000 7f74853cfb80 1 -- start start 2023-07-21T19:27:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.103+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.103+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee90ac830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f74853cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee90690a0 con 0x557ee8eff6b0 2023-07-21T19:27:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f74853cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee90ad5f0 con 0x557ee90a3580 2023-07-21T19:27:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f74853cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee909ad00 con 0x557ee90a92d0 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90b3cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51382/0 (socket says 172.21.15.5:51382) 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f746b7fe700 1 -- 172.21.15.5:0/1504483973 learned_addr learned my addr 172.21.15.5:0/1504483973 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f746a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee90ac830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.104+0000 7f746affd700 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 -- 172.21.15.5:0/1504483973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 msgr2=0x557ee90b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 -- 172.21.15.5:0/1504483973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 msgr2=0x557ee90a3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 -- 172.21.15.5:0/1504483973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557ee8e0bc00 con 0x557ee90a3580 2023-07-21T19:27:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee90ac830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x557ee90bf680 tx=0x7f7460004420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46dce54f853346c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f74527fc700 1 -- 172.21.15.5:0/1504483973 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460017040 con 0x557ee90a3580 2023-07-21T19:27:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f74527fc700 1 -- 172.21.15.5:0/1504483973 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7460019040 con 0x557ee90a3580 2023-07-21T19:27:39.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.105+0000 7f74527fc700 1 -- 172.21.15.5:0/1504483973 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460010630 con 0x557ee90a3580 2023-07-21T19:27:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 -- 172.21.15.5:0/1504483973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 msgr2=0x557ee90ac830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee90ac830 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x557ee90bf680 tx=0x7f7460004420 comp rx=0 tx=0).stop 2023-07-21T19:27:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 -- 172.21.15.5:0/1504483973 shutdown_connections 2023-07-21T19:27:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee90a92d0 0x557ee90b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee8eff6b0 0x557ee90a3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1504483973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee90ac830 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 -- 172.21.15.5:0/1504483973 >> 172.21.15.5:0/1504483973 conn(0x557ee9098be0 msgr2=0x557ee90ac120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 -- 172.21.15.5:0/1504483973 shutdown_connections 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.106+0000 7f74853cfb80 1 -- 172.21.15.5:0/1504483973 wait complete. 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.107+0000 7f74853cfb80 1 -- start start 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.107+0000 7f74853cfb80 1 Processor -- start 2023-07-21T19:27:39.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee8eff6b0 0x557ee91049d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 0x557ee910f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee9105620 con 0x557ee90a92d0 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee910fda0 con 0x557ee90a3580 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557ee910ff10 con 0x557ee8eff6b0 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.108+0000 7f74853cfb80 10 client.0 _ll_register_callbacks cb 0x7f746409b5a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:39.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee8eff6b0 0x557ee91049d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 0x557ee910f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43606/0 (socket says 172.21.15.5:43606) 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 -- 172.21.15.5:0/1336372069 learned_addr learned my addr 172.21.15.5:0/1336372069 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 -- 172.21.15.5:0/1336372069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee8eff6b0 msgr2=0x557ee91049d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee8eff6b0 0x557ee91049d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 -- 172.21.15.5:0/1336372069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 msgr2=0x557ee910f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:39.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 0x557ee910f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.109+0000 7f746a7fc700 1 -- 172.21.15.5:0/1336372069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746000bdd0 con 0x557ee90a3580 2023-07-21T19:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.110+0000 7f746b7fe700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 0x557ee910f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.110+0000 7f746a7fc700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x557ee90a8a60 tx=0x7f746001fd20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5147ac4d2be48f3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f744affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7460017040 con 0x557ee90a3580 2023-07-21T19:27:39.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f746001e5c0 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f745c004cc0 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f746001eb70 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f74853cfb80 10 client.29138 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x557ee8e491f0 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.111+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f745c004cc0 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f7460004790 con 0x557ee90a3580 2023-07-21T19:27:39.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 10 client.29138 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 10 client.29138 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 20 client.29138 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee91115d0 0x557ee91139e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 4 client.29138 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557ee8fafff0 con 0x557ee91115d0 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.112+0000 7f746affd700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee91115d0 0x557ee91139e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.113+0000 7f746affd700 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee91115d0 0x557ee91139e0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a1e3276a08eeed9 server_cookie=ff979bf3f6a4c8d7 in_seq=0 out_seq=0 2023-07-21T19:27:39.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.113+0000 7f744affd700 10 client.29138 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f744affd700 1 -- 172.21.15.5:0/1336372069 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557ee8fafff0 con 0x557ee91115d0 2023-07-21T19:27:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f744affd700 10 client.29138 handle_command_reply: tid=0 2023-07-21T19:27:39.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 client.29138 shutdown 2023-07-21T19:27:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 msgr2=0x557ee9104f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x557ee90a8a60 tx=0x7f746001fd20 comp rx=0 tx=0).stop 2023-07-21T19:27:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 shutdown_connections 2023-07-21T19:27:39.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557ee91115d0 0x557ee91139e0 crc :-1 s=READY pgs=404 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557ee90a92d0 0x557ee910f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.140+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557ee8eff6b0 0x557ee91049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.141+0000 7f74853cfb80 1 --2- 172.21.15.5:0/1336372069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557ee90a3580 0x557ee9104f10 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.141+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 >> 172.21.15.5:0/1336372069 conn(0x557ee9098be0 msgr2=0x557ee90a7a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.141+0000 7f744affd700 0 client.29138 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:39.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.141+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 shutdown_connections 2023-07-21T19:27:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.141+0000 7f74853cfb80 1 -- 172.21.15.5:0/1336372069 wait complete. 2023-07-21T19:27:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.142+0000 7f74853cfb80 20 client.29138 trim_cache size 0 max 16384 2023-07-21T19:27:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:27:39.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:27:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:27:39.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:27:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.144+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 msgr2=0x7f74540a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.144+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 0x7f74540a3980 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f745c0046e0 tx=0x7f745c000bc0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.145+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 msgr2=0x7f746c11c3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:39.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.145+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7464014ef0 tx=0x7f7464012500 comp rx=0 tx=0).stop 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 shutdown_connections 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74540a14d0 0x7f74540a3980 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f746c110b50 0x7f746c117260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f746c107780 0x7f746c11c3b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 --2- 172.21.15.5:0/2832712704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f746c108090 0x7f746c11c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:39.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.146+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 >> 172.21.15.5:0/2832712704 conn(0x7f746c06c270 msgr2=0x7f746c1084b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:39.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.148+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 shutdown_connections 2023-07-21T19:27:39.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:39.149+0000 7f74719ff700 1 -- 172.21.15.5:0/2832712704 wait complete. 2023-07-21T19:27:39.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:39 smithi005 ceph-mon[126460]: pgmap v632: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2023-07-21T19:27:39.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:39 smithi156 ceph-mon[128316]: pgmap v632: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2023-07-21T19:27:40.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:39 smithi101 ceph-mon[128136]: pgmap v632: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 628 KiB/s rd, 22 MiB/s wr, 1.51k op/s 2023-07-21T19:27:41.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:41 smithi005 ceph-mon[126460]: pgmap v633: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-21T19:27:41.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:41 smithi156 ceph-mon[128316]: pgmap v633: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-21T19:27:41.937 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 70.0 s, 315.6 tps, lat 241.736 ms stddev 365.997 2023-07-21T19:27:42.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:41 smithi101 ceph-mon[128136]: pgmap v633: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 455 KiB/s rd, 16 MiB/s wr, 1.05k op/s 2023-07-21T19:27:42.221 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:27:42.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 -- 172.21.15.5:0/2961955992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e00ffad0 msgr2=0x7fa6e00fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/2961955992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e00ffad0 0x7fa6e00fff40 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fa6d4009990 tx=0x7fa6d4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 -- 172.21.15.5:0/2961955992 shutdown_connections 2023-07-21T19:27:42.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/2961955992 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e0100480 0x7fa6e010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/2961955992 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 0x7fa6e0105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/2961955992 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e00ffad0 0x7fa6e00fff40 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 -- 172.21.15.5:0/2961955992 >> 172.21.15.5:0/2961955992 conn(0x7fa6e00fb5f0 msgr2=0x7fa6e00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 -- 172.21.15.5:0/2961955992 shutdown_connections 2023-07-21T19:27:42.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.456+0000 7fa6e5fff700 1 -- 172.21.15.5:0/2961955992 wait complete. 2023-07-21T19:27:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.457+0000 7fa6e5fff700 1 Processor -- start 2023-07-21T19:27:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 -- start start 2023-07-21T19:27:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 0x7fa6e0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 0x7fa6e01a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e0073580 con 0x7fa6e0100480 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01a41e0 con 0x7fa6e0105ae0 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.458+0000 7fa6e5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01a4320 con 0x7fa6e00ffad0 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6deffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 0x7fa6e0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51412/0 (socket says 172.21.15.5:51412) 2023-07-21T19:27:42.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1151134499 learned_addr learned my addr 172.21.15.5:0/1151134499 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6dffff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 0x7fa6e01a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1151134499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 msgr2=0x7fa6e01a3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 0x7fa6e01a3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 msgr2=0x7fa6e0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 0x7fa6e0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1151134499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d40095f0 con 0x7fa6e00ffad0 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6deffd700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 0x7fa6e0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:42.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.459+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c9b0 tx=0x7fa6d000ccc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.460+0000 7fa6dcff9700 1 -- 172.21.15.5:0/1151134499 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0007b00 con 0x7fa6e00ffad0 2023-07-21T19:27:42.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.460+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01a4580 con 0x7fa6e00ffad0 2023-07-21T19:27:42.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.460+0000 7fa6dcff9700 1 -- 172.21.15.5:0/1151134499 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6d0007c60 con 0x7fa6e00ffad0 2023-07-21T19:27:42.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.460+0000 7fa6dcff9700 1 -- 172.21.15.5:0/1151134499 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0018660 con 0x7fa6e00ffad0 2023-07-21T19:27:42.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.460+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e01a4a60 con 0x7fa6e00ffad0 2023-07-21T19:27:42.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.463+0000 7fa6dcff9700 1 -- 172.21.15.5:0/1151134499 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa6d000f450 con 0x7fa6e00ffad0 2023-07-21T19:27:42.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.464+0000 7fa6f99a7b80 1 Processor -- start 2023-07-21T19:27:42.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.464+0000 7fa6f99a7b80 1 -- start start 2023-07-21T19:27:42.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6dcff9700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 0x7fa6c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c94910be0 0x558c9491b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c9491bad0 0x558c9491deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94816cb0 con 0x558c949102d0 2023-07-21T19:27:42.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94869a40 con 0x558c9491bad0 2023-07-21T19:27:42.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94862ed0 con 0x558c94910be0 2023-07-21T19:27:42.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6dcff9700 1 -- 172.21.15.5:0/1151134499 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa6d00ccd30 con 0x7fa6e00ffad0 2023-07-21T19:27:42.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6dffff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 0x7fa6c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59280/0 (socket says 172.21.15.5:59280) 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1965254455 learned_addr learned my addr 172.21.15.5:0/1965254455 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6deffd700 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c94910be0 0x558c9491b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.465+0000 7fa6dffff700 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c9491bad0 0x558c9491deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1965254455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c94910be0 msgr2=0x558c9491b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c94910be0 0x558c9491b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1965254455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c9491bad0 msgr2=0x558c9491deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c9491bad0 0x558c9491deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 -- 172.21.15.5:0/1965254455 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c94690de0 con 0x558c949102d0 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6dffff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 0x7fa6c80a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80060b0 tx=0x7fa6d8009c10 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0096810 tx=0x7fa6d0099960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10855c8284e3d7d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6c5ffb700 1 -- 172.21.15.5:0/1965254455 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00a90b0 con 0x558c949102d0 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6c5ffb700 1 -- 172.21.15.5:0/1965254455 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6d00a6c00 con 0x558c949102d0 2023-07-21T19:27:42.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.466+0000 7fa6c5ffb700 1 -- 172.21.15.5:0/1965254455 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0099da0 con 0x558c949102d0 2023-07-21T19:27:42.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.467+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/1965254455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 msgr2=0x558c949106a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.467+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0096810 tx=0x7fa6d0099960 comp rx=0 tx=0).stop 2023-07-21T19:27:42.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/1965254455 shutdown_connections 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c9491bad0 0x558c9491deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949106a0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/1965254455 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c94910be0 0x558c9491b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/1965254455 >> 172.21.15.5:0/1965254455 conn(0x558c94906da0 msgr2=0x558c9490c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/1965254455 shutdown_connections 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.468+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/1965254455 wait complete. 2023-07-21T19:27:42.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.469+0000 7fa6f99a7b80 1 -- start start 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.469+0000 7fa6f99a7b80 1 Processor -- start 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c94910be0 0x558c94918770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c94918e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94802ac0 con 0x558c94910be0 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c948027c0 con 0x558c949102d0 2023-07-21T19:27:42.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c946e6920 con 0x558c9491bad0 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6f99a7b80 10 client.0 _ll_register_callbacks cb 0x558c94800760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43650/0 (socket says 172.21.15.5:43650) 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6df7fe700 1 -- 172.21.15.5:0/4170131122 learned_addr learned my addr 172.21.15.5:0/4170131122 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6dffff700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c94918e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.470+0000 7fa6deffd700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c94910be0 0x558c94918770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 -- 172.21.15.5:0/4170131122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 msgr2=0x558c94918e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c94918e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 -- 172.21.15.5:0/4170131122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c94910be0 msgr2=0x558c94918770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c94910be0 0x558c94918770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 -- 172.21.15.5:0/4170131122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e01a4a60 con 0x558c949102d0 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6dffff700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c94918e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0007fd0 tx=0x7fa6d000f910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ba7e31027b9d4c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.471+0000 7fa6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:42.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00ad070 con 0x558c949102d0 2023-07-21T19:27:42.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa6d001f300 con 0x558c949102d0 2023-07-21T19:27:42.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c947bc4f0 con 0x558c949102d0 2023-07-21T19:27:42.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6d00ca8b0 con 0x558c949102d0 2023-07-21T19:27:42.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6f99a7b80 10 client.29147 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:42.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.472+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558c94679c60 con 0x558c949102d0 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c947bc4f0 con 0x558c949102d0 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa6d00a93f0 con 0x558c949102d0 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 10 client.29147 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 10 client.29147 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 20 client.29147 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:42.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c94921ef0 0x558c94924300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 4 client.29147 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:42.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.473+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c9485beb0 con 0x558c94921ef0 2023-07-21T19:27:42.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.475+0000 7fa6deffd700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c94921ef0 0x558c94924300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.476+0000 7fa6beffd700 10 client.29147 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:42.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.476+0000 7fa6deffd700 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c94921ef0 0x558c94924300 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48db818362bb0bd8 server_cookie=54c9d15d487abb19 in_seq=0 out_seq=0 2023-07-21T19:27:42.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.477+0000 7fa6beffd700 1 -- 172.21.15.5:0/4170131122 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558c9485beb0 con 0x558c94921ef0 2023-07-21T19:27:42.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.477+0000 7fa6beffd700 10 client.29147 handle_command_reply: tid=0 2023-07-21T19:27:42.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.477+0000 7fa6f99a7b80 1 client.29147 shutdown 2023-07-21T19:27:42.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 msgr2=0x558c9491b4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa6d0007fd0 tx=0x7fa6d000f910 comp rx=0 tx=0).stop 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 shutdown_connections 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c94918e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c949102d0 0x558c9491b4f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c94921ef0 0x558c94924300 crc :-1 s=READY pgs=405 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/4170131122 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c94910be0 0x558c94918770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 >> 172.21.15.5:0/4170131122 conn(0x558c94906da0 msgr2=0x558c9490c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6beffd700 0 client.29147 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:42.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 shutdown_connections 2023-07-21T19:27:42.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.478+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/4170131122 wait complete. 2023-07-21T19:27:42.470 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.479+0000 7fa6f99a7b80 20 client.29147 trim_cache size 0 max 16384 2023-07-21T19:27:42.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.496+0000 7fa6f99a7b80 1 Processor -- start 2023-07-21T19:27:42.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 -- start start 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c949102d0 0x558c9491b090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94917f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94910fb0 con 0x558c9491bad0 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94919d50 con 0x558c94910be0 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c94903ef0 con 0x558c949102d0 2023-07-21T19:27:42.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6dffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59306/0 (socket says 172.21.15.5:59306) 2023-07-21T19:27:42.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6dffff700 1 -- 172.21.15.5:0/3651292539 learned_addr learned my addr 172.21.15.5:0/3651292539 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6deffd700 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94917f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.497+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c949102d0 0x558c9491b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 -- 172.21.15.5:0/3651292539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c949102d0 msgr2=0x558c9491b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c949102d0 0x558c9491b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 -- 172.21.15.5:0/3651292539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 msgr2=0x558c94917f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94917f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 -- 172.21.15.5:0/3651292539 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c949085e0 con 0x558c9491bad0 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6dffff700 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009410 tx=0x7fa6d8009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b13a443c0bd599e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6c67fc700 1 -- 172.21.15.5:0/3651292539 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d801b680 con 0x558c9491bad0 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6c67fc700 1 -- 172.21.15.5:0/3651292539 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6d801bc70 con 0x558c9491bad0 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.498+0000 7fa6c67fc700 1 -- 172.21.15.5:0/3651292539 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8028cb0 con 0x558c9491bad0 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/3651292539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 msgr2=0x558c94918470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fa6d8009410 tx=0x7fa6d8009040 comp rx=0 tx=0).stop 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/3651292539 shutdown_connections 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c9491bad0 0x558c94918470 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c949102d0 0x558c9491b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/3651292539 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94917f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/3651292539 >> 172.21.15.5:0/3651292539 conn(0x558c94906da0 msgr2=0x558c94923f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/3651292539 shutdown_connections 2023-07-21T19:27:42.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.499+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/3651292539 wait complete. 2023-07-21T19:27:42.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.500+0000 7fa6f99a7b80 1 -- start start 2023-07-21T19:27:42.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.500+0000 7fa6f99a7b80 1 Processor -- start 2023-07-21T19:27:42.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.500+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.500+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94926b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6f99a7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c9497df40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c949273f0 con 0x558c949102d0 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9497e480 con 0x558c94910be0 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6f99a7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c9497e5c0 con 0x558c9491bad0 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6f99a7b80 10 client.0 _ll_register_callbacks cb 0x7fa6d8007580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59322/0 (socket says 172.21.15.5:59322) 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 -- 172.21.15.5:0/2298191196 learned_addr learned my addr 172.21.15.5:0/2298191196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6deffd700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94926b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6dffff700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c9497df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 -- 172.21.15.5:0/2298191196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 msgr2=0x558c9497df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c9497df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 -- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 msgr2=0x558c94926b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94926b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 -- 172.21.15.5:0/2298191196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6d8011070 con 0x558c949102d0 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6df7fe700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00962a0 tx=0x7fa6d00a1f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e630345534de030 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.501+0000 7fa6beffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00ad040 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa6d009a050 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa6d00a6c00 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa6d009ab80 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 10 client.19494 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x558c946b7250 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa6d00a6c00 con 0x558c949102d0 2023-07-21T19:27:42.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa6d00ca050 con 0x558c949102d0 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 10 client.19494 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 10 client.19494 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 20 client.19494 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c9497f7b0 0x558c94981b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 4 client.19494 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.502+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c9481e050 con 0x558c9497f7b0 2023-07-21T19:27:42.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.503+0000 7fa6deffd700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c9497f7b0 0x558c94981b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.503+0000 7fa6beffd700 10 client.19494 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.503+0000 7fa6deffd700 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c9497f7b0 0x558c94981b90 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2260f69d8e62482 server_cookie=73142c00965ab7ee in_seq=0 out_seq=0 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6beffd700 1 -- 172.21.15.5:0/2298191196 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x558c9481e050 con 0x558c9497f7b0 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6beffd700 10 client.19494 handle_command_reply: tid=0 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 client.19494 shutdown 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 msgr2=0x558c949265d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fa6d00962a0 tx=0x7fa6d00a1f70 comp rx=0 tx=0).stop 2023-07-21T19:27:42.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 shutdown_connections 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c9491bad0 0x558c9497df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c9497f7b0 0x558c94981b90 crc :-1 s=READY pgs=406 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c949102d0 0x558c949265d0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 --2- 172.21.15.5:0/2298191196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c94910be0 0x558c94926b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6beffd700 0 client.19494 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 >> 172.21.15.5:0/2298191196 conn(0x558c94906da0 msgr2=0x558c94933330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 shutdown_connections 2023-07-21T19:27:42.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 1 -- 172.21.15.5:0/2298191196 wait complete. 2023-07-21T19:27:42.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.504+0000 7fa6f99a7b80 20 client.19494 trim_cache size 0 max 16384 2023-07-21T19:27:42.501 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:42.501 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:27:42.501 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:27:42.501 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 msgr2=0x7fa6c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 0x7fa6c80a3770 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa6d80060b0 tx=0x7fa6d8009c10 comp rx=0 tx=0).stop 2023-07-21T19:27:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 msgr2=0x7fa6e0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000c9b0 tx=0x7fa6d000ccc0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 shutdown_connections 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa6e0105ae0 0x7fa6e01a3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa6e00ffad0 0x7fa6e0072810 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa6c80a12c0 0x7fa6c80a3770 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 --2- 172.21.15.5:0/1151134499 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa6e0100480 0x7fa6e0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 >> 172.21.15.5:0/1151134499 conn(0x7fa6e00fb5f0 msgr2=0x7fa6e00fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 shutdown_connections 2023-07-21T19:27:42.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.505+0000 7fa6e5fff700 1 -- 172.21.15.5:0/1151134499 wait complete. 2023-07-21T19:27:42.511 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:27:42.512 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:27:42.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.733+0000 7f0402fff700 1 -- 172.21.15.5:0/1665362808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc0ffad0 msgr2=0x7f03fc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.733+0000 7f0402fff700 1 --2- 172.21.15.5:0/1665362808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc0ffad0 0x7f03fc0fff40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f03e8009990 tx=0x7f03e8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 -- 172.21.15.5:0/1665362808 shutdown_connections 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 --2- 172.21.15.5:0/1665362808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 0x7f03fc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 --2- 172.21.15.5:0/1665362808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc105ae0 0x7f03fc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 --2- 172.21.15.5:0/1665362808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc0ffad0 0x7f03fc0fff40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 -- 172.21.15.5:0/1665362808 >> 172.21.15.5:0/1665362808 conn(0x7f03fc0fb590 msgr2=0x7f03fc0fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 -- 172.21.15.5:0/1665362808 shutdown_connections 2023-07-21T19:27:42.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.734+0000 7f0402fff700 1 -- 172.21.15.5:0/1665362808 wait complete. 2023-07-21T19:27:42.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 Processor -- start 2023-07-21T19:27:42.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 -- start start 2023-07-21T19:27:42.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc0ffad0 0x7f03fc19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 0x7f03fc19df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc198150 con 0x7f03fc105ae0 2023-07-21T19:27:42.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc1982c0 con 0x7f03fc0ffad0 2023-07-21T19:27:42.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.735+0000 7f0402fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03fc198480 con 0x7f03fc100480 2023-07-21T19:27:42.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59342/0 (socket says 172.21.15.5:59342) 2023-07-21T19:27:42.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 -- 172.21.15.5:0/1667275020 learned_addr learned my addr 172.21.15.5:0/1667275020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:42.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f03fbfff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 0x7f03fc19df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f0400d9b700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc0ffad0 0x7f03fc19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 -- 172.21.15.5:0/1667275020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 msgr2=0x7f03fc19df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 0x7f03fc19df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 -- 172.21.15.5:0/1667275020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc0ffad0 msgr2=0x7f03fc19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc0ffad0 0x7f03fc19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 -- 172.21.15.5:0/1667275020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80095f0 con 0x7f03fc105ae0 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.736+0000 7f040159c700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f03f4007ee0 tx=0x7f03f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.737+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4017070 con 0x7f03fc105ae0 2023-07-21T19:27:42.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.737+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03fc198760 con 0x7f03fc105ae0 2023-07-21T19:27:42.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.737+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f03f400f040 con 0x7f03fc105ae0 2023-07-21T19:27:42.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.737+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4005710 con 0x7f03fc105ae0 2023-07-21T19:27:42.730 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.737+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03fc1a4ac0 con 0x7f03fc105ae0 2023-07-21T19:27:42.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.738+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03fc067300 con 0x7f03fc105ae0 2023-07-21T19:27:42.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.740+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f03f4005870 con 0x7f03fc105ae0 2023-07-21T19:27:42.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.741+0000 7f03f9ffb700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 0x7f03ec0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:42.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.741+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f03f40cc590 con 0x7f03fc105ae0 2023-07-21T19:27:42.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.742+0000 7f0400d9b700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 0x7f03ec0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:42.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.742+0000 7f0400d9b700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 0x7f03ec0a37b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03f0005950 tx=0x7f03f00058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:42.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:42.746+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f03f4095940 con 0x7f03fc105ae0 2023-07-21T19:27:42.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.005+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03fc198fe0 con 0x7f03fc105ae0 2023-07-21T19:27:42.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.006+0000 7f03f9ffb700 1 -- 172.21.15.5:0/1667275020 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f03f4098f60 con 0x7f03fc105ae0 2023-07-21T19:27:42.995 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:27:42.995 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:25:53.614690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:27:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 msgr2=0x7f03ec0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 0x7f03ec0a37b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03f0005950 tx=0x7f03f00058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 msgr2=0x7f03fc197af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f03f4007ee0 tx=0x7f03f400d3b0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 shutdown_connections 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03ec0a1300 0x7f03ec0a37b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f03fc105ae0 0x7f03fc197af0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f03fc0ffad0 0x7f03fc19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 --2- 172.21.15.5:0/1667275020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f03fc100480 0x7f03fc19df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 >> 172.21.15.5:0/1667275020 conn(0x7f03fc0fb590 msgr2=0x7f03fc10ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 shutdown_connections 2023-07-21T19:27:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.008+0000 7f0402fff700 1 -- 172.21.15.5:0/1667275020 wait complete. 2023-07-21T19:27:42.999 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 56 2023-07-21T19:27:43.015 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:27:43.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/879744830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb8100480 msgr2=0x7fcfb810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/879744830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb8100480 0x7fcfb810c910 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b0a0 tx=0x7fcfb000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/879744830 shutdown_connections 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/879744830 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb8100480 0x7fcfb810c910 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/879744830 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb8105ae0 0x7fcfb8105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/879744830 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb80ffad0 0x7fcfb80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/879744830 >> 172.21.15.5:0/879744830 conn(0x7fcfb80fb5f0 msgr2=0x7fcfb80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/879744830 shutdown_connections 2023-07-21T19:27:43.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.233+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/879744830 wait complete. 2023-07-21T19:27:43.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 Processor -- start 2023-07-21T19:27:43.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 -- start start 2023-07-21T19:27:43.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb8100480 0x7fcfb8198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb8105ae0 0x7fcfb819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819cde0 con 0x7fcfb80ffad0 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819cf50 con 0x7fcfb8100480 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.234+0000 7fcfbd3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfb819d110 con 0x7fcfb8105ae0 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59356/0 (socket says 172.21.15.5:59356) 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/928416196 learned_addr learned my addr 172.21.15.5:0/928416196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb8100480 0x7fcfb8198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/928416196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb8105ae0 msgr2=0x7fcfb819c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb8105ae0 0x7fcfb819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/928416196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb8100480 msgr2=0x7fcfb8198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb8100480 0x7fcfb8198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/928416196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfb0009d00 con 0x7fcfb80ffad0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d950 tx=0x7fcfa800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.235+0000 7fcf9ffff700 1 -- 172.21.15.5:0/928416196 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa8009b40 con 0x7fcfb80ffad0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.236+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfb819d3f0 con 0x7fcfb80ffad0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.236+0000 7fcf9ffff700 1 -- 172.21.15.5:0/928416196 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfa8009ca0 con 0x7fcfb80ffad0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.236+0000 7fcf9ffff700 1 -- 172.21.15.5:0/928416196 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa800f620 con 0x7fcfb80ffad0 2023-07-21T19:27:43.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.236+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb81a4c30 con 0x7fcfb80ffad0 2023-07-21T19:27:43.229 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.239+0000 7fcf9ffff700 1 -- 172.21.15.5:0/928416196 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcfa8010460 con 0x7fcfb80ffad0 2023-07-21T19:27:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 Processor -- start 2023-07-21T19:27:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcf9ffff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 0x7fcfa00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 -- start start 2023-07-21T19:27:43.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcf9ffff700 1 -- 172.21.15.5:0/928416196 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcfa80cd090 con 0x7fcfb80ffad0 2023-07-21T19:27:43.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 0x7fcfa00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae7be0 0x561636af2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636af4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616369edcb0 con 0x561636af2ad0 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636a40a40 con 0x561636ae7be0 2023-07-21T19:27:43.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.241+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636a39ed0 con 0x561636ae72d0 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51496/0 (socket says 172.21.15.5:51496) 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/1671500552 learned_addr learned my addr 172.21.15.5:0/1671500552 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae7be0 0x561636af2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636af4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/1671500552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae7be0 msgr2=0x561636af2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 0x7fcfa00a3770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b070 tx=0x7fcfb0009c70 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae7be0 0x561636af2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/1671500552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 msgr2=0x561636af4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636af4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.242+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/1671500552 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561636867de0 con 0x561636ae72d0 2023-07-21T19:27:43.235 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.243+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8010ab0 tx=0x7fcfa809a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c77bb8f9f9756e4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.243+0000 7fcf9d7fa700 1 -- 172.21.15.5:0/1671500552 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa809a100 con 0x561636ae72d0 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.243+0000 7fcf9d7fa700 1 -- 172.21.15.5:0/1671500552 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfa809e130 con 0x561636ae72d0 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.243+0000 7fcf9d7fa700 1 -- 172.21.15.5:0/1671500552 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfa809a360 con 0x561636ae72d0 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1671500552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 msgr2=0x561636ae76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fcfa8010ab0 tx=0x7fcfa809a000 comp rx=0 tx=0).stop 2023-07-21T19:27:43.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1671500552 shutdown_connections 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636af4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae72d0 0x561636ae76a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1671500552 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae7be0 0x561636af2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1671500552 >> 172.21.15.5:0/1671500552 conn(0x561636addda0 msgr2=0x561636ae36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1671500552 shutdown_connections 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.244+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1671500552 wait complete. 2023-07-21T19:27:43.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.245+0000 7fcfd0cb0b80 1 -- start start 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.245+0000 7fcfd0cb0b80 1 Processor -- start 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636aef890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616369d9ac0 con 0x561636ae7be0 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616369d97c0 con 0x561636ae72d0 2023-07-21T19:27:43.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5616368bd920 con 0x561636af2ad0 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfd0cb0b80 10 client.0 _ll_register_callbacks cb 0x5616369d7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfb67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfb67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59370/0 (socket says 172.21.15.5:59370) 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1308899793 learned_addr learned my addr 172.21.15.5:0/1308899793 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636aef890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.246+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 msgr2=0x561636af0310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af0310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1308899793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 msgr2=0x561636aef890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636aef890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1308899793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfb81a4c30 con 0x561636ae7be0 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00bf50 tx=0x7fcfac012730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e35fbafbba2648 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcf9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:27:43.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac0107e0 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5616369934f0 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.247+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcfac010940 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfac010b00 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 10 client.19512 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x561636850c60 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfac027080 con 0x561636ae7be0 2023-07-21T19:27:43.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcfac021030 con 0x561636ae7be0 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 10 client.19512 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 10 client.19512 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 20 client.19512 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b02430 0x561636b04840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 4 client.19512 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.248+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561636a32eb0 con 0x561636b02430 2023-07-21T19:27:43.244 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.249+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b02430 0x561636b04840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.249+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b02430 0x561636b04840 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=30632888ba6845e3 server_cookie=7e0e98c4e866c17e in_seq=0 out_seq=0 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.249+0000 7fcf9aff5700 10 client.19512 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1308899793 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561636a32eb0 con 0x561636b02430 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcf9aff5700 10 client.19512 handle_command_reply: tid=0 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 client.19512 shutdown 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 msgr2=0x561636aefdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.245 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fcfac00bf50 tx=0x7fcfac012730 comp rx=0 tx=0).stop 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 shutdown_connections 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af0310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b02430 0x561636b04840 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636aef890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1308899793 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636aefdd0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcf9aff5700 0 client.19512 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.250+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 >> 172.21.15.5:0/1308899793 conn(0x561636addda0 msgr2=0x561636ae3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.246 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.251+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 shutdown_connections 2023-07-21T19:27:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.251+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1308899793 wait complete. 2023-07-21T19:27:43.247 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.251+0000 7fcfd0cb0b80 20 client.19512 trim_cache size 0 max 16384 2023-07-21T19:27:43.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.284+0000 7fcfd0cb0b80 1 Processor -- start 2023-07-21T19:27:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.285+0000 7fcfd0cb0b80 1 -- start start 2023-07-21T19:27:43.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 0x561636af0140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636aecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636ae8540 con 0x561636af2ad0 2023-07-21T19:27:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636aea6a0 con 0x561636ae72d0 2023-07-21T19:27:43.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636ae8100 con 0x561636ae7be0 2023-07-21T19:27:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfb6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43718/0 (socket says 172.21.15.5:43718) 2023-07-21T19:27:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.286+0000 7fcfb6ffd700 1 -- 172.21.15.5:0/1071849361 learned_addr learned my addr 172.21.15.5:0/1071849361 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636aecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 0x561636af0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1071849361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 msgr2=0x561636af94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1071849361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 msgr2=0x561636aecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636aecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 -- 172.21.15.5:0/1071849361 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56163688e250 con 0x561636ae7be0 2023-07-21T19:27:43.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.287+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 0x561636af0140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcfac01bec0 tx=0x7fcfac005910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35b1136ba22059f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.288+0000 7fcf9dffb700 1 -- 172.21.15.5:0/1071849361 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac01b640 con 0x561636ae7be0 2023-07-21T19:27:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.288+0000 7fcf9dffb700 1 -- 172.21.15.5:0/1071849361 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5616369934f0 con 0x561636ae7be0 2023-07-21T19:27:43.279 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.288+0000 7fcf9dffb700 1 -- 172.21.15.5:0/1071849361 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfac010d60 con 0x561636ae7be0 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1071849361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 msgr2=0x561636af0140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 0x561636af0140 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fcfac01bec0 tx=0x7fcfac005910 comp rx=0 tx=0).stop 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1071849361 shutdown_connections 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636af2ad0 0x561636aecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1071849361 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636ae7be0 0x561636af0140 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1071849361 >> 172.21.15.5:0/1071849361 conn(0x561636addda0 msgr2=0x561636afe760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1071849361 shutdown_connections 2023-07-21T19:27:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.289+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1071849361 wait complete. 2023-07-21T19:27:43.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.290+0000 7fcfd0cb0b80 1 -- start start 2023-07-21T19:27:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.290+0000 7fcfd0cb0b80 1 Processor -- start 2023-07-21T19:27:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636af75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636af81b0 con 0x561636ae7be0 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636af8320 con 0x561636ae72d0 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561636b56300 con 0x561636af2ad0 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfd0cb0b80 10 client.0 _ll_register_callbacks cb 0x7fcfac00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfb77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51522/0 (socket says 172.21.15.5:51522) 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.291+0000 7fcfb77fe700 1 -- 172.21.15.5:0/1968027892 learned_addr learned my addr 172.21.15.5:0/1968027892 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636af75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb77fe700 1 -- 172.21.15.5:0/1968027892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 msgr2=0x561636af92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb77fe700 1 -- 172.21.15.5:0/1968027892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 msgr2=0x561636af75a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636af75a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb77fe700 1 -- 172.21.15.5:0/1968027892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561636850c60 con 0x561636af2ad0 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.292+0000 7fcfb67fc700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636af75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:43.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.293+0000 7fcfb77fe700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b0a0 tx=0x7fcfb0006000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=111a760cfc0f19f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.293+0000 7fcf9aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.293+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfb000e040 con 0x561636af2ad0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.293+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcfb001ecf0 con 0x561636af2ad0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.294+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcfa809e130 con 0x561636af2ad0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.294+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcfb002b470 con 0x561636af2ad0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.294+0000 7fcfd0cb0b80 10 client.29095 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.294+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5616368662b0 con 0x561636af2ad0 2023-07-21T19:27:43.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcfa809e130 con 0x561636af2ad0 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcfb0019070 con 0x561636af2ad0 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 10 client.29095 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 10 client.29095 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 20 client.29095 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b57b90 0x561636b57f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 4 client.29095 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:43.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.295+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x5616369f5050 con 0x561636b57b90 2023-07-21T19:27:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.296+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b57b90 0x561636b57f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.297+0000 7fcfb6ffd700 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b57b90 0x561636b57f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6ac7e7d2a96b701f server_cookie=71f869458faebbbd in_seq=0 out_seq=0 2023-07-21T19:27:43.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.297+0000 7fcf9aff5700 10 client.29095 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcf9aff5700 1 -- 172.21.15.5:0/1968027892 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x5616369f5050 con 0x561636b57b90 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcf9aff5700 10 client.29095 handle_command_reply: tid=0 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 client.29095 shutdown 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 msgr2=0x561636af7ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b0a0 tx=0x7fcfb0006000 comp rx=0 tx=0).stop 2023-07-21T19:27:43.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 shutdown_connections 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561636af2ad0 0x561636af7ae0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x561636b57b90 0x561636b57f90 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561636ae72d0 0x561636af92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 --2- 172.21.15.5:0/1968027892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561636ae7be0 0x561636af75a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcf9aff5700 0 client.29095 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.298+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 >> 172.21.15.5:0/1968027892 conn(0x561636addda0 msgr2=0x561636ae93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.299+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 shutdown_connections 2023-07-21T19:27:43.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.299+0000 7fcfd0cb0b80 1 -- 172.21.15.5:0/1968027892 wait complete. 2023-07-21T19:27:43.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.299+0000 7fcfd0cb0b80 20 client.29095 trim_cache size 0 max 16384 2023-07-21T19:27:43.291 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 msgr2=0x7fcfa00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 0x7fcfa00a3770 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fcfb000b070 tx=0x7fcfb0009c70 comp rx=0 tx=0).stop 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 msgr2=0x7fcfb8197e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7fcfa800d950 tx=0x7fcfa800dd10 comp rx=0 tx=0).stop 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 shutdown_connections 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcfb8105ae0 0x7fcfb819c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcfa00a12c0 0x7fcfa00a3770 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcfb80ffad0 0x7fcfb8197e00 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 --2- 172.21.15.5:0/928416196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcfb8100480 0x7fcfb8198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.300+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 >> 172.21.15.5:0/928416196 conn(0x7fcfb80fb5f0 msgr2=0x7fcfb80fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.301+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 shutdown_connections 2023-07-21T19:27:43.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.301+0000 7fcfbd3ff700 1 -- 172.21.15.5:0/928416196 wait complete. 2023-07-21T19:27:43.322 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:27:43.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:43 smithi005 ceph-mon[126460]: pgmap v634: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-07-21T19:27:43.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:43 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1667275020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:43.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 -- 172.21.15.5:0/2263915126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f230406cee0 msgr2=0x7f230406d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 --2- 172.21.15.5:0/2263915126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f230406cee0 0x7f230406d350 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f22f80098f0 tx=0x7f22f8009c00 comp rx=0 tx=0).stop 2023-07-21T19:27:43.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 -- 172.21.15.5:0/2263915126 shutdown_connections 2023-07-21T19:27:43.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 --2- 172.21.15.5:0/2263915126 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2304114c80 0x7f2304117060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 --2- 172.21.15.5:0/2263915126 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 0x7f230406c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 --2- 172.21.15.5:0/2263915126 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f230406cee0 0x7f230406d350 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 -- 172.21.15.5:0/2263915126 >> 172.21.15.5:0/2263915126 conn(0x7f230406bf70 msgr2=0x7f2304070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.664+0000 7f230b9ff700 1 -- 172.21.15.5:0/2263915126 shutdown_connections 2023-07-21T19:27:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.665+0000 7f230b9ff700 1 -- 172.21.15.5:0/2263915126 wait complete. 2023-07-21T19:27:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.666+0000 7f230b9ff700 1 Processor -- start 2023-07-21T19:27:43.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.666+0000 7f230b9ff700 1 -- start start 2023-07-21T19:27:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 0x7f230411fef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 0x7f2304120430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041148b0 con 0x7f230410a190 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2304114a20 con 0x7f2304114c80 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f230b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23041a8800 con 0x7f230406cee0 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.667+0000 7f2309f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 0x7f230411fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f2309f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43752/0 (socket says 172.21.15.5:43752) 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f2309f9c700 1 -- 172.21.15.5:0/3305344043 learned_addr learned my addr 172.21.15.5:0/3305344043 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 0x7f2304120430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 -- 172.21.15.5:0/3305344043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 msgr2=0x7f23041142b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 -- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 msgr2=0x7f2304120430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 0x7f2304120430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f230979b700 1 -- 172.21.15.5:0/3305344043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22f8009520 con 0x7f230406cee0 2023-07-21T19:27:43.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.668+0000 7f2309f9c700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.669+0000 7f230979b700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 0x7f230411fef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.669+0000 7f23027fc700 1 -- 172.21.15.5:0/3305344043 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0096d0 con 0x7f230406cee0 2023-07-21T19:27:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.669+0000 7f23027fc700 1 -- 172.21.15.5:0/3305344043 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22fc00cbd0 con 0x7f230406cee0 2023-07-21T19:27:43.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.669+0000 7f23027fc700 1 -- 172.21.15.5:0/3305344043 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc018680 con 0x7f230406cee0 2023-07-21T19:27:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.669+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23041a8ae0 con 0x7f230406cee0 2023-07-21T19:27:43.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.670+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23041a8f70 con 0x7f230406cee0 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.676+0000 7f231f345b80 1 Processor -- start 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 -- start start 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bb33cb0 con 0x55ed0bc38ad0 2023-07-21T19:27:43.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bb86a40 con 0x55ed0bc2dbe0 2023-07-21T19:27:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.677+0000 7f231f345b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bb7fed0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.678+0000 7f2308f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.678+0000 7f2308f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59416/0 (socket says 172.21.15.5:59416) 2023-07-21T19:27:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.678+0000 7f2308f9a700 1 -- 172.21.15.5:0/1616236739 learned_addr learned my addr 172.21.15.5:0/1616236739 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.678+0000 7f230979b700 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 -- 172.21.15.5:0/1616236739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 msgr2=0x55ed0bc2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 -- 172.21.15.5:0/1616236739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 msgr2=0x55ed0bc38590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 -- 172.21.15.5:0/1616236739 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0b9adde0 con 0x55ed0bc38ad0 2023-07-21T19:27:43.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.679+0000 7f2308f9a700 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eff95b41f7db9a9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.680+0000 7f22eb7fe700 1 -- 172.21.15.5:0/1616236739 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f800fe20 con 0x55ed0bc38ad0 2023-07-21T19:27:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.680+0000 7f22eb7fe700 1 -- 172.21.15.5:0/1616236739 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22f801d040 con 0x55ed0bc38ad0 2023-07-21T19:27:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.680+0000 7f22eb7fe700 1 -- 172.21.15.5:0/1616236739 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22f8005040 con 0x55ed0bc38ad0 2023-07-21T19:27:43.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.682+0000 7f23027fc700 1 -- 172.21.15.5:0/3305344043 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f22fc010430 con 0x7f230406cee0 2023-07-21T19:27:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.682+0000 7f231f345b80 1 -- 172.21.15.5:0/1616236739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 msgr2=0x55ed0bc3aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.682+0000 7f231f345b80 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f22f8000c00 tx=0x7f22f8005dc0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.683+0000 7f231f345b80 1 -- 172.21.15.5:0/1616236739 shutdown_connections 2023-07-21T19:27:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.683+0000 7f231f345b80 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc3aeb0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.683+0000 7f231f345b80 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.683+0000 7f231f345b80 1 --2- 172.21.15.5:0/1616236739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.683+0000 7f231f345b80 1 -- 172.21.15.5:0/1616236739 >> 172.21.15.5:0/1616236739 conn(0x55ed0bc23da0 msgr2=0x55ed0bc296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.684+0000 7f231f345b80 1 -- 172.21.15.5:0/1616236739 shutdown_connections 2023-07-21T19:27:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.684+0000 7f23027fc700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 0x7f22ec0a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.685+0000 7f23027fc700 1 -- 172.21.15.5:0/3305344043 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f22fc014070 con 0x7f230406cee0 2023-07-21T19:27:43.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.685+0000 7f231f345b80 1 -- 172.21.15.5:0/1616236739 wait complete. 2023-07-21T19:27:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.685+0000 7f231f345b80 1 -- start start 2023-07-21T19:27:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.685+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 0x7f22ec0a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.686+0000 7f231f345b80 1 Processor -- start 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.686+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 0x7f22ec0a3700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005a40 tx=0x7f22f8009fd0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.686+0000 7f231f345b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.686+0000 7f231f345b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc35730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.686+0000 7f231f345b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc35d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f231f345b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bb1fac0 con 0x55ed0bc38ad0 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f231f345b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bb1f7c0 con 0x55ed0bc2dbe0 2023-07-21T19:27:43.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f231f345b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0ba03920 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f231f345b80 10 client.0 _ll_register_callbacks cb 0x55ed0bb1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f230979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f230979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51558/0 (socket says 172.21.15.5:51558) 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f230979b700 1 -- 172.21.15.5:0/3444319575 learned_addr learned my addr 172.21.15.5:0/3444319575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f2309f9c700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc35d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.687+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc35730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 -- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 msgr2=0x55ed0bc35730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc35730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 -- 172.21.15.5:0/3444319575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 msgr2=0x55ed0bc35d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc35d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 -- 172.21.15.5:0/3444319575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2304117be0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f230979b700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9c0 tx=0x7f22fc010990 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ea6b9ae4d37b1b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f22e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:43.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc070070 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.688+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f22fc0a0d00 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed0bad94f0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22fc0a0660 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f231f345b80 10 client.29104 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55ed0b996c60 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ed0bad94f0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.689+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f22fc0a8cc0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 10 client.29104 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 10 client.29104 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 20 client.29104 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bc6a2c0 0x55ed0bc6c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 4 client.29104 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ed0bb78eb0 con 0x55ed0bc6a2c0 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.690+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bc6a2c0 0x55ed0bc6c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.691+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bc6a2c0 0x55ed0bc6c6d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=70073ba1159c5ab2 server_cookie=c06b3f028041d416 in_seq=0 out_seq=0 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.691+0000 7f22e8ff9700 10 client.29104 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.691+0000 7f22e8ff9700 1 -- 172.21.15.5:0/3444319575 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ed0bb78eb0 con 0x55ed0bc6a2c0 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.692+0000 7f22e8ff9700 10 client.29104 handle_command_reply: tid=0 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.692+0000 7f231f345b80 1 client.29104 shutdown 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.692+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 msgr2=0x55ed0bc384c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.692+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9c0 tx=0x7f22fc010990 comp rx=0 tx=0).stop 2023-07-21T19:27:43.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 shutdown_connections 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc35d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc384c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bc6a2c0 0x55ed0bc6c6d0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 --2- 172.21.15.5:0/3444319575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc35730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 >> 172.21.15.5:0/3444319575 conn(0x55ed0bc23da0 msgr2=0x55ed0bc29290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f22e8ff9700 0 client.29104 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 shutdown_connections 2023-07-21T19:27:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 1 -- 172.21.15.5:0/3444319575 wait complete. 2023-07-21T19:27:43.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.693+0000 7f231f345b80 20 client.29104 trim_cache size 0 max 16384 2023-07-21T19:27:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.714+0000 7f231f345b80 1 Processor -- start 2023-07-21T19:27:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.714+0000 7f231f345b80 1 -- start start 2023-07-21T19:27:43.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc34ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bc36cf0 con 0x55ed0bc2dbe0 2023-07-21T19:27:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bc2f1a0 con 0x55ed0bc38ad0 2023-07-21T19:27:43.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.715+0000 7f231f345b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bc2edd0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51566/0 (socket says 172.21.15.5:51566) 2023-07-21T19:27:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 -- 172.21.15.5:0/3908952218 learned_addr learned my addr 172.21.15.5:0/3908952218 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f2309f9c700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc34ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 -- 172.21.15.5:0/3908952218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 msgr2=0x55ed0bc34ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc34ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 -- 172.21.15.5:0/3908952218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 msgr2=0x55ed0bc38030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f230979b700 1 -- 172.21.15.5:0/3908952218 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ed0b9d4250 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.716+0000 7f2308f9a700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:43.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.717+0000 7f230979b700 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22fc09d170 tx=0x7f22fc00c5b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fac231d0dba42cc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.717+0000 7f22ebfff700 1 -- 172.21.15.5:0/3908952218 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc070070 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.717+0000 7f22ebfff700 1 -- 172.21.15.5:0/3908952218 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22fc0a0660 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.717+0000 7f22ebfff700 1 -- 172.21.15.5:0/3908952218 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc0a8cc0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 -- 172.21.15.5:0/3908952218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 msgr2=0x55ed0bc614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f22fc09d170 tx=0x7f22fc00c5b0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 -- 172.21.15.5:0/3908952218 shutdown_connections 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc34ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc614f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 --2- 172.21.15.5:0/3908952218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc38030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.718+0000 7f231f345b80 1 -- 172.21.15.5:0/3908952218 >> 172.21.15.5:0/3908952218 conn(0x55ed0bc23da0 msgr2=0x55ed0bc66730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.719+0000 7f231f345b80 1 -- 172.21.15.5:0/3908952218 shutdown_connections 2023-07-21T19:27:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.719+0000 7f231f345b80 1 -- 172.21.15.5:0/3908952218 wait complete. 2023-07-21T19:27:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.719+0000 7f231f345b80 1 -- start start 2023-07-21T19:27:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 Processor -- start 2023-07-21T19:27:43.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc61500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc31e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc32340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bc32a10 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bc32b80 con 0x55ed0bc38ad0 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ed0bcbe140 con 0x55ed0bc2dbe0 2023-07-21T19:27:43.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.720+0000 7f231f345b80 10 client.0 _ll_register_callbacks cb 0x55ed0bb1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.721+0000 7f2309f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc32340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.721+0000 7f2309f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc32340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43788/0 (socket says 172.21.15.5:43788) 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.721+0000 7f2309f9c700 1 -- 172.21.15.5:0/2323657162 learned_addr learned my addr 172.21.15.5:0/2323657162 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.721+0000 7f230979b700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc61500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.721+0000 7f2308f9a700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc31e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 -- 172.21.15.5:0/2323657162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2dbe0 msgr2=0x55ed0bc31e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc31e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 -- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 msgr2=0x55ed0bc32340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:43.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc32340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 -- 172.21.15.5:0/2323657162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ed0b9adde0 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f230979b700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc61500 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f22fc09d690 tx=0x7f22fc09d060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de168129369d06cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.722+0000 7f22e8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22fc070070 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f22fc0a0d00 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ed0bb3b050 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22fc0d1250 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f231f345b80 10 client.19533 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:43.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.723+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55ed0b996c60 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f22fc06a070 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f22fc09cb70 con 0x55ed0bc2d2d0 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 10 client.19533 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 10 client.19533 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 20 client.19533 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:43.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bcbf1a0 0x55ed0bcc1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 4 client.19533 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55ed0b9cef70 con 0x55ed0bcbf1a0 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.724+0000 7f2308f9a700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bcbf1a0 0x55ed0bcc1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.725+0000 7f2308f9a700 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bcbf1a0 0x55ed0bcc1580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e63808adb8a708ef server_cookie=d90283c6042bd852 in_seq=0 out_seq=0 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.725+0000 7f22e8ff9700 10 client.19533 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f22e8ff9700 1 -- 172.21.15.5:0/2323657162 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55ed0b9cef70 con 0x55ed0bcbf1a0 2023-07-21T19:27:43.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f22e8ff9700 10 client.19533 handle_command_reply: tid=0 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 client.19533 shutdown 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 msgr2=0x55ed0bc61500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc61500 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f22fc09d690 tx=0x7f22fc09d060 comp rx=0 tx=0).stop 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 shutdown_connections 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x55ed0bcbf1a0 0x55ed0bcc1580 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ed0bc38ad0 0x55ed0bc32340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.718 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ed0bc2d2d0 0x55ed0bc61500 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 --2- 172.21.15.5:0/2323657162 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ed0bc2dbe0 0x55ed0bc31e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 >> 172.21.15.5:0/2323657162 conn(0x55ed0bc23da0 msgr2=0x55ed0bc2f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f22e8ff9700 0 client.19533 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:27:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 shutdown_connections 2023-07-21T19:27:43.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.726+0000 7f231f345b80 1 -- 172.21.15.5:0/2323657162 wait complete. 2023-07-21T19:27:43.720 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f231f345b80 20 client.19533 trim_cache size 0 max 16384 2023-07-21T19:27:43.720 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 msgr2=0x7f22ec0a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 0x7f22ec0a3700 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f22f8005a40 tx=0x7f22f8009fd0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 msgr2=0x7f230411fef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 0x7f230411fef0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f22fc00e9f0 tx=0x7f22fc00ed00 comp rx=0 tx=0).stop 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 shutdown_connections 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2304114c80 0x7f23041142b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f230406cee0 0x7f230411fef0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22ec0a1250 0x7f22ec0a3700 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.727+0000 7f230b9ff700 1 --2- 172.21.15.5:0/3305344043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f230410a190 0x7f2304120430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:43.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.728+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 >> 172.21.15.5:0/3305344043 conn(0x7f230406bf70 msgr2=0x7f23041154b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:43.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.728+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 shutdown_connections 2023-07-21T19:27:43.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:43.728+0000 7f230b9ff700 1 -- 172.21.15.5:0/3305344043 wait complete. 2023-07-21T19:27:43.744 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:27:43.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:43 smithi156 ceph-mon[128316]: pgmap v634: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-07-21T19:27:43.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:43 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1667275020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:44.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:43 smithi101 ceph-mon[128136]: pgmap v634: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 668 KiB/s rd, 20 MiB/s wr, 1.36k op/s 2023-07-21T19:27:44.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:43 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1667275020' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:27:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.112+0000 7f357d5ff700 1 -- 172.21.15.5:0/119653694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 msgr2=0x7f3578104db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.112+0000 7f357d5ff700 1 --2- 172.21.15.5:0/119653694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f3578104db0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3560009990 tx=0x7f3560009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 -- 172.21.15.5:0/119653694 shutdown_connections 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 --2- 172.21.15.5:0/119653694 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780ff380 0x7f357810b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 --2- 172.21.15.5:0/119653694 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f3578104db0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 --2- 172.21.15.5:0/119653694 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35780fe9d0 0x7f35780fee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 -- 172.21.15.5:0/119653694 >> 172.21.15.5:0/119653694 conn(0x7f35780fa4b0 msgr2=0x7f35780fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 -- 172.21.15.5:0/119653694 shutdown_connections 2023-07-21T19:27:44.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.113+0000 7f357d5ff700 1 -- 172.21.15.5:0/119653694 wait complete. 2023-07-21T19:27:44.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.115+0000 7f357d5ff700 1 Processor -- start 2023-07-21T19:27:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.115+0000 7f357d5ff700 1 -- start start 2023-07-21T19:27:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.115+0000 7f357d5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.115+0000 7f357d5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35780ff380 0x7f3578071ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.115+0000 7f357d5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f357806a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f357d5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357806ac80 con 0x7f35780ff380 2023-07-21T19:27:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f357d5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357806adc0 con 0x7f35781049e0 2023-07-21T19:27:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f357d5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357806af80 con 0x7f35780fe9d0 2023-07-21T19:27:44.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f35777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f357806a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51592/0 (socket says 172.21.15.5:51592) 2023-07-21T19:27:44.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.116+0000 7f3576ffd700 1 -- 172.21.15.5:0/2542248537 learned_addr learned my addr 172.21.15.5:0/2542248537 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:44.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f3576ffd700 1 -- 172.21.15.5:0/2542248537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 msgr2=0x7f357806a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f357806a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f3576ffd700 1 -- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35780ff380 msgr2=0x7f3578071ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35780ff380 0x7f3578071ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f3576ffd700 1 -- 172.21.15.5:0/2542248537 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3568009520 con 0x7f35780fe9d0 2023-07-21T19:27:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.117+0000 7f35777fe700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f357806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.118+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f35600098e0 tx=0x7f3560005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.119+0000 7f3574ff9700 1 -- 172.21.15.5:0/2542248537 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356001d070 con 0x7f35780fe9d0 2023-07-21T19:27:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.119+0000 7f3574ff9700 1 -- 172.21.15.5:0/2542248537 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35600173f0 con 0x7f35780fe9d0 2023-07-21T19:27:44.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.119+0000 7f3574ff9700 1 -- 172.21.15.5:0/2542248537 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356000e640 con 0x7f35780fe9d0 2023-07-21T19:27:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.120+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35600095f0 con 0x7f35780fe9d0 2023-07-21T19:27:44.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.120+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357806b3e0 con 0x7f35780fe9d0 2023-07-21T19:27:44.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.125+0000 7f3574ff9700 1 -- 172.21.15.5:0/2542248537 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3560017560 con 0x7f35780fe9d0 2023-07-21T19:27:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 Processor -- start 2023-07-21T19:27:44.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 -- start start 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 0x563746b826a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b8dad0 0x563746b8feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746a88cb0 con 0x563746b822d0 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746adba40 con 0x563746b82be0 2023-07-21T19:27:44.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.127+0000 7f3590f13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746ad4ed0 con 0x563746b8dad0 2023-07-21T19:27:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.128+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.128+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43816/0 (socket says 172.21.15.5:43816) 2023-07-21T19:27:44.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.128+0000 7f3576ffd700 1 -- 172.21.15.5:0/2550456056 learned_addr learned my addr 172.21.15.5:0/2550456056 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.128+0000 7f35777fe700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 0x563746b826a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.129+0000 7f356ffff700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b8dad0 0x563746b8feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.130+0000 7f3574ff9700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 0x7f35640a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.131+0000 7f3574ff9700 1 -- 172.21.15.5:0/2542248537 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f35600ce690 con 0x7f35780fe9d0 2023-07-21T19:27:44.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.131+0000 7f35777fe700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 0x7f35640a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 -- 172.21.15.5:0/2550456056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b8dad0 msgr2=0x563746b8feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b8dad0 0x563746b8feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 -- 172.21.15.5:0/2550456056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 msgr2=0x563746b826a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 0x563746b826a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 -- 172.21.15.5:0/2550456056 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563746902de0 con 0x563746b82be0 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f35777fe700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 0x563746b826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:44.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.132+0000 7f3576ffd700 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f356009b720 tx=0x7f3560097490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1eaa29877070b8c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f356d7fa700 1 -- 172.21.15.5:0/2550456056 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356009b0f0 con 0x563746b82be0 2023-07-21T19:27:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f35777fe700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 0x7f35640a3900 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f35700195b0 tx=0x7f3570004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f356d7fa700 1 -- 172.21.15.5:0/2550456056 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f356009b790 con 0x563746b82be0 2023-07-21T19:27:44.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f356d7fa700 1 -- 172.21.15.5:0/2550456056 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356009b9f0 con 0x563746b82be0 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f3590f13b80 1 -- 172.21.15.5:0/2550456056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 msgr2=0x563746b8d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.133+0000 7f3590f13b80 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f356009b720 tx=0x7f3560097490 comp rx=0 tx=0).stop 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 -- 172.21.15.5:0/2550456056 shutdown_connections 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b8dad0 0x563746b8feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b822d0 0x563746b826a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 --2- 172.21.15.5:0/2550456056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746b8d590 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 -- 172.21.15.5:0/2550456056 >> 172.21.15.5:0/2550456056 conn(0x563746b78da0 msgr2=0x563746b7e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 -- 172.21.15.5:0/2550456056 shutdown_connections 2023-07-21T19:27:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.134+0000 7f3590f13b80 1 -- 172.21.15.5:0/2550456056 wait complete. 2023-07-21T19:27:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.135+0000 7f3590f13b80 1 -- start start 2023-07-21T19:27:44.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.135+0000 7f3590f13b80 1 Processor -- start 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746b92aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b82be0 0x563746b92fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746a74ac0 con 0x563746b82be0 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746a747c0 con 0x563746b8dad0 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746958920 con 0x563746b822d0 2023-07-21T19:27:44.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f3590f13b80 10 client.0 _ll_register_callbacks cb 0x563746a72760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.136+0000 7f35777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43822/0 (socket says 172.21.15.5:43822) 2023-07-21T19:27:44.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 -- 172.21.15.5:0/3444722420 learned_addr learned my addr 172.21.15.5:0/3444722420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f356ffff700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b82be0 0x563746b92fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746b92aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 -- 172.21.15.5:0/3444722420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 msgr2=0x563746b92aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746b92aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 -- 172.21.15.5:0/3444722420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b82be0 msgr2=0x563746b92fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b82be0 0x563746b92fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.137+0000 7f35777fe700 1 -- 172.21.15.5:0/3444722420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3570007e20 con 0x563746b8dad0 2023-07-21T19:27:44.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f35777fe700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3570019f60 tx=0x7f3570004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6930a89c4e5a453c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f355e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570022070 con 0x563746b8dad0 2023-07-21T19:27:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3570004bb0 con 0x563746b8dad0 2023-07-21T19:27:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563746a2e4f0 con 0x563746b8dad0 2023-07-21T19:27:44.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.138+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3570004d70 con 0x563746b8dad0 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 10 client.29204 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x7f357806b3e0 con 0x563746b8dad0 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563746a2e4f0 con 0x563746b8dad0 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f357001b7c0 con 0x563746b8dad0 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 10 client.29204 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 10 client.29204 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 20 client.29204 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:44.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746b9d060 0x563746b9f470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 4 client.29204 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.139+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563746acdeb0 con 0x563746b9d060 2023-07-21T19:27:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.140+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746b9d060 0x563746b9f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.140+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746b9d060 0x563746b9f470 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c351fb596971443 server_cookie=c6c495c48d7a34ad in_seq=0 out_seq=0 2023-07-21T19:27:44.132 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.140+0000 7f355e7fc700 10 client.29204 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.141+0000 7f355e7fc700 1 -- 172.21.15.5:0/3444722420 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563746acdeb0 con 0x563746b9d060 2023-07-21T19:27:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.141+0000 7f355e7fc700 10 client.29204 handle_command_reply: tid=0 2023-07-21T19:27:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.142+0000 7f3590f13b80 1 client.29204 shutdown 2023-07-21T19:27:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.142+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 msgr2=0x563746b8c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.142+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f3570019f60 tx=0x7f3570004ad0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 shutdown_connections 2023-07-21T19:27:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b8dad0 0x563746b8c950 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746b92aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746b9d060 0x563746b9f470 crc :-1 s=READY pgs=407 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3444722420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b82be0 0x563746b92fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 >> 172.21.15.5:0/3444722420 conn(0x563746b78da0 msgr2=0x563746b7e290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f355e7fc700 0 client.29204 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.144+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 shutdown_connections 2023-07-21T19:27:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.145+0000 7f3590f13b80 1 -- 172.21.15.5:0/3444722420 wait complete. 2023-07-21T19:27:44.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.145+0000 7f3590f13b80 20 client.29204 trim_cache size 0 max 16384 2023-07-21T19:27:44.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.170+0000 7f3590f13b80 1 Processor -- start 2023-07-21T19:27:44.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.170+0000 7f3590f13b80 1 -- start start 2023-07-21T19:27:44.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.170+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.170+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b82be0 0x563746b92bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746b89c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3590f13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746b78970 con 0x563746b8dad0 2023-07-21T19:27:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3590f13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746b8aed0 con 0x563746b822d0 2023-07-21T19:27:44.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3590f13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746ba4150 con 0x563746b82be0 2023-07-21T19:27:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43830/0 (socket says 172.21.15.5:43830) 2023-07-21T19:27:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f3576ffd700 1 -- 172.21.15.5:0/3227501796 learned_addr learned my addr 172.21.15.5:0/3227501796 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:44.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f35777fe700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746b89c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.171+0000 7f356ffff700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b82be0 0x563746b92bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 -- 172.21.15.5:0/3227501796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b82be0 msgr2=0x563746b92bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b82be0 0x563746b92bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 -- 172.21.15.5:0/3227501796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 msgr2=0x563746b89c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746b89c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 -- 172.21.15.5:0/3227501796 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637468ebc60 con 0x563746b822d0 2023-07-21T19:27:44.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f35777fe700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746b89c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:27:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.172+0000 7f3576ffd700 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f356000b2e0 tx=0x7f3560009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8e092ada3f964e40 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.173+0000 7f356dffb700 1 -- 172.21.15.5:0/3227501796 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356009f840 con 0x563746b822d0 2023-07-21T19:27:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.173+0000 7f356dffb700 1 -- 172.21.15.5:0/3227501796 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3560099b30 con 0x563746b822d0 2023-07-21T19:27:44.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.173+0000 7f356dffb700 1 -- 172.21.15.5:0/3227501796 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600b9af0 con 0x563746b822d0 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 -- 172.21.15.5:0/3227501796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 msgr2=0x563746b92670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f356000b2e0 tx=0x7f3560009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 -- 172.21.15.5:0/3227501796 shutdown_connections 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746b89c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b822d0 0x563746b92670 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 --2- 172.21.15.5:0/3227501796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b82be0 0x563746b92bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.174+0000 7f3590f13b80 1 -- 172.21.15.5:0/3227501796 >> 172.21.15.5:0/3227501796 conn(0x563746b78da0 msgr2=0x563746b7ce10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.175+0000 7f3590f13b80 1 -- 172.21.15.5:0/3227501796 shutdown_connections 2023-07-21T19:27:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.176+0000 7f3590f13b80 1 -- 172.21.15.5:0/3227501796 wait complete. 2023-07-21T19:27:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.176+0000 7f3590f13b80 1 -- start start 2023-07-21T19:27:44.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.177+0000 7f3590f13b80 1 Processor -- start 2023-07-21T19:27:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.177+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746bea6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746be8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746be9460 con 0x563746b8dad0 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746be95d0 con 0x563746b82be0 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563746be9790 con 0x563746b822d0 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3590f13b80 10 client.0 _ll_register_callbacks cb 0x563746a72760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51624/0 (socket says 172.21.15.5:51624) 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 -- 172.21.15.5:0/1923233071 learned_addr learned my addr 172.21.15.5:0/1923233071 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 -- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 msgr2=0x563746bea6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746bea6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 -- 172.21.15.5:0/1923233071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 msgr2=0x563746be8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746be8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.178+0000 7f3576ffd700 1 -- 172.21.15.5:0/1923233071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357806b3e0 con 0x563746b822d0 2023-07-21T19:27:44.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.179+0000 7f3576ffd700 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3560097c40 tx=0x7f356009b0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1ff4b89f77375f09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.179+0000 7f355e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.179+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35600178d0 con 0x563746b822d0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.179+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f35600b9890 con 0x563746b822d0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.179+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f356009b790 con 0x563746b822d0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.180+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35600b9a50 con 0x563746b822d0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.180+0000 7f3590f13b80 10 client.29128 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.180+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563746902de0 con 0x563746b822d0 2023-07-21T19:27:44.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.180+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f35600bf030 con 0x563746b822d0 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.180+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f356009baf0 con 0x563746b822d0 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 10 client.29128 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 10 client.29128 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 20 client.29128 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746bf1c50 0x563746bf4030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 4 client.29128 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:27:44.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563746a90050 con 0x563746bf1c50 2023-07-21T19:27:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.181+0000 7f356ffff700 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746bf1c50 0x563746bf4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:44.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f355e7fc700 10 client.29128 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f356ffff700 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746bf1c50 0x563746bf4030 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2dc7791a4ea2436b server_cookie=4a81839acfc96d2e in_seq=0 out_seq=0 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f355e7fc700 1 -- 172.21.15.5:0/1923233071 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563746a90050 con 0x563746bf1c50 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f355e7fc700 10 client.29128 handle_command_reply: tid=0 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f3590f13b80 1 client.29128 shutdown 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 msgr2=0x563746bea190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.182+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f3560097c40 tx=0x7f356009b0f0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 shutdown_connections 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563746bf1c50 0x563746bf4030 crc :-1 s=READY pgs=408 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563746b8dad0 0x563746be8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563746b822d0 0x563746bea190 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 --2- 172.21.15.5:0/1923233071 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563746b82be0 0x563746bea6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 >> 172.21.15.5:0/1923233071 conn(0x563746b78da0 msgr2=0x563746b84a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f355e7fc700 0 client.29128 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:44.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.183+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 shutdown_connections 2023-07-21T19:27:44.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.184+0000 7f3590f13b80 1 -- 172.21.15.5:0/1923233071 wait complete. 2023-07-21T19:27:44.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.185+0000 7f3590f13b80 20 client.29128 trim_cache size 0 max 16384 2023-07-21T19:27:44.176 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 msgr2=0x7f35640a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 0x7f35640a3900 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f35700195b0 tx=0x7f3570004b40 comp rx=0 tx=0).stop 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 msgr2=0x7f35780717a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f35600098e0 tx=0x7f3560005740 comp rx=0 tx=0).stop 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 shutdown_connections 2023-07-21T19:27:44.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f35781049e0 0x7f357806a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f35780fe9d0 0x7f35780717a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f35640a1450 0x7f35640a3900 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 --2- 172.21.15.5:0/2542248537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f35780ff380 0x7f3578071ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 >> 172.21.15.5:0/2542248537 conn(0x7f35780fa4b0 msgr2=0x7f35780fbb40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 shutdown_connections 2023-07-21T19:27:44.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:44.186+0000 7f357d5ff700 1 -- 172.21.15.5:0/2542248537 wait complete. 2023-07-21T19:27:45.203 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:27:45.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:27:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.597+0000 7f86491ff700 1 -- 172.21.15.5:0/2386843638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 msgr2=0x7f864410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.597+0000 7f86491ff700 1 --2- 172.21.15.5:0/2386843638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864410c610 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f863400b0a0 tx=0x7f863400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.602+0000 7f86491ff700 1 -- 172.21.15.5:0/2386843638 shutdown_connections 2023-07-21T19:27:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.602+0000 7f86491ff700 1 --2- 172.21.15.5:0/2386843638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f8644112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.602+0000 7f86491ff700 1 --2- 172.21.15.5:0/2386843638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864410c610 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.602+0000 7f86491ff700 1 --2- 172.21.15.5:0/2386843638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f864410cb50 0x7f8644106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.602+0000 7f86491ff700 1 -- 172.21.15.5:0/2386843638 >> 172.21.15.5:0/2386843638 conn(0x7f86440759e0 msgr2=0x7f8644077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.603+0000 7f86491ff700 1 -- 172.21.15.5:0/2386843638 shutdown_connections 2023-07-21T19:27:45.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.603+0000 7f86491ff700 1 -- 172.21.15.5:0/2386843638 wait complete. 2023-07-21T19:27:45.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.604+0000 7f86491ff700 1 Processor -- start 2023-07-21T19:27:45.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.604+0000 7f86491ff700 1 -- start start 2023-07-21T19:27:45.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f864406f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864406fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.594 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f864410cb50 0x7f864406a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864406af60 con 0x7f864410cb50 2023-07-21T19:27:45.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864406b0d0 con 0x7f864410c240 2023-07-21T19:27:45.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f86491ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f864406b290 con 0x7f8644106a70 2023-07-21T19:27:45.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.605+0000 7f863bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864406fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f863bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864406fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43856/0 (socket says 172.21.15.5:43856) 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f863bfff700 1 -- 172.21.15.5:0/1784382827 learned_addr learned my addr 172.21.15.5:0/1784382827 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f864406f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 -- 172.21.15.5:0/1784382827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 msgr2=0x7f864406fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864406fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 -- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f864410cb50 msgr2=0x7f864406a920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f864410cb50 0x7f864406a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.606+0000 7f8642d9d700 1 -- 172.21.15.5:0/1784382827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8634009d00 con 0x7f8644106a70 2023-07-21T19:27:45.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.607+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f864406f730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f863400b0a0 tx=0x7f8634004db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.607+0000 7f8640d99700 1 -- 172.21.15.5:0/1784382827 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863400e040 con 0x7f8644106a70 2023-07-21T19:27:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.607+0000 7f8640d99700 1 -- 172.21.15.5:0/1784382827 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f86340079d0 con 0x7f8644106a70 2023-07-21T19:27:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.607+0000 7f8640d99700 1 -- 172.21.15.5:0/1784382827 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86340052a0 con 0x7f8644106a70 2023-07-21T19:27:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.608+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f864406b510 con 0x7f8644106a70 2023-07-21T19:27:45.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.608+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f864406e670 con 0x7f8644106a70 2023-07-21T19:27:45.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.614+0000 7f865cbc0b80 1 Processor -- start 2023-07-21T19:27:45.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.615+0000 7f865cbc0b80 1 -- start start 2023-07-21T19:27:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.615+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 0x55b551988720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.616+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b551993610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.616+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 0x55b551995f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.616+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55188ed30 con 0x55b551988350 2023-07-21T19:27:45.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.616+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5518e1ac0 con 0x55b551988c60 2023-07-21T19:27:45.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.616+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5518daf50 con 0x55b551993b50 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 0x55b551988720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f863bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 0x55b551995f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f863bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 0x55b551995f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51636/0 (socket says 172.21.15.5:51636) 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f863bfff700 1 -- 172.21.15.5:0/77355450 learned_addr learned my addr 172.21.15.5:0/77355450 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f8640d99700 1 -- 172.21.15.5:0/1784382827 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8634005400 con 0x7f8644106a70 2023-07-21T19:27:45.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 -- 172.21.15.5:0/77355450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 msgr2=0x55b551995f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 0x55b551995f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 -- 172.21.15.5:0/77355450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 msgr2=0x55b551993610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:27:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b551993610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.617+0000 7f864359e700 1 -- 172.21.15.5:0/77355450 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b551708e60 con 0x55b551988350 2023-07-21T19:27:45.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.618+0000 7f864359e700 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 0x55b551988720 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f863c009bc0 tx=0x7f863c009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=174225fa128e41c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.620+0000 7f86397fa700 1 -- 172.21.15.5:0/77355450 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c009710 con 0x55b551988350 2023-07-21T19:27:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.620+0000 7f86397fa700 1 -- 172.21.15.5:0/77355450 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f863c004830 con 0x55b551988350 2023-07-21T19:27:45.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.620+0000 7f86397fa700 1 -- 172.21.15.5:0/77355450 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c00edf0 con 0x55b551988350 2023-07-21T19:27:45.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.620+0000 7f8640d99700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 0x7f86280a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.620+0000 7f8640d99700 1 -- 172.21.15.5:0/1784382827 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f86340ce300 con 0x7f8644106a70 2023-07-21T19:27:45.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 -- 172.21.15.5:0/77355450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 msgr2=0x55b551988720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 0x55b551988720 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7f863c009bc0 tx=0x7f863c009ed0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 -- 172.21.15.5:0/77355450 shutdown_connections 2023-07-21T19:27:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551993b50 0x55b551995f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551988350 0x55b551988720 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/77355450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b551993610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f865cbc0b80 1 -- 172.21.15.5:0/77355450 >> 172.21.15.5:0/77355450 conn(0x55b55197ee20 msgr2=0x55b551984720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.621+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 0x7f86280a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.622+0000 7f865cbc0b80 1 -- 172.21.15.5:0/77355450 shutdown_connections 2023-07-21T19:27:45.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.622+0000 7f865cbc0b80 1 -- 172.21.15.5:0/77355450 wait complete. 2023-07-21T19:27:45.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.623+0000 7f865cbc0b80 1 -- start start 2023-07-21T19:27:45.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.623+0000 7f865cbc0b80 1 Processor -- start 2023-07-21T19:27:45.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b551993510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551990e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55187ab40 con 0x55b551993b50 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55187a840 con 0x55b551988350 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55175e9a0 con 0x55b551988c60 2023-07-21T19:27:45.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f865cbc0b80 10 client.0 _ll_register_callbacks cb 0x55b5518787e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.625+0000 7f8642d9d700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 0x7f86280a37f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f80 tx=0x7f86340099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51642/0 (socket says 172.21.15.5:51642) 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 -- 172.21.15.5:0/2501257019 learned_addr learned my addr 172.21.15.5:0/2501257019 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f864359e700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b551993510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f864359e700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551990e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 -- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 msgr2=0x55b551993510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b551993510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 -- 172.21.15.5:0/2501257019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 msgr2=0x55b551990e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.626+0000 7f863bfff700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551990e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.627+0000 7f863bfff700 1 -- 172.21.15.5:0/2501257019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f863c009870 con 0x55b551988c60 2023-07-21T19:27:45.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.627+0000 7f864359e700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551990e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.627+0000 7f863bfff700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8630006db0 tx=0x7f8630010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddac13f8a1f9a4d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.627+0000 7f86267fc700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.628+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630014640 con 0x55b551988c60 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.628+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f86300147a0 con 0x55b551988c60 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.628+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b551834570 con 0x55b551988c60 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.629+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8630014960 con 0x55b551988c60 2023-07-21T19:27:45.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.629+0000 7f865cbc0b80 10 client.29140 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:45.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.629+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b5516f1ce0 con 0x55b551988c60 2023-07-21T19:27:45.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b551834570 con 0x55b551988c60 2023-07-21T19:27:45.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f863001dc30 con 0x55b551988c60 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 10 client.29140 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 10 client.29140 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 20 client.29140 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5519a3560 0x55b5519a5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 4 client.29140 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b5518d3f30 con 0x55b5519a3560 2023-07-21T19:27:45.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.630+0000 7f864359e700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5519a3560 0x55b5519a5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.632+0000 7f864359e700 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5519a3560 0x55b5519a5970 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=103afbcc5fe6c0ea server_cookie=69614221baf5f685 in_seq=0 out_seq=0 2023-07-21T19:27:45.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.633+0000 7f86267fc700 10 client.29140 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:45.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.633+0000 7f86267fc700 1 -- 172.21.15.5:0/2501257019 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b5518d3f30 con 0x55b5519a3560 2023-07-21T19:27:45.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.633+0000 7f86267fc700 10 client.29140 handle_command_reply: tid=0 2023-07-21T19:27:45.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.633+0000 7f865cbc0b80 1 client.29140 shutdown 2023-07-21T19:27:45.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 msgr2=0x55b551990790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8630006db0 tx=0x7f8630010860 comp rx=0 tx=0).stop 2023-07-21T19:27:45.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 shutdown_connections 2023-07-21T19:27:45.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b5519a3560 0x55b5519a5970 crc :-1 s=READY pgs=409 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551990e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b551993510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.634+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2501257019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990790 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.635+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 >> 172.21.15.5:0/2501257019 conn(0x55b55197ee20 msgr2=0x55b551984310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.635+0000 7f86267fc700 0 client.29140 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.635+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 shutdown_connections 2023-07-21T19:27:45.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.635+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2501257019 wait complete. 2023-07-21T19:27:45.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.635+0000 7f865cbc0b80 20 client.29140 trim_cache size 0 max 16384 2023-07-21T19:27:45.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.666+0000 7f865cbc0b80 1 Processor -- start 2023-07-21T19:27:45.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 -- start start 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b55198daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5519a0c10 con 0x55b551993b50 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b551980f40 con 0x55b551988350 2023-07-21T19:27:45.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.667+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55198edf0 con 0x55b551988c60 2023-07-21T19:27:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.668+0000 7f864359e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.668+0000 7f864359e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43868/0 (socket says 172.21.15.5:43868) 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.668+0000 7f864359e700 1 -- 172.21.15.5:0/430466138 learned_addr learned my addr 172.21.15.5:0/430466138 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.668+0000 7f863bfff700 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 -- 172.21.15.5:0/430466138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 msgr2=0x55b551990510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 -- 172.21.15.5:0/430466138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 msgr2=0x55b55198daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b55198daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 -- 172.21.15.5:0/430466138 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b55172f2d0 con 0x55b551988350 2023-07-21T19:27:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f864359e700 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f863c005dc0 tx=0x7f863c004f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94c5b659ad79282a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f8639ffb700 1 -- 172.21.15.5:0/430466138 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c01c040 con 0x55b551988350 2023-07-21T19:27:45.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f8639ffb700 1 -- 172.21.15.5:0/430466138 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f863c00e900 con 0x55b551988350 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.669+0000 7f8639ffb700 1 -- 172.21.15.5:0/430466138 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c005000 con 0x55b551988350 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 -- 172.21.15.5:0/430466138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 msgr2=0x55b55198ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f863c005dc0 tx=0x7f863c004f90 comp rx=0 tx=0).stop 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 -- 172.21.15.5:0/430466138 shutdown_connections 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b55198daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988350 0x55b55198ffd0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.670+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/430466138 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988c60 0x55b551990510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.671+0000 7f865cbc0b80 1 -- 172.21.15.5:0/430466138 >> 172.21.15.5:0/430466138 conn(0x55b55197ee20 msgr2=0x55b55198bb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.671+0000 7f865cbc0b80 1 -- 172.21.15.5:0/430466138 shutdown_connections 2023-07-21T19:27:45.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.671+0000 7f865cbc0b80 1 -- 172.21.15.5:0/430466138 wait complete. 2023-07-21T19:27:45.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.671+0000 7f865cbc0b80 1 -- start start 2023-07-21T19:27:45.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.672+0000 7f865cbc0b80 1 Processor -- start 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988350 0x55b5519f2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551999c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55199a140 con 0x55b551993b50 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55199a2b0 con 0x55b551988c60 2023-07-21T19:27:45.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b55199a470 con 0x55b551988350 2023-07-21T19:27:45.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f865cbc0b80 10 client.0 _ll_register_callbacks cb 0x7f8630007b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:45.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.673+0000 7f864359e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988350 0x55b5519f2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43874/0 (socket says 172.21.15.5:43874) 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 -- 172.21.15.5:0/2928973794 learned_addr learned my addr 172.21.15.5:0/2928973794 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f864359e700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551999c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 -- 172.21.15.5:0/2928973794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988350 msgr2=0x55b5519f2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988350 0x55b5519f2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 -- 172.21.15.5:0/2928973794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 msgr2=0x55b551999c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551999c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f863bfff700 1 -- 172.21.15.5:0/2928973794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b551708e60 con 0x55b551988c60 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.674+0000 7f864359e700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551999c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.675+0000 7f863bfff700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8630007c60 tx=0x7f8630007120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67f36dc11d892fa9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.675+0000 7f86267fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.675+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8630007300 con 0x55b551988c60 2023-07-21T19:27:45.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.675+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8630013690 con 0x55b551988c60 2023-07-21T19:27:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.675+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f863c004830 con 0x55b551988c60 2023-07-21T19:27:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.676+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8630006a80 con 0x55b551988c60 2023-07-21T19:27:45.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.676+0000 7f865cbc0b80 10 client.29222 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.676+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55b551707330 con 0x55b551988c60 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f863c004830 con 0x55b551988c60 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f863001d8b0 con 0x55b551988c60 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 10 client.29222 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 10 client.29222 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:45.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 20 client.29222 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b551a0a7c0 0x55b551a0cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 4 client.29222 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:27:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.678+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x55b5518960d0 con 0x55b551a0a7c0 2023-07-21T19:27:45.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.679+0000 7f864359e700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b551a0a7c0 0x55b551a0cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:45.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.681+0000 7f864359e700 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b551a0a7c0 0x55b551a0cbd0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6eebc3d3b55d8fa8 server_cookie=433d69775cb9acf7 in_seq=0 out_seq=0 2023-07-21T19:27:45.671 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.681+0000 7f86267fc700 10 client.29222 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:45.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.683+0000 7f86267fc700 1 -- 172.21.15.5:0/2928973794 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55b5518960d0 con 0x55b551a0a7c0 2023-07-21T19:27:45.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.683+0000 7f86267fc700 10 client.29222 handle_command_reply: tid=0 2023-07-21T19:27:45.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.683+0000 7f865cbc0b80 1 client.29222 shutdown 2023-07-21T19:27:45.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.683+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 msgr2=0x55b5519996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.683+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8630007c60 tx=0x7f8630007120 comp rx=0 tx=0).stop 2023-07-21T19:27:45.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 shutdown_connections 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b551a0a7c0 0x55b551a0cbd0 crc :-1 s=READY pgs=410 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b551993b50 0x55b551999c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b551988350 0x55b5519f2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f86267fc700 0 client.29222 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 --2- 172.21.15.5:0/2928973794 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b551988c60 0x55b5519996c0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.684+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 >> 172.21.15.5:0/2928973794 conn(0x55b55197ee20 msgr2=0x55b5519920a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.685+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 shutdown_connections 2023-07-21T19:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.685+0000 7f865cbc0b80 1 -- 172.21.15.5:0/2928973794 wait complete. 2023-07-21T19:27:45.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.685+0000 7f865cbc0b80 20 client.29222 trim_cache size 0 max 16384 2023-07-21T19:27:45.676 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:27:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "fae1d037-a31e-4d31-ba4b-04ac5477c5cf", 2023-07-21T19:27:45.676 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:27:45.676 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.686+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 msgr2=0x7f86280a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.686+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 0x7f86280a37f0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f8634000f80 tx=0x7f86340099e0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.686+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 msgr2=0x7f864406f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.686+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f864406f730 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f863400b0a0 tx=0x7f8634004db0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 shutdown_connections 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f86280a1340 0x7f86280a37f0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f864410cb50 0x7f864406a920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8644106a70 0x7f864406f730 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 --2- 172.21.15.5:0/1784382827 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f864410c240 0x7f864406fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.687+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 >> 172.21.15.5:0/1784382827 conn(0x7f86440759e0 msgr2=0x7f86440770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.688+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 shutdown_connections 2023-07-21T19:27:45.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:45.688+0000 7f86491ff700 1 -- 172.21.15.5:0/1784382827 wait complete. 2023-07-21T19:27:45.711 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:27:45.900 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:45 smithi005 ceph-mon[126460]: pgmap v635: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:45.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:45 smithi156 ceph-mon[128316]: pgmap v635: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:46.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:45 smithi101 ceph-mon[128136]: pgmap v635: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.106+0000 7f4898bff700 1 -- 172.21.15.5:0/1264640446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48940731c0 msgr2=0x7f48941105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.095 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.106+0000 7f4898bff700 1 --2- 172.21.15.5:0/1264640446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48940731c0 0x7f48941105a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f4884009990 tx=0x7f4884009ca0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.109+0000 7f4898bff700 1 -- 172.21.15.5:0/1264640446 shutdown_connections 2023-07-21T19:27:46.099 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.109+0000 7f4898bff700 1 --2- 172.21.15.5:0/1264640446 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4894110ae0 0x7f4894112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.109+0000 7f4898bff700 1 --2- 172.21.15.5:0/1264640446 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940728b0 0x7f4894072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.109+0000 7f4898bff700 1 --2- 172.21.15.5:0/1264640446 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f48940731c0 0x7f48941105a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.109+0000 7f4898bff700 1 -- 172.21.15.5:0/1264640446 >> 172.21.15.5:0/1264640446 conn(0x7f48940fb5f0 msgr2=0x7f48940fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.110+0000 7f4898bff700 1 -- 172.21.15.5:0/1264640446 shutdown_connections 2023-07-21T19:27:46.100 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.110+0000 7f4898bff700 1 -- 172.21.15.5:0/1264640446 wait complete. 2023-07-21T19:27:46.101 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.112+0000 7f4898bff700 1 Processor -- start 2023-07-21T19:27:46.102 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 -- start start 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940731c0 0x7f489419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4894110ae0 0x7f48941a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941a1190 con 0x7f48940728b0 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941a1300 con 0x7f4894110ae0 2023-07-21T19:27:46.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.113+0000 7f4898bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48941a14c0 con 0x7f48940731c0 2023-07-21T19:27:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.114+0000 7f4889d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940731c0 0x7f489419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.114+0000 7f489259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.114+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4894110ae0 0x7f48941a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.114+0000 7f489259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59564/0 (socket says 172.21.15.5:59564) 2023-07-21T19:27:46.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.114+0000 7f489259c700 1 -- 172.21.15.5:0/1156441731 learned_addr learned my addr 172.21.15.5:0/1156441731 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.115+0000 7f489259c700 1 -- 172.21.15.5:0/1156441731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940731c0 msgr2=0x7f489419c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.115+0000 7f489259c700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940731c0 0x7f489419c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.115+0000 7f489259c700 1 -- 172.21.15.5:0/1156441731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4894110ae0 msgr2=0x7f48941a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.115+0000 7f489259c700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4894110ae0 0x7f48941a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.115+0000 7f489259c700 1 -- 172.21.15.5:0/1156441731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48840095f0 con 0x7f48940728b0 2023-07-21T19:27:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.116+0000 7f489259c700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d950 tx=0x7f487c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.116+0000 7f488bfff700 1 -- 172.21.15.5:0/1156441731 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c00f890 con 0x7f48940728b0 2023-07-21T19:27:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.116+0000 7f488bfff700 1 -- 172.21.15.5:0/1156441731 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f487c00fe80 con 0x7f48940728b0 2023-07-21T19:27:46.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.116+0000 7f488bfff700 1 -- 172.21.15.5:0/1156441731 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f487c00e3b0 con 0x7f48940728b0 2023-07-21T19:27:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.117+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48941a17a0 con 0x7f48940728b0 2023-07-21T19:27:46.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.117+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48941a8fe0 con 0x7f48940728b0 2023-07-21T19:27:46.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.124+0000 7f488bfff700 1 -- 172.21.15.5:0/1156441731 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f487c010460 con 0x7f48940728b0 2023-07-21T19:27:46.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.125+0000 7f48ac436b80 1 Processor -- start 2023-07-21T19:27:46.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 -- start start 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cdbe0 0x5564003d8590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003daeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564002d3cb0 con 0x5564003d8ad0 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556400326a40 con 0x5564003cdbe0 2023-07-21T19:27:46.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.126+0000 7f48ac436b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640031fed0 con 0x5564003cd2d0 2023-07-21T19:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.127+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.127+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51676/0 (socket says 172.21.15.5:51676) 2023-07-21T19:27:46.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.127+0000 7f4892d9d700 1 -- 172.21.15.5:0/3010440600 learned_addr learned my addr 172.21.15.5:0/3010440600 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.127+0000 7f489259c700 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cdbe0 0x5564003d8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 -- 172.21.15.5:0/3010440600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cdbe0 msgr2=0x5564003d8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cdbe0 0x5564003d8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 -- 172.21.15.5:0/3010440600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 msgr2=0x5564003daeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003daeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 -- 172.21.15.5:0/3010440600 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55640014dde0 con 0x5564003cd2d0 2023-07-21T19:27:46.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f488bfff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 0x7f48800a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f488c003f40 tx=0x7f488c009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7160bb6245db4e84 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.128+0000 7f488bfff700 1 -- 172.21.15.5:0/1156441731 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f487c0cc9f0 con 0x7f48940728b0 2023-07-21T19:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.129+0000 7f4888d99700 1 -- 172.21.15.5:0/3010440600 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c019070 con 0x5564003cd2d0 2023-07-21T19:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.129+0000 7f4888d99700 1 -- 172.21.15.5:0/3010440600 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488c00f040 con 0x5564003cd2d0 2023-07-21T19:27:46.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.129+0000 7f4888d99700 1 -- 172.21.15.5:0/3010440600 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c007a50 con 0x5564003cd2d0 2023-07-21T19:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.129+0000 7f489259c700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 0x7f48800a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.130+0000 7f48ac436b80 1 -- 172.21.15.5:0/3010440600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 msgr2=0x5564003cd6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.130+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f488c003f40 tx=0x7f488c009ba0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.131+0000 7f489259c700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 0x7f48800a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f487c000c00 tx=0x7f487c010000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.134+0000 7f48ac436b80 1 -- 172.21.15.5:0/3010440600 shutdown_connections 2023-07-21T19:27:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.134+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003daeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.134+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cd2d0 0x5564003cd6a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.134+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3010440600 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cdbe0 0x5564003d8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.134+0000 7f48ac436b80 1 -- 172.21.15.5:0/3010440600 >> 172.21.15.5:0/3010440600 conn(0x5564003c3da0 msgr2=0x5564003c96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.135+0000 7f48ac436b80 1 -- 172.21.15.5:0/3010440600 shutdown_connections 2023-07-21T19:27:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.135+0000 7f48ac436b80 1 -- 172.21.15.5:0/3010440600 wait complete. 2023-07-21T19:27:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.136+0000 7f48ac436b80 1 -- start start 2023-07-21T19:27:46.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.136+0000 7f48ac436b80 1 Processor -- start 2023-07-21T19:27:46.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003d5f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564002bfac0 con 0x5564003d8ad0 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564002bf7c0 con 0x5564003cd2d0 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564001a3920 con 0x5564003cdbe0 2023-07-21T19:27:46.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.137+0000 7f48ac436b80 10 client.0 _ll_register_callbacks cb 0x5564002bd760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.138+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.138+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43898/0 (socket says 172.21.15.5:43898) 2023-07-21T19:27:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.138+0000 7f4892d9d700 1 -- 172.21.15.5:0/1051381923 learned_addr learned my addr 172.21.15.5:0/1051381923 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:46.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 -- 172.21.15.5:0/1051381923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 msgr2=0x5564003d57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 -- 172.21.15.5:0/1051381923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 msgr2=0x5564003d5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003d5f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 -- 172.21.15.5:0/1051381923 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f488c003b70 con 0x5564003cd2d0 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.139+0000 7f4892d9d700 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f488c0047c0 tx=0x7f488c0048a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc57a4314a5e9fc9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.140+0000 7f4879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.140+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c019040 con 0x5564003cd2d0 2023-07-21T19:27:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.140+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f488c007c60 con 0x5564003cd2d0 2023-07-21T19:27:46.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.141+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5564002794f0 con 0x5564003cd2d0 2023-07-21T19:27:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.142+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488c007e20 con 0x5564003cd2d0 2023-07-21T19:27:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.142+0000 7f48ac436b80 10 client.29231 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:46.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.142+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x556400136c60 con 0x5564003cd2d0 2023-07-21T19:27:46.133 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.144+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5564002794f0 con 0x5564003cd2d0 2023-07-21T19:27:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.144+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f488c020430 con 0x5564003cd2d0 2023-07-21T19:27:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 10 client.29231 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 10 client.29231 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 20 client.29231 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:46.134 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5564003f9440 0x5564003fb850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 4 client.29231 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556400318eb0 con 0x5564003f9440 2023-07-21T19:27:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.145+0000 7f4889d9b700 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5564003f9440 0x5564003fb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.146+0000 7f4889d9b700 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5564003f9440 0x5564003fb850 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=120e271a761cc1e server_cookie=390cc62a071c148 in_seq=0 out_seq=0 2023-07-21T19:27:46.135 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.146+0000 7f4879ffb700 10 client.29231 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.147+0000 7f4879ffb700 1 -- 172.21.15.5:0/1051381923 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556400318eb0 con 0x5564003f9440 2023-07-21T19:27:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.147+0000 7f4879ffb700 10 client.29231 handle_command_reply: tid=0 2023-07-21T19:27:46.136 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.147+0000 7f48ac436b80 1 client.29231 shutdown 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.148+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 msgr2=0x5564003d8560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.148+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f488c0047c0 tx=0x7f488c0048a0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 shutdown_connections 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5564003f9440 0x5564003fb850 crc :-1 s=READY pgs=411 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003d8ad0 0x5564003d5f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003d8560 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 --2- 172.21.15.5:0/1051381923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.149+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 >> 172.21.15.5:0/1051381923 conn(0x5564003c3da0 msgr2=0x5564003c9230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.150+0000 7f4879ffb700 0 client.29231 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:46.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.150+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 shutdown_connections 2023-07-21T19:27:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.150+0000 7f48ac436b80 1 -- 172.21.15.5:0/1051381923 wait complete. 2023-07-21T19:27:46.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.151+0000 7f48ac436b80 20 client.29231 trim_cache size 0 max 16384 2023-07-21T19:27:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 Processor -- start 2023-07-21T19:27:46.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 -- start start 2023-07-21T19:27:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003cd2d0 0x5564003d8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 0x5564003d54e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564003d6dc0 con 0x5564003cd2d0 2023-07-21T19:27:46.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564003d3d40 con 0x5564003d8ad0 2023-07-21T19:27:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.188+0000 7f48ac436b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564003c8490 con 0x5564003cdbe0 2023-07-21T19:27:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003cd2d0 0x5564003d8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51698/0 (socket says 172.21.15.5:51698) 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 -- 172.21.15.5:0/3978070355 learned_addr learned my addr 172.21.15.5:0/3978070355 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 0x5564003d54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 -- 172.21.15.5:0/3978070355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 msgr2=0x5564003d54e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 0x5564003d54e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 -- 172.21.15.5:0/3978070355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003cd2d0 msgr2=0x5564003d8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.189+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003cd2d0 0x5564003d8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.190+0000 7f4889d9b700 1 -- 172.21.15.5:0/3978070355 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556400174250 con 0x5564003cdbe0 2023-07-21T19:27:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.190+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 0x5564003d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:27:46.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.191+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840058e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2354cc2b1afdacf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.191+0000 7f488959a700 1 -- 172.21.15.5:0/3978070355 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488401d030 con 0x5564003cdbe0 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.191+0000 7f488959a700 1 -- 172.21.15.5:0/3978070355 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488402a070 con 0x5564003cdbe0 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.191+0000 7f488959a700 1 -- 172.21.15.5:0/3978070355 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884005690 con 0x5564003cdbe0 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 -- 172.21.15.5:0/3978070355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 msgr2=0x5564003d4fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 -- 172.21.15.5:0/3978070355 shutdown_connections 2023-07-21T19:27:46.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003d8ad0 0x5564003d54e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003cd2d0 0x5564003d8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3978070355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003cdbe0 0x5564003d4fa0 secure :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f48840098e0 tx=0x7f48840058e0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.192+0000 7f48ac436b80 1 -- 172.21.15.5:0/3978070355 >> 172.21.15.5:0/3978070355 conn(0x5564003c3da0 msgr2=0x5564003d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.193+0000 7f48ac436b80 1 -- 172.21.15.5:0/3978070355 shutdown_connections 2023-07-21T19:27:46.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.193+0000 7f48ac436b80 1 -- 172.21.15.5:0/3978070355 wait complete. 2023-07-21T19:27:46.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.193+0000 7f48ac436b80 1 -- start start 2023-07-21T19:27:46.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.194+0000 7f48ac436b80 1 Processor -- start 2023-07-21T19:27:46.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.194+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003d8ad0 0x5564003dd630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003ddb70 0x5564003ddfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564003dfad0 con 0x5564003ddb70 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640044d330 con 0x5564003cd2d0 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55640045e480 con 0x5564003d8ad0 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f48ac436b80 10 client.0 _ll_register_callbacks cb 0x7f488c003d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:46.185 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.195+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43914/0 (socket says 172.21.15.5:43914) 2023-07-21T19:27:46.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 -- 172.21.15.5:0/3706150267 learned_addr learned my addr 172.21.15.5:0/3706150267 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003d8ad0 0x5564003dd630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003ddb70 0x5564003ddfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 -- 172.21.15.5:0/3706150267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003d8ad0 msgr2=0x5564003dd630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003d8ad0 0x5564003dd630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 -- 172.21.15.5:0/3706150267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003ddb70 msgr2=0x5564003ddfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003ddb70 0x5564003ddfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.196+0000 7f4892d9d700 1 -- 172.21.15.5:0/3706150267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4894113780 con 0x5564003cd2d0 2023-07-21T19:27:46.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.197+0000 7f4892d9d700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f488c0076a0 tx=0x7f488c007630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6924bc4db0fbcaff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.197+0000 7f4879ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:27:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.197+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c019040 con 0x5564003cd2d0 2023-07-21T19:27:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.197+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f488c007a60 con 0x5564003cd2d0 2023-07-21T19:27:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.197+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f488c00f040 con 0x5564003cd2d0 2023-07-21T19:27:46.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.198+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f488c007c20 con 0x5564003cd2d0 2023-07-21T19:27:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.200+0000 7f48ac436b80 10 client.29237 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:46.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.200+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55640014c2b0 con 0x5564003cd2d0 2023-07-21T19:27:46.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f488c00f040 con 0x5564003cd2d0 2023-07-21T19:27:46.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f488c03e050 con 0x5564003cd2d0 2023-07-21T19:27:46.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 10 client.29237 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 10 client.29237 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 20 client.29237 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55640045f820 0x556400461c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 4 client.29237 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.202+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5564002db050 con 0x55640045f820 2023-07-21T19:27:46.192 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.203+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55640045f820 0x556400461c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:46.193 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.204+0000 7f4889d9b700 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55640045f820 0x556400461c00 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0e16b617b1eb14f server_cookie=7e7bde61f29c3850 in_seq=0 out_seq=0 2023-07-21T19:27:46.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.205+0000 7f4879ffb700 10 client.29237 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:46.194 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.205+0000 7f4879ffb700 1 -- 172.21.15.5:0/3706150267 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5564002db050 con 0x55640045f820 2023-07-21T19:27:46.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.205+0000 7f4879ffb700 10 client.29237 handle_command_reply: tid=0 2023-07-21T19:27:46.195 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.206+0000 7f48ac436b80 1 client.29237 shutdown 2023-07-21T19:27:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 msgr2=0x5564003df3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f488c0076a0 tx=0x7f488c007630 comp rx=0 tx=0).stop 2023-07-21T19:27:46.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 shutdown_connections 2023-07-21T19:27:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55640045f820 0x556400461c00 crc :-1 s=READY pgs=412 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5564003ddb70 0x5564003ddfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5564003cd2d0 0x5564003df3a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 --2- 172.21.15.5:0/3706150267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5564003d8ad0 0x5564003dd630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.208+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 >> 172.21.15.5:0/3706150267 conn(0x5564003c3da0 msgr2=0x5564003cf3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.210+0000 7f4879ffb700 0 client.29237 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.210+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 shutdown_connections 2023-07-21T19:27:46.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.210+0000 7f48ac436b80 1 -- 172.21.15.5:0/3706150267 wait complete. 2023-07-21T19:27:46.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.210+0000 7f48ac436b80 20 client.29237 trim_cache size 0 max 16384 2023-07-21T19:27:46.200 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:27:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:27:46.200 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "fae1d037-a31e-4d31-ba4b-04ac5477c5cf": { 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "fae1d037-a31e-4d31-ba4b-04ac5477c5cf", 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:27:46.201 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:27:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 msgr2=0x7f48800a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 0x7f48800a3940 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f487c000c00 tx=0x7f487c010000 comp rx=0 tx=0).stop 2023-07-21T19:27:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 msgr2=0x7f489419c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7f487c00d950 tx=0x7f487c00dc60 comp rx=0 tx=0).stop 2023-07-21T19:27:46.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 shutdown_connections 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4894110ae0 0x7f48941a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f48800a1490 0x7f48800a3940 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f48940728b0 0x7f489419c1b0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 --2- 172.21.15.5:0/1156441731 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f48940731c0 0x7f489419c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.211+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 >> 172.21.15.5:0/1156441731 conn(0x7f48940fb5f0 msgr2=0x7f48940fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:46.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.213+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 shutdown_connections 2023-07-21T19:27:46.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:27:46.214+0000 7f4898bff700 1 -- 172.21.15.5:0/1156441731 wait complete. 2023-07-21T19:27:46.240 INFO:tasks.cephfs.filesystem:scrub status for tag:fae1d037-a31e-4d31-ba4b-04ac5477c5cf - {'path': '/', 'tag': 'fae1d037-a31e-4d31-ba4b-04ac5477c5cf', 'options': 'recursive,force'} 2023-07-21T19:27:46.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:27:46.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:46.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:46.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:46.910 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:46 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:46.938 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 75.0 s, 485.1 tps, lat 254.887 ms stddev 496.630 2023-07-21T19:27:47.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:27:47.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:47.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:47.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:47.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:27:47.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:46 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:27:47.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:46 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:27:47.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:47 smithi156 ceph-mon[128316]: pgmap v636: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:48.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:47 smithi101 ceph-mon[128136]: pgmap v636: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:48.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:47 smithi005 ceph-mon[126460]: pgmap v636: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 13 MiB/s wr, 906 op/s 2023-07-21T19:27:49.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:48 smithi101 ceph-mon[128136]: pgmap v637: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-07-21T19:27:49.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:48 smithi005 ceph-mon[126460]: pgmap v637: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-07-21T19:27:49.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:48 smithi156 ceph-mon[128316]: pgmap v637: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 795 KiB/s rd, 20 MiB/s wr, 1.49k op/s 2023-07-21T19:27:51.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:51 smithi101 ceph-mon[128136]: pgmap v638: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 12 MiB/s wr, 893 op/s 2023-07-21T19:27:51.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:51 smithi005 ceph-mon[126460]: pgmap v638: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 12 MiB/s wr, 893 op/s 2023-07-21T19:27:51.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:51 smithi156 ceph-mon[128316]: pgmap v638: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 492 KiB/s rd, 12 MiB/s wr, 893 op/s 2023-07-21T19:27:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 80.0 s, 681.2 tps, lat 147.430 ms stddev 35.391 2023-07-21T19:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.505+0000 7f59a3bff700 1 -- 172.21.15.5:0/1077718619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c104a30 msgr2=0x7f599c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.505+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1077718619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c104a30 0x7f599c104e00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f599800b0a0 tx=0x7f599800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 -- 172.21.15.5:0/1077718619 shutdown_connections 2023-07-21T19:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1077718619 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c10b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1077718619 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c104a30 0x7f599c104e00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1077718619 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c0fea30 0x7f599c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 -- 172.21.15.5:0/1077718619 >> 172.21.15.5:0/1077718619 conn(0x7f599c074760 msgr2=0x7f599c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 -- 172.21.15.5:0/1077718619 shutdown_connections 2023-07-21T19:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.506+0000 7f59a3bff700 1 -- 172.21.15.5:0/1077718619 wait complete. 2023-07-21T19:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.508+0000 7f59a3bff700 1 Processor -- start 2023-07-21T19:27:53.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.508+0000 7f59a3bff700 1 -- start start 2023-07-21T19:27:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.508+0000 7f59a3bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.508+0000 7f59a3bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c1071d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a3bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c104a30 0x7f599c107710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a3bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c107d10 con 0x7f599c104a30 2023-07-21T19:27:53.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a3bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c107e80 con 0x7f599c0fea30 2023-07-21T19:27:53.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a3bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599c108040 con 0x7f599c0ff3e0 2023-07-21T19:27:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35660/0 (socket says 172.21.15.5:35660) 2023-07-21T19:27:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 -- 172.21.15.5:0/1646275469 learned_addr learned my addr 172.21.15.5:0/1646275469 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a119a700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c1071d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a219c700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c104a30 0x7f599c107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 -- 172.21.15.5:0/1646275469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 msgr2=0x7f599c1071d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c1071d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 -- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c104a30 msgr2=0x7f599c107710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c104a30 0x7f599c107710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a199b700 1 -- 172.21.15.5:0/1646275469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5998009d00 con 0x7f599c0fea30 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.509+0000 7f59a119a700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:27:53.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f59a199b700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f599800b830 tx=0x7f5998012750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f5992ffd700 1 -- 172.21.15.5:0/1646275469 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599800e040 con 0x7f599c0fea30 2023-07-21T19:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f599c1a8bc0 con 0x7f599c0fea30 2023-07-21T19:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f5992ffd700 1 -- 172.21.15.5:0/1646275469 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5998003e90 con 0x7f599c0fea30 2023-07-21T19:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f5992ffd700 1 -- 172.21.15.5:0/1646275469 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5998012e40 con 0x7f599c0fea30 2023-07-21T19:27:53.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.510+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599c1a9040 con 0x7f599c0fea30 2023-07-21T19:27:53.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.516+0000 7f5992ffd700 1 -- 172.21.15.5:0/1646275469 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5998004c60 con 0x7f599c0fea30 2023-07-21T19:27:53.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.518+0000 7f59b75d3b80 1 Processor -- start 2023-07-21T19:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.519+0000 7f59b75d3b80 1 -- start start 2023-07-21T19:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.519+0000 7f5992ffd700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 0x7f59880a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f5992ffd700 1 -- 172.21.15.5:0/1646275469 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f59980ce280 con 0x7f599c0fea30 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86253d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a862542d0 0x562a8624e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a86153c50 con 0x562a860aa6b0 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a861a69e0 con 0x562a8624e580 2023-07-21T19:27:53.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.520+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8619fe70 con 0x562a862542d0 2023-07-21T19:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.521+0000 7f59a219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.521+0000 7f59a219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35664/0 (socket says 172.21.15.5:35664) 2023-07-21T19:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.521+0000 7f59a219c700 1 -- 172.21.15.5:0/3147396427 learned_addr learned my addr 172.21.15.5:0/3147396427 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:53.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.521+0000 7f59a119a700 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a862542d0 0x562a8624e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 -- 172.21.15.5:0/3147396427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a862542d0 msgr2=0x562a8624e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a862542d0 0x562a8624e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 -- 172.21.15.5:0/3147396427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 msgr2=0x562a86253d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86253d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 -- 172.21.15.5:0/3147396427 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a85fcdd80 con 0x562a8624e580 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f599400cde0 tx=0x7f59940116c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ba09df6cd448365 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.522+0000 7f59a219c700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 0x7f59880a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.523+0000 7f5987fff700 1 -- 172.21.15.5:0/3147396427 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5994011e20 con 0x562a8624e580 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.523+0000 7f5987fff700 1 -- 172.21.15.5:0/3147396427 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59940128e0 con 0x562a8624e580 2023-07-21T19:27:53.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.523+0000 7f5987fff700 1 -- 172.21.15.5:0/3147396427 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f599400f7a0 con 0x562a8624e580 2023-07-21T19:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 -- 172.21.15.5:0/3147396427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 msgr2=0x562a8624e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59a219c700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 0x7f59880a3870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5994019000 tx=0x7f599400fb50 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f599400cde0 tx=0x7f59940116c0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 -- 172.21.15.5:0/3147396427 shutdown_connections 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8624e9d0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86253d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/3147396427 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a862542d0 0x562a8624e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 -- 172.21.15.5:0/3147396427 >> 172.21.15.5:0/3147396427 conn(0x562a86243be0 msgr2=0x562a86248f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 -- 172.21.15.5:0/3147396427 shutdown_connections 2023-07-21T19:27:53.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.524+0000 7f59b75d3b80 1 -- 172.21.15.5:0/3147396427 wait complete. 2023-07-21T19:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.525+0000 7f59b75d3b80 1 -- start start 2023-07-21T19:27:53.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.525+0000 7f59b75d3b80 1 Processor -- start 2023-07-21T19:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86257a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a86258490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8613fa60 con 0x562a860aa6b0 2023-07-21T19:27:53.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8613f760 con 0x562a862542d0 2023-07-21T19:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a860238c0 con 0x562a8624e580 2023-07-21T19:27:53.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59b75d3b80 10 client.0 _ll_register_callbacks cb 0x562a8613d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59a119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59a119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60022/0 (socket says 172.21.15.5:60022) 2023-07-21T19:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59a119a700 1 -- 172.21.15.5:0/129955580 learned_addr learned my addr 172.21.15.5:0/129955580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:53.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59a219c700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a86258490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.529+0000 7f59a199b700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86257a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 -- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 msgr2=0x562a86258490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a86258490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 -- 172.21.15.5:0/129955580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 msgr2=0x562a86257a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86257a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 -- 172.21.15.5:0/129955580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f599400ca10 con 0x562a8624e580 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a199b700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86257a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59a119a700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f598c00bad0 tx=0x7f598c00bde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7a6320441a2caa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59857fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c01b440 con 0x562a8624e580 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.530+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f598c01b5a0 con 0x562a8624e580 2023-07-21T19:27:53.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.531+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a860f9490 con 0x562a8624e580 2023-07-21T19:27:53.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.531+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c01b760 con 0x562a8624e580 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.531+0000 7f59b75d3b80 10 client.29173 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.531+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x562a85fb6c00 con 0x562a8624e580 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562a860f9490 con 0x562a8624e580 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f598c0117e0 con 0x562a8624e580 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 10 client.29173 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 10 client.29173 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:53.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 20 client.29173 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a86278d70 0x562a8627b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 4 client.29173 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.532+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a86198e50 con 0x562a86278d70 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.533+0000 7f59a199b700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a86278d70 0x562a8627b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.534+0000 7f59a199b700 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a86278d70 0x562a8627b180 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1195ff96d7267a57 server_cookie=b9d4efdbae4db567 in_seq=0 out_seq=0 2023-07-21T19:27:53.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.534+0000 7f59857fa700 10 client.29173 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.536+0000 7f59857fa700 1 -- 172.21.15.5:0/129955580 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562a86198e50 con 0x562a86278d70 2023-07-21T19:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.536+0000 7f59857fa700 10 client.29173 handle_command_reply: tid=0 2023-07-21T19:27:53.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.537+0000 7f59b75d3b80 1 client.29173 shutdown 2023-07-21T19:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.537+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 msgr2=0x562a86257f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.537+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f598c00bad0 tx=0x7f598c00bde0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 shutdown_connections 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a86258490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a86278d70 0x562a8627b180 crc :-1 s=READY pgs=413 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a86257a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/129955580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a86257f50 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 >> 172.21.15.5:0/129955580 conn(0x562a86243be0 msgr2=0x562a862484e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59857fa700 0 client.29173 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:53.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 shutdown_connections 2023-07-21T19:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.538+0000 7f59b75d3b80 1 -- 172.21.15.5:0/129955580 wait complete. 2023-07-21T19:27:53.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.539+0000 7f59b75d3b80 20 client.29173 trim_cache size 0 max 16384 2023-07-21T19:27:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.564+0000 7f59b75d3b80 1 Processor -- start 2023-07-21T19:27:53.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.564+0000 7f59b75d3b80 1 -- start start 2023-07-21T19:27:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 0x562a8624e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8626fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a862542d0 0x562a86252b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a862455f0 con 0x562a862542d0 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a8624efd0 con 0x562a8624e580 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a86249b20 con 0x562a860aa6b0 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59a199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 0x562a8624e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59a199b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 0x562a8624e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60026/0 (socket says 172.21.15.5:60026) 2023-07-21T19:27:53.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59a199b700 1 -- 172.21.15.5:0/303519104 learned_addr learned my addr 172.21.15.5:0/303519104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.565+0000 7f59a119a700 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8626fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a219c700 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a862542d0 0x562a86252b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 -- 172.21.15.5:0/303519104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 msgr2=0x562a8624e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 0x562a8624e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 -- 172.21.15.5:0/303519104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a862542d0 msgr2=0x562a86252b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a862542d0 0x562a86252b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 -- 172.21.15.5:0/303519104 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a85ff41f0 con 0x562a8624e580 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.566+0000 7f59a119a700 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8626fc70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f598c011bc0 tx=0x7f598c005a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4d6d0c0b2efad1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.567+0000 7f5990bff700 1 -- 172.21.15.5:0/303519104 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0234a0 con 0x562a8624e580 2023-07-21T19:27:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.567+0000 7f5990bff700 1 -- 172.21.15.5:0/303519104 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f598c023600 con 0x562a8624e580 2023-07-21T19:27:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.567+0000 7f5990bff700 1 -- 172.21.15.5:0/303519104 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c0056c0 con 0x562a8624e580 2023-07-21T19:27:53.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.568+0000 7f59b75d3b80 1 -- 172.21.15.5:0/303519104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 msgr2=0x562a8626fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.568+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8626fc70 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f598c011bc0 tx=0x7f598c005a20 comp rx=0 tx=0).stop 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 -- 172.21.15.5:0/303519104 shutdown_connections 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a862542d0 0x562a86252b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a860aa6b0 0x562a8624e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/303519104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a8624e580 0x562a8626fc70 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 -- 172.21.15.5:0/303519104 >> 172.21.15.5:0/303519104 conn(0x562a86243be0 msgr2=0x562a86251c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 -- 172.21.15.5:0/303519104 shutdown_connections 2023-07-21T19:27:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 -- 172.21.15.5:0/303519104 wait complete. 2023-07-21T19:27:53.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.569+0000 7f59b75d3b80 1 -- start start 2023-07-21T19:27:53.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.570+0000 7f59b75d3b80 1 Processor -- start 2023-07-21T19:27:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a8626dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a8626e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a8626e6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a862cbc70 con 0x562a860aa6b0 2023-07-21T19:27:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a862cbde0 con 0x562a862542d0 2023-07-21T19:27:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a862cbfa0 con 0x562a8624e580 2023-07-21T19:27:53.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59b75d3b80 10 client.0 _ll_register_callbacks cb 0x7f598c0042c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59a219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a8626e6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59a219c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a8626e6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35700/0 (socket says 172.21.15.5:35700) 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59a219c700 1 -- 172.21.15.5:0/1796661850 learned_addr learned my addr 172.21.15.5:0/1796661850 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.574+0000 7f59a119a700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a8626e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a199b700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a8626dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a119a700 1 -- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 msgr2=0x562a8626e6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a119a700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a8626e6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a119a700 1 -- 172.21.15.5:0/1796661850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 msgr2=0x562a8626dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:27:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a119a700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a8626dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.575+0000 7f59a119a700 1 -- 172.21.15.5:0/1796661850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a85fb6c00 con 0x562a8624e580 2023-07-21T19:27:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.576+0000 7f59a119a700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a8626e1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x562a86243aa0 tx=0x7f598c004180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52fe9864c59517ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:27:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.576+0000 7f59857fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:27:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.576+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f598c003d80 con 0x562a8624e580 2023-07-21T19:27:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.576+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(116..116 src has 1..116) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f598c000e40 con 0x562a8624e580 2023-07-21T19:27:53.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.576+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59940128e0 con 0x562a8624e580 2023-07-21T19:27:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.577+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f598c003ee0 con 0x562a8624e580 2023-07-21T19:27:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.577+0000 7f59b75d3b80 10 client.29179 fetch_fsmap learned FSMap version 56 2023-07-21T19:27:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.577+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x562a85fcc250 con 0x562a8624e580 2023-07-21T19:27:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.577+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f59940128e0 con 0x562a8624e580 2023-07-21T19:27:53.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f598c023cd0 con 0x562a8624e580 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 10 client.29179 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 10 client.29179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 20 client.29179 populate_metadata read hostname 'smithi005' 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a862cd250 0x562a862cf630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 4 client.29179 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:27:53.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.578+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562a8615aff0 con 0x562a862cd250 2023-07-21T19:27:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.580+0000 7f59a199b700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a862cd250 0x562a862cf630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:27:53.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.581+0000 7f59a199b700 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a862cd250 0x562a862cf630 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c3d72577b482f30 server_cookie=5c3432265cc197ce in_seq=0 out_seq=0 2023-07-21T19:27:53.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.581+0000 7f59857fa700 10 client.29179 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.615+0000 7f59857fa700 1 -- 172.21.15.5:0/1796661850 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562a8615aff0 con 0x562a862cd250 2023-07-21T19:27:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.615+0000 7f59857fa700 10 client.29179 handle_command_reply: tid=0 2023-07-21T19:27:53.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 client.29179 shutdown 2023-07-21T19:27:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 msgr2=0x562a8626e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a8626e1a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x562a86243aa0 tx=0x7f598c004180 comp rx=0 tx=0).stop 2023-07-21T19:27:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 shutdown_connections 2023-07-21T19:27:53.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562a862542d0 0x562a8626e6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562a862cd250 0x562a862cf630 crc :-1 s=READY pgs=414 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562a860aa6b0 0x562a8626dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 --2- 172.21.15.5:0/1796661850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562a8624e580 0x562a8626e1a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.616+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 >> 172.21.15.5:0/1796661850 conn(0x562a86243be0 msgr2=0x562a86252b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.617+0000 7f59857fa700 0 client.29179 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.617+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 shutdown_connections 2023-07-21T19:27:53.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.617+0000 7f59b75d3b80 1 -- 172.21.15.5:0/1796661850 wait complete. 2023-07-21T19:27:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.617+0000 7f59b75d3b80 20 client.29179 trim_cache size 0 max 16384 2023-07-21T19:27:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:27:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:27:53.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:27:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:27:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.618+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 msgr2=0x7f59880a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.618+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 0x7f59880a3870 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5994019000 tx=0x7f599400fb50 comp rx=0 tx=0).stop 2023-07-21T19:27:53.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.618+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 msgr2=0x7f599c10b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.618+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f599800b830 tx=0x7f5998012750 comp rx=0 tx=0).stop 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 shutdown_connections 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f59880a13c0 0x7f59880a3870 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f599c104a30 0x7f599c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f599c0fea30 0x7f599c10b3e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 --2- 172.21.15.5:0/1646275469 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f599c0ff3e0 0x7f599c1071d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.619+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 >> 172.21.15.5:0/1646275469 conn(0x7f599c074760 msgr2=0x7f599c0fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:27:53.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.620+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 shutdown_connections 2023-07-21T19:27:53.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:27:53.622+0000 7f59a3bff700 1 -- 172.21.15.5:0/1646275469 wait complete. 2023-07-21T19:27:53.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:53 smithi005 ceph-mon[126460]: pgmap v639: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-21T19:27:53.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:53 smithi156 ceph-mon[128316]: pgmap v639: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-21T19:27:54.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:53 smithi101 ceph-mon[128136]: pgmap v639: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 605 KiB/s rd, 20 MiB/s wr, 1.48k op/s 2023-07-21T19:27:55.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:55 smithi005 ceph-mon[126460]: pgmap v640: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:55.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:55 smithi156 ceph-mon[128316]: pgmap v640: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:56.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:55 smithi101 ceph-mon[128136]: pgmap v640: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:56.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:56.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:56.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:56 smithi005 ceph-mon[126460]: scrub summary: idle 2023-07-21T19:27:56.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:56.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:56.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:56 smithi156 ceph-mon[128316]: scrub summary: idle 2023-07-21T19:27:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 85.0 s, 650.2 tps, lat 151.996 ms stddev 51.957 2023-07-21T19:27:57.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:57.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:57.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:56 smithi101 ceph-mon[128136]: scrub summary: idle 2023-07-21T19:27:57.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:57 smithi156 ceph-mon[128316]: pgmap v641: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:58.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:57 smithi101 ceph-mon[128136]: pgmap v641: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:58.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:57 smithi005 ceph-mon[126460]: pgmap v641: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 392 KiB/s rd, 16 MiB/s wr, 1.17k op/s 2023-07-21T19:27:59.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:27:59 smithi156 ceph-mon[128316]: pgmap v642: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-07-21T19:28:00.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:27:59 smithi101 ceph-mon[128136]: pgmap v642: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-07-21T19:28:00.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:27:59 smithi005 ceph-mon[126460]: pgmap v642: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 24 MiB/s wr, 1.66k op/s 2023-07-21T19:28:01.939 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 90.0 s, 464.4 tps, lat 213.970 ms stddev 59.092 2023-07-21T19:28:02.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:01 smithi101 ceph-mon[128136]: pgmap v643: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-07-21T19:28:02.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:01 smithi005 ceph-mon[126460]: pgmap v643: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-07-21T19:28:02.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:01 smithi156 ceph-mon[128316]: pgmap v643: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 241 KiB/s rd, 17 MiB/s wr, 1.08k op/s 2023-07-21T19:28:03.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:03 smithi005 ceph-mon[126460]: pgmap v644: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-07-21T19:28:03.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:03 smithi156 ceph-mon[128316]: pgmap v644: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-07-21T19:28:04.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:03 smithi101 ceph-mon[128136]: pgmap v644: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 517 KiB/s rd, 23 MiB/s wr, 1.50k op/s 2023-07-21T19:28:05.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:05 smithi005 ceph-mon[126460]: pgmap v645: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:05.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:05 smithi156 ceph-mon[128316]: pgmap v645: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:06.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:05 smithi101 ceph-mon[128136]: pgmap v645: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 404 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:06.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 -- 172.21.15.5:0/3313454167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 msgr2=0x7ff410067900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 --2- 172.21.15.5:0/3313454167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410067900 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7ff408009f90 tx=0x7ff40800b330 comp rx=0 tx=0).stop 2023-07-21T19:28:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 -- 172.21.15.5:0/3313454167 shutdown_connections 2023-07-21T19:28:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 --2- 172.21.15.5:0/3313454167 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410067900 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 --2- 172.21.15.5:0/3313454167 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4101048a0 0x7ff410104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 --2- 172.21.15.5:0/3313454167 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4101051b0 0x7ff4100fe9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 -- 172.21.15.5:0/3313454167 >> 172.21.15.5:0/3313454167 conn(0x7ff410074770 msgr2=0x7ff410074b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:06.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.917+0000 7ff414f9d700 1 -- 172.21.15.5:0/3313454167 shutdown_connections 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.918+0000 7ff414f9d700 1 -- 172.21.15.5:0/3313454167 wait complete. 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 Processor -- start 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 -- start start 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4101048a0 0x7ff41010e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4101051b0 0x7ff41010e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41010ed10 con 0x7ff4100ff100 2023-07-21T19:28:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41010ee80 con 0x7ff4101048a0 2023-07-21T19:28:06.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.919+0000 7ff414f9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff41010f040 con 0x7ff4101051b0 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46766/0 (socket says 172.21.15.5:46766) 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2507604778 learned_addr learned my addr 172.21.15.5:0/2507604778 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2507604778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4101051b0 msgr2=0x7ff41010e7d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4101051b0 0x7ff41010e7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2507604778 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4101048a0 msgr2=0x7ff41010e290 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:28:06.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.920+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4101048a0 0x7ff41010e290 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2507604778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff408009c40 con 0x7ff4100ff100 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800e9a0 tx=0x7ff3f800ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff40cff9700 1 -- 172.21.15.5:0/2507604778 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f800cbd0 con 0x7ff4100ff100 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff41019e130 con 0x7ff4100ff100 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff40cff9700 1 -- 172.21.15.5:0/2507604778 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3f800cd30 con 0x7ff4100ff100 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff40cff9700 1 -- 172.21.15.5:0/2507604778 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8018680 con 0x7ff4100ff100 2023-07-21T19:28:06.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.921+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41019e640 con 0x7ff4100ff100 2023-07-21T19:28:06.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.927+0000 7ff40cff9700 1 -- 172.21.15.5:0/2507604778 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff3f8010430 con 0x7ff4100ff100 2023-07-21T19:28:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.927+0000 7ff429b49b80 1 Processor -- start 2023-07-21T19:28:06.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 -- start start 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeaf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfe5700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45deeac50 con 0x55c45dfeb8c0 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45df3d9e0 con 0x55c45de416b0 2023-07-21T19:28:06.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.928+0000 7ff429b49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45df36e70 con 0x55c45dfeb470 2023-07-21T19:28:06.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35408/0 (socket says 172.21.15.5:35408) 2023-07-21T19:28:06.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2541680715 learned_addr learned my addr 172.21.15.5:0/2541680715 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40ffff700 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeaf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2541680715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 msgr2=0x55c45dfeaf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeaf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2541680715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 msgr2=0x55c45dfe5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfe5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.929+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2541680715 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45dd64d80 con 0x55c45dfeb470 2023-07-21T19:28:06.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.930+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ca770 tx=0x7ff3f80d53c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bacda53abb01c0dc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.931+0000 7ff405ffb700 1 -- 172.21.15.5:0/2541680715 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80d5d30 con 0x55c45dfeb470 2023-07-21T19:28:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.931+0000 7ff405ffb700 1 -- 172.21.15.5:0/2541680715 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3f80d5e90 con 0x55c45dfeb470 2023-07-21T19:28:06.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.931+0000 7ff405ffb700 1 -- 172.21.15.5:0/2541680715 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80da600 con 0x55c45dfeb470 2023-07-21T19:28:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.932+0000 7ff40cff9700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 0x7ff3fc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.932+0000 7ff429b49b80 1 -- 172.21.15.5:0/2541680715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 msgr2=0x55c45dfe51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.932+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7ff3f80ca770 tx=0x7ff3f80d53c0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.932+0000 7ff40cff9700 1 -- 172.21.15.5:0/2507604778 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff3f8014070 con 0x7ff4100ff100 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff40ffff700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 0x7ff3fc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff429b49b80 1 -- 172.21.15.5:0/2541680715 shutdown_connections 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfe5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeaf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2541680715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfe51a0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.933+0000 7ff429b49b80 1 -- 172.21.15.5:0/2541680715 >> 172.21.15.5:0/2541680715 conn(0x55c45dfdad40 msgr2=0x55c45dfe0340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.934+0000 7ff40ffff700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 0x7ff3fc0a37f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff40800b6c0 tx=0x7ff408007bf0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.934+0000 7ff429b49b80 1 -- 172.21.15.5:0/2541680715 shutdown_connections 2023-07-21T19:28:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.934+0000 7ff429b49b80 1 -- 172.21.15.5:0/2541680715 wait complete. 2023-07-21T19:28:06.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.935+0000 7ff429b49b80 1 -- start start 2023-07-21T19:28:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.935+0000 7ff429b49b80 1 Processor -- start 2023-07-21T19:28:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeeac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfef000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfef540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45ded6a60 con 0x55c45dfeb8c0 2023-07-21T19:28:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45ded6760 con 0x55c45de416b0 2023-07-21T19:28:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45ddba8c0 con 0x55c45dfeb470 2023-07-21T19:28:06.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff429b49b80 10 client.0 _ll_register_callbacks cb 0x55c45ded4700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.936+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeeac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60956/0 (socket says 172.21.15.5:60956) 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40f7fe700 1 -- 172.21.15.5:0/2711882892 learned_addr learned my addr 172.21.15.5:0/2711882892 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40ffff700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfef540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfef000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 -- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 msgr2=0x55c45dfeeac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeeac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 -- 172.21.15.5:0/2711882892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 msgr2=0x55c45dfef540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfef540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.937+0000 7ff40effd700 1 -- 172.21.15.5:0/2711882892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff41019e640 con 0x55c45dfeb470 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff40effd700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfef000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55c45dfebcd0 tx=0x7ff40000e6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf614c831c4a3a03 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff3eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:28:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff400010630 con 0x55c45dfeb470 2023-07-21T19:28:06.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff400014070 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c45de90490 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff40000ee80 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.938+0000 7ff429b49b80 10 client.29188 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c45dd4dc00 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff400024080 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff400020890 con 0x55c45dfeb470 2023-07-21T19:28:06.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 10 client.29188 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 10 client.29188 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 20 client.29188 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45dffedc0 0x55c45e0011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 4 client.29188 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.939+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c45df2fe50 con 0x55c45dffedc0 2023-07-21T19:28:06.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.940+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45dffedc0 0x55c45e0011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.941+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45dffedc0 0x55c45e0011d0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbc1bc46695b4da9 server_cookie=8a90af11176aa797 in_seq=0 out_seq=0 2023-07-21T19:28:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.941+0000 7ff3eeffd700 10 client.29188 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:06.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.941+0000 7ff3eeffd700 1 -- 172.21.15.5:0/2711882892 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c45df2fe50 con 0x55c45dffedc0 2023-07-21T19:28:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.941+0000 7ff3eeffd700 10 client.29188 handle_command_reply: tid=0 2023-07-21T19:28:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 client.29188 shutdown 2023-07-21T19:28:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 msgr2=0x55c45dfef000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfef000 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x55c45dfebcd0 tx=0x7ff40000e6a0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 shutdown_connections 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45dffedc0 0x55c45e0011d0 crc :-1 s=READY pgs=415 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfef540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45de416b0 0x55c45dfeeac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 --2- 172.21.15.5:0/2711882892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dfef000 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 >> 172.21.15.5:0/2711882892 conn(0x55c45dfdad40 msgr2=0x55c45dfdf640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.942+0000 7ff3eeffd700 0 client.29188 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.943+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 shutdown_connections 2023-07-21T19:28:06.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.943+0000 7ff429b49b80 1 -- 172.21.15.5:0/2711882892 wait complete. 2023-07-21T19:28:06.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.943+0000 7ff429b49b80 20 client.29188 trim_cache size 0 max 16384 2023-07-21T19:28:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 95.0 s, 298.6 tps, lat 319.588 ms stddev 70.930 2023-07-21T19:28:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.961+0000 7ff429b49b80 1 Processor -- start 2023-07-21T19:28:06.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.961+0000 7ff429b49b80 1 -- start start 2023-07-21T19:28:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.961+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.961+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb470 0x55c45dfef370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfec410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff429b49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45dfd7d80 con 0x55c45dfeb8c0 2023-07-21T19:28:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff429b49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45dfe5a90 con 0x55c45dfeb470 2023-07-21T19:28:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff429b49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45dfe1170 con 0x55c45de416b0 2023-07-21T19:28:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfec410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35430/0 (socket says 172.21.15.5:35430) 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 -- 172.21.15.5:0/1053619022 learned_addr learned my addr 172.21.15.5:0/1053619022 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40effd700 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb470 0x55c45dfef370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 -- 172.21.15.5:0/1053619022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb470 msgr2=0x55c45dfef370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb470 0x55c45dfef370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 -- 172.21.15.5:0/1053619022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 msgr2=0x55c45dfec410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfec410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.962+0000 7ff40f7fe700 1 -- 172.21.15.5:0/1053619022 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c45dfe56b0 con 0x55c45de416b0 2023-07-21T19:28:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.963+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff3f809a140 tx=0x7ff3f80dbe30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=628dfa61a42bf205 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.963+0000 7ff4067fc700 1 -- 172.21.15.5:0/1053619022 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f80d0910 con 0x55c45de416b0 2023-07-21T19:28:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.963+0000 7ff4067fc700 1 -- 172.21.15.5:0/1053619022 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3f80964f0 con 0x55c45de416b0 2023-07-21T19:28:06.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.963+0000 7ff4067fc700 1 -- 172.21.15.5:0/1053619022 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f809a240 con 0x55c45de416b0 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- 172.21.15.5:0/1053619022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 msgr2=0x55c45dff5de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7ff3f809a140 tx=0x7ff3f80dbe30 comp rx=0 tx=0).stop 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- 172.21.15.5:0/1053619022 shutdown_connections 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45dfeb8c0 0x55c45dfec410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45de416b0 0x55c45dff5de0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 --2- 172.21.15.5:0/1053619022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb470 0x55c45dfef370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- 172.21.15.5:0/1053619022 >> 172.21.15.5:0/1053619022 conn(0x55c45dfdad40 msgr2=0x55c45dfebd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- 172.21.15.5:0/1053619022 shutdown_connections 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- 172.21.15.5:0/1053619022 wait complete. 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 -- start start 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.964+0000 7ff429b49b80 1 Processor -- start 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dff22a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb8c0 0x55c45e051c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45dff2a70 con 0x55c45de416b0 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45dff2be0 con 0x55c45dfeb8c0 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c45e0521b0 con 0x55c45dfeb470 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff429b49b80 10 client.0 _ll_register_callbacks cb 0x7ff400009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff40f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46806/0 (socket says 172.21.15.5:46806) 2023-07-21T19:28:06.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.965+0000 7ff40f7fe700 1 -- 172.21.15.5:0/644005760 learned_addr learned my addr 172.21.15.5:0/644005760 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40effd700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dff22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 -- 172.21.15.5:0/644005760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 msgr2=0x55c45dff22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dff22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 -- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb8c0 msgr2=0x55c45e051c70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb8c0 0x55c45e051c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 -- 172.21.15.5:0/644005760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c45dd64d80 con 0x55c45de416b0 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff40f7fe700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009230 tx=0x7ff3f80d5840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd07b12053e0481b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:06.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff3eeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.966+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3f8099b10 con 0x55c45de416b0 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff3f80d5e90 con 0x55c45de416b0 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(117..117 src has 1..117) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff3f809e1e0 con 0x55c45de416b0 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3f8099f80 con 0x55c45de416b0 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff429b49b80 10 client.19587 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:06.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55c45dd8b1f0 con 0x55c45de416b0 2023-07-21T19:28:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff3f80d5e90 con 0x55c45de416b0 2023-07-21T19:28:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.967+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ff3f80968c0 con 0x55c45de416b0 2023-07-21T19:28:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 10 client.19587 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:06.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 10 client.19587 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 20 client.19587 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45e0533f0 0x55c45e0557d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 4 client.19587 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c45def1ff0 con 0x55c45e0533f0 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.968+0000 7ff40effd700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45e0533f0 0x55c45e0557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.969+0000 7ff40effd700 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45e0533f0 0x55c45e0557d0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2643b54535555f7e server_cookie=eaa107a139775a58 in_seq=0 out_seq=0 2023-07-21T19:28:06.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:06.969+0000 7ff3eeffd700 10 client.19587 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:07.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.193+0000 7ff3eeffd700 1 -- 172.21.15.5:0/644005760 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c45def1ff0 con 0x55c45e0533f0 2023-07-21T19:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.193+0000 7ff3eeffd700 10 client.19587 handle_command_reply: tid=0 2023-07-21T19:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.193+0000 7ff429b49b80 1 client.19587 shutdown 2023-07-21T19:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 msgr2=0x55c45dff1d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:07.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009230 tx=0x7ff3f80d5840 comp rx=0 tx=0).stop 2023-07-21T19:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 shutdown_connections 2023-07-21T19:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c45dfeb8c0 0x55c45e051c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c45e0533f0 0x55c45e0557d0 crc :-1 s=READY pgs=416 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c45de416b0 0x55c45dff1d60 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 --2- 172.21.15.5:0/644005760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c45dfeb470 0x55c45dff22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 >> 172.21.15.5:0/644005760 conn(0x55c45dfdad40 msgr2=0x55c45dfec590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.194+0000 7ff3eeffd700 0 client.19587 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.195+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 shutdown_connections 2023-07-21T19:28:07.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.195+0000 7ff429b49b80 1 -- 172.21.15.5:0/644005760 wait complete. 2023-07-21T19:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.196+0000 7ff429b49b80 20 client.19587 trim_cache size 0 max 16384 2023-07-21T19:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:28:07.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:28:07.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.198+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 msgr2=0x7ff3fc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.198+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 0x7ff3fc0a37f0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff40800b6c0 tx=0x7ff408007bf0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.198+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 msgr2=0x7ff410111280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:07.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.198+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800e9a0 tx=0x7ff3f800ed60 comp rx=0 tx=0).stop 2023-07-21T19:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 shutdown_connections 2023-07-21T19:28:07.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4101051b0 0x7ff41010e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff3fc0a1340 0x7ff3fc0a37f0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff4100ff100 0x7ff410111280 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 --2- 172.21.15.5:0/2507604778 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4101048a0 0x7ff41010e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 >> 172.21.15.5:0/2507604778 conn(0x7ff410074770 msgr2=0x7ff4101023d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:07.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.205+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 shutdown_connections 2023-07-21T19:28:07.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:07.206+0000 7ff414f9d700 1 -- 172.21.15.5:0/2507604778 wait complete. 2023-07-21T19:28:08.039 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:07 smithi101 ceph-mon[128136]: pgmap v646: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:08.039 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:07 smithi101 ceph-mon[128136]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:28:08.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:07 smithi005 ceph-mon[126460]: pgmap v646: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:08.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:07 smithi005 ceph-mon[126460]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:28:08.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:07 smithi156 ceph-mon[128316]: pgmap v646: 97 pgs: 97 active+clean; 16 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 403 KiB/s rd, 14 MiB/s wr, 915 op/s 2023-07-21T19:28:08.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:07 smithi156 ceph-mon[128316]: osdmap e117: 12 total, 12 up, 12 in 2023-07-21T19:28:09.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:08 smithi101 ceph-mon[128136]: pgmap v648: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:09.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:08 smithi005 ceph-mon[126460]: pgmap v648: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:09.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:08 smithi156 ceph-mon[128316]: pgmap v648: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:11.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:11 smithi101 ceph-mon[128136]: pgmap v649: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:11.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:11 smithi005 ceph-mon[126460]: pgmap v649: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:11.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:11 smithi156 ceph-mon[128316]: pgmap v649: 97 pgs: 38 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 38 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 916 op/s 2023-07-21T19:28:11.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 100.0 s, 355.4 tps, lat 293.306 ms stddev 156.642 2023-07-21T19:28:13.894 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:13 smithi101 ceph-mon[128136]: pgmap v650: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 860 op/s 2023-07-21T19:28:13.895 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:13 smithi101 ceph-mon[128136]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:28:13.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:13 smithi005 ceph-mon[126460]: pgmap v650: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 860 op/s 2023-07-21T19:28:13.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:13 smithi005 ceph-mon[126460]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:28:13.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:13 smithi156 ceph-mon[128316]: pgmap v650: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 595 KiB/s rd, 12 MiB/s wr, 860 op/s 2023-07-21T19:28:13.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:13 smithi156 ceph-mon[128316]: osdmap e118: 12 total, 12 up, 12 in 2023-07-21T19:28:15.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:15 smithi005 ceph-mon[126460]: pgmap v652: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-21T19:28:15.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:15 smithi156 ceph-mon[128316]: pgmap v652: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-21T19:28:16.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:15 smithi101 ceph-mon[128136]: pgmap v652: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 744 KiB/s rd, 15 MiB/s wr, 1.07k op/s 2023-07-21T19:28:16.241 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:28:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.620+0000 7fb7697ff700 1 -- 172.21.15.5:0/991656240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 msgr2=0x7fb764117300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 --2- 172.21.15.5:0/991656240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb764117300 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7fb75c009990 tx=0x7fb75c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 -- 172.21.15.5:0/991656240 shutdown_connections 2023-07-21T19:28:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 --2- 172.21.15.5:0/991656240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb764117300 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 --2- 172.21.15.5:0/991656240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb76410f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 --2- 172.21.15.5:0/991656240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 0x7fb76410a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 -- 172.21.15.5:0/991656240 >> 172.21.15.5:0/991656240 conn(0x7fb76406a950 msgr2=0x7fb76406ad50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.621+0000 7fb7697ff700 1 -- 172.21.15.5:0/991656240 shutdown_connections 2023-07-21T19:28:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.622+0000 7fb7697ff700 1 -- 172.21.15.5:0/991656240 wait complete. 2023-07-21T19:28:16.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 Processor -- start 2023-07-21T19:28:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 -- start start 2023-07-21T19:28:16.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb7641a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 0x7fb7641a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641a5550 con 0x7fb76410fd20 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641a56c0 con 0x7fb76410aca0 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.624+0000 7fb7697ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7641a5880 con 0x7fb76410f410 2023-07-21T19:28:16.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.625+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.625+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38916/0 (socket says 172.21.15.5:38916) 2023-07-21T19:28:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.625+0000 7fb75a7fc700 1 -- 172.21.15.5:0/4195943790 learned_addr learned my addr 172.21.15.5:0/4195943790 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:16.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.625+0000 7fb7637fe700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 0x7fb7641a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.625+0000 7fb762ffd700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb7641a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 -- 172.21.15.5:0/4195943790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 msgr2=0x7fb7641a0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb7641a0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 -- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 msgr2=0x7fb7641a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 0x7fb7641a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 -- 172.21.15.5:0/4195943790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c0095f0 con 0x7fb76410f410 2023-07-21T19:28:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.626+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d980 tx=0x7fb74c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.627+0000 7fb760ff9700 1 -- 172.21.15.5:0/4195943790 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00f890 con 0x7fb76410f410 2023-07-21T19:28:16.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.627+0000 7fb760ff9700 1 -- 172.21.15.5:0/4195943790 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb74c00fe80 con 0x7fb76410f410 2023-07-21T19:28:16.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.627+0000 7fb760ff9700 1 -- 172.21.15.5:0/4195943790 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00e3b0 con 0x7fb76410f410 2023-07-21T19:28:16.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.627+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7641a5b60 con 0x7fb76410f410 2023-07-21T19:28:16.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.627+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7641ad3a0 con 0x7fb76410f410 2023-07-21T19:28:16.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.633+0000 7fb760ff9700 1 -- 172.21.15.5:0/4195943790 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb74c00f9f0 con 0x7fb76410f410 2023-07-21T19:28:16.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.635+0000 7fb77d060b80 1 Processor -- start 2023-07-21T19:28:16.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 -- start start 2023-07-21T19:28:16.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x56330502d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x56330503aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304f33cb0 con 0x56330502d2d0 2023-07-21T19:28:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304f86a40 con 0x56330502dbe0 2023-07-21T19:28:16.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb77d060b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304f7fed0 con 0x563305038ad0 2023-07-21T19:28:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60018/0 (socket says 172.21.15.5:60018) 2023-07-21T19:28:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.637+0000 7fb762ffd700 1 -- 172.21.15.5:0/53763340 learned_addr learned my addr 172.21.15.5:0/53763340 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.638+0000 7fb7637fe700 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x56330502d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.638+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x56330503aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.638+0000 7fb760ff9700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 0x7fb7500a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb760ff9700 1 -- 172.21.15.5:0/4195943790 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb74c0cbfa0 con 0x7fb76410f410 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 -- 172.21.15.5:0/53763340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 msgr2=0x56330503aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x56330503aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 -- 172.21.15.5:0/53763340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 msgr2=0x56330502d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x56330502d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 -- 172.21.15.5:0/53763340 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563304dadde0 con 0x56330502dbe0 2023-07-21T19:28:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb7637fe700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 0x7fb7500a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.639+0000 7fb762ffd700 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb754009990 tx=0x7fb754009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5e9b4da8d1dd4a1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.640+0000 7fb7597fa700 1 -- 172.21.15.5:0/53763340 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754004780 con 0x56330502dbe0 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.640+0000 7fb7597fa700 1 -- 172.21.15.5:0/53763340 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75400f040 con 0x56330502dbe0 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.640+0000 7fb7597fa700 1 -- 172.21.15.5:0/53763340 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7540156e0 con 0x56330502dbe0 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 -- 172.21.15.5:0/53763340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 msgr2=0x563305038590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb7637fe700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 0x7fb7500a3980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00a010 tx=0x7fb75c009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fb754009990 tx=0x7fb754009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 -- 172.21.15.5:0/53763340 shutdown_connections 2023-07-21T19:28:16.631 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x56330503aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x56330502d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 --2- 172.21.15.5:0/53763340 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038590 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 -- 172.21.15.5:0/53763340 >> 172.21.15.5:0/53763340 conn(0x563305023da0 msgr2=0x5633050296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 -- 172.21.15.5:0/53763340 shutdown_connections 2023-07-21T19:28:16.632 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.641+0000 7fb77d060b80 1 -- 172.21.15.5:0/53763340 wait complete. 2023-07-21T19:28:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.642+0000 7fb77d060b80 1 -- start start 2023-07-21T19:28:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.642+0000 7fb77d060b80 1 Processor -- start 2023-07-21T19:28:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502d2d0 0x5633050384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502dbe0 0x563305035700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.633 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304f1fac0 con 0x563305038ad0 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304f1f7c0 con 0x56330502d2d0 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563304e03920 con 0x56330502dbe0 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.643+0000 7fb77d060b80 10 client.0 _ll_register_callbacks cb 0x563304f1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36086/0 (socket says 172.21.15.5:36086) 2023-07-21T19:28:16.634 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 -- 172.21.15.5:0/2286972395 learned_addr learned my addr 172.21.15.5:0/2286972395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 -- 172.21.15.5:0/2286972395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502dbe0 msgr2=0x563305035700 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:28:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502dbe0 0x563305035700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 -- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502d2d0 msgr2=0x5633050384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502d2d0 0x5633050384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.635 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 -- 172.21.15.5:0/2286972395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb754003cf0 con 0x563305038ad0 2023-07-21T19:28:16.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.644+0000 7fb7637fe700 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb75c015000 tx=0x7fb75c009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=805f1952457496ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.645+0000 7fb74a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.645+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb75c01d040 con 0x563305038ad0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.645+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb75c02c040 con 0x563305038ad0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.645+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563304ed94f0 con 0x563305038ad0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.646+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb75c0047d0 con 0x563305038ad0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.646+0000 7fb77d060b80 10 client.19599 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.646+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563304d96c60 con 0x563305038ad0 2023-07-21T19:28:16.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.646+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563304ed94f0 con 0x563305038ad0 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb75c003d10 con 0x563305038ad0 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 10 client.19599 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 10 client.19599 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 20 client.19599 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563305050a60 0x563305052e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 4 client.19599 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563304f78eb0 con 0x563305050a60 2023-07-21T19:28:16.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.647+0000 7fb762ffd700 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563305050a60 0x563305052e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.648+0000 7fb762ffd700 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563305050a60 0x563305052e70 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72b5cd2bb08c4efb server_cookie=2cf24eee7c6f8dd8 in_seq=0 out_seq=0 2023-07-21T19:28:16.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.648+0000 7fb74a7fc700 10 client.19599 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.651+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2286972395 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563304f78eb0 con 0x563305050a60 2023-07-21T19:28:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.651+0000 7fb74a7fc700 10 client.19599 handle_command_reply: tid=0 2023-07-21T19:28:16.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.651+0000 7fb77d060b80 1 client.19599 shutdown 2023-07-21T19:28:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.651+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 msgr2=0x563305035d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.651+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fb75c015000 tx=0x7fb75c009ee0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 shutdown_connections 2023-07-21T19:28:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563305050a60 0x563305052e70 crc :-1 s=READY pgs=417 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x563305035d30 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502d2d0 0x5633050384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2286972395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502dbe0 0x563305035700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 >> 172.21.15.5:0/2286972395 conn(0x563305023da0 msgr2=0x563305029230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb74a7fc700 0 client.19599 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:16.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 shutdown_connections 2023-07-21T19:28:16.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.652+0000 7fb77d060b80 1 -- 172.21.15.5:0/2286972395 wait complete. 2023-07-21T19:28:16.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.653+0000 7fb77d060b80 20 client.19599 trim_cache size 0 max 16384 2023-07-21T19:28:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.683+0000 7fb77d060b80 1 Processor -- start 2023-07-21T19:28:16.672 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.683+0000 7fb77d060b80 1 -- start start 2023-07-21T19:28:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 0x563305034ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.673 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x5633050356c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330502e5b0 con 0x563305038ad0 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56330504cb50 con 0x56330502dbe0 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb77d060b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633050301b0 con 0x56330502d2d0 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb7637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x5633050356c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 0x563305034ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb7637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x5633050356c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36088/0 (socket says 172.21.15.5:36088) 2023-07-21T19:28:16.674 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb7637fe700 1 -- 172.21.15.5:0/184435658 learned_addr learned my addr 172.21.15.5:0/184435658 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:16.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.684+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 -- 172.21.15.5:0/184435658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 msgr2=0x563305038020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.675 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 -- 172.21.15.5:0/184435658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 msgr2=0x5633050356c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x5633050356c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 -- 172.21.15.5:0/184435658 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563304dd4250 con 0x56330502d2d0 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb762ffd700 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 0x563305034ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x563305053790 tx=0x7fb754007e10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=def36161fd32c182 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb759ffb700 1 -- 172.21.15.5:0/184435658 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754006000 con 0x56330502d2d0 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb759ffb700 1 -- 172.21.15.5:0/184435658 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb7540058f0 con 0x56330502d2d0 2023-07-21T19:28:16.676 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.685+0000 7fb759ffb700 1 -- 172.21.15.5:0/184435658 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb754015b30 con 0x56330502d2d0 2023-07-21T19:28:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.686+0000 7fb77d060b80 1 -- 172.21.15.5:0/184435658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 msgr2=0x563305034ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.686+0000 7fb77d060b80 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 0x563305034ee0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x563305053790 tx=0x7fb754007e10 comp rx=0 tx=0).stop 2023-07-21T19:28:16.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 -- 172.21.15.5:0/184435658 shutdown_connections 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563305038ad0 0x5633050356c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56330502d2d0 0x563305034ee0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 --2- 172.21.15.5:0/184435658 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305038020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 -- 172.21.15.5:0/184435658 >> 172.21.15.5:0/184435658 conn(0x563305023da0 msgr2=0x563305030a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 -- 172.21.15.5:0/184435658 shutdown_connections 2023-07-21T19:28:16.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.688+0000 7fb77d060b80 1 -- 172.21.15.5:0/184435658 wait complete. 2023-07-21T19:28:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.689+0000 7fb77d060b80 1 -- start start 2023-07-21T19:28:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.689+0000 7fb77d060b80 1 Processor -- start 2023-07-21T19:28:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x563305046320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x563305046da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633050a4970 con 0x56330502d2d0 2023-07-21T19:28:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633050a4ab0 con 0x56330502dbe0 2023-07-21T19:28:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5633050a4bf0 con 0x563305038ad0 2023-07-21T19:28:16.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb77d060b80 10 client.0 _ll_register_callbacks cb 0x7fb75c005750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb762ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x563305046320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb75a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60058/0 (socket says 172.21.15.5:60058) 2023-07-21T19:28:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb75a7fc700 1 -- 172.21.15.5:0/2644174575 learned_addr learned my addr 172.21.15.5:0/2644174575 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:16.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.690+0000 7fb7637fe700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x563305046da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 -- 172.21.15.5:0/2644174575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 msgr2=0x563305046da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x563305046da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 -- 172.21.15.5:0/2644174575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 msgr2=0x563305046320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x563305046320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 -- 172.21.15.5:0/2644174575 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563304dadde0 con 0x56330502dbe0 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.691+0000 7fb75a7fc700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0a0f30 tx=0x7fb74c095410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d6222fc8e459173d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb74a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:16.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c099470 con 0x56330502dbe0 2023-07-21T19:28:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb75400f040 con 0x56330502dbe0 2023-07-21T19:28:16.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb74c0106b0 con 0x56330502dbe0 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb74c098970 con 0x56330502dbe0 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb77d060b80 10 client.29279 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.692+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563304dac2b0 con 0x56330502dbe0 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb75400f040 con 0x56330502dbe0 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fb74c098500 con 0x56330502dbe0 2023-07-21T19:28:16.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 10 client.29279 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 10 client.29279 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 20 client.29279 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5633050a6250 0x5633050a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 4 client.29279 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563304f3b050 con 0x5633050a6250 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.693+0000 7fb762ffd700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5633050a6250 0x5633050a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:16.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.694+0000 7fb762ffd700 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5633050a6250 0x5633050a8660 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec50fc3b407ede97 server_cookie=c5988d4fae6be2a7 in_seq=0 out_seq=0 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.694+0000 7fb74a7fc700 10 client.29279 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.694+0000 7fb74a7fc700 1 -- 172.21.15.5:0/2644174575 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x563304f3b050 con 0x5633050a6250 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.694+0000 7fb74a7fc700 10 client.29279 handle_command_reply: tid=0 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 client.29279 shutdown 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 msgr2=0x563305046860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fb74c0a0f30 tx=0x7fb74c095410 comp rx=0 tx=0).stop 2023-07-21T19:28:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 shutdown_connections 2023-07-21T19:28:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563305038ad0 0x563305046da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5633050a6250 0x5633050a8660 crc :-1 s=READY pgs=418 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56330502d2d0 0x563305046320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 --2- 172.21.15.5:0/2644174575 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56330502dbe0 0x563305046860 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 >> 172.21.15.5:0/2644174575 conn(0x563305023da0 msgr2=0x5633050355d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb74a7fc700 0 client.29279 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 shutdown_connections 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.695+0000 7fb77d060b80 1 -- 172.21.15.5:0/2644174575 wait complete. 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:28:16.688 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "no active scrubs running", 2023-07-21T19:28:16.689 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": {} 2023-07-21T19:28:16.689 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:28:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.696+0000 7fb77d060b80 20 client.29279 trim_cache size 0 max 16384 2023-07-21T19:28:16.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 msgr2=0x7fb7500a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 0x7fb7500a3980 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb75c00a010 tx=0x7fb75c009310 comp rx=0 tx=0).stop 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 msgr2=0x7fb7641a0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb74c00d980 tx=0x7fb74c00dc90 comp rx=0 tx=0).stop 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 shutdown_connections 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb7500a14d0 0x7fb7500a3980 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb76410fd20 0x7fb7641a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb76410aca0 0x7fb7641a0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 --2- 172.21.15.5:0/4195943790 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb76410f410 0x7fb7641a0ab0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 >> 172.21.15.5:0/4195943790 conn(0x7fb76406a950 msgr2=0x7fb7641083a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 shutdown_connections 2023-07-21T19:28:16.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:16.698+0000 7fb7697ff700 1 -- 172.21.15.5:0/4195943790 wait complete. 2023-07-21T19:28:16.714 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-07-21T19:28:16.714 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-07-21T19:28:16.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 105.0 s, 168.6 tps, lat 598.784 ms stddev 787.977 2023-07-21T19:28:17.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 -- 172.21.15.5:0/1395711150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106cf0 msgr2=0x7f122c112f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 --2- 172.21.15.5:0/1395711150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106cf0 0x7f122c112f40 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f1224009fc0 tx=0x7f122400b330 comp rx=0 tx=0).stop 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 -- 172.21.15.5:0/1395711150 shutdown_connections 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 --2- 172.21.15.5:0/1395711150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106cf0 0x7f122c112f40 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 --2- 172.21.15.5:0/1395711150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c10c1c0 0x7f122c10c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 --2- 172.21.15.5:0/1395711150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c106340 0x7f122c1067b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.047+0000 7f1233bff700 1 -- 172.21.15.5:0/1395711150 >> 172.21.15.5:0/1395711150 conn(0x7f122c0759e0 msgr2=0x7f122c077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.048+0000 7f1233bff700 1 -- 172.21.15.5:0/1395711150 shutdown_connections 2023-07-21T19:28:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.048+0000 7f1233bff700 1 -- 172.21.15.5:0/1395711150 wait complete. 2023-07-21T19:28:17.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.049+0000 7f1233bff700 1 Processor -- start 2023-07-21T19:28:17.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.049+0000 7f1233bff700 1 -- start start 2023-07-21T19:28:17.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 0x7f122c19c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:17.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:17.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c10c1c0 0x7f122c1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:17.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a1200 con 0x7f122c10c1c0 2023-07-21T19:28:17.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a1370 con 0x7f122c106340 2023-07-21T19:28:17.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f1233bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f122c1a1530 con 0x7f122c106cf0 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f123119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f123119a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38972/0 (socket says 172.21.15.5:38972) 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.050+0000 7f123119a700 1 -- 172.21.15.5:0/1879674849 learned_addr learned my addr 172.21.15.5:0/1879674849 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123199b700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 0x7f122c19c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123219c700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c10c1c0 0x7f122c1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:17.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 -- 172.21.15.5:0/1879674849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 msgr2=0x7f122c19c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 0x7f122c19c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 -- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c10c1c0 msgr2=0x7f122c1a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c10c1c0 0x7f122c1a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 -- 172.21.15.5:0/1879674849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1224009c40 con 0x7f122c106cf0 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123199b700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 0x7f122c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.051+0000 7f123119a700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9c0 tx=0x7f121c00ed80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:17.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.052+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c00cbd0 con 0x7f122c106cf0 2023-07-21T19:28:17.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.052+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f121c00cd30 con 0x7f122c106cf0 2023-07-21T19:28:17.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.052+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f121c005520 con 0x7f122c106cf0 2023-07-21T19:28:17.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.052+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f122c1a1810 con 0x7f122c106cf0 2023-07-21T19:28:17.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.052+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f122c1a9050 con 0x7f122c106cf0 2023-07-21T19:28:17.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.054+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f122c067750 con 0x7f122c106cf0 2023-07-21T19:28:17.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.057+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f121c004750 con 0x7f122c106cf0 2023-07-21T19:28:17.048 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.060+0000 7f1222ffd700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 0x7f12180a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.060+0000 7f123199b700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 0x7f12180a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:17.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.060+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f121c014070 con 0x7f122c106cf0 2023-07-21T19:28:17.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.061+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f121c09a570 con 0x7f122c106cf0 2023-07-21T19:28:17.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.062+0000 7f123199b700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 0x7f12180a3870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f122800bb90 tx=0x7f1228005d80 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:17.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.538+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f122c19d570 con 0x7f122c106cf0 2023-07-21T19:28:17.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.539+0000 7f1222ffd700 1 -- 172.21.15.5:0/1879674849 <== mon.1 v2:172.21.15.156:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+10839 (secure 0 0 0) 0x7f121c09a100 con 0x7f122c106cf0 2023-07-21T19:28:17.530 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:28:17.531 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":56,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14604,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6835/3927516205","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6834","nonce":3927516205},{"type":"v1","addr":"172.21.15.5:6835","nonce":3927516205}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14622,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6837/3579918721","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6836","nonce":3579918721},{"type":"v1","addr":"172.21.15.5:6837","nonce":3579918721}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14694,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6839/666617424","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6838","nonce":666617424},{"type":"v1","addr":"172.21.15.5:6839","nonce":666617424}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14706,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.5:6841/2768414598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6840","nonce":2768414598},{"type":"v1","addr":"172.21.15.5:6841","nonce":2768414598}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24451,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.156:6833/2408636934","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6832","nonce":2408636934},{"type":"v1","addr":"172.21.15.156:6833","nonce":2408636934}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24482,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.101:6833/2387040321","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6832","nonce":2387040321},{"type":"v1","addr":"172.21.15.101:6833","nonce":2387040321}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":56,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-07-21T19:05:37.254064+0000","modified":"2023-07-21T19:25:53.614690+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24614,"mds_1":24566,"mds_2":24553},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24511":{"gid":24511,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6835/2855288918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6834","nonce":2855288918},{"type":"v1","addr":"172.21.15.156:6835","nonce":2855288918}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24523":{"gid":24523,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.156:6837/1947602937","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6836","nonce":1947602937},{"type":"v1","addr":"172.21.15.156:6837","nonce":1947602937}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24524":{"gid":24524,"name":"e","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.101:6835/153908041","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6834","nonce":153908041},{"type":"v1","addr":"172.21.15.101:6835","nonce":153908041}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24553":{"gid":24553,"name":"i","rank":2,"incarnation":21,"state":"up:active","state_seq":7,"addr":"172.21.15.156:6839/2252036160","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6838","nonce":2252036160},{"type":"v1","addr":"172.21.15.156:6839","nonce":2252036160}]},"join_fscid":-1,"export_targets":[0,1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24566":{"gid":24566,"name":"h","rank":1,"incarnation":19,"state":"up:active","state_seq":7,"addr":"172.21.15.101:6837/3565857598","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6836","nonce":3565857598},{"type":"v1","addr":"172.21.15.101:6837","nonce":3565857598}]},"join_fscid":-1,"export_targets":[0,2],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24614":{"gid":24614,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.101:6839/715074805","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6838","nonce":715074805},{"type":"v1","addr":"172.21.15.101:6839","nonce":715074805}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 msgr2=0x7f12180a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 0x7f12180a3870 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f122800bb90 tx=0x7f1228005d80 comp rx=0 tx=0).stop 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 msgr2=0x7f122c19c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f121c00e9c0 tx=0x7f121c00ed80 comp rx=0 tx=0).stop 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 shutdown_connections 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f12180a13c0 0x7f12180a3870 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f122c10c1c0 0x7f122c1a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f122c106340 0x7f122c19c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.545+0000 7f1233bff700 1 --2- 172.21.15.5:0/1879674849 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f122c106cf0 0x7f122c19c760 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.546+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 >> 172.21.15.5:0/1879674849 conn(0x7f122c0759e0 msgr2=0x7f122c076eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:17.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.546+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 shutdown_connections 2023-07-21T19:28:17.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:17.546+0000 7f1233bff700 1 -- 172.21.15.5:0/1879674849 wait complete. 2023-07-21T19:28:17.541 INFO:teuthology.orchestra.run.smithi005.stderr:dumped fsmap epoch 56 2023-07-21T19:28:17.566 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2023-07-21T19:28:17.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:17 smithi005 ceph-mon[126460]: pgmap v653: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 12 MiB/s wr, 883 op/s 2023-07-21T19:28:17.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:17 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1879674849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:28:17.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:17 smithi156 ceph-mon[128316]: pgmap v653: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 12 MiB/s wr, 883 op/s 2023-07-21T19:28:17.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:17 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1879674849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:28:18.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:17 smithi101 ceph-mon[128136]: pgmap v653: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 611 KiB/s rd, 12 MiB/s wr, 883 op/s 2023-07-21T19:28:18.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:17 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1879674849' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-07-21T19:28:18.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 -- 172.21.15.5:0/3185876454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 msgr2=0x7fc2600fec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 --2- 172.21.15.5:0/3185876454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc2600fec70 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fc250009990 tx=0x7fc250009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 -- 172.21.15.5:0/3185876454 shutdown_connections 2023-07-21T19:28:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 --2- 172.21.15.5:0/3185876454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 0x7fc26010e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 --2- 172.21.15.5:0/3185876454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc2600fec70 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 --2- 172.21.15.5:0/3185876454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2600ff1b0 0x7fc26010bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.523+0000 7fc264dff700 1 -- 172.21.15.5:0/3185876454 >> 172.21.15.5:0/3185876454 conn(0x7fc2600fa4b0 msgr2=0x7fc2600fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.524+0000 7fc264dff700 1 -- 172.21.15.5:0/3185876454 shutdown_connections 2023-07-21T19:28:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.524+0000 7fc264dff700 1 -- 172.21.15.5:0/3185876454 wait complete. 2023-07-21T19:28:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.525+0000 7fc264dff700 1 Processor -- start 2023-07-21T19:28:18.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.525+0000 7fc264dff700 1 -- start start 2023-07-21T19:28:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2600ff1b0 0x7fc260198360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 0x7fc26019c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019ce00 con 0x7fc2600ff1b0 2023-07-21T19:28:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019cf70 con 0x7fc2600fe8a0 2023-07-21T19:28:18.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc264dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc26019d130 con 0x7fc26010c450 2023-07-21T19:28:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60100/0 (socket says 172.21.15.5:60100) 2023-07-21T19:28:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.526+0000 7fc25e59c700 1 -- 172.21.15.5:0/448829887 learned_addr learned my addr 172.21.15.5:0/448829887 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 0x7fc26019c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 -- 172.21.15.5:0/448829887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 msgr2=0x7fc26019c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 0x7fc26019c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 -- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2600ff1b0 msgr2=0x7fc260198360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2600ff1b0 0x7fc260198360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 -- 172.21.15.5:0/448829887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500095f0 con 0x7fc2600fe8a0 2023-07-21T19:28:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.527+0000 7fc25e59c700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc250005740 tx=0x7fc250005e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.528+0000 7fc2577fe700 1 -- 172.21.15.5:0/448829887 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25001d070 con 0x7fc2600fe8a0 2023-07-21T19:28:18.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.528+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc26019d3b0 con 0x7fc2600fe8a0 2023-07-21T19:28:18.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.528+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2601a4c50 con 0x7fc2600fe8a0 2023-07-21T19:28:18.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.528+0000 7fc2577fe700 1 -- 172.21.15.5:0/448829887 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc25000fbb0 con 0x7fc2600fe8a0 2023-07-21T19:28:18.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.528+0000 7fc2577fe700 1 -- 172.21.15.5:0/448829887 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc250017650 con 0x7fc2600fe8a0 2023-07-21T19:28:18.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.534+0000 7fc278705b80 1 Processor -- start 2023-07-21T19:28:18.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.535+0000 7fc2577fe700 1 -- 172.21.15.5:0/448829887 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc250022470 con 0x7fc2600fe8a0 2023-07-21T19:28:18.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.535+0000 7fc278705b80 1 -- start start 2023-07-21T19:28:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf48eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 0x55e6eaf53da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 0x55e6eaf566c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eae46cb0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eae99a40 con 0x55e6eaf493f0 2023-07-21T19:28:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc278705b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eae92ed0 con 0x55e6eaf542e0 2023-07-21T19:28:18.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.536+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 0x55e6eaf53da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 0x55e6eaf53da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60116/0 (socket says 172.21.15.5:60116) 2023-07-21T19:28:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc25e59c700 1 -- 172.21.15.5:0/717885207 learned_addr learned my addr 172.21.15.5:0/717885207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 0x55e6eaf566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf48eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 -- 172.21.15.5:0/717885207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 msgr2=0x55e6eaf53da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 0x55e6eaf53da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 -- 172.21.15.5:0/717885207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 msgr2=0x55e6eaf48eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf48eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.537+0000 7fc257fff700 1 -- 172.21.15.5:0/717885207 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6eacc0de0 con 0x55e6eaf542e0 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.538+0000 7fc257fff700 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 0x55e6eaf566c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ba0 tx=0x7fc248009eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1bef4427030a0694 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.539+0000 7fc2577fe700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 0x7fc24c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.539+0000 7fc254ff9700 1 -- 172.21.15.5:0/717885207 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248004230 con 0x55e6eaf542e0 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.539+0000 7fc2577fe700 1 -- 172.21.15.5:0/448829887 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc2500ce3a0 con 0x7fc2600fe8a0 2023-07-21T19:28:18.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.539+0000 7fc254ff9700 1 -- 172.21.15.5:0/717885207 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc24800ed30 con 0x55e6eaf542e0 2023-07-21T19:28:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.539+0000 7fc254ff9700 1 -- 172.21.15.5:0/717885207 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc248004e20 con 0x55e6eaf542e0 2023-07-21T19:28:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 0x7fc24c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 -- 172.21.15.5:0/717885207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 msgr2=0x55e6eaf566c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 0x55e6eaf566c0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc248009ba0 tx=0x7fc248009eb0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 -- 172.21.15.5:0/717885207 shutdown_connections 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf542e0 0x55e6eaf566c0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf48eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 --2- 172.21.15.5:0/717885207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf493f0 0x55e6eaf53da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.540+0000 7fc278705b80 1 -- 172.21.15.5:0/717885207 >> 172.21.15.5:0/717885207 conn(0x55e6eaf36da0 msgr2=0x55e6eaf3c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.541+0000 7fc278705b80 1 -- 172.21.15.5:0/717885207 shutdown_connections 2023-07-21T19:28:18.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.541+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 0x7fc24c0a37f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc260069c40 tx=0x7fc258009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.541+0000 7fc278705b80 1 -- 172.21.15.5:0/717885207 wait complete. 2023-07-21T19:28:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.542+0000 7fc278705b80 1 -- start start 2023-07-21T19:28:18.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.542+0000 7fc278705b80 1 Processor -- start 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf50f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eaf51660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eae32ac0 con 0x55e6eaf542e0 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eae327c0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6ead16920 con 0x55e6eaf493f0 2023-07-21T19:28:18.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.543+0000 7fc278705b80 10 client.0 _ll_register_callbacks cb 0x55e6eae30760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60128/0 (socket says 172.21.15.5:60128) 2023-07-21T19:28:18.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 -- 172.21.15.5:0/2816239989 learned_addr learned my addr 172.21.15.5:0/2816239989 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eaf51660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 -- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 msgr2=0x55e6eaf50f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf50f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 -- 172.21.15.5:0/2816239989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 msgr2=0x55e6eaf51660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eaf51660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.544+0000 7fc25e59c700 1 -- 172.21.15.5:0/2816239989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc248009880 con 0x55e6eaf48ae0 2023-07-21T19:28:18.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2500a3390 tx=0x7fc250009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=10aa253e623e1250 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25009b040 con 0x55e6eaf48ae0 2023-07-21T19:28:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc25009b500 con 0x55e6eaf48ae0 2023-07-21T19:28:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e6eadec4f0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc25009ae60 con 0x55e6eaf48ae0 2023-07-21T19:28:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc278705b80 10 client.29297 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.545+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e6eaca9c60 con 0x55e6eaf48ae0 2023-07-21T19:28:18.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55e6eadec4f0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc25009a9f0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 10 client.29297 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 10 client.29297 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 20 client.29297 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eaf638e0 0x55e6eaf65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 4 client.29297 mds_command: new command op to 24553 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.546+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e6eae8beb0 con 0x55e6eaf638e0 2023-07-21T19:28:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.547+0000 7fc257fff700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eaf638e0 0x55e6eaf65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.548+0000 7fc257fff700 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eaf638e0 0x55e6eaf65cf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=cc5580aad192149 server_cookie=9ae378072c2ef317 in_seq=0 out_seq=0 2023-07-21T19:28:18.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.548+0000 7fc245ffb700 10 client.29297 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:28:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.549+0000 7fc245ffb700 1 -- 172.21.15.5:0/2816239989 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e6eae8beb0 con 0x55e6eaf638e0 2023-07-21T19:28:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.549+0000 7fc245ffb700 10 client.29297 handle_command_reply: tid=0 2023-07-21T19:28:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.549+0000 7fc278705b80 1 client.29297 shutdown 2023-07-21T19:28:18.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.549+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 msgr2=0x55e6eaf53d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.549+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fc2500a3390 tx=0x7fc250009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 shutdown_connections 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eaf51660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eaf638e0 0x55e6eaf65cf0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53d10 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 --2- 172.21.15.5:0/2816239989 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf50f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 >> 172.21.15.5:0/2816239989 conn(0x55e6eaf36da0 msgr2=0x55e6eaf3c290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc245ffb700 0 client.29297 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:28:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 shutdown_connections 2023-07-21T19:28:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.550+0000 7fc278705b80 1 -- 172.21.15.5:0/2816239989 wait complete. 2023-07-21T19:28:18.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.551+0000 7fc278705b80 20 client.29297 trim_cache size 0 max 16384 2023-07-21T19:28:18.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 Processor -- start 2023-07-21T19:28:18.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 -- start start 2023-07-21T19:28:18.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf4c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf542e0 0x55e6eaf4c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eaf49dc0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eaf369a0 con 0x55e6eaf542e0 2023-07-21T19:28:18.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc278705b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eaf4a9e0 con 0x55e6eaf493f0 2023-07-21T19:28:18.560 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.569+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36160/0 (socket says 172.21.15.5:36160) 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 -- 172.21.15.5:0/2051326395 learned_addr learned my addr 172.21.15.5:0/2051326395 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf542e0 0x55e6eaf4c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc257fff700 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf4c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 -- 172.21.15.5:0/2051326395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 msgr2=0x55e6eaf4c240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf4c240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 -- 172.21.15.5:0/2051326395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf542e0 msgr2=0x55e6eaf4c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf542e0 0x55e6eaf4c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 -- 172.21.15.5:0/2051326395 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e6eace7250 con 0x55e6eaf48ae0 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc25e59c700 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc250097f70 tx=0x7fc250099ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8acb4e37a7cd5c74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc2557fa700 1 -- 172.21.15.5:0/2051326395 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500d5070 con 0x55e6eaf48ae0 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc2557fa700 1 -- 172.21.15.5:0/2051326395 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc25009ae60 con 0x55e6eaf48ae0 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.570+0000 7fc2557fa700 1 -- 172.21.15.5:0/2051326395 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25009a9f0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 -- 172.21.15.5:0/2051326395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 msgr2=0x55e6eaf53840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fc250097f70 tx=0x7fc250099ab0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 -- 172.21.15.5:0/2051326395 shutdown_connections 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf542e0 0x55e6eaf4c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf53840 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 --2- 172.21.15.5:0/2051326395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf4c240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 -- 172.21.15.5:0/2051326395 >> 172.21.15.5:0/2051326395 conn(0x55e6eaf36da0 msgr2=0x55e6eaf60000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 -- 172.21.15.5:0/2051326395 shutdown_connections 2023-07-21T19:28:18.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.571+0000 7fc278705b80 1 -- 172.21.15.5:0/2051326395 wait complete. 2023-07-21T19:28:18.564 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 -- start start 2023-07-21T19:28:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 Processor -- start 2023-07-21T19:28:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf58f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.566 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eaf59c30 con 0x55e6eaf542e0 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eafb6d10 con 0x55e6eaf48ae0 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e6eafb6ed0 con 0x55e6eaf493f0 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.572+0000 7fc278705b80 10 client.0 _ll_register_callbacks cb 0x55e6eae30760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.567 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf58f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36168/0 (socket says 172.21.15.5:36168) 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25ed9d700 1 -- 172.21.15.5:0/187945769 learned_addr learned my addr 172.21.15.5:0/187945769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc257fff700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 -- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 msgr2=0x55e6eaf594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf594a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 -- 172.21.15.5:0/187945769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 msgr2=0x55e6eafb67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 -- 172.21.15.5:0/187945769 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e6eaca9c60 con 0x55e6eaf48ae0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25ed9d700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc25e59c700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf58f60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc250097f70 tx=0x7fc25000ff50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4677a4a797d89dcc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.573+0000 7fc245ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2500d5050 con 0x55e6eaf48ae0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc25009b500 con 0x55e6eaf48ae0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc24800ed30 con 0x55e6eaf48ae0 2023-07-21T19:28:18.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc25009b040 con 0x55e6eaf48ae0 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 10 client.29303 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55e6eacbf2b0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc24800ed30 con 0x55e6eaf48ae0 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fc2500972f0 con 0x55e6eaf48ae0 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 10 client.29303 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 10 client.29303 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.i 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 20 client.29303 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:18.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eafb8120 0x55e6eafba500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 4 client.29303 mds_command: new command op to 24553 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.574+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 --> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55e6eae4e050 con 0x55e6eafb8120 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc257fff700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eafb8120 0x55e6eafba500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc245ffb700 10 client.29303 ms_handle_connect on v2:172.21.15.156:6838/2252036160 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc257fff700 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eafb8120 0x55e6eafba500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5fedea61f9b74f53 server_cookie=be03c69f5bbdd8b4 in_seq=0 out_seq=0 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc245ffb700 1 -- 172.21.15.5:0/187945769 <== mds.2 v2:172.21.15.156:6838/2252036160 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55e6eae4e050 con 0x55e6eafb8120 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc245ffb700 10 client.29303 handle_command_reply: tid=0 2023-07-21T19:28:18.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.575+0000 7fc278705b80 1 client.29303 shutdown 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 msgr2=0x55e6eaf58f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf58f60 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fc250097f70 tx=0x7fc25000ff50 comp rx=0 tx=0).stop 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 shutdown_connections 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55e6eaf542e0 0x55e6eafb67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:6838/2252036160,v1:172.21.15.156:6839/2252036160] conn(0x55e6eafb8120 0x55e6eafba500 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55e6eaf48ae0 0x55e6eaf58f60 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 --2- 172.21.15.5:0/187945769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55e6eaf493f0 0x55e6eaf594a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc245ffb700 0 client.29303 ms_handle_reset on v2:172.21.15.156:6838/2252036160 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 >> 172.21.15.5:0/187945769 conn(0x55e6eaf36da0 msgr2=0x55e6eaf4c240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 shutdown_connections 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 1 -- 172.21.15.5:0/187945769 wait complete. 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.576+0000 7fc278705b80 20 client.29303 trim_cache size 0 max 16384 2023-07-21T19:28:18.573 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:28:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 msgr2=0x7fc24c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 0x7fc24c0a37f0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc260069c40 tx=0x7fc258009040 comp rx=0 tx=0).stop 2023-07-21T19:28:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 msgr2=0x7fc260197e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fc250005740 tx=0x7fc250005e80 comp rx=0 tx=0).stop 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 shutdown_connections 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc26010c450 0x7fc26019c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2600fe8a0 0x7fc260197e20 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc24c0a1340 0x7fc24c0a37f0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 --2- 172.21.15.5:0/448829887 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2600ff1b0 0x7fc260198360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 >> 172.21.15.5:0/448829887 conn(0x7fc2600fa4b0 msgr2=0x7fc26010e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 shutdown_connections 2023-07-21T19:28:18.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.577+0000 7fc264dff700 1 -- 172.21.15.5:0/448829887 wait complete. 2023-07-21T19:28:18.583 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2023-07-21T19:28:18.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 -- 172.21.15.5:0/1617336859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 msgr2=0x7f4278074e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1617336859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f4278074e10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f426c00b0a0 tx=0x7f426c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 -- 172.21.15.5:0/1617336859 shutdown_connections 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1617336859 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 0x7f4278112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1617336859 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f4278074e10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1617336859 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.933+0000 7f427f3ff700 1 -- 172.21.15.5:0/1617336859 >> 172.21.15.5:0/1617336859 conn(0x7f42780fd7e0 msgr2=0x7f42780ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.934+0000 7f427f3ff700 1 -- 172.21.15.5:0/1617336859 shutdown_connections 2023-07-21T19:28:18.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.934+0000 7f427f3ff700 1 -- 172.21.15.5:0/1617336859 wait complete. 2023-07-21T19:28:18.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.935+0000 7f427f3ff700 1 Processor -- start 2023-07-21T19:28:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.936+0000 7f427f3ff700 1 -- start start 2023-07-21T19:28:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.936+0000 7f427f3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f427810fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.936+0000 7f427f3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.936+0000 7f427f3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 0x7f4278108cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.936+0000 7f427f3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4278109480 con 0x7f4278075350 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427f3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42781095f0 con 0x7f4278074a40 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427f3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42781097b0 con 0x7f42781109a0 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427c99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427c99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36184/0 (socket says 172.21.15.5:36184) 2023-07-21T19:28:18.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427c99a700 1 -- 172.21.15.5:0/1951989825 learned_addr learned my addr 172.21.15.5:0/1951989825 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427d19b700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f427810fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.937+0000 7f427d99c700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 0x7f4278108cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.938+0000 7f427c99a700 1 -- 172.21.15.5:0/1951989825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 msgr2=0x7f4278108cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.938+0000 7f427c99a700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 0x7f4278108cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.938+0000 7f427c99a700 1 -- 172.21.15.5:0/1951989825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 msgr2=0x7f427810fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.938+0000 7f427c99a700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f427810fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.938+0000 7f427c99a700 1 -- 172.21.15.5:0/1951989825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f426c009d00 con 0x7f4278075350 2023-07-21T19:28:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.939+0000 7f427c99a700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f426400d950 tx=0x7f426400dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.939+0000 7f42767fc700 1 -- 172.21.15.5:0/1951989825 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4264009b40 con 0x7f4278075350 2023-07-21T19:28:18.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.939+0000 7f42767fc700 1 -- 172.21.15.5:0/1951989825 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4264009ca0 con 0x7f4278075350 2023-07-21T19:28:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.939+0000 7f42767fc700 1 -- 172.21.15.5:0/1951989825 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426400f620 con 0x7f4278075350 2023-07-21T19:28:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.940+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4278109a90 con 0x7f4278075350 2023-07-21T19:28:18.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.940+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f427810dad0 con 0x7f4278075350 2023-07-21T19:28:18.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.946+0000 7f4292c50b80 1 Processor -- start 2023-07-21T19:28:18.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 -- start start 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c38590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c3aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741b33cb0 con 0x563741c38ad0 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741b86a40 con 0x563741c2dbe0 2023-07-21T19:28:18.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.947+0000 7f4292c50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741b7fed0 con 0x563741c2d2d0 2023-07-21T19:28:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c38590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39054/0 (socket says 172.21.15.5:39054) 2023-07-21T19:28:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d99c700 1 -- 172.21.15.5:0/2555146046 learned_addr learned my addr 172.21.15.5:0/2555146046 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d19b700 1 -- 172.21.15.5:0/2555146046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 msgr2=0x563741c2d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.948+0000 7f427d19b700 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.949+0000 7f427d19b700 1 -- 172.21.15.5:0/2555146046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 msgr2=0x563741c3aeb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.949+0000 7f427d19b700 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c3aeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.949+0000 7f427d19b700 1 -- 172.21.15.5:0/2555146046 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637419adde0 con 0x563741c2dbe0 2023-07-21T19:28:18.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.949+0000 7f427d99c700 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.949+0000 7f42767fc700 1 -- 172.21.15.5:0/1951989825 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4264010460 con 0x7f4278075350 2023-07-21T19:28:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.950+0000 7f427d19b700 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c38590 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d010 tx=0x7f426c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=56b856a1e59f5061 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.950+0000 7f42637fe700 1 -- 172.21.15.5:0/2555146046 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e040 con 0x563741c2dbe0 2023-07-21T19:28:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.950+0000 7f42637fe700 1 -- 172.21.15.5:0/2555146046 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f426c007d90 con 0x563741c2dbe0 2023-07-21T19:28:18.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.950+0000 7f42637fe700 1 -- 172.21.15.5:0/2555146046 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c004cb0 con 0x563741c2dbe0 2023-07-21T19:28:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.951+0000 7f4292c50b80 1 -- 172.21.15.5:0/2555146046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 msgr2=0x563741c38590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.951+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c38590 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f426c00d010 tx=0x7f426c009c00 comp rx=0 tx=0).stop 2023-07-21T19:28:18.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.951+0000 7f42767fc700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 0x7f42680a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f42767fc700 1 -- 172.21.15.5:0/1951989825 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f42640ccbf0 con 0x7f4278075350 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f427d99c700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 0x7f42680a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f4292c50b80 1 -- 172.21.15.5:0/2555146046 shutdown_connections 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c3aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c2d6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2555146046 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c38590 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.952+0000 7f4292c50b80 1 -- 172.21.15.5:0/2555146046 >> 172.21.15.5:0/2555146046 conn(0x563741c23da0 msgr2=0x563741c296a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.953+0000 7f4292c50b80 1 -- 172.21.15.5:0/2555146046 shutdown_connections 2023-07-21T19:28:18.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.953+0000 7f4292c50b80 1 -- 172.21.15.5:0/2555146046 wait complete. 2023-07-21T19:28:18.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.953+0000 7f427d99c700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 0x7f42680a3980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4270009cf0 tx=0x7f4270009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.954+0000 7f4292c50b80 1 -- start start 2023-07-21T19:28:18.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.954+0000 7f4292c50b80 1 Processor -- start 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.954+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c38560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.954+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.954+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c35f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f4292c50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741b1fac0 con 0x563741c38ad0 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f4292c50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741b1f7c0 con 0x563741c2d2d0 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f4292c50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741a03920 con 0x563741c2dbe0 2023-07-21T19:28:18.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f4292c50b80 10 client.0 _ll_register_callbacks cb 0x563741b1d760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39056/0 (socket says 172.21.15.5:39056) 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 -- 172.21.15.5:0/2311506785 learned_addr learned my addr 172.21.15.5:0/2311506785 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427d99c700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c35f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427d19b700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c38560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 -- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 msgr2=0x563741c38560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c38560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 -- 172.21.15.5:0/2311506785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 msgr2=0x563741c35f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c35f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.955+0000 7f427c99a700 1 -- 172.21.15.5:0/2311506785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42781137b0 con 0x563741c2dbe0 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f427d99c700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c35f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f427c99a700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f42640966d0 tx=0x7f42640998d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbff01d755aba36a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f4260ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:28:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426409a0c0 con 0x563741c2dbe0 2023-07-21T19:28:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f42640a20b0 con 0x563741c2dbe0 2023-07-21T19:28:18.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.956+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563741ad94f0 con 0x563741c2dbe0 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f42640ca770 con 0x563741c2dbe0 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 10 client.29242 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563741996c60 con 0x563741c2dbe0 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f42640b6070 con 0x563741c2dbe0 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4264099150 con 0x563741c2dbe0 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 10 client.29242 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:18.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 10 client.29242 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 20 client.29242 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c48320 0x563741c4a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 4 client.29242 mds_command: new command op to 24566 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.957+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563741b78eb0 con 0x563741c48320 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.958+0000 7f427d19b700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c48320 0x563741c4a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.959+0000 7f427d19b700 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c48320 0x563741c4a730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8d3a52a1ab26e397 server_cookie=5dc4648d1a3ae02f in_seq=0 out_seq=0 2023-07-21T19:28:18.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.959+0000 7f4260ff9700 10 client.29242 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:28:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.960+0000 7f4260ff9700 1 -- 172.21.15.5:0/2311506785 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563741b78eb0 con 0x563741c48320 2023-07-21T19:28:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.960+0000 7f4260ff9700 10 client.29242 handle_command_reply: tid=0 2023-07-21T19:28:18.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.960+0000 7f4292c50b80 1 client.29242 shutdown 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.961+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 msgr2=0x563741c357e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.961+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f42640966d0 tx=0x7f42640998d0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 shutdown_connections 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c35f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c48320 0x563741c4a730 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c38560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2311506785 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2dbe0 0x563741c357e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 >> 172.21.15.5:0/2311506785 conn(0x563741c23da0 msgr2=0x563741c29290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4260ff9700 0 client.29242 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:28:18.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 shutdown_connections 2023-07-21T19:28:18.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.962+0000 7f4292c50b80 1 -- 172.21.15.5:0/2311506785 wait complete. 2023-07-21T19:28:18.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.963+0000 7f4292c50b80 20 client.29242 trim_cache size 0 max 16384 2023-07-21T19:28:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.984+0000 7f4292c50b80 1 Processor -- start 2023-07-21T19:28:18.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.984+0000 7f4292c50b80 1 -- start start 2023-07-21T19:28:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.984+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.984+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c30b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c31070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f4292c50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c2e160 con 0x563741c38ad0 2023-07-21T19:28:18.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f4292c50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c20ef0 con 0x563741c2dbe0 2023-07-21T19:28:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f4292c50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c45a80 con 0x563741c2d2d0 2023-07-21T19:28:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f427d19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f427d19b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39062/0 (socket says 172.21.15.5:39062) 2023-07-21T19:28:18.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f427d19b700 1 -- 172.21.15.5:0/2229948143 learned_addr learned my addr 172.21.15.5:0/2229948143 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f427c99a700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c30b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.985+0000 7f427d99c700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c31070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 -- 172.21.15.5:0/2229948143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 msgr2=0x563741c30b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c30b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 -- 172.21.15.5:0/2229948143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 msgr2=0x563741c31070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c31070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 -- 172.21.15.5:0/2229948143 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5637419d4250 con 0x563741c2d2d0 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d99c700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c31070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f427d19b700 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f426c0062a0 tx=0x7f426c0099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5abb65191ee8a98a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f4263fff700 1 -- 172.21.15.5:0/2229948143 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e040 con 0x563741c2d2d0 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f4263fff700 1 -- 172.21.15.5:0/2229948143 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f426c029070 con 0x563741c2d2d0 2023-07-21T19:28:18.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.986+0000 7f4263fff700 1 -- 172.21.15.5:0/2229948143 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c030480 con 0x563741c2d2d0 2023-07-21T19:28:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.987+0000 7f4292c50b80 1 -- 172.21.15.5:0/2229948143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 msgr2=0x563741c4a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.987+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f426c0062a0 tx=0x7f426c0099e0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 -- 172.21.15.5:0/2229948143 shutdown_connections 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c38ad0 0x563741c31070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c2d2d0 0x563741c4a330 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 --2- 172.21.15.5:0/2229948143 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2dbe0 0x563741c30b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 -- 172.21.15.5:0/2229948143 >> 172.21.15.5:0/2229948143 conn(0x563741c23da0 msgr2=0x563741c44720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 -- 172.21.15.5:0/2229948143 shutdown_connections 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 -- 172.21.15.5:0/2229948143 wait complete. 2023-07-21T19:28:18.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.988+0000 7f4292c50b80 1 -- start start 2023-07-21T19:28:18.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 Processor -- start 2023-07-21T19:28:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c3d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c2dbe0 0x563741c3dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c9b6e0 con 0x563741c2dbe0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c9b820 con 0x563741c2d2d0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.989+0000 7f4292c50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563741c9b960 con 0x563741c38ad0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f4292c50b80 10 client.0 _ll_register_callbacks cb 0x7f4264099a30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f427d19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c3d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f427d99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f427d99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39072/0 (socket says 172.21.15.5:39072) 2023-07-21T19:28:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f427d99c700 1 -- 172.21.15.5:0/1676097625 learned_addr learned my addr 172.21.15.5:0/1676097625 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:18.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.990+0000 7f427c99a700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c2dbe0 0x563741c3dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 -- 172.21.15.5:0/1676097625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 msgr2=0x563741c9b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 -- 172.21.15.5:0/1676097625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c2dbe0 msgr2=0x563741c3dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:18.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c2dbe0 0x563741c3dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 -- 172.21.15.5:0/1676097625 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5637419adde0 con 0x563741c2d2d0 2023-07-21T19:28:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d99c700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f427d19b700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c3d680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f426c009610 tx=0x7f426c0094a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab03427299f48934 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:18.985 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f4260ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f426c00e050 con 0x563741c2d2d0 2023-07-21T19:28:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.991+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f426c007d90 con 0x563741c2d2d0 2023-07-21T19:28:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f426c02b070 con 0x563741c2d2d0 2023-07-21T19:28:18.986 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f426c007850 con 0x563741c2d2d0 2023-07-21T19:28:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 10 client.29315 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5637419ac2b0 con 0x563741c2d2d0 2023-07-21T19:28:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f426c007d90 con 0x563741c2d2d0 2023-07-21T19:28:18.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f426c019040 con 0x563741c2d2d0 2023-07-21T19:28:18.989 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 10 client.29315 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 10 client.29315 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.h 2023-07-21T19:28:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 20 client.29315 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:18.990 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c9cc50 0x563741c9f030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 4 client.29315 mds_command: new command op to 24566 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:28:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.992+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 --> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x563741b3b050 con 0x563741c9cc50 2023-07-21T19:28:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.993+0000 7f427c99a700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c9cc50 0x563741c9f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:18.991 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.993+0000 7f427c99a700 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c9cc50 0x563741c9f030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=730a283e73895bd0 server_cookie=687800731aecada5 in_seq=0 out_seq=0 2023-07-21T19:28:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.993+0000 7f4260ff9700 10 client.29315 ms_handle_connect on v2:172.21.15.101:6836/3565857598 2023-07-21T19:28:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4260ff9700 1 -- 172.21.15.5:0/1676097625 <== mds.1 v2:172.21.15.101:6836/3565857598 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x563741b3b050 con 0x563741c9cc50 2023-07-21T19:28:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4260ff9700 10 client.29315 handle_command_reply: tid=0 2023-07-21T19:28:18.992 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 client.29315 shutdown 2023-07-21T19:28:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 msgr2=0x563741c3d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c3d680 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f426c009610 tx=0x7f426c0094a0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.993 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 shutdown_connections 2023-07-21T19:28:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563741c38ad0 0x563741c9b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:6836/3565857598,v1:172.21.15.101:6837/3565857598] conn(0x563741c9cc50 0x563741c9f030 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563741c2d2d0 0x563741c3d680 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.994 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 --2- 172.21.15.5:0/1676097625 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563741c2dbe0 0x563741c3dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4260ff9700 0 client.29315 ms_handle_reset on v2:172.21.15.101:6836/3565857598 2023-07-21T19:28:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.994+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 >> 172.21.15.5:0/1676097625 conn(0x563741c23da0 msgr2=0x563741c30b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.995+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 shutdown_connections 2023-07-21T19:28:18.995 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.995+0000 7f4292c50b80 1 -- 172.21.15.5:0/1676097625 wait complete. 2023-07-21T19:28:18.996 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.995+0000 7f4292c50b80 20 client.29315 trim_cache size 0 max 16384 2023-07-21T19:28:18.996 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:28:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 msgr2=0x7f42680a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 0x7f42680a3980 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f4270009cf0 tx=0x7f4270009260 comp rx=0 tx=0).stop 2023-07-21T19:28:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 msgr2=0x7f4278110220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f426400d950 tx=0x7f426400dd10 comp rx=0 tx=0).stop 2023-07-21T19:28:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 shutdown_connections 2023-07-21T19:28:18.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f42781109a0 0x7f4278108cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4278074a40 0x7f427810fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f42680a14d0 0x7f42680a3980 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 --2- 172.21.15.5:0/1951989825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4278075350 0x7f4278110220 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:18.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.996+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 >> 172.21.15.5:0/1951989825 conn(0x7f42780fd7e0 msgr2=0x7f4278111600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.997+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 shutdown_connections 2023-07-21T19:28:19.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:18.997+0000 7f427f3ff700 1 -- 172.21.15.5:0/1951989825 wait complete. 2023-07-21T19:28:19.012 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2023-07-21T19:28:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 -- 172.21.15.5:0/232224703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 msgr2=0x7f92b81105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 --2- 172.21.15.5:0/232224703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b81105a0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f92a80098f0 tx=0x7f92a8009c00 comp rx=0 tx=0).stop 2023-07-21T19:28:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 -- 172.21.15.5:0/232224703 shutdown_connections 2023-07-21T19:28:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 --2- 172.21.15.5:0/232224703 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b8110ae0 0x7f92b8112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 --2- 172.21.15.5:0/232224703 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b8072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 --2- 172.21.15.5:0/232224703 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b81105a0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.364+0000 7f92bffff700 1 -- 172.21.15.5:0/232224703 >> 172.21.15.5:0/232224703 conn(0x7f92b80fb5f0 msgr2=0x7f92b80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.365+0000 7f92bffff700 1 -- 172.21.15.5:0/232224703 shutdown_connections 2023-07-21T19:28:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.365+0000 7f92bffff700 1 -- 172.21.15.5:0/232224703 wait complete. 2023-07-21T19:28:19.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 Processor -- start 2023-07-21T19:28:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 -- start start 2023-07-21T19:28:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b819c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b819c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b8110ae0 0x7f92b81a0c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a1240 con 0x7f92b8110ae0 2023-07-21T19:28:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a13b0 con 0x7f92b80728b0 2023-07-21T19:28:19.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.367+0000 7f92bffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b81a1570 con 0x7f92b80731c0 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.368+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b819c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.368+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b819c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60232/0 (socket says 172.21.15.5:60232) 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.368+0000 7f92bdd9b700 1 -- 172.21.15.5:0/1619061565 learned_addr learned my addr 172.21.15.5:0/1619061565 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.368+0000 7f92bd59a700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b819c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 -- 172.21.15.5:0/1619061565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 msgr2=0x7f92b819c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b819c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 -- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b8110ae0 msgr2=0x7f92b81a0c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b8110ae0 0x7f92b81a0c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 -- 172.21.15.5:0/1619061565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a8009520 con 0x7f92b80731c0 2023-07-21T19:28:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.369+0000 7f92bd59a700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b819c7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009230 tx=0x7f92a800f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.370+0000 7f92aeffd700 1 -- 172.21.15.5:0/1619061565 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a801c070 con 0x7f92b80731c0 2023-07-21T19:28:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.370+0000 7f92aeffd700 1 -- 172.21.15.5:0/1619061565 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92a800bba0 con 0x7f92b80731c0 2023-07-21T19:28:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.370+0000 7f92aeffd700 1 -- 172.21.15.5:0/1619061565 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a80038e0 con 0x7f92b80731c0 2023-07-21T19:28:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.370+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b81a17f0 con 0x7f92b80731c0 2023-07-21T19:28:19.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.370+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b81a1a70 con 0x7f92b80731c0 2023-07-21T19:28:19.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.376+0000 7f92aeffd700 1 -- 172.21.15.5:0/1619061565 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f92a8020030 con 0x7f92b80731c0 2023-07-21T19:28:19.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.377+0000 7f92d3a00b80 1 Processor -- start 2023-07-21T19:28:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.377+0000 7f92d3a00b80 1 -- start start 2023-07-21T19:28:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0261590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0263eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d015ccb0 con 0x55d8d0261ad0 2023-07-21T19:28:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d01afa40 con 0x55d8d02562d0 2023-07-21T19:28:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d01a8ed0 con 0x55d8d0256be0 2023-07-21T19:28:19.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.378+0000 7f92be59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92bd59a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0263eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0261590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60234/0 (socket says 172.21.15.5:60234) 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0261590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39096/0 (socket says 172.21.15.5:39096) 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 -- 172.21.15.5:0/3350334284 learned_addr learned my addr 172.21.15.5:0/3350334284 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 -- 172.21.15.5:0/3350334284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 msgr2=0x55d8d0261590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0261590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92aeffd700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 0x7f92a40a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 -- 172.21.15.5:0/3350334284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 msgr2=0x55d8d0263eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0263eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92be59c700 1 -- 172.21.15.5:0/3350334284 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cffd6de0 con 0x55d8d02562d0 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.379+0000 7f92aeffd700 1 -- 172.21.15.5:0/1619061565 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f92a80cd9c0 con 0x7f92b80731c0 2023-07-21T19:28:19.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.380+0000 7f92be59c700 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009c00 tx=0x7f92b0009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3eea8a0e34a8b6b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.380+0000 7f92a3fff700 1 -- 172.21.15.5:0/3350334284 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0007e00 con 0x55d8d02562d0 2023-07-21T19:28:19.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.380+0000 7f92be59c700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 0x7f92a40a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.380+0000 7f92a3fff700 1 -- 172.21.15.5:0/3350334284 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92b000f040 con 0x55d8d02562d0 2023-07-21T19:28:19.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.380+0000 7f92a3fff700 1 -- 172.21.15.5:0/3350334284 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00125f0 con 0x55d8d02562d0 2023-07-21T19:28:19.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.381+0000 7f92d3a00b80 1 -- 172.21.15.5:0/3350334284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 msgr2=0x55d8d02566a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.381+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009c00 tx=0x7f92b0009510 comp rx=0 tx=0).stop 2023-07-21T19:28:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92be59c700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 0x7f92a40a3830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f92b0013fd0 tx=0x7f92b00129e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 -- 172.21.15.5:0/3350334284 shutdown_connections 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0263eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d02566a0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/3350334284 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0261590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 -- 172.21.15.5:0/3350334284 >> 172.21.15.5:0/3350334284 conn(0x55d8d024cda0 msgr2=0x55d8d02526a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 -- 172.21.15.5:0/3350334284 shutdown_connections 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 -- 172.21.15.5:0/3350334284 wait complete. 2023-07-21T19:28:19.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.382+0000 7f92d3a00b80 1 -- start start 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 Processor -- start 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0261510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d0148ac0 con 0x55d8d02562d0 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d01487c0 con 0x55d8d0256be0 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.383+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d002c920 con 0x55d8d0261ad0 2023-07-21T19:28:19.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92d3a00b80 10 client.0 _ll_register_callbacks cb 0x55d8d0146760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:19.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60242/0 (socket says 172.21.15.5:60242) 2023-07-21T19:28:19.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 -- 172.21.15.5:0/167596884 learned_addr learned my addr 172.21.15.5:0/167596884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 -- 172.21.15.5:0/167596884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 msgr2=0x55d8d025ef40 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025ef40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 -- 172.21.15.5:0/167596884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 msgr2=0x55d8d0261510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0261510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.384+0000 7f92bd59a700 1 -- 172.21.15.5:0/167596884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0009860 con 0x55d8d0256be0 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.385+0000 7f92bd59a700 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f92a80cb170 tx=0x7f92a8009880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69d89726edf65b17 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.385+0000 7f92a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:19.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.385+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a809a480 con 0x55d8d0256be0 2023-07-21T19:28:19.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.385+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f92a809ab20 con 0x55d8d0256be0 2023-07-21T19:28:19.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.386+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d8d01024f0 con 0x55d8d0256be0 2023-07-21T19:28:19.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.386+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92a8005040 con 0x55d8d0256be0 2023-07-21T19:28:19.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.387+0000 7f92d3a00b80 10 client.29327 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.387+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d8cffbfc60 con 0x55d8d0256be0 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d8d01024f0 con 0x55d8d0256be0 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f92a809ad80 con 0x55d8d0256be0 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 10 client.29327 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 10 client.29327 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 20 client.29327 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:19.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d0270fb0 0x55d8d02733c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 4 client.29327 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.388+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d8d01a1eb0 con 0x55d8d0270fb0 2023-07-21T19:28:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.389+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d0270fb0 0x55d8d02733c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.390+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d0270fb0 0x55d8d02733c0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d94aaf318740682f server_cookie=89fa078ad424669b in_seq=0 out_seq=0 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.390+0000 7f92a17fa700 10 client.29327 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.391+0000 7f92a17fa700 1 -- 172.21.15.5:0/167596884 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d8d01a1eb0 con 0x55d8d0270fb0 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.391+0000 7f92a17fa700 10 client.29327 handle_command_reply: tid=0 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.391+0000 7f92d3a00b80 1 client.29327 shutdown 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.391+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 msgr2=0x55d8d025e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.391+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f92a80cb170 tx=0x7f92a8009880 comp rx=0 tx=0).stop 2023-07-21T19:28:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 shutdown_connections 2023-07-21T19:28:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d0270fb0 0x55d8d02733c0 crc :-1 s=READY pgs=419 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0261510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/167596884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025e800 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 >> 172.21.15.5:0/167596884 conn(0x55d8d024cda0 msgr2=0x55d8d0252290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92a17fa700 0 client.29327 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.392+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 shutdown_connections 2023-07-21T19:28:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.393+0000 7f92d3a00b80 1 -- 172.21.15.5:0/167596884 wait complete. 2023-07-21T19:28:19.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.393+0000 7f92d3a00b80 20 client.29327 trim_cache size 0 max 16384 2023-07-21T19:28:19.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.417+0000 7f92d3a00b80 1 Processor -- start 2023-07-21T19:28:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 -- start start 2023-07-21T19:28:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0259b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.407 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d0261850 con 0x55d8d02562d0 2023-07-21T19:28:19.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d025cd60 con 0x55d8d0256be0 2023-07-21T19:28:19.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.418+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d0253c70 con 0x55d8d0261ad0 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39108/0 (socket says 172.21.15.5:39108) 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 -- 172.21.15.5:0/153928 learned_addr learned my addr 172.21.15.5:0/153928 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92bd59a700 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0259b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 -- 172.21.15.5:0/153928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 msgr2=0x55d8d025a070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025a070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 -- 172.21.15.5:0/153928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 msgr2=0x55d8d0259b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0259b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.419+0000 7f92be59c700 1 -- 172.21.15.5:0/153928 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d8cfffd250 con 0x55d8d0261ad0 2023-07-21T19:28:19.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.420+0000 7f92be59c700 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009b50 tx=0x7f92b0021000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=185cac0176ced793 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.420+0000 7f92acff9700 1 -- 172.21.15.5:0/153928 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b002d070 con 0x55d8d0261ad0 2023-07-21T19:28:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.420+0000 7f92acff9700 1 -- 172.21.15.5:0/153928 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92b0012b50 con 0x55d8d0261ad0 2023-07-21T19:28:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.420+0000 7f92acff9700 1 -- 172.21.15.5:0/153928 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0029430 con 0x55d8d0261ad0 2023-07-21T19:28:19.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 -- 172.21.15.5:0/153928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 msgr2=0x55d8d025dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f92b0009b50 tx=0x7f92b0021000 comp rx=0 tx=0).stop 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 -- 172.21.15.5:0/153928 shutdown_connections 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0261ad0 0x55d8d025dfc0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d02562d0 0x55d8d0259b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/153928 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d0256be0 0x55d8d025a070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 -- 172.21.15.5:0/153928 >> 172.21.15.5:0/153928 conn(0x55d8d024cda0 msgr2=0x55d8d0261140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.421+0000 7f92d3a00b80 1 -- 172.21.15.5:0/153928 shutdown_connections 2023-07-21T19:28:19.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.422+0000 7f92d3a00b80 1 -- 172.21.15.5:0/153928 wait complete. 2023-07-21T19:28:19.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.422+0000 7f92d3a00b80 1 -- start start 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.422+0000 7f92d3a00b80 1 Processor -- start 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0267010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0267550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d0268a90 con 0x55d8d0261ad0 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d02c4e80 con 0x55d8d02562d0 2023-07-21T19:28:19.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d8d02c4fc0 con 0x55d8d0256be0 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92d3a00b80 10 client.0 _ll_register_callbacks cb 0x55d8d0146760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.423+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60270/0 (socket says 172.21.15.5:60270) 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 -- 172.21.15.5:0/2229105697 learned_addr learned my addr 172.21.15.5:0/2229105697 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 -- 172.21.15.5:0/2229105697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 msgr2=0x55d8d0267010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bd59a700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0267010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0267010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 -- 172.21.15.5:0/2229105697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 msgr2=0x55d8d0267550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:19.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0267550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 -- 172.21.15.5:0/2229105697 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b0024070 con 0x55d8d02562d0 2023-07-21T19:28:19.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bd59a700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0267010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:28:19.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92bdd9b700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55d8d0273ca0 tx=0x7f92b4010710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=168640dfb601b982 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:19.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.424+0000 7f92a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:19.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.425+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b4017030 con 0x55d8d02562d0 2023-07-21T19:28:19.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.425+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f92b000f040 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.425+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f92b401b4a0 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.426+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f92b401b780 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.426+0000 7f92d3a00b80 10 client.29333 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.426+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55d8cffd52b0 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.426+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f92b000f040 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.426+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f92b4005400 con 0x55d8d02562d0 2023-07-21T19:28:19.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 10 client.29333 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 10 client.29333 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 20 client.29333 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d02c6740 0x55d8d02c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 4 client.29333 mds_command: new command op to 24614 tid=0[{"prefix": "damage ls"}] 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "damage ls"}) v1 -- 0x55d8d0164050 con 0x55d8d02c6740 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.427+0000 7f92bd59a700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d02c6740 0x55d8d02c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.428+0000 7f92bd59a700 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d02c6740 0x55d8d02c8b50 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc871b0d21e98f92 server_cookie=40c752cfb8ff6786 in_seq=0 out_seq=0 2023-07-21T19:28:19.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.428+0000 7f92a17fa700 10 client.29333 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:19.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.428+0000 7f92a17fa700 1 -- 172.21.15.5:0/2229105697 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+3 (crc 0 0 0) 0x55d8d0164050 con 0x55d8d02c6740 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.428+0000 7f92a17fa700 10 client.29333 handle_command_reply: tid=0 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.429+0000 7f92d3a00b80 1 client.29333 shutdown 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.429+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 msgr2=0x55d8d0266ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.429+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x55d8d0273ca0 tx=0x7f92b4010710 comp rx=0 tx=0).stop 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 shutdown_connections 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d8d02c6740 0x55d8d02c8b50 crc :-1 s=READY pgs=420 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d8d0261ad0 0x55d8d0267550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d8d02562d0 0x55d8d0266ad0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92a17fa700 0 client.29333 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 --2- 172.21.15.5:0/2229105697 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d8d0256be0 0x55d8d0267010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.430+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 >> 172.21.15.5:0/2229105697 conn(0x55d8d024cda0 msgr2=0x55d8d0259b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.431+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 shutdown_connections 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.431+0000 7f92d3a00b80 1 -- 172.21.15.5:0/2229105697 wait complete. 2023-07-21T19:28:19.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.432+0000 7f92d3a00b80 20 client.29333 trim_cache size 0 max 16384 2023-07-21T19:28:19.423 INFO:teuthology.orchestra.run.smithi005.stdout:[] 2023-07-21T19:28:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.432+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 msgr2=0x7f92a40a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.432+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 0x7f92a40a3830 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f92b0013fd0 tx=0x7f92b00129e0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.432+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 msgr2=0x7f92b819c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.432+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b819c7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009230 tx=0x7f92a800f9e0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 shutdown_connections 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f92a40a1380 0x7f92a40a3830 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f92b8110ae0 0x7f92b81a0c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f92b80728b0 0x7f92b819c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 --2- 172.21.15.5:0/1619061565 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f92b80731c0 0x7f92b819c7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.433+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 >> 172.21.15.5:0/1619061565 conn(0x7f92b80fb5f0 msgr2=0x7f92b80fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.434+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 shutdown_connections 2023-07-21T19:28:19.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:19.434+0000 7f92bffff700 1 -- 172.21.15.5:0/1619061565 wait complete. 2023-07-21T19:28:19.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:19 smithi005 ceph-mon[126460]: pgmap v654: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:19.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:19 smithi156 ceph-mon[128316]: pgmap v654: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:20.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:19 smithi101 ceph-mon[128136]: pgmap v654: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:20.452 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2023-07-21T19:28:20.452 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2023-07-21T19:28:21.138 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.149+0000 7fcf94bff700 1 -- 172.21.15.5:0/442655947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 msgr2=0x7fcf9010c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.149+0000 7fcf94bff700 1 --2- 172.21.15.5:0/442655947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf9010c920 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009990 tx=0x7fcf84009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.139 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.150+0000 7fcf94bff700 1 -- 172.21.15.5:0/442655947 shutdown_connections 2023-07-21T19:28:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.150+0000 7fcf94bff700 1 --2- 172.21.15.5:0/442655947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf9010c920 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.150+0000 7fcf94bff700 1 --2- 172.21.15.5:0/442655947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf90105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.140 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.150+0000 7fcf94bff700 1 --2- 172.21.15.5:0/442655947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf900fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.150+0000 7fcf94bff700 1 -- 172.21.15.5:0/442655947 >> 172.21.15.5:0/442655947 conn(0x7fcf900fb5e0 msgr2=0x7fcf900fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.141 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.151+0000 7fcf94bff700 1 -- 172.21.15.5:0/442655947 shutdown_connections 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.150+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2565270307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64107780 msgr2=0x7ffb64107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.150+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2565270307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64107780 0x7ffb64107b50 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009990 tx=0x7ffb4c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.151+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2565270307 shutdown_connections 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.151+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2565270307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb64110b50 0x7ffb64112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.151+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2565270307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64107780 0x7ffb64107b50 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.151+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2565270307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64108090 0x7ffb64110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.151+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2565270307 >> 172.21.15.5:0/2565270307 conn(0x7ffb6406c270 msgr2=0x7ffb6406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.152+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2565270307 shutdown_connections 2023-07-21T19:28:21.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.152+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2565270307 wait complete. 2023-07-21T19:28:21.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.151+0000 7fcf94bff700 1 -- 172.21.15.5:0/442655947 wait complete. 2023-07-21T19:28:21.143 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.153+0000 7fcf94bff700 1 Processor -- start 2023-07-21T19:28:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.154+0000 7ffb6a3ff700 1 Processor -- start 2023-07-21T19:28:21.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.154+0000 7ffb6a3ff700 1 -- start start 2023-07-21T19:28:21.144 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 -- start start 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf90198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf90198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf901937f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90193e80 con 0x7fcf90105af0 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90193ff0 con 0x7fcf90100490 2023-07-21T19:28:21.145 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.154+0000 7fcf94bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf901941b0 con 0x7fcf900ffae0 2023-07-21T19:28:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.155+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf90198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.155+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf90198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39136/0 (socket says 172.21.15.5:39136) 2023-07-21T19:28:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.155+0000 7fcf8e59c700 1 -- 172.21.15.5:0/306712236 learned_addr learned my addr 172.21.15.5:0/306712236 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.155+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf90198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.146 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf901937f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8dd9b700 1 -- 172.21.15.5:0/306712236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 msgr2=0x7fcf90198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf90198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8dd9b700 1 -- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 msgr2=0x7fcf901937f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf901937f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8dd9b700 1 -- 172.21.15.5:0/306712236 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf840095f0 con 0x7fcf90100490 2023-07-21T19:28:21.147 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.156+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf901937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.157+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf90198bb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800bd50 tx=0x7fcf8800bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.157+0000 7fcf7f7fe700 1 -- 172.21.15.5:0/306712236 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8800cc90 con 0x7fcf90100490 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.157+0000 7fcf7f7fe700 1 -- 172.21.15.5:0/306712236 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf8800cdf0 con 0x7fcf90100490 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.157+0000 7fcf7f7fe700 1 -- 172.21.15.5:0/306712236 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf88012670 con 0x7fcf90100490 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.157+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf90194490 con 0x7fcf90100490 2023-07-21T19:28:21.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.158+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf901975b0 con 0x7fcf90100490 2023-07-21T19:28:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64107780 0x7ffb641aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb64110b50 0x7ffb641a48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb641a4f10 con 0x7ffb64110b50 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb641a5080 con 0x7ffb64108090 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb6a3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb641a5240 con 0x7ffb64107780 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb5bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60302/0 (socket says 172.21.15.5:60302) 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.158+0000 7ffb5bfff700 1 -- 172.21.15.5:0/2849444858 learned_addr learned my addr 172.21.15.5:0/2849444858 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.159+0000 7ffb63fff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64107780 0x7ffb641aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 -- 172.21.15.5:0/2849444858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64107780 msgr2=0x7ffb641aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64107780 0x7ffb641aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 -- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb64110b50 msgr2=0x7ffb641a48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb64110b50 0x7ffb641a48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 -- 172.21.15.5:0/2849444858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb4c0095f0 con 0x7ffb64108090 2023-07-21T19:28:21.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400b550 tx=0x7ffb5400b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.160+0000 7ffb61ffb700 1 -- 172.21.15.5:0/2849444858 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54011830 con 0x7ffb64108090 2023-07-21T19:28:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.161+0000 7ffb61ffb700 1 -- 172.21.15.5:0/2849444858 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb54011e20 con 0x7ffb64108090 2023-07-21T19:28:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.161+0000 7ffb61ffb700 1 -- 172.21.15.5:0/2849444858 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb54010dd0 con 0x7ffb64108090 2023-07-21T19:28:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.162+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb641a5520 con 0x7ffb64108090 2023-07-21T19:28:21.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.162+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb641b1a30 con 0x7ffb64108090 2023-07-21T19:28:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.167+0000 7fcfa859db80 1 Processor -- start 2023-07-21T19:28:21.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.167+0000 7fcfa859db80 1 -- start start 2023-07-21T19:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd31610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd33f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedc2cd30 con 0x559eedd26c60 2023-07-21T19:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedc7fac0 con 0x559eedd31b50 2023-07-21T19:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcfa859db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedc78f50 con 0x559eedd26350 2023-07-21T19:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.168+0000 7fcf8ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39148/0 (socket says 172.21.15.5:39148) 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 -- 172.21.15.5:0/966745679 learned_addr learned my addr 172.21.15.5:0/966745679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 -- 172.21.15.5:0/966745679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 msgr2=0x559eedd33f30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd33f30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 -- 172.21.15.5:0/966745679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 msgr2=0x559eedd31610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd31610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.169+0000 7fcf8ed9d700 1 -- 172.21.15.5:0/966745679 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eedaa6e60 con 0x559eedd26350 2023-07-21T19:28:21.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.169+0000 7ffb61ffb700 1 -- 172.21.15.5:0/2849444858 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ffb54011990 con 0x7ffb64108090 2023-07-21T19:28:21.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.170+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005cd0 tx=0x7fcf84009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4e7d9843f53a26c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.170+0000 7ffb7dd6db80 1 Processor -- start 2023-07-21T19:28:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.170+0000 7fcf7cff9700 1 -- 172.21.15.5:0/966745679 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf84005430 con 0x559eedd26350 2023-07-21T19:28:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.170+0000 7fcf7cff9700 1 -- 172.21.15.5:0/966745679 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf8401e040 con 0x559eedd26350 2023-07-21T19:28:21.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.170+0000 7fcf7cff9700 1 -- 172.21.15.5:0/966745679 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf840048c0 con 0x559eedd26350 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 -- start start 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2227ee6b0 0x55f222997d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f222897c50 con 0x55f2227ee6b0 2023-07-21T19:28:21.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2228ea9e0 con 0x55f2229982d0 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.171+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2228e3e70 con 0x55f222992580 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60312/0 (socket says 172.21.15.5:60312) 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 -- 172.21.15.5:0/68397091 learned_addr learned my addr 172.21.15.5:0/68397091 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 -- 172.21.15.5:0/68397091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 msgr2=0x55f2229929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 -- 172.21.15.5:0/68397091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2227ee6b0 msgr2=0x55f222997d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2227ee6b0 0x55f222997d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.172+0000 7ffb63fff700 1 -- 172.21.15.5:0/68397091 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f222711d80 con 0x55f2229982d0 2023-07-21T19:28:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.171+0000 7fcfa859db80 1 -- 172.21.15.5:0/966745679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 msgr2=0x559eedd26720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.171+0000 7fcfa859db80 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcf84005cd0 tx=0x7fcf84009ee0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.173+0000 7ffb63fff700 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009f50 tx=0x7ffb4c009f80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ab0342461a02a44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.173+0000 7ffb5a7fc700 1 -- 172.21.15.5:0/68397091 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c00bb30 con 0x55f2229982d0 2023-07-21T19:28:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.173+0000 7ffb61ffb700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 0x7ffb500a38f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.173+0000 7ffb5a7fc700 1 -- 172.21.15.5:0/68397091 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c01e040 con 0x55f2229982d0 2023-07-21T19:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.173+0000 7ffb5a7fc700 1 -- 172.21.15.5:0/68397091 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c004920 con 0x55f2229982d0 2023-07-21T19:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.174+0000 7ffb61ffb700 1 -- 172.21.15.5:0/2849444858 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb540cc400 con 0x7ffb64108090 2023-07-21T19:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.174+0000 7ffb6899c700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 0x7ffb500a38f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.174+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/68397091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 msgr2=0x55f222992040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.174+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009f50 tx=0x7ffb4c009f80 comp rx=0 tx=0).stop 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.173+0000 7fcfa859db80 1 -- 172.21.15.5:0/966745679 shutdown_connections 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.173+0000 7fcfa859db80 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd33f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.173+0000 7fcfa859db80 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd26720 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.173+0000 7fcfa859db80 1 --2- 172.21.15.5:0/966745679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd31610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.173+0000 7fcfa859db80 1 -- 172.21.15.5:0/966745679 >> 172.21.15.5:0/966745679 conn(0x559eedd1ce20 msgr2=0x559eedd22720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.174+0000 7fcfa859db80 1 -- 172.21.15.5:0/966745679 shutdown_connections 2023-07-21T19:28:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.174+0000 7fcfa859db80 1 -- 172.21.15.5:0/966745679 wait complete. 2023-07-21T19:28:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.174+0000 7fcf7f7fe700 1 -- 172.21.15.5:0/306712236 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcf88014440 con 0x7fcf90100490 2023-07-21T19:28:21.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.174+0000 7fcfa859db80 1 -- start start 2023-07-21T19:28:21.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/68397091 shutdown_connections 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2227ee6b0 0x55f222997d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/68397091 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2229982d0 0x55f222992040 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/68397091 >> 172.21.15.5:0/68397091 conn(0x55f222987be0 msgr2=0x55f22298cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/68397091 shutdown_connections 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.175+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/68397091 wait complete. 2023-07-21T19:28:21.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.176+0000 7ffb6899c700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 0x7ffb500a38f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffb641ab740 tx=0x7ffb5c009250 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.176+0000 7ffb7dd6db80 1 -- start start 2023-07-21T19:28:21.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.177+0000 7ffb7dd6db80 1 Processor -- start 2023-07-21T19:28:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.177+0000 7fcfa859db80 1 Processor -- start 2023-07-21T19:28:21.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd315e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd2efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedc18b40 con 0x559eedd26c60 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedc18840 con 0x559eedd31b50 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedafc9a0 con 0x559eedd26350 2023-07-21T19:28:21.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.178+0000 7fcfa859db80 10 client.0 _ll_register_callbacks cb 0x559eedc167e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.172 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2227ee6b0 0x55f22299e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f222992580 0x55f22299b910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f222883a60 con 0x55f2229982d0 2023-07-21T19:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f222883760 con 0x55f222992580 2023-07-21T19:28:21.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2227678c0 con 0x55f2227ee6b0 2023-07-21T19:28:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.178+0000 7ffb7dd6db80 10 client.0 _ll_register_callbacks cb 0x55f222881700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.179+0000 7ffb6899c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.179+0000 7ffb6899c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36318/0 (socket says 172.21.15.5:36318) 2023-07-21T19:28:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.179+0000 7ffb6899c700 1 -- 172.21.15.5:0/1536400089 learned_addr learned my addr 172.21.15.5:0/1536400089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.179+0000 7ffb63fff700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2227ee6b0 0x55f22299e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.180+0000 7ffb6899c700 1 -- 172.21.15.5:0/1536400089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2227ee6b0 msgr2=0x55f22299e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.180+0000 7ffb6899c700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2227ee6b0 0x55f22299e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.180+0000 7ffb6899c700 1 -- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f222992580 msgr2=0x55f22299b910 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:21.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.180+0000 7ffb6899c700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f222992580 0x55f22299b910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.180+0000 7ffb6899c700 1 -- 172.21.15.5:0/1536400089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb64113be0 con 0x55f2229982d0 2023-07-21T19:28:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.180+0000 7fcf8dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd2efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.176 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.180+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.180+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36326/0 (socket says 172.21.15.5:36326) 2023-07-21T19:28:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.180+0000 7fcf8e59c700 1 -- 172.21.15.5:0/1514830559 learned_addr learned my addr 172.21.15.5:0/1514830559 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.181+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd315e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.181+0000 7fcf7f7fe700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 0x7fcf780a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.181+0000 7ffb6899c700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00d6d0 tx=0x7ffb5c004cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1a94f9d9ecb566 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.181+0000 7ffb437fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.181+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 0x7fcf780a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 -- 172.21.15.5:0/1514830559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 msgr2=0x559eedd315e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf7f7fe700 1 -- 172.21.15.5:0/306712236 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcf880cc4e0 con 0x7fcf90100490 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd315e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 -- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 msgr2=0x559eedd2efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd2efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 -- 172.21.15.5:0/1514830559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf9010d220 con 0x559eedd26c60 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd2efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:21.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcf800103d0 tx=0x7fcf800106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0057943aec14d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.182+0000 7fcf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.183+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb5c01a830 con 0x55f2229982d0 2023-07-21T19:28:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.183+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb5c013070 con 0x55f2229982d0 2023-07-21T19:28:21.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.183+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f22283d490 con 0x55f2229982d0 2023-07-21T19:28:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.183+0000 7fcf8ed9d700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 0x7fcf780a3890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ce0 tx=0x7fcf8400b580 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.183+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80010ee0 con 0x559eedd26c60 2023-07-21T19:28:21.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.183+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcf80011440 con 0x559eedd26c60 2023-07-21T19:28:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.183+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559eedbd2570 con 0x559eedd26c60 2023-07-21T19:28:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.184+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf80011600 con 0x559eedd26c60 2023-07-21T19:28:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.184+0000 7fcfa859db80 10 client.19662 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.182 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.184+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559eeda8fce0 con 0x559eedd26c60 2023-07-21T19:28:21.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.184+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb5c01a990 con 0x55f2229982d0 2023-07-21T19:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.185+0000 7ffb7dd6db80 10 client.19659 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.185+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55f2226fac00 con 0x55f2229982d0 2023-07-21T19:28:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.184+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559eedbd2570 con 0x559eedd26c60 2023-07-21T19:28:21.183 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.184+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcf80024a00 con 0x559eedd26c60 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 10 client.19662 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 10 client.19662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 20 client.19662 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd41480 0x559eedd43890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 4 client.19662 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559eedc71f30 con 0x559eedd41480 2023-07-21T19:28:21.184 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.185+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd41480 0x559eedd43890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.185+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f22283d490 con 0x55f2229982d0 2023-07-21T19:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb5c020020 con 0x55f2229982d0 2023-07-21T19:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 10 client.19659 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 10 client.19659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 20 client.19659 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2229abc80 0x55f2229ae090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 4 client.19659 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.186+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2228dce50 con 0x55f2229abc80 2023-07-21T19:28:21.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.187+0000 7ffb63fff700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2229abc80 0x55f2229ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.186 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.187+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd41480 0x559eedd43890 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12bd5ce3d145dac8 server_cookie=94ee824919a1340 in_seq=0 out_seq=0 2023-07-21T19:28:21.187 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.187+0000 7fcf75ffb700 10 client.19662 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.187+0000 7ffb63fff700 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2229abc80 0x55f2229ae090 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da065774c49996f3 server_cookie=bb528345df099192 in_seq=0 out_seq=0 2023-07-21T19:28:21.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.188+0000 7ffb437fe700 10 client.19659 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.188+0000 7fcf75ffb700 1 -- 172.21.15.5:0/1514830559 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559eedc71f30 con 0x559eedd41480 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.188+0000 7fcf75ffb700 10 client.19662 handle_command_reply: tid=0 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.188+0000 7fcfa859db80 1 client.19662 shutdown 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 msgr2=0x559eedd2e860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fcf800103d0 tx=0x7fcf800106e0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 shutdown_connections 2023-07-21T19:28:21.188 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd31b50 0x559eedd2efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd26350 0x559eedd315e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd41480 0x559eedd43890 crc :-1 s=READY pgs=421 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 --2- 172.21.15.5:0/1514830559 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd2e860 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 >> 172.21.15.5:0/1514830559 conn(0x559eedd1ce20 msgr2=0x559eedd222b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.189 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.189+0000 7fcf75ffb700 0 client.19662 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.190+0000 7ffb437fe700 1 -- 172.21.15.5:0/1536400089 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f2228dce50 con 0x55f2229abc80 2023-07-21T19:28:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.190+0000 7ffb437fe700 10 client.19659 handle_command_reply: tid=0 2023-07-21T19:28:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.190+0000 7ffb7dd6db80 1 client.19659 shutdown 2023-07-21T19:28:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.191+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 msgr2=0x55f22299c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.191+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7ffb5c00d6d0 tx=0x7ffb5c004cc0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.190 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.190+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 shutdown_connections 2023-07-21T19:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.192+0000 7fcfa859db80 1 -- 172.21.15.5:0/1514830559 wait complete. 2023-07-21T19:28:21.191 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.192+0000 7fcfa859db80 20 client.19662 trim_cache size 0 max 16384 2023-07-21T19:28:21.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 shutdown_connections 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2229abc80 0x55f2229ae090 crc :-1 s=READY pgs=422 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f22299c000 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2227ee6b0 0x55f22299e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/1536400089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f222992580 0x55f22299b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 >> 172.21.15.5:0/1536400089 conn(0x55f222987be0 msgr2=0x55f22298bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.193+0000 7ffb437fe700 0 client.19659 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.194+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 shutdown_connections 2023-07-21T19:28:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.194+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/1536400089 wait complete. 2023-07-21T19:28:21.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.194+0000 7ffb7dd6db80 20 client.19659 trim_cache size 0 max 16384 2023-07-21T19:28:21.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.219+0000 7fcfa859db80 1 Processor -- start 2023-07-21T19:28:21.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.219+0000 7fcfa859db80 1 -- start start 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 0x559eedd38530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd28b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd2e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd29700 con 0x559eedd26c60 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd3ec00 con 0x559eedd26350 2023-07-21T19:28:21.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcfa859db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd271e0 con 0x559eedd31b50 2023-07-21T19:28:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcf8ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd2e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcf8ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd2e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39184/0 (socket says 172.21.15.5:39184) 2023-07-21T19:28:21.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.220+0000 7fcf8ed9d700 1 -- 172.21.15.5:0/2298763903 learned_addr learned my addr 172.21.15.5:0/2298763903 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 0x559eedd38530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 -- 172.21.15.5:0/2298763903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 msgr2=0x559eedd2e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd2e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 -- 172.21.15.5:0/2298763903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 msgr2=0x559eedd28b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd28b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.221+0000 7fcf8e59c700 1 -- 172.21.15.5:0/2298763903 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559eedacd2d0 con 0x559eedd26350 2023-07-21T19:28:21.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.222+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 0x559eedd38530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcf80024d90 tx=0x7fcf800057d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f53ad5fbfe95412 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.222+0000 7ffb7dd6db80 1 Processor -- start 2023-07-21T19:28:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.222+0000 7fcf7d7fa700 1 -- 172.21.15.5:0/2298763903 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80010ee0 con 0x559eedd26350 2023-07-21T19:28:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.223+0000 7fcf7d7fa700 1 -- 172.21.15.5:0/2298763903 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf80015070 con 0x559eedd26350 2023-07-21T19:28:21.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.223+0000 7fcf7d7fa700 1 -- 172.21.15.5:0/2298763903 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8001aa10 con 0x559eedd26350 2023-07-21T19:28:21.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.222+0000 7ffb7dd6db80 1 -- start start 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f222992580 0x55f22298cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2229982d0 0x55f22299e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f22299c3e0 con 0x55f222992580 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2229986d0 con 0x55f2227ee6b0 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.223+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f222989d50 con 0x55f2229982d0 2023-07-21T19:28:21.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 -- 172.21.15.5:0/2298763903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 msgr2=0x559eedd38530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 0x559eedd38530 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fcf80024d90 tx=0x7fcf800057d0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60344/0 (socket says 172.21.15.5:60344) 2023-07-21T19:28:21.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 -- 172.21.15.5:0/3727367595 learned_addr learned my addr 172.21.15.5:0/3727367595 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb6899c700 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2229982d0 0x55f22299e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 -- 172.21.15.5:0/2298763903 shutdown_connections 2023-07-21T19:28:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd2e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26350 0x559eedd38530 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 --2- 172.21.15.5:0/2298763903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd26c60 0x559eedd28b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 -- 172.21.15.5:0/2298763903 >> 172.21.15.5:0/2298763903 conn(0x559eedd1ce20 msgr2=0x559eedd28460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 -- 172.21.15.5:0/2298763903 shutdown_connections 2023-07-21T19:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 -- 172.21.15.5:0/3727367595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2229982d0 msgr2=0x55f22299e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2229982d0 0x55f22299e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 -- 172.21.15.5:0/3727367595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f222992580 msgr2=0x55f22298cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f222992580 0x55f22298cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.224+0000 7ffb63fff700 1 -- 172.21.15.5:0/3727367595 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2227381f0 con 0x55f2227ee6b0 2023-07-21T19:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.225+0000 7ffb63fff700 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006010 tx=0x7ffb4c01a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=900b4b61ead94fa0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.225+0000 7ffb5affd700 1 -- 172.21.15.5:0/3727367595 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c005630 con 0x55f2227ee6b0 2023-07-21T19:28:21.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.224+0000 7fcfa859db80 1 -- 172.21.15.5:0/2298763903 wait complete. 2023-07-21T19:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.225+0000 7fcfa859db80 1 -- start start 2023-07-21T19:28:21.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.225+0000 7fcfa859db80 1 Processor -- start 2023-07-21T19:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.225+0000 7ffb5affd700 1 -- 172.21.15.5:0/3727367595 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c00bbb0 con 0x55f2227ee6b0 2023-07-21T19:28:21.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.225+0000 7ffb5affd700 1 -- 172.21.15.5:0/3727367595 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c02f540 con 0x55f2227ee6b0 2023-07-21T19:28:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.226+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/3727367595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 msgr2=0x55f222997240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.226+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c006010 tx=0x7ffb4c01a040 comp rx=0 tx=0).stop 2023-07-21T19:28:21.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd37650 0x559eedd95310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd958f0 con 0x559eedd37650 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd95a30 con 0x559eedd26c60 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559eedd95bf0 con 0x559eedd31b50 2023-07-21T19:28:21.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcfa859db80 10 client.0 _ll_register_callbacks cb 0x559eedc167e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcf8e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60346/0 (socket says 172.21.15.5:60346) 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.226+0000 7fcf8e59c700 1 -- 172.21.15.5:0/665314375 learned_addr learned my addr 172.21.15.5:0/665314375 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 -- 172.21.15.5:0/665314375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 msgr2=0x559eedd370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 -- 172.21.15.5:0/665314375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd37650 msgr2=0x559eedd95310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd37650 0x559eedd95310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 -- 172.21.15.5:0/665314375 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559eeda8fce0 con 0x559eedd26c60 2023-07-21T19:28:21.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:28:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.227+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/3727367595 shutdown_connections 2023-07-21T19:28:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.227+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2229982d0 0x55f22299e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.227+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f222997240 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.227+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/3727367595 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f222992580 0x55f22298cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.227+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/3727367595 >> 172.21.15.5:0/3727367595 conn(0x55f222987be0 msgr2=0x55f222996b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf8e59c700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcf80024fd0 tx=0x7fcf800046b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35301e9eff94f0d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.227+0000 7fcf75ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.228+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/3727367595 shutdown_connections 2023-07-21T19:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.228+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/3727367595 wait complete. 2023-07-21T19:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.228+0000 7ffb7dd6db80 1 -- start start 2023-07-21T19:28:21.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.229+0000 7ffb7dd6db80 1 Processor -- start 2023-07-21T19:28:21.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.228+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf80024670 con 0x559eedd26c60 2023-07-21T19:28:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.228+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcf80029030 con 0x559eedd26c60 2023-07-21T19:28:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.228+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcf8401e040 con 0x559eedd26c60 2023-07-21T19:28:21.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.229+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcf800247d0 con 0x559eedd26c60 2023-07-21T19:28:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.229+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.229+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229a1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb7dd6db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f2229a16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2229a3250 con 0x55f2229982d0 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2229feb40 con 0x55f2227ee6b0 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb7dd6db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f222a0fc90 con 0x55f222992580 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb7dd6db80 10 client.0 _ll_register_callbacks cb 0x7ffb5c00cf70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb63fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60348/0 (socket says 172.21.15.5:60348) 2023-07-21T19:28:21.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb63fff700 1 -- 172.21.15.5:0/2250757165 learned_addr learned my addr 172.21.15.5:0/2250757165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.230+0000 7fcfa859db80 10 client.29360 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.230+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x559eedaa5330 con 0x559eedd26c60 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.230+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229a1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 -- 172.21.15.5:0/2250757165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 msgr2=0x55f2229a1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229a1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 -- 172.21.15.5:0/2250757165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 msgr2=0x55f2229a16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f2229a16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 -- 172.21.15.5:0/2250757165 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb4c027040 con 0x55f2227ee6b0 2023-07-21T19:28:21.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb63fff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009990 tx=0x7ffb4c009fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e4f2857f486e50b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.231+0000 7ffb437fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcf8401e040 con 0x559eedd26c60 2023-07-21T19:28:21.230 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fcf8002e480 con 0x559eedd26c60 2023-07-21T19:28:21.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.232+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c02fad0 con 0x55f2227ee6b0 2023-07-21T19:28:21.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.232+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb4c02fc30 con 0x55f2227ee6b0 2023-07-21T19:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 10 client.29360 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 10 client.29360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 20 client.29360 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.231 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd96ed0 0x559eedd992b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 4 client.29360 mds_command: new command op to 24614 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2023-07-21T19:28:21.232 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.232+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) v1 -- 0x559eedc340d0 con 0x559eedd96ed0 2023-07-21T19:28:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.232+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb4c01e040 con 0x55f2227ee6b0 2023-07-21T19:28:21.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.233+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb4c02fdf0 con 0x55f2227ee6b0 2023-07-21T19:28:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.233+0000 7ffb7dd6db80 10 client.29363 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.233+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x55f222710250 con 0x55f2227ee6b0 2023-07-21T19:28:21.233 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.233+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd96ed0 0x559eedd992b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.233+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb4c047030 con 0x55f2227ee6b0 2023-07-21T19:28:21.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.233+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7ffb4c00b260 con 0x55f2227ee6b0 2023-07-21T19:28:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.234+0000 7fcf8dd9b700 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd96ed0 0x559eedd992b0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e548e43693dd9894 server_cookie=1572cd9ff26c80b in_seq=0 out_seq=0 2023-07-21T19:28:21.234 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.234+0000 7fcf75ffb700 10 client.29360 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 10 client.29363 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 10 client.29363 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 20 client.29363 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f222a10ed0 0x55f222a132b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 4 client.29363 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f22289eff0 con 0x55f222a10ed0 2023-07-21T19:28:21.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.234+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f222a10ed0 0x55f222a132b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.235+0000 7fcf75ffb700 1 -- 172.21.15.5:0/665314375 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x559eedc340d0 con 0x559eedd96ed0 2023-07-21T19:28:21.236 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.235+0000 7fcf75ffb700 10 client.29360 handle_command_reply: tid=0 2023-07-21T19:28:21.237 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.235+0000 7fcfa859db80 1 client.29360 shutdown 2023-07-21T19:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.235+0000 7ffb437fe700 10 client.29363 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.235+0000 7ffb5bfff700 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f222a10ed0 0x55f222a132b0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d35b3ccc487fa1e7 server_cookie=2097b11d8b7befc2 in_seq=0 out_seq=0 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 msgr2=0x559eedd36b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fcf80024fd0 tx=0x7fcf800046b0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 shutdown_connections 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559eedd96ed0 0x559eedd992b0 crc :-1 s=READY pgs=423 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559eedd37650 0x559eedd95310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559eedd26c60 0x559eedd36b80 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.238 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 --2- 172.21.15.5:0/665314375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559eedd31b50 0x559eedd370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 >> 172.21.15.5:0/665314375 conn(0x559eedd1ce20 msgr2=0x559eedd28cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcf75ffb700 0 client.29360 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.236+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 shutdown_connections 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.237+0000 7fcfa859db80 1 -- 172.21.15.5:0/665314375 wait complete. 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.237+0000 7fcfa859db80 20 client.29360 trim_cache size 0 max 16384 2023-07-21T19:28:21.239 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:28:21.240 INFO:teuthology.orchestra.run.smithi005.stdout: "return_code": 0, 2023-07-21T19:28:21.240 INFO:teuthology.orchestra.run.smithi005.stdout: "scrub_tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:28:21.240 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": "asynchronous" 2023-07-21T19:28:21.240 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:28:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 msgr2=0x7fcf780a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 0x7fcf780a3890 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fcf84009ce0 tx=0x7fcf8400b580 comp rx=0 tx=0).stop 2023-07-21T19:28:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 msgr2=0x7fcf90198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.241 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf90198bb0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fcf8800bd50 tx=0x7fcf8800bd80 comp rx=0 tx=0).stop 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 shutdown_connections 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcf780a13e0 0x7fcf780a3890 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcf90105af0 0x7fcf901937f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcf900ffae0 0x7fcf90198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 --2- 172.21.15.5:0/306712236 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcf90100490 0x7fcf90198bb0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 >> 172.21.15.5:0/306712236 conn(0x7fcf900fb5e0 msgr2=0x7fcf900fbb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.242 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 shutdown_connections 2023-07-21T19:28:21.243 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.238+0000 7fcf94bff700 1 -- 172.21.15.5:0/306712236 wait complete. 2023-07-21T19:28:21.258 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:28:21.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.288+0000 7ffb437fe700 1 -- 172.21.15.5:0/2250757165 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f22289eff0 con 0x55f222a10ed0 2023-07-21T19:28:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.288+0000 7ffb437fe700 10 client.29363 handle_command_reply: tid=0 2023-07-21T19:28:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.288+0000 7ffb7dd6db80 1 client.29363 shutdown 2023-07-21T19:28:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.288+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 msgr2=0x55f2229a0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.288+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c009990 tx=0x7ffb4c009fd0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 shutdown_connections 2023-07-21T19:28:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f222a10ed0 0x55f222a132b0 crc :-1 s=READY pgs=424 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2229982d0 0x55f2229a16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2227ee6b0 0x55f2229a0c50 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb437fe700 0 client.29363 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 --2- 172.21.15.5:0/2250757165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f222992580 0x55f2229a1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 >> 172.21.15.5:0/2250757165 conn(0x55f222987be0 msgr2=0x55f222998840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 shutdown_connections 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 1 -- 172.21.15.5:0/2250757165 wait complete. 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.289+0000 7ffb7dd6db80 20 client.29363 trim_cache size 0 max 16384 2023-07-21T19:28:21.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:28:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:28:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:28:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:28:21.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.290+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 msgr2=0x7ffb500a38f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.290+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 0x7ffb500a38f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7ffb641ab740 tx=0x7ffb5c009250 comp rx=0 tx=0).stop 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.290+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 msgr2=0x7ffb641aaf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.290+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400b550 tx=0x7ffb5400b910 comp rx=0 tx=0).stop 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 shutdown_connections 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffb500a1440 0x7ffb500a38f0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb64110b50 0x7ffb641a48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb64107780 0x7ffb641aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 --2- 172.21.15.5:0/2849444858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb64108090 0x7ffb641aaf80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 >> 172.21.15.5:0/2849444858 conn(0x7ffb6406c270 msgr2=0x7ffb6406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 shutdown_connections 2023-07-21T19:28:21.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:21.291+0000 7ffb6a3ff700 1 -- 172.21.15.5:0/2849444858 wait complete. 2023-07-21T19:28:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 -- 172.21.15.5:0/832771128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc1059e0 msgr2=0x7f6edc105db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 --2- 172.21.15.5:0/832771128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc1059e0 0x7f6edc105db0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009990 tx=0x7f6ecc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 -- 172.21.15.5:0/832771128 shutdown_connections 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 --2- 172.21.15.5:0/832771128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 --2- 172.21.15.5:0/832771128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc1059e0 0x7f6edc105db0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 --2- 172.21.15.5:0/832771128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc0ff9d0 0x7f6edc0ffe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 -- 172.21.15.5:0/832771128 >> 172.21.15.5:0/832771128 conn(0x7f6edc0fb4b0 msgr2=0x7f6edc0fd8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 -- 172.21.15.5:0/832771128 shutdown_connections 2023-07-21T19:28:21.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.591+0000 7f6edb59e700 1 -- 172.21.15.5:0/832771128 wait complete. 2023-07-21T19:28:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.593+0000 7f6edb59e700 1 Processor -- start 2023-07-21T19:28:21.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.593+0000 7f6edb59e700 1 -- start start 2023-07-21T19:28:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.593+0000 7f6edb59e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc0ff9d0 0x7f6edc10c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edb59e700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edb59e700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edb59e700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc109a90 con 0x7f6edc100380 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edb59e700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc109c00 con 0x7f6edc1059e0 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edb59e700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6edc109dc0 con 0x7f6edc0ff9d0 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60352/0 (socket says 172.21.15.5:60352) 2023-07-21T19:28:21.585 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6edad9d700 1 -- 172.21.15.5:0/2047688632 learned_addr learned my addr 172.21.15.5:0/2047688632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6ed9d9b700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.594+0000 7f6eda59c700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc0ff9d0 0x7f6edc10c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 -- 172.21.15.5:0/2047688632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc0ff9d0 msgr2=0x7f6edc10c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc0ff9d0 0x7f6edc10c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 -- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 msgr2=0x7f6edc10c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 -- 172.21.15.5:0/2047688632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ecc0095f0 con 0x7f6edc1059e0 2023-07-21T19:28:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6ed9d9b700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.595+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400ef30 tx=0x7f6ed400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.596+0000 7f6ecb7fe700 1 -- 172.21.15.5:0/2047688632 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4018070 con 0x7f6edc1059e0 2023-07-21T19:28:21.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.596+0000 7f6ecb7fe700 1 -- 172.21.15.5:0/2047688632 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ed400f040 con 0x7f6edc1059e0 2023-07-21T19:28:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.596+0000 7f6ecb7fe700 1 -- 172.21.15.5:0/2047688632 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed400cbc0 con 0x7f6edc1059e0 2023-07-21T19:28:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.596+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6edc10a040 con 0x7f6edc1059e0 2023-07-21T19:28:21.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.596+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc1125a0 con 0x7f6edc1059e0 2023-07-21T19:28:21.590 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.600+0000 7f6ecb7fe700 1 -- 172.21.15.5:0/2047688632 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6ed4007500 con 0x7f6edc1059e0 2023-07-21T19:28:21.592 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.603+0000 7f6ef448eb80 1 Processor -- start 2023-07-21T19:28:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.603+0000 7f6ecb7fe700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 0x7f6ec40a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.603+0000 7f6ef448eb80 1 -- start start 2023-07-21T19:28:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.603+0000 7f6ecb7fe700 1 -- 172.21.15.5:0/2047688632 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ed40cc690 con 0x7f6edc1059e0 2023-07-21T19:28:21.593 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.604+0000 7f6eda59c700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 0x7f6ec40a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.604+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 0x555fd165d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.595 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165de00 0x555fd16677f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 0x555fd166a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd1562cb0 con 0x555fd165de00 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd15b5a40 con 0x555fd165d4f0 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd15aeed0 con 0x555fd1667d30 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 0x555fd165d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 0x555fd166a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 0x555fd165d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60354/0 (socket says 172.21.15.5:60354) 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6eda59c700 1 -- 172.21.15.5:0/1261994128 learned_addr learned my addr 172.21.15.5:0/1261994128 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 -- 172.21.15.5:0/1261994128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 msgr2=0x555fd165d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 0x555fd165d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 -- 172.21.15.5:0/1261994128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165de00 msgr2=0x555fd16677f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165de00 0x555fd16677f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.605+0000 7f6edad9d700 1 -- 172.21.15.5:0/1261994128 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd13dcde0 con 0x555fd1667d30 2023-07-21T19:28:21.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.606+0000 7f6edad9d700 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 0x555fd166a110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9d80 tx=0x7f6ed4004cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b58d8bade9e664 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.606+0000 7f6eda59c700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 0x7f6ec40a3890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009990 tx=0x7f6ecc005490 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.606+0000 7f6ec8ff9700 1 -- 172.21.15.5:0/1261994128 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed409d7c0 con 0x555fd1667d30 2023-07-21T19:28:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.607+0000 7f6ec8ff9700 1 -- 172.21.15.5:0/1261994128 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ed40a1b50 con 0x555fd1667d30 2023-07-21T19:28:21.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.607+0000 7f6ec8ff9700 1 -- 172.21.15.5:0/1261994128 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40999f0 con 0x555fd1667d30 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1261994128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 msgr2=0x555fd166a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 0x555fd166a110 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f6ed40c9d80 tx=0x7f6ed4004cd0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1261994128 shutdown_connections 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1667d30 0x555fd166a110 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165d4f0 0x555fd165d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1261994128 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165de00 0x555fd16677f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1261994128 >> 172.21.15.5:0/1261994128 conn(0x555fd1652ff0 msgr2=0x555fd1658930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1261994128 shutdown_connections 2023-07-21T19:28:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.608+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1261994128 wait complete. 2023-07-21T19:28:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.609+0000 7f6ef448eb80 1 -- start start 2023-07-21T19:28:21.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.609+0000 7f6ef448eb80 1 Processor -- start 2023-07-21T19:28:21.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1667790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165de00 0x555fd16649a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd154eac0 con 0x555fd1667d30 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd154e7c0 con 0x555fd165de00 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd1432920 con 0x555fd165d4f0 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6ef448eb80 10 client.0 _ll_register_callbacks cb 0x555fd154c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.610+0000 7f6edad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36402/0 (socket says 172.21.15.5:36402) 2023-07-21T19:28:21.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 -- 172.21.15.5:0/2399180541 learned_addr learned my addr 172.21.15.5:0/2399180541 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1667790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6ed9d9b700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165de00 0x555fd16649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 -- 172.21.15.5:0/2399180541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 msgr2=0x555fd1667790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1667790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 -- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165de00 msgr2=0x555fd16649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165de00 0x555fd16649a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 -- 172.21.15.5:0/2399180541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6edc1125a0 con 0x555fd1667d30 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c6f0 tx=0x7f6ed40998c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c1b58193e35a0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.611+0000 7f6ec1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40b4060 con 0x555fd1667d30 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ed4097bf0 con 0x555fd1667d30 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555fd15084f0 con 0x555fd1667d30 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ed4098e80 con 0x555fd1667d30 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ef448eb80 10 client.19671 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.612+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x555fd13c5c60 con 0x555fd1667d30 2023-07-21T19:28:21.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555fd15084f0 con 0x555fd1667d30 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6ed4095580 con 0x555fd1667d30 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 10 client.19671 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 10 client.19671 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 20 client.19671 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd1677310 0x555fd16796f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 4 client.19671 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:21.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.613+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555fd15a7eb0 con 0x555fd1677310 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.614+0000 7f6eda59c700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd1677310 0x555fd16796f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.615+0000 7f6eda59c700 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd1677310 0x555fd16796f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1efdeda8e63f48d server_cookie=f63ca14434107a04 in_seq=0 out_seq=0 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.615+0000 7f6ec1ffb700 10 client.19671 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.616+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/2399180541 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555fd15a7eb0 con 0x555fd1677310 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.616+0000 7f6ec1ffb700 10 client.19671 handle_command_reply: tid=0 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.616+0000 7f6ef448eb80 1 client.19671 shutdown 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 msgr2=0x555fd16650e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400c6f0 tx=0x7f6ed40998c0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 shutdown_connections 2023-07-21T19:28:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd1677310 0x555fd16796f0 crc :-1 s=READY pgs=425 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1667d30 0x555fd16650e0 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1667790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2399180541 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd165de00 0x555fd16649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ec1ffb700 0 client.19671 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.617+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 >> 172.21.15.5:0/2399180541 conn(0x555fd1652ff0 msgr2=0x555fd1658520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.618+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 shutdown_connections 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.618+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2399180541 wait complete. 2023-07-21T19:28:21.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.618+0000 7f6ef448eb80 20 client.19671 trim_cache size 0 max 16384 2023-07-21T19:28:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.647+0000 7f6ef448eb80 1 Processor -- start 2023-07-21T19:28:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.647+0000 7f6ef448eb80 1 -- start start 2023-07-21T19:28:21.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 0x555fd16672c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1652ff0 0x555fd1664240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1664780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd1652b70 con 0x555fd1652ff0 2023-07-21T19:28:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd165f590 con 0x555fd1665250 2023-07-21T19:28:21.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.648+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd1664cc0 con 0x555fd165d4f0 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 0x555fd16672c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6ed9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1652ff0 0x555fd1664240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 0x555fd16672c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60382/0 (socket says 172.21.15.5:60382) 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6eda59c700 1 -- 172.21.15.5:0/2903997670 learned_addr learned my addr 172.21.15.5:0/2903997670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1664780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6edad9d700 1 -- 172.21.15.5:0/2903997670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 msgr2=0x555fd16672c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.649+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 0x555fd16672c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.650+0000 7f6edad9d700 1 -- 172.21.15.5:0/2903997670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1652ff0 msgr2=0x555fd1664240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.650+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1652ff0 0x555fd1664240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.650+0000 7f6edad9d700 1 -- 172.21.15.5:0/2903997670 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555fd1403250 con 0x555fd165d4f0 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.650+0000 7f6edad9d700 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1664780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4004db0 tx=0x7f6ed4082f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=16c99cbec17d653d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.650+0000 7f6ec97fa700 1 -- 172.21.15.5:0/2903997670 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed40b4070 con 0x555fd165d4f0 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.651+0000 7f6ec97fa700 1 -- 172.21.15.5:0/2903997670 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ed4097bf0 con 0x555fd165d4f0 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.651+0000 7f6ec97fa700 1 -- 172.21.15.5:0/2903997670 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed4099c50 con 0x555fd165d4f0 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.651+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2903997670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 msgr2=0x555fd1664780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.651+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1664780 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f6ed4004db0 tx=0x7f6ed4082f90 comp rx=0 tx=0).stop 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2903997670 shutdown_connections 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd165d4f0 0x555fd1664780 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1665250 0x555fd16672c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/2903997670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd1652ff0 0x555fd1664240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2903997670 >> 172.21.15.5:0/2903997670 conn(0x555fd166a3e0 msgr2=0x555fd1673ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2903997670 shutdown_connections 2023-07-21T19:28:21.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 -- 172.21.15.5:0/2903997670 wait complete. 2023-07-21T19:28:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.652+0000 7f6ef448eb80 1 -- start start 2023-07-21T19:28:21.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 Processor -- start 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165d4f0 0x555fd166cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1665250 0x555fd166d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd166d6e0 con 0x555fd165d4f0 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd166d850 con 0x555fd1652ff0 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555fd16cb1b0 con 0x555fd1665250 2023-07-21T19:28:21.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.653+0000 7f6ef448eb80 10 client.0 _ll_register_callbacks cb 0x7f6ed40097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.654+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.654+0000 7f6ed9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165d4f0 0x555fd166cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.654+0000 7f6eda59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60390/0 (socket says 172.21.15.5:60390) 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.654+0000 7f6eda59c700 1 -- 172.21.15.5:0/1776312181 learned_addr learned my addr 172.21.15.5:0/1776312181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.654+0000 7f6edad9d700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1665250 0x555fd166d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 -- 172.21.15.5:0/1776312181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1665250 msgr2=0x555fd166d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1665250 0x555fd166d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 -- 172.21.15.5:0/1776312181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165d4f0 msgr2=0x555fd166cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165d4f0 0x555fd166cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 -- 172.21.15.5:0/1776312181 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555fd13c5c60 con 0x555fd1652ff0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6eda59c700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005910 tx=0x7f6ecc009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a70ade8adcc8cd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.655+0000 7f6ec1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.656+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ecc036610 con 0x555fd1652ff0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.656+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6ecc0369f0 con 0x555fd1652ff0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.656+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ed40a1b50 con 0x555fd1652ff0 2023-07-21T19:28:21.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.656+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ecc036bb0 con 0x555fd1652ff0 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 10 client.29381 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x555fd13db2b0 con 0x555fd1652ff0 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6ed40a1b50 con 0x555fd1652ff0 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f6ecc029470 con 0x555fd1652ff0 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 10 client.29381 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 10 client.29381 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 20 client.29381 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:21.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd165de00 0x555fd16ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 4 client.29381 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:28:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.657+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555fd156a050 con 0x555fd165de00 2023-07-21T19:28:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.658+0000 7f6ed9d9b700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd165de00 0x555fd16ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:21.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.659+0000 7f6ec1ffb700 10 client.29381 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.660+0000 7f6ed9d9b700 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd165de00 0x555fd16ce7b0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16c54d078ba9d13 server_cookie=bbb73fdf08aa6133 in_seq=0 out_seq=0 2023-07-21T19:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.661+0000 7f6ec1ffb700 1 -- 172.21.15.5:0/1776312181 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555fd156a050 con 0x555fd165de00 2023-07-21T19:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.661+0000 7f6ec1ffb700 10 client.29381 handle_command_reply: tid=0 2023-07-21T19:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.661+0000 7f6ef448eb80 1 client.29381 shutdown 2023-07-21T19:28:21.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.661+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 msgr2=0x555fd166e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.661+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc005910 tx=0x7f6ecc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 shutdown_connections 2023-07-21T19:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555fd1665250 0x555fd166d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555fd1652ff0 0x555fd166e850 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555fd165de00 0x555fd16ce7b0 crc :-1 s=READY pgs=426 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 --2- 172.21.15.5:0/1776312181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555fd165d4f0 0x555fd166cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 >> 172.21.15.5:0/1776312181 conn(0x555fd166a3e0 msgr2=0x555fd1664240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ec1ffb700 0 client.29381 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 shutdown_connections 2023-07-21T19:28:21.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.662+0000 7f6ef448eb80 1 -- 172.21.15.5:0/1776312181 wait complete. 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.663+0000 7f6ef448eb80 20 client.29381 trim_cache size 0 max 16384 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:28:21.653 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:28:21.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:28:21.654 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:28:21.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:28:21.654 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:28:21.654 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:28:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.665+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 msgr2=0x7f6ec40a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.665+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 0x7f6ec40a3890 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6ecc009990 tx=0x7f6ecc005490 comp rx=0 tx=0).stop 2023-07-21T19:28:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.665+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 msgr2=0x7f6edc1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.665+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f6ed400ef30 tx=0x7f6ed400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 shutdown_connections 2023-07-21T19:28:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6edc1059e0 0x7f6edc1092c0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6edc0ff9d0 0x7f6edc10c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6ec40a13e0 0x7f6ec40a3890 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 --2- 172.21.15.5:0/2047688632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6edc100380 0x7f6edc10c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:21.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.666+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 >> 172.21.15.5:0/2047688632 conn(0x7f6edc0fb4b0 msgr2=0x7f6edc0fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.669+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 shutdown_connections 2023-07-21T19:28:21.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:21.671+0000 7f6edb59e700 1 -- 172.21.15.5:0/2047688632 wait complete. 2023-07-21T19:28:21.688 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:28:21.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 110.0 s, 378.4 tps, lat 262.312 ms stddev 80.839 2023-07-21T19:28:22.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:21 smithi101 ceph-mon[128136]: pgmap v655: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:21 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:21 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:21 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:22.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:21 smithi005 ceph-mon[126460]: pgmap v655: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:22.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:21 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:22.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:21 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:22.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:21 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:22.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:21 smithi156 ceph-mon[128316]: pgmap v655: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 436 KiB/s rd, 8.7 MiB/s wr, 605 op/s 2023-07-21T19:28:22.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:21 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:22.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:21 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:22.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:21 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub queued for path: / 2023-07-21T19:28:23.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:23.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:23.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:22 smithi101 ceph-mon[128136]: pgmap v656: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 8.4 MiB/s wr, 663 op/s 2023-07-21T19:28:23.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub queued for path: / 2023-07-21T19:28:23.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:23.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:23.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:22 smithi005 ceph-mon[126460]: pgmap v656: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 8.4 MiB/s wr, 663 op/s 2023-07-21T19:28:23.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub queued for path: / 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:23.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:22 smithi156 ceph-mon[128316]: pgmap v656: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 545 KiB/s rd, 8.4 MiB/s wr, 663 op/s 2023-07-21T19:28:25.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:25 smithi101 ceph-mon[128136]: pgmap v657: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 7.2 MiB/s wr, 564 op/s 2023-07-21T19:28:25.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:25 smithi005 ceph-mon[126460]: pgmap v657: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 7.2 MiB/s wr, 564 op/s 2023-07-21T19:28:25.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:25 smithi156 ceph-mon[128316]: pgmap v657: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 464 KiB/s rd, 7.2 MiB/s wr, 564 op/s 2023-07-21T19:28:26.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:26 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:28:26.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:26 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:28:26.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:26 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:28:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 115.0 s, 56.4 tps, lat 1683.805 ms stddev 2018.022 2023-07-21T19:28:27.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:27 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:27.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:27 smithi101 ceph-mon[128136]: scrub summary: active paths [/] 2023-07-21T19:28:27.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:27 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:27.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:27 smithi101 ceph-mon[128136]: pgmap v658: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.0 MiB/s wr, 552 op/s 2023-07-21T19:28:27.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:27 smithi101 ceph-mon[128136]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:27 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:27 smithi005 ceph-mon[126460]: scrub summary: active paths [/] 2023-07-21T19:28:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:27 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:27 smithi005 ceph-mon[126460]: pgmap v658: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.0 MiB/s wr, 552 op/s 2023-07-21T19:28:27.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:27 smithi005 ceph-mon[126460]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:27.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:27 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:27 smithi156 ceph-mon[128316]: scrub summary: active paths [/] 2023-07-21T19:28:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:27 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:27 smithi156 ceph-mon[128316]: pgmap v658: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 7.0 MiB/s wr, 552 op/s 2023-07-21T19:28:27.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:27 smithi156 ceph-mon[128316]: scrub summary: idle+waiting paths [/] 2023-07-21T19:28:29.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:29 smithi156 ceph-mon[128316]: pgmap v659: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-07-21T19:28:30.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:29 smithi101 ceph-mon[128136]: pgmap v659: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-07-21T19:28:30.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:29 smithi005 ceph-mon[126460]: pgmap v659: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 507 KiB/s rd, 7.8 MiB/s wr, 645 op/s 2023-07-21T19:28:31.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: pgmap v660: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2023-07-21T19:28:31.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:28:31.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:30 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:28:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: pgmap v660: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:28:31.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:30 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:28:31.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: pgmap v660: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 327 KiB/s rd, 5.7 MiB/s wr, 520 op/s 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:28:31.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:30 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:28:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 120.0 s, 66.4 tps, lat 1605.300 ms stddev 2161.523 2023-07-21T19:28:32.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.071 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.072 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:28:32.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:28:32.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.074 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.075 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:28:32.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:28:32.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.150 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.151 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:28:32.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:32.161 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:28:33.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:33.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:32 smithi101 ceph-mon[128136]: pgmap v661: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.8 MiB/s wr, 594 op/s 2023-07-21T19:28:33.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:28:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:28:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:28:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:28:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:33.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:32 smithi005 ceph-mon[126460]: pgmap v661: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.8 MiB/s wr, 594 op/s 2023-07-21T19:28:33.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:28:33.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:28:33.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:28:33.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:28:33.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:28:33.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:32 smithi156 ceph-mon[128316]: pgmap v661: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 393 KiB/s rd, 6.8 MiB/s wr, 594 op/s 2023-07-21T19:28:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 -- 172.21.15.5:0/2288001013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b70105b60 msgr2=0x7f4b70105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 --2- 172.21.15.5:0/2288001013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b70105b60 0x7f4b70105f30 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f4b600098f0 tx=0x7f4b60009c00 comp rx=0 tx=0).stop 2023-07-21T19:28:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 -- 172.21.15.5:0/2288001013 shutdown_connections 2023-07-21T19:28:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 --2- 172.21.15.5:0/2288001013 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701004e0 0x7f4b7010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 --2- 172.21.15.5:0/2288001013 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b70105b60 0x7f4b70105f30 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 --2- 172.21.15.5:0/2288001013 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b700ffb30 0x7f4b700fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.534+0000 7f4b765ff700 1 -- 172.21.15.5:0/2288001013 >> 172.21.15.5:0/2288001013 conn(0x7f4b700fb650 msgr2=0x7f4b700fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.535+0000 7f4b765ff700 1 -- 172.21.15.5:0/2288001013 shutdown_connections 2023-07-21T19:28:34.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.535+0000 7f4b765ff700 1 -- 172.21.15.5:0/2288001013 wait complete. 2023-07-21T19:28:34.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.537+0000 7f4b765ff700 1 Processor -- start 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.537+0000 7f4b765ff700 1 -- start start 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b700ffb30 0x7f4b70109090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b701004e0 0x7f4b701095f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701a2d20 con 0x7f4b700ffb30 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701a2e60 con 0x7f4b701a2280 2023-07-21T19:28:34.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b765ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b701a2ff0 con 0x7f4b701004e0 2023-07-21T19:28:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.538+0000 7f4b74b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48620/0 (socket says 172.21.15.5:48620) 2023-07-21T19:28:34.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 -- 172.21.15.5:0/737899956 learned_addr learned my addr 172.21.15.5:0/737899956 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b701004e0 0x7f4b701095f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 -- 172.21.15.5:0/737899956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b701004e0 msgr2=0x7f4b701095f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b701004e0 0x7f4b701095f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 -- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b700ffb30 msgr2=0x7f4b70109090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b700ffb30 0x7f4b70109090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.539+0000 7f4b74b9c700 1 -- 172.21.15.5:0/737899956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b60009520 con 0x7f4b701a2280 2023-07-21T19:28:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.540+0000 7f4b74b9c700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c110 tx=0x7f4b6800c4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.540+0000 7f4b6d7fa700 1 -- 172.21.15.5:0/737899956 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68017070 con 0x7f4b701a2280 2023-07-21T19:28:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.540+0000 7f4b6d7fa700 1 -- 172.21.15.5:0/737899956 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6800e050 con 0x7f4b701a2280 2023-07-21T19:28:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.540+0000 7f4b6d7fa700 1 -- 172.21.15.5:0/737899956 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b68007950 con 0x7f4b701a2280 2023-07-21T19:28:34.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.540+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b70069f30 con 0x7f4b701a2280 2023-07-21T19:28:34.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.541+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b7006a3f0 con 0x7f4b701a2280 2023-07-21T19:28:34.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.546+0000 7f4b6d7fa700 1 -- 172.21.15.5:0/737899956 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4b68004ad0 con 0x7f4b701a2280 2023-07-21T19:28:34.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.547+0000 7f4b89f0ab80 1 Processor -- start 2023-07-21T19:28:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 -- start start 2023-07-21T19:28:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a871b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a871b580 0x5648a871b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8620c50 con 0x5648a87212d0 2023-07-21T19:28:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a86739e0 con 0x5648a871b580 2023-07-21T19:28:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.548+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a866ce70 con 0x5648a85776b0 2023-07-21T19:28:34.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53420/0 (socket says 172.21.15.5:53420) 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3405011131 learned_addr learned my addr 172.21.15.5:0/3405011131 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a871b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6d7fa700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 0x7f4b580a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a871b580 0x5648a871b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.549+0000 7f4b6d7fa700 1 -- 172.21.15.5:0/737899956 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b680cc890 con 0x7f4b701a2280 2023-07-21T19:28:34.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3405011131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a871b580 msgr2=0x5648a871b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a871b580 0x5648a871b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3405011131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 msgr2=0x5648a871b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a871b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3405011131 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a849ad80 con 0x5648a85776b0 2023-07-21T19:28:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b74b9c700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 0x7f4b580a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.550+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b640148d0 tx=0x7f4b640096a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f56d968805dc0b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.551+0000 7f4b567fc700 1 -- 172.21.15.5:0/3405011131 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640142d0 con 0x5648a85776b0 2023-07-21T19:28:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.551+0000 7f4b567fc700 1 -- 172.21.15.5:0/3405011131 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b6401a070 con 0x5648a85776b0 2023-07-21T19:28:34.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.551+0000 7f4b567fc700 1 -- 172.21.15.5:0/3405011131 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64003dc0 con 0x5648a85776b0 2023-07-21T19:28:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3405011131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 msgr2=0x5648a8720d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f4b640148d0 tx=0x7f4b640096a0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b74b9c700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 0x7f4b580a38c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4b680a1880 tx=0x7f4b68006240 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3405011131 shutdown_connections 2023-07-21T19:28:34.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a871b580 0x5648a871b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a85776b0 0x5648a8720d90 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3405011131 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a871b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.552+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3405011131 >> 172.21.15.5:0/3405011131 conn(0x5648a8710be0 msgr2=0x5648a8715f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.543 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.553+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3405011131 shutdown_connections 2023-07-21T19:28:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.553+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3405011131 wait complete. 2023-07-21T19:28:34.544 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.553+0000 7f4b89f0ab80 1 -- start start 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.554+0000 7f4b89f0ab80 1 Processor -- start 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.554+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a8724940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.554+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a87253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a860ca60 con 0x5648a87212d0 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a860c760 con 0x5648a85776b0 2023-07-21T19:28:34.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a84f08c0 con 0x5648a871b580 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b89f0ab80 10 client.0 _ll_register_callbacks cb 0x5648a860a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53422/0 (socket says 172.21.15.5:53422) 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 -- 172.21.15.5:0/822163392 learned_addr learned my addr 172.21.15.5:0/822163392 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a8724940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 -- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 msgr2=0x5648a8724940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a8724940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6ffff700 1 -- 172.21.15.5:0/822163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 msgr2=0x5648a87253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.555+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a87253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a87253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b6ffff700 1 -- 172.21.15.5:0/822163392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b640145b0 con 0x5648a871b580 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a87253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b600098c0 tx=0x7f4b6000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cbb59ad950bf2f2b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000b970 con 0x5648a871b580 2023-07-21T19:28:34.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.556+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b6000f8b0 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5648a85c6490 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b6000bec0 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b89f0ab80 10 client.29314 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5648a8483c00 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648a85c6490 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.557+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4b60029070 con 0x5648a871b580 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 10 client.29314 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:34.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 10 client.29314 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 20 client.29314 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a8734ba0 0x5648a8736fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 4 client.29314 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5648a8665e50 con 0x5648a8734ba0 2023-07-21T19:28:34.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.558+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a8734ba0 0x5648a8736fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.559+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a8734ba0 0x5648a8736fb0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc6b5c55674b3fad server_cookie=d65d78c94bd7731f in_seq=0 out_seq=0 2023-07-21T19:28:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.559+0000 7f4b4f7fe700 10 client.29314 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.560+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/822163392 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7f4b6402a780 con 0x5648a8734ba0 2023-07-21T19:28:34.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.560+0000 7f4b4f7fe700 10 client.29314 handle_command_reply: tid=0 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.560+0000 7f4b89f0ab80 1 client.29314 shutdown 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 msgr2=0x5648a8724e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4b600098c0 tx=0x7f4b6000f690 comp rx=0 tx=0).stop 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 shutdown_connections 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a8734ba0 0x5648a8736fb0 crc :-1 s=READY pgs=427 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a87253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a8724940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/822163392 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a871b580 0x5648a8724e80 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 >> 172.21.15.5:0/822163392 conn(0x5648a8710be0 msgr2=0x5648a87154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b4f7fe700 0 client.29314 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 shutdown_connections 2023-07-21T19:28:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/822163392 wait complete. 2023-07-21T19:28:34.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.561+0000 7f4b89f0ab80 20 client.29314 trim_cache size 0 max 16384 2023-07-21T19:28:34.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.589+0000 7f4b89f0ab80 1 Processor -- start 2023-07-21T19:28:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.589+0000 7f4b89f0ab80 1 -- start start 2023-07-21T19:28:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.589+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a85776b0 0x5648a871b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.589+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 0x5648a872bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 0x5648a8720230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a87279d0 con 0x5648a85776b0 2023-07-21T19:28:34.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a87216a0 con 0x5648a8727260 2023-07-21T19:28:34.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8730aa0 con 0x5648a87212d0 2023-07-21T19:28:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 0x5648a872bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.590+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a85776b0 0x5648a871b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.591+0000 7f4b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 0x5648a872bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53434/0 (socket says 172.21.15.5:53434) 2023-07-21T19:28:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.591+0000 7f4b6ffff700 1 -- 172.21.15.5:0/422665020 learned_addr learned my addr 172.21.15.5:0/422665020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:34.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.591+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 0x5648a8720230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.592+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/422665020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 msgr2=0x5648a872bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.592+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 0x5648a872bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.592+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/422665020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a85776b0 msgr2=0x5648a871b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.592+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a85776b0 0x5648a871b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.592+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/422665020 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5648a84c11f0 con 0x5648a8727260 2023-07-21T19:28:34.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.593+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 0x5648a8720230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009c50 tx=0x7f4b64021040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=800035589b5e03ec server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.593+0000 7f4b56ffd700 1 -- 172.21.15.5:0/422665020 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64014110 con 0x5648a8727260 2023-07-21T19:28:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.593+0000 7f4b56ffd700 1 -- 172.21.15.5:0/422665020 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5648a8665e50 con 0x5648a8727260 2023-07-21T19:28:34.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.593+0000 7f4b56ffd700 1 -- 172.21.15.5:0/422665020 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b64012e60 con 0x5648a8727260 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/422665020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 msgr2=0x5648a8720230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 0x5648a8720230 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009c50 tx=0x7f4b64021040 comp rx=0 tx=0).stop 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/422665020 shutdown_connections 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a8727260 0x5648a8720230 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a85776b0 0x5648a871b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/422665020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a87212d0 0x5648a872bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/422665020 >> 172.21.15.5:0/422665020 conn(0x5648a8710be0 msgr2=0x5648a871fb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/422665020 shutdown_connections 2023-07-21T19:28:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.595+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/422665020 wait complete. 2023-07-21T19:28:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.596+0000 7f4b89f0ab80 1 -- start start 2023-07-21T19:28:34.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.596+0000 7f4b89f0ab80 1 Processor -- start 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a8780e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a8727260 0x5648a8781360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8781a30 con 0x5648a87212d0 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8781ba0 con 0x5648a85776b0 2023-07-21T19:28:34.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648a8799a20 con 0x5648a8727260 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.597+0000 7f4b89f0ab80 10 client.0 _ll_register_callbacks cb 0x7f4b60009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.598+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a8727260 0x5648a8781360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.598+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.598+0000 7f4b6f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48638/0 (socket says 172.21.15.5:48638) 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.598+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3998753798 learned_addr learned my addr 172.21.15.5:0/3998753798 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:34.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a8780e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3998753798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a8727260 msgr2=0x5648a8781360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a8727260 0x5648a8781360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3998753798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 msgr2=0x5648a8780e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:34.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a8780e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 -- 172.21.15.5:0/3998753798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b64028040 con 0x5648a85776b0 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.599+0000 7f4b6f7fe700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009890 tx=0x7f4b64000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2ccfba8b40b5083 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.600+0000 7f4b4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.600+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b640141d0 con 0x5648a85776b0 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.600+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4b64000e40 con 0x5648a85776b0 2023-07-21T19:28:34.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.600+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4b6401a070 con 0x5648a85776b0 2023-07-21T19:28:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.601+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4b64014330 con 0x5648a85776b0 2023-07-21T19:28:34.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.602+0000 7f4b89f0ab80 10 client.29399 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.602+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x5648a8499250 con 0x5648a85776b0 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4b64044030 con 0x5648a85776b0 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f4b6402c070 con 0x5648a85776b0 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 10 client.29399 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 10 client.29399 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 20 client.29399 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:34.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a879ac10 0x5648a879d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 4 client.29399 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:28:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.603+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5648a8627ff0 con 0x5648a879ac10 2023-07-21T19:28:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.604+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a879ac10 0x5648a879d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.605+0000 7f4b6ffff700 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a879ac10 0x5648a879d020 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6190ac00bbd7a5f7 server_cookie=988e61a27520c880 in_seq=0 out_seq=0 2023-07-21T19:28:34.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.605+0000 7f4b4f7fe700 10 client.29399 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:34.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.641+0000 7f4b4f7fe700 1 -- 172.21.15.5:0/3998753798 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5648a8627ff0 con 0x5648a879ac10 2023-07-21T19:28:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.641+0000 7f4b4f7fe700 10 client.29399 handle_command_reply: tid=0 2023-07-21T19:28:34.631 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.641+0000 7f4b89f0ab80 1 client.29399 shutdown 2023-07-21T19:28:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.641+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 msgr2=0x5648a872bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009890 tx=0x7f4b64000bc0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 shutdown_connections 2023-07-21T19:28:34.633 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5648a8727260 0x5648a8781360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5648a85776b0 0x5648a872bd90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5648a879ac10 0x5648a879d020 crc :-1 s=READY pgs=428 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 --2- 172.21.15.5:0/3998753798 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5648a87212d0 0x5648a8780e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.634 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 >> 172.21.15.5:0/3998753798 conn(0x5648a8710be0 msgr2=0x5648a8726ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b4f7fe700 0 client.29399 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 shutdown_connections 2023-07-21T19:28:34.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.642+0000 7f4b89f0ab80 1 -- 172.21.15.5:0/3998753798 wait complete. 2023-07-21T19:28:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.643+0000 7f4b89f0ab80 20 client.29399 trim_cache size 0 max 16384 2023-07-21T19:28:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:28:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:28:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:28:34.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:28:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 msgr2=0x7f4b580a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 0x7f4b580a38c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f4b680a1880 tx=0x7f4b68006240 comp rx=0 tx=0).stop 2023-07-21T19:28:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 msgr2=0x7f4b701a2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:34.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f4b6800c110 tx=0x7f4b6800c4d0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 shutdown_connections 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4b701a2280 0x7f4b701a2720 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4b580a1410 0x7f4b580a38c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4b700ffb30 0x7f4b70109090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 --2- 172.21.15.5:0/737899956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4b701004e0 0x7f4b701095f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 >> 172.21.15.5:0/737899956 conn(0x7f4b700fb650 msgr2=0x7f4b70103650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:34.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 shutdown_connections 2023-07-21T19:28:34.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:34.644+0000 7f4b765ff700 1 -- 172.21.15.5:0/737899956 wait complete. 2023-07-21T19:28:35.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:35 smithi101 ceph-mon[128136]: pgmap v662: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:35.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:35 smithi005 ceph-mon[126460]: pgmap v662: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:35.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:35 smithi156 ceph-mon[128316]: pgmap v662: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 125.0 s, 421.8 tps, lat 235.549 ms stddev 84.470 2023-07-21T19:28:37.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:37 smithi101 ceph-mon[128136]: pgmap v663: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:37.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:37 smithi005 ceph-mon[126460]: pgmap v663: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:37.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:37 smithi156 ceph-mon[128316]: pgmap v663: 97 pgs: 97 active+clean; 14 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 118 KiB/s rd, 1.9 MiB/s wr, 167 op/s 2023-07-21T19:28:39.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:39 smithi156 ceph-mon[128316]: pgmap v664: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 7.1 MiB/s wr, 543 op/s 2023-07-21T19:28:40.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:39 smithi101 ceph-mon[128136]: pgmap v664: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 7.1 MiB/s wr, 543 op/s 2023-07-21T19:28:40.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:39 smithi005 ceph-mon[126460]: pgmap v664: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 7.1 MiB/s wr, 543 op/s 2023-07-21T19:28:41.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 130.0 s, 420.4 tps, lat 238.105 ms stddev 55.956 2023-07-21T19:28:42.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:41 smithi101 ceph-mon[128136]: pgmap v665: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T19:28:42.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:41 smithi005 ceph-mon[126460]: pgmap v665: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T19:28:42.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:41 smithi156 ceph-mon[128316]: pgmap v665: 97 pgs: 97 active+clean; 15 GiB data, 11 GiB used, 1.0 TiB / 1.0 TiB avail; 149 KiB/s rd, 6.4 MiB/s wr, 450 op/s 2023-07-21T19:28:43.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:43 smithi156 ceph-mon[128316]: pgmap v666: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 854 op/s 2023-07-21T19:28:44.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:43 smithi101 ceph-mon[128136]: pgmap v666: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 854 op/s 2023-07-21T19:28:44.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:43 smithi005 ceph-mon[126460]: pgmap v666: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 189 KiB/s rd, 12 MiB/s wr, 854 op/s 2023-07-21T19:28:45.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:45 smithi156 ceph-mon[128316]: pgmap v667: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:46.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:45 smithi101 ceph-mon[128136]: pgmap v667: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:46.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:45 smithi005 ceph-mon[126460]: pgmap v667: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:46.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 135.0 s, 439.6 tps, lat 227.499 ms stddev 63.811 2023-07-21T19:28:48.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:47 smithi101 ceph-mon[128136]: pgmap v668: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:48.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:47 smithi005 ceph-mon[126460]: pgmap v668: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:48.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:47 smithi156 ceph-mon[128316]: pgmap v668: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 11 MiB/s wr, 780 op/s 2023-07-21T19:28:48.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.509+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/3154613500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 msgr2=0x7fe5b41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.509+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/3154613500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b41023a0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fe5a40099c0 tx=0x7fe5a4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/3154613500 shutdown_connections 2023-07-21T19:28:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/3154613500 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/3154613500 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b41023a0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/3154613500 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/3154613500 >> 172.21.15.5:0/3154613500 conn(0x7fe5b40fd840 msgr2=0x7fe5b40ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/3154613500 shutdown_connections 2023-07-21T19:28:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.510+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/3154613500 wait complete. 2023-07-21T19:28:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.512+0000 7fe5bc7ff700 1 Processor -- start 2023-07-21T19:28:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.512+0000 7fe5bc7ff700 1 -- start start 2023-07-21T19:28:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4071f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b410dce0 con 0x7fe5b4101fd0 2023-07-21T19:28:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b410de20 con 0x7fe5b4102970 2023-07-21T19:28:48.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bc7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b4110cb0 con 0x7fe5b4107010 2023-07-21T19:28:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5ba59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4071f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5ba59b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52158/0 (socket says 172.21.15.5:52158) 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5ba59b700 1 -- 172.21.15.5:0/2372813187 learned_addr learned my addr 172.21.15.5:0/2372813187 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.513+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/2372813187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 msgr2=0x7fe5b410d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 msgr2=0x7fe5b410d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.505 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/2372813187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5a40095f0 con 0x7fe5b4102970 2023-07-21T19:28:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:48.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.514+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4071f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00bdf0 tx=0x7fe5ac009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.515+0000 7fe5ab7fe700 1 -- 172.21.15.5:0/2372813187 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac00cbb0 con 0x7fe5b4102970 2023-07-21T19:28:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.515+0000 7fe5ab7fe700 1 -- 172.21.15.5:0/2372813187 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5ac00cd10 con 0x7fe5b4102970 2023-07-21T19:28:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.515+0000 7fe5ab7fe700 1 -- 172.21.15.5:0/2372813187 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac0125c0 con 0x7fe5b4102970 2023-07-21T19:28:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.515+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5b4110f90 con 0x7fe5b4102970 2023-07-21T19:28:48.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.515+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b4111450 con 0x7fe5b4102970 2023-07-21T19:28:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.523+0000 7fe5ab7fe700 1 -- 172.21.15.5:0/2372813187 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe5ac014440 con 0x7fe5b4102970 2023-07-21T19:28:48.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.524+0000 7fe5d0184b80 1 Processor -- start 2023-07-21T19:28:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.524+0000 7fe5d0184b80 1 -- start start 2023-07-21T19:28:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.524+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x563890320d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.524+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389031b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563890220c50 con 0x56389031b580 2023-07-21T19:28:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638902739e0 con 0x5638901776b0 2023-07-21T19:28:48.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389026ce70 con 0x5638903212d0 2023-07-21T19:28:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5ba59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389031b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5ba59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46046/0 (socket says 172.21.15.5:46046) 2023-07-21T19:28:48.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5ba59b700 1 -- 172.21.15.5:0/1403004656 learned_addr learned my addr 172.21.15.5:0/1403004656 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:48.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.525+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x563890320d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 -- 172.21.15.5:0/1403004656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 msgr2=0x563890320d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x563890320d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 -- 172.21.15.5:0/1403004656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 msgr2=0x56389031b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389031b9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 -- 172.21.15.5:0/1403004656 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56389009ad80 con 0x5638903212d0 2023-07-21T19:28:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400bc30 tx=0x7fe5a400bc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ea6f7fcad2c425b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.526+0000 7fe5ab7fe700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 0x7fe5a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.527+0000 7fe5ab7fe700 1 -- 172.21.15.5:0/2372813187 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe5ac0cc420 con 0x7fe5b4102970 2023-07-21T19:28:48.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.527+0000 7fe5a8ff9700 1 -- 172.21.15.5:0/1403004656 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a401d030 con 0x5638903212d0 2023-07-21T19:28:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.527+0000 7fe5a8ff9700 1 -- 172.21.15.5:0/1403004656 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5a401e040 con 0x5638903212d0 2023-07-21T19:28:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.527+0000 7fe5a8ff9700 1 -- 172.21.15.5:0/1403004656 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5a4023460 con 0x5638903212d0 2023-07-21T19:28:48.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.527+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 0x7fe5a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1403004656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 msgr2=0x56389031b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fe5a400bc30 tx=0x7fe5a400bc60 comp rx=0 tx=0).stop 2023-07-21T19:28:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 0x7fe5a00a37f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009fd0 tx=0x7fe5b0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1403004656 shutdown_connections 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389031b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x563890320d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1403004656 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389031b040 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.528+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1403004656 >> 172.21.15.5:0/1403004656 conn(0x563890310be0 msgr2=0x563890315f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.529+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1403004656 shutdown_connections 2023-07-21T19:28:48.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.529+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1403004656 wait complete. 2023-07-21T19:28:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.529+0000 7fe5d0184b80 1 -- start start 2023-07-21T19:28:48.524 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.530+0000 7fe5d0184b80 1 Processor -- start 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638901776b0 0x563890324930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x5638903253b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389020ca60 con 0x5638901776b0 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389020c760 con 0x56389031b580 2023-07-21T19:28:48.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638900f08c0 con 0x5638903212d0 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5d0184b80 10 client.0 _ll_register_callbacks cb 0x56389020a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54696/0 (socket says 172.21.15.5:54696) 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.531+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1249746442 learned_addr learned my addr 172.21.15.5:0/1249746442 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1249746442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 msgr2=0x5638903253b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x5638903253b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1249746442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638901776b0 msgr2=0x563890324930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638901776b0 0x563890324930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1249746442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5b410f420 con 0x56389031b580 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.532+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00be20 tx=0x7fe5ac009940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b13330e46ad3d281 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.533+0000 7fe59dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.533+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac099140 con 0x56389031b580 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.533+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe5ac0b90b0 con 0x56389031b580 2023-07-21T19:28:48.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.533+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638901c6490 con 0x56389031b580 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.534+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ac098f60 con 0x56389031b580 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.534+0000 7fe5d0184b80 10 client.29408 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.534+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563890083c00 con 0x56389031b580 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.534+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638901c6490 con 0x56389031b580 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe5ac0840f0 con 0x56389031b580 2023-07-21T19:28:48.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 10 client.29408 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 10 client.29408 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 20 client.29408 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890334bf0 0x563890337000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 4 client.29408 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:48.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.535+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563890265e50 con 0x563890334bf0 2023-07-21T19:28:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.536+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890334bf0 0x563890337000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.537+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890334bf0 0x563890337000 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df22558f11ba6af server_cookie=9a28255f9abd576c in_seq=0 out_seq=0 2023-07-21T19:28:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.537+0000 7fe59dffb700 10 client.29408 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.538+0000 7fe59dffb700 1 -- 172.21.15.5:0/1249746442 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563890265e50 con 0x563890334bf0 2023-07-21T19:28:48.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.538+0000 7fe59dffb700 10 client.29408 handle_command_reply: tid=0 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.538+0000 7fe5d0184b80 1 client.29408 shutdown 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 msgr2=0x563890324e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00be20 tx=0x7fe5ac009940 comp rx=0 tx=0).stop 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 shutdown_connections 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x5638903253b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890334bf0 0x563890337000 crc :-1 s=READY pgs=429 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638901776b0 0x563890324930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1249746442 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56389031b580 0x563890324e70 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 >> 172.21.15.5:0/1249746442 conn(0x563890310be0 msgr2=0x5638903154e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe59dffb700 0 client.29408 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.539+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 shutdown_connections 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.540+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1249746442 wait complete. 2023-07-21T19:28:48.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.540+0000 7fe5d0184b80 20 client.29408 trim_cache size 0 max 16384 2023-07-21T19:28:48.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.565+0000 7fe5d0184b80 1 Processor -- start 2023-07-21T19:28:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 -- start start 2023-07-21T19:28:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 0x56389031b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638903212d0 0x563890315c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563890327530 con 0x56389031b580 2023-07-21T19:28:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56389030dbc0 con 0x5638903212d0 2023-07-21T19:28:48.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.566+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638903209e0 con 0x5638901776b0 2023-07-21T19:28:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5ba59b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 0x56389031b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52178/0 (socket says 172.21.15.5:52178) 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1657269105 learned_addr learned my addr 172.21.15.5:0/1657269105 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638903212d0 0x563890315c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1657269105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 msgr2=0x56389031b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 0x56389031b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1657269105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638903212d0 msgr2=0x563890315c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638903212d0 0x563890315c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1657269105 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638900c11f0 con 0x56389031b580 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 0x56389031b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.567+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac014d50 tx=0x7fe5ac00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41e3e29b28d61bfd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.568+0000 7fe5a97fa700 1 -- 172.21.15.5:0/1657269105 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac014820 con 0x56389031b580 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.568+0000 7fe5a97fa700 1 -- 172.21.15.5:0/1657269105 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5ac098f60 con 0x56389031b580 2023-07-21T19:28:48.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.568+0000 7fe5a97fa700 1 -- 172.21.15.5:0/1657269105 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac09d630 con 0x56389031b580 2023-07-21T19:28:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1657269105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 msgr2=0x56389032bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac014d50 tx=0x7fe5ac00bf90 comp rx=0 tx=0).stop 2023-07-21T19:28:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1657269105 shutdown_connections 2023-07-21T19:28:48.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638903212d0 0x563890315c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638901776b0 0x56389031b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1657269105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389032bbc0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1657269105 >> 172.21.15.5:0/1657269105 conn(0x563890310be0 msgr2=0x56389031eb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1657269105 shutdown_connections 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.569+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1657269105 wait complete. 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.570+0000 7fe5d0184b80 1 -- start start 2023-07-21T19:28:48.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.570+0000 7fe5d0184b80 1 Processor -- start 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x56389037ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389037d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563890387b00 con 0x56389031b580 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563890387c40 con 0x5638901776b0 2023-07-21T19:28:48.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563890387e00 con 0x5638903212d0 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5d0184b80 10 client.0 _ll_register_callbacks cb 0x7fe5ac004b10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5b9d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52188/0 (socket says 172.21.15.5:52188) 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1007008438 learned_addr learned my addr 172.21.15.5:0/1007008438 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.571+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389037d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x56389037ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1007008438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 msgr2=0x56389037d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389037d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 msgr2=0x56389037ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x56389037ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 -- 172.21.15.5:0/1007008438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563890083c00 con 0x56389031b580 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5bad9c700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389037d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:48.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe5b9d9a700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00be20 tx=0x7fe5ac014760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=531e1d856fd134f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.572+0000 7fe59dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:28:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.573+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5ac095940 con 0x56389031b580 2023-07-21T19:28:48.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.573+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe5a401e040 con 0x56389031b580 2023-07-21T19:28:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.573+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe5ac0b90b0 con 0x56389031b580 2023-07-21T19:28:48.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.573+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=56) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5ac099600 con 0x56389031b580 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.574+0000 7fe5d0184b80 10 client.19704 fetch_fsmap learned FSMap version 56 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.574+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=56,monmap=4+}) v3 -- 0x563890099250 con 0x56389031b580 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe5a401e040 con 0x56389031b580 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 56) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fe5ac099d00 con 0x56389031b580 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 10 client.19704 fetch_fsmap finished waiting for FSMap version 56 2023-07-21T19:28:48.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 10 client.19704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 20 client.19704 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890389060 0x56389038b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 4 client.19704 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.575+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563890227ff0 con 0x563890389060 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.576+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890389060 0x56389038b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.576+0000 7fe5ba59b700 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890389060 0x56389038b440 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50dde9e46ed6e319 server_cookie=23c25cf2cbbf9d93 in_seq=0 out_seq=0 2023-07-21T19:28:48.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.576+0000 7fe59dffb700 10 client.19704 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.609+0000 7fe59dffb700 1 -- 172.21.15.5:0/1007008438 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563890227ff0 con 0x563890389060 2023-07-21T19:28:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.609+0000 7fe59dffb700 10 client.19704 handle_command_reply: tid=0 2023-07-21T19:28:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.609+0000 7fe5d0184b80 1 client.19704 shutdown 2023-07-21T19:28:48.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.609+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 msgr2=0x56389037cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.609+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00be20 tx=0x7fe5ac014760 comp rx=0 tx=0).stop 2023-07-21T19:28:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 shutdown_connections 2023-07-21T19:28:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638903212d0 0x56389037d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638901776b0 0x56389037ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563890389060 0x56389038b440 crc :-1 s=READY pgs=430 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 --2- 172.21.15.5:0/1007008438 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56389031b580 0x56389037cf80 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 >> 172.21.15.5:0/1007008438 conn(0x563890310be0 msgr2=0x56389031fab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe59dffb700 0 client.19704 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 shutdown_connections 2023-07-21T19:28:48.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.610+0000 7fe5d0184b80 1 -- 172.21.15.5:0/1007008438 wait complete. 2023-07-21T19:28:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:28:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:28:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:28:48.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:28:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.611+0000 7fe5d0184b80 20 client.19704 trim_cache size 0 max 16384 2023-07-21T19:28:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.613+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 msgr2=0x7fe5a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.613+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 0x7fe5a00a37f0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fe5b0009fd0 tx=0x7fe5b0009260 comp rx=0 tx=0).stop 2023-07-21T19:28:48.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.613+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 msgr2=0x7fe5b4071f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.613+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4071f90 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fe5ac00bdf0 tx=0x7fe5ac009f70 comp rx=0 tx=0).stop 2023-07-21T19:28:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 shutdown_connections 2023-07-21T19:28:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe5b4107010 0x7fe5b410d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe5a00a1340 0x7fe5a00a37f0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe5b4101fd0 0x7fe5b410d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 --2- 172.21.15.5:0/2372813187 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe5b4102970 0x7fe5b4071f90 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.614+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 >> 172.21.15.5:0/2372813187 conn(0x7fe5b40fd840 msgr2=0x7fe5b4078330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:48.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.615+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 shutdown_connections 2023-07-21T19:28:48.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:28:48.615+0000 7fe5bc7ff700 1 -- 172.21.15.5:0/2372813187 wait complete. 2023-07-21T19:28:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:49 smithi101 ceph-mon[128136]: pgmap v669: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-21T19:28:50.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:49 smithi005 ceph-mon[126460]: pgmap v669: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-21T19:28:50.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:49 smithi156 ceph-mon[128316]: pgmap v669: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 222 KiB/s rd, 17 MiB/s wr, 1.20k op/s 2023-07-21T19:28:51.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:50 smithi101 ceph-mon[128136]: pgmap v670: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-21T19:28:51.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:50 smithi005 ceph-mon[126460]: pgmap v670: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-21T19:28:51.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:50 smithi156 ceph-mon[128316]: pgmap v670: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 138 KiB/s rd, 11 MiB/s wr, 826 op/s 2023-07-21T19:28:51.689 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:28:51.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 140.0 s, 430.2 tps, lat 230.998 ms stddev 42.511 2023-07-21T19:28:52.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.061+0000 7fc81cdff700 1 -- 172.21.15.5:0/3692470629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818100480 msgr2=0x7fc81810c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.061+0000 7fc81cdff700 1 --2- 172.21.15.5:0/3692470629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818100480 0x7fc81810c910 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b0a0 tx=0x7fc80800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.062+0000 7fc81cdff700 1 -- 172.21.15.5:0/3692470629 shutdown_connections 2023-07-21T19:28:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.062+0000 7fc81cdff700 1 --2- 172.21.15.5:0/3692470629 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818100480 0x7fc81810c910 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.062+0000 7fc81cdff700 1 --2- 172.21.15.5:0/3692470629 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818105ae0 0x7fc818105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.062+0000 7fc81cdff700 1 --2- 172.21.15.5:0/3692470629 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc8180fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.062+0000 7fc81cdff700 1 -- 172.21.15.5:0/3692470629 >> 172.21.15.5:0/3692470629 conn(0x7fc8180fb5f0 msgr2=0x7fc8180fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.063+0000 7fc81cdff700 1 -- 172.21.15.5:0/3692470629 shutdown_connections 2023-07-21T19:28:52.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.063+0000 7fc81cdff700 1 -- 172.21.15.5:0/3692470629 wait complete. 2023-07-21T19:28:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.064+0000 7fc81cdff700 1 Processor -- start 2023-07-21T19:28:52.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 -- start start 2023-07-21T19:28:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818100480 0x7fc818072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 0x7fc8181a3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818073580 con 0x7fc818100480 2023-07-21T19:28:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181a4360 con 0x7fc8180ffad0 2023-07-21T19:28:52.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.065+0000 7fc81cdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8181a44a0 con 0x7fc818105ae0 2023-07-21T19:28:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc81659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc81659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54718/0 (socket says 172.21.15.5:54718) 2023-07-21T19:28:52.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc81659c700 1 -- 172.21.15.5:0/545413896 learned_addr learned my addr 172.21.15.5:0/545413896 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc815d9b700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818100480 0x7fc818072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 0x7fc8181a3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 -- 172.21.15.5:0/545413896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 msgr2=0x7fc818072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 -- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818100480 msgr2=0x7fc818072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818100480 0x7fc818072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc816d9d700 1 -- 172.21.15.5:0/545413896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808009d00 con 0x7fc818105ae0 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.066+0000 7fc81659c700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc816d9d700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 0x7fc8181a3e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b070 tx=0x7fc80800bce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc8077fe700 1 -- 172.21.15.5:0/545413896 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800e040 con 0x7fc818105ae0 2023-07-21T19:28:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc8077fe700 1 -- 172.21.15.5:0/545413896 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc8080094a0 con 0x7fc818105ae0 2023-07-21T19:28:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc8077fe700 1 -- 172.21.15.5:0/545413896 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc808005040 con 0x7fc818105ae0 2023-07-21T19:28:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8181a46a0 con 0x7fc818105ae0 2023-07-21T19:28:52.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.067+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8181a4a70 con 0x7fc818105ae0 2023-07-21T19:28:52.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.073+0000 7fc8077fe700 1 -- 172.21.15.5:0/545413896 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc80801d480 con 0x7fc818105ae0 2023-07-21T19:28:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.074+0000 7fc83077eb80 1 Processor -- start 2023-07-21T19:28:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.074+0000 7fc83077eb80 1 -- start start 2023-07-21T19:28:52.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc57590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc59eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdb52cb0 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdba5a40 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc83077eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdb9eed0 con 0x55f5fdc57ad0 2023-07-21T19:28:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54732/0 (socket says 172.21.15.5:54732) 2023-07-21T19:28:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc815d9b700 1 -- 172.21.15.5:0/929271100 learned_addr learned my addr 172.21.15.5:0/929271100 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:52.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.075+0000 7fc815d9b700 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc59eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 -- 172.21.15.5:0/929271100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 msgr2=0x55f5fdc4c6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4c6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 -- 172.21.15.5:0/929271100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4cbe0 msgr2=0x55f5fdc57590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc57590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 -- 172.21.15.5:0/929271100 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5fd9ccde0 con 0x55f5fdc57ad0 2023-07-21T19:28:52.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc8077fe700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 0x7fc8000a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc815d9b700 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc59eb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc80c011880 tx=0x7fc80c009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca276a6b9add256c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc8077fe700 1 -- 172.21.15.5:0/545413896 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc8080cdc10 con 0x7fc818105ae0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc804ff9700 1 -- 172.21.15.5:0/929271100 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c011ea0 con 0x55f5fdc57ad0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc804ff9700 1 -- 172.21.15.5:0/929271100 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc80c011540 con 0x55f5fdc57ad0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.076+0000 7fc804ff9700 1 -- 172.21.15.5:0/929271100 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c005280 con 0x55f5fdc57ad0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.077+0000 7fc81659c700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 0x7fc8000a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.077+0000 7fc83077eb80 1 -- 172.21.15.5:0/929271100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 msgr2=0x55f5fdc59eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.077+0000 7fc83077eb80 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc59eb0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fc80c011880 tx=0x7fc80c009260 comp rx=0 tx=0).stop 2023-07-21T19:28:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 -- 172.21.15.5:0/929271100 shutdown_connections 2023-07-21T19:28:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc59eb0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 --2- 172.21.15.5:0/929271100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc57590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 -- 172.21.15.5:0/929271100 >> 172.21.15.5:0/929271100 conn(0x55f5fdc42da0 msgr2=0x55f5fdc486a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 -- 172.21.15.5:0/929271100 shutdown_connections 2023-07-21T19:28:52.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc83077eb80 1 -- 172.21.15.5:0/929271100 wait complete. 2023-07-21T19:28:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.078+0000 7fc81659c700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 0x7fc8000a3830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ea0 tx=0x7fc810009290 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.079+0000 7fc83077eb80 1 -- start start 2023-07-21T19:28:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.079+0000 7fc83077eb80 1 Processor -- start 2023-07-21T19:28:52.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc57500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc54ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdb3eac0 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdb3e7c0 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fda22920 con 0x55f5fdc57ad0 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.080+0000 7fc83077eb80 10 client.0 _ll_register_callbacks cb 0x55f5fdb3c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.081+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.081+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54744/0 (socket says 172.21.15.5:54744) 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.081+0000 7fc815d9b700 1 -- 172.21.15.5:0/2408585547 learned_addr learned my addr 172.21.15.5:0/2408585547 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc81659c700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc57500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc815d9b700 1 -- 172.21.15.5:0/2408585547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 msgr2=0x55f5fdc54ec0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:28:52.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc815d9b700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc54ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc815d9b700 1 -- 172.21.15.5:0/2408585547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 msgr2=0x55f5fdc57500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc815d9b700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc57500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc815d9b700 1 -- 172.21.15.5:0/2408585547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80c019040 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.082+0000 7fc81659c700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc57500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:28:52.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.083+0000 7fc815d9b700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005950 tx=0x7fc80c009930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3855678b7cad0de8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.083+0000 7fc7fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:28:52.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c025070 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc80c000e40 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f5fdaf84f0 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80c011ea0 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc83077eb80 10 client.29426 fetch_fsmap learned FSMap version 57 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.084+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55f5fd9b5c60 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.085+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f5fdaf84f0 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc80c01d070 con 0x55f5fdc4cbe0 2023-07-21T19:28:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 10 client.29426 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:28:52.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 10 client.29426 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 20 client.29426 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdc783e0 0x55f5fdc7a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 4 client.29426 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.086+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f5fdb97eb0 con 0x55f5fdc783e0 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.087+0000 7fc81659c700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdc783e0 0x55f5fdc7a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.087+0000 7fc81659c700 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdc783e0 0x55f5fdc7a7f0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ecc4c1cd2022ecf server_cookie=8349fa44ce75b783 in_seq=0 out_seq=0 2023-07-21T19:28:52.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.087+0000 7fc7fdffb700 10 client.29426 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.089+0000 7fc7fdffb700 1 -- 172.21.15.5:0/2408585547 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f5fdb97eb0 con 0x55f5fdc783e0 2023-07-21T19:28:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.089+0000 7fc7fdffb700 10 client.29426 handle_command_reply: tid=0 2023-07-21T19:28:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.089+0000 7fc83077eb80 1 client.29426 shutdown 2023-07-21T19:28:52.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 msgr2=0x55f5fdc54780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005950 tx=0x7fc80c009930 comp rx=0 tx=0).stop 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 shutdown_connections 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc54ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdc783e0 0x55f5fdc7a7f0 crc :-1 s=READY pgs=431 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc57500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 --2- 172.21.15.5:0/2408585547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc4cbe0 0x55f5fdc54780 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fc80c005950 tx=0x7fc80c009930 comp rx=0 tx=0).stop 2023-07-21T19:28:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 >> 172.21.15.5:0/2408585547 conn(0x55f5fdc42da0 msgr2=0x55f5fdc48230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc7fdffb700 0 client.29426 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 shutdown_connections 2023-07-21T19:28:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.090+0000 7fc83077eb80 1 -- 172.21.15.5:0/2408585547 wait complete. 2023-07-21T19:28:52.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.091+0000 7fc83077eb80 20 client.29426 trim_cache size 0 max 16384 2023-07-21T19:28:52.103 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 Processor -- start 2023-07-21T19:28:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 -- start start 2023-07-21T19:28:52.104 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4fa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc4ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 0x55f5fdc51a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdc4e1a0 con 0x55f5fdc488c0 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdc4d540 con 0x55f5fdc57ad0 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.116+0000 7fc83077eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdc4ddd0 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc816d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 0x55f5fdc51a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.105 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc816d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 0x55f5fdc51a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52212/0 (socket says 172.21.15.5:52212) 2023-07-21T19:28:52.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc816d9d700 1 -- 172.21.15.5:0/4216749087 learned_addr learned my addr 172.21.15.5:0/4216749087 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:52.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc815d9b700 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc4ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.106 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 -- 172.21.15.5:0/4216749087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc57ad0 msgr2=0x55f5fdc4ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc4ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 -- 172.21.15.5:0/4216749087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 msgr2=0x55f5fdc51a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 0x55f5fdc51a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.117+0000 7fc81659c700 1 -- 172.21.15.5:0/4216749087 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f5fd9f3250 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.118+0000 7fc81659c700 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4fa90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f5fdc7add0 tx=0x7fc810011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668c720f82fbac4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.118+0000 7fc8057fa700 1 -- 172.21.15.5:0/4216749087 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000ee80 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.118+0000 7fc8057fa700 1 -- 172.21.15.5:0/4216749087 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc810024540 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.118+0000 7fc8057fa700 1 -- 172.21.15.5:0/4216749087 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc810015b60 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- 172.21.15.5:0/4216749087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 msgr2=0x55f5fdc4fa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4fa90 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x55f5fdc7add0 tx=0x7fc810011000 comp rx=0 tx=0).stop 2023-07-21T19:28:52.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- 172.21.15.5:0/4216749087 shutdown_connections 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc488c0 0x55f5fdc51a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc4fa90 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 --2- 172.21.15.5:0/4216749087 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc4ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- 172.21.15.5:0/4216749087 >> 172.21.15.5:0/4216749087 conn(0x55f5fdc42da0 msgr2=0x55f5fdc57130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- 172.21.15.5:0/4216749087 shutdown_connections 2023-07-21T19:28:52.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- 172.21.15.5:0/4216749087 wait complete. 2023-07-21T19:28:52.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.119+0000 7fc83077eb80 1 -- start start 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 Processor -- start 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 0x55f5fdc5cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc5d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdcd4a80 con 0x55f5fdc57ad0 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdcd4bc0 con 0x55f5fdc488c0 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f5fdcd4d00 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.120+0000 7fc83077eb80 10 client.0 _ll_register_callbacks cb 0x7fc80c009b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc81659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 0x55f5fdc5cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46108/0 (socket says 172.21.15.5:46108) 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc81659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 0x55f5fdc5cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54756/0 (socket says 172.21.15.5:54756) 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 -- 172.21.15.5:0/352546735 learned_addr learned my addr 172.21.15.5:0/352546735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc816d9d700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc5d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 -- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 msgr2=0x55f5fdc5cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 0x55f5fdc5cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 -- 172.21.15.5:0/352546735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 msgr2=0x55f5fdc5d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc5d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc815d9b700 1 -- 172.21.15.5:0/352546735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc81001a040 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.121+0000 7fc816d9d700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc5d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc815d9b700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f5fdc42ab0 tx=0x7fc80c005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ffecdc302f4a29d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:28:52.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc7fdffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80c025070 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc80c011540 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc80c000e40 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc80c009260 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc83077eb80 10 client.29350 fetch_fsmap learned FSMap version 57 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.122+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x55f5fd9cb2b0 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.114 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc80c011540 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc80c011b90 con 0x55f5fdc4c2d0 2023-07-21T19:28:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 10 client.29350 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:28:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 10 client.29350 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:28:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 20 client.29350 populate_metadata read hostname 'smithi005' 2023-07-21T19:28:52.115 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdcd6120 0x55f5fdcd8530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 4 client.29350 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f5fdb5a050 con 0x55f5fdcd6120 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.123+0000 7fc81659c700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdcd6120 0x55f5fdcd8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.124+0000 7fc81659c700 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdcd6120 0x55f5fdcd8530 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d19e5c92a129215a server_cookie=8a8967c7dbd5b176 in_seq=0 out_seq=0 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.124+0000 7fc7fdffb700 10 client.29350 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.124+0000 7fc7fdffb700 1 -- 172.21.15.5:0/352546735 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f5fdb5a050 con 0x55f5fdcd6120 2023-07-21T19:28:52.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.124+0000 7fc7fdffb700 10 client.29350 handle_command_reply: tid=0 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.124+0000 7fc83077eb80 1 client.29350 shutdown 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 msgr2=0x55f5fdc5d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x55f5fdc42ab0 tx=0x7fc80c005800 comp rx=0 tx=0).stop 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 shutdown_connections 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f5fdcd6120 0x55f5fdcd8530 crc :-1 s=READY pgs=432 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f5fdc57ad0 0x55f5fdc5d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f5fdc488c0 0x55f5fdc5cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 --2- 172.21.15.5:0/352546735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f5fdc4c2d0 0x55f5fdc5d0f0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 >> 172.21.15.5:0/352546735 conn(0x55f5fdc42da0 msgr2=0x55f5fdc4fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.126+0000 7fc7fdffb700 0 client.29350 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:28:52.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.129+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 shutdown_connections 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:28:52.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:28:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:28:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:28:52.121 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:28:52.121 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.129+0000 7fc83077eb80 1 -- 172.21.15.5:0/352546735 wait complete. 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.130+0000 7fc83077eb80 20 client.29350 trim_cache size 0 max 16384 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.133+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 msgr2=0x7fc8000a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.133+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 0x7fc8000a3830 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fc810009ea0 tx=0x7fc810009290 comp rx=0 tx=0).stop 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.133+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 msgr2=0x7fc8181a3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.133+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 0x7fc8181a3e20 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fc80800b070 tx=0x7fc80800bce0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 shutdown_connections 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc818105ae0 0x7fc8181a3e20 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8180ffad0 0x7fc818072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8000a1380 0x7fc8000a3830 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 --2- 172.21.15.5:0/545413896 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc818100480 0x7fc818072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 >> 172.21.15.5:0/545413896 conn(0x7fc8180fb5f0 msgr2=0x7fc8180fbba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 shutdown_connections 2023-07-21T19:28:52.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:28:52.134+0000 7fc81cdff700 1 -- 172.21.15.5:0/545413896 wait complete. 2023-07-21T19:28:52.151 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:28:53.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:52 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:28:53.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:52 smithi101 ceph-mon[128136]: pgmap v671: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-21T19:28:53.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:52 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:28:53.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:52 smithi005 ceph-mon[126460]: pgmap v671: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-21T19:28:53.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:52 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:28:53.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:52 smithi156 ceph-mon[128316]: pgmap v671: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 246 KiB/s rd, 17 MiB/s wr, 1.24k op/s 2023-07-21T19:28:55.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:55 smithi101 ceph-mon[128136]: pgmap v672: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:28:55.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:55 smithi005 ceph-mon[126460]: pgmap v672: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:28:55.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:55 smithi156 ceph-mon[128316]: pgmap v672: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:28:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 145.0 s, 439.0 tps, lat 229.070 ms stddev 40.027 2023-07-21T19:28:57.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:57 smithi101 ceph-mon[128136]: pgmap v673: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:28:57.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:57 smithi005 ceph-mon[126460]: pgmap v673: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:28:57.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:57 smithi156 ceph-mon[128316]: pgmap v673: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 206 KiB/s rd, 12 MiB/s wr, 833 op/s 2023-07-21T19:29:00.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:28:59 smithi101 ceph-mon[128136]: pgmap v674: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-21T19:29:00.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:28:59 smithi005 ceph-mon[126460]: pgmap v674: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-21T19:29:00.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:28:59 smithi156 ceph-mon[128316]: pgmap v674: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 17 MiB/s wr, 1.27k op/s 2023-07-21T19:29:01.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 150.0 s, 385.8 tps, lat 255.354 ms stddev 73.600 2023-07-21T19:29:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.073+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3817195332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4107780 msgr2=0x7f4ac4107b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:02.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.073+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3817195332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4107780 0x7f4ac4107b50 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f4ab4009990 tx=0x7f4ab4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.074+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3817195332 shutdown_connections 2023-07-21T19:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.074+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3817195332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ac4110b50 0x7f4ac4112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.074+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3817195332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4107780 0x7f4ac4107b50 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.074+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3817195332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4108090 0x7f4ac4110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.074+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3817195332 >> 172.21.15.5:0/3817195332 conn(0x7f4ac406c270 msgr2=0x7f4ac406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:02.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.076+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3817195332 shutdown_connections 2023-07-21T19:29:02.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.076+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3817195332 wait complete. 2023-07-21T19:29:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.078+0000 7f4ac9dff700 1 Processor -- start 2023-07-21T19:29:02.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.078+0000 7f4ac9dff700 1 -- start start 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ac4107780 0x7f4ac4110290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4108090 0x7f4ac410d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410dcc0 con 0x7f4ac4110b50 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac410de30 con 0x7f4ac4107780 2023-07-21T19:29:02.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.079+0000 7f4ac9dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac411acf0 con 0x7f4ac4108090 2023-07-21T19:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.080+0000 7f4ac3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.080+0000 7f4ac3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33974/0 (socket says 172.21.15.5:33974) 2023-07-21T19:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.080+0000 7f4ac3fff700 1 -- 172.21.15.5:0/3543289957 learned_addr learned my addr 172.21.15.5:0/3543289957 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:02.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 -- 172.21.15.5:0/3543289957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4108090 msgr2=0x7f4ac410d0e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4108090 0x7f4ac410d0e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 -- 172.21.15.5:0/3543289957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ac4107780 msgr2=0x7f4ac4110290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ac4107780 0x7f4ac4110290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 -- 172.21.15.5:0/3543289957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab40095f0 con 0x7f4ac4110b50 2023-07-21T19:29:02.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:01 smithi101 ceph-mon[128136]: pgmap v675: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-21T19:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.081+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80077c0 tx=0x7f4ab8007b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.082+0000 7f4ac0ff9700 1 -- 172.21.15.5:0/3543289957 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab800f040 con 0x7f4ac4110b50 2023-07-21T19:29:02.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.082+0000 7f4ac0ff9700 1 -- 172.21.15.5:0/3543289957 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ab8004cc0 con 0x7f4ac4110b50 2023-07-21T19:29:02.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.082+0000 7f4ac0ff9700 1 -- 172.21.15.5:0/3543289957 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8013870 con 0x7f4ac4110b50 2023-07-21T19:29:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.083+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac411afd0 con 0x7f4ac4110b50 2023-07-21T19:29:02.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.083+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac411b490 con 0x7f4ac4110b50 2023-07-21T19:29:02.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.091+0000 7f4add7c6b80 1 Processor -- start 2023-07-21T19:29:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 -- start start 2023-07-21T19:29:02.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4ac0ff9700 1 -- 172.21.15.5:0/3543289957 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4ab8005000 con 0x7f4ac4110b50 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1abb580 0x560db1abb9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db19c0c50 con 0x560db19176b0 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1a139e0 con 0x560db1abb580 2023-07-21T19:29:02.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.092+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1a0ce70 con 0x560db1ac12d0 2023-07-21T19:29:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.093+0000 7f4ac2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33988/0 (socket says 172.21.15.5:33988) 2023-07-21T19:29:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/808107873 learned_addr learned my addr 172.21.15.5:0/808107873 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/808107873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 msgr2=0x560db1abb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/808107873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1abb580 msgr2=0x560db1abb9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1abb580 0x560db1abb9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.094+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/808107873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db183ad80 con 0x560db19176b0 2023-07-21T19:29:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.095+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009c00 tx=0x7f4abc009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90610c05e536eb60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.095+0000 7f4aa9ffb700 1 -- 172.21.15.5:0/808107873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc003c00 con 0x560db19176b0 2023-07-21T19:29:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.095+0000 7f4aa9ffb700 1 -- 172.21.15.5:0/808107873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4abc00ee60 con 0x560db19176b0 2023-07-21T19:29:02.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.095+0000 7f4aa9ffb700 1 -- 172.21.15.5:0/808107873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0047f0 con 0x560db19176b0 2023-07-21T19:29:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.096+0000 7f4add7c6b80 1 -- 172.21.15.5:0/808107873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 msgr2=0x560db1ac0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.096+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f4abc009c00 tx=0x7f4abc009f10 comp rx=0 tx=0).stop 2023-07-21T19:29:02.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.097+0000 7f4ac0ff9700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 0x7f4aac0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.098+0000 7f4ac0ff9700 1 -- 172.21.15.5:0/3543289957 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4ab80cc0f0 con 0x7f4ac4110b50 2023-07-21T19:29:02.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.100+0000 7f4add7c6b80 1 -- 172.21.15.5:0/808107873 shutdown_connections 2023-07-21T19:29:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.100+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1abb580 0x560db1abb9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.100+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac0d90 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.100+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/808107873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.100+0000 7f4add7c6b80 1 -- 172.21.15.5:0/808107873 >> 172.21.15.5:0/808107873 conn(0x560db1ab0be0 msgr2=0x560db1ab5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:02.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.103+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 0x7f4aac0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.111+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 0x7f4aac0a3980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4abc004c00 tx=0x7f4abc004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.115+0000 7f4add7c6b80 1 -- 172.21.15.5:0/808107873 shutdown_connections 2023-07-21T19:29:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.115+0000 7f4add7c6b80 1 -- 172.21.15.5:0/808107873 wait complete. 2023-07-21T19:29:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.116+0000 7f4add7c6b80 1 -- start start 2023-07-21T19:29:02.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.116+0000 7f4add7c6b80 1 Processor -- start 2023-07-21T19:29:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac7580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac4e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db19aca60 con 0x560db19176b0 2023-07-21T19:29:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db19ac760 con 0x560db1ac12d0 2023-07-21T19:29:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db18908c0 con 0x560db1abb580 2023-07-21T19:29:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.118+0000 7f4add7c6b80 10 client.0 _ll_register_callbacks cb 0x560db19aa700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.120+0000 7f4ac2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.120+0000 7f4ac2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38068/0 (socket says 172.21.15.5:38068) 2023-07-21T19:29:02.108 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.120+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/4227281592 learned_addr learned my addr 172.21.15.5:0/4227281592 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.122+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 msgr2=0x560db1ac4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.122+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.122+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/4227281592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 msgr2=0x560db1ac7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.122+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.122+0000 7f4ac2ffd700 1 -- 172.21.15.5:0/4227281592 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4abc0098b0 con 0x560db1abb580 2023-07-21T19:29:02.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.130+0000 7f4ac2ffd700 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4abc004570 tx=0x7f4abc009f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61e7745ebd75ab7c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.130+0000 7f4aa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:02.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.132+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4abc0062e0 con 0x560db1abb580 2023-07-21T19:29:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.132+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4abc006440 con 0x560db1abb580 2023-07-21T19:29:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.132+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560db1966490 con 0x560db1abb580 2023-07-21T19:29:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.133+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4abc0092a0 con 0x560db1abb580 2023-07-21T19:29:02.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.133+0000 7f4add7c6b80 10 client.29356 fetch_fsmap learned FSMap version 57 2023-07-21T19:29:02.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.133+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560db1823c00 con 0x560db1abb580 2023-07-21T19:29:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560db1966490 con 0x560db1abb580 2023-07-21T19:29:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4abc01fcb0 con 0x560db1abb580 2023-07-21T19:29:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 10 client.29356 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:29:02.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 10 client.29356 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 20 client.29356 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1ad4b90 0x560db1ad6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 4 client.29356 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:02.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.135+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560db1a05e50 con 0x560db1ad4b90 2023-07-21T19:29:02.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.139+0000 7f4ac37fe700 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1ad4b90 0x560db1ad6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.141+0000 7f4ac37fe700 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1ad4b90 0x560db1ad6fc0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c9e1cf52e876511 server_cookie=ab5c0ed9a4385455 in_seq=0 out_seq=0 2023-07-21T19:29:02.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.143+0000 7f4aa2ffd700 10 client.29356 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.144+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/4227281592 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560db1a05e50 con 0x560db1ad4b90 2023-07-21T19:29:02.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.144+0000 7f4aa2ffd700 10 client.29356 handle_command_reply: tid=0 2023-07-21T19:29:02.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.145+0000 7f4add7c6b80 1 client.29356 shutdown 2023-07-21T19:29:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 msgr2=0x560db1ac4800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4abc004570 tx=0x7f4abc009f10 comp rx=0 tx=0).stop 2023-07-21T19:29:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 shutdown_connections 2023-07-21T19:29:02.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1ad4b90 0x560db1ad6fc0 crc :-1 s=READY pgs=433 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db19176b0 0x560db1ac7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/4227281592 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1abb580 0x560db1ac4800 secure :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f4abc004570 tx=0x7f4abc009f10 comp rx=0 tx=0).stop 2023-07-21T19:29:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.146+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 >> 172.21.15.5:0/4227281592 conn(0x560db1ab0be0 msgr2=0x560db1ab4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:02.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.148+0000 7f4aa2ffd700 0 client.29356 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.148+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 shutdown_connections 2023-07-21T19:29:02.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.149+0000 7f4add7c6b80 1 -- 172.21.15.5:0/4227281592 wait complete. 2023-07-21T19:29:02.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.149+0000 7f4add7c6b80 20 client.29356 trim_cache size 0 max 16384 2023-07-21T19:29:02.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:01 smithi005 ceph-mon[126460]: pgmap v675: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-21T19:29:02.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:01 smithi156 ceph-mon[128316]: pgmap v675: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 289 KiB/s rd, 11 MiB/s wr, 843 op/s 2023-07-21T19:29:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.177+0000 7f4add7c6b80 1 Processor -- start 2023-07-21T19:29:02.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.177+0000 7f4add7c6b80 1 -- start start 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.177+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 0x560db1acbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.177+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.178+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1ab5c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.178+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1ab2d00 con 0x560db1ac7660 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.178+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1ac5de0 con 0x560db1ac12d0 2023-07-21T19:29:02.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.178+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1ab25f0 con 0x560db19176b0 2023-07-21T19:29:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.180+0000 7f4ac37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 0x560db1acbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.180+0000 7f4ac37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 0x560db1acbc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38070/0 (socket says 172.21.15.5:38070) 2023-07-21T19:29:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.180+0000 7f4ac37fe700 1 -- 172.21.15.5:0/520996371 learned_addr learned my addr 172.21.15.5:0/520996371 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:02.169 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.181+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1ab5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.182+0000 7f4ac3fff700 1 -- 172.21.15.5:0/520996371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 msgr2=0x560db1acbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.182+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 0x560db1acbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.182+0000 7f4ac3fff700 1 -- 172.21.15.5:0/520996371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 msgr2=0x560db1ac7120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.182+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.170 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.182+0000 7f4ac3fff700 1 -- 172.21.15.5:0/520996371 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560db18611f0 con 0x560db1ac7660 2023-07-21T19:29:02.171 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.183+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1ab5c60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c97b0 tx=0x7f4ab8098ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cac03282f57b6ce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.185+0000 7f4aaa7fc700 1 -- 172.21.15.5:0/520996371 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab809d190 con 0x560db1ac7660 2023-07-21T19:29:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.185+0000 7f4aaa7fc700 1 -- 172.21.15.5:0/520996371 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ab80a15b0 con 0x560db1ac7660 2023-07-21T19:29:02.173 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.185+0000 7f4aaa7fc700 1 -- 172.21.15.5:0/520996371 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8099620 con 0x560db1ac7660 2023-07-21T19:29:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/520996371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 msgr2=0x560db1ab5c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:02.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1ab5c60 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c97b0 tx=0x7f4ab8098ac0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 -- 172.21.15.5:0/520996371 shutdown_connections 2023-07-21T19:29:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1ab5c60 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db19176b0 0x560db1acbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/520996371 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db1ac12d0 0x560db1ac7120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 -- 172.21.15.5:0/520996371 >> 172.21.15.5:0/520996371 conn(0x560db1ab0be0 msgr2=0x560db1ac1900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.188+0000 7f4add7c6b80 1 -- 172.21.15.5:0/520996371 shutdown_connections 2023-07-21T19:29:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.189+0000 7f4add7c6b80 1 -- 172.21.15.5:0/520996371 wait complete. 2023-07-21T19:29:02.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.189+0000 7f4add7c6b80 1 -- start start 2023-07-21T19:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.189+0000 7f4add7c6b80 1 Processor -- start 2023-07-21T19:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db19176b0 0x560db1acbe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abcc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1abd920 con 0x560db1ac7660 2023-07-21T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1abda90 con 0x560db19176b0 2023-07-21T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560db1b28ab0 con 0x560db1ac12d0 2023-07-21T19:29:02.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.190+0000 7f4add7c6b80 10 client.0 _ll_register_callbacks cb 0x7f4abc00ece0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:34016/0 (socket says 172.21.15.5:34016) 2023-07-21T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 -- 172.21.15.5:0/559821975 learned_addr learned my addr 172.21.15.5:0/559821975 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 -- 172.21.15.5:0/559821975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 msgr2=0x560db1abcc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abcc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 -- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db19176b0 msgr2=0x560db1acbe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db19176b0 0x560db1acbe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.191+0000 7f4ac3fff700 1 -- 172.21.15.5:0/559821975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac411b490 con 0x560db1ac7660 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4ac3fff700 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c9780 tx=0x7f4ab80992d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c36661805f8007eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4aa2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab8098650 con 0x560db1ac7660 2023-07-21T19:29:02.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(118..118 src has 1..118) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4ab80987b0 con 0x560db1ac7660 2023-07-21T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4abc00ee60 con 0x560db1ac7660 2023-07-21T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ab80993c0 con 0x560db1ac7660 2023-07-21T19:29:02.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4add7c6b80 10 client.19725 fetch_fsmap learned FSMap version 57 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.192+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560db1839250 con 0x560db1ac7660 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4abc00ee60 con 0x560db1ac7660 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f4ab80a1120 con 0x560db1ac7660 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 10 client.19725 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 10 client.19725 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 20 client.19725 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1b29d30 0x560db1b2c140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:02.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 4 client.19725 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560db19c7ff0 con 0x560db1b29d30 2023-07-21T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.194+0000 7f4ac37fe700 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1b29d30 0x560db1b2c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.196+0000 7f4ac37fe700 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1b29d30 0x560db1b2c140 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=49b71adf81af7551 server_cookie=cd6dbeba8a56365 in_seq=0 out_seq=0 2023-07-21T19:29:02.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:02.196+0000 7f4aa2ffd700 10 client.19725 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:03.174 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.186+0000 7f4aa2ffd700 1 -- 172.21.15.5:0/559821975 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560db19c7ff0 con 0x560db1b29d30 2023-07-21T19:29:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.186+0000 7f4aa2ffd700 10 client.19725 handle_command_reply: tid=0 2023-07-21T19:29:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.186+0000 7f4add7c6b80 1 client.19725 shutdown 2023-07-21T19:29:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 msgr2=0x560db1abd210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:03.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80c9780 tx=0x7f4ab80992d0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 shutdown_connections 2023-07-21T19:29:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560db1b29d30 0x560db1b2c140 crc :-1 s=READY pgs=434 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560db1ac7660 0x560db1abd210 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560db19176b0 0x560db1acbe60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 --2- 172.21.15.5:0/559821975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560db1ac12d0 0x560db1abcc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 >> 172.21.15.5:0/559821975 conn(0x560db1ab0be0 msgr2=0x560db1ac2190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4aa2ffd700 0 client.19725 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 shutdown_connections 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.187+0000 7f4add7c6b80 1 -- 172.21.15.5:0/559821975 wait complete. 2023-07-21T19:29:03.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.188+0000 7f4add7c6b80 20 client.19725 trim_cache size 0 max 16384 2023-07-21T19:29:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:29:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:29:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:29:03.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:29:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.188+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 msgr2=0x7f4aac0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.188+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 0x7f4aac0a3980 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f4abc004c00 tx=0x7f4abc004ad0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.188+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 msgr2=0x7f4ac410d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.188+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f4ab80077c0 tx=0x7f4ab8007b80 comp rx=0 tx=0).stop 2023-07-21T19:29:03.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 shutdown_connections 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4aac0a14d0 0x7f4aac0a3980 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ac4110b50 0x7f4ac410d620 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ac4107780 0x7f4ac4110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 --2- 172.21.15.5:0/3543289957 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ac4108090 0x7f4ac410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 >> 172.21.15.5:0/3543289957 conn(0x7f4ac406c270 msgr2=0x7f4ac406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 shutdown_connections 2023-07-21T19:29:03.180 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:03.189+0000 7f4ac9dff700 1 -- 172.21.15.5:0/3543289957 wait complete. 2023-07-21T19:29:03.686 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: *** Caught signal (Segmentation fault) ** 2023-07-21T19:29:03.686 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: in thread 7fad19010700 thread_name:md_log_replay 2023-07-21T19:29:03.686 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T19:29:03.686 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad29bbecf0] 2023-07-21T19:29:03.686 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2: /usr/bin/ceph-mds(+0x4c518f) [0x555bff2da18f] 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555bff2dc3c3] 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 4: (MDLog::_replay_thread()+0x66d) [0x555bff34932d] 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555bfeff5d71] 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fad29bb41ca] 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 7: clone() 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.396+0000 7fad19010700 -1 *** Caught signal (Segmentation fault) ** 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: in thread 7fad19010700 thread_name:md_log_replay 2023-07-21T19:29:03.687 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad29bbecf0] 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2: /usr/bin/ceph-mds(+0x4c518f) [0x555bff2da18f] 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555bff2dc3c3] 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 4: (MDLog::_replay_thread()+0x66d) [0x555bff34932d] 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555bfeff5d71] 2023-07-21T19:29:03.688 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fad29bb41ca] 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 7: clone() 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 0> 2023-07-21T19:29:03.396+0000 7fad19010700 -1 *** Caught signal (Segmentation fault) ** 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: in thread 7fad19010700 thread_name:md_log_replay 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.689 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad29bbecf0] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2: /usr/bin/ceph-mds(+0x4c518f) [0x555bff2da18f] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555bff2dc3c3] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 4: (MDLog::_replay_thread()+0x66d) [0x555bff34932d] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555bfeff5d71] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fad29bb41ca] 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 7: clone() 2023-07-21T19:29:03.690 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 0> 2023-07-21T19:29:03.396+0000 7fad19010700 -1 *** Caught signal (Segmentation fault) ** 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: in thread 7fad19010700 thread_name:md_log_replay 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: ceph version 18.0.0-4746-g81cf785a (81cf785aa7781c4e08381c540f890cb9bc987ef9) reef (dev) 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 1: /lib64/libpthread.so.0(+0x12cf0) [0x7fad29bbecf0] 2023-07-21T19:29:03.691 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2: /usr/bin/ceph-mds(+0x4c518f) [0x555bff2da18f] 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 3: (MDSTableClient::got_journaled_ack(unsigned long)+0x173) [0x555bff2dc3c3] 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 4: (MDLog::_replay_thread()+0x66d) [0x555bff34932d] 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 5: (MDLog::ReplayThread::entry()+0x11) [0x555bfeff5d71] 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 6: /lib64/libpthread.so.0(+0x81ca) [0x7fad29bb41ca] 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 7: clone() 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: NOTE: a copy of the executable, or `objdump -rdS ` is needed to interpret this. 2023-07-21T19:29:03.692 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[153048]: 2023-07-21T19:29:03.693 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:03 smithi101 ceph-mon[128136]: pgmap v676: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-21T19:29:04.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:03 smithi005 ceph-mon[126460]: pgmap v676: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-21T19:29:04.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:03 smithi156 ceph-mon[128316]: pgmap v676: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 473 KiB/s rd, 18 MiB/s wr, 1.37k op/s 2023-07-21T19:29:04.570 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:04 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service: Main process exited, code=exited, status=139/n/a 2023-07-21T19:29:05.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:04 smithi101 ceph-mon[128136]: pgmap v677: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 12 MiB/s wr, 956 op/s 2023-07-21T19:29:05.070 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:04 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service: Failed with result 'exit-code'. 2023-07-21T19:29:05.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:04 smithi005 ceph-mon[126460]: pgmap v677: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 12 MiB/s wr, 956 op/s 2023-07-21T19:29:05.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:04 smithi156 ceph-mon[128316]: pgmap v677: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 365 KiB/s rd, 12 MiB/s wr, 956 op/s 2023-07-21T19:29:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 155.0 s, 254.5 tps, lat 364.197 ms stddev 136.544 2023-07-21T19:29:07.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:06 smithi005 ceph-mon[126460]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:29:07.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:06 smithi005 ceph-mon[126460]: pgmap v679: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-21T19:29:07.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:06 smithi156 ceph-mon[128316]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:29:07.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:06 smithi156 ceph-mon[128316]: pgmap v679: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-21T19:29:07.564 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:06 smithi101 ceph-mon[128136]: osdmap e119: 12 total, 12 up, 12 in 2023-07-21T19:29:07.565 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:06 smithi101 ceph-mon[128136]: pgmap v679: 97 pgs: 97 active+clean; 15 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 438 KiB/s rd, 15 MiB/s wr, 1.15k op/s 2023-07-21T19:29:10.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:09 smithi005 ceph-mon[126460]: pgmap v680: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:10.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:09 smithi156 ceph-mon[128316]: pgmap v680: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:11.449 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:09 smithi101 ceph-mon[128136]: pgmap v680: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:11.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 160.0 s, 406.5 tps, lat 267.325 ms stddev 157.145 2023-07-21T19:29:12.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:11 smithi101 ceph-mon[128136]: pgmap v681: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:12.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:11 smithi005 ceph-mon[126460]: pgmap v681: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:12.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:11 smithi156 ceph-mon[128316]: pgmap v681: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 501 KiB/s rd, 12 MiB/s wr, 1.01k op/s 2023-07-21T19:29:14.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:13 smithi101 ceph-mon[128136]: pgmap v682: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:14.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:13 smithi101 ceph-mon[128136]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:29:14.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:13 smithi005 ceph-mon[126460]: pgmap v682: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:14.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:13 smithi005 ceph-mon[126460]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:29:14.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:13 smithi156 ceph-mon[128316]: pgmap v682: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:14.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:13 smithi156 ceph-mon[128316]: osdmap e120: 12 total, 12 up, 12 in 2023-07-21T19:29:15.319 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:14 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service: Service RestartSec=10s expired, scheduling restart. 2023-07-21T19:29:15.319 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:14 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service: Scheduled restart job, restart counter is at 1. 2023-07-21T19:29:15.319 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:14 smithi101 systemd[1]: Stopped Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:29:15.320 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:14 smithi101 systemd[1]: Starting Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:29:16.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:15 smithi101 ceph-mon[128136]: pgmap v684: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-21T19:29:16.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:15 smithi005 ceph-mon[126460]: pgmap v684: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-21T19:29:16.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:15 smithi156 ceph-mon[128316]: pgmap v684: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 706 KiB/s rd, 13 MiB/s wr, 939 op/s 2023-07-21T19:29:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.518+0000 7fc2b5fff700 1 -- 172.21.15.5:0/1295230624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b010e2a0 msgr2=0x7fc2b010e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.518+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/1295230624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b010e2a0 0x7fc2b010e670 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0009990 tx=0x7fc2a0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 -- 172.21.15.5:0/1295230624 shutdown_connections 2023-07-21T19:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/1295230624 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 0x7fc2b011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/1295230624 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b010e2a0 0x7fc2b010e670 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/1295230624 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b006c910 0x7fc2b006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 -- 172.21.15.5:0/1295230624 >> 172.21.15.5:0/1295230624 conn(0x7fc2b006bd70 msgr2=0x7fc2b0070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.519+0000 7fc2b5fff700 1 -- 172.21.15.5:0/1295230624 shutdown_connections 2023-07-21T19:29:16.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.520+0000 7fc2b5fff700 1 -- 172.21.15.5:0/1295230624 wait complete. 2023-07-21T19:29:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.521+0000 7fc2b5fff700 1 Processor -- start 2023-07-21T19:29:16.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.522+0000 7fc2b5fff700 1 -- start start 2023-07-21T19:29:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.522+0000 7fc2b5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.522+0000 7fc2b5fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 0x7fc2b01aaff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.522+0000 7fc2b5fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 0x7fc2b01a4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2b5fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01a5050 con 0x7fc2b010e2a0 2023-07-21T19:29:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2b5fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01a51c0 con 0x7fc2b006d3a0 2023-07-21T19:29:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2b5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b01a5380 con 0x7fc2b006c910 2023-07-21T19:29:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60500/0 (socket says 172.21.15.5:60500) 2023-07-21T19:29:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.523+0000 7fc2af7fe700 1 -- 172.21.15.5:0/2976239319 learned_addr learned my addr 172.21.15.5:0/2976239319 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 0x7fc2b01aaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2affff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 0x7fc2b01a4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2af7fe700 1 -- 172.21.15.5:0/2976239319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 msgr2=0x7fc2b01aaff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 0x7fc2b01aaff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2af7fe700 1 -- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 msgr2=0x7fc2b01a4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 0x7fc2b01a4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2af7fe700 1 -- 172.21.15.5:0/2976239319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2a00095f0 con 0x7fc2b006c910 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.524+0000 7fc2affff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 0x7fc2b01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0000c00 tx=0x7fc2a0009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2ad7fa700 1 -- 172.21.15.5:0/2976239319 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a001d070 con 0x7fc2b006c910 2023-07-21T19:29:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2ad7fa700 1 -- 172.21.15.5:0/2976239319 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a0004000 con 0x7fc2b006c910 2023-07-21T19:29:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2ad7fa700 1 -- 172.21.15.5:0/2976239319 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000fc30 con 0x7fc2b006c910 2023-07-21T19:29:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b01a5600 con 0x7fc2b006c910 2023-07-21T19:29:16.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.525+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b01b1a30 con 0x7fc2b006c910 2023-07-21T19:29:16.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.532+0000 7fc2ad7fa700 1 -- 172.21.15.5:0/2976239319 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc2a0003b70 con 0x7fc2b006c910 2023-07-21T19:29:16.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.534+0000 7fc2c9956b80 1 Processor -- start 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 -- start start 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d85d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 0x560232d809d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232c85c50 con 0x560232d80580 2023-07-21T19:29:16.525 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232cd89e0 con 0x560232d862d0 2023-07-21T19:29:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.535+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232cd1e70 con 0x560232bdc6b0 2023-07-21T19:29:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.536+0000 7fc2affff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d85d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.536+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.536+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d80040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59964/0 (socket says 172.21.15.5:59964) 2023-07-21T19:29:16.526 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.536+0000 7fc2af7fe700 1 -- 172.21.15.5:0/3873937215 learned_addr learned my addr 172.21.15.5:0/3873937215 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.536+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 0x560232d809d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2ad7fa700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 0x7fc2900a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2ad7fa700 1 -- 172.21.15.5:0/2976239319 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc2a00cdd70 con 0x7fc2b006c910 2023-07-21T19:29:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/3873937215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 msgr2=0x560232d85d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.527 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d85d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/3873937215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 msgr2=0x560232d80040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d80040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.537+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/3873937215 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560232affd80 con 0x560232d80580 2023-07-21T19:29:16.528 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.538+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 0x560232d809d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc2b01a6350 tx=0x7fc298007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ca42708a76efdc3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.538+0000 7fc2affff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 0x7fc2900a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.538+0000 7fc2a5ffb700 1 -- 172.21.15.5:0/3873937215 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2980125f0 con 0x560232d80580 2023-07-21T19:29:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.538+0000 7fc2a5ffb700 1 -- 172.21.15.5:0/3873937215 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc298012be0 con 0x560232d80580 2023-07-21T19:29:16.529 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.538+0000 7fc2a5ffb700 1 -- 172.21.15.5:0/3873937215 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298013c40 con 0x560232d80580 2023-07-21T19:29:16.530 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 -- 172.21.15.5:0/3873937215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 msgr2=0x560232d809d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 0x560232d809d0 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7fc2b01a6350 tx=0x7fc298007a00 comp rx=0 tx=0).stop 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 -- 172.21.15.5:0/3873937215 shutdown_connections 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d80580 0x560232d809d0 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d85d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/3873937215 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d80040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.531 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 -- 172.21.15.5:0/3873937215 >> 172.21.15.5:0/3873937215 conn(0x560232d75be0 msgr2=0x560232d7af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2affff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 0x7fc2900a3830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2b01ab740 tx=0x7fc2a8004ad0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.539+0000 7fc2c9956b80 1 -- 172.21.15.5:0/3873937215 shutdown_connections 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.540+0000 7fc2c9956b80 1 -- 172.21.15.5:0/3873937215 wait complete. 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.540+0000 7fc2c9956b80 1 -- start start 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 Processor -- start 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232bdc6b0 0x560232d8c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.532 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d89dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232c71a60 con 0x560232bdc6b0 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232c71760 con 0x560232d862d0 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232b558c0 con 0x560232d80580 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.541+0000 7fc2c9956b80 10 client.0 _ll_register_callbacks cb 0x560232c6f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.533 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60520/0 (socket says 172.21.15.5:60520) 2023-07-21T19:29:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2607843962 learned_addr learned my addr 172.21.15.5:0/2607843962 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:16.534 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2affff700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d89dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 msgr2=0x560232d89dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d89dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2607843962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232bdc6b0 msgr2=0x560232d8c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.535 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232bdc6b0 0x560232d8c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.542+0000 7fc2a6ffd700 1 -- 172.21.15.5:0/2607843962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29800e040 con 0x560232d80580 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.543+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc298012f80 tx=0x7fc298012fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25741ccc3e4d60af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.543+0000 7fc29effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc298013820 con 0x560232d80580 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc298013980 con 0x560232d80580 2023-07-21T19:29:16.536 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560232c2b490 con 0x560232d80580 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc298013b40 con 0x560232d80580 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc2c9956b80 10 client.29371 fetch_fsmap learned FSMap version 57 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.544+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560232ae8c00 con 0x560232d80580 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.545+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560232c2b490 con 0x560232d80580 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc29802d020 con 0x560232d80580 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 10 client.29371 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 10 client.29371 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:16.537 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 20 client.29371 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232d99a50 0x560232d9be60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 4 client.29371 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:16.538 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.546+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560232ccae50 con 0x560232d99a50 2023-07-21T19:29:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.547+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232d99a50 0x560232d9be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.548+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232d99a50 0x560232d9be60 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f0d9df231c48a2b server_cookie=bd4a8c7c5d7320d8 in_seq=0 out_seq=0 2023-07-21T19:29:16.539 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.548+0000 7fc29effd700 10 client.29371 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.549+0000 7fc29effd700 1 -- 172.21.15.5:0/2607843962 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560232ccae50 con 0x560232d99a50 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.549+0000 7fc29effd700 10 client.29371 handle_command_reply: tid=0 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.549+0000 7fc2c9956b80 1 client.29371 shutdown 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.549+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 msgr2=0x560232d897a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.549+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc298012f80 tx=0x7fc298012fb0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.540 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 shutdown_connections 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d862d0 0x560232d89dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232d99a50 0x560232d9be60 crc :-1 s=READY pgs=435 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232bdc6b0 0x560232d8c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/2607843962 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232d80580 0x560232d897a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 >> 172.21.15.5:0/2607843962 conn(0x560232d75be0 msgr2=0x560232d79c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc29effd700 0 client.29371 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:16.541 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 shutdown_connections 2023-07-21T19:29:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.550+0000 7fc2c9956b80 1 -- 172.21.15.5:0/2607843962 wait complete. 2023-07-21T19:29:16.542 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.551+0000 7fc2c9956b80 20 client.29371 trim_cache size 0 max 16384 2023-07-21T19:29:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.573+0000 7fc2c9956b80 1 Processor -- start 2023-07-21T19:29:16.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.573+0000 7fc2c9956b80 1 -- start start 2023-07-21T19:29:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d87010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d8c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d7ba00 con 0x560232d862d0 2023-07-21T19:29:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d77d20 con 0x560232d8c0c0 2023-07-21T19:29:16.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d7bfb0 con 0x560232bdc6b0 2023-07-21T19:29:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60522/0 (socket says 172.21.15.5:60522) 2023-07-21T19:29:16.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.574+0000 7fc2af7fe700 1 -- 172.21.15.5:0/4025599886 learned_addr learned my addr 172.21.15.5:0/4025599886 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:16.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2affff700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d8c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d87010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 -- 172.21.15.5:0/4025599886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 msgr2=0x560232d8c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d8c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 -- 172.21.15.5:0/4025599886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 msgr2=0x560232d87010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d87010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 -- 172.21.15.5:0/4025599886 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560232b261f0 con 0x560232bdc6b0 2023-07-21T19:29:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:29:16.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.575+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000b5c0 tx=0x7fc2a00d0240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76b5d1bf5ae561de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.576+0000 7fc2a67fc700 1 -- 172.21.15.5:0/4025599886 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a000f460 con 0x560232bdc6b0 2023-07-21T19:29:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.576+0000 7fc2a67fc700 1 -- 172.21.15.5:0/4025599886 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a0003e20 con 0x560232bdc6b0 2023-07-21T19:29:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.576+0000 7fc2a67fc700 1 -- 172.21.15.5:0/4025599886 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a0099270 con 0x560232bdc6b0 2023-07-21T19:29:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 -- 172.21.15.5:0/4025599886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 msgr2=0x560232d90c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fc2a000b5c0 tx=0x7fc2a00d0240 comp rx=0 tx=0).stop 2023-07-21T19:29:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 -- 172.21.15.5:0/4025599886 shutdown_connections 2023-07-21T19:29:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d8c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d90c60 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/4025599886 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d87010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 -- 172.21.15.5:0/4025599886 >> 172.21.15.5:0/4025599886 conn(0x560232d75be0 msgr2=0x560232d86900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 -- 172.21.15.5:0/4025599886 shutdown_connections 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.577+0000 7fc2c9956b80 1 -- 172.21.15.5:0/4025599886 wait complete. 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.578+0000 7fc2c9956b80 1 -- start start 2023-07-21T19:29:16.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.578+0000 7fc2c9956b80 1 Processor -- start 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d83090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d81790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d81cd0 con 0x560232d862d0 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d81e40 con 0x560232d8c0c0 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560232d82000 con 0x560232bdc6b0 2023-07-21T19:29:16.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2c9956b80 10 client.0 _ll_register_callbacks cb 0x7fc2980075a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2af7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60526/0 (socket says 172.21.15.5:60526) 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2af7fe700 1 -- 172.21.15.5:0/715682884 learned_addr learned my addr 172.21.15.5:0/715682884 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.579+0000 7fc2affff700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d81790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 -- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 msgr2=0x560232d81790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d81790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 -- 172.21.15.5:0/715682884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 msgr2=0x560232d83090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d83090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 -- 172.21.15.5:0/715682884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2a00a0040 con 0x560232bdc6b0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2af7fe700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00cbac0 tx=0x7fc2a00cb450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=934454e68937f49 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc29effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2a009aed0 con 0x560232bdc6b0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc2a0004170 con 0x560232bdc6b0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.580+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc298012be0 con 0x560232bdc6b0 2023-07-21T19:29:16.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.581+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=57) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2a00a3290 con 0x560232bdc6b0 2023-07-21T19:29:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.581+0000 7fc2c9956b80 10 client.29377 fetch_fsmap learned FSMap version 57 2023-07-21T19:29:16.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.581+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=57,monmap=4+}) v3 -- 0x560232afe250 con 0x560232bdc6b0 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.581+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc2a007d080 con 0x560232bdc6b0 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.581+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 57) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc2a009b1c0 con 0x560232bdc6b0 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 10 client.29377 fetch_fsmap finished waiting for FSMap version 57 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 10 client.29377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 20 client.29377 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232dee2f0 0x560232df06d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:16.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 4 client.29377 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:29:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560232c8cff0 con 0x560232dee2f0 2023-07-21T19:29:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.582+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232dee2f0 0x560232df06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.583+0000 7fc29effd700 10 client.29377 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:16.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.583+0000 7fc2a6ffd700 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232dee2f0 0x560232df06d0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=21b614a5eb334e37 server_cookie=f97a787849104e5f in_seq=0 out_seq=0 2023-07-21T19:29:16.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.607+0000 7fc29effd700 1 -- 172.21.15.5:0/715682884 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560232c8cff0 con 0x560232dee2f0 2023-07-21T19:29:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.607+0000 7fc29effd700 10 client.29377 handle_command_reply: tid=0 2023-07-21T19:29:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.607+0000 7fc2c9956b80 1 client.29377 shutdown 2023-07-21T19:29:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 msgr2=0x560232d82b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc2a00cbac0 tx=0x7fc2a00cb450 comp rx=0 tx=0).stop 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 shutdown_connections 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560232d8c0c0 0x560232d81790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560232bdc6b0 0x560232d82b50 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560232dee2f0 0x560232df06d0 crc :-1 s=READY pgs=436 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 --2- 172.21.15.5:0/715682884 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560232d862d0 0x560232d83090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 >> 172.21.15.5:0/715682884 conn(0x560232d75be0 msgr2=0x560232d8b940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc29effd700 0 client.29377 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 shutdown_connections 2023-07-21T19:29:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 1 -- 172.21.15.5:0/715682884 wait complete. 2023-07-21T19:29:16.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.608+0000 7fc2c9956b80 20 client.29377 trim_cache size 0 max 16384 2023-07-21T19:29:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:29:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:29:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:29:16.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 msgr2=0x7fc2900a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 0x7fc2900a3830 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc2b01ab740 tx=0x7fc2a8004ad0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 msgr2=0x7fc2b01aaab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc2a0000c00 tx=0x7fc2a0009580 comp rx=0 tx=0).stop 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 shutdown_connections 2023-07-21T19:29:16.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc2900a1380 0x7fc2900a3830 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc2b010e2a0 0x7fc2b01a4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc2b006c910 0x7fc2b01aaab0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 --2- 172.21.15.5:0/2976239319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc2b006d3a0 0x7fc2b01aaff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 >> 172.21.15.5:0/2976239319 conn(0x7fc2b006bd70 msgr2=0x7fc2b0070800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.609+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 shutdown_connections 2023-07-21T19:29:16.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:16.610+0000 7fc2b5fff700 1 -- 172.21.15.5:0/2976239319 wait complete. 2023-07-21T19:29:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 165.0 s, 422.8 tps, lat 237.272 ms stddev 78.460 2023-07-21T19:29:18.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:17 smithi101 ceph-mon[128136]: pgmap v685: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:18.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:17 smithi101 ceph-mon[128136]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-07-21T19:29:18.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:17 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2023-07-21T19:29:18.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:17 smithi005 ceph-mon[126460]: pgmap v685: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:18.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:17 smithi005 ceph-mon[126460]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-07-21T19:29:18.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:17 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2023-07-21T19:29:18.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:17 smithi156 ceph-mon[128316]: pgmap v685: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 608 KiB/s rd, 11 MiB/s wr, 809 op/s 2023-07-21T19:29:18.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:17 smithi156 ceph-mon[128316]: MDS daemon mds.e is removed because it is dead or otherwise unavailable. 2023-07-21T19:29:18.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:17 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 5 up:standby 2023-07-21T19:29:19.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:18 smithi101 ceph-mon[128136]: pgmap v686: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:18 smithi005 ceph-mon[126460]: pgmap v686: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:19.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:18 smithi156 ceph-mon[128316]: pgmap v686: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:21.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:21 smithi101 ceph-mon[128136]: pgmap v687: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:21.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:21 smithi005 ceph-mon[126460]: pgmap v687: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:21.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:21 smithi156 ceph-mon[128316]: pgmap v687: 97 pgs: 97 active+clean; 13 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 733 KiB/s rd, 12 MiB/s wr, 846 op/s 2023-07-21T19:29:21.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 170.0 s, 397.2 tps, lat 251.828 ms stddev 80.383 2023-07-21T19:29:22.152 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:29:22.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 -- 172.21.15.5:0/3102843333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef8106a70 msgr2=0x7fdef8112f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 --2- 172.21.15.5:0/3102843333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef8106a70 0x7fdef8112f70 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009920 tx=0x7fdee8009c30 comp rx=0 tx=0).stop 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 -- 172.21.15.5:0/3102843333 shutdown_connections 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 --2- 172.21.15.5:0/3102843333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef8106a70 0x7fdef8112f70 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 --2- 172.21.15.5:0/3102843333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef810c240 0x7fdef810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 --2- 172.21.15.5:0/3102843333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810cb50 0x7fdef8106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 -- 172.21.15.5:0/3102843333 >> 172.21.15.5:0/3102843333 conn(0x7fdef80759e0 msgr2=0x7fdef8077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 -- 172.21.15.5:0/3102843333 shutdown_connections 2023-07-21T19:29:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.404+0000 7fdefefff700 1 -- 172.21.15.5:0/3102843333 wait complete. 2023-07-21T19:29:22.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.406+0000 7fdefefff700 1 Processor -- start 2023-07-21T19:29:22.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.406+0000 7fdefefff700 1 -- start start 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 0x7fdef819c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 0x7fdef819c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef810cb50 0x7fdef81a0c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81a1280 con 0x7fdef810c240 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81a13f0 con 0x7fdef8106a70 2023-07-21T19:29:22.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.407+0000 7fdefefff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef81a15b0 con 0x7fdef810cb50 2023-07-21T19:29:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.408+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 0x7fdef819c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.408+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 0x7fdef819c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53870/0 (socket says 172.21.15.5:53870) 2023-07-21T19:29:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.408+0000 7fdef7fff700 1 -- 172.21.15.5:0/1709660174 learned_addr learned my addr 172.21.15.5:0/1709660174 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:22.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.408+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 0x7fdef819c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 -- 172.21.15.5:0/1709660174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef810cb50 msgr2=0x7fdef81a0c40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:29:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef810cb50 0x7fdef81a0c40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 -- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 msgr2=0x7fdef819c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 0x7fdef819c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 -- 172.21.15.5:0/1709660174 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdef0009c40 con 0x7fdef8106a70 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 0x7fdef819c2a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdeec005fd0 tx=0x7fdeec00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdef5ffb700 1 -- 172.21.15.5:0/1709660174 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec004220 con 0x7fdef8106a70 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdef5ffb700 1 -- 172.21.15.5:0/1709660174 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdeec004380 con 0x7fdef8106a70 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.409+0000 7fdef5ffb700 1 -- 172.21.15.5:0/1709660174 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec003ad0 con 0x7fdef8106a70 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.410+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee8009520 con 0x7fdef8106a70 2023-07-21T19:29:22.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.410+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef81a8ef0 con 0x7fdef8106a70 2023-07-21T19:29:22.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.415+0000 7fdef5ffb700 1 -- 172.21.15.5:0/1709660174 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdeec003c30 con 0x7fdef8106a70 2023-07-21T19:29:22.406 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.417+0000 7fdf129b6b80 1 Processor -- start 2023-07-21T19:29:22.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 -- start start 2023-07-21T19:29:22.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f3b2cb0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.419+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f405a40 con 0x55cd7f4acbe0 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f3feed0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdefd59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdefd59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60542/0 (socket says 172.21.15.5:60542) 2023-07-21T19:29:22.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdefd59c700 1 -- 172.21.15.5:0/780535848 learned_addr learned my addr 172.21.15.5:0/780535848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdef5ffb700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 0x7fdee00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdef7fff700 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.420+0000 7fdef5ffb700 1 -- 172.21.15.5:0/1709660174 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdeec014070 con 0x7fdef8106a70 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 -- 172.21.15.5:0/780535848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 msgr2=0x55cd7f4b7590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 -- 172.21.15.5:0/780535848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 msgr2=0x55cd7f4b9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 -- 172.21.15.5:0/780535848 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7f22cde0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee800bf20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=148550b5a6d68baa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.421+0000 7fdefd59c700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 0x7fdee00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.422+0000 7fdedeffd700 1 -- 172.21.15.5:0/780535848 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee800f460 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.422+0000 7fdedeffd700 1 -- 172.21.15.5:0/780535848 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdee800b560 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.422+0000 7fdedeffd700 1 -- 172.21.15.5:0/780535848 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8005540 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.423+0000 7fdf129b6b80 1 -- 172.21.15.5:0/780535848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 msgr2=0x55cd7f4ac6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.423+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdee8009230 tx=0x7fdee800bf20 comp rx=0 tx=0).stop 2023-07-21T19:29:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdf129b6b80 1 -- 172.21.15.5:0/780535848 shutdown_connections 2023-07-21T19:29:22.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4ac6a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/780535848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdf129b6b80 1 -- 172.21.15.5:0/780535848 >> 172.21.15.5:0/780535848 conn(0x55cd7f4a2da0 msgr2=0x55cd7f4a86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.424+0000 7fdefd59c700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 0x7fdee00a3870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005850 tx=0x7fdee8005780 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.425+0000 7fdf129b6b80 1 -- 172.21.15.5:0/780535848 shutdown_connections 2023-07-21T19:29:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.426+0000 7fdf129b6b80 1 -- 172.21.15.5:0/780535848 wait complete. 2023-07-21T19:29:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.426+0000 7fdf129b6b80 1 -- start start 2023-07-21T19:29:22.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.426+0000 7fdf129b6b80 1 Processor -- start 2023-07-21T19:29:22.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.427+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.427+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f39eac0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f39e7c0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f282920 con 0x55cd7f4acbe0 2023-07-21T19:29:22.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdf129b6b80 10 client.0 _ll_register_callbacks cb 0x55cd7f39c760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdefcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdefcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60028/0 (socket says 172.21.15.5:60028) 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdefcd9b700 1 -- 172.21.15.5:0/193153903 learned_addr learned my addr 172.21.15.5:0/193153903 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdef7fff700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.428+0000 7fdefd59c700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 -- 172.21.15.5:0/193153903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 msgr2=0x55cd7f4b47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 -- 172.21.15.5:0/193153903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 msgr2=0x55cd7f4b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 -- 172.21.15.5:0/193153903 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef8113800 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0a1d60 tx=0x7fdeec099a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d05c84439280542 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fded7fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.429+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec09a2a0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.430+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdeec0044f0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.430+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cd7f3584f0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.430+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeec01bdb0 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.430+0000 7fdf129b6b80 10 client.29462 fetch_fsmap learned FSMap version 58 2023-07-21T19:29:22.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.430+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x55cd7f215c60 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdeec0b6080 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 58) v1 ==== 6929+0+0 (secure 0 0 0) 0x7fdeec099330 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 10 client.29462 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T19:29:22.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 10 client.29462 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 20 client.29462 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f4d8420 0x55cd7f4da830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 4 client.29462 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.431+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cd7f3f7eb0 con 0x55cd7f4d8420 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.432+0000 7fdef7fff700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f4d8420 0x55cd7f4da830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.433+0000 7fdef7fff700 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f4d8420 0x55cd7f4da830 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f117213386b0bcd2 server_cookie=6058a326b51d87b3 in_seq=0 out_seq=0 2023-07-21T19:29:22.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.433+0000 7fded7fff700 10 client.29462 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.434+0000 7fded7fff700 1 -- 172.21.15.5:0/193153903 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cd7f3f7eb0 con 0x55cd7f4d8420 2023-07-21T19:29:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.434+0000 7fded7fff700 10 client.29462 handle_command_reply: tid=0 2023-07-21T19:29:22.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.434+0000 7fdf129b6b80 1 client.29462 shutdown 2023-07-21T19:29:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.435+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 msgr2=0x55cd7f4b74e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.435+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0a1d60 tx=0x7fdeec099a70 comp rx=0 tx=0).stop 2023-07-21T19:29:22.424 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 shutdown_connections 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f4d8420 0x55cd7f4da830 crc :-1 s=READY pgs=437 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4b74e0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/193153903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 >> 172.21.15.5:0/193153903 conn(0x55cd7f4a2da0 msgr2=0x55cd7f4a8230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.425 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fded7fff700 0 client.29462 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.436+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 shutdown_connections 2023-07-21T19:29:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.437+0000 7fdf129b6b80 1 -- 172.21.15.5:0/193153903 wait complete. 2023-07-21T19:29:22.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.438+0000 7fdf129b6b80 20 client.29462 trim_cache size 0 max 16384 2023-07-21T19:29:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.472+0000 7fdf129b6b80 1 Processor -- start 2023-07-21T19:29:22.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 -- start start 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f4a2970 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f4ae1a0 con 0x55cd7f4acbe0 2023-07-21T19:29:22.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.473+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f4a7490 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53904/0 (socket says 172.21.15.5:53904) 2023-07-21T19:29:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefcd9b700 1 -- 172.21.15.5:0/3740917 learned_addr learned my addr 172.21.15.5:0/3740917 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:22.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 -- 172.21.15.5:0/3740917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 msgr2=0x55cd7f4b7080 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7080 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 -- 172.21.15.5:0/3740917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 msgr2=0x55cd7f4cf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefd59c700 1 -- 172.21.15.5:0/3740917 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cd7f4a45e0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.464 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.474+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.475+0000 7fdefd59c700 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b1a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdee8025040 tx=0x7fdee802d000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5fa44d27b8fb2e44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.475+0000 7fdedf7fe700 1 -- 172.21.15.5:0/3740917 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee8005920 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.465 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.475+0000 7fdedf7fe700 1 -- 172.21.15.5:0/3740917 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdee8010cb0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.475+0000 7fdedf7fe700 1 -- 172.21.15.5:0/3740917 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee800f8c0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.476+0000 7fdf129b6b80 1 -- 172.21.15.5:0/3740917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 msgr2=0x55cd7f4b1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.476+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b1a40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fdee8025040 tx=0x7fdee802d000 comp rx=0 tx=0).stop 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 -- 172.21.15.5:0/3740917 shutdown_connections 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f4b1a40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f4cf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/3740917 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4acbe0 0x55cd7f4b7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.466 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 -- 172.21.15.5:0/3740917 >> 172.21.15.5:0/3740917 conn(0x55cd7f4a2da0 msgr2=0x55cd7f4afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 -- 172.21.15.5:0/3740917 shutdown_connections 2023-07-21T19:29:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 -- 172.21.15.5:0/3740917 wait complete. 2023-07-21T19:29:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.477+0000 7fdf129b6b80 1 -- start start 2023-07-21T19:29:22.467 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.478+0000 7fdf129b6b80 1 Processor -- start 2023-07-21T19:29:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f525670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f520230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.468 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f5208d0 con 0x55cd7f4b7ad0 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f520a40 con 0x55cd7f4ac2d0 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cd7f520c00 con 0x55cd7f4acbe0 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdf129b6b80 10 client.0 _ll_register_callbacks cb 0x7fdeec096270 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.479+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60572/0 (socket says 172.21.15.5:60572) 2023-07-21T19:29:22.469 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 -- 172.21.15.5:0/1955210879 learned_addr learned my addr 172.21.15.5:0/1955210879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f525670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 -- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 msgr2=0x55cd7f525670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f525670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 -- 172.21.15.5:0/1955210879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 msgr2=0x55cd7f520230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f520230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 -- 172.21.15.5:0/1955210879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cd7f215c60 con 0x55cd7f4acbe0 2023-07-21T19:29:22.471 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fdef7fff700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55cd7f4db100 tx=0x7fdef0009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c5e814baf7dc06 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.480+0000 7fded7fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef001e5e0 con 0x55cd7f4acbe0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdef001e860 con 0x55cd7f4acbe0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdee800b560 con 0x55cd7f4acbe0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdef001ea20 con 0x55cd7f4acbe0 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fdf129b6b80 10 client.29392 fetch_fsmap learned FSMap version 58 2023-07-21T19:29:22.472 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=4+}) v3 -- 0x7fdee801c040 con 0x55cd7f4acbe0 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.481+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdee800b560 con 0x55cd7f4acbe0 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 58) v1 ==== 6929+0+0 (secure 0 0 0) 0x7fdef0030b40 con 0x55cd7f4acbe0 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 10 client.29392 fetch_fsmap finished waiting for FSMap version 58 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 10 client.29392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 20 client.29392 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f52d4a0 0x55cd7f52f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 4 client.29392 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:29:22.473 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55cd7f3ba050 con 0x55cd7f52d4a0 2023-07-21T19:29:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.482+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f52d4a0 0x55cd7f52f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:22.474 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.483+0000 7fded7fff700 10 client.29392 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.483+0000 7fdefcd9b700 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f52d4a0 0x55cd7f52f8b0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4391766877a4ff9d server_cookie=3dd94f38145ee81 in_seq=0 out_seq=0 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.483+0000 7fded7fff700 1 -- 172.21.15.5:0/1955210879 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55cd7f3ba050 con 0x55cd7f52d4a0 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.483+0000 7fded7fff700 10 client.29392 handle_command_reply: tid=0 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.483+0000 7fdf129b6b80 1 client.29392 shutdown 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 msgr2=0x55cd7f525bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x55cd7f4db100 tx=0x7fdef0009920 comp rx=0 tx=0).stop 2023-07-21T19:29:22.475 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 shutdown_connections 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cd7f52d4a0 0x55cd7f52f8b0 crc :-1 s=READY pgs=438 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cd7f4b7ad0 0x55cd7f520230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cd7f4ac2d0 0x55cd7f525670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fded7fff700 0 client.29392 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 --2- 172.21.15.5:0/1955210879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cd7f4acbe0 0x55cd7f525bb0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 >> 172.21.15.5:0/1955210879 conn(0x55cd7f4a2da0 msgr2=0x55cd7f4ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 shutdown_connections 2023-07-21T19:29:22.476 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 1 -- 172.21.15.5:0/1955210879 wait complete. 2023-07-21T19:29:22.477 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:29:22.477 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:29:22.477 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:29:22.477 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:29:22.477 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:29:22.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:29:22.478 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:29:22.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:29:22.478 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:29:22.478 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.484+0000 7fdf129b6b80 20 client.29392 trim_cache size 0 max 16384 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 msgr2=0x7fdee00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 0x7fdee00a3870 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fdee8005850 tx=0x7fdee8005780 comp rx=0 tx=0).stop 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 msgr2=0x7fdef819c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 0x7fdef819c2a0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fdeec005fd0 tx=0x7fdeec00eca0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 shutdown_connections 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdef810cb50 0x7fdef81a0c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdef8106a70 0x7fdef819c2a0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdee00a13c0 0x7fdee00a3870 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 --2- 172.21.15.5:0/1709660174 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdef810c240 0x7fdef819c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 >> 172.21.15.5:0/1709660174 conn(0x7fdef80759e0 msgr2=0x7fdef80770a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 shutdown_connections 2023-07-21T19:29:22.480 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:22.485+0000 7fdefefff700 1 -- 172.21.15.5:0/1709660174 wait complete. 2023-07-21T19:29:22.503 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:29:23.070 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:22 smithi101 podman[160947]: 2023-07-21T19:29:24.071 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:23 smithi101 bash[160947]: 499339a374b4c15fc56a19aedda6f5b013fc6ace5892ce1a09ff554ceae712f0 2023-07-21T19:29:24.072 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:23 smithi101 ceph-mds[160989]: -- 172.21.15.101:0/1853818855 <== mon.1 v2:172.21.15.156:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5595278c31e0 con 0x559527898800 2023-07-21T19:29:24.072 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[160963]: starting mds.e at 2023-07-21T19:29:24.072 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:29:23 smithi101 systemd[1]: Started Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:29:24.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:23 smithi101 ceph-mon[128136]: pgmap v688: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-07-21T19:29:24.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:23 smithi005 ceph-mon[126460]: pgmap v688: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-07-21T19:29:24.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:23 smithi156 ceph-mon[128316]: pgmap v688: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 931 KiB/s rd, 12 MiB/s wr, 889 op/s 2023-07-21T19:29:26.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:25 smithi101 ceph-mon[128136]: pgmap v689: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-07-21T19:29:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:25 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:29:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:25 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6834/1958170175,v1:172.21.15.101:6835/1958170175] up:boot 2023-07-21T19:29:26.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:25 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:26.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:25 smithi005 ceph-mon[126460]: pgmap v689: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-07-21T19:29:26.161 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:25 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:29:26.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:25 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6834/1958170175,v1:172.21.15.101:6835/1958170175] up:boot 2023-07-21T19:29:26.162 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:25 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:26.163 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:25 smithi156 ceph-mon[128316]: pgmap v689: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 831 KiB/s rd, 10 MiB/s wr, 781 op/s 2023-07-21T19:29:26.163 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:25 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-07-21T19:29:26.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:25 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6834/1958170175,v1:172.21.15.101:6835/1958170175] up:boot 2023-07-21T19:29:26.164 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:25 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:26.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 175.0 s, 392.2 tps, lat 253.783 ms stddev 66.579 2023-07-21T19:29:28.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:27 smithi101 ceph-mon[128136]: pgmap v690: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 10 MiB/s wr, 765 op/s 2023-07-21T19:29:28.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:27 smithi005 ceph-mon[126460]: pgmap v690: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 10 MiB/s wr, 765 op/s 2023-07-21T19:29:28.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:27 smithi156 ceph-mon[128316]: pgmap v690: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 814 KiB/s rd, 10 MiB/s wr, 765 op/s 2023-07-21T19:29:29.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:28 smithi101 ceph-mon[128136]: pgmap v691: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-21T19:29:29.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:28 smithi005 ceph-mon[126460]: pgmap v691: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-21T19:29:29.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:28 smithi156 ceph-mon[128316]: pgmap v691: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 16 MiB/s wr, 1.19k op/s 2023-07-21T19:29:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.852+0000 7f66f75ff700 1 -- 172.21.15.5:0/3062362029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f0104a30 msgr2=0x7f66f0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.852+0000 7f66f75ff700 1 --2- 172.21.15.5:0/3062362029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f0104a30 0x7f66f0104e00 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f66ec009990 tx=0x7f66ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 -- 172.21.15.5:0/3062362029 shutdown_connections 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 --2- 172.21.15.5:0/3062362029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 --2- 172.21.15.5:0/3062362029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f0104a30 0x7f66f0104e00 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 --2- 172.21.15.5:0/3062362029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f00fea30 0x7f66f00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 -- 172.21.15.5:0/3062362029 >> 172.21.15.5:0/3062362029 conn(0x7f66f0074760 msgr2=0x7f66f0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 -- 172.21.15.5:0/3062362029 shutdown_connections 2023-07-21T19:29:29.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.853+0000 7f66f75ff700 1 -- 172.21.15.5:0/3062362029 wait complete. 2023-07-21T19:29:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.854+0000 7f66f75ff700 1 Processor -- start 2023-07-21T19:29:29.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.856+0000 7f66f75ff700 1 -- start start 2023-07-21T19:29:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 0x7f66f0113b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f01140d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 0x7f66f010ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f010f1f0 con 0x7f66f0104a30 2023-07-21T19:29:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f010f360 con 0x7f66f00ff3e0 2023-07-21T19:29:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f75ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f010f520 con 0x7f66f00fea30 2023-07-21T19:29:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f01140d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f4b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f01140d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60700/0 (socket says 172.21.15.5:60700) 2023-07-21T19:29:29.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.857+0000 7f66f4b9a700 1 -- 172.21.15.5:0/4186845414 learned_addr learned my addr 172.21.15.5:0/4186845414 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 0x7f66f0113b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 0x7f66f010ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 -- 172.21.15.5:0/4186845414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 msgr2=0x7f66f01140d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f01140d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 -- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 msgr2=0x7f66f010ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 0x7f66f010ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f539b700 1 -- 172.21.15.5:0/4186845414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009c40 con 0x7f66f00fea30 2023-07-21T19:29:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.858+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 0x7f66f010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:29:29.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.859+0000 7f66f539b700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 0x7f66f0113b70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0098e0 tx=0x7f66ec005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.859+0000 7f66e67fc700 1 -- 172.21.15.5:0/4186845414 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec01d070 con 0x7f66f00fea30 2023-07-21T19:29:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.859+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66ec0095f0 con 0x7f66f00fea30 2023-07-21T19:29:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.860+0000 7f66e67fc700 1 -- 172.21.15.5:0/4186845414 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66ec022470 con 0x7f66f00fea30 2023-07-21T19:29:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.859+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f0117ac0 con 0x7f66f00fea30 2023-07-21T19:29:29.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.860+0000 7f66e67fc700 1 -- 172.21.15.5:0/4186845414 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec017650 con 0x7f66f00fea30 2023-07-21T19:29:29.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.865+0000 7f66e67fc700 1 -- 172.21.15.5:0/4186845414 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f66ec00fb10 con 0x7f66f00fea30 2023-07-21T19:29:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.866+0000 7f670af50b80 1 Processor -- start 2023-07-21T19:29:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.867+0000 7f670af50b80 1 -- start start 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982b2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf981b2c50 con 0x55bf982ad580 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982059e0 con 0x55bf982b32d0 2023-07-21T19:29:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f670af50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf981fee70 con 0x55bf981096b0 2023-07-21T19:29:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f66e67fc700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.858 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.868+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60706/0 (socket says 172.21.15.5:60706) 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f4b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f539b700 1 -- 172.21.15.5:0/3851320537 learned_addr learned my addr 172.21.15.5:0/3851320537 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982b2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.859 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66e67fc700 1 -- 172.21.15.5:0/4186845414 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66ec0cdf60 con 0x7f66f00fea30 2023-07-21T19:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 -- 172.21.15.5:0/3851320537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 msgr2=0x55bf982ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 -- 172.21.15.5:0/3851320537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 msgr2=0x55bf982ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.860 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.869+0000 7f66f5b9c700 1 -- 172.21.15.5:0/3851320537 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf9802cd80 con 0x55bf981096b0 2023-07-21T19:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.870+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982b2d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66f0114800 tx=0x7f66e8017350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=89f4200d4cfa43ce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.870+0000 7f66db7fe700 1 -- 172.21.15.5:0/3851320537 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e800e040 con 0x55bf981096b0 2023-07-21T19:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.870+0000 7f66db7fe700 1 -- 172.21.15.5:0/3851320537 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66e8021410 con 0x55bf981096b0 2023-07-21T19:29:29.861 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.870+0000 7f66db7fe700 1 -- 172.21.15.5:0/3851320537 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8020600 con 0x55bf981096b0 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.871+0000 7f670af50b80 1 -- 172.21.15.5:0/3851320537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 msgr2=0x55bf982b2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.871+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 0x7f66dc0a37f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66e8022000 tx=0x7f66e8020000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.871+0000 7f670af50b80 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982b2d90 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f66f0114800 tx=0x7f66e8017350 comp rx=0 tx=0).stop 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.872+0000 7f670af50b80 1 -- 172.21.15.5:0/3851320537 shutdown_connections 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.872+0000 7f670af50b80 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.862 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.872+0000 7f670af50b80 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982b2d90 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.872+0000 7f670af50b80 1 --2- 172.21.15.5:0/3851320537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.872+0000 7f670af50b80 1 -- 172.21.15.5:0/3851320537 >> 172.21.15.5:0/3851320537 conn(0x55bf982a2be0 msgr2=0x55bf982a7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.873+0000 7f670af50b80 1 -- 172.21.15.5:0/3851320537 shutdown_connections 2023-07-21T19:29:29.863 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.873+0000 7f670af50b80 1 -- 172.21.15.5:0/3851320537 wait complete. 2023-07-21T19:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.874+0000 7f670af50b80 1 -- start start 2023-07-21T19:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.875+0000 7f670af50b80 1 Processor -- start 2023-07-21T19:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.875+0000 7f670af50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.875+0000 7f670af50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982b6850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.875+0000 7f670af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982b6e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f670af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9819ea60 con 0x55bf982b32d0 2023-07-21T19:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f670af50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf9819e760 con 0x55bf981096b0 2023-07-21T19:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f670af50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf980828c0 con 0x55bf982ad580 2023-07-21T19:29:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f670af50b80 10 client.0 _ll_register_callbacks cb 0x55bf9819c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60716/0 (socket says 172.21.15.5:60716) 2023-07-21T19:29:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f66f539b700 1 -- 172.21.15.5:0/2033658713 learned_addr learned my addr 172.21.15.5:0/2033658713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.876+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 -- 172.21.15.5:0/2033658713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 msgr2=0x55bf982b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 -- 172.21.15.5:0/2033658713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 msgr2=0x55bf982b6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982b6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 -- 172.21.15.5:0/2033658713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66e80108f0 con 0x55bf981096b0 2023-07-21T19:29:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66f539b700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0976e0 tx=0x7f66ec005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3444ac894024f529 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.877+0000 7f66d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec022ea0 con 0x55bf981096b0 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66ec09b1e0 con 0x55bf981096b0 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bf98158490 con 0x55bf981096b0 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec097520 con 0x55bf981096b0 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f670af50b80 10 client.29483 fetch_fsmap learned FSMap version 59 2023-07-21T19:29:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.878+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55bf98015c00 con 0x55bf981096b0 2023-07-21T19:29:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bf98158490 con 0x55bf981096b0 2023-07-21T19:29:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 59) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f66ec09b440 con 0x55bf981096b0 2023-07-21T19:29:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 10 client.29483 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T19:29:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 10 client.29483 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 20 client.29483 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf982d78a0 0x55bf982d9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 4 client.29483 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.879+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bf981f7e50 con 0x55bf982d78a0 2023-07-21T19:29:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.880+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf982d78a0 0x55bf982d9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.881+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf982d78a0 0x55bf982d9cb0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9911b6a84f9b7c0 server_cookie=a43eee892e43c444 in_seq=0 out_seq=0 2023-07-21T19:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.881+0000 7f66d8ff9700 10 client.29483 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.882+0000 7f66d8ff9700 1 -- 172.21.15.5:0/2033658713 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bf981f7e50 con 0x55bf982d78a0 2023-07-21T19:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.882+0000 7f66d8ff9700 10 client.29483 handle_command_reply: tid=0 2023-07-21T19:29:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.882+0000 7f670af50b80 1 client.29483 shutdown 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 msgr2=0x55bf982b95d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0976e0 tx=0x7f66ec005c00 comp rx=0 tx=0).stop 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 shutdown_connections 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf982d78a0 0x55bf982d9cb0 crc :-1 s=READY pgs=439 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982b6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982b95d0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 --2- 172.21.15.5:0/2033658713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 >> 172.21.15.5:0/2033658713 conn(0x55bf982a2be0 msgr2=0x55bf982a6c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.883+0000 7f66d8ff9700 0 client.29483 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.884+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 shutdown_connections 2023-07-21T19:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.884+0000 7f670af50b80 1 -- 172.21.15.5:0/2033658713 wait complete. 2023-07-21T19:29:29.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.884+0000 7f670af50b80 20 client.29483 trim_cache size 0 max 16384 2023-07-21T19:29:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 Processor -- start 2023-07-21T19:29:29.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 -- start start 2023-07-21T19:29:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982bdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982b7480 con 0x55bf982ad580 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982b29e0 con 0x55bf982b32d0 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.906+0000 7f670af50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982a45f0 con 0x55bf981096b0 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52414/0 (socket says 172.21.15.5:52414) 2023-07-21T19:29:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 -- 172.21.15.5:0/1154207338 learned_addr learned my addr 172.21.15.5:0/1154207338 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f5b9c700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982bdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 -- 172.21.15.5:0/1154207338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 msgr2=0x55bf982bdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982bdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 -- 172.21.15.5:0/1154207338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 msgr2=0x55bf982b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f539b700 1 -- 172.21.15.5:0/1154207338 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bf980531f0 con 0x55bf981096b0 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.907+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.908+0000 7f66f539b700 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d458f7e6e8a8bed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.908+0000 7f66dbfff700 1 -- 172.21.15.5:0/1154207338 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec022ea0 con 0x55bf981096b0 2023-07-21T19:29:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.909+0000 7f66dbfff700 1 -- 172.21.15.5:0/1154207338 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66ec09afb0 con 0x55bf981096b0 2023-07-21T19:29:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.909+0000 7f66dbfff700 1 -- 172.21.15.5:0/1154207338 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec09b440 con 0x55bf981096b0 2023-07-21T19:29:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.910+0000 7f670af50b80 1 -- 172.21.15.5:0/1154207338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 msgr2=0x55bf982ad040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.910+0000 7f670af50b80 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f66ec000c00 tx=0x7f66ec009ca0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 -- 172.21.15.5:0/1154207338 shutdown_connections 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf982b32d0 0x55bf982bdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf981096b0 0x55bf982ad040 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 --2- 172.21.15.5:0/1154207338 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982ad580 0x55bf982b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 -- 172.21.15.5:0/1154207338 >> 172.21.15.5:0/1154207338 conn(0x55bf982a2be0 msgr2=0x55bf982b60f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 -- 172.21.15.5:0/1154207338 shutdown_connections 2023-07-21T19:29:29.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.914+0000 7f670af50b80 1 -- 172.21.15.5:0/1154207338 wait complete. 2023-07-21T19:29:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.915+0000 7f670af50b80 1 -- start start 2023-07-21T19:29:29.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.915+0000 7f670af50b80 1 Processor -- start 2023-07-21T19:29:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982ae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982aecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982b0360 con 0x55bf982b32d0 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982b04d0 con 0x55bf981096b0 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bf982af210 con 0x55bf982ad580 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f670af50b80 10 client.0 _ll_register_callbacks cb 0x55bf9819c700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:29.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.917+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:60742/0 (socket says 172.21.15.5:60742) 2023-07-21T19:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 -- 172.21.15.5:0/3766162973 learned_addr learned my addr 172.21.15.5:0/3766162973 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982ae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 -- 172.21.15.5:0/3766162973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 msgr2=0x55bf982ae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982ae790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 -- 172.21.15.5:0/3766162973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 msgr2=0x55bf982aecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982aecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.918+0000 7f66f539b700 1 -- 172.21.15.5:0/3766162973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55bf98015c00 con 0x55bf981096b0 2023-07-21T19:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.919+0000 7f66f539b700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66ec09b120 tx=0x7f66ec005c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9a7766dfc02ac6bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.919+0000 7f66d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.919+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec09b1e0 con 0x55bf981096b0 2023-07-21T19:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f66ec09f210 con 0x55bf981096b0 2023-07-21T19:29:29.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f66e8021410 con 0x55bf981096b0 2023-07-21T19:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f66ec0a10e0 con 0x55bf981096b0 2023-07-21T19:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f670af50b80 10 client.29489 fetch_fsmap learned FSMap version 59 2023-07-21T19:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55bf9802b250 con 0x55bf981096b0 2023-07-21T19:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.920+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f66e8021410 con 0x55bf981096b0 2023-07-21T19:29:29.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 59) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f66ec0b9050 con 0x55bf981096b0 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 10 client.29489 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 10 client.29489 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 20 client.29489 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf9832c310 0x55bf9832e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 4 client.29489 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.921+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bf981b9ff0 con 0x55bf9832c310 2023-07-21T19:29:29.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.922+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf9832c310 0x55bf9832e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.923+0000 7f66f4b9a700 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf9832c310 0x55bf9832e6f0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6dd372a2f2cb8d5 server_cookie=e484d42c86fbd6e7 in_seq=0 out_seq=0 2023-07-21T19:29:29.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.923+0000 7f66d8ff9700 10 client.29489 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:29.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.969+0000 7f66d8ff9700 1 -- 172.21.15.5:0/3766162973 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bf981b9ff0 con 0x55bf9832c310 2023-07-21T19:29:29.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.969+0000 7f66d8ff9700 10 client.29489 handle_command_reply: tid=0 2023-07-21T19:29:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.970+0000 7f670af50b80 1 client.29489 shutdown 2023-07-21T19:29:29.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.971+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 msgr2=0x55bf982afd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.971+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f66ec09b120 tx=0x7f66ec005c00 comp rx=0 tx=0).stop 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 shutdown_connections 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bf9832c310 0x55bf9832e6f0 crc :-1 s=READY pgs=440 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bf982b32d0 0x55bf982aecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bf981096b0 0x55bf982afd60 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 --2- 172.21.15.5:0/3766162973 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bf982ad580 0x55bf982ae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 >> 172.21.15.5:0/3766162973 conn(0x55bf982a2be0 msgr2=0x55bf982b1a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f66d8ff9700 0 client.29489 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 shutdown_connections 2023-07-21T19:29:29.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.973+0000 7f670af50b80 1 -- 172.21.15.5:0/3766162973 wait complete. 2023-07-21T19:29:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:29:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:29:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:29:29.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:29:29.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.974+0000 7f670af50b80 20 client.29489 trim_cache size 0 max 16384 2023-07-21T19:29:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.978+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 msgr2=0x7f66dc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.978+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 0x7f66dc0a37f0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f66e8022000 tx=0x7f66e8020000 comp rx=0 tx=0).stop 2023-07-21T19:29:29.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.978+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 msgr2=0x7f66f0113b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.978+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 0x7f66f0113b70 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f66ec0098e0 tx=0x7f66ec005740 comp rx=0 tx=0).stop 2023-07-21T19:29:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 shutdown_connections 2023-07-21T19:29:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f66dc0a1340 0x7f66dc0a37f0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f66f0104a30 0x7f66f010ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f66f00fea30 0x7f66f0113b70 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 --2- 172.21.15.5:0/4186845414 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f66f00ff3e0 0x7f66f01140d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.979+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 >> 172.21.15.5:0/4186845414 conn(0x7f66f0074760 msgr2=0x7f66f00fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.980+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 shutdown_connections 2023-07-21T19:29:29.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:29.980+0000 7f66f75ff700 1 -- 172.21.15.5:0/4186845414 wait complete. 2023-07-21T19:29:31.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:31 smithi101 ceph-mon[128136]: pgmap v692: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-07-21T19:29:31.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:31 smithi005 ceph-mon[126460]: pgmap v692: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-07-21T19:29:31.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:31 smithi156 ceph-mon[128316]: pgmap v692: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 975 KiB/s rd, 11 MiB/s wr, 836 op/s 2023-07-21T19:29:31.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 180.0 s, 428.6 tps, lat 233.321 ms stddev 41.393 2023-07-21T19:29:32.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:32 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:29:32.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:32 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:29:32.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:32 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:29:34.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: pgmap v693: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-07-21T19:29:34.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:33 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: pgmap v693: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:33 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: pgmap v693: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 16 MiB/s wr, 1.26k op/s 2023-07-21T19:29:34.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:34.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:33 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:36.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:35 smithi101 ceph-mon[128136]: pgmap v694: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 907 op/s 2023-07-21T19:29:36.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:35 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:29:36.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:35 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:29:36.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:35 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:36.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:35 smithi005 ceph-mon[126460]: pgmap v694: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 907 op/s 2023-07-21T19:29:36.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:35 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:29:36.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:35 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:29:36.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:35 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:36.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:35 smithi156 ceph-mon[128316]: pgmap v694: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 11 MiB/s wr, 907 op/s 2023-07-21T19:29:36.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:35 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:29:36.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:35 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:29:36.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:35 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:29:36.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 185.0 s, 303.0 tps, lat 259.161 ms stddev 71.586 2023-07-21T19:29:37.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:36 smithi101 ceph-mon[128136]: pgmap v695: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 883 op/s 2023-07-21T19:29:37.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:36 smithi005 ceph-mon[126460]: pgmap v695: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 883 op/s 2023-07-21T19:29:37.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:36 smithi156 ceph-mon[128316]: pgmap v695: 97 pgs: 97 active+clean; 14 GiB data, 12 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 11 MiB/s wr, 883 op/s 2023-07-21T19:29:40.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:39 smithi101 ceph-mon[128136]: pgmap v696: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-07-21T19:29:40.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:39 smithi005 ceph-mon[126460]: pgmap v696: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-07-21T19:29:40.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:39 smithi156 ceph-mon[128316]: pgmap v696: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 15 MiB/s wr, 1.22k op/s 2023-07-21T19:29:41.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 190.0 s, 57.2 tps, lat 1768.157 ms stddev 1078.419 2023-07-21T19:29:42.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:41 smithi101 ceph-mon[128136]: pgmap v697: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.5 MiB/s wr, 793 op/s 2023-07-21T19:29:42.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:41 smithi005 ceph-mon[126460]: pgmap v697: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.5 MiB/s wr, 793 op/s 2023-07-21T19:29:42.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:41 smithi156 ceph-mon[128316]: pgmap v697: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 9.5 MiB/s wr, 793 op/s 2023-07-21T19:29:43.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 -- 172.21.15.5:0/3390779892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7748067c10 msgr2=0x7f7748067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 --2- 172.21.15.5:0/3390779892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7748067c10 0x7f7748067fe0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f77440099c0 tx=0x7f7744009cd0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 -- 172.21.15.5:0/3390779892 shutdown_connections 2023-07-21T19:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 --2- 172.21.15.5:0/3390779892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 0x7f774810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 --2- 172.21.15.5:0/3390779892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7748067c10 0x7f7748067fe0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 --2- 172.21.15.5:0/3390779892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f77480685b0 0x7f7748104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 -- 172.21.15.5:0/3390779892 >> 172.21.15.5:0/3390779892 conn(0x7f7748074760 msgr2=0x7f7748074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 -- 172.21.15.5:0/3390779892 shutdown_connections 2023-07-21T19:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 -- 172.21.15.5:0/3390779892 wait complete. 2023-07-21T19:29:43.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.155+0000 7f774f3ff700 1 Processor -- start 2023-07-21T19:29:43.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 -- start start 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 0x7f774819c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 0x7f7748195a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748196090 con 0x7f77480685b0 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7748196200 con 0x7f7748067c10 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774f3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77481963c0 con 0x7f7748104e50 2023-07-21T19:29:43.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774d19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774c99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 0x7f774819c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774d19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57340/0 (socket says 172.21.15.5:57340) 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774d19b700 1 -- 172.21.15.5:0/1012400274 learned_addr learned my addr 172.21.15.5:0/1012400274 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.156+0000 7f774d99c700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 0x7f7748195a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 -- 172.21.15.5:0/1012400274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 msgr2=0x7f7748195a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 0x7f7748195a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 -- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 msgr2=0x7f774819c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 0x7f774819c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 -- 172.21.15.5:0/1012400274 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77440095f0 con 0x7f7748067c10 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774c99a700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 0x7f774819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774d19b700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7744005910 tx=0x7f774400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f773e7fc700 1 -- 172.21.15.5:0/1012400274 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774401d070 con 0x7f7748067c10 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f773e7fc700 1 -- 172.21.15.5:0/1012400274 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7744005050 con 0x7f7748067c10 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7748196640 con 0x7f7748067c10 2023-07-21T19:29:43.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f773e7fc700 1 -- 172.21.15.5:0/1012400274 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7744004950 con 0x7f7748067c10 2023-07-21T19:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.158+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77481a2bb0 con 0x7f7748067c10 2023-07-21T19:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.161+0000 7f773e7fc700 1 -- 172.21.15.5:0/1012400274 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f77440051c0 con 0x7f7748067c10 2023-07-21T19:29:43.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.162+0000 7f7762d12b80 1 Processor -- start 2023-07-21T19:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.162+0000 7f7762d12b80 1 -- start start 2023-07-21T19:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e95450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f773e7fc700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 0x7f77300a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7d96c50 con 0x55f7c7e9b680 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7de99e0 con 0x55f7c7ced6b0 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f7762d12b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7de2e70 con 0x55f7c7e9bad0 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f773e7fc700 1 -- 172.21.15.5:0/1012400274 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f77440cda20 con 0x7f7748067c10 2023-07-21T19:29:43.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e95450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57350/0 (socket says 172.21.15.5:57350) 2023-07-21T19:29:43.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d99c700 1 -- 172.21.15.5:0/2951248492 learned_addr learned my addr 172.21.15.5:0/2951248492 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d99c700 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d19b700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 0x7f77300a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 -- 172.21.15.5:0/2951248492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 msgr2=0x55f7c7e959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 -- 172.21.15.5:0/2951248492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 msgr2=0x55f7c7e9b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 -- 172.21.15.5:0/2951248492 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c7c10d80 con 0x55f7c7e9b680 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d99c700 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:43.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774c99a700 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e95450 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f77480733d0 tx=0x7f77380117a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d298abab0002a9f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f774d19b700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 0x7f77300a37f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f774409a820 tx=0x7f7744005c00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f772f7fe700 1 -- 172.21.15.5:0/2951248492 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738011310 con 0x55f7c7e9b680 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f772f7fe700 1 -- 172.21.15.5:0/2951248492 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77380044c0 con 0x55f7c7e9b680 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.164+0000 7f772f7fe700 1 -- 172.21.15.5:0/2951248492 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7738010430 con 0x55f7c7e9b680 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- 172.21.15.5:0/2951248492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 msgr2=0x55f7c7e95450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e95450 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f77480733d0 tx=0x7f77380117a0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- 172.21.15.5:0/2951248492 shutdown_connections 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 --2- 172.21.15.5:0/2951248492 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e95450 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- 172.21.15.5:0/2951248492 >> 172.21.15.5:0/2951248492 conn(0x55f7c7e86be0 msgr2=0x55f7c7e8bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- 172.21.15.5:0/2951248492 shutdown_connections 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- 172.21.15.5:0/2951248492 wait complete. 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.166+0000 7f7762d12b80 1 -- start start 2023-07-21T19:29:43.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 Processor -- start 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e9ec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e9f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7d82a60 con 0x55f7c7e9b680 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7d82760 con 0x55f7c7ced6b0 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7c668c0 con 0x55f7c7e9bad0 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f7762d12b80 10 client.0 _ll_register_callbacks cb 0x55f7c7d80700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:43.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774c99a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e9ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57356/0 (socket says 172.21.15.5:57356) 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 -- 172.21.15.5:0/1305674524 learned_addr learned my addr 172.21.15.5:0/1305674524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e9f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 -- 172.21.15.5:0/1305674524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 msgr2=0x55f7c7e9f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e9f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 -- 172.21.15.5:0/1305674524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 msgr2=0x55f7c7e9ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e9ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 -- 172.21.15.5:0/1305674524 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7740008910 con 0x55f7c7ced6b0 2023-07-21T19:29:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774c99a700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e9ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f774d99c700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7740008eb0 tx=0x7f7740004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b1df325ea168184 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.161 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f772cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:29:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f774000e070 con 0x55f7c7ced6b0 2023-07-21T19:29:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.167+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7740003c10 con 0x55f7c7ced6b0 2023-07-21T19:29:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f7c7d3c490 con 0x55f7c7ced6b0 2023-07-21T19:29:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7740003dd0 con 0x55f7c7ced6b0 2023-07-21T19:29:43.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 10 client.29501 fetch_fsmap learned FSMap version 59 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x7f7738009c90 con 0x55f7c7ced6b0 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f7c7d3c490 con 0x55f7c7ced6b0 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 59) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f7740004670 con 0x55f7c7ced6b0 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 10 client.29501 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 10 client.29501 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 20 client.29501 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7eaefd0 0x55f7c7eb13e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 4 client.29501 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f7c7ddbe50 con 0x55f7c7eaefd0 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.169+0000 7f774c99a700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7eaefd0 0x55f7c7eb13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.171+0000 7f774c99a700 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7eaefd0 0x55f7c7eb13e0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f15fb3983e94bd4f server_cookie=4f930003b98123bb in_seq=0 out_seq=0 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.171+0000 7f772cff9700 10 client.29501 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f772cff9700 1 -- 172.21.15.5:0/1305674524 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f7c7ddbe50 con 0x55f7c7eaefd0 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f772cff9700 10 client.29501 handle_command_reply: tid=0 2023-07-21T19:29:43.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 client.29501 shutdown 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 msgr2=0x55f7c7ea1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f7740008eb0 tx=0x7f7740004b40 comp rx=0 tx=0).stop 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 shutdown_connections 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e9f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7ea1a40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7eaefd0 0x55f7c7eb13e0 crc :-1 s=READY pgs=441 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1305674524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9b680 0x55f7c7e9ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 >> 172.21.15.5:0/1305674524 conn(0x55f7c7e86be0 msgr2=0x55f7c7e8b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f772cff9700 0 client.29501 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 shutdown_connections 2023-07-21T19:29:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 1 -- 172.21.15.5:0/1305674524 wait complete. 2023-07-21T19:29:43.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.172+0000 7f7762d12b80 20 client.29501 trim_cache size 0 max 16384 2023-07-21T19:29:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 Processor -- start 2023-07-21T19:29:43.175 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 -- start start 2023-07-21T19:29:43.176 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7ea60a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7e8ca00 con 0x55f7c7e9bad0 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7e8d800 con 0x55f7c7ced6b0 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f7762d12b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7eab900 con 0x55f7c7e9b680 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.188+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774d99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.177 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7ea60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774d99c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54924/0 (socket says 172.21.15.5:54924) 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774d99c700 1 -- 172.21.15.5:0/885119098 learned_addr learned my addr 172.21.15.5:0/885119098 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 -- 172.21.15.5:0/885119098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 msgr2=0x55f7c7e9ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 -- 172.21.15.5:0/885119098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 msgr2=0x55f7c7e8bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.178 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 -- 172.21.15.5:0/885119098 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f7c7bf9c00 con 0x55f7c7e9b680 2023-07-21T19:29:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774d99c700 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:43.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f774c99a700 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7ea60a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7738000c00 tx=0x7f7738009fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4446bbc56e896293 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.189+0000 7f772ffff700 1 -- 172.21.15.5:0/885119098 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773802e030 con 0x55f7c7e9b680 2023-07-21T19:29:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f772ffff700 1 -- 172.21.15.5:0/885119098 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7738020040 con 0x55f7c7e9b680 2023-07-21T19:29:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f772ffff700 1 -- 172.21.15.5:0/885119098 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77380054e0 con 0x55f7c7e9b680 2023-07-21T19:29:43.181 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 -- 172.21.15.5:0/885119098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 msgr2=0x55f7c7ea60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7ea60a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f7738000c00 tx=0x7f7738009fb0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 -- 172.21.15.5:0/885119098 shutdown_connections 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e8bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e9ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 --2- 172.21.15.5:0/885119098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7ea60a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 -- 172.21.15.5:0/885119098 >> 172.21.15.5:0/885119098 conn(0x55f7c7e86be0 msgr2=0x55f7c7e9e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.182 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 -- 172.21.15.5:0/885119098 shutdown_connections 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.190+0000 7f7762d12b80 1 -- 172.21.15.5:0/885119098 wait complete. 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 -- start start 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 Processor -- start 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7e983f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e96ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7e970e0 con 0x55f7c7e9bad0 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7e97250 con 0x55f7c7ced6b0 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.191+0000 7f7762d12b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f7c7e97410 con 0x55f7c7e9b680 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f7762d12b80 10 client.0 _ll_register_callbacks cb 0x55f7c7d80700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7e983f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774d99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57380/0 (socket says 172.21.15.5:57380) 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774d99c700 1 -- 172.21.15.5:0/1388976707 learned_addr learned my addr 172.21.15.5:0/1388976707 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:43.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774d99c700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e96ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 -- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 msgr2=0x55f7c7e97eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 -- 172.21.15.5:0/1388976707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 msgr2=0x55f7c7e96ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e96ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 -- 172.21.15.5:0/1388976707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7738009c90 con 0x55f7c7e9b680 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774d99c700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:43.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f774c99a700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7e983f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55f7c7d3c4b0 tx=0x7f773802ab30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b082d654f98235d1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.192+0000 7f772cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773802e040 con 0x55f7c7e9b680 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f773802f070 con 0x55f7c7e9b680 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f77380044c0 con 0x55f7c7e9b680 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=59) v2 ==== 24+0+0 (secure 0 0 0) 0x7f773802ade0 con 0x55f7c7e9b680 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f7762d12b80 10 client.29434 fetch_fsmap learned FSMap version 59 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=59,monmap=4+}) v3 -- 0x55f7c7c10d80 con 0x55f7c7e9b680 2023-07-21T19:29:43.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.193+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f77380044c0 con 0x55f7c7e9b680 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 59) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f77380046a0 con 0x55f7c7e9b680 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 10 client.29434 fetch_fsmap finished waiting for FSMap version 59 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 10 client.29434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 20 client.29434 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7f03970 0x55f7c7f05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 4 client.29434 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f7c7d9dff0 con 0x55f7c7f03970 2023-07-21T19:29:43.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f774d99c700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7f03970 0x55f7c7f05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f772cff9700 10 client.29434 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:43.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.194+0000 7f774d99c700 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7f03970 0x55f7c7f05d50 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df6a4f2e7b432499 server_cookie=87ada648144b3f87 in_seq=0 out_seq=0 2023-07-21T19:29:43.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.220+0000 7f772cff9700 1 -- 172.21.15.5:0/1388976707 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f7c7d9dff0 con 0x55f7c7f03970 2023-07-21T19:29:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.220+0000 7f772cff9700 10 client.29434 handle_command_reply: tid=0 2023-07-21T19:29:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.220+0000 7f7762d12b80 1 client.29434 shutdown 2023-07-21T19:29:43.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 msgr2=0x55f7c7e983f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7e983f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x55f7c7d3c4b0 tx=0x7f773802ab30 comp rx=0 tx=0).stop 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 shutdown_connections 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f7c7f03970 0x55f7c7f05d50 crc :-1 s=READY pgs=442 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f7c7e9bad0 0x55f7c7e96ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f7c7ced6b0 0x55f7c7e97eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 --2- 172.21.15.5:0/1388976707 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f7c7e9b680 0x55f7c7e983f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f772cff9700 0 client.29434 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 >> 172.21.15.5:0/1388976707 conn(0x55f7c7e86be0 msgr2=0x55f7c7e99d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 shutdown_connections 2023-07-21T19:29:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.221+0000 7f7762d12b80 1 -- 172.21.15.5:0/1388976707 wait complete. 2023-07-21T19:29:43.210 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.222+0000 7f7762d12b80 20 client.29434 trim_cache size 0 max 16384 2023-07-21T19:29:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:29:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:29:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:29:43.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.222+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 msgr2=0x7f77300a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.222+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 0x7f77300a37f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f774409a820 tx=0x7f7744005c00 comp rx=0 tx=0).stop 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.222+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 msgr2=0x7f774819bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.222+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f7744005910 tx=0x7f774400f690 comp rx=0 tx=0).stop 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 shutdown_connections 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7748104e50 0x7f7748195a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7748067c10 0x7f774819bbc0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f77300a1340 0x7f77300a37f0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 --2- 172.21.15.5:0/1012400274 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f77480685b0 0x7f774819c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.223+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 >> 172.21.15.5:0/1012400274 conn(0x7f7748074760 msgr2=0x7f77480fc640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.224+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 shutdown_connections 2023-07-21T19:29:43.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:43.224+0000 7f774f3ff700 1 -- 172.21.15.5:0/1012400274 wait complete. 2023-07-21T19:29:44.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:43 smithi101 ceph-mon[128136]: pgmap v698: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 863 op/s 2023-07-21T19:29:44.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:43 smithi005 ceph-mon[126460]: pgmap v698: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 863 op/s 2023-07-21T19:29:44.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:43 smithi156 ceph-mon[128316]: pgmap v698: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 10 MiB/s wr, 863 op/s 2023-07-21T19:29:45.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:44 smithi101 ceph-mon[128136]: pgmap v699: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.8 MiB/s wr, 439 op/s 2023-07-21T19:29:45.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:44 smithi005 ceph-mon[126460]: pgmap v699: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.8 MiB/s wr, 439 op/s 2023-07-21T19:29:45.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:44 smithi156 ceph-mon[128316]: pgmap v699: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 665 KiB/s rd, 4.8 MiB/s wr, 439 op/s 2023-07-21T19:29:46.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 195.0 s, 93.8 tps, lat 1268.628 ms stddev 1907.828 2023-07-21T19:29:47.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:47 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:47.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:47 smithi101 ceph-mon[128136]: pgmap v700: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-07-21T19:29:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:47 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:47 smithi005 ceph-mon[126460]: pgmap v700: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-07-21T19:29:47.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:47 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:29:47.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:47 smithi156 ceph-mon[128316]: pgmap v700: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 613 KiB/s rd, 4.6 MiB/s wr, 409 op/s 2023-07-21T19:29:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:49 smithi101 ceph-mon[128136]: pgmap v701: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-07-21T19:29:50.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:49 smithi005 ceph-mon[126460]: pgmap v701: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-07-21T19:29:50.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:49 smithi156 ceph-mon[128316]: pgmap v701: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 782 KiB/s rd, 5.8 MiB/s wr, 507 op/s 2023-07-21T19:29:51.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 200.0 s, 310.2 tps, lat 328.305 ms stddev 346.472 2023-07-21T19:29:52.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:51 smithi101 ceph-mon[128136]: pgmap v702: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2023-07-21T19:29:52.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:51 smithi005 ceph-mon[126460]: pgmap v702: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2023-07-21T19:29:52.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:51 smithi156 ceph-mon[128316]: pgmap v702: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 294 KiB/s rd, 1.9 MiB/s wr, 171 op/s 2023-07-21T19:29:52.504 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:29:52.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.874+0000 7f7bc0fff700 1 -- 172.21.15.5:0/68866409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 msgr2=0x7f7bbc108b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.874+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/68866409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc108b60 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f7bac009990 tx=0x7f7bac009ca0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.862 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.874+0000 7f7bc0fff700 1 -- 172.21.15.5:0/68866409 shutdown_connections 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/68866409 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc10d630 0x7f7bbc115200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/68866409 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc108b60 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/68866409 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc109130 0x7f7bbc10cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 -- 172.21.15.5:0/68866409 >> 172.21.15.5:0/68866409 conn(0x7f7bbc071570 msgr2=0x7f7bbc071970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 -- 172.21.15.5:0/68866409 shutdown_connections 2023-07-21T19:29:52.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.875+0000 7f7bc0fff700 1 -- 172.21.15.5:0/68866409 wait complete. 2023-07-21T19:29:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 Processor -- start 2023-07-21T19:29:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 -- start start 2023-07-21T19:29:52.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc109130 0x7f7bbc111d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc10d630 0x7f7bbc112290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc1127d0 con 0x7f7bbc109130 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc112940 con 0x7f7bbc10d630 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.876+0000 7f7bc0fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bbc112b00 con 0x7f7bbc108790 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.865 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56236/0 (socket says 172.21.15.5:56236) 2023-07-21T19:29:52.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 -- 172.21.15.5:0/773978963 learned_addr learned my addr 172.21.15.5:0/773978963 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:52.866 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc109130 0x7f7bbc111d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc10d630 0x7f7bbc112290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 -- 172.21.15.5:0/773978963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc10d630 msgr2=0x7f7bbc112290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc10d630 0x7f7bbc112290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 -- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc109130 msgr2=0x7f7bbc111d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc109130 0x7f7bbc111d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.867 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.877+0000 7f7bba59c700 1 -- 172.21.15.5:0/773978963 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bac0095f0 con 0x7f7bbc108790 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bba59c700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b650 tx=0x7f7bac00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bab7fe700 1 -- 172.21.15.5:0/773978963 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac01d070 con 0x7f7bbc108790 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bab7fe700 1 -- 172.21.15.5:0/773978963 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bac00beb0 con 0x7f7bbc108790 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bab7fe700 1 -- 172.21.15.5:0/773978963 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac004c10 con 0x7f7bbc108790 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc1b1220 con 0x7f7bbc108790 2023-07-21T19:29:52.868 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.878+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bbc1b1650 con 0x7f7bbc108790 2023-07-21T19:29:52.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 Processor -- start 2023-07-21T19:29:52.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 -- start start 2023-07-21T19:29:52.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f43c5cb0 con 0x55c4f44bfbe0 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.885+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4418a40 con 0x55c4f44caad0 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4411ed0 con 0x55c4f44bf2d0 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bab7fe700 1 -- 172.21.15.5:0/773978963 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7bac005050 con 0x7f7bbc108790 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56248/0 (socket says 172.21.15.5:56248) 2023-07-21T19:29:52.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3466767578 learned_addr learned my addr 172.21.15.5:0/3466767578 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:52.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3466767578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 msgr2=0x55c4f44cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44cceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3466767578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 msgr2=0x55c4f44ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.886+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3466767578 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f423fde0 con 0x55c4f44bf2d0 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.887+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009440 tx=0x7f7bb4009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e246aa0b9174c410 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.887+0000 7f7ba8ff9700 1 -- 172.21.15.5:0/3466767578 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400e070 con 0x55c4f44bf2d0 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.887+0000 7f7ba8ff9700 1 -- 172.21.15.5:0/3466767578 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bb400f040 con 0x55c4f44bf2d0 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.887+0000 7f7ba8ff9700 1 -- 172.21.15.5:0/3466767578 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb4015a90 con 0x55c4f44bf2d0 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3466767578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 msgr2=0x55c4f44bf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009440 tx=0x7f7bb4009920 comp rx=0 tx=0).stop 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3466767578 shutdown_connections 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44cceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44bf6a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3466767578 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44ca590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3466767578 >> 172.21.15.5:0/3466767578 conn(0x55c4f44b5da0 msgr2=0x55c4f44bb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.888+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3466767578 shutdown_connections 2023-07-21T19:29:52.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.889+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3466767578 wait complete. 2023-07-21T19:29:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.889+0000 7f7bab7fe700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 0x7f7ba40a3780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.889+0000 7f7bab7fe700 1 -- 172.21.15.5:0/773978963 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7bac0cdfb0 con 0x7f7bbc108790 2023-07-21T19:29:52.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.889+0000 7f7bd48c0b80 1 -- start start 2023-07-21T19:29:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.889+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 0x7f7ba40a3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.890+0000 7f7bd48c0b80 1 Processor -- start 2023-07-21T19:29:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.890+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 0x7f7ba40a3780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc113540 tx=0x7f7bb000b580 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f43b1ac0 con 0x55c4f44bfbe0 2023-07-21T19:29:52.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f43b17c0 con 0x55c4f44caad0 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f4295920 con 0x55c4f44bf2d0 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bd48c0b80 10 client.0 _ll_register_callbacks cb 0x55c4f43af760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bb9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bb9d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57464/0 (socket says 172.21.15.5:57464) 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.891+0000 7f7bb9d9b700 1 -- 172.21.15.5:0/3088782524 learned_addr learned my addr 172.21.15.5:0/3088782524 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 -- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 msgr2=0x55c4f44c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 -- 172.21.15.5:0/3088782524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 msgr2=0x55c4f44c77c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bba59c700 1 -- 172.21.15.5:0/3088782524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bb40090f0 con 0x55c4f44bf2d0 2023-07-21T19:29:52.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.892+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44ca540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cb6a0 tx=0x7f7bac09af50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35bc16fa48218e54 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac005500 con 0x55c4f44bf2d0 2023-07-21T19:29:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c4f436b4f0 con 0x55c4f44bf2d0 2023-07-21T19:29:52.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7bac09f0e0 con 0x55c4f44bf2d0 2023-07-21T19:29:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bac0a3470 con 0x55c4f44bf2d0 2023-07-21T19:29:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7bd48c0b80 10 client.29446 fetch_fsmap learned FSMap version 60 2023-07-21T19:29:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.893+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55c4f4228c60 con 0x55c4f44bf2d0 2023-07-21T19:29:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.894+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bac0b7080 con 0x55c4f44bf2d0 2023-07-21T19:29:52.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.894+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bac0aed20 con 0x55c4f44bf2d0 2023-07-21T19:29:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 10 client.29446 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:29:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 10 client.29446 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 20 client.29446 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:52.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f44eb150 0x55c4f44ed530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 4 client.29446 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c4f440aeb0 con 0x55c4f44eb150 2023-07-21T19:29:52.888 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.895+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f44eb150 0x55c4f44ed530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.896+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f44eb150 0x55c4f44ed530 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eee2f628b195a861 server_cookie=846a1e2f94215b99 in_seq=0 out_seq=0 2023-07-21T19:29:52.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.896+0000 7f7ba1ffb700 10 client.29446 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:52.890 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.897+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3088782524 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c4f440aeb0 con 0x55c4f44eb150 2023-07-21T19:29:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.897+0000 7f7ba1ffb700 10 client.29446 handle_command_reply: tid=0 2023-07-21T19:29:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.897+0000 7f7bd48c0b80 1 client.29446 shutdown 2023-07-21T19:29:52.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 msgr2=0x55c4f44ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44ca540 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0cb6a0 tx=0x7f7bac09af50 comp rx=0 tx=0).stop 2023-07-21T19:29:52.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 shutdown_connections 2023-07-21T19:29:52.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44caad0 0x55c4f44c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44bf2d0 0x55c4f44ca540 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f44eb150 0x55c4f44ed530 crc :-1 s=READY pgs=443 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3088782524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c77c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 >> 172.21.15.5:0/3088782524 conn(0x55c4f44b5da0 msgr2=0x55c4f44bb290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7ba1ffb700 0 client.29446 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.898+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 shutdown_connections 2023-07-21T19:29:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.899+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3088782524 wait complete. 2023-07-21T19:29:52.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.900+0000 7f7bd48c0b80 20 client.29446 trim_cache size 0 max 16384 2023-07-21T19:29:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.931+0000 7f7bd48c0b80 1 Processor -- start 2023-07-21T19:29:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 -- start start 2023-07-21T19:29:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44ca0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44c3240 0x55c4f44c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44b7ec0 con 0x55c4f44bfbe0 2023-07-21T19:29:52.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44bcd20 con 0x55c4f44c3240 2023-07-21T19:29:52.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.932+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44c8da0 con 0x55c4f44caad0 2023-07-21T19:29:52.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.933+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.933+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57480/0 (socket says 172.21.15.5:57480) 2023-07-21T19:29:52.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.933+0000 7f7bba59c700 1 -- 172.21.15.5:0/1760348075 learned_addr learned my addr 172.21.15.5:0/1760348075 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:52.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.933+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44c3240 0x55c4f44c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.933+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44ca0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 -- 172.21.15.5:0/1760348075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 msgr2=0x55c4f44ca0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44ca0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 -- 172.21.15.5:0/1760348075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44c3240 msgr2=0x55c4f44c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44c3240 0x55c4f44c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 -- 172.21.15.5:0/1760348075 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4f4266250 con 0x55c4f44bfbe0 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:29:52.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7bba59c700 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0a2f40 tx=0x7f7bac005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e2d252893128286 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.934+0000 7f7ba97fa700 1 -- 172.21.15.5:0/1760348075 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac09a5a0 con 0x55c4f44bfbe0 2023-07-21T19:29:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.935+0000 7f7ba97fa700 1 -- 172.21.15.5:0/1760348075 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c4f436b4f0 con 0x55c4f44bfbe0 2023-07-21T19:29:52.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.935+0000 7f7ba97fa700 1 -- 172.21.15.5:0/1760348075 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bac0a3270 con 0x55c4f44bfbe0 2023-07-21T19:29:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.935+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/1760348075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 msgr2=0x55c4f44c6a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.935+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f7bac0a2f40 tx=0x7f7bac005e80 comp rx=0 tx=0).stop 2023-07-21T19:29:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/1760348075 shutdown_connections 2023-07-21T19:29:52.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44c3240 0x55c4f44c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44bfbe0 0x55c4f44c6a90 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/1760348075 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44ca0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/1760348075 >> 172.21.15.5:0/1760348075 conn(0x55c4f44b5da0 msgr2=0x55c4f44c2b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/1760348075 shutdown_connections 2023-07-21T19:29:52.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.936+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/1760348075 wait complete. 2023-07-21T19:29:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.937+0000 7f7bd48c0b80 1 -- start start 2023-07-21T19:29:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.937+0000 7f7bd48c0b80 1 Processor -- start 2023-07-21T19:29:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 0x55c4f44e2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 0x55c4f44e0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44e1230 con 0x55c4f44c3240 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44e1370 con 0x55c4f44bfbe0 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c4f44e1530 con 0x55c4f44caad0 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bd48c0b80 10 client.0 _ll_register_callbacks cb 0x7f7bac0973f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:52.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.938+0000 7f7bbad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bba59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 0x55c4f44e2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56276/0 (socket says 172.21.15.5:56276) 2023-07-21T19:29:52.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3116322165 learned_addr learned my addr 172.21.15.5:0/3116322165 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 0x55c4f44e0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 msgr2=0x55c4f44e2410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 0x55c4f44e2410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3116322165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 msgr2=0x55c4f44e0770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 0x55c4f44e0770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 -- 172.21.15.5:0/3116322165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4f4228c60 con 0x55c4f44caad0 2023-07-21T19:29:52.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 0x55c4f44e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bb9d9b700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 0x55c4f44e0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.939+0000 7f7bbad9d700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009440 tx=0x7f7bb40138b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dd53a601c86286 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:52.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7ba1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:52.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bb400e040 con 0x55c4f44caad0 2023-07-21T19:29:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7bb4003a00 con 0x55c4f44caad0 2023-07-21T19:29:52.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7bb400f040 con 0x55c4f44caad0 2023-07-21T19:29:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7bb4013ee0 con 0x55c4f44caad0 2023-07-21T19:29:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.940+0000 7f7bd48c0b80 10 client.29452 fetch_fsmap learned FSMap version 60 2023-07-21T19:29:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55c4f423e2b0 con 0x55c4f44caad0 2023-07-21T19:29:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7bb400f040 con 0x55c4f44caad0 2023-07-21T19:29:52.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f7bb401c070 con 0x55c4f44caad0 2023-07-21T19:29:52.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 10 client.29452 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:29:52.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 10 client.29452 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:52.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 20 client.29452 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:52.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f4540360 0x55c4f4542770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:52.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 4 client.29452 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:29:52.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.941+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55c4f43cd050 con 0x55c4f4540360 2023-07-21T19:29:52.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.942+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f4540360 0x55c4f4542770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.942+0000 7f7bba59c700 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f4540360 0x55c4f4542770 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=191b6e504b65075 server_cookie=e4c076611c3997d1 in_seq=0 out_seq=0 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.942+0000 7f7ba1ffb700 10 client.29452 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.942+0000 7f7ba1ffb700 1 -- 172.21.15.5:0/3116322165 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55c4f43cd050 con 0x55c4f4540360 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7ba1ffb700 10 client.29452 handle_command_reply: tid=0 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 client.29452 shutdown 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 msgr2=0x55c4f44e0cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f7bb4009440 tx=0x7f7bb40138b0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 shutdown_connections 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c4f44caad0 0x55c4f44e0cb0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c4f44bfbe0 0x55c4f44e2410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c4f4540360 0x55c4f4542770 crc :-1 s=READY pgs=444 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 --2- 172.21.15.5:0/3116322165 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c4f44c3240 0x55c4f44e0770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 >> 172.21.15.5:0/3116322165 conn(0x55c4f44b5da0 msgr2=0x55c4f44c7020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7ba1ffb700 0 client.29452 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 shutdown_connections 2023-07-21T19:29:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.943+0000 7f7bd48c0b80 1 -- 172.21.15.5:0/3116322165 wait complete. 2023-07-21T19:29:52.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.944+0000 7f7bd48c0b80 20 client.29452 trim_cache size 0 max 16384 2023-07-21T19:29:52.943 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:29:52.943 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:29:52.943 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:29:52.943 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:29:52.943 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:29:52.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:29:52.944 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:29:52.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:29:52.944 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:29:52.944 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:29:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 msgr2=0x7f7ba40a3780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 0x7f7ba40a3780 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc113540 tx=0x7f7bb000b580 comp rx=0 tx=0).stop 2023-07-21T19:29:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 msgr2=0x7f7bbc114d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:52.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f7bac00b650 tx=0x7f7bac00f740 comp rx=0 tx=0).stop 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 shutdown_connections 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7bbc10d630 0x7f7bbc112290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7bbc108790 0x7f7bbc114d40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ba40a12d0 0x7f7ba40a3780 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 --2- 172.21.15.5:0/773978963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7bbc109130 0x7f7bbc111d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:52.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 >> 172.21.15.5:0/773978963 conn(0x7f7bbc071570 msgr2=0x7f7bbc06d9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 shutdown_connections 2023-07-21T19:29:52.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:29:52.945+0000 7f7bc0fff700 1 -- 172.21.15.5:0/773978963 wait complete. 2023-07-21T19:29:52.974 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:29:54.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:53 smithi101 ceph-mon[128136]: pgmap v703: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 5.4 MiB/s wr, 453 op/s 2023-07-21T19:29:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:53 smithi005 ceph-mon[126460]: pgmap v703: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 5.4 MiB/s wr, 453 op/s 2023-07-21T19:29:54.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:53 smithi156 ceph-mon[128316]: pgmap v703: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 800 KiB/s rd, 5.4 MiB/s wr, 453 op/s 2023-07-21T19:29:56.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:55 smithi101 ceph-mon[128136]: pgmap v704: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 5.3 MiB/s wr, 412 op/s 2023-07-21T19:29:56.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:55 smithi005 ceph-mon[126460]: pgmap v704: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 5.3 MiB/s wr, 412 op/s 2023-07-21T19:29:56.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:55 smithi156 ceph-mon[128316]: pgmap v704: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 724 KiB/s rd, 5.3 MiB/s wr, 412 op/s 2023-07-21T19:29:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.393+0000 7ff43afff700 1 -- 172.21.15.5:0/3720626742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 msgr2=0x7ff434105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.393+0000 7ff43afff700 1 --2- 172.21.15.5:0/3720626742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434105f30 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7ff428009f90 tx=0x7ff42800b330 comp rx=0 tx=0).stop 2023-07-21T19:29:56.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 -- 172.21.15.5:0/3720626742 shutdown_connections 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 --2- 172.21.15.5:0/3720626742 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4341004e0 0x7ff43410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 --2- 172.21.15.5:0/3720626742 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434105f30 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 --2- 172.21.15.5:0/3720626742 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4340ffb30 0x7ff4340fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 -- 172.21.15.5:0/3720626742 >> 172.21.15.5:0/3720626742 conn(0x7ff4340fb5f0 msgr2=0x7ff4340fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 -- 172.21.15.5:0/3720626742 shutdown_connections 2023-07-21T19:29:56.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.394+0000 7ff43afff700 1 -- 172.21.15.5:0/3720626742 wait complete. 2023-07-21T19:29:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.395+0000 7ff43afff700 1 Processor -- start 2023-07-21T19:29:56.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.396+0000 7ff43afff700 1 -- start start 2023-07-21T19:29:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.396+0000 7ff43afff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 0x7ff43419bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.396+0000 7ff43afff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4341004e0 0x7ff43419c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.396+0000 7ff43afff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434195a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43afff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff434196080 con 0x7ff434105b60 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43afff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4341961f0 con 0x7ff4340ffb30 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43afff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4341963b0 con 0x7ff4341004e0 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 0x7ff43419bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434195a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 0x7ff43419bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43862/0 (socket says 172.21.15.5:43862) 2023-07-21T19:29:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff438d9b700 1 -- 172.21.15.5:0/2531621168 learned_addr learned my addr 172.21.15.5:0/2531621168 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff433fff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4341004e0 0x7ff43419c0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 -- 172.21.15.5:0/2531621168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4341004e0 msgr2=0x7ff43419c0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4341004e0 0x7ff43419c0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 -- 172.21.15.5:0/2531621168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 msgr2=0x7ff43419bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 0x7ff43419bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 -- 172.21.15.5:0/2531621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200095f0 con 0x7ff434105b60 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.397+0000 7ff43959c700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434195a10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e970 tx=0x7ff42c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.398+0000 7ff431ffb700 1 -- 172.21.15.5:0/2531621168 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c004220 con 0x7ff434105b60 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.398+0000 7ff431ffb700 1 -- 172.21.15.5:0/2531621168 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff42c004cc0 con 0x7ff434105b60 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.398+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff428009c40 con 0x7ff434105b60 2023-07-21T19:29:56.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.398+0000 7ff431ffb700 1 -- 172.21.15.5:0/2531621168 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42c005590 con 0x7ff434105b60 2023-07-21T19:29:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.398+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4341a29c0 con 0x7ff434105b60 2023-07-21T19:29:56.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.402+0000 7ff431ffb700 1 -- 172.21.15.5:0/2531621168 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff42c004750 con 0x7ff434105b60 2023-07-21T19:29:56.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.403+0000 7ff44e933b80 1 Processor -- start 2023-07-21T19:29:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 -- start start 2023-07-21T19:29:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5d076b0 0x560ba5eb0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eab9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5db0c50 con 0x560ba5d076b0 2023-07-21T19:29:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5e039e0 con 0x560ba5eb12d0 2023-07-21T19:29:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff44e933b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5dfce70 con 0x560ba5eab580 2023-07-21T19:29:56.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.404+0000 7ff431ffb700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 0x7ff4240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43878/0 (socket says 172.21.15.5:43878) 2023-07-21T19:29:56.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 -- 172.21.15.5:0/167158111 learned_addr learned my addr 172.21.15.5:0/167158111 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff431ffb700 1 -- 172.21.15.5:0/2531621168 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff42c014070 con 0x7ff434105b60 2023-07-21T19:29:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff433fff700 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eab9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff433fff700 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5d076b0 0x560ba5eb0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff43959c700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 0x7ff4240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 -- 172.21.15.5:0/167158111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 msgr2=0x560ba5eab9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eab9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 -- 172.21.15.5:0/167158111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5d076b0 msgr2=0x560ba5eb0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5d076b0 0x560ba5eb0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.405+0000 7ff438d9b700 1 -- 172.21.15.5:0/167158111 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba5c2ad80 con 0x560ba5eb12d0 2023-07-21T19:29:56.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.406+0000 7ff438d9b700 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff428009490 tx=0x7ff4280094c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f10d4db97ca22d84 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.406+0000 7ff43959c700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 0x7ff4240a3770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0ca210 tx=0x7ff42c005020 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.406+0000 7ff41effd700 1 -- 172.21.15.5:0/167158111 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800e040 con 0x560ba5eb12d0 2023-07-21T19:29:56.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.406+0000 7ff41effd700 1 -- 172.21.15.5:0/167158111 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff428009ac0 con 0x560ba5eb12d0 2023-07-21T19:29:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.406+0000 7ff41effd700 1 -- 172.21.15.5:0/167158111 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42800bd70 con 0x560ba5eb12d0 2023-07-21T19:29:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 -- 172.21.15.5:0/167158111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 msgr2=0x560ba5eab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7ff428009490 tx=0x7ff4280094c0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 -- 172.21.15.5:0/167158111 shutdown_connections 2023-07-21T19:29:56.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eab9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5d076b0 0x560ba5eb0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 --2- 172.21.15.5:0/167158111 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5eab040 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.407+0000 7ff44e933b80 1 -- 172.21.15.5:0/167158111 >> 172.21.15.5:0/167158111 conn(0x560ba5ea0be0 msgr2=0x560ba5ea5f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.408+0000 7ff44e933b80 1 -- 172.21.15.5:0/167158111 shutdown_connections 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.408+0000 7ff44e933b80 1 -- 172.21.15.5:0/167158111 wait complete. 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.408+0000 7ff44e933b80 1 -- start start 2023-07-21T19:29:56.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.409+0000 7ff44e933b80 1 Processor -- start 2023-07-21T19:29:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ebeb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eb8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5d9ca60 con 0x560ba5eb12d0 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5d9c760 con 0x560ba5eab580 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5c808c0 con 0x560ba5d076b0 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff44e933b80 10 client.0 _ll_register_callbacks cb 0x560ba5d9a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.410+0000 7ff438d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43892/0 (socket says 172.21.15.5:43892) 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 -- 172.21.15.5:0/1066867613 learned_addr learned my addr 172.21.15.5:0/1066867613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff433fff700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eb8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff433fff700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ebeb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 -- 172.21.15.5:0/1066867613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 msgr2=0x560ba5ebeb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ebeb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 -- 172.21.15.5:0/1066867613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 msgr2=0x560ba5eb8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eb8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff438d9b700 1 -- 172.21.15.5:0/1066867613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff43410d220 con 0x560ba5eab580 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.411+0000 7ff433fff700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eb8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.412+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff42800b370 tx=0x7ff4280078b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c6e1f5a51e2278a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.412+0000 7ff413fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.412+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42802b070 con 0x560ba5eab580 2023-07-21T19:29:56.406 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.412+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ba5df5e50 con 0x560ba5eab580 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.412+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff428007df0 con 0x560ba5eab580 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.413+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff42800e040 con 0x560ba5eab580 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.413+0000 7ff44e933b80 10 client.29537 fetch_fsmap learned FSMap version 60 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.413+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x560ba5c13c00 con 0x560ba5eab580 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ba5df5e50 con 0x560ba5eab580 2023-07-21T19:29:56.407 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff42802e020 con 0x560ba5eab580 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 10 client.29537 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 10 client.29537 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 20 client.29537 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5ec8b10 0x560ba5ecaf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 4 client.29537 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.414+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560ba5db7ff0 con 0x560ba5ec8b10 2023-07-21T19:29:56.408 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.415+0000 7ff433fff700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5ec8b10 0x560ba5ecaf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.416+0000 7ff433fff700 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5ec8b10 0x560ba5ecaf20 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bd5845b2e732118 server_cookie=9bdcfe1023e64db1 in_seq=0 out_seq=0 2023-07-21T19:29:56.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.416+0000 7ff413fff700 10 client.29537 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.419+0000 7ff413fff700 1 -- 172.21.15.5:0/1066867613 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560ba5db7ff0 con 0x560ba5ec8b10 2023-07-21T19:29:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.419+0000 7ff413fff700 10 client.29537 handle_command_reply: tid=0 2023-07-21T19:29:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.419+0000 7ff44e933b80 1 client.29537 shutdown 2023-07-21T19:29:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.419+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 msgr2=0x560ba5ebf080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.419+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7ff42800b370 tx=0x7ff4280078b0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 shutdown_connections 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5ec8b10 0x560ba5ecaf20 crc :-1 s=READY pgs=445 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eb8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ebeb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff413fff700 0 client.29537 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1066867613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eab580 0x560ba5ebf080 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 >> 172.21.15.5:0/1066867613 conn(0x560ba5ea0be0 msgr2=0x560ba5ea4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 shutdown_connections 2023-07-21T19:29:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.421+0000 7ff44e933b80 1 -- 172.21.15.5:0/1066867613 wait complete. 2023-07-21T19:29:56.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.422+0000 7ff44e933b80 20 client.29537 trim_cache size 0 max 16384 2023-07-21T19:29:56.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.448+0000 7ff44e933b80 1 Processor -- start 2023-07-21T19:29:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 -- start start 2023-07-21T19:29:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eb97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eacee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5eb0800 con 0x560ba5eb12d0 2023-07-21T19:29:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5ea2d00 con 0x560ba5d076b0 2023-07-21T19:29:56.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.449+0000 7ff44e933b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5eba410 con 0x560ba5eab580 2023-07-21T19:29:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43896/0 (socket says 172.21.15.5:43896) 2023-07-21T19:29:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 -- 172.21.15.5:0/271079418 learned_addr learned my addr 172.21.15.5:0/271079418 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eacee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff438d9b700 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eb97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 -- 172.21.15.5:0/271079418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 msgr2=0x560ba5eb97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eb97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 -- 172.21.15.5:0/271079418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 msgr2=0x560ba5eacee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eacee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.450+0000 7ff433fff700 1 -- 172.21.15.5:0/271079418 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560ba5eb89a0 con 0x560ba5d076b0 2023-07-21T19:29:56.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.451+0000 7ff433fff700 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff420009890 tx=0x7ff420004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f412e11036a44cc4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.451+0000 7ff41f7fe700 1 -- 172.21.15.5:0/271079418 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420016070 con 0x560ba5d076b0 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.451+0000 7ff41f7fe700 1 -- 172.21.15.5:0/271079418 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff420029070 con 0x560ba5d076b0 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.451+0000 7ff41f7fe700 1 -- 172.21.15.5:0/271079418 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420030430 con 0x560ba5d076b0 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 -- 172.21.15.5:0/271079418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 msgr2=0x560ba5eab040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7ff420009890 tx=0x7ff420004750 comp rx=0 tx=0).stop 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 -- 172.21.15.5:0/271079418 shutdown_connections 2023-07-21T19:29:56.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eb12d0 0x560ba5eacee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5d076b0 0x560ba5eab040 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 --2- 172.21.15.5:0/271079418 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5eab580 0x560ba5eb97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.452+0000 7ff44e933b80 1 -- 172.21.15.5:0/271079418 >> 172.21.15.5:0/271079418 conn(0x560ba5ea0be0 msgr2=0x560ba5ecf880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.453+0000 7ff44e933b80 1 -- 172.21.15.5:0/271079418 shutdown_connections 2023-07-21T19:29:56.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.453+0000 7ff44e933b80 1 -- 172.21.15.5:0/271079418 wait complete. 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.453+0000 7ff44e933b80 1 -- start start 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 Processor -- start 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eab580 0x560ba5ec1db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5f1b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5ec2670 con 0x560ba5eab580 2023-07-21T19:29:56.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5f1bf00 con 0x560ba5eb12d0 2023-07-21T19:29:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560ba5f1c040 con 0x560ba5d076b0 2023-07-21T19:29:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.454+0000 7ff44e933b80 10 client.0 _ll_register_callbacks cb 0x7ff428009e60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:29:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:40908/0 (socket says 172.21.15.5:40908) 2023-07-21T19:29:56.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 -- 172.21.15.5:0/1830200512 learned_addr learned my addr 172.21.15.5:0/1830200512 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:29:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5f1b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eab580 0x560ba5ec1db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 -- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 msgr2=0x560ba5f1b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5f1b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 -- 172.21.15.5:0/1830200512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eab580 msgr2=0x560ba5ec1db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:29:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eab580 0x560ba5ec1db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.455+0000 7ff433fff700 1 -- 172.21.15.5:0/1830200512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff420023af0 con 0x560ba5d076b0 2023-07-21T19:29:56.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff433fff700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff420023750 tx=0x7ff420005a00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f9f5bf7d820a654f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:29:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff413fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:29:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff420016070 con 0x560ba5d076b0 2023-07-21T19:29:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(120..120 src has 1..120) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff420020740 con 0x560ba5d076b0 2023-07-21T19:29:56.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560ba5d56490 con 0x560ba5d076b0 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.456+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff420020b00 con 0x560ba5d076b0 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 10 client.29470 fetch_fsmap learned FSMap version 60 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x560ba5c511f0 con 0x560ba5d076b0 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560ba5d56490 con 0x560ba5d076b0 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff42003c020 con 0x560ba5d076b0 2023-07-21T19:29:56.452 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 10 client.29470 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 10 client.29470 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 20 client.29470 populate_metadata read hostname 'smithi005' 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5f1d590 0x560ba5f1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 4 client.29470 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.457+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560ba5c4bf10 con 0x560ba5f1d590 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.458+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5f1d590 0x560ba5f1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:29:56.453 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.458+0000 7ff413fff700 10 client.29470 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:56.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.458+0000 7ff438d9b700 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5f1d590 0x560ba5f1f9a0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2b7f6764c4facdc server_cookie=2f1a35ef072d97b3 in_seq=0 out_seq=0 2023-07-21T19:29:56.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.482+0000 7ff413fff700 1 -- 172.21.15.5:0/1830200512 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560ba5c4bf10 con 0x560ba5f1d590 2023-07-21T19:29:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.482+0000 7ff413fff700 10 client.29470 handle_command_reply: tid=0 2023-07-21T19:29:56.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.482+0000 7ff44e933b80 1 client.29470 shutdown 2023-07-21T19:29:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 msgr2=0x560ba5ec1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7ff420023750 tx=0x7ff420005a00 comp rx=0 tx=0).stop 2023-07-21T19:29:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 shutdown_connections 2023-07-21T19:29:56.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560ba5eb12d0 0x560ba5f1b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560ba5d076b0 0x560ba5ec1870 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560ba5f1d590 0x560ba5f1f9a0 crc :-1 s=READY pgs=446 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 --2- 172.21.15.5:0/1830200512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560ba5eab580 0x560ba5ec1db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 >> 172.21.15.5:0/1830200512 conn(0x560ba5ea0be0 msgr2=0x560ba5eb97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff413fff700 0 client.29470 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 shutdown_connections 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.483+0000 7ff44e933b80 1 -- 172.21.15.5:0/1830200512 wait complete. 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.484+0000 7ff44e933b80 20 client.29470 trim_cache size 0 max 16384 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:29:56.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:29:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:29:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.484+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 msgr2=0x7ff4240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.484+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 0x7ff4240a3770 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7ff42c0ca210 tx=0x7ff42c005020 comp rx=0 tx=0).stop 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.484+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 msgr2=0x7ff434195a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434195a10 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7ff42c00e970 tx=0x7ff42c00ec80 comp rx=0 tx=0).stop 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 shutdown_connections 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff4240a12c0 0x7ff4240a3770 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff434105b60 0x7ff434195a10 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff4340ffb30 0x7ff43419bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 --2- 172.21.15.5:0/2531621168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff4341004e0 0x7ff43419c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:29:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 >> 172.21.15.5:0/2531621168 conn(0x7ff4340fb5f0 msgr2=0x7ff4341037a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:29:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 shutdown_connections 2023-07-21T19:29:56.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:29:56.485+0000 7ff43afff700 1 -- 172.21.15.5:0/2531621168 wait complete. 2023-07-21T19:29:56.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 205.0 s, 287.6 tps, lat 346.839 ms stddev 450.880 2023-07-21T19:29:57.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:56 smithi101 ceph-mon[128136]: pgmap v705: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.3 MiB/s wr, 408 op/s 2023-07-21T19:29:57.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:56 smithi005 ceph-mon[126460]: pgmap v705: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.3 MiB/s wr, 408 op/s 2023-07-21T19:29:57.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:56 smithi156 ceph-mon[128316]: pgmap v705: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 720 KiB/s rd, 5.3 MiB/s wr, 408 op/s 2023-07-21T19:30:00.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:29:59 smithi101 ceph-mon[128136]: pgmap v706: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 727 op/s 2023-07-21T19:30:00.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:29:59 smithi005 ceph-mon[126460]: pgmap v706: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 727 op/s 2023-07-21T19:30:00.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:29:59 smithi156 ceph-mon[128316]: pgmap v706: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 8.9 MiB/s wr, 727 op/s 2023-07-21T19:30:01.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:00 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T19:30:01.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:00 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T19:30:01.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:00 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T19:30:01.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 210.0 s, 414.7 tps, lat 241.649 ms stddev 82.655 2023-07-21T19:30:02.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:01 smithi101 ceph-mon[128136]: pgmap v707: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 630 op/s 2023-07-21T19:30:02.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:01 smithi005 ceph-mon[126460]: pgmap v707: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 630 op/s 2023-07-21T19:30:02.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:01 smithi156 ceph-mon[128316]: pgmap v707: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 7.7 MiB/s wr, 630 op/s 2023-07-21T19:30:04.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:03 smithi101 ceph-mon[128136]: pgmap v708: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:04.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:03 smithi005 ceph-mon[126460]: pgmap v708: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:04.159 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:03 smithi156 ceph-mon[128316]: pgmap v708: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:05.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:04 smithi101 ceph-mon[128136]: pgmap v709: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 850 op/s 2023-07-21T19:30:05.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:04 smithi005 ceph-mon[126460]: pgmap v709: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 850 op/s 2023-07-21T19:30:05.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:04 smithi156 ceph-mon[128316]: pgmap v709: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 10 MiB/s wr, 850 op/s 2023-07-21T19:30:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 215.0 s, 256.8 tps, lat 352.967 ms stddev 166.673 2023-07-21T19:30:07.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:06 smithi005 ceph-mon[126460]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:30:07.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:06 smithi005 ceph-mon[126460]: pgmap v711: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-07-21T19:30:07.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:06 smithi156 ceph-mon[128316]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:30:07.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:06 smithi156 ceph-mon[128316]: pgmap v711: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-07-21T19:30:07.318 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:06 smithi101 ceph-mon[128136]: osdmap e121: 12 total, 12 up, 12 in 2023-07-21T19:30:07.319 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:06 smithi101 ceph-mon[128136]: pgmap v711: 97 pgs: 97 active+clean; 14 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 985 op/s 2023-07-21T19:30:09.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:08 smithi101 ceph-mon[128136]: pgmap v712: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:09.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:08 smithi005 ceph-mon[126460]: pgmap v712: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:09.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:08 smithi156 ceph-mon[128316]: pgmap v712: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.728+0000 7f64771ff700 1 -- 172.21.15.5:0/1655307620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647010b090 msgr2=0x7f647010b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.728+0000 7f64771ff700 1 --2- 172.21.15.5:0/1655307620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647010b090 0x7f647010b460 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f6468009990 tx=0x7f6468009ca0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.739+0000 7f64771ff700 1 -- 172.21.15.5:0/1655307620 shutdown_connections 2023-07-21T19:30:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.739+0000 7f64771ff700 1 --2- 172.21.15.5:0/1655307620 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f647007a4f0 0x7f6470112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.739+0000 7f64771ff700 1 --2- 172.21.15.5:0/1655307620 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647010b090 0x7f647010b460 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.739+0000 7f64771ff700 1 --2- 172.21.15.5:0/1655307620 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6470079b60 0x7f6470079fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.739+0000 7f64771ff700 1 -- 172.21.15.5:0/1655307620 >> 172.21.15.5:0/1655307620 conn(0x7f6470075600 msgr2=0x7f6470077a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.740+0000 7f64771ff700 1 -- 172.21.15.5:0/1655307620 shutdown_connections 2023-07-21T19:30:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.740+0000 7f64771ff700 1 -- 172.21.15.5:0/1655307620 wait complete. 2023-07-21T19:30:09.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.741+0000 7f64771ff700 1 Processor -- start 2023-07-21T19:30:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 -- start start 2023-07-21T19:30:09.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6470079b60 0x7f647019c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 0x7f647019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a1340 con 0x7f647010b090 2023-07-21T19:30:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a14b0 con 0x7f647007a4f0 2023-07-21T19:30:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.742+0000 7f64771ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64701a1670 con 0x7f6470079b60 2023-07-21T19:30:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.743+0000 7f647579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.743+0000 7f646ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 0x7f647019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.743+0000 7f647579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58730/0 (socket says 172.21.15.5:58730) 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.743+0000 7f646ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 0x7f647019c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51948/0 (socket says 172.21.15.5:51948) 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.743+0000 7f647579c700 1 -- 172.21.15.5:0/2117290903 learned_addr learned my addr 172.21.15.5:0/2117290903 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f6474f9b700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6470079b60 0x7f647019c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 -- 172.21.15.5:0/2117290903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6470079b60 msgr2=0x7f647019c150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6470079b60 0x7f647019c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 -- 172.21.15.5:0/2117290903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 msgr2=0x7f647019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 0x7f647019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 -- 172.21.15.5:0/2117290903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64680095f0 con 0x7f647010b090 2023-07-21T19:30:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.744+0000 7f647579c700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6460007ee0 tx=0x7f646000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.745+0000 7f646dffb700 1 -- 172.21.15.5:0/2117290903 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460017070 con 0x7f647010b090 2023-07-21T19:30:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.745+0000 7f646dffb700 1 -- 172.21.15.5:0/2117290903 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f646000f040 con 0x7f647010b090 2023-07-21T19:30:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.745+0000 7f646dffb700 1 -- 172.21.15.5:0/2117290903 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460005710 con 0x7f647010b090 2023-07-21T19:30:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.745+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64701a1950 con 0x7f647010b090 2023-07-21T19:30:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.746+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64701a9190 con 0x7f647010b090 2023-07-21T19:30:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.751+0000 7f646dffb700 1 -- 172.21.15.5:0/2117290903 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6460005870 con 0x7f647010b090 2023-07-21T19:30:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.752+0000 7f648aa51b80 1 Processor -- start 2023-07-21T19:30:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.753+0000 7f646dffb700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 0x7f645c0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.754+0000 7f646dffb700 1 -- 172.21.15.5:0/2117290903 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f64600cbeb0 con 0x7f647010b090 2023-07-21T19:30:09.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.755+0000 7f646ffff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 0x7f645c0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.756+0000 7f648aa51b80 1 -- start start 2023-07-21T19:30:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca67770d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca677712d0 0x55ca6776b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca67670c50 con 0x55ca6776b580 2023-07-21T19:30:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca676c39e0 con 0x55ca675c76b0 2023-07-21T19:30:09.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f648aa51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca676bce70 con 0x55ca677712d0 2023-07-21T19:30:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f647579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f647579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58736/0 (socket says 172.21.15.5:58736) 2023-07-21T19:30:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.757+0000 7f647579c700 1 -- 172.21.15.5:0/1479181602 learned_addr learned my addr 172.21.15.5:0/1479181602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f646ffff700 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca677712d0 0x55ca6776b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f647579c700 1 -- 172.21.15.5:0/1479181602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca677712d0 msgr2=0x55ca6776b040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f647579c700 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca677712d0 0x55ca6776b040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f647579c700 1 -- 172.21.15.5:0/1479181602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 msgr2=0x55ca67770d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:30:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f647579c700 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca67770d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.758+0000 7f647579c700 1 -- 172.21.15.5:0/1479181602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca674ead80 con 0x55ca6776b580 2023-07-21T19:30:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.759+0000 7f647579c700 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f64600a0e20 tx=0x7f6460007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4d74890f060fa32 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.760+0000 7f6466ffd700 1 -- 172.21.15.5:0/1479181602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460098f00 con 0x55ca6776b580 2023-07-21T19:30:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.760+0000 7f6466ffd700 1 -- 172.21.15.5:0/1479181602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f646009cf30 con 0x55ca6776b580 2023-07-21T19:30:09.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.760+0000 7f6466ffd700 1 -- 172.21.15.5:0/1479181602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6460099160 con 0x55ca6776b580 2023-07-21T19:30:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.761+0000 7f648aa51b80 1 -- 172.21.15.5:0/1479181602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 msgr2=0x55ca6776b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.761+0000 7f648aa51b80 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f64600a0e20 tx=0x7f6460007f90 comp rx=0 tx=0).stop 2023-07-21T19:30:09.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.763+0000 7f648aa51b80 1 -- 172.21.15.5:0/1479181602 shutdown_connections 2023-07-21T19:30:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.763+0000 7f648aa51b80 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca6776b580 0x55ca6776b9d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.763+0000 7f648aa51b80 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca67770d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.763+0000 7f648aa51b80 1 --2- 172.21.15.5:0/1479181602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca677712d0 0x55ca6776b040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.763+0000 7f648aa51b80 1 -- 172.21.15.5:0/1479181602 >> 172.21.15.5:0/1479181602 conn(0x55ca67760be0 msgr2=0x55ca67765f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.754 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.766+0000 7f648aa51b80 1 -- 172.21.15.5:0/1479181602 shutdown_connections 2023-07-21T19:30:09.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.769+0000 7f648aa51b80 1 -- 172.21.15.5:0/1479181602 wait complete. 2023-07-21T19:30:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.770+0000 7f646ffff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 0x7f645c0a3830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55ca67767b40 tx=0x7f645800a7d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.770+0000 7f648aa51b80 1 -- start start 2023-07-21T19:30:09.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.772+0000 7f648aa51b80 1 Processor -- start 2023-07-21T19:30:09.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca675c76b0 0x55ca677749b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca67775430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6765ca60 con 0x55ca677712d0 2023-07-21T19:30:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6765c760 con 0x55ca6776b580 2023-07-21T19:30:09.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca675408c0 con 0x55ca675c76b0 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.774+0000 7f648aa51b80 10 client.0 _ll_register_callbacks cb 0x55ca6765a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.775+0000 7f646ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.775+0000 7f646ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51962/0 (socket says 172.21.15.5:51962) 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.775+0000 7f646ffff700 1 -- 172.21.15.5:0/303452800 learned_addr learned my addr 172.21.15.5:0/303452800 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.775+0000 7f647579c700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca67775430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.776+0000 7f646ffff700 1 -- 172.21.15.5:0/303452800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca675c76b0 msgr2=0x55ca677749b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.776+0000 7f646ffff700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca675c76b0 0x55ca677749b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.776+0000 7f646ffff700 1 -- 172.21.15.5:0/303452800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 msgr2=0x55ca67775430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.776+0000 7f646ffff700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca67775430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.776+0000 7f646ffff700 1 -- 172.21.15.5:0/303452800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64701a9190 con 0x55ca6776b580 2023-07-21T19:30:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.777+0000 7f646ffff700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f645800b010 tx=0x7f6458004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18db0998c53a6c60 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.777+0000 7f644ffff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:30:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.778+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800fcc0 con 0x55ca6776b580 2023-07-21T19:30:09.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.778+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f645800fe20 con 0x55ca6776b580 2023-07-21T19:30:09.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.780+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ca67616490 con 0x55ca6776b580 2023-07-21T19:30:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.784+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6458004560 con 0x55ca6776b580 2023-07-21T19:30:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.784+0000 7f648aa51b80 10 client.29552 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:09.772 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.784+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55ca674d3c00 con 0x55ca6776b580 2023-07-21T19:30:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ca67616490 con 0x55ca6776b580 2023-07-21T19:30:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f645801f520 con 0x55ca6776b580 2023-07-21T19:30:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 10 client.29552 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:09.775 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 10 client.29552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 20 client.29552 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca67795d70 0x55ca67798180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 4 client.29552 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:09.776 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.787+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ca676b5e50 con 0x55ca67795d70 2023-07-21T19:30:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.788+0000 7f6474f9b700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca67795d70 0x55ca67798180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.789+0000 7f6474f9b700 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca67795d70 0x55ca67798180 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=731283c43cc96295 server_cookie=bb9c37f2abcc52e0 in_seq=0 out_seq=0 2023-07-21T19:30:09.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.789+0000 7f644ffff700 10 client.29552 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:09.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.790+0000 7f644ffff700 1 -- 172.21.15.5:0/303452800 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ca676b5e50 con 0x55ca67795d70 2023-07-21T19:30:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.790+0000 7f644ffff700 10 client.29552 handle_command_reply: tid=0 2023-07-21T19:30:09.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.790+0000 7f648aa51b80 1 client.29552 shutdown 2023-07-21T19:30:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.791+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 msgr2=0x55ca67774ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.791+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f645800b010 tx=0x7f6458004750 comp rx=0 tx=0).stop 2023-07-21T19:30:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 shutdown_connections 2023-07-21T19:30:09.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca67795d70 0x55ca67798180 crc :-1 s=READY pgs=447 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca67775430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca675c76b0 0x55ca677749b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 --2- 172.21.15.5:0/303452800 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca6776b580 0x55ca67774ef0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.792+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 >> 172.21.15.5:0/303452800 conn(0x55ca67760be0 msgr2=0x55ca677654e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.793+0000 7f644ffff700 0 client.29552 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:09.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.793+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 shutdown_connections 2023-07-21T19:30:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.793+0000 7f648aa51b80 1 -- 172.21.15.5:0/303452800 wait complete. 2023-07-21T19:30:09.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.794+0000 7f648aa51b80 20 client.29552 trim_cache size 0 max 16384 2023-07-21T19:30:09.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.824+0000 7f648aa51b80 1 Processor -- start 2023-07-21T19:30:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.824+0000 7f648aa51b80 1 -- start start 2023-07-21T19:30:09.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 0x55ca6776b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca6778cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca677712d0 0x55ca67770260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca67767b70 con 0x55ca675c76b0 2023-07-21T19:30:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca677625f0 con 0x55ca677712d0 2023-07-21T19:30:09.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.825+0000 7f648aa51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca6776f740 con 0x55ca6776b580 2023-07-21T19:30:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.826+0000 7f647579c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca677712d0 0x55ca67770260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.826+0000 7f646ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca6778cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.826+0000 7f646ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca6778cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48262/0 (socket says 172.21.15.5:48262) 2023-07-21T19:30:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.826+0000 7f646ffff700 1 -- 172.21.15.5:0/2488787003 learned_addr learned my addr 172.21.15.5:0/2488787003 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:09.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.826+0000 7f6474f9b700 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 0x55ca6776b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 -- 172.21.15.5:0/2488787003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 msgr2=0x55ca6778cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca6778cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 -- 172.21.15.5:0/2488787003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca677712d0 msgr2=0x55ca67770260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca677712d0 0x55ca67770260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 -- 172.21.15.5:0/2488787003 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ca675111f0 con 0x55ca675c76b0 2023-07-21T19:30:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.829+0000 7f6474f9b700 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 0x55ca6776b040 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f6468005a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c22171293abd031 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.830+0000 7f64677fe700 1 -- 172.21.15.5:0/2488787003 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f646801e040 con 0x55ca675c76b0 2023-07-21T19:30:09.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.830+0000 7f64677fe700 1 -- 172.21.15.5:0/2488787003 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6468022570 con 0x55ca675c76b0 2023-07-21T19:30:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.830+0000 7f64677fe700 1 -- 172.21.15.5:0/2488787003 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6468022780 con 0x55ca675c76b0 2023-07-21T19:30:09.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.832+0000 7f648aa51b80 1 -- 172.21.15.5:0/2488787003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 msgr2=0x55ca6776b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.832+0000 7f648aa51b80 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 0x55ca6776b040 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f6468006010 tx=0x7f6468005a90 comp rx=0 tx=0).stop 2023-07-21T19:30:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 -- 172.21.15.5:0/2488787003 shutdown_connections 2023-07-21T19:30:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca677712d0 0x55ca67770260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca675c76b0 0x55ca6776b040 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 --2- 172.21.15.5:0/2488787003 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca6778cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 -- 172.21.15.5:0/2488787003 >> 172.21.15.5:0/2488787003 conn(0x55ca67760be0 msgr2=0x55ca6776fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 -- 172.21.15.5:0/2488787003 shutdown_connections 2023-07-21T19:30:09.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.833+0000 7f648aa51b80 1 -- 172.21.15.5:0/2488787003 wait complete. 2023-07-21T19:30:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.834+0000 7f648aa51b80 1 -- start start 2023-07-21T19:30:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.834+0000 7f648aa51b80 1 Processor -- start 2023-07-21T19:30:09.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca677dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca677e2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca677e31f0 con 0x55ca677712d0 2023-07-21T19:30:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca677e3360 con 0x55ca675c76b0 2023-07-21T19:30:09.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ca677e3520 con 0x55ca6776b580 2023-07-21T19:30:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.835+0000 7f648aa51b80 10 client.0 _ll_register_callbacks cb 0x7f64580044b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.836+0000 7f646ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.836+0000 7f646ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48276/0 (socket says 172.21.15.5:48276) 2023-07-21T19:30:09.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.836+0000 7f646ffff700 1 -- 172.21.15.5:0/4267885109 learned_addr learned my addr 172.21.15.5:0/4267885109 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:09.828 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f6474f9b700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca677dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f646ffff700 1 -- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 msgr2=0x55ca677dddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f646ffff700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca677dddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f646ffff700 1 -- 172.21.15.5:0/4267885109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 msgr2=0x55ca677e2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f646ffff700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca677e2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f646ffff700 1 -- 172.21.15.5:0/4267885109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6470113870 con 0x55ca6776b580 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.837+0000 7f6474f9b700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca677dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.838+0000 7f646ffff700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f645801f850 tx=0x7f6458005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76ad951e217d36a6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.838+0000 7f644ffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.838+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f645800fcd0 con 0x55ca6776b580 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.838+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(121..121 src has 1..121) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f645800fe30 con 0x55ca6776b580 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.839+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f646009cf30 con 0x55ca6776b580 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.839+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6458005030 con 0x55ca6776b580 2023-07-21T19:30:09.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.839+0000 7f648aa51b80 10 client.29482 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.839+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55ca674e9250 con 0x55ca6776b580 2023-07-21T19:30:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f646009cf30 con 0x55ca6776b580 2023-07-21T19:30:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6458004c90 con 0x55ca6776b580 2023-07-21T19:30:09.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 10 client.29482 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 10 client.29482 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 20 client.29482 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca677fbc10 0x55ca677fe020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 4 client.29482 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:30:09.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.842+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ca67677ff0 con 0x55ca677fbc10 2023-07-21T19:30:09.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.844+0000 7f6474f9b700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca677fbc10 0x55ca677fe020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.847+0000 7f6474f9b700 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca677fbc10 0x55ca677fe020 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e04e100ec8cbc7e8 server_cookie=6c58c5063e7cbd1d in_seq=0 out_seq=0 2023-07-21T19:30:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.847+0000 7f644ffff700 10 client.29482 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:09.881 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.893+0000 7f644ffff700 1 -- 172.21.15.5:0/4267885109 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ca67677ff0 con 0x55ca677fbc10 2023-07-21T19:30:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.893+0000 7f644ffff700 10 client.29482 handle_command_reply: tid=0 2023-07-21T19:30:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.893+0000 7f648aa51b80 1 client.29482 shutdown 2023-07-21T19:30:09.882 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 msgr2=0x55ca677de2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f645801f850 tx=0x7f6458005ee0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 shutdown_connections 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ca677fbc10 0x55ca677fe020 crc :-1 s=READY pgs=448 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ca677712d0 0x55ca677e2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ca675c76b0 0x55ca677dddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 --2- 172.21.15.5:0/4267885109 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ca6776b580 0x55ca677de2f0 secure :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f645801f850 tx=0x7f6458005ee0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 >> 172.21.15.5:0/4267885109 conn(0x55ca67760be0 msgr2=0x55ca677703e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f644ffff700 0 client.29482 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 shutdown_connections 2023-07-21T19:30:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.894+0000 7f648aa51b80 1 -- 172.21.15.5:0/4267885109 wait complete. 2023-07-21T19:30:09.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.895+0000 7f648aa51b80 20 client.29482 trim_cache size 0 max 16384 2023-07-21T19:30:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:30:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:30:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:30:09.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 msgr2=0x7f645c0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 0x7f645c0a3830 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55ca67767b40 tx=0x7f645800a7d0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 msgr2=0x7f64701a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f6460007ee0 tx=0x7f646000d3b0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 shutdown_connections 2023-07-21T19:30:09.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f645c0a1380 0x7f645c0a3830 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f647010b090 0x7f64701a0d00 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6470079b60 0x7f647019c150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 --2- 172.21.15.5:0/2117290903 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f647007a4f0 0x7f647019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 >> 172.21.15.5:0/2117290903 conn(0x7f6470075600 msgr2=0x7f6470076c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.896+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 shutdown_connections 2023-07-21T19:30:09.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:09.897+0000 7f64771ff700 1 -- 172.21.15.5:0/2117290903 wait complete. 2023-07-21T19:30:11.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:11 smithi101 ceph-mon[128136]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:11.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:11 smithi005 ceph-mon[126460]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:11.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:11 smithi156 ceph-mon[128316]: pgmap v713: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 11 MiB/s wr, 948 op/s 2023-07-21T19:30:11.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 220.0 s, 350.0 tps, lat 310.763 ms stddev 159.445 2023-07-21T19:30:14.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:13 smithi101 ceph-mon[128136]: pgmap v714: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T19:30:14.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:13 smithi101 ceph-mon[128136]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:30:14.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:13 smithi005 ceph-mon[126460]: pgmap v714: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T19:30:14.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:13 smithi005 ceph-mon[126460]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:30:14.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:13 smithi156 ceph-mon[128316]: pgmap v714: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 10 MiB/s wr, 855 op/s 2023-07-21T19:30:14.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:13 smithi156 ceph-mon[128316]: osdmap e122: 12 total, 12 up, 12 in 2023-07-21T19:30:14.704 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:14 smithi101 ceph-mon[128136]: pgmap v716: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 978 op/s 2023-07-21T19:30:15.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:14 smithi005 ceph-mon[126460]: pgmap v716: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 978 op/s 2023-07-21T19:30:15.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:14 smithi156 ceph-mon[128316]: pgmap v716: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 11 MiB/s wr, 978 op/s 2023-07-21T19:30:16.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 225.0 s, 400.6 tps, lat 232.169 ms stddev 78.239 2023-07-21T19:30:17.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:17 smithi101 ceph-mon[128136]: pgmap v717: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 845 op/s 2023-07-21T19:30:17.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:17 smithi005 ceph-mon[126460]: pgmap v717: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 845 op/s 2023-07-21T19:30:17.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:17 smithi156 ceph-mon[128316]: pgmap v717: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 9.5 MiB/s wr, 845 op/s 2023-07-21T19:30:19.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:18 smithi101 ceph-mon[128136]: pgmap v718: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:19.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:18 smithi005 ceph-mon[126460]: pgmap v718: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:19.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:18 smithi156 ceph-mon[128316]: pgmap v718: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:21.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:21 smithi101 ceph-mon[128136]: pgmap v719: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:21.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:21 smithi005 ceph-mon[126460]: pgmap v719: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:21.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:21 smithi156 ceph-mon[128316]: pgmap v719: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.02k op/s 2023-07-21T19:30:21.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 230.0 s, 106.8 tps, lat 993.685 ms stddev 1702.714 2023-07-21T19:30:22.976 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:30:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 -- 172.21.15.5:0/3422842831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac067c10 msgr2=0x7f01ac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3422842831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac067c10 0x7f01ac067fe0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f019c009990 tx=0x7f019c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 -- 172.21.15.5:0/3422842831 shutdown_connections 2023-07-21T19:30:23.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3422842831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 0x7f01ac10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3422842831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac067c10 0x7f01ac067fe0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3422842831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac0685b0 0x7f01ac104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 -- 172.21.15.5:0/3422842831 >> 172.21.15.5:0/3422842831 conn(0x7f01ac074760 msgr2=0x7f01ac074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 -- 172.21.15.5:0/3422842831 shutdown_connections 2023-07-21T19:30:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.136+0000 7f01b1fff700 1 -- 172.21.15.5:0/3422842831 wait complete. 2023-07-21T19:30:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.138+0000 7f01b1fff700 1 Processor -- start 2023-07-21T19:30:23.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 -- start start 2023-07-21T19:30:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0685b0 0x7f01ac198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 0x7f01ac19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19cf20 con 0x7f01ac104e50 2023-07-21T19:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19d090 con 0x7f01ac067c10 2023-07-21T19:30:23.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.139+0000 7f01b1fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ac19d250 con 0x7f01ac0685b0 2023-07-21T19:30:23.129 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54372/0 (socket says 172.21.15.5:54372) 2023-07-21T19:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 -- 172.21.15.5:0/3403034543 learned_addr learned my addr 172.21.15.5:0/3403034543 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01aaffd700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0685b0 0x7f01ac198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.130 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01abfff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 0x7f01ac19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 -- 172.21.15.5:0/3403034543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0685b0 msgr2=0x7f01ac198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0685b0 0x7f01ac198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 -- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 msgr2=0x7f01ac19c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.140+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 0x7f01ac19c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.141+0000 7f01ab7fe700 1 -- 172.21.15.5:0/3403034543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0095f0 con 0x7f01ac067c10 2023-07-21T19:30:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.141+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f019c0036c0 tx=0x7f019c005720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.141+0000 7f01a8ff9700 1 -- 172.21.15.5:0/3403034543 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c01d070 con 0x7f01ac067c10 2023-07-21T19:30:23.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.142+0000 7f01a8ff9700 1 -- 172.21.15.5:0/3403034543 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f019c003ec0 con 0x7f01ac067c10 2023-07-21T19:30:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.142+0000 7f01a8ff9700 1 -- 172.21.15.5:0/3403034543 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f019c00fbe0 con 0x7f01ac067c10 2023-07-21T19:30:23.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.142+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01ac19d4d0 con 0x7f01ac067c10 2023-07-21T19:30:23.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.142+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01ac19d750 con 0x7f01ac067c10 2023-07-21T19:30:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.148+0000 7f01a8ff9700 1 -- 172.21.15.5:0/3403034543 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f019c00fd40 con 0x7f01ac067c10 2023-07-21T19:30:23.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.148+0000 7f01c59dfb80 1 Processor -- start 2023-07-21T19:30:23.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 -- start start 2023-07-21T19:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd50a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd551a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dc4dc50 con 0x56279dba46b0 2023-07-21T19:30:23.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dca09e0 con 0x56279dd50f80 2023-07-21T19:30:23.140 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.150+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dc99e70 con 0x56279dd513f0 2023-07-21T19:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.151+0000 7f01aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.151+0000 7f01aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:40300/0 (socket says 172.21.15.5:40300) 2023-07-21T19:30:23.141 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.151+0000 7f01aaffd700 1 -- 172.21.15.5:0/2659216226 learned_addr learned my addr 172.21.15.5:0/2659216226 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.151+0000 7f01abfff700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd50a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd551a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01a8ff9700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 0x7f01940a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.142 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01aaffd700 1 -- 172.21.15.5:0/2659216226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 msgr2=0x56279dd551a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01aaffd700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd551a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01aaffd700 1 -- 172.21.15.5:0/2659216226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 msgr2=0x56279dd50a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01aaffd700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd50a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01aaffd700 1 -- 172.21.15.5:0/2659216226 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56279dac7d80 con 0x56279dd513f0 2023-07-21T19:30:23.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.152+0000 7f01a8ff9700 1 -- 172.21.15.5:0/3403034543 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f019c0cd9e0 con 0x7f01ac067c10 2023-07-21T19:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.153+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.153+0000 7f01abfff700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd50a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:30:23.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.153+0000 7f01aaffd700 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bcb0 tx=0x7f01a00103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=315b354cf0a4852f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.154+0000 7f0191ffb700 1 -- 172.21.15.5:0/2659216226 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0010dc0 con 0x56279dd513f0 2023-07-21T19:30:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.154+0000 7f0191ffb700 1 -- 172.21.15.5:0/2659216226 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01a0015070 con 0x56279dd513f0 2023-07-21T19:30:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.154+0000 7f0191ffb700 1 -- 172.21.15.5:0/2659216226 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0005020 con 0x56279dd513f0 2023-07-21T19:30:23.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01abfff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 0x7f01940a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 -- 172.21.15.5:0/2659216226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 msgr2=0x56279dd556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f01a000bcb0 tx=0x7f01a00103d0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 -- 172.21.15.5:0/2659216226 shutdown_connections 2023-07-21T19:30:23.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd556e0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd50a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/2659216226 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd551a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.155+0000 7f01c59dfb80 1 -- 172.21.15.5:0/2659216226 >> 172.21.15.5:0/2659216226 conn(0x56279dd3dbe0 msgr2=0x56279dd42f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.156+0000 7f01c59dfb80 1 -- 172.21.15.5:0/2659216226 shutdown_connections 2023-07-21T19:30:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.157+0000 7f01c59dfb80 1 -- 172.21.15.5:0/2659216226 wait complete. 2023-07-21T19:30:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.157+0000 7f01c59dfb80 1 -- start start 2023-07-21T19:30:23.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.158+0000 7f01c59dfb80 1 Processor -- start 2023-07-21T19:30:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.158+0000 7f01abfff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 0x7f01940a3870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009550 tx=0x7f01a400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5cea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd5a120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.151 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dc39a60 con 0x56279dba46b0 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dc39760 con 0x56279dd50f80 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279db1d8c0 con 0x56279dd513f0 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01c59dfb80 10 client.0 _ll_register_callbacks cb 0x56279dc37700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01abfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01abfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:40312/0 (socket says 172.21.15.5:40312) 2023-07-21T19:30:23.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01abfff700 1 -- 172.21.15.5:0/3601194946 learned_addr learned my addr 172.21.15.5:0/3601194946 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.159+0000 7f01aaffd700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd5a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 -- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 msgr2=0x56279dd5a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd5a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 -- 172.21.15.5:0/3601194946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 msgr2=0x56279dd5cea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 -- 172.21.15.5:0/3601194946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01a000b8e0 con 0x56279dd513f0 2023-07-21T19:30:23.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f01abfff700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01a4016c00 tx=0x7f01a4009040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d7c99eae8c7e97c6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.160+0000 7f018affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.161+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a4015a00 con 0x56279dd513f0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.161+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f01a4015b60 con 0x56279dd513f0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.161+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56279dbf3490 con 0x56279dd513f0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a4015d20 con 0x56279dd513f0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 10 client.29494 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56279dab0c00 con 0x56279dd513f0 2023-07-21T19:30:23.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56279dbf3490 con 0x56279dd513f0 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01a402a3b0 con 0x56279dd513f0 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 10 client.29494 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 10 client.29494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 20 client.29494 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd6a4d0 0x56279dd6c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 4 client.29494 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:23.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.162+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56279dc92e50 con 0x56279dd6a4d0 2023-07-21T19:30:23.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.163+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd6a4d0 0x56279dd6c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.163+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd6a4d0 0x56279dd6c8e0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d25a6ed214dce6f6 server_cookie=d692472d60b8483d in_seq=0 out_seq=0 2023-07-21T19:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.163+0000 7f018affd700 10 client.29494 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.164+0000 7f018affd700 1 -- 172.21.15.5:0/3601194946 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56279dc92e50 con 0x56279dd6a4d0 2023-07-21T19:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.164+0000 7f018affd700 10 client.29494 handle_command_reply: tid=0 2023-07-21T19:30:23.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.164+0000 7f01c59dfb80 1 client.29494 shutdown 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 msgr2=0x56279dd5a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f01a4016c00 tx=0x7f01a4009040 comp rx=0 tx=0).stop 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 shutdown_connections 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd513f0 0x56279dd5a860 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd6a4d0 0x56279dd6c8e0 crc :-1 s=READY pgs=449 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5cea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/3601194946 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd50f80 0x56279dd5a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 >> 172.21.15.5:0/3601194946 conn(0x56279dd3dbe0 msgr2=0x56279dd424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f018affd700 0 client.29494 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.165+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 shutdown_connections 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.166+0000 7f01c59dfb80 1 -- 172.21.15.5:0/3601194946 wait complete. 2023-07-21T19:30:23.159 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.166+0000 7f01c59dfb80 20 client.29494 trim_cache size 0 max 16384 2023-07-21T19:30:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.194+0000 7f01c59dfb80 1 Processor -- start 2023-07-21T19:30:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 -- start start 2023-07-21T19:30:23.183 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd55480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279dd52fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dd55f50 con 0x56279dba46b0 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dd5ae60 con 0x56279dd513f0 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.195+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279dd54690 con 0x56279dd50f80 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd55480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:40328/0 (socket says 172.21.15.5:40328) 2023-07-21T19:30:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01aaffd700 1 -- 172.21.15.5:0/4135674169 learned_addr learned my addr 172.21.15.5:0/4135674169 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279dd52fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 -- 172.21.15.5:0/4135674169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 msgr2=0x56279dd5c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 -- 172.21.15.5:0/4135674169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 msgr2=0x56279dd55480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd55480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01abfff700 1 -- 172.21.15.5:0/4135674169 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56279daee1f0 con 0x56279dd513f0 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01aaffd700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:30:23.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.196+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd55480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:30:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.197+0000 7f01abfff700 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279dd52fd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009010 tx=0x7f01a4004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aedbc8690369f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.197+0000 7f01927fc700 1 -- 172.21.15.5:0/4135674169 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40093e0 con 0x56279dd513f0 2023-07-21T19:30:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.197+0000 7f01927fc700 1 -- 172.21.15.5:0/4135674169 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01a400f030 con 0x56279dd513f0 2023-07-21T19:30:23.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.197+0000 7f01927fc700 1 -- 172.21.15.5:0/4135674169 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40216c0 con 0x56279dd513f0 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 -- 172.21.15.5:0/4135674169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 msgr2=0x56279dd52fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279dd52fd0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009010 tx=0x7f01a4004ad0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 -- 172.21.15.5:0/4135674169 shutdown_connections 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279dd52fd0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd55480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/4135674169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.198+0000 7f01c59dfb80 1 -- 172.21.15.5:0/4135674169 >> 172.21.15.5:0/4135674169 conn(0x56279dd3dbe0 msgr2=0x56279dd54d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.199+0000 7f01c59dfb80 1 -- 172.21.15.5:0/4135674169 shutdown_connections 2023-07-21T19:30:23.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.199+0000 7f01c59dfb80 1 -- 172.21.15.5:0/4135674169 wait complete. 2023-07-21T19:30:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.200+0000 7f01c59dfb80 1 -- start start 2023-07-21T19:30:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.200+0000 7f01c59dfb80 1 Processor -- start 2023-07-21T19:30:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279ddb7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279ddb7970 con 0x56279dba46b0 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279ddb7ae0 con 0x56279dd513f0 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56279ddb7ca0 con 0x56279dd50f80 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01c59dfb80 10 client.0 _ll_register_callbacks cb 0x7f01a40167b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5d6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41766/0 (socket says 172.21.15.5:41766) 2023-07-21T19:30:23.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01ab7fe700 1 -- 172.21.15.5:0/1870965382 learned_addr learned my addr 172.21.15.5:0/1870965382 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.201+0000 7f01abfff700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279ddb7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 -- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 msgr2=0x56279ddb7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279ddb7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 -- 172.21.15.5:0/1870965382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 msgr2=0x56279dd5d6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5d6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 -- 172.21.15.5:0/1870965382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56279dac7d80 con 0x56279dd50f80 2023-07-21T19:30:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f01aaffd700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5dc10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005d90 tx=0x7f01a00038b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b31ea01472c746c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.202+0000 7f018affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a0011440 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f01a0015070 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f01a0017070 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f01a00115a0 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f01c59dfb80 10 client.29500 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.203+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x56279dab0c00 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f01a0032080 con 0x56279dd50f80 2023-07-21T19:30:23.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f01a002c030 con 0x56279dd50f80 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 10 client.29500 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 10 client.29500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 20 client.29500 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd48de0 0x56279dd4b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 4 client.29500 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.204+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56279dc54ff0 con 0x56279dd48de0 2023-07-21T19:30:23.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.205+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd48de0 0x56279dd4b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.206+0000 7f01ab7fe700 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd48de0 0x56279dd4b1f0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e64850a0922d2e3e server_cookie=e10f6410eeab1742 in_seq=0 out_seq=0 2023-07-21T19:30:23.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.206+0000 7f018affd700 10 client.29500 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.242+0000 7f018affd700 1 -- 172.21.15.5:0/1870965382 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56279dc54ff0 con 0x56279dd48de0 2023-07-21T19:30:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.242+0000 7f018affd700 10 client.29500 handle_command_reply: tid=0 2023-07-21T19:30:23.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 client.29500 shutdown 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 msgr2=0x56279dd5dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5dc10 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005d90 tx=0x7f01a00038b0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 shutdown_connections 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56279dd513f0 0x56279ddb7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56279dd48de0 0x56279dd4b1f0 crc :-1 s=READY pgs=450 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56279dba46b0 0x56279dd5d6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 --2- 172.21.15.5:0/1870965382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56279dd50f80 0x56279dd5dc10 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.243+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 >> 172.21.15.5:0/1870965382 conn(0x56279dd3dbe0 msgr2=0x56279dd478c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.244+0000 7f018affd700 0 client.29500 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.244+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 shutdown_connections 2023-07-21T19:30:23.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.259+0000 7f01c59dfb80 1 -- 172.21.15.5:0/1870965382 wait complete. 2023-07-21T19:30:23.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.272+0000 7f01c59dfb80 20 client.29500 trim_cache size 0 max 16384 2023-07-21T19:30:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:30:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:30:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:30:23.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:30:23.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.277+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 msgr2=0x7f01940a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.277+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 0x7f01940a3870 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009550 tx=0x7f01a400b040 comp rx=0 tx=0).stop 2023-07-21T19:30:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.277+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 msgr2=0x7f01ac197d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.277+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f019c0036c0 tx=0x7f019c005720 comp rx=0 tx=0).stop 2023-07-21T19:30:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 shutdown_connections 2023-07-21T19:30:23.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f01940a13c0 0x7f01940a3870 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01ac104e50 0x7f01ac19c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f01ac067c10 0x7f01ac197d30 secure :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f019c0036c0 tx=0x7f019c005720 comp rx=0 tx=0).stop 2023-07-21T19:30:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 --2- 172.21.15.5:0/3403034543 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01ac0685b0 0x7f01ac198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.282+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 >> 172.21.15.5:0/3403034543 conn(0x7f01ac074760 msgr2=0x7f01ac0fc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.287+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 shutdown_connections 2023-07-21T19:30:23.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:23.290+0000 7f01b1fff700 1 -- 172.21.15.5:0/3403034543 wait complete. 2023-07-21T19:30:23.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.340+0000 7f1009fff700 1 -- 172.21.15.5:0/2339089118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1004105ae0 msgr2=0x7f1004105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.340+0000 7f1009fff700 1 --2- 172.21.15.5:0/2339089118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1004105ae0 0x7f1004105eb0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009990 tx=0x7f0ff4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 -- 172.21.15.5:0/2339089118 shutdown_connections 2023-07-21T19:30:23.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 --2- 172.21.15.5:0/2339089118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 --2- 172.21.15.5:0/2339089118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1004105ae0 0x7f1004105eb0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 --2- 172.21.15.5:0/2339089118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f10040ffad0 0x7f10040fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 -- 172.21.15.5:0/2339089118 >> 172.21.15.5:0/2339089118 conn(0x7f10040fb5f0 msgr2=0x7f10040fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 -- 172.21.15.5:0/2339089118 shutdown_connections 2023-07-21T19:30:23.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.341+0000 7f1009fff700 1 -- 172.21.15.5:0/2339089118 wait complete. 2023-07-21T19:30:23.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.342+0000 7f1009fff700 1 Processor -- start 2023-07-21T19:30:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.342+0000 7f1009fff700 1 -- start start 2023-07-21T19:30:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10040ffad0 0x7f100410ea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100410f800 con 0x7f10040ffad0 2023-07-21T19:30:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004113b10 con 0x7f1004105ae0 2023-07-21T19:30:23.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1009fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1004113cd0 con 0x7f1004100480 2023-07-21T19:30:23.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1003fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f10037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10040ffad0 0x7f100410ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1003fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54448/0 (socket says 172.21.15.5:54448) 2023-07-21T19:30:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1003fff700 1 -- 172.21.15.5:0/3493665583 learned_addr learned my addr 172.21.15.5:0/3493665583 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.343+0000 7f1002ffd700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 -- 172.21.15.5:0/3493665583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 msgr2=0x7f100410ef90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410ef90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 -- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10040ffad0 msgr2=0x7f100410ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10040ffad0 0x7f100410ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 -- 172.21.15.5:0/3493665583 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff40095f0 con 0x7f1004105ae0 2023-07-21T19:30:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1002ffd700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:30:23.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.344+0000 7f1003fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007ee0 tx=0x7f0ffc00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.345+0000 7f1000ff9700 1 -- 172.21.15.5:0/3493665583 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc017070 con 0x7f1004105ae0 2023-07-21T19:30:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.345+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1004113fb0 con 0x7f1004105ae0 2023-07-21T19:30:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.345+0000 7f1000ff9700 1 -- 172.21.15.5:0/3493665583 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ffc00f040 con 0x7f1004105ae0 2023-07-21T19:30:23.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.345+0000 7f1000ff9700 1 -- 172.21.15.5:0/3493665583 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc005570 con 0x7f1004105ae0 2023-07-21T19:30:23.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.345+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10041b1960 con 0x7f1004105ae0 2023-07-21T19:30:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.349+0000 7f1000ff9700 1 -- 172.21.15.5:0/3493665583 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0ffc00db20 con 0x7f1004105ae0 2023-07-21T19:30:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.350+0000 7f101d957b80 1 Processor -- start 2023-07-21T19:30:23.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.350+0000 7f101d957b80 1 -- start start 2023-07-21T19:30:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaae590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aab0eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a9a9cb0 con 0x55798aaa3be0 2023-07-21T19:30:23.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a9fca40 con 0x55798aaa32d0 2023-07-21T19:30:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f101d957b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a9f5ed0 con 0x55798aaaead0 2023-07-21T19:30:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54456/0 (socket says 172.21.15.5:54456) 2023-07-21T19:30:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f1002ffd700 1 -- 172.21.15.5:0/2508341845 learned_addr learned my addr 172.21.15.5:0/2508341845 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aab0eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.351+0000 7f10037fe700 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1000ff9700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 0x7f0fec0a37d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1000ff9700 1 -- 172.21.15.5:0/3493665583 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0ffc0cc3b0 con 0x7f1004105ae0 2023-07-21T19:30:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1002ffd700 1 -- 172.21.15.5:0/2508341845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 msgr2=0x55798aab0eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aab0eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1002ffd700 1 -- 172.21.15.5:0/2508341845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 msgr2=0x55798aaae590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaae590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.352+0000 7f1002ffd700 1 -- 172.21.15.5:0/2508341845 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798a823de0 con 0x55798aaa32d0 2023-07-21T19:30:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.353+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8011c70 tx=0x7f0ff8009c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbddf37a21c3c4a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.353+0000 7f0fe9ffb700 1 -- 172.21.15.5:0/2508341845 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff800f6d0 con 0x55798aaa32d0 2023-07-21T19:30:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.353+0000 7f1003fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 0x7f0fec0a37d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.353+0000 7f0fe9ffb700 1 -- 172.21.15.5:0/2508341845 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff800fcc0 con 0x55798aaa32d0 2023-07-21T19:30:23.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.353+0000 7f0fe9ffb700 1 -- 172.21.15.5:0/2508341845 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff801fb50 con 0x55798aaa32d0 2023-07-21T19:30:23.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.354+0000 7f101d957b80 1 -- 172.21.15.5:0/2508341845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 msgr2=0x55798aaa36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.354+0000 7f101d957b80 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8011c70 tx=0x7f0ff8009c30 comp rx=0 tx=0).stop 2023-07-21T19:30:23.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f1003fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 0x7f0fec0a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0993b0 tx=0x7f0ffc00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 -- 172.21.15.5:0/2508341845 shutdown_connections 2023-07-21T19:30:23.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aab0eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaa36a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 --2- 172.21.15.5:0/2508341845 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaae590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 -- 172.21.15.5:0/2508341845 >> 172.21.15.5:0/2508341845 conn(0x55798aa99da0 msgr2=0x55798aa9f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 -- 172.21.15.5:0/2508341845 shutdown_connections 2023-07-21T19:30:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.355+0000 7f101d957b80 1 -- 172.21.15.5:0/2508341845 wait complete. 2023-07-21T19:30:23.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.356+0000 7f101d957b80 1 -- start start 2023-07-21T19:30:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.356+0000 7f101d957b80 1 Processor -- start 2023-07-21T19:30:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaab7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaabf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a995ac0 con 0x55798aaa3be0 2023-07-21T19:30:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a9957c0 con 0x55798aaa32d0 2023-07-21T19:30:23.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798a879920 con 0x55798aaaead0 2023-07-21T19:30:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f101d957b80 10 client.0 _ll_register_callbacks cb 0x55798a993760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54458/0 (socket says 172.21.15.5:54458) 2023-07-21T19:30:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f1002ffd700 1 -- 172.21.15.5:0/827970831 learned_addr learned my addr 172.21.15.5:0/827970831 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f1002ffd700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaabf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.357+0000 7f10037fe700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaab7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 -- 172.21.15.5:0/827970831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 msgr2=0x55798aaabf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaabf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 -- 172.21.15.5:0/827970831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 msgr2=0x55798aaab7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaab7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 -- 172.21.15.5:0/827970831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff8019040 con 0x55798aaa32d0 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f10037fe700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:30:23.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f1002ffd700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ff801f750 tx=0x7f0ff8011a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=702a61271bf8c592 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f0fe2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.358+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8023070 con 0x55798aaa32d0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55798a94f4f0 con 0x55798aaa32d0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0ff8005050 con 0x55798aaa32d0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff8005210 con 0x55798aaa32d0 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f101d957b80 10 client.29585 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:23.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55798a80cc60 con 0x55798aaa32d0 2023-07-21T19:30:23.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.359+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55798a94f4f0 con 0x55798aaa32d0 2023-07-21T19:30:23.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ff8004750 con 0x55798aaa32d0 2023-07-21T19:30:23.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 10 client.29585 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:23.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 10 client.29585 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 20 client.29585 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798aabe2b0 0x55798aac06c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 4 client.29585 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55798a9eeeb0 con 0x55798aabe2b0 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.360+0000 7f10037fe700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798aabe2b0 0x55798aac06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.361+0000 7f0fe2ffd700 10 client.29585 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.361+0000 7f10037fe700 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798aabe2b0 0x55798aac06c0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79e342128683e01 server_cookie=13131da24bb10db1 in_seq=0 out_seq=0 2023-07-21T19:30:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/827970831 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55798a9eeeb0 con 0x55798aabe2b0 2023-07-21T19:30:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f0fe2ffd700 10 client.29585 handle_command_reply: tid=0 2023-07-21T19:30:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 client.29585 shutdown 2023-07-21T19:30:23.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 msgr2=0x55798aaae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f0ff801f750 tx=0x7f0ff8011a40 comp rx=0 tx=0).stop 2023-07-21T19:30:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 shutdown_connections 2023-07-21T19:30:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaabf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa32d0 0x55798aaae570 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798aabe2b0 0x55798aac06c0 crc :-1 s=READY pgs=451 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f101d957b80 1 --2- 172.21.15.5:0/827970831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aaab7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.362+0000 7f0fe2ffd700 0 client.29585 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.363+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 >> 172.21.15.5:0/827970831 conn(0x55798aa99da0 msgr2=0x55798aa9f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.363+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 shutdown_connections 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.363+0000 7f101d957b80 1 -- 172.21.15.5:0/827970831 wait complete. 2023-07-21T19:30:23.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.363+0000 7f101d957b80 20 client.29585 trim_cache size 0 max 16384 2023-07-21T19:30:23.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 Processor -- start 2023-07-21T19:30:23.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 -- start start 2023-07-21T19:30:23.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa32d0 0x55798aab54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaa5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798aaa4d60 con 0x55798aaa32d0 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798aaac740 con 0x55798aaa3be0 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.392+0000 7f101d957b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798aaa9d50 con 0x55798aaaead0 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f10037fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f10037fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:54464/0 (socket says 172.21.15.5:54464) 2023-07-21T19:30:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f10037fe700 1 -- 172.21.15.5:0/2034461363 learned_addr learned my addr 172.21.15.5:0/2034461363 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaa5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f10037fe700 1 -- 172.21.15.5:0/2034461363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 msgr2=0x55798aaa5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.393+0000 7f10037fe700 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaa5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.394+0000 7f10037fe700 1 -- 172.21.15.5:0/2034461363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa32d0 msgr2=0x55798aab54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.394+0000 7f10037fe700 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa32d0 0x55798aab54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.394+0000 7f10037fe700 1 -- 172.21.15.5:0/2034461363 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55798a84a250 con 0x55798aaa3be0 2023-07-21T19:30:23.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.394+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaa5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:30:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.394+0000 7f10037fe700 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009990 tx=0x7f0ff4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=907144feb70e8cc5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.395+0000 7f0fea7fc700 1 -- 172.21.15.5:0/2034461363 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff402e070 con 0x55798aaa3be0 2023-07-21T19:30:23.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.395+0000 7f0fea7fc700 1 -- 172.21.15.5:0/2034461363 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff401e040 con 0x55798aaa3be0 2023-07-21T19:30:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.395+0000 7f0fea7fc700 1 -- 172.21.15.5:0/2034461363 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff401d030 con 0x55798aaa3be0 2023-07-21T19:30:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- 172.21.15.5:0/2034461363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 msgr2=0x55798aaba700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f0ff4009990 tx=0x7f0ff4009310 comp rx=0 tx=0).stop 2023-07-21T19:30:23.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- 172.21.15.5:0/2034461363 shutdown_connections 2023-07-21T19:30:23.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaaead0 0x55798aaa5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa32d0 0x55798aab54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 --2- 172.21.15.5:0/2034461363 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaa3be0 0x55798aaba700 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- 172.21.15.5:0/2034461363 >> 172.21.15.5:0/2034461363 conn(0x55798aa99da0 msgr2=0x55798aa9de10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- 172.21.15.5:0/2034461363 shutdown_connections 2023-07-21T19:30:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- 172.21.15.5:0/2034461363 wait complete. 2023-07-21T19:30:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.396+0000 7f101d957b80 1 -- start start 2023-07-21T19:30:23.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.397+0000 7f101d957b80 1 Processor -- start 2023-07-21T19:30:23.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aab3610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaaead0 0x55798aab3b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798aab4220 con 0x55798aaa3be0 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798aab4390 con 0x55798aaaead0 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55798ab12140 con 0x55798aaa32d0 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f101d957b80 10 client.0 _ll_register_callbacks cb 0x7f0ff8011b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:23.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaaead0 0x55798aab3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f1002ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:40398/0 (socket says 172.21.15.5:40398) 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.398+0000 7f1002ffd700 1 -- 172.21.15.5:0/2992999923 learned_addr learned my addr 172.21.15.5:0/2992999923 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f10037fe700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aab3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f1002ffd700 1 -- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaaead0 msgr2=0x55798aab3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaaead0 0x55798aab3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f1002ffd700 1 -- 172.21.15.5:0/2992999923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 msgr2=0x55798aab3610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aab3610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.399+0000 7f1002ffd700 1 -- 172.21.15.5:0/2992999923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100410d210 con 0x55798aaa32d0 2023-07-21T19:30:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.400+0000 7f1002ffd700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8001eb0 tx=0x7f0ff8011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da841f7cc65181c4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:23.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.400+0000 7f0fe2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.400+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff8023040 con 0x55798aaa32d0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.400+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0ff8000e40 con 0x55798aaa32d0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.400+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0ff800fcc0 con 0x55798aaa32d0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.401+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=60) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0ff8011310 con 0x55798aaa32d0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.401+0000 7f101d957b80 10 client.29518 fetch_fsmap learned FSMap version 60 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.401+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=60,monmap=4+}) v3 -- 0x55798a8222b0 con 0x55798aaa32d0 2023-07-21T19:30:23.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.401+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0ff800fcc0 con 0x55798aaa32d0 2023-07-21T19:30:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.401+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 60) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0ff800f3c0 con 0x55798aaa32d0 2023-07-21T19:30:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 10 client.29518 fetch_fsmap finished waiting for FSMap version 60 2023-07-21T19:30:23.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 10 client.29518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 20 client.29518 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798ab135b0 0x55798ab159c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 4 client.29518 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:30:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55798a9b1050 con 0x55798ab135b0 2023-07-21T19:30:23.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.402+0000 7f10037fe700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798ab135b0 0x55798ab159c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.403+0000 7f10037fe700 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798ab135b0 0x55798ab159c0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=82ee20d633a551b4 server_cookie=210c587aded8e81b in_seq=0 out_seq=0 2023-07-21T19:30:23.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.403+0000 7f0fe2ffd700 10 client.29518 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.403+0000 7f0fe2ffd700 1 -- 172.21.15.5:0/2992999923 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55798a9b1050 con 0x55798ab135b0 2023-07-21T19:30:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.403+0000 7f0fe2ffd700 10 client.29518 handle_command_reply: tid=0 2023-07-21T19:30:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.403+0000 7f101d957b80 1 client.29518 shutdown 2023-07-21T19:30:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 msgr2=0x55798aab52e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f0ff8001eb0 tx=0x7f0ff8011a40 comp rx=0 tx=0).stop 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 shutdown_connections 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55798aaaead0 0x55798aab3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55798aaa32d0 0x55798aab52e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55798ab135b0 0x55798ab159c0 crc :-1 s=READY pgs=452 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 --2- 172.21.15.5:0/2992999923 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55798aaa3be0 0x55798aab3610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 >> 172.21.15.5:0/2992999923 conn(0x55798aa99da0 msgr2=0x55798aaa5270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f0fe2ffd700 0 client.29518 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 shutdown_connections 2023-07-21T19:30:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.404+0000 7f101d957b80 1 -- 172.21.15.5:0/2992999923 wait complete. 2023-07-21T19:30:23.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f101d957b80 20 client.29518 trim_cache size 0 max 16384 2023-07-21T19:30:23.401 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:30:23.402 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:30:23.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:30:23.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:30:23.403 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:30:23.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 msgr2=0x7f0fec0a37d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 0x7f0fec0a37d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc0993b0 tx=0x7f0ffc00b040 comp rx=0 tx=0).stop 2023-07-21T19:30:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 msgr2=0x7f10041135d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc007ee0 tx=0x7f0ffc00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.405+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 shutdown_connections 2023-07-21T19:30:23.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1004105ae0 0x7f10041135d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0fec0a1320 0x7f0fec0a37d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f10040ffad0 0x7f100410ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 --2- 172.21.15.5:0/3493665583 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1004100480 0x7f100410ef90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 >> 172.21.15.5:0/3493665583 conn(0x7f10040fb5f0 msgr2=0x7f10041035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 shutdown_connections 2023-07-21T19:30:23.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:23.406+0000 7f1009fff700 1 -- 172.21.15.5:0/3493665583 wait complete. 2023-07-21T19:30:23.422 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:30:24.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:23 smithi101 ceph-mon[128136]: pgmap v720: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.3 MiB/s wr, 763 op/s 2023-07-21T19:30:24.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:23 smithi005 ceph-mon[126460]: pgmap v720: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.3 MiB/s wr, 763 op/s 2023-07-21T19:30:24.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:23 smithi156 ceph-mon[128316]: pgmap v720: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 8.3 MiB/s wr, 763 op/s 2023-07-21T19:30:25.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:24 smithi101 ceph-mon[128136]: pgmap v721: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 627 op/s 2023-07-21T19:30:25.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:24 smithi005 ceph-mon[126460]: pgmap v721: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 627 op/s 2023-07-21T19:30:25.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:24 smithi156 ceph-mon[128316]: pgmap v721: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.9 MiB/s wr, 627 op/s 2023-07-21T19:30:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 235.0 s, 434.8 tps, lat 231.712 ms stddev 49.495 2023-07-21T19:30:27.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:26 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:27.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:26 smithi101 ceph-mon[128136]: pgmap v722: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 614 op/s 2023-07-21T19:30:27.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:26 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:27.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:26 smithi005 ceph-mon[126460]: pgmap v722: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 614 op/s 2023-07-21T19:30:27.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:26 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:27.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:26 smithi156 ceph-mon[128316]: pgmap v722: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 6.8 MiB/s wr, 614 op/s 2023-07-21T19:30:29.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:28 smithi101 ceph-mon[128136]: pgmap v723: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-21T19:30:29.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:28 smithi005 ceph-mon[126460]: pgmap v723: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-21T19:30:29.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:28 smithi156 ceph-mon[128316]: pgmap v723: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 12 MiB/s wr, 1.10k op/s 2023-07-21T19:30:31.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:31 smithi101 ceph-mon[128136]: pgmap v724: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 667 op/s 2023-07-21T19:30:31.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:31 smithi005 ceph-mon[126460]: pgmap v724: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 667 op/s 2023-07-21T19:30:31.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:31 smithi156 ceph-mon[128316]: pgmap v724: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 7.6 MiB/s wr, 667 op/s 2023-07-21T19:30:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 240.0 s, 389.4 tps, lat 257.272 ms stddev 44.325 2023-07-21T19:30:32.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:32 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:32.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:32 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:32.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:32 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:30:33.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:33 smithi101 ceph-mon[128136]: pgmap v725: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 1.19k op/s 2023-07-21T19:30:33.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:33 smithi005 ceph-mon[126460]: pgmap v725: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 1.19k op/s 2023-07-21T19:30:33.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:33 smithi156 ceph-mon[128316]: pgmap v725: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 13 MiB/s wr, 1.19k op/s 2023-07-21T19:30:35.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:35 smithi101 ceph-mon[128136]: pgmap v726: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:35.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:35 smithi005 ceph-mon[126460]: pgmap v726: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:35.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:35 smithi156 ceph-mon[128316]: pgmap v726: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.08k op/s 2023-07-21T19:30:36.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 -- 172.21.15.5:0/4160069124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa714114c80 msgr2=0x7fa714117060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/4160069124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa714114c80 0x7fa714117060 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0098f0 tx=0x7fa70c009c00 comp rx=0 tx=0).stop 2023-07-21T19:30:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 -- 172.21.15.5:0/4160069124 shutdown_connections 2023-07-21T19:30:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/4160069124 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa714114c80 0x7fa714117060 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/4160069124 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71410a1b0 0x7fa71406c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/4160069124 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa71406ce50 0x7fa71406d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.736+0000 7fa71b9ff700 1 -- 172.21.15.5:0/4160069124 >> 172.21.15.5:0/4160069124 conn(0x7fa71406bd70 msgr2=0x7fa714070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.737+0000 7fa71b9ff700 1 -- 172.21.15.5:0/4160069124 shutdown_connections 2023-07-21T19:30:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.737+0000 7fa71b9ff700 1 -- 172.21.15.5:0/4160069124 wait complete. 2023-07-21T19:30:36.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.738+0000 7fa71b9ff700 1 Processor -- start 2023-07-21T19:30:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.738+0000 7fa71b9ff700 1 -- start start 2023-07-21T19:30:36.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71406ce50 0x7fa7141a6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa714114c80 0x7fa7141a0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141a0bf0 con 0x7fa71406ce50 2023-07-21T19:30:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141a0d60 con 0x7fa71410a1b0 2023-07-21T19:30:36.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.739+0000 7fa71b9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7141a0f20 con 0x7fa714114c80 2023-07-21T19:30:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.740+0000 7fa718f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.740+0000 7fa718f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41374/0 (socket says 172.21.15.5:41374) 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.740+0000 7fa718f9a700 1 -- 172.21.15.5:0/1656157326 learned_addr learned my addr 172.21.15.5:0/1656157326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.740+0000 7fa71979b700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71406ce50 0x7fa7141a6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.740+0000 7fa719f9c700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa714114c80 0x7fa7141a0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 -- 172.21.15.5:0/1656157326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa714114c80 msgr2=0x7fa7141a0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa714114c80 0x7fa7141a0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 -- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71406ce50 msgr2=0x7fa7141a6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71406ce50 0x7fa7141a6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 -- 172.21.15.5:0/1656157326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c009520 con 0x7fa71410a1b0 2023-07-21T19:30:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.741+0000 7fa718f9a700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa70400b550 tx=0x7fa70400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.742+0000 7fa70a7fc700 1 -- 172.21.15.5:0/1656157326 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704011830 con 0x7fa71410a1b0 2023-07-21T19:30:36.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.742+0000 7fa70a7fc700 1 -- 172.21.15.5:0/1656157326 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa704011e20 con 0x7fa71410a1b0 2023-07-21T19:30:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.742+0000 7fa70a7fc700 1 -- 172.21.15.5:0/1656157326 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704010dc0 con 0x7fa71410a1b0 2023-07-21T19:30:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.742+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7141a1180 con 0x7fa71410a1b0 2023-07-21T19:30:36.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.742+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71410aa10 con 0x7fa71410a1b0 2023-07-21T19:30:36.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.751+0000 7fa70a7fc700 1 -- 172.21.15.5:0/1656157326 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa704010380 con 0x7fa71410a1b0 2023-07-21T19:30:36.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.760+0000 7fa70a7fc700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 0x7fa7000a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.760+0000 7fa70a7fc700 1 -- 172.21.15.5:0/1656157326 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa7040cc5b0 con 0x7fa71410a1b0 2023-07-21T19:30:36.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.761+0000 7fa72f31cb80 1 Processor -- start 2023-07-21T19:30:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.761+0000 7fa71979b700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 0x7fa7000a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 -- start start 2023-07-21T19:30:36.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805ca9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805ca4580 0x558805ca49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805ba9c50 con 0x558805ca4580 2023-07-21T19:30:36.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805bfc9e0 con 0x558805caa2d0 2023-07-21T19:30:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805bf5e70 con 0x558805b006b0 2023-07-21T19:30:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.762+0000 7fa718f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41378/0 (socket says 172.21.15.5:41378) 2023-07-21T19:30:36.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 -- 172.21.15.5:0/4125455028 learned_addr learned my addr 172.21.15.5:0/4125455028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 -- 172.21.15.5:0/4125455028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 msgr2=0x558805ca9d90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805ca9d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 -- 172.21.15.5:0/4125455028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805ca4580 msgr2=0x558805ca49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805ca4580 0x558805ca49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 -- 172.21.15.5:0/4125455028 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558805a23d80 con 0x558805caa2d0 2023-07-21T19:30:36.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.763+0000 7fa718f9a700 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa704095c30 tx=0x7fa704095ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f949e10d5d88c83f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.764+0000 7fa6ff7fe700 1 -- 172.21.15.5:0/4125455028 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704099730 con 0x558805caa2d0 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.764+0000 7fa6ff7fe700 1 -- 172.21.15.5:0/4125455028 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa70409d760 con 0x558805caa2d0 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.764+0000 7fa6ff7fe700 1 -- 172.21.15.5:0/4125455028 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704098c20 con 0x558805caa2d0 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 -- 172.21.15.5:0/4125455028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 msgr2=0x558805ca4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7fa704095c30 tx=0x7fa704095ba0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa71979b700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 0x7fa7000a3870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa71000f540 tx=0x7fa71000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.759 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 -- 172.21.15.5:0/4125455028 shutdown_connections 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805ca4580 0x558805ca49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805ca9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/4125455028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805caa2d0 0x558805ca4040 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 -- 172.21.15.5:0/4125455028 >> 172.21.15.5:0/4125455028 conn(0x558805c99be0 msgr2=0x558805c9ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 -- 172.21.15.5:0/4125455028 shutdown_connections 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.765+0000 7fa72f31cb80 1 -- 172.21.15.5:0/4125455028 wait complete. 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.766+0000 7fa72f31cb80 1 -- start start 2023-07-21T19:30:36.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.766+0000 7fa72f31cb80 1 Processor -- start 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805cb0650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cad860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805caa2d0 0x558805cadfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805b95a60 con 0x558805caa2d0 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805b95760 con 0x558805ca4580 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805a798c0 con 0x558805b006b0 2023-07-21T19:30:36.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.767+0000 7fa72f31cb80 10 client.0 _ll_register_callbacks cb 0x558805b93700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.768+0000 7fa71979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805cb0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.768+0000 7fa71979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805cb0650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54988/0 (socket says 172.21.15.5:54988) 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.768+0000 7fa71979b700 1 -- 172.21.15.5:0/270637029 learned_addr learned my addr 172.21.15.5:0/270637029 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.768+0000 7fa718f9a700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cad860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa719f9c700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805caa2d0 0x558805cadfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa718f9a700 1 -- 172.21.15.5:0/270637029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 msgr2=0x558805cb0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa718f9a700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805cb0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa718f9a700 1 -- 172.21.15.5:0/270637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805caa2d0 msgr2=0x558805cadfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa718f9a700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805caa2d0 0x558805cadfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.770+0000 7fa718f9a700 1 -- 172.21.15.5:0/270637029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa71410aa10 con 0x558805ca4580 2023-07-21T19:30:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.771+0000 7fa718f9a700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cad860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa704099200 tx=0x7fa704004790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c69879ef6e958d2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.771+0000 7fa6fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:30:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.771+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040b6070 con 0x558805ca4580 2023-07-21T19:30:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.771+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa704097ad0 con 0x558805ca4580 2023-07-21T19:30:36.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.772+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558805b4f490 con 0x558805ca4580 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.772+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa704098e30 con 0x558805ca4580 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.772+0000 7fa72f31cb80 10 client.29603 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.772+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x558805a0cc00 con 0x558805ca4580 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.773+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558805b4f490 con 0x558805ca4580 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.773+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa7040a17c0 con 0x558805ca4580 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.773+0000 7fa72f31cb80 10 client.29603 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.773+0000 7fa72f31cb80 10 client.29603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:36.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.773+0000 7fa72f31cb80 20 client.29603 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.774+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805cce850 0x558805cd0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.774+0000 7fa72f31cb80 4 client.29603 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.774+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558805beee50 con 0x558805cce850 2023-07-21T19:30:36.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.776+0000 7fa71979b700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805cce850 0x558805cd0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.777+0000 7fa71979b700 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805cce850 0x558805cd0c60 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=612833e9a83129e6 server_cookie=b82b294dd29fa4a6 in_seq=0 out_seq=0 2023-07-21T19:30:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.778+0000 7fa6fcff9700 10 client.29603 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.779+0000 7fa6fcff9700 1 -- 172.21.15.5:0/270637029 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558805beee50 con 0x558805cce850 2023-07-21T19:30:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.779+0000 7fa6fcff9700 10 client.29603 handle_command_reply: tid=0 2023-07-21T19:30:36.767 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.779+0000 7fa72f31cb80 1 client.29603 shutdown 2023-07-21T19:30:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 msgr2=0x558805cad860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cad860 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa704099200 tx=0x7fa704004790 comp rx=0 tx=0).stop 2023-07-21T19:30:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 shutdown_connections 2023-07-21T19:30:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805cce850 0x558805cd0c60 crc :-1 s=READY pgs=453 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.768 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805caa2d0 0x558805cadfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805b006b0 0x558805cb0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/270637029 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cad860 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 >> 172.21.15.5:0/270637029 conn(0x558805c99be0 msgr2=0x558805c9dc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.780+0000 7fa6fcff9700 0 client.29603 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.781+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 shutdown_connections 2023-07-21T19:30:36.769 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.781+0000 7fa72f31cb80 1 -- 172.21.15.5:0/270637029 wait complete. 2023-07-21T19:30:36.770 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.781+0000 7fa72f31cb80 20 client.29603 trim_cache size 0 max 16384 2023-07-21T19:30:36.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.810+0000 7fa72f31cb80 1 Processor -- start 2023-07-21T19:30:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.810+0000 7fa72f31cb80 1 -- start start 2023-07-21T19:30:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805cb4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cb0180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805ca9a10 con 0x558805b006b0 2023-07-21T19:30:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805c9bd00 con 0x558805ca4580 2023-07-21T19:30:36.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805caa6a0 con 0x558805c99be0 2023-07-21T19:30:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa71979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa71979b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54998/0 (socket says 172.21.15.5:54998) 2023-07-21T19:30:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.811+0000 7fa71979b700 1 -- 172.21.15.5:0/1540658330 learned_addr learned my addr 172.21.15.5:0/1540658330 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa718f9a700 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805cb4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa719f9c700 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cb0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa71979b700 1 -- 172.21.15.5:0/1540658330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 msgr2=0x558805cb0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa71979b700 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cb0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa71979b700 1 -- 172.21.15.5:0/1540658330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 msgr2=0x558805cb4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa71979b700 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805cb4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.812+0000 7fa71979b700 1 -- 172.21.15.5:0/1540658330 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558805cd5850 con 0x558805c99be0 2023-07-21T19:30:36.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.813+0000 7fa71979b700 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa710006280 tx=0x7fa710014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c06d87a82c17846f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.818+0000 7fa6fffff700 1 -- 172.21.15.5:0/1540658330 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71001c040 con 0x558805c99be0 2023-07-21T19:30:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.818+0000 7fa6fffff700 1 -- 172.21.15.5:0/1540658330 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa71000f040 con 0x558805c99be0 2023-07-21T19:30:36.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.818+0000 7fa6fffff700 1 -- 172.21.15.5:0/1540658330 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa71000e040 con 0x558805c99be0 2023-07-21T19:30:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.819+0000 7fa72f31cb80 1 -- 172.21.15.5:0/1540658330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 msgr2=0x558805ca4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.819+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fa710006280 tx=0x7fa710014000 comp rx=0 tx=0).stop 2023-07-21T19:30:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.826+0000 7fa72f31cb80 1 -- 172.21.15.5:0/1540658330 shutdown_connections 2023-07-21T19:30:36.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.826+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805ca4580 0x558805cb0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.826+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805c99be0 0x558805ca4040 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.826+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/1540658330 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805cb4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.826+0000 7fa72f31cb80 1 -- 172.21.15.5:0/1540658330 >> 172.21.15.5:0/1540658330 conn(0x558805ca4cf0 msgr2=0x558805ca7ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.828+0000 7fa72f31cb80 1 -- 172.21.15.5:0/1540658330 shutdown_connections 2023-07-21T19:30:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.828+0000 7fa72f31cb80 1 -- 172.21.15.5:0/1540658330 wait complete. 2023-07-21T19:30:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.829+0000 7fa72f31cb80 1 -- start start 2023-07-21T19:30:36.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.829+0000 7fa72f31cb80 1 Processor -- start 2023-07-21T19:30:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805c99be0 0x558805d16d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805ca4580 0x558805d216e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805d21c20 con 0x558805b006b0 2023-07-21T19:30:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805d21d60 con 0x558805c99be0 2023-07-21T19:30:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558805d21ef0 con 0x558805ca4580 2023-07-21T19:30:36.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.830+0000 7fa72f31cb80 10 client.0 _ll_register_callbacks cb 0x558805b93700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:36.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:36 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:30:36.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.831+0000 7fa71979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.831+0000 7fa71979b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:46596/0 (socket says 172.21.15.5:46596) 2023-07-21T19:30:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.831+0000 7fa71979b700 1 -- 172.21.15.5:0/3590258604 learned_addr learned my addr 172.21.15.5:0/3590258604 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.831+0000 7fa718f9a700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805c99be0 0x558805d16d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa719f9c700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805ca4580 0x558805d216e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa71979b700 1 -- 172.21.15.5:0/3590258604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805ca4580 msgr2=0x558805d216e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa71979b700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805ca4580 0x558805d216e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa71979b700 1 -- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805c99be0 msgr2=0x558805d16d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa71979b700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805c99be0 0x558805d16d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.832+0000 7fa71979b700 1 -- 172.21.15.5:0/3590258604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa710003a00 con 0x558805b006b0 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.833+0000 7fa71979b700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa71002c040 tx=0x7fa710004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f4b94dcb907873b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.833+0000 7fa6fcff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.834+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa710029660 con 0x558805b006b0 2023-07-21T19:30:36.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.834+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa71001c040 con 0x558805b006b0 2023-07-21T19:30:36.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.834+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa70409d760 con 0x558805b006b0 2023-07-21T19:30:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.835+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa710003f60 con 0x558805b006b0 2023-07-21T19:30:36.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.835+0000 7fa72f31cb80 10 client.19869 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.835+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x558805a4a1f0 con 0x558805b006b0 2023-07-21T19:30:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.835+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa70409d760 con 0x558805b006b0 2023-07-21T19:30:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.835+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fa710017710 con 0x558805b006b0 2023-07-21T19:30:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 10 client.19869 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:36.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 10 client.19869 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 20 client.19869 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805caa2d0 0x558805d256a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 4 client.19869 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558805bb0ff0 con 0x558805caa2d0 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.836+0000 7fa718f9a700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805caa2d0 0x558805d256a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:36.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.837+0000 7fa718f9a700 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805caa2d0 0x558805d256a0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c1bd90ea0202c4b server_cookie=f11fc3e0746e49dd in_seq=0 out_seq=0 2023-07-21T19:30:36.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.837+0000 7fa6fcff9700 10 client.19869 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.864+0000 7fa6fcff9700 1 -- 172.21.15.5:0/3590258604 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558805bb0ff0 con 0x558805caa2d0 2023-07-21T19:30:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.864+0000 7fa6fcff9700 10 client.19869 handle_command_reply: tid=0 2023-07-21T19:30:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 client.19869 shutdown 2023-07-21T19:30:36.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 msgr2=0x558805d16850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7fa71002c040 tx=0x7fa710004db0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 shutdown_connections 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558805ca4580 0x558805d216e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558805caa2d0 0x558805d256a0 crc :-1 s=READY pgs=454 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558805b006b0 0x558805d16850 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 --2- 172.21.15.5:0/3590258604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558805c99be0 0x558805d16d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 >> 172.21.15.5:0/3590258604 conn(0x558805ca4cf0 msgr2=0x558805cad100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa6fcff9700 0 client.19869 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 shutdown_connections 2023-07-21T19:30:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.865+0000 7fa72f31cb80 1 -- 172.21.15.5:0/3590258604 wait complete. 2023-07-21T19:30:36.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.866+0000 7fa72f31cb80 20 client.19869 trim_cache size 0 max 16384 2023-07-21T19:30:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:30:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:30:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:30:36.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:30:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.866+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 msgr2=0x7fa7000a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.866+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 0x7fa7000a3870 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa71000f540 tx=0x7fa71000b040 comp rx=0 tx=0).stop 2023-07-21T19:30:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.866+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 msgr2=0x7fa7141a6b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.866+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fa70400b550 tx=0x7fa70400b860 comp rx=0 tx=0).stop 2023-07-21T19:30:36.856 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 shutdown_connections 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa714114c80 0x7fa7141a0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa7000a13c0 0x7fa7000a3870 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa71406ce50 0x7fa7141a6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 --2- 172.21.15.5:0/1656157326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa71410a1b0 0x7fa7141a6b90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 >> 172.21.15.5:0/1656157326 conn(0x7fa71406bd70 msgr2=0x7fa714115380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 shutdown_connections 2023-07-21T19:30:36.857 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:36.867+0000 7fa71b9ff700 1 -- 172.21.15.5:0/1656157326 wait complete. 2023-07-21T19:30:36.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:36 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:30:36.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:36 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:30:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 245.0 s, 391.2 tps, lat 254.766 ms stddev 74.480 2023-07-21T19:30:37.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:37 smithi101 ceph-mon[128136]: pgmap v727: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-07-21T19:30:37.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:37 smithi005 ceph-mon[126460]: pgmap v727: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-07-21T19:30:37.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:37 smithi156 ceph-mon[128316]: pgmap v727: 97 pgs: 97 active+clean; 13 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 12 MiB/s wr, 1.06k op/s 2023-07-21T19:30:39.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:38 smithi101 ceph-mon[128136]: pgmap v728: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-21T19:30:39.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:38 smithi005 ceph-mon[126460]: pgmap v728: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-21T19:30:39.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:38 smithi156 ceph-mon[128316]: pgmap v728: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 17 MiB/s wr, 1.53k op/s 2023-07-21T19:30:41.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:41 smithi101 ceph-mon[128136]: pgmap v729: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-07-21T19:30:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:41 smithi005 ceph-mon[126460]: pgmap v729: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-07-21T19:30:41.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:41 smithi156 ceph-mon[128316]: pgmap v729: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 12 MiB/s wr, 1.04k op/s 2023-07-21T19:30:41.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 250.0 s, 439.6 tps, lat 229.597 ms stddev 65.078 2023-07-21T19:30:43.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:43 smithi101 ceph-mon[128136]: pgmap v730: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:30:43.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:43 smithi005 ceph-mon[126460]: pgmap v730: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:30:43.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:43 smithi156 ceph-mon[128316]: pgmap v730: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:30:45.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:45 smithi101 ceph-mon[128136]: pgmap v731: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:45.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:45 smithi005 ceph-mon[126460]: pgmap v731: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:45.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:45 smithi156 ceph-mon[128316]: pgmap v731: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 255.0 s, 403.6 tps, lat 246.166 ms stddev 87.053 2023-07-21T19:30:47.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:47 smithi101 ceph-mon[128136]: pgmap v732: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 876 op/s 2023-07-21T19:30:47.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:30:47.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:30:47.819 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:30:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:47 smithi005 ceph-mon[126460]: pgmap v732: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 876 op/s 2023-07-21T19:30:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:30:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:30:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:30:47.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:47 smithi156 ceph-mon[128316]: pgmap v732: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 876 op/s 2023-07-21T19:30:47.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:30:47.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:30:47.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:30:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:48 smithi101 ceph-mon[128136]: pgmap v733: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-07-21T19:30:49.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:48 smithi005 ceph-mon[126460]: pgmap v733: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-07-21T19:30:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:48 smithi156 ceph-mon[128316]: pgmap v733: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 16 MiB/s wr, 1.32k op/s 2023-07-21T19:30:50.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 -- 172.21.15.5:0/1298987388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4101f70 msgr2=0x7f00b4102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/1298987388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4101f70 0x7f00b4102340 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f00a40099c0 tx=0x7f00a4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 -- 172.21.15.5:0/1298987388 shutdown_connections 2023-07-21T19:30:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/1298987388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4106fb0 0x7f00b410eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/1298987388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4101f70 0x7f00b4102340 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/1298987388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4102910 0x7f00b41068d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 -- 172.21.15.5:0/1298987388 >> 172.21.15.5:0/1298987388 conn(0x7f00b40fd7e0 msgr2=0x7f00b40ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 -- 172.21.15.5:0/1298987388 shutdown_connections 2023-07-21T19:30:50.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.124+0000 7f00ba7ff700 1 -- 172.21.15.5:0/1298987388 wait complete. 2023-07-21T19:30:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.126+0000 7f00ba7ff700 1 Processor -- start 2023-07-21T19:30:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.126+0000 7f00ba7ff700 1 -- start start 2023-07-21T19:30:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4101f70 0x7f00b4197e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 0x7f00b4198340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b419cff0 con 0x7f00b4106fb0 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b419d160 con 0x7f00b4101f70 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00ba7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b419d320 con 0x7f00b4102910 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00b8d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00b8d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58772/0 (socket says 172.21.15.5:58772) 2023-07-21T19:30:50.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00b8d9c700 1 -- 172.21.15.5:0/3371997449 learned_addr learned my addr 172.21.15.5:0/3371997449 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.127+0000 7f00b3fff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4101f70 0x7f00b4197e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 0x7f00b4198340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 -- 172.21.15.5:0/3371997449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 msgr2=0x7f00b4198340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 0x7f00b4198340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 -- 172.21.15.5:0/3371997449 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4101f70 msgr2=0x7f00b4197e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4101f70 0x7f00b4197e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 -- 172.21.15.5:0/3371997449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a40095f0 con 0x7f00b4106fb0 2023-07-21T19:30:50.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 0x7f00b4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:30:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.128+0000 7f00b8d9c700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f00ac007ee0 tx=0x7f00ac00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.129+0000 7f00b17fa700 1 -- 172.21.15.5:0/3371997449 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac017070 con 0x7f00b4106fb0 2023-07-21T19:30:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.129+0000 7f00b17fa700 1 -- 172.21.15.5:0/3371997449 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00ac00f040 con 0x7f00b4106fb0 2023-07-21T19:30:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.129+0000 7f00b17fa700 1 -- 172.21.15.5:0/3371997449 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac005710 con 0x7f00b4106fb0 2023-07-21T19:30:50.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.129+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b419d600 con 0x7f00b4106fb0 2023-07-21T19:30:50.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.129+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b4074b30 con 0x7f00b4106fb0 2023-07-21T19:30:50.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.135+0000 7f00b17fa700 1 -- 172.21.15.5:0/3371997449 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f00ac005870 con 0x7f00b4106fb0 2023-07-21T19:30:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.136+0000 7f00ce1cdb80 1 Processor -- start 2023-07-21T19:30:50.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.136+0000 7f00ce1cdb80 1 -- start start 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bd1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bcc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2ad1c50 con 0x5571b2bd22d0 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2b249e0 con 0x5571b2a286b0 2023-07-21T19:30:50.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.137+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2b1de70 con 0x5571b2bcc580 2023-07-21T19:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42402/0 (socket says 172.21.15.5:42402) 2023-07-21T19:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 -- 172.21.15.5:0/3268461051 learned_addr learned my addr 172.21.15.5:0/3268461051 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:50.126 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bd1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 -- 172.21.15.5:0/3268461051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 msgr2=0x5571b2bd1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bd1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 -- 172.21.15.5:0/3268461051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 msgr2=0x5571b2bcc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bcc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.127 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.138+0000 7f00b37fe700 1 -- 172.21.15.5:0/3268461051 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571b294bd80 con 0x5571b2bcc580 2023-07-21T19:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.139+0000 7f00b17fa700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 0x7f009c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.128 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.139+0000 7f00b17fa700 1 -- 172.21.15.5:0/3371997449 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00ac0cc4d0 con 0x7f00b4106fb0 2023-07-21T19:30:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.140+0000 7f00b3fff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 0x7f009c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.142+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a800c230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c520a318aef9bb1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.142+0000 7f009a7fc700 1 -- 172.21.15.5:0/3268461051 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a800c030 con 0x5571b2bcc580 2023-07-21T19:30:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.143+0000 7f009a7fc700 1 -- 172.21.15.5:0/3268461051 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00a800c9e0 con 0x5571b2bcc580 2023-07-21T19:30:50.131 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.143+0000 7f009a7fc700 1 -- 172.21.15.5:0/3268461051 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8003df0 con 0x5571b2bcc580 2023-07-21T19:30:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.144+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3268461051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 msgr2=0x5571b2bcc9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.132 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.144+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f00a8006fd0 tx=0x7f00a800c230 comp rx=0 tx=0).stop 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00b3fff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 0x7f009c0a38b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009990 tx=0x7f00a4005a90 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3268461051 shutdown_connections 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bcc580 0x5571b2bcc9d0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bd1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3268461051 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bcc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.145+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3268461051 >> 172.21.15.5:0/3268461051 conn(0x5571b2bc1be0 msgr2=0x5571b2bc6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.148+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3268461051 shutdown_connections 2023-07-21T19:30:50.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.149+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3268461051 wait complete. 2023-07-21T19:30:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.150+0000 7f00ce1cdb80 1 -- start start 2023-07-21T19:30:50.139 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.150+0000 7f00ce1cdb80 1 Processor -- start 2023-07-21T19:30:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2a286b0 0x5571b2bd59f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.143 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bd6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2abda60 con 0x5571b2bd22d0 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2abd760 con 0x5571b2bcc580 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b29a18c0 con 0x5571b2a286b0 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.151+0000 7f00ce1cdb80 10 client.0 _ll_register_callbacks cb 0x5571b2abb700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.152+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.152+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48828/0 (socket says 172.21.15.5:48828) 2023-07-21T19:30:50.144 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.152+0000 7f00b37fe700 1 -- 172.21.15.5:0/1321039860 learned_addr learned my addr 172.21.15.5:0/1321039860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.152+0000 7f00b3fff700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2a286b0 0x5571b2bd59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.153+0000 7f00b37fe700 1 -- 172.21.15.5:0/1321039860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2a286b0 msgr2=0x5571b2bd59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.153+0000 7f00b37fe700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2a286b0 0x5571b2bd59f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.153+0000 7f00b37fe700 1 -- 172.21.15.5:0/1321039860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 msgr2=0x5571b2bd6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.153+0000 7f00b37fe700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bd6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.153+0000 7f00b37fe700 1 -- 172.21.15.5:0/1321039860 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a8015320 con 0x5571b2bcc580 2023-07-21T19:30:50.145 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.155+0000 7f00b37fe700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c160 tx=0x7f00a8006c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ca31e2bff9904cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.155+0000 7f00937fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:30:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.156+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8004220 con 0x5571b2bcc580 2023-07-21T19:30:50.146 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.157+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00a8004380 con 0x5571b2bcc580 2023-07-21T19:30:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.157+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5571b2a77490 con 0x5571b2bcc580 2023-07-21T19:30:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.158+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a8004580 con 0x5571b2bcc580 2023-07-21T19:30:50.147 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.159+0000 7f00ce1cdb80 10 client.29615 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.159+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x5571b2934c00 con 0x5571b2bcc580 2023-07-21T19:30:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.159+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00a8051030 con 0x5571b2bcc580 2023-07-21T19:30:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f00a8048020 con 0x5571b2bcc580 2023-07-21T19:30:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 10 client.29615 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:50.148 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 10 client.29615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 20 client.29615 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2be4380 0x5571b2be6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 4 client.29615 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.160+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5571b2b16e50 con 0x5571b2be4380 2023-07-21T19:30:50.149 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.161+0000 7f00b3fff700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2be4380 0x5571b2be6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.162+0000 7f00b3fff700 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2be4380 0x5571b2be6790 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34223a5c95c2ed03 server_cookie=7ea76cb4c907d57b in_seq=0 out_seq=0 2023-07-21T19:30:50.150 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.162+0000 7f00937fe700 10 client.29615 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.164+0000 7f00937fe700 1 -- 172.21.15.5:0/1321039860 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5571b2b16e50 con 0x5571b2be4380 2023-07-21T19:30:50.152 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.164+0000 7f00937fe700 10 client.29615 handle_command_reply: tid=0 2023-07-21T19:30:50.153 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.164+0000 7f00ce1cdb80 1 client.29615 shutdown 2023-07-21T19:30:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.165+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 msgr2=0x5571b2bd5f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.165+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f00a800c160 tx=0x7f00a8006c70 comp rx=0 tx=0).stop 2023-07-21T19:30:50.154 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 shutdown_connections 2023-07-21T19:30:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2be4380 0x5571b2be6790 crc :-1 s=READY pgs=455 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bd22d0 0x5571b2bd6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.155 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2a286b0 0x5571b2bd59f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/1321039860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2bcc580 0x5571b2bd5f30 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 >> 172.21.15.5:0/1321039860 conn(0x5571b2bc1be0 msgr2=0x5571b2bc64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.156 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.166+0000 7f00937fe700 0 client.29615 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:50.157 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.167+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 shutdown_connections 2023-07-21T19:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.167+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/1321039860 wait complete. 2023-07-21T19:30:50.158 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.169+0000 7f00ce1cdb80 20 client.29615 trim_cache size 0 max 16384 2023-07-21T19:30:50.184 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.196+0000 7f00ce1cdb80 1 Processor -- start 2023-07-21T19:30:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 -- start start 2023-07-21T19:30:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bcc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.185 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bf1fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2b16e50 con 0x5571b2bcc580 2023-07-21T19:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2bbec00 con 0x5571b2a286b0 2023-07-21T19:30:50.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.197+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2bc3d00 con 0x5571b2bd22d0 2023-07-21T19:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.198+0000 7f00b8d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bf1fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bcc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58786/0 (socket says 172.21.15.5:58786) 2023-07-21T19:30:50.187 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 -- 172.21.15.5:0/3860283765 learned_addr learned my addr 172.21.15.5:0/3860283765 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 -- 172.21.15.5:0/3860283765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 msgr2=0x5571b2bf1fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.188 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bf1fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 -- 172.21.15.5:0/3860283765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 msgr2=0x5571b2bcc040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bcc040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.199+0000 7f00b37fe700 1 -- 172.21.15.5:0/3860283765 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5571b29721f0 con 0x5571b2bcc580 2023-07-21T19:30:50.189 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.200+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f00a8015610 tx=0x7f00a800cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591cb276f19fdd67 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.201+0000 7f009affd700 1 -- 172.21.15.5:0/3860283765 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a80174e0 con 0x5571b2bcc580 2023-07-21T19:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.201+0000 7f009affd700 1 -- 172.21.15.5:0/3860283765 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00a8051040 con 0x5571b2bcc580 2023-07-21T19:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.201+0000 7f009affd700 1 -- 172.21.15.5:0/3860283765 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a8017a50 con 0x5571b2bcc580 2023-07-21T19:30:50.190 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.202+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3860283765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 msgr2=0x5571b2bdcbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.202+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f00a8015610 tx=0x7f00a800cf70 comp rx=0 tx=0).stop 2023-07-21T19:30:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.203+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3860283765 shutdown_connections 2023-07-21T19:30:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.203+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bf1fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.203+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2bcc040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.191 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.203+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3860283765 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2bdcbd0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.203+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3860283765 >> 172.21.15.5:0/3860283765 conn(0x5571b2bc1be0 msgr2=0x5571b2bcfc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.192 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.204+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3860283765 shutdown_connections 2023-07-21T19:30:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.204+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3860283765 wait complete. 2023-07-21T19:30:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.204+0000 7f00ce1cdb80 1 -- start start 2023-07-21T19:30:50.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 Processor -- start 2023-07-21T19:30:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2c337b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bc5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2bdcf30 con 0x5571b2bcc580 2023-07-21T19:30:50.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2bdd0a0 con 0x5571b2a286b0 2023-07-21T19:30:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5571b2bc63d0 con 0x5571b2bd22d0 2023-07-21T19:30:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.205+0000 7f00ce1cdb80 10 client.0 _ll_register_callbacks cb 0x7f00a800cc90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.206+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.206+0000 7f00b8d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bc5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.206+0000 7f00b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58798/0 (socket says 172.21.15.5:58798) 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.206+0000 7f00b37fe700 1 -- 172.21.15.5:0/3662366693 learned_addr learned my addr 172.21.15.5:0/3662366693 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b37fe700 1 -- 172.21.15.5:0/3662366693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 msgr2=0x5571b2bc5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bc5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b37fe700 1 -- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 msgr2=0x5571b2c337b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2c337b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b37fe700 1 -- 172.21.15.5:0/3662366693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5571b2934c00 con 0x5571b2bcc580 2023-07-21T19:30:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.207+0000 7f00b8d9c700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bc5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:30:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.208+0000 7f00b37fe700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f00a8048320 tx=0x7f00a8004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbf853ab089c123 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:50.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.208+0000 7f00937fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a80043e0 con 0x5571b2bcc580 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f00a80174e0 con 0x5571b2bcc580 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f00a800c9e0 con 0x5571b2bcc580 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f00a8017930 con 0x5571b2bcc580 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00ce1cdb80 10 client.19881 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:50.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.209+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x5571b294a250 con 0x5571b2bcc580 2023-07-21T19:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.210+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f00a800c9e0 con 0x5571b2bcc580 2023-07-21T19:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.210+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f00a8040040 con 0x5571b2bcc580 2023-07-21T19:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 10 client.19881 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 10 client.19881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:50.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 20 client.19881 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2bdc2b0 0x5571b2bdc6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 4 client.19881 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.211+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5571b2ad8ff0 con 0x5571b2bdc2b0 2023-07-21T19:30:50.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.213+0000 7f00b3fff700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2bdc2b0 0x5571b2bdc6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.213+0000 7f00b3fff700 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2bdc2b0 0x5571b2bdc6b0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8255a3f97fac63db server_cookie=7fd641771770de6b in_seq=0 out_seq=0 2023-07-21T19:30:50.201 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.214+0000 7f00937fe700 10 client.19881 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:50.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.251+0000 7f00937fe700 1 -- 172.21.15.5:0/3662366693 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5571b2ad8ff0 con 0x5571b2bdc2b0 2023-07-21T19:30:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.251+0000 7f00937fe700 10 client.19881 handle_command_reply: tid=0 2023-07-21T19:30:50.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.251+0000 7f00ce1cdb80 1 client.19881 shutdown 2023-07-21T19:30:50.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.252+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 msgr2=0x5571b2c33cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.252+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f00a8048320 tx=0x7f00a8004800 comp rx=0 tx=0).stop 2023-07-21T19:30:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 shutdown_connections 2023-07-21T19:30:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5571b2bd22d0 0x5571b2bc5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5571b2a286b0 0x5571b2c337b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5571b2bdc2b0 0x5571b2bdc6b0 crc :-1 s=READY pgs=456 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 --2- 172.21.15.5:0/3662366693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5571b2bcc580 0x5571b2c33cf0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 >> 172.21.15.5:0/3662366693 conn(0x5571b2bc1be0 msgr2=0x5571b2bf1fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00937fe700 0 client.19881 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 shutdown_connections 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.253+0000 7f00ce1cdb80 1 -- 172.21.15.5:0/3662366693 wait complete. 2023-07-21T19:30:50.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.254+0000 7f00ce1cdb80 20 client.19881 trim_cache size 0 max 16384 2023-07-21T19:30:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:30:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:30:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:30:50.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:30:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.254+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 msgr2=0x7f009c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.254+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 0x7f009c0a38b0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f00a4009990 tx=0x7f00a4005a90 comp rx=0 tx=0).stop 2023-07-21T19:30:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.254+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 msgr2=0x7f00b419c9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:50.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.254+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7f00ac007ee0 tx=0x7f00ac00d3b0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 shutdown_connections 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f009c0a1400 0x7f009c0a38b0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f00b4106fb0 0x7f00b419c9b0 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f00b4101f70 0x7f00b4197e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 --2- 172.21.15.5:0/3371997449 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f00b4102910 0x7f00b4198340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 >> 172.21.15.5:0/3371997449 conn(0x7f00b40fd7e0 msgr2=0x7f00b41058b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:50.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 shutdown_connections 2023-07-21T19:30:50.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:30:50.255+0000 7f00ba7ff700 1 -- 172.21.15.5:0/3371997449 wait complete. 2023-07-21T19:30:51.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:51 smithi101 ceph-mon[128136]: pgmap v734: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 848 op/s 2023-07-21T19:30:51.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:51 smithi005 ceph-mon[126460]: pgmap v734: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 848 op/s 2023-07-21T19:30:51.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:51 smithi156 ceph-mon[128316]: pgmap v734: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 10 MiB/s wr, 848 op/s 2023-07-21T19:30:51.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 260.0 s, 414.8 tps, lat 240.727 ms stddev 43.793 2023-07-21T19:30:53.423 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:30:53.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.777+0000 7f12145ff700 1 -- 172.21.15.5:0/77036844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 msgr2=0x7f120c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 --2- 172.21.15.5:0/77036844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c0fff70 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009990 tx=0x7f11f8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 -- 172.21.15.5:0/77036844 shutdown_connections 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 --2- 172.21.15.5:0/77036844 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 --2- 172.21.15.5:0/77036844 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 0x7f120c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 --2- 172.21.15.5:0/77036844 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c0fff70 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.778+0000 7f12145ff700 1 -- 172.21.15.5:0/77036844 >> 172.21.15.5:0/77036844 conn(0x7f120c0fb600 msgr2=0x7f120c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.779+0000 7f12145ff700 1 -- 172.21.15.5:0/77036844 shutdown_connections 2023-07-21T19:30:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.779+0000 7f12145ff700 1 -- 172.21.15.5:0/77036844 wait complete. 2023-07-21T19:30:53.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.780+0000 7f12145ff700 1 Processor -- start 2023-07-21T19:30:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 -- start start 2023-07-21T19:30:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.770 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 0x7f120c10dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c070890 con 0x7f120c1004b0 2023-07-21T19:30:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c070a00 con 0x7f120c105b10 2023-07-21T19:30:53.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.782+0000 7f12145ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f120c070bc0 con 0x7f120c0ffb00 2023-07-21T19:30:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.783+0000 7f121239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.783+0000 7f121239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42450/0 (socket says 172.21.15.5:42450) 2023-07-21T19:30:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.783+0000 7f121239b700 1 -- 172.21.15.5:0/1188593334 learned_addr learned my addr 172.21.15.5:0/1188593334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.783+0000 7f1211b9a700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.783+0000 7f1212b9c700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 0x7f120c10dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 -- 172.21.15.5:0/1188593334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 msgr2=0x7f120c10dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 0x7f120c10dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 -- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 msgr2=0x7f120c10d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 -- 172.21.15.5:0/1188593334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80095f0 con 0x7f120c0ffb00 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f1211b9a700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f121239b700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f120000ef10 tx=0x7f120000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f120b7fe700 1 -- 172.21.15.5:0/1188593334 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120000c9c0 con 0x7f120c0ffb00 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f120b7fe700 1 -- 172.21.15.5:0/1188593334 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f120000f040 con 0x7f120c0ffb00 2023-07-21T19:30:53.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f120b7fe700 1 -- 172.21.15.5:0/1188593334 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200014480 con 0x7f120c0ffb00 2023-07-21T19:30:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.784+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f120c070e40 con 0x7f120c0ffb00 2023-07-21T19:30:53.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.785+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0712d0 con 0x7f120c0ffb00 2023-07-21T19:30:53.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.789+0000 7f120b7fe700 1 -- 172.21.15.5:0/1188593334 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1200007690 con 0x7f120c0ffb00 2023-07-21T19:30:53.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.791+0000 7f1227fd1b80 1 Processor -- start 2023-07-21T19:30:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.791+0000 7f120b7fe700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 0x7f11fc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.791+0000 7f120b7fe700 1 -- 172.21.15.5:0/1188593334 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f12000ccf10 con 0x7f120c0ffb00 2023-07-21T19:30:53.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.791+0000 7f1227fd1b80 1 -- start start 2023-07-21T19:30:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1212b9c700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 0x7f11fc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 0x55b4917d1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4916cccb0 con 0x55b4917c6be0 2023-07-21T19:30:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b49171fa40 con 0x55b4917c62d0 2023-07-21T19:30:53.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.792+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b491718ed0 con 0x55b4917d1ad0 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f1211b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 0x55b4917d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48878/0 (socket says 172.21.15.5:48878) 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 -- 172.21.15.5:0/724162155 learned_addr learned my addr 172.21.15.5:0/724162155 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f1211b9a700 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 0x55b4917d1590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58818/0 (socket says 172.21.15.5:58818) 2023-07-21T19:30:53.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f1212b9c700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 0x7f11fc0a3830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f120400abe0 tx=0x7f120400a2d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f1212b9c700 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 -- 172.21.15.5:0/724162155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 msgr2=0x55b4917d3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917d3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 -- 172.21.15.5:0/724162155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 msgr2=0x55b4917d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 0x55b4917d1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.793+0000 7f121239b700 1 -- 172.21.15.5:0/724162155 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b491546de0 con 0x55b4917c62d0 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.794+0000 7f121239b700 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1200099f10 tx=0x7f1200099e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6281e674f27be876 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.794+0000 7f1208bfe700 1 -- 172.21.15.5:0/724162155 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1200099f80 con 0x55b4917c62d0 2023-07-21T19:30:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.794+0000 7f1208bfe700 1 -- 172.21.15.5:0/724162155 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f120009dfb0 con 0x55b4917c62d0 2023-07-21T19:30:53.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.794+0000 7f1208bfe700 1 -- 172.21.15.5:0/724162155 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12000996d0 con 0x55b4917c62d0 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.795+0000 7f1227fd1b80 1 -- 172.21.15.5:0/724162155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 msgr2=0x55b4917c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.795+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1200099f10 tx=0x7f1200099e20 comp rx=0 tx=0).stop 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 -- 172.21.15.5:0/724162155 shutdown_connections 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917d3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917c66a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/724162155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c6be0 0x55b4917d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 -- 172.21.15.5:0/724162155 >> 172.21.15.5:0/724162155 conn(0x55b4917bcda0 msgr2=0x55b4917c26a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 -- 172.21.15.5:0/724162155 shutdown_connections 2023-07-21T19:30:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.796+0000 7f1227fd1b80 1 -- 172.21.15.5:0/724162155 wait complete. 2023-07-21T19:30:53.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.797+0000 7f1227fd1b80 1 -- start start 2023-07-21T19:30:53.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 Processor -- start 2023-07-21T19:30:53.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917ce8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b4917cee00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4916b8ac0 con 0x55b4917c62d0 2023-07-21T19:30:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4916b87c0 con 0x55b4917d1ad0 2023-07-21T19:30:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.798+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b49159c920 con 0x55b4917c6be0 2023-07-21T19:30:53.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.799+0000 7f1227fd1b80 10 client.0 _ll_register_callbacks cb 0x55b4916b6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.799+0000 7f1212b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48884/0 (socket says 172.21.15.5:48884) 2023-07-21T19:30:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 -- 172.21.15.5:0/3071160858 learned_addr learned my addr 172.21.15.5:0/3071160858 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1211b9a700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b4917cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 -- 172.21.15.5:0/3071160858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 msgr2=0x55b4917cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b4917cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 -- 172.21.15.5:0/3071160858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 msgr2=0x55b4917ce8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917ce8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.800+0000 7f1212b9c700 1 -- 172.21.15.5:0/3071160858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f120c0712d0 con 0x55b4917d1ad0 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.801+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1204010a90 tx=0x7f12040047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b8234635c722b7ae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.801+0000 7f11f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:30:53.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.802+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204021070 con 0x55b4917d1ad0 2023-07-21T19:30:53.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.803+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f120401c630 con 0x55b4917d1ad0 2023-07-21T19:30:53.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.803+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b4916724f0 con 0x55b4917d1ad0 2023-07-21T19:30:53.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.803+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f120401c830 con 0x55b4917d1ad0 2023-07-21T19:30:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.803+0000 7f1227fd1b80 10 client.29630 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.803+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55b49152fc60 con 0x55b4917d1ad0 2023-07-21T19:30:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b4916724f0 con 0x55b4917d1ad0 2023-07-21T19:30:53.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f12040259f0 con 0x55b4917d1ad0 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 10 client.29630 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 10 client.29630 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 20 client.29630 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4917e1530 0x55b4917e3940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 4 client.29630 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:30:53.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.804+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b491711eb0 con 0x55b4917e1530 2023-07-21T19:30:53.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.805+0000 7f121239b700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4917e1530 0x55b4917e3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.807+0000 7f121239b700 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4917e1530 0x55b4917e3940 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70741672690cd031 server_cookie=7f699a74de924fc2 in_seq=0 out_seq=0 2023-07-21T19:30:53.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.808+0000 7f11f5ffb700 10 client.29630 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:53.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f11f5ffb700 1 -- 172.21.15.5:0/3071160858 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b491711eb0 con 0x55b4917e1530 2023-07-21T19:30:53.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f11f5ffb700 10 client.29630 handle_command_reply: tid=0 2023-07-21T19:30:53.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 client.29630 shutdown 2023-07-21T19:30:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 msgr2=0x55b4917cf340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f1204010a90 tx=0x7f12040047c0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 shutdown_connections 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917d1ad0 0x55b4917cf340 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4917e1530 0x55b4917e3940 crc :-1 s=READY pgs=457 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.810+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917ce8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.811+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3071160858 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b4917cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.811+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 >> 172.21.15.5:0/3071160858 conn(0x55b4917bcda0 msgr2=0x55b4917c2230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.811+0000 7f11f5ffb700 0 client.29630 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:53.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.811+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 shutdown_connections 2023-07-21T19:30:53.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.812+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3071160858 wait complete. 2023-07-21T19:30:53.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.813+0000 7f1227fd1b80 20 client.29630 trim_cache size 0 max 16384 2023-07-21T19:30:53.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:53 smithi101 ceph-mon[128136]: pgmap v735: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-07-21T19:30:53.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.848+0000 7f1227fd1b80 1 Processor -- start 2023-07-21T19:30:53.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.848+0000 7f1227fd1b80 1 -- start start 2023-07-21T19:30:53.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c6be0 0x55b4917cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4917cfc10 con 0x55b4917c62d0 2023-07-21T19:30:53.838 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4917bc970 con 0x55b4917c6be0 2023-07-21T19:30:53.839 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4917c7d60 con 0x55b4917d1ad0 2023-07-21T19:30:53.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.849+0000 7f1212b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42486/0 (socket says 172.21.15.5:42486) 2023-07-21T19:30:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 -- 172.21.15.5:0/3290330505 learned_addr learned my addr 172.21.15.5:0/3290330505 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1211b9a700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c6be0 0x55b4917cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 -- 172.21.15.5:0/3290330505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c6be0 msgr2=0x55b4917cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c6be0 0x55b4917cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f121239b700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 -- 172.21.15.5:0/3290330505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 msgr2=0x55b4917d84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917d84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.850+0000 7f1212b9c700 1 -- 172.21.15.5:0/3290330505 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b49156d250 con 0x55b4917d1ad0 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.851+0000 7f121239b700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.851+0000 7f1212b9c700 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f120400a9c0 tx=0x7f120400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb5ae82fad1b81d7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.851+0000 7f12093ff700 1 -- 172.21.15.5:0/3290330505 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1204021030 con 0x55b4917d1ad0 2023-07-21T19:30:53.842 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.852+0000 7f12093ff700 1 -- 172.21.15.5:0/3290330505 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1204014070 con 0x55b4917d1ad0 2023-07-21T19:30:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.852+0000 7f12093ff700 1 -- 172.21.15.5:0/3290330505 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f120401b980 con 0x55b4917d1ad0 2023-07-21T19:30:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3290330505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 msgr2=0x55b4917cbbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.843 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f120400a9c0 tx=0x7f120400a340 comp rx=0 tx=0).stop 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3290330505 shutdown_connections 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917d1ad0 0x55b4917cbbc0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917c62d0 0x55b4917d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/3290330505 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c6be0 0x55b4917cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.853+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3290330505 >> 172.21.15.5:0/3290330505 conn(0x55b4917bcda0 msgr2=0x55b4917c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.854+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3290330505 shutdown_connections 2023-07-21T19:30:53.844 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.854+0000 7f1227fd1b80 1 -- 172.21.15.5:0/3290330505 wait complete. 2023-07-21T19:30:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.855+0000 7f1227fd1b80 1 -- start start 2023-07-21T19:30:53.845 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.855+0000 7f1227fd1b80 1 Processor -- start 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917d86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917d1ad0 0x55b49182dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b49182e370 con 0x55b4917d1ad0 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b49182e4e0 con 0x55b4917c62d0 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4918463f0 con 0x55b4917c6be0 2023-07-21T19:30:53.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.856+0000 7f1227fd1b80 10 client.0 _ll_register_callbacks cb 0x55b4916b6760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1212b9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917d1ad0 0x55b49182dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42492/0 (socket says 172.21.15.5:42492) 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 -- 172.21.15.5:0/100959325 learned_addr learned my addr 172.21.15.5:0/100959325 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f121239b700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917d86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 -- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 msgr2=0x55b4917d86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917d86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 -- 172.21.15.5:0/100959325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917d1ad0 msgr2=0x55b49182dc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:30:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917d1ad0 0x55b49182dc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.857+0000 7f1211b9a700 1 -- 172.21.15.5:0/100959325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b49152fc60 con 0x55b4917c6be0 2023-07-21T19:30:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.858+0000 7f1211b9a700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009990 tx=0x7f11f8005410 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b918d43335e2702 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:30:53.848 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.858+0000 7f11f5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:30:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.859+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f801d040 con 0x55b4917c6be0 2023-07-21T19:30:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.859+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f11f8004ad0 con 0x55b4917c6be0 2023-07-21T19:30:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.859+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f120009dfb0 con 0x55b4917c6be0 2023-07-21T19:30:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.860+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11f8004c90 con 0x55b4917c6be0 2023-07-21T19:30:53.849 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.860+0000 7f1227fd1b80 10 client.29560 fetch_fsmap learned FSMap version 62 2023-07-21T19:30:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.860+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x55b4915452b0 con 0x55b4917c6be0 2023-07-21T19:30:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.861+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f120009dfb0 con 0x55b4917c6be0 2023-07-21T19:30:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.861+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f11f8023050 con 0x55b4917c6be0 2023-07-21T19:30:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 10 client.29560 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:30:53.850 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 10 client.29560 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:30:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 20 client.29560 populate_metadata read hostname 'smithi005' 2023-07-21T19:30:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4918479b0 0x55b491849dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:30:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 4 client.29560 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:30:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.862+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55b4916d4050 con 0x55b4918479b0 2023-07-21T19:30:53.851 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.863+0000 7f121239b700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4918479b0 0x55b491849dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:30:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.864+0000 7f121239b700 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4918479b0 0x55b491849dc0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=541913d5a4675913 server_cookie=ffec22dfd01fd5b in_seq=0 out_seq=0 2023-07-21T19:30:53.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.864+0000 7f11f5ffb700 10 client.29560 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f11f5ffb700 1 -- 172.21.15.5:0/100959325 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55b4916d4050 con 0x55b4918479b0 2023-07-21T19:30:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f11f5ffb700 10 client.29560 handle_command_reply: tid=0 2023-07-21T19:30:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 client.29560 shutdown 2023-07-21T19:30:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 msgr2=0x55b49182d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.853 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f11f8009990 tx=0x7f11f8005410 comp rx=0 tx=0).stop 2023-07-21T19:30:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 shutdown_connections 2023-07-21T19:30:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b4918479b0 0x55b491849dc0 crc :-1 s=READY pgs=458 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.854 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4917d1ad0 0x55b49182dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f11f5ffb700 0 client.29560 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4917c62d0 0x55b4917d86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 --2- 172.21.15.5:0/100959325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4917c6be0 0x55b49182d720 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.865+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 >> 172.21.15.5:0/100959325 conn(0x55b4917bcda0 msgr2=0x55b4917ceea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.866+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 shutdown_connections 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.866+0000 7f1227fd1b80 1 -- 172.21.15.5:0/100959325 wait complete. 2023-07-21T19:30:53.855 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.866+0000 7f1227fd1b80 20 client.29560 trim_cache size 0 max 16384 2023-07-21T19:30:53.856 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:30:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:30:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:30:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:30:53.856 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:30:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:30:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:30:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:30:53.857 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:30:53.857 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:30:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.868+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 msgr2=0x7f11fc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.868+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 0x7f11fc0a3830 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f120400abe0 tx=0x7f120400a2d0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.868+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 msgr2=0x7f120c10d1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:30:53.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.868+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f120000ef10 tx=0x7f120000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 shutdown_connections 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f120c105b10 0x7f120c10dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f120c0ffb00 0x7f120c10d1d0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11fc0a1380 0x7f11fc0a3830 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 --2- 172.21.15.5:0/1188593334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f120c1004b0 0x7f120c10d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:30:53.859 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 >> 172.21.15.5:0/1188593334 conn(0x7f120c0fb600 msgr2=0x7f120c0fbb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:30:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 shutdown_connections 2023-07-21T19:30:53.860 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:30:53.869+0000 7f12145ff700 1 -- 172.21.15.5:0/1188593334 wait complete. 2023-07-21T19:30:53.889 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:30:53.889 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:53 smithi005 ceph-mon[126460]: pgmap v735: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-07-21T19:30:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:53 smithi156 ceph-mon[128316]: pgmap v735: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 15 MiB/s wr, 1.29k op/s 2023-07-21T19:30:55.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:55 smithi101 ceph-mon[128136]: pgmap v736: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 954 op/s 2023-07-21T19:30:55.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:55 smithi005 ceph-mon[126460]: pgmap v736: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 954 op/s 2023-07-21T19:30:55.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:55 smithi156 ceph-mon[128316]: pgmap v736: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 11 MiB/s wr, 954 op/s 2023-07-21T19:30:56.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 265.0 s, 405.4 tps, lat 245.360 ms stddev 46.031 2023-07-21T19:30:57.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:57 smithi101 ceph-mon[128136]: pgmap v737: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:57.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:57 smithi005 ceph-mon[126460]: pgmap v737: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:57.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:57 smithi156 ceph-mon[128316]: pgmap v737: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 927 op/s 2023-07-21T19:30:59.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:30:58 smithi101 ceph-mon[128136]: pgmap v738: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:30:59.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:30:58 smithi005 ceph-mon[126460]: pgmap v738: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:30:59.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:30:58 smithi156 ceph-mon[128316]: pgmap v738: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 16 MiB/s wr, 1.42k op/s 2023-07-21T19:31:01.825 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:01 smithi101 ceph-mon[128136]: pgmap v739: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 977 op/s 2023-07-21T19:31:01.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:01 smithi005 ceph-mon[126460]: pgmap v739: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 977 op/s 2023-07-21T19:31:01.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:01 smithi156 ceph-mon[128316]: pgmap v739: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 977 op/s 2023-07-21T19:31:01.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 270.0 s, 368.4 tps, lat 271.226 ms stddev 127.192 2023-07-21T19:31:03.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.505+0000 7fd4f6fff700 1 -- 172.21.15.5:0/2535639775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f0100760 msgr2=0x7fd4f0104720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.505+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/2535639775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f0100760 0x7fd4f0104720 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009920 tx=0x7fd4dc009c30 comp rx=0 tx=0).stop 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 -- 172.21.15.5:0/2535639775 shutdown_connections 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/2535639775 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 0x7fd4f010c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/2535639775 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f0100190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/2535639775 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f0100760 0x7fd4f0104720 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 -- 172.21.15.5:0/2535639775 >> 172.21.15.5:0/2535639775 conn(0x7fd4f00fb650 msgr2=0x7fd4f00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 -- 172.21.15.5:0/2535639775 shutdown_connections 2023-07-21T19:31:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.506+0000 7fd4f6fff700 1 -- 172.21.15.5:0/2535639775 wait complete. 2023-07-21T19:31:03.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.507+0000 7fd4f6fff700 1 Processor -- start 2023-07-21T19:31:03.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 -- start start 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f0100760 0x7fd4f01a27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 0x7fd4f010b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f010be30 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f010bfa0 con 0x7fd4f0104e00 2023-07-21T19:31:03.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.508+0000 7fd4f6fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f010c160 con 0x7fd4f0100760 2023-07-21T19:31:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.509+0000 7fd4effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f0100760 0x7fd4f01a27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.509+0000 7fd4f4d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.509+0000 7fd4f4d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38980/0 (socket says 172.21.15.5:38980) 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.509+0000 7fd4f4d9b700 1 -- 172.21.15.5:0/1343862262 learned_addr learned my addr 172.21.15.5:0/1343862262 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.509+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 0x7fd4f010b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 -- 172.21.15.5:0/1343862262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f0100760 msgr2=0x7fd4f01a27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f0100760 0x7fd4f01a27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 -- 172.21.15.5:0/1343862262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 msgr2=0x7fd4f010b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 0x7fd4f010b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 -- 172.21.15.5:0/1343862262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e40095f0 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.510+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ec10 tx=0x7fd4e400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.511+0000 7fd4edffb700 1 -- 172.21.15.5:0/1343862262 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e400cc90 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.511+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc009520 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.511+0000 7fd4edffb700 1 -- 172.21.15.5:0/1343862262 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e400cdf0 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.511+0000 7fd4edffb700 1 -- 172.21.15.5:0/1343862262 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e4003e80 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.511+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f0195240 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.504 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.517+0000 7fd4edffb700 1 -- 172.21.15.5:0/1343862262 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fd4e4003fe0 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.518+0000 7fd50a8cdb80 1 Processor -- start 2023-07-21T19:31:03.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 -- start start 2023-07-21T19:31:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7448040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb7448580 0x562bb74489d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb734dc50 con 0x562bb7448580 2023-07-21T19:31:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb73a09e0 con 0x562bb744e2d0 2023-07-21T19:31:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.519+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7399e70 con 0x562bb72a46b0 2023-07-21T19:31:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37550/0 (socket says 172.21.15.5:37550) 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 -- 172.21.15.5:0/1177829342 learned_addr learned my addr 172.21.15.5:0/1177829342 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7448040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4effff700 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb7448580 0x562bb74489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 -- 172.21.15.5:0/1177829342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 msgr2=0x562bb7448040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7448040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 -- 172.21.15.5:0/1177829342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb7448580 msgr2=0x562bb74489d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb7448580 0x562bb74489d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4f559c700 1 -- 172.21.15.5:0/1177829342 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bb71c7d80 con 0x562bb72a46b0 2023-07-21T19:31:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.520+0000 7fd4edffb700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 0x7fd4e00b48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8011680 tx=0x7fd4e8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=33b0a423550e5515 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.510 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4edffb700 1 -- 172.21.15.5:0/1343862262 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4e4014070 con 0x7fd4f00ffdc0 2023-07-21T19:31:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4daffd700 1 -- 172.21.15.5:0/1177829342 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800e040 con 0x562bb72a46b0 2023-07-21T19:31:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4daffd700 1 -- 172.21.15.5:0/1177829342 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e800f040 con 0x562bb72a46b0 2023-07-21T19:31:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4daffd700 1 -- 172.21.15.5:0/1177829342 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8004790 con 0x562bb72a46b0 2023-07-21T19:31:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.521+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 0x7fd4e00b48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.511 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.522+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1177829342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 msgr2=0x562bb744dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.522+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8011680 tx=0x7fd4e8009920 comp rx=0 tx=0).stop 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1177829342 shutdown_connections 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb7448580 0x562bb74489d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb744dd90 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1177829342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7448040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1177829342 >> 172.21.15.5:0/1177829342 conn(0x562bb743dbe0 msgr2=0x562bb7442f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.512 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd4f4d9b700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 0x7fd4e00b48c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x562bb7444b40 tx=0x7fd4e40d0970 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1177829342 shutdown_connections 2023-07-21T19:31:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.523+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1177829342 wait complete. 2023-07-21T19:31:03.513 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.524+0000 7fd50a8cdb80 1 -- start start 2023-07-21T19:31:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.524+0000 7fd50a8cdb80 1 Processor -- start 2023-07-21T19:31:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb7454650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.514 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7451fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7339a60 con 0x562bb72a46b0 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7339760 con 0x562bb744e2d0 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb721d8c0 con 0x562bb7448580 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.525+0000 7fd50a8cdb80 10 client.0 _ll_register_callbacks cb 0x562bb7337700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37562/0 (socket says 172.21.15.5:37562) 2023-07-21T19:31:03.515 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 -- 172.21.15.5:0/4121111419 learned_addr learned my addr 172.21.15.5:0/4121111419 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4f559c700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7451fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4f559c700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb7454650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 -- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 msgr2=0x562bb7451fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7451fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 -- 172.21.15.5:0/4121111419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 msgr2=0x562bb7454650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb7454650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.516 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4effff700 1 -- 172.21.15.5:0/4121111419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e8011330 con 0x562bb7448580 2023-07-21T19:31:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.526+0000 7fd4f559c700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb7454650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:31:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.527+0000 7fd4effff700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009920 tx=0x7fd4dc00f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1fb72a16657c1030 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.517 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.527+0000 7fd4cffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.527+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc00b970 con 0x562bb7448580 2023-07-21T19:31:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.527+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4dc00bad0 con 0x562bb7448580 2023-07-21T19:31:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.527+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562bb72f3490 con 0x562bb7448580 2023-07-21T19:31:03.518 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.528+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4dc00bec0 con 0x562bb7448580 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.528+0000 7fd50a8cdb80 10 client.29572 fetch_fsmap learned FSMap version 62 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.528+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x562bb71b0c00 con 0x562bb7448580 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.528+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562bb72f3490 con 0x562bb7448580 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4dc015020 con 0x562bb7448580 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 10 client.29572 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 10 client.29572 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 20 client.29572 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:03.519 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb7461c00 0x562bb7464010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 4 client.29572 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562bb7392e50 con 0x562bb7461c00 2023-07-21T19:31:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.529+0000 7fd4f559c700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb7461c00 0x562bb7464010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.530+0000 7fd4f559c700 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb7461c00 0x562bb7464010 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd8fdceee9a80050 server_cookie=4bd5ce04543bf43 in_seq=0 out_seq=0 2023-07-21T19:31:03.520 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.530+0000 7fd4cffff700 10 client.29572 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.531+0000 7fd4cffff700 1 -- 172.21.15.5:0/4121111419 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562bb7392e50 con 0x562bb7461c00 2023-07-21T19:31:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.531+0000 7fd4cffff700 10 client.29572 handle_command_reply: tid=0 2023-07-21T19:31:03.521 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.531+0000 7fd50a8cdb80 1 client.29572 shutdown 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 msgr2=0x562bb7451860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fd4dc009920 tx=0x7fd4dc00f740 comp rx=0 tx=0).stop 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 shutdown_connections 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb744e2d0 0x562bb7451fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb7461c00 0x562bb7464010 crc :-1 s=READY pgs=459 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb7454650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.522 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd4cffff700 0 client.29572 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/4121111419 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb7448580 0x562bb7451860 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 >> 172.21.15.5:0/4121111419 conn(0x562bb743dbe0 msgr2=0x562bb74424e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 shutdown_connections 2023-07-21T19:31:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.532+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/4121111419 wait complete. 2023-07-21T19:31:03.523 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.533+0000 7fd50a8cdb80 20 client.29572 trim_cache size 0 max 16384 2023-07-21T19:31:03.545 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 Processor -- start 2023-07-21T19:31:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 -- start start 2023-07-21T19:31:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb744d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.546 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7442c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb745e5d0 con 0x562bb72a46b0 2023-07-21T19:31:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb744da40 con 0x562bb7448580 2023-07-21T19:31:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.558+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb74446f0 con 0x562bb743dbe0 2023-07-21T19:31:03.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37566/0 (socket says 172.21.15.5:37566) 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 -- 172.21.15.5:0/2241790254 learned_addr learned my addr 172.21.15.5:0/2241790254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7442c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4effff700 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb744d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 -- 172.21.15.5:0/2241790254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 msgr2=0x562bb7442c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7442c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 -- 172.21.15.5:0/2241790254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 msgr2=0x562bb744d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb744d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.559+0000 7fd4f559c700 1 -- 172.21.15.5:0/2241790254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562bb7468c40 con 0x562bb743dbe0 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.560+0000 7fd4f559c700 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8011680 tx=0x7fd4e8017040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=518b9cad83896ce5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.560+0000 7fd4db7fe700 1 -- 172.21.15.5:0/2241790254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e800e040 con 0x562bb743dbe0 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.560+0000 7fd4db7fe700 1 -- 172.21.15.5:0/2241790254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e802d020 con 0x562bb743dbe0 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.560+0000 7fd4db7fe700 1 -- 172.21.15.5:0/2241790254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4e8025a50 con 0x562bb743dbe0 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/2241790254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 msgr2=0x562bb7448040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8011680 tx=0x7fd4e8017040 comp rx=0 tx=0).stop 2023-07-21T19:31:03.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/2241790254 shutdown_connections 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7442c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb743dbe0 0x562bb7448040 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/2241790254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb72a46b0 0x562bb744d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/2241790254 >> 172.21.15.5:0/2241790254 conn(0x562bb745e200 msgr2=0x562bb744c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/2241790254 shutdown_connections 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.561+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/2241790254 wait complete. 2023-07-21T19:31:03.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.562+0000 7fd50a8cdb80 1 -- start start 2023-07-21T19:31:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.562+0000 7fd50a8cdb80 1 Processor -- start 2023-07-21T19:31:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.563+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb7458660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7456c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7457220 con 0x562bb743dbe0 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7457390 con 0x562bb7448580 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562bb7457550 con 0x562bb72a46b0 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd50a8cdb80 10 client.0 _ll_register_callbacks cb 0x7fd4dc009740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd4effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd4effff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39014/0 (socket says 172.21.15.5:39014) 2023-07-21T19:31:03.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.564+0000 7fd4effff700 1 -- 172.21.15.5:0/1360910735 learned_addr learned my addr 172.21.15.5:0/1360910735 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7456c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb7458660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4effff700 1 -- 172.21.15.5:0/1360910735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 msgr2=0x562bb7458660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4effff700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb7458660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4effff700 1 -- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 msgr2=0x562bb7456c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4effff700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7456c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4effff700 1 -- 172.21.15.5:0/1360910735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562bb71c7d80 con 0x562bb743dbe0 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.565+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb7458660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:31:03.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd4effff700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x562bb7452f10 tx=0x7fd4dc0097f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63d661190fa98ae1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd4cffff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc005920 con 0x562bb743dbe0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(122..122 src has 1..122) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fd4dc000e40 con 0x562bb743dbe0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fd4e800f040 con 0x562bb743dbe0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.566+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd4dc003bc0 con 0x562bb743dbe0 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 10 client.19905 fetch_fsmap learned FSMap version 62 2023-07-21T19:31:03.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=4+}) v3 -- 0x562bb71ee1f0 con 0x562bb743dbe0 2023-07-21T19:31:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fd4e800f040 con 0x562bb743dbe0 2023-07-21T19:31:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 62) v1 ==== 7478+0+0 (secure 0 0 0) 0x7fd4dc015070 con 0x562bb743dbe0 2023-07-21T19:31:03.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 10 client.19905 fetch_fsmap finished waiting for FSMap version 62 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 10 client.19905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 20 client.19905 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.567+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb744e2d0 0x562bb74b8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.568+0000 7fd50a8cdb80 4 client.19905 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.568+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562bb7354ff0 con 0x562bb744e2d0 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.568+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb744e2d0 0x562bb74b8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.569+0000 7fd4f559c700 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb744e2d0 0x562bb74b8930 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e660d904270f859a server_cookie=16fae31586b9fdf0 in_seq=0 out_seq=0 2023-07-21T19:31:03.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.569+0000 7fd4cffff700 10 client.19905 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.602+0000 7fd4cffff700 1 -- 172.21.15.5:0/1360910735 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562bb7354ff0 con 0x562bb744e2d0 2023-07-21T19:31:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.602+0000 7fd4cffff700 10 client.19905 handle_command_reply: tid=0 2023-07-21T19:31:03.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.602+0000 7fd50a8cdb80 1 client.19905 shutdown 2023-07-21T19:31:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 msgr2=0x562bb7458ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x562bb7452f10 tx=0x7fd4dc0097f0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 shutdown_connections 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562bb7448580 0x562bb7456c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562bb72a46b0 0x562bb7458660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562bb744e2d0 0x562bb74b8930 crc :-1 s=READY pgs=460 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 --2- 172.21.15.5:0/1360910735 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562bb743dbe0 0x562bb7458ba0 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 >> 172.21.15.5:0/1360910735 conn(0x562bb745e200 msgr2=0x562bb744d280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd4cffff700 0 client.19905 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 shutdown_connections 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 1 -- 172.21.15.5:0/1360910735 wait complete. 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.603+0000 7fd50a8cdb80 20 client.19905 trim_cache size 0 max 16384 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:31:03.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:31:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:31:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 msgr2=0x7fd4e00b48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 0x7fd4e00b48c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x562bb7444b40 tx=0x7fd4e40d0970 comp rx=0 tx=0).stop 2023-07-21T19:31:03.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 msgr2=0x7fd4f01a2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fd4e400ec10 tx=0x7fd4e400ef20 comp rx=0 tx=0).stop 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 shutdown_connections 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fd4f0104e00 0x7fd4f010b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fd4e00b2410 0x7fd4e00b48c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fd4f00ffdc0 0x7fd4f01a2270 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 --2- 172.21.15.5:0/1343862262 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fd4f0100760 0x7fd4f01a27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:03.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.604+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 >> 172.21.15.5:0/1343862262 conn(0x7fd4f00fb650 msgr2=0x7fd4f00fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.605+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 shutdown_connections 2023-07-21T19:31:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:03.605+0000 7fd4f6fff700 1 -- 172.21.15.5:0/1343862262 wait complete. 2023-07-21T19:31:04.073 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:03 smithi101 ceph-mon[128136]: pgmap v740: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.38k op/s 2023-07-21T19:31:04.148 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:03 smithi005 ceph-mon[126460]: pgmap v740: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.38k op/s 2023-07-21T19:31:04.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:03 smithi156 ceph-mon[128316]: pgmap v740: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 15 MiB/s wr, 1.38k op/s 2023-07-21T19:31:05.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:04 smithi101 ceph-mon[128136]: pgmap v741: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-07-21T19:31:05.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:04 smithi005 ceph-mon[126460]: pgmap v741: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-07-21T19:31:05.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:04 smithi156 ceph-mon[128316]: pgmap v741: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 11 MiB/s wr, 1.01k op/s 2023-07-21T19:31:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 275.0 s, 212.2 tps, lat 446.149 ms stddev 311.496 2023-07-21T19:31:07.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:06 smithi101 ceph-mon[128136]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:31:07.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:06 smithi101 ceph-mon[128136]: pgmap v743: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2023-07-21T19:31:07.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:06 smithi005 ceph-mon[126460]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:31:07.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:06 smithi005 ceph-mon[126460]: pgmap v743: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2023-07-21T19:31:07.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:06 smithi156 ceph-mon[128316]: osdmap e123: 12 total, 12 up, 12 in 2023-07-21T19:31:07.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:06 smithi156 ceph-mon[128316]: pgmap v743: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 12 MiB/s wr, 1.16k op/s 2023-07-21T19:31:09.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:08 smithi101 ceph-mon[128136]: pgmap v744: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:09.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:08 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:09.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:08 smithi005 ceph-mon[126460]: pgmap v744: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:09.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:08 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:09.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:08 smithi156 ceph-mon[128316]: pgmap v744: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:09.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:08 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:11.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:11 smithi101 ceph-mon[128136]: pgmap v745: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:11.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:11 smithi005 ceph-mon[126460]: pgmap v745: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:11.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:11 smithi156 ceph-mon[128316]: pgmap v745: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.7 MiB/s wr, 915 op/s 2023-07-21T19:31:11.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 280.0 s, 248.6 tps, lat 338.543 ms stddev 170.234 2023-07-21T19:31:14.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:13 smithi101 ceph-mon[128136]: pgmap v746: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2023-07-21T19:31:14.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:13 smithi101 ceph-mon[128136]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:31:14.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:13 smithi005 ceph-mon[126460]: pgmap v746: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2023-07-21T19:31:14.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:13 smithi005 ceph-mon[126460]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:31:14.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:13 smithi156 ceph-mon[128316]: pgmap v746: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 726 op/s 2023-07-21T19:31:14.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:13 smithi156 ceph-mon[128316]: osdmap e124: 12 total, 12 up, 12 in 2023-07-21T19:31:15.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:14 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:15.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:14 smithi101 ceph-mon[128136]: pgmap v748: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 794 op/s 2023-07-21T19:31:15.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:14 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:15.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:14 smithi005 ceph-mon[126460]: pgmap v748: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 794 op/s 2023-07-21T19:31:15.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:14 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:15.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:14 smithi156 ceph-mon[128316]: pgmap v748: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.2 MiB/s wr, 794 op/s 2023-07-21T19:31:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 -- 172.21.15.5:0/88078502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa574105b60 msgr2=0x7fa574105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 --2- 172.21.15.5:0/88078502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa574105b60 0x7fa574105f30 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7fa564009990 tx=0x7fa564009ca0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 -- 172.21.15.5:0/88078502 shutdown_connections 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 --2- 172.21.15.5:0/88078502 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57410c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 --2- 172.21.15.5:0/88078502 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa574105b60 0x7fa574105f30 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 --2- 172.21.15.5:0/88078502 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa5740ffb30 0x7fa5740fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 -- 172.21.15.5:0/88078502 >> 172.21.15.5:0/88078502 conn(0x7fa5740fb5f0 msgr2=0x7fa5740fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 -- 172.21.15.5:0/88078502 shutdown_connections 2023-07-21T19:31:16.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.806+0000 7fa5789ff700 1 -- 172.21.15.5:0/88078502 wait complete. 2023-07-21T19:31:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.808+0000 7fa5789ff700 1 Processor -- start 2023-07-21T19:31:16.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.808+0000 7fa5789ff700 1 -- start start 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57419c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574105b60 0x7fa574195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574195fc0 con 0x7fa5741004e0 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa574196130 con 0x7fa5740ffb30 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa5789ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5741962f0 con 0x7fa574105b60 2023-07-21T19:31:16.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa571d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57419c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa572d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574105b60 0x7fa574195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43932/0 (socket says 172.21.15.5:43932) 2023-07-21T19:31:16.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa57259c700 1 -- 172.21.15.5:0/127338674 learned_addr learned my addr 172.21.15.5:0/127338674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.809+0000 7fa571d9b700 1 -- 172.21.15.5:0/127338674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574105b60 msgr2=0x7fa574195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa571d9b700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574105b60 0x7fa574195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa571d9b700 1 -- 172.21.15.5:0/127338674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 msgr2=0x7fa57419baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa571d9b700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa571d9b700 1 -- 172.21.15.5:0/127338674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640095f0 con 0x7fa5741004e0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa57259c700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa571d9b700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57419c030 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa5637fe700 1 -- 172.21.15.5:0/127338674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56800c9c0 con 0x7fa5741004e0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa5637fe700 1 -- 172.21.15.5:0/127338674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa568010430 con 0x7fa5741004e0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa5637fe700 1 -- 172.21.15.5:0/127338674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa568018680 con 0x7fa5741004e0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.810+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5741965d0 con 0x7fa5741004e0 2023-07-21T19:31:16.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.811+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5741a2ae0 con 0x7fa5741004e0 2023-07-21T19:31:16.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.815+0000 7fa5637fe700 1 -- 172.21.15.5:0/127338674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa5680105a0 con 0x7fa5741004e0 2023-07-21T19:31:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.816+0000 7fa58c323b80 1 Processor -- start 2023-07-21T19:31:16.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 -- start start 2023-07-21T19:31:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7ddde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7eecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d6dfc50 con 0x55d46d6366b0 2023-07-21T19:31:16.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d7329e0 con 0x55d46d7da040 2023-07-21T19:31:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa58c323b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d72be70 con 0x55d46d7da9f0 2023-07-21T19:31:16.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.817+0000 7fa5637fe700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 0x7fa55c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43946/0 (socket says 172.21.15.5:43946) 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 -- 172.21.15.5:0/3311085605 learned_addr learned my addr 172.21.15.5:0/3311085605 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa571d9b700 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7eecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa5637fe700 1 -- 172.21.15.5:0/127338674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa568014070 con 0x7fa5741004e0 2023-07-21T19:31:16.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa572d9d700 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7ddde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa572d9d700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 0x7fa55c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 -- 172.21.15.5:0/3311085605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 msgr2=0x55d46d7eecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7eecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 -- 172.21.15.5:0/3311085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 msgr2=0x55d46d7ddde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7ddde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.818+0000 7fa57259c700 1 -- 172.21.15.5:0/3311085605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46d559d80 con 0x55d46d7da040 2023-07-21T19:31:16.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.819+0000 7fa57259c700 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa564005c70 tx=0x7fa564004500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0fc01299d745bd2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.819+0000 7fa560ff9700 1 -- 172.21.15.5:0/3311085605 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56401d050 con 0x55d46d7da040 2023-07-21T19:31:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.819+0000 7fa560ff9700 1 -- 172.21.15.5:0/3311085605 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa56401e040 con 0x55d46d7da040 2023-07-21T19:31:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.819+0000 7fa572d9d700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 0x7fa55c0a3870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009ec0 tx=0x7fa56c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.819+0000 7fa560ff9700 1 -- 172.21.15.5:0/3311085605 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564004d00 con 0x55d46d7da040 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 -- 172.21.15.5:0/3311085605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 msgr2=0x55d46d7da4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fa564005c70 tx=0x7fa564004500 comp rx=0 tx=0).stop 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 -- 172.21.15.5:0/3311085605 shutdown_connections 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7eecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7ddde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3311085605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7da4b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 -- 172.21.15.5:0/3311085605 >> 172.21.15.5:0/3311085605 conn(0x55d46d7cfbe0 msgr2=0x55d46d7d4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 -- 172.21.15.5:0/3311085605 shutdown_connections 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.820+0000 7fa58c323b80 1 -- 172.21.15.5:0/3311085605 wait complete. 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.821+0000 7fa58c323b80 1 -- start start 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.821+0000 7fa58c323b80 1 Processor -- start 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 0x55d46d7ebf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da9f0 0x55d46d7ec640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d6cba60 con 0x55d46d7da9f0 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d6cb760 con 0x55d46d6366b0 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d5af8c0 con 0x55d46d7da040 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa58c323b80 10 client.0 _ll_register_callbacks cb 0x55d46d6c9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43948/0 (socket says 172.21.15.5:43948) 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa57259c700 1 -- 172.21.15.5:0/2870152586 learned_addr learned my addr 172.21.15.5:0/2870152586 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:16.813 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa572d9d700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da9f0 0x55d46d7ec640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.822+0000 7fa571d9b700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 0x55d46d7ebf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 -- 172.21.15.5:0/2870152586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 msgr2=0x55d46d7ebf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 0x55d46d7ebf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 -- 172.21.15.5:0/2870152586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da9f0 msgr2=0x55d46d7ec640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da9f0 0x55d46d7ec640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 -- 172.21.15.5:0/2870152586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa57410d220 con 0x55d46d6366b0 2023-07-21T19:31:16.814 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa571d9b700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 0x55d46d7ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:31:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa57259c700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ce0 tx=0x7fa564024ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=32b75a177c1ac89d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.823+0000 7fa559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:31:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56400f660 con 0x55d46d6366b0 2023-07-21T19:31:16.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa564004570 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d46d685490 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa56400f7c0 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa58c323b80 10 client.29666 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55d46d542c00 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.824+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d46d685490 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa56402d6b0 con 0x55d46d6366b0 2023-07-21T19:31:16.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 10 client.29666 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 10 client.29666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 20 client.29666 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d7fbec0 0x55d46d7fe2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 4 client.29666 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d46d724e50 con 0x55d46d7fbec0 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.825+0000 7fa571d9b700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d7fbec0 0x55d46d7fe2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.826+0000 7fa571d9b700 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d7fbec0 0x55d46d7fe2d0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f45ff4833ddfc551 server_cookie=f15469a7f47b4c3e in_seq=0 out_seq=0 2023-07-21T19:31:16.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.826+0000 7fa559ffb700 10 client.29666 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.826+0000 7fa559ffb700 1 -- 172.21.15.5:0/2870152586 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d46d724e50 con 0x55d46d7fbec0 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.826+0000 7fa559ffb700 10 client.29666 handle_command_reply: tid=0 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 client.29666 shutdown 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 msgr2=0x55d46d7eec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fa564009ce0 tx=0x7fa564024ad0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 shutdown_connections 2023-07-21T19:31:16.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d7fbec0 0x55d46d7fe2d0 crc :-1 s=READY pgs=461 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da9f0 0x55d46d7ec640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7eec80 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 --2- 172.21.15.5:0/2870152586 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da040 0x55d46d7ebf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 >> 172.21.15.5:0/2870152586 conn(0x55d46d7cfbe0 msgr2=0x55d46d7d44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa559ffb700 0 client.29666 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 shutdown_connections 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.827+0000 7fa58c323b80 1 -- 172.21.15.5:0/2870152586 wait complete. 2023-07-21T19:31:16.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.828+0000 7fa58c323b80 20 client.29666 trim_cache size 0 max 16384 2023-07-21T19:31:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.845+0000 7fa58c323b80 1 Processor -- start 2023-07-21T19:31:16.833 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 -- start start 2023-07-21T19:31:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.834 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da040 0x55d46d7f3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7d9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d7db1e0 con 0x55d46d7da040 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d7ccbb0 con 0x55d46d6366b0 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa58c323b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d7dda10 con 0x55d46d7da9f0 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa572d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7d9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa571d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da040 0x55d46d7f3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.835 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa57259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43956/0 (socket says 172.21.15.5:43956) 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.846+0000 7fa57259c700 1 -- 172.21.15.5:0/390368014 learned_addr learned my addr 172.21.15.5:0/390368014 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 -- 172.21.15.5:0/390368014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 msgr2=0x55d46d7d9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7d9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 -- 172.21.15.5:0/390368014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da040 msgr2=0x55d46d7f3440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da040 0x55d46d7f3440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 -- 172.21.15.5:0/390368014 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d46d5801f0 con 0x55d46d6366b0 2023-07-21T19:31:16.836 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa572d9d700 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:16.837 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa57259c700 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa564009520 tx=0x7fa564005c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7a5131a4e8182d9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa5617fa700 1 -- 172.21.15.5:0/390368014 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56401d070 con 0x55d46d6366b0 2023-07-21T19:31:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa5617fa700 1 -- 172.21.15.5:0/390368014 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa564024bf0 con 0x55d46d6366b0 2023-07-21T19:31:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.847+0000 7fa5617fa700 1 -- 172.21.15.5:0/390368014 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa564005490 con 0x55d46d6366b0 2023-07-21T19:31:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- 172.21.15.5:0/390368014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 msgr2=0x55d46d7fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fa564009520 tx=0x7fa564005c40 comp rx=0 tx=0).stop 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- 172.21.15.5:0/390368014 shutdown_connections 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7d9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d6366b0 0x55d46d7fe2a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 --2- 172.21.15.5:0/390368014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d7da040 0x55d46d7f3440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- 172.21.15.5:0/390368014 >> 172.21.15.5:0/390368014 conn(0x55d46d7cfbe0 msgr2=0x55d46d802c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- 172.21.15.5:0/390368014 shutdown_connections 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- 172.21.15.5:0/390368014 wait complete. 2023-07-21T19:31:16.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.848+0000 7fa58c323b80 1 -- start start 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 Processor -- start 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7f0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7f10d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d7f1810 con 0x55d46d6366b0 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d84ed80 con 0x55d46d7da040 2023-07-21T19:31:16.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d46d84ef40 con 0x55d46d7da9f0 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.849+0000 7fa58c323b80 10 client.0 _ll_register_callbacks cb 0x7fa568010b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa572d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7f10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa571d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7f0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40120/0 (socket says 172.21.15.5:40120) 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 -- 172.21.15.5:0/3134195078 learned_addr learned my addr 172.21.15.5:0/3134195078 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 -- 172.21.15.5:0/3134195078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 msgr2=0x55d46d7f10d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7f10d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 -- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 msgr2=0x55d46d7f0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7f0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 -- 172.21.15.5:0/3134195078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa56c020040 con 0x55d46d6366b0 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa571d9b700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa57259c700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa564006010 tx=0x7fa56402d440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6075bbe82165c2f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa559ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.850+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56401d040 con 0x55d46d6366b0 2023-07-21T19:31:16.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa56401e040 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa564000e40 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa564005240 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 10 client.19923 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x55d46d542c00 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa56401e040 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa564004ca0 con 0x55d46d6366b0 2023-07-21T19:31:16.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 10 client.19923 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 10 client.19923 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 20 client.19923 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d850230 0x55d46d852640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 4 client.19923 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:31:16.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.851+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d46d6e6ff0 con 0x55d46d850230 2023-07-21T19:31:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.852+0000 7fa571d9b700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d850230 0x55d46d852640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.852+0000 7fa571d9b700 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d850230 0x55d46d852640 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d0de78d03827536 server_cookie=98ee41e1a09819be in_seq=0 out_seq=0 2023-07-21T19:31:16.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.852+0000 7fa559ffb700 10 client.19923 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.885+0000 7fa559ffb700 1 -- 172.21.15.5:0/3134195078 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d46d6e6ff0 con 0x55d46d850230 2023-07-21T19:31:16.873 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.885+0000 7fa559ffb700 10 client.19923 handle_command_reply: tid=0 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.885+0000 7fa58c323b80 1 client.19923 shutdown 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.885+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 msgr2=0x55d46d7f3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.885+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7fa564006010 tx=0x7fa56402d440 comp rx=0 tx=0).stop 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 shutdown_connections 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d46d7da9f0 0x55d46d7f10d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d46d850230 0x55d46d852640 crc :-1 s=READY pgs=462 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.874 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d46d6366b0 0x55d46d7f3670 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 --2- 172.21.15.5:0/3134195078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d46d7da040 0x55d46d7f0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa559ffb700 0 client.19923 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 >> 172.21.15.5:0/3134195078 conn(0x55d46d7cfbe0 msgr2=0x55d46d7d9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 shutdown_connections 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 1 -- 172.21.15.5:0/3134195078 wait complete. 2023-07-21T19:31:16.875 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.886+0000 7fa58c323b80 20 client.19923 trim_cache size 0 max 16384 2023-07-21T19:31:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:31:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:31:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:31:16.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:31:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 msgr2=0x7fa55c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 0x7fa55c0a3870 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa56c009ec0 tx=0x7fa56c00b040 comp rx=0 tx=0).stop 2023-07-21T19:31:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 msgr2=0x7fa57419c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:16.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57419c030 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7fa56800e9f0 tx=0x7fa56800ed00 comp rx=0 tx=0).stop 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 shutdown_connections 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa574105b60 0x7fa574195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa5740ffb30 0x7fa57419baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa55c0a13c0 0x7fa55c0a3870 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 --2- 172.21.15.5:0/127338674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa5741004e0 0x7fa57419c030 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 >> 172.21.15.5:0/127338674 conn(0x7fa5740fb5f0 msgr2=0x7fa5740fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 shutdown_connections 2023-07-21T19:31:16.878 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:16.887+0000 7fa5789ff700 1 -- 172.21.15.5:0/127338674 wait complete. 2023-07-21T19:31:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 285.0 s, 68.2 tps, lat 1816.151 ms stddev 2212.560 2023-07-21T19:31:17.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:17 smithi101 ceph-mon[128136]: pgmap v749: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 688 op/s 2023-07-21T19:31:17.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:17 smithi005 ceph-mon[126460]: pgmap v749: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 688 op/s 2023-07-21T19:31:17.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:17 smithi156 ceph-mon[128316]: pgmap v749: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.1 MiB/s wr, 688 op/s 2023-07-21T19:31:19.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:18 smithi101 ceph-mon[128136]: pgmap v750: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:18 smithi005 ceph-mon[126460]: pgmap v750: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:19.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:18 smithi156 ceph-mon[128316]: pgmap v750: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:21.820 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:21 smithi101 ceph-mon[128136]: pgmap v751: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:21.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:21 smithi005 ceph-mon[126460]: pgmap v751: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:21.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:21 smithi156 ceph-mon[128316]: pgmap v751: 97 pgs: 97 active+clean; 13 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 5.1 MiB/s wr, 530 op/s 2023-07-21T19:31:21.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 290.0 s, 320.0 tps, lat 307.702 ms stddev 301.444 2023-07-21T19:31:23.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:23 smithi101 ceph-mon[128136]: pgmap v752: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-07-21T19:31:23.890 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:31:23.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:23 smithi005 ceph-mon[126460]: pgmap v752: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-07-21T19:31:23.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:23 smithi156 ceph-mon[128316]: pgmap v752: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.8 MiB/s wr, 573 op/s 2023-07-21T19:31:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.265+0000 7f154e5ff700 1 -- 172.21.15.5:0/2791445299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 msgr2=0x7f1548072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.265+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2791445299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f1548072c80 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f1538009f90 tx=0x7f153800b330 comp rx=0 tx=0).stop 2023-07-21T19:31:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 -- 172.21.15.5:0/2791445299 shutdown_connections 2023-07-21T19:31:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2791445299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1548110ae0 0x7f1548112ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2791445299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f1548072c80 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2791445299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f15480731c0 0x7f15481105a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 -- 172.21.15.5:0/2791445299 >> 172.21.15.5:0/2791445299 conn(0x7f15480fb5f0 msgr2=0x7f15480fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 -- 172.21.15.5:0/2791445299 shutdown_connections 2023-07-21T19:31:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.266+0000 7f154e5ff700 1 -- 172.21.15.5:0/2791445299 wait complete. 2023-07-21T19:31:24.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.268+0000 7f154e5ff700 1 Processor -- start 2023-07-21T19:31:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.268+0000 7f154e5ff700 1 -- start start 2023-07-21T19:31:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480731c0 0x7f15481103e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1548110ae0 0x7f1548108ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481096c0 con 0x7f1548110ae0 2023-07-21T19:31:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548109830 con 0x7f15480728b0 2023-07-21T19:31:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f154e5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481099f0 con 0x7f15480731c0 2023-07-21T19:31:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f1547fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f1547fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43982/0 (socket says 172.21.15.5:43982) 2023-07-21T19:31:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.269+0000 7f1547fff700 1 -- 172.21.15.5:0/2210824157 learned_addr learned my addr 172.21.15.5:0/2210824157 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.270+0000 7f1547fff700 1 -- 172.21.15.5:0/2210824157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480731c0 msgr2=0x7f15481103e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:31:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.270+0000 7f1547fff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480731c0 0x7f15481103e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.270+0000 7f1547fff700 1 -- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1548110ae0 msgr2=0x7f1548108ef0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:31:24.258 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.270+0000 7f1547fff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1548110ae0 0x7f1548108ef0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.270+0000 7f1547fff700 1 -- 172.21.15.5:0/2210824157 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1538009c40 con 0x7f15480728b0 2023-07-21T19:31:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f1547fff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f153800d010 tx=0x7f1538007860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f1545ffb700 1 -- 172.21.15.5:0/2210824157 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153800e040 con 0x7f15480728b0 2023-07-21T19:31:24.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f1545ffb700 1 -- 172.21.15.5:0/2210824157 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1538004e40 con 0x7f15480728b0 2023-07-21T19:31:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f1545ffb700 1 -- 172.21.15.5:0/2210824157 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153801b6b0 con 0x7f15480728b0 2023-07-21T19:31:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1548109c70 con 0x7f15480728b0 2023-07-21T19:31:24.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.271+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154810dc30 con 0x7f15480728b0 2023-07-21T19:31:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.277+0000 7f1561ec6b80 1 Processor -- start 2023-07-21T19:31:24.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.278+0000 7f1545ffb700 1 -- 172.21.15.5:0/2210824157 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f153801b810 con 0x7f15480728b0 2023-07-21T19:31:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 -- start start 2023-07-21T19:31:24.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1545ffb700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 0x7f15340a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e759b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1545ffb700 1 -- 172.21.15.5:0/2210824157 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f15380ce5a0 con 0x7f15480728b0 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7496cb0 con 0x5652e7590be0 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e74e9a40 con 0x5652e75902d0 2023-07-21T19:31:24.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.280+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e74e2ed0 con 0x5652e759bad0 2023-07-21T19:31:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.281+0000 7f1547fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.281+0000 7f1547fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43994/0 (socket says 172.21.15.5:43994) 2023-07-21T19:31:24.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.281+0000 7f1547fff700 1 -- 172.21.15.5:0/1120482904 learned_addr learned my addr 172.21.15.5:0/1120482904 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 -- 172.21.15.5:0/1120482904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 msgr2=0x5652e759deb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759deb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 -- 172.21.15.5:0/1120482904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 msgr2=0x5652e759b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e759b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 -- 172.21.15.5:0/1120482904 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e7310de0 con 0x5652e75902d0 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.282+0000 7f1547fff700 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f15380a3530 tx=0x7f1538009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ed35bd751189689 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.283+0000 7f154cb9c700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 0x7f15340a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.283+0000 7f153e7fc700 1 -- 172.21.15.5:0/1120482904 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538084ea0 con 0x5652e75902d0 2023-07-21T19:31:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.283+0000 7f153e7fc700 1 -- 172.21.15.5:0/1120482904 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153809f6d0 con 0x5652e75902d0 2023-07-21T19:31:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.283+0000 7f153e7fc700 1 -- 172.21.15.5:0/1120482904 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15380cbf10 con 0x5652e75902d0 2023-07-21T19:31:24.272 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.284+0000 7f1561ec6b80 1 -- 172.21.15.5:0/1120482904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 msgr2=0x5652e75906a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.284+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f15380a3530 tx=0x7f1538009920 comp rx=0 tx=0).stop 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.284+0000 7f154cb9c700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 0x7f15340a3870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1548110ab0 tx=0x7f1540009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 -- 172.21.15.5:0/1120482904 shutdown_connections 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75906a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/1120482904 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e759b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 -- 172.21.15.5:0/1120482904 >> 172.21.15.5:0/1120482904 conn(0x5652e7586da0 msgr2=0x5652e758c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 -- 172.21.15.5:0/1120482904 shutdown_connections 2023-07-21T19:31:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.285+0000 7f1561ec6b80 1 -- 172.21.15.5:0/1120482904 wait complete. 2023-07-21T19:31:24.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.286+0000 7f1561ec6b80 1 -- start start 2023-07-21T19:31:24.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.286+0000 7f1561ec6b80 1 Processor -- start 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e7590be0 0x5652e75985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e7598b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7482ac0 con 0x5652e75902d0 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e74827c0 con 0x5652e7590be0 2023-07-21T19:31:24.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7366920 con 0x5652e759bad0 2023-07-21T19:31:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1561ec6b80 10 client.0 _ll_register_callbacks cb 0x5652e7480760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1547fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1547fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40128/0 (socket says 172.21.15.5:40128) 2023-07-21T19:31:24.277 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.288+0000 7f1547fff700 1 -- 172.21.15.5:0/871171265 learned_addr learned my addr 172.21.15.5:0/871171265 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f154cb9c700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e7598b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f153f7fe700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e7590be0 0x5652e75985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f1547fff700 1 -- 172.21.15.5:0/871171265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 msgr2=0x5652e7598b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f1547fff700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e7598b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f1547fff700 1 -- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e7590be0 msgr2=0x5652e75985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f1547fff700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e7590be0 0x5652e75985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.278 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.289+0000 7f1547fff700 1 -- 172.21.15.5:0/871171265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154810dc30 con 0x5652e75902d0 2023-07-21T19:31:24.280 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.290+0000 7f1547fff700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1538097420 tx=0x7f153809b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9240aa09ddb6a674 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.281 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.290+0000 7f15277fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:31:24.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.294+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538084ea0 con 0x5652e75902d0 2023-07-21T19:31:24.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.294+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f153809bb00 con 0x5652e75902d0 2023-07-21T19:31:24.282 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.294+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5652e743c4f0 con 0x5652e75902d0 2023-07-21T19:31:24.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.295+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15380cc120 con 0x5652e75902d0 2023-07-21T19:31:24.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.295+0000 7f1561ec6b80 10 client.19926 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:24.283 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.295+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5652e72f9c60 con 0x5652e75902d0 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f15380c9030 con 0x5652e75902d0 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f15380860b0 con 0x5652e75902d0 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 10 client.19926 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 10 client.19926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 20 client.19926 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e75bbcb0 0x5652e75be0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.284 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 4 client.19926 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:24.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.296+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5652e74dbeb0 con 0x5652e75bbcb0 2023-07-21T19:31:24.285 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.297+0000 7f153f7fe700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e75bbcb0 0x5652e75be0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.302+0000 7f153f7fe700 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e75bbcb0 0x5652e75be0c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39d5ae6210014cf3 server_cookie=9e7ffe7dd03c788c in_seq=0 out_seq=0 2023-07-21T19:31:24.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.302+0000 7f15277fe700 10 client.19926 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:24.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.305+0000 7f15277fe700 1 -- 172.21.15.5:0/871171265 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5652e74dbeb0 con 0x5652e75bbcb0 2023-07-21T19:31:24.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.305+0000 7f15277fe700 10 client.19926 handle_command_reply: tid=0 2023-07-21T19:31:24.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.305+0000 7f1561ec6b80 1 client.19926 shutdown 2023-07-21T19:31:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.306+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 msgr2=0x5652e759b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.306+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f1538097420 tx=0x7f153809b820 comp rx=0 tx=0).stop 2023-07-21T19:31:24.296 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 shutdown_connections 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e7598b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e75bbcb0 0x5652e75be0c0 crc :-1 s=READY pgs=463 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e75902d0 0x5652e759b370 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/871171265 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e7590be0 0x5652e75985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 >> 172.21.15.5:0/871171265 conn(0x5652e7586da0 msgr2=0x5652e758c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f15277fe700 0 client.19926 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 shutdown_connections 2023-07-21T19:31:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.309+0000 7f1561ec6b80 1 -- 172.21.15.5:0/871171265 wait complete. 2023-07-21T19:31:24.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.310+0000 7f1561ec6b80 20 client.19926 trim_cache size 0 max 16384 2023-07-21T19:31:24.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 Processor -- start 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 -- start start 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75b3420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e7597db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7599bd0 con 0x5652e7590be0 2023-07-21T19:31:24.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7596b50 con 0x5652e75902d0 2023-07-21T19:31:24.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.345+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e7598e60 con 0x5652e759bad0 2023-07-21T19:31:24.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.346+0000 7f154cb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.346+0000 7f154cb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51060/0 (socket says 172.21.15.5:51060) 2023-07-21T19:31:24.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.346+0000 7f154cb9c700 1 -- 172.21.15.5:0/3338285857 learned_addr learned my addr 172.21.15.5:0/3338285857 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:24.336 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.346+0000 7f1547fff700 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75b3420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.347+0000 7f154cb9c700 1 -- 172.21.15.5:0/3338285857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 msgr2=0x5652e75b3420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.347+0000 7f154cb9c700 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75b3420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.347+0000 7f154cb9c700 1 -- 172.21.15.5:0/3338285857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 msgr2=0x5652e7597db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.347+0000 7f154cb9c700 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e7597db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.347+0000 7f154cb9c700 1 -- 172.21.15.5:0/3338285857 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652e75885e0 con 0x5652e759bad0 2023-07-21T19:31:24.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.351+0000 7f154cb9c700 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f154000ebe0 tx=0x7f1540011000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=905011172fe61538 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.351+0000 7f153effd700 1 -- 172.21.15.5:0/3338285857 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400106b0 con 0x5652e759bad0 2023-07-21T19:31:24.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.352+0000 7f153effd700 1 -- 172.21.15.5:0/3338285857 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1540021070 con 0x5652e759bad0 2023-07-21T19:31:24.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.352+0000 7f153effd700 1 -- 172.21.15.5:0/3338285857 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154001da70 con 0x5652e759bad0 2023-07-21T19:31:24.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 -- 172.21.15.5:0/3338285857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 msgr2=0x5652e759af10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f154000ebe0 tx=0x7f1540011000 comp rx=0 tx=0).stop 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 -- 172.21.15.5:0/3338285857 shutdown_connections 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e759bad0 0x5652e759af10 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e75902d0 0x5652e75b3420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/3338285857 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e7597db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 -- 172.21.15.5:0/3338285857 >> 172.21.15.5:0/3338285857 conn(0x5652e7586da0 msgr2=0x5652e758ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 -- 172.21.15.5:0/3338285857 shutdown_connections 2023-07-21T19:31:24.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.353+0000 7f1561ec6b80 1 -- 172.21.15.5:0/3338285857 wait complete. 2023-07-21T19:31:24.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.354+0000 7f1561ec6b80 1 -- start start 2023-07-21T19:31:24.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.354+0000 7f1561ec6b80 1 Processor -- start 2023-07-21T19:31:24.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e75af5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e759bad0 0x5652e75afb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e75b0250 con 0x5652e7590be0 2023-07-21T19:31:24.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e75b03c0 con 0x5652e759bad0 2023-07-21T19:31:24.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652e760fb70 con 0x5652e75902d0 2023-07-21T19:31:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.355+0000 7f1561ec6b80 10 client.0 _ll_register_callbacks cb 0x5652e7480760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51072/0 (socket says 172.21.15.5:51072) 2023-07-21T19:31:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 -- 172.21.15.5:0/2862163938 learned_addr learned my addr 172.21.15.5:0/2862163938 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:24.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f153f7fe700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e75af5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f154cb9c700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e759bad0 0x5652e75afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 -- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e759bad0 msgr2=0x5652e75afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e759bad0 0x5652e75afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 -- 172.21.15.5:0/2862163938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 msgr2=0x5652e75af5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e75af5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.356+0000 7f1547fff700 1 -- 172.21.15.5:0/2862163938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f154000e890 con 0x5652e75902d0 2023-07-21T19:31:24.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.357+0000 7f1547fff700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f153809c0a0 tx=0x7f15380a3bc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd4ecb060aae97b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:24.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.357+0000 7f15277fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.357+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1538084ea0 con 0x5652e75902d0 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.357+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f15380c5e40 con 0x5652e75902d0 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.357+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f153809f6d0 con 0x5652e75902d0 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.359+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f15380cc120 con 0x5652e75902d0 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.359+0000 7f1561ec6b80 10 client.29608 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:24.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.359+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5652e7337250 con 0x5652e75902d0 2023-07-21T19:31:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153809f6d0 con 0x5652e75902d0 2023-07-21T19:31:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f153809ab90 con 0x5652e75902d0 2023-07-21T19:31:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 10 client.29608 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 10 client.29608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:24.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 20 client.29608 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e7610d60 0x5652e7613140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 4 client.29608 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:31:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.360+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5652e749e050 con 0x5652e7610d60 2023-07-21T19:31:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.362+0000 7f153f7fe700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e7610d60 0x5652e7613140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:24.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.363+0000 7f15277fe700 10 client.29608 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.364+0000 7f153f7fe700 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e7610d60 0x5652e7613140 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bc396497c7368a7 server_cookie=8fd3ada965104e92 in_seq=0 out_seq=0 2023-07-21T19:31:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.364+0000 7f15277fe700 1 -- 172.21.15.5:0/2862163938 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5652e749e050 con 0x5652e7610d60 2023-07-21T19:31:24.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.364+0000 7f15277fe700 10 client.29608 handle_command_reply: tid=0 2023-07-21T19:31:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 client.29608 shutdown 2023-07-21T19:31:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 msgr2=0x5652e75b32a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f153809c0a0 tx=0x7f15380a3bc0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 shutdown_connections 2023-07-21T19:31:24.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5652e759bad0 0x5652e75afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5652e75902d0 0x5652e75b32a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5652e7610d60 0x5652e7613140 crc :-1 s=READY pgs=464 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 --2- 172.21.15.5:0/2862163938 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5652e7590be0 0x5652e75af5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.366+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 >> 172.21.15.5:0/2862163938 conn(0x5652e7586da0 msgr2=0x5652e75b7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.367+0000 7f15277fe700 0 client.29608 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.367+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 shutdown_connections 2023-07-21T19:31:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.367+0000 7f1561ec6b80 1 -- 172.21.15.5:0/2862163938 wait complete. 2023-07-21T19:31:24.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.368+0000 7f1561ec6b80 20 client.29608 trim_cache size 0 max 16384 2023-07-21T19:31:24.356 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:31:24.356 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:31:24.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:31:24.358 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:31:24.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 msgr2=0x7f15340a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 0x7f15340a3870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f1548110ab0 tx=0x7f1540009310 comp rx=0 tx=0).stop 2023-07-21T19:31:24.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 msgr2=0x7f154810fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f153800d010 tx=0x7f1538007860 comp rx=0 tx=0).stop 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 shutdown_connections 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15340a13c0 0x7f15340a3870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1548110ae0 0x7f1548108ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f15480728b0 0x7f154810fe60 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 --2- 172.21.15.5:0/2210824157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f15480731c0 0x7f15481103e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:24.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 >> 172.21.15.5:0/2210824157 conn(0x7f15480fb5f0 msgr2=0x7f15480fc250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 shutdown_connections 2023-07-21T19:31:24.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:24.369+0000 7f154e5ff700 1 -- 172.21.15.5:0/2210824157 wait complete. 2023-07-21T19:31:24.387 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:31:25.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:25 smithi101 ceph-mon[128136]: pgmap v753: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 494 op/s 2023-07-21T19:31:25.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:25 smithi005 ceph-mon[126460]: pgmap v753: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 494 op/s 2023-07-21T19:31:25.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:25 smithi156 ceph-mon[128316]: pgmap v753: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 494 op/s 2023-07-21T19:31:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 295.0 s, 394.4 tps, lat 253.410 ms stddev 47.079 2023-07-21T19:31:27.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:27 smithi101 ceph-mon[128136]: pgmap v754: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 484 op/s 2023-07-21T19:31:27.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:27 smithi005 ceph-mon[126460]: pgmap v754: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 484 op/s 2023-07-21T19:31:27.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:27 smithi156 ceph-mon[128316]: pgmap v754: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.0 MiB/s wr, 484 op/s 2023-07-21T19:31:30.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.057+0000 7f3b763ff700 1 -- 172.21.15.5:0/1661199664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b70105b60 msgr2=0x7f3b70105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.057+0000 7f3b763ff700 1 --2- 172.21.15.5:0/1661199664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b70105b60 0x7f3b70105f30 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f3b60009990 tx=0x7f3b60009ca0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.057+0000 7f3b763ff700 1 -- 172.21.15.5:0/1661199664 shutdown_connections 2023-07-21T19:31:30.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.057+0000 7f3b763ff700 1 --2- 172.21.15.5:0/1661199664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b701004e0 0x7f3b7010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.058+0000 7f3b763ff700 1 --2- 172.21.15.5:0/1661199664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b70105b60 0x7f3b70105f30 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.058+0000 7f3b763ff700 1 --2- 172.21.15.5:0/1661199664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.058+0000 7f3b763ff700 1 -- 172.21.15.5:0/1661199664 >> 172.21.15.5:0/1661199664 conn(0x7f3b700fb5f0 msgr2=0x7f3b700fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.058+0000 7f3b763ff700 1 -- 172.21.15.5:0/1661199664 shutdown_connections 2023-07-21T19:31:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.058+0000 7f3b763ff700 1 -- 172.21.15.5:0/1661199664 wait complete. 2023-07-21T19:31:30.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 Processor -- start 2023-07-21T19:31:30.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 -- start start 2023-07-21T19:31:30.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 0x7f3b70072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b70105b60 0x7f3b701a3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701a4180 con 0x7f3b701004e0 2023-07-21T19:31:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701a42c0 con 0x7f3b70105b60 2023-07-21T19:31:30.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.059+0000 7f3b763ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b701a4430 con 0x7f3b700ffb30 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 0x7f3b70072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45232/0 (socket says 172.21.15.5:45232) 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6ffff700 1 -- 172.21.15.5:0/27747874 learned_addr learned my addr 172.21.15.5:0/27747874 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b7499c700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b70105b60 0x7f3b701a3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 -- 172.21.15.5:0/27747874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 msgr2=0x7f3b700729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 -- 172.21.15.5:0/27747874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b70105b60 msgr2=0x7f3b701a3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b70105b60 0x7f3b701a3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 -- 172.21.15.5:0/27747874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b600095f0 con 0x7f3b701004e0 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.060+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 0x7f3b70072f30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400e9f0 tx=0x7f3b6400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.061+0000 7f3b6d7fa700 1 -- 172.21.15.5:0/27747874 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6400c9c0 con 0x7f3b701004e0 2023-07-21T19:31:30.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.061+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b701a4710 con 0x7f3b701004e0 2023-07-21T19:31:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.061+0000 7f3b6d7fa700 1 -- 172.21.15.5:0/27747874 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b64010430 con 0x7f3b701004e0 2023-07-21T19:31:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.061+0000 7f3b6d7fa700 1 -- 172.21.15.5:0/27747874 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b64018680 con 0x7f3b701004e0 2023-07-21T19:31:30.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.061+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b701a4c20 con 0x7f3b701004e0 2023-07-21T19:31:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.066+0000 7f3b6d7fa700 1 -- 172.21.15.5:0/27747874 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3b640105a0 con 0x7f3b701004e0 2023-07-21T19:31:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.066+0000 7f3b89cc7b80 1 Processor -- start 2023-07-21T19:31:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 -- start start 2023-07-21T19:31:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e32172040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e32172580 0x561e321729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e32077c50 con 0x561e31fce6b0 2023-07-21T19:31:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e320ca9e0 con 0x561e32172580 2023-07-21T19:31:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e320c3e70 con 0x561e321782d0 2023-07-21T19:31:30.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e32172040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b7499c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e32172580 0x561e321729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b7499c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56166/0 (socket says 172.21.15.5:56166) 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.067+0000 7f3b7499c700 1 -- 172.21.15.5:0/1598891747 learned_addr learned my addr 172.21.15.5:0/1598891747 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 -- 172.21.15.5:0/1598891747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e32172580 msgr2=0x561e321729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e32172580 0x561e321729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 -- 172.21.15.5:0/1598891747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 msgr2=0x561e32177d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 -- 172.21.15.5:0/1598891747 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e31ef1d80 con 0x561e321782d0 2023-07-21T19:31:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b7499c700 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6d7fa700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 0x7f3b580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e32172040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3b60000c00 tx=0x7f3b60005f00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2de715c204ef4276 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b6d7fa700 1 -- 172.21.15.5:0/27747874 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b64014070 con 0x7f3b701004e0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b7499c700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 0x7f3b580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.068+0000 7f3b567fc700 1 -- 172.21.15.5:0/1598891747 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6001d070 con 0x561e321782d0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b567fc700 1 -- 172.21.15.5:0/1598891747 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b6001e040 con 0x561e321782d0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b567fc700 1 -- 172.21.15.5:0/1598891747 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60004760 con 0x561e321782d0 2023-07-21T19:31:30.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/1598891747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 msgr2=0x561e32172040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b7499c700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 0x7f3b580a3770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007610 tx=0x7f3b6800b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e32172040 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f3b60000c00 tx=0x7f3b60005f00 comp rx=0 tx=0).stop 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/1598891747 shutdown_connections 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e32172580 0x561e321729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32177d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/1598891747 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e32172040 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.069+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/1598891747 >> 172.21.15.5:0/1598891747 conn(0x561e32167be0 msgr2=0x561e3216cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.070+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/1598891747 shutdown_connections 2023-07-21T19:31:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.070+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/1598891747 wait complete. 2023-07-21T19:31:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.070+0000 7f3b89cc7b80 1 -- start start 2023-07-21T19:31:30.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.070+0000 7f3b89cc7b80 1 Processor -- start 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3217bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e32063a60 con 0x561e32172580 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e32063760 con 0x561e321782d0 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e31f478c0 con 0x561e31fce6b0 2023-07-21T19:31:30.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b89cc7b80 10 client.0 _ll_register_callbacks cb 0x561e32061700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45254/0 (socket says 172.21.15.5:45254) 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b6ffff700 1 -- 172.21.15.5:0/3246062591 learned_addr learned my addr 172.21.15.5:0/3246062591 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.071+0000 7f3b7499c700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3217bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 -- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 msgr2=0x561e3217bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3217bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 -- 172.21.15.5:0/3246062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 msgr2=0x561e3217b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 -- 172.21.15.5:0/3246062591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b68011330 con 0x561e31fce6b0 2023-07-21T19:31:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006010 tx=0x7f3b6000b940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3a3d81564f28e3c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6001d030 con 0x561e31fce6b0 2023-07-21T19:31:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e3201d490 con 0x561e31fce6b0 2023-07-21T19:31:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b6000bb30 con 0x561e31fce6b0 2023-07-21T19:31:30.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b60004710 con 0x561e31fce6b0 2023-07-21T19:31:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b89cc7b80 10 client.29620 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.072+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f3b7010d220 con 0x561e31fce6b0 2023-07-21T19:31:30.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x561e3201d490 con 0x561e31fce6b0 2023-07-21T19:31:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b6000f950 con 0x561e31fce6b0 2023-07-21T19:31:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 10 client.29620 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 10 client.29620 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:30.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 20 client.29620 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e3218b700 0x561e3218db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 4 client.29620 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:30.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e320bce50 con 0x561e3218b700 2023-07-21T19:31:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.073+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e3218b700 0x561e3218db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.074+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e3218b700 0x561e3218db10 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f46391079815898 server_cookie=d67698bb2e73852d in_seq=0 out_seq=0 2023-07-21T19:31:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.074+0000 7f3b4f7fe700 10 client.29620 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:30.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/3246062591 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561e320bce50 con 0x561e3218b700 2023-07-21T19:31:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b4f7fe700 10 client.29620 handle_command_reply: tid=0 2023-07-21T19:31:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 client.29620 shutdown 2023-07-21T19:31:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 msgr2=0x561e3217e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f3b60006010 tx=0x7f3b6000b940 comp rx=0 tx=0).stop 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 shutdown_connections 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3217bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e31fce6b0 0x561e3217e4f0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e3218b700 0x561e3218db10 crc :-1 s=READY pgs=465 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/3246062591 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b4f7fe700 0 client.29620 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:30.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 >> 172.21.15.5:0/3246062591 conn(0x561e32167be0 msgr2=0x561e3216bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 shutdown_connections 2023-07-21T19:31:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.075+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/3246062591 wait complete. 2023-07-21T19:31:30.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.076+0000 7f3b89cc7b80 20 client.29620 trim_cache size 0 max 16384 2023-07-21T19:31:30.076 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:29 smithi101 ceph-mon[128136]: pgmap v755: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-07-21T19:31:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 Processor -- start 2023-07-21T19:31:30.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 -- start start 2023-07-21T19:31:30.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32182c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3216cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e3217cd20 con 0x561e31fce6b0 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e32176500 con 0x561e321782d0 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e32164b40 con 0x561e32172580 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.091+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32182c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b7499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3216cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:45258/0 (socket says 172.21.15.5:45258) 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6f7fe700 1 -- 172.21.15.5:0/458742334 learned_addr learned my addr 172.21.15.5:0/458742334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b7499c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3216cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50430/0 (socket says 172.21.15.5:50430) 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 -- 172.21.15.5:0/458742334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 msgr2=0x561e32178fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 -- 172.21.15.5:0/458742334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 msgr2=0x561e3216cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3216cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 -- 172.21.15.5:0/458742334 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e32172c20 con 0x561e31fce6b0 2023-07-21T19:31:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:30.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32182c60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000fce0 tx=0x7f3b60009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17949a35dc0b6ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b56ffd700 1 -- 172.21.15.5:0/458742334 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6001d040 con 0x561e31fce6b0 2023-07-21T19:31:30.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b56ffd700 1 -- 172.21.15.5:0/458742334 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b6000f460 con 0x561e31fce6b0 2023-07-21T19:31:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.092+0000 7f3b56ffd700 1 -- 172.21.15.5:0/458742334 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b60004990 con 0x561e31fce6b0 2023-07-21T19:31:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/458742334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 msgr2=0x561e32182c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32182c60 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000fce0 tx=0x7f3b60009ca0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/458742334 shutdown_connections 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e321782d0 0x561e3216cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e31fce6b0 0x561e32182c60 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/458742334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e32172580 0x561e32178fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/458742334 >> 172.21.15.5:0/458742334 conn(0x561e32167be0 msgr2=0x561e321788a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/458742334 shutdown_connections 2023-07-21T19:31:30.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.093+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/458742334 wait complete. 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 -- start start 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 Processor -- start 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e321de570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e321deab0 con 0x561e32172580 2023-07-21T19:31:30.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e321debf0 con 0x561e31fce6b0 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e321ded30 con 0x561e321782d0 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b89cc7b80 10 client.0 _ll_register_callbacks cb 0x7f3b68009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.094+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50436/0 (socket says 172.21.15.5:50436) 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 -- 172.21.15.5:0/655743679 learned_addr learned my addr 172.21.15.5:0/655743679 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:30.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b7499c700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e321de570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 -- 172.21.15.5:0/655743679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 msgr2=0x561e321de570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e321de570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 -- 172.21.15.5:0/655743679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 msgr2=0x561e3217f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 -- 172.21.15.5:0/655743679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b701a4c20 con 0x561e31fce6b0 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b6ffff700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000f800 tx=0x7f3b6000f830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=224336ff4c930927 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:30.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b4f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:31:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b6000bb30 con 0x561e31fce6b0 2023-07-21T19:31:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3b60000e40 con 0x561e31fce6b0 2023-07-21T19:31:30.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.095+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3b6001e040 con 0x561e31fce6b0 2023-07-21T19:31:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b6000bc90 con 0x561e31fce6b0 2023-07-21T19:31:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 10 client.29708 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x7f3b7010d220 con 0x561e31fce6b0 2023-07-21T19:31:30.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3b6001e040 con 0x561e31fce6b0 2023-07-21T19:31:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f3b6002c8a0 con 0x561e31fce6b0 2023-07-21T19:31:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 10 client.29708 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:30.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 10 client.29708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 20 client.29708 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e321dff70 0x561e321e2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:30.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 4 client.29708 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:31:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.096+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561e3207eff0 con 0x561e321dff70 2023-07-21T19:31:30.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.097+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e321dff70 0x561e321e2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.097+0000 7f3b6f7fe700 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e321dff70 0x561e321e2380 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3823062ca6498ce4 server_cookie=7cd102e408e74a2e in_seq=0 out_seq=0 2023-07-21T19:31:30.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.097+0000 7f3b4f7fe700 10 client.29708 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:30.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b4f7fe700 1 -- 172.21.15.5:0/655743679 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561e3207eff0 con 0x561e321dff70 2023-07-21T19:31:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b4f7fe700 10 client.29708 handle_command_reply: tid=0 2023-07-21T19:31:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 client.29708 shutdown 2023-07-21T19:31:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 msgr2=0x561e3217ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f3b6000f800 tx=0x7f3b6000f830 comp rx=0 tx=0).stop 2023-07-21T19:31:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 shutdown_connections 2023-07-21T19:31:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x561e321782d0 0x561e321de570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x561e31fce6b0 0x561e3217ee30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x561e321dff70 0x561e321e2380 crc :-1 s=READY pgs=466 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 --2- 172.21.15.5:0/655743679 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x561e32172580 0x561e3217f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b4f7fe700 0 client.29708 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 >> 172.21.15.5:0/655743679 conn(0x561e32167be0 msgr2=0x561e32179130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 shutdown_connections 2023-07-21T19:31:30.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.129+0000 7f3b89cc7b80 1 -- 172.21.15.5:0/655743679 wait complete. 2023-07-21T19:31:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.130+0000 7f3b89cc7b80 20 client.29708 trim_cache size 0 max 16384 2023-07-21T19:31:30.120 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:31:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:31:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:31:30.121 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:31:30.122 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.130+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 msgr2=0x7f3b580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.130+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 0x7f3b580a3770 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f3b68007610 tx=0x7f3b6800b040 comp rx=0 tx=0).stop 2023-07-21T19:31:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.130+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 msgr2=0x7f3b70072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.130+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 0x7f3b70072f30 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f3b6400e9f0 tx=0x7f3b6400ed00 comp rx=0 tx=0).stop 2023-07-21T19:31:30.123 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 shutdown_connections 2023-07-21T19:31:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3b70105b60 0x7f3b701a3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3b700ffb30 0x7f3b700729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3b580a12c0 0x7f3b580a3770 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 --2- 172.21.15.5:0/27747874 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3b701004e0 0x7f3b70072f30 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 >> 172.21.15.5:0/27747874 conn(0x7f3b700fb5f0 msgr2=0x7f3b700fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 shutdown_connections 2023-07-21T19:31:30.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:30.131+0000 7f3b763ff700 1 -- 172.21.15.5:0/27747874 wait complete. 2023-07-21T19:31:30.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:29 smithi005 ceph-mon[126460]: pgmap v755: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-07-21T19:31:30.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:29 smithi156 ceph-mon[128316]: pgmap v755: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.9 MiB/s wr, 947 op/s 2023-07-21T19:31:31.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:30 smithi101 ceph-mon[128136]: pgmap v756: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 794 op/s 2023-07-21T19:31:31.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:30 smithi005 ceph-mon[126460]: pgmap v756: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 794 op/s 2023-07-21T19:31:31.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:30 smithi156 ceph-mon[128316]: pgmap v756: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.6 MiB/s wr, 794 op/s 2023-07-21T19:31:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 300.0 s, 171.6 tps, lat 581.738 ms stddev 986.035 2023-07-21T19:31:34.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:33 smithi101 ceph-mon[128136]: pgmap v757: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 994 op/s 2023-07-21T19:31:34.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:33 smithi005 ceph-mon[126460]: pgmap v757: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 994 op/s 2023-07-21T19:31:34.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:33 smithi156 ceph-mon[128316]: pgmap v757: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 994 op/s 2023-07-21T19:31:35.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:34 smithi101 ceph-mon[128136]: pgmap v758: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2023-07-21T19:31:35.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:34 smithi005 ceph-mon[126460]: pgmap v758: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2023-07-21T19:31:35.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:34 smithi156 ceph-mon[128316]: pgmap v758: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.8 MiB/s wr, 764 op/s 2023-07-21T19:31:36.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 305.0 s, 257.4 tps, lat 338.559 ms stddev 98.438 2023-07-21T19:31:37.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:37 smithi101 ceph-mon[128136]: pgmap v759: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 711 op/s 2023-07-21T19:31:37.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:37 smithi005 ceph-mon[126460]: pgmap v759: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 711 op/s 2023-07-21T19:31:37.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:37 smithi156 ceph-mon[128316]: pgmap v759: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 711 op/s 2023-07-21T19:31:39.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:38 smithi101 ceph-mon[128136]: pgmap v760: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:31:39.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:38 smithi005 ceph-mon[126460]: pgmap v760: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:31:39.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:38 smithi156 ceph-mon[128316]: pgmap v760: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:31:41.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:41 smithi101 ceph-mon[128136]: pgmap v761: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 630 op/s 2023-07-21T19:31:41.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:41 smithi005 ceph-mon[126460]: pgmap v761: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 630 op/s 2023-07-21T19:31:41.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:41 smithi156 ceph-mon[128316]: pgmap v761: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.1 MiB/s wr, 630 op/s 2023-07-21T19:31:41.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 310.0 s, 145.6 tps, lat 27.499 ms stddev 9.592 2023-07-21T19:31:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.340+0000 7f01377ff700 1 -- 172.21.15.5:0/830936704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 msgr2=0x7f0130103900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.340+0000 7f01377ff700 1 --2- 172.21.15.5:0/830936704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 0x7f0130103900 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f0120009990 tx=0x7f0120009ca0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 -- 172.21.15.5:0/830936704 shutdown_connections 2023-07-21T19:31:43.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 --2- 172.21.15.5:0/830936704 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 0x7f01301042e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 --2- 172.21.15.5:0/830936704 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f01300fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 --2- 172.21.15.5:0/830936704 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 0x7f0130103900 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 -- 172.21.15.5:0/830936704 >> 172.21.15.5:0/830936704 conn(0x7f0130074680 msgr2=0x7f0130074a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 -- 172.21.15.5:0/830936704 shutdown_connections 2023-07-21T19:31:43.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.342+0000 7f01377ff700 1 -- 172.21.15.5:0/830936704 wait complete. 2023-07-21T19:31:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 Processor -- start 2023-07-21T19:31:43.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 -- start start 2023-07-21T19:31:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 0x7f013019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 0x7f01301a0b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301a1170 con 0x7f01300fe940 2023-07-21T19:31:43.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301a12e0 con 0x7f0130103e40 2023-07-21T19:31:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.344+0000 7f01377ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01301a14a0 con 0x7f01300ff250 2023-07-21T19:31:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.345+0000 7f013559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.345+0000 7f013559b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39982/0 (socket says 172.21.15.5:39982) 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.345+0000 7f013559b700 1 -- 172.21.15.5:0/2992616684 learned_addr learned my addr 172.21.15.5:0/2992616684 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.345+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 0x7f01301a0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 -- 172.21.15.5:0/2992616684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 msgr2=0x7f013019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 0x7f013019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 -- 172.21.15.5:0/2992616684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 msgr2=0x7f01301a0b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 0x7f01301a0b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 -- 172.21.15.5:0/2992616684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01200095f0 con 0x7f01300fe940 2023-07-21T19:31:43.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.346+0000 7f013559b700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d950 tx=0x7f012c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.351+0000 7f01267fc700 1 -- 172.21.15.5:0/2992616684 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c009b40 con 0x7f01300fe940 2023-07-21T19:31:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.351+0000 7f01267fc700 1 -- 172.21.15.5:0/2992616684 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f012c009ca0 con 0x7f01300fe940 2023-07-21T19:31:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.351+0000 7f01267fc700 1 -- 172.21.15.5:0/2992616684 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f012c00f620 con 0x7f01300fe940 2023-07-21T19:31:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.351+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01301a1780 con 0x7f01300fe940 2023-07-21T19:31:43.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.351+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01301a8fc0 con 0x7f01300fe940 2023-07-21T19:31:43.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.358+0000 7f014b1cbb80 1 Processor -- start 2023-07-21T19:31:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.359+0000 7f014b1cbb80 1 -- start start 2023-07-21T19:31:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x559860570040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605709d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860475c50 con 0x559860570580 2023-07-21T19:31:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598604c89e0 con 0x5598603cc6b0 2023-07-21T19:31:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598604c1e70 con 0x5598605762d0 2023-07-21T19:31:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f0134d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605709d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f0135d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f0135d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56344/0 (socket says 172.21.15.5:56344) 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.360+0000 7f0135d9c700 1 -- 172.21.15.5:0/2610740758 learned_addr learned my addr 172.21.15.5:0/2610740758 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 -- 172.21.15.5:0/2610740758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 msgr2=0x559860570040 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x559860570040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 -- 172.21.15.5:0/2610740758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 msgr2=0x5598605709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605709d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 -- 172.21.15.5:0/2610740758 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5598602efd80 con 0x5598603cc6b0 2023-07-21T19:31:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.361+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0128009f60 tx=0x7f0128009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96b728794aaa45c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.362+0000 7f011b7fe700 1 -- 172.21.15.5:0/2610740758 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128011070 con 0x5598603cc6b0 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.362+0000 7f011b7fe700 1 -- 172.21.15.5:0/2610740758 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f012800c0d0 con 0x5598603cc6b0 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.362+0000 7f011b7fe700 1 -- 172.21.15.5:0/2610740758 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128015be0 con 0x5598603cc6b0 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 -- 172.21.15.5:0/2610740758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 msgr2=0x559860575d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f0128009f60 tx=0x7f0128009f90 comp rx=0 tx=0).stop 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f01267fc700 1 -- 172.21.15.5:0/2992616684 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f012c00f780 con 0x7f01300fe940 2023-07-21T19:31:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 -- 172.21.15.5:0/2610740758 shutdown_connections 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605709d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860575d90 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/2610740758 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x559860570040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 -- 172.21.15.5:0/2610740758 >> 172.21.15.5:0/2610740758 conn(0x559860565be0 msgr2=0x55986056af10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 -- 172.21.15.5:0/2610740758 shutdown_connections 2023-07-21T19:31:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.363+0000 7f014b1cbb80 1 -- 172.21.15.5:0/2610740758 wait complete. 2023-07-21T19:31:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.364+0000 7f014b1cbb80 1 -- start start 2023-07-21T19:31:43.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.364+0000 7f014b1cbb80 1 Processor -- start 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605797b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860461a60 con 0x559860570580 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860461760 con 0x5598605762d0 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598603458c0 con 0x5598603cc6b0 2023-07-21T19:31:43.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.366+0000 7f014b1cbb80 10 client.0 _ll_register_callbacks cb 0x55986045f700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56350/0 (socket says 172.21.15.5:56350) 2023-07-21T19:31:43.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 -- 172.21.15.5:0/177518177 learned_addr learned my addr 172.21.15.5:0/177518177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f013559b700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605797b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0135d9c700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f01267fc700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 0x7f011c0a3810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 -- 172.21.15.5:0/177518177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 msgr2=0x55986057c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 -- 172.21.15.5:0/177518177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 msgr2=0x5598605797b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605797b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f0134d9a700 1 -- 172.21.15.5:0/177518177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0128009c10 con 0x5598605762d0 2023-07-21T19:31:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.367+0000 7f01267fc700 1 -- 172.21.15.5:0/2992616684 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f012c0cc820 con 0x7f01300fe940 2023-07-21T19:31:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 0x7f011c0a3810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f0134d9a700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0120009990 tx=0x7f012000bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f4b92f4580c4bfd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f0118ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:31:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120005450 con 0x5598605762d0 2023-07-21T19:31:43.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f01200055b0 con 0x5598605762d0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.368+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55986041b490 con 0x5598605762d0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0120004500 con 0x5598605762d0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f0135d9c700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 0x7f011c0a3810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0128015780 tx=0x7f0128009f90 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f014b1cbb80 10 client.29717 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5598602d8c00 con 0x5598605762d0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55986041b490 con 0x5598605762d0 2023-07-21T19:31:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.369+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0120003ad0 con 0x5598605762d0 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 10 client.29717 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 10 client.29717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 20 client.29717 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559860589c30 0x55986058c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 4 client.29717 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5598604bae50 con 0x559860589c30 2023-07-21T19:31:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.370+0000 7f013559b700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559860589c30 0x55986058c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.371+0000 7f013559b700 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559860589c30 0x55986058c040 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5374897142062b83 server_cookie=1c33b0ed9a7b4d61 in_seq=0 out_seq=0 2023-07-21T19:31:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.371+0000 7f0118ff9700 10 client.29717 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f0118ff9700 1 -- 172.21.15.5:0/177518177 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5598604bae50 con 0x559860589c30 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f0118ff9700 10 client.29717 handle_command_reply: tid=0 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 client.29717 shutdown 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 msgr2=0x559860579de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f0120009990 tx=0x7f012000bf90 comp rx=0 tx=0).stop 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 shutdown_connections 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x559860579de0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559860589c30 0x55986058c040 crc :-1 s=READY pgs=467 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/177518177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x5598605797b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 >> 172.21.15.5:0/177518177 conn(0x559860565be0 msgr2=0x559860569c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f0118ff9700 0 client.29717 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 shutdown_connections 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.372+0000 7f014b1cbb80 1 -- 172.21.15.5:0/177518177 wait complete. 2023-07-21T19:31:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.373+0000 7f014b1cbb80 20 client.29717 trim_cache size 0 max 16384 2023-07-21T19:31:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.401+0000 7f014b1cbb80 1 Processor -- start 2023-07-21T19:31:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.401+0000 7f014b1cbb80 1 -- start start 2023-07-21T19:31:43.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860570040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x559860580ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x55986056ac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860577d10 con 0x559860570580 2023-07-21T19:31:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559860574520 con 0x5598603cc6b0 2023-07-21T19:31:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55986057ab00 con 0x5598605762d0 2023-07-21T19:31:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f013559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860570040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f0135d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x55986056ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f013559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860570040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56356/0 (socket says 172.21.15.5:56356) 2023-07-21T19:31:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f013559b700 1 -- 172.21.15.5:0/3804281716 learned_addr learned my addr 172.21.15.5:0/3804281716 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.402+0000 7f0134d9a700 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x559860580ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 -- 172.21.15.5:0/3804281716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 msgr2=0x55986056ac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x55986056ac60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 -- 172.21.15.5:0/3804281716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 msgr2=0x559860570040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860570040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 -- 172.21.15.5:0/3804281716 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559860590c30 con 0x559860570580 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.403+0000 7f0134d9a700 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x559860580ca0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0120003ee0 tx=0x7f01200041d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5801e6e069b2ff44 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.404+0000 7f011bfff700 1 -- 172.21.15.5:0/3804281716 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120024e80 con 0x559860570580 2023-07-21T19:31:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.404+0000 7f011bfff700 1 -- 172.21.15.5:0/3804281716 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f012001e040 con 0x559860570580 2023-07-21T19:31:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.404+0000 7f011bfff700 1 -- 172.21.15.5:0/3804281716 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0120004500 con 0x559860570580 2023-07-21T19:31:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 -- 172.21.15.5:0/3804281716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 msgr2=0x559860580ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x559860580ca0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f0120003ee0 tx=0x7f01200041d0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 -- 172.21.15.5:0/3804281716 shutdown_connections 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598605762d0 0x55986056ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598603cc6b0 0x559860570040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/3804281716 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x559860580ca0 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 -- 172.21.15.5:0/3804281716 >> 172.21.15.5:0/3804281716 conn(0x559860565be0 msgr2=0x559860573a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 -- 172.21.15.5:0/3804281716 shutdown_connections 2023-07-21T19:31:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.405+0000 7f014b1cbb80 1 -- 172.21.15.5:0/3804281716 wait complete. 2023-07-21T19:31:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.406+0000 7f014b1cbb80 1 -- start start 2023-07-21T19:31:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.406+0000 7f014b1cbb80 1 Processor -- start 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x55986057f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598605ee000 con 0x559860570580 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598605ee140 con 0x5598605762d0 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5598605ee280 con 0x5598603cc6b0 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f014b1cbb80 10 client.0 _ll_register_callbacks cb 0x7f012000bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f0135d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f0135d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56372/0 (socket says 172.21.15.5:56372) 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f0135d9c700 1 -- 172.21.15.5:0/869245687 learned_addr learned my addr 172.21.15.5:0/869245687 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.407+0000 7f013559b700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.408+0000 7f0135d9c700 1 -- 172.21.15.5:0/869245687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 msgr2=0x55986057ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.408+0000 7f0135d9c700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.408+0000 7f0135d9c700 1 -- 172.21.15.5:0/869245687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 msgr2=0x55986057f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.408+0000 7f0135d9c700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x55986057f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.408+0000 7f0135d9c700 1 -- 172.21.15.5:0/869245687 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5598602d8c00 con 0x5598605762d0 2023-07-21T19:31:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.409+0000 7f0135d9c700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0128018040 tx=0x7f0128007040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e54fadec0f1023a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.409+0000 7f0118ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:31:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.410+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0128011030 con 0x5598605762d0 2023-07-21T19:31:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.410+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0128012070 con 0x5598605762d0 2023-07-21T19:31:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.410+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f012800c0d0 con 0x5598605762d0 2023-07-21T19:31:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.411+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0128004240 con 0x5598605762d0 2023-07-21T19:31:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.411+0000 7f014b1cbb80 10 client.29723 fetch_fsmap learned FSMap version 64 2023-07-21T19:31:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.411+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=4+}) v3 -- 0x5598603161f0 con 0x5598605762d0 2023-07-21T19:31:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.412+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f012800c0d0 con 0x5598605762d0 2023-07-21T19:31:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.412+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 64) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f012801e850 con 0x5598605762d0 2023-07-21T19:31:43.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 10 client.29723 fetch_fsmap finished waiting for FSMap version 64 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 10 client.29723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 20 client.29723 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5598605ef980 0x5598605f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 4 client.29723 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55986047cff0 con 0x5598605ef980 2023-07-21T19:31:43.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.413+0000 7f013559b700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5598605ef980 0x5598605f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.414+0000 7f013559b700 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5598605ef980 0x5598605f1d90 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb0f881c8526bace server_cookie=627df4d6ecb436c6 in_seq=0 out_seq=0 2023-07-21T19:31:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.414+0000 7f0118ff9700 10 client.29723 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.439+0000 7f0118ff9700 1 -- 172.21.15.5:0/869245687 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55986047cff0 con 0x5598605ef980 2023-07-21T19:31:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.439+0000 7f0118ff9700 10 client.29723 handle_command_reply: tid=0 2023-07-21T19:31:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.439+0000 7f014b1cbb80 1 client.29723 shutdown 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.439+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 msgr2=0x5598605edac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.439+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f0128018040 tx=0x7f0128007040 comp rx=0 tx=0).stop 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 shutdown_connections 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5598605762d0 0x5598605edac0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5598603cc6b0 0x55986057ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5598605ef980 0x5598605f1d90 crc :-1 s=READY pgs=468 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 --2- 172.21.15.5:0/869245687 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559860570580 0x55986057f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 >> 172.21.15.5:0/869245687 conn(0x559860565be0 msgr2=0x55986057b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f0118ff9700 0 client.29723 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 shutdown_connections 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 1 -- 172.21.15.5:0/869245687 wait complete. 2023-07-21T19:31:43.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.440+0000 7f014b1cbb80 20 client.29723 trim_cache size 0 max 16384 2023-07-21T19:31:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:31:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:31:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:31:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:31:43.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.441+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 msgr2=0x7f011c0a3810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.441+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 0x7f011c0a3810 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0128015780 tx=0x7f0128009f90 comp rx=0 tx=0).stop 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.441+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 msgr2=0x7f013019c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.441+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f012c00d950 tx=0x7f012c00dd10 comp rx=0 tx=0).stop 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 shutdown_connections 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0130103e40 0x7f01301a0b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f011c0a1360 0x7f011c0a3810 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f01300fe940 0x7f013019c190 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 --2- 172.21.15.5:0/2992616684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f01300ff250 0x7f013019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 >> 172.21.15.5:0/2992616684 conn(0x7f0130074680 msgr2=0x7f0130101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 shutdown_connections 2023-07-21T19:31:43.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:43.442+0000 7f01377ff700 1 -- 172.21.15.5:0/2992616684 wait complete. 2023-07-21T19:31:44.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:43 smithi101 ceph-mon[128136]: pgmap v762: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 880 op/s 2023-07-21T19:31:44.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:43 smithi005 ceph-mon[126460]: pgmap v762: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 880 op/s 2023-07-21T19:31:44.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:43 smithi156 ceph-mon[128316]: pgmap v762: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.0 MiB/s wr, 880 op/s 2023-07-21T19:31:45.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:44 smithi101 ceph-mon[128136]: pgmap v763: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 742 op/s 2023-07-21T19:31:45.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:44 smithi005 ceph-mon[126460]: pgmap v763: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 742 op/s 2023-07-21T19:31:45.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:44 smithi156 ceph-mon[128316]: pgmap v763: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.9 MiB/s wr, 742 op/s 2023-07-21T19:31:46.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 315.0 s, 215.8 tps, lat 959.095 ms stddev 2282.961 2023-07-21T19:31:47.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:47 smithi101 ceph-mon[128136]: pgmap v764: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2023-07-21T19:31:47.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:31:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:47 smithi005 ceph-mon[126460]: pgmap v764: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2023-07-21T19:31:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:31:47.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:47 smithi156 ceph-mon[128316]: pgmap v764: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 6.3 MiB/s wr, 695 op/s 2023-07-21T19:31:47.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:31:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:48 smithi101 ceph-mon[128136]: pgmap v765: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2023-07-21T19:31:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:48 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:31:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:48 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:31:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:48 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:31:49.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:48 smithi005 ceph-mon[126460]: pgmap v765: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2023-07-21T19:31:49.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:48 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:31:49.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:48 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:31:49.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:48 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:31:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:48 smithi156 ceph-mon[128316]: pgmap v765: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 9.2 MiB/s wr, 998 op/s 2023-07-21T19:31:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:48 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:31:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:48 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:31:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:48 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:31:51.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:51 smithi101 ceph-mon[128136]: pgmap v766: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2023-07-21T19:31:51.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:51 smithi005 ceph-mon[126460]: pgmap v766: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2023-07-21T19:31:51.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:51 smithi156 ceph-mon[128316]: pgmap v766: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.5 MiB/s wr, 616 op/s 2023-07-21T19:31:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 320.0 s, 250.2 tps, lat 403.485 ms stddev 58.462 2023-07-21T19:31:52.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:52 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:52.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:52 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:52.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:52 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:53.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:53 smithi101 ceph-mon[128136]: pgmap v767: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 917 op/s 2023-07-21T19:31:53.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:53 smithi005 ceph-mon[126460]: pgmap v767: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 917 op/s 2023-07-21T19:31:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:53 smithi156 ceph-mon[128316]: pgmap v767: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.4 MiB/s wr, 917 op/s 2023-07-21T19:31:54.387 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:31:54.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 -- 172.21.15.5:0/3049486437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 msgr2=0x7f57841105b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3049486437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f57841105b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f5778009990 tx=0x7f5778009ca0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 -- 172.21.15.5:0/3049486437 shutdown_connections 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3049486437 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5784110af0 0x7f5784112ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3049486437 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57840728b0 0x7f5784072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3049486437 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f57841105b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 -- 172.21.15.5:0/3049486437 >> 172.21.15.5:0/3049486437 conn(0x7f57840fb5e0 msgr2=0x7f57840fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 -- 172.21.15.5:0/3049486437 shutdown_connections 2023-07-21T19:31:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.722+0000 7f578a3ff700 1 -- 172.21.15.5:0/3049486437 wait complete. 2023-07-21T19:31:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.724+0000 7f578a3ff700 1 Processor -- start 2023-07-21T19:31:54.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.724+0000 7f578a3ff700 1 -- start start 2023-07-21T19:31:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f578419c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5784110af0 0x7f57841a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57841a1160 con 0x7f57840728b0 2023-07-21T19:31:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57841a12d0 con 0x7f5784110af0 2023-07-21T19:31:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578a3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57841a1490 con 0x7f57840731c0 2023-07-21T19:31:54.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f5783fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f57837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f578419c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f5783fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36824/0 (socket says 172.21.15.5:36824) 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f5783fff700 1 -- 172.21.15.5:0/3059496477 learned_addr learned my addr 172.21.15.5:0/3059496477 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.725+0000 7f578899c700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5784110af0 0x7f57841a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 -- 172.21.15.5:0/3059496477 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5784110af0 msgr2=0x7f57841a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5784110af0 0x7f57841a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 -- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 msgr2=0x7f578419c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 -- 172.21.15.5:0/3059496477 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57780095f0 con 0x7f57840731c0 2023-07-21T19:31:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f5783fff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.726+0000 7f57837fe700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f578419c6c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5778006010 tx=0x7f5778005740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.727+0000 7f57817fa700 1 -- 172.21.15.5:0/3059496477 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577801d070 con 0x7f57840731c0 2023-07-21T19:31:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.727+0000 7f57817fa700 1 -- 172.21.15.5:0/3059496477 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f57780173f0 con 0x7f57840731c0 2023-07-21T19:31:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.727+0000 7f57817fa700 1 -- 172.21.15.5:0/3059496477 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577800e640 con 0x7f57840731c0 2023-07-21T19:31:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.727+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57841a1710 con 0x7f57840731c0 2023-07-21T19:31:54.717 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.727+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57841a8dd0 con 0x7f57840731c0 2023-07-21T19:31:54.721 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.733+0000 7f57817fa700 1 -- 172.21.15.5:0/3059496477 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f577800e7a0 con 0x7f57840731c0 2023-07-21T19:31:54.722 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.735+0000 7f579de06b80 1 Processor -- start 2023-07-21T19:31:54.723 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 -- start start 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f57817fa700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 0x7f576c0a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2e2d0 0x5641d1e2e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e3beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1d34cb0 con 0x5641d1e2e2d0 2023-07-21T19:31:54.724 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1d87a40 con 0x5641d1e2ebe0 2023-07-21T19:31:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f579de06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1d80ed0 con 0x5641d1e39ad0 2023-07-21T19:31:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.736+0000 7f57817fa700 1 -- 172.21.15.5:0/3059496477 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57780cde00 con 0x7f57840731c0 2023-07-21T19:31:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51516/0 (socket says 172.21.15.5:51516) 2023-07-21T19:31:54.725 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 -- 172.21.15.5:0/3919946382 learned_addr learned my addr 172.21.15.5:0/3919946382 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:54.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f5783fff700 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2e2d0 0x5641d1e2e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 -- 172.21.15.5:0/3919946382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 msgr2=0x5641d1e3beb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:31:54.726 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e3beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 -- 172.21.15.5:0/3919946382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2e2d0 msgr2=0x5641d1e2e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2e2d0 0x5641d1e2e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 -- 172.21.15.5:0/3919946382 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641d1baede0 con 0x5641d1e2ebe0 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f57837fe700 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5778097970 tx=0x7f577800bfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4cf5df6210630dde server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.737+0000 7f578899c700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 0x7f576c0a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.727 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.738+0000 7f576a7fc700 1 -- 172.21.15.5:0/3919946382 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577809a860 con 0x5641d1e2ebe0 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.738+0000 7f576a7fc700 1 -- 172.21.15.5:0/3919946382 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5778003680 con 0x5641d1e2ebe0 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.738+0000 7f576a7fc700 1 -- 172.21.15.5:0/3919946382 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577809a3f0 con 0x5641d1e2ebe0 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f578899c700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 0x7f576c0a38c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f577c009ec0 tx=0x7f577c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 -- 172.21.15.5:0/3919946382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 msgr2=0x5641d1e39590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f5778097970 tx=0x7f577800bfb0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.728 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 -- 172.21.15.5:0/3919946382 shutdown_connections 2023-07-21T19:31:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e3beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2e2d0 0x5641d1e2e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 --2- 172.21.15.5:0/3919946382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2ebe0 0x5641d1e39590 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.729 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 -- 172.21.15.5:0/3919946382 >> 172.21.15.5:0/3919946382 conn(0x5641d1e24da0 msgr2=0x5641d1e2a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 -- 172.21.15.5:0/3919946382 shutdown_connections 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.739+0000 7f579de06b80 1 -- 172.21.15.5:0/3919946382 wait complete. 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.740+0000 7f579de06b80 1 -- start start 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.740+0000 7f579de06b80 1 Processor -- start 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e39ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.731 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e39ad0 0x5641d1e3a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1d20ac0 con 0x5641d1e39ad0 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1d207c0 con 0x5641d1e2e2d0 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1c04920 con 0x5641d1e2ebe0 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.741+0000 7f579de06b80 10 client.0 _ll_register_callbacks cb 0x5641d1d1e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:54.732 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:38842/0 (socket says 172.21.15.5:38842) 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 -- 172.21.15.5:0/1388791105 learned_addr learned my addr 172.21.15.5:0/1388791105 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f5783fff700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e39ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f578899c700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e39ad0 0x5641d1e3a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 -- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 msgr2=0x5641d1e39ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e39ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.733 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 -- 172.21.15.5:0/1388791105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e39ad0 msgr2=0x5641d1e3a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e39ad0 0x5641d1e3a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f57837fe700 1 -- 172.21.15.5:0/1388791105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57841a8dd0 con 0x5641d1e2ebe0 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.742+0000 7f5783fff700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e39ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f57837fe700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f57780a2d90 tx=0x7f5778005ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d83960b9dd34c76 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f57637fe700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577809a860 con 0x5641d1e2ebe0 2023-07-21T19:31:54.734 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57780a0050 con 0x5641d1e2ebe0 2023-07-21T19:31:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5641d1cda4f0 con 0x5641d1e2ebe0 2023-07-21T19:31:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f577809ef30 con 0x5641d1e2ebe0 2023-07-21T19:31:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f579de06b80 10 client.29647 fetch_fsmap learned FSMap version 65 2023-07-21T19:31:54.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.743+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5641d1b97c60 con 0x5641d1e2ebe0 2023-07-21T19:31:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5641d1cda4f0 con 0x5641d1e2ebe0 2023-07-21T19:31:54.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 65) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f5778017560 con 0x5641d1e2ebe0 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 10 client.29647 fetch_fsmap finished waiting for FSMap version 65 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 10 client.29647 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 20 client.29647 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e48d30 0x5641d1e4b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 4 client.29647 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5641d1d79eb0 con 0x5641d1e48d30 2023-07-21T19:31:54.737 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.744+0000 7f5783fff700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e48d30 0x5641d1e4b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.745+0000 7f5783fff700 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e48d30 0x5641d1e4b140 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3ffd8cf33ea446e server_cookie=ed124a7ce36e16a in_seq=0 out_seq=0 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.745+0000 7f57637fe700 10 client.29647 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f57637fe700 1 -- 172.21.15.5:0/1388791105 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5641d1d79eb0 con 0x5641d1e48d30 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f57637fe700 10 client.29647 handle_command_reply: tid=0 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f579de06b80 1 client.29647 shutdown 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 msgr2=0x5641d1e3a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.738 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f57780a2d90 tx=0x7f5778005ad0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 shutdown_connections 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.746+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e48d30 0x5641d1e4b140 crc :-1 s=READY pgs=469 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e39ad0 0x5641d1e3a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e39ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 --2- 172.21.15.5:0/1388791105 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3a3e0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 >> 172.21.15.5:0/1388791105 conn(0x5641d1e24da0 msgr2=0x5641d1e2a290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f57637fe700 0 client.29647 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:54.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 shutdown_connections 2023-07-21T19:31:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 1 -- 172.21.15.5:0/1388791105 wait complete. 2023-07-21T19:31:54.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.747+0000 7f579de06b80 20 client.29647 trim_cache size 0 max 16384 2023-07-21T19:31:54.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.778+0000 7f579de06b80 1 Processor -- start 2023-07-21T19:31:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.778+0000 7f579de06b80 1 -- start start 2023-07-21T19:31:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.778+0000 7f579de06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f579de06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e44690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f579de06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e32540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f579de06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e2bd20 con 0x5641d1e2ebe0 2023-07-21T19:31:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f579de06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e2f540 con 0x5641d1e2e2d0 2023-07-21T19:31:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f579de06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e38f70 con 0x5641d1e39ad0 2023-07-21T19:31:54.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f5783fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f5783fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51534/0 (socket says 172.21.15.5:51534) 2023-07-21T19:31:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.779+0000 7f5783fff700 1 -- 172.21.15.5:0/1812284164 learned_addr learned my addr 172.21.15.5:0/1812284164 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:54.768 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.780+0000 7f57837fe700 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e44690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.780+0000 7f5783fff700 1 -- 172.21.15.5:0/1812284164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 msgr2=0x5641d1e32540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.780+0000 7f5783fff700 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e32540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.780+0000 7f5783fff700 1 -- 172.21.15.5:0/1812284164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 msgr2=0x5641d1e44690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.780+0000 7f5783fff700 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e44690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.781+0000 7f5783fff700 1 -- 172.21.15.5:0/1812284164 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5641d1bd5250 con 0x5641d1e2e2d0 2023-07-21T19:31:54.769 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.781+0000 7f5783fff700 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5774005d90 tx=0x7f5774009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffecacc416d656f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.783+0000 7f576affd700 1 -- 172.21.15.5:0/1812284164 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774017070 con 0x5641d1e2e2d0 2023-07-21T19:31:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.783+0000 7f576affd700 1 -- 172.21.15.5:0/1812284164 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5774018040 con 0x5641d1e2e2d0 2023-07-21T19:31:54.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.783+0000 7f576affd700 1 -- 172.21.15.5:0/1812284164 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5774021510 con 0x5641d1e2e2d0 2023-07-21T19:31:54.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.784+0000 7f579de06b80 1 -- 172.21.15.5:0/1812284164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 msgr2=0x5641d1e3eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.784+0000 7f579de06b80 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f5774005d90 tx=0x7f5774009260 comp rx=0 tx=0).stop 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 -- 172.21.15.5:0/1812284164 shutdown_connections 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e39ad0 0x5641d1e32540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3eed0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 --2- 172.21.15.5:0/1812284164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e44690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 -- 172.21.15.5:0/1812284164 >> 172.21.15.5:0/1812284164 conn(0x5641d1e24da0 msgr2=0x5641d1e33340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.785+0000 7f579de06b80 1 -- 172.21.15.5:0/1812284164 shutdown_connections 2023-07-21T19:31:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.786+0000 7f579de06b80 1 -- 172.21.15.5:0/1812284164 wait complete. 2023-07-21T19:31:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.786+0000 7f579de06b80 1 -- start start 2023-07-21T19:31:54.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.787+0000 7f579de06b80 1 Processor -- start 2023-07-21T19:31:54.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3ec00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e3b990 con 0x5641d1e2ebe0 2023-07-21T19:31:54.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e3bb00 con 0x5641d1e39ad0 2023-07-21T19:31:54.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5641d1e3bcc0 con 0x5641d1e2e2d0 2023-07-21T19:31:54.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.788+0000 7f579de06b80 10 client.0 _ll_register_callbacks cb 0x5641d1d1e760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51536/0 (socket says 172.21.15.5:51536) 2023-07-21T19:31:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 -- 172.21.15.5:0/1916781883 learned_addr learned my addr 172.21.15.5:0/1916781883 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f57837fe700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 -- 172.21.15.5:0/1916781883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2e2d0 msgr2=0x5641d1e3ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 -- 172.21.15.5:0/1916781883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 msgr2=0x5641d1e3af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.789+0000 7f578899c700 1 -- 172.21.15.5:0/1916781883 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5774012a40 con 0x5641d1e39ad0 2023-07-21T19:31:54.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.790+0000 7f578899c700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f577c008ac0 tx=0x7f577c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b63fbbc74b98deb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:54.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.791+0000 7f57637fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:31:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.796+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577c026070 con 0x5641d1e39ad0 2023-07-21T19:31:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.796+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f577c00fe80 con 0x5641d1e39ad0 2023-07-21T19:31:54.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.796+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5778003680 con 0x5641d1e39ad0 2023-07-21T19:31:54.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.797+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=65) v2 ==== 24+0+0 (secure 0 0 0) 0x7f577c01c040 con 0x5641d1e39ad0 2023-07-21T19:31:54.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.799+0000 7f579de06b80 10 client.29741 fetch_fsmap learned FSMap version 65 2023-07-21T19:31:54.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.799+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=65,monmap=4+}) v3 -- 0x5641d1bad2b0 con 0x5641d1e39ad0 2023-07-21T19:31:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5778003680 con 0x5641d1e39ad0 2023-07-21T19:31:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 65) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f577c029020 con 0x5641d1e39ad0 2023-07-21T19:31:54.789 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 10 client.29741 fetch_fsmap finished waiting for FSMap version 65 2023-07-21T19:31:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 10 client.29741 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 20 client.29741 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e9e030 0x5641d1ea0440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:54.790 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 4 client.29741 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:31:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.802+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5641d1d3c050 con 0x5641d1e9e030 2023-07-21T19:31:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.803+0000 7f5783fff700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e9e030 0x5641d1ea0440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:54.791 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.804+0000 7f5783fff700 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e9e030 0x5641d1ea0440 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f860b5fb492d255 server_cookie=38ee75727b0a973 in_seq=0 out_seq=0 2023-07-21T19:31:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.804+0000 7f57637fe700 10 client.29741 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.804+0000 7f57637fe700 1 -- 172.21.15.5:0/1916781883 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5641d1d3c050 con 0x5641d1e9e030 2023-07-21T19:31:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.804+0000 7f57637fe700 10 client.29741 handle_command_reply: tid=0 2023-07-21T19:31:54.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.804+0000 7f579de06b80 1 client.29741 shutdown 2023-07-21T19:31:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 msgr2=0x5641d1e3b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f577c008ac0 tx=0x7f577c009920 comp rx=0 tx=0).stop 2023-07-21T19:31:54.793 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 shutdown_connections 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5641d1e39ad0 0x5641d1e3b450 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5641d1e2e2d0 0x5641d1e3ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5641d1e9e030 0x5641d1ea0440 crc :-1 s=READY pgs=470 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 --2- 172.21.15.5:0/1916781883 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5641d1e2ebe0 0x5641d1e3af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 >> 172.21.15.5:0/1916781883 conn(0x5641d1e24da0 msgr2=0x5641d1e39ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f57637fe700 0 client.29741 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.805+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 shutdown_connections 2023-07-21T19:31:54.794 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.806+0000 7f579de06b80 1 -- 172.21.15.5:0/1916781883 wait complete. 2023-07-21T19:31:54.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.806+0000 7f579de06b80 20 client.29741 trim_cache size 0 max 16384 2023-07-21T19:31:54.795 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:31:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:31:54.795 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:31:54.796 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:31:54.797 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:31:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 msgr2=0x7f576c0a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 0x7f576c0a38c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f577c009ec0 tx=0x7f577c00b040 comp rx=0 tx=0).stop 2023-07-21T19:31:54.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 msgr2=0x7f578419c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f578419c6c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f5778006010 tx=0x7f5778005740 comp rx=0 tx=0).stop 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 shutdown_connections 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5784110af0 0x7f57841a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f576c0a1410 0x7f576c0a38c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57840728b0 0x7f578419c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 --2- 172.21.15.5:0/3059496477 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f57840731c0 0x7f578419c6c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 >> 172.21.15.5:0/3059496477 conn(0x7f57840fb5e0 msgr2=0x7f57840fc240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:54.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.807+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 shutdown_connections 2023-07-21T19:31:54.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:31:54.808+0000 7f578a3ff700 1 -- 172.21.15.5:0/3059496477 wait complete. 2023-07-21T19:31:54.831 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:31:55.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:55 smithi101 ceph-mon[128136]: pgmap v768: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 731 op/s 2023-07-21T19:31:55.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:55 smithi005 ceph-mon[126460]: pgmap v768: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 731 op/s 2023-07-21T19:31:55.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:55 smithi156 ceph-mon[128316]: pgmap v768: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 731 op/s 2023-07-21T19:31:56.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 -- 172.21.15.5:0/3619170342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac067c10 msgr2=0x7f40ac067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 --2- 172.21.15.5:0/3619170342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac067c10 0x7f40ac067fe0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f40a80099c0 tx=0x7f40a8009cd0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 -- 172.21.15.5:0/3619170342 shutdown_connections 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 --2- 172.21.15.5:0/3619170342 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac104e50 0x7f40ac10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 --2- 172.21.15.5:0/3619170342 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac067c10 0x7f40ac067fe0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 --2- 172.21.15.5:0/3619170342 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac0685b0 0x7f40ac104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 -- 172.21.15.5:0/3619170342 >> 172.21.15.5:0/3619170342 conn(0x7f40ac074760 msgr2=0x7f40ac074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 -- 172.21.15.5:0/3619170342 shutdown_connections 2023-07-21T19:31:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.691+0000 7f40b37ff700 1 -- 172.21.15.5:0/3619170342 wait complete. 2023-07-21T19:31:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.692+0000 7f40b37ff700 1 Processor -- start 2023-07-21T19:31:56.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.693+0000 7f40b37ff700 1 -- start start 2023-07-21T19:31:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 0x7f40ac0fec40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 0x7f40ac0ff180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac104e50 0x7f40ac1a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1a4240 con 0x7f40ac104e50 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1a4380 con 0x7f40ac0685b0 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40ac1a44f0 con 0x7f40ac067c10 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 0x7f40ac0ff180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 0x7f40ac0fec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 0x7f40ac0ff180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49248/0 (socket says 172.21.15.5:49248) 2023-07-21T19:31:56.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b0d9a700 1 -- 172.21.15.5:0/645911582 learned_addr learned my addr 172.21.15.5:0/645911582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b159b700 1 -- 172.21.15.5:0/645911582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 msgr2=0x7f40ac0ff180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b159b700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 0x7f40ac0ff180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b159b700 1 -- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac104e50 msgr2=0x7f40ac1a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.694+0000 7f40b159b700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac104e50 0x7f40ac1a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40b159b700 1 -- 172.21.15.5:0/645911582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a80095f0 con 0x7f40ac067c10 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40b159b700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 0x7f40ac0fec40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a800f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40a27fc700 1 -- 172.21.15.5:0/645911582 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a801d070 con 0x7f40ac067c10 2023-07-21T19:31:56.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40ac195880 con 0x7f40ac067c10 2023-07-21T19:31:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac195d80 con 0x7f40ac067c10 2023-07-21T19:31:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40a27fc700 1 -- 172.21.15.5:0/645911582 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a8005050 con 0x7f40ac067c10 2023-07-21T19:31:56.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.695+0000 7f40a27fc700 1 -- 172.21.15.5:0/645911582 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8004a00 con 0x7f40ac067c10 2023-07-21T19:31:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.703+0000 7f40a27fc700 1 -- 172.21.15.5:0/645911582 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f40a80054a0 con 0x7f40ac067c10 2023-07-21T19:31:56.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.703+0000 7f40c7034b80 1 Processor -- start 2023-07-21T19:31:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.703+0000 7f40c7034b80 1 -- start start 2023-07-21T19:31:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b66d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b61580 0x55bd60b619d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60a66c50 con 0x55bd60b61580 2023-07-21T19:31:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60ab99e0 con 0x55bd609bd6b0 2023-07-21T19:31:56.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40c7034b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60ab2e70 con 0x55bd60b672d0 2023-07-21T19:31:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40b159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40b159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33204/0 (socket says 172.21.15.5:33204) 2023-07-21T19:31:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40b159b700 1 -- 172.21.15.5:0/2763449160 learned_addr learned my addr 172.21.15.5:0/2763449160 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b66d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.704+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b61580 0x55bd60b619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 -- 172.21.15.5:0/2763449160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 msgr2=0x55bd60b61040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 -- 172.21.15.5:0/2763449160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b61580 msgr2=0x55bd60b619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b61580 0x55bd60b619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 -- 172.21.15.5:0/2763449160 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd608e0d80 con 0x55bd609bd6b0 2023-07-21T19:31:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b159b700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:31:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.705+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b66d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e490 tx=0x7f40a400e7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29661d08b8606bb8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.706+0000 7f40977fe700 1 -- 172.21.15.5:0/2763449160 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4004db0 con 0x55bd609bd6b0 2023-07-21T19:31:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.706+0000 7f40977fe700 1 -- 172.21.15.5:0/2763449160 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a400f040 con 0x55bd609bd6b0 2023-07-21T19:31:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.706+0000 7f40977fe700 1 -- 172.21.15.5:0/2763449160 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a4004410 con 0x55bd609bd6b0 2023-07-21T19:31:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.706+0000 7f40a27fc700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 0x7f40980a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.706+0000 7f40a27fc700 1 -- 172.21.15.5:0/645911582 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f40a80cd9e0 con 0x7f40ac067c10 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 -- 172.21.15.5:0/2763449160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 msgr2=0x55bd60b66d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b66d90 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f40a400e490 tx=0x7f40a400e7a0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 -- 172.21.15.5:0/2763449160 shutdown_connections 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b61580 0x55bd60b619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b66d90 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 --2- 172.21.15.5:0/2763449160 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b672d0 0x55bd60b61040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 -- 172.21.15.5:0/2763449160 >> 172.21.15.5:0/2763449160 conn(0x55bd60b56be0 msgr2=0x55bd60b5bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40b159b700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 0x7f40980a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.707+0000 7f40c7034b80 1 -- 172.21.15.5:0/2763449160 shutdown_connections 2023-07-21T19:31:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.708+0000 7f40c7034b80 1 -- 172.21.15.5:0/2763449160 wait complete. 2023-07-21T19:31:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.708+0000 7f40c7034b80 1 -- start start 2023-07-21T19:31:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.708+0000 7f40c7034b80 1 Processor -- start 2023-07-21T19:31:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40b159b700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 0x7f40980a3870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40a809a7e0 tx=0x7f40a80d3530 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd60b61580 0x55bd60b6a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60a52a60 con 0x55bd60b672d0 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60a52760 con 0x55bd60b61580 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd609368c0 con 0x55bd609bd6b0 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.709+0000 7f40c7034b80 10 client.0 _ll_register_callbacks cb 0x55bd60a50700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.710+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.710+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:33216/0 (socket says 172.21.15.5:33216) 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.710+0000 7f40b1d9c700 1 -- 172.21.15.5:0/223076088 learned_addr learned my addr 172.21.15.5:0/223076088 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 -- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd60b61580 msgr2=0x55bd60b6a860 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd60b61580 0x55bd60b6a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 -- 172.21.15.5:0/223076088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 msgr2=0x55bd60b6afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 -- 172.21.15.5:0/223076088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40ac195d80 con 0x55bd609bd6b0 2023-07-21T19:31:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.711+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009760 tx=0x7f40a4004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b0f05455a0a2e3e6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.712+0000 7f4094ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.712+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40043e0 con 0x55bd609bd6b0 2023-07-21T19:31:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.712+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f40a4004800 con 0x55bd609bd6b0 2023-07-21T19:31:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.712+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55bd60a0c490 con 0x55bd609bd6b0 2023-07-21T19:31:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.713+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40a4003a00 con 0x55bd609bd6b0 2023-07-21T19:31:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.713+0000 7f40c7034b80 10 client.29659 fetch_fsmap learned FSMap version 66 2023-07-21T19:31:56.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.713+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x7f40a400e140 con 0x55bd609bd6b0 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.713+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55bd60a0c490 con 0x55bd609bd6b0 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 66) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f40a4007ad0 con 0x55bd609bd6b0 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 10 client.29659 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 10 client.29659 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 20 client.29659 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:56.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60b9cb80 0x55bd60b9ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 4 client.29659 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:31:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.714+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55bd60aabe50 con 0x55bd60b9cb80 2023-07-21T19:31:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.715+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60b9cb80 0x55bd60b9ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.715+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60b9cb80 0x55bd60b9ef90 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccf754e14ff1506b server_cookie=79cdabc6745c0010 in_seq=0 out_seq=0 2023-07-21T19:31:56.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.715+0000 7f4094ff9700 10 client.29659 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.716+0000 7f4094ff9700 1 -- 172.21.15.5:0/223076088 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55bd60aabe50 con 0x55bd60b9cb80 2023-07-21T19:31:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.716+0000 7f4094ff9700 10 client.29659 handle_command_reply: tid=0 2023-07-21T19:31:56.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.717+0000 7f40c7034b80 1 client.29659 shutdown 2023-07-21T19:31:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 msgr2=0x55bd60b6d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009760 tx=0x7f40a4004ad0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 shutdown_connections 2023-07-21T19:31:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60b9cb80 0x55bd60b9ef90 crc :-1 s=READY pgs=471 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd609bd6b0 0x55bd60b6d5e0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 --2- 172.21.15.5:0/223076088 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd60b61580 0x55bd60b6a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 >> 172.21.15.5:0/223076088 conn(0x55bd60b56be0 msgr2=0x55bd60b5ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.720+0000 7f4094ff9700 0 client.29659 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:56.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.721+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 shutdown_connections 2023-07-21T19:31:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.721+0000 7f40c7034b80 1 -- 172.21.15.5:0/223076088 wait complete. 2023-07-21T19:31:56.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.722+0000 7f40c7034b80 20 client.29659 trim_cache size 0 max 16384 2023-07-21T19:31:56.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.747+0000 7f40c7034b80 1 Processor -- start 2023-07-21T19:31:56.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.747+0000 7f40c7034b80 1 -- start start 2023-07-21T19:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60b82c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6d110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60b66a50 con 0x55bd60b672d0 2023-07-21T19:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60b676a0 con 0x55bd609bd6b0 2023-07-21T19:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40c7034b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60b6bd20 con 0x55bd60b61580 2023-07-21T19:31:56.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.748+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49286/0 (socket says 172.21.15.5:49286) 2023-07-21T19:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 -- 172.21.15.5:0/3321727037 learned_addr learned my addr 172.21.15.5:0/3321727037 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60b82c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 -- 172.21.15.5:0/3321727037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 msgr2=0x55bd60b82c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60b82c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 -- 172.21.15.5:0/3321727037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 msgr2=0x55bd60b6d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.749+0000 7f40b1d9c700 1 -- 172.21.15.5:0/3321727037 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55bd608c9c00 con 0x55bd609bd6b0 2023-07-21T19:31:56.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.750+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009410 tx=0x7f40a40096c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25d375aba457702c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.750+0000 7f4097fff700 1 -- 172.21.15.5:0/3321727037 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40043c0 con 0x55bd609bd6b0 2023-07-21T19:31:56.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.750+0000 7f4097fff700 1 -- 172.21.15.5:0/3321727037 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a4003e90 con 0x55bd609bd6b0 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.750+0000 7f4097fff700 1 -- 172.21.15.5:0/3321727037 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a40078c0 con 0x55bd609bd6b0 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.751+0000 7f40c7034b80 1 -- 172.21.15.5:0/3321727037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 msgr2=0x55bd60b61040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.751+0000 7f40c7034b80 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009410 tx=0x7f40a40096c0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.752+0000 7f40c7034b80 1 -- 172.21.15.5:0/3321727037 shutdown_connections 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.752+0000 7f40c7034b80 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60b6d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.752+0000 7f40c7034b80 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60b61040 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.752+0000 7f40c7034b80 1 --2- 172.21.15.5:0/3321727037 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60b82c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.752+0000 7f40c7034b80 1 -- 172.21.15.5:0/3321727037 >> 172.21.15.5:0/3321727037 conn(0x55bd60b56be0 msgr2=0x55bd60b64ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.754+0000 7f40c7034b80 1 -- 172.21.15.5:0/3321727037 shutdown_connections 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.754+0000 7f40c7034b80 1 -- 172.21.15.5:0/3321727037 wait complete. 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.754+0000 7f40c7034b80 1 -- start start 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 Processor -- start 2023-07-21T19:31:56.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60be4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60be4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60be5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60befb00 con 0x55bd60b672d0 2023-07-21T19:31:56.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60befc40 con 0x55bd609bd6b0 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55bd60befe00 con 0x55bd60b61580 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.755+0000 7f40c7034b80 10 client.0 _ll_register_callbacks cb 0x7f409c00b8a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.756+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60be4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.756+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60be5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.756+0000 7f40b1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60be5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54040/0 (socket says 172.21.15.5:54040) 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.756+0000 7f40b1d9c700 1 -- 172.21.15.5:0/451894454 learned_addr learned my addr 172.21.15.5:0/451894454 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:31:56.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.756+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60be4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 -- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 msgr2=0x55bd60be4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60be4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 -- 172.21.15.5:0/451894454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 msgr2=0x55bd60be5410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60be5410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 -- 172.21.15.5:0/451894454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a400e140 con 0x55bd60b61580 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f40b0d9a700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60be4ed0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55bd60b9f830 tx=0x7f409c01cd50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c76b8075a7f644b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f4094ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.757+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c016040 con 0x55bd60b61580 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.758+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(124..124 src has 1..124) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f409c01c580 con 0x55bd60b61580 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.758+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40a400f040 con 0x55bd60b61580 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.758+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f409c005020 con 0x55bd60b61580 2023-07-21T19:31:56.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.758+0000 7f40c7034b80 10 client.29665 fetch_fsmap learned FSMap version 66 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.758+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x55bd609071f0 con 0x55bd60b61580 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f40a400f040 con 0x55bd60b61580 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 66) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f409c003680 con 0x55bd60b61580 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 10 client.29665 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 10 client.29665 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 20 client.29665 populate_metadata read hostname 'smithi005' 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60bf1620 0x55bd60bf3a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:31:56.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 4 client.29665 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.759+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55bd60a6dff0 con 0x55bd60bf1620 2023-07-21T19:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.760+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60bf1620 0x55bd60bf3a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.760+0000 7f40b1d9c700 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60bf1620 0x55bd60bf3a30 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce3304da363e5903 server_cookie=669b7dadba43ce25 in_seq=0 out_seq=0 2023-07-21T19:31:56.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.761+0000 7f4094ff9700 10 client.29665 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.789+0000 7f4094ff9700 1 -- 172.21.15.5:0/451894454 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55bd60a6dff0 con 0x55bd60bf1620 2023-07-21T19:31:56.777 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.789+0000 7f4094ff9700 10 client.29665 handle_command_reply: tid=0 2023-07-21T19:31:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.789+0000 7f40c7034b80 1 client.29665 shutdown 2023-07-21T19:31:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 msgr2=0x55bd60be4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60be4ed0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x55bd60b9f830 tx=0x7f409c01cd50 comp rx=0 tx=0).stop 2023-07-21T19:31:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 shutdown_connections 2023-07-21T19:31:56.778 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55bd60bf1620 0x55bd60bf3a30 crc :-1 s=READY pgs=472 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55bd60b672d0 0x55bd60be5410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55bd609bd6b0 0x55bd60be4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 --2- 172.21.15.5:0/451894454 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55bd60b61580 0x55bd60be4ed0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 >> 172.21.15.5:0/451894454 conn(0x55bd60b56be0 msgr2=0x55bd60b6a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f4094ff9700 0 client.29665 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:31:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 shutdown_connections 2023-07-21T19:31:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.790+0000 7f40c7034b80 1 -- 172.21.15.5:0/451894454 wait complete. 2023-07-21T19:31:56.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.791+0000 7f40c7034b80 20 client.29665 trim_cache size 0 max 16384 2023-07-21T19:31:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:31:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:31:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:31:56.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:31:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.791+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 msgr2=0x7f40980a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.791+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 0x7f40980a3870 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f40a809a7e0 tx=0x7f40a80d3530 comp rx=0 tx=0).stop 2023-07-21T19:31:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.791+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 msgr2=0x7f40ac0fec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:31:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.791+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 0x7f40ac0fec40 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f40a8000c00 tx=0x7f40a800f690 comp rx=0 tx=0).stop 2023-07-21T19:31:56.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 shutdown_connections 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f40980a13c0 0x7f40980a3870 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f40ac104e50 0x7f40ac1a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f40ac067c10 0x7f40ac0fec40 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 --2- 172.21.15.5:0/645911582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f40ac0685b0 0x7f40ac0ff180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.792+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 >> 172.21.15.5:0/645911582 conn(0x7f40ac074760 msgr2=0x7f40ac0fc6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.793+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 shutdown_connections 2023-07-21T19:31:56.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:31:56.795+0000 7f40b37ff700 1 -- 172.21.15.5:0/645911582 wait complete. 2023-07-21T19:31:56.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 325.0 s, 246.4 tps, lat 397.127 ms stddev 104.797 2023-07-21T19:31:57.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:57 smithi005 ceph-mon[126460]: pgmap v769: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-07-21T19:31:57.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:57 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:57.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:57 smithi156 ceph-mon[128316]: pgmap v769: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-07-21T19:31:57.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:57 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:58.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:57 smithi101 ceph-mon[128136]: pgmap v769: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 668 op/s 2023-07-21T19:31:58.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:57 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:31:59.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:31:58 smithi101 ceph-mon[128136]: pgmap v770: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-07-21T19:31:59.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:31:58 smithi005 ceph-mon[126460]: pgmap v770: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-07-21T19:31:59.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:31:58 smithi156 ceph-mon[128316]: pgmap v770: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 1.03k op/s 2023-07-21T19:32:01.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:01 smithi005 ceph-mon[126460]: pgmap v771: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-07-21T19:32:01.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:01 smithi156 ceph-mon[128316]: pgmap v771: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-07-21T19:32:01.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 330.0 s, 205.0 tps, lat 485.341 ms stddev 191.703 2023-07-21T19:32:02.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:01 smithi101 ceph-mon[128136]: pgmap v771: 97 pgs: 97 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.0 MiB/s wr, 732 op/s 2023-07-21T19:32:04.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:03 smithi101 ceph-mon[128136]: pgmap v772: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-07-21T19:32:04.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:03 smithi005 ceph-mon[126460]: pgmap v772: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-07-21T19:32:04.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:03 smithi156 ceph-mon[128316]: pgmap v772: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.06k op/s 2023-07-21T19:32:06.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:05 smithi101 ceph-mon[128136]: pgmap v773: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 803 op/s 2023-07-21T19:32:06.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:05 smithi005 ceph-mon[126460]: pgmap v773: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 803 op/s 2023-07-21T19:32:06.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:05 smithi156 ceph-mon[128316]: pgmap v773: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.5 MiB/s wr, 803 op/s 2023-07-21T19:32:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 335.0 s, 156.4 tps, lat 620.197 ms stddev 185.744 2023-07-21T19:32:07.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:06 smithi101 ceph-mon[128136]: pgmap v774: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 739 op/s 2023-07-21T19:32:07.149 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:06 smithi005 ceph-mon[126460]: pgmap v774: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 739 op/s 2023-07-21T19:32:07.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:06 smithi156 ceph-mon[128316]: pgmap v774: 97 pgs: 97 active+clean; 14 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.9 MiB/s wr, 739 op/s 2023-07-21T19:32:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:07 smithi101 ceph-mon[128136]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:32:08.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:07 smithi005 ceph-mon[126460]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:32:08.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:07 smithi156 ceph-mon[128316]: osdmap e125: 12 total, 12 up, 12 in 2023-07-21T19:32:09.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:08 smithi101 ceph-mon[128136]: pgmap v776: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:09.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:08 smithi005 ceph-mon[126460]: pgmap v776: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:09.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:08 smithi156 ceph-mon[128316]: pgmap v776: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:10.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 -- 172.21.15.5:0/287493199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064067c10 msgr2=0x7f8064067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 --2- 172.21.15.5:0/287493199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064067c10 0x7f8064067fe0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f80540099c0 tx=0x7f8054009cd0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 -- 172.21.15.5:0/287493199 shutdown_connections 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 --2- 172.21.15.5:0/287493199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064104e50 0x7f806410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 --2- 172.21.15.5:0/287493199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064067c10 0x7f8064067fe0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 --2- 172.21.15.5:0/287493199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 -- 172.21.15.5:0/287493199 >> 172.21.15.5:0/287493199 conn(0x7f8064074760 msgr2=0x7f8064074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 -- 172.21.15.5:0/287493199 shutdown_connections 2023-07-21T19:32:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.037+0000 7f80699ff700 1 -- 172.21.15.5:0/287493199 wait complete. 2023-07-21T19:32:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.039+0000 7f80699ff700 1 Processor -- start 2023-07-21T19:32:10.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.039+0000 7f80699ff700 1 -- start start 2023-07-21T19:32:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064067c10 0x7f8064197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064104e50 0x7f806419c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419cf20 con 0x7f80640685b0 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419d090 con 0x7f8064104e50 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.040+0000 7f80699ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f806419d250 con 0x7f8064067c10 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064104e50 0x7f806419c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57390/0 (socket says 172.21.15.5:57390) 2023-07-21T19:32:10.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 -- 172.21.15.5:0/3609493296 learned_addr learned my addr 172.21.15.5:0/3609493296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f8062ffd700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064067c10 0x7f8064197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 -- 172.21.15.5:0/3609493296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064067c10 msgr2=0x7f8064197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064067c10 0x7f8064197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 -- 172.21.15.5:0/3609493296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064104e50 msgr2=0x7f806419c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064104e50 0x7f806419c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.041+0000 7f80627fc700 1 -- 172.21.15.5:0/3609493296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80540095f0 con 0x7f80640685b0 2023-07-21T19:32:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80627fc700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f805800e940 tx=0x7f805800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80689fd700 1 -- 172.21.15.5:0/3609493296 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058009730 con 0x7f80640685b0 2023-07-21T19:32:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f806419d530 con 0x7f80640685b0 2023-07-21T19:32:10.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80641a4d70 con 0x7f80640685b0 2023-07-21T19:32:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80689fd700 1 -- 172.21.15.5:0/3609493296 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f805800ce70 con 0x7f80640685b0 2023-07-21T19:32:10.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.042+0000 7f80689fd700 1 -- 172.21.15.5:0/3609493296 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8058018420 con 0x7f80640685b0 2023-07-21T19:32:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.047+0000 7f80689fd700 1 -- 172.21.15.5:0/3609493296 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f805800c9e0 con 0x7f80640685b0 2023-07-21T19:32:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.048+0000 7f807d249b80 1 Processor -- start 2023-07-21T19:32:10.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.048+0000 7f807d249b80 1 -- start start 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.048+0000 7f807d249b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f807d249b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f807d249b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354b19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f807d249b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f353b6c50 con 0x562f3530d6b0 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f807d249b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354099e0 con 0x562f354b1580 2023-07-21T19:32:10.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f807d249b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f35402e70 con 0x562f354b72d0 2023-07-21T19:32:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f8062ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f8062ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49096/0 (socket says 172.21.15.5:49096) 2023-07-21T19:32:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f8062ffd700 1 -- 172.21.15.5:0/581894020 learned_addr learned my addr 172.21.15.5:0/581894020 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f80627fc700 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354b19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.049+0000 7f80637fe700 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 -- 172.21.15.5:0/581894020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 msgr2=0x562f354b19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354b19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 -- 172.21.15.5:0/581894020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 msgr2=0x562f354b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 -- 172.21.15.5:0/581894020 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f35230d80 con 0x562f354b72d0 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f80689fd700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 0x7f804c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f8062ffd700 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80540099c0 tx=0x7f8054005c70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e741fb979cddb7b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f80689fd700 1 -- 172.21.15.5:0/3609493296 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8058014070 con 0x7f80640685b0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.050+0000 7f80637fe700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 0x7f804c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f8049ffb700 1 -- 172.21.15.5:0/581894020 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805400ba10 con 0x562f354b72d0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f8049ffb700 1 -- 172.21.15.5:0/581894020 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f805401e040 con 0x562f354b72d0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f8049ffb700 1 -- 172.21.15.5:0/581894020 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054003b20 con 0x562f354b72d0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f807d249b80 1 -- 172.21.15.5:0/581894020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 msgr2=0x562f354b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f807d249b80 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f80540099c0 tx=0x7f8054005c70 comp rx=0 tx=0).stop 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.051+0000 7f80637fe700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 0x7f804c0a38b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f805c007500 tx=0x7f805c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 -- 172.21.15.5:0/581894020 shutdown_connections 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354b19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 --2- 172.21.15.5:0/581894020 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354b1040 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 -- 172.21.15.5:0/581894020 >> 172.21.15.5:0/581894020 conn(0x562f354a6be0 msgr2=0x562f354abf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 -- 172.21.15.5:0/581894020 shutdown_connections 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.052+0000 7f807d249b80 1 -- 172.21.15.5:0/581894020 wait complete. 2023-07-21T19:32:10.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.053+0000 7f807d249b80 1 -- start start 2023-07-21T19:32:10.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.053+0000 7f807d249b80 1 Processor -- start 2023-07-21T19:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354ba800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354bae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f353a2a60 con 0x562f354b1580 2023-07-21T19:32:10.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f353a2760 con 0x562f3530d6b0 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f352868c0 con 0x562f354b72d0 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f807d249b80 10 client.0 _ll_register_callbacks cb 0x562f353a0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f8062ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f8062ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35210/0 (socket says 172.21.15.5:35210) 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f8062ffd700 1 -- 172.21.15.5:0/3384622050 learned_addr learned my addr 172.21.15.5:0/3384622050 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.054+0000 7f80637fe700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354bae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 -- 172.21.15.5:0/3384622050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 msgr2=0x562f354bae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354bae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f80627fc700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354ba800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 -- 172.21.15.5:0/3384622050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 msgr2=0x562f354ba800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354ba800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 -- 172.21.15.5:0/3384622050 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f806410d210 con 0x562f3530d6b0 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f80627fc700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8062ffd700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8054009990 tx=0x7f805400bca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf8f4f77b51a325f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.055+0000 7f8042ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:10.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8054005600 con 0x562f3530d6b0 2023-07-21T19:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562f3535c490 con 0x562f3530d6b0 2023-07-21T19:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8054003760 con 0x562f3530d6b0 2023-07-21T19:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8054003920 con 0x562f3530d6b0 2023-07-21T19:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f807d249b80 10 client.29768 fetch_fsmap learned FSMap version 66 2023-07-21T19:32:10.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.056+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x562f35219c00 con 0x562f3530d6b0 2023-07-21T19:32:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.057+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562f3535c490 con 0x562f3530d6b0 2023-07-21T19:32:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.057+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 66) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f805400fb60 con 0x562f3530d6b0 2023-07-21T19:32:10.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 10 client.29768 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 10 client.29768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 20 client.29768 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354d2f10 0x562f354d5320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 4 client.29768 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562f353fbe50 con 0x562f354d2f10 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.058+0000 7f80627fc700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354d2f10 0x562f354d5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.059+0000 7f80627fc700 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354d2f10 0x562f354d5320 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fa6f71a16b86c78 server_cookie=d897a54a356e55b2 in_seq=0 out_seq=0 2023-07-21T19:32:10.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.059+0000 7f8042ffd700 10 client.29768 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.060+0000 7f8042ffd700 1 -- 172.21.15.5:0/3384622050 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562f353fbe50 con 0x562f354d2f10 2023-07-21T19:32:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.060+0000 7f8042ffd700 10 client.29768 handle_command_reply: tid=0 2023-07-21T19:32:10.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.061+0000 7f807d249b80 1 client.29768 shutdown 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.061+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 msgr2=0x562f354bd580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.061+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f8054009990 tx=0x7f805400bca0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 shutdown_connections 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354b72d0 0x562f354bae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f3530d6b0 0x562f354bd580 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354d2f10 0x562f354d5320 crc :-1 s=READY pgs=473 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 --2- 172.21.15.5:0/3384622050 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354ba800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 >> 172.21.15.5:0/3384622050 conn(0x562f354a6be0 msgr2=0x562f354aac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f8042ffd700 0 client.29768 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 shutdown_connections 2023-07-21T19:32:10.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.062+0000 7f807d249b80 1 -- 172.21.15.5:0/3384622050 wait complete. 2023-07-21T19:32:10.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.063+0000 7f807d249b80 20 client.29768 trim_cache size 0 max 16384 2023-07-21T19:32:10.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.096+0000 7f807d249b80 1 Processor -- start 2023-07-21T19:32:10.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.099+0000 7f807d249b80 1 -- start start 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354a6be0 0x562f354ca4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354a8d70 con 0x562f3530d6b0 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354b6a00 con 0x562f354b1580 2023-07-21T19:32:10.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f807d249b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354bb790 con 0x562f354a6be0 2023-07-21T19:32:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f80637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f80637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35220/0 (socket says 172.21.15.5:35220) 2023-07-21T19:32:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.100+0000 7f80637fe700 1 -- 172.21.15.5:0/664488582 learned_addr learned my addr 172.21.15.5:0/664488582 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.101+0000 7f80637fe700 1 -- 172.21.15.5:0/664488582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354a6be0 msgr2=0x562f354ca4b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:32:10.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.101+0000 7f80637fe700 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354a6be0 0x562f354ca4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.101+0000 7f80637fe700 1 -- 172.21.15.5:0/664488582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 msgr2=0x562f354b7fb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:32:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.101+0000 7f80637fe700 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b7fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.101+0000 7f80637fe700 1 -- 172.21.15.5:0/664488582 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562f354d9f70 con 0x562f354b1580 2023-07-21T19:32:10.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.103+0000 7f80637fe700 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f805c007e40 tx=0x7f805c016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c2da4f130994ecda server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.106+0000 7f804a7fc700 1 -- 172.21.15.5:0/664488582 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c00e040 con 0x562f354b1580 2023-07-21T19:32:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.106+0000 7f804a7fc700 1 -- 172.21.15.5:0/664488582 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f805c00f040 con 0x562f354b1580 2023-07-21T19:32:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.106+0000 7f804a7fc700 1 -- 172.21.15.5:0/664488582 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f805c0243f0 con 0x562f354b1580 2023-07-21T19:32:10.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.107+0000 7f807d249b80 1 -- 172.21.15.5:0/664488582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 msgr2=0x562f354bd120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.107+0000 7f807d249b80 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f805c007e40 tx=0x7f805c016000 comp rx=0 tx=0).stop 2023-07-21T19:32:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.108+0000 7f807d249b80 1 -- 172.21.15.5:0/664488582 shutdown_connections 2023-07-21T19:32:10.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.108+0000 7f807d249b80 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354b1580 0x562f354bd120 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.108+0000 7f807d249b80 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f354a6be0 0x562f354ca4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.108+0000 7f807d249b80 1 --2- 172.21.15.5:0/664488582 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f3530d6b0 0x562f354b7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.108+0000 7f807d249b80 1 -- 172.21.15.5:0/664488582 >> 172.21.15.5:0/664488582 conn(0x562f354cefc0 msgr2=0x562f354b78a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.109+0000 7f807d249b80 1 -- 172.21.15.5:0/664488582 shutdown_connections 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.109+0000 7f807d249b80 1 -- 172.21.15.5:0/664488582 wait complete. 2023-07-21T19:32:10.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.109+0000 7f807d249b80 1 -- start start 2023-07-21T19:32:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.110+0000 7f807d249b80 1 Processor -- start 2023-07-21T19:32:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.110+0000 7f807d249b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 0x562f354b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.110+0000 7f807d249b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354a6be0 0x562f354b3f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.098 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.110+0000 7f807d249b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354b2790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.111+0000 7f807d249b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354b4510 con 0x562f354b1580 2023-07-21T19:32:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.111+0000 7f807d249b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354b2cd0 con 0x562f354a6be0 2023-07-21T19:32:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.111+0000 7f807d249b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562f354b2e90 con 0x562f3530d6b0 2023-07-21T19:32:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.111+0000 7f807d249b80 10 client.0 _ll_register_callbacks cb 0x7f80580d0370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:10.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.112+0000 7f8062ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 0x562f354b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.113+0000 7f80637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354b2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.113+0000 7f80637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354b2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57422/0 (socket says 172.21.15.5:57422) 2023-07-21T19:32:10.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.113+0000 7f80637fe700 1 -- 172.21.15.5:0/1209889763 learned_addr learned my addr 172.21.15.5:0/1209889763 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.113+0000 7f80627fc700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354a6be0 0x562f354b3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 -- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354a6be0 msgr2=0x562f354b3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354a6be0 0x562f354b3f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 -- 172.21.15.5:0/1209889763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 msgr2=0x562f354b2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354b2790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 -- 172.21.15.5:0/1209889763 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f805c007b20 con 0x562f3530d6b0 2023-07-21T19:32:10.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.114+0000 7f8062ffd700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 0x562f354b39d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f80540098e0 tx=0x7f8054005610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87f4c6618e491da server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:10.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.115+0000 7f8042ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:32:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.117+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80540235e0 con 0x562f3530d6b0 2023-07-21T19:32:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.117+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(125..125 src has 1..125) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f8054000e40 con 0x562f3530d6b0 2023-07-21T19:32:10.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.118+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f805401e040 con 0x562f3530d6b0 2023-07-21T19:32:10.106 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.118+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=66) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8054005c00 con 0x562f3530d6b0 2023-07-21T19:32:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.120+0000 7f807d249b80 10 client.29680 fetch_fsmap learned FSMap version 66 2023-07-21T19:32:10.107 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.120+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=66,monmap=4+}) v3 -- 0x562f352571f0 con 0x562f3530d6b0 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8054044030 con 0x562f3530d6b0 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 66) v1 ==== 7478+0+0 (secure 0 0 0) 0x7f8054003760 con 0x562f3530d6b0 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 10 client.29680 fetch_fsmap finished waiting for FSMap version 66 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 10 client.29680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 20 client.29680 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:10.109 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354b72d0 0x562f35529750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 4 client.29680 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:32:10.110 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.121+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562f353bdff0 con 0x562f354b72d0 2023-07-21T19:32:10.112 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.123+0000 7f80627fc700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354b72d0 0x562f35529750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:10.124 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.126+0000 7f80627fc700 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354b72d0 0x562f35529750 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e89ae3347f1165d server_cookie=1b1993a8ebf4874d in_seq=0 out_seq=0 2023-07-21T19:32:10.125 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.126+0000 7f8042ffd700 10 client.29680 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.205+0000 7f8042ffd700 1 -- 172.21.15.5:0/1209889763 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562f353bdff0 con 0x562f354b72d0 2023-07-21T19:32:10.193 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.205+0000 7f8042ffd700 10 client.29680 handle_command_reply: tid=0 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.205+0000 7f807d249b80 1 client.29680 shutdown 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 msgr2=0x562f354b39d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 0x562f354b39d0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f80540098e0 tx=0x7f8054005610 comp rx=0 tx=0).stop 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 shutdown_connections 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562f354b72d0 0x562f35529750 crc :-1 s=READY pgs=474 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562f354b1580 0x562f354b2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.194 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562f3530d6b0 0x562f354b39d0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 --2- 172.21.15.5:0/1209889763 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562f354a6be0 0x562f354b3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 >> 172.21.15.5:0/1209889763 conn(0x562f354cefc0 msgr2=0x562f354b6250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.195 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f8042ffd700 0 client.29680 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 shutdown_connections 2023-07-21T19:32:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 1 -- 172.21.15.5:0/1209889763 wait complete. 2023-07-21T19:32:10.196 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.206+0000 7f807d249b80 20 client.29680 trim_cache size 0 max 16384 2023-07-21T19:32:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:32:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:32:10.197 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:32:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:32:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 msgr2=0x7f804c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.198 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 0x7f804c0a38b0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f805c007500 tx=0x7f805c00b040 comp rx=0 tx=0).stop 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 msgr2=0x7f8064198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7f805800e940 tx=0x7f805800ec50 comp rx=0 tx=0).stop 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 shutdown_connections 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8064104e50 0x7f806419c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8064067c10 0x7f8064197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f804c0a1400 0x7f804c0a38b0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.199 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 --2- 172.21.15.5:0/3609493296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f80640685b0 0x7f8064198270 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 >> 172.21.15.5:0/3609493296 conn(0x7f8064074760 msgr2=0x7f80640fc980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 shutdown_connections 2023-07-21T19:32:10.200 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:10.207+0000 7f80699ff700 1 -- 172.21.15.5:0/3609493296 wait complete. 2023-07-21T19:32:11.068 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:32:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: ignoring --setuser ceph since I am not root 2023-07-21T19:32:11.068 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:32:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: ignoring --setgroup ceph since I am not root 2023-07-21T19:32:11.068 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:32:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: starting mds.k at 2023-07-21T19:32:11.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:11 smithi101 ceph-mon[128136]: pgmap v777: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:11.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:11 smithi101 ceph-mon[128136]: daemon mds.k restarted 2023-07-21T19:32:11.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:11 smithi005 ceph-mon[126460]: pgmap v777: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:11.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:11 smithi005 ceph-mon[126460]: daemon mds.k restarted 2023-07-21T19:32:11.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:11 smithi156 ceph-mon[128316]: pgmap v777: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 13 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 684 op/s 2023-07-21T19:32:11.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:11 smithi156 ceph-mon[128316]: daemon mds.k restarted 2023-07-21T19:32:11.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 340.0 s, 124.4 tps, lat 837.511 ms stddev 452.343 2023-07-21T19:32:12.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:12 smithi101 ceph-mon[128136]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:32:12.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:12 smithi101 ceph-mon[128136]: mds.? [v2:172.21.15.101:6832/219183820,v1:172.21.15.101:6833/219183820] up:boot 2023-07-21T19:32:12.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T19:32:12.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:12 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:32:12.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:12.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:12 smithi005 ceph-mon[126460]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:32:12.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:12 smithi005 ceph-mon[126460]: mds.? [v2:172.21.15.101:6832/219183820,v1:172.21.15.101:6833/219183820] up:boot 2023-07-21T19:32:12.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T19:32:12.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:12 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:32:12.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:12.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:12 smithi156 ceph-mon[128316]: osdmap e126: 12 total, 12 up, 12 in 2023-07-21T19:32:12.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:12 smithi156 ceph-mon[128316]: mds.? [v2:172.21.15.101:6832/219183820,v1:172.21.15.101:6833/219183820] up:boot 2023-07-21T19:32:12.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 2 up:standby-replay 7 up:standby 2023-07-21T19:32:12.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:12 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-07-21T19:32:12.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:13.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:13 smithi005 ceph-mon[126460]: pgmap v779: 97 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 85 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 615 op/s 2023-07-21T19:32:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:13 smithi156 ceph-mon[128316]: pgmap v779: 97 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 85 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 615 op/s 2023-07-21T19:32:14.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:13 smithi101 ceph-mon[128136]: pgmap v779: 97 pgs: 4 active+clean+snaptrim, 8 active+clean+snaptrim_wait, 85 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.4 MiB/s wr, 615 op/s 2023-07-21T19:32:15.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:15 smithi005 ceph-mon[126460]: pgmap v780: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 674 op/s 2023-07-21T19:32:15.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:15 smithi156 ceph-mon[128316]: pgmap v780: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 674 op/s 2023-07-21T19:32:16.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:15 smithi101 ceph-mon[128136]: pgmap v780: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.7 MiB/s wr, 674 op/s 2023-07-21T19:32:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 345.0 s, 246.6 tps, lat 410.098 ms stddev 92.606 2023-07-21T19:32:17.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:17 smithi005 ceph-mon[126460]: pgmap v781: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-07-21T19:32:17.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:17 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:17.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:17 smithi156 ceph-mon[128316]: pgmap v781: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-07-21T19:32:17.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:17 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:18.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:17 smithi101 ceph-mon[128136]: pgmap v781: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.5 MiB/s wr, 553 op/s 2023-07-21T19:32:18.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:17 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:19.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:18 smithi101 ceph-mon[128136]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T19:32:19.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:18 smithi101 ceph-mon[128136]: pgmap v783: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 939 op/s 2023-07-21T19:32:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:18 smithi005 ceph-mon[126460]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T19:32:19.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:18 smithi005 ceph-mon[126460]: pgmap v783: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 939 op/s 2023-07-21T19:32:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:18 smithi156 ceph-mon[128316]: osdmap e127: 12 total, 12 up, 12 in 2023-07-21T19:32:19.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:18 smithi156 ceph-mon[128316]: pgmap v783: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 9.3 MiB/s wr, 939 op/s 2023-07-21T19:32:21.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:21 smithi101 ceph-mon[128136]: pgmap v784: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 MiB/s wr, 841 op/s 2023-07-21T19:32:21.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:21 smithi005 ceph-mon[126460]: pgmap v784: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 MiB/s wr, 841 op/s 2023-07-21T19:32:21.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:21 smithi156 ceph-mon[128316]: pgmap v784: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 8.4 MiB/s wr, 841 op/s 2023-07-21T19:32:21.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 350.0 s, 174.5 tps, lat 295.960 ms stddev 204.320 2023-07-21T19:32:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.412+0000 7fc6397ff700 1 -- 172.21.15.5:0/418148994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 msgr2=0x7fc63411b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.412+0000 7fc6397ff700 1 --2- 172.21.15.5:0/418148994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b470 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fc62c009f90 tx=0x7fc62c00b330 comp rx=0 tx=0).stop 2023-07-21T19:32:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.418+0000 7fc6397ff700 1 -- 172.21.15.5:0/418148994 shutdown_connections 2023-07-21T19:32:23.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.418+0000 7fc6397ff700 1 --2- 172.21.15.5:0/418148994 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b470 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.418+0000 7fc6397ff700 1 --2- 172.21.15.5:0/418148994 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 0x7fc63410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.418+0000 7fc6397ff700 1 --2- 172.21.15.5:0/418148994 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.418+0000 7fc6397ff700 1 -- 172.21.15.5:0/418148994 >> 172.21.15.5:0/418148994 conn(0x7fc63406bd70 msgr2=0x7fc634070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:23.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.425+0000 7fc6397ff700 1 -- 172.21.15.5:0/418148994 shutdown_connections 2023-07-21T19:32:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.426+0000 7fc6397ff700 1 -- 172.21.15.5:0/418148994 wait complete. 2023-07-21T19:32:23.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.428+0000 7fc6397ff700 1 Processor -- start 2023-07-21T19:32:23.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.428+0000 7fc6397ff700 1 -- start start 2023-07-21T19:32:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63411ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 0x7fc634113b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634114320 con 0x7fc63406d3a0 2023-07-21T19:32:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634114490 con 0x7fc63406c910 2023-07-21T19:32:23.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.432+0000 7fc6397ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc634114650 con 0x7fc63410e2a0 2023-07-21T19:32:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc6327fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc6337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 0x7fc634113b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc6327fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58160/0 (socket says 172.21.15.5:58160) 2023-07-21T19:32:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc6327fc700 1 -- 172.21.15.5:0/3920635934 learned_addr learned my addr 172.21.15.5:0/3920635934 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:23.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63411ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 -- 172.21.15.5:0/3920635934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 msgr2=0x7fc634113b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 0x7fc634113b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 -- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 msgr2=0x7fc63411b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.434+0000 7fc632ffd700 1 -- 172.21.15.5:0/3920635934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc62c009c40 con 0x7fc63406c910 2023-07-21T19:32:23.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.435+0000 7fc632ffd700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63411ad30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc628006fd0 tx=0x7fc62800b7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.436+0000 7fc61bfff700 1 -- 172.21.15.5:0/3920635934 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628011830 con 0x7fc63406c910 2023-07-21T19:32:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.436+0000 7fc61bfff700 1 -- 172.21.15.5:0/3920635934 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc628011e20 con 0x7fc63406c910 2023-07-21T19:32:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.436+0000 7fc61bfff700 1 -- 172.21.15.5:0/3920635934 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628010dc0 con 0x7fc63406c910 2023-07-21T19:32:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.436+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc634114930 con 0x7fc63406c910 2023-07-21T19:32:23.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.436+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc634118b90 con 0x7fc63406c910 2023-07-21T19:32:23.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.443+0000 7fc61bfff700 1 -- 172.21.15.5:0/3920635934 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc628011990 con 0x7fc63406c910 2023-07-21T19:32:23.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.444+0000 7fc64d05ab80 1 Processor -- start 2023-07-21T19:32:23.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 -- start start 2023-07-21T19:32:23.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a913102d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a9130fd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a913002c50 con 0x55a912f596b0 2023-07-21T19:32:23.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9130559e0 con 0x55a9130fd580 2023-07-21T19:32:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91304ee70 con 0x55a9131032d0 2023-07-21T19:32:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.445+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53538/0 (socket says 172.21.15.5:53538) 2023-07-21T19:32:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc632ffd700 1 -- 172.21.15.5:0/167603632 learned_addr learned my addr 172.21.15.5:0/167603632 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a9130fd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6337fe700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a913102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc61bfff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 0x7fc61c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 -- 172.21.15.5:0/167603632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 msgr2=0x55a9130fd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 -- 172.21.15.5:0/167603632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 msgr2=0x55a913102d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a913102d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc6327fc700 1 -- 172.21.15.5:0/167603632 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a912e7cd80 con 0x55a9130fd580 2023-07-21T19:32:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc61bfff700 1 -- 172.21.15.5:0/3920635934 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc6280cc380 con 0x7fc63406c910 2023-07-21T19:32:23.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.446+0000 7fc632ffd700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:32:23.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.447+0000 7fc6327fc700 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a9130fd9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bd00 tx=0x7fc62400f640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5db19b7377e47097 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.447+0000 7fc6337fe700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 0x7fc61c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.447+0000 7fc6197fa700 1 -- 172.21.15.5:0/167603632 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62400fda0 con 0x55a9130fd580 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.447+0000 7fc6197fa700 1 -- 172.21.15.5:0/167603632 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc62400be50 con 0x55a9130fd580 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.447+0000 7fc6197fa700 1 -- 172.21.15.5:0/167603632 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624003d90 con 0x55a9130fd580 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.448+0000 7fc6337fe700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 0x7fc61c0a3980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b770 tx=0x7fc62c009920 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.448+0000 7fc64d05ab80 1 -- 172.21.15.5:0/167603632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 msgr2=0x55a9130fd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.448+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a9130fd9d0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bd00 tx=0x7fc62400f640 comp rx=0 tx=0).stop 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 -- 172.21.15.5:0/167603632 shutdown_connections 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a9130fd9d0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a913102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/167603632 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130fd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 -- 172.21.15.5:0/167603632 >> 172.21.15.5:0/167603632 conn(0x55a9130f2be0 msgr2=0x55a9130f7f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.449+0000 7fc64d05ab80 1 -- 172.21.15.5:0/167603632 shutdown_connections 2023-07-21T19:32:23.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.450+0000 7fc64d05ab80 1 -- 172.21.15.5:0/167603632 wait complete. 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.450+0000 7fc64d05ab80 1 -- start start 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 Processor -- start 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9130fd580 0x55a913106810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9131032d0 0x55a913106e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912feea60 con 0x55a9130fd580 2023-07-21T19:32:23.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912fee760 con 0x55a9131032d0 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.451+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a912ed28c0 con 0x55a912f596b0 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc64d05ab80 10 client.0 _ll_register_callbacks cb 0x55a912fec700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53544/0 (socket says 172.21.15.5:53544) 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 -- 172.21.15.5:0/965304079 learned_addr learned my addr 172.21.15.5:0/965304079 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 -- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9131032d0 msgr2=0x55a913106e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9131032d0 0x55a913106e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 -- 172.21.15.5:0/965304079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9130fd580 msgr2=0x55a913106810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9130fd580 0x55a913106810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.452+0000 7fc632ffd700 1 -- 172.21.15.5:0/965304079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc634118b90 con 0x55a912f596b0 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc632ffd700 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc628095f70 tx=0x7fc6280cfa60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fbb3cd75c7eb3a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62809d320 con 0x55a912f596b0 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a912fa8490 con 0x55a912f596b0 2023-07-21T19:32:23.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc628010480 con 0x55a912f596b0 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc628095840 con 0x55a912f596b0 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc64d05ab80 10 client.29695 fetch_fsmap learned FSMap version 69 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.453+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x7fc62400b960 con 0x55a912f596b0 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a912fa8490 con 0x55a912f596b0 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 69) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc628099760 con 0x55a912f596b0 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 10 client.29695 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T19:32:23.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 10 client.29695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 20 client.29695 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a913116ae0 0x55a913118f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 4 client.29695 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.454+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a913047e50 con 0x55a913116ae0 2023-07-21T19:32:23.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.455+0000 7fc6327fc700 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a913116ae0 0x55a913118f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.456+0000 7fc6327fc700 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a913116ae0 0x55a913118f30 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37c9dfa26dcb88ff server_cookie=9a889e9194a4b49d in_seq=0 out_seq=0 2023-07-21T19:32:23.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.456+0000 7fc616ff5700 10 client.29695 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.457+0000 7fc616ff5700 1 -- 172.21.15.5:0/965304079 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a913047e50 con 0x55a913116ae0 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.457+0000 7fc616ff5700 10 client.29695 handle_command_reply: tid=0 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.457+0000 7fc64d05ab80 1 client.29695 shutdown 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.457+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 msgr2=0x55a913109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.457+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fc628095f70 tx=0x7fc6280cfa60 comp rx=0 tx=0).stop 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 shutdown_connections 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9131032d0 0x55a913106e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.450 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a913109590 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a913116ae0 0x55a913118f30 crc :-1 s=READY pgs=475 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/965304079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9130fd580 0x55a913106810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 >> 172.21.15.5:0/965304079 conn(0x55a9130f2be0 msgr2=0x55a9130f74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc616ff5700 0 client.29695 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 shutdown_connections 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 1 -- 172.21.15.5:0/965304079 wait complete. 2023-07-21T19:32:23.451 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.458+0000 7fc64d05ab80 20 client.29695 trim_cache size 0 max 16384 2023-07-21T19:32:23.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.487+0000 7fc64d05ab80 1 Processor -- start 2023-07-21T19:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.487+0000 7fc64d05ab80 1 -- start start 2023-07-21T19:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a9131020e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.474 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a913109130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130f7c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9130f45f0 con 0x55a912f596b0 2023-07-21T19:32:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a9131013a0 con 0x55a9130fd580 2023-07-21T19:32:23.475 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.488+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a913104d70 con 0x55a9131032d0 2023-07-21T19:32:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.489+0000 7fc6337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130f7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.489+0000 7fc6337fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130f7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53560/0 (socket says 172.21.15.5:53560) 2023-07-21T19:32:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.489+0000 7fc6337fe700 1 -- 172.21.15.5:0/944935309 learned_addr learned my addr 172.21.15.5:0/944935309 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:23.476 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.490+0000 7fc632ffd700 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a9131020e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.490+0000 7fc6327fc700 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a913109130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.491+0000 7fc632ffd700 1 -- 172.21.15.5:0/944935309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 msgr2=0x55a9130f7c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.491+0000 7fc632ffd700 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130f7c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.477 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.491+0000 7fc632ffd700 1 -- 172.21.15.5:0/944935309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 msgr2=0x55a913109130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.491+0000 7fc632ffd700 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a913109130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.491+0000 7fc632ffd700 1 -- 172.21.15.5:0/944935309 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a912e65c00 con 0x55a912f596b0 2023-07-21T19:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.492+0000 7fc632ffd700 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a9131020e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc628009fd0 tx=0x7fc628097ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7c21df773786f3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.478 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.492+0000 7fc619ffb700 1 -- 172.21.15.5:0/944935309 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc62809d320 con 0x55a912f596b0 2023-07-21T19:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.492+0000 7fc619ffb700 1 -- 172.21.15.5:0/944935309 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc6280cfad0 con 0x55a912f596b0 2023-07-21T19:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.492+0000 7fc619ffb700 1 -- 172.21.15.5:0/944935309 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc628098c50 con 0x55a912f596b0 2023-07-21T19:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 -- 172.21.15.5:0/944935309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 msgr2=0x55a9131020e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:23.479 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a9131020e0 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fc628009fd0 tx=0x7fc628097ab0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 -- 172.21.15.5:0/944935309 shutdown_connections 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a9131032d0 0x55a9130f7c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a912f596b0 0x55a9131020e0 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/944935309 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a913109130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 -- 172.21.15.5:0/944935309 >> 172.21.15.5:0/944935309 conn(0x55a9130f2be0 msgr2=0x55a9131019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:23.480 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 -- 172.21.15.5:0/944935309 shutdown_connections 2023-07-21T19:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.493+0000 7fc64d05ab80 1 -- 172.21.15.5:0/944935309 wait complete. 2023-07-21T19:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.494+0000 7fc64d05ab80 1 -- start start 2023-07-21T19:32:23.481 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.494+0000 7fc64d05ab80 1 Processor -- start 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a91310dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9131032d0 0x55a91310c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91310c860 con 0x55a9131032d0 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a91310c9d0 con 0x55a9130fd580 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a913169a20 con 0x55a912f596b0 2023-07-21T19:32:23.482 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.495+0000 7fc64d05ab80 10 client.0 _ll_register_callbacks cb 0x55a912fec700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.496+0000 7fc6327fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.496+0000 7fc6327fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32818/0 (socket says 172.21.15.5:32818) 2023-07-21T19:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.496+0000 7fc6327fc700 1 -- 172.21.15.5:0/213025303 learned_addr learned my addr 172.21.15.5:0/213025303 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:23.483 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.497+0000 7fc6327fc700 1 -- 172.21.15.5:0/213025303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 msgr2=0x55a91310dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.497+0000 7fc6327fc700 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a91310dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.497+0000 7fc6327fc700 1 -- 172.21.15.5:0/213025303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9131032d0 msgr2=0x55a91310c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.497+0000 7fc6327fc700 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9131032d0 0x55a91310c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:23.484 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.497+0000 7fc6327fc700 1 -- 172.21.15.5:0/213025303 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62400b960 con 0x55a9130fd580 2023-07-21T19:32:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.498+0000 7fc6327fc700 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bc50 tx=0x7fc62400b650 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1349997f3cd2f2cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.498+0000 7fc616ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.498+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc624027cc0 con 0x55a9130fd580 2023-07-21T19:32:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.498+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc62400be50 con 0x55a9130fd580 2023-07-21T19:32:23.485 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.499+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc624027e20 con 0x55a9130fd580 2023-07-21T19:32:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.499+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc6240039a0 con 0x55a9130fd580 2023-07-21T19:32:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.499+0000 7fc64d05ab80 10 client.29795 fetch_fsmap learned FSMap version 69 2023-07-21T19:32:23.486 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.499+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x55a912ea31f0 con 0x55a9130fd580 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc624035020 con 0x55a9130fd580 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 69) v1 ==== 7474+0+0 (secure 0 0 0) 0x7fc6240051a0 con 0x55a9130fd580 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 10 client.29795 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 10 client.29795 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 20 client.29795 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:23.487 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a91316b0f0 0x55a91316d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 4 client.29795 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:32:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.500+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a913009ff0 con 0x55a91316b0f0 2023-07-21T19:32:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.501+0000 7fc632ffd700 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a91316b0f0 0x55a91316d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.501+0000 7fc616ff5700 10 client.29795 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:23.488 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:23.501+0000 7fc632ffd700 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a91316b0f0 0x55a91316d500 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edfc0da2c5f590c server_cookie=b9a1bf386d9dfb6f in_seq=0 out_seq=0 2023-07-21T19:32:23.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:23 smithi101 ceph-mon[128136]: pgmap v785: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 721 op/s 2023-07-21T19:32:23.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:23 smithi005 ceph-mon[126460]: pgmap v785: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 721 op/s 2023-07-21T19:32:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:23 smithi156 ceph-mon[128316]: pgmap v785: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 721 op/s 2023-07-21T19:32:24.832 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:32:25.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 -- 172.21.15.5:0/1509554723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398010c3d0 msgr2=0x7f398010e7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 --2- 172.21.15.5:0/1509554723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398010c3d0 0x7f398010e7b0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f397800b0a0 tx=0x7f397800b3b0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 -- 172.21.15.5:0/1509554723 shutdown_connections 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 --2- 172.21.15.5:0/1509554723 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f398010c3d0 0x7f398010e7b0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 --2- 172.21.15.5:0/1509554723 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800fe820 0x7f39800febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 --2- 172.21.15.5:0/1509554723 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39800ff1c0 0x7f398010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.212+0000 7f39879ff700 1 -- 172.21.15.5:0/1509554723 >> 172.21.15.5:0/1509554723 conn(0x7f39800fa4b0 msgr2=0x7f39800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.213+0000 7f39879ff700 1 -- 172.21.15.5:0/1509554723 shutdown_connections 2023-07-21T19:32:25.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.213+0000 7f39879ff700 1 -- 172.21.15.5:0/1509554723 wait complete. 2023-07-21T19:32:25.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.215+0000 7f39879ff700 1 Processor -- start 2023-07-21T19:32:25.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.215+0000 7f39879ff700 1 -- start start 2023-07-21T19:32:25.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39800fe820 0x7f398010ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800ff1c0 0x7f3980108a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3980109580 con 0x7f39800ff1c0 2023-07-21T19:32:25.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801096f0 con 0x7f398010c3d0 2023-07-21T19:32:25.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.216+0000 7f39879ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39801098b0 con 0x7f39800fe820 2023-07-21T19:32:25.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800ff1c0 0x7f3980108a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32834/0 (socket says 172.21.15.5:32834) 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 -- 172.21.15.5:0/665327389 learned_addr learned my addr 172.21.15.5:0/665327389 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f398579b700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39800fe820 0x7f398010ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 -- 172.21.15.5:0/665327389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39800fe820 msgr2=0x7f398010ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39800fe820 0x7f398010ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 -- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800ff1c0 msgr2=0x7f3980108a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800ff1c0 0x7f3980108a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.217+0000 7f3985f9c700 1 -- 172.21.15.5:0/665327389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3970009520 con 0x7f398010c3d0 2023-07-21T19:32:25.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f3985f9c700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3978009fd0 tx=0x7f3978007930 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f39767fc700 1 -- 172.21.15.5:0/665327389 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397800e040 con 0x7f398010c3d0 2023-07-21T19:32:25.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f39767fc700 1 -- 172.21.15.5:0/665327389 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f397800bdc0 con 0x7f398010c3d0 2023-07-21T19:32:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f39767fc700 1 -- 172.21.15.5:0/665327389 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3978005040 con 0x7f398010c3d0 2023-07-21T19:32:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3978009d00 con 0x7f398010c3d0 2023-07-21T19:32:25.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.218+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39801a8cc0 con 0x7f398010c3d0 2023-07-21T19:32:25.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.224+0000 7f39767fc700 1 -- 172.21.15.5:0/665327389 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f397801d480 con 0x7f398010c3d0 2023-07-21T19:32:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.225+0000 7f399b29ab80 1 Processor -- start 2023-07-21T19:32:25.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.225+0000 7f399b29ab80 1 -- start start 2023-07-21T19:32:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.225+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f98b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f996590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f998eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f399b29ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f891cb0 con 0x56010f98b2d0 2023-07-21T19:32:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f399b29ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f8e4a40 con 0x56010f98bbe0 2023-07-21T19:32:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f399b29ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f8dded0 con 0x56010f996ad0 2023-07-21T19:32:25.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f398579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f996590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f398579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f996590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32842/0 (socket says 172.21.15.5:32842) 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f398579b700 1 -- 172.21.15.5:0/3759326473 learned_addr learned my addr 172.21.15.5:0/3759326473 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f998eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.226+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f98b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 -- 172.21.15.5:0/3759326473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 msgr2=0x56010f996590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f996590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 -- 172.21.15.5:0/3759326473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 msgr2=0x56010f98b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f98b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 -- 172.21.15.5:0/3759326473 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f70bde0 con 0x56010f996ad0 2023-07-21T19:32:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f998eb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f398010a260 tx=0x7f3970009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef77ca428524467e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.227+0000 7f39767fc700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 0x7f396c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.228+0000 7f396b7fe700 1 -- 172.21.15.5:0/3759326473 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970004830 con 0x56010f996ad0 2023-07-21T19:32:25.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.228+0000 7f39767fc700 1 -- 172.21.15.5:0/665327389 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f39780cdce0 con 0x7f398010c3d0 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.228+0000 7f396b7fe700 1 -- 172.21.15.5:0/3759326473 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3970009e80 con 0x56010f996ad0 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.228+0000 7f396b7fe700 1 -- 172.21.15.5:0/3759326473 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970010460 con 0x56010f996ad0 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.228+0000 7f398579b700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 0x7f396c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 -- 172.21.15.5:0/3759326473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 msgr2=0x56010f998eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f998eb0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f398010a260 tx=0x7f3970009400 comp rx=0 tx=0).stop 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 -- 172.21.15.5:0/3759326473 shutdown_connections 2023-07-21T19:32:25.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f998eb0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f98b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3759326473 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f996590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 -- 172.21.15.5:0/3759326473 >> 172.21.15.5:0/3759326473 conn(0x56010f981da0 msgr2=0x56010f9876a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f398579b700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 0x7f396c0a37f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f397c005c50 tx=0x7f397c009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.229+0000 7f399b29ab80 1 -- 172.21.15.5:0/3759326473 shutdown_connections 2023-07-21T19:32:25.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.230+0000 7f399b29ab80 1 -- 172.21.15.5:0/3759326473 wait complete. 2023-07-21T19:32:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.230+0000 7f399b29ab80 1 -- start start 2023-07-21T19:32:25.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 Processor -- start 2023-07-21T19:32:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f996510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f993ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f87dac0 con 0x56010f98b2d0 2023-07-21T19:32:25.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f87d7c0 con 0x56010f98bbe0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f761920 con 0x56010f996ad0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.231+0000 7f399b29ab80 10 client.0 _ll_register_callbacks cb 0x56010f87b760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3985f9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f993ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32846/0 (socket says 172.21.15.5:32846) 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 -- 172.21.15.5:0/1241773025 learned_addr learned my addr 172.21.15.5:0/1241773025 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f398579b700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f996510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 -- 172.21.15.5:0/1241773025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 msgr2=0x56010f993ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f993ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 -- 172.21.15.5:0/1241773025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 msgr2=0x56010f996510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.232+0000 7f3984f9a700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f996510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f3984f9a700 1 -- 172.21.15.5:0/1241773025 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3970013460 con 0x56010f98bbe0 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f3984f9a700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3970005950 tx=0x7f39700057b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=69f65f68b40bc2a4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f3968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970004c20 con 0x56010f98bbe0 2023-07-21T19:32:25.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010f8d6eb0 con 0x56010f98bbe0 2023-07-21T19:32:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.233+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3970000e40 con 0x56010f98bbe0 2023-07-21T19:32:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3970004d80 con 0x56010f98bbe0 2023-07-21T19:32:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f399b29ab80 10 client.29807 fetch_fsmap learned FSMap version 69 2023-07-21T19:32:25.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x56010f6f4c60 con 0x56010f98bbe0 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56010f8d6eb0 con 0x56010f98bbe0 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 69) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f397001f450 con 0x56010f98bbe0 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f399b29ab80 10 client.29807 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.234+0000 7f399b29ab80 10 client.29807 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.235+0000 7f399b29ab80 20 client.29807 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:25.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.235+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f9b74c0 0x56010f9b98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.235+0000 7f399b29ab80 4 client.29807 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.235+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010f899050 con 0x56010f9b74c0 2023-07-21T19:32:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.235+0000 7f398579b700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f9b74c0 0x56010f9b98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.236+0000 7f398579b700 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f9b74c0 0x56010f9b98d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b08a760fe28cb7c server_cookie=3103aa928b58fb6c in_seq=0 out_seq=0 2023-07-21T19:32:25.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.236+0000 7f3968ff9700 10 client.29807 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc616ff5700 1 -- 172.21.15.5:0/213025303 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a913009ff0 con 0x55a91316b0f0 2023-07-21T19:32:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc616ff5700 10 client.29795 handle_command_reply: tid=0 2023-07-21T19:32:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 client.29795 shutdown 2023-07-21T19:32:25.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 msgr2=0x55a91310bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fc62400bc50 tx=0x7fc62400b650 comp rx=0 tx=0).stop 2023-07-21T19:32:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 shutdown_connections 2023-07-21T19:32:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a91316b0f0 0x55a91316d500 crc :-1 s=READY pgs=476 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a9131032d0 0x55a91310c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a912f596b0 0x55a91310dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc616ff5700 0 client.29795 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 --2- 172.21.15.5:0/213025303 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a9130fd580 0x55a91310bc50 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 >> 172.21.15.5:0/213025303 conn(0x55a9130f2be0 msgr2=0x55a913102260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f3968ff9700 1 -- 172.21.15.5:0/1241773025 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56010f899050 con 0x56010f9b74c0 2023-07-21T19:32:25.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f3968ff9700 10 client.29807 handle_command_reply: tid=0 2023-07-21T19:32:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 shutdown_connections 2023-07-21T19:32:25.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 1 -- 172.21.15.5:0/213025303 wait complete. 2023-07-21T19:32:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f399b29ab80 1 client.29807 shutdown 2023-07-21T19:32:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 msgr2=0x56010f993790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f3970005950 tx=0x7f39700057b0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.793+0000 7fc64d05ab80 20 client.29795 trim_cache size 0 max 16384 2023-07-21T19:32:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 shutdown_connections 2023-07-21T19:32:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.793+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f996ad0 0x56010f993ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f9b74c0 0x56010f9b98d0 crc :-1 s=READY pgs=477 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:32:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:32:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:32:25.785 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98b2d0 0x56010f996510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f3968ff9700 0 client.29807 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 --2- 172.21.15.5:0/1241773025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98bbe0 0x56010f993790 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 >> 172.21.15.5:0/1241773025 conn(0x56010f981da0 msgr2=0x56010f987290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 shutdown_connections 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 1 -- 172.21.15.5:0/1241773025 wait complete. 2023-07-21T19:32:25.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.794+0000 7f399b29ab80 20 client.29807 trim_cache size 0 max 16384 2023-07-21T19:32:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.794+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 msgr2=0x7fc61c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.794+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 0x7fc61c0a3980 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc62c00b770 tx=0x7fc62c009920 comp rx=0 tx=0).stop 2023-07-21T19:32:25.787 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.794+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 msgr2=0x7fc63411ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.794+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63411ad30 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fc628006fd0 tx=0x7fc62800b7e0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 shutdown_connections 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc63410e2a0 0x7fc634113b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc63406c910 0x7fc63411ad30 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc61c0a14d0 0x7fc61c0a3980 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 --2- 172.21.15.5:0/3920635934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc63406d3a0 0x7fc63411b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 >> 172.21.15.5:0/3920635934 conn(0x7fc63406bd70 msgr2=0x7fc63406f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 shutdown_connections 2023-07-21T19:32:25.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:25.795+0000 7fc6397ff700 1 -- 172.21.15.5:0/3920635934 wait complete. 2023-07-21T19:32:25.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.822+0000 7f399b29ab80 1 Processor -- start 2023-07-21T19:32:25.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 -- start start 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 0x56010f99d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f990a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f8d6eb0 con 0x56010f98bbe0 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f983ec0 con 0x56010f981da0 2023-07-21T19:32:25.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f399b29ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f988d20 con 0x56010f98b2d0 2023-07-21T19:32:25.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f398579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 0x56010f99d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53610/0 (socket says 172.21.15.5:53610) 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f398579b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 0x56010f99d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32856/0 (socket says 172.21.15.5:32856) 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.823+0000 7f3984f9a700 1 -- 172.21.15.5:0/2441721993 learned_addr learned my addr 172.21.15.5:0/2441721993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3985f9c700 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f990a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 -- 172.21.15.5:0/2441721993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 msgr2=0x56010f99d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 0x56010f99d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 -- 172.21.15.5:0/2441721993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 msgr2=0x56010f990a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f990a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 -- 172.21.15.5:0/2441721993 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010f732250 con 0x56010f98b2d0 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3985f9c700 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f990a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.824+0000 7f3984f9a700 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f397001f8b0 tx=0x7f39700092d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d713b1e73fd18c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.825+0000 7f396bfff700 1 -- 172.21.15.5:0/2441721993 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970009aa0 con 0x56010f98b2d0 2023-07-21T19:32:25.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.825+0000 7f396bfff700 1 -- 172.21.15.5:0/2441721993 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3970004eb0 con 0x56010f98b2d0 2023-07-21T19:32:25.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.825+0000 7f396bfff700 1 -- 172.21.15.5:0/2441721993 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39700108c0 con 0x56010f98b2d0 2023-07-21T19:32:25.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 -- 172.21.15.5:0/2441721993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 msgr2=0x56010f996140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f397001f8b0 tx=0x7f39700092d0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 -- 172.21.15.5:0/2441721993 shutdown_connections 2023-07-21T19:32:25.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f990a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f981da0 0x56010f99d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 --2- 172.21.15.5:0/2441721993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f98b2d0 0x56010f996140 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 -- 172.21.15.5:0/2441721993 >> 172.21.15.5:0/2441721993 conn(0x56010f983690 msgr2=0x56010f98ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 -- 172.21.15.5:0/2441721993 shutdown_connections 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.826+0000 7f399b29ab80 1 -- 172.21.15.5:0/2441721993 wait complete. 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.827+0000 7f399b29ab80 1 -- start start 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.827+0000 7f399b29ab80 1 Processor -- start 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 0x56010fa04ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f9ff2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f9ff8e0 con 0x56010f98bbe0 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f9ffa50 con 0x56010f98b2d0 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010f9ffc10 con 0x56010f981da0 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f399b29ab80 10 client.0 _ll_register_callbacks cb 0x7f397c009830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f398579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 0x56010fa04ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f3984f9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:32858/0 (socket says 172.21.15.5:32858) 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f3984f9a700 1 -- 172.21.15.5:0/3694215240 learned_addr learned my addr 172.21.15.5:0/3694215240 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f398579b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 0x56010fa04ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53618/0 (socket says 172.21.15.5:53618) 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.828+0000 7f3985f9c700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f9ff2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 -- 172.21.15.5:0/3694215240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 msgr2=0x56010fa04ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 0x56010fa04ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 -- 172.21.15.5:0/3694215240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 msgr2=0x56010f9ff2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:25.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f9ff2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 -- 172.21.15.5:0/3694215240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010f6f4c60 con 0x56010f98b2d0 2023-07-21T19:32:25.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3985f9c700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f9ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:32:25.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3984f9a700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3970016040 tx=0x7f3970005480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3208a4576298fe6a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:25.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.829+0000 7f3968ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:25.821 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:25 smithi101 ceph-mon[128136]: pgmap v786: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:25.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3970010870 con 0x56010f98b2d0 2023-07-21T19:32:25.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3970000e40 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010f8374f0 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=69) v2 ==== 24+0+0 (secure 0 0 0) 0x7f397001fb20 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f399b29ab80 10 client.29813 fetch_fsmap learned FSMap version 69 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=69,monmap=4+}) v3 -- 0x56010f70a2b0 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.830+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3970046020 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 69) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f3970037070 con 0x56010f98b2d0 2023-07-21T19:32:25.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 10 client.29813 fetch_fsmap finished waiting for FSMap version 69 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 10 client.29813 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 20 client.29813 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f996ad0 0x56010fa0e650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 4 client.29813 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56010f72cf70 con 0x56010f996ad0 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.831+0000 7f398579b700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f996ad0 0x56010fa0e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:25.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f3968ff9700 10 client.29813 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f398579b700 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f996ad0 0x56010fa0e650 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ce2aa223696efe2 server_cookie=b71aef9cb6696365 in_seq=0 out_seq=0 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f3968ff9700 1 -- 172.21.15.5:0/3694215240 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56010f72cf70 con 0x56010f996ad0 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f3968ff9700 10 client.29813 handle_command_reply: tid=0 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 client.29813 shutdown 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 msgr2=0x56010fa050e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f3970016040 tx=0x7f3970005480 comp rx=0 tx=0).stop 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 shutdown_connections 2023-07-21T19:32:25.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56010f996ad0 0x56010fa0e650 crc :-1 s=READY pgs=478 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56010f98bbe0 0x56010f9ff2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56010f981da0 0x56010fa04ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.832+0000 7f399b29ab80 1 --2- 172.21.15.5:0/3694215240 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56010f98b2d0 0x56010fa050e0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.833+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 >> 172.21.15.5:0/3694215240 conn(0x56010f983690 msgr2=0x56010f98d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.833+0000 7f3968ff9700 0 client.29813 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:25.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.833+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 shutdown_connections 2023-07-21T19:32:25.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.833+0000 7f399b29ab80 1 -- 172.21.15.5:0/3694215240 wait complete. 2023-07-21T19:32:25.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.833+0000 7f399b29ab80 20 client.29813 trim_cache size 0 max 16384 2023-07-21T19:32:25.827 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:32:25.827 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:32:25.828 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:32:25.828 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:32:25.828 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:32:25.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:32:25.828 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:32:25.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:32:25.829 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:32:25.829 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:32:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 msgr2=0x7f396c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 0x7f396c0a37f0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f397c005c50 tx=0x7f397c009260 comp rx=0 tx=0).stop 2023-07-21T19:32:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 msgr2=0x7f3980108fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f3978009fd0 tx=0x7f3978007930 comp rx=0 tx=0).stop 2023-07-21T19:32:25.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 shutdown_connections 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f398010c3d0 0x7f3980108fb0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f39800fe820 0x7f398010ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f396c0a1340 0x7f396c0a37f0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 --2- 172.21.15.5:0/665327389 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39800ff1c0 0x7f3980108a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 >> 172.21.15.5:0/665327389 conn(0x7f39800fa4b0 msgr2=0x7f398010d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 shutdown_connections 2023-07-21T19:32:25.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:25.834+0000 7f39879ff700 1 -- 172.21.15.5:0/665327389 wait complete. 2023-07-21T19:32:25.845 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:25 smithi005 ceph-mon[126460]: pgmap v786: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:25.849 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:32:25.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:25 smithi156 ceph-mon[128316]: pgmap v786: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 355.0 s, 184.5 tps, lat 801.450 ms stddev 1217.687 2023-07-21T19:32:27.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:27 smithi101 ceph-mon[128136]: pgmap v787: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:27.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:27 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:27.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:27 smithi005 ceph-mon[126460]: pgmap v787: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:27.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:27 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:27.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:27 smithi156 ceph-mon[128316]: pgmap v787: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 710 op/s 2023-07-21T19:32:27.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:27 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:32:29.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:28 smithi101 ceph-mon[128136]: pgmap v788: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 568 op/s 2023-07-21T19:32:29.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:28 smithi005 ceph-mon[126460]: pgmap v788: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 568 op/s 2023-07-21T19:32:29.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:28 smithi156 ceph-mon[128316]: pgmap v788: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.8 MiB/s wr, 568 op/s 2023-07-21T19:32:31.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:31 smithi101 ceph-mon[128136]: pgmap v789: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2023-07-21T19:32:31.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:31 smithi005 ceph-mon[126460]: pgmap v789: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2023-07-21T19:32:31.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:31 smithi156 ceph-mon[128316]: pgmap v789: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.2 MiB/s wr, 509 op/s 2023-07-21T19:32:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 360.0 s, 259.8 tps, lat 386.208 ms stddev 97.728 2023-07-21T19:32:33.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:33 smithi101 ceph-mon[128136]: pgmap v790: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-07-21T19:32:33.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:33 smithi005 ceph-mon[126460]: pgmap v790: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-07-21T19:32:33.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:33 smithi156 ceph-mon[128316]: pgmap v790: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 819 op/s 2023-07-21T19:32:35.818 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:35 smithi101 ceph-mon[128136]: pgmap v791: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 688 op/s 2023-07-21T19:32:35.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:35 smithi005 ceph-mon[126460]: pgmap v791: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 688 op/s 2023-07-21T19:32:35.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:35 smithi156 ceph-mon[128316]: pgmap v791: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 688 op/s 2023-07-21T19:32:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 365.0 s, 279.0 tps, lat 358.256 ms stddev 64.634 2023-07-21T19:32:37.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:37 smithi101 ceph-mon[128136]: pgmap v792: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2023-07-21T19:32:37.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:37 smithi005 ceph-mon[126460]: pgmap v792: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2023-07-21T19:32:37.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:37 smithi156 ceph-mon[128316]: pgmap v792: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 657 op/s 2023-07-21T19:32:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 -- 172.21.15.5:0/1170280359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f0104a30 msgr2=0x7f72f0104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 --2- 172.21.15.5:0/1170280359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f0104a30 0x7f72f0104e00 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0098f0 tx=0x7f72ec009c00 comp rx=0 tx=0).stop 2023-07-21T19:32:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 -- 172.21.15.5:0/1170280359 shutdown_connections 2023-07-21T19:32:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 --2- 172.21.15.5:0/1170280359 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f010b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 --2- 172.21.15.5:0/1170280359 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f0104a30 0x7f72f0104e00 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 --2- 172.21.15.5:0/1170280359 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f00fea30 0x7f72f00feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.021+0000 7f72f77ff700 1 -- 172.21.15.5:0/1170280359 >> 172.21.15.5:0/1170280359 conn(0x7f72f0074760 msgr2=0x7f72f0074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.022+0000 7f72f77ff700 1 -- 172.21.15.5:0/1170280359 shutdown_connections 2023-07-21T19:32:39.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.022+0000 7f72f77ff700 1 -- 172.21.15.5:0/1170280359 wait complete. 2023-07-21T19:32:39.011 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.024+0000 7f72f77ff700 1 Processor -- start 2023-07-21T19:32:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.024+0000 7f72f77ff700 1 -- start start 2023-07-21T19:32:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f00fea30 0x7f72f0197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.012 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f01982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019cfa0 con 0x7f72f00fea30 2023-07-21T19:32:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019d110 con 0x7f72f00ff3e0 2023-07-21T19:32:39.013 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f77ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72f019d2d0 con 0x7f72f0104a30 2023-07-21T19:32:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.025+0000 7f72f5d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39478/0 (socket says 172.21.15.5:39478) 2023-07-21T19:32:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 -- 172.21.15.5:0/401756014 learned_addr learned my addr 172.21.15.5:0/401756014 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f559b700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f00fea30 0x7f72f0197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f01982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 -- 172.21.15.5:0/401756014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 msgr2=0x7f72f01982f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f01982f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 -- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f00fea30 msgr2=0x7f72f0197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f00fea30 0x7f72f0197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f5d9c700 1 -- 172.21.15.5:0/401756014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009520 con 0x7f72f0104a30 2023-07-21T19:32:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.026+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:32:39.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72e800ef30 tx=0x7f72e800c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72e67fc700 1 -- 172.21.15.5:0/401756014 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8018070 con 0x7f72f0104a30 2023-07-21T19:32:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72e67fc700 1 -- 172.21.15.5:0/401756014 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72e800f040 con 0x7f72f0104a30 2023-07-21T19:32:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72f019d550 con 0x7f72f0104a30 2023-07-21T19:32:39.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72e67fc700 1 -- 172.21.15.5:0/401756014 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8003d00 con 0x7f72f0104a30 2023-07-21T19:32:39.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.027+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72f006a110 con 0x7f72f0104a30 2023-07-21T19:32:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.034+0000 7f72e67fc700 1 -- 172.21.15.5:0/401756014 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f72e800caa0 con 0x7f72f0104a30 2023-07-21T19:32:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.035+0000 7f730b056b80 1 Processor -- start 2023-07-21T19:32:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 -- start start 2023-07-21T19:32:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d1246b0 0x55c81d2d65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d6ae0 0x55c81d2d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d1cdc50 con 0x55c81d1246b0 2023-07-21T19:32:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2209e0 con 0x55c81d2d6ae0 2023-07-21T19:32:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.036+0000 7f730b056b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d219e70 con 0x55c81d2d0d90 2023-07-21T19:32:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.037+0000 7f72f4d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.037+0000 7f72f4d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39490/0 (socket says 172.21.15.5:39490) 2023-07-21T19:32:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.037+0000 7f72f4d9a700 1 -- 172.21.15.5:0/2935956837 learned_addr learned my addr 172.21.15.5:0/2935956837 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.037+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d1246b0 0x55c81d2d65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.037+0000 7f72f559b700 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d6ae0 0x55c81d2d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 -- 172.21.15.5:0/2935956837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d6ae0 msgr2=0x55c81d2d0850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d6ae0 0x55c81d2d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 -- 172.21.15.5:0/2935956837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d1246b0 msgr2=0x55c81d2d65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d1246b0 0x55c81d2d65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 -- 172.21.15.5:0/2935956837 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c81d047d80 con 0x55c81d2d0d90 2023-07-21T19:32:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.038+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009f30 tx=0x7f72e00096f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6dc9b6f974f2497b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.039+0000 7f72db7fe700 1 -- 172.21.15.5:0/2935956837 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e00142f0 con 0x55c81d2d0d90 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.039+0000 7f72e67fc700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 0x7f72dc0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.039+0000 7f72db7fe700 1 -- 172.21.15.5:0/2935956837 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72e00044c0 con 0x55c81d2d0d90 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.039+0000 7f72db7fe700 1 -- 172.21.15.5:0/2935956837 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0012450 con 0x55c81d2d0d90 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.039+0000 7f72e67fc700 1 -- 172.21.15.5:0/401756014 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f72e80cc9a0 con 0x7f72f0104a30 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f72f559b700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 0x7f72dc0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 -- 172.21.15.5:0/2935956837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 msgr2=0x55c81d2d11e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009f30 tx=0x7f72e00096f0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 -- 172.21.15.5:0/2935956837 shutdown_connections 2023-07-21T19:32:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d11e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d1246b0 0x55c81d2d65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 --2- 172.21.15.5:0/2935956837 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d6ae0 0x55c81d2d0850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.040+0000 7f730b056b80 1 -- 172.21.15.5:0/2935956837 >> 172.21.15.5:0/2935956837 conn(0x55c81d2bdbe0 msgr2=0x55c81d2c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.041+0000 7f730b056b80 1 -- 172.21.15.5:0/2935956837 shutdown_connections 2023-07-21T19:32:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.041+0000 7f730b056b80 1 -- 172.21.15.5:0/2935956837 wait complete. 2023-07-21T19:32:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.041+0000 7f72f559b700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 0x7f72dc0a3940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0098c0 tx=0x7f72ec019040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.042+0000 7f730b056b80 1 -- start start 2023-07-21T19:32:39.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.042+0000 7f730b056b80 1 Processor -- start 2023-07-21T19:32:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2da6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2dac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d1b9a60 con 0x55c81d2d6ae0 2023-07-21T19:32:39.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d1b9760 con 0x55c81d1246b0 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d09d8c0 con 0x55c81d2d0d90 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f730b056b80 10 client.0 _ll_register_callbacks cb 0x55c81d1b7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.043+0000 7f72f559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42126/0 (socket says 172.21.15.5:42126) 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 -- 172.21.15.5:0/988781168 learned_addr learned my addr 172.21.15.5:0/988781168 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:39.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2da6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 -- 172.21.15.5:0/988781168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 msgr2=0x55c81d2da6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2da6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 -- 172.21.15.5:0/988781168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 msgr2=0x55c81d2dac10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2dac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.044+0000 7f72f559b700 1 -- 172.21.15.5:0/988781168 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72e0014470 con 0x55c81d1246b0 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.045+0000 7f72f559b700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005e90 tx=0x7f72ec009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1edebde4678f32de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.045+0000 7f72d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.045+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec028070 con 0x55c81d1246b0 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.045+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f72ec00fe60 con 0x55c81d1246b0 2023-07-21T19:32:39.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.045+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c81d173490 con 0x55c81d1246b0 2023-07-21T19:32:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.046+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72ec01c040 con 0x55c81d1246b0 2023-07-21T19:32:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.046+0000 7f730b056b80 10 client.29819 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.046+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55c81d030c00 con 0x55c81d1246b0 2023-07-21T19:32:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c81d173490 con 0x55c81d1246b0 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72ec035070 con 0x55c81d1246b0 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 10 client.29819 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 10 client.29819 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 20 client.29819 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d2ea410 0x55c81d2ec820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 4 client.29819 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.047+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c81d212e50 con 0x55c81d2ea410 2023-07-21T19:32:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.048+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d2ea410 0x55c81d2ec820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.050+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d2ea410 0x55c81d2ec820 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78063f7ae44c0dea server_cookie=5834e23c8f3155 in_seq=0 out_seq=0 2023-07-21T19:32:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.050+0000 7f72d8ff9700 10 client.29819 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.051+0000 7f72d8ff9700 1 -- 172.21.15.5:0/988781168 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c81d212e50 con 0x55c81d2ea410 2023-07-21T19:32:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.051+0000 7f72d8ff9700 10 client.29819 handle_command_reply: tid=0 2023-07-21T19:32:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.051+0000 7f730b056b80 1 client.29819 shutdown 2023-07-21T19:32:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.051+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 msgr2=0x55c81d2da190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.051+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f72ec005e90 tx=0x7f72ec009c00 comp rx=0 tx=0).stop 2023-07-21T19:32:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 shutdown_connections 2023-07-21T19:32:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d2ea410 0x55c81d2ec820 crc :-1 s=READY pgs=479 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2dac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2da190 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 --2- 172.21.15.5:0/988781168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2da6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 >> 172.21.15.5:0/988781168 conn(0x55c81d2bdbe0 msgr2=0x55c81d2c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f72d8ff9700 0 client.29819 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 shutdown_connections 2023-07-21T19:32:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.052+0000 7f730b056b80 1 -- 172.21.15.5:0/988781168 wait complete. 2023-07-21T19:32:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.053+0000 7f730b056b80 20 client.29819 trim_cache size 0 max 16384 2023-07-21T19:32:39.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.074+0000 7f730b056b80 1 Processor -- start 2023-07-21T19:32:39.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 -- start start 2023-07-21T19:32:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2daa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.064 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2dd1d0 con 0x55c81d2d6ae0 2023-07-21T19:32:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2d6220 con 0x55c81d1246b0 2023-07-21T19:32:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.075+0000 7f730b056b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2d6eb0 con 0x55c81d2d0d90 2023-07-21T19:32:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42140/0 (socket says 172.21.15.5:42140) 2023-07-21T19:32:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 -- 172.21.15.5:0/1536229787 learned_addr learned my addr 172.21.15.5:0/1536229787 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2daa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 -- 172.21.15.5:0/1536229787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 msgr2=0x55c81d2d5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 -- 172.21.15.5:0/1536229787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 msgr2=0x55c81d2daa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2daa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 -- 172.21.15.5:0/1536229787 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c81d06e1f0 con 0x55c81d1246b0 2023-07-21T19:32:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2daa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:32:39.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:38 smithi101 ceph-mon[128136]: pgmap v793: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 963 op/s 2023-07-21T19:32:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.076+0000 7f72f559b700 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009810 tx=0x7f72ec005c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=91d9bda214fd17fa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.077+0000 7f72dbfff700 1 -- 172.21.15.5:0/1536229787 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec028070 con 0x55c81d1246b0 2023-07-21T19:32:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.077+0000 7f72dbfff700 1 -- 172.21.15.5:0/1536229787 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72ec022410 con 0x55c81d1246b0 2023-07-21T19:32:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.077+0000 7f72dbfff700 1 -- 172.21.15.5:0/1536229787 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec01c040 con 0x55c81d1246b0 2023-07-21T19:32:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- 172.21.15.5:0/1536229787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 msgr2=0x55c81d2d0850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f72ec009810 tx=0x7f72ec005c80 comp rx=0 tx=0).stop 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- 172.21.15.5:0/1536229787 shutdown_connections 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2daa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d1246b0 0x55c81d2d0850 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 --2- 172.21.15.5:0/1536229787 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d2d0d90 0x55c81d2d5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- 172.21.15.5:0/1536229787 >> 172.21.15.5:0/1536229787 conn(0x55c81d2bdbe0 msgr2=0x55c81d2d7110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- 172.21.15.5:0/1536229787 shutdown_connections 2023-07-21T19:32:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- 172.21.15.5:0/1536229787 wait complete. 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.078+0000 7f730b056b80 1 -- start start 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 Processor -- start 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 0x55c81d2e1140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2df600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2dfcd0 con 0x55c81d2d6ae0 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2dfe40 con 0x55c81d2d0d90 2023-07-21T19:32:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c81d2e0000 con 0x55c81d1246b0 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.079+0000 7f730b056b80 10 client.0 _ll_register_callbacks cb 0x7f72ec00f770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42146/0 (socket says 172.21.15.5:42146) 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 -- 172.21.15.5:0/3421516468 learned_addr learned my addr 172.21.15.5:0/3421516468 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f559b700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 0x55c81d2e1140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2df600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 -- 172.21.15.5:0/3421516468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 msgr2=0x55c81d2e1140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 0x55c81d2e1140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 -- 172.21.15.5:0/3421516468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 msgr2=0x55c81d2df600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2df600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 -- 172.21.15.5:0/3421516468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c81d030c00 con 0x55c81d2d0d90 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f5d9c700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f559b700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 0x55c81d2e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:32:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72f4d9a700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f72e0016010 tx=0x7f72e0005bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4df3632c941bc6eb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.080+0000 7f72d8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e0024040 con 0x55c81d2d0d90 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f72e00044c0 con 0x55c81d2d0d90 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f72e0000e40 con 0x55c81d2d0d90 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e0028780 con 0x55c81d2d0d90 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f730b056b80 10 client.29825 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55c81d046250 con 0x55c81d2d0d90 2023-07-21T19:32:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.081+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f72e00044c0 con 0x55c81d2d0d90 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f72e001c020 con 0x55c81d2d0d90 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 10 client.29825 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 10 client.29825 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 20 client.29825 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d33f540 0x55c81d341950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 4 client.29825 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c81d1d4ff0 con 0x55c81d33f540 2023-07-21T19:32:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.082+0000 7f72f559b700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d33f540 0x55c81d341950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.083+0000 7f72f559b700 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d33f540 0x55c81d341950 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c800bc4030a7a9b server_cookie=cc7928349a9bdee7 in_seq=0 out_seq=0 2023-07-21T19:32:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.083+0000 7f72d8ff9700 10 client.29825 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f72d8ff9700 1 -- 172.21.15.5:0/3421516468 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c81d1d4ff0 con 0x55c81d33f540 2023-07-21T19:32:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f72d8ff9700 10 client.29825 handle_command_reply: tid=0 2023-07-21T19:32:39.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 client.29825 shutdown 2023-07-21T19:32:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 msgr2=0x55c81d2e1680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f72e0016010 tx=0x7f72e0005bc0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 shutdown_connections 2023-07-21T19:32:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c81d33f540 0x55c81d341950 crc :-1 s=READY pgs=480 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c81d2d6ae0 0x55c81d2df600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c81d1246b0 0x55c81d2e1140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 --2- 172.21.15.5:0/3421516468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c81d2d0d90 0x55c81d2e1680 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 >> 172.21.15.5:0/3421516468 conn(0x55c81d2bdbe0 msgr2=0x55c81d2d79a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f72d8ff9700 0 client.29825 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 shutdown_connections 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.103+0000 7f730b056b80 1 -- 172.21.15.5:0/3421516468 wait complete. 2023-07-21T19:32:39.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.104+0000 7f730b056b80 20 client.29825 trim_cache size 0 max 16384 2023-07-21T19:32:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:32:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:32:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:32:39.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:32:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 msgr2=0x7f72dc0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 0x7f72dc0a3940 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0098c0 tx=0x7f72ec019040 comp rx=0 tx=0).stop 2023-07-21T19:32:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 msgr2=0x7f72f019c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f72e800ef30 tx=0x7f72e800c450 comp rx=0 tx=0).stop 2023-07-21T19:32:39.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 shutdown_connections 2023-07-21T19:32:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f72f0104a30 0x7f72f019c960 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f72dc0a1490 0x7f72dc0a3940 secure :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0098c0 tx=0x7f72ec019040 comp rx=0 tx=0).stop 2023-07-21T19:32:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f72f00fea30 0x7f72f0197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 --2- 172.21.15.5:0/401756014 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f72f00ff3e0 0x7f72f01982f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:39.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.105+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 >> 172.21.15.5:0/401756014 conn(0x7f72f0074760 msgr2=0x7f72f0067940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.106+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 shutdown_connections 2023-07-21T19:32:39.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:39.106+0000 7f72f77ff700 1 -- 172.21.15.5:0/401756014 wait complete. 2023-07-21T19:32:39.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:38 smithi005 ceph-mon[126460]: pgmap v793: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 963 op/s 2023-07-21T19:32:39.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:38 smithi156 ceph-mon[128316]: pgmap v793: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 963 op/s 2023-07-21T19:32:41.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:41 smithi101 ceph-mon[128136]: pgmap v794: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 672 op/s 2023-07-21T19:32:41.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:41 smithi005 ceph-mon[126460]: pgmap v794: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 672 op/s 2023-07-21T19:32:41.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:41 smithi156 ceph-mon[128316]: pgmap v794: 97 pgs: 97 active+clean; 12 GiB data, 14 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.1 MiB/s wr, 672 op/s 2023-07-21T19:32:41.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 370.0 s, 263.2 tps, lat 376.494 ms stddev 96.537 2023-07-21T19:32:43.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:43 smithi101 ceph-mon[128136]: pgmap v795: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 940 op/s 2023-07-21T19:32:43.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:43 smithi005 ceph-mon[126460]: pgmap v795: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 940 op/s 2023-07-21T19:32:43.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:43 smithi156 ceph-mon[128316]: pgmap v795: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 10 MiB/s wr, 940 op/s 2023-07-21T19:32:45.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:45 smithi101 ceph-mon[128136]: pgmap v796: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 687 op/s 2023-07-21T19:32:45.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:45 smithi005 ceph-mon[126460]: pgmap v796: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 687 op/s 2023-07-21T19:32:45.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:45 smithi156 ceph-mon[128316]: pgmap v796: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.0 MiB/s wr, 687 op/s 2023-07-21T19:32:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 375.0 s, 269.4 tps, lat 375.805 ms stddev 95.920 2023-07-21T19:32:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:47 smithi005 ceph-mon[126460]: pgmap v797: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2023-07-21T19:32:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:47 smithi156 ceph-mon[128316]: pgmap v797: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2023-07-21T19:32:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:47 smithi101 ceph-mon[128136]: pgmap v797: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.7 MiB/s wr, 630 op/s 2023-07-21T19:32:49.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:48 smithi101 ceph-mon[128136]: pgmap v798: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-07-21T19:32:49.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:48 smithi005 ceph-mon[126460]: pgmap v798: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-07-21T19:32:49.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:48 smithi156 ceph-mon[128316]: pgmap v798: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 10 MiB/s wr, 988 op/s 2023-07-21T19:32:50.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:32:50.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:32:50.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:32:51.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:50 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:32:51.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:50 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:32:51.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:50 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:32:51.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:50 smithi101 ceph-mon[128136]: pgmap v799: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 682 op/s 2023-07-21T19:32:51.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:50 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:32:51.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:50 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:32:51.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:50 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:32:51.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:50 smithi005 ceph-mon[126460]: pgmap v799: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 682 op/s 2023-07-21T19:32:51.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:50 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:32:51.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:50 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:32:51.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:50 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:32:51.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:50 smithi156 ceph-mon[128316]: pgmap v799: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 682 op/s 2023-07-21T19:32:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 380.0 s, 261.0 tps, lat 381.699 ms stddev 60.118 2023-07-21T19:32:52.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 -- 172.21.15.5:0/4233891216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 msgr2=0x7f1934105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 --2- 172.21.15.5:0/4233891216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f1934105ee0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f1928009990 tx=0x7f1928009ca0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 -- 172.21.15.5:0/4233891216 shutdown_connections 2023-07-21T19:32:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 --2- 172.21.15.5:0/4233891216 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19341004b0 0x7f193410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 --2- 172.21.15.5:0/4233891216 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f1934105ee0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 --2- 172.21.15.5:0/4233891216 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19340ffb00 0x7f19340fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 -- 172.21.15.5:0/4233891216 >> 172.21.15.5:0/4233891216 conn(0x7f19340fb600 msgr2=0x7f19340fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 -- 172.21.15.5:0/4233891216 shutdown_connections 2023-07-21T19:32:52.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.349+0000 7f193bbff700 1 -- 172.21.15.5:0/4233891216 wait complete. 2023-07-21T19:32:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.351+0000 7f193bbff700 1 Processor -- start 2023-07-21T19:32:52.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.351+0000 7f193bbff700 1 -- start start 2023-07-21T19:32:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 0x7f1934197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19341004b0 0x7f19341982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f193419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193419cf70 con 0x7f1934105b10 2023-07-21T19:32:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193419d0e0 con 0x7f19340ffb00 2023-07-21T19:32:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.352+0000 7f193bbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193419d2a0 con 0x7f19341004b0 2023-07-21T19:32:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 0x7f1934197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 0x7f1934197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51330/0 (socket says 172.21.15.5:51330) 2023-07-21T19:32:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193999b700 1 -- 172.21.15.5:0/772145879 learned_addr learned my addr 172.21.15.5:0/772145879 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f193419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193919a700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19341004b0 0x7f19341982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 -- 172.21.15.5:0/772145879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19341004b0 msgr2=0x7f19341982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19341004b0 0x7f19341982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 -- 172.21.15.5:0/772145879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 msgr2=0x7f1934197d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 0x7f1934197d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.353+0000 7f193a19c700 1 -- 172.21.15.5:0/772145879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19280095f0 con 0x7f1934105b10 2023-07-21T19:32:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.354+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f193419c930 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e970 tx=0x7f192c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.355+0000 7f1926ffd700 1 -- 172.21.15.5:0/772145879 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c004220 con 0x7f1934105b10 2023-07-21T19:32:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.355+0000 7f1926ffd700 1 -- 172.21.15.5:0/772145879 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f192c004cc0 con 0x7f1934105b10 2023-07-21T19:32:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.355+0000 7f1926ffd700 1 -- 172.21.15.5:0/772145879 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c005590 con 0x7f1934105b10 2023-07-21T19:32:52.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.355+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193419d580 con 0x7f1934105b10 2023-07-21T19:32:52.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.355+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19341a4dc0 con 0x7f1934105b10 2023-07-21T19:32:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.360+0000 7f1926ffd700 1 -- 172.21.15.5:0/772145879 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f192c004750 con 0x7f1934105b10 2023-07-21T19:32:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.360+0000 7f194f490b80 1 Processor -- start 2023-07-21T19:32:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 -- start start 2023-07-21T19:32:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac59f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac49cc50 con 0x55d9ac3f36b0 2023-07-21T19:32:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac4ef9e0 con 0x55d9ac5a5ae0 2023-07-21T19:32:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.361+0000 7f194f490b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac4e8e70 con 0x55d9ac59fd90 2023-07-21T19:32:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac59f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac59f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51338/0 (socket says 172.21.15.5:51338) 2023-07-21T19:32:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f193999b700 1 -- 172.21.15.5:0/935885510 learned_addr learned my addr 172.21.15.5:0/935885510 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f193919a700 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f193919a700 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.362+0000 7f1926ffd700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 0x7f19200a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f193919a700 1 -- 172.21.15.5:0/935885510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 msgr2=0x55d9ac5a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f193919a700 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f193919a700 1 -- 172.21.15.5:0/935885510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 msgr2=0x55d9ac59f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f193919a700 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac59f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f193919a700 1 -- 172.21.15.5:0/935885510 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ac316d80 con 0x55d9ac3f36b0 2023-07-21T19:32:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.363+0000 7f1926ffd700 1 -- 172.21.15.5:0/772145879 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f192c014070 con 0x7f1934105b10 2023-07-21T19:32:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.364+0000 7f193919a700 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a55a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1930008c10 tx=0x7f1930009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66508cd9cb5db490 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.364+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 0x7f19200a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.364+0000 7f191ffff700 1 -- 172.21.15.5:0/935885510 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000e050 con 0x55d9ac3f36b0 2023-07-21T19:32:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.364+0000 7f191ffff700 1 -- 172.21.15.5:0/935885510 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f193000f040 con 0x55d9ac3f36b0 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.364+0000 7f191ffff700 1 -- 172.21.15.5:0/935885510 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930018b70 con 0x55d9ac3f36b0 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.365+0000 7f194f490b80 1 -- 172.21.15.5:0/935885510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 msgr2=0x55d9ac5a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.365+0000 7f194f490b80 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a55a0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1930008c10 tx=0x7f1930009f90 comp rx=0 tx=0).stop 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.365+0000 7f193a19c700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 0x7f19200a3940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f192c0a1920 tx=0x7f192c005020 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 -- 172.21.15.5:0/935885510 shutdown_connections 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a55a0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 --2- 172.21.15.5:0/935885510 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac59f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 -- 172.21.15.5:0/935885510 >> 172.21.15.5:0/935885510 conn(0x55d9ac58cbe0 msgr2=0x55d9ac591f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 -- 172.21.15.5:0/935885510 shutdown_connections 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.366+0000 7f194f490b80 1 -- 172.21.15.5:0/935885510 wait complete. 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.367+0000 7f194f490b80 1 -- start start 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.367+0000 7f194f490b80 1 Processor -- start 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.368+0000 7f194f490b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.368+0000 7f194f490b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a9610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f194f490b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a9b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f194f490b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac488a60 con 0x55d9ac59fd90 2023-07-21T19:32:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f194f490b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac488760 con 0x55d9ac3f36b0 2023-07-21T19:32:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f194f490b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac36c8c0 con 0x55d9ac5a5ae0 2023-07-21T19:32:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f194f490b80 10 client.0 _ll_register_callbacks cb 0x55d9ac486700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f193919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f193919a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51348/0 (socket says 172.21.15.5:51348) 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f193919a700 1 -- 172.21.15.5:0/3838801100 learned_addr learned my addr 172.21.15.5:0/3838801100 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f193919a700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.369+0000 7f193999b700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 -- 172.21.15.5:0/3838801100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 msgr2=0x55d9ac5a9b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a9b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 -- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 msgr2=0x55d9ac5a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 -- 172.21.15.5:0/3838801100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300088f0 con 0x55d9ac59fd90 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f193999b700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a9610 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1928005800 tx=0x7f1928005c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fafbf1e2799ea68e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.370+0000 7f191d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192801d070 con 0x55d9ac59fd90 2023-07-21T19:32:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1928004500 con 0x55d9ac59fd90 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d9ac442490 con 0x55d9ac59fd90 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f19280046c0 con 0x55d9ac59fd90 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f194f490b80 10 client.20055 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.371+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55d9ac2ffc00 con 0x55d9ac59fd90 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d9ac442490 con 0x55d9ac59fd90 2023-07-21T19:32:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f192800f940 con 0x55d9ac59fd90 2023-07-21T19:32:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 10 client.20055 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 10 client.20055 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 20 client.20055 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac5b93b0 0x55d9ac5bb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 4 client.20055 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.372+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d9ac4e1e50 con 0x55d9ac5b93b0 2023-07-21T19:32:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.373+0000 7f193919a700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac5b93b0 0x55d9ac5bb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.373+0000 7f193919a700 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac5b93b0 0x55d9ac5bb7c0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3345c2e984e88be3 server_cookie=2554823d8913c7de in_seq=0 out_seq=0 2023-07-21T19:32:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.373+0000 7f191d7fa700 10 client.20055 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.374+0000 7f191d7fa700 1 -- 172.21.15.5:0/3838801100 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d9ac4e1e50 con 0x55d9ac5b93b0 2023-07-21T19:32:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.374+0000 7f191d7fa700 10 client.20055 handle_command_reply: tid=0 2023-07-21T19:32:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.374+0000 7f194f490b80 1 client.20055 shutdown 2023-07-21T19:32:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 msgr2=0x55d9ac5a9610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a9610 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f1928005800 tx=0x7f1928005c40 comp rx=0 tx=0).stop 2023-07-21T19:32:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 shutdown_connections 2023-07-21T19:32:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a9b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac5b93b0 0x55d9ac5bb7c0 crc :-1 s=READY pgs=481 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 --2- 172.21.15.5:0/3838801100 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac59fd90 0x55d9ac5a9610 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f191d7fa700 0 client.20055 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.375+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 >> 172.21.15.5:0/3838801100 conn(0x55d9ac58cbe0 msgr2=0x55d9ac5914e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.376+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 shutdown_connections 2023-07-21T19:32:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.376+0000 7f194f490b80 1 -- 172.21.15.5:0/3838801100 wait complete. 2023-07-21T19:32:52.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.376+0000 7f194f490b80 20 client.20055 trim_cache size 0 max 16384 2023-07-21T19:32:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.398+0000 7f194f490b80 1 Processor -- start 2023-07-21T19:32:52.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.398+0000 7f194f490b80 1 -- start start 2023-07-21T19:32:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac59f850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac5a5220 con 0x55d9ac3f36b0 2023-07-21T19:32:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac5a5eb0 con 0x55d9ac59fd90 2023-07-21T19:32:52.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f194f490b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac593ab0 con 0x55d9ac5a5ae0 2023-07-21T19:32:52.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f193999b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51354/0 (socket says 172.21.15.5:51354) 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f193999b700 1 -- 172.21.15.5:0/3118349733 learned_addr learned my addr 172.21.15.5:0/3118349733 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f193919a700 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.399+0000 7f193919a700 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac59f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 -- 172.21.15.5:0/3118349733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 msgr2=0x55d9ac5a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 -- 172.21.15.5:0/3118349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 msgr2=0x55d9ac59f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac59f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 -- 172.21.15.5:0/3118349733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d9ac33d1f0 con 0x55d9ac59fd90 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.400+0000 7f193999b700 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928001e70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e8c738e53bb7621a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.401+0000 7f1924bff700 1 -- 172.21.15.5:0/3118349733 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192801d070 con 0x55d9ac59fd90 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.401+0000 7f1924bff700 1 -- 172.21.15.5:0/3118349733 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f192801e040 con 0x55d9ac59fd90 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.401+0000 7f1924bff700 1 -- 172.21.15.5:0/3118349733 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192800e920 con 0x55d9ac59fd90 2023-07-21T19:32:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 -- 172.21.15.5:0/3118349733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 msgr2=0x55d9ac5b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f1928000c00 tx=0x7f1928001e70 comp rx=0 tx=0).stop 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 -- 172.21.15.5:0/3118349733 shutdown_connections 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac5a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac59f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 --2- 172.21.15.5:0/3118349733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac59fd90 0x55d9ac5b0470 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 -- 172.21.15.5:0/3118349733 >> 172.21.15.5:0/3118349733 conn(0x55d9ac58cbe0 msgr2=0x55d9ac5a42a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 -- 172.21.15.5:0/3118349733 shutdown_connections 2023-07-21T19:32:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.402+0000 7f194f490b80 1 -- 172.21.15.5:0/3118349733 wait complete. 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.403+0000 7f194f490b80 1 -- start start 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.403+0000 7f194f490b80 1 Processor -- start 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5ae650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac60c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac60c7b0 con 0x55d9ac3f36b0 2023-07-21T19:32:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac60c8f0 con 0x55d9ac5a5ae0 2023-07-21T19:32:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d9ac60ca30 con 0x55d9ac59fd90 2023-07-21T19:32:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f194f490b80 10 client.0 _ll_register_callbacks cb 0x7f1928005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f193999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f193999b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46670/0 (socket says 172.21.15.5:46670) 2023-07-21T19:32:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f193999b700 1 -- 172.21.15.5:0/304483829 learned_addr learned my addr 172.21.15.5:0/304483829 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f193919a700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac60c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.404+0000 7f193919a700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5ae650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 -- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 msgr2=0x55d9ac60c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac60c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 -- 172.21.15.5:0/304483829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 msgr2=0x55d9ac5ae650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5ae650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 -- 172.21.15.5:0/304483829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d9ac2ffc00 con 0x55d9ac59fd90 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f193999b700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ce0 tx=0x7f192800ba80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=309333dbde1746b8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.405+0000 7f191d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192801d040 con 0x55d9ac59fd90 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f19280245a0 con 0x55d9ac59fd90 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f193000f040 con 0x55d9ac59fd90 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1928024760 con 0x55d9ac59fd90 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f194f490b80 10 client.29752 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55d9ac315250 con 0x55d9ac59fd90 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.406+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f192804c080 con 0x55d9ac59fd90 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f19280248c0 con 0x55d9ac59fd90 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 10 client.29752 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 10 client.29752 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 20 client.29752 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac60dc50 0x55d9ac610060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 4 client.29752 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:32:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d9ac4a3ff0 con 0x55d9ac60dc50 2023-07-21T19:32:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.408+0000 7f193919a700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac60dc50 0x55d9ac610060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.409+0000 7f193919a700 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac60dc50 0x55d9ac610060 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c1772be9e7ee84ee server_cookie=5306d7e200746917 in_seq=0 out_seq=0 2023-07-21T19:32:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.409+0000 7f191d7fa700 10 client.29752 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:52.418 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f191d7fa700 1 -- 172.21.15.5:0/304483829 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d9ac4a3ff0 con 0x55d9ac60dc50 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f191d7fa700 10 client.29752 handle_command_reply: tid=0 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 client.29752 shutdown 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 msgr2=0x55d9ac5aeb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f1928009ce0 tx=0x7f192800ba80 comp rx=0 tx=0).stop 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 shutdown_connections 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d9ac5a5ae0 0x55d9ac60c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.419 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.431+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d9ac60dc50 0x55d9ac610060 crc :-1 s=READY pgs=482 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d9ac3f36b0 0x55d9ac5ae650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 1 --2- 172.21.15.5:0/304483829 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d9ac59fd90 0x55d9ac5aeb90 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f191d7fa700 0 client.29752 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:52.420 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 >> 172.21.15.5:0/304483829 conn(0x55d9ac58cbe0 msgr2=0x55d9ac5a8130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 shutdown_connections 2023-07-21T19:32:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 1 -- 172.21.15.5:0/304483829 wait complete. 2023-07-21T19:32:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.432+0000 7f194f490b80 20 client.29752 trim_cache size 0 max 16384 2023-07-21T19:32:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:32:52.421 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:32:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:32:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:32:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 msgr2=0x7f19200a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.422 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 0x7f19200a3940 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f192c0a1920 tx=0x7f192c005020 comp rx=0 tx=0).stop 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 msgr2=0x7f193419c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f193419c930 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f192c00e970 tx=0x7f192c00ec80 comp rx=0 tx=0).stop 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 shutdown_connections 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f19200a1490 0x7f19200a3940 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1934105b10 0x7f193419c930 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19340ffb00 0x7f1934197d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.423 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 --2- 172.21.15.5:0/772145879 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19341004b0 0x7f19341982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 >> 172.21.15.5:0/772145879 conn(0x7f19340fb600 msgr2=0x7f193410ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.433+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 shutdown_connections 2023-07-21T19:32:52.424 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:32:52.434+0000 7f193bbff700 1 -- 172.21.15.5:0/772145879 wait complete. 2023-07-21T19:32:53.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:53 smithi101 ceph-mon[128136]: pgmap v800: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-07-21T19:32:53.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:53 smithi005 ceph-mon[126460]: pgmap v800: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-07-21T19:32:53.937 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:53 smithi156 ceph-mon[128316]: pgmap v800: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 9.7 MiB/s wr, 978 op/s 2023-07-21T19:32:55.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:55 smithi101 ceph-mon[128136]: pgmap v801: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 776 op/s 2023-07-21T19:32:55.850 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:32:55.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:55 smithi005 ceph-mon[126460]: pgmap v801: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 776 op/s 2023-07-21T19:32:56.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.210+0000 7f1b703ff700 1 -- 172.21.15.5:0/1088516395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b6810a070 msgr2=0x7f1b6810c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.210+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1088516395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b6810a070 0x7f1b6810c4b0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b6000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 -- 172.21.15.5:0/1088516395 shutdown_connections 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1088516395 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b6810a070 0x7f1b6810c4b0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1088516395 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b68100ca0 0x7f1b68101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1088516395 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b68109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 -- 172.21.15.5:0/1088516395 >> 172.21.15.5:0/1088516395 conn(0x7f1b680fa4b0 msgr2=0x7f1b680fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 -- 172.21.15.5:0/1088516395 shutdown_connections 2023-07-21T19:32:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.211+0000 7f1b703ff700 1 -- 172.21.15.5:0/1088516395 wait complete. 2023-07-21T19:32:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.212+0000 7f1b703ff700 1 Processor -- start 2023-07-21T19:32:56.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.212+0000 7f1b703ff700 1 -- start start 2023-07-21T19:32:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 0x7f1b680fe8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b680fee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b6810a070 0x7f1b681a3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b680ff660 con 0x7f1b68100ca0 2023-07-21T19:32:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681a43f0 con 0x7f1b681015b0 2023-07-21T19:32:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.213+0000 7f1b703ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b681a4530 con 0x7f1b6810a070 2023-07-21T19:32:56.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6e19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 0x7f1b680fe8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b680fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6e19b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 0x7f1b680fe8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54992/0 (socket says 172.21.15.5:54992) 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6e19b700 1 -- 172.21.15.5:0/1771500335 learned_addr learned my addr 172.21.15.5:0/1771500335 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b6810a070 0x7f1b681a3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 -- 172.21.15.5:0/1771500335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b6810a070 msgr2=0x7f1b681a3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b6810a070 0x7f1b681a3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 -- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 msgr2=0x7f1b680fe8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 0x7f1b680fe8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.214+0000 7f1b6d99a700 1 -- 172.21.15.5:0/1771500335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b64009520 con 0x7f1b681015b0 2023-07-21T19:32:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.215+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b680fee30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e9f0 tx=0x7f1b5800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.215+0000 7f1b5f7fe700 1 -- 172.21.15.5:0/1771500335 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b5800c9c0 con 0x7f1b681015b0 2023-07-21T19:32:56.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.215+0000 7f1b5f7fe700 1 -- 172.21.15.5:0/1771500335 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b58010430 con 0x7f1b681015b0 2023-07-21T19:32:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.215+0000 7f1b5f7fe700 1 -- 172.21.15.5:0/1771500335 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58018680 con 0x7f1b681015b0 2023-07-21T19:32:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.216+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b60009d00 con 0x7f1b681015b0 2023-07-21T19:32:56.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.216+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b681a4a90 con 0x7f1b681015b0 2023-07-21T19:32:56.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.222+0000 7f1b5f7fe700 1 -- 172.21.15.5:0/1771500335 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1b580105a0 con 0x7f1b681015b0 2023-07-21T19:32:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.223+0000 7f1b83d97b80 1 Processor -- start 2023-07-21T19:32:56.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.223+0000 7f1b83d97b80 1 -- start start 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e80be0 0x563338e8b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e8bad0 0x563338e8deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338d86cb0 con 0x563338e8bad0 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338dd9a40 con 0x563338e802d0 2023-07-21T19:32:56.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338dd2ed0 con 0x563338e80be0 2023-07-21T19:32:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b6e99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b6e99c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51362/0 (socket says 172.21.15.5:51362) 2023-07-21T19:32:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b6e99c700 1 -- 172.21.15.5:0/4094353532 learned_addr learned my addr 172.21.15.5:0/4094353532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:56.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.224+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e80be0 0x563338e8b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e8bad0 0x563338e8deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6e99c700 1 -- 172.21.15.5:0/4094353532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e80be0 msgr2=0x563338e8b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e80be0 0x563338e8b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6e99c700 1 -- 172.21.15.5:0/4094353532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e8bad0 msgr2=0x563338e8deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e8bad0 0x563338e8deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.225+0000 7f1b6e99c700 1 -- 172.21.15.5:0/4094353532 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563338c00de0 con 0x563338e802d0 2023-07-21T19:32:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b60009c40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87e4ba60537f56af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b5f7fe700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 0x7f1b540a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b5cbfe700 1 -- 172.21.15.5:0/4094353532 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000e040 con 0x563338e802d0 2023-07-21T19:32:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b5cbfe700 1 -- 172.21.15.5:0/4094353532 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b600094c0 con 0x563338e802d0 2023-07-21T19:32:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b5cbfe700 1 -- 172.21.15.5:0/4094353532 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b60003d30 con 0x563338e802d0 2023-07-21T19:32:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.226+0000 7f1b5f7fe700 1 -- 172.21.15.5:0/1771500335 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1b58014070 con 0x7f1b681015b0 2023-07-21T19:32:56.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.227+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 0x7f1b540a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.227+0000 7f1b83d97b80 1 -- 172.21.15.5:0/4094353532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 msgr2=0x563338e806a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.227+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b60009c40 comp rx=0 tx=0).stop 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 -- 172.21.15.5:0/4094353532 shutdown_connections 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e8bad0 0x563338e8deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e806a0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/4094353532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e80be0 0x563338e8b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 -- 172.21.15.5:0/4094353532 >> 172.21.15.5:0/4094353532 conn(0x563338e76da0 msgr2=0x563338e7c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 0x7f1b540a37f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b64012510 tx=0x7f1b64009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 -- 172.21.15.5:0/4094353532 shutdown_connections 2023-07-21T19:32:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.228+0000 7f1b83d97b80 1 -- 172.21.15.5:0/4094353532 wait complete. 2023-07-21T19:32:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.229+0000 7f1b83d97b80 1 -- start start 2023-07-21T19:32:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.229+0000 7f1b83d97b80 1 Processor -- start 2023-07-21T19:32:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.230+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 0x563338e8b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.230+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.230+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.230+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338d72ac0 con 0x563338e802d0 2023-07-21T19:32:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.230+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338d727c0 con 0x563338e80be0 2023-07-21T19:32:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338c56920 con 0x563338e8bad0 2023-07-21T19:32:56.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b83d97b80 10 client.0 _ll_register_callbacks cb 0x563338d70760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b6d99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b6e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b6d99a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51366/0 (socket says 172.21.15.5:51366) 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.231+0000 7f1b6d99a700 1 -- 172.21.15.5:0/2991772123 learned_addr learned my addr 172.21.15.5:0/2991772123 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 0x563338e8b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 -- 172.21.15.5:0/2991772123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 msgr2=0x563338e88ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 -- 172.21.15.5:0/2991772123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 msgr2=0x563338e8b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 0x563338e8b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 -- 172.21.15.5:0/2991772123 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6810d170 con 0x563338e80be0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 0x563338e8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009230 tx=0x7f1b580caad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85766176a81a2432 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.232+0000 7f1b51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b58099ef0 con 0x563338e80be0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1b5809df20 con 0x563338e80be0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563338dcbeb0 con 0x563338e80be0 2023-07-21T19:32:56.222 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b580a2340 con 0x563338e80be0 2023-07-21T19:32:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b83d97b80 10 client.29855 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:56.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.233+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563338be9c60 con 0x563338e80be0 2023-07-21T19:32:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563338dcbeb0 con 0x563338e80be0 2023-07-21T19:32:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b58094b00 con 0x563338e80be0 2023-07-21T19:32:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 10 client.29855 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 10 client.29855 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:56.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 20 client.29855 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338eac2e0 0x563338eae6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 4 client.29855 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563338d8e050 con 0x563338eac2e0 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.234+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338eac2e0 0x563338eae6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.235+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338eac2e0 0x563338eae6f0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4923974cdab7333b server_cookie=9cad801b12b786b5 in_seq=0 out_seq=0 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.235+0000 7f1b51ffb700 10 client.29855 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.236+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2991772123 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563338d8e050 con 0x563338eac2e0 2023-07-21T19:32:56.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.236+0000 7f1b51ffb700 10 client.29855 handle_command_reply: tid=0 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.236+0000 7f1b83d97b80 1 client.29855 shutdown 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 msgr2=0x563338e88790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1b58009230 tx=0x7f1b580caad0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 shutdown_connections 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338eac2e0 0x563338eae6f0 crc :-1 s=READY pgs=483 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e802d0 0x563338e8b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.226 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2991772123 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e80be0 0x563338e88790 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 >> 172.21.15.5:0/2991772123 conn(0x563338e76da0 msgr2=0x563338e7c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.227 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b51ffb700 0 client.29855 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 shutdown_connections 2023-07-21T19:32:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.237+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2991772123 wait complete. 2023-07-21T19:32:56.228 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.238+0000 7f1b83d97b80 20 client.29855 trim_cache size 0 max 16384 2023-07-21T19:32:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 Processor -- start 2023-07-21T19:32:56.251 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 -- start start 2023-07-21T19:32:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.252 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e88030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338e7dd20 con 0x563338e80be0 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.264+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338e81540 con 0x563338e802d0 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338eac0c0 con 0x563338e8bad0 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51368/0 (socket says 172.21.15.5:51368) 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 -- 172.21.15.5:0/1335466475 learned_addr learned my addr 172.21.15.5:0/1335466475 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:56.253 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e88030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 -- 172.21.15.5:0/1335466475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 msgr2=0x563338e88570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 -- 172.21.15.5:0/1335466475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 msgr2=0x563338e88030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e88030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e19b700 1 -- 172.21.15.5:0/1335466475 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563338c27250 con 0x563338e802d0 2023-07-21T19:32:56.254 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.265+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:32:56.255 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.266+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b64012c60 tx=0x7f1b6401b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2481c219f3d61243 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.266+0000 7f1b5d3ff700 1 -- 172.21.15.5:0/1335466475 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64022040 con 0x563338e802d0 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.266+0000 7f1b5d3ff700 1 -- 172.21.15.5:0/1335466475 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b6401dd90 con 0x563338e802d0 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.266+0000 7f1b5d3ff700 1 -- 172.21.15.5:0/1335466475 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b64017070 con 0x563338e802d0 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 -- 172.21.15.5:0/1335466475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 msgr2=0x563338e8b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f1b64012c60 tx=0x7f1b6401b000 comp rx=0 tx=0).stop 2023-07-21T19:32:56.256 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 -- 172.21.15.5:0/1335466475 shutdown_connections 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e88570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e8b040 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/1335466475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e88030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 -- 172.21.15.5:0/1335466475 >> 172.21.15.5:0/1335466475 conn(0x563338e76da0 msgr2=0x563338e83a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 -- 172.21.15.5:0/1335466475 shutdown_connections 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.267+0000 7f1b83d97b80 1 -- 172.21.15.5:0/1335466475 wait complete. 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.268+0000 7f1b83d97b80 1 -- start start 2023-07-21T19:32:56.257 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.268+0000 7f1b83d97b80 1 Processor -- start 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e90a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e90fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338e92ab0 con 0x563338e80be0 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338f001b0 con 0x563338e802d0 2023-07-21T19:32:56.259 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563338f00370 con 0x563338e8bad0 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.269+0000 7f1b83d97b80 10 client.0 _ll_register_callbacks cb 0x7f1b58096230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46714/0 (socket says 172.21.15.5:46714) 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 -- 172.21.15.5:0/2840245529 learned_addr learned my addr 172.21.15.5:0/2840245529 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e90a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e90fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 -- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 msgr2=0x563338e90a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e90a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 -- 172.21.15.5:0/2840245529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 msgr2=0x563338e90fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e90fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6e99c700 1 -- 172.21.15.5:0/2840245529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b6402a9d0 con 0x563338e8bad0 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.270+0000 7f1b6d99a700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e90fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.271+0000 7f1b6e99c700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b60007dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84d76828a0932903 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.271+0000 7f1b51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:32:56.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.271+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b6000e040 con 0x563338e8bad0 2023-07-21T19:32:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.271+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1b60003a00 con 0x563338e8bad0 2023-07-21T19:32:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.271+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563338d2c4f0 con 0x563338e8bad0 2023-07-21T19:32:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.272+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b60003be0 con 0x563338e8bad0 2023-07-21T19:32:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.272+0000 7f1b83d97b80 10 client.29770 fetch_fsmap learned FSMap version 70 2023-07-21T19:32:56.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.272+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563338bff2b0 con 0x563338e8bad0 2023-07-21T19:32:56.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.272+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b60037020 con 0x563338e8bad0 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1b60019050 con 0x563338e8bad0 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 10 client.29770 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 10 client.29770 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 20 client.29770 populate_metadata read hostname 'smithi005' 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338f09f00 0x563338f0c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:32:56.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 4 client.29770 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:32:56.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563338c21f70 con 0x563338f09f00 2023-07-21T19:32:56.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.273+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338f09f00 0x563338f0c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:32:56.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.274+0000 7f1b6e19b700 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338f09f00 0x563338f0c310 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7b380e954dec18 server_cookie=cdb261f8bdb5ded0 in_seq=0 out_seq=0 2023-07-21T19:32:56.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.274+0000 7f1b51ffb700 10 client.29770 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:56.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b51ffb700 1 -- 172.21.15.5:0/2840245529 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563338c21f70 con 0x563338f09f00 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b51ffb700 10 client.29770 handle_command_reply: tid=0 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 client.29770 shutdown 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 msgr2=0x563338e914e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1b6000b0a0 tx=0x7f1b60007dc0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 shutdown_connections 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563338e8bad0 0x563338e914e0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563338e802d0 0x563338e90a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563338f09f00 0x563338f0c310 crc :-1 s=READY pgs=484 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 --2- 172.21.15.5:0/2840245529 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563338e80be0 0x563338e90fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b51ffb700 0 client.29770 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.275+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 >> 172.21.15.5:0/2840245529 conn(0x563338e76da0 msgr2=0x563338e88030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.276+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 shutdown_connections 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.276+0000 7f1b83d97b80 1 -- 172.21.15.5:0/2840245529 wait complete. 2023-07-21T19:32:56.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.276+0000 7f1b83d97b80 20 client.29770 trim_cache size 0 max 16384 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:32:56.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:32:56.269 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:32:56.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:32:56.269 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:32:56.269 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:32:56.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 msgr2=0x7f1b540a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 0x7f1b540a37f0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1b64012510 tx=0x7f1b64009260 comp rx=0 tx=0).stop 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 msgr2=0x7f1b680fee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b680fee30 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f1b5800e9f0 tx=0x7f1b5800ed00 comp rx=0 tx=0).stop 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 shutdown_connections 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b6810a070 0x7f1b681a3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1b540a1340 0x7f1b540a37f0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b68100ca0 0x7f1b680fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 --2- 172.21.15.5:0/1771500335 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b681015b0 0x7f1b680fee30 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:32:56.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.277+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 >> 172.21.15.5:0/1771500335 conn(0x7f1b680fa4b0 msgr2=0x7f1b6810c1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:32:56.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.278+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 shutdown_connections 2023-07-21T19:32:56.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:32:56.278+0000 7f1b703ff700 1 -- 172.21.15.5:0/1771500335 wait complete. 2023-07-21T19:32:56.292 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:32:56.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 385.0 s, 242.4 tps, lat 410.779 ms stddev 71.733 2023-07-21T19:32:57.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:57 smithi005 ceph-mon[126460]: pgmap v802: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2023-07-21T19:32:58.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:57 smithi101 ceph-mon[128136]: pgmap v802: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2023-07-21T19:32:58.715 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:55 smithi156 ceph-mon[128316]: pgmap v801: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 776 op/s 2023-07-21T19:32:58.716 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:57 smithi156 ceph-mon[128316]: pgmap v802: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.1 MiB/s wr, 720 op/s 2023-07-21T19:32:59.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:32:58 smithi101 ceph-mon[128136]: pgmap v803: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:32:59.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:32:58 smithi005 ceph-mon[126460]: pgmap v803: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:32:59.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:32:58 smithi156 ceph-mon[128316]: pgmap v803: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:33:01.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:01 smithi101 ceph-mon[128136]: pgmap v804: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T19:33:01.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:01 smithi005 ceph-mon[126460]: pgmap v804: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T19:33:01.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:01 smithi156 ceph-mon[128316]: pgmap v804: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.1 MiB/s wr, 716 op/s 2023-07-21T19:33:01.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 390.0 s, 265.2 tps, lat 383.014 ms stddev 104.426 2023-07-21T19:33:03.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:03 smithi101 ceph-mon[128136]: pgmap v805: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:33:03.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:03 smithi005 ceph-mon[126460]: pgmap v805: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:33:03.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:03 smithi156 ceph-mon[128316]: pgmap v805: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.681+0000 7faf0b1ff700 1 -- 172.21.15.5:0/753379512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf04067c10 msgr2=0x7faf04067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.681+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/753379512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf04067c10 0x7faf04067fe0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7faf000099c0 tx=0x7faf00009cd0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 -- 172.21.15.5:0/753379512 shutdown_connections 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/753379512 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 0x7faf0410ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/753379512 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf04067c10 0x7faf04067fe0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/753379512 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf040685b0 0x7faf04104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 -- 172.21.15.5:0/753379512 >> 172.21.15.5:0/753379512 conn(0x7faf04074760 msgr2=0x7faf04074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 -- 172.21.15.5:0/753379512 shutdown_connections 2023-07-21T19:33:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.682+0000 7faf0b1ff700 1 -- 172.21.15.5:0/753379512 wait complete. 2023-07-21T19:33:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.684+0000 7faf0b1ff700 1 Processor -- start 2023-07-21T19:33:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.684+0000 7faf0b1ff700 1 -- start start 2023-07-21T19:33:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 0x7faf041a15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 0x7faf0410b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410be20 con 0x7faf04067c10 2023-07-21T19:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410bf90 con 0x7faf040685b0 2023-07-21T19:33:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0b1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf0410c150 con 0x7faf04104e50 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faefbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 0x7faf041a15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53768/0 (socket says 172.21.15.5:53768) 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf08f9b700 1 -- 172.21.15.5:0/834697383 learned_addr learned my addr 172.21.15.5:0/834697383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.685+0000 7faf0979c700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 0x7faf0410b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 -- 172.21.15.5:0/834697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 msgr2=0x7faf0410b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 0x7faf0410b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 -- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 msgr2=0x7faf041a10b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 -- 172.21.15.5:0/834697383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf000095f0 con 0x7faf040685b0 2023-07-21T19:33:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faf08f9b700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:33:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.686+0000 7faefbfff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 0x7faf041a15f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faf04068980 tx=0x7faef400eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.687+0000 7faef9ffb700 1 -- 172.21.15.5:0/834697383 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef400cbd0 con 0x7faf040685b0 2023-07-21T19:33:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.687+0000 7faef9ffb700 1 -- 172.21.15.5:0/834697383 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faef400ee80 con 0x7faf040685b0 2023-07-21T19:33:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.687+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf0410c430 con 0x7faf040685b0 2023-07-21T19:33:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.687+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf041a9200 con 0x7faf040685b0 2023-07-21T19:33:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.687+0000 7faef9ffb700 1 -- 172.21.15.5:0/834697383 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4018680 con 0x7faf040685b0 2023-07-21T19:33:05.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.692+0000 7faef9ffb700 1 -- 172.21.15.5:0/834697383 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7faef400cd30 con 0x7faf040685b0 2023-07-21T19:33:05.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.693+0000 7faf1eb8cb80 1 Processor -- start 2023-07-21T19:33:05.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.695+0000 7faf1eb8cb80 1 -- start start 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca24976b0 0x55eca2640d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca263b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca2540c50 con 0x55eca263b580 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca25939e0 con 0x55eca24976b0 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.696+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca258ce70 con 0x55eca26412d0 2023-07-21T19:33:05.684 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faefbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca263b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58060/0 (socket says 172.21.15.5:58060) 2023-07-21T19:33:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faf08f9b700 1 -- 172.21.15.5:0/4103381312 learned_addr learned my addr 172.21.15.5:0/4103381312 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faef9ffb700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 0x7faeec0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.685 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.697+0000 7faef9ffb700 1 -- 172.21.15.5:0/834697383 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faef4014070 con 0x7faf040685b0 2023-07-21T19:33:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 -- 172.21.15.5:0/4103381312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca24976b0 msgr2=0x55eca2640d90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:33:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca24976b0 0x55eca2640d90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 -- 172.21.15.5:0/4103381312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 msgr2=0x55eca263b9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca263b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.686 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 -- 172.21.15.5:0/4103381312 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca23bad80 con 0x55eca26412d0 2023-07-21T19:33:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.698+0000 7faf08f9b700 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faf00005910 tx=0x7faf00005c80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8514cd719d26cb5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.699+0000 7faeeaffd700 1 -- 172.21.15.5:0/4103381312 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0001d040 con 0x55eca26412d0 2023-07-21T19:33:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.699+0000 7faeeaffd700 1 -- 172.21.15.5:0/4103381312 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf0000b840 con 0x55eca26412d0 2023-07-21T19:33:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.699+0000 7faeeaffd700 1 -- 172.21.15.5:0/4103381312 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf00003b10 con 0x55eca26412d0 2023-07-21T19:33:05.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.700+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/4103381312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 msgr2=0x55eca263b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.700+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7faf00005910 tx=0x7faf00005c80 comp rx=0 tx=0).stop 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/4103381312 shutdown_connections 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca263b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca24976b0 0x55eca2640d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/4103381312 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca26412d0 0x55eca263b040 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/4103381312 >> 172.21.15.5:0/4103381312 conn(0x55eca2630be0 msgr2=0x55eca2635f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.701+0000 7faf0979c700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 0x7faeec0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.702+0000 7faf0979c700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 0x7faeec0a37f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faefc011460 tx=0x7faefc00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.702+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/4103381312 shutdown_connections 2023-07-21T19:33:05.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.702+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/4103381312 wait complete. 2023-07-21T19:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.703+0000 7faf1eb8cb80 1 -- start start 2023-07-21T19:33:05.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 Processor -- start 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca2644800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca26412d0 0x55eca2644de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca252ca60 con 0x55eca263b580 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca252c760 con 0x55eca26412d0 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.704+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca24108c0 con 0x55eca24976b0 2023-07-21T19:33:05.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faf1eb8cb80 10 client.0 _ll_register_callbacks cb 0x55eca252a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58072/0 (socket says 172.21.15.5:58072) 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faf08f9b700 1 -- 172.21.15.5:0/591791299 learned_addr learned my addr 172.21.15.5:0/591791299 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faf0979c700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca26412d0 0x55eca2644de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.705+0000 7faefbfff700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca2644800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 -- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca26412d0 msgr2=0x55eca2644de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca26412d0 0x55eca2644de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 -- 172.21.15.5:0/591791299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 msgr2=0x55eca2644800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca2644800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 -- 172.21.15.5:0/591791299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf0410d210 con 0x55eca24976b0 2023-07-21T19:33:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faf08f9b700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf00009510 tx=0x7faf00009310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a917d2c591c0b83a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.706+0000 7faee3fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:33:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf0001d040 con 0x55eca24976b0 2023-07-21T19:33:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faf00005480 con 0x55eca24976b0 2023-07-21T19:33:05.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eca24e6490 con 0x55eca24976b0 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7faf000038b0 con 0x55eca24976b0 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faf1eb8cb80 10 client.29782 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55eca23a3c00 con 0x55eca24976b0 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.707+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf00038020 con 0x55eca24976b0 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.708+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faf00023b20 con 0x55eca24976b0 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.708+0000 7faf1eb8cb80 10 client.29782 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.708+0000 7faf1eb8cb80 10 client.29782 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:05.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.709+0000 7faf1eb8cb80 20 client.29782 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.709+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26765c0 0x55eca26789a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.709+0000 7faf1eb8cb80 4 client.29782 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.709+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eca2585e50 con 0x55eca26765c0 2023-07-21T19:33:05.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.709+0000 7faefbfff700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26765c0 0x55eca26789a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.710+0000 7faefbfff700 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26765c0 0x55eca26789a0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8c08445cb7cf10 server_cookie=edaed6262c231eec in_seq=0 out_seq=0 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.710+0000 7faee3fff700 10 client.29782 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.711+0000 7faee3fff700 1 -- 172.21.15.5:0/591791299 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eca2585e50 con 0x55eca26765c0 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.711+0000 7faee3fff700 10 client.29782 handle_command_reply: tid=0 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.711+0000 7faf1eb8cb80 1 client.29782 shutdown 2023-07-21T19:33:05.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.711+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 msgr2=0x55eca2647510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.711+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7faf00009510 tx=0x7faf00009310 comp rx=0 tx=0).stop 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 shutdown_connections 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca26412d0 0x55eca2644de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2647510 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26765c0 0x55eca26789a0 crc :-1 s=READY pgs=485 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/591791299 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca263b580 0x55eca2644800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 >> 172.21.15.5:0/591791299 conn(0x55eca2630be0 msgr2=0x55eca2634c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faee3fff700 0 client.29782 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 shutdown_connections 2023-07-21T19:33:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.712+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/591791299 wait complete. 2023-07-21T19:33:05.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.713+0000 7faf1eb8cb80 20 client.29782 trim_cache size 0 max 16384 2023-07-21T19:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 Processor -- start 2023-07-21T19:33:05.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 -- start start 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca265cc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2635c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca26470b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca2632d00 con 0x55eca2630be0 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca2636b70 con 0x55eca263b580 2023-07-21T19:33:05.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.737+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca26325f0 con 0x55eca24976b0 2023-07-21T19:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca265cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca265cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53788/0 (socket says 172.21.15.5:53788) 2023-07-21T19:33:05.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faf08f9b700 1 -- 172.21.15.5:0/3065005810 learned_addr learned my addr 172.21.15.5:0/3065005810 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2635c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faf0979c700 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca26470b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 -- 172.21.15.5:0/3065005810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 msgr2=0x55eca26470b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca26470b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 -- 172.21.15.5:0/3065005810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 msgr2=0x55eca265cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca265cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.738+0000 7faefbfff700 1 -- 172.21.15.5:0/3065005810 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eca267d530 con 0x55eca24976b0 2023-07-21T19:33:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.739+0000 7faefbfff700 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2635c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faef40ca480 tx=0x7faef40cce00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5467e6a61110fb29 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.739+0000 7faeeb7fe700 1 -- 172.21.15.5:0/3065005810 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef409de30 con 0x55eca24976b0 2023-07-21T19:33:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.739+0000 7faeeb7fe700 1 -- 172.21.15.5:0/3065005810 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faef40a2250 con 0x55eca24976b0 2023-07-21T19:33:05.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faeeb7fe700 1 -- 172.21.15.5:0/3065005810 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4099550 con 0x55eca24976b0 2023-07-21T19:33:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/3065005810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 msgr2=0x55eca2635c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2635c60 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7faef40ca480 tx=0x7faef40cce00 comp rx=0 tx=0).stop 2023-07-21T19:33:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/3065005810 shutdown_connections 2023-07-21T19:33:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca26470b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca265cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/3065005810 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca2635c60 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/3065005810 >> 172.21.15.5:0/3065005810 conn(0x55eca2673120 msgr2=0x55eca26418a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.740+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/3065005810 shutdown_connections 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.741+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/3065005810 wait complete. 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.741+0000 7faf1eb8cb80 1 -- start start 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.741+0000 7faf1eb8cb80 1 Processor -- start 2023-07-21T19:33:05.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca265cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca263c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca263d4b0 con 0x55eca2630be0 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca263d620 con 0x55eca263b580 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eca263d7e0 con 0x55eca24976b0 2023-07-21T19:33:05.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf1eb8cb80 10 client.0 _ll_register_callbacks cb 0x7faef4005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:05.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf0979c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf08f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca265cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf0979c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52506/0 (socket says 172.21.15.5:52506) 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.742+0000 7faf0979c700 1 -- 172.21.15.5:0/2755307964 learned_addr learned my addr 172.21.15.5:0/2755307964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faefbfff700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca263c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 -- 172.21.15.5:0/2755307964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 msgr2=0x55eca265cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca265cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 -- 172.21.15.5:0/2755307964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 msgr2=0x55eca263c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca263c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 -- 172.21.15.5:0/2755307964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf041a9200 con 0x55eca263b580 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faefbfff700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca263c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faf0979c700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faefc017fb0 tx=0x7faefc004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1e112b3d7a1758c5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.743+0000 7faee3fff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faefc00e040 con 0x55eca263b580 2023-07-21T19:33:05.734 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eca26734f0 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(127..127 src has 1..127) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faefc01c040 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7faefc015e40 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faf1eb8cb80 10 client.29876 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55eca23e11f0 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.744+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faefc038080 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7faefc032030 con 0x55eca263b580 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 10 client.29876 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:05.735 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 10 client.29876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 20 client.29876 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26412d0 0x55eca26cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 4 client.29876 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.745+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7faf0000b840 con 0x55eca26412d0 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.746+0000 7faf08f9b700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26412d0 0x55eca26cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:05.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.746+0000 7faee3fff700 10 client.29876 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:05.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.746+0000 7faf08f9b700 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26412d0 0x55eca26cd9b0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7af6c3ff51a22cbb server_cookie=b8d68e08b8b67464 in_seq=0 out_seq=0 2023-07-21T19:33:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.791+0000 7faee3fff700 1 -- 172.21.15.5:0/2755307964 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7faf0000b840 con 0x55eca26412d0 2023-07-21T19:33:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.791+0000 7faee3fff700 10 client.29876 handle_command_reply: tid=0 2023-07-21T19:33:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 client.29876 shutdown 2023-07-21T19:33:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 msgr2=0x55eca263cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.779 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7faefc017fb0 tx=0x7faefc004ad0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 shutdown_connections 2023-07-21T19:33:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eca263b580 0x55eca263cf30 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eca24976b0 0x55eca265cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eca26412d0 0x55eca26cd9b0 crc :-1 s=READY pgs=486 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 --2- 172.21.15.5:0/2755307964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eca2630be0 0x55eca263c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 >> 172.21.15.5:0/2755307964 conn(0x55eca2673120 msgr2=0x55eca26401f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faee3fff700 0 client.29876 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 shutdown_connections 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.792+0000 7faf1eb8cb80 1 -- 172.21.15.5:0/2755307964 wait complete. 2023-07-21T19:33:05.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.793+0000 7faf1eb8cb80 20 client.29876 trim_cache size 0 max 16384 2023-07-21T19:33:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:33:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:33:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:33:05.782 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.794+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 msgr2=0x7faeec0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.794+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 0x7faeec0a37f0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7faefc011460 tx=0x7faefc00b040 comp rx=0 tx=0).stop 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.794+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 msgr2=0x7faf041a15f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.794+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 0x7faf041a15f0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7faf04068980 tx=0x7faef400eca0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 shutdown_connections 2023-07-21T19:33:05.783 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf04104e50 0x7faf0410b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faeec0a1340 0x7faeec0a37f0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf04067c10 0x7faf041a10b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 --2- 172.21.15.5:0/834697383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf040685b0 0x7faf041a15f0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 >> 172.21.15.5:0/834697383 conn(0x7faf04074760 msgr2=0x7faf040fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 shutdown_connections 2023-07-21T19:33:05.784 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:05.795+0000 7faf0b1ff700 1 -- 172.21.15.5:0/834697383 wait complete. 2023-07-21T19:33:05.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:05 smithi005 ceph-mon[126460]: pgmap v806: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 824 op/s 2023-07-21T19:33:05.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:05 smithi156 ceph-mon[128316]: pgmap v806: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 824 op/s 2023-07-21T19:33:06.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:05 smithi101 ceph-mon[128136]: pgmap v806: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 8.3 MiB/s wr, 824 op/s 2023-07-21T19:33:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 395.0 s, 198.2 tps, lat 461.804 ms stddev 130.067 2023-07-21T19:33:07.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:07 smithi005 ceph-mon[126460]: pgmap v807: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2023-07-21T19:33:07.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:07 smithi005 ceph-mon[126460]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T19:33:07.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:07 smithi156 ceph-mon[128316]: pgmap v807: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2023-07-21T19:33:07.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:07 smithi156 ceph-mon[128316]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T19:33:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:07 smithi101 ceph-mon[128136]: pgmap v807: 97 pgs: 97 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 7.6 MiB/s wr, 757 op/s 2023-07-21T19:33:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:07 smithi101 ceph-mon[128136]: osdmap e128: 12 total, 12 up, 12 in 2023-07-21T19:33:09.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:08 smithi005 ceph-mon[126460]: pgmap v809: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:09.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:08 smithi156 ceph-mon[128316]: pgmap v809: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:09.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:08 smithi101 ceph-mon[128136]: pgmap v809: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:11.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:11 smithi005 ceph-mon[126460]: pgmap v810: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:11.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:11 smithi156 ceph-mon[128316]: pgmap v810: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:11.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 400.0 s, 123.4 tps, lat 836.690 ms stddev 462.627 2023-07-21T19:33:12.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:11 smithi101 ceph-mon[128136]: pgmap v810: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 12 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.9 MiB/s wr, 770 op/s 2023-07-21T19:33:13.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:13 smithi005 ceph-mon[126460]: pgmap v811: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 639 op/s 2023-07-21T19:33:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:13 smithi156 ceph-mon[128316]: pgmap v811: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 639 op/s 2023-07-21T19:33:14.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:13 smithi101 ceph-mon[128136]: pgmap v811: 97 pgs: 6 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 81 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.1 MiB/s wr, 639 op/s 2023-07-21T19:33:15.899 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:15 smithi005 ceph-mon[126460]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:15.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:15 smithi156 ceph-mon[128316]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:16.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:15 smithi101 ceph-mon[128136]: pgmap v812: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:16.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 405.0 s, 163.0 tps, lat 589.983 ms stddev 497.613 2023-07-21T19:33:17.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:17 smithi005 ceph-mon[126460]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:17.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:17 smithi156 ceph-mon[128316]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:18.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:17 smithi101 ceph-mon[128136]: pgmap v813: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 597 op/s 2023-07-21T19:33:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.032+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1578210017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 msgr2=0x7ffb14106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.018 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.032+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1578210017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 0x7ffb14106930 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7ffb080099c0 tx=0x7ffb08009cd0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1578210017 shutdown_connections 2023-07-21T19:33:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1578210017 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14107010 0x7ffb1410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1578210017 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14101fd0 0x7ffb141023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1578210017 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 0x7ffb14106930 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1578210017 >> 172.21.15.5:0/1578210017 conn(0x7ffb140fd840 msgr2=0x7ffb140ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1578210017 shutdown_connections 2023-07-21T19:33:19.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.033+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1578210017 wait complete. 2023-07-21T19:33:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.035+0000 7ffb1b7ff700 1 Processor -- start 2023-07-21T19:33:19.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.035+0000 7ffb1b7ff700 1 -- start start 2023-07-21T19:33:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14101fd0 0x7ffb14197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 0x7ffb14198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419cfd0 con 0x7ffb14102970 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419d140 con 0x7ffb14101fd0 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.036+0000 7ffb1b7ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1419d300 con 0x7ffb14107010 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49784/0 (socket says 172.21.15.5:49784) 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 -- 172.21.15.5:0/1735680802 learned_addr learned my addr 172.21.15.5:0/1735680802 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:19.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb1959b700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14101fd0 0x7ffb14197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 -- 172.21.15.5:0/1735680802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14101fd0 msgr2=0x7ffb14197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14101fd0 0x7ffb14197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 -- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 msgr2=0x7ffb14198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 0x7ffb14198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.037+0000 7ffb19d9c700 1 -- 172.21.15.5:0/1735680802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb080095f0 con 0x7ffb14107010 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.038+0000 7ffb19d9c700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d980 tx=0x7ffb0c00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.038+0000 7ffb127fc700 1 -- 172.21.15.5:0/1735680802 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c004a30 con 0x7ffb14107010 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.038+0000 7ffb127fc700 1 -- 172.21.15.5:0/1735680802 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0c00b5c0 con 0x7ffb14107010 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.038+0000 7ffb127fc700 1 -- 172.21.15.5:0/1735680802 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0c009a40 con 0x7ffb14107010 2023-07-21T19:33:19.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.038+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1419d5e0 con 0x7ffb14107010 2023-07-21T19:33:19.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.039+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb141a4e20 con 0x7ffb14107010 2023-07-21T19:33:19.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.044+0000 7ffb127fc700 1 -- 172.21.15.5:0/1735680802 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ffb0c009ba0 con 0x7ffb14107010 2023-07-21T19:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.045+0000 7ffb2f18fb80 1 Processor -- start 2023-07-21T19:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.045+0000 7ffb2f18fb80 1 -- start start 2023-07-21T19:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5c8580 0x557c7c5c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c4cdc50 con 0x557c7c5c8580 2023-07-21T19:33:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5209e0 con 0x557c7c5ce2d0 2023-07-21T19:33:19.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c519e70 con 0x557c7c4246b0 2023-07-21T19:33:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb1959b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb1959b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47062/0 (socket says 172.21.15.5:47062) 2023-07-21T19:33:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb1959b700 1 -- 172.21.15.5:0/2834115464 learned_addr learned my addr 172.21.15.5:0/2834115464 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.046+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb127fc700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 0x7ffafc0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb1959b700 1 -- 172.21.15.5:0/2834115464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 msgr2=0x557c7c5cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb1959b700 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb1959b700 1 -- 172.21.15.5:0/2834115464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5c8580 msgr2=0x557c7c5c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb1959b700 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5c8580 0x557c7c5c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb1959b700 1 -- 172.21.15.5:0/2834115464 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c7c347d80 con 0x557c7c5ce2d0 2023-07-21T19:33:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.047+0000 7ffb127fc700 1 -- 172.21.15.5:0/1735680802 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb0c0cc950 con 0x7ffb14107010 2023-07-21T19:33:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.048+0000 7ffb1959b700 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011650 tx=0x7ffb04009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b01dec99b8bbb2b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.048+0000 7ffafb7fe700 1 -- 172.21.15.5:0/2834115464 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x557c7c5ce2d0 2023-07-21T19:33:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.048+0000 7ffafb7fe700 1 -- 172.21.15.5:0/2834115464 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0400f040 con 0x557c7c5ce2d0 2023-07-21T19:33:19.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.048+0000 7ffafb7fe700 1 -- 172.21.15.5:0/2834115464 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb04015670 con 0x557c7c5ce2d0 2023-07-21T19:33:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.048+0000 7ffb19d9c700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 0x7ffafc0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.049+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/2834115464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 msgr2=0x557c7c5c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.049+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011650 tx=0x7ffb04009920 comp rx=0 tx=0).stop 2023-07-21T19:33:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.049+0000 7ffb19d9c700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 0x7ffafc0a3830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0a18e0 tx=0x7ffb0c000bc0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/2834115464 shutdown_connections 2023-07-21T19:33:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5c8580 0x557c7c5c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/2834115464 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5ce2d0 0x557c7c5c8040 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/2834115464 >> 172.21.15.5:0/2834115464 conn(0x557c7c5bdbe0 msgr2=0x557c7c5c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/2834115464 shutdown_connections 2023-07-21T19:33:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.050+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/2834115464 wait complete. 2023-07-21T19:33:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.051+0000 7ffb2f18fb80 1 -- start start 2023-07-21T19:33:19.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.051+0000 7ffb2f18fb80 1 Processor -- start 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.052+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.052+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 0x557c7c5d1820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5ce2d0 0x557c7c5d1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c4b9a60 con 0x557c7c5ce2d0 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c4b9760 con 0x557c7c5c8580 2023-07-21T19:33:19.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c39d8c0 con 0x557c7c4246b0 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb2f18fb80 10 client.0 _ll_register_callbacks cb 0x557c7c4b7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49806/0 (socket says 172.21.15.5:49806) 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb18d9a700 1 -- 172.21.15.5:0/3010661532 learned_addr learned my addr 172.21.15.5:0/3010661532 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5ce2d0 0x557c7c5d1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.053+0000 7ffb1959b700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 0x557c7c5d1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 -- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 msgr2=0x557c7c5d1820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 0x557c7c5d1820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 -- 172.21.15.5:0/3010661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5ce2d0 msgr2=0x557c7c5d1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5ce2d0 0x557c7c5d1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 -- 172.21.15.5:0/3010661532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb04011330 con 0x557c7c4246b0 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb1959b700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 0x557c7c5d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:19.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006010 tx=0x7ffb08009510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51a84d97ba5c904c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.054+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08010750 con 0x557c7c4246b0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.055+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb080108b0 con 0x557c7c4246b0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.055+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557c7c473490 con 0x557c7c4246b0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.055+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb08010a70 con 0x557c7c4246b0 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.055+0000 7ffb2f18fb80 10 client.29800 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:19.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.055+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x557c7c330c00 con 0x557c7c4246b0 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb08035020 con 0x557c7c4246b0 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb08017970 con 0x557c7c4246b0 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 10 client.29800 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 10 client.29800 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 20 client.29800 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ea330 0x557c7c5ec740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 4 client.29800 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:19.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557c7c512e50 con 0x557c7c5ea330 2023-07-21T19:33:19.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.056+0000 7ffb1959b700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ea330 0x557c7c5ec740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.057+0000 7ffb1959b700 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ea330 0x557c7c5ec740 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e010d90b64085f0 server_cookie=c444be7fe95f74c in_seq=0 out_seq=0 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.057+0000 7ffaf8ff9700 10 client.29800 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.058+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/3010661532 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557c7c512e50 con 0x557c7c5ea330 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.058+0000 7ffaf8ff9700 10 client.29800 handle_command_reply: tid=0 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.058+0000 7ffb2f18fb80 1 client.29800 shutdown 2023-07-21T19:33:19.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 msgr2=0x557c7c5d4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7ffb08006010 tx=0x7ffb08009510 comp rx=0 tx=0).stop 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 shutdown_connections 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ea330 0x557c7c5ec740 crc :-1 s=READY pgs=487 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c5ce2d0 0x557c7c5d1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c4246b0 0x557c7c5d4530 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/3010661532 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5c8580 0x557c7c5d1820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 >> 172.21.15.5:0/3010661532 conn(0x557c7c5bdbe0 msgr2=0x557c7c5c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffaf8ff9700 0 client.29800 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 shutdown_connections 2023-07-21T19:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/3010661532 wait complete. 2023-07-21T19:33:19.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.059+0000 7ffb2f18fb80 20 client.29800 trim_cache size 0 max 16384 2023-07-21T19:33:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.081+0000 7ffb2f18fb80 1 Processor -- start 2023-07-21T19:33:19.067 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.081+0000 7ffb2f18fb80 1 -- start start 2023-07-21T19:33:19.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:18 smithi101 ceph-mon[128136]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T19:33:19.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:18 smithi101 ceph-mon[128136]: pgmap v815: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.081+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5e1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.081+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5c3ff0 con 0x557c7c4246b0 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5bfd00 con 0x557c7c5bdbe0 2023-07-21T19:33:19.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5c4990 con 0x557c7c5c8580 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5e1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55802/0 (socket says 172.21.15.5:55802) 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5e1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47068/0 (socket says 172.21.15.5:47068) 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 -- 172.21.15.5:0/1329265749 learned_addr learned my addr 172.21.15.5:0/1329265749 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 -- 172.21.15.5:0/1329265749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 msgr2=0x557c7c5c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 -- 172.21.15.5:0/1329265749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 msgr2=0x557c7c5e1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5e1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.082+0000 7ffb1959b700 1 -- 172.21.15.5:0/1329265749 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557c7c5f1390 con 0x557c7c4246b0 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.083+0000 7ffb1959b700 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011c40 tx=0x7ffb0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50690c86ffc80d1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.083+0000 7ffafbfff700 1 -- 172.21.15.5:0/1329265749 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0402b070 con 0x557c7c4246b0 2023-07-21T19:33:19.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.083+0000 7ffafbfff700 1 -- 172.21.15.5:0/1329265749 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb04026cf0 con 0x557c7c4246b0 2023-07-21T19:33:19.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.083+0000 7ffafbfff700 1 -- 172.21.15.5:0/1329265749 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb0400e040 con 0x557c7c4246b0 2023-07-21T19:33:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/1329265749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 msgr2=0x557c7c5cd100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7ffb04011c40 tx=0x7ffb0400b040 comp rx=0 tx=0).stop 2023-07-21T19:33:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/1329265749 shutdown_connections 2023-07-21T19:33:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5e1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/1329265749 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cd100 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/1329265749 >> 172.21.15.5:0/1329265749 conn(0x557c7c5e6040 msgr2=0x557c7c5cc9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/1329265749 shutdown_connections 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.084+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/1329265749 wait complete. 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.085+0000 7ffb2f18fb80 1 -- start start 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.085+0000 7ffb2f18fb80 1 Processor -- start 2023-07-21T19:33:19.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5cb150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5c9cd0 con 0x557c7c4246b0 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5c9e40 con 0x557c7c5bdbe0 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x557c7c5ca000 con 0x557c7c5c8580 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb2f18fb80 10 client.0 _ll_register_callbacks cb 0x7ffb0800bb20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb18d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55812/0 (socket says 172.21.15.5:55812) 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb18d9a700 1 -- 172.21.15.5:0/47964554 learned_addr learned my addr 172.21.15.5:0/47964554 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.086+0000 7ffb1959b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 -- 172.21.15.5:0/47964554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 msgr2=0x557c7c5c9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c9790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 -- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 msgr2=0x557c7c5cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:19.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.087+0000 7ffb18d9a700 1 -- 172.21.15.5:0/47964554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557c7c347d80 con 0x557c7c4246b0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffb18d9a700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800bd70 tx=0x7ffb08009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac76ae7a959aa20 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffaf8ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080175f0 con 0x557c7c4246b0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ffb0400f040 con 0x557c7c4246b0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ffb08000e40 con 0x557c7c4246b0 2023-07-21T19:33:19.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffb08017750 con 0x557c7c4246b0 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffb2f18fb80 10 client.20097 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.088+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x557c7c36e1f0 con 0x557c7c4246b0 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ffb0400f040 con 0x557c7c4246b0 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ffb08005040 con 0x557c7c4246b0 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 10 client.20097 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 10 client.20097 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:19.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 20 client.20097 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ce2d0 0x557c7c640d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 4 client.20097 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557c7c4d4ff0 con 0x557c7c5ce2d0 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.089+0000 7ffb1959b700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ce2d0 0x557c7c640d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.090+0000 7ffb1959b700 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ce2d0 0x557c7c640d70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f8fbe1e2dd8ff88 server_cookie=55ce1379418c7b2 in_seq=0 out_seq=0 2023-07-21T19:33:19.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.090+0000 7ffaf8ff9700 10 client.20097 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffaf8ff9700 1 -- 172.21.15.5:0/47964554 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557c7c4d4ff0 con 0x557c7c5ce2d0 2023-07-21T19:33:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffaf8ff9700 10 client.20097 handle_command_reply: tid=0 2023-07-21T19:33:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 client.20097 shutdown 2023-07-21T19:33:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 msgr2=0x557c7c5cac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7ffb0800bd70 tx=0x7ffb08009330 comp rx=0 tx=0).stop 2023-07-21T19:33:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 shutdown_connections 2023-07-21T19:33:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x557c7c5c8580 0x557c7c5c9790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x557c7c5ce2d0 0x557c7c640d70 crc :-1 s=READY pgs=488 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x557c7c4246b0 0x557c7c5cac10 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 --2- 172.21.15.5:0/47964554 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x557c7c5bdbe0 0x557c7c5cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 >> 172.21.15.5:0/47964554 conn(0x557c7c5e6040 msgr2=0x557c7c5cd280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.128+0000 7ffaf8ff9700 0 client.20097 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.129+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 shutdown_connections 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.129+0000 7ffb2f18fb80 1 -- 172.21.15.5:0/47964554 wait complete. 2023-07-21T19:33:19.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.129+0000 7ffb2f18fb80 20 client.20097 trim_cache size 0 max 16384 2023-07-21T19:33:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:33:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:33:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:33:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:33:19.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 msgr2=0x7ffafc0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 0x7ffafc0a3830 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c0a18e0 tx=0x7ffb0c000bc0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 msgr2=0x7ffb1419c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7ffb0c00d980 tx=0x7ffb0c00dc90 comp rx=0 tx=0).stop 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 shutdown_connections 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ffb14107010 0x7ffb1419c990 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ffb14101fd0 0x7ffb14197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ffafc0a1380 0x7ffafc0a3830 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 --2- 172.21.15.5:0/1735680802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ffb14102970 0x7ffb14198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 >> 172.21.15.5:0/1735680802 conn(0x7ffb140fd840 msgr2=0x7ffb140fe300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 shutdown_connections 2023-07-21T19:33:19.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:19.130+0000 7ffb1b7ff700 1 -- 172.21.15.5:0/1735680802 wait complete. 2023-07-21T19:33:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:18 smithi005 ceph-mon[126460]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T19:33:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:18 smithi005 ceph-mon[126460]: pgmap v815: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:18 smithi156 ceph-mon[128316]: osdmap e129: 12 total, 12 up, 12 in 2023-07-21T19:33:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:18 smithi156 ceph-mon[128316]: pgmap v815: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:21.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:21 smithi101 ceph-mon[128136]: pgmap v816: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:21.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:21 smithi005 ceph-mon[126460]: pgmap v816: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:21.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:21 smithi156 ceph-mon[128316]: pgmap v816: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.3 MiB/s wr, 532 op/s 2023-07-21T19:33:21.936 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 410.0 s, 48.0 tps, lat 1133.093 ms stddev 1070.054 2023-07-21T19:33:23.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:23 smithi005 ceph-mon[126460]: pgmap v817: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2023-07-21T19:33:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:23 smithi156 ceph-mon[128316]: pgmap v817: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2023-07-21T19:33:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:23 smithi101 ceph-mon[128136]: pgmap v817: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.7 MiB/s wr, 402 op/s 2023-07-21T19:33:25.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:25 smithi005 ceph-mon[126460]: pgmap v818: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:25.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:25 smithi156 ceph-mon[128316]: pgmap v818: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:26.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:25 smithi101 ceph-mon[128136]: pgmap v818: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:26.294 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:33:26.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 -- 172.21.15.5:0/2340190972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 msgr2=0x7fc958072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 --2- 172.21.15.5:0/2340190972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958072c80 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fc94c009990 tx=0x7fc94c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 -- 172.21.15.5:0/2340190972 shutdown_connections 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 --2- 172.21.15.5:0/2340190972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc95810c320 0x7fc95810e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 --2- 172.21.15.5:0/2340190972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958072c80 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 --2- 172.21.15.5:0/2340190972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc9580731c0 0x7fc95810bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 -- 172.21.15.5:0/2340190972 >> 172.21.15.5:0/2340190972 conn(0x7fc9580fb5e0 msgr2=0x7fc9580fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 -- 172.21.15.5:0/2340190972 shutdown_connections 2023-07-21T19:33:26.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.521+0000 7fc95ebff700 1 -- 172.21.15.5:0/2340190972 wait complete. 2023-07-21T19:33:26.510 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.523+0000 7fc95ebff700 1 Processor -- start 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.523+0000 7fc95ebff700 1 -- start start 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.523+0000 7fc95ebff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95ebff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9580731c0 0x7fc958198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95ebff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc95810c320 0x7fc95819c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95ebff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95819cd20 con 0x7fc9580728b0 2023-07-21T19:33:26.511 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95ebff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95819ce90 con 0x7fc95810c320 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95ebff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95819d050 con 0x7fc9580731c0 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44590/0 (socket says 172.21.15.5:44590) 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 -- 172.21.15.5:0/1803703222 learned_addr learned my addr 172.21.15.5:0/1803703222 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc957fff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9580731c0 0x7fc958198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.512 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95d19c700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc95810c320 0x7fc95819c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 -- 172.21.15.5:0/1803703222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9580731c0 msgr2=0x7fc958198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9580731c0 0x7fc958198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.513 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 -- 172.21.15.5:0/1803703222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc95810c320 msgr2=0x7fc95819c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc95810c320 0x7fc95819c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.524+0000 7fc95c99b700 1 -- 172.21.15.5:0/1803703222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc94c0095f0 con 0x7fc9580728b0 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc94c005740 tx=0x7fc94c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc955ffb700 1 -- 172.21.15.5:0/1803703222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c01d070 con 0x7fc9580728b0 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc955ffb700 1 -- 172.21.15.5:0/1803703222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94c022470 con 0x7fc9580728b0 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc955ffb700 1 -- 172.21.15.5:0/1803703222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c017650 con 0x7fc9580728b0 2023-07-21T19:33:26.514 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95819d2d0 con 0x7fc9580728b0 2023-07-21T19:33:26.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.525+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95819d550 con 0x7fc9580728b0 2023-07-21T19:33:26.515 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.528+0000 7fc955ffb700 1 -- 172.21.15.5:0/1803703222 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc94c00fb00 con 0x7fc9580728b0 2023-07-21T19:33:26.516 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc972603b80 1 Processor -- start 2023-07-21T19:33:26.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc955ffb700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 0x7fc9480a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc972603b80 1 -- start start 2023-07-21T19:33:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc955ffb700 1 -- 172.21.15.5:0/1803703222 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc94c0ce0c0 con 0x7fc9580728b0 2023-07-21T19:33:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc95d19c700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 0x7fc9480a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc972603b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.530+0000 7fc972603b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549da590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc972603b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc972603b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e548d5cb0 con 0x564e549daad0 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc972603b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e54928a40 con 0x564e549cf2d0 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc972603b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e54921ed0 con 0x564e549cfbe0 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36682/0 (socket says 172.21.15.5:36682) 2023-07-21T19:33:26.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 -- 172.21.15.5:0/3773915773 learned_addr learned my addr 172.21.15.5:0/3773915773 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc957fff700 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549da590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95d19c700 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 -- 172.21.15.5:0/3773915773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 msgr2=0x564e549da590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549da590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 -- 172.21.15.5:0/3773915773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 msgr2=0x564e549dceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.520 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95d19c700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 0x7fc9480a37f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc950003ee0 tx=0x7fc950009ba0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.521 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.531+0000 7fc95c99b700 1 -- 172.21.15.5:0/3773915773 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e5474fde0 con 0x564e549cf2d0 2023-07-21T19:33:26.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc95c99b700 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0a3050 tx=0x7fc94c09add0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=27d350752541ccf8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc942ffd700 1 -- 172.21.15.5:0/3773915773 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c09f1f0 con 0x564e549cf2d0 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc942ffd700 1 -- 172.21.15.5:0/3773915773 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94c0a3580 con 0x564e549cf2d0 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc942ffd700 1 -- 172.21.15.5:0/3773915773 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c09b420 con 0x564e549cf2d0 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc972603b80 1 -- 172.21.15.5:0/3773915773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 msgr2=0x564e549cf6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.532+0000 7fc972603b80 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fc94c0a3050 tx=0x7fc94c09add0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 -- 172.21.15.5:0/3773915773 shutdown_connections 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549cf6a0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 --2- 172.21.15.5:0/3773915773 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549da590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 -- 172.21.15.5:0/3773915773 >> 172.21.15.5:0/3773915773 conn(0x564e549c5da0 msgr2=0x564e549cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 -- 172.21.15.5:0/3773915773 shutdown_connections 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 -- 172.21.15.5:0/3773915773 wait complete. 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 -- start start 2023-07-21T19:33:26.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.533+0000 7fc972603b80 1 Processor -- start 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e549d99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e548c1ac0 con 0x564e549cf2d0 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e548c17c0 con 0x564e549daad0 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e547a5920 con 0x564e549cfbe0 2023-07-21T19:33:26.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc972603b80 10 client.0 _ll_register_callbacks cb 0x564e548bf760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc95d19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e549d99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44602/0 (socket says 172.21.15.5:44602) 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc95c99b700 1 -- 172.21.15.5:0/1228707663 learned_addr learned my addr 172.21.15.5:0/1228707663 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.534+0000 7fc957fff700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 -- 172.21.15.5:0/1228707663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 msgr2=0x564e549e0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549e0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 -- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 msgr2=0x564e549d99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e549d99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 -- 172.21.15.5:0/1228707663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95819d2d0 con 0x564e549cf2d0 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95d19c700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e549d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc94c097c30 tx=0x7fc94c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a317b3bec5d7250c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc937fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c09f1f0 con 0x564e549cf2d0 2023-07-21T19:33:26.528 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc94c099560 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564e5487b4f0 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc94c022850 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.535+0000 7fc972603b80 10 client.20109 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x564e54738c60 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94c0cb020 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc94c096fb0 con 0x564e549cf2d0 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 10 client.20109 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:26.529 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 10 client.20109 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 20 client.20109 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e549e9ae0 0x564e549ebef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 4 client.20109 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564e5491aeb0 con 0x564e549e9ae0 2023-07-21T19:33:26.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.536+0000 7fc957fff700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e549e9ae0 0x564e549ebef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.537+0000 7fc957fff700 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e549e9ae0 0x564e549ebef0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a126b387633516a server_cookie=b621e3647d2af090 in_seq=0 out_seq=0 2023-07-21T19:33:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.537+0000 7fc937fff700 10 client.20109 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:26.531 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc937fff700 1 -- 172.21.15.5:0/1228707663 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564e5491aeb0 con 0x564e549e9ae0 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc937fff700 10 client.20109 handle_command_reply: tid=0 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 client.20109 shutdown 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 msgr2=0x564e549dfb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fc94c097c30 tx=0x7fc94c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 shutdown_connections 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e549d99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e549e9ae0 0x564e549ebef0 crc :-1 s=READY pgs=489 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cf2d0 0x564e549dfb20 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 --2- 172.21.15.5:0/1228707663 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549e0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc937fff700 0 client.20109 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 >> 172.21.15.5:0/1228707663 conn(0x564e549c5da0 msgr2=0x564e549cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 shutdown_connections 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 1 -- 172.21.15.5:0/1228707663 wait complete. 2023-07-21T19:33:26.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.538+0000 7fc972603b80 20 client.20109 trim_cache size 0 max 16384 2023-07-21T19:33:26.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 Processor -- start 2023-07-21T19:33:26.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 -- start start 2023-07-21T19:33:26.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549d9b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e549d7f40 con 0x564e549daad0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e549d55f0 con 0x564e549cf2d0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc972603b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e549e0170 con 0x564e549cfbe0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.556+0000 7fc95d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549d9b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95d19c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44616/0 (socket says 172.21.15.5:44616) 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95d19c700 1 -- 172.21.15.5:0/109498848 learned_addr learned my addr 172.21.15.5:0/109498848 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc957fff700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 -- 172.21.15.5:0/109498848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 msgr2=0x564e549df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549df6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 -- 172.21.15.5:0/109498848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 msgr2=0x564e549dfc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 -- 172.21.15.5:0/109498848 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564e54776250 con 0x564e549cf2d0 2023-07-21T19:33:26.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc957fff700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:26.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95d19c700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:26.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc95c99b700 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549d9b10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc94c09b150 tx=0x7fc94c005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9dcfc1ecee89065 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc9437fe700 1 -- 172.21.15.5:0/109498848 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c097500 con 0x564e549cf2d0 2023-07-21T19:33:26.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc9437fe700 1 -- 172.21.15.5:0/109498848 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94c0cb040 con 0x564e549cf2d0 2023-07-21T19:33:26.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.557+0000 7fc9437fe700 1 -- 172.21.15.5:0/109498848 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc94c0a30f0 con 0x564e549cf2d0 2023-07-21T19:33:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 -- 172.21.15.5:0/109498848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 msgr2=0x564e549d9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549d9b10 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fc94c09b150 tx=0x7fc94c005fd0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 -- 172.21.15.5:0/109498848 shutdown_connections 2023-07-21T19:33:26.550 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549daad0 0x564e549dfc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549cf2d0 0x564e549d9b10 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 --2- 172.21.15.5:0/109498848 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cfbe0 0x564e549df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 -- 172.21.15.5:0/109498848 >> 172.21.15.5:0/109498848 conn(0x564e549c5da0 msgr2=0x564e549c9e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 -- 172.21.15.5:0/109498848 shutdown_connections 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.558+0000 7fc972603b80 1 -- 172.21.15.5:0/109498848 wait complete. 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 -- start start 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 Processor -- start 2023-07-21T19:33:26.551 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cf2d0 0x564e54a370e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e54a32030 con 0x564e549cfbe0 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e54a321a0 con 0x564e549daad0 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564e54a32360 con 0x564e549cf2d0 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.559+0000 7fc972603b80 10 client.0 _ll_register_callbacks cb 0x7fc950009620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:26.552 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc95d19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44618/0 (socket says 172.21.15.5:44618) 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 -- 172.21.15.5:0/1743357990 learned_addr learned my addr 172.21.15.5:0/1743357990 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc95d19c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36716/0 (socket says 172.21.15.5:36716) 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cf2d0 0x564e54a370e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 -- 172.21.15.5:0/1743357990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cf2d0 msgr2=0x564e54a370e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.553 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cf2d0 0x564e54a370e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 -- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 msgr2=0x564e54a31930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 -- 172.21.15.5:0/1743357990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9500047a0 con 0x564e549cfbe0 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc95d19c700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc957fff700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x564e549db410 tx=0x7fc944010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2d92684717e0682 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc937fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.560+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc944017040 con 0x564e549cfbe0 2023-07-21T19:33:26.554 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc94c0a3580 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc944010c00 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc944010dc0 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 10 client.20115 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x564e54738c60 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc94c0a3580 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc944024030 con 0x564e549cfbe0 2023-07-21T19:33:26.555 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 10 client.20115 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 10 client.20115 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 20 client.20115 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e54a3ef30 0x564e54a41340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 4 client.20115 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.561+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564e548dd050 con 0x564e54a3ef30 2023-07-21T19:33:26.556 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e54a3ef30 0x564e54a41340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc95c99b700 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e54a3ef30 0x564e54a41340 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=549114caaf954106 server_cookie=b0d63e83bfee60bd in_seq=0 out_seq=0 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc937fff700 10 client.20115 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc937fff700 1 -- 172.21.15.5:0/1743357990 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564e548dd050 con 0x564e54a3ef30 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc937fff700 10 client.20115 handle_command_reply: tid=0 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.562+0000 7fc972603b80 1 client.20115 shutdown 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 msgr2=0x564e54a37640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.557 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x564e549db410 tx=0x7fc944010520 comp rx=0 tx=0).stop 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 shutdown_connections 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564e549daad0 0x564e54a31930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564e549cf2d0 0x564e54a370e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564e54a3ef30 0x564e54a41340 crc :-1 s=READY pgs=490 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 --2- 172.21.15.5:0/1743357990 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564e549cfbe0 0x564e54a37640 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc937fff700 0 client.20115 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:26.558 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 >> 172.21.15.5:0/1743357990 conn(0x564e549c5da0 msgr2=0x564e549d8190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 shutdown_connections 2023-07-21T19:33:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 1 -- 172.21.15.5:0/1743357990 wait complete. 2023-07-21T19:33:26.559 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.563+0000 7fc972603b80 20 client.20115 trim_cache size 0 max 16384 2023-07-21T19:33:26.559 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:33:26.559 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:33:26.560 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:33:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 msgr2=0x7fc9480a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 0x7fc9480a37f0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc950003ee0 tx=0x7fc950009ba0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.561 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 msgr2=0x7fc958197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fc94c005740 tx=0x7fc94c005770 comp rx=0 tx=0).stop 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 shutdown_connections 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc95810c320 0x7fc95819c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc9480a1340 0x7fc9480a37f0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc9580728b0 0x7fc958197d40 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 --2- 172.21.15.5:0/1803703222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc9580731c0 0x7fc958198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 >> 172.21.15.5:0/1803703222 conn(0x7fc9580fb5e0 msgr2=0x7fc95810e4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:26.562 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 shutdown_connections 2023-07-21T19:33:26.563 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:26.564+0000 7fc95ebff700 1 -- 172.21.15.5:0/1803703222 wait complete. 2023-07-21T19:33:26.568 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:33:26.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 415.0 s, 21.6 tps, lat 6173.902 ms stddev 1279.351 2023-07-21T19:33:27.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:27 smithi005 ceph-mon[126460]: pgmap v819: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:27.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:27 smithi156 ceph-mon[128316]: pgmap v819: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:28.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:27 smithi101 ceph-mon[128136]: pgmap v819: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 3.5 MiB/s wr, 358 op/s 2023-07-21T19:33:29.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:28 smithi101 ceph-mon[128136]: pgmap v820: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-07-21T19:33:29.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:28 smithi005 ceph-mon[126460]: pgmap v820: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-07-21T19:33:29.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:28 smithi156 ceph-mon[128316]: pgmap v820: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 432 KiB/s rd, 1.5 MiB/s wr, 180 op/s 2023-07-21T19:33:31.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:31 smithi005 ceph-mon[126460]: pgmap v821: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-07-21T19:33:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:31 smithi156 ceph-mon[128316]: pgmap v821: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-07-21T19:33:31.934 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 420.0 s, 189.6 tps, lat 537.338 ms stddev 740.968 2023-07-21T19:33:32.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:31 smithi101 ceph-mon[128136]: pgmap v821: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 387 KiB/s rd, 1.4 MiB/s wr, 161 op/s 2023-07-21T19:33:32.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 -- 172.21.15.5:0/113758337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 msgr2=0x7f43ac105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 --2- 172.21.15.5:0/113758337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac105f30 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f439c009990 tx=0x7f439c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 -- 172.21.15.5:0/113758337 shutdown_connections 2023-07-21T19:33:32.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 --2- 172.21.15.5:0/113758337 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 --2- 172.21.15.5:0/113758337 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac105f30 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 --2- 172.21.15.5:0/113758337 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 0x7f43ac0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 -- 172.21.15.5:0/113758337 >> 172.21.15.5:0/113758337 conn(0x7f43ac0fb5f0 msgr2=0x7f43ac0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 -- 172.21.15.5:0/113758337 shutdown_connections 2023-07-21T19:33:32.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.288+0000 7f43b1dff700 1 -- 172.21.15.5:0/113758337 wait complete. 2023-07-21T19:33:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.289+0000 7f43b1dff700 1 Processor -- start 2023-07-21T19:33:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.289+0000 7f43b1dff700 1 -- start start 2023-07-21T19:33:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 0x7f43ac0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac1a3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac1a41b0 con 0x7f43ac105b60 2023-07-21T19:33:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac1a42f0 con 0x7f43ac1004e0 2023-07-21T19:33:32.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43b1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac1a4460 con 0x7f43ac0ffb30 2023-07-21T19:33:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac1a3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36734/0 (socket says 172.21.15.5:36734) 2023-07-21T19:33:32.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43aaffd700 1 -- 172.21.15.5:0/2942150372 learned_addr learned my addr 172.21.15.5:0/2942150372 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 0x7f43ac0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43abfff700 1 -- 172.21.15.5:0/2942150372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 msgr2=0x7f43ac0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.290+0000 7f43abfff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 0x7f43ac0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43abfff700 1 -- 172.21.15.5:0/2942150372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 msgr2=0x7f43ac072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43abfff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43abfff700 1 -- 172.21.15.5:0/2942150372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f439c0095f0 con 0x7f43ac105b60 2023-07-21T19:33:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43aaffd700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:32.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43abfff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac1a3c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f43a400efc0 tx=0x7f43a400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43a8ff9700 1 -- 172.21.15.5:0/2942150372 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4018070 con 0x7f43ac105b60 2023-07-21T19:33:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43a8ff9700 1 -- 172.21.15.5:0/2942150372 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43a400f040 con 0x7f43ac105b60 2023-07-21T19:33:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac1a46e0 con 0x7f43ac105b60 2023-07-21T19:33:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43a8ff9700 1 -- 172.21.15.5:0/2942150372 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400cba0 con 0x7f43ac105b60 2023-07-21T19:33:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.291+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac1a4bf0 con 0x7f43ac105b60 2023-07-21T19:33:32.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.294+0000 7f43a8ff9700 1 -- 172.21.15.5:0/2942150372 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f43a4003a00 con 0x7f43ac105b60 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 Processor -- start 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 -- start start 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43a8ff9700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 0x7f43940a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a87040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a879d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996898cc50 con 0x559968a8d2d0 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599689df9e0 con 0x5599688e36b0 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5599689d8e70 con 0x559968a87580 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43a8ff9700 1 -- 172.21.15.5:0/2942150372 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f43a40cc820 con 0x7f43ac105b60 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43ab7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43ab7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36746/0 (socket says 172.21.15.5:36746) 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 -- 172.21.15.5:0/3497476821 learned_addr learned my addr 172.21.15.5:0/3497476821 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:32.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.296+0000 7f43aaffd700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 0x7f43940a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43abfff700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a879d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43aaffd700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a87040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 -- 172.21.15.5:0/3497476821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 msgr2=0x559968a879d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a879d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 -- 172.21.15.5:0/3497476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 msgr2=0x559968a87040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a87040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 -- 172.21.15.5:0/3497476821 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559968806d80 con 0x5599688e36b0 2023-07-21T19:33:32.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43aaffd700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a87040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:32.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f439c015000 tx=0x7f439c005f00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c9de0c1cfa6ff8e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.297+0000 7f43aaffd700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 0x7f43940a3770 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559968a83b40 tx=0x7f43a00095c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f4391ffb700 1 -- 172.21.15.5:0/3497476821 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c01d070 con 0x5599688e36b0 2023-07-21T19:33:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f4391ffb700 1 -- 172.21.15.5:0/3497476821 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f439c01e040 con 0x5599688e36b0 2023-07-21T19:33:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f4391ffb700 1 -- 172.21.15.5:0/3497476821 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f439c004760 con 0x5599688e36b0 2023-07-21T19:33:32.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 -- 172.21.15.5:0/3497476821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 msgr2=0x559968a8cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f439c015000 tx=0x7f439c005f00 comp rx=0 tx=0).stop 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 -- 172.21.15.5:0/3497476821 shutdown_connections 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a879d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a8cd90 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/3497476821 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a87040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 -- 172.21.15.5:0/3497476821 >> 172.21.15.5:0/3497476821 conn(0x559968a7cbe0 msgr2=0x559968a81f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.298+0000 7f43c56e7b80 1 -- 172.21.15.5:0/3497476821 shutdown_connections 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.299+0000 7f43c56e7b80 1 -- 172.21.15.5:0/3497476821 wait complete. 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.299+0000 7f43c56e7b80 1 -- start start 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.299+0000 7f43c56e7b80 1 Processor -- start 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a93520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a90730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968978a60 con 0x559968a8d2d0 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968978760 con 0x5599688e36b0 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55996885c8c0 con 0x559968a87580 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43c56e7b80 10 client.0 _ll_register_callbacks cb 0x559968976700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43aaffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a90730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43ab7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a93520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44658/0 (socket says 172.21.15.5:44658) 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 -- 172.21.15.5:0/4154413606 learned_addr learned my addr 172.21.15.5:0/4154413606 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 -- 172.21.15.5:0/4154413606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 msgr2=0x559968a90730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a90730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 -- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 msgr2=0x559968a93520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a93520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.300+0000 7f43abfff700 1 -- 172.21.15.5:0/4154413606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac10d220 con 0x559968a8d2d0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f43abfff700 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f43a4095e20 tx=0x7f43a40992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ec3510ff6f2587c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f438affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4099d80 con 0x559968a8d2d0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f43a409d950 con 0x559968a8d2d0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559968932490 con 0x559968a8d2d0 2023-07-21T19:33:32.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a40a1ce0 con 0x559968a8d2d0 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f43c56e7b80 10 client.20124 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x5599687efc00 con 0x559968a8d2d0 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.301+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43a405e030 con 0x559968a8d2d0 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43a40a18e0 con 0x559968a8d2d0 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 10 client.20124 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 10 client.20124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:32.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 20 client.20124 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968aa0890 0x559968aa2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 4 client.20124 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5599689d1e50 con 0x559968aa0890 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968aa0890 0x559968aa2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968aa0890 0x559968aa2ca0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72956e8196bff3b6 server_cookie=9074d14388dc67bb in_seq=0 out_seq=0 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.302+0000 7f438affd700 10 client.20124 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:32.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.303+0000 7f438affd700 1 -- 172.21.15.5:0/4154413606 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5599689d1e50 con 0x559968aa0890 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.303+0000 7f438affd700 10 client.20124 handle_command_reply: tid=0 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 client.20124 shutdown 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 msgr2=0x559968a90cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f43a4095e20 tx=0x7f43a40992f0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 shutdown_connections 2023-07-21T19:33:32.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968aa0890 0x559968aa2ca0 crc :-1 s=READY pgs=491 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a90cb0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a93520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/4154413606 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a90730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 >> 172.21.15.5:0/4154413606 conn(0x559968a7cbe0 msgr2=0x559968a80c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f438affd700 0 client.20124 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 shutdown_connections 2023-07-21T19:33:32.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 1 -- 172.21.15.5:0/4154413606 wait complete. 2023-07-21T19:33:32.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.304+0000 7f43c56e7b80 20 client.20124 trim_cache size 0 max 16384 2023-07-21T19:33:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.320+0000 7f43c56e7b80 1 Processor -- start 2023-07-21T19:33:32.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.320+0000 7f43c56e7b80 1 -- start start 2023-07-21T19:33:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a87040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a8d2d0 0x559968a97c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968a4d060 con 0x559968a87580 2023-07-21T19:33:32.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968a8ca10 con 0x5599688e36b0 2023-07-21T19:33:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.321+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968a7ed00 con 0x559968a8d2d0 2023-07-21T19:33:32.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44666/0 (socket says 172.21.15.5:44666) 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 -- 172.21.15.5:0/581191835 learned_addr learned my addr 172.21.15.5:0/581191835 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a87040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43abfff700 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a8d2d0 0x559968a97c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 -- 172.21.15.5:0/581191835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a8d2d0 msgr2=0x559968a97c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a8d2d0 0x559968a97c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 -- 172.21.15.5:0/581191835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 msgr2=0x559968a87040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a87040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.322+0000 7f43aaffd700 1 -- 172.21.15.5:0/581191835 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55996882d1f0 con 0x559968a87580 2023-07-21T19:33:32.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.323+0000 7f43aaffd700 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f43ac074210 tx=0x7f43a001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f126f7fa262c65c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.324+0000 7f43927fc700 1 -- 172.21.15.5:0/581191835 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a0023040 con 0x559968a87580 2023-07-21T19:33:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.324+0000 7f43927fc700 1 -- 172.21.15.5:0/581191835 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f43a0017030 con 0x559968a87580 2023-07-21T19:33:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.324+0000 7f43927fc700 1 -- 172.21.15.5:0/581191835 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000e7d0 con 0x559968a87580 2023-07-21T19:33:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 -- 172.21.15.5:0/581191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 msgr2=0x559968a8c360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f43ac074210 tx=0x7f43a001a000 comp rx=0 tx=0).stop 2023-07-21T19:33:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 -- 172.21.15.5:0/581191835 shutdown_connections 2023-07-21T19:33:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a8d2d0 0x559968a97c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a87040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/581191835 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a87580 0x559968a8c360 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.325+0000 7f43c56e7b80 1 -- 172.21.15.5:0/581191835 >> 172.21.15.5:0/581191835 conn(0x559968a7cbe0 msgr2=0x559968a8f770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.326+0000 7f43c56e7b80 1 -- 172.21.15.5:0/581191835 shutdown_connections 2023-07-21T19:33:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.326+0000 7f43c56e7b80 1 -- 172.21.15.5:0/581191835 wait complete. 2023-07-21T19:33:32.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.326+0000 7f43c56e7b80 1 -- start start 2023-07-21T19:33:32.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.327+0000 7f43c56e7b80 1 Processor -- start 2023-07-21T19:33:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.327+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a97d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a94090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968a94d00 con 0x559968a8d2d0 2023-07-21T19:33:32.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968a94e70 con 0x5599688e36b0 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559968af4750 con 0x559968a87580 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43c56e7b80 10 client.0 _ll_register_callbacks cb 0x7f43a40997c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:44674/0 (socket says 172.21.15.5:44674) 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 -- 172.21.15.5:0/2070756163 learned_addr learned my addr 172.21.15.5:0/2070756163 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43aaffd700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a94090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 -- 172.21.15.5:0/2070756163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 msgr2=0x559968a94090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a94090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 -- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 msgr2=0x559968a97d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a97d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.328+0000 7f43abfff700 1 -- 172.21.15.5:0/2070756163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43a001d4a0 con 0x559968a8d2d0 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f43abfff700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f43a40090a0 tx=0x7f43a40990f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2050bfa8a96b44a7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:32.318 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f438affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a409dca0 con 0x559968a8d2d0 2023-07-21T19:33:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f439c01e040 con 0x559968a8d2d0 2023-07-21T19:33:32.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f43a409d950 con 0x559968a8d2d0 2023-07-21T19:33:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f43a40a1ce0 con 0x559968a8d2d0 2023-07-21T19:33:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f43c56e7b80 10 client.20130 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x559968805250 con 0x559968a8d2d0 2023-07-21T19:33:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.329+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f439c01e040 con 0x559968a8d2d0 2023-07-21T19:33:32.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f43a40957a0 con 0x559968a8d2d0 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 10 client.20130 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 10 client.20130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 20 client.20130 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968af59e0 0x559968af7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 4 client.20130 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559968993ff0 con 0x559968af59e0 2023-07-21T19:33:32.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.330+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968af59e0 0x559968af7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.332+0000 7f43ab7fe700 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968af59e0 0x559968af7dc0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dddbdd1fa3b8bf50 server_cookie=d68900339eb7d5ef in_seq=0 out_seq=0 2023-07-21T19:33:32.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.332+0000 7f438affd700 10 client.20130 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.376+0000 7f438affd700 1 -- 172.21.15.5:0/2070756163 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559968993ff0 con 0x559968af59e0 2023-07-21T19:33:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.376+0000 7f438affd700 10 client.20130 handle_command_reply: tid=0 2023-07-21T19:33:32.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.376+0000 7f43c56e7b80 1 client.20130 shutdown 2023-07-21T19:33:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.377+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 msgr2=0x559968a945f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.377+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f43a40090a0 tx=0x7f43a40990f0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 shutdown_connections 2023-07-21T19:33:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559968af59e0 0x559968af7dc0 crc :-1 s=READY pgs=492 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559968a8d2d0 0x559968a945f0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5599688e36b0 0x559968a97d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 --2- 172.21.15.5:0/2070756163 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559968a87580 0x559968a94090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.378+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 >> 172.21.15.5:0/2070756163 conn(0x559968a7cbe0 msgr2=0x559968a9c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.379+0000 7f438affd700 0 client.20130 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.379+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 shutdown_connections 2023-07-21T19:33:32.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.379+0000 7f43c56e7b80 1 -- 172.21.15.5:0/2070756163 wait complete. 2023-07-21T19:33:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.380+0000 7f43c56e7b80 20 client.20130 trim_cache size 0 max 16384 2023-07-21T19:33:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:33:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:33:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:33:32.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:33:32.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 msgr2=0x7f43940a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 0x7f43940a3770 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x559968a83b40 tx=0x7f43a00095c0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 msgr2=0x7f43ac1a3c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac1a3c70 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f43a400efc0 tx=0x7f43a400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 shutdown_connections 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f43940a12c0 0x7f43940a3770 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f43ac105b60 0x7f43ac1a3c70 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f43ac0ffb30 0x7f43ac0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 --2- 172.21.15.5:0/2942150372 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f43ac1004e0 0x7f43ac072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.381+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 >> 172.21.15.5:0/2942150372 conn(0x7f43ac0fb5f0 msgr2=0x7f43ac103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.382+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 shutdown_connections 2023-07-21T19:33:32.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:32.382+0000 7f43b1dff700 1 -- 172.21.15.5:0/2942150372 wait complete. 2023-07-21T19:33:33.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:33 smithi005 ceph-mon[126460]: pgmap v822: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 288 op/s 2023-07-21T19:33:33.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:33 smithi156 ceph-mon[128316]: pgmap v822: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 288 op/s 2023-07-21T19:33:34.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:33 smithi101 ceph-mon[128136]: pgmap v822: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 2.7 MiB/s wr, 288 op/s 2023-07-21T19:33:35.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:35 smithi005 ceph-mon[126460]: pgmap v823: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T19:33:35.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:35 smithi156 ceph-mon[128316]: pgmap v823: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T19:33:36.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:35 smithi101 ceph-mon[128136]: pgmap v823: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1018 KiB/s rd, 2.4 MiB/s wr, 225 op/s 2023-07-21T19:33:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 425.0 s, 133.6 tps, lat 896.820 ms stddev 1062.729 2023-07-21T19:33:37.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:37 smithi005 ceph-mon[126460]: pgmap v824: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2023-07-21T19:33:37.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:37 smithi156 ceph-mon[128316]: pgmap v824: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2023-07-21T19:33:38.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:37 smithi101 ceph-mon[128136]: pgmap v824: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 996 KiB/s rd, 2.3 MiB/s wr, 206 op/s 2023-07-21T19:33:39.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:38 smithi101 ceph-mon[128136]: pgmap v825: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2023-07-21T19:33:39.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:38 smithi005 ceph-mon[126460]: pgmap v825: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2023-07-21T19:33:39.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:38 smithi156 ceph-mon[128316]: pgmap v825: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 350 op/s 2023-07-21T19:33:41.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:41 smithi005 ceph-mon[126460]: pgmap v826: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 302 op/s 2023-07-21T19:33:41.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:41 smithi156 ceph-mon[128316]: pgmap v826: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 302 op/s 2023-07-21T19:33:41.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 430.0 s, 239.2 tps, lat 348.917 ms stddev 78.341 2023-07-21T19:33:42.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:41 smithi101 ceph-mon[128136]: pgmap v826: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 3.5 MiB/s wr, 302 op/s 2023-07-21T19:33:43.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:43 smithi005 ceph-mon[126460]: pgmap v827: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 617 op/s 2023-07-21T19:33:43.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:43 smithi156 ceph-mon[128316]: pgmap v827: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 617 op/s 2023-07-21T19:33:44.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:43 smithi101 ceph-mon[128136]: pgmap v827: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 6.5 MiB/s wr, 617 op/s 2023-07-21T19:33:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 -- 172.21.15.5:0/2071081475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c641004e0 msgr2=0x7f2c6410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 --2- 172.21.15.5:0/2071081475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c641004e0 0x7f2c6410c920 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00b0a0 tx=0x7f2c5c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 -- 172.21.15.5:0/2071081475 shutdown_connections 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 --2- 172.21.15.5:0/2071081475 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c641004e0 0x7f2c6410c920 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 --2- 172.21.15.5:0/2071081475 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 0x7f2c64105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 --2- 172.21.15.5:0/2071081475 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c640ffb30 0x7f2c640fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 -- 172.21.15.5:0/2071081475 >> 172.21.15.5:0/2071081475 conn(0x7f2c640fb5f0 msgr2=0x7f2c640fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.568+0000 7f2c695ff700 1 -- 172.21.15.5:0/2071081475 shutdown_connections 2023-07-21T19:33:45.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.569+0000 7f2c695ff700 1 -- 172.21.15.5:0/2071081475 wait complete. 2023-07-21T19:33:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.569+0000 7f2c695ff700 1 Processor -- start 2023-07-21T19:33:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 -- start start 2023-07-21T19:33:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c641004e0 0x7f2c6419c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 0x7f2c641959b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64196020 con 0x7f2c640ffb30 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64196190 con 0x7f2c641004e0 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c695ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c64196350 con 0x7f2c64105b60 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56920/0 (socket says 172.21.15.5:56920) 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c62ffd700 1 -- 172.21.15.5:0/1880035664 learned_addr learned my addr 172.21.15.5:0/1880035664 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:45.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.570+0000 7f2c627fc700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c641004e0 0x7f2c6419c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c637fe700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 0x7f2c641959b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 -- 172.21.15.5:0/1880035664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 msgr2=0x7f2c641959b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 0x7f2c641959b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 -- 172.21.15.5:0/1880035664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c641004e0 msgr2=0x7f2c6419c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c641004e0 0x7f2c6419c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 -- 172.21.15.5:0/1880035664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c5c009d00 con 0x7f2c640ffb30 2023-07-21T19:33:45.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d950 tx=0x7f2c5400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c4bfff700 1 -- 172.21.15.5:0/1880035664 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400f890 con 0x7f2c640ffb30 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c4bfff700 1 -- 172.21.15.5:0/1880035664 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c5400fe80 con 0x7f2c640ffb30 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c4bfff700 1 -- 172.21.15.5:0/1880035664 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5400e3b0 con 0x7f2c640ffb30 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c64196630 con 0x7f2c640ffb30 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.571+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641a2b40 con 0x7f2c640ffb30 2023-07-21T19:33:45.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.575+0000 7f2c4bfff700 1 -- 172.21.15.5:0/1880035664 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2c54010460 con 0x7f2c640ffb30 2023-07-21T19:33:45.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 Processor -- start 2023-07-21T19:33:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 -- start start 2023-07-21T19:33:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c4bfff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 0x7f2c4c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f9201791a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f920173450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920173de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c4bfff700 1 -- 172.21.15.5:0/1880035664 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2c540ccae0 con 0x7f2c640ffb30 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f920074c50 con 0x55f91ffcb6b0 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9200c79e0 con 0x55f920173990 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.576+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9200c0e70 con 0x55f9201796e0 2023-07-21T19:33:45.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c637fe700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 0x7f2c4c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920173de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f9201791a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c637fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920173de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55732/0 (socket says 172.21.15.5:55732) 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c637fe700 1 -- 172.21.15.5:0/2063766388 learned_addr learned my addr 172.21.15.5:0/2063766388 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c627fc700 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f920173450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 -- 172.21.15.5:0/2063766388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 msgr2=0x55f920173450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f920173450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 -- 172.21.15.5:0/2063766388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 msgr2=0x55f920173de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920173de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.577+0000 7f2c62ffd700 1 -- 172.21.15.5:0/2063766388 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91feeed80 con 0x55f91ffcb6b0 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.578+0000 7f2c637fe700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 0x7f2c4c0a37f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d010 tx=0x7f2c5c0094d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.578+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f9201791a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2c540ca1d0 tx=0x7f2c54010760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fffb34df8ac1734 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.578+0000 7f2c497fa700 1 -- 172.21.15.5:0/2063766388 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c5409db80 con 0x55f91ffcb6b0 2023-07-21T19:33:45.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.578+0000 7f2c497fa700 1 -- 172.21.15.5:0/2063766388 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c540a1fa0 con 0x55f91ffcb6b0 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.578+0000 7f2c497fa700 1 -- 172.21.15.5:0/2063766388 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540a1b10 con 0x55f91ffcb6b0 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/2063766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 msgr2=0x55f9201791a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f9201791a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f2c540ca1d0 tx=0x7f2c54010760 comp rx=0 tx=0).stop 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/2063766388 shutdown_connections 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920173de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f9201791a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/2063766388 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f920173450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/2063766388 >> 172.21.15.5:0/2063766388 conn(0x55f920164be0 msgr2=0x55f920169f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/2063766388 shutdown_connections 2023-07-21T19:33:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/2063766388 wait complete. 2023-07-21T19:33:45.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.579+0000 7f2c7ceddb80 1 -- start start 2023-07-21T19:33:45.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 Processor -- start 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f92017cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f920060a60 con 0x55f91ffcb6b0 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f920060760 con 0x55f920173990 2023-07-21T19:33:45.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f91ff448c0 con 0x55f9201796e0 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c7ceddb80 10 client.0 _ll_register_callbacks cb 0x55f92005e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56930/0 (socket says 172.21.15.5:56930) 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55736/0 (socket says 172.21.15.5:55736) 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.580+0000 7f2c62ffd700 1 -- 172.21.15.5:0/61867536 learned_addr learned my addr 172.21.15.5:0/61867536 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c637fe700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f92017cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 -- 172.21.15.5:0/61867536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 msgr2=0x55f92017cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f92017cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 -- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 msgr2=0x55f920182e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 -- 172.21.15.5:0/61867536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c641a2b40 con 0x55f91ffcb6b0 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c627fc700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c637fe700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f92017cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:45.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2c54099850 tx=0x7f2c540992f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83da6e6e63a88896 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c540b22e0 con 0x55f91ffcb6b0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2c54099db0 con 0x55f91ffcb6b0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f92001a490 con 0x55f91ffcb6b0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c54099f70 con 0x55f91ffcb6b0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c7ceddb80 10 client.20142 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.581+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55f91fed7c00 con 0x55f91ffcb6b0 2023-07-21T19:33:45.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f92001a490 con 0x55f91ffcb6b0 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c54098ab0 con 0x55f91ffcb6b0 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 10 client.20142 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 10 client.20142 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 20 client.20142 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f92018c9b0 0x55f92018ed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 4 client.20142 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:45.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f9200b9e50 con 0x55f92018c9b0 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.582+0000 7f2c627fc700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f92018c9b0 0x55f92018ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.583+0000 7f2c627fc700 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f92018c9b0 0x55f92018ed90 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db0827d4fcc0f3bb server_cookie=60c53f9f68946c77 in_seq=0 out_seq=0 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.583+0000 7f2c46ff5700 10 client.20142 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c46ff5700 1 -- 172.21.15.5:0/61867536 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f9200b9e50 con 0x55f92018c9b0 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c46ff5700 10 client.20142 handle_command_reply: tid=0 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 client.20142 shutdown 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 msgr2=0x55f920182940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f2c54099850 tx=0x7f2c540992f0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 shutdown_connections 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f92017cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f92018c9b0 0x55f92018ed90 crc :-1 s=READY pgs=493 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f91ffcb6b0 0x55f920182940 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/61867536 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f920182e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 >> 172.21.15.5:0/61867536 conn(0x55f920164be0 msgr2=0x55f9201694e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c46ff5700 0 client.20142 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:45.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 shutdown_connections 2023-07-21T19:33:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/61867536 wait complete. 2023-07-21T19:33:45.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.584+0000 7f2c7ceddb80 20 client.20142 trim_cache size 0 max 16384 2023-07-21T19:33:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.600+0000 7f2c7ceddb80 1 Processor -- start 2023-07-21T19:33:45.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.600+0000 7f2c7ceddb80 1 -- start start 2023-07-21T19:33:45.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.600+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 0x55f920173450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9201796e0 0x55f920169c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f920166d00 con 0x55f9201796e0 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f92017e1e0 con 0x55f920173990 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9201665f0 con 0x55f91ffcb6b0 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c637fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9201796e0 0x55f920169c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55742/0 (socket says 172.21.15.5:55742) 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 0x55f920173450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 -- 172.21.15.5:0/3542969278 learned_addr learned my addr 172.21.15.5:0/3542969278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 -- 172.21.15.5:0/3542969278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 msgr2=0x55f920173450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 0x55f920173450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 -- 172.21.15.5:0/3542969278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9201796e0 msgr2=0x55f920169c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9201796e0 0x55f920169c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c627fc700 1 -- 172.21.15.5:0/3542969278 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f91ff151f0 con 0x55f920173990 2023-07-21T19:33:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.601+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 0x55f920173450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.602+0000 7f2c627fc700 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c640fef20 tx=0x7f2c58004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=76049c96797b9f7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.602+0000 7f2c49ffb700 1 -- 172.21.15.5:0/3542969278 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58016070 con 0x55f920173990 2023-07-21T19:33:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.602+0000 7f2c49ffb700 1 -- 172.21.15.5:0/3542969278 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c58017040 con 0x55f920173990 2023-07-21T19:33:45.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.602+0000 7f2c49ffb700 1 -- 172.21.15.5:0/3542969278 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c580105d0 con 0x55f920173990 2023-07-21T19:33:45.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/3542969278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 msgr2=0x55f9201785b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f2c640fef20 tx=0x7f2c58004750 comp rx=0 tx=0).stop 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/3542969278 shutdown_connections 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f9201796e0 0x55f920169c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f91ffcb6b0 0x55f920173450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/3542969278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f920173990 0x55f9201785b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/3542969278 >> 172.21.15.5:0/3542969278 conn(0x55f920164be0 msgr2=0x55f920177ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/3542969278 shutdown_connections 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/3542969278 wait complete. 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 -- start start 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.603+0000 7f2c7ceddb80 1 Processor -- start 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 0x55f9201d47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f9201d5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9201df9d0 con 0x55f920173990 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9201dfb40 con 0x55f91ffcb6b0 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f9201dfd00 con 0x55f9201796e0 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c7ceddb80 10 client.0 _ll_register_callbacks cb 0x7f2c58005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c62ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 0x55f9201d47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c627fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56940/0 (socket says 172.21.15.5:56940) 2023-07-21T19:33:45.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c627fc700 1 -- 172.21.15.5:0/1065612356 learned_addr learned my addr 172.21.15.5:0/1065612356 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.604+0000 7f2c637fe700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f9201d5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 -- 172.21.15.5:0/1065612356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 msgr2=0x55f9201d5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f9201d5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 -- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 msgr2=0x55f9201d47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 0x55f9201d47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 -- 172.21.15.5:0/1065612356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f91fed7c00 con 0x55f920173990 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 0x55f9201d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:45.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c627fc700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c580092a0 tx=0x7f2c5801d5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96691f0e4549cb85 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c46ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c58016070 con 0x55f920173990 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2c540a1fa0 con 0x55f920173990 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2c58005450 con 0x55f920173990 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2c58005610 con 0x55f920173990 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c7ceddb80 10 client.20148 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.605+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f2c5801d9d0 con 0x55f920173990 2023-07-21T19:33:45.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2c540a1fa0 con 0x55f920173990 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2c580038e0 con 0x55f920173990 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 10 client.20148 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 10 client.20148 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 20 client.20148 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f9201e1470 0x55f9201e3880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 4 client.20148 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:33:45.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f92007bff0 con 0x55f9201e1470 2023-07-21T19:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.606+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f9201e1470 0x55f9201e3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.607+0000 7f2c62ffd700 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f9201e1470 0x55f9201e3880 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cbf81c4eec40325 server_cookie=a384be4831c08dbb in_seq=0 out_seq=0 2023-07-21T19:33:45.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.607+0000 7f2c46ff5700 10 client.20148 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c46ff5700 1 -- 172.21.15.5:0/1065612356 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f92007bff0 con 0x55f9201e1470 2023-07-21T19:33:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c46ff5700 10 client.20148 handle_command_reply: tid=0 2023-07-21T19:33:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 client.20148 shutdown 2023-07-21T19:33:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 msgr2=0x55f9201d4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f2c580092a0 tx=0x7f2c5801d5c0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 shutdown_connections 2023-07-21T19:33:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f9201796e0 0x55f9201d5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f91ffcb6b0 0x55f9201d47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f9201e1470 0x55f9201e3880 crc :-1 s=READY pgs=494 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 --2- 172.21.15.5:0/1065612356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f920173990 0x55f9201d4d10 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.749+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 >> 172.21.15.5:0/1065612356 conn(0x55f920164be0 msgr2=0x55f92017cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.750+0000 7f2c46ff5700 0 client.20148 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.750+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 shutdown_connections 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.750+0000 7f2c7ceddb80 1 -- 172.21.15.5:0/1065612356 wait complete. 2023-07-21T19:33:45.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.751+0000 7f2c7ceddb80 20 client.20148 trim_cache size 0 max 16384 2023-07-21T19:33:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:33:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:33:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:33:45.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:33:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 msgr2=0x7f2c4c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 0x7f2c4c0a37f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c00d010 tx=0x7f2c5c0094d0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 msgr2=0x7f2c6419bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f2c5400d950 tx=0x7f2c5400dc60 comp rx=0 tx=0).stop 2023-07-21T19:33:45.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 shutdown_connections 2023-07-21T19:33:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2c64105b60 0x7f2c641959b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2c4c0a1340 0x7f2c4c0a37f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2c640ffb30 0x7f2c6419bb50 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 --2- 172.21.15.5:0/1880035664 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2c641004e0 0x7f2c6419c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:45.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.752+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 >> 172.21.15.5:0/1880035664 conn(0x7f2c640fb5f0 msgr2=0x7f2c640fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.753+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 shutdown_connections 2023-07-21T19:33:45.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:45.753+0000 7f2c695ff700 1 -- 172.21.15.5:0/1880035664 wait complete. 2023-07-21T19:33:45.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:45 smithi005 ceph-mon[126460]: pgmap v828: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-07-21T19:33:45.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:45 smithi156 ceph-mon[128316]: pgmap v828: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-07-21T19:33:46.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:45 smithi101 ceph-mon[128136]: pgmap v828: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 573 op/s 2023-07-21T19:33:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 435.0 s, 243.0 tps, lat 473.478 ms stddev 407.297 2023-07-21T19:33:47.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:47 smithi005 ceph-mon[126460]: pgmap v829: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 541 op/s 2023-07-21T19:33:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:47 smithi156 ceph-mon[128316]: pgmap v829: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 541 op/s 2023-07-21T19:33:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:47 smithi101 ceph-mon[128136]: pgmap v829: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.6 MiB/s wr, 541 op/s 2023-07-21T19:33:49.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:48 smithi101 ceph-mon[128136]: pgmap v830: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 876 op/s 2023-07-21T19:33:49.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:48 smithi005 ceph-mon[126460]: pgmap v830: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 876 op/s 2023-07-21T19:33:49.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:48 smithi156 ceph-mon[128316]: pgmap v830: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.1 MiB/s wr, 876 op/s 2023-07-21T19:33:51.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:51 smithi005 ceph-mon[126460]: pgmap v831: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 732 op/s 2023-07-21T19:33:51.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:51 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:33:51.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:51 smithi156 ceph-mon[128316]: pgmap v831: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 732 op/s 2023-07-21T19:33:51.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:51 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:33:51.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 440.0 s, 238.8 tps, lat 414.543 ms stddev 88.134 2023-07-21T19:33:52.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:51 smithi101 ceph-mon[128136]: pgmap v831: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.5 MiB/s wr, 732 op/s 2023-07-21T19:33:52.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:51 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:33:53.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:53 smithi005 ceph-mon[126460]: pgmap v832: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-07-21T19:33:53.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:53 smithi156 ceph-mon[128316]: pgmap v832: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-07-21T19:33:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:53 smithi101 ceph-mon[128136]: pgmap v832: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 11 MiB/s wr, 1.02k op/s 2023-07-21T19:33:55.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:55 smithi005 ceph-mon[126460]: pgmap v833: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-07-21T19:33:55.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:55 smithi156 ceph-mon[128316]: pgmap v833: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-07-21T19:33:56.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:55 smithi101 ceph-mon[128136]: pgmap v833: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.8 MiB/s wr, 736 op/s 2023-07-21T19:33:56.570 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:33:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 -- 172.21.15.5:0/2915053210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c110b50 msgr2=0x7feb3c112f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 --2- 172.21.15.5:0/2915053210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c110b50 0x7feb3c112f30 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7feb3400b280 tx=0x7feb340089d0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 -- 172.21.15.5:0/2915053210 shutdown_connections 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 --2- 172.21.15.5:0/2915053210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c110b50 0x7feb3c112f30 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 --2- 172.21.15.5:0/2915053210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c107780 0x7feb3c107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 --2- 172.21.15.5:0/2915053210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb3c108090 0x7feb3c110580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.926+0000 7feb41fff700 1 -- 172.21.15.5:0/2915053210 >> 172.21.15.5:0/2915053210 conn(0x7feb3c06c200 msgr2=0x7feb3c06c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.927+0000 7feb41fff700 1 -- 172.21.15.5:0/2915053210 shutdown_connections 2023-07-21T19:33:56.914 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.927+0000 7feb41fff700 1 -- 172.21.15.5:0/2915053210 wait complete. 2023-07-21T19:33:56.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.928+0000 7feb41fff700 1 Processor -- start 2023-07-21T19:33:56.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.928+0000 7feb41fff700 1 -- start start 2023-07-21T19:33:56.915 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb3c107780 0x7feb3c19c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 0x7feb3c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 0x7feb3c1a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1a12a0 con 0x7feb3c110b50 2023-07-21T19:33:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1a1410 con 0x7feb3c107780 2023-07-21T19:33:56.916 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.929+0000 7feb41fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c1a15d0 con 0x7feb3c108090 2023-07-21T19:33:56.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 0x7feb3c1a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 0x7feb3c1a0c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55350/0 (socket says 172.21.15.5:55350) 2023-07-21T19:33:56.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3bfff700 1 -- 172.21.15.5:0/4107864096 learned_addr learned my addr 172.21.15.5:0/4107864096 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:56.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 0x7feb3c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 -- 172.21.15.5:0/4107864096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb3c107780 msgr2=0x7feb3c19c2c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:33:56.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb3c107780 0x7feb3c19c2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 -- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 msgr2=0x7feb3c1a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.918 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 0x7feb3c1a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.930+0000 7feb3affd700 1 -- 172.21.15.5:0/4107864096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34008cf0 con 0x7feb3c108090 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.931+0000 7feb3affd700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 0x7feb3c19c800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feb2c010aa0 tx=0x7feb2c010db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.931+0000 7feb38ff9700 1 -- 172.21.15.5:0/4107864096 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00e9c0 con 0x7feb3c108090 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.931+0000 7feb38ff9700 1 -- 172.21.15.5:0/4107864096 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb2c011040 con 0x7feb3c108090 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.931+0000 7feb38ff9700 1 -- 172.21.15.5:0/4107864096 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c0153e0 con 0x7feb3c108090 2023-07-21T19:33:56.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.931+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c1a18b0 con 0x7feb3c108090 2023-07-21T19:33:56.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.932+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c1a90f0 con 0x7feb3c108090 2023-07-21T19:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.939+0000 7feb5587db80 1 Processor -- start 2023-07-21T19:33:56.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.940+0000 7feb38ff9700 1 -- 172.21.15.5:0/4107864096 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7feb2c00eb20 con 0x7feb3c108090 2023-07-21T19:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.940+0000 7feb5587db80 1 -- start start 2023-07-21T19:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf0200be0 0x55ebf020f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf02122c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0106cb0 con 0x55ebf0200be0 2023-07-21T19:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0159a40 con 0x55ebf020fee0 2023-07-21T19:33:56.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb5587db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0152ed0 con 0x55ebf02002d0 2023-07-21T19:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb3affd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf02122c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35058/0 (socket says 172.21.15.5:35058) 2023-07-21T19:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.941+0000 7feb3bfff700 1 -- 172.21.15.5:0/2824106334 learned_addr learned my addr 172.21.15.5:0/2824106334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:56.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf0200be0 0x55ebf020f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 -- 172.21.15.5:0/2824106334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 msgr2=0x55ebf02122c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf02122c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 -- 172.21.15.5:0/2824106334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf0200be0 msgr2=0x55ebf020f9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf0200be0 0x55ebf020f9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 -- 172.21.15.5:0/2824106334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebeff80de0 con 0x55ebf02002d0 2023-07-21T19:33:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb38ff9700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 0x7feb240a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.942+0000 7feb3bfff700 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feb34007570 tx=0x7feb34008b70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f9102633424b1d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.943+0000 7feb38ff9700 1 -- 172.21.15.5:0/4107864096 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7feb2c0cf0f0 con 0x7feb3c108090 2023-07-21T19:33:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.943+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 0x7feb240a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.943+0000 7feb21ffb700 1 -- 172.21.15.5:0/2824106334 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340123e0 con 0x55ebf02002d0 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.943+0000 7feb21ffb700 1 -- 172.21.15.5:0/2824106334 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb34003e90 con 0x55ebf02002d0 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.943+0000 7feb21ffb700 1 -- 172.21.15.5:0/2824106334 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3401e6e0 con 0x55ebf02002d0 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.944+0000 7feb5587db80 1 -- 172.21.15.5:0/2824106334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 msgr2=0x55ebf02006a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.944+0000 7feb5587db80 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7feb34007570 tx=0x7feb34008b70 comp rx=0 tx=0).stop 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 0x7feb240a3870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb30009bf0 tx=0x7feb30009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 -- 172.21.15.5:0/2824106334 shutdown_connections 2023-07-21T19:33:56.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf02122c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf02006a0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 --2- 172.21.15.5:0/2824106334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf0200be0 0x55ebf020f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 -- 172.21.15.5:0/2824106334 >> 172.21.15.5:0/2824106334 conn(0x55ebf01f6da0 msgr2=0x55ebf01fc6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 -- 172.21.15.5:0/2824106334 shutdown_connections 2023-07-21T19:33:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 -- 172.21.15.5:0/2824106334 wait complete. 2023-07-21T19:33:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.945+0000 7feb5587db80 1 -- start start 2023-07-21T19:33:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 Processor -- start 2023-07-21T19:33:56.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf0200be0 0x55ebf020cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf00f2ac0 con 0x55ebf02002d0 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf00f27c0 con 0x55ebf020fee0 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebeffd6920 con 0x55ebf0200be0 2023-07-21T19:33:56.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.946+0000 7feb5587db80 10 client.0 _ll_register_callbacks cb 0x55ebf00f0760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:56.937 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 445.0 s, 231.0 tps, lat 431.373 ms stddev 112.532 2023-07-21T19:33:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52022/0 (socket says 172.21.15.5:52022) 2023-07-21T19:33:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 -- 172.21.15.5:0/1796648416 learned_addr learned my addr 172.21.15.5:0/1796648416 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:56.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3affd700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf0200be0 0x55ebf020cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 -- 172.21.15.5:0/1796648416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf0200be0 msgr2=0x55ebf020cbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf0200be0 0x55ebf020cbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 -- 172.21.15.5:0/1796648416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 msgr2=0x55ebf020f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.947+0000 7feb3bfff700 1 -- 172.21.15.5:0/1796648416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c113bd0 con 0x55ebf020fee0 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.948+0000 7feb3bfff700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb34016040 tx=0x7feb34006000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=111ddc393a5326af server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.948+0000 7feb1affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:33:56.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.948+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34003a00 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.948+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7feb34003b60 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.949+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ebf00ac4f0 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.949+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb34003d20 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.949+0000 7feb5587db80 10 client.29951 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.949+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55ebeff69c60 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55ebf00ac4f0 con 0x55ebf020fee0 2023-07-21T19:33:56.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb34013480 con 0x55ebf020fee0 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 10 client.29951 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 10 client.29951 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 20 client.29951 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf021f3e0 0x55ebf02217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 4 client.29951 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.950+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ebf014beb0 con 0x55ebf021f3e0 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.951+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf021f3e0 0x55ebf02217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.951+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf021f3e0 0x55ebf02217f0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af4540f31a44639 server_cookie=cf4291b6f9d228fc in_seq=0 out_seq=0 2023-07-21T19:33:56.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.951+0000 7feb1affd700 10 client.29951 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:56.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.952+0000 7feb1affd700 1 -- 172.21.15.5:0/1796648416 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ebf014beb0 con 0x55ebf021f3e0 2023-07-21T19:33:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.952+0000 7feb1affd700 10 client.29951 handle_command_reply: tid=0 2023-07-21T19:33:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.952+0000 7feb5587db80 1 client.29951 shutdown 2023-07-21T19:33:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 msgr2=0x55ebf020d280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7feb34016040 tx=0x7feb34006000 comp rx=0 tx=0).stop 2023-07-21T19:33:56.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 shutdown_connections 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf020fee0 0x55ebf020d280 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf021f3e0 0x55ebf02217f0 crc :-1 s=READY pgs=495 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 --2- 172.21.15.5:0/1796648416 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf0200be0 0x55ebf020cbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 >> 172.21.15.5:0/1796648416 conn(0x55ebf01f6da0 msgr2=0x55ebf01fc230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb1affd700 0 client.29951 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:56.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 shutdown_connections 2023-07-21T19:33:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 1 -- 172.21.15.5:0/1796648416 wait complete. 2023-07-21T19:33:56.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.953+0000 7feb5587db80 20 client.29951 trim_cache size 0 max 16384 2023-07-21T19:33:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 Processor -- start 2023-07-21T19:33:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 -- start start 2023-07-21T19:33:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf020c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.968 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf01fdd20 con 0x55ebf02002d0 2023-07-21T19:33:56.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0201540 con 0x55ebf0200be0 2023-07-21T19:33:56.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.981+0000 7feb5587db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf020e150 con 0x55ebf020fee0 2023-07-21T19:33:56.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35080/0 (socket says 172.21.15.5:35080) 2023-07-21T19:33:56.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 -- 172.21.15.5:0/714747334 learned_addr learned my addr 172.21.15.5:0/714747334 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:56.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3affd700 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf020c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 -- 172.21.15.5:0/714747334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 msgr2=0x55ebf020c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf020c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 -- 172.21.15.5:0/714747334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 msgr2=0x55ebf020c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.982+0000 7feb3bfff700 1 -- 172.21.15.5:0/714747334 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ebeffa7250 con 0x55ebf020fee0 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.983+0000 7feb3bfff700 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feb34034040 tx=0x7feb34006070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e46cb73ea935b002 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.983+0000 7feb227fc700 1 -- 172.21.15.5:0/714747334 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34007530 con 0x55ebf020fee0 2023-07-21T19:33:56.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.983+0000 7feb227fc700 1 -- 172.21.15.5:0/714747334 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb34012720 con 0x55ebf020fee0 2023-07-21T19:33:56.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.983+0000 7feb227fc700 1 -- 172.21.15.5:0/714747334 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340054d0 con 0x55ebf020fee0 2023-07-21T19:33:56.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 -- 172.21.15.5:0/714747334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 msgr2=0x55ebf0209e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7feb34034040 tx=0x7feb34006070 comp rx=0 tx=0).stop 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 -- 172.21.15.5:0/714747334 shutdown_connections 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf020fee0 0x55ebf0209e70 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf02002d0 0x55ebf020c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 --2- 172.21.15.5:0/714747334 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf020c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 -- 172.21.15.5:0/714747334 >> 172.21.15.5:0/714747334 conn(0x55ebf01f6da0 msgr2=0x55ebf020f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 -- 172.21.15.5:0/714747334 shutdown_connections 2023-07-21T19:33:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.984+0000 7feb5587db80 1 -- 172.21.15.5:0/714747334 wait complete. 2023-07-21T19:33:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.985+0000 7feb5587db80 1 -- start start 2023-07-21T19:33:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.985+0000 7feb5587db80 1 Processor -- start 2023-07-21T19:33:56.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf0216960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf0214cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf020fee0 0x55ebf0215230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0215920 con 0x55ebf020fee0 2023-07-21T19:33:56.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0215a90 con 0x55ebf0200be0 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ebf0273270 con 0x55ebf02002d0 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.986+0000 7feb5587db80 10 client.0 _ll_register_callbacks cb 0x7feb2c0d07d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf0216960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf0216960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35090/0 (socket says 172.21.15.5:35090) 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3b7fe700 1 -- 172.21.15.5:0/3558206302 learned_addr learned my addr 172.21.15.5:0/3558206302 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3affd700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf0214cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3bfff700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf020fee0 0x55ebf0215230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3affd700 1 -- 172.21.15.5:0/3558206302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 msgr2=0x55ebf0216960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.987+0000 7feb3affd700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf0216960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.988+0000 7feb3affd700 1 -- 172.21.15.5:0/3558206302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf020fee0 msgr2=0x55ebf0215230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.988+0000 7feb3affd700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf020fee0 0x55ebf0215230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.988+0000 7feb3affd700 1 -- 172.21.15.5:0/3558206302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ebeff69c60 con 0x55ebf0200be0 2023-07-21T19:33:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb3affd700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf0214cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0cc7e0 tx=0x7feb2c0d0500 comp rx=0 tx=0).ready entity=mon.2 client_cookie=40b495de9ce576c9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb1affd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:33:56.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c09b850 con 0x55ebf0200be0 2023-07-21T19:33:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7feb2c016680 con 0x55ebf0200be0 2023-07-21T19:33:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feb34003e90 con 0x55ebf0200be0 2023-07-21T19:33:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.991+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7feb2c09bf30 con 0x55ebf0200be0 2023-07-21T19:33:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 10 client.29957 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:56.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55ebeff7f2b0 con 0x55ebf0200be0 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feb34003e90 con 0x55ebf0200be0 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7feb2c0a4120 con 0x55ebf0200be0 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 10 client.29957 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 10 client.29957 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 20 client.29957 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:56.980 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.992+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf0203c60 0x55ebf0206070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.993+0000 7feb5587db80 4 client.29957 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:33:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.993+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55ebf010e050 con 0x55ebf0203c60 2023-07-21T19:33:56.981 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.993+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf0203c60 0x55ebf0206070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb1affd700 10 client.29957 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb3b7fe700 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf0203c60 0x55ebf0206070 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb39e7a5e04a48ef server_cookie=b3572eb1f41b5a9a in_seq=0 out_seq=0 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb1affd700 1 -- 172.21.15.5:0/3558206302 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55ebf010e050 con 0x55ebf0203c60 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb1affd700 10 client.29957 handle_command_reply: tid=0 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb5587db80 1 client.29957 shutdown 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 msgr2=0x55ebf0214cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.994+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf0214cf0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7feb2c0cc7e0 tx=0x7feb2c0d0500 comp rx=0 tx=0).stop 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 shutdown_connections 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55ebf0203c60 0x55ebf0206070 crc :-1 s=READY pgs=496 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55ebf020fee0 0x55ebf0215230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55ebf02002d0 0x55ebf0216960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 --2- 172.21.15.5:0/3558206302 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55ebf0200be0 0x55ebf0214cf0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 >> 172.21.15.5:0/3558206302 conn(0x55ebf01f6da0 msgr2=0x55ebf0202c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb1affd700 0 client.29957 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 shutdown_connections 2023-07-21T19:33:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 1 -- 172.21.15.5:0/3558206302 wait complete. 2023-07-21T19:33:56.984 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.995+0000 7feb5587db80 20 client.29957 trim_cache size 0 max 16384 2023-07-21T19:33:56.985 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:33:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:33:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:33:56.985 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:33:56.986 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:33:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 msgr2=0x7feb240a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 0x7feb240a3870 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7feb30009bf0 tx=0x7feb30009260 comp rx=0 tx=0).stop 2023-07-21T19:33:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 msgr2=0x7feb3c19c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 0x7feb3c19c800 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feb2c010aa0 tx=0x7feb2c010db0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.987 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 shutdown_connections 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feb240a13c0 0x7feb240a3870 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feb3c110b50 0x7feb3c1a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feb3c107780 0x7feb3c19c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 --2- 172.21.15.5:0/4107864096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feb3c108090 0x7feb3c19c800 secure :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7feb2c010aa0 tx=0x7feb2c010db0 comp rx=0 tx=0).stop 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.996+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 >> 172.21.15.5:0/4107864096 conn(0x7feb3c06c200 msgr2=0x7feb3c06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.997+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 shutdown_connections 2023-07-21T19:33:56.988 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:33:56.997+0000 7feb41fff700 1 -- 172.21.15.5:0/4107864096 wait complete. 2023-07-21T19:33:57.012 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:33:57.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:57 smithi005 ceph-mon[126460]: pgmap v834: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 654 op/s 2023-07-21T19:33:57.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:57 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:57.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:57 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:57.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:57 smithi156 ceph-mon[128316]: pgmap v834: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 654 op/s 2023-07-21T19:33:57.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:57 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:57.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:57 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:58.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:57 smithi101 ceph-mon[128136]: pgmap v834: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.8 MiB/s wr, 654 op/s 2023-07-21T19:33:58.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:57 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:58.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:57 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 -- 172.21.15.5:0/2926431056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 msgr2=0x7f4c1c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:58.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 --2- 172.21.15.5:0/2926431056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c0fff70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009990 tx=0x7f4c10009ca0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 -- 172.21.15.5:0/2926431056 shutdown_connections 2023-07-21T19:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 --2- 172.21.15.5:0/2926431056 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 --2- 172.21.15.5:0/2926431056 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c105b10 0x7f4c1c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 --2- 172.21.15.5:0/2926431056 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c0fff70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 -- 172.21.15.5:0/2926431056 >> 172.21.15.5:0/2926431056 conn(0x7f4c1c0fb600 msgr2=0x7f4c1c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 -- 172.21.15.5:0/2926431056 shutdown_connections 2023-07-21T19:33:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.994+0000 7f4c219ff700 1 -- 172.21.15.5:0/2926431056 wait complete. 2023-07-21T19:33:58.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.996+0000 7f4c219ff700 1 Processor -- start 2023-07-21T19:33:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.996+0000 7f4c219ff700 1 -- start start 2023-07-21T19:33:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c109100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 0x7f4c1c1a8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1099f0 con 0x7f4c1c105b10 2023-07-21T19:33:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a8750 con 0x7f4c1c1004b0 2023-07-21T19:33:58.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c219ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c1a88c0 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35110/0 (socket says 172.21.15.5:35110) 2023-07-21T19:33:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1affd700 1 -- 172.21.15.5:0/3401604155 learned_addr learned my addr 172.21.15.5:0/3401604155 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:58.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c109100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 0x7f4c1c1a8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.997+0000 7f4c1affd700 1 -- 172.21.15.5:0/3401604155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 msgr2=0x7f4c1c109100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1affd700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c109100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1affd700 1 -- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 msgr2=0x7f4c1c1a8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1affd700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 0x7f4c1c1a8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1affd700 1 -- 172.21.15.5:0/3401604155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100095f0 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 0x7f4c1c1a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c1affd700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9a0 tx=0x7f4c0c00ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c209fd700 1 -- 172.21.15.5:0/3401604155 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c009790 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c209fd700 1 -- 172.21.15.5:0/3401604155 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c0c00cbd0 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.998+0000 7f4c209fd700 1 -- 172.21.15.5:0/3401604155 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c018680 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.999+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c1a8ba0 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:58.999+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c1a9030 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.005+0000 7f4c209fd700 1 -- 172.21.15.5:0/3401604155 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4c0c00cd40 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.006+0000 7f4c35360b80 1 Processor -- start 2023-07-21T19:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.006+0000 7f4c35360b80 1 -- start start 2023-07-21T19:33:58.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c209fd700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 0x7f4c040a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 0x55c31c5ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c4b3c50 con 0x55c31c40a6b0 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c5069e0 con 0x55c31c5b42d0 2023-07-21T19:33:58.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c35360b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c4ffe70 con 0x55c31c5ae580 2023-07-21T19:33:58.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c209fd700 1 -- 172.21.15.5:0/3401604155 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4c0c014070 con 0x7f4c1c0ffb00 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c1a7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 0x55c31c5ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.007+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55400/0 (socket says 172.21.15.5:55400) 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 -- 172.21.15.5:0/2130832547 learned_addr learned my addr 172.21.15.5:0/2130832547 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 -- 172.21.15.5:0/2130832547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 msgr2=0x55c31c5ae9d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5ae9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 -- 172.21.15.5:0/2130832547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 msgr2=0x55c31c5ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 0x55c31c5ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 -- 172.21.15.5:0/2130832547 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c31c32dd80 con 0x55c31c40a6b0 2023-07-21T19:33:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 0x55c31c5ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:33:58.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.008+0000 7f4c1affd700 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1f10 tx=0x7f4c0c099bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bce226ae532d48e1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.009+0000 7f4c01ffb700 1 -- 172.21.15.5:0/2130832547 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c099ff0 con 0x55c31c40a6b0 2023-07-21T19:33:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.009+0000 7f4c01ffb700 1 -- 172.21.15.5:0/2130832547 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c0c09e020 con 0x55c31c40a6b0 2023-07-21T19:33:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.009+0000 7f4c01ffb700 1 -- 172.21.15.5:0/2130832547 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c0c08a7e0 con 0x55c31c40a6b0 2023-07-21T19:33:58.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.009+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 0x7f4c040a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 -- 172.21.15.5:0/2130832547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 msgr2=0x55c31c5b3d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0a1f10 tx=0x7f4c0c099bd0 comp rx=0 tx=0).stop 2023-07-21T19:33:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 0x7f4c040a3770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009480 tx=0x7f4c1400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:58.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 -- 172.21.15.5:0/2130832547 shutdown_connections 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5b3d90 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2130832547 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5b42d0 0x55c31c5ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 -- 172.21.15.5:0/2130832547 >> 172.21.15.5:0/2130832547 conn(0x55c31c5a3be0 msgr2=0x55c31c5a8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 -- 172.21.15.5:0/2130832547 shutdown_connections 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.010+0000 7f4c35360b80 1 -- 172.21.15.5:0/2130832547 wait complete. 2023-07-21T19:33:59.000 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.011+0000 7f4c35360b80 1 -- start start 2023-07-21T19:33:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.012+0000 7f4c35360b80 1 Processor -- start 2023-07-21T19:33:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5b7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b8310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c49fa60 con 0x55c31c5b42d0 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c49f760 con 0x55c31c40a6b0 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c3838c0 con 0x55c31c5ae580 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c35360b80 10 client.0 _ll_register_callbacks cb 0x55c31c49d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c1a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c1a7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35124/0 (socket says 172.21.15.5:35124) 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.013+0000 7f4c1a7fc700 1 -- 172.21.15.5:0/1901636398 learned_addr learned my addr 172.21.15.5:0/1901636398 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b8310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 -- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 msgr2=0x55c31c5b7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1affd700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5b7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5b7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 -- 172.21.15.5:0/1901636398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 msgr2=0x55c31c5b8310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b8310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 -- 172.21.15.5:0/1901636398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c10d240 con 0x55c31c5ae580 2023-07-21T19:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1affd700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009990 tx=0x7f4c10005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f8fd7937ae58fa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:59.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.014+0000 7f4bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:33:59.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.015+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1001d070 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.015+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4c100049d0 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.016+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c31c459490 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.017+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c10004b90 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.017+0000 7f4c35360b80 10 client.29881 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.017+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f4c1c1a9030 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c31c459490 con 0x55c31c5ae580 2023-07-21T19:33:59.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c1000fcd0 con 0x55c31c5ae580 2023-07-21T19:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 10 client.29881 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 10 client.29881 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:59.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 20 client.29881 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c5c7c00 0x55c31c5ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 4 client.29881 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.018+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c31c4f8e50 con 0x55c31c5c7c00 2023-07-21T19:33:59.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.019+0000 7f4c1affd700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c5c7c00 0x55c31c5ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.021+0000 7f4c1affd700 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c5c7c00 0x55c31c5ca010 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f604e2b19d6e2a4c server_cookie=43e59c1f1f444a34 in_seq=0 out_seq=0 2023-07-21T19:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.021+0000 7f4bfaffd700 10 client.29881 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.023+0000 7f4bfaffd700 1 -- 172.21.15.5:0/1901636398 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c31c4f8e50 con 0x55c31c5c7c00 2023-07-21T19:33:59.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.023+0000 7f4bfaffd700 10 client.29881 handle_command_reply: tid=0 2023-07-21T19:33:59.010 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.023+0000 7f4c35360b80 1 client.29881 shutdown 2023-07-21T19:33:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 msgr2=0x55c31c5b7dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:59.014 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4c10009990 tx=0x7f4c10005320 comp rx=0 tx=0).stop 2023-07-21T19:33:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 shutdown_connections 2023-07-21T19:33:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c5c7c00 0x55c31c5ca010 crc :-1 s=READY pgs=497 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b8310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.015 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5b7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 --2- 172.21.15.5:0/1901636398 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5b7dd0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.028+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 >> 172.21.15.5:0/1901636398 conn(0x55c31c5a3be0 msgr2=0x55c31c5a84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:59.016 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.029+0000 7f4bfaffd700 0 client.29881 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.029+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 shutdown_connections 2023-07-21T19:33:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.029+0000 7f4c35360b80 1 -- 172.21.15.5:0/1901636398 wait complete. 2023-07-21T19:33:59.017 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.029+0000 7f4c35360b80 20 client.29881 trim_cache size 0 max 16384 2023-07-21T19:33:59.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.056+0000 7f4c35360b80 1 Processor -- start 2023-07-21T19:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.056+0000 7f4c35360b80 1 -- start start 2023-07-21T19:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.056+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5ae040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.056+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c35360b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c5aa800 con 0x55c31c5b42d0 2023-07-21T19:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c35360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c5a55f0 con 0x55c31c40a6b0 2023-07-21T19:33:59.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c35360b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c5c4560 con 0x55c31c5ae580 2023-07-21T19:33:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55416/0 (socket says 172.21.15.5:55416) 2023-07-21T19:33:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/2362245440 learned_addr learned my addr 172.21.15.5:0/2362245440 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c1affd700 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.057+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/2362245440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 msgr2=0x55c31c5a8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5a8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/2362245440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 msgr2=0x55c31c5ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/2362245440 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c31c5cec40 con 0x55c31c5b42d0 2023-07-21T19:33:59.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.058+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007ed0 tx=0x7f4c140094b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6adc5921a578a5d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:59.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.059+0000 7f4c027fc700 1 -- 172.21.15.5:0/2362245440 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e050 con 0x55c31c5b42d0 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.059+0000 7f4c027fc700 1 -- 172.21.15.5:0/2362245440 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c1400f040 con 0x55c31c5b42d0 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.059+0000 7f4c027fc700 1 -- 172.21.15.5:0/2362245440 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14016650 con 0x55c31c5b42d0 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 -- 172.21.15.5:0/2362245440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 msgr2=0x55c31c5b2a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007ed0 tx=0x7f4c140094b0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 -- 172.21.15.5:0/2362245440 shutdown_connections 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c5b42d0 0x55c31c5b2a90 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c40a6b0 0x55c31c5ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 --2- 172.21.15.5:0/2362245440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5ae580 0x55c31c5a8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 -- 172.21.15.5:0/2362245440 >> 172.21.15.5:0/2362245440 conn(0x55c31c5a3be0 msgr2=0x55c31c5b7e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 -- 172.21.15.5:0/2362245440 shutdown_connections 2023-07-21T19:33:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.060+0000 7f4c35360b80 1 -- 172.21.15.5:0/2362245440 wait complete. 2023-07-21T19:33:59.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.061+0000 7f4c35360b80 1 -- start start 2023-07-21T19:33:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.061+0000 7f4c35360b80 1 Processor -- start 2023-07-21T19:33:59.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5ae580 0x55c31c5bd190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c61aac0 con 0x55c31c40a6b0 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c61ac00 con 0x55c31c5ae580 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c31c62bd50 con 0x55c31c5b42d0 2023-07-21T19:33:59.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.062+0000 7f4c35360b80 10 client.0 _ll_register_callbacks cb 0x7f4c10009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35144/0 (socket says 172.21.15.5:35144) 2023-07-21T19:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/3137190570 learned_addr learned my addr 172.21.15.5:0/3137190570 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:33:59.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1a7fc700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5ae580 0x55c31c5bd190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5ae580 msgr2=0x55c31c5bd190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5ae580 0x55c31c5bd190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/3137190570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 msgr2=0x55c31c5bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.063+0000 7f4c1b7fe700 1 -- 172.21.15.5:0/3137190570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1401e040 con 0x55c31c5b42d0 2023-07-21T19:33:59.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.064+0000 7f4c1b7fe700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007c20 tx=0x7f4c14022bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a007096ddbde737d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.064+0000 7f4bfaffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:33:59.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.064+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400e040 con 0x55c31c5b42d0 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.064+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(129..129 src has 1..129) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4c14004d60 con 0x55c31c5b42d0 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.064+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4c0c09e020 con 0x55c31c5b42d0 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.065+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c14022420 con 0x55c31c5b42d0 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.065+0000 7f4c35360b80 10 client.29887 fetch_fsmap learned FSMap version 70 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.065+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x55c31c316c00 con 0x55c31c5b42d0 2023-07-21T19:33:59.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4c0c09e020 con 0x55c31c5b42d0 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f4c14005840 con 0x55c31c5b42d0 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 10 client.29887 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 10 client.29887 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 20 client.29887 populate_metadata read hostname 'smithi005' 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c62d300 0x55c31c62f710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 4 client.29887 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.066+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c31c4baff0 con 0x55c31c62d300 2023-07-21T19:33:59.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.068+0000 7f4c1affd700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c62d300 0x55c31c62f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:33:59.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.069+0000 7f4c1affd700 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c62d300 0x55c31c62f710 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d2cd25b9e9aa4a5 server_cookie=416c2704c13bde81 in_seq=0 out_seq=0 2023-07-21T19:33:59.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.069+0000 7f4bfaffd700 10 client.29887 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:59.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:58 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:33:59.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:58 smithi101 ceph-mon[128136]: pgmap v835: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 946 op/s 2023-07-21T19:33:59.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:58 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:33:59.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:33:58 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:59.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.101+0000 7f4bfaffd700 1 -- 172.21.15.5:0/3137190570 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c31c4baff0 con 0x55c31c62d300 2023-07-21T19:33:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.101+0000 7f4bfaffd700 10 client.29887 handle_command_reply: tid=0 2023-07-21T19:33:59.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.101+0000 7f4c35360b80 1 client.29887 shutdown 2023-07-21T19:33:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.101+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 msgr2=0x55c31c5bd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.101+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007c20 tx=0x7f4c14022bb0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 shutdown_connections 2023-07-21T19:33:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c31c5b42d0 0x55c31c5bd6d0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c31c62d300 0x55c31c62f710 crc :-1 s=READY pgs=498 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c31c40a6b0 0x55c31c5bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 --2- 172.21.15.5:0/3137190570 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c31c5ae580 0x55c31c5bd190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.090 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4bfaffd700 0 client.29887 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:33:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 >> 172.21.15.5:0/3137190570 conn(0x55c31c5a3be0 msgr2=0x55c31c5b4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 shutdown_connections 2023-07-21T19:33:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.102+0000 7f4c35360b80 1 -- 172.21.15.5:0/3137190570 wait complete. 2023-07-21T19:33:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.103+0000 7f4c35360b80 20 client.29887 trim_cache size 0 max 16384 2023-07-21T19:33:59.091 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:33:59.092 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 msgr2=0x7f4c040a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 0x7f4c040a3770 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f4c14009480 tx=0x7f4c1400b040 comp rx=0 tx=0).stop 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 msgr2=0x7f4c1c108bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c00e9a0 tx=0x7f4c0c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 shutdown_connections 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4c040a12c0 0x7f4c040a3770 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4c1c105b10 0x7f4c1c1a8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4c1c0ffb00 0x7f4c1c108bc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 --2- 172.21.15.5:0/3401604155 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4c1c1004b0 0x7f4c1c109100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 >> 172.21.15.5:0/3401604155 conn(0x7f4c1c0fb600 msgr2=0x7f4c1c0fd350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.104+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 shutdown_connections 2023-07-21T19:33:59.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:33:59.105+0000 7f4c219ff700 1 -- 172.21.15.5:0/3401604155 wait complete. 2023-07-21T19:33:59.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:58 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:33:59.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:58 smithi005 ceph-mon[126460]: pgmap v835: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 946 op/s 2023-07-21T19:33:59.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:58 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:33:59.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:33:58 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:33:59.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:58 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:33:59.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:58 smithi156 ceph-mon[128316]: pgmap v835: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 9.8 MiB/s wr, 946 op/s 2023-07-21T19:33:59.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:58 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:33:59.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:33:58 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:34:01.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:01 smithi005 ceph-mon[126460]: pgmap v836: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 611 op/s 2023-07-21T19:34:01.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:01 smithi156 ceph-mon[128316]: pgmap v836: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 611 op/s 2023-07-21T19:34:01.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 450.0 s, 200.0 tps, lat 506.846 ms stddev 232.984 2023-07-21T19:34:02.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:01 smithi101 ceph-mon[128136]: pgmap v836: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.3 MiB/s wr, 611 op/s 2023-07-21T19:34:03.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:03 smithi005 ceph-mon[126460]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 907 op/s 2023-07-21T19:34:03.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:03 smithi156 ceph-mon[128316]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 907 op/s 2023-07-21T19:34:04.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:03 smithi101 ceph-mon[128136]: pgmap v837: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.0 MiB/s wr, 907 op/s 2023-07-21T19:34:05.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:05 smithi005 ceph-mon[126460]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 665 op/s 2023-07-21T19:34:05.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:05 smithi156 ceph-mon[128316]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 665 op/s 2023-07-21T19:34:06.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:05 smithi101 ceph-mon[128136]: pgmap v838: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.5 MiB/s wr, 665 op/s 2023-07-21T19:34:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 455.0 s, 209.2 tps, lat 450.818 ms stddev 114.604 2023-07-21T19:34:07.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:07 smithi005 ceph-mon[126460]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2023-07-21T19:34:07.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:07 smithi005 ceph-mon[126460]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T19:34:07.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:07 smithi156 ceph-mon[128316]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2023-07-21T19:34:07.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:07 smithi156 ceph-mon[128316]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T19:34:08.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:07 smithi101 ceph-mon[128136]: pgmap v839: 97 pgs: 97 active+clean; 12 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.4 MiB/s wr, 636 op/s 2023-07-21T19:34:08.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:07 smithi101 ceph-mon[128136]: osdmap e130: 12 total, 12 up, 12 in 2023-07-21T19:34:09.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:08 smithi005 ceph-mon[126460]: pgmap v841: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:09.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:08 smithi156 ceph-mon[128316]: pgmap v841: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:09.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:08 smithi101 ceph-mon[128136]: pgmap v841: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:11.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:11 smithi005 ceph-mon[126460]: pgmap v842: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:11.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:11 smithi156 ceph-mon[128316]: pgmap v842: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:11.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 460.0 s, 134.4 tps, lat 788.342 ms stddev 544.299 2023-07-21T19:34:12.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:11 smithi101 ceph-mon[128136]: pgmap v842: 97 pgs: 23 active+clean+snaptrim, 41 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.4 MiB/s wr, 674 op/s 2023-07-21T19:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 -- 172.21.15.5:0/928998151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 msgr2=0x7fcc201105d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 --2- 172.21.15.5:0/928998151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc201105d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c009990 tx=0x7fcc0c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 -- 172.21.15.5:0/928998151 shutdown_connections 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 --2- 172.21.15.5:0/928998151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 0x7fcc20112ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 --2- 172.21.15.5:0/928998151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc20072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 --2- 172.21.15.5:0/928998151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc201105d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 -- 172.21.15.5:0/928998151 >> 172.21.15.5:0/928998151 conn(0x7fcc200fb600 msgr2=0x7fcc200fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.345+0000 7fcc271ff700 1 -- 172.21.15.5:0/928998151 shutdown_connections 2023-07-21T19:34:12.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.346+0000 7fcc271ff700 1 -- 172.21.15.5:0/928998151 wait complete. 2023-07-21T19:34:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.347+0000 7fcc271ff700 1 Processor -- start 2023-07-21T19:34:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.347+0000 7fcc271ff700 1 -- start start 2023-07-21T19:34:12.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc2019c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 0x7fcc201a0db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a13f0 con 0x7fcc200731c0 2023-07-21T19:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a1560 con 0x7fcc20110b10 2023-07-21T19:34:12.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc271ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc201a1720 con 0x7fcc200728b0 2023-07-21T19:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc24f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc24f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59414/0 (socket says 172.21.15.5:59414) 2023-07-21T19:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc24f9b700 1 -- 172.21.15.5:0/1489783476 learned_addr learned my addr 172.21.15.5:0/1489783476 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:12.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc2579c700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 0x7fcc201a0db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.348+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc2019c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 -- 172.21.15.5:0/1489783476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 msgr2=0x7fcc201a0db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 0x7fcc201a0db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 -- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 msgr2=0x7fcc2019c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc2019c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 -- 172.21.15.5:0/1489783476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c0095f0 con 0x7fcc200728b0 2023-07-21T19:34:12.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc24f9b700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400d950 tx=0x7fcc1400dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc1dffb700 1 -- 172.21.15.5:0/1489783476 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400f890 con 0x7fcc200728b0 2023-07-21T19:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.349+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc201a1a00 con 0x7fcc200728b0 2023-07-21T19:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.350+0000 7fcc1dffb700 1 -- 172.21.15.5:0/1489783476 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc1400fe80 con 0x7fcc200728b0 2023-07-21T19:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.350+0000 7fcc1dffb700 1 -- 172.21.15.5:0/1489783476 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc1400e3b0 con 0x7fcc200728b0 2023-07-21T19:34:12.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.350+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc201a9240 con 0x7fcc200728b0 2023-07-21T19:34:12.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.358+0000 7fcc3aae7b80 1 Processor -- start 2023-07-21T19:34:12.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 -- start start 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b319d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68a36c50 con 0x563d68b31580 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68a899e0 con 0x563d68b372d0 2023-07-21T19:34:12.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.359+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68a82e70 con 0x563d6898d6b0 2023-07-21T19:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.360+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.360+0000 7fcc1dffb700 1 -- 172.21.15.5:0/1489783476 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcc14010460 con 0x7fcc200728b0 2023-07-21T19:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.360+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59418/0 (socket says 172.21.15.5:59418) 2023-07-21T19:34:12.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.360+0000 7fcc2579c700 1 -- 172.21.15.5:0/3202051634 learned_addr learned my addr 172.21.15.5:0/3202051634 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.360+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b319d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 -- 172.21.15.5:0/3202051634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 msgr2=0x563d68b31040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 -- 172.21.15.5:0/3202051634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 msgr2=0x563d68b319d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b319d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 -- 172.21.15.5:0/3202051634 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d688b0d80 con 0x563d6898d6b0 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.361+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a4b0 tx=0x7fcc18009650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=677662cdfcc9abeb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.362+0000 7fcc0affd700 1 -- 172.21.15.5:0/3202051634 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18010040 con 0x563d6898d6b0 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.362+0000 7fcc0affd700 1 -- 172.21.15.5:0/3202051634 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc1800ae20 con 0x563d6898d6b0 2023-07-21T19:34:12.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.362+0000 7fcc0affd700 1 -- 172.21.15.5:0/3202051634 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc180138c0 con 0x563d6898d6b0 2023-07-21T19:34:12.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.363+0000 7fcc1dffb700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 0x7fcc100a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.363+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3202051634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 msgr2=0x563d68b36d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.363+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fcc1800a4b0 tx=0x7fcc18009650 comp rx=0 tx=0).stop 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.363+0000 7fcc1dffb700 1 -- 172.21.15.5:0/1489783476 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcc140ccac0 con 0x7fcc200728b0 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.363+0000 7fcc24f9b700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 0x7fcc100a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3202051634 shutdown_connections 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b319d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b36d90 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3202051634 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3202051634 >> 172.21.15.5:0/3202051634 conn(0x563d68b26be0 msgr2=0x563d68b2bf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3202051634 shutdown_connections 2023-07-21T19:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.364+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3202051634 wait complete. 2023-07-21T19:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.365+0000 7fcc24f9b700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 0x7fcc100a3870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563d68b2db40 tx=0x7fcc14095ed0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.365+0000 7fcc3aae7b80 1 -- start start 2023-07-21T19:34:12.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 Processor -- start 2023-07-21T19:34:12.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b3a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b372d0 0x563d68b3af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68a22a60 con 0x563d68b372d0 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68a22760 con 0x563d68b31580 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.366+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d689068c0 con 0x563d6898d6b0 2023-07-21T19:34:12.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc3aae7b80 10 client.0 _ll_register_callbacks cb 0x563d68a20700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59434/0 (socket says 172.21.15.5:59434) 2023-07-21T19:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc2579c700 1 -- 172.21.15.5:0/2849467387 learned_addr learned my addr 172.21.15.5:0/2849467387 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc2579c700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b372d0 0x563d68b3af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.367+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b3a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 -- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 msgr2=0x563d68b3a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b3a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 -- 172.21.15.5:0/2849467387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b372d0 msgr2=0x563d68b3af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b372d0 0x563d68b3af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 -- 172.21.15.5:0/2849467387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc18009c80 con 0x563d6898d6b0 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcc2579c700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc18016030 tx=0x7fcc18013420 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da2d2fdeb09263bf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcbfffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:34:12.357 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.368+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18013660 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.369+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563d689dc490 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.369+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcc180137c0 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.369+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc18013980 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.369+0000 7fcc3aae7b80 10 client.29899 fetch_fsmap learned FSMap version 70 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.369+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563d68899c00 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563d689dc490 con 0x563d6898d6b0 2023-07-21T19:34:12.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc18008030 con 0x563d6898d6b0 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 10 client.29899 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 10 client.29899 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 20 client.29899 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b4ab20 0x563d68b4cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 4 client.29899 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563d68a7be50 con 0x563d68b4ab20 2023-07-21T19:34:12.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.370+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b4ab20 0x563d68b4cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.371+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b4ab20 0x563d68b4cf30 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=78c9e272393f5ab server_cookie=e5f9f0b8e071bcdd in_seq=0 out_seq=0 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.371+0000 7fcbfffff700 10 client.29899 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.372+0000 7fcbfffff700 1 -- 172.21.15.5:0/2849467387 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563d68a7be50 con 0x563d68b4ab20 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.372+0000 7fcbfffff700 10 client.29899 handle_command_reply: tid=0 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.372+0000 7fcc3aae7b80 1 client.29899 shutdown 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 msgr2=0x563d68b3d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fcc18016030 tx=0x7fcc18013420 comp rx=0 tx=0).stop 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 shutdown_connections 2023-07-21T19:34:12.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b4ab20 0x563d68b4cf30 crc :-1 s=READY pgs=499 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b372d0 0x563d68b3af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3d5b0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/2849467387 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b3a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 >> 172.21.15.5:0/2849467387 conn(0x563d68b26be0 msgr2=0x563d68b2b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.373+0000 7fcbfffff700 0 client.29899 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.374+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 shutdown_connections 2023-07-21T19:34:12.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.374+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/2849467387 wait complete. 2023-07-21T19:34:12.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.374+0000 7fcc3aae7b80 20 client.29899 trim_cache size 0 max 16384 2023-07-21T19:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.394+0000 7fcc3aae7b80 1 Processor -- start 2023-07-21T19:34:12.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.394+0000 7fcc3aae7b80 1 -- start start 2023-07-21T19:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 0x563d68b31040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b362e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d68b372d0 0x563d68b41c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b3b830 con 0x563d6898d6b0 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b28d00 con 0x563d68b31580 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b3be10 con 0x563d68b372d0 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 0x563d68b31040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b362e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc2579c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 0x563d68b31040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51974/0 (socket says 172.21.15.5:51974) 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.395+0000 7fcc2579c700 1 -- 172.21.15.5:0/1512461149 learned_addr learned my addr 172.21.15.5:0/1512461149 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:12.382 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc2579c700 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d68b372d0 0x563d68b41c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 -- 172.21.15.5:0/1512461149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d68b372d0 msgr2=0x563d68b41c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d68b372d0 0x563d68b41c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 -- 172.21.15.5:0/1512461149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 msgr2=0x563d68b31040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 0x563d68b31040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 -- 172.21.15.5:0/1512461149 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563d688d71f0 con 0x563d68b31580 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.396+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b362e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006010 tx=0x7fcc0c0098c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ad1d5bee108a1b8f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.397+0000 7fcc0b7fe700 1 -- 172.21.15.5:0/1512461149 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c01d040 con 0x563d68b31580 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.397+0000 7fcc0b7fe700 1 -- 172.21.15.5:0/1512461149 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc0c01e040 con 0x563d68b31580 2023-07-21T19:34:12.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.397+0000 7fcc0b7fe700 1 -- 172.21.15.5:0/1512461149 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0c0274d0 con 0x563d68b31580 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.397+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/1512461149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 msgr2=0x563d68b362e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.397+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b362e0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c006010 tx=0x7fcc0c0098c0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/1512461149 shutdown_connections 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d68b372d0 0x563d68b41c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d6898d6b0 0x563d68b31040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/1512461149 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b31580 0x563d68b362e0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/1512461149 >> 172.21.15.5:0/1512461149 conn(0x563d68b26be0 msgr2=0x563d68b37900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/1512461149 shutdown_connections 2023-07-21T19:34:12.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.398+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/1512461149 wait complete. 2023-07-21T19:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.399+0000 7fcc3aae7b80 1 -- start start 2023-07-21T19:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.399+0000 7fcc3aae7b80 1 Processor -- start 2023-07-21T19:34:12.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.399+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.399+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc3aae7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b97a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b97fd0 con 0x563d68b31580 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b98110 con 0x563d68b372d0 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc3aae7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563d68b982d0 con 0x563d6898d6b0 2023-07-21T19:34:12.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc3aae7b80 10 client.0 _ll_register_callbacks cb 0x7fcc18009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc1ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51982/0 (socket says 172.21.15.5:51982) 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc1ffff700 1 -- 172.21.15.5:0/3690005846 learned_addr learned my addr 172.21.15.5:0/3690005846 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.400+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b97a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 -- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 msgr2=0x563d68b97a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b97a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 -- 172.21.15.5:0/3690005846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 msgr2=0x563d68b40380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 -- 172.21.15.5:0/3690005846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc201137b0 con 0x563d6898d6b0 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc2579c700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3fe40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x563d68b368f0 tx=0x7fcc18013380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d6c6e679dd727a10 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcbfffff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:34:12.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc18010040 con 0x563d6898d6b0 2023-07-21T19:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcc1800ae20 con 0x563d6898d6b0 2023-07-21T19:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.401+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(130..130 src has 1..130) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fcc1801ce50 con 0x563d6898d6b0 2023-07-21T19:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcc18003aa0 con 0x563d6898d6b0 2023-07-21T19:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 10 client.29905 fetch_fsmap learned FSMap version 70 2023-07-21T19:34:12.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x563d688af250 con 0x563d6898d6b0 2023-07-21T19:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcc1800ae20 con 0x563d6898d6b0 2023-07-21T19:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fcc1800a4e0 con 0x563d6898d6b0 2023-07-21T19:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 10 client.29905 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 10 client.29905 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:12.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 20 client.29905 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b9f750 0x563d68ba1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 4 client.29905 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.402+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563d68a3dff0 con 0x563d68b9f750 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.403+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b9f750 0x563d68ba1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.403+0000 7fcc1ffff700 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b9f750 0x563d68ba1b30 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84cf9a6da00affed server_cookie=a4342f7cab65251c in_seq=0 out_seq=0 2023-07-21T19:34:12.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.403+0000 7fcbfffff700 10 client.29905 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:12.425 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcbfffff700 1 -- 172.21.15.5:0/3690005846 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563d68a3dff0 con 0x563d68b9f750 2023-07-21T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcbfffff700 10 client.29905 handle_command_reply: tid=0 2023-07-21T19:34:12.426 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcc3aae7b80 1 client.29905 shutdown 2023-07-21T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 msgr2=0x563d68b3fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3fe40 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x563d68b368f0 tx=0x7fcc18013380 comp rx=0 tx=0).stop 2023-07-21T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 shutdown_connections 2023-07-21T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.440+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563d68b372d0 0x563d68b97a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.427 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563d6898d6b0 0x563d68b3fe40 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563d68b9f750 0x563d68ba1b30 crc :-1 s=READY pgs=500 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 --2- 172.21.15.5:0/3690005846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563d68b31580 0x563d68b40380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 >> 172.21.15.5:0/3690005846 conn(0x563d68b26be0 msgr2=0x563d68b308c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcbfffff700 0 client.29905 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 shutdown_connections 2023-07-21T19:34:12.428 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.441+0000 7fcc3aae7b80 1 -- 172.21.15.5:0/3690005846 wait complete. 2023-07-21T19:34:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.442+0000 7fcc3aae7b80 20 client.29905 trim_cache size 0 max 16384 2023-07-21T19:34:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:34:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:34:12.429 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 msgr2=0x7fcc100a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.430 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 0x7fcc100a3870 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x563d68b2db40 tx=0x7fcc14095ed0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 msgr2=0x7fcc2019c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fcc1400d950 tx=0x7fcc1400dc60 comp rx=0 tx=0).stop 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 shutdown_connections 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcc20110b10 0x7fcc201a0db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcc200728b0 0x7fcc2019c200 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.431 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcc100a13c0 0x7fcc100a3870 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 --2- 172.21.15.5:0/1489783476 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcc200731c0 0x7fcc2019c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 >> 172.21.15.5:0/1489783476 conn(0x7fcc200fb600 msgr2=0x7fcc200fc260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 shutdown_connections 2023-07-21T19:34:12.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:12.443+0000 7fcc271ff700 1 -- 172.21.15.5:0/1489783476 wait complete. 2023-07-21T19:34:13.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:13 smithi005 ceph-mon[126460]: pgmap v843: 97 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 79 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 499 op/s 2023-07-21T19:34:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:13 smithi156 ceph-mon[128316]: pgmap v843: 97 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 79 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 499 op/s 2023-07-21T19:34:14.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:13 smithi101 ceph-mon[128136]: pgmap v843: 97 pgs: 8 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 79 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 499 op/s 2023-07-21T19:34:15.898 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:15 smithi005 ceph-mon[126460]: pgmap v844: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:15.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:15 smithi156 ceph-mon[128316]: pgmap v844: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:16.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:15 smithi101 ceph-mon[128136]: pgmap v844: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:16.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 465.0 s, 255.1 tps, lat 392.465 ms stddev 81.294 2023-07-21T19:34:17.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:17 smithi005 ceph-mon[126460]: pgmap v845: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:17.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:17 smithi156 ceph-mon[128316]: pgmap v845: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:18.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:17 smithi101 ceph-mon[128136]: pgmap v845: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.9 MiB/s wr, 541 op/s 2023-07-21T19:34:19.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:18 smithi101 ceph-mon[128136]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T19:34:19.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:18 smithi101 ceph-mon[128136]: pgmap v847: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:18 smithi005 ceph-mon[126460]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T19:34:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:18 smithi005 ceph-mon[126460]: pgmap v847: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:18 smithi156 ceph-mon[128316]: osdmap e131: 12 total, 12 up, 12 in 2023-07-21T19:34:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:18 smithi156 ceph-mon[128316]: pgmap v847: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:21.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:21 smithi005 ceph-mon[126460]: pgmap v848: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:21.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:21 smithi156 ceph-mon[128316]: pgmap v848: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:21.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 470.0 s, 255.0 tps, lat 372.066 ms stddev 102.033 2023-07-21T19:34:22.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:21 smithi101 ceph-mon[128136]: pgmap v848: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.6 MiB/s wr, 691 op/s 2023-07-21T19:34:23.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:23 smithi005 ceph-mon[126460]: pgmap v849: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 790 op/s 2023-07-21T19:34:23.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:23 smithi156 ceph-mon[128316]: pgmap v849: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 790 op/s 2023-07-21T19:34:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:23 smithi101 ceph-mon[128136]: pgmap v849: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.2 MiB/s wr, 790 op/s 2023-07-21T19:34:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 -- 172.21.15.5:0/3378572125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc1004e0 msgr2=0x7f50fc10c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 --2- 172.21.15.5:0/3378572125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc1004e0 0x7f50fc10c920 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f50f400b0a0 tx=0x7f50f400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 -- 172.21.15.5:0/3378572125 shutdown_connections 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 --2- 172.21.15.5:0/3378572125 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc1004e0 0x7f50fc10c920 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 --2- 172.21.15.5:0/3378572125 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc105b60 0x7f50fc105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 --2- 172.21.15.5:0/3378572125 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc0ffb30 0x7f50fc0fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 -- 172.21.15.5:0/3378572125 >> 172.21.15.5:0/3378572125 conn(0x7f50fc0fb5f0 msgr2=0x7f50fc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.603+0000 7f51047ff700 1 -- 172.21.15.5:0/3378572125 shutdown_connections 2023-07-21T19:34:25.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.604+0000 7f51047ff700 1 -- 172.21.15.5:0/3378572125 wait complete. 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 Processor -- start 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 -- start start 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc0ffb30 0x7f50fc0729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 0x7f50fc1a3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1a4330 con 0x7f50fc105b60 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1a4470 con 0x7f50fc1004e0 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.605+0000 7f51047ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc1a45e0 con 0x7f50fc0ffb30 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5102d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 0x7f50fc1a3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59766/0 (socket says 172.21.15.5:59766) 2023-07-21T19:34:25.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 -- 172.21.15.5:0/2152893612 learned_addr learned my addr 172.21.15.5:0/2152893612 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f510259b700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc0ffb30 0x7f50fc0729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 -- 172.21.15.5:0/2152893612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc0ffb30 msgr2=0x7f50fc0729f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc0ffb30 0x7f50fc0729f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 -- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 msgr2=0x7f50fc1a3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 0x7f50fc1a3df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 -- 172.21.15.5:0/2152893612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f4009d00 con 0x7f50fc1004e0 2023-07-21T19:34:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5102d9c700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 0x7f50fc1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:34:25.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.606+0000 7f5101d9a700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e9a0 tx=0x7f50ec00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.607+0000 7f50f37fe700 1 -- 172.21.15.5:0/2152893612 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec009790 con 0x7f50fc1004e0 2023-07-21T19:34:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.607+0000 7f50f37fe700 1 -- 172.21.15.5:0/2152893612 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50ec010430 con 0x7f50fc1004e0 2023-07-21T19:34:25.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.607+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc1a48c0 con 0x7f50fc1004e0 2023-07-21T19:34:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.607+0000 7f50f37fe700 1 -- 172.21.15.5:0/2152893612 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec00cea0 con 0x7f50fc1004e0 2023-07-21T19:34:25.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.607+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc1a4e20 con 0x7f50fc1004e0 2023-07-21T19:34:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.612+0000 7f50f37fe700 1 -- 172.21.15.5:0/2152893612 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f50ec0105a0 con 0x7f50fc1004e0 2023-07-21T19:34:25.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.613+0000 7f5118047b80 1 Processor -- start 2023-07-21T19:34:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 -- start start 2023-07-21T19:34:25.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787da4ae0 0x556787d9e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787c9bc50 con 0x556787d9ed90 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787cee9e0 con 0x556787bf26b0 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f5118047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787ce7e70 con 0x556787da4ae0 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.614+0000 7f50f37fe700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 0x7f50e80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5102d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5102d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59778/0 (socket says 172.21.15.5:59778) 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5102d9c700 1 -- 172.21.15.5:0/1813613005 learned_addr learned my addr 172.21.15.5:0/1813613005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60434/0 (socket says 172.21.15.5:60434) 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f50f37fe700 1 -- 172.21.15.5:0/2152893612 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50ec014070 con 0x7f50fc1004e0 2023-07-21T19:34:25.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5102d9c700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 0x7f50e80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 -- 172.21.15.5:0/1813613005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787da4ae0 msgr2=0x556787d9e850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787da4ae0 0x556787d9e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 -- 172.21.15.5:0/1813613005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 msgr2=0x556787da45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5101d9a700 1 -- 172.21.15.5:0/1813613005 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556787b15d80 con 0x556787d9ed90 2023-07-21T19:34:25.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.615+0000 7f5102d9c700 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:34:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.616+0000 7f5101d9a700 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0ca600 tx=0x7f50ec007ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4ce60b7bf299860 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.616+0000 7f5102d9c700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 0x7f50e80a3870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006130 tx=0x7f50f400b720 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.616+0000 7f50f0ff9700 1 -- 172.21.15.5:0/1813613005 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec099d50 con 0x556787d9ed90 2023-07-21T19:34:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.616+0000 7f50f0ff9700 1 -- 172.21.15.5:0/1813613005 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50ec099ac0 con 0x556787d9ed90 2023-07-21T19:34:25.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.616+0000 7f50f0ff9700 1 -- 172.21.15.5:0/1813613005 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec0a23d0 con 0x556787d9ed90 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 -- 172.21.15.5:0/1813613005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 msgr2=0x556787d9f1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f50ec0ca600 tx=0x7f50ec007ef0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 -- 172.21.15.5:0/1813613005 shutdown_connections 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787d9f1e0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787bf26b0 0x556787da45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 --2- 172.21.15.5:0/1813613005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787da4ae0 0x556787d9e850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 -- 172.21.15.5:0/1813613005 >> 172.21.15.5:0/1813613005 conn(0x556787d8bbe0 msgr2=0x556787d90f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 -- 172.21.15.5:0/1813613005 shutdown_connections 2023-07-21T19:34:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.617+0000 7f5118047b80 1 -- 172.21.15.5:0/1813613005 wait complete. 2023-07-21T19:34:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.618+0000 7f5118047b80 1 -- start start 2023-07-21T19:34:25.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.618+0000 7f5118047b80 1 Processor -- start 2023-07-21T19:34:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787da8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787da86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787da8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787c87a60 con 0x556787d9ed90 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787c87760 con 0x556787da4ae0 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787b6b8c0 con 0x556787bf26b0 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5118047b80 10 client.0 _ll_register_callbacks cb 0x556787c85700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787da86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f510259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787da8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f510259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787da8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60156/0 (socket says 172.21.15.5:60156) 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.619+0000 7f510259b700 1 -- 172.21.15.5:0/3679308308 learned_addr learned my addr 172.21.15.5:0/3679308308 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:25.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 -- 172.21.15.5:0/3679308308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 msgr2=0x556787da8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787da8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 -- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 msgr2=0x556787da8c00 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787da8c00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 -- 172.21.15.5:0/3679308308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc10d220 con 0x556787d9ed90 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f5101d9a700 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787da86c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x556787d9f830 tx=0x7f50ec09a4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88fb496c2f5f118 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f50e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:34:25.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50ec099d50 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.620+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50ec010880 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556787c41490 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50ec0998e0 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 10 client.20208 fetch_fsmap learned FSMap version 70 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x7f50fc1a4e20 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50ec0b6080 con 0x556787d9ed90 2023-07-21T19:34:25.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50ec0a1f40 con 0x556787d9ed90 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 10 client.20208 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 10 client.20208 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 20 client.20208 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787db81b0 0x556787dba5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 4 client.20208 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.621+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556787ce0e50 con 0x556787db81b0 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.622+0000 7f510259b700 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787db81b0 0x556787dba5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.623+0000 7f510259b700 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787db81b0 0x556787dba5c0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd88c3c305f5582 server_cookie=8188fb5d0a3b904c in_seq=0 out_seq=0 2023-07-21T19:34:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.623+0000 7f50e5ffb700 10 client.20208 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3679308308 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556787ce0e50 con 0x556787db81b0 2023-07-21T19:34:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f50e5ffb700 10 client.20208 handle_command_reply: tid=0 2023-07-21T19:34:25.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 client.20208 shutdown 2023-07-21T19:34:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 msgr2=0x556787da86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787da86c0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x556787d9f830 tx=0x7f50ec09a4c0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 shutdown_connections 2023-07-21T19:34:25.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787da8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787da8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787db81b0 0x556787dba5c0 crc :-1 s=READY pgs=501 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 --2- 172.21.15.5:0/3679308308 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787d9ed90 0x556787da86c0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f50e5ffb700 0 client.20208 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.624+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 >> 172.21.15.5:0/3679308308 conn(0x556787d8bbe0 msgr2=0x556787d904e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.625+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 shutdown_connections 2023-07-21T19:34:25.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.625+0000 7f5118047b80 1 -- 172.21.15.5:0/3679308308 wait complete. 2023-07-21T19:34:25.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.625+0000 7f5118047b80 20 client.20208 trim_cache size 0 max 16384 2023-07-21T19:34:25.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 Processor -- start 2023-07-21T19:34:25.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 -- start start 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 0x556787d90c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 0x556787da3300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787da2f30 con 0x556787da4ae0 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787da6120 con 0x556787d9ed90 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.652+0000 7f5118047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787d88b40 con 0x556787bf26b0 2023-07-21T19:34:25.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f5102d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 0x556787da3300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 0x556787d90c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f510259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f5102d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 0x556787da3300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:60454/0 (socket says 172.21.15.5:60454) 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 0x556787d90c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59792/0 (socket says 172.21.15.5:59792) 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f5102d9c700 1 -- 172.21.15.5:0/498141686 learned_addr learned my addr 172.21.15.5:0/498141686 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:25.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f510259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60172/0 (socket says 172.21.15.5:60172) 2023-07-21T19:34:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f510259b700 1 -- 172.21.15.5:0/498141686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 msgr2=0x556787d90c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.653+0000 7f510259b700 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 0x556787d90c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.654+0000 7f510259b700 1 -- 172.21.15.5:0/498141686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 msgr2=0x556787da3300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.654+0000 7f510259b700 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 0x556787da3300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.654+0000 7f510259b700 1 -- 172.21.15.5:0/498141686 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556787d9f450 con 0x556787bf26b0 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.654+0000 7f510259b700 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x556787db8580 tx=0x7f50f8004320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1bebe417c8146f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.654+0000 7f50f17fa700 1 -- 172.21.15.5:0/498141686 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f801f070 con 0x556787bf26b0 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.655+0000 7f50f17fa700 1 -- 172.21.15.5:0/498141686 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50f801bd70 con 0x556787bf26b0 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.655+0000 7f50f17fa700 1 -- 172.21.15.5:0/498141686 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f8017040 con 0x556787bf26b0 2023-07-21T19:34:25.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.655+0000 7f5118047b80 1 -- 172.21.15.5:0/498141686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 msgr2=0x556787d9e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.655+0000 7f5118047b80 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x556787db8580 tx=0x7f50f8004320 comp rx=0 tx=0).stop 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.656+0000 7f5118047b80 1 -- 172.21.15.5:0/498141686 shutdown_connections 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.656+0000 7f5118047b80 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787da4ae0 0x556787da3300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.656+0000 7f5118047b80 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787bf26b0 0x556787d9e850 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.656+0000 7f5118047b80 1 --2- 172.21.15.5:0/498141686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787d9ed90 0x556787d90c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.656+0000 7f5118047b80 1 -- 172.21.15.5:0/498141686 >> 172.21.15.5:0/498141686 conn(0x556787d8bbe0 msgr2=0x556787da2380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.657+0000 7f5118047b80 1 -- 172.21.15.5:0/498141686 shutdown_connections 2023-07-21T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.657+0000 7f5118047b80 1 -- 172.21.15.5:0/498141686 wait complete. 2023-07-21T19:34:25.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.657+0000 7f5118047b80 1 -- start start 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 Processor -- start 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787bf26b0 0x556787e05320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 0x556787e05860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787e040b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787e04620 con 0x556787bf26b0 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787e04790 con 0x556787da4ae0 2023-07-21T19:34:25.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x556787e04950 con 0x556787d9ed90 2023-07-21T19:34:25.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.658+0000 7f5118047b80 10 client.0 _ll_register_callbacks cb 0x556787c85700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.659+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 0x556787e05860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.659+0000 7f5101d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 0x556787e05860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60186/0 (socket says 172.21.15.5:60186) 2023-07-21T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.659+0000 7f5101d9a700 1 -- 172.21.15.5:0/3945246425 learned_addr learned my addr 172.21.15.5:0/3945246425 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.659+0000 7f5102d9c700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787e040b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.659+0000 7f510259b700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787bf26b0 0x556787e05320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 -- 172.21.15.5:0/3945246425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 msgr2=0x556787e05860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 0x556787e05860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 -- 172.21.15.5:0/3945246425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787bf26b0 msgr2=0x556787e05320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787bf26b0 0x556787e05320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 -- 172.21.15.5:0/3945246425 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50f8009e20 con 0x556787da4ae0 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f5102d9c700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787e040b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007550 tx=0x7f50f40099e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12420d9c940f591e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.660+0000 7f50e5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:34:25.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.661+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f400e040 con 0x556787da4ae0 2023-07-21T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.661+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f50f4020930 con 0x556787da4ae0 2023-07-21T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.661+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f50ec099ac0 con 0x556787da4ae0 2023-07-21T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.661+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f50f4020af0 con 0x556787da4ae0 2023-07-21T19:34:25.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 10 client.29996 fetch_fsmap learned FSMap version 70 2023-07-21T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=4+}) v3 -- 0x556787afec00 con 0x556787da4ae0 2023-07-21T19:34:25.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f50ec099ac0 con 0x556787da4ae0 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 70) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f50f401fc10 con 0x556787da4ae0 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 10 client.29996 fetch_fsmap finished waiting for FSMap version 70 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 10 client.29996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 20 client.29996 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787e0d310 0x556787e0f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 4 client.29996 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:34:25.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.662+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556787ca2ff0 con 0x556787e0d310 2023-07-21T19:34:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.664+0000 7f510259b700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787e0d310 0x556787e0f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.665+0000 7f510259b700 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787e0d310 0x556787e0f720 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=819813a6c806321a server_cookie=f8fe2c158a0b9041 in_seq=0 out_seq=0 2023-07-21T19:34:25.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.665+0000 7f50e5ffb700 10 client.29996 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:25.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f50e5ffb700 1 -- 172.21.15.5:0/3945246425 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556787ca2ff0 con 0x556787e0d310 2023-07-21T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f50e5ffb700 10 client.29996 handle_command_reply: tid=0 2023-07-21T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 client.29996 shutdown 2023-07-21T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 msgr2=0x556787e040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787e040b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f50f4007550 tx=0x7f50f40099e0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 shutdown_connections 2023-07-21T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x556787da4ae0 0x556787e040b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x556787e0d310 0x556787e0f720 crc :-1 s=READY pgs=502 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x556787bf26b0 0x556787e05320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 --2- 172.21.15.5:0/3945246425 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x556787d9ed90 0x556787e05860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f50e5ffb700 0 client.29996 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.690+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 >> 172.21.15.5:0/3945246425 conn(0x556787d8bbe0 msgr2=0x556787da9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.691+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 shutdown_connections 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.691+0000 7f5118047b80 1 -- 172.21.15.5:0/3945246425 wait complete. 2023-07-21T19:34:25.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.691+0000 7f5118047b80 20 client.29996 trim_cache size 0 max 16384 2023-07-21T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:34:25.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:34:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 msgr2=0x7f50e80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 0x7f50e80a3870 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f50f4006130 tx=0x7f50f400b720 comp rx=0 tx=0).stop 2023-07-21T19:34:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 msgr2=0x7f50fc072f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f50ec00e9a0 tx=0x7f50ec00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 shutdown_connections 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f50e80a13c0 0x7f50e80a3870 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f50fc105b60 0x7f50fc1a3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f50fc0ffb30 0x7f50fc0729f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 --2- 172.21.15.5:0/2152893612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f50fc1004e0 0x7f50fc072f30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 >> 172.21.15.5:0/2152893612 conn(0x7f50fc0fb5f0 msgr2=0x7f50fc103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 shutdown_connections 2023-07-21T19:34:25.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:25.692+0000 7f51047ff700 1 -- 172.21.15.5:0/2152893612 wait complete. 2023-07-21T19:34:25.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:25 smithi005 ceph-mon[126460]: pgmap v850: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:25.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:25 smithi156 ceph-mon[128316]: pgmap v850: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:26.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:25 smithi101 ceph-mon[128136]: pgmap v850: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:26.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 475.0 s, 233.4 tps, lat 437.796 ms stddev 123.737 2023-07-21T19:34:27.013 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:34:27.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/489337060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c040fe9d0 msgr2=0x7f8c040fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/489337060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c040fe9d0 0x7f8c040fee40 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f8bf80098f0 tx=0x7f8bf8009c00 comp rx=0 tx=0).stop 2023-07-21T19:34:27.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/489337060 shutdown_connections 2023-07-21T19:34:27.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/489337060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c040ff380 0x7f8c0410b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/489337060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c041049e0 0x7f8c04104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/489337060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c040fe9d0 0x7f8c040fee40 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/489337060 >> 172.21.15.5:0/489337060 conn(0x7f8c040fa4b0 msgr2=0x7f8c040fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/489337060 shutdown_connections 2023-07-21T19:34:27.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.365+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/489337060 wait complete. 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.367+0000 7f8c0a1ff700 1 Processor -- start 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.367+0000 7f8c0a1ff700 1 -- start start 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.367+0000 7f8c0a1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c040fe9d0 0x7f8c0406e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c0a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c040ff380 0x7f8c0406eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c0a1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c0a1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0406aee0 con 0x7f8c040fe9d0 2023-07-21T19:34:27.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c0a1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0406b050 con 0x7f8c041049e0 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c0a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c0406b210 con 0x7f8c040ff380 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.368+0000 7f8c03fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57706/0 (socket says 172.21.15.5:57706) 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 -- 172.21.15.5:0/274158479 learned_addr learned my addr 172.21.15.5:0/274158479 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c040ff380 0x7f8c0406eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 -- 172.21.15.5:0/274158479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c040ff380 msgr2=0x7f8c0406eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c040ff380 0x7f8c0406eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 -- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c040fe9d0 msgr2=0x7f8c0406e630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:34:27.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c040fe9d0 0x7f8c0406e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 -- 172.21.15.5:0/274158479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf400b730 con 0x7f8c041049e0 2023-07-21T19:34:27.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.369+0000 7f8c03fff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005c50 tx=0x7f8bf4009bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.360 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.371+0000 7f8c00ff9700 1 -- 172.21.15.5:0/274158479 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4018070 con 0x7f8c041049e0 2023-07-21T19:34:27.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.371+0000 7f8c00ff9700 1 -- 172.21.15.5:0/274158479 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bf40044b0 con 0x7f8c041049e0 2023-07-21T19:34:27.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.372+0000 7f8c00ff9700 1 -- 172.21.15.5:0/274158479 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf4010040 con 0x7f8c041049e0 2023-07-21T19:34:27.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.372+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf8009520 con 0x7f8c041049e0 2023-07-21T19:34:27.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.372+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c0406f4f0 con 0x7f8c041049e0 2023-07-21T19:34:27.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.378+0000 7f8c00ff9700 1 -- 172.21.15.5:0/274158479 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8bf4003ee0 con 0x7f8c041049e0 2023-07-21T19:34:27.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.379+0000 7f8c1db27b80 1 Processor -- start 2023-07-21T19:34:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 -- start start 2023-07-21T19:34:27.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf406a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf4deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be46cb0 con 0x56296bf4bad0 2023-07-21T19:34:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be99a40 con 0x56296bf402d0 2023-07-21T19:34:27.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.381+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be92ed0 con 0x56296bf40be0 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58476/0 (socket says 172.21.15.5:58476) 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c037fe700 1 -- 172.21.15.5:0/4278868296 learned_addr learned my addr 172.21.15.5:0/4278868296 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf406a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf4deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c00ff9700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 0x7f8bec0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 -- 172.21.15.5:0/4278868296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 msgr2=0x56296bf4b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 -- 172.21.15.5:0/4278868296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 msgr2=0x56296bf4deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf4deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c02ffd700 1 -- 172.21.15.5:0/4278868296 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bcc0de0 con 0x56296bf402d0 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.382+0000 7f8c037fe700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:34:27.370 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8c00ff9700 1 -- 172.21.15.5:0/274158479 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bf40cc380 con 0x7f8c041049e0 2023-07-21T19:34:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf406a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009810 tx=0x7f8bf8019040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1bcda890c6b54b16 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8c03fff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 0x7f8bec0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8be9ffb700 1 -- 172.21.15.5:0/4278868296 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8010de0 con 0x56296bf402d0 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8be9ffb700 1 -- 172.21.15.5:0/4278868296 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bf800f8f0 con 0x56296bf402d0 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.383+0000 7f8be9ffb700 1 -- 172.21.15.5:0/4278868296 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf8005450 con 0x56296bf402d0 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.384+0000 7f8c1db27b80 1 -- 172.21.15.5:0/4278868296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 msgr2=0x56296bf406a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.384+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf406a0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009810 tx=0x7f8bf8019040 comp rx=0 tx=0).stop 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.384+0000 7f8c03fff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 0x7f8bec0a3980 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c9ad0 tx=0x7f8bf40cc590 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 -- 172.21.15.5:0/4278868296 shutdown_connections 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf4deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf406a0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/4278868296 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf40be0 0x56296bf4b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 -- 172.21.15.5:0/4278868296 >> 172.21.15.5:0/4278868296 conn(0x56296bf36da0 msgr2=0x56296bf3c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 -- 172.21.15.5:0/4278868296 shutdown_connections 2023-07-21T19:34:27.373 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 -- 172.21.15.5:0/4278868296 wait complete. 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.385+0000 7f8c1db27b80 1 -- start start 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.386+0000 7f8c1db27b80 1 Processor -- start 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 0x56296bf4b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf48dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be32ac0 con 0x56296bf402d0 2023-07-21T19:34:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296be327c0 con 0x56296bf40be0 2023-07-21T19:34:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bd16920 con 0x56296bf4bad0 2023-07-21T19:34:27.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c1db27b80 10 client.0 _ll_register_callbacks cb 0x56296be30760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 0x56296bf4b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57722/0 (socket says 172.21.15.5:57722) 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 -- 172.21.15.5:0/204253553 learned_addr learned my addr 172.21.15.5:0/204253553 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.387+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 0x56296bf4b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49242/0 (socket says 172.21.15.5:49242) 2023-07-21T19:34:27.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf48dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 -- 172.21.15.5:0/204253553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 msgr2=0x56296bf48dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf48dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 -- 172.21.15.5:0/204253553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 msgr2=0x56296bf4b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 0x56296bf4b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 -- 172.21.15.5:0/204253553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bfc00e0c0 con 0x56296bf40be0 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8c037fe700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007a50 tx=0x7f8bfc007e10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6957d64d660f4f34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.377 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.388+0000 7f8be2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc010040 con 0x56296bf40be0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56296be8beb0 con 0x56296bf40be0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bfc00ead0 con 0x56296bf40be0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc00ecb0 con 0x56296bf40be0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8c1db27b80 10 client.30008 fetch_fsmap learned FSMap version 71 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.389+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x7f8bf801f040 con 0x56296bf40be0 2023-07-21T19:34:27.378 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56296be8beb0 con 0x56296bf40be0 2023-07-21T19:34:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 71) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8bfc015390 con 0x56296bf40be0 2023-07-21T19:34:27.379 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 10 client.30008 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 10 client.30008 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 20 client.30008 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bf5b320 0x56296bf5d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 4 client.30008 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56296be4e050 con 0x56296bf5b320 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.390+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bf5b320 0x56296bf5d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.391+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bf5b320 0x56296bf5d730 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=239a49543e1fd619 server_cookie=f886738fa403e8b0 in_seq=0 out_seq=0 2023-07-21T19:34:27.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.391+0000 7f8be2ffd700 10 client.30008 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.392+0000 7f8be2ffd700 1 -- 172.21.15.5:0/204253553 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7f8bf8033780 con 0x56296bf5b320 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.392+0000 7f8be2ffd700 10 client.30008 handle_command_reply: tid=0 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.392+0000 7f8c1db27b80 1 client.30008 shutdown 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.392+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 msgr2=0x56296bf48750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.392+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc007a50 tx=0x7f8bfc007e10 comp rx=0 tx=0).stop 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 shutdown_connections 2023-07-21T19:34:27.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf48dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bf5b320 0x56296bf5d730 crc :-1 s=READY pgs=503 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf402d0 0x56296bf4b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/204253553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf48750 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 >> 172.21.15.5:0/204253553 conn(0x56296bf36da0 msgr2=0x56296bf3c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8be2ffd700 0 client.30008 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 shutdown_connections 2023-07-21T19:34:27.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 1 -- 172.21.15.5:0/204253553 wait complete. 2023-07-21T19:34:27.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.393+0000 7f8c1db27b80 20 client.30008 trim_cache size 0 max 16384 2023-07-21T19:34:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 Processor -- start 2023-07-21T19:34:27.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 -- start start 2023-07-21T19:34:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf524b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf43a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf58a80 con 0x56296bf40be0 2023-07-21T19:34:27.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf46cb0 con 0x56296bf402d0 2023-07-21T19:34:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.422+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf529f0 con 0x56296bf4bad0 2023-07-21T19:34:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.423+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.423+0000 7f8c037fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49256/0 (socket says 172.21.15.5:49256) 2023-07-21T19:34:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.423+0000 7f8c037fe700 1 -- 172.21.15.5:0/457058947 learned_addr learned my addr 172.21.15.5:0/457058947 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:27.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.423+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.423+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf43a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 -- 172.21.15.5:0/457058947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 msgr2=0x56296bf43a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.411 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf43a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 -- 172.21.15.5:0/457058947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 msgr2=0x56296bf524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 -- 172.21.15.5:0/457058947 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56296bca9c60 con 0x56296bf40be0 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.424+0000 7f8c037fe700 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc002230 tx=0x7f8bfc005a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3610698c86c222de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.425+0000 7f8bea7fc700 1 -- 172.21.15.5:0/457058947 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc02d070 con 0x56296bf40be0 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.425+0000 7f8bea7fc700 1 -- 172.21.15.5:0/457058947 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bfc009640 con 0x56296bf40be0 2023-07-21T19:34:27.412 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.425+0000 7f8bea7fc700 1 -- 172.21.15.5:0/457058947 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc010070 con 0x56296bf40be0 2023-07-21T19:34:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 -- 172.21.15.5:0/457058947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 msgr2=0x56296bf421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f8bfc002230 tx=0x7f8bfc005a80 comp rx=0 tx=0).stop 2023-07-21T19:34:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 -- 172.21.15.5:0/457058947 shutdown_connections 2023-07-21T19:34:27.413 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf4bad0 0x56296bf43a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf402d0 0x56296bf524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/457058947 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf40be0 0x56296bf421a0 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 -- 172.21.15.5:0/457058947 >> 172.21.15.5:0/457058947 conn(0x56296bf36da0 msgr2=0x56296bf57660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 -- 172.21.15.5:0/457058947 shutdown_connections 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.426+0000 7f8c1db27b80 1 -- 172.21.15.5:0/457058947 wait complete. 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 -- start start 2023-07-21T19:34:27.414 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 Processor -- start 2023-07-21T19:34:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 0x56296bf50840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf50d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.415 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf512c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bf52890 con 0x56296bf4bad0 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bfaf1f0 con 0x56296bf40be0 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56296bfaf360 con 0x56296bf402d0 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.427+0000 7f8c1db27b80 10 client.0 _ll_register_callbacks cb 0x56296be30760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 0x56296bf50840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c02ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 0x56296bf50840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58494/0 (socket says 172.21.15.5:58494) 2023-07-21T19:34:27.416 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c02ffd700 1 -- 172.21.15.5:0/712069304 learned_addr learned my addr 172.21.15.5:0/712069304 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf50d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf512c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 -- 172.21.15.5:0/712069304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 msgr2=0x56296bf50840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 0x56296bf50840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 -- 172.21.15.5:0/712069304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 msgr2=0x56296bf512c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf512c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.417 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.428+0000 7f8c037fe700 1 -- 172.21.15.5:0/712069304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf801f040 con 0x56296bf40be0 2023-07-21T19:34:27.418 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.429+0000 7f8c037fe700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf50d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x56296bf3ddf0 tx=0x7f8bfc015620 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9145f860cc55cf99 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.429+0000 7f8be2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.429+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bfc010070 con 0x56296bf40be0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.429+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8bfc00ede0 con 0x56296bf40be0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.430+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56296bdec4f0 con 0x56296bf40be0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.430+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bfc02d040 con 0x56296bf40be0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.430+0000 7f8c1db27b80 10 client.30014 fetch_fsmap learned FSMap version 71 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.430+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=4+}) v3 -- 0x56296bce7250 con 0x56296bf40be0 2023-07-21T19:34:27.419 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8bfc03e020 con 0x56296bf40be0 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 71) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f8bfc029a10 con 0x56296bf40be0 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 10 client.30014 fetch_fsmap finished waiting for FSMap version 71 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 10 client.30014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 20 client.30014 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bfb06e0 0x56296bfb2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 4 client.30014 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:34:27.420 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56296bce1f70 con 0x56296bfb06e0 2023-07-21T19:34:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.431+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bfb06e0 0x56296bfb2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.432+0000 7f8c02ffd700 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bfb06e0 0x56296bfb2ac0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79cf9d7af39b3bbf server_cookie=2b48623d0d324c54 in_seq=0 out_seq=0 2023-07-21T19:34:27.421 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.432+0000 7f8be2ffd700 10 client.30014 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8be2ffd700 1 -- 172.21.15.5:0/712069304 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56296bce1f70 con 0x56296bfb06e0 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8be2ffd700 10 client.30014 handle_command_reply: tid=0 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 client.30014 shutdown 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 msgr2=0x56296bf50d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf50d80 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x56296bf3ddf0 tx=0x7f8bfc015620 comp rx=0 tx=0).stop 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 shutdown_connections 2023-07-21T19:34:27.422 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56296bfb06e0 0x56296bfb2ac0 crc :-1 s=READY pgs=504 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56296bf4bad0 0x56296bf512c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56296bf402d0 0x56296bf50840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 --2- 172.21.15.5:0/712069304 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56296bf40be0 0x56296bf50d80 secure :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0x56296bf3ddf0 tx=0x7f8bfc015620 comp rx=0 tx=0).stop 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 >> 172.21.15.5:0/712069304 conn(0x56296bf36da0 msgr2=0x56296bf56c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.433+0000 7f8be2ffd700 0 client.30014 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.434+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 shutdown_connections 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.434+0000 7f8c1db27b80 1 -- 172.21.15.5:0/712069304 wait complete. 2023-07-21T19:34:27.423 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.434+0000 7f8c1db27b80 20 client.30014 trim_cache size 0 max 16384 2023-07-21T19:34:27.424 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:34:27.424 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:34:27.424 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:34:27.424 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:34:27.424 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:34:27.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:34:27.425 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:34:27.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:34:27.425 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:34:27.425 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.435+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 msgr2=0x7f8bec0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.435+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 0x7f8bec0a3980 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40c9ad0 tx=0x7f8bf40cc590 comp rx=0 tx=0).stop 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.435+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 msgr2=0x7f8c0406a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.435+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f8bf4005c50 tx=0x7f8bf4009bf0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 shutdown_connections 2023-07-21T19:34:27.426 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8c041049e0 0x7f8c0406a8e0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8bec0a14d0 0x7f8bec0a3980 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8c040fe9d0 0x7f8c0406e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 --2- 172.21.15.5:0/274158479 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8c040ff380 0x7f8c0406eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 >> 172.21.15.5:0/274158479 conn(0x7f8c040fa4b0 msgr2=0x7f8c040fba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 shutdown_connections 2023-07-21T19:34:27.427 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:27.436+0000 7f8c0a1ff700 1 -- 172.21.15.5:0/274158479 wait complete. 2023-07-21T19:34:27.450 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:34:27.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:27 smithi005 ceph-mon[126460]: pgmap v851: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:27.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:27 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:27.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:27 smithi156 ceph-mon[128316]: pgmap v851: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:27.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:27 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:28.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:27 smithi101 ceph-mon[128136]: pgmap v851: 97 pgs: 97 active+clean; 11 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.9 MiB/s wr, 798 op/s 2023-07-21T19:34:28.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:27 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:29.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:28 smithi101 ceph-mon[128136]: pgmap v852: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 697 op/s 2023-07-21T19:34:29.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:28 smithi005 ceph-mon[126460]: pgmap v852: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 697 op/s 2023-07-21T19:34:29.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:28 smithi156 ceph-mon[128316]: pgmap v852: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 697 op/s 2023-07-21T19:34:31.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:31 smithi005 ceph-mon[126460]: pgmap v853: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 624 op/s 2023-07-21T19:34:31.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:31 smithi156 ceph-mon[128316]: pgmap v853: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 624 op/s 2023-07-21T19:34:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 480.0 s, 244.0 tps, lat 414.649 ms stddev 83.339 2023-07-21T19:34:32.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:31 smithi101 ceph-mon[128136]: pgmap v853: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 624 op/s 2023-07-21T19:34:33.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:33 smithi005 ceph-mon[126460]: pgmap v854: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 895 op/s 2023-07-21T19:34:33.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:33 smithi156 ceph-mon[128316]: pgmap v854: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 895 op/s 2023-07-21T19:34:34.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:33 smithi101 ceph-mon[128136]: pgmap v854: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.9 MiB/s wr, 895 op/s 2023-07-21T19:34:35.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:35 smithi005 ceph-mon[126460]: pgmap v855: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2023-07-21T19:34:35.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:35 smithi156 ceph-mon[128316]: pgmap v855: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2023-07-21T19:34:36.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:35 smithi101 ceph-mon[128136]: pgmap v855: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.4 MiB/s wr, 762 op/s 2023-07-21T19:34:36.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 485.0 s, 234.4 tps, lat 421.040 ms stddev 80.510 2023-07-21T19:34:37.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:37 smithi005 ceph-mon[126460]: pgmap v856: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 674 op/s 2023-07-21T19:34:37.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:37 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:37.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:37 smithi156 ceph-mon[128316]: pgmap v856: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 674 op/s 2023-07-21T19:34:37.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:37 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:38.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:37 smithi101 ceph-mon[128136]: pgmap v856: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.6 MiB/s wr, 674 op/s 2023-07-21T19:34:38.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:37 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:34:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 -- 172.21.15.5:0/431610604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 msgr2=0x7f751410e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 --2- 172.21.15.5:0/431610604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f751410e720 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f750c00b0a0 tx=0x7f750c00b3b0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 -- 172.21.15.5:0/431610604 shutdown_connections 2023-07-21T19:34:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 --2- 172.21.15.5:0/431610604 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f751410e720 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 --2- 172.21.15.5:0/431610604 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140728b0 0x7f7514072c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 --2- 172.21.15.5:0/431610604 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140731c0 0x7f751410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 -- 172.21.15.5:0/431610604 >> 172.21.15.5:0/431610604 conn(0x7f75140fb600 msgr2=0x7f75140fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.923+0000 7f75197ff700 1 -- 172.21.15.5:0/431610604 shutdown_connections 2023-07-21T19:34:38.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.924+0000 7f75197ff700 1 -- 172.21.15.5:0/431610604 wait complete. 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 Processor -- start 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 -- start start 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140731c0 0x7f75140fffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f75141a8440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514100800 con 0x7f751410c340 2023-07-21T19:34:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514197be0 con 0x7f75140728b0 2023-07-21T19:34:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.925+0000 7f75197ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7514197da0 con 0x7f75140731c0 2023-07-21T19:34:38.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f75127fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140731c0 0x7f75140fffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33260/0 (socket says 172.21.15.5:33260) 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 -- 172.21.15.5:0/1450289580 learned_addr learned my addr 172.21.15.5:0/1450289580 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f75137fe700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f75141a8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 -- 172.21.15.5:0/1450289580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140731c0 msgr2=0x7f75140fffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140731c0 0x7f75140fffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 -- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 msgr2=0x7f75141a8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f75141a8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.926+0000 7f7512ffd700 1 -- 172.21.15.5:0/1450289580 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f750c009d00 con 0x7f75140728b0 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f7512ffd700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9a0 tx=0x7f750400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f74fbfff700 1 -- 172.21.15.5:0/1450289580 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750400ca10 con 0x7f75140728b0 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f74fbfff700 1 -- 172.21.15.5:0/1450289580 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7504010430 con 0x7f75140728b0 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f74fbfff700 1 -- 172.21.15.5:0/1450289580 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504018680 con 0x7f75140728b0 2023-07-21T19:34:38.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7514198080 con 0x7f75140728b0 2023-07-21T19:34:38.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.927+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7514198540 con 0x7f75140728b0 2023-07-21T19:34:38.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.932+0000 7f74fbfff700 1 -- 172.21.15.5:0/1450289580 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f75040105a0 con 0x7f75140728b0 2023-07-21T19:34:38.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.933+0000 7f752d143b80 1 Processor -- start 2023-07-21T19:34:38.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.934+0000 7f752d143b80 1 -- start start 2023-07-21T19:34:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fac040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fac580 0x564390fac9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390eb1c50 con 0x564390fb22d0 2023-07-21T19:34:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390f049e0 con 0x564390fac580 2023-07-21T19:34:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f752d143b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390efde70 con 0x564390e086b0 2023-07-21T19:34:38.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f74fbfff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 0x7f74fc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f75137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f75137fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55018/0 (socket says 172.21.15.5:55018) 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.935+0000 7f75137fe700 1 -- 172.21.15.5:0/2389510158 learned_addr learned my addr 172.21.15.5:0/2389510158 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f74fbfff700 1 -- 172.21.15.5:0/1450289580 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7504014070 con 0x7f75140728b0 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75127fc700 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fac580 0x564390fac9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f7512ffd700 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 0x7f74fc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 -- 172.21.15.5:0/2389510158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fac580 msgr2=0x564390fac9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fac580 0x564390fac9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 -- 172.21.15.5:0/2389510158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 msgr2=0x564390fac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.936+0000 7f75137fe700 1 -- 172.21.15.5:0/2389510158 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564390d2bd80 con 0x564390e086b0 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.937+0000 7f75137fe700 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f750c009cd0 tx=0x7f750c00bdf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=469b5d5269ab805f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.937+0000 7f75137fe700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 0x7f74fc0a37f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d010 tx=0x7f750c019000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.937+0000 7f74f97fa700 1 -- 172.21.15.5:0/2389510158 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c00e040 con 0x564390e086b0 2023-07-21T19:34:38.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.937+0000 7f74f97fa700 1 -- 172.21.15.5:0/2389510158 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f750c00bab0 con 0x564390e086b0 2023-07-21T19:34:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.937+0000 7f74f97fa700 1 -- 172.21.15.5:0/2389510158 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c025070 con 0x564390e086b0 2023-07-21T19:34:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 -- 172.21.15.5:0/2389510158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 msgr2=0x564390fb1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f750c009cd0 tx=0x7f750c00bdf0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 -- 172.21.15.5:0/2389510158 shutdown_connections 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fac580 0x564390fac9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390e086b0 0x564390fb1d90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 --2- 172.21.15.5:0/2389510158 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.938+0000 7f752d143b80 1 -- 172.21.15.5:0/2389510158 >> 172.21.15.5:0/2389510158 conn(0x564390fa1be0 msgr2=0x564390fa6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.939+0000 7f752d143b80 1 -- 172.21.15.5:0/2389510158 shutdown_connections 2023-07-21T19:34:38.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.939+0000 7f752d143b80 1 -- 172.21.15.5:0/2389510158 wait complete. 2023-07-21T19:34:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.939+0000 7f752d143b80 1 -- start start 2023-07-21T19:34:38.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.940+0000 7f752d143b80 1 Processor -- start 2023-07-21T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fb8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fb58a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.929 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390e9da60 con 0x564390fb22d0 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390e9d760 con 0x564390e086b0 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390d818c0 con 0x564390fac580 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f752d143b80 10 client.0 _ll_register_callbacks cb 0x564390e9b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f75137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f75137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38448/0 (socket says 172.21.15.5:38448) 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.941+0000 7f75137fe700 1 -- 172.21.15.5:0/2770818322 learned_addr learned my addr 172.21.15.5:0/2770818322 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:38.930 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75127fc700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fb58a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f7512ffd700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fb8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 -- 172.21.15.5:0/2770818322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 msgr2=0x564390fb58a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fb58a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 -- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 msgr2=0x564390fb8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fb8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 -- 172.21.15.5:0/2770818322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f751410f3c0 con 0x564390fb22d0 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f7512ffd700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fb8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:34:38.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f75137fe700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f750c0097c0 tx=0x7f750c01f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b44ba360527a6186 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.942+0000 7f74f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:34:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750c034070 con 0x564390fb22d0 2023-07-21T19:34:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f750c003c10 con 0x564390fb22d0 2023-07-21T19:34:38.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564390ef6e50 con 0x564390fb22d0 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f750c00e070 con 0x564390fb22d0 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f752d143b80 10 client.20241 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.943+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564390d14c00 con 0x564390fb22d0 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564390ef6e50 con 0x564390fb22d0 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f750c037030 con 0x564390fb22d0 2023-07-21T19:34:38.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 10 client.20241 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 10 client.20241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 20 client.20241 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564390fc5800 0x564390fc7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 4 client.20241 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.944+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564390eb8ff0 con 0x564390fc5800 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.945+0000 7f7512ffd700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564390fc5800 0x564390fc7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.945+0000 7f7512ffd700 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564390fc5800 0x564390fc7c10 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c077a26d5ab6106b server_cookie=ed63542472c8c5b0 in_seq=0 out_seq=0 2023-07-21T19:34:38.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.945+0000 7f74f6ff5700 10 client.20241 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.946+0000 7f74f6ff5700 1 -- 172.21.15.5:0/2770818322 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564390eb8ff0 con 0x564390fc5800 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.946+0000 7f74f6ff5700 10 client.20241 handle_command_reply: tid=0 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.946+0000 7f752d143b80 1 client.20241 shutdown 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 msgr2=0x564390fb5f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f750c0097c0 tx=0x7f750c01f670 comp rx=0 tx=0).stop 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 shutdown_connections 2023-07-21T19:34:38.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564390fc5800 0x564390fc7c10 crc :-1 s=READY pgs=505 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564390fb5f90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fb8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 --2- 172.21.15.5:0/2770818322 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fb58a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 >> 172.21.15.5:0/2770818322 conn(0x564390fa1be0 msgr2=0x564390fa64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f74f6ff5700 0 client.20241 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 shutdown_connections 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 1 -- 172.21.15.5:0/2770818322 wait complete. 2023-07-21T19:34:38.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.947+0000 7f752d143b80 20 client.20241 trim_cache size 0 max 16384 2023-07-21T19:34:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.970+0000 7f752d143b80 1 Processor -- start 2023-07-21T19:34:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.970+0000 7f752d143b80 1 -- start start 2023-07-21T19:34:38.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 0x564390fb1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fb22d0 0x564390fb81c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390fb3e50 con 0x564390e086b0 2023-07-21T19:34:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390fc1aa0 con 0x564390fb22d0 2023-07-21T19:34:38.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.971+0000 7f752d143b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564390fb0640 con 0x564390fac580 2023-07-21T19:34:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:55046/0 (socket says 172.21.15.5:55046) 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 -- 172.21.15.5:0/1699634054 learned_addr learned my addr 172.21.15.5:0/1699634054 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75137fe700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fb22d0 0x564390fb81c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 -- 172.21.15.5:0/1699634054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fb22d0 msgr2=0x564390fb81c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fb22d0 0x564390fb81c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f7512ffd700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 0x564390fb1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 -- 172.21.15.5:0/1699634054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 msgr2=0x564390fb1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 0x564390fb1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f75127fc700 1 -- 172.21.15.5:0/1699634054 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564390d521f0 con 0x564390fac580 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.972+0000 7f7512ffd700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 0x564390fb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.973+0000 7f75127fc700 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7508009b10 tx=0x7f7508009190 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab2124cb6d8c96c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.973+0000 7f74f9ffb700 1 -- 172.21.15.5:0/1699634054 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508014a80 con 0x564390fac580 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.973+0000 7f74f9ffb700 1 -- 172.21.15.5:0/1699634054 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7508014be0 con 0x564390fac580 2023-07-21T19:34:38.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.973+0000 7f74f9ffb700 1 -- 172.21.15.5:0/1699634054 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7508016690 con 0x564390fac580 2023-07-21T19:34:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 -- 172.21.15.5:0/1699634054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 msgr2=0x564390fa6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f7508009b10 tx=0x7f7508009190 comp rx=0 tx=0).stop 2023-07-21T19:34:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 -- 172.21.15.5:0/1699634054 shutdown_connections 2023-07-21T19:34:38.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390fb22d0 0x564390fb81c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390e086b0 0x564390fb1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 --2- 172.21.15.5:0/1699634054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fa6c60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 -- 172.21.15.5:0/1699634054 >> 172.21.15.5:0/1699634054 conn(0x564390fa1be0 msgr2=0x564390fb0a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 -- 172.21.15.5:0/1699634054 shutdown_connections 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.974+0000 7f752d143b80 1 -- 172.21.15.5:0/1699634054 wait complete. 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.975+0000 7f752d143b80 1 -- start start 2023-07-21T19:34:38.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.975+0000 7f752d143b80 1 Processor -- start 2023-07-21T19:34:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fbaf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564391018680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564391018bc0 con 0x564390fb22d0 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564391018d30 con 0x564390e086b0 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564391018ef0 con 0x564390fac580 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f752d143b80 10 client.0 _ll_register_callbacks cb 0x7f750c020ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f7512ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33288/0 (socket says 172.21.15.5:33288) 2023-07-21T19:34:38.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.976+0000 7f7512ffd700 1 -- 172.21.15.5:0/3211972332 learned_addr learned my addr 172.21.15.5:0/3211972332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f75137fe700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564391018680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f7512ffd700 1 -- 172.21.15.5:0/3211972332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 msgr2=0x564390fbaf80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f7512ffd700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fbaf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f7512ffd700 1 -- 172.21.15.5:0/3211972332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 msgr2=0x564391018680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f7512ffd700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564391018680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f7512ffd700 1 -- 172.21.15.5:0/3211972332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75080097c0 con 0x564390e086b0 2023-07-21T19:34:38.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.977+0000 7f75137fe700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564391018680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:34:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.978+0000 7f7512ffd700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7504099ec0 tx=0x7f75040092d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e9c772c0a36a9d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:38.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.978+0000 7f74f6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.978+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f750409a390 con 0x564390e086b0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.978+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f750409df60 con 0x564390e086b0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.978+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564390e57490 con 0x564390e086b0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75040a2380 con 0x564390e086b0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 10 client.30026 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564390d2a250 con 0x564390e086b0 2023-07-21T19:34:38.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f75040b8030 con 0x564390e086b0 2023-07-21T19:34:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f75040a1ef0 con 0x564390e086b0 2023-07-21T19:34:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 10 client.30026 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 10 client.30026 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:38.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 20 client.30026 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56439101a670 0x56439101ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 4 client.30026 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:34:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.979+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564390d4cf10 con 0x56439101a670 2023-07-21T19:34:38.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.980+0000 7f75127fc700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56439101a670 0x56439101ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.981+0000 7f75127fc700 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56439101a670 0x56439101ca80 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=353bd96c038fd71f server_cookie=5323b34a4d124eea in_seq=0 out_seq=0 2023-07-21T19:34:38.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:38.981+0000 7f74f6ff5700 10 client.30026 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:39.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.016+0000 7f74f6ff5700 1 -- 172.21.15.5:0/3211972332 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564390d4cf10 con 0x56439101a670 2023-07-21T19:34:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.016+0000 7f74f6ff5700 10 client.30026 handle_command_reply: tid=0 2023-07-21T19:34:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.016+0000 7f752d143b80 1 client.30026 shutdown 2023-07-21T19:34:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.016+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 msgr2=0x564390fbaa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.016+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f7504099ec0 tx=0x7f75040092d0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 shutdown_connections 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56439101a670 0x56439101ca80 crc :-1 s=READY pgs=506 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564390fb22d0 0x564391018680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f74f6ff5700 0 client.30026 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564390e086b0 0x564390fbaa40 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 --2- 172.21.15.5:0/3211972332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564390fac580 0x564390fbaf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 >> 172.21.15.5:0/3211972332 conn(0x564390fa1be0 msgr2=0x564390fb12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:39.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 shutdown_connections 2023-07-21T19:34:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 1 -- 172.21.15.5:0/3211972332 wait complete. 2023-07-21T19:34:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.017+0000 7f752d143b80 20 client.30026 trim_cache size 0 max 16384 2023-07-21T19:34:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:34:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:34:39.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:34:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:34:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 msgr2=0x7f74fc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 0x7f74fc0a37f0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f750c00d010 tx=0x7f750c019000 comp rx=0 tx=0).stop 2023-07-21T19:34:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 msgr2=0x7f75140ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:39.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f750400e9a0 tx=0x7f750400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 shutdown_connections 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f74fc0a1340 0x7f74fc0a37f0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f751410c340 0x7f75141a8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f75140728b0 0x7f75140ffa60 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 --2- 172.21.15.5:0/1450289580 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f75140731c0 0x7f75140fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 >> 172.21.15.5:0/1450289580 conn(0x7f75140fb600 msgr2=0x7f751410e400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:39.008 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 shutdown_connections 2023-07-21T19:34:39.009 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:39.018+0000 7f75197ff700 1 -- 172.21.15.5:0/1450289580 wait complete. 2023-07-21T19:34:39.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:38 smithi101 ceph-mon[128136]: pgmap v857: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-07-21T19:34:39.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:38 smithi005 ceph-mon[126460]: pgmap v857: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-07-21T19:34:39.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:38 smithi156 ceph-mon[128316]: pgmap v857: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 10 MiB/s wr, 1.04k op/s 2023-07-21T19:34:41.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:41 smithi005 ceph-mon[126460]: pgmap v858: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2023-07-21T19:34:41.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:41 smithi156 ceph-mon[128316]: pgmap v858: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2023-07-21T19:34:41.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 490.0 s, 190.4 tps, lat 176.633 ms stddev 228.487 2023-07-21T19:34:42.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:41 smithi101 ceph-mon[128136]: pgmap v858: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 7.3 MiB/s wr, 738 op/s 2023-07-21T19:34:43.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:43 smithi005 ceph-mon[126460]: pgmap v859: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:34:43.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:43 smithi156 ceph-mon[128316]: pgmap v859: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:34:44.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:43 smithi101 ceph-mon[128136]: pgmap v859: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.9 MiB/s wr, 1.03k op/s 2023-07-21T19:34:45.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:45 smithi005 ceph-mon[126460]: pgmap v860: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 833 op/s 2023-07-21T19:34:45.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:45 smithi156 ceph-mon[128316]: pgmap v860: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 833 op/s 2023-07-21T19:34:46.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:45 smithi101 ceph-mon[128136]: pgmap v860: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.9 MiB/s wr, 833 op/s 2023-07-21T19:34:46.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 495.0 s, 195.8 tps, lat 45.873 ms stddev 11.498 2023-07-21T19:34:47.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:47 smithi005 ceph-mon[126460]: pgmap v861: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-07-21T19:34:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:47 smithi156 ceph-mon[128316]: pgmap v861: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-07-21T19:34:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:47 smithi101 ceph-mon[128136]: pgmap v861: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 7.0 MiB/s wr, 733 op/s 2023-07-21T19:34:49.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:48 smithi101 ceph-mon[128136]: pgmap v862: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 1.02k op/s 2023-07-21T19:34:49.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:48 smithi005 ceph-mon[126460]: pgmap v862: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 1.02k op/s 2023-07-21T19:34:49.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:48 smithi156 ceph-mon[128316]: pgmap v862: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 9.7 MiB/s wr, 1.02k op/s 2023-07-21T19:34:51.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:51 smithi005 ceph-mon[126460]: pgmap v863: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2023-07-21T19:34:51.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:51 smithi156 ceph-mon[128316]: pgmap v863: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2023-07-21T19:34:51.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 500.0 s, 204.2 tps, lat 44.120 ms stddev 11.862 2023-07-21T19:34:52.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:51 smithi101 ceph-mon[128136]: pgmap v863: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 6.1 MiB/s wr, 650 op/s 2023-07-21T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 -- 172.21.15.5:0/1814686152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 msgr2=0x7fc4e010ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/1814686152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e010ebe0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009f90 tx=0x7fc4d800b330 comp rx=0 tx=0).stop 2023-07-21T19:34:52.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 -- 172.21.15.5:0/1814686152 shutdown_connections 2023-07-21T19:34:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/1814686152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e010ebe0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/1814686152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e01023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/1814686152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 0x7fc4e0106930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 -- 172.21.15.5:0/1814686152 >> 172.21.15.5:0/1814686152 conn(0x7fc4e00fd840 msgr2=0x7fc4e00ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 -- 172.21.15.5:0/1814686152 shutdown_connections 2023-07-21T19:34:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.232+0000 7fc4e4fff700 1 -- 172.21.15.5:0/1814686152 wait complete. 2023-07-21T19:34:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.233+0000 7fc4e4fff700 1 Processor -- start 2023-07-21T19:34:52.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 -- start start 2023-07-21T19:34:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 0x7fc4e0198450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e019cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d100 con 0x7fc4e0102970 2023-07-21T19:34:52.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d270 con 0x7fc4e0107010 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.234+0000 7fc4e4fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e019d430 con 0x7fc4e0101fd0 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e019cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44078/0 (socket says 172.21.15.5:44078) 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e019cac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34230/0 (socket says 172.21.15.5:34230) 2023-07-21T19:34:52.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 -- 172.21.15.5:0/2643005210 learned_addr learned my addr 172.21.15.5:0/2643005210 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 0x7fc4e0198450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 -- 172.21.15.5:0/2643005210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 msgr2=0x7fc4e019cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e019cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 -- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 msgr2=0x7fc4e0198450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 0x7fc4e0198450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.235+0000 7fc4de59c700 1 -- 172.21.15.5:0/2643005210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d8009c40 con 0x7fc4e0101fd0 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4de59c700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d950 tx=0x7fc4d000dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4cf7fe700 1 -- 172.21.15.5:0/2643005210 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d0009940 con 0x7fc4e0101fd0 2023-07-21T19:34:52.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e019d710 con 0x7fc4e0101fd0 2023-07-21T19:34:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4cf7fe700 1 -- 172.21.15.5:0/2643005210 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d0010460 con 0x7fc4e0101fd0 2023-07-21T19:34:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4cf7fe700 1 -- 172.21.15.5:0/2643005210 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d000f620 con 0x7fc4e0101fd0 2023-07-21T19:34:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.236+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e01a4f50 con 0x7fc4e0101fd0 2023-07-21T19:34:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.241+0000 7fc4cf7fe700 1 -- 172.21.15.5:0/2643005210 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc4d00105d0 con 0x7fc4e0101fd0 2023-07-21T19:34:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.242+0000 7fc4f8913b80 1 Processor -- start 2023-07-21T19:34:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.243+0000 7fc4f8913b80 1 -- start start 2023-07-21T19:34:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173495a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc17343850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc17240c50 con 0x55cc17343d90 2023-07-21T19:34:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc172939e0 con 0x55cc171976b0 2023-07-21T19:34:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1728ce70 con 0x55cc17349ae0 2023-07-21T19:34:52.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4cf7fe700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 0x7fc4c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc17343850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.244+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173495a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42274/0 (socket says 172.21.15.5:42274) 2023-07-21T19:34:52.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/1929521043 learned_addr learned my addr 172.21.15.5:0/1929521043 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 0x7fc4c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/1929521043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 msgr2=0x55cc17343850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc17343850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/1929521043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 msgr2=0x55cc173495a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173495a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/1929521043 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc170bad80 con 0x55cc17343d90 2023-07-21T19:34:52.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.245+0000 7fc4cf7fe700 1 -- 172.21.15.5:0/2643005210 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc4d00ccb10 con 0x7fc4e0101fd0 2023-07-21T19:34:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.246+0000 7fc4de59c700 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc17343850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:34:52.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.246+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009bc0 tx=0x7fc4d4009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75548dc62eff4f7a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.246+0000 7fc4ccff9700 1 -- 172.21.15.5:0/1929521043 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011b90 con 0x55cc17343d90 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.246+0000 7fc4ccff9700 1 -- 172.21.15.5:0/1929521043 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d40044e0 con 0x55cc17343d90 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.246+0000 7fc4ccff9700 1 -- 172.21.15.5:0/1929521043 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4003e40 con 0x55cc17343d90 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.247+0000 7fc4f8913b80 1 -- 172.21.15.5:0/1929521043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 msgr2=0x55cc173441e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.247+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fc4d4009bc0 tx=0x7fc4d4009ed0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.247+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 0x7fc4c80a37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009790 tx=0x7fc4d80096c0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 -- 172.21.15.5:0/1929521043 shutdown_connections 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc173441e0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173495a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/1929521043 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc17343850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 -- 172.21.15.5:0/1929521043 >> 172.21.15.5:0/1929521043 conn(0x55cc17330be0 msgr2=0x55cc17335f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 -- 172.21.15.5:0/1929521043 shutdown_connections 2023-07-21T19:34:52.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 -- 172.21.15.5:0/1929521043 wait complete. 2023-07-21T19:34:52.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.248+0000 7fc4f8913b80 1 -- start start 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.249+0000 7fc4f8913b80 1 Processor -- start 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc1734fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc1734d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1722ca60 con 0x55cc171976b0 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1722c760 con 0x55cc17343d90 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc171108c0 con 0x55cc17349ae0 2023-07-21T19:34:52.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4f8913b80 10 client.0 _ll_register_callbacks cb 0x55cc1722a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44106/0 (socket says 172.21.15.5:44106) 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.250+0000 7fc4ded9d700 1 -- 172.21.15.5:0/2467493196 learned_addr learned my addr 172.21.15.5:0/2467493196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 -- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 msgr2=0x55cc1734d0c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc1734d0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 -- 172.21.15.5:0/2467493196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 msgr2=0x55cc1734fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc1734fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 -- 172.21.15.5:0/2467493196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4d40097f0 con 0x55cc17349ae0 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a00 tx=0x7fc4d800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19ad91fcbde10620 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d800e040 con 0x55cc17349ae0 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55cc171e6490 con 0x55cc17349ae0 2023-07-21T19:34:52.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.251+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc4d80050e0 con 0x55cc17349ae0 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d800be80 con 0x55cc17349ae0 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 10 client.29965 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55cc170a3c00 con 0x55cc17349ae0 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d8039020 con 0x55cc17349ae0 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4d80082e0 con 0x55cc17349ae0 2023-07-21T19:34:52.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 10 client.29965 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 10 client.29965 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 20 client.29965 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc1735d450 0x55cc1735f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 4 client.29965 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.252+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55cc17285e50 con 0x55cc1735d450 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.253+0000 7fc4de59c700 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc1735d450 0x55cc1735f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.253+0000 7fc4c5ffb700 10 client.29965 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.253+0000 7fc4de59c700 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc1735d450 0x55cc1735f880 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f24cfc576f658278 server_cookie=503bf59ee006b5dc in_seq=0 out_seq=0 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.254+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/2467493196 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55cc17285e50 con 0x55cc1735d450 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.254+0000 7fc4c5ffb700 10 client.29965 handle_command_reply: tid=0 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.254+0000 7fc4f8913b80 1 client.29965 shutdown 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 msgr2=0x55cc1734d760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8003a00 tx=0x7fc4d800b330 comp rx=0 tx=0).stop 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 shutdown_connections 2023-07-21T19:34:52.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17349ae0 0x55cc1734d760 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc1735d450 0x55cc1735f880 crc :-1 s=READY pgs=507 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc1734fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/2467493196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc1734d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 >> 172.21.15.5:0/2467493196 conn(0x55cc17330be0 msgr2=0x55cc17334c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4c5ffb700 0 client.29965 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 shutdown_connections 2023-07-21T19:34:52.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 1 -- 172.21.15.5:0/2467493196 wait complete. 2023-07-21T19:34:52.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.255+0000 7fc4f8913b80 20 client.29965 trim_cache size 0 max 16384 2023-07-21T19:34:52.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.278+0000 7fc4f8913b80 1 Processor -- start 2023-07-21T19:34:52.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.278+0000 7fc4f8913b80 1 -- start start 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.278+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17343850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc17354480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc1734e630 con 0x55cc171976b0 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc17349920 con 0x55cc17343d90 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc17336e70 con 0x55cc17330be0 2023-07-21T19:34:52.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17343850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:42292/0 (socket says 172.21.15.5:42292) 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ded9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc17354480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/93528159 learned_addr learned my addr 172.21.15.5:0/93528159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/93528159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 msgr2=0x55cc17343850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17343850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/93528159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 msgr2=0x55cc17354480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.279+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc17354480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.280+0000 7fc4ddd9b700 1 -- 172.21.15.5:0/93528159 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55cc17344430 con 0x55cc171976b0 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.280+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc4d401a000 tx=0x7fc4d40059a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1332b68d801382bf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.280+0000 7fc4cd7fa700 1 -- 172.21.15.5:0/93528159 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4020070 con 0x55cc171976b0 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.280+0000 7fc4cd7fa700 1 -- 172.21.15.5:0/93528159 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d40056e0 con 0x55cc171976b0 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.280+0000 7fc4cd7fa700 1 -- 172.21.15.5:0/93528159 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4003b90 con 0x55cc171976b0 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 -- 172.21.15.5:0/93528159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 msgr2=0x55cc17348af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fc4d401a000 tx=0x7fc4d40059a0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 -- 172.21.15.5:0/93528159 shutdown_connections 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc17343d90 0x55cc17354480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17343850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/93528159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc171976b0 0x55cc17348af0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 -- 172.21.15.5:0/93528159 >> 172.21.15.5:0/93528159 conn(0x55cc17359a50 msgr2=0x55cc1734a110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.281+0000 7fc4f8913b80 1 -- 172.21.15.5:0/93528159 shutdown_connections 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.282+0000 7fc4f8913b80 1 -- 172.21.15.5:0/93528159 wait complete. 2023-07-21T19:34:52.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.282+0000 7fc4f8913b80 1 -- start start 2023-07-21T19:34:52.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 Processor -- start 2023-07-21T19:34:52.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17352620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc17352b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc173530a0 con 0x55cc17343d90 2023-07-21T19:34:52.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc17353210 con 0x55cc171976b0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55cc173533d0 con 0x55cc17330be0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4f8913b80 10 client.0 _ll_register_callbacks cb 0x55cc1722a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.283+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34258/0 (socket says 172.21.15.5:34258) 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 -- 172.21.15.5:0/3637993404 learned_addr learned my addr 172.21.15.5:0/3637993404 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4ded9d700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc17352b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17352620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 -- 172.21.15.5:0/3637993404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 msgr2=0x55cc17352620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17352620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 -- 172.21.15.5:0/3637993404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 msgr2=0x55cc17352b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc17352b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 -- 172.21.15.5:0/3637993404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55cc170bad80 con 0x55cc171976b0 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17352620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4de59c700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0010880 tx=0x7fc4d0095fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=66482b9aff2e9cf7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:52.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.284+0000 7fc4c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d00a1da0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc4d00108d0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc4d40044e0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4d0099be0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4f8913b80 10 client.30041 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55cc170e11f0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4d40044e0 con 0x55cc171976b0 2023-07-21T19:34:52.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.285+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc4d00990d0 con 0x55cc171976b0 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 10 client.30041 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 10 client.30041 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 20 client.30041 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc17349ae0 0x55cc173b47e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 4 client.30041 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55cc17247ff0 con 0x55cc17349ae0 2023-07-21T19:34:52.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.286+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc17349ae0 0x55cc173b47e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.287+0000 7fc4ddd9b700 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc17349ae0 0x55cc173b47e0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a7bbb3018a8c49 server_cookie=7e927bc00bad1139 in_seq=0 out_seq=0 2023-07-21T19:34:52.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.287+0000 7fc4c5ffb700 10 client.30041 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.316+0000 7fc4c5ffb700 1 -- 172.21.15.5:0/3637993404 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55cc17247ff0 con 0x55cc17349ae0 2023-07-21T19:34:52.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.316+0000 7fc4c5ffb700 10 client.30041 handle_command_reply: tid=0 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.316+0000 7fc4f8913b80 1 client.30041 shutdown 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 msgr2=0x55cc173542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fc4d0010880 tx=0x7fc4d0095fd0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 shutdown_connections 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55cc17349ae0 0x55cc173b47e0 crc :-1 s=READY pgs=508 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55cc17343d90 0x55cc17352b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55cc171976b0 0x55cc173542f0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4c5ffb700 0 client.30041 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 --2- 172.21.15.5:0/3637993404 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55cc17330be0 0x55cc17352620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 >> 172.21.15.5:0/3637993404 conn(0x55cc17359a50 msgr2=0x55cc17348960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 shutdown_connections 2023-07-21T19:34:52.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 1 -- 172.21.15.5:0/3637993404 wait complete. 2023-07-21T19:34:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.317+0000 7fc4f8913b80 20 client.30041 trim_cache size 0 max 16384 2023-07-21T19:34:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:34:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:34:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:34:52.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:34:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 msgr2=0x7fc4c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 0x7fc4c80a37f0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fc4d8009790 tx=0x7fc4d80096c0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 msgr2=0x7fc4e0197f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fc4d000d950 tx=0x7fc4d000dc60 comp rx=0 tx=0).stop 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 shutdown_connections 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4e0107010 0x7fc4e019cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4e0101fd0 0x7fc4e0197f10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc4c80a1340 0x7fc4c80a37f0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 --2- 172.21.15.5:0/2643005210 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4e0102970 0x7fc4e0198450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 >> 172.21.15.5:0/2643005210 conn(0x7fc4e00fd840 msgr2=0x7fc4e00fe270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 shutdown_connections 2023-07-21T19:34:52.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:34:52.318+0000 7fc4e4fff700 1 -- 172.21.15.5:0/2643005210 wait complete. 2023-07-21T19:34:53.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:53 smithi005 ceph-mon[126460]: pgmap v864: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.7 MiB/s wr, 912 op/s 2023-07-21T19:34:53.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:53 smithi156 ceph-mon[128316]: pgmap v864: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.7 MiB/s wr, 912 op/s 2023-07-21T19:34:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:53 smithi101 ceph-mon[128136]: pgmap v864: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.7 MiB/s wr, 912 op/s 2023-07-21T19:34:55.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:55 smithi005 ceph-mon[126460]: pgmap v865: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 673 op/s 2023-07-21T19:34:55.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:55 smithi156 ceph-mon[128316]: pgmap v865: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 673 op/s 2023-07-21T19:34:56.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:55 smithi101 ceph-mon[128136]: pgmap v865: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 6.4 MiB/s wr, 673 op/s 2023-07-21T19:34:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 505.0 s, 250.6 tps, lat 1402.149 ms stddev 3790.782 2023-07-21T19:34:57.451 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:34:57.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:57 smithi005 ceph-mon[126460]: pgmap v866: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2023-07-21T19:34:57.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:57 smithi156 ceph-mon[128316]: pgmap v866: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2023-07-21T19:34:58.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:57 smithi101 ceph-mon[128136]: pgmap v866: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.6 MiB/s wr, 594 op/s 2023-07-21T19:34:58.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 -- 172.21.15.5:0/80932179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c073160 msgr2=0x7f023c10be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 --2- 172.21.15.5:0/80932179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c073160 0x7f023c10be60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f02280098f0 tx=0x7f0228009c00 comp rx=0 tx=0).stop 2023-07-21T19:34:58.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 -- 172.21.15.5:0/80932179 shutdown_connections 2023-07-21T19:34:58.596 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 --2- 172.21.15.5:0/80932179 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c10c3a0 0x7f023c10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 --2- 172.21.15.5:0/80932179 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 0x7f023c072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 --2- 172.21.15.5:0/80932179 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c073160 0x7f023c10be60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.610+0000 7f02431ff700 1 -- 172.21.15.5:0/80932179 >> 172.21.15.5:0/80932179 conn(0x7f023c0fb600 msgr2=0x7f023c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.597 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.611+0000 7f02431ff700 1 -- 172.21.15.5:0/80932179 shutdown_connections 2023-07-21T19:34:58.598 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.611+0000 7f02431ff700 1 -- 172.21.15.5:0/80932179 wait complete. 2023-07-21T19:34:58.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.612+0000 7f02431ff700 1 Processor -- start 2023-07-21T19:34:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 -- start start 2023-07-21T19:34:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 0x7f023c197ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 0x7f023c1983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c19ce80 con 0x7f023c072850 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c19cff0 con 0x7f023c10c3a0 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.613+0000 7f02431ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023c19d1b0 con 0x7f023c073160 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49772/0 (socket says 172.21.15.5:49772) 2023-07-21T19:34:58.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 -- 172.21.15.5:0/2934261348 learned_addr learned my addr 172.21.15.5:0/2934261348 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f023bfff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 0x7f023c1983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f0240f9b700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 0x7f023c197ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 -- 172.21.15.5:0/2934261348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 msgr2=0x7f023c1983e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 0x7f023c1983e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 -- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 msgr2=0x7f023c197ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 0x7f023c197ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f024179c700 1 -- 172.21.15.5:0/2934261348 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228009520 con 0x7f023c10c3a0 2023-07-21T19:34:58.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.614+0000 7f023bfff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 0x7f023c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f024179c700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f023400d720 tx=0x7f023400da30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f0239ffb700 1 -- 172.21.15.5:0/2934261348 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0234004a30 con 0x7f023c10c3a0 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f0239ffb700 1 -- 172.21.15.5:0/2934261348 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02340054b0 con 0x7f023c10c3a0 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f0239ffb700 1 -- 172.21.15.5:0/2934261348 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023400fb90 con 0x7f023c10c3a0 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f023c19d490 con 0x7f023c10c3a0 2023-07-21T19:34:58.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.615+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f023c1a4cd0 con 0x7f023c10c3a0 2023-07-21T19:34:58.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.620+0000 7f0239ffb700 1 -- 172.21.15.5:0/2934261348 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f023400fcf0 con 0x7f023c10c3a0 2023-07-21T19:34:58.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.621+0000 7f0256b47b80 1 Processor -- start 2023-07-21T19:34:58.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.622+0000 7f0256b47b80 1 -- start start 2023-07-21T19:34:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x5584096116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x55840961c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x55840961eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409517cb0 con 0x5584096112d0 2023-07-21T19:34:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840956aa40 con 0x55840961cad0 2023-07-21T19:34:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0256b47b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409563ed0 con 0x558409611be0 2023-07-21T19:34:58.609 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0239ffb700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 0x7f022c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.623+0000 7f0240f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x55840961c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f0240f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x55840961c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35666/0 (socket says 172.21.15.5:35666) 2023-07-21T19:34:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f0240f9b700 1 -- 172.21.15.5:0/4250352169 learned_addr learned my addr 172.21.15.5:0/4250352169 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f0239ffb700 1 -- 172.21.15.5:0/2934261348 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02340cc540 con 0x7f023c10c3a0 2023-07-21T19:34:58.610 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f024179c700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 0x7f022c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x55840961eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x5584096116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 -- 172.21.15.5:0/4250352169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 msgr2=0x55840961c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.611 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x55840961c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 -- 172.21.15.5:0/4250352169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 msgr2=0x55840961eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.624+0000 7f023bfff700 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x55840961eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.625+0000 7f023bfff700 1 -- 172.21.15.5:0/4250352169 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558409391de0 con 0x5584096112d0 2023-07-21T19:34:58.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.625+0000 7f024179c700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 0x7f022c0a3870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f023409d040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.625+0000 7f023bfff700 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x5584096116a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0228009810 tx=0x7f0228005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0205092729503b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.626+0000 7f0226ffd700 1 -- 172.21.15.5:0/4250352169 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022800f8b0 con 0x5584096112d0 2023-07-21T19:34:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.626+0000 7f0226ffd700 1 -- 172.21.15.5:0/4250352169 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f022801d070 con 0x5584096112d0 2023-07-21T19:34:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.626+0000 7f0226ffd700 1 -- 172.21.15.5:0/4250352169 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228003680 con 0x5584096112d0 2023-07-21T19:34:58.613 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 -- 172.21.15.5:0/4250352169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 msgr2=0x5584096116a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x5584096116a0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f0228009810 tx=0x7f0228005dc0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 -- 172.21.15.5:0/4250352169 shutdown_connections 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x55840961eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x5584096116a0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 --2- 172.21.15.5:0/4250352169 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x55840961c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 -- 172.21.15.5:0/4250352169 >> 172.21.15.5:0/4250352169 conn(0x558409607da0 msgr2=0x55840960d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.614 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 -- 172.21.15.5:0/4250352169 shutdown_connections 2023-07-21T19:34:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.627+0000 7f0256b47b80 1 -- 172.21.15.5:0/4250352169 wait complete. 2023-07-21T19:34:58.615 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.628+0000 7f0256b47b80 1 -- start start 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.629+0000 7f0256b47b80 1 Processor -- start 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.629+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.629+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409619850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x558409619f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0256b47b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409503ac0 con 0x5584096112d0 2023-07-21T19:34:58.616 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0256b47b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584095037c0 con 0x55840961cad0 2023-07-21T19:34:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0256b47b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584093e7920 con 0x558409611be0 2023-07-21T19:34:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0256b47b80 10 client.0 _ll_register_callbacks cb 0x558409501760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f023bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.617 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f023bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50468/0 (socket says 172.21.15.5:50468) 2023-07-21T19:34:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f023bfff700 1 -- 172.21.15.5:0/117597796 learned_addr learned my addr 172.21.15.5:0/117597796 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f023bfff700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x558409619f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.618 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.630+0000 7f0240f9b700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409619850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 -- 172.21.15.5:0/117597796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 msgr2=0x558409619850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409619850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 -- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 msgr2=0x558409619f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.619 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x558409619f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 -- 172.21.15.5:0/117597796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f022801f040 con 0x5584096112d0 2023-07-21T19:34:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f023bfff700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0228005340 tx=0x7f0228009400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2824f9f486990aa7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.631+0000 7f021bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:34:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.632+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022802c070 con 0x5584096112d0 2023-07-21T19:34:58.620 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.632+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0228005670 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.632+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5584094bd4f0 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.632+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f022800b560 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 10 client.20271 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55840937ac60 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0228043020 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0228015020 con 0x5584096112d0 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 10 client.20271 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:58.621 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 10 client.20271 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 20 client.20271 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584096347a0 0x558409636bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 4 client.20271 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:34:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.633+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55840955ceb0 con 0x5584096347a0 2023-07-21T19:34:58.622 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.634+0000 7f0240f9b700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584096347a0 0x558409636bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.634+0000 7f0240f9b700 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584096347a0 0x558409636bb0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=646b6437fb5821a server_cookie=6d5a6972e01a0ca8 in_seq=0 out_seq=0 2023-07-21T19:34:58.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.635+0000 7f021bfff700 10 client.20271 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:58.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.635+0000 7f021bfff700 1 -- 172.21.15.5:0/117597796 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55840955ceb0 con 0x5584096347a0 2023-07-21T19:34:58.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.635+0000 7f021bfff700 10 client.20271 handle_command_reply: tid=0 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.635+0000 7f0256b47b80 1 client.20271 shutdown 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 msgr2=0x55840961c560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f0228005340 tx=0x7f0228009400 comp rx=0 tx=0).stop 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 shutdown_connections 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55840961cad0 0x558409619f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5584096347a0 0x558409636bb0 crc :-1 s=READY pgs=509 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584096112d0 0x55840961c560 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 --2- 172.21.15.5:0/117597796 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409619850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f021bfff700 0 client.20271 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 >> 172.21.15.5:0/117597796 conn(0x558409607da0 msgr2=0x55840960d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 shutdown_connections 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.636+0000 7f0256b47b80 1 -- 172.21.15.5:0/117597796 wait complete. 2023-07-21T19:34:58.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.637+0000 7f0256b47b80 20 client.20271 trim_cache size 0 max 16384 2023-07-21T19:34:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.662+0000 7f0256b47b80 1 Processor -- start 2023-07-21T19:34:58.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 -- start start 2023-07-21T19:34:58.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584096112d0 0x55840961c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x558409617070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409604fb0 con 0x55840961cad0 2023-07-21T19:34:58.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409607970 con 0x5584096112d0 2023-07-21T19:34:58.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.663+0000 7f0256b47b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409637210 con 0x558409611be0 2023-07-21T19:34:58.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f023bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584096112d0 0x55840961c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35680/0 (socket says 172.21.15.5:35680) 2023-07-21T19:34:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 -- 172.21.15.5:0/2937569968 learned_addr learned my addr 172.21.15.5:0/2937569968 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f023bfff700 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x558409617070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 -- 172.21.15.5:0/2937569968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584096112d0 msgr2=0x55840961c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584096112d0 0x55840961c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 -- 172.21.15.5:0/2937569968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 msgr2=0x558409617070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x558409617070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.664+0000 7f0240f9b700 1 -- 172.21.15.5:0/2937569968 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584093b8250 con 0x558409611be0 2023-07-21T19:34:58.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.665+0000 7f0240f9b700 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0230004a30 tx=0x7f023000b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d56c1b030315c346 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.665+0000 7f02277fe700 1 -- 172.21.15.5:0/2937569968 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023001c040 con 0x558409611be0 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.665+0000 7f02277fe700 1 -- 172.21.15.5:0/2937569968 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f023000f040 con 0x558409611be0 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.665+0000 7f02277fe700 1 -- 172.21.15.5:0/2937569968 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023000e040 con 0x558409611be0 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 -- 172.21.15.5:0/2937569968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 msgr2=0x558409616b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f0230004a30 tx=0x7f023000b040 comp rx=0 tx=0).stop 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 -- 172.21.15.5:0/2937569968 shutdown_connections 2023-07-21T19:34:58.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x558409617070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584096112d0 0x55840961c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 --2- 172.21.15.5:0/2937569968 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558409611be0 0x558409616b30 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 -- 172.21.15.5:0/2937569968 >> 172.21.15.5:0/2937569968 conn(0x558409607da0 msgr2=0x558409630fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 -- 172.21.15.5:0/2937569968 shutdown_connections 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.666+0000 7f0256b47b80 1 -- 172.21.15.5:0/2937569968 wait complete. 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.667+0000 7f0256b47b80 1 -- start start 2023-07-21T19:34:58.657 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.667+0000 7f0256b47b80 1 Processor -- start 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558409611be0 0x55840962aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x55840962af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409688670 con 0x55840961cad0 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584096887b0 con 0x558409611be0 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558409688970 con 0x5584096112d0 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f0256b47b80 10 client.0 _ll_register_callbacks cb 0x7f022800f800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:34:58.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f023bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f023bfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35690/0 (socket says 172.21.15.5:35690) 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f023bfff700 1 -- 172.21.15.5:0/1733048553 learned_addr learned my addr 172.21.15.5:0/1733048553 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.668+0000 7f023bfff700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x55840962af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f0240f9b700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558409611be0 0x55840962aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 -- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558409611be0 msgr2=0x55840962aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558409611be0 0x55840962aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 -- 172.21.15.5:0/1733048553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 msgr2=0x55840962af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:34:58.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x55840962af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 -- 172.21.15.5:0/1733048553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0230009420 con 0x5584096112d0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f023bfff700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0228009470 tx=0x7f0228009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=935048f72a8ddf0c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.669+0000 7f021bfff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022802c030 con 0x5584096112d0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f022801d070 con 0x5584096112d0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0228000e40 con 0x5584096112d0 2023-07-21T19:34:58.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f022800f590 con 0x5584096112d0 2023-07-21T19:34:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f0256b47b80 10 client.29989 fetch_fsmap learned FSMap version 72 2023-07-21T19:34:58.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.670+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5584093902b0 con 0x5584096112d0 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0228059020 con 0x5584096112d0 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0228015040 con 0x5584096112d0 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 10 client.29989 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 10 client.29989 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 20 client.29989 populate_metadata read hostname 'smithi005' 2023-07-21T19:34:58.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558409689c00 0x55840968bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 4 client.29989 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.671+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55840951f050 con 0x558409689c00 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.672+0000 7f0240f9b700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558409689c00 0x55840968bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.673+0000 7f0240f9b700 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558409689c00 0x55840968bfe0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bec79ecb4bf8da35 server_cookie=490c8d920a13f3bb in_seq=0 out_seq=0 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.673+0000 7f021bfff700 10 client.29989 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f021bfff700 1 -- 172.21.15.5:0/1733048553 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55840951f050 con 0x558409689c00 2023-07-21T19:34:58.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f021bfff700 10 client.29989 handle_command_reply: tid=0 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 client.29989 shutdown 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 msgr2=0x55840962a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f0228009470 tx=0x7f0228009c00 comp rx=0 tx=0).stop 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 shutdown_connections 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558409689c00 0x55840968bfe0 crc :-1 s=READY pgs=510 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55840961cad0 0x55840962af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584096112d0 0x55840962a4e0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 --2- 172.21.15.5:0/1733048553 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558409611be0 0x55840962aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 >> 172.21.15.5:0/1733048553 conn(0x558409607da0 msgr2=0x5584096131a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.674+0000 7f021bfff700 0 client.29989 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.675+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 shutdown_connections 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.675+0000 7f0256b47b80 1 -- 172.21.15.5:0/1733048553 wait complete. 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.675+0000 7f0256b47b80 20 client.29989 trim_cache size 0 max 16384 2023-07-21T19:34:58.665 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:34:58.666 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:34:58.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:34:58.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:34:58.667 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:34:58.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 msgr2=0x7f022c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 0x7f022c0a3870 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f0234000c00 tx=0x7f023409d040 comp rx=0 tx=0).stop 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 msgr2=0x7f023c19c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f023400d720 tx=0x7f023400da30 comp rx=0 tx=0).stop 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 shutdown_connections 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f023c10c3a0 0x7f023c19c840 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f022c0a13c0 0x7f022c0a3870 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f023c072850 0x7f023c197ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 --2- 172.21.15.5:0/2934261348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f023c073160 0x7f023c1983e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:34:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 >> 172.21.15.5:0/2934261348 conn(0x7f023c0fb600 msgr2=0x7f023c10e4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:34:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 shutdown_connections 2023-07-21T19:34:58.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:34:58.676+0000 7f02431ff700 1 -- 172.21.15.5:0/2934261348 wait complete. 2023-07-21T19:34:58.687 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:34:59.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:34:58 smithi101 ceph-mon[128136]: pgmap v867: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T19:34:59.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:34:58 smithi005 ceph-mon[126460]: pgmap v867: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T19:34:59.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:34:58 smithi156 ceph-mon[128316]: pgmap v867: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.8 MiB/s wr, 882 op/s 2023-07-21T19:35:00.567 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:00 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:35:00.647 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:00 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:35:00.656 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:00 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:35:01.567 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:01 smithi101 ceph-mon[128136]: pgmap v868: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 599 op/s 2023-07-21T19:35:01.644 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:01 smithi005 ceph-mon[126460]: pgmap v868: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 599 op/s 2023-07-21T19:35:01.656 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:01 smithi156 ceph-mon[128316]: pgmap v868: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.1 MiB/s wr, 599 op/s 2023-07-21T19:35:01.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 510.0 s, 236.2 tps, lat 371.668 ms stddev 104.479 2023-07-21T19:35:03.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:03 smithi005 ceph-mon[126460]: pgmap v869: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 857 op/s 2023-07-21T19:35:03.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:03 smithi156 ceph-mon[128316]: pgmap v869: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 857 op/s 2023-07-21T19:35:04.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:03 smithi101 ceph-mon[128136]: pgmap v869: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.5 MiB/s wr, 857 op/s 2023-07-21T19:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 -- 172.21.15.5:0/102547383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 msgr2=0x7f368406cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 --2- 172.21.15.5:0/102547383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f368406cd80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f3674009990 tx=0x7f3674009ca0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 -- 172.21.15.5:0/102547383 shutdown_connections 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 --2- 172.21.15.5:0/102547383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f368411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 --2- 172.21.15.5:0/102547383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 0x7f368410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 --2- 172.21.15.5:0/102547383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f368406cd80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 -- 172.21.15.5:0/102547383 >> 172.21.15.5:0/102547383 conn(0x7f368406bd70 msgr2=0x7f3684070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.579+0000 7f368c1ff700 1 -- 172.21.15.5:0/102547383 shutdown_connections 2023-07-21T19:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.580+0000 7f368c1ff700 1 -- 172.21.15.5:0/102547383 wait complete. 2023-07-21T19:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.581+0000 7f368c1ff700 1 Processor -- start 2023-07-21T19:35:05.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.581+0000 7f368c1ff700 1 -- start start 2023-07-21T19:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f36841aaa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 0x7f36841a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36841a5000 con 0x7f368406d3a0 2023-07-21T19:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36841a5170 con 0x7f368406c910 2023-07-21T19:35:05.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.582+0000 7f368c1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36841a5330 con 0x7f368410e2a0 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50500/0 (socket says 172.21.15.5:50500) 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 -- 172.21.15.5:0/1275489036 learned_addr learned my addr 172.21.15.5:0/1275489036 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368a79c700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 0x7f36841a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f36841aaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 -- 172.21.15.5:0/1275489036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 msgr2=0x7f36841a4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 0x7f36841a4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 -- 172.21.15.5:0/1275489036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 msgr2=0x7f36841aaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f36841aaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.583+0000 7f368979a700 1 -- 172.21.15.5:0/1275489036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36740095f0 con 0x7f368406d3a0 2023-07-21T19:35:05.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.584+0000 7f368979a700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674004710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.584+0000 7f367affd700 1 -- 172.21.15.5:0/1275489036 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367401d070 con 0x7f368406d3a0 2023-07-21T19:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.584+0000 7f367affd700 1 -- 172.21.15.5:0/1275489036 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3674003fc0 con 0x7f368406d3a0 2023-07-21T19:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.585+0000 7f367affd700 1 -- 172.21.15.5:0/1275489036 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3674022c80 con 0x7f368406d3a0 2023-07-21T19:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.585+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36841a55b0 con 0x7f368406d3a0 2023-07-21T19:35:05.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.585+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36841b19e0 con 0x7f368406d3a0 2023-07-21T19:35:05.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.590+0000 7f367affd700 1 -- 172.21.15.5:0/1275489036 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3674022470 con 0x7f368406d3a0 2023-07-21T19:35:05.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.591+0000 7f369faa9b80 1 Processor -- start 2023-07-21T19:35:05.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 -- start start 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aa1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402a9c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402a9c9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584029a1c50 con 0x558402a9c580 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584029f49e0 con 0x5584028f86b0 2023-07-21T19:35:05.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.592+0000 7f369faa9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584029ede70 con 0x558402aa22d0 2023-07-21T19:35:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f368a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aa1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402a9c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f368a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aa1d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49850/0 (socket says 172.21.15.5:49850) 2023-07-21T19:35:05.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f368a79c700 1 -- 172.21.15.5:0/2223720613 learned_addr learned my addr 172.21.15.5:0/2223720613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f368979a700 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402a9c9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f367affd700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 0x7f36700a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 -- 172.21.15.5:0/2223720613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 msgr2=0x558402aa1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aa1d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 -- 172.21.15.5:0/2223720613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 msgr2=0x558402a9c9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402a9c9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f3689f9b700 1 -- 172.21.15.5:0/2223720613 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55840281bd80 con 0x558402aa22d0 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.593+0000 7f367affd700 1 -- 172.21.15.5:0/1275489036 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f36740ce270 con 0x7f368406d3a0 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.594+0000 7f3689f9b700 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402a9c040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f367c00a430 tx=0x7f367c0096d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f0fae3fe0b47632 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.594+0000 7f366ffff700 1 -- 172.21.15.5:0/2223720613 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c010040 con 0x558402aa22d0 2023-07-21T19:35:05.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.594+0000 7f368a79c700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 0x7f36700a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.594+0000 7f366ffff700 1 -- 172.21.15.5:0/2223720613 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f367c014ab0 con 0x558402aa22d0 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.594+0000 7f366ffff700 1 -- 172.21.15.5:0/2223720613 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c0139b0 con 0x558402aa22d0 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 -- 172.21.15.5:0/2223720613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 msgr2=0x558402a9c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402a9c040 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f367c00a430 tx=0x7f367c0096d0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f368a79c700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 0x7f36700a3940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3680005950 tx=0x7f3680004750 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 -- 172.21.15.5:0/2223720613 shutdown_connections 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402a9c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aa1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 --2- 172.21.15.5:0/2223720613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402a9c040 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 -- 172.21.15.5:0/2223720613 >> 172.21.15.5:0/2223720613 conn(0x558402a91be0 msgr2=0x558402a96f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 -- 172.21.15.5:0/2223720613 shutdown_connections 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.595+0000 7f369faa9b80 1 -- 172.21.15.5:0/2223720613 wait complete. 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.596+0000 7f369faa9b80 1 -- start start 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.596+0000 7f369faa9b80 1 Processor -- start 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584028f86b0 0x558402aa5910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa6390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840298da60 con 0x5584028f86b0 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55840298d760 con 0x558402aa22d0 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5584028718c0 con 0x558402a9c580 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.597+0000 7f369faa9b80 10 client.0 _ll_register_callbacks cb 0x55840298b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f3689f9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584028f86b0 0x558402aa5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:35740/0 (socket says 172.21.15.5:35740) 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 -- 172.21.15.5:0/1602862797 learned_addr learned my addr 172.21.15.5:0/1602862797 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 -- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 msgr2=0x558402aa6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 -- 172.21.15.5:0/1602862797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584028f86b0 msgr2=0x558402aa5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584028f86b0 0x558402aa5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 -- 172.21.15.5:0/1602862797 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c009c20 con 0x558402a9c580 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f368979a700 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3674097de0 tx=0x7f367409af30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1041574c228f724b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.598+0000 7f366d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367409b370 con 0x558402a9c580 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558402947490 con 0x558402a9c580 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f367409f3a0 con 0x558402a9c580 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f36740a3730 con 0x558402a9c580 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f369faa9b80 10 client.29998 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:05.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x558402804c00 con 0x558402a9c580 2023-07-21T19:35:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558402947490 con 0x558402a9c580 2023-07-21T19:35:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.599+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f367409a860 con 0x558402a9c580 2023-07-21T19:35:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 10 client.29998 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 10 client.29998 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:05.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 20 client.29998 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402ab5800 0x558402ab7c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 4 client.29998 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5584029e6e50 con 0x558402ab5800 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.600+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402ab5800 0x558402ab7c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.601+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402ab5800 0x558402ab7c10 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fd80018547cb9741 server_cookie=6c1af3a10a79d59 in_seq=0 out_seq=0 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.601+0000 7f366d7fa700 10 client.29998 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.602+0000 7f366d7fa700 1 -- 172.21.15.5:0/1602862797 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5584029e6e50 con 0x558402ab5800 2023-07-21T19:35:05.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.602+0000 7f366d7fa700 10 client.29998 handle_command_reply: tid=0 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.602+0000 7f369faa9b80 1 client.29998 shutdown 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 msgr2=0x558402aa5e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f3674097de0 tx=0x7f367409af30 comp rx=0 tx=0).stop 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 shutdown_connections 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402ab5800 0x558402ab7c10 crc :-1 s=READY pgs=511 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5584028f86b0 0x558402aa5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1602862797 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402a9c580 0x558402aa5e50 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 >> 172.21.15.5:0/1602862797 conn(0x558402a91be0 msgr2=0x558402a95c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f366d7fa700 0 client.29998 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 shutdown_connections 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 1 -- 172.21.15.5:0/1602862797 wait complete. 2023-07-21T19:35:05.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.603+0000 7f369faa9b80 20 client.29998 trim_cache size 0 max 16384 2023-07-21T19:35:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 Processor -- start 2023-07-21T19:35:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 -- start start 2023-07-21T19:35:05.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584028f86b0 0x558402a9c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa61c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402a98b70 con 0x558402a9c580 2023-07-21T19:35:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402aa1a10 con 0x558402aa22d0 2023-07-21T19:35:05.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f369faa9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402a935f0 con 0x5584028f86b0 2023-07-21T19:35:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f368979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f368979a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50524/0 (socket says 172.21.15.5:50524) 2023-07-21T19:35:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f368979a700 1 -- 172.21.15.5:0/1300438612 learned_addr learned my addr 172.21.15.5:0/1300438612 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f368a79c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.631+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584028f86b0 0x558402a9c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 -- 172.21.15.5:0/1300438612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584028f86b0 msgr2=0x558402a9c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584028f86b0 0x558402a9c040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 -- 172.21.15.5:0/1300438612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 msgr2=0x558402aa61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 -- 172.21.15.5:0/1300438612 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5584028421f0 con 0x558402a9c580 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.632+0000 7f368979a700 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f36740996c0 tx=0x7f367409afc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd82b558aaadd5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.633+0000 7f3678ff9700 1 -- 172.21.15.5:0/1300438612 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367409f3a0 con 0x558402a9c580 2023-07-21T19:35:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.633+0000 7f3678ff9700 1 -- 172.21.15.5:0/1300438612 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f36740a3730 con 0x558402a9c580 2023-07-21T19:35:05.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.633+0000 7f3678ff9700 1 -- 172.21.15.5:0/1300438612 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36740a33c0 con 0x558402a9c580 2023-07-21T19:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.633+0000 7f369faa9b80 1 -- 172.21.15.5:0/1300438612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 msgr2=0x558402aa1370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.633+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f36740996c0 tx=0x7f367409afc0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 -- 172.21.15.5:0/1300438612 shutdown_connections 2023-07-21T19:35:05.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558402aa22d0 0x558402aa61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5584028f86b0 0x558402a9c040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1300438612 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402a9c580 0x558402aa1370 secure :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7f36740996c0 tx=0x7f367409afc0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 -- 172.21.15.5:0/1300438612 >> 172.21.15.5:0/1300438612 conn(0x558402a91be0 msgr2=0x558402aa2900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 -- 172.21.15.5:0/1300438612 shutdown_connections 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 -- 172.21.15.5:0/1300438612 wait complete. 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.634+0000 7f369faa9b80 1 -- start start 2023-07-21T19:35:05.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 Processor -- start 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aaaa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402aaaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402b03040 con 0x558402b02690 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402b031b0 con 0x5584028f86b0 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558402b03370 con 0x558402aa22d0 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f369faa9b80 10 client.0 _ll_register_callbacks cb 0x55840298b700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:05.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.635+0000 7f368a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f3689f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aaaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50538/0 (socket says 172.21.15.5:50538) 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f3689f9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aaaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49884/0 (socket says 172.21.15.5:49884) 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 -- 172.21.15.5:0/1368151521 learned_addr learned my addr 172.21.15.5:0/1368151521 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368979a700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402aaaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 -- 172.21.15.5:0/1368151521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 msgr2=0x558402aaaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.624 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402aaaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 -- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 msgr2=0x558402aaaa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aaaa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.636+0000 7f368a79c700 1 -- 172.21.15.5:0/1368151521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558402804c00 con 0x558402b02690 2023-07-21T19:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f368a79c700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f3680010e30 tx=0x7f36800047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=905b3bb769f08929 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f366d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f368001fea0 con 0x558402b02690 2023-07-21T19:35:05.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(131..131 src has 1..131) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3680012260 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f367c014ab0 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3680004830 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f369faa9b80 10 client.20292 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.637+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55840281a250 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3680036020 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f368000f530 con 0x558402b02690 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 10 client.20292 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:05.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 10 client.20292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 20 client.20292 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402b0aaf0 0x558402b0cf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 4 client.20292 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5584029a8ff0 con 0x558402b0aaf0 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.638+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402b0aaf0 0x558402b0cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.639+0000 7f3689f9b700 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402b0aaf0 0x558402b0cf00 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c68478c02c18839e server_cookie=93d54c05e1916eeb in_seq=0 out_seq=0 2023-07-21T19:35:05.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.639+0000 7f366d7fa700 10 client.20292 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:05.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.684+0000 7f366d7fa700 1 -- 172.21.15.5:0/1368151521 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5584029a8ff0 con 0x558402b0aaf0 2023-07-21T19:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.684+0000 7f366d7fa700 10 client.20292 handle_command_reply: tid=0 2023-07-21T19:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.684+0000 7f369faa9b80 1 client.20292 shutdown 2023-07-21T19:35:05.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 msgr2=0x558402b02b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f3680010e30 tx=0x7f36800047c0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 shutdown_connections 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558402b0aaf0 0x558402b0cf00 crc :-1 s=READY pgs=512 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558402b02690 0x558402b02b00 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5584028f86b0 0x558402aaaa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 --2- 172.21.15.5:0/1368151521 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558402aa22d0 0x558402aaaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 >> 172.21.15.5:0/1368151521 conn(0x558402a91be0 msgr2=0x558402aa11e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f366d7fa700 0 client.20292 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 shutdown_connections 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 1 -- 172.21.15.5:0/1368151521 wait complete. 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.685+0000 7f369faa9b80 20 client.20292 trim_cache size 0 max 16384 2023-07-21T19:35:05.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:35:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:35:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:35:05.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:35:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 msgr2=0x7f36700a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 0x7f36700a3940 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3680005950 tx=0x7f3680004750 comp rx=0 tx=0).stop 2023-07-21T19:35:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 msgr2=0x7f36841aafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f3674000c00 tx=0x7f3674004710 comp rx=0 tx=0).stop 2023-07-21T19:35:05.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 shutdown_connections 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f368410e2a0 0x7f36841a4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f368406c910 0x7f36841aaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36700a1490 0x7f36700a3940 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 --2- 172.21.15.5:0/1275489036 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f368406d3a0 0x7f36841aafa0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.686+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 >> 172.21.15.5:0/1275489036 conn(0x7f368406bd70 msgr2=0x7f3684070800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:05.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.687+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 shutdown_connections 2023-07-21T19:35:05.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:05.687+0000 7f368c1ff700 1 -- 172.21.15.5:0/1275489036 wait complete. 2023-07-21T19:35:05.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:05 smithi005 ceph-mon[126460]: pgmap v870: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 665 op/s 2023-07-21T19:35:05.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:05 smithi156 ceph-mon[128316]: pgmap v870: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 665 op/s 2023-07-21T19:35:06.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:05 smithi101 ceph-mon[128136]: pgmap v870: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 665 op/s 2023-07-21T19:35:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 515.0 s, 162.2 tps, lat 637.485 ms stddev 311.813 2023-07-21T19:35:07.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:07 smithi005 ceph-mon[126460]: pgmap v871: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 617 op/s 2023-07-21T19:35:07.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:07 smithi005 ceph-mon[126460]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T19:35:07.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:07 smithi156 ceph-mon[128316]: pgmap v871: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 617 op/s 2023-07-21T19:35:07.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:07 smithi156 ceph-mon[128316]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T19:35:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:07 smithi101 ceph-mon[128136]: pgmap v871: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.4 MiB/s wr, 617 op/s 2023-07-21T19:35:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:07 smithi101 ceph-mon[128136]: osdmap e132: 12 total, 12 up, 12 in 2023-07-21T19:35:09.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:08 smithi005 ceph-mon[126460]: pgmap v873: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:09.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:08 smithi156 ceph-mon[128316]: pgmap v873: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:09.318 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:08 smithi101 ceph-mon[128136]: pgmap v873: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:11.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:11 smithi005 ceph-mon[126460]: pgmap v874: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:11.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:11 smithi156 ceph-mon[128316]: pgmap v874: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:11.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 520.0 s, 66.0 tps, lat 1588.727 ms stddev 746.125 2023-07-21T19:35:12.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:11 smithi101 ceph-mon[128136]: pgmap v874: 97 pgs: 24 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 33 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.3 MiB/s wr, 637 op/s 2023-07-21T19:35:13.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:13 smithi005 ceph-mon[126460]: pgmap v875: 97 pgs: 14 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 62 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2023-07-21T19:35:13.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:13 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:13.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:13 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:13 smithi156 ceph-mon[128316]: pgmap v875: 97 pgs: 14 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 62 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2023-07-21T19:35:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:13 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:13.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:13 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:14.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:13 smithi101 ceph-mon[128136]: pgmap v875: 97 pgs: 14 active+clean+snaptrim, 21 active+clean+snaptrim_wait, 62 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.8 MiB/s wr, 424 op/s 2023-07-21T19:35:14.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:13 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:14.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:13 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:15.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:15 smithi005 ceph-mon[126460]: pgmap v876: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:15.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:15 smithi156 ceph-mon[128316]: pgmap v876: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:16.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:15 smithi101 ceph-mon[128136]: pgmap v876: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 525.0 s, 179.0 tps, lat 574.634 ms stddev 179.879 2023-07-21T19:35:17.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:17 smithi005 ceph-mon[126460]: pgmap v877: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:17.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:17 smithi156 ceph-mon[128316]: pgmap v877: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:18.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:17 smithi101 ceph-mon[128136]: pgmap v877: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 4.2 MiB/s wr, 425 op/s 2023-07-21T19:35:19.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:18 smithi101 ceph-mon[128136]: osdmap e133: 12 total, 12 up, 12 in 2023-07-21T19:35:19.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:18 smithi101 ceph-mon[128136]: pgmap v879: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2023-07-21T19:35:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:18 smithi005 ceph-mon[126460]: osdmap e133: 12 total, 12 up, 12 in 2023-07-21T19:35:19.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:18 smithi005 ceph-mon[126460]: pgmap v879: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2023-07-21T19:35:19.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:18 smithi156 ceph-mon[128316]: osdmap e133: 12 total, 12 up, 12 in 2023-07-21T19:35:19.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:18 smithi156 ceph-mon[128316]: pgmap v879: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 539 op/s 2023-07-21T19:35:19.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.702+0000 7f345e1ff700 1 -- 172.21.15.5:0/1114308225 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 msgr2=0x7f3458110460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.702+0000 7f345e1ff700 1 --2- 172.21.15.5:0/1114308225 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f3458110460 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f3440009990 tx=0x7f3440009ca0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 -- 172.21.15.5:0/1114308225 shutdown_connections 2023-07-21T19:35:19.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 --2- 172.21.15.5:0/1114308225 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f34581109a0 0x7f3458112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 --2- 172.21.15.5:0/1114308225 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3458074a40 0x7f3458074e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 --2- 172.21.15.5:0/1114308225 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f3458110460 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 -- 172.21.15.5:0/1114308225 >> 172.21.15.5:0/1114308225 conn(0x7f34580fd7e0 msgr2=0x7f34580ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:19.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 -- 172.21.15.5:0/1114308225 shutdown_connections 2023-07-21T19:35:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.704+0000 7f345e1ff700 1 -- 172.21.15.5:0/1114308225 wait complete. 2023-07-21T19:35:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.706+0000 7f345e1ff700 1 Processor -- start 2023-07-21T19:35:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.706+0000 7f345e1ff700 1 -- start start 2023-07-21T19:35:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f345819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34581109a0 0x7f34581a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581a1340 con 0x7f34581109a0 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581a14b0 con 0x7f3458075350 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f345e1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34581a1670 con 0x7f3458074a40 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f34577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f34577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37794/0 (socket says 172.21.15.5:37794) 2023-07-21T19:35:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.707+0000 7f34577fe700 1 -- 172.21.15.5:0/4003961660 learned_addr learned my addr 172.21.15.5:0/4003961660 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f344ffff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f345819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f3457fff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34581109a0 0x7f34581a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f34577fe700 1 -- 172.21.15.5:0/4003961660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 msgr2=0x7f345819c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f34577fe700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f345819c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f34577fe700 1 -- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34581109a0 msgr2=0x7f34581a0d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f34577fe700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34581109a0 0x7f34581a0d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.708+0000 7f34577fe700 1 -- 172.21.15.5:0/4003961660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34400095f0 con 0x7f3458074a40 2023-07-21T19:35:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f34577fe700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f344800ef10 tx=0x7f344800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f34557fa700 1 -- 172.21.15.5:0/4003961660 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344800c9c0 con 0x7f3458074a40 2023-07-21T19:35:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f34557fa700 1 -- 172.21.15.5:0/4003961660 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f344800f040 con 0x7f3458074a40 2023-07-21T19:35:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f34557fa700 1 -- 172.21.15.5:0/4003961660 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448014480 con 0x7f3458074a40 2023-07-21T19:35:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34581a18f0 con 0x7f3458074a40 2023-07-21T19:35:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.709+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581a9190 con 0x7f3458074a40 2023-07-21T19:35:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.714+0000 7f34557fa700 1 -- 172.21.15.5:0/4003961660 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3448007690 con 0x7f3458074a40 2023-07-21T19:35:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.716+0000 7f34557fa700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.716+0000 7f34557fa700 1 -- 172.21.15.5:0/4003961660 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f34480ccf50 con 0x7f3458074a40 2023-07-21T19:35:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.716+0000 7f3471a7db80 1 Processor -- start 2023-07-21T19:35:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.716+0000 7f344ffff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.717+0000 7f3471a7db80 1 -- start start 2023-07-21T19:35:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.717+0000 7f344ffff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f3440005960 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.717+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad13d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.717+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad0e580 0x55b8aad0e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f3471a7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aac13c50 con 0x55b8aab6a6b0 2023-07-21T19:35:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f3471a7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aac669e0 con 0x55b8aad0e580 2023-07-21T19:35:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f3471a7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aac5fe70 con 0x55b8aad142d0 2023-07-21T19:35:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f344ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f34577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad13d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f344ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37798/0 (socket says 172.21.15.5:37798) 2023-07-21T19:35:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f344ffff700 1 -- 172.21.15.5:0/957549078 learned_addr learned my addr 172.21.15.5:0/957549078 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.718+0000 7f3457fff700 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad0e580 0x55b8aad0e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 -- 172.21.15.5:0/957549078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad0e580 msgr2=0x55b8aad0e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad0e580 0x55b8aad0e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 -- 172.21.15.5:0/957549078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 msgr2=0x55b8aad13d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad13d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 -- 172.21.15.5:0/957549078 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8aaa8dd80 con 0x55b8aad142d0 2023-07-21T19:35:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344ffff700 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3440009990 tx=0x7f3440020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40d1b6660cf63492 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344dffb700 1 -- 172.21.15.5:0/957549078 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3440004700 con 0x55b8aad142d0 2023-07-21T19:35:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344dffb700 1 -- 172.21.15.5:0/957549078 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f344001e040 con 0x55b8aad142d0 2023-07-21T19:35:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.719+0000 7f344dffb700 1 -- 172.21.15.5:0/957549078 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f344000f9b0 con 0x55b8aad142d0 2023-07-21T19:35:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.720+0000 7f3471a7db80 1 -- 172.21.15.5:0/957549078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 msgr2=0x55b8aad0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.720+0000 7f3471a7db80 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f3440009990 tx=0x7f3440020000 comp rx=0 tx=0).stop 2023-07-21T19:35:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 -- 172.21.15.5:0/957549078 shutdown_connections 2023-07-21T19:35:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad0e580 0x55b8aad0e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad13d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 --2- 172.21.15.5:0/957549078 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad0e040 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 -- 172.21.15.5:0/957549078 >> 172.21.15.5:0/957549078 conn(0x55b8aad03be0 msgr2=0x55b8aad08f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 -- 172.21.15.5:0/957549078 shutdown_connections 2023-07-21T19:35:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.721+0000 7f3471a7db80 1 -- 172.21.15.5:0/957549078 wait complete. 2023-07-21T19:35:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.722+0000 7f3471a7db80 1 -- start start 2023-07-21T19:35:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.722+0000 7f3471a7db80 1 Processor -- start 2023-07-21T19:35:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.713 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad17880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad17fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aabffa60 con 0x55b8aad0e580 2023-07-21T19:35:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aabff760 con 0x55b8aab6a6b0 2023-07-21T19:35:19.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aaae38c0 con 0x55b8aad142d0 2023-07-21T19:35:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f3471a7db80 10 client.0 _ll_register_callbacks cb 0x55b8aabfd700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:39140/0 (socket says 172.21.15.5:39140) 2023-07-21T19:35:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 -- 172.21.15.5:0/666814618 learned_addr learned my addr 172.21.15.5:0/666814618 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 -- 172.21.15.5:0/666814618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 msgr2=0x55b8aad17fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad17fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 -- 172.21.15.5:0/666814618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 msgr2=0x55b8aad17880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.723+0000 7f34577fe700 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad17880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f34577fe700 1 -- 172.21.15.5:0/666814618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34581137b0 con 0x55b8aab6a6b0 2023-07-21T19:35:19.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f34577fe700 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3448099d20 tx=0x7f3448099a30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=161011c919bf5762 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:35:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480b1070 con 0x55b8aab6a6b0 2023-07-21T19:35:19.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f344809e050 con 0x55b8aab6a6b0 2023-07-21T19:35:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b8aabb9490 con 0x55b8aab6a6b0 2023-07-21T19:35:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34480a2470 con 0x55b8aab6a6b0 2023-07-21T19:35:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.724+0000 7f3471a7db80 10 client.30089 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55b8aaa76c00 con 0x55b8aab6a6b0 2023-07-21T19:35:19.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f34480c8080 con 0x55b8aab6a6b0 2023-07-21T19:35:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f34480add20 con 0x55b8aab6a6b0 2023-07-21T19:35:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 10 client.30089 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:19.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 10 client.30089 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:19.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 20 client.30089 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad27ab0 0x55b8aad29ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 4 client.30089 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.725+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b8aac58e50 con 0x55b8aad27ab0 2023-07-21T19:35:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.726+0000 7f344ffff700 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad27ab0 0x55b8aad29ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.726+0000 7f344ffff700 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad27ab0 0x55b8aad29ec0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2da3ca4f81dfdb98 server_cookie=ee3fa3496ef79644 in_seq=0 out_seq=0 2023-07-21T19:35:19.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.726+0000 7f3436ffd700 10 client.30089 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.727+0000 7f3436ffd700 1 -- 172.21.15.5:0/666814618 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b8aac58e50 con 0x55b8aad27ab0 2023-07-21T19:35:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.727+0000 7f3436ffd700 10 client.30089 handle_command_reply: tid=0 2023-07-21T19:35:19.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.727+0000 7f3471a7db80 1 client.30089 shutdown 2023-07-21T19:35:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 msgr2=0x55b8aad1a600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f3448099d20 tx=0x7f3448099a30 comp rx=0 tx=0).stop 2023-07-21T19:35:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 shutdown_connections 2023-07-21T19:35:19.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad142d0 0x55b8aad17fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1a600 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad27ab0 0x55b8aad29ec0 crc :-1 s=READY pgs=513 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 --2- 172.21.15.5:0/666814618 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad17880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 >> 172.21.15.5:0/666814618 conn(0x55b8aad03be0 msgr2=0x55b8aad07c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3436ffd700 0 client.30089 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.728+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 shutdown_connections 2023-07-21T19:35:19.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.729+0000 7f3471a7db80 1 -- 172.21.15.5:0/666814618 wait complete. 2023-07-21T19:35:19.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.729+0000 7f3471a7db80 20 client.30089 trim_cache size 0 max 16384 2023-07-21T19:35:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.751+0000 7f3471a7db80 1 Processor -- start 2023-07-21T19:35:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.751+0000 7f3471a7db80 1 -- start start 2023-07-21T19:35:19.738 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad17830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad08c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aac58e50 con 0x55b8aad0e580 2023-07-21T19:35:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aad09b20 con 0x55b8aab6a6b0 2023-07-21T19:35:19.739 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3471a7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aad146a0 con 0x55b8aad03be0 2023-07-21T19:35:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f344ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad17830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f34577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f34577fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37812/0 (socket says 172.21.15.5:37812) 2023-07-21T19:35:19.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f34577fe700 1 -- 172.21.15.5:0/3125402214 learned_addr learned my addr 172.21.15.5:0/3125402214 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.752+0000 7f3457fff700 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad08c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 -- 172.21.15.5:0/3125402214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 msgr2=0x55b8aad17830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad17830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 -- 172.21.15.5:0/3125402214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 msgr2=0x55b8aad08c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad08c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 -- 172.21.15.5:0/3125402214 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b8aaab41f0 con 0x55b8aad03be0 2023-07-21T19:35:19.741 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.753+0000 7f34577fe700 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3448099fb0 tx=0x7f34480985d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e5412ebf9fc73f26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.754+0000 7f344e7fc700 1 -- 172.21.15.5:0/3125402214 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34480b1070 con 0x55b8aad03be0 2023-07-21T19:35:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.754+0000 7f344e7fc700 1 -- 172.21.15.5:0/3125402214 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55b8aabb9490 con 0x55b8aad03be0 2023-07-21T19:35:19.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.754+0000 7f344e7fc700 1 -- 172.21.15.5:0/3125402214 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3448099510 con 0x55b8aad03be0 2023-07-21T19:35:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 -- 172.21.15.5:0/3125402214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 msgr2=0x55b8aad0e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:19.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f3448099fb0 tx=0x7f34480985d0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 -- 172.21.15.5:0/3125402214 shutdown_connections 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aad0e580 0x55b8aad08c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad03be0 0x55b8aad0e040 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 --2- 172.21.15.5:0/3125402214 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aab6a6b0 0x55b8aad17830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 -- 172.21.15.5:0/3125402214 >> 172.21.15.5:0/3125402214 conn(0x55b8aad24170 msgr2=0x55b8aad17120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 -- 172.21.15.5:0/3125402214 shutdown_connections 2023-07-21T19:35:19.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.755+0000 7f3471a7db80 1 -- 172.21.15.5:0/3125402214 wait complete. 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.756+0000 7f3471a7db80 1 -- start start 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.756+0000 7f3471a7db80 1 Processor -- start 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 0x55b8aad1c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aad7a9e0 con 0x55b8aab6a6b0 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aad7ab50 con 0x55b8aad03be0 2023-07-21T19:35:19.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b8aad7ad10 con 0x55b8aad0e580 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3471a7db80 10 client.0 _ll_register_callbacks cb 0x7f3448099e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f34577fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3457fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3457fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:37814/0 (socket says 172.21.15.5:37814) 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f3457fff700 1 -- 172.21.15.5:0/735485170 learned_addr learned my addr 172.21.15.5:0/735485170 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:19.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.757+0000 7f344ffff700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 0x55b8aad1c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 -- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 msgr2=0x55b8aad1c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 0x55b8aad1c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 -- 172.21.15.5:0/735485170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 msgr2=0x55b8aad1c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:19.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 -- 172.21.15.5:0/735485170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b8aaa76c00 con 0x55b8aad0e580 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f34577fe700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f344ffff700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 0x55b8aad1c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3457fff700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3450005430 tx=0x7f34500057f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=53d4a1d00a4592b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3436ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3450017040 con 0x55b8aad0e580 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f344001e040 con 0x55b8aad0e580 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.758+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f345001c680 con 0x55b8aad0e580 2023-07-21T19:35:19.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3450004020 con 0x55b8aad0e580 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 10 client.30019 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55b8aaa8c250 con 0x55b8aad0e580 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f344001e040 con 0x55b8aad0e580 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f34500036f0 con 0x55b8aad0e580 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 10 client.30019 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 10 client.30019 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:19.750 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 20 client.30019 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad142d0 0x55b8aad7e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 4 client.30019 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.759+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b8aac1aff0 con 0x55b8aad142d0 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.760+0000 7f34577fe700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad142d0 0x55b8aad7e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.761+0000 7f34577fe700 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad142d0 0x55b8aad7e500 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bb9dcd21fbbf7a1 server_cookie=7291d281bce2e8f in_seq=0 out_seq=0 2023-07-21T19:35:19.751 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:19.761+0000 7f3436ffd700 10 client.30019 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.804+0000 7f3436ffd700 1 -- 172.21.15.5:0/735485170 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b8aac1aff0 con 0x55b8aad142d0 2023-07-21T19:35:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.804+0000 7f3436ffd700 10 client.30019 handle_command_reply: tid=0 2023-07-21T19:35:20.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.804+0000 7f3471a7db80 1 client.30019 shutdown 2023-07-21T19:35:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 msgr2=0x55b8aad1cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f3450005430 tx=0x7f34500057f0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 shutdown_connections 2023-07-21T19:35:20.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b8aad0e580 0x55b8aad1cd60 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b8aad142d0 0x55b8aad7e500 crc :-1 s=READY pgs=514 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b8aab6a6b0 0x55b8aad1c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 --2- 172.21.15.5:0/735485170 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b8aad03be0 0x55b8aad1c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3436ffd700 0 client.30019 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.805+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 >> 172.21.15.5:0/735485170 conn(0x55b8aad24170 msgr2=0x55b8aad14a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.806+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 shutdown_connections 2023-07-21T19:35:20.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.806+0000 7f3471a7db80 1 -- 172.21.15.5:0/735485170 wait complete. 2023-07-21T19:35:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.809+0000 7f3471a7db80 20 client.30019 trim_cache size 0 max 16384 2023-07-21T19:35:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:35:20.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:35:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:35:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:35:20.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 msgr2=0x7f34440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f3440006010 tx=0x7f3440005960 comp rx=0 tx=0).stop 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 msgr2=0x7f345819c150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f344800ef10 tx=0x7f344800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 shutdown_connections 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f34440a1340 0x7f34440a37f0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f34581109a0 0x7f34581a0d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3458074a40 0x7f345819c150 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 --2- 172.21.15.5:0/4003961660 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3458075350 0x7f345819c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.810+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 >> 172.21.15.5:0/4003961660 conn(0x7f34580fd7e0 msgr2=0x7f3458111600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:20.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.813+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 shutdown_connections 2023-07-21T19:35:20.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:20.814+0000 7f345e1ff700 1 -- 172.21.15.5:0/4003961660 wait complete. 2023-07-21T19:35:21.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:21 smithi005 ceph-mon[126460]: pgmap v880: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 540 op/s 2023-07-21T19:35:21.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:21 smithi156 ceph-mon[128316]: pgmap v880: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 540 op/s 2023-07-21T19:35:21.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 530.0 s, 220.4 tps, lat 459.888 ms stddev 179.813 2023-07-21T19:35:22.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:21 smithi101 ceph-mon[128136]: pgmap v880: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.1 MiB/s wr, 540 op/s 2023-07-21T19:35:23.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: pgmap v881: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 634 op/s 2023-07-21T19:35:23.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:23.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:23.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:35:23.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:35:23.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:23 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:23.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: pgmap v881: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 634 op/s 2023-07-21T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:35:23.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:23 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: pgmap v881: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.1 MiB/s wr, 634 op/s 2023-07-21T19:35:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:24.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:35:24.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:35:24.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:23 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:35:25.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:25 smithi005 ceph-mon[126460]: pgmap v882: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:25.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:25 smithi156 ceph-mon[128316]: pgmap v882: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:26.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:25 smithi101 ceph-mon[128136]: pgmap v882: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 535.0 s, 233.6 tps, lat 422.985 ms stddev 88.111 2023-07-21T19:35:27.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:27 smithi005 ceph-mon[126460]: pgmap v883: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:27.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:27 smithi156 ceph-mon[128316]: pgmap v883: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:28.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:27 smithi101 ceph-mon[128136]: pgmap v883: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.9 MiB/s wr, 715 op/s 2023-07-21T19:35:28.688 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/2560079878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 msgr2=0x7fbe94112f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/2560079878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe94112f70 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c009fc0 tx=0x7fbe8c00b330 comp rx=0 tx=0).stop 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/2560079878 shutdown_connections 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/2560079878 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe94112f70 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/2560079878 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410c240 0x7fbe9410c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/2560079878 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410cb50 0x7fbe94106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.018+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/2560079878 >> 172.21.15.5:0/2560079878 conn(0x7fbe940759e0 msgr2=0x7fbe94077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.019+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/2560079878 shutdown_connections 2023-07-21T19:35:29.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.019+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/2560079878 wait complete. 2023-07-21T19:35:29.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 Processor -- start 2023-07-21T19:35:29.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 -- start start 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe9419c320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410c240 0x7fbe9419c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a1300 con 0x7fbe9410cb50 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a1470 con 0x7fbe9410c240 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9b3ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe941a1630 con 0x7fbe94106a70 2023-07-21T19:35:29.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9999c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9999c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49880/0 (socket says 172.21.15.5:49880) 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9999c700 1 -- 172.21.15.5:0/1132968193 learned_addr learned my addr 172.21.15.5:0/1132968193 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.020+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410c240 0x7fbe9419c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe9419c320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 -- 172.21.15.5:0/1132968193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 msgr2=0x7fbe9419c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe9419c320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 -- 172.21.15.5:0/1132968193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410c240 msgr2=0x7fbe9419c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410c240 0x7fbe9419c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 -- 172.21.15.5:0/1132968193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe840095f0 con 0x7fbe9410cb50 2023-07-21T19:35:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9999c700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004f90 tx=0x7fbe8c008a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe8a7fc700 1 -- 172.21.15.5:0/1132968193 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c00e040 con 0x7fbe9410cb50 2023-07-21T19:35:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe8a7fc700 1 -- 172.21.15.5:0/1132968193 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe8c008ee0 con 0x7fbe9410cb50 2023-07-21T19:35:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe8a7fc700 1 -- 172.21.15.5:0/1132968193 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8c005040 con 0x7fbe9410cb50 2023-07-21T19:35:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.021+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe8c009c40 con 0x7fbe9410cb50 2023-07-21T19:35:29.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.022+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe941a8f70 con 0x7fbe9410cb50 2023-07-21T19:35:29.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.026+0000 7fbe8a7fc700 1 -- 172.21.15.5:0/1132968193 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fbe8c00bbf0 con 0x7fbe9410cb50 2023-07-21T19:35:29.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.027+0000 7fbeaedebb80 1 Processor -- start 2023-07-21T19:35:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 -- start start 2023-07-21T19:35:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d8582d0 0x564c0d8586a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 0x564c0d865eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d75ecb0 con 0x564c0d863ad0 2023-07-21T19:35:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d7b1a40 con 0x564c0d858be0 2023-07-21T19:35:29.015 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.028+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d7aaed0 con 0x564c0d8582d0 2023-07-21T19:35:29.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d8582d0 0x564c0d8586a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57600/0 (socket says 172.21.15.5:57600) 2023-07-21T19:35:29.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 -- 172.21.15.5:0/1627741181 learned_addr learned my addr 172.21.15.5:0/1627741181 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 0x564c0d865eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe8a7fc700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 0x7fbe800a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe8a7fc700 1 -- 172.21.15.5:0/1132968193 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbe8c0ce220 con 0x7fbe9410cb50 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 -- 172.21.15.5:0/1627741181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d8582d0 msgr2=0x564c0d8586a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d8582d0 0x564c0d8586a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 -- 172.21.15.5:0/1627741181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 msgr2=0x564c0d865eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 0x564c0d865eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9919b700 1 -- 172.21.15.5:0/1627741181 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0d5d8de0 con 0x564c0d858be0 2023-07-21T19:35:29.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.029+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 0x564c0d865eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:29.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.030+0000 7fbe9999c700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 0x7fbe800a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.030+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d30 tx=0x7fbe90011720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2cf5bacbbd9ed2bd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.031+0000 7fbe7f7fe700 1 -- 172.21.15.5:0/1627741181 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004220 con 0x564c0d858be0 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.031+0000 7fbe7f7fe700 1 -- 172.21.15.5:0/1627741181 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe90004380 con 0x564c0d858be0 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.031+0000 7fbe7f7fe700 1 -- 172.21.15.5:0/1627741181 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900197a0 con 0x564c0d858be0 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.032+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1627741181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 msgr2=0x564c0d863590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.032+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d30 tx=0x7fbe90011720 comp rx=0 tx=0).stop 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.032+0000 7fbe9999c700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 0x7fbe800a3870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c097820 tx=0x7fbe8c097040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1627741181 shutdown_connections 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d863ad0 0x564c0d865eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d8582d0 0x564c0d8586a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1627741181 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d863590 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1627741181 >> 172.21.15.5:0/1627741181 conn(0x564c0d84eda0 msgr2=0x564c0d8546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1627741181 shutdown_connections 2023-07-21T19:35:29.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.033+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1627741181 wait complete. 2023-07-21T19:35:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.034+0000 7fbeaedebb80 1 -- start start 2023-07-21T19:35:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.034+0000 7fbeaedebb80 1 Processor -- start 2023-07-21T19:35:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d8582d0 0x564c0d863580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d860800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d74aac0 con 0x564c0d8582d0 2023-07-21T19:35:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d74a7c0 con 0x564c0d858be0 2023-07-21T19:35:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d62e920 con 0x564c0d863ad0 2023-07-21T19:35:29.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.035+0000 7fbeaedebb80 10 client.0 _ll_register_callbacks cb 0x564c0d748760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.036+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.036+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39904/0 (socket says 172.21.15.5:39904) 2023-07-21T19:35:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.036+0000 7fbe9899a700 1 -- 172.21.15.5:0/494762297 learned_addr learned my addr 172.21.15.5:0/494762297 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:29.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.036+0000 7fbe9919b700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d860800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 -- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 msgr2=0x564c0d860800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d860800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 -- 172.21.15.5:0/494762297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d8582d0 msgr2=0x564c0d863580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d8582d0 0x564c0d863580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 -- 172.21.15.5:0/494762297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe84010750 con 0x564c0d863ad0 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe9899a700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbe840099b0 tx=0x7fbe8400f920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e923931898b3a98 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.037+0000 7fbe7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe8401e400 con 0x564c0d863ad0 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbe8401e560 con 0x564c0d863ad0 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c0d7044f0 con 0x564c0d863ad0 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8401e720 con 0x564c0d863ad0 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbeaedebb80 10 client.30031 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:29.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.038+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fbe90009960 con 0x564c0d863ad0 2023-07-21T19:35:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe8402b020 con 0x564c0d863ad0 2023-07-21T19:35:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe8400e3c0 con 0x564c0d863ad0 2023-07-21T19:35:29.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 10 client.30031 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 10 client.30031 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 20 client.30031 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d884420 0x564c0d886830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 4 client.30031 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.039+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c0d7a3eb0 con 0x564c0d884420 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.040+0000 7fbe9919b700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d884420 0x564c0d886830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.040+0000 7fbe9919b700 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d884420 0x564c0d886830 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8698d1731b84796 server_cookie=e149e188edc7c42e in_seq=0 out_seq=0 2023-07-21T19:35:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.040+0000 7fbe7cff9700 10 client.30031 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:29.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbe7cff9700 1 -- 172.21.15.5:0/494762297 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c0d7a3eb0 con 0x564c0d884420 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbe7cff9700 10 client.30031 handle_command_reply: tid=0 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 client.30031 shutdown 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 msgr2=0x564c0d860f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fbe840099b0 tx=0x7fbe8400f920 comp rx=0 tx=0).stop 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 shutdown_connections 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d860f40 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d884420 0x564c0d886830 crc :-1 s=READY pgs=515 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d8582d0 0x564c0d863580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/494762297 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d858be0 0x564c0d860800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbe7cff9700 0 client.30031 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.042+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 >> 172.21.15.5:0/494762297 conn(0x564c0d84eda0 msgr2=0x564c0d854290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.043+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 shutdown_connections 2023-07-21T19:35:29.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.043+0000 7fbeaedebb80 1 -- 172.21.15.5:0/494762297 wait complete. 2023-07-21T19:35:29.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.043+0000 7fbeaedebb80 20 client.30031 trim_cache size 0 max 16384 2023-07-21T19:35:29.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 Processor -- start 2023-07-21T19:35:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 -- start start 2023-07-21T19:35:29.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d85b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d85c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d880e20 con 0x564c0d858be0 2023-07-21T19:35:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d85ed60 con 0x564c0d8582d0 2023-07-21T19:35:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.074+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d886800 con 0x564c0d863ad0 2023-07-21T19:35:29.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d85c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49900/0 (socket says 172.21.15.5:49900) 2023-07-21T19:35:29.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 -- 172.21.15.5:0/2625116429 learned_addr learned my addr 172.21.15.5:0/2625116429 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 -- 172.21.15.5:0/2625116429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 msgr2=0x564c0d85c070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d85c070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 -- 172.21.15.5:0/2625116429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 msgr2=0x564c0d85b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d85b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 -- 172.21.15.5:0/2625116429 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c0d5c1c60 con 0x564c0d858be0 2023-07-21T19:35:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.075+0000 7fbe9919b700 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbe90014040 tx=0x7fbe90005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2156c2b1093be0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.076+0000 7fbe7ffff700 1 -- 172.21.15.5:0/2625116429 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004110 con 0x564c0d858be0 2023-07-21T19:35:29.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.076+0000 7fbe7ffff700 1 -- 172.21.15.5:0/2625116429 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe9002d070 con 0x564c0d858be0 2023-07-21T19:35:29.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.076+0000 7fbe7ffff700 1 -- 172.21.15.5:0/2625116429 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe900055c0 con 0x564c0d858be0 2023-07-21T19:35:29.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:28 smithi101 ceph-mon[128136]: pgmap v884: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 -- 172.21.15.5:0/2625116429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 msgr2=0x564c0d85bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fbe90014040 tx=0x7fbe90005ce0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 -- 172.21.15.5:0/2625116429 shutdown_connections 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d85c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d85b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/2625116429 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d85bb30 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 -- 172.21.15.5:0/2625116429 >> 172.21.15.5:0/2625116429 conn(0x564c0d84eda0 msgr2=0x564c0d880710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 -- 172.21.15.5:0/2625116429 shutdown_connections 2023-07-21T19:35:29.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.077+0000 7fbeaedebb80 1 -- 172.21.15.5:0/2625116429 wait complete. 2023-07-21T19:35:29.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.078+0000 7fbeaedebb80 1 -- start start 2023-07-21T19:35:29.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.078+0000 7fbeaedebb80 1 Processor -- start 2023-07-21T19:35:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d868690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d868bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d8691f0 con 0x564c0d858be0 2023-07-21T19:35:29.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d869330 con 0x564c0d8582d0 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c0d8694f0 con 0x564c0d863ad0 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbeaedebb80 10 client.0 _ll_register_callbacks cb 0x564c0d748760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbe9899a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57620/0 (socket says 172.21.15.5:57620) 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbe9899a700 1 -- 172.21.15.5:0/1814157279 learned_addr learned my addr 172.21.15.5:0/1814157279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:29.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.079+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d868bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d868690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9899a700 1 -- 172.21.15.5:0/1814157279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 msgr2=0x564c0d868bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d868bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9899a700 1 -- 172.21.15.5:0/1814157279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 msgr2=0x564c0d868690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d868690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9899a700 1 -- 172.21.15.5:0/1814157279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c0d5d8de0 con 0x564c0d8582d0 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.080+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d868690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbe9899a700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbe84010e40 tx=0x7fbe84005ce0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ece5e63a1f5ed2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:29.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbe7cff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:35:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84005220 con 0x564c0d8582d0 2023-07-21T19:35:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fbe84000e40 con 0x564c0d8582d0 2023-07-21T19:35:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbe90004380 con 0x564c0d8582d0 2023-07-21T19:35:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.081+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbe8400ee60 con 0x564c0d8582d0 2023-07-21T19:35:29.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.082+0000 7fbeaedebb80 10 client.30110 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.082+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564c0d5ff250 con 0x564c0d8582d0 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.082+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fbe90004380 con 0x564c0d8582d0 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fbe8400f640 con 0x564c0d8582d0 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 10 client.30110 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 10 client.30110 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 20 client.30110 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:29.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d8d97c0 0x564c0d8dbbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 4 client.30110 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:35:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.083+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c0d766050 con 0x564c0d8d97c0 2023-07-21T19:35:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.084+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d8d97c0 0x564c0d8dbbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbe9919b700 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d8d97c0 0x564c0d8dbbd0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=298fc46f951484a9 server_cookie=81e84a5f6b366d41 in_seq=0 out_seq=0 2023-07-21T19:35:29.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbe7cff9700 10 client.30110 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:29.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbe7cff9700 1 -- 172.21.15.5:0/1814157279 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c0d766050 con 0x564c0d8d97c0 2023-07-21T19:35:29.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbe7cff9700 10 client.30110 handle_command_reply: tid=0 2023-07-21T19:35:29.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbeaedebb80 1 client.30110 shutdown 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 msgr2=0x564c0d86a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.086+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbe84010e40 tx=0x7fbe84005ce0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 shutdown_connections 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c0d863ad0 0x564c0d868bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c0d8582d0 0x564c0d86a3d0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c0d8d97c0 0x564c0d8dbbd0 crc :-1 s=READY pgs=516 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 --2- 172.21.15.5:0/1814157279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c0d858be0 0x564c0d868690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 >> 172.21.15.5:0/1814157279 conn(0x564c0d84eda0 msgr2=0x564c0d85bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbe7cff9700 0 client.30110 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 shutdown_connections 2023-07-21T19:35:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 1 -- 172.21.15.5:0/1814157279 wait complete. 2023-07-21T19:35:29.087 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.087+0000 7fbeaedebb80 20 client.30110 trim_cache size 0 max 16384 2023-07-21T19:35:29.087 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:35:29.088 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:35:29.089 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:35:29.089 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:35:29.089 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 msgr2=0x7fbe800a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 0x7fbe800a3870 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c097820 tx=0x7fbe8c097040 comp rx=0 tx=0).stop 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 msgr2=0x7fbe941a0cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fbe8c004f90 tx=0x7fbe8c008a70 comp rx=0 tx=0).stop 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 shutdown_connections 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fbe800a13c0 0x7fbe800a3870 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fbe9410cb50 0x7fbe941a0cc0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.090 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fbe94106a70 0x7fbe9419c320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 --2- 172.21.15.5:0/1132968193 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fbe9410c240 0x7fbe9419c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:29.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 >> 172.21.15.5:0/1132968193 conn(0x7fbe940759e0 msgr2=0x7fbe94077080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:29.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.088+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 shutdown_connections 2023-07-21T19:35:29.091 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:29.089+0000 7fbe9b3ff700 1 -- 172.21.15.5:0/1132968193 wait complete. 2023-07-21T19:35:29.096 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:28 smithi005 ceph-mon[126460]: pgmap v884: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:35:29.099 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:35:29.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:28 smithi156 ceph-mon[128316]: pgmap v884: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:35:31.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:31 smithi005 ceph-mon[126460]: pgmap v885: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2023-07-21T19:35:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:31 smithi156 ceph-mon[128316]: pgmap v885: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2023-07-21T19:35:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 540.0 s, 272.4 tps, lat 370.715 ms stddev 55.070 2023-07-21T19:35:32.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:31 smithi101 ceph-mon[128136]: pgmap v885: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.2 MiB/s wr, 630 op/s 2023-07-21T19:35:33.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:33 smithi005 ceph-mon[126460]: pgmap v886: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 852 op/s 2023-07-21T19:35:33.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:33 smithi156 ceph-mon[128316]: pgmap v886: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 852 op/s 2023-07-21T19:35:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 -- 172.21.15.5:0/3432793972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c067c10 msgr2=0x7f588c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 --2- 172.21.15.5:0/3432793972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c067c10 0x7f588c067fe0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f58880099c0 tx=0x7f5888009cd0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 -- 172.21.15.5:0/3432793972 shutdown_connections 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 --2- 172.21.15.5:0/3432793972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c104e50 0x7f588c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 --2- 172.21.15.5:0/3432793972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c067c10 0x7f588c067fe0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 --2- 172.21.15.5:0/3432793972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.040+0000 7f58947ff700 1 -- 172.21.15.5:0/3432793972 >> 172.21.15.5:0/3432793972 conn(0x7f588c074760 msgr2=0x7f588c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.041+0000 7f58947ff700 1 -- 172.21.15.5:0/3432793972 shutdown_connections 2023-07-21T19:35:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.041+0000 7f58947ff700 1 -- 172.21.15.5:0/3432793972 wait complete. 2023-07-21T19:35:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.042+0000 7f58947ff700 1 Processor -- start 2023-07-21T19:35:34.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.042+0000 7f58947ff700 1 -- start start 2023-07-21T19:35:34.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c104e50 0x7f588c1959a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c196010 con 0x7f588c0685b0 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c196180 con 0x7f588c104e50 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f58947ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f588c196340 con 0x7f588c067c10 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f5891d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f589259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f589259b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39948/0 (socket says 172.21.15.5:39948) 2023-07-21T19:35:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f589259b700 1 -- 172.21.15.5:0/3621032494 learned_addr learned my addr 172.21.15.5:0/3621032494 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f5891d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49930/0 (socket says 172.21.15.5:49930) 2023-07-21T19:35:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.043+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c104e50 0x7f588c1959a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 -- 172.21.15.5:0/3621032494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c104e50 msgr2=0x7f588c1959a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c104e50 0x7f588c1959a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 -- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 msgr2=0x7f588c19c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 -- 172.21.15.5:0/3621032494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c009520 con 0x7f588c067c10 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f5891d9a700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.044+0000 7f589259b700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588800f670 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.045+0000 7f58837fe700 1 -- 172.21.15.5:0/3621032494 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588801d070 con 0x7f588c067c10 2023-07-21T19:35:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.045+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58880095f0 con 0x7f588c067c10 2023-07-21T19:35:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.045+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c1a2950 con 0x7f588c067c10 2023-07-21T19:35:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.045+0000 7f58837fe700 1 -- 172.21.15.5:0/3621032494 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5888005030 con 0x7f588c067c10 2023-07-21T19:35:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.045+0000 7f58837fe700 1 -- 172.21.15.5:0/3621032494 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588802a610 con 0x7f588c067c10 2023-07-21T19:35:34.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.050+0000 7f58837fe700 1 -- 172.21.15.5:0/3621032494 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f588800fa30 con 0x7f588c067c10 2023-07-21T19:35:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.051+0000 7f58a81eeb80 1 Processor -- start 2023-07-21T19:35:34.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.051+0000 7f58a81eeb80 1 -- start start 2023-07-21T19:35:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb795040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7959d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb69ac50 con 0x55a5bb795580 2023-07-21T19:35:34.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb6ed9e0 con 0x55a5bb79b2d0 2023-07-21T19:35:34.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb6e6e70 con 0x55a5bb5f16b0 2023-07-21T19:35:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f5892d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f5892d9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39952/0 (socket says 172.21.15.5:39952) 2023-07-21T19:35:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f5892d9c700 1 -- 172.21.15.5:0/53458168 learned_addr learned my addr 172.21.15.5:0/53458168 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.052+0000 7f5891d9a700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7959d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 -- 172.21.15.5:0/53458168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 msgr2=0x55a5bb795040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f58837fe700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 0x7f58780a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f589259b700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb795040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb795040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 -- 172.21.15.5:0/53458168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 msgr2=0x55a5bb7959d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7959d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 -- 172.21.15.5:0/53458168 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5bb514d80 con 0x55a5bb5f16b0 2023-07-21T19:35:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5891d9a700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:35:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f58837fe700 1 -- 172.21.15.5:0/3621032494 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f58880cd7c0 con 0x7f588c067c10 2023-07-21T19:35:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f589259b700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb795040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:35:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.053+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 0x7f58780a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.054+0000 7f5892d9c700 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5884011600 tx=0x7f5884009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=171687ccb4018198 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.054+0000 7f5880ff9700 1 -- 172.21.15.5:0/53458168 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f588400e040 con 0x55a5bb5f16b0 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.054+0000 7f5880ff9700 1 -- 172.21.15.5:0/53458168 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f588400f040 con 0x55a5bb5f16b0 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.054+0000 7f5880ff9700 1 -- 172.21.15.5:0/53458168 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884015740 con 0x55a5bb5f16b0 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.054+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 0x7f58780a37f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5884009510 tx=0x7f5884023000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 -- 172.21.15.5:0/53458168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 msgr2=0x55a5bb79ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f5884011600 tx=0x7f5884009920 comp rx=0 tx=0).stop 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 -- 172.21.15.5:0/53458168 shutdown_connections 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7959d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb5f16b0 0x55a5bb79ad90 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/53458168 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb795040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.055+0000 7f58a81eeb80 1 -- 172.21.15.5:0/53458168 >> 172.21.15.5:0/53458168 conn(0x55a5bb78abe0 msgr2=0x55a5bb78ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.056+0000 7f58a81eeb80 1 -- 172.21.15.5:0/53458168 shutdown_connections 2023-07-21T19:35:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.056+0000 7f58a81eeb80 1 -- 172.21.15.5:0/53458168 wait complete. 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.056+0000 7f58a81eeb80 1 -- start start 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 Processor -- start 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a14f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb686a60 con 0x55a5bb5f16b0 2023-07-21T19:35:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb686760 con 0x55a5bb79b2d0 2023-07-21T19:35:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.057+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb56a8c0 con 0x55a5bb795580 2023-07-21T19:35:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f58a81eeb80 10 client.0 _ll_register_callbacks cb 0x55a5bb684700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:34.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f589259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39954/0 (socket says 172.21.15.5:39954) 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 -- 172.21.15.5:0/2039483243 learned_addr learned my addr 172.21.15.5:0/2039483243 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5892d9c700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 -- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 msgr2=0x55a5bb79ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 -- 172.21.15.5:0/2039483243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 msgr2=0x55a5bb7a14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a14f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.058+0000 7f5891d9a700 1 -- 172.21.15.5:0/2039483243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58840112b0 con 0x55a5bb795580 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.059+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f587c011d10 tx=0x7f587c00f6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa1db502803a6cea server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.059+0000 7f5875ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.059+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587c016070 con 0x55a5bb795580 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.059+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f587c00fd70 con 0x55a5bb795580 2023-07-21T19:35:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.059+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a5bb640490 con 0x55a5bb795580 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f587c004030 con 0x55a5bb795580 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f58a81eeb80 10 client.30049 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55a5bb4fdc00 con 0x55a5bb795580 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f587c027020 con 0x55a5bb795580 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f587c003c50 con 0x55a5bb795580 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f58a81eeb80 10 client.30049 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f58a81eeb80 10 client.30049 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.060+0000 7f58a81eeb80 20 client.30049 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.061+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb7b2ff0 0x55a5bb7b5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.061+0000 7f58a81eeb80 4 client.30049 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.061+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a5bb6dfe50 con 0x55a5bb7b2ff0 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.061+0000 7f589259b700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb7b2ff0 0x55a5bb7b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.062+0000 7f589259b700 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb7b2ff0 0x55a5bb7b5400 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=475235232ef594df server_cookie=6e735ccc75d73227 in_seq=0 out_seq=0 2023-07-21T19:35:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.062+0000 7f5875ffb700 10 client.30049 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f5875ffb700 1 -- 172.21.15.5:0/2039483243 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a5bb6dfe50 con 0x55a5bb7b2ff0 2023-07-21T19:35:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f5875ffb700 10 client.30049 handle_command_reply: tid=0 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 client.30049 shutdown 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 msgr2=0x55a5bb79e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f587c011d10 tx=0x7f587c00f6a0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 shutdown_connections 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb7b2ff0 0x55a5bb7b5400 crc :-1 s=READY pgs=517 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2039483243 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb79e770 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 >> 172.21.15.5:0/2039483243 conn(0x55a5bb78abe0 msgr2=0x55a5bb78ec50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.063+0000 7f5875ffb700 0 client.30049 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.064+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 shutdown_connections 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.064+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2039483243 wait complete. 2023-07-21T19:35:34.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.064+0000 7f58a81eeb80 20 client.30049 trim_cache size 0 max 16384 2023-07-21T19:35:34.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:33 smithi101 ceph-mon[128136]: pgmap v886: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 8.7 MiB/s wr, 852 op/s 2023-07-21T19:35:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.086+0000 7f58a81eeb80 1 Processor -- start 2023-07-21T19:35:34.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.086+0000 7f58a81eeb80 1 -- start start 2023-07-21T19:35:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7aa070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79df30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb7b05c0 con 0x55a5bb795580 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb790fb0 con 0x55a5bb5f16b0 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb79ccd0 con 0x55a5bb79b2d0 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f589259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f589259b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57658/0 (socket says 172.21.15.5:57658) 2023-07-21T19:35:34.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f589259b700 1 -- 172.21.15.5:0/3016686915 learned_addr learned my addr 172.21.15.5:0/3016686915 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f5891d9a700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.087+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79df30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 -- 172.21.15.5:0/3016686915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 msgr2=0x55a5bb795040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 -- 172.21.15.5:0/3016686915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 msgr2=0x55a5bb7aa070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7aa070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 -- 172.21.15.5:0/3016686915 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a5bb53b1f0 con 0x55a5bb79b2d0 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f589259b700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.088+0000 7f5892d9c700 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79df30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5884009110 tx=0x7f5884009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e7a096ae218a5806 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.089+0000 7f58817fa700 1 -- 172.21.15.5:0/3016686915 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884015650 con 0x55a5bb79b2d0 2023-07-21T19:35:34.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.089+0000 7f58817fa700 1 -- 172.21.15.5:0/3016686915 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5884014e80 con 0x55a5bb79b2d0 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.089+0000 7f58817fa700 1 -- 172.21.15.5:0/3016686915 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5884007760 con 0x55a5bb79b2d0 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- 172.21.15.5:0/3016686915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 msgr2=0x55a5bb79df30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79df30 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f5884009110 tx=0x7f5884009400 comp rx=0 tx=0).stop 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- 172.21.15.5:0/3016686915 shutdown_connections 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb79b2d0 0x55a5bb79df30 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb5f16b0 0x55a5bb795040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/3016686915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb795580 0x55a5bb7aa070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- 172.21.15.5:0/3016686915 >> 172.21.15.5:0/3016686915 conn(0x55a5bb78abe0 msgr2=0x55a5bb79e730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- 172.21.15.5:0/3016686915 shutdown_connections 2023-07-21T19:35:34.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- 172.21.15.5:0/3016686915 wait complete. 2023-07-21T19:35:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.090+0000 7f58a81eeb80 1 -- start start 2023-07-21T19:35:34.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 Processor -- start 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb7a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb7a66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb7a6c30 con 0x55a5bb5f16b0 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb7a6da0 con 0x55a5bb79b2d0 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.091+0000 7f58a81eeb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a5bb7a6f10 con 0x55a5bb795580 2023-07-21T19:35:34.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f58a81eeb80 10 client.0 _ll_register_callbacks cb 0x7f587c0117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:34.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f5891d9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb7a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49974/0 (socket says 172.21.15.5:49974) 2023-07-21T19:35:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 -- 172.21.15.5:0/2369318396 learned_addr learned my addr 172.21.15.5:0/2369318396 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f5892d9c700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb7a66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 -- 172.21.15.5:0/2369318396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 msgr2=0x55a5bb7a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb7a61b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 -- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 msgr2=0x55a5bb7a66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb7a66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f589259b700 1 -- 172.21.15.5:0/2369318396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f588c1a2950 con 0x55a5bb5f16b0 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb7a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:34.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.092+0000 7f5892d9c700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb7a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f589259b700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5888096dc0 tx=0x7f5888009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d76fc8307b0362 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f5875ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58880cb330 con 0x55a5bb5f16b0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f588400f040 con 0x55a5bb5f16b0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f58880ba0c0 con 0x55a5bb5f16b0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f58880ba280 con 0x55a5bb5f16b0 2023-07-21T19:35:34.086 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f58a81eeb80 10 client.20337 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.093+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55a5bb514d80 con 0x55a5bb5f16b0 2023-07-21T19:35:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f588400f040 con 0x55a5bb5f16b0 2023-07-21T19:35:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f58880b5050 con 0x55a5bb5f16b0 2023-07-21T19:35:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 10 client.20337 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:34.087 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 10 client.20337 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 20 client.20337 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb807f10 0x55a5bb80a320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 4 client.20337 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:35:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.094+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a5bb6a1ff0 con 0x55a5bb807f10 2023-07-21T19:35:34.088 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.095+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb807f10 0x55a5bb80a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.095+0000 7f5875ffb700 10 client.20337 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:34.089 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.095+0000 7f5891d9a700 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb807f10 0x55a5bb80a320 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bbbc354193b1207 server_cookie=ed92c320c649ceb8 in_seq=0 out_seq=0 2023-07-21T19:35:34.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.126+0000 7f5875ffb700 1 -- 172.21.15.5:0/2369318396 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a5bb6a1ff0 con 0x55a5bb807f10 2023-07-21T19:35:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.126+0000 7f5875ffb700 10 client.20337 handle_command_reply: tid=0 2023-07-21T19:35:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.126+0000 7f58a81eeb80 1 client.20337 shutdown 2023-07-21T19:35:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 msgr2=0x55a5bb7a9ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7f5888096dc0 tx=0x7f5888009310 comp rx=0 tx=0).stop 2023-07-21T19:35:34.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 shutdown_connections 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a5bb79b2d0 0x55a5bb7a66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a5bb807f10 0x55a5bb80a320 crc :-1 s=READY pgs=518 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a5bb5f16b0 0x55a5bb7a9ea0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 --2- 172.21.15.5:0/2369318396 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a5bb795580 0x55a5bb7a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f5875ffb700 0 client.20337 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 >> 172.21.15.5:0/2369318396 conn(0x55a5bb78abe0 msgr2=0x55a5bb7a0910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 shutdown_connections 2023-07-21T19:35:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 1 -- 172.21.15.5:0/2369318396 wait complete. 2023-07-21T19:35:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.127+0000 7f58a81eeb80 20 client.20337 trim_cache size 0 max 16384 2023-07-21T19:35:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:35:34.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:35:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:35:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:35:34.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 msgr2=0x7f58780a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 0x7f58780a37f0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5884009510 tx=0x7f5884023000 comp rx=0 tx=0).stop 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 msgr2=0x7f588c19bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f5888006010 tx=0x7f588800f670 comp rx=0 tx=0).stop 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 shutdown_connections 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f588c104e50 0x7f588c1959a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f588c067c10 0x7f588c19bb40 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f58780a1340 0x7f58780a37f0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 --2- 172.21.15.5:0/3621032494 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f588c0685b0 0x7f588c19c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.128+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 >> 172.21.15.5:0/3621032494 conn(0x7f588c074760 msgr2=0x7f588c0fc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.129+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 shutdown_connections 2023-07-21T19:35:34.119 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:34.129+0000 7f58947ff700 1 -- 172.21.15.5:0/3621032494 wait complete. 2023-07-21T19:35:35.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:35 smithi005 ceph-mon[126460]: pgmap v887: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.7 MiB/s wr, 829 op/s 2023-07-21T19:35:35.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:35 smithi156 ceph-mon[128316]: pgmap v887: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.7 MiB/s wr, 829 op/s 2023-07-21T19:35:36.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:35 smithi101 ceph-mon[128136]: pgmap v887: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.7 MiB/s wr, 829 op/s 2023-07-21T19:35:36.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 545.0 s, 279.8 tps, lat 356.865 ms stddev 52.408 2023-07-21T19:35:37.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:37 smithi005 ceph-mon[126460]: pgmap v888: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 689 op/s 2023-07-21T19:35:37.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:37 smithi156 ceph-mon[128316]: pgmap v888: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 689 op/s 2023-07-21T19:35:38.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:37 smithi101 ceph-mon[128136]: pgmap v888: 97 pgs: 97 active+clean; 11 GiB data, 16 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 689 op/s 2023-07-21T19:35:39.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:38 smithi005 ceph-mon[126460]: pgmap v889: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:35:39.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:38 smithi156 ceph-mon[128316]: pgmap v889: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:35:39.320 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:38 smithi101 ceph-mon[128136]: pgmap v889: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.07k op/s 2023-07-21T19:35:41.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:41 smithi005 ceph-mon[126460]: pgmap v890: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 739 op/s 2023-07-21T19:35:41.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:41 smithi156 ceph-mon[128316]: pgmap v890: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 739 op/s 2023-07-21T19:35:41.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 550.0 s, 246.4 tps, lat 407.122 ms stddev 122.138 2023-07-21T19:35:42.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:41 smithi101 ceph-mon[128136]: pgmap v890: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 739 op/s 2023-07-21T19:35:43.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:43 smithi005 ceph-mon[126460]: pgmap v891: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 918 op/s 2023-07-21T19:35:43.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:43 smithi156 ceph-mon[128316]: pgmap v891: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 918 op/s 2023-07-21T19:35:44.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:43 smithi101 ceph-mon[128136]: pgmap v891: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 918 op/s 2023-07-21T19:35:45.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:45 smithi005 ceph-mon[126460]: pgmap v892: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 865 op/s 2023-07-21T19:35:45.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:45 smithi156 ceph-mon[128316]: pgmap v892: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 865 op/s 2023-07-21T19:35:46.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:45 smithi101 ceph-mon[128136]: pgmap v892: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 865 op/s 2023-07-21T19:35:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 555.0 s, 240.0 tps, lat 394.884 ms stddev 94.751 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.291+0000 7f6837bff700 1 -- 172.21.15.5:0/2205881063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68300685b0 msgr2=0x7f6830104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.291+0000 7f6837bff700 1 --2- 172.21.15.5:0/2205881063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68300685b0 0x7f6830104770 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f682c0099c0 tx=0x7f682c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 -- 172.21.15.5:0/2205881063 shutdown_connections 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 --2- 172.21.15.5:0/2205881063 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830104e50 0x7f683010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 --2- 172.21.15.5:0/2205881063 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6830067c10 0x7f6830067fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 --2- 172.21.15.5:0/2205881063 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f68300685b0 0x7f6830104770 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 -- 172.21.15.5:0/2205881063 >> 172.21.15.5:0/2205881063 conn(0x7f6830074760 msgr2=0x7f6830074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 -- 172.21.15.5:0/2205881063 shutdown_connections 2023-07-21T19:35:47.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.292+0000 7f6837bff700 1 -- 172.21.15.5:0/2205881063 wait complete. 2023-07-21T19:35:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.293+0000 7f6837bff700 1 Processor -- start 2023-07-21T19:35:47.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.293+0000 7f6837bff700 1 -- start start 2023-07-21T19:35:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68300685b0 0x7f683019e220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 0x7f6830197b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301981b0 con 0x7f6830104e50 2023-07-21T19:35:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6830198320 con 0x7f68300685b0 2023-07-21T19:35:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.294+0000 7f6837bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68301984e0 con 0x7f6830067c10 2023-07-21T19:35:47.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34864/0 (socket says 172.21.15.5:34864) 2023-07-21T19:35:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 -- 172.21.15.5:0/4290581190 learned_addr learned my addr 172.21.15.5:0/4290581190 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683519a700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68300685b0 0x7f683019e220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683619c700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 0x7f6830197b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 -- 172.21.15.5:0/4290581190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68300685b0 msgr2=0x7f683019e220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68300685b0 0x7f683019e220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 -- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 msgr2=0x7f6830197b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 0x7f6830197b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683599b700 1 -- 172.21.15.5:0/4290581190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f682c0095f0 con 0x7f6830067c10 2023-07-21T19:35:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.295+0000 7f683619c700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 0x7f6830197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:35:47.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f683599b700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9a0 tx=0x7f682400ed60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f6822ffd700 1 -- 172.21.15.5:0/4290581190 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824009790 con 0x7f6830067c10 2023-07-21T19:35:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f6822ffd700 1 -- 172.21.15.5:0/4290581190 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f682400cbd0 con 0x7f6830067c10 2023-07-21T19:35:47.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f6822ffd700 1 -- 172.21.15.5:0/4290581190 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6824018680 con 0x7f6830067c10 2023-07-21T19:35:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68301987c0 con 0x7f6830067c10 2023-07-21T19:35:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.296+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830105a40 con 0x7f6830067c10 2023-07-21T19:35:47.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.303+0000 7f6822ffd700 1 -- 172.21.15.5:0/4290581190 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f682400cd40 con 0x7f6830067c10 2023-07-21T19:35:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.305+0000 7f6822ffd700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 0x7f681c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.305+0000 7f684b5d8b80 1 Processor -- start 2023-07-21T19:35:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.305+0000 7f6822ffd700 1 -- 172.21.15.5:0/4290581190 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6824014070 con 0x7f6830067c10 2023-07-21T19:35:47.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 -- start start 2023-07-21T19:35:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f683519a700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 0x7f681c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe64c2d0 0x55eebe646040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe6469d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe54bc50 con 0x55eebe4a26b0 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe59e9e0 con 0x55eebe646580 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.306+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe597e70 con 0x55eebe64c2d0 2023-07-21T19:35:47.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:43392/0 (socket says 172.21.15.5:43392) 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 -- 172.21.15.5:0/3301318833 learned_addr learned my addr 172.21.15.5:0/3301318833 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683619c700 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe6469d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683519a700 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe64c2d0 0x55eebe646040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 -- 172.21.15.5:0/3301318833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe64c2d0 msgr2=0x55eebe646040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe64c2d0 0x55eebe646040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 -- 172.21.15.5:0/3301318833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 msgr2=0x55eebe6469d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe6469d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 -- 172.21.15.5:0/3301318833 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eebe3c5d80 con 0x55eebe4a26b0 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683519a700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 0x7f681c0a37f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f682c009990 tx=0x7f682c000bc0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.307+0000 7f683599b700 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f68240a1fd0 tx=0x7f6824099d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d51c3f420e8c7a2a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f681bfff700 1 -- 172.21.15.5:0/3301318833 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a2500 con 0x55eebe4a26b0 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f681bfff700 1 -- 172.21.15.5:0/3301318833 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f682409a310 con 0x55eebe4a26b0 2023-07-21T19:35:47.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f681bfff700 1 -- 172.21.15.5:0/3301318833 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a2290 con 0x55eebe4a26b0 2023-07-21T19:35:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3301318833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 msgr2=0x55eebe64bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7f68240a1fd0 tx=0x7f6824099d20 comp rx=0 tx=0).stop 2023-07-21T19:35:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3301318833 shutdown_connections 2023-07-21T19:35:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe6469d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe4a26b0 0x55eebe64bd90 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3301318833 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe64c2d0 0x55eebe646040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.308+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3301318833 >> 172.21.15.5:0/3301318833 conn(0x55eebe63bbe0 msgr2=0x55eebe640f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.309+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3301318833 shutdown_connections 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.309+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3301318833 wait complete. 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.309+0000 7f684b5d8b80 1 -- start start 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 Processor -- start 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe64fe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe650360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe537a60 con 0x55eebe64c2d0 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe537760 con 0x55eebe646580 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.310+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe41b8c0 con 0x55eebe4a26b0 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f684b5d8b80 10 client.0 _ll_register_callbacks cb 0x55eebe535700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34890/0 (socket says 172.21.15.5:34890) 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 -- 172.21.15.5:0/1930634159 learned_addr learned my addr 172.21.15.5:0/1930634159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:47.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683519a700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe64fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683619c700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe650360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 -- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 msgr2=0x55eebe64fe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe64fe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 -- 172.21.15.5:0/1930634159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 msgr2=0x55eebe650360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe650360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.311+0000 7f683599b700 1 -- 172.21.15.5:0/1930634159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6830105a40 con 0x55eebe4a26b0 2023-07-21T19:35:47.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f683599b700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6824099ff0 tx=0x7f6824010b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a263b98ef8ca8419 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f68197fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a2500 con 0x55eebe4a26b0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eebe4f1490 con 0x55eebe4a26b0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f68240a2280 con 0x55eebe4a26b0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6824095ea0 con 0x55eebe4a26b0 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f684b5d8b80 10 client.30067 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:47.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.312+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55eebe3aec00 con 0x55eebe4a26b0 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f682407c080 con 0x55eebe4a26b0 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6824099010 con 0x55eebe4a26b0 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 10 client.30067 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 10 client.30067 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 20 client.30067 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:47.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe65f780 0x55eebe661b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 4 client.30067 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:47.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eebe590e50 con 0x55eebe65f780 2023-07-21T19:35:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.313+0000 7f683519a700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe65f780 0x55eebe661b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.314+0000 7f683519a700 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe65f780 0x55eebe661b60 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd7d3168767c24fa server_cookie=3031ee357b54f0b3 in_seq=0 out_seq=0 2023-07-21T19:35:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.314+0000 7f68197fa700 10 client.30067 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:47.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.315+0000 7f68197fa700 1 -- 172.21.15.5:0/1930634159 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eebe590e50 con 0x55eebe65f780 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.315+0000 7f68197fa700 10 client.30067 handle_command_reply: tid=0 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.315+0000 7f684b5d8b80 1 client.30067 shutdown 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 msgr2=0x55eebe64f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f6824099ff0 tx=0x7f6824010b60 comp rx=0 tx=0).stop 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 shutdown_connections 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe65f780 0x55eebe661b60 crc :-1 s=READY pgs=519 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe650360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe64f8e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/1930634159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe646580 0x55eebe64fe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 >> 172.21.15.5:0/1930634159 conn(0x55eebe63bbe0 msgr2=0x55eebe6404e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f68197fa700 0 client.30067 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 shutdown_connections 2023-07-21T19:35:47.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.316+0000 7f684b5d8b80 1 -- 172.21.15.5:0/1930634159 wait complete. 2023-07-21T19:35:47.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.317+0000 7f684b5d8b80 20 client.30067 trim_cache size 0 max 16384 2023-07-21T19:35:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 Processor -- start 2023-07-21T19:35:47.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 -- start start 2023-07-21T19:35:47.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 0x55eebe656c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe64b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe63d680 con 0x55eebe64c2d0 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe64a6d0 con 0x55eebe4a26b0 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.344+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe638b40 con 0x55eebe646580 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 0x55eebe656c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37418/0 (socket says 172.21.15.5:37418) 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 -- 172.21.15.5:0/3044814368 learned_addr learned my addr 172.21.15.5:0/3044814368 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683519a700 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 0x55eebe656c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:34902/0 (socket says 172.21.15.5:34902) 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683619c700 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe64b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 -- 172.21.15.5:0/3044814368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 msgr2=0x55eebe656c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 0x55eebe656c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 -- 172.21.15.5:0/3044814368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 msgr2=0x55eebe64b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe64b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.345+0000 7f683599b700 1 -- 172.21.15.5:0/3044814368 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eebe3ec1f0 con 0x55eebe4a26b0 2023-07-21T19:35:47.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.346+0000 7f683599b700 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f68240ca790 tx=0x7f6824096520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=daf0aba5f16b010d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.346+0000 7f6820bff700 1 -- 172.21.15.5:0/3044814368 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240a2500 con 0x55eebe4a26b0 2023-07-21T19:35:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.346+0000 7f6820bff700 1 -- 172.21.15.5:0/3044814368 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55eebe4f1490 con 0x55eebe4a26b0 2023-07-21T19:35:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.346+0000 7f6820bff700 1 -- 172.21.15.5:0/3044814368 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68240b8120 con 0x55eebe4a26b0 2023-07-21T19:35:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3044814368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 msgr2=0x55eebe646040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f68240ca790 tx=0x7f6824096520 comp rx=0 tx=0).stop 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3044814368 shutdown_connections 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe64c2d0 0x55eebe64b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe4a26b0 0x55eebe646040 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/3044814368 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe646580 0x55eebe656c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3044814368 >> 172.21.15.5:0/3044814368 conn(0x55eebe63bbe0 msgr2=0x55eebe64aab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3044814368 shutdown_connections 2023-07-21T19:35:47.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.347+0000 7f684b5d8b80 1 -- 172.21.15.5:0/3044814368 wait complete. 2023-07-21T19:35:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.348+0000 7f684b5d8b80 1 -- start start 2023-07-21T19:35:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.348+0000 7f684b5d8b80 1 Processor -- start 2023-07-21T19:35:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe654980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 0x55eebe654ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 0x55eebe655440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe6b2830 con 0x55eebe646580 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe6b29a0 con 0x55eebe64c2d0 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eebe6b2b60 con 0x55eebe4a26b0 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.349+0000 7f684b5d8b80 10 client.0 _ll_register_callbacks cb 0x7f6828009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.350+0000 7f683619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 0x55eebe655440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.350+0000 7f683619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 0x55eebe655440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:37434/0 (socket says 172.21.15.5:37434) 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.350+0000 7f683619c700 1 -- 172.21.15.5:0/123696804 learned_addr learned my addr 172.21.15.5:0/123696804 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.350+0000 7f683519a700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 0x55eebe654ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.351+0000 7f683519a700 1 -- 172.21.15.5:0/123696804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 msgr2=0x55eebe654980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.351+0000 7f683519a700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe654980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.351+0000 7f683519a700 1 -- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 msgr2=0x55eebe655440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.351+0000 7f683519a700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 0x55eebe655440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.351+0000 7f683519a700 1 -- 172.21.15.5:0/123696804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eebe3aec00 con 0x55eebe646580 2023-07-21T19:35:47.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.352+0000 7f683519a700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 0x55eebe654ec0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=435e0ee81ad252c3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.352+0000 7f68197fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:35:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.352+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f682c01d040 con 0x55eebe646580 2023-07-21T19:35:47.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.353+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f682c00f870 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.353+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f682409a310 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.353+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f682c0335d0 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.353+0000 7f684b5d8b80 10 client.20355 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.353+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55eebe3c4250 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f682c048020 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f682c030a30 con 0x55eebe646580 2023-07-21T19:35:47.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 10 client.20355 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 10 client.20355 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 20 client.20355 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe6b4130 0x55eebe6b6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 4 client.20355 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:35:47.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.354+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eebe552ff0 con 0x55eebe6b4130 2023-07-21T19:35:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.355+0000 7f683599b700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe6b4130 0x55eebe6b6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.356+0000 7f68197fa700 10 client.20355 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:47.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.356+0000 7f683599b700 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe6b4130 0x55eebe6b6540 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c604146a53a3803e server_cookie=96bfabeaa863591b in_seq=0 out_seq=0 2023-07-21T19:35:47.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f68197fa700 1 -- 172.21.15.5:0/123696804 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eebe552ff0 con 0x55eebe6b4130 2023-07-21T19:35:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f68197fa700 10 client.20355 handle_command_reply: tid=0 2023-07-21T19:35:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 client.20355 shutdown 2023-07-21T19:35:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 msgr2=0x55eebe654ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 0x55eebe654ec0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f682c0098e0 tx=0x7f682c026000 comp rx=0 tx=0).stop 2023-07-21T19:35:47.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 shutdown_connections 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55eebe64c2d0 0x55eebe655440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55eebe4a26b0 0x55eebe654980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55eebe6b4130 0x55eebe6b6540 crc :-1 s=READY pgs=520 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 --2- 172.21.15.5:0/123696804 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55eebe646580 0x55eebe654ec0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 >> 172.21.15.5:0/123696804 conn(0x55eebe63bbe0 msgr2=0x55eebe64e920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.400+0000 7f68197fa700 0 client.20355 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:47.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 shutdown_connections 2023-07-21T19:35:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f684b5d8b80 1 -- 172.21.15.5:0/123696804 wait complete. 2023-07-21T19:35:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f684b5d8b80 20 client.20355 trim_cache size 0 max 16384 2023-07-21T19:35:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:35:47.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:35:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:35:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:35:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 msgr2=0x7f681c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 0x7f681c0a37f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f682c009990 tx=0x7f682c000bc0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 msgr2=0x7f683019dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f682400e9a0 tx=0x7f682400ed60 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 shutdown_connections 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f681c0a1340 0x7f681c0a37f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6830104e50 0x7f6830197b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6830067c10 0x7f683019dce0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 --2- 172.21.15.5:0/4290581190 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f68300685b0 0x7f683019e220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:47.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.401+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 >> 172.21.15.5:0/4290581190 conn(0x7f6830074760 msgr2=0x7f683010bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.402+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 shutdown_connections 2023-07-21T19:35:47.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:35:47.402+0000 7f6837bff700 1 -- 172.21.15.5:0/4290581190 wait complete. 2023-07-21T19:35:47.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:47 smithi005 ceph-mon[126460]: pgmap v893: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 728 op/s 2023-07-21T19:35:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:47 smithi156 ceph-mon[128316]: pgmap v893: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 728 op/s 2023-07-21T19:35:48.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:47 smithi101 ceph-mon[128136]: pgmap v893: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.5 MiB/s wr, 728 op/s 2023-07-21T19:35:49.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:48 smithi005 ceph-mon[126460]: pgmap v894: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-07-21T19:35:49.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:48 smithi156 ceph-mon[128316]: pgmap v894: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-07-21T19:35:49.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:48 smithi101 ceph-mon[128136]: pgmap v894: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 11 MiB/s wr, 1.06k op/s 2023-07-21T19:35:51.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:51 smithi005 ceph-mon[126460]: pgmap v895: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 677 op/s 2023-07-21T19:35:51.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:51 smithi156 ceph-mon[128316]: pgmap v895: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 677 op/s 2023-07-21T19:35:51.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 560.0 s, 252.0 tps, lat 421.058 ms stddev 185.112 2023-07-21T19:35:52.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:51 smithi101 ceph-mon[128136]: pgmap v895: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.8 MiB/s wr, 677 op/s 2023-07-21T19:35:53.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:53 smithi005 ceph-mon[126460]: pgmap v896: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 865 op/s 2023-07-21T19:35:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:53 smithi156 ceph-mon[128316]: pgmap v896: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 865 op/s 2023-07-21T19:35:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:53 smithi101 ceph-mon[128136]: pgmap v896: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.9 MiB/s wr, 865 op/s 2023-07-21T19:35:55.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:55 smithi005 ceph-mon[126460]: pgmap v897: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 821 op/s 2023-07-21T19:35:55.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:55 smithi156 ceph-mon[128316]: pgmap v897: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 821 op/s 2023-07-21T19:35:56.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:55 smithi101 ceph-mon[128136]: pgmap v897: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.8 MiB/s wr, 821 op/s 2023-07-21T19:35:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 565.0 s, 261.4 tps, lat 379.265 ms stddev 72.389 2023-07-21T19:35:57.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:57 smithi005 ceph-mon[126460]: pgmap v898: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 652 op/s 2023-07-21T19:35:57.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:57 smithi156 ceph-mon[128316]: pgmap v898: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 652 op/s 2023-07-21T19:35:58.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:57 smithi101 ceph-mon[128136]: pgmap v898: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.5 MiB/s wr, 652 op/s 2023-07-21T19:35:59.101 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:35:59.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:35:58 smithi005 ceph-mon[126460]: pgmap v899: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:35:59.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:35:58 smithi156 ceph-mon[128316]: pgmap v899: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:35:59.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:35:58 smithi101 ceph-mon[128136]: pgmap v899: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:35:59.435 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 -- 172.21.15.5:0/4096660977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0fe9d0 msgr2=0x7ff6ac0fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/4096660977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac0fee40 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0009990 tx=0x7ff6a0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 -- 172.21.15.5:0/4096660977 shutdown_connections 2023-07-21T19:35:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/4096660977 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0ff380 0x7ff6ac10b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/4096660977 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.436 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/4096660977 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac0fee40 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 -- 172.21.15.5:0/4096660977 >> 172.21.15.5:0/4096660977 conn(0x7ff6ac0fa4b0 msgr2=0x7ff6ac0fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.449+0000 7ff6b1dff700 1 -- 172.21.15.5:0/4096660977 shutdown_connections 2023-07-21T19:35:59.437 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.450+0000 7ff6b1dff700 1 -- 172.21.15.5:0/4096660977 wait complete. 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.451+0000 7ff6b1dff700 1 Processor -- start 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.451+0000 7ff6b1dff700 1 -- start start 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac197de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0ff380 0x7ff6ac198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.438 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19cdc0 con 0x7ff6ac0ff380 2023-07-21T19:35:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19cf30 con 0x7ff6ac1049e0 2023-07-21T19:35:59.439 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6b1dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ac19d0f0 con 0x7ff6ac0fe9d0 2023-07-21T19:35:59.440 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0ff380 0x7ff6ac198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6abfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47898/0 (socket says 172.21.15.5:47898) 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6abfff700 1 -- 172.21.15.5:0/1946341733 learned_addr learned my addr 172.21.15.5:0/1946341733 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.452+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac197de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 -- 172.21.15.5:0/1946341733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0fe9d0 msgr2=0x7ff6ac197de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.441 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac197de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 -- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0ff380 msgr2=0x7ff6ac198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0ff380 0x7ff6ac198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 -- 172.21.15.5:0/1946341733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6a00095f0 con 0x7ff6ac1049e0 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.453+0000 7ff6abfff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400ef30 tx=0x7ff6a400c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.454+0000 7ff6a8ff9700 1 -- 172.21.15.5:0/1946341733 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a4018070 con 0x7ff6ac1049e0 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.454+0000 7ff6a8ff9700 1 -- 172.21.15.5:0/1946341733 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6a400f040 con 0x7ff6ac1049e0 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.454+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ac19d370 con 0x7ff6ac1049e0 2023-07-21T19:35:59.442 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.454+0000 7ff6a8ff9700 1 -- 172.21.15.5:0/1946341733 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a400cbc0 con 0x7ff6ac1049e0 2023-07-21T19:35:59.443 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.454+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ac1a4c10 con 0x7ff6ac1049e0 2023-07-21T19:35:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.459+0000 7ff6a8ff9700 1 -- 172.21.15.5:0/1946341733 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff6a4007500 con 0x7ff6ac1049e0 2023-07-21T19:35:59.445 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.460+0000 7ff6c579db80 1 Processor -- start 2023-07-21T19:35:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.460+0000 7ff6c579db80 1 -- start start 2023-07-21T19:35:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 0x564c604746a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.446 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 0x564c6047f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c6047fad0 0x564c60481eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6037acb0 con 0x564c60474be0 2023-07-21T19:35:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c603cda40 con 0x564c604742d0 2023-07-21T19:35:59.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6c579db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c603c6ed0 con 0x564c6047fad0 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 0x564c604746a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 0x564c604746a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47914/0 (socket says 172.21.15.5:47914) 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6aaffd700 1 -- 172.21.15.5:0/4194146922 learned_addr learned my addr 172.21.15.5:0/4194146922 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 0x564c6047f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.461+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c6047fad0 0x564c60481eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.448 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/4194146922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c6047fad0 msgr2=0x564c60481eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c6047fad0 0x564c60481eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/4194146922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 msgr2=0x564c604746a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 0x564c604746a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.449 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/4194146922 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c601f4de0 con 0x564c60474be0 2023-07-21T19:35:59.450 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 0x564c6047f590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009c80 tx=0x7ff69c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31d1b34c355e6e5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff691ffb700 1 -- 172.21.15.5:0/4194146922 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c011c50 con 0x564c60474be0 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff6a8ff9700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 0x7ff6940a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff691ffb700 1 -- 172.21.15.5:0/4194146922 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff69c0044c0 con 0x564c60474be0 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.462+0000 7ff691ffb700 1 -- 172.21.15.5:0/4194146922 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003df0 con 0x564c60474be0 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.463+0000 7ff6a8ff9700 1 -- 172.21.15.5:0/1946341733 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff6a40cc790 con 0x7ff6ac1049e0 2023-07-21T19:35:59.451 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.463+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 0x7ff6940a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.463+0000 7ff6c579db80 1 -- 172.21.15.5:0/4194146922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 msgr2=0x564c6047f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.463+0000 7ff6c579db80 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 0x564c6047f590 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009c80 tx=0x7ff69c009f90 comp rx=0 tx=0).stop 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 -- 172.21.15.5:0/4194146922 shutdown_connections 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c6047fad0 0x564c60481eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c604742d0 0x564c604746a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 --2- 172.21.15.5:0/4194146922 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c60474be0 0x564c6047f590 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.452 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 -- 172.21.15.5:0/4194146922 >> 172.21.15.5:0/4194146922 conn(0x564c6046ada0 msgr2=0x564c604706a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 -- 172.21.15.5:0/4194146922 shutdown_connections 2023-07-21T19:35:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 0x7ff6940a37f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005910 tx=0x7ff6a001a040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 -- 172.21.15.5:0/4194146922 wait complete. 2023-07-21T19:35:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.464+0000 7ff6c579db80 1 -- start start 2023-07-21T19:35:59.453 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.465+0000 7ff6c579db80 1 Processor -- start 2023-07-21T19:35:59.454 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.465+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60366ac0 con 0x564c6047fad0 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c603667c0 con 0x564c60474be0 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6024a920 con 0x564c604742d0 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6c579db80 10 client.0 _ll_register_callbacks cb 0x564c60364760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:59.455 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59942/0 (socket says 172.21.15.5:59942) 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/2810402424 learned_addr learned my addr 172.21.15.5:0/2810402424 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6abfff700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 msgr2=0x564c6047c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/2810402424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 msgr2=0x564c6047ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.466+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/2810402424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69c0098e0 con 0x564c604742d0 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff6abfff700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009230 tx=0x7ff69c005320 comp rx=0 tx=0).ready entity=mon.1 client_cookie=693a3059319138e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff68affd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:35:59.457 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c004950 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c603204f0 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.467+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff69c005ce0 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.468+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c011e80 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.468+0000 7ff6c579db80 10 client.30082 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.468+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564c601ddc60 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.468+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c603204f0 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.468+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff69c028020 con 0x564c604742d0 2023-07-21T19:35:59.458 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 10 client.30082 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 10 client.30082 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 20 client.30082 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c6048f350 0x564c60491760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 4 client.30082 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:35:59.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c603bfeb0 con 0x564c6048f350 2023-07-21T19:35:59.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.469+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c6048f350 0x564c60491760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.470+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c6048f350 0x564c60491760 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=65fe3d1f94c67a78 server_cookie=19628db64501b5f2 in_seq=0 out_seq=0 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.470+0000 7ff68affd700 10 client.30082 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff68affd700 1 -- 172.21.15.5:0/2810402424 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c603bfeb0 con 0x564c6048f350 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff68affd700 10 client.30082 handle_command_reply: tid=0 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 client.30082 shutdown 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 msgr2=0x564c6047f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009230 tx=0x7ff69c005320 comp rx=0 tx=0).stop 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 shutdown_connections 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c6048f350 0x564c60491760 crc :-1 s=READY pgs=521 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6047f4c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 --2- 172.21.15.5:0/2810402424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 >> 172.21.15.5:0/2810402424 conn(0x564c6046ada0 msgr2=0x564c60470230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.471+0000 7ff68affd700 0 client.30082 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.472+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 shutdown_connections 2023-07-21T19:35:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.472+0000 7ff6c579db80 1 -- 172.21.15.5:0/2810402424 wait complete. 2023-07-21T19:35:59.463 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.472+0000 7ff6c579db80 20 client.30082 trim_cache size 0 max 16384 2023-07-21T19:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.499+0000 7ff6c579db80 1 Processor -- start 2023-07-21T19:35:59.485 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.499+0000 7ff6c579db80 1 -- start start 2023-07-21T19:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.486 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6047d740 con 0x564c6047fad0 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6047dd20 con 0x564c60474be0 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6c579db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c6046a970 con 0x564c604742d0 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6aaffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.487 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59958/0 (socket says 172.21.15.5:59958) 2023-07-21T19:35:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/1086085295 learned_addr learned my addr 172.21.15.5:0/1086085295 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:59.488 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.500+0000 7ff6abfff700 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6aaffd700 1 -- 172.21.15.5:0/1086085295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 msgr2=0x564c6048b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.489 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6aaffd700 1 -- 172.21.15.5:0/1086085295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 msgr2=0x564c6047bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6aaffd700 1 -- 172.21.15.5:0/1086085295 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c6021b250 con 0x564c60474be0 2023-07-21T19:35:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.501+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:35:59.490 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.502+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047f060 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x564c6047dcf0 tx=0x7ff6a0009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=430e2b9ca26afc1e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.502+0000 7ff6927fc700 1 -- 172.21.15.5:0/1086085295 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a0003a10 con 0x564c60474be0 2023-07-21T19:35:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.502+0000 7ff6927fc700 1 -- 172.21.15.5:0/1086085295 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff6a0003d70 con 0x564c60474be0 2023-07-21T19:35:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.502+0000 7ff6927fc700 1 -- 172.21.15.5:0/1086085295 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6a00382d0 con 0x564c60474be0 2023-07-21T19:35:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 -- 172.21.15.5:0/1086085295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 msgr2=0x564c6047f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.491 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047f060 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x564c6047dcf0 tx=0x7ff6a0009310 comp rx=0 tx=0).stop 2023-07-21T19:35:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 -- 172.21.15.5:0/1086085295 shutdown_connections 2023-07-21T19:35:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c6047fad0 0x564c6047bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c604742d0 0x564c6048b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 --2- 172.21.15.5:0/1086085295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c60474be0 0x564c6047f060 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.492 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.503+0000 7ff6c579db80 1 -- 172.21.15.5:0/1086085295 >> 172.21.15.5:0/1086085295 conn(0x564c6046ada0 msgr2=0x564c60496080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.506+0000 7ff6c579db80 1 -- 172.21.15.5:0/1086085295 shutdown_connections 2023-07-21T19:35:59.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.506+0000 7ff6c579db80 1 -- 172.21.15.5:0/1086085295 wait complete. 2023-07-21T19:35:59.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.507+0000 7ff6c579db80 1 -- start start 2023-07-21T19:35:59.493 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.507+0000 7ff6c579db80 1 Processor -- start 2023-07-21T19:35:59.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c60474be0 0x564c60484950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 0x564c60484fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c604856a0 con 0x564c604742d0 2023-07-21T19:35:59.494 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c60485810 con 0x564c6047fad0 2023-07-21T19:35:59.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c604d91f0 con 0x564c60474be0 2023-07-21T19:35:59.495 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6c579db80 10 client.0 _ll_register_callbacks cb 0x7ff69c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:35:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.508+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49284/0 (socket says 172.21.15.5:49284) 2023-07-21T19:35:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/753103702 learned_addr learned my addr 172.21.15.5:0/753103702 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:35:59.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c60474be0 0x564c60484950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/753103702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c60474be0 msgr2=0x564c60484950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c60474be0 0x564c60484950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 msgr2=0x564c60484fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6abfff700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 0x564c60484fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 0x564c60484fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6ab7fe700 1 -- 172.21.15.5:0/753103702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x564c601ddc60 con 0x564c604742d0 2023-07-21T19:35:59.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.509+0000 7ff6abfff700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 0x564c60484fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff6ab7fe700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009bd0 tx=0x7ff69c005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ee714994e583ae0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff68affd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff69c003bf0 con 0x564c604742d0 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff69c000e40 con 0x564c604742d0 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff69c0044c0 con 0x564c604742d0 2023-07-21T19:35:59.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.510+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff69c005440 con 0x564c604742d0 2023-07-21T19:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 10 client.20373 fetch_fsmap learned FSMap version 72 2023-07-21T19:35:59.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7ff6a0030040 con 0x564c604742d0 2023-07-21T19:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff69c0044c0 con 0x564c604742d0 2023-07-21T19:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff69c011440 con 0x564c604742d0 2023-07-21T19:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 10 client.20373 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:35:59.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 10 client.20373 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 20 client.20373 populate_metadata read hostname 'smithi005' 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c604da590 0x564c604dc970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 4 client.20373 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.511+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c60382050 con 0x564c604da590 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.512+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c604da590 0x564c604dc970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.512+0000 7ff6aaffd700 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c604da590 0x564c604dc970 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38e5dba691d86f4a server_cookie=bcc06b9106ad19b7 in_seq=0 out_seq=0 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.512+0000 7ff68affd700 10 client.20373 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:59.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff68affd700 1 -- 172.21.15.5:0/753103702 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c60382050 con 0x564c604da590 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff68affd700 10 client.20373 handle_command_reply: tid=0 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 client.20373 shutdown 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 msgr2=0x564c60486660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7ff69c009bd0 tx=0x7ff69c005e30 comp rx=0 tx=0).stop 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 shutdown_connections 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c6047fad0 0x564c60484fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c604da590 0x564c604dc970 crc :-1 s=READY pgs=522 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c604742d0 0x564c60486660 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 --2- 172.21.15.5:0/753103702 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c60474be0 0x564c60484950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff68affd700 0 client.20373 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.513+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 >> 172.21.15.5:0/753103702 conn(0x564c6046ada0 msgr2=0x564c6048b780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.514+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 shutdown_connections 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.514+0000 7ff6c579db80 1 -- 172.21.15.5:0/753103702 wait complete. 2023-07-21T19:35:59.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.514+0000 7ff6c579db80 20 client.20373 trim_cache size 0 max 16384 2023-07-21T19:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:35:59.505 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:35:59.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:35:59.506 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:35:59.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:35:59.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:35:59.506 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.515+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 msgr2=0x7ff6940a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.515+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 0x7ff6940a37f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff6a0005910 tx=0x7ff6a001a040 comp rx=0 tx=0).stop 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.515+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 msgr2=0x7ff6ac19c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.515+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7ff6a400ef30 tx=0x7ff6a400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 shutdown_connections 2023-07-21T19:35:59.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff6ac1049e0 0x7ff6ac19c780 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6ac0fe9d0 0x7ff6ac197de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6940a1340 0x7ff6940a37f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 --2- 172.21.15.5:0/1946341733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6ac0ff380 0x7ff6ac198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:35:59.508 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 >> 172.21.15.5:0/1946341733 conn(0x7ff6ac0fa4b0 msgr2=0x7ff6ac10be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:35:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 shutdown_connections 2023-07-21T19:35:59.509 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:35:59.518+0000 7ff6b1dff700 1 -- 172.21.15.5:0/1946341733 wait complete. 2023-07-21T19:35:59.532 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:36:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.694+0000 7f8f92bff700 1 -- 172.21.15.5:0/2833072231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06d3a0 msgr2=0x7f8f8c11b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.694+0000 7f8f92bff700 1 --2- 172.21.15.5:0/2833072231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c11b470 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009990 tx=0x7f8f74009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.695+0000 7f8f92bff700 1 -- 172.21.15.5:0/2833072231 shutdown_connections 2023-07-21T19:36:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.695+0000 7f8f92bff700 1 --2- 172.21.15.5:0/2833072231 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c11b470 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.695+0000 7f8f92bff700 1 --2- 172.21.15.5:0/2833072231 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c10e2a0 0x7f8f8c10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.695+0000 7f8f92bff700 1 --2- 172.21.15.5:0/2833072231 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8c06c910 0x7f8f8c06cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.695+0000 7f8f92bff700 1 -- 172.21.15.5:0/2833072231 >> 172.21.15.5:0/2833072231 conn(0x7f8f8c06bd70 msgr2=0x7f8f8c070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.697+0000 7f8f92bff700 1 -- 172.21.15.5:0/2833072231 shutdown_connections 2023-07-21T19:36:00.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.704+0000 7f8f92bff700 1 -- 172.21.15.5:0/2833072231 wait complete. 2023-07-21T19:36:00.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.706+0000 7f8f92bff700 1 Processor -- start 2023-07-21T19:36:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.708+0000 7f8f92bff700 1 -- start start 2023-07-21T19:36:00.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06c910 0x7f8f8c114e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8c10e2a0 0x7f8f8c11a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c11b0d0 con 0x7f8f8c10e2a0 2023-07-21T19:36:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c11b240 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.709+0000 7f8f92bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f8c11b400 con 0x7f8f8c06c910 2023-07-21T19:36:00.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47964/0 (socket says 172.21.15.5:47964) 2023-07-21T19:36:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 -- 172.21.15.5:0/538540777 learned_addr learned my addr 172.21.15.5:0/538540777 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:00.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 -- 172.21.15.5:0/538540777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06c910 msgr2=0x7f8f8c114e50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:36:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06c910 0x7f8f8c114e50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 -- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8c10e2a0 msgr2=0x7f8f8c11a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8c10e2a0 0x7f8f8c11a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.711+0000 7f8f8bfff700 1 -- 172.21.15.5:0/538540777 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740095f0 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.712+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba50 tx=0x7f8f7c00be10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.712+0000 7f8f89ffb700 1 -- 172.21.15.5:0/538540777 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c00cab0 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.714+0000 7f8f89ffb700 1 -- 172.21.15.5:0/538540777 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f7c00cc10 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.714+0000 7f8f89ffb700 1 -- 172.21.15.5:0/538540777 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0075b0 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.714+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f8c11b6e0 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.714+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1ba080 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.724+0000 7f8f89ffb700 1 -- 172.21.15.5:0/538540777 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8f7c014440 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.725+0000 7f8fa653cb80 1 Processor -- start 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 -- start start 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544ef9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4543f4c50 con 0x55d4544ef580 2023-07-21T19:36:00.714 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4544479e0 con 0x55d45434b6b0 2023-07-21T19:36:00.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.727+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d454440e70 con 0x55d4544f52d0 2023-07-21T19:36:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9099b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.716 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9099b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59990/0 (socket says 172.21.15.5:59990) 2023-07-21T19:36:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9099b700 1 -- 172.21.15.5:0/2978243044 learned_addr learned my addr 172.21.15.5:0/2978243044 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f89ffb700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 0x7f8f780a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f89ffb700 1 -- 172.21.15.5:0/538540777 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8f7c0cc2f0 con 0x7f8f8c06d3a0 2023-07-21T19:36:00.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 -- 172.21.15.5:0/2978243044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 msgr2=0x55d4544ef040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 -- 172.21.15.5:0/2978243044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 msgr2=0x55d4544ef9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544ef9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.730+0000 7f8f9119c700 1 -- 172.21.15.5:0/2978243044 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d45426ed80 con 0x55d45434b6b0 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.731+0000 7f8f9099b700 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.731+0000 7f8f9119c700 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544f4d90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009990 tx=0x7f8f74005600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a72a9b8b1febcfd7 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.731+0000 7f8f9119c700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 0x7f8f780a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.731+0000 7f8f82ffd700 1 -- 172.21.15.5:0/2978243044 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f740039a0 con 0x55d45434b6b0 2023-07-21T19:36:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.732+0000 7f8f82ffd700 1 -- 172.21.15.5:0/2978243044 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f74003b80 con 0x55d45434b6b0 2023-07-21T19:36:00.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.732+0000 7f8f82ffd700 1 -- 172.21.15.5:0/2978243044 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400f940 con 0x55d45434b6b0 2023-07-21T19:36:00.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 -- 172.21.15.5:0/2978243044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 msgr2=0x55d4544f4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544f4d90 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8f74009990 tx=0x7f8f74005600 comp rx=0 tx=0).stop 2023-07-21T19:36:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8f9119c700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 0x7f8f780a38b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005850 tx=0x7f8f7400fbe0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 -- 172.21.15.5:0/2978243044 shutdown_connections 2023-07-21T19:36:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544ef9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544f4d90 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/2978243044 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544ef040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 -- 172.21.15.5:0/2978243044 >> 172.21.15.5:0/2978243044 conn(0x55d4544e4be0 msgr2=0x55d4544e9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 -- 172.21.15.5:0/2978243044 shutdown_connections 2023-07-21T19:36:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.733+0000 7f8fa653cb80 1 -- 172.21.15.5:0/2978243044 wait complete. 2023-07-21T19:36:00.725 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.734+0000 7f8fa653cb80 1 -- start start 2023-07-21T19:36:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.735+0000 7f8fa653cb80 1 Processor -- start 2023-07-21T19:36:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.735+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544fb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544ef580 0x55d4544f8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4543e0a60 con 0x55d4544f52d0 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4543e0760 con 0x55d45434b6b0 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4542c48c0 con 0x55d4544ef580 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.736+0000 7f8fa653cb80 10 client.0 _ll_register_callbacks cb 0x55d4543de700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.738+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.738+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49318/0 (socket says 172.21.15.5:49318) 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.738+0000 7f8f9119c700 1 -- 172.21.15.5:0/650912157 learned_addr learned my addr 172.21.15.5:0/650912157 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.739+0000 7f8f9119c700 1 -- 172.21.15.5:0/650912157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544ef580 msgr2=0x55d4544f8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.739+0000 7f8f9119c700 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544ef580 0x55d4544f8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.739+0000 7f8f9119c700 1 -- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 msgr2=0x55d4544fb610 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.739+0000 7f8f9119c700 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544fb610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.739+0000 7f8f9119c700 1 -- 172.21.15.5:0/650912157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f84009910 con 0x55d4544f52d0 2023-07-21T19:36:00.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.740+0000 7f8f9119c700 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005640 tx=0x7f8f74024000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be251ea8756de670 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.750+0000 7f8f6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:36:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.750+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400b910 con 0x55d4544f52d0 2023-07-21T19:36:00.736 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.750+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8f740039b0 con 0x55d4544f52d0 2023-07-21T19:36:00.737 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.750+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d45439a490 con 0x55d4544f52d0 2023-07-21T19:36:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.754+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f7400ba70 con 0x55d4544f52d0 2023-07-21T19:36:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.754+0000 7f8fa653cb80 10 client.20376 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:00.740 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.754+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f8f8c11bfb0 con 0x55d4544f52d0 2023-07-21T19:36:00.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.755+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f74044030 con 0x55d4544f52d0 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.755+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f74023ce0 con 0x55d4544f52d0 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 10 client.20376 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 10 client.20376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 20 client.20376 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d454508870 0x55d45450ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.744 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 4 client.20376 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:00.745 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.756+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d454439e50 con 0x55d454508870 2023-07-21T19:36:00.752 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.764+0000 7f8f9099b700 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d454508870 0x55d45450ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.777+0000 7f8f9099b700 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d454508870 0x55d45450ac80 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8209b17c8cc94f0 server_cookie=9d8577f554f9ba96 in_seq=0 out_seq=0 2023-07-21T19:36:00.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.777+0000 7f8f6bfff700 10 client.20376 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.778+0000 7f8f6bfff700 1 -- 172.21.15.5:0/650912157 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d454439e50 con 0x55d454508870 2023-07-21T19:36:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.778+0000 7f8f6bfff700 10 client.20376 handle_command_reply: tid=0 2023-07-21T19:36:00.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.778+0000 7f8fa653cb80 1 client.20376 shutdown 2023-07-21T19:36:00.780 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.785+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 msgr2=0x55d4544f9040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.781 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.785+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005640 tx=0x7f8f74024000 comp rx=0 tx=0).stop 2023-07-21T19:36:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 shutdown_connections 2023-07-21T19:36:00.788 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d454508870 0x55d45450ac80 crc :-1 s=READY pgs=523 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544f9040 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544fb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/650912157 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544ef580 0x55d4544f8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.801+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 >> 172.21.15.5:0/650912157 conn(0x55d4544e4be0 msgr2=0x55d4544e94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.802+0000 7f8f6bfff700 0 client.20376 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:00.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.831+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 shutdown_connections 2023-07-21T19:36:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.843+0000 7f8fa653cb80 1 -- 172.21.15.5:0/650912157 wait complete. 2023-07-21T19:36:00.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.843+0000 7f8fa653cb80 20 client.20376 trim_cache size 0 max 16384 2023-07-21T19:36:00.864 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.873+0000 7f8fa653cb80 1 Processor -- start 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.873+0000 7f8fa653cb80 1 -- start start 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.873+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544e9c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.874+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544fb1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.874+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544f80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.874+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4544b5060 con 0x55d4544ef580 2023-07-21T19:36:00.865 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.874+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4544f4a10 con 0x55d45434b6b0 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.874+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d4544f9e70 con 0x55d4544f52d0 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.876+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544f80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.876+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544f80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:60020/0 (socket says 172.21.15.5:60020) 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.876+0000 7f8f9119c700 1 -- 172.21.15.5:0/618002562 learned_addr learned my addr 172.21.15.5:0/618002562 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.876+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544fb1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.876+0000 7f8f9099b700 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544e9c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 -- 172.21.15.5:0/618002562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 msgr2=0x55d4544f80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:00.866 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544f80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 -- 172.21.15.5:0/618002562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 msgr2=0x55d4544e9c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544e9c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 -- 172.21.15.5:0/618002562 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d45450f870 con 0x55d4544ef580 2023-07-21T19:36:00.867 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.877+0000 7f8f8bfff700 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544fb1b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0c9af0 tx=0x7f8f7c095830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e1d162816bf81c0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.890+0000 7f8f837fe700 1 -- 172.21.15.5:0/618002562 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c0cb070 con 0x55d4544ef580 2023-07-21T19:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.890+0000 7f8f837fe700 1 -- 172.21.15.5:0/618002562 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f7c098dd0 con 0x55d4544ef580 2023-07-21T19:36:00.876 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.890+0000 7f8f837fe700 1 -- 172.21.15.5:0/618002562 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c097810 con 0x55d4544ef580 2023-07-21T19:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.891+0000 7f8fa653cb80 1 -- 172.21.15.5:0/618002562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 msgr2=0x55d4544fb1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.877 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.891+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544fb1b0 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c0c9af0 tx=0x7f8f7c095830 comp rx=0 tx=0).stop 2023-07-21T19:36:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.893+0000 7f8fa653cb80 1 -- 172.21.15.5:0/618002562 shutdown_connections 2023-07-21T19:36:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.893+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d4544f52d0 0x55d4544f80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.893+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d45434b6b0 0x55d4544e9c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.893+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/618002562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544ef580 0x55d4544fb1b0 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.879 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.893+0000 7f8fa653cb80 1 -- 172.21.15.5:0/618002562 >> 172.21.15.5:0/618002562 conn(0x55d4544e4be0 msgr2=0x55d4544f2b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.899+0000 7f8fa653cb80 1 -- 172.21.15.5:0/618002562 shutdown_connections 2023-07-21T19:36:00.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.904+0000 7f8fa653cb80 1 -- 172.21.15.5:0/618002562 wait complete. 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.905+0000 7f8fa653cb80 1 -- start start 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.905+0000 7f8fa653cb80 1 Processor -- start 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d45434b6b0 0x55d4544fdc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4544ef580 0x55d4544fe190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d45455b780 con 0x55d4544f52d0 2023-07-21T19:36:00.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d45455b8f0 con 0x55d4544ef580 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d45455bab0 con 0x55d45434b6b0 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.906+0000 7f8fa653cb80 10 client.0 _ll_register_callbacks cb 0x7f8f84009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.908+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.908+0000 7f8f9119c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49346/0 (socket says 172.21.15.5:49346) 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.908+0000 7f8f9119c700 1 -- 172.21.15.5:0/3052245958 learned_addr learned my addr 172.21.15.5:0/3052245958 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.909+0000 7f8f9119c700 1 -- 172.21.15.5:0/3052245958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d45434b6b0 msgr2=0x55d4544fdc50 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:36:00.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.909+0000 7f8f9119c700 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d45434b6b0 0x55d4544fdc50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.909+0000 7f8f9119c700 1 -- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4544ef580 msgr2=0x55d4544fe190 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.909+0000 7f8f9119c700 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4544ef580 0x55d4544fe190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.909+0000 7f8f9119c700 1 -- 172.21.15.5:0/3052245958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f8c1ba080 con 0x55d4544f52d0 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.910+0000 7f8f9119c700 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8f74036d90 tx=0x7f8f74036e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64063ea81c09902 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.911+0000 7f8f6bfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.912+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7400bb20 con 0x55d4544f52d0 2023-07-21T19:36:00.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.912+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(133..133 src has 1..133) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8f74000e40 con 0x55d4544f52d0 2023-07-21T19:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.912+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8f74003b80 con 0x55d4544f52d0 2023-07-21T19:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.912+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8f7400f9e0 con 0x55d4544f52d0 2023-07-21T19:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.913+0000 7f8fa653cb80 10 client.20382 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:00.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.913+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55d454257c00 con 0x55d4544f52d0 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.918+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8f74003b80 con 0x55d4544f52d0 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.918+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8f74050050 con 0x55d4544f52d0 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 10 client.20382 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 10 client.20382 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 20 client.20382 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d45455ccf0 0x55d45455f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:00.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 4 client.20382 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:36:00.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.919+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d4543fbff0 con 0x55d45455ccf0 2023-07-21T19:36:00.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.928+0000 7f8f9099b700 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d45455ccf0 0x55d45455f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.930+0000 7f8f9099b700 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d45455ccf0 0x55d45455f0d0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8c2a46b028f89f server_cookie=951f0216f90ac427 in_seq=0 out_seq=0 2023-07-21T19:36:00.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.930+0000 7f8f6bfff700 10 client.20382 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:00.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8f6bfff700 1 -- 172.21.15.5:0/3052245958 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d4543fbff0 con 0x55d45455ccf0 2023-07-21T19:36:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8f6bfff700 10 client.20382 handle_command_reply: tid=0 2023-07-21T19:36:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8fa653cb80 1 client.20382 shutdown 2023-07-21T19:36:00.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 msgr2=0x55d4544fe6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f8f74036d90 tx=0x7f8f74036e70 comp rx=0 tx=0).stop 2023-07-21T19:36:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 shutdown_connections 2023-07-21T19:36:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.983+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d45455ccf0 0x55d45455f0d0 crc :-1 s=READY pgs=524 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d4544f52d0 0x55d4544fe6d0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d45434b6b0 0x55d4544fdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 --2- 172.21.15.5:0/3052245958 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d4544ef580 0x55d4544fe190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 >> 172.21.15.5:0/3052245958 conn(0x55d4544e4be0 msgr2=0x55d4544f3a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8f6bfff700 0 client.20382 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 shutdown_connections 2023-07-21T19:36:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 1 -- 172.21.15.5:0/3052245958 wait complete. 2023-07-21T19:36:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.984+0000 7f8fa653cb80 20 client.20382 trim_cache size 0 max 16384 2023-07-21T19:36:00.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:36:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:36:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:36:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:36:00.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.985+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 msgr2=0x7f8f780a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.985+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 0x7f8f780a38b0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f8f74005850 tx=0x7f8f7400fbe0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.985+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 msgr2=0x7f8f8c115390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f8f7c00ba50 tx=0x7f8f7c00be10 comp rx=0 tx=0).stop 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 shutdown_connections 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8f780a1400 0x7f8f780a38b0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8f8c10e2a0 0x7f8f8c11a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8f8c06c910 0x7f8f8c114e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 --2- 172.21.15.5:0/538540777 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8f8c06d3a0 0x7f8f8c115390 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 >> 172.21.15.5:0/538540777 conn(0x7f8f8c06bd70 msgr2=0x7f8f8c06f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 shutdown_connections 2023-07-21T19:36:00.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:00.986+0000 7f8f92bff700 1 -- 172.21.15.5:0/538540777 wait complete. 2023-07-21T19:36:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 570.0 s, 260.2 tps, lat 383.802 ms stddev 147.553 2023-07-21T19:36:02.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:01 smithi101 ceph-mon[128136]: pgmap v900: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-07-21T19:36:02.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:01 smithi005 ceph-mon[126460]: pgmap v900: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-07-21T19:36:02.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:01 smithi156 ceph-mon[128316]: pgmap v900: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 699 op/s 2023-07-21T19:36:04.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:03 smithi101 ceph-mon[128136]: pgmap v901: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 933 op/s 2023-07-21T19:36:04.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:03 smithi005 ceph-mon[126460]: pgmap v901: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 933 op/s 2023-07-21T19:36:04.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:03 smithi156 ceph-mon[128316]: pgmap v901: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.5 MiB/s wr, 933 op/s 2023-07-21T19:36:06.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:05 smithi101 ceph-mon[128136]: pgmap v902: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 896 op/s 2023-07-21T19:36:06.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:05 smithi005 ceph-mon[126460]: pgmap v902: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 896 op/s 2023-07-21T19:36:06.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:05 smithi156 ceph-mon[128316]: pgmap v902: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.3 MiB/s wr, 896 op/s 2023-07-21T19:36:06.938 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 575.0 s, 190.6 tps, lat 463.417 ms stddev 127.926 2023-07-21T19:36:08.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:07 smithi101 ceph-mon[128136]: pgmap v903: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-07-21T19:36:08.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:07 smithi101 ceph-mon[128136]: osdmap e134: 12 total, 12 up, 12 in 2023-07-21T19:36:08.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:07 smithi005 ceph-mon[126460]: pgmap v903: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-07-21T19:36:08.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:07 smithi005 ceph-mon[126460]: osdmap e134: 12 total, 12 up, 12 in 2023-07-21T19:36:08.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:07 smithi156 ceph-mon[128316]: pgmap v903: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.1 MiB/s wr, 762 op/s 2023-07-21T19:36:08.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:07 smithi156 ceph-mon[128316]: osdmap e134: 12 total, 12 up, 12 in 2023-07-21T19:36:09.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:08 smithi005 ceph-mon[126460]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 703 op/s 2023-07-21T19:36:09.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:08 smithi156 ceph-mon[128316]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 703 op/s 2023-07-21T19:36:09.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:08 smithi101 ceph-mon[128136]: pgmap v905: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 703 op/s 2023-07-21T19:36:11.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:11 smithi005 ceph-mon[126460]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 704 op/s 2023-07-21T19:36:11.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:11 smithi156 ceph-mon[128316]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 704 op/s 2023-07-21T19:36:11.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 580.0 s, 108.9 tps, lat 977.600 ms stddev 763.903 2023-07-21T19:36:12.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:11 smithi101 ceph-mon[128136]: pgmap v906: 97 pgs: 40 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 34 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 704 op/s 2023-07-21T19:36:13.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:13 smithi156 ceph-mon[128316]: pgmap v907: 97 pgs: 28 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-07-21T19:36:14.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:13 smithi101 ceph-mon[128136]: pgmap v907: 97 pgs: 28 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-07-21T19:36:14.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:13 smithi005 ceph-mon[126460]: pgmap v907: 97 pgs: 28 active+clean+snaptrim_wait, 17 active+clean+snaptrim, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.8 MiB/s wr, 490 op/s 2023-07-21T19:36:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 -- 172.21.15.5:0/3227828358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc006d8c0 msgr2=0x7f0bc0107510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/3227828358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc006d8c0 0x7f0bc0107510 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009990 tx=0x7f0bb0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 -- 172.21.15.5:0/3227828358 shutdown_connections 2023-07-21T19:36:14.206 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/3227828358 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0112920 0x7f0bc0114d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/3227828358 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc006d8c0 0x7f0bc0107510 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/3227828358 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc0107a50 0x7f0bc0107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 -- 172.21.15.5:0/3227828358 >> 172.21.15.5:0/3227828358 conn(0x7f0bc006c270 msgr2=0x7f0bc006c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.207 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.220+0000 7f0bc5dff700 1 -- 172.21.15.5:0/3227828358 shutdown_connections 2023-07-21T19:36:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.221+0000 7f0bc5dff700 1 -- 172.21.15.5:0/3227828358 wait complete. 2023-07-21T19:36:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.222+0000 7f0bc5dff700 1 Processor -- start 2023-07-21T19:36:14.208 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.222+0000 7f0bc5dff700 1 -- start start 2023-07-21T19:36:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc006d8c0 0x7f0bc01a65d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a0bc0 con 0x7f0bc0107a50 2023-07-21T19:36:14.209 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a0d30 con 0x7f0bc006d8c0 2023-07-21T19:36:14.211 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.223+0000 7f0bc5dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc01a0ef0 con 0x7f0bc0112920 2023-07-21T19:36:14.212 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc006d8c0 0x7f0bc01a65d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbeffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40230/0 (socket says 172.21.15.5:40230) 2023-07-21T19:36:14.213 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42856/0 (socket says 172.21.15.5:42856) 2023-07-21T19:36:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbeffd700 1 -- 172.21.15.5:0/2512639817 learned_addr learned my addr 172.21.15.5:0/2512639817 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbeffd700 1 -- 172.21.15.5:0/2512639817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 msgr2=0x7f0bc01a03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.224+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbeffd700 1 -- 172.21.15.5:0/2512639817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc006d8c0 msgr2=0x7f0bc01a65d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc006d8c0 0x7f0bc01a65d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbeffd700 1 -- 172.21.15.5:0/2512639817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bb00095f0 con 0x7f0bc0107a50 2023-07-21T19:36:14.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbffff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:36:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9a0 tx=0x7f0bb400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbcff9700 1 -- 172.21.15.5:0/2512639817 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb4009790 con 0x7f0bc0107a50 2023-07-21T19:36:14.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bbcff9700 1 -- 172.21.15.5:0/2512639817 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bb4010430 con 0x7f0bc0107a50 2023-07-21T19:36:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.225+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc01a11d0 con 0x7f0bc0107a50 2023-07-21T19:36:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.226+0000 7f0bbcff9700 1 -- 172.21.15.5:0/2512639817 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb400cea0 con 0x7f0bc0107a50 2023-07-21T19:36:14.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.226+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc01ad5a0 con 0x7f0bc0107a50 2023-07-21T19:36:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.233+0000 7f0bd9818b80 1 Processor -- start 2023-07-21T19:36:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.233+0000 7f0bd9818b80 1 -- start start 2023-07-21T19:36:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fba850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fbad90 0x559d09fbb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09eb7c50 con 0x559d09fbad90 2023-07-21T19:36:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bbcff9700 1 -- 172.21.15.5:0/2512639817 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f0bb40105a0 con 0x7f0bc0107a50 2023-07-21T19:36:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09f0a9e0 con 0x559d09fc0ae0 2023-07-21T19:36:14.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.234+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09f03e70 con 0x559d09e0e6b0 2023-07-21T19:36:14.222 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42860/0 (socket says 172.21.15.5:42860) 2023-07-21T19:36:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 -- 172.21.15.5:0/1152452356 learned_addr learned my addr 172.21.15.5:0/1152452356 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 -- 172.21.15.5:0/1152452356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 msgr2=0x559d09fba850 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:36:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fba850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.223 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 -- 172.21.15.5:0/1152452356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fbad90 msgr2=0x559d09fbb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fbad90 0x559d09fbb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.235+0000 7f0bbffff700 1 -- 172.21.15.5:0/1152452356 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d09d31d80 con 0x559d09e0e6b0 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.236+0000 7f0bbffff700 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8003e30 tx=0x7f0bb8009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2c2490b878a0ef69 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.236+0000 7f0ba5ffb700 1 -- 172.21.15.5:0/1152452356 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb80092d0 con 0x559d09e0e6b0 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.237+0000 7f0ba5ffb700 1 -- 172.21.15.5:0/1152452356 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bb800f040 con 0x559d09e0e6b0 2023-07-21T19:36:14.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.237+0000 7f0ba5ffb700 1 -- 172.21.15.5:0/1152452356 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb800e750 con 0x559d09e0e6b0 2023-07-21T19:36:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.238+0000 7f0bd9818b80 1 -- 172.21.15.5:0/1152452356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 msgr2=0x559d09fc05a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.238+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8003e30 tx=0x7f0bb8009920 comp rx=0 tx=0).stop 2023-07-21T19:36:14.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.238+0000 7f0bbcff9700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 0x7f0ba80a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.238+0000 7f0bbcff9700 1 -- 172.21.15.5:0/2512639817 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f0bb4014070 con 0x7f0bc0107a50 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bd9818b80 1 -- 172.21.15.5:0/1152452356 shutdown_connections 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fbad90 0x559d09fbb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09e0e6b0 0x559d09fc05a0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/1152452356 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fba850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bd9818b80 1 -- 172.21.15.5:0/1152452356 >> 172.21.15.5:0/1152452356 conn(0x559d09fa7be0 msgr2=0x559d09facf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.239+0000 7f0bbffff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 0x7f0ba80a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.240+0000 7f0bd9818b80 1 -- 172.21.15.5:0/1152452356 shutdown_connections 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.240+0000 7f0bd9818b80 1 -- 172.21.15.5:0/1152452356 wait complete. 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.241+0000 7f0bbffff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 0x7f0ba80a3870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007d30 tx=0x7f0bb8000f40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.241+0000 7f0bd9818b80 1 -- start start 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 Processor -- start 2023-07-21T19:36:14.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 0x559d09fc6de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09ea3a60 con 0x559d09fc0ae0 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09ea3760 con 0x559d09e0e6b0 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.242+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09d878c0 con 0x559d09fbad90 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bd9818b80 10 client.0 _ll_register_callbacks cb 0x559d09ea1700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:14.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bbffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40246/0 (socket says 172.21.15.5:40246) 2023-07-21T19:36:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bbffff700 1 -- 172.21.15.5:0/4229162472 learned_addr learned my addr 172.21.15.5:0/4229162472 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.243+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 0x559d09fc6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/4229162472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 msgr2=0x559d09fc40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/4229162472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 msgr2=0x559d09fc4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/4229162472 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bb8003ae0 con 0x559d09e0e6b0 2023-07-21T19:36:14.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.244+0000 7f0bbffff700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 0x559d09fc6de0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098e0 tx=0x7f0bb0005320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cc139ed807f7d442 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0b9effd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb001d070 con 0x559d09e0e6b0 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f0bb0003a90 con 0x559d09e0e6b0 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559d09e5d490 con 0x559d09e0e6b0 2023-07-21T19:36:14.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.245+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bb0003c50 con 0x559d09e0e6b0 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 10 client.30179 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x559d09d1ac00 con 0x559d09e0e6b0 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559d09e5d490 con 0x559d09e0e6b0 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bb0004500 con 0x559d09e0e6b0 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 10 client.30179 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 10 client.30179 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:14.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 20 client.30179 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d09fd4360 0x559d09fd6770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 4 client.30179 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.246+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559d09efce50 con 0x559d09fd4360 2023-07-21T19:36:14.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.247+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d09fd4360 0x559d09fd6770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.248+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d09fd4360 0x559d09fd6770 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14c2e573bbc1067b server_cookie=de5153b7e15ccfb8 in_seq=0 out_seq=0 2023-07-21T19:36:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.248+0000 7f0b9effd700 10 client.30179 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:14.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.249+0000 7f0b9effd700 1 -- 172.21.15.5:0/4229162472 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559d09efce50 con 0x559d09fd4360 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.249+0000 7f0b9effd700 10 client.30179 handle_command_reply: tid=0 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.249+0000 7f0bd9818b80 1 client.30179 shutdown 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 msgr2=0x559d09fc6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 0x559d09fc6de0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0bb00098e0 tx=0x7f0bb0005320 comp rx=0 tx=0).stop 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 shutdown_connections 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d09fd4360 0x559d09fd6770 crc :-1 s=READY pgs=525 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09fc0ae0 0x559d09fc4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09e0e6b0 0x559d09fc6de0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/4229162472 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 >> 172.21.15.5:0/4229162472 conn(0x559d09fa7be0 msgr2=0x559d09fabc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0b9effd700 0 client.30179 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 shutdown_connections 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.250+0000 7f0bd9818b80 1 -- 172.21.15.5:0/4229162472 wait complete. 2023-07-21T19:36:14.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.251+0000 7f0bd9818b80 20 client.30179 trim_cache size 0 max 16384 2023-07-21T19:36:14.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.275+0000 7f0bd9818b80 1 Processor -- start 2023-07-21T19:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 -- start start 2023-07-21T19:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fbad90 0x559d09fc6980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 0x559d09fc3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09fa9d00 con 0x559d09e0e6b0 2023-07-21T19:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09fbee40 con 0x559d09fbad90 2023-07-21T19:36:14.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.276+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09fa4b60 con 0x559d09fc0ae0 2023-07-21T19:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40250/0 (socket says 172.21.15.5:40250) 2023-07-21T19:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/388733282 learned_addr learned my addr 172.21.15.5:0/388733282 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbffff700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 0x559d09fc3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fbad90 0x559d09fc6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/388733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 msgr2=0x559d09fc3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 0x559d09fc3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/388733282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fbad90 msgr2=0x559d09fc6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fbad90 0x559d09fc6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/388733282 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559d09d581f0 con 0x559d09e0e6b0 2023-07-21T19:36:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.277+0000 7f0bbffff700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 0x559d09fc3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:36:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.278+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009ce0 tx=0x7f0bb0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f679871cef48b457 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.278+0000 7f0ba67fc700 1 -- 172.21.15.5:0/388733282 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb001d040 con 0x559d09e0e6b0 2023-07-21T19:36:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.278+0000 7f0ba67fc700 1 -- 172.21.15.5:0/388733282 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bb001e040 con 0x559d09e0e6b0 2023-07-21T19:36:14.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.278+0000 7f0ba67fc700 1 -- 172.21.15.5:0/388733282 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb000b710 con 0x559d09e0e6b0 2023-07-21T19:36:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 -- 172.21.15.5:0/388733282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 msgr2=0x559d09facc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0009ce0 tx=0x7f0bb0009310 comp rx=0 tx=0).stop 2023-07-21T19:36:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 -- 172.21.15.5:0/388733282 shutdown_connections 2023-07-21T19:36:14.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fc0ae0 0x559d09fc3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09facc60 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/388733282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fbad90 0x559d09fc6980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 -- 172.21.15.5:0/388733282 >> 172.21.15.5:0/388733282 conn(0x559d09fa7be0 msgr2=0x559d09fbe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 -- 172.21.15.5:0/388733282 shutdown_connections 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.279+0000 7f0bd9818b80 1 -- 172.21.15.5:0/388733282 wait complete. 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.280+0000 7f0bd9818b80 1 -- start start 2023-07-21T19:36:14.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.280+0000 7f0bd9818b80 1 Processor -- start 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc99a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fc9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d09fcbaa0 con 0x559d09e0e6b0 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0a027250 con 0x559d09fc0ae0 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559d0a027410 con 0x559d09fbad90 2023-07-21T19:36:14.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.281+0000 7f0bd9818b80 10 client.0 _ll_register_callbacks cb 0x7f0bb0009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:14.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:40266/0 (socket says 172.21.15.5:40266) 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/2609339104 learned_addr learned my addr 172.21.15.5:0/2609339104 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/2609339104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 msgr2=0x559d09fc99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.282+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 msgr2=0x559d09fc9ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:36:14.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fc9ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0bbf7fe700 1 -- 172.21.15.5:0/2609339104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559d09d31d80 con 0x559d09e0e6b0 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0bbf7fe700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb0009950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72cbbc7af0592a8d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.283+0000 7f0b9effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bb0042030 con 0x559d09e0e6b0 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(134..134 src has 1..134) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f0bb0043070 con 0x559d09e0e6b0 2023-07-21T19:36:14.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0bb800f040 con 0x559d09e0e6b0 2023-07-21T19:36:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bb0004e50 con 0x559d09e0e6b0 2023-07-21T19:36:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0bd9818b80 10 client.20397 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.284+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x559d09d1ac00 con 0x559d09e0e6b0 2023-07-21T19:36:14.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0bb800f040 con 0x559d09e0e6b0 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0bb00173f0 con 0x559d09e0e6b0 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 10 client.20397 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 10 client.20397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 20 client.20397 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d0a028650 0x559d0a02aa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 4 client.20397 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:36:14.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559d09ebeff0 con 0x559d0a028650 2023-07-21T19:36:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.285+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d0a028650 0x559d0a02aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.286+0000 7f0bbeffd700 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d0a028650 0x559d0a02aa30 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=12ef36d3e234aa5b server_cookie=bb49fa7a7e91e436 in_seq=0 out_seq=0 2023-07-21T19:36:14.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.286+0000 7f0b9effd700 10 client.20397 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.333+0000 7f0b9effd700 1 -- 172.21.15.5:0/2609339104 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559d09ebeff0 con 0x559d0a028650 2023-07-21T19:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.333+0000 7f0b9effd700 10 client.20397 handle_command_reply: tid=0 2023-07-21T19:36:14.319 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.333+0000 7f0bd9818b80 1 client.20397 shutdown 2023-07-21T19:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.333+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 msgr2=0x559d09fc9460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.333+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f0bb0006010 tx=0x7f0bb0009950 comp rx=0 tx=0).stop 2023-07-21T19:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 shutdown_connections 2023-07-21T19:36:14.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559d09fc0ae0 0x559d09fc9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559d0a028650 0x559d0a02aa30 crc :-1 s=READY pgs=526 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559d09e0e6b0 0x559d09fc9460 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 --2- 172.21.15.5:0/2609339104 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559d09fbad90 0x559d09fc99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 >> 172.21.15.5:0/2609339104 conn(0x559d09fa7be0 msgr2=0x559d09fbf250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0b9effd700 0 client.20397 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 shutdown_connections 2023-07-21T19:36:14.321 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.334+0000 7f0bd9818b80 1 -- 172.21.15.5:0/2609339104 wait complete. 2023-07-21T19:36:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.335+0000 7f0bd9818b80 20 client.20397 trim_cache size 0 max 16384 2023-07-21T19:36:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:36:14.322 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:36:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:36:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:36:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.336+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 msgr2=0x7f0ba80a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.323 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.336+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 0x7f0ba80a3870 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8007d30 tx=0x7f0bb8000f40 comp rx=0 tx=0).stop 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.336+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 msgr2=0x7f0bc01a6b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.336+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7f0bb400e9a0 tx=0x7f0bb400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 shutdown_connections 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0bc0112920 0x7f0bc01a03f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0bc006d8c0 0x7f0bc01a65d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f0ba80a13c0 0x7f0ba80a3870 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 --2- 172.21.15.5:0/2512639817 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0bc0107a50 0x7f0bc01a6b10 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.337+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 >> 172.21.15.5:0/2512639817 conn(0x7f0bc006c270 msgr2=0x7f0bc0071770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.338+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 shutdown_connections 2023-07-21T19:36:14.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:14.338+0000 7f0bc5dff700 1 -- 172.21.15.5:0/2512639817 wait complete. 2023-07-21T19:36:15.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:15 smithi156 ceph-mon[128316]: pgmap v908: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:16.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:15 smithi101 ceph-mon[128136]: pgmap v908: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:16.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:15 smithi005 ceph-mon[126460]: pgmap v908: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 585.0 s, 249.8 tps, lat 421.149 ms stddev 107.150 2023-07-21T19:36:17.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:17 smithi156 ceph-mon[128316]: pgmap v909: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:18.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:17 smithi101 ceph-mon[128136]: pgmap v909: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:18.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:17 smithi005 ceph-mon[126460]: pgmap v909: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:36:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:18 smithi005 ceph-mon[126460]: osdmap e135: 12 total, 12 up, 12 in 2023-07-21T19:36:19.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:18 smithi005 ceph-mon[126460]: pgmap v911: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:19.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:18 smithi156 ceph-mon[128316]: osdmap e135: 12 total, 12 up, 12 in 2023-07-21T19:36:19.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:18 smithi156 ceph-mon[128316]: pgmap v911: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:19.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:18 smithi101 ceph-mon[128136]: osdmap e135: 12 total, 12 up, 12 in 2023-07-21T19:36:19.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:18 smithi101 ceph-mon[128136]: pgmap v911: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:21.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 590.0 s, 230.2 tps, lat 433.150 ms stddev 132.340 2023-07-21T19:36:22.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:21 smithi101 ceph-mon[128136]: pgmap v912: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:22.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:21 smithi005 ceph-mon[126460]: pgmap v912: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:22.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:21 smithi156 ceph-mon[128316]: pgmap v912: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 589 op/s 2023-07-21T19:36:24.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:23 smithi101 ceph-mon[128136]: pgmap v913: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 612 op/s 2023-07-21T19:36:24.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:23 smithi005 ceph-mon[126460]: pgmap v913: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 612 op/s 2023-07-21T19:36:24.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:23 smithi156 ceph-mon[128316]: pgmap v913: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.2 MiB/s wr, 612 op/s 2023-07-21T19:36:25.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:24 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:36:25.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:24 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:36:25.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:24 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:36:26.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:25 smithi101 ceph-mon[128136]: pgmap v914: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:26.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:25 smithi005 ceph-mon[126460]: pgmap v914: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:26.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:25 smithi156 ceph-mon[128316]: pgmap v914: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 595.0 s, 208.0 tps, lat 481.209 ms stddev 139.598 2023-07-21T19:36:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.607+0000 7f16bc5ff700 1 -- 172.21.15.5:0/191367379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4108090 msgr2=0x7f16b4110580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.607+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/191367379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4108090 0x7f16b4110580 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009f90 tx=0x7f16b000b330 comp rx=0 tx=0).stop 2023-07-21T19:36:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 -- 172.21.15.5:0/191367379 shutdown_connections 2023-07-21T19:36:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/191367379 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b4110b50 0x7f16b4112f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/191367379 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16b4107780 0x7f16b4107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/191367379 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4108090 0x7f16b4110580 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 -- 172.21.15.5:0/191367379 >> 172.21.15.5:0/191367379 conn(0x7f16b406c270 msgr2=0x7f16b406c670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 -- 172.21.15.5:0/191367379 shutdown_connections 2023-07-21T19:36:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.608+0000 7f16bc5ff700 1 -- 172.21.15.5:0/191367379 wait complete. 2023-07-21T19:36:27.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.610+0000 7f16bc5ff700 1 Processor -- start 2023-07-21T19:36:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.610+0000 7f16bc5ff700 1 -- start start 2023-07-21T19:36:27.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16b4107780 0x7f16b4113f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b4108090 0x7f16b4114440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b4114ca0 con 0x7f16b4110b50 2023-07-21T19:36:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41a0430 con 0x7f16b4107780 2023-07-21T19:36:27.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.611+0000 7f16bc5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16b41a05f0 con 0x7f16b4108090 2023-07-21T19:36:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50772/0 (socket says 172.21.15.5:50772) 2023-07-21T19:36:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 -- 172.21.15.5:0/2767864674 learned_addr learned my addr 172.21.15.5:0/2767864674 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 -- 172.21.15.5:0/2767864674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b4108090 msgr2=0x7f16b4114440 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:36:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b4108090 0x7f16b4114440 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 -- 172.21.15.5:0/2767864674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16b4107780 msgr2=0x7f16b4113f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:36:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16b4107780 0x7f16b4113f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.613+0000 7f16bab9c700 1 -- 172.21.15.5:0/2767864674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b0009c40 con 0x7f16b4110b50 2023-07-21T19:36:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.614+0000 7f16bab9c700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0046e0 tx=0x7f16ac009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.615+0000 7f16a77fe700 1 -- 172.21.15.5:0/2767864674 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac004830 con 0x7f16b4110b50 2023-07-21T19:36:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.615+0000 7f16a77fe700 1 -- 172.21.15.5:0/2767864674 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16ac010040 con 0x7f16b4110b50 2023-07-21T19:36:27.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.615+0000 7f16a77fe700 1 -- 172.21.15.5:0/2767864674 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ac00e780 con 0x7f16b4110b50 2023-07-21T19:36:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.616+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16b41a08d0 con 0x7f16b4110b50 2023-07-21T19:36:27.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.616+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b41a0d90 con 0x7f16b4110b50 2023-07-21T19:36:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.622+0000 7f16a77fe700 1 -- 172.21.15.5:0/2767864674 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f16ac00a040 con 0x7f16b4110b50 2023-07-21T19:36:27.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.623+0000 7f16cfe1fb80 1 Processor -- start 2023-07-21T19:36:27.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.623+0000 7f16cfe1fb80 1 -- start start 2023-07-21T19:36:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b07040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b07580 0x565166b079d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166a0cc50 con 0x565166b0d2d0 2023-07-21T19:36:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166a5f9e0 con 0x5651669636b0 2023-07-21T19:36:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.624+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166a58e70 con 0x565166b07580 2023-07-21T19:36:27.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.625+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.625+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43900/0 (socket says 172.21.15.5:43900) 2023-07-21T19:36:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.625+0000 7f16b9b9a700 1 -- 172.21.15.5:0/3686871507 learned_addr learned my addr 172.21.15.5:0/3686871507 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:27.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.625+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b07580 0x565166b079d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16a77fe700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 0x7f16a00a3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 -- 172.21.15.5:0/3686871507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b07580 msgr2=0x565166b079d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b07580 0x565166b079d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 -- 172.21.15.5:0/3686871507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 msgr2=0x565166b07040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b07040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 -- 172.21.15.5:0/3686871507 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x565166886d80 con 0x5651669636b0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005bd0 tx=0x7f16b0005b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=273060de958e8750 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16bab9c700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 0x7f16a00a3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16a77fe700 1 -- 172.21.15.5:0/2767864674 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f16ac0cc250 con 0x7f16b4110b50 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.626+0000 7f16a4bfe700 1 -- 172.21.15.5:0/3686871507 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000e040 con 0x5651669636b0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.627+0000 7f16a4bfe700 1 -- 172.21.15.5:0/3686871507 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b0007720 con 0x5651669636b0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.627+0000 7f16a4bfe700 1 -- 172.21.15.5:0/3686871507 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000bc50 con 0x5651669636b0 2023-07-21T19:36:27.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/3686871507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 msgr2=0x565166b0cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f16b0005bd0 tx=0x7f16b0005b00 comp rx=0 tx=0).stop 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16bab9c700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 0x7f16a00a3920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16ac098fc0 tx=0x7f16ac005280 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/3686871507 shutdown_connections 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b07580 0x565166b079d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b0cd90 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/3686871507 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b07040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/3686871507 >> 172.21.15.5:0/3686871507 conn(0x565166afcbe0 msgr2=0x565166b01f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/3686871507 shutdown_connections 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.628+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/3686871507 wait complete. 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.629+0000 7f16cfe1fb80 1 -- start start 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.629+0000 7f16cfe1fb80 1 Processor -- start 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b07580 0x565166b16c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b10a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651669f8a60 con 0x565166b0d2d0 2023-07-21T19:36:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651669f8760 con 0x565166b07580 2023-07-21T19:36:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651668dc8c0 con 0x5651669636b0 2023-07-21T19:36:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16cfe1fb80 10 client.0 _ll_register_callbacks cb 0x5651669f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:27.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.630+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36062/0 (socket says 172.21.15.5:36062) 2023-07-21T19:36:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1345965786 learned_addr learned my addr 172.21.15.5:0/1345965786 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16ba39b700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b07580 0x565166b16c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b10a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b07580 msgr2=0x565166b16c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b07580 0x565166b16c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1345965786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 msgr2=0x565166b10a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b10a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.631+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1345965786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16b4113be0 con 0x5651669636b0 2023-07-21T19:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.632+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009550 tx=0x7f16b0007b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7faa8cd52d86bc0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.632+0000 7f169dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:36:27.625 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.632+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000e070 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.632+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f16b0004c90 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.632+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651669b2490 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.633+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b0004e50 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.633+0000 7f16cfe1fb80 10 client.30118 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.633+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56516686fc00 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651669b2490 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f16b0019040 con 0x5651669636b0 2023-07-21T19:36:27.626 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 10 client.30118 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 10 client.30118 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 20 client.30118 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b20780 0x565166b22b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 4 client.30118 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x565166a51e50 con 0x565166b20780 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.634+0000 7f16ba39b700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b20780 0x565166b22b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.627 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.635+0000 7f169dffb700 10 client.30118 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.635+0000 7f16ba39b700 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b20780 0x565166b22b90 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94a0486cfb671a81 server_cookie=2da02da3a7a88ddc in_seq=0 out_seq=0 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.636+0000 7f169dffb700 1 -- 172.21.15.5:0/1345965786 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x565166a51e50 con 0x565166b20780 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.636+0000 7f169dffb700 10 client.30118 handle_command_reply: tid=0 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.636+0000 7f16cfe1fb80 1 client.30118 shutdown 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.636+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 msgr2=0x565166b16720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.636+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009550 tx=0x7f16b0007b40 comp rx=0 tx=0).stop 2023-07-21T19:36:27.628 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 shutdown_connections 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b20780 0x565166b22b90 crc :-1 s=READY pgs=527 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b0d2d0 0x565166b10a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b16720 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1345965786 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b07580 0x565166b16c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 >> 172.21.15.5:0/1345965786 conn(0x565166afcbe0 msgr2=0x565166b014e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f169dffb700 0 client.30118 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 shutdown_connections 2023-07-21T19:36:27.629 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.637+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1345965786 wait complete. 2023-07-21T19:36:27.630 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.638+0000 7f16cfe1fb80 20 client.30118 trim_cache size 0 max 16384 2023-07-21T19:36:27.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 Processor -- start 2023-07-21T19:36:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 -- start start 2023-07-21T19:36:27.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b0c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b0d2d0 0x565166b01c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651669b2490 con 0x565166b07580 2023-07-21T19:36:27.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166afed00 con 0x5651669636b0 2023-07-21T19:36:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.668+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166af9bb0 con 0x565166b0d2d0 2023-07-21T19:36:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16ba39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b0c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43926/0 (socket says 172.21.15.5:43926) 2023-07-21T19:36:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1847671761 learned_addr learned my addr 172.21.15.5:0/1847671761 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:27.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b0d2d0 0x565166b01c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1847671761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b0d2d0 msgr2=0x565166b01c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b0d2d0 0x565166b01c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1847671761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 msgr2=0x565166b0c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b0c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.669+0000 7f16b9b9a700 1 -- 172.21.15.5:0/1847671761 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651668ad1f0 con 0x5651669636b0 2023-07-21T19:36:27.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.670+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009520 tx=0x7f16b0000f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a2171afdad80393b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.670+0000 7f16a53ff700 1 -- 172.21.15.5:0/1847671761 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b000e040 con 0x5651669636b0 2023-07-21T19:36:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.670+0000 7f16a53ff700 1 -- 172.21.15.5:0/1847671761 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b0007bb0 con 0x5651669636b0 2023-07-21T19:36:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.670+0000 7f16a53ff700 1 -- 172.21.15.5:0/1847671761 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0003c60 con 0x5651669636b0 2023-07-21T19:36:27.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1847671761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 msgr2=0x565166b07040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f16b0009520 tx=0x7f16b0000f40 comp rx=0 tx=0).stop 2023-07-21T19:36:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1847671761 shutdown_connections 2023-07-21T19:36:27.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x565166b0d2d0 0x565166b01c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651669636b0 0x565166b07040 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/1847671761 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b0c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1847671761 >> 172.21.15.5:0/1847671761 conn(0x565166afcbe0 msgr2=0x565166b0bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1847671761 shutdown_connections 2023-07-21T19:36:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.671+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/1847671761 wait complete. 2023-07-21T19:36:27.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.672+0000 7f16cfe1fb80 1 -- start start 2023-07-21T19:36:27.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.672+0000 7f16cfe1fb80 1 Processor -- start 2023-07-21T19:36:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b6c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b0d2d0 0x565166b15fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166b165a0 con 0x565166b07580 2023-07-21T19:36:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166b16710 con 0x565166b0d2d0 2023-07-21T19:36:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x565166b168d0 con 0x5651669636b0 2023-07-21T19:36:27.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16cfe1fb80 10 client.0 _ll_register_callbacks cb 0x7f16b000bd00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16b9b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36072/0 (socket says 172.21.15.5:36072) 2023-07-21T19:36:27.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.673+0000 7f16b9b9a700 1 -- 172.21.15.5:0/2178092738 learned_addr learned my addr 172.21.15.5:0/2178092738 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b0d2d0 0x565166b15fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16ba39b700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b6c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 -- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b0d2d0 msgr2=0x565166b15fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b0d2d0 0x565166b15fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 -- 172.21.15.5:0/2178092738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 msgr2=0x565166b6c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b6c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 -- 172.21.15.5:0/2178092738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56516686fc00 con 0x5651669636b0 2023-07-21T19:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f16b9b9a700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007b80 tx=0x7f16b0007eb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=39c55e74b263eb61 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.674+0000 7f169dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:36:27.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b0004a90 con 0x5651669636b0 2023-07-21T19:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f16b0004bf0 con 0x5651669636b0 2023-07-21T19:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f16b0007720 con 0x5651669636b0 2023-07-21T19:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f16b0004db0 con 0x5651669636b0 2023-07-21T19:36:27.678 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f16cfe1fb80 10 client.30124 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.675+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x565166885250 con 0x5651669636b0 2023-07-21T19:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f16b0007720 con 0x5651669636b0 2023-07-21T19:36:27.679 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f16b000b3a0 con 0x5651669636b0 2023-07-21T19:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 10 client.30124 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 10 client.30124 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:27.680 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 20 client.30124 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b749e0 0x565166b76df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 4 client.30124 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:36:27.681 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x565166a13ff0 con 0x565166b749e0 2023-07-21T19:36:27.682 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.676+0000 7f16ba39b700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b749e0 0x565166b76df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.677+0000 7f16ba39b700 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b749e0 0x565166b76df0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45ff2cc5cfad4ffc server_cookie=b2c315fe00a819fa in_seq=0 out_seq=0 2023-07-21T19:36:27.683 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.677+0000 7f169dffb700 10 client.30124 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:27.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.714+0000 7f169dffb700 1 -- 172.21.15.5:0/2178092738 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x565166a13ff0 con 0x565166b749e0 2023-07-21T19:36:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.714+0000 7f169dffb700 10 client.30124 handle_command_reply: tid=0 2023-07-21T19:36:27.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 client.30124 shutdown 2023-07-21T19:36:27.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 msgr2=0x565166b6bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f16b0007b80 tx=0x7f16b0007eb0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 shutdown_connections 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x565166b0d2d0 0x565166b15fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651669636b0 0x565166b6bf80 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x565166b749e0 0x565166b76df0 crc :-1 s=READY pgs=528 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 --2- 172.21.15.5:0/2178092738 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x565166b07580 0x565166b6c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 >> 172.21.15.5:0/2178092738 conn(0x565166afcbe0 msgr2=0x565166b10970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.715+0000 7f169dffb700 0 client.30124 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.716+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 shutdown_connections 2023-07-21T19:36:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.716+0000 7f16cfe1fb80 1 -- 172.21.15.5:0/2178092738 wait complete. 2023-07-21T19:36:27.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.716+0000 7f16cfe1fb80 20 client.30124 trim_cache size 0 max 16384 2023-07-21T19:36:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:36:27.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:36:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:36:27.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:36:27.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 msgr2=0x7f16a00a3920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 0x7f16a00a3920 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f16ac098fc0 tx=0x7f16ac005280 comp rx=0 tx=0).stop 2023-07-21T19:36:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 msgr2=0x7f16b41b0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:27.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f16ac0046e0 tx=0x7f16ac009d40 comp rx=0 tx=0).stop 2023-07-21T19:36:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 shutdown_connections 2023-07-21T19:36:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f16a00a1470 0x7f16a00a3920 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f16b4110b50 0x7f16b41b0c90 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f16b4107780 0x7f16b4113f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 --2- 172.21.15.5:0/2767864674 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f16b4108090 0x7f16b4114440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:27.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.717+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 >> 172.21.15.5:0/2767864674 conn(0x7f16b406c270 msgr2=0x7f16b406eed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.718+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 shutdown_connections 2023-07-21T19:36:27.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:27.718+0000 7f16bc5ff700 1 -- 172.21.15.5:0/2767864674 wait complete. 2023-07-21T19:36:28.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:27 smithi101 ceph-mon[128136]: pgmap v915: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:28.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:27 smithi005 ceph-mon[126460]: pgmap v915: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:28.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:27 smithi156 ceph-mon[128316]: pgmap v915: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.3 MiB/s wr, 750 op/s 2023-07-21T19:36:29.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:28 smithi005 ceph-mon[126460]: pgmap v916: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 754 op/s 2023-07-21T19:36:29.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:28 smithi156 ceph-mon[128316]: pgmap v916: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 754 op/s 2023-07-21T19:36:29.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:28 smithi101 ceph-mon[128136]: pgmap v916: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.4 MiB/s wr, 754 op/s 2023-07-21T19:36:29.533 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:36:29.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 -- 172.21.15.5:0/3058437185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 msgr2=0x7f8d80101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 --2- 172.21.15.5:0/3058437185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d80101070 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b0a0 tx=0x7f8d7000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 -- 172.21.15.5:0/3058437185 shutdown_connections 2023-07-21T19:36:29.872 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 --2- 172.21.15.5:0/3058437185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 0x7f8d8010c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 --2- 172.21.15.5:0/3058437185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d80101070 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 --2- 172.21.15.5:0/3058437185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 0x7f8d80109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 -- 172.21.15.5:0/3058437185 >> 172.21.15.5:0/3058437185 conn(0x7f8d800fa4b0 msgr2=0x7f8d800fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 -- 172.21.15.5:0/3058437185 shutdown_connections 2023-07-21T19:36:29.873 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.886+0000 7f8d859ff700 1 -- 172.21.15.5:0/3058437185 wait complete. 2023-07-21T19:36:29.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.887+0000 7f8d859ff700 1 Processor -- start 2023-07-21T19:36:29.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 -- start start 2023-07-21T19:36:29.874 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 0x7f8d8019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 0x7f8d80195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80196120 con 0x7f8d801015b0 2023-07-21T19:36:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80196290 con 0x7f8d80100ca0 2023-07-21T19:36:29.875 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.888+0000 7f8d859ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d80196450 con 0x7f8d8010a070 2023-07-21T19:36:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.876 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43946/0 (socket says 172.21.15.5:43946) 2023-07-21T19:36:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 -- 172.21.15.5:0/683714728 learned_addr learned my addr 172.21.15.5:0/683714728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7f7fe700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 0x7f8d80195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 0x7f8d8019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 -- 172.21.15.5:0/683714728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 msgr2=0x7f8d80195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.877 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 0x7f8d80195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 -- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 msgr2=0x7f8d8019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 0x7f8d8019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.889+0000 7f8d7effd700 1 -- 172.21.15.5:0/683714728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d70009d00 con 0x7f8d80100ca0 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d7effd700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b070 tx=0x7f8d7000bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d849fd700 1 -- 172.21.15.5:0/683714728 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7000e040 con 0x7f8d80100ca0 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d801966d0 con 0x7f8d80100ca0 2023-07-21T19:36:29.878 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d849fd700 1 -- 172.21.15.5:0/683714728 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d70009480 con 0x7f8d80100ca0 2023-07-21T19:36:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d849fd700 1 -- 172.21.15.5:0/683714728 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d700052a0 con 0x7f8d80100ca0 2023-07-21T19:36:29.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.890+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d800fea00 con 0x7f8d80100ca0 2023-07-21T19:36:29.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.895+0000 7f8d849fd700 1 -- 172.21.15.5:0/683714728 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8d7001b450 con 0x7f8d80100ca0 2023-07-21T19:36:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.897+0000 7f8d992c7b80 1 Processor -- start 2023-07-21T19:36:29.883 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.897+0000 7f8d992c7b80 1 -- start start 2023-07-21T19:36:29.884 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0636a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.885 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5df69cb0 con 0x563a5e063be0 2023-07-21T19:36:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5dfbca40 con 0x563a5e0632d0 2023-07-21T19:36:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5dfb5ed0 con 0x563a5e06ead0 2023-07-21T19:36:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.886 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50844/0 (socket says 172.21.15.5:50844) 2023-07-21T19:36:29.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36108/0 (socket says 172.21.15.5:36108) 2023-07-21T19:36:29.887 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.898+0000 7f8d7effd700 1 -- 172.21.15.5:0/23783740 learned_addr learned my addr 172.21.15.5:0/23783740 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7effd700 1 -- 172.21.15.5:0/23783740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 msgr2=0x563a5e070eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7effd700 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7effd700 1 -- 172.21.15.5:0/23783740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 msgr2=0x563a5e0636a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7effd700 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0636a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7effd700 1 -- 172.21.15.5:0/23783740 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5dde3de0 con 0x563a5e063be0 2023-07-21T19:36:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d849fd700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 0x7f8d680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.899+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:36:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.900+0000 7f8d849fd700 1 -- 172.21.15.5:0/683714728 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d700ce1b0 con 0x7f8d80100ca0 2023-07-21T19:36:29.891 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.900+0000 7f8d7f7fe700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 0x7f8d680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.900+0000 7f8d7effd700 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cb7e0 tx=0x7f8d700d61b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b5a34ed1129f7cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.900+0000 7f8d65ffb700 1 -- 172.21.15.5:0/23783740 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d70009b80 con 0x563a5e063be0 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.901+0000 7f8d65ffb700 1 -- 172.21.15.5:0/23783740 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d700975f0 con 0x563a5e063be0 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.901+0000 7f8d65ffb700 1 -- 172.21.15.5:0/23783740 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7009adf0 con 0x563a5e063be0 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.901+0000 7f8d992c7b80 1 -- 172.21.15.5:0/23783740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 msgr2=0x563a5e06e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.901+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f8d700cb7e0 tx=0x7f8d700d61b0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d7f7fe700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 0x7f8d680a37f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d8019c740 tx=0x7f8d78009870 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 -- 172.21.15.5:0/23783740 shutdown_connections 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e070eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0636a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/23783740 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e06e590 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 -- 172.21.15.5:0/23783740 >> 172.21.15.5:0/23783740 conn(0x563a5e059da0 msgr2=0x563a5e05f6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 -- 172.21.15.5:0/23783740 shutdown_connections 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.902+0000 7f8d992c7b80 1 -- 172.21.15.5:0/23783740 wait complete. 2023-07-21T19:36:29.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.903+0000 7f8d992c7b80 1 -- start start 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.903+0000 7f8d992c7b80 1 Processor -- start 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 0x563a5e06f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e06ead0 0x563a5e079f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5df55ac0 con 0x563a5e0632d0 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5df557c0 con 0x563a5e06ead0 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5de39920 con 0x563a5e063be0 2023-07-21T19:36:29.894 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.904+0000 7f8d992c7b80 10 client.0 _ll_register_callbacks cb 0x563a5df53760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7effd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e06eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50858/0 (socket says 172.21.15.5:50858) 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 0x563a5e06f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7effd700 1 -- 172.21.15.5:0/2130456183 learned_addr learned my addr 172.21.15.5:0/2130456183 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7f7fe700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e06ead0 0x563a5e079f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e06ead0 msgr2=0x563a5e079f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.895 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e06ead0 0x563a5e079f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2130456183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 msgr2=0x563a5e06eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e06eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.896 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.905+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2130456183 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d7400b8e0 con 0x563a5e063be0 2023-07-21T19:36:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 0x563a5e06f3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005950 tx=0x7f8d740045a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d253c853abd97b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d5effd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:36:29.897 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016070 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d74003d00 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563a5df0f4f0 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d74004830 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d992c7b80 10 client.30136 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.906+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f8d800fea00 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563a5df0f4f0 con 0x563a5e063be0 2023-07-21T19:36:29.898 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d74011440 con 0x563a5e063be0 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 10 client.30136 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 10 client.30136 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 20 client.30136 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e07e200 0x563a5e080610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 4 client.30136 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.907+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563a5dfaeeb0 con 0x563a5e07e200 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.908+0000 7f8d7effd700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e07e200 0x563a5e080610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.899 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.909+0000 7f8d7effd700 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e07e200 0x563a5e080610 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e52780a3f442d899 server_cookie=9ca7d47e2a42944d in_seq=0 out_seq=0 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.909+0000 7f8d5effd700 10 client.30136 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.910+0000 7f8d5effd700 1 -- 172.21.15.5:0/2130456183 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563a5dfaeeb0 con 0x563a5e07e200 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.910+0000 7f8d5effd700 10 client.30136 handle_command_reply: tid=0 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.910+0000 7f8d992c7b80 1 client.30136 shutdown 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.910+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 msgr2=0x563a5e06f3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.910+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 0x563a5e06f3e0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005950 tx=0x7f8d740045a0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.900 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 shutdown_connections 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e06ead0 0x563a5e079f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e07e200 0x563a5e080610 crc :-1 s=READY pgs=529 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e06eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2130456183 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e063be0 0x563a5e06f3e0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 >> 172.21.15.5:0/2130456183 conn(0x563a5e059da0 msgr2=0x563a5e05f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.901 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d5effd700 0 client.30136 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 shutdown_connections 2023-07-21T19:36:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2130456183 wait complete. 2023-07-21T19:36:29.902 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.911+0000 7f8d992c7b80 20 client.30136 trim_cache size 0 max 16384 2023-07-21T19:36:29.924 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.938+0000 7f8d992c7b80 1 Processor -- start 2023-07-21T19:36:29.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 -- start start 2023-07-21T19:36:29.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e068400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e0694d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e059970 con 0x563a5e0632d0 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e06e220 con 0x563a5e063be0 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.939+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e06c210 con 0x563a5e06ead0 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43976/0 (socket says 172.21.15.5:43976) 2023-07-21T19:36:29.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2243011748 learned_addr learned my addr 172.21.15.5:0/2243011748 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7f7fe700 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e0694d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2243011748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 msgr2=0x563a5e0694d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.940+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e0694d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2243011748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 msgr2=0x563a5e068400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e068400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/2243011748 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563a5e05b5e0 con 0x563a5e063be0 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d7400b480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a139f31ec22b94e2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d667fc700 1 -- 172.21.15.5:0/2243011748 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016070 con 0x563a5e063be0 2023-07-21T19:36:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.941+0000 7f8d667fc700 1 -- 172.21.15.5:0/2243011748 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d74017040 con 0x563a5e063be0 2023-07-21T19:36:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d667fc700 1 -- 172.21.15.5:0/2243011748 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d7401bb60 con 0x563a5e063be0 2023-07-21T19:36:29.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2243011748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 msgr2=0x563a5e068f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8d74000c00 tx=0x7f8d7400b480 comp rx=0 tx=0).stop 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2243011748 shutdown_connections 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e0694d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e0632d0 0x563a5e068400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/2243011748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e063be0 0x563a5e068f90 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.942+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2243011748 >> 172.21.15.5:0/2243011748 conn(0x563a5e059da0 msgr2=0x563a5e06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.943+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2243011748 shutdown_connections 2023-07-21T19:36:29.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.943+0000 7f8d992c7b80 1 -- 172.21.15.5:0/2243011748 wait complete. 2023-07-21T19:36:29.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.943+0000 7f8d992c7b80 1 -- start start 2023-07-21T19:36:29.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.944+0000 7f8d992c7b80 1 Processor -- start 2023-07-21T19:36:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.944+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0738d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e071d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e072430 con 0x563a5e063be0 2023-07-21T19:36:29.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e0725a0 con 0x563a5e0632d0 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563a5e072760 con 0x563a5e06ead0 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d992c7b80 10 client.0 _ll_register_callbacks cb 0x7f8d7400bbc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7e7fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50874/0 (socket says 172.21.15.5:50874) 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/50624853 learned_addr learned my addr 172.21.15.5:0/50624853 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:29.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7f7fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:36130/0 (socket says 172.21.15.5:36130) 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.945+0000 7f8d7effd700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0738d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/50624853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 msgr2=0x563a5e071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 msgr2=0x563a5e0738d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0738d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 -- 172.21.15.5:0/50624853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d800fea00 con 0x563a5e063be0 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d7e7fc700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8d740118f0 tx=0x7f8d74003d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6da36a5e5271b3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:29.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.946+0000 7f8d5effd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:36:29.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.947+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74016040 con 0x563a5e063be0 2023-07-21T19:36:29.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.947+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8d7401b670 con 0x563a5e063be0 2023-07-21T19:36:29.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.947+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8d700975f0 con 0x563a5e063be0 2023-07-21T19:36:29.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.947+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d7401b830 con 0x563a5e063be0 2023-07-21T19:36:29.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.947+0000 7f8d992c7b80 10 client.20427 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563a5ddccc60 con 0x563a5e063be0 2023-07-21T19:36:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8d700975f0 con 0x563a5e063be0 2023-07-21T19:36:29.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f8d7403c070 con 0x563a5e063be0 2023-07-21T19:36:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 10 client.20427 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 10 client.20427 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:29.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 20 client.20427 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e0db730 0x563a5e0ddb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 4 client.20427 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:36:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.948+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x563a5df71050 con 0x563a5e0db730 2023-07-21T19:36:29.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.949+0000 7f8d7effd700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e0db730 0x563a5e0ddb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.949+0000 7f8d7effd700 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e0db730 0x563a5e0ddb40 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24e7b2aebb699b18 server_cookie=6c57250de928976c in_seq=0 out_seq=0 2023-07-21T19:36:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.949+0000 7f8d5effd700 10 client.20427 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d5effd700 1 -- 172.21.15.5:0/50624853 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x563a5df71050 con 0x563a5e0db730 2023-07-21T19:36:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d5effd700 10 client.20427 handle_command_reply: tid=0 2023-07-21T19:36:29.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 client.20427 shutdown 2023-07-21T19:36:29.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 msgr2=0x563a5e073e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f8d740118f0 tx=0x7f8d74003d90 comp rx=0 tx=0).stop 2023-07-21T19:36:29.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 shutdown_connections 2023-07-21T19:36:29.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563a5e06ead0 0x563a5e071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563a5e0632d0 0x563a5e0738d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563a5e0db730 0x563a5e0ddb40 crc :-1 s=READY pgs=530 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 --2- 172.21.15.5:0/50624853 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563a5e063be0 0x563a5e073e10 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.950+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 >> 172.21.15.5:0/50624853 conn(0x563a5e059da0 msgr2=0x563a5e06eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.951+0000 7f8d5effd700 0 client.20427 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.951+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 shutdown_connections 2023-07-21T19:36:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.951+0000 7f8d992c7b80 1 -- 172.21.15.5:0/50624853 wait complete. 2023-07-21T19:36:29.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.951+0000 7f8d992c7b80 20 client.20427 trim_cache size 0 max 16384 2023-07-21T19:36:29.947 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:36:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:36:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:36:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:36:29.947 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:36:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:36:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:36:29.948 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:36:29.949 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:36:29.949 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:36:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.952+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 msgr2=0x7f8d680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.952+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 0x7f8d680a37f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8d8019c740 tx=0x7f8d78009870 comp rx=0 tx=0).stop 2023-07-21T19:36:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.952+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 msgr2=0x7f8d8019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:29.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.952+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8d7000b070 tx=0x7f8d7000bd60 comp rx=0 tx=0).stop 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 shutdown_connections 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8d8010a070 0x7f8d80195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8d80100ca0 0x7f8d8019ba40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8d680a1340 0x7f8d680a37f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 --2- 172.21.15.5:0/683714728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8d801015b0 0x7f8d8019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 >> 172.21.15.5:0/683714728 conn(0x7f8d800fa4b0 msgr2=0x7f8d800fb110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:29.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.953+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 shutdown_connections 2023-07-21T19:36:29.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:36:29.954+0000 7f8d859ff700 1 -- 172.21.15.5:0/683714728 wait complete. 2023-07-21T19:36:29.964 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:36:31.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:31 smithi005 ceph-mon[126460]: pgmap v917: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-07-21T19:36:31.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:36:31.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:31 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:36:31.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:31 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:36:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:31 smithi156 ceph-mon[128316]: pgmap v917: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-07-21T19:36:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:36:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:31 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:36:31.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:31 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:36:31.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 600.0 s, 252.0 tps, lat 399.018 ms stddev 106.086 2023-07-21T19:36:32.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:31 smithi101 ceph-mon[128136]: pgmap v917: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 6.0 MiB/s wr, 614 op/s 2023-07-21T19:36:32.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:36:32.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:31 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:36:32.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:31 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:36:33.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:33 smithi005 ceph-mon[126460]: pgmap v918: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2023-07-21T19:36:33.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:33 smithi156 ceph-mon[128316]: pgmap v918: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2023-07-21T19:36:34.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:33 smithi101 ceph-mon[128136]: pgmap v918: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 704 op/s 2023-07-21T19:36:35.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:35 smithi005 ceph-mon[126460]: pgmap v919: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 873 op/s 2023-07-21T19:36:35.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:35 smithi156 ceph-mon[128316]: pgmap v919: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 873 op/s 2023-07-21T19:36:36.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:35 smithi101 ceph-mon[128136]: pgmap v919: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 873 op/s 2023-07-21T19:36:36.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 605.0 s, 260.2 tps, lat 368.241 ms stddev 54.818 2023-07-21T19:36:37.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:37 smithi156 ceph-mon[128316]: pgmap v920: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 639 op/s 2023-07-21T19:36:38.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:37 smithi101 ceph-mon[128136]: pgmap v920: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 639 op/s 2023-07-21T19:36:38.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:37 smithi005 ceph-mon[126460]: pgmap v920: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.1 MiB/s wr, 639 op/s 2023-07-21T19:36:39.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:38 smithi005 ceph-mon[126460]: pgmap v921: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.4 MiB/s wr, 985 op/s 2023-07-21T19:36:39.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:38 smithi156 ceph-mon[128316]: pgmap v921: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.4 MiB/s wr, 985 op/s 2023-07-21T19:36:39.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:38 smithi101 ceph-mon[128136]: pgmap v921: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.4 MiB/s wr, 985 op/s 2023-07-21T19:36:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.948+0000 7f9dc99ff700 1 -- 172.21.15.5:0/4228616522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 msgr2=0x7f9dc4110940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.948+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/4228616522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc4110940 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009990 tx=0x7f9db4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 -- 172.21.15.5:0/4228616522 shutdown_connections 2023-07-21T19:36:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/4228616522 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410af00 0x7f9dc4117330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/4228616522 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc4110940 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/4228616522 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410a550 0x7f9dc410a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 -- 172.21.15.5:0/4228616522 >> 172.21.15.5:0/4228616522 conn(0x7f9dc406a900 msgr2=0x7f9dc406ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:40.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 -- 172.21.15.5:0/4228616522 shutdown_connections 2023-07-21T19:36:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.950+0000 7f9dc99ff700 1 -- 172.21.15.5:0/4228616522 wait complete. 2023-07-21T19:36:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.951+0000 7f9dc99ff700 1 Processor -- start 2023-07-21T19:36:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 -- start start 2023-07-21T19:36:40.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410af00 0x7f9dc41a0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc41a3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41a45d0 con 0x7f9dc410af00 2023-07-21T19:36:40.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41a4740 con 0x7f9dc410a550 2023-07-21T19:36:40.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.952+0000 7f9dc99ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9dc41a4900 con 0x7f9dc4110570 2023-07-21T19:36:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.953+0000 7f9dc37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc41a3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.953+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.953+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56142/0 (socket says 172.21.15.5:56142) 2023-07-21T19:36:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.953+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/1026039651 learned_addr learned my addr 172.21.15.5:0/1026039651 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:40.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/1026039651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 msgr2=0x7f9dc41a3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc41a3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410af00 msgr2=0x7f9dc41a0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410af00 0x7f9dc41a0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/1026039651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db40095f0 con 0x7f9dc410a550 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.954+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9db4000c00 tx=0x7f9db4004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.955+0000 7f9dc89fd700 1 -- 172.21.15.5:0/1026039651 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db401d070 con 0x7f9dc410a550 2023-07-21T19:36:40.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.955+0000 7f9dc89fd700 1 -- 172.21.15.5:0/1026039651 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9db4022470 con 0x7f9dc410a550 2023-07-21T19:36:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.955+0000 7f9dc89fd700 1 -- 172.21.15.5:0/1026039651 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db4017650 con 0x7f9dc410a550 2023-07-21T19:36:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.955+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9dc41a4b80 con 0x7f9dc410a550 2023-07-21T19:36:40.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.955+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc41ad360 con 0x7f9dc410a550 2023-07-21T19:36:40.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.960+0000 7f9dc89fd700 1 -- 172.21.15.5:0/1026039651 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f9db40225e0 con 0x7f9dc410a550 2023-07-21T19:36:40.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.964+0000 7f9dc89fd700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 0x7f9db00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.965+0000 7f9dc89fd700 1 -- 172.21.15.5:0/1026039651 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9db40ce160 con 0x7f9dc410a550 2023-07-21T19:36:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.965+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 0x7f9db00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.965+0000 7f9ddd223b80 1 Processor -- start 2023-07-21T19:36:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.965+0000 7f9ddd223b80 1 -- start start 2023-07-21T19:36:40.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 0x7f9db00a3870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005950 tx=0x7f9dac0058e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee9cae0 0x55865ee96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 0x55865ee971e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ed93c50 con 0x55865ee96d90 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ede69e0 con 0x55865ee9cae0 2023-07-21T19:36:40.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.966+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865eddfe70 con 0x55865ecea6b0 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48348/0 (socket says 172.21.15.5:48348) 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/4105442860 learned_addr learned my addr 172.21.15.5:0/4105442860 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee9cae0 0x55865ee96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 0x55865ee971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/4105442860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee9cae0 msgr2=0x55865ee96850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee9cae0 0x55865ee96850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/4105442860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 msgr2=0x55865ee971e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 0x55865ee971e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/4105442860 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55865ec0dd80 con 0x55865ecea6b0 2023-07-21T19:36:40.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.967+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 0x55865ee971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.968+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009350 tx=0x7f9db4096f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7352ae47a516b5b4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.968+0000 7f9db9ffb700 1 -- 172.21.15.5:0/4105442860 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db409abc0 con 0x55865ecea6b0 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.968+0000 7f9db9ffb700 1 -- 172.21.15.5:0/4105442860 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9db40228c0 con 0x55865ecea6b0 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.968+0000 7f9db9ffb700 1 -- 172.21.15.5:0/4105442860 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db409b720 con 0x55865ecea6b0 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 -- 172.21.15.5:0/4105442860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 msgr2=0x55865ee9c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009350 tx=0x7f9db4096f90 comp rx=0 tx=0).stop 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 -- 172.21.15.5:0/4105442860 shutdown_connections 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee96d90 0x55865ee971e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee9c5a0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/4105442860 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee9cae0 0x55865ee96850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 -- 172.21.15.5:0/4105442860 >> 172.21.15.5:0/4105442860 conn(0x55865ee83be0 msgr2=0x55865ee88f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:40.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 -- 172.21.15.5:0/4105442860 shutdown_connections 2023-07-21T19:36:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.969+0000 7f9ddd223b80 1 -- 172.21.15.5:0/4105442860 wait complete. 2023-07-21T19:36:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.970+0000 7f9ddd223b80 1 -- start start 2023-07-21T19:36:40.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.970+0000 7f9ddd223b80 1 Processor -- start 2023-07-21T19:36:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ecea6b0 0x55865eea5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 0x55865ee9fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ed7fa60 con 0x55865ecea6b0 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ed7f760 con 0x55865ee96d90 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ec638c0 con 0x55865ee9cae0 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9ddd223b80 10 client.0 _ll_register_callbacks cb 0x55865ed7d700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56154/0 (socket says 172.21.15.5:56154) 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.971+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1967814052 learned_addr learned my addr 172.21.15.5:0/1967814052 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ecea6b0 0x55865eea5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 0x55865ee9fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1967814052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 msgr2=0x55865ee9fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 0x55865ee9fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1967814052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ecea6b0 msgr2=0x55865eea5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ecea6b0 0x55865eea5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1967814052 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9dc41ad360 con 0x55865ee96d90 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 0x55865ee9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9dac01d6b0 tx=0x7f9dac004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=451cefee49257249 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.972+0000 7f9da2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac004a30 con 0x55865ee96d90 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9dac004b90 con 0x55865ee96d90 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55865ed39490 con 0x55865ee96d90 2023-07-21T19:36:40.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9dac00eec0 con 0x55865ee96d90 2023-07-21T19:36:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9ddd223b80 10 client.30224 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:40.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.973+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55865ebf6c00 con 0x55865ee96d90 2023-07-21T19:36:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55865ed39490 con 0x55865ee96d90 2023-07-21T19:36:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9dac018070 con 0x55865ee96d90 2023-07-21T19:36:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 10 client.30224 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 10 client.30224 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:40.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 20 client.30224 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865eeb0040 0x55865eeb2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 4 client.30224 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.974+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55865edd8e50 con 0x55865eeb0040 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.975+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865eeb0040 0x55865eeb2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.976+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865eeb0040 0x55865eeb2450 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c7ff70e706a03569 server_cookie=636dd3aa7110db5f in_seq=0 out_seq=0 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.976+0000 7f9da2ffd700 10 client.30224 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:40.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.976+0000 7f9da2ffd700 1 -- 172.21.15.5:0/1967814052 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55865edd8e50 con 0x55865eeb0040 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9da2ffd700 10 client.30224 handle_command_reply: tid=0 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 client.30224 shutdown 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 msgr2=0x55865eea5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f9dac01d6b0 tx=0x7f9dac004750 comp rx=0 tx=0).stop 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 shutdown_connections 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee9cae0 0x55865ee9fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.965 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865eeb0040 0x55865eeb2450 crc :-1 s=READY pgs=531 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ecea6b0 0x55865eea5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1967814052 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865eea5fb0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9da2ffd700 0 client.30224 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 >> 172.21.15.5:0/1967814052 conn(0x55865ee83be0 msgr2=0x55865ee87c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 shutdown_connections 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.977+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1967814052 wait complete. 2023-07-21T19:36:40.966 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:40.978+0000 7f9ddd223b80 20 client.30224 trim_cache size 0 max 16384 2023-07-21T19:36:40.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.005+0000 7f9ddd223b80 1 Processor -- start 2023-07-21T19:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.005+0000 7f9ddd223b80 1 -- start start 2023-07-21T19:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee96850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865ee98fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ee89a00 con 0x55865ee9cae0 2023-07-21T19:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ee80bb0 con 0x55865ee96d90 2023-07-21T19:36:40.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865ee977e0 con 0x55865ecea6b0 2023-07-21T19:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56164/0 (socket says 172.21.15.5:56164) 2023-07-21T19:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1105004683 learned_addr learned my addr 172.21.15.5:0/1105004683 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.006+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee96850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865ee98fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1105004683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 msgr2=0x55865ee96850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee96850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1105004683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 msgr2=0x55865ee98fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865ee98fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.007+0000 7f9dbbfff700 1 -- 172.21.15.5:0/1105004683 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55865ec341f0 con 0x55865ee96d90 2023-07-21T19:36:40.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.008+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9dac013040 tx=0x7f9dac01de80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c7c09c79cdc2c501 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.008+0000 7f9dba7fc700 1 -- 172.21.15.5:0/1105004683 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac010840 con 0x55865ee96d90 2023-07-21T19:36:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.008+0000 7f9dba7fc700 1 -- 172.21.15.5:0/1105004683 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9dac00eea0 con 0x55865ee96d90 2023-07-21T19:36:40.996 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.008+0000 7f9dba7fc700 1 -- 172.21.15.5:0/1105004683 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dac0053a0 con 0x55865ee96d90 2023-07-21T19:36:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1105004683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 msgr2=0x55865ee9fd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9dac013040 tx=0x7f9dac01de80 comp rx=0 tx=0).stop 2023-07-21T19:36:40.997 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1105004683 shutdown_connections 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865ee98fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ecea6b0 0x55865ee96850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/1105004683 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ee96d90 0x55865ee9fd50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1105004683 >> 172.21.15.5:0/1105004683 conn(0x55865ee83be0 msgr2=0x55865ee9f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1105004683 shutdown_connections 2023-07-21T19:36:40.998 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.010+0000 7f9ddd223b80 1 -- 172.21.15.5:0/1105004683 wait complete. 2023-07-21T19:36:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.011+0000 7f9ddd223b80 1 -- start start 2023-07-21T19:36:40.999 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.011+0000 7f9ddd223b80 1 Processor -- start 2023-07-21T19:36:41.001 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.012+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 0x55865eea56e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.012+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 0x55865eea5c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.012+0000 7f9ddd223b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865eea3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.012+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865eea42d0 con 0x55865ee9cae0 2023-07-21T19:36:41.002 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.012+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865eea4440 con 0x55865ecea6b0 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9ddd223b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55865eea4600 con 0x55865ee96d90 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9ddd223b80 10 client.0 _ll_register_callbacks cb 0x7f9dac00ed20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 0x55865eea5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9dbbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 0x55865eea5c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:48388/0 (socket says 172.21.15.5:48388) 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9dbbfff700 1 -- 172.21.15.5:0/2177601925 learned_addr learned my addr 172.21.15.5:0/2177601925 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9dc2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 0x55865eea56e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.013+0000 7f9dc37fe700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865eea3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:41.003 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.014+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/2177601925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 msgr2=0x55865eea5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.014+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 0x55865eea5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.014+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/2177601925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 msgr2=0x55865eea3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.014+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865eea3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.014+0000 7f9dc2ffd700 1 -- 172.21.15.5:0/2177601925 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55865ebf6c00 con 0x55865ecea6b0 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.015+0000 7f9dc2ffd700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 0x55865eea56e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009350 tx=0x7f9db4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7b243a8922a35f2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.015+0000 7f9da2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:36:41.004 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.015+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db409b260 con 0x55865ecea6b0 2023-07-21T19:36:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.015+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f9db409f290 con 0x55865ecea6b0 2023-07-21T19:36:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.015+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9db40228c0 con 0x55865ecea6b0 2023-07-21T19:36:41.005 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.017+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9db409a750 con 0x55865ecea6b0 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.017+0000 7f9ddd223b80 10 client.30230 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55865ec0c250 con 0x55865ecea6b0 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f9db40228c0 con 0x55865ecea6b0 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f9db40cddd0 con 0x55865ecea6b0 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 10 client.30230 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:41.006 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 10 client.30230 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 20 client.30230 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865ee8bd20 0x55865ee8e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 4 client.30230 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.018+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55865ed9aff0 con 0x55865ee8bd20 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.020+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865ee8bd20 0x55865ee8e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.021+0000 7f9dbbfff700 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865ee8bd20 0x55865ee8e130 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20c63db6eae2df2b server_cookie=1dc0df424bfff0e5 in_seq=0 out_seq=0 2023-07-21T19:36:41.007 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.021+0000 7f9da2ffd700 10 client.30230 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.045+0000 7f9da2ffd700 1 -- 172.21.15.5:0/2177601925 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55865ed9aff0 con 0x55865ee8bd20 2023-07-21T19:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.045+0000 7f9da2ffd700 10 client.30230 handle_command_reply: tid=0 2023-07-21T19:36:41.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.045+0000 7f9ddd223b80 1 client.30230 shutdown 2023-07-21T19:36:41.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.045+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 msgr2=0x55865eea56e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:41.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.045+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 0x55865eea56e0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9db4009350 tx=0x7f9db4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 shutdown_connections 2023-07-21T19:36:41.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55865ee8bd20 0x55865ee8e130 crc :-1 s=READY pgs=532 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55865ee9cae0 0x55865eea3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55865ecea6b0 0x55865eea56e0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 --2- 172.21.15.5:0/2177601925 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55865ee96d90 0x55865eea5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9da2ffd700 0 client.30230 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 >> 172.21.15.5:0/2177601925 conn(0x55865ee83be0 msgr2=0x55865ee884e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 shutdown_connections 2023-07-21T19:36:41.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 1 -- 172.21.15.5:0/2177601925 wait complete. 2023-07-21T19:36:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.046+0000 7f9ddd223b80 20 client.30230 trim_cache size 0 max 16384 2023-07-21T19:36:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:36:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:36:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:36:41.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:36:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.047+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 msgr2=0x7f9db00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.047+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 0x7f9db00a3870 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f9dac005950 tx=0x7f9dac0058e0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.047+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 msgr2=0x7f9dc41a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.047+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f9db4000c00 tx=0x7f9db4004730 comp rx=0 tx=0).stop 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 shutdown_connections 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f9dc4110570 0x7f9dc41a3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f9dc410a550 0x7f9dc41a04d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f9db00a13c0 0x7f9db00a3870 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 --2- 172.21.15.5:0/1026039651 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f9dc410af00 0x7f9dc41a0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:41.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 >> 172.21.15.5:0/1026039651 conn(0x7f9dc406a900 msgr2=0x7f9dc4109070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.048+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 shutdown_connections 2023-07-21T19:36:41.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:41.049+0000 7f9dc99ff700 1 -- 172.21.15.5:0/1026039651 wait complete. 2023-07-21T19:36:41.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:41 smithi005 ceph-mon[126460]: pgmap v922: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T19:36:41.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:41 smithi156 ceph-mon[128316]: pgmap v922: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T19:36:41.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 610.0 s, 259.0 tps, lat 400.750 ms stddev 97.080 2023-07-21T19:36:42.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:41 smithi101 ceph-mon[128136]: pgmap v922: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 680 op/s 2023-07-21T19:36:43.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:43 smithi005 ceph-mon[126460]: pgmap v923: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.0 MiB/s wr, 799 op/s 2023-07-21T19:36:43.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:43 smithi156 ceph-mon[128316]: pgmap v923: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.0 MiB/s wr, 799 op/s 2023-07-21T19:36:44.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:43 smithi101 ceph-mon[128136]: pgmap v923: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.0 MiB/s wr, 799 op/s 2023-07-21T19:36:45.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:45 smithi156 ceph-mon[128316]: pgmap v924: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 944 op/s 2023-07-21T19:36:46.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:45 smithi101 ceph-mon[128136]: pgmap v924: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 944 op/s 2023-07-21T19:36:46.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:45 smithi005 ceph-mon[126460]: pgmap v924: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 944 op/s 2023-07-21T19:36:46.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 615.0 s, 261.6 tps, lat 380.885 ms stddev 66.817 2023-07-21T19:36:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:47 smithi156 ceph-mon[128316]: pgmap v925: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-07-21T19:36:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:47 smithi101 ceph-mon[128136]: pgmap v925: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-07-21T19:36:48.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:47 smithi005 ceph-mon[126460]: pgmap v925: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.8 MiB/s wr, 700 op/s 2023-07-21T19:36:49.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:48 smithi005 ceph-mon[126460]: pgmap v926: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:36:49.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:48 smithi156 ceph-mon[128316]: pgmap v926: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:36:49.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:48 smithi101 ceph-mon[128136]: pgmap v926: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.09k op/s 2023-07-21T19:36:51.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:51 smithi005 ceph-mon[126460]: pgmap v927: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 740 op/s 2023-07-21T19:36:51.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:51 smithi156 ceph-mon[128316]: pgmap v927: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 740 op/s 2023-07-21T19:36:51.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 620.0 s, 246.6 tps, lat 406.246 ms stddev 67.685 2023-07-21T19:36:52.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:51 smithi101 ceph-mon[128136]: pgmap v927: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 7.3 MiB/s wr, 740 op/s 2023-07-21T19:36:53.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:53 smithi005 ceph-mon[126460]: pgmap v928: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 859 op/s 2023-07-21T19:36:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:53 smithi156 ceph-mon[128316]: pgmap v928: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 859 op/s 2023-07-21T19:36:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:53 smithi101 ceph-mon[128136]: pgmap v928: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 8.8 MiB/s wr, 859 op/s 2023-07-21T19:36:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 -- 172.21.15.5:0/1146096350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8067c10 msgr2=0x7f49d8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1146096350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8067c10 0x7f49d8067fe0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f49d4009990 tx=0x7f49d4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 -- 172.21.15.5:0/1146096350 shutdown_connections 2023-07-21T19:36:54.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1146096350 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8104e50 0x7f49d810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1146096350 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8067c10 0x7f49d8067fe0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1146096350 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 -- 172.21.15.5:0/1146096350 >> 172.21.15.5:0/1146096350 conn(0x7f49d8074760 msgr2=0x7f49d8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 -- 172.21.15.5:0/1146096350 shutdown_connections 2023-07-21T19:36:54.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.262+0000 7f49e05ff700 1 -- 172.21.15.5:0/1146096350 wait complete. 2023-07-21T19:36:54.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 Processor -- start 2023-07-21T19:36:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 -- start start 2023-07-21T19:36:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8067c10 0x7f49d819cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8104e50 0x7f49d81984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8198c10 con 0x7f49d80685b0 2023-07-21T19:36:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8198d80 con 0x7f49d8067c10 2023-07-21T19:36:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.264+0000 7f49e05ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8198f40 con 0x7f49d8104e50 2023-07-21T19:36:54.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49deb9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8104e50 0x7f49d81984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58644/0 (socket says 172.21.15.5:58644) 2023-07-21T19:36:54.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 -- 172.21.15.5:0/1162831481 learned_addr learned my addr 172.21.15.5:0/1162831481 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:54.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49de39b700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8067c10 0x7f49d819cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 -- 172.21.15.5:0/1162831481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8104e50 msgr2=0x7f49d81984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8104e50 0x7f49d81984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 -- 172.21.15.5:0/1162831481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8067c10 msgr2=0x7f49d819cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8067c10 0x7f49d819cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.265+0000 7f49ddb9a700 1 -- 172.21.15.5:0/1162831481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d40095f0 con 0x7f49d80685b0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e940 tx=0x7f49c800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49cf7fe700 1 -- 172.21.15.5:0/1162831481 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8009730 con 0x7f49d80685b0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d81991c0 con 0x7f49d80685b0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d81a4d20 con 0x7f49d80685b0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49cf7fe700 1 -- 172.21.15.5:0/1162831481 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49c800ce70 con 0x7f49d80685b0 2023-07-21T19:36:54.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.266+0000 7f49cf7fe700 1 -- 172.21.15.5:0/1162831481 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8018420 con 0x7f49d80685b0 2023-07-21T19:36:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.272+0000 7f49cf7fe700 1 -- 172.21.15.5:0/1162831481 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f49c800c9e0 con 0x7f49d80685b0 2023-07-21T19:36:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.273+0000 7f49f3f5fb80 1 Processor -- start 2023-07-21T19:36:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.273+0000 7f49f3f5fb80 1 -- start start 2023-07-21T19:36:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.273+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.273+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee982d0 0x55a54ee92040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.273+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54ee929d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ed97c50 con 0x55a54ecee6b0 2023-07-21T19:36:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54edea9e0 con 0x55a54ee92580 2023-07-21T19:36:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ede3e70 con 0x55a54ee982d0 2023-07-21T19:36:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58656/0 (socket says 172.21.15.5:58656) 2023-07-21T19:36:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 -- 172.21.15.5:0/358329975 learned_addr learned my addr 172.21.15.5:0/358329975 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49de39b700 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee982d0 0x55a54ee92040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54ee929d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 -- 172.21.15.5:0/358329975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee982d0 msgr2=0x55a54ee92040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee982d0 0x55a54ee92040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 -- 172.21.15.5:0/358329975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 msgr2=0x55a54ee929d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54ee929d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.274+0000 7f49deb9c700 1 -- 172.21.15.5:0/358329975 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a54ec11d80 con 0x55a54ecee6b0 2023-07-21T19:36:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.275+0000 7f49cf7fe700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 0x7f49c40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.275+0000 7f49cf7fe700 1 -- 172.21.15.5:0/1162831481 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f49c8014070 con 0x7f49d80685b0 2023-07-21T19:36:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.276+0000 7f49de39b700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 0x7f49c40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.276+0000 7f49deb9c700 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011680 tx=0x7f49d0009920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a5b34938a456c4b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.276+0000 7f49ccbfe700 1 -- 172.21.15.5:0/358329975 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d000e040 con 0x55a54ecee6b0 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.276+0000 7f49ccbfe700 1 -- 172.21.15.5:0/358329975 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49d000f040 con 0x55a54ecee6b0 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.276+0000 7f49ccbfe700 1 -- 172.21.15.5:0/358329975 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d0004790 con 0x55a54ecee6b0 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.277+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/358329975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 msgr2=0x55a54ee97d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.277+0000 7f49de39b700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 0x7f49c40a3870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b2e0 tx=0x7f49d4009510 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.277+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f49d0011680 tx=0x7f49d0009920 comp rx=0 tx=0).stop 2023-07-21T19:36:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/358329975 shutdown_connections 2023-07-21T19:36:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54ee929d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ecee6b0 0x55a54ee97d90 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/358329975 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee982d0 0x55a54ee92040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/358329975 >> 172.21.15.5:0/358329975 conn(0x55a54ee87be0 msgr2=0x55a54ee8cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/358329975 shutdown_connections 2023-07-21T19:36:54.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/358329975 wait complete. 2023-07-21T19:36:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.278+0000 7f49f3f5fb80 1 -- start start 2023-07-21T19:36:54.272 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.279+0000 7f49f3f5fb80 1 Processor -- start 2023-07-21T19:36:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee9b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9bf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ed83a60 con 0x55a54ee982d0 2023-07-21T19:36:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ed83760 con 0x55a54ecee6b0 2023-07-21T19:36:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ec678c0 con 0x55a54ee92580 2023-07-21T19:36:54.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49f3f5fb80 10 client.0 _ll_register_callbacks cb 0x55a54ed81700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:54.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49de39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49de39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:53346/0 (socket says 172.21.15.5:53346) 2023-07-21T19:36:54.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49de39b700 1 -- 172.21.15.5:0/4177279267 learned_addr learned my addr 172.21.15.5:0/4177279267 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee9b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.280+0000 7f49deb9c700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 -- 172.21.15.5:0/4177279267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 msgr2=0x55a54ee9b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee9b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 -- 172.21.15.5:0/4177279267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 msgr2=0x55a54ee9bf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9bf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 -- 172.21.15.5:0/4177279267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d0011330 con 0x55a54ecee6b0 2023-07-21T19:36:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49de39b700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f49d4015000 tx=0x7f49d4009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=230507ebb28273cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.281+0000 7f49c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:36:54.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400bee0 con 0x55a54ecee6b0 2023-07-21T19:36:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f49d400f460 con 0x55a54ecee6b0 2023-07-21T19:36:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a54ed3d490 con 0x55a54ecee6b0 2023-07-21T19:36:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d4004ea0 con 0x55a54ecee6b0 2023-07-21T19:36:54.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49f3f5fb80 10 client.30242 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55a54ebfac00 con 0x55a54ecee6b0 2023-07-21T19:36:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.282+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a54ed3d490 con 0x55a54ecee6b0 2023-07-21T19:36:54.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49d402f4b0 con 0x55a54ecee6b0 2023-07-21T19:36:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 10 client.30242 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 10 client.30242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 20 client.30242 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:54.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54eeb43e0 0x55a54eeb67f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 4 client.30242 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:36:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a54eddce50 con 0x55a54eeb43e0 2023-07-21T19:36:54.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.283+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54eeb43e0 0x55a54eeb67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.284+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54eeb43e0 0x55a54eeb67f0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66d7a18291fc28f5 server_cookie=1dde9d290cc60aca in_seq=0 out_seq=0 2023-07-21T19:36:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.284+0000 7f49c1ffb700 10 client.30242 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49c1ffb700 1 -- 172.21.15.5:0/4177279267 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a54eddce50 con 0x55a54eeb43e0 2023-07-21T19:36:54.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49c1ffb700 10 client.30242 handle_command_reply: tid=0 2023-07-21T19:36:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49f3f5fb80 1 client.30242 shutdown 2023-07-21T19:36:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 msgr2=0x55a54ee9e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f49d4015000 tx=0x7f49d4009310 comp rx=0 tx=0).stop 2023-07-21T19:36:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.285+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 shutdown_connections 2023-07-21T19:36:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54eeb43e0 0x55a54eeb67f0 crc :-1 s=READY pgs=533 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9bf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee9e620 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/4177279267 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee9b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49c1ffb700 0 client.30242 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:54.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 >> 172.21.15.5:0/4177279267 conn(0x55a54ee87be0 msgr2=0x55a54ee8bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 shutdown_connections 2023-07-21T19:36:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/4177279267 wait complete. 2023-07-21T19:36:54.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.286+0000 7f49f3f5fb80 20 client.30242 trim_cache size 0 max 16384 2023-07-21T19:36:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 Processor -- start 2023-07-21T19:36:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 -- start start 2023-07-21T19:36:54.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee8cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ee99eb0 con 0x55a54ee982d0 2023-07-21T19:36:54.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ee96650 con 0x55a54ecee6b0 2023-07-21T19:36:54.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.312+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54eeb0dd0 con 0x55a54ee92580 2023-07-21T19:36:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49ddb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee8cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49ddb9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee8cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57486/0 (socket says 172.21.15.5:57486) 2023-07-21T19:36:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49ddb9a700 1 -- 172.21.15.5:0/2375304605 learned_addr learned my addr 172.21.15.5:0/2375304605 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49deb9c700 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 -- 172.21.15.5:0/2375304605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 msgr2=0x55a54ee8cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee8cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 -- 172.21.15.5:0/2375304605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 msgr2=0x55a54ee9e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 -- 172.21.15.5:0/2375304605 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a54ec381f0 con 0x55a54ecee6b0 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.313+0000 7f49de39b700 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee97170 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f49d402f7e0 tx=0x7f49d4009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39e38f95ae42020a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.314+0000 7f49cd3ff700 1 -- 172.21.15.5:0/2375304605 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400bee0 con 0x55a54ecee6b0 2023-07-21T19:36:54.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.314+0000 7f49cd3ff700 1 -- 172.21.15.5:0/2375304605 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49d401e040 con 0x55a54ecee6b0 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.314+0000 7f49cd3ff700 1 -- 172.21.15.5:0/2375304605 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d400f6c0 con 0x55a54ecee6b0 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/2375304605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 msgr2=0x55a54ee97170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee97170 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f49d402f7e0 tx=0x7f49d4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/2375304605 shutdown_connections 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ee9e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ecee6b0 0x55a54ee97170 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.306 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/2375304605 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ee92580 0x55a54ee8cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/2375304605 >> 172.21.15.5:0/2375304605 conn(0x55a54ee87be0 msgr2=0x55a54ee96a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/2375304605 shutdown_connections 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/2375304605 wait complete. 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.315+0000 7f49f3f5fb80 1 -- start start 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.316+0000 7f49f3f5fb80 1 Processor -- start 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.316+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.307 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54eea9c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ef07260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54eeaa3e0 con 0x55a54ee982d0 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ef077d0 con 0x55a54ee92580 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a54ef07990 con 0x55a54ecee6b0 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49f3f5fb80 10 client.0 _ll_register_callbacks cb 0x7f49d400bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.308 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57498/0 (socket says 172.21.15.5:57498) 2023-07-21T19:36:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 -- 172.21.15.5:0/404970196 learned_addr learned my addr 172.21.15.5:0/404970196 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:36:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54eea9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.309 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49deb9c700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ef07260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.310 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 -- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 msgr2=0x55a54eea9c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54eea9c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.311 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 -- 172.21.15.5:0/404970196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 msgr2=0x55a54ef07260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ef07260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.317+0000 7f49de39b700 1 -- 172.21.15.5:0/404970196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a54ebfac00 con 0x55a54ecee6b0 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.318+0000 7f49de39b700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006010 tx=0x7f49d400bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3045a4fc978f9b4d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.318+0000 7f49c1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.318+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d404e070 con 0x55a54ecee6b0 2023-07-21T19:36:54.312 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.318+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f49d4004b70 con 0x55a54ecee6b0 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.318+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f49d000f040 con 0x55a54ecee6b0 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.319+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49d4004d70 con 0x55a54ecee6b0 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.319+0000 7f49f3f5fb80 10 client.30178 fetch_fsmap learned FSMap version 72 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.319+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55a54ec10250 con 0x55a54ecee6b0 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.319+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f49d4056020 con 0x55a54ecee6b0 2023-07-21T19:36:54.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f49d400f460 con 0x55a54ecee6b0 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 10 client.30178 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 10 client.30178 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 20 client.30178 populate_metadata read hostname 'smithi005' 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54ef08c40 0x55a54ef0b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 4 client.30178 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a54ed9eff0 con 0x55a54ef08c40 2023-07-21T19:36:54.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.320+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54ef08c40 0x55a54ef0b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:36:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.321+0000 7f49ddb9a700 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54ef08c40 0x55a54ef0b020 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4631f4a85aa3ff29 server_cookie=52e343114104b32e in_seq=0 out_seq=0 2023-07-21T19:36:54.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.321+0000 7f49c1ffb700 10 client.30178 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:54.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.398+0000 7f49c1ffb700 1 -- 172.21.15.5:0/404970196 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a54ed9eff0 con 0x55a54ef08c40 2023-07-21T19:36:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.398+0000 7f49c1ffb700 10 client.30178 handle_command_reply: tid=0 2023-07-21T19:36:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.398+0000 7f49f3f5fb80 1 client.30178 shutdown 2023-07-21T19:36:54.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.398+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 msgr2=0x55a54eea9710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.398+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f49d4006010 tx=0x7f49d400bee0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 shutdown_connections 2023-07-21T19:36:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a54ef08c40 0x55a54ef0b020 crc :-1 s=READY pgs=534 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a54ee982d0 0x55a54ef07260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a54ecee6b0 0x55a54eea9710 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 --2- 172.21.15.5:0/404970196 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a54ee92580 0x55a54eea9c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 >> 172.21.15.5:0/404970196 conn(0x55a54ee87be0 msgr2=0x55a54ee972f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49c1ffb700 0 client.30178 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:36:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 shutdown_connections 2023-07-21T19:36:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.399+0000 7f49f3f5fb80 1 -- 172.21.15.5:0/404970196 wait complete. 2023-07-21T19:36:54.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.400+0000 7f49f3f5fb80 20 client.30178 trim_cache size 0 max 16384 2023-07-21T19:36:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:36:54.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:36:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:36:54.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:36:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 msgr2=0x7f49c40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 0x7f49c40a3870 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f49d400b2e0 tx=0x7f49d4009510 comp rx=0 tx=0).stop 2023-07-21T19:36:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 msgr2=0x7f49d819d160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:36:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f49c800e940 tx=0x7f49c800ec50 comp rx=0 tx=0).stop 2023-07-21T19:36:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 shutdown_connections 2023-07-21T19:36:54.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f49d8104e50 0x7f49d81984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f49d8067c10 0x7f49d819cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f49c40a13c0 0x7f49c40a3870 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 --2- 172.21.15.5:0/1162831481 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f49d80685b0 0x7f49d819d160 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:36:54.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.401+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 >> 172.21.15.5:0/1162831481 conn(0x7f49d8074760 msgr2=0x7f49d81036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:36:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.402+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 shutdown_connections 2023-07-21T19:36:54.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:36:54.402+0000 7f49e05ff700 1 -- 172.21.15.5:0/1162831481 wait complete. 2023-07-21T19:36:55.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:55 smithi156 ceph-mon[128316]: pgmap v929: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 1.02k op/s 2023-07-21T19:36:56.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:55 smithi101 ceph-mon[128136]: pgmap v929: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 1.02k op/s 2023-07-21T19:36:56.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:55 smithi005 ceph-mon[126460]: pgmap v929: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 9.8 MiB/s wr, 1.02k op/s 2023-07-21T19:36:56.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 625.0 s, 226.0 tps, lat 437.717 ms stddev 83.124 2023-07-21T19:36:57.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:57 smithi156 ceph-mon[128316]: pgmap v930: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 781 op/s 2023-07-21T19:36:58.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:57 smithi101 ceph-mon[128136]: pgmap v930: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 781 op/s 2023-07-21T19:36:58.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:57 smithi005 ceph-mon[126460]: pgmap v930: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.9 MiB/s wr, 781 op/s 2023-07-21T19:36:59.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:36:58 smithi005 ceph-mon[126460]: pgmap v931: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.15k op/s 2023-07-21T19:36:59.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:36:58 smithi156 ceph-mon[128316]: pgmap v931: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.15k op/s 2023-07-21T19:36:59.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:36:58 smithi101 ceph-mon[128136]: pgmap v931: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 11 MiB/s wr, 1.15k op/s 2023-07-21T19:36:59.966 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:37:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.010+0000 7ff69c5ff700 1 -- 172.21.15.5:0/281328713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 msgr2=0x7ff69410be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.010+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/281328713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff69410be90 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7ff6840098f0 tx=0x7ff684009c00 comp rx=0 tx=0).stop 2023-07-21T19:37:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 -- 172.21.15.5:0/281328713 shutdown_connections 2023-07-21T19:37:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/281328713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff69410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:00.997 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/281328713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 0x7ff6940febf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:00.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/281328713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff69410be90 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:00.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 -- 172.21.15.5:0/281328713 >> 172.21.15.5:0/281328713 conn(0x7ff6940fa4b0 msgr2=0x7ff6940fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:00.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 -- 172.21.15.5:0/281328713 shutdown_connections 2023-07-21T19:37:00.998 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.011+0000 7ff69c5ff700 1 -- 172.21.15.5:0/281328713 wait complete. 2023-07-21T19:37:00.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.012+0000 7ff69c5ff700 1 Processor -- start 2023-07-21T19:37:00.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 -- start start 2023-07-21T19:37:00.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 0x7ff694105060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:00.999 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff6941055a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff694106540 con 0x7ff6940fe820 2023-07-21T19:37:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6941066b0 con 0x7ff69410c3d0 2023-07-21T19:37:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.013+0000 7ff69c5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff694106870 con 0x7ff6940ff1c0 2023-07-21T19:37:01.000 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff69ab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.001 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff69ab9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56184/0 (socket says 172.21.15.5:56184) 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff69ab9c700 1 -- 172.21.15.5:0/2257117890 learned_addr learned my addr 172.21.15.5:0/2257117890 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff6941055a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff69a39b700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 0x7ff694105060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff699b9a700 1 -- 172.21.15.5:0/2257117890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 msgr2=0x7ff694105de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.014+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff699b9a700 1 -- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 msgr2=0x7ff694105060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.002 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 0x7ff694105060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff699b9a700 1 -- 172.21.15.5:0/2257117890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff684009520 con 0x7ff6940ff1c0 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff6941055a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff684009810 tx=0x7ff68400fa00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff68b7fe700 1 -- 172.21.15.5:0/2257117890 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68401c070 con 0x7ff6940ff1c0 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff68b7fe700 1 -- 172.21.15.5:0/2257117890 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff68400bb70 con 0x7ff6940ff1c0 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff68b7fe700 1 -- 172.21.15.5:0/2257117890 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840038c0 con 0x7ff6940ff1c0 2023-07-21T19:37:01.003 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.015+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff694106af0 con 0x7ff6940ff1c0 2023-07-21T19:37:01.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.016+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69410b150 con 0x7ff6940ff1c0 2023-07-21T19:37:01.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.020+0000 7ff68b7fe700 1 -- 172.21.15.5:0/2257117890 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff684020030 con 0x7ff6940ff1c0 2023-07-21T19:37:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.022+0000 7ff6afe8bb80 1 Processor -- start 2023-07-21T19:37:01.009 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.022+0000 7ff6afe8bb80 1 -- start start 2023-07-21T19:37:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff68b7fe700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 0x7ff6800a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff68b7fe700 1 -- 172.21.15.5:0/2257117890 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff6840cdb70 con 0x7ff6940ff1c0 2023-07-21T19:37:01.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c377072d0 0x559c377076a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37714eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3760dcb0 con 0x559c37707be0 2023-07-21T19:37:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 0x7ff6800a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c37660a40 con 0x559c377072d0 2023-07-21T19:37:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c37659ed0 con 0x559c37712ad0 2023-07-21T19:37:01.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.023+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53680/0 (socket says 172.21.15.5:53680) 2023-07-21T19:37:01.014 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 -- 172.21.15.5:0/1601680061 learned_addr learned my addr 172.21.15.5:0/1601680061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:01.016 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 -- 172.21.15.5:0/1601680061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 msgr2=0x559c37714eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37714eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 -- 172.21.15.5:0/1601680061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c377072d0 msgr2=0x559c377076a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c377072d0 0x559c377076a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 -- 172.21.15.5:0/1601680061 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c37487de0 con 0x559c37707be0 2023-07-21T19:37:01.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff699b9a700 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff6840976e0 tx=0x7ff68409a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d90dfdb766e3acad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.024+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 0x7ff6800a37f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0060b0 tx=0x7ff68c0099e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.025+0000 7ff688bfe700 1 -- 172.21.15.5:0/1601680061 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68409ac70 con 0x559c37707be0 2023-07-21T19:37:01.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.025+0000 7ff688bfe700 1 -- 172.21.15.5:0/1601680061 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff68409eca0 con 0x559c37707be0 2023-07-21T19:37:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.025+0000 7ff688bfe700 1 -- 172.21.15.5:0/1601680061 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840cb4e0 con 0x559c37707be0 2023-07-21T19:37:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/1601680061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 msgr2=0x559c37712590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7ff6840976e0 tx=0x7ff68409a5d0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/1601680061 shutdown_connections 2023-07-21T19:37:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37714eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c377072d0 0x559c377076a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/1601680061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c37707be0 0x559c37712590 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/1601680061 >> 172.21.15.5:0/1601680061 conn(0x559c376fdda0 msgr2=0x559c377036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:01.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/1601680061 shutdown_connections 2023-07-21T19:37:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/1601680061 wait complete. 2023-07-21T19:37:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.026+0000 7ff6afe8bb80 1 -- start start 2023-07-21T19:37:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 Processor -- start 2023-07-21T19:37:01.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37712ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c377133e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37713920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c375f9ac0 con 0x559c377072d0 2023-07-21T19:37:01.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c375f97c0 con 0x559c37707be0 2023-07-21T19:37:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c374dd920 con 0x559c37712ad0 2023-07-21T19:37:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.027+0000 7ff6afe8bb80 10 client.0 _ll_register_callbacks cb 0x559c375f7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37713920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37713920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47218/0 (socket says 172.21.15.5:47218) 2023-07-21T19:37:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69ab9c700 1 -- 172.21.15.5:0/549816421 learned_addr learned my addr 172.21.15.5:0/549816421 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37712ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff699b9a700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c377133e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 -- 172.21.15.5:0/549816421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 msgr2=0x559c37713920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37713920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 -- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 msgr2=0x559c377133e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c377133e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff69a39b700 1 -- 172.21.15.5:0/549816421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff69410b150 con 0x559c377072d0 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.028+0000 7ff699b9a700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c377133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:01.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.029+0000 7ff69a39b700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37712ea0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff690011410 tx=0x7ff690011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953cfa3057a98825 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.029+0000 7ff67dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.029+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff690004250 con 0x559c377072d0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.029+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff6900043b0 con 0x559c377072d0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.029+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c375b34f0 con 0x559c377072d0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff690004570 con 0x559c377072d0 2023-07-21T19:37:01.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 10 client.20472 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x559c37470c60 con 0x559c377072d0 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x559c375b34f0 con 0x559c377072d0 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff69001b7b0 con 0x559c377072d0 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 10 client.20472 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 10 client.20472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 20 client.20472 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37722120 0x559c37724530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 4 client.20472 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.030+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c37652eb0 con 0x559c37722120 2023-07-21T19:37:01.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.031+0000 7ff699b9a700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37722120 0x559c37724530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.032+0000 7ff699b9a700 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37722120 0x559c37724530 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cf26f68ad18b7c3 server_cookie=b546d8a9275be2fc in_seq=0 out_seq=0 2023-07-21T19:37:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.032+0000 7ff67dffb700 10 client.20472 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.032+0000 7ff67dffb700 1 -- 172.21.15.5:0/549816421 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559c37652eb0 con 0x559c37722120 2023-07-21T19:37:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff67dffb700 10 client.20472 handle_command_reply: tid=0 2023-07-21T19:37:01.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 client.20472 shutdown 2023-07-21T19:37:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 msgr2=0x559c37712ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37712ea0 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7ff690011410 tx=0x7ff690011720 comp rx=0 tx=0).stop 2023-07-21T19:37:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 shutdown_connections 2023-07-21T19:37:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c37713920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37722120 0x559c37724530 crc :-1 s=READY pgs=535 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37712ea0 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/549816421 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c377133e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 >> 172.21.15.5:0/549816421 conn(0x559c376fdda0 msgr2=0x559c37703290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:01.033 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff67dffb700 0 client.20472 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.033+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 shutdown_connections 2023-07-21T19:37:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.034+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/549816421 wait complete. 2023-07-21T19:37:01.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.034+0000 7ff6afe8bb80 20 client.20472 trim_cache size 0 max 16384 2023-07-21T19:37:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 Processor -- start 2023-07-21T19:37:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 -- start start 2023-07-21T19:37:01.049 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37717ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3771d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3770c9c0 con 0x559c377072d0 2023-07-21T19:37:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c376fd970 con 0x559c37707be0 2023-07-21T19:37:01.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.063+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c37712180 con 0x559c37712ad0 2023-07-21T19:37:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:56212/0 (socket says 172.21.15.5:56212) 2023-07-21T19:37:01.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff699b9a700 1 -- 172.21.15.5:0/2967332384 learned_addr learned my addr 172.21.15.5:0/2967332384 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff69a39b700 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37717ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3771d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff699b9a700 1 -- 172.21.15.5:0/2967332384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 msgr2=0x559c3771d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.064+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3771d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.065+0000 7ff699b9a700 1 -- 172.21.15.5:0/2967332384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 msgr2=0x559c37717ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.065+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37717ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.052 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.065+0000 7ff699b9a700 1 -- 172.21.15.5:0/2967332384 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c374ae250 con 0x559c37707be0 2023-07-21T19:37:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.065+0000 7ff699b9a700 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff68409a920 tx=0x7ff6840a0040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ecd0afb6be40f34d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.065+0000 7ff6893ff700 1 -- 172.21.15.5:0/2967332384 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840050c0 con 0x559c37707be0 2023-07-21T19:37:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.066+0000 7ff6893ff700 1 -- 172.21.15.5:0/2967332384 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff684097380 con 0x559c37707be0 2023-07-21T19:37:01.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.066+0000 7ff6893ff700 1 -- 172.21.15.5:0/2967332384 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840b3380 con 0x559c37707be0 2023-07-21T19:37:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2967332384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 msgr2=0x559c3771e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7ff68409a920 tx=0x7ff6840a0040 comp rx=0 tx=0).stop 2023-07-21T19:37:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2967332384 shutdown_connections 2023-07-21T19:37:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3771d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37717ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2967332384 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c3771e3f0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.067+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2967332384 >> 172.21.15.5:0/2967332384 conn(0x559c376fdda0 msgr2=0x559c37701e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.068+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2967332384 shutdown_connections 2023-07-21T19:37:01.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.068+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2967332384 wait complete. 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.069+0000 7ff6afe8bb80 1 -- start start 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.069+0000 7ff6afe8bb80 1 Processor -- start 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37713d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c37714260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3776efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3776f500 con 0x559c377072d0 2023-07-21T19:37:01.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3776f640 con 0x559c37707be0 2023-07-21T19:37:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c3776f800 con 0x559c37712ad0 2023-07-21T19:37:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff6afe8bb80 10 client.0 _ll_register_callbacks cb 0x7ff690009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff69a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37713d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff699b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c37714260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff69ab9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3776efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff69a39b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37713d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:53700/0 (socket says 172.21.15.5:53700) 2023-07-21T19:37:01.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.070+0000 7ff69a39b700 1 -- 172.21.15.5:0/2109668403 learned_addr learned my addr 172.21.15.5:0/2109668403 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 -- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 msgr2=0x559c37714260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c37714260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 -- 172.21.15.5:0/2109668403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 msgr2=0x559c37713d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37713d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 -- 172.21.15.5:0/2109668403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c37487de0 con 0x559c37712ad0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.071+0000 7ff69ab9c700 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3776efc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009110 tx=0x7ff68c016000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=23feb6f23f3f602c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff67dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68c017ae0 con 0x559c37712ad0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff68409eca0 con 0x559c37712ad0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(135..135 src has 1..135) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff68c017c40 con 0x559c37712ad0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff68c017e00 con 0x559c37712ad0 2023-07-21T19:37:01.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.072+0000 7ff6afe8bb80 10 client.30193 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x559c374862b0 con 0x559c37712ad0 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff68409eca0 con 0x559c37712ad0 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff68c0255d0 con 0x559c37712ad0 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 10 client.30193 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 10 client.30193 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 20 client.30193 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37777080 0x559c37779490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:01.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 4 client.30193 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:37:01.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.073+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c37615050 con 0x559c37777080 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.074+0000 7ff69a39b700 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37777080 0x559c37779490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.075+0000 7ff69a39b700 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37777080 0x559c37779490 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9da188903995d881 server_cookie=8ebef8b4fc9f8e in_seq=0 out_seq=0 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.075+0000 7ff67dffb700 10 client.30193 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.075+0000 7ff67dffb700 1 -- 172.21.15.5:0/2109668403 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x559c37615050 con 0x559c37777080 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.075+0000 7ff67dffb700 10 client.30193 handle_command_reply: tid=0 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.075+0000 7ff6afe8bb80 1 client.30193 shutdown 2023-07-21T19:37:01.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 msgr2=0x559c3776efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3776efc0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7ff68c009110 tx=0x7ff68c016000 comp rx=0 tx=0).stop 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 shutdown_connections 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x559c37712ad0 0x559c3776efc0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x559c37777080 0x559c37779490 crc :-1 s=READY pgs=536 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x559c377072d0 0x559c37713d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 --2- 172.21.15.5:0/2109668403 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x559c37707be0 0x559c37714260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 >> 172.21.15.5:0/2109668403 conn(0x559c376fdda0 msgr2=0x559c377093e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff67dffb700 0 client.30193 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.076+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 shutdown_connections 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.077+0000 7ff6afe8bb80 1 -- 172.21.15.5:0/2109668403 wait complete. 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.077+0000 7ff6afe8bb80 20 client.30193 trim_cache size 0 max 16384 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:37:01.065 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:37:01.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:37:01.067 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:37:01.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 msgr2=0x7ff6800a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 0x7ff6800a37f0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff68c0060b0 tx=0x7ff68c0099e0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 msgr2=0x7ff6941055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff6941055a0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7ff684009810 tx=0x7ff68400fa00 comp rx=0 tx=0).stop 2023-07-21T19:37:01.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 shutdown_connections 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff69410c3d0 0x7ff694105de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff6800a1340 0x7ff6800a37f0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff6940fe820 0x7ff694105060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 --2- 172.21.15.5:0/2257117890 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff6940ff1c0 0x7ff6941055a0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 >> 172.21.15.5:0/2257117890 conn(0x7ff6940fa4b0 msgr2=0x7ff69410d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 shutdown_connections 2023-07-21T19:37:01.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:01.078+0000 7ff69c5ff700 1 -- 172.21.15.5:0/2257117890 wait complete. 2023-07-21T19:37:01.106 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:37:01.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:01 smithi005 ceph-mon[126460]: pgmap v932: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 762 op/s 2023-07-21T19:37:01.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:01 smithi156 ceph-mon[128316]: pgmap v932: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 762 op/s 2023-07-21T19:37:01.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 630.0 s, 228.6 tps, lat 443.145 ms stddev 141.438 2023-07-21T19:37:02.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:01 smithi101 ceph-mon[128136]: pgmap v932: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.5 MiB/s wr, 762 op/s 2023-07-21T19:37:03.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:03 smithi156 ceph-mon[128316]: pgmap v933: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 887 op/s 2023-07-21T19:37:04.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:03 smithi101 ceph-mon[128136]: pgmap v933: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 887 op/s 2023-07-21T19:37:04.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:03 smithi005 ceph-mon[126460]: pgmap v933: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 9.2 MiB/s wr, 887 op/s 2023-07-21T19:37:05.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:05 smithi156 ceph-mon[128316]: pgmap v934: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-07-21T19:37:06.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:05 smithi101 ceph-mon[128136]: pgmap v934: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-07-21T19:37:06.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:05 smithi005 ceph-mon[126460]: pgmap v934: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.4 MiB/s wr, 992 op/s 2023-07-21T19:37:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 635.0 s, 231.7 tps, lat 397.879 ms stddev 103.516 2023-07-21T19:37:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.699+0000 7fb3b9dff700 1 -- 172.21.15.5:0/934101840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 msgr2=0x7fb3b411b470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.699+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/934101840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b411b470 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb3a8009990 tx=0x7fb3a8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.687 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.700+0000 7fb3b9dff700 1 -- 172.21.15.5:0/934101840 shutdown_connections 2023-07-21T19:37:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.700+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/934101840 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b411b470 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.700+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/934101840 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3b410e2a0 0x7fb3b410e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.700+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/934101840 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b406c910 0x7fb3b406cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.700+0000 7fb3b9dff700 1 -- 172.21.15.5:0/934101840 >> 172.21.15.5:0/934101840 conn(0x7fb3b406bd70 msgr2=0x7fb3b4070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.703+0000 7fb3b9dff700 1 -- 172.21.15.5:0/934101840 shutdown_connections 2023-07-21T19:37:07.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.703+0000 7fb3b9dff700 1 -- 172.21.15.5:0/934101840 wait complete. 2023-07-21T19:37:07.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.705+0000 7fb3b9dff700 1 Processor -- start 2023-07-21T19:37:07.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.705+0000 7fb3b9dff700 1 -- start start 2023-07-21T19:37:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3b406c910 0x7fb3b41aab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b41ab040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 0x7fb3b41a4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41a4fd0 con 0x7fb3b410e2a0 2023-07-21T19:37:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41a5140 con 0x7fb3b406c910 2023-07-21T19:37:07.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.706+0000 7fb3b9dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b41a5300 con 0x7fb3b406d3a0 2023-07-21T19:37:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 0x7fb3b41a4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b41ab040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b41ab040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59050/0 (socket says 172.21.15.5:59050) 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/3635452805 learned_addr learned my addr 172.21.15.5:0/3635452805 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b3fff700 1 -- 172.21.15.5:0/3635452805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 msgr2=0x7fb3b41ab040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.707+0000 7fb3b3fff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b41ab040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.708+0000 7fb3b3fff700 1 -- 172.21.15.5:0/3635452805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3b406c910 msgr2=0x7fb3b41aab00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.708+0000 7fb3b3fff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3b406c910 0x7fb3b41aab00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.708+0000 7fb3b3fff700 1 -- 172.21.15.5:0/3635452805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a80095f0 con 0x7fb3b410e2a0 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.708+0000 7fb3b3fff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 0x7fb3b41a4960 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80098e0 tx=0x7fb3a8005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.695 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.709+0000 7fb3b0ff9700 1 -- 172.21.15.5:0/3635452805 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a801d070 con 0x7fb3b410e2a0 2023-07-21T19:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.709+0000 7fb3b0ff9700 1 -- 172.21.15.5:0/3635452805 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3a8003ec0 con 0x7fb3b410e2a0 2023-07-21T19:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.709+0000 7fb3b0ff9700 1 -- 172.21.15.5:0/3635452805 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a800fa80 con 0x7fb3b410e2a0 2023-07-21T19:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.709+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b41a5580 con 0x7fb3b410e2a0 2023-07-21T19:37:07.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.709+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b41b1af0 con 0x7fb3b410e2a0 2023-07-21T19:37:07.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.716+0000 7fb3cd7bcb80 1 Processor -- start 2023-07-21T19:37:07.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.717+0000 7fb3cd7bcb80 1 -- start start 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3b0ff9700 1 -- 172.21.15.5:0/3635452805 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb3a8004030 con 0x7fb3b410e2a0 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3f4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3f4580 0x55d74b3f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b2f9c50 con 0x55d74b2506b0 2023-07-21T19:37:07.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b34c9e0 con 0x55d74b3f4580 2023-07-21T19:37:07.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.718+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b345e70 con 0x55d74b3fa2d0 2023-07-21T19:37:07.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.720+0000 7fb3b0ff9700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 0x7fb39c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.720+0000 7fb3b0ff9700 1 -- 172.21.15.5:0/3635452805 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb3a80cdcf0 con 0x7fb3b410e2a0 2023-07-21T19:37:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.723+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.723+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51282/0 (socket says 172.21.15.5:51282) 2023-07-21T19:37:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.723+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/1345728207 learned_addr learned my addr 172.21.15.5:0/1345728207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.723+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3f4580 0x55d74b3f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/1345728207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 msgr2=0x55d74b3f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/1345728207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3f4580 msgr2=0x55d74b3f49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3f4580 0x55d74b3f49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/1345728207 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d74b173d80 con 0x55d74b2506b0 2023-07-21T19:37:07.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.724+0000 7fb3b3fff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 0x7fb39c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.726+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40154a0 tx=0x7fb3a4009660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5c5ffc21d1f9fd8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.726+0000 7fb399ffb700 1 -- 172.21.15.5:0/1345728207 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a4017470 con 0x55d74b2506b0 2023-07-21T19:37:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.727+0000 7fb399ffb700 1 -- 172.21.15.5:0/1345728207 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3a4017a60 con 0x55d74b2506b0 2023-07-21T19:37:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.727+0000 7fb399ffb700 1 -- 172.21.15.5:0/1345728207 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a40138f0 con 0x55d74b2506b0 2023-07-21T19:37:07.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.727+0000 7fb3b3fff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 0x7fb39c0a38b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80a2bc0 tx=0x7fb3a8005a00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.728+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/1345728207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 msgr2=0x55d74b3f9d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:07.715 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.728+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7fb3a40154a0 tx=0x7fb3a4009660 comp rx=0 tx=0).stop 2023-07-21T19:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.732+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/1345728207 shutdown_connections 2023-07-21T19:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.732+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3f4580 0x55d74b3f49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.717 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.732+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b2506b0 0x55d74b3f9d90 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.732+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/1345728207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.732+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/1345728207 >> 172.21.15.5:0/1345728207 conn(0x55d74b3e9be0 msgr2=0x55d74b3eef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:07.718 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.733+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/1345728207 shutdown_connections 2023-07-21T19:37:07.719 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.733+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/1345728207 wait complete. 2023-07-21T19:37:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.734+0000 7fb3cd7bcb80 1 -- start start 2023-07-21T19:37:07.720 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.734+0000 7fb3cd7bcb80 1 Processor -- start 2023-07-21T19:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.735+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 0x55d74b400630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.735+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3fd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.721 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.735+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3fe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.735+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b2e5a60 con 0x55d74b3f4580 2023-07-21T19:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.735+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b2e5760 con 0x55d74b2506b0 2023-07-21T19:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.736+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b1c98c0 con 0x55d74b3fa2d0 2023-07-21T19:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.736+0000 7fb3cd7bcb80 10 client.0 _ll_register_callbacks cb 0x55d74b2e3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:07.722 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.737+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3fe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.737+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 0x55d74b400630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.737+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 0x55d74b400630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:47318/0 (socket says 172.21.15.5:47318) 2023-07-21T19:37:07.723 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.737+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/2368888218 learned_addr learned my addr 172.21.15.5:0/2368888218 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:07.724 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.739+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3fd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.726 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.741+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2368888218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 msgr2=0x55d74b3fe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.741+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3fe010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.741+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 msgr2=0x55d74b400630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.741+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 0x55d74b400630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.741+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2368888218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3a4009e20 con 0x55d74b3f4580 2023-07-21T19:37:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.742+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3fd920 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00ca10 tx=0x7fb3ac00ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c8243f6266f8a1f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.742+0000 7fb392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:37:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.743+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac011070 con 0x55d74b3f4580 2023-07-21T19:37:07.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.743+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb3ac006e40 con 0x55d74b3f4580 2023-07-21T19:37:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.743+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d74b29f490 con 0x55d74b3f4580 2023-07-21T19:37:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.743+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3ac0098d0 con 0x55d74b3f4580 2023-07-21T19:37:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.743+0000 7fb3cd7bcb80 10 client.20487 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:07.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55d74b15cc00 con 0x55d74b3f4580 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d74b29f490 con 0x55d74b3f4580 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3ac01d070 con 0x55d74b3f4580 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb3cd7bcb80 10 client.20487 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb3cd7bcb80 10 client.20487 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb3cd7bcb80 20 client.20487 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:07.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.744+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b41ebf0 0x55d74b421000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.745+0000 7fb3cd7bcb80 4 client.20487 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:07.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.745+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d74b33ee50 con 0x55d74b41ebf0 2023-07-21T19:37:07.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.747+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b41ebf0 0x55d74b421000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.742 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.756+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b41ebf0 0x55d74b421000 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88945333e17600ac server_cookie=45ba091f5a72ce14 in_seq=0 out_seq=0 2023-07-21T19:37:07.743 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.757+0000 7fb392ffd700 10 client.20487 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:07.746 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.761+0000 7fb392ffd700 1 -- 172.21.15.5:0/2368888218 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d74b33ee50 con 0x55d74b41ebf0 2023-07-21T19:37:07.747 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.761+0000 7fb392ffd700 10 client.20487 handle_command_reply: tid=0 2023-07-21T19:37:07.748 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.763+0000 7fb3cd7bcb80 1 client.20487 shutdown 2023-07-21T19:37:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.764+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 msgr2=0x55d74b3fd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:07.749 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.764+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3fd920 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00ca10 tx=0x7fb3ac00ca40 comp rx=0 tx=0).stop 2023-07-21T19:37:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 shutdown_connections 2023-07-21T19:37:07.756 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b3fa2d0 0x55d74b3fe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b2506b0 0x55d74b400630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b41ebf0 0x55d74b421000 crc :-1 s=READY pgs=537 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2368888218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3fd920 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb392ffd700 0 client.20487 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.765+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 >> 172.21.15.5:0/2368888218 conn(0x55d74b3e9be0 msgr2=0x55d74b3edc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.767+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 shutdown_connections 2023-07-21T19:37:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.767+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2368888218 wait complete. 2023-07-21T19:37:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.768+0000 7fb3cd7bcb80 20 client.20487 trim_cache size 0 max 16384 2023-07-21T19:37:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.803+0000 7fb3cd7bcb80 1 Processor -- start 2023-07-21T19:37:07.789 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 -- start start 2023-07-21T19:37:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b3eec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3f9200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b4001d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.790 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b3ebd50 con 0x55d74b3f4580 2023-07-21T19:37:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b3feec0 con 0x55d74b3fa2d0 2023-07-21T19:37:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.804+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b3e6c30 con 0x55d74b2506b0 2023-07-21T19:37:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.805+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b3eec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.805+0000 7fb3b2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b3eec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59084/0 (socket says 172.21.15.5:59084) 2023-07-21T19:37:07.791 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.805+0000 7fb3b2ffd700 1 -- 172.21.15.5:0/2016685028 learned_addr learned my addr 172.21.15.5:0/2016685028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:07.792 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.805+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b4001d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2016685028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 msgr2=0x55d74b3eec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b3eec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.793 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2016685028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 msgr2=0x55d74b4001d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b4001d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.806+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2016685028 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d74b19a1f0 con 0x55d74b3f4580 2023-07-21T19:37:07.794 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.807+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3f9200 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00a5f0 tx=0x7fb3ac009390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bdaf38ef7ede8be server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.808+0000 7fb39a7fc700 1 -- 172.21.15.5:0/2016685028 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac011070 con 0x55d74b3f4580 2023-07-21T19:37:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.808+0000 7fb39a7fc700 1 -- 172.21.15.5:0/2016685028 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3ac00a860 con 0x55d74b3f4580 2023-07-21T19:37:07.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.808+0000 7fb39a7fc700 1 -- 172.21.15.5:0/2016685028 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac01d320 con 0x55d74b3f4580 2023-07-21T19:37:07.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2016685028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 msgr2=0x55d74b3f9200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3f9200 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00a5f0 tx=0x7fb3ac009390 comp rx=0 tx=0).stop 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2016685028 shutdown_connections 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b4001d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b3eec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2016685028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b3f9200 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.809+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2016685028 >> 172.21.15.5:0/2016685028 conn(0x55d74b3e9be0 msgr2=0x55d74b3f8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.810+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2016685028 shutdown_connections 2023-07-21T19:37:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.810+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2016685028 wait complete. 2023-07-21T19:37:07.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.810+0000 7fb3cd7bcb80 1 -- start start 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 Processor -- start 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b4046a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b402c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b403250 con 0x55d74b3f4580 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b4033c0 con 0x55d74b3fa2d0 2023-07-21T19:37:07.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d74b403580 con 0x55d74b2506b0 2023-07-21T19:37:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.811+0000 7fb3cd7bcb80 10 client.0 _ll_register_callbacks cb 0x7fb3ac00c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.813+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.813+0000 7fb3b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:51300/0 (socket says 172.21.15.5:51300) 2023-07-21T19:37:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.813+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2273331237 learned_addr learned my addr 172.21.15.5:0/2273331237 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:07.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.813+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b4046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b402c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2273331237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 msgr2=0x55d74b4046a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b4046a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 msgr2=0x55d74b402c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b402c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.814+0000 7fb3b37fe700 1 -- 172.21.15.5:0/2273331237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d74b15cc00 con 0x55d74b3f4580 2023-07-21T19:37:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.815+0000 7fb3b37fe700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac002230 tx=0x7fb3ac00ce40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80019a235258c5f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:07.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.816+0000 7fb392ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:37:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.817+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac011050 con 0x55d74b3f4580 2023-07-21T19:37:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.817+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(136..136 src has 1..136) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb3ac01f630 con 0x55d74b3f4580 2023-07-21T19:37:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.817+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3a4017a60 con 0x55d74b3f4580 2023-07-21T19:37:07.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.818+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3ac01f810 con 0x55d74b3f4580 2023-07-21T19:37:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.818+0000 7fb3cd7bcb80 10 client.20493 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:07.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.818+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55d74b172250 con 0x55d74b3f4580 2023-07-21T19:37:07.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.818+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb3a4017a60 con 0x55d74b3f4580 2023-07-21T19:37:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.818+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb3ac008040 con 0x55d74b3f4580 2023-07-21T19:37:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 10 client.20493 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 10 client.20493 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:07.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 20 client.20493 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b473a10 0x55d74b475df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 4 client.20493 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:37:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.819+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d74b300ff0 con 0x55d74b473a10 2023-07-21T19:37:07.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.820+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b473a10 0x55d74b475df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.821+0000 7fb3b2ffd700 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b473a10 0x55d74b475df0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f98e9bec9d072ba1 server_cookie=341a7df21e1bf6b2 in_seq=0 out_seq=0 2023-07-21T19:37:07.808 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:07.821+0000 7fb392ffd700 10 client.20493 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:07 smithi101 ceph-mon[128136]: pgmap v935: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:37:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:07 smithi101 ceph-mon[128136]: osdmap e136: 12 total, 12 up, 12 in 2023-07-21T19:37:08.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:07 smithi005 ceph-mon[126460]: pgmap v935: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:37:08.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:07 smithi005 ceph-mon[126460]: osdmap e136: 12 total, 12 up, 12 in 2023-07-21T19:37:08.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:07 smithi156 ceph-mon[128316]: pgmap v935: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 7.0 MiB/s wr, 716 op/s 2023-07-21T19:37:08.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:07 smithi156 ceph-mon[128316]: osdmap e136: 12 total, 12 up, 12 in 2023-07-21T19:37:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.175+0000 7fb392ffd700 1 -- 172.21.15.5:0/2273331237 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d74b300ff0 con 0x55d74b473a10 2023-07-21T19:37:08.160 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.175+0000 7fb392ffd700 10 client.20493 handle_command_reply: tid=0 2023-07-21T19:37:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.175+0000 7fb3cd7bcb80 1 client.20493 shutdown 2023-07-21T19:37:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.176+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 msgr2=0x55d74b404be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:08.162 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.176+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac002230 tx=0x7fb3ac00ce40 comp rx=0 tx=0).stop 2023-07-21T19:37:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 shutdown_connections 2023-07-21T19:37:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d74b3fa2d0 0x55d74b402c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.163 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d74b2506b0 0x55d74b4046a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d74b473a10 0x55d74b475df0 crc :-1 s=READY pgs=538 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 --2- 172.21.15.5:0/2273331237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d74b3f4580 0x55d74b404be0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 >> 172.21.15.5:0/2273331237 conn(0x55d74b3e9be0 msgr2=0x55d74b3f9380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb392ffd700 0 client.20493 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.177+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 shutdown_connections 2023-07-21T19:37:08.164 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.178+0000 7fb3cd7bcb80 1 -- 172.21.15.5:0/2273331237 wait complete. 2023-07-21T19:37:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.178+0000 7fb3cd7bcb80 20 client.20493 trim_cache size 0 max 16384 2023-07-21T19:37:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:37:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:37:08.165 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:37:08.166 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:37:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.181+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 msgr2=0x7fb39c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.181+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 0x7fb39c0a38b0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80a2bc0 tx=0x7fb3a8005a00 comp rx=0 tx=0).stop 2023-07-21T19:37:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.181+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 msgr2=0x7fb3b41a4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:08.167 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.181+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 0x7fb3b41a4960 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7fb3a80098e0 tx=0x7fb3a8005740 comp rx=0 tx=0).stop 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 shutdown_connections 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb39c0a1400 0x7fb39c0a38b0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb3b410e2a0 0x7fb3b41a4960 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb3b406c910 0x7fb3b41aab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 --2- 172.21.15.5:0/3635452805 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb3b406d3a0 0x7fb3b41ab040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:08.168 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.182+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 >> 172.21.15.5:0/3635452805 conn(0x7fb3b406bd70 msgr2=0x7fb3b406f200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:08.179 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.187+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 shutdown_connections 2023-07-21T19:37:08.186 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:08.200+0000 7fb3b9dff700 1 -- 172.21.15.5:0/3635452805 wait complete. 2023-07-21T19:37:09.322 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:08 smithi101 ceph-mon[128136]: pgmap v937: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:09.398 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:08 smithi005 ceph-mon[126460]: pgmap v937: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:09.405 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:08 smithi156 ceph-mon[128316]: pgmap v937: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:11.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:11 smithi156 ceph-mon[128316]: pgmap v938: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:11.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 640.0 s, 119.3 tps, lat 883.687 ms stddev 669.224 2023-07-21T19:37:12.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:11 smithi101 ceph-mon[128136]: pgmap v938: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:12.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:11 smithi005 ceph-mon[126460]: pgmap v938: 97 pgs: 23 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 34 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.8 MiB/s wr, 674 op/s 2023-07-21T19:37:13.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:13 smithi156 ceph-mon[128316]: pgmap v939: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2023-07-21T19:37:14.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:13 smithi101 ceph-mon[128136]: pgmap v939: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2023-07-21T19:37:14.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:13 smithi005 ceph-mon[126460]: pgmap v939: 97 pgs: 17 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 52 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.1 MiB/s wr, 574 op/s 2023-07-21T19:37:15.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:15 smithi156 ceph-mon[128316]: pgmap v940: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:16.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:15 smithi101 ceph-mon[128136]: pgmap v940: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:16.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:15 smithi005 ceph-mon[126460]: pgmap v940: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 645.0 s, 228.4 tps, lat 422.763 ms stddev 131.097 2023-07-21T19:37:18.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:17 smithi101 ceph-mon[128136]: pgmap v941: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:18.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:17 smithi005 ceph-mon[126460]: pgmap v941: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:18.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:17 smithi156 ceph-mon[128316]: pgmap v941: 97 pgs: 97 active+clean; 11 GiB data, 17 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.7 MiB/s wr, 451 op/s 2023-07-21T19:37:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:18 smithi005 ceph-mon[126460]: osdmap e137: 12 total, 12 up, 12 in 2023-07-21T19:37:19.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:18 smithi005 ceph-mon[126460]: pgmap v943: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 514 op/s 2023-07-21T19:37:19.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:18 smithi156 ceph-mon[128316]: osdmap e137: 12 total, 12 up, 12 in 2023-07-21T19:37:19.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:18 smithi156 ceph-mon[128316]: pgmap v943: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 514 op/s 2023-07-21T19:37:19.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:18 smithi101 ceph-mon[128136]: osdmap e137: 12 total, 12 up, 12 in 2023-07-21T19:37:19.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:18 smithi101 ceph-mon[128136]: pgmap v943: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.3 MiB/s wr, 514 op/s 2023-07-21T19:37:21.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 -- 172.21.15.5:0/789522538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd01043f0 msgr2=0x7f4dd01047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/789522538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd01043f0 0x7f4dd01047c0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009990 tx=0x7f4dc0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 -- 172.21.15.5:0/789522538 shutdown_connections 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/789522538 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd010b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/789522538 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd01043f0 0x7f4dd01047c0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/789522538 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd00fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.442 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 -- 172.21.15.5:0/789522538 >> 172.21.15.5:0/789522538 conn(0x7f4dd0073d50 msgr2=0x7f4dd0074150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 -- 172.21.15.5:0/789522538 shutdown_connections 2023-07-21T19:37:21.443 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.455+0000 7f4dd61ff700 1 -- 172.21.15.5:0/789522538 wait complete. 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.457+0000 7f4dd61ff700 1 Processor -- start 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.457+0000 7f4dd61ff700 1 -- start start 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd01984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 0x7f4dd019cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.444 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019d160 con 0x7f4dd00fe7a0 2023-07-21T19:37:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019d2d0 con 0x7f4dd00ff150 2023-07-21T19:37:21.445 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.458+0000 7f4dd61ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dd019d490 con 0x7f4dd01043f0 2023-07-21T19:37:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.459+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.459+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59318/0 (socket says 172.21.15.5:59318) 2023-07-21T19:37:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.459+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/2134422727 learned_addr learned my addr 172.21.15.5:0/2134422727 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.459+0000 7f4dcffff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 0x7f4dd019cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.446 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.459+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd01984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 -- 172.21.15.5:0/2134422727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 msgr2=0x7f4dd01984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd01984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 -- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 msgr2=0x7f4dd0197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.447 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 -- 172.21.15.5:0/2134422727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00095f0 con 0x7f4dd01043f0 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcffff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 0x7f4dd019cb20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800efc0 tx=0x7f4dc800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcd7fa700 1 -- 172.21.15.5:0/2134422727 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc8018070 con 0x7f4dd01043f0 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcd7fa700 1 -- 172.21.15.5:0/2134422727 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc800f040 con 0x7f4dd01043f0 2023-07-21T19:37:21.448 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.460+0000 7f4dcd7fa700 1 -- 172.21.15.5:0/2134422727 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc800cba0 con 0x7f4dd01043f0 2023-07-21T19:37:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.461+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dd019d710 con 0x7f4dd01043f0 2023-07-21T19:37:21.449 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.461+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd01a4fb0 con 0x7f4dd01043f0 2023-07-21T19:37:21.454 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.468+0000 7f4dcd7fa700 1 -- 172.21.15.5:0/2134422727 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4dc8003a00 con 0x7f4dd01043f0 2023-07-21T19:37:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 Processor -- start 2023-07-21T19:37:21.455 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 -- start start 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a74246b0 0x55f2a75cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a74cdc50 con 0x55f2a74246b0 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a75209e0 con 0x55f2a75ce2d0 2023-07-21T19:37:21.456 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.470+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a7519e70 con 0x55f2a75c8580 2023-07-21T19:37:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35728/0 (socket says 172.21.15.5:35728) 2023-07-21T19:37:21.457 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/761957831 learned_addr learned my addr 172.21.15.5:0/761957831 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a74246b0 0x55f2a75cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/761957831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 msgr2=0x55f2a75c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.458 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/761957831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a74246b0 msgr2=0x55f2a75cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a74246b0 0x55f2a75cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.471+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/761957831 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a7347d80 con 0x55f2a75ce2d0 2023-07-21T19:37:21.459 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.472+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000bbd0 tx=0x7f4dc000bc00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=52e7d5a01955fbe3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.472+0000 7f4dc5ffb700 1 -- 172.21.15.5:0/761957831 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc001d070 con 0x55f2a75ce2d0 2023-07-21T19:37:21.460 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.472+0000 7f4dc5ffb700 1 -- 172.21.15.5:0/761957831 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc001e040 con 0x55f2a75ce2d0 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.472+0000 7f4dc5ffb700 1 -- 172.21.15.5:0/761957831 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc0004ee0 con 0x55f2a75ce2d0 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.473+0000 7f4dcd7fa700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 0x7f4db00a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.473+0000 7f4dcd7fa700 1 -- 172.21.15.5:0/2134422727 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4dc80cc7f0 con 0x7f4dd01043f0 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.473+0000 7f4de9b87b80 1 -- 172.21.15.5:0/761957831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 msgr2=0x55f2a75c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.473+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000bbd0 tx=0x7f4dc000bc00 comp rx=0 tx=0).stop 2023-07-21T19:37:21.461 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.473+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 0x7f4db00a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 -- 172.21.15.5:0/761957831 shutdown_connections 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a74246b0 0x55f2a75cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/761957831 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a75ce2d0 0x55f2a75c8040 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 -- 172.21.15.5:0/761957831 >> 172.21.15.5:0/761957831 conn(0x55f2a75bdbe0 msgr2=0x55f2a75c2f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 -- 172.21.15.5:0/761957831 shutdown_connections 2023-07-21T19:37:21.462 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4de9b87b80 1 -- 172.21.15.5:0/761957831 wait complete. 2023-07-21T19:37:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.474+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 0x7f4db00a3870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4db80093b0 tx=0x7f4db8009190 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.475+0000 7f4de9b87b80 1 -- start start 2023-07-21T19:37:21.463 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.475+0000 7f4de9b87b80 1 Processor -- start 2023-07-21T19:37:21.464 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75c8580 0x55f2a75d18a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75ce2d0 0x55f2a75d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a74b9a60 con 0x55f2a75c8580 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a74b9760 con 0x55f2a74246b0 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a739d8c0 con 0x55f2a75ce2d0 2023-07-21T19:37:21.465 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4de9b87b80 10 client.0 _ll_register_callbacks cb 0x55f2a74b7700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35730/0 (socket says 172.21.15.5:35730) 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1366182616 learned_addr learned my addr 172.21.15.5:0/1366182616 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.476+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75c8580 0x55f2a75d18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcffff700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75ce2d0 0x55f2a75d1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1366182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75ce2d0 msgr2=0x55f2a75d1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.466 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75ce2d0 0x55f2a75d1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1366182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75c8580 msgr2=0x55f2a75d18a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75c8580 0x55f2a75d18a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1366182616 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dd00fe290 con 0x55f2a74246b0 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00048c0 tx=0x7f4dc0004cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fcfe01d61c612126 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.477+0000 7f4dbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc001d070 con 0x55f2a74246b0 2023-07-21T19:37:21.467 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f2a7473490 con 0x55f2a74246b0 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4dc00058e0 con 0x55f2a74246b0 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4dc0005550 con 0x55f2a74246b0 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4de9b87b80 10 client.30287 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55f2a7330c00 con 0x55f2a74246b0 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.478+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f2a7473490 con 0x55f2a74246b0 2023-07-21T19:37:21.468 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4dc002c440 con 0x55f2a74246b0 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 10 client.30287 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 10 client.30287 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 20 client.30287 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75e15d0 0x55f2a75e39e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 4 client.30287 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f2a7512e50 con 0x55f2a75e15d0 2023-07-21T19:37:21.469 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.479+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75e15d0 0x55f2a75e39e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.470 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.480+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75e15d0 0x55f2a75e39e0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=26a43a38b54468c4 server_cookie=4727a9ab5d4adc03 in_seq=0 out_seq=0 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.480+0000 7f4dbeffd700 10 client.30287 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.481+0000 7f4dbeffd700 1 -- 172.21.15.5:0/1366182616 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f2a7512e50 con 0x55f2a75e15d0 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.481+0000 7f4dbeffd700 10 client.30287 handle_command_reply: tid=0 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.481+0000 7f4de9b87b80 1 client.30287 shutdown 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.481+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 msgr2=0x55f2a75d4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f4dc00048c0 tx=0x7f4dc0004cf0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.471 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 shutdown_connections 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75ce2d0 0x55f2a75d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d4580 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75e15d0 0x55f2a75e39e0 crc :-1 s=READY pgs=539 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1366182616 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75c8580 0x55f2a75d18a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 >> 172.21.15.5:0/1366182616 conn(0x55f2a75bdbe0 msgr2=0x55f2a75c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4dbeffd700 0 client.30287 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:21.472 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 shutdown_connections 2023-07-21T19:37:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1366182616 wait complete. 2023-07-21T19:37:21.473 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.482+0000 7f4de9b87b80 20 client.30287 trim_cache size 0 max 16384 2023-07-21T19:37:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 Processor -- start 2023-07-21T19:37:21.492 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 -- start start 2023-07-21T19:37:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.493 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c2c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a75bac30 con 0x55f2a75bdbe0 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a75c3b50 con 0x55f2a74246b0 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.507+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a75ce190 con 0x55f2a75c8580 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.494 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c2c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75cd160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35746/0 (socket says 172.21.15.5:35746) 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c2c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:44102/0 (socket says 172.21.15.5:44102) 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59330/0 (socket says 172.21.15.5:59330) 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dc7fff700 1 -- 172.21.15.5:0/1016011864 learned_addr learned my addr 172.21.15.5:0/1016011864 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:21.495 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1016011864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 msgr2=0x55f2a75c2c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c2c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1016011864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 msgr2=0x55f2a75cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.496 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.508+0000 7f4dcf7fe700 1 -- 172.21.15.5:0/1016011864 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f2a75e8610 con 0x55f2a75bdbe0 2023-07-21T19:37:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.509+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009ce0 tx=0x7f4dc002ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8693e89816c95986 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.497 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.509+0000 7f4dc67fc700 1 -- 172.21.15.5:0/1016011864 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc001d070 con 0x55f2a75bdbe0 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.509+0000 7f4dc67fc700 1 -- 172.21.15.5:0/1016011864 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc0004e10 con 0x55f2a75bdbe0 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.509+0000 7f4dc67fc700 1 -- 172.21.15.5:0/1016011864 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000f730 con 0x55f2a75bdbe0 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.510+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1016011864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 msgr2=0x55f2a75d8c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.510+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f4dc0009ce0 tx=0x7f4dc002ca40 comp rx=0 tx=0).stop 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1016011864 shutdown_connections 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a75c2c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.498 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d8c70 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/1016011864 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1016011864 >> 172.21.15.5:0/1016011864 conn(0x55f2a75dd770 msgr2=0x55f2a75cca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1016011864 shutdown_connections 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 -- 172.21.15.5:0/1016011864 wait complete. 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.511+0000 7f4de9b87b80 1 -- start start 2023-07-21T19:37:21.499 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.512+0000 7f4de9b87b80 1 Processor -- start 2023-07-21T19:37:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a762e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a762e9a0 con 0x55f2a75bdbe0 2023-07-21T19:37:21.500 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a762eae0 con 0x55f2a74246b0 2023-07-21T19:37:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f2a762ec50 con 0x55f2a75c8580 2023-07-21T19:37:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.513+0000 7f4de9b87b80 10 client.0 _ll_register_callbacks cb 0x7f4dc000baf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.516+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.516+0000 7f4dc7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59346/0 (socket says 172.21.15.5:59346) 2023-07-21T19:37:21.501 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.516+0000 7f4dc7fff700 1 -- 172.21.15.5:0/2830521643 learned_addr learned my addr 172.21.15.5:0/2830521643 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.516+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 -- 172.21.15.5:0/2830521643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 msgr2=0x55f2a762e460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a762e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.502 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 -- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 msgr2=0x55f2a75d6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 -- 172.21.15.5:0/2830521643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f2a7330c00 con 0x55f2a75bdbe0 2023-07-21T19:37:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.517+0000 7f4dc7fff700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009ac0 tx=0x7f4db8004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3800cb0e985037a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:21.503 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.518+0000 7f4dbeffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:37:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.520+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8029040 con 0x55f2a75bdbe0 2023-07-21T19:37:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.520+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f4db802f070 con 0x55f2a75bdbe0 2023-07-21T19:37:21.506 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.521+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4dc001e040 con 0x55f2a75bdbe0 2023-07-21T19:37:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.521+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4db80295b0 con 0x55f2a75bdbe0 2023-07-21T19:37:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.521+0000 7f4de9b87b80 10 client.20511 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55f2a736e1f0 con 0x55f2a75bdbe0 2023-07-21T19:37:21.507 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4dc001e040 con 0x55f2a75bdbe0 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f4db8034780 con 0x55f2a75bdbe0 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 10 client.20511 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 10 client.20511 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 20 client.20511 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75ce2d0 0x55f2a76494b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:21.508 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 4 client.20511 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:37:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.522+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55f2a74d4ff0 con 0x55f2a75ce2d0 2023-07-21T19:37:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.523+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75ce2d0 0x55f2a76494b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.523+0000 7f4dbeffd700 10 client.20511 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:21.509 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:21.523+0000 7f4dcf7fe700 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75ce2d0 0x55f2a76494b0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f081316f21a7dfeb server_cookie=d70de9aac89b7983 in_seq=0 out_seq=0 2023-07-21T19:37:21.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:21 smithi156 ceph-mon[128316]: pgmap v944: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2023-07-21T19:37:21.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 650.0 s, 241.4 tps, lat 435.061 ms stddev 152.604 2023-07-21T19:37:22.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:21 smithi101 ceph-mon[128136]: pgmap v944: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2023-07-21T19:37:22.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:21 smithi005 ceph-mon[126460]: pgmap v944: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.4 MiB/s wr, 533 op/s 2023-07-21T19:37:24.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:23 smithi101 ceph-mon[128136]: pgmap v945: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 585 op/s 2023-07-21T19:37:24.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:23 smithi005 ceph-mon[126460]: pgmap v945: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 585 op/s 2023-07-21T19:37:24.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:23 smithi156 ceph-mon[128316]: pgmap v945: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.1 MiB/s wr, 585 op/s 2023-07-21T19:37:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4dbeffd700 1 -- 172.21.15.5:0/2830521643 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55f2a74d4ff0 con 0x55f2a75ce2d0 2023-07-21T19:37:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4dbeffd700 10 client.20511 handle_command_reply: tid=0 2023-07-21T19:37:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 client.20511 shutdown 2023-07-21T19:37:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 msgr2=0x55f2a75d7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:25.795 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f4db8009ac0 tx=0x7f4db8004860 comp rx=0 tx=0).stop 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 shutdown_connections 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f2a75c8580 0x55f2a762e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f2a74246b0 0x55f2a75d6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f2a75ce2d0 0x55f2a76494b0 crc :-1 s=READY pgs=540 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 --2- 172.21.15.5:0/2830521643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f2a75bdbe0 0x55f2a75d7380 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 >> 172.21.15.5:0/2830521643 conn(0x55f2a75dd770 msgr2=0x55f2a75c7030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:25.796 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.809+0000 7f4dbeffd700 0 client.20511 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.810+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 shutdown_connections 2023-07-21T19:37:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.810+0000 7f4de9b87b80 1 -- 172.21.15.5:0/2830521643 wait complete. 2023-07-21T19:37:25.797 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.810+0000 7f4de9b87b80 20 client.20511 trim_cache size 0 max 16384 2023-07-21T19:37:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:37:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:37:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:37:25.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.810+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 msgr2=0x7f4db00a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.810+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 0x7f4db00a3870 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4db80093b0 tx=0x7f4db8009190 comp rx=0 tx=0).stop 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 msgr2=0x7f4dd019cb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 0x7f4dd019cb20 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4dc800efc0 tx=0x7f4dc800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 shutdown_connections 2023-07-21T19:37:25.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4dd01043f0 0x7f4dd019cb20 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4db00a13c0 0x7f4db00a3870 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4dd00fe7a0 0x7f4dd0197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 --2- 172.21.15.5:0/2134422727 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4dd00ff150 0x7f4dd01984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 >> 172.21.15.5:0/2134422727 conn(0x7f4dd0073d50 msgr2=0x7f4dd00fc470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 shutdown_connections 2023-07-21T19:37:25.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:25.811+0000 7f4dd61ff700 1 -- 172.21.15.5:0/2134422727 wait complete. 2023-07-21T19:37:26.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:25 smithi101 ceph-mon[128136]: pgmap v946: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:26.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:25 smithi005 ceph-mon[126460]: pgmap v946: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:26.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:25 smithi156 ceph-mon[128316]: pgmap v946: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 655.0 s, 259.4 tps, lat 385.763 ms stddev 68.267 2023-07-21T19:37:28.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:27 smithi101 ceph-mon[128136]: pgmap v947: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:28.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:27 smithi005 ceph-mon[126460]: pgmap v947: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:28.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:27 smithi156 ceph-mon[128316]: pgmap v947: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.4 MiB/s wr, 702 op/s 2023-07-21T19:37:29.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:28 smithi005 ceph-mon[126460]: pgmap v948: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 784 op/s 2023-07-21T19:37:29.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:28 smithi156 ceph-mon[128316]: pgmap v948: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 784 op/s 2023-07-21T19:37:29.318 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:28 smithi101 ceph-mon[128136]: pgmap v948: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.6 MiB/s wr, 784 op/s 2023-07-21T19:37:31.107 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:37:31.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 660.0 s, 225.7 tps, lat 444.681 ms stddev 106.411 2023-07-21T19:37:32.040 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:31 smithi005 ceph-mon[126460]: pgmap v949: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 670 op/s 2023-07-21T19:37:32.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:31 smithi101 ceph-mon[128136]: pgmap v949: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 670 op/s 2023-07-21T19:37:32.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:31 smithi156 ceph-mon[128316]: pgmap v949: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.6 MiB/s wr, 670 op/s 2023-07-21T19:37:32.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 -- 172.21.15.5:0/1153315263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00fe9d0 msgr2=0x7f19a00fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 --2- 172.21.15.5:0/1153315263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00fe9d0 0x7f19a00fee40 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f19900099c0 tx=0x7f1990009cd0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.286 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 -- 172.21.15.5:0/1153315263 shutdown_connections 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 --2- 172.21.15.5:0/1153315263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a00ff380 0x7f19a0067670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 --2- 172.21.15.5:0/1153315263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a01049e0 0x7f19a0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 --2- 172.21.15.5:0/1153315263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00fe9d0 0x7f19a00fee40 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 -- 172.21.15.5:0/1153315263 >> 172.21.15.5:0/1153315263 conn(0x7f19a00fa4b0 msgr2=0x7f19a00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.301+0000 7f19a7bff700 1 -- 172.21.15.5:0/1153315263 shutdown_connections 2023-07-21T19:37:32.287 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.302+0000 7f19a7bff700 1 -- 172.21.15.5:0/1153315263 wait complete. 2023-07-21T19:37:32.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.303+0000 7f19a7bff700 1 Processor -- start 2023-07-21T19:37:32.288 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.304+0000 7f19a7bff700 1 -- start start 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.304+0000 7f19a7bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 0x7f19a01a1f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.304+0000 7f19a7bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a7bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 0x7f19a0105ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a7bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01064b0 con 0x7f19a00fe9d0 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a7bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a0106620 con 0x7f19a01049e0 2023-07-21T19:37:32.289 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a7bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19a01067e0 con 0x7f19a00ff380 2023-07-21T19:37:32.290 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.291 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 0x7f19a0105ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49950/0 (socket says 172.21.15.5:49950) 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 0x7f19a0105ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34882/0 (socket says 172.21.15.5:34882) 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a519a700 1 -- 172.21.15.5:0/3420580282 learned_addr learned my addr 172.21.15.5:0/3420580282 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.305+0000 7f19a599b700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 0x7f19a01a1f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 -- 172.21.15.5:0/3420580282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 msgr2=0x7f19a0105ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.292 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 0x7f19a0105ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 -- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 msgr2=0x7f19a01a1f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 0x7f19a01a1f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 -- 172.21.15.5:0/3420580282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19900095f0 con 0x7f19a00ff380 2023-07-21T19:37:32.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a599b700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 0x7f19a01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:37:32.293 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.306+0000 7f19a519a700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.307+0000 7f1996ffd700 1 -- 172.21.15.5:0/3420580282 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199001d070 con 0x7f19a00ff380 2023-07-21T19:37:32.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.307+0000 7f1996ffd700 1 -- 172.21.15.5:0/3420580282 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1990022470 con 0x7f19a00ff380 2023-07-21T19:37:32.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.307+0000 7f1996ffd700 1 -- 172.21.15.5:0/3420580282 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1990017650 con 0x7f19a00ff380 2023-07-21T19:37:32.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.307+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a0106a60 con 0x7f19a00ff380 2023-07-21T19:37:32.294 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.308+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19a01a8e90 con 0x7f19a00ff380 2023-07-21T19:37:32.297 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.313+0000 7f1996ffd700 1 -- 172.21.15.5:0/3420580282 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f199000fa90 con 0x7f19a00ff380 2023-07-21T19:37:32.299 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.315+0000 7f19bb61eb80 1 Processor -- start 2023-07-21T19:37:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.315+0000 7f19bb61eb80 1 -- start start 2023-07-21T19:37:32.300 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555650371be0 0x55565037c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x55565037eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650277cb0 con 0x555650371be0 2023-07-21T19:37:32.301 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556502caa40 con 0x55565037cad0 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556502c3ed0 con 0x5556503712d0 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f1996ffd700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 0x7f198c0a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19a619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.316+0000 7f19a619c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49952/0 (socket says 172.21.15.5:49952) 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 -- 172.21.15.5:0/875563467 learned_addr learned my addr 172.21.15.5:0/875563467 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a599b700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555650371be0 0x55565037c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.302 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f1996ffd700 1 -- 172.21.15.5:0/3420580282 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f19900cdbd0 con 0x7f19a00ff380 2023-07-21T19:37:32.303 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a519a700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x55565037eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 -- 172.21.15.5:0/875563467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 msgr2=0x55565037eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x55565037eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 -- 172.21.15.5:0/875563467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555650371be0 msgr2=0x55565037c590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555650371be0 0x55565037c590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 -- 172.21.15.5:0/875563467 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5556500f1de0 con 0x5556503712d0 2023-07-21T19:37:32.304 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a519a700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x55565037eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.317+0000 7f19a619c700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 0x7f198c0a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.318+0000 7f19a619c700 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b840 tx=0x7f199c00bb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f27cb8786a76c733 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.318+0000 7f198bfff700 1 -- 172.21.15.5:0/875563467 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c003c40 con 0x5556503712d0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.318+0000 7f198bfff700 1 -- 172.21.15.5:0/875563467 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f199c003e20 con 0x5556503712d0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.318+0000 7f198bfff700 1 -- 172.21.15.5:0/875563467 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c012980 con 0x5556503712d0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.319+0000 7f19a619c700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 0x7f198c0a3870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f199c01f000 tx=0x7f199c012f00 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.319+0000 7f19bb61eb80 1 -- 172.21.15.5:0/875563467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 msgr2=0x5556503716a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.305 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.319+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f199c00b840 tx=0x7f199c00bb50 comp rx=0 tx=0).stop 2023-07-21T19:37:32.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 -- 172.21.15.5:0/875563467 shutdown_connections 2023-07-21T19:37:32.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x55565037eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.306 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5556503712d0 0x5556503716a0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/875563467 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555650371be0 0x55565037c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 -- 172.21.15.5:0/875563467 >> 172.21.15.5:0/875563467 conn(0x555650367da0 msgr2=0x55565036d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 -- 172.21.15.5:0/875563467 shutdown_connections 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.320+0000 7f19bb61eb80 1 -- 172.21.15.5:0/875563467 wait complete. 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.321+0000 7f19bb61eb80 1 -- start start 2023-07-21T19:37:32.307 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 Processor -- start 2023-07-21T19:37:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556503712d0 0x55565037c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650379800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.308 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650263ac0 con 0x5556503712d0 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5556502637c0 con 0x55565037cad0 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.322+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650147920 con 0x555650371be0 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.323+0000 7f19bb61eb80 10 client.0 _ll_register_callbacks cb 0x555650261760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.323+0000 7f19a619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.323+0000 7f19a619c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34902/0 (socket says 172.21.15.5:34902) 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.323+0000 7f19a619c700 1 -- 172.21.15.5:0/2305376745 learned_addr learned my addr 172.21.15.5:0/2305376745 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.323+0000 7f19a519a700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650379800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 -- 172.21.15.5:0/2305376745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 msgr2=0x555650379800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650379800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 -- 172.21.15.5:0/2305376745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556503712d0 msgr2=0x55565037c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556503712d0 0x55565037c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.310 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 -- 172.21.15.5:0/2305376745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199c004450 con 0x55565037cad0 2023-07-21T19:37:32.311 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19a619c700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f199c012670 tx=0x7f199c00d000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7697b31868811079 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.324+0000 7f19897fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c01ce00 con 0x55565037cad0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55565021d4f0 con 0x55565037cad0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f199c003c40 con 0x55565037cad0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199c003a00 con 0x55565037cad0 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19bb61eb80 10 client.30302 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:32.312 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.325+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5556500dac60 con 0x55565037cad0 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f199c03d020 con 0x55565037cad0 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f199c024050 con 0x55565037cad0 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 10 client.30302 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 10 client.30302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 20 client.30302 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55565038c2a0 0x55565038e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 4 client.30302 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:32.313 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.326+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5556502bceb0 con 0x55565038c2a0 2023-07-21T19:37:32.314 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.327+0000 7f19a599b700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55565038c2a0 0x55565038e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.328+0000 7f19a599b700 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55565038c2a0 0x55565038e6b0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd0043bc64c06cc2 server_cookie=531c5dbe0a2caec5 in_seq=0 out_seq=0 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.328+0000 7f19897fa700 10 client.30302 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.329+0000 7f19897fa700 1 -- 172.21.15.5:0/2305376745 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5556502bceb0 con 0x55565038c2a0 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.329+0000 7f19897fa700 10 client.30302 handle_command_reply: tid=0 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.329+0000 7f19bb61eb80 1 client.30302 shutdown 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.329+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 msgr2=0x555650379f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.315 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.329+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f199c012670 tx=0x7f199c00d000 comp rx=0 tx=0).stop 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 shutdown_connections 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55565037cad0 0x555650379f40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55565038c2a0 0x55565038e6b0 crc :-1 s=READY pgs=541 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5556503712d0 0x55565037c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/2305376745 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650379800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 >> 172.21.15.5:0/2305376745 conn(0x555650367da0 msgr2=0x55565036d230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.316 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19897fa700 0 client.30302 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 shutdown_connections 2023-07-21T19:37:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 1 -- 172.21.15.5:0/2305376745 wait complete. 2023-07-21T19:37:32.317 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.330+0000 7f19bb61eb80 20 client.30302 trim_cache size 0 max 16384 2023-07-21T19:37:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 Processor -- start 2023-07-21T19:37:32.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 -- start start 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x55565037c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650378fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x555650379500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650372d60 con 0x55565037cad0 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650364f80 con 0x5556503712d0 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.363+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55565037a780 con 0x555650371be0 2023-07-21T19:37:32.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650378fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a519a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650378fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:49976/0 (socket says 172.21.15.5:49976) 2023-07-21T19:37:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a519a700 1 -- 172.21.15.5:0/1879055534 learned_addr learned my addr 172.21.15.5:0/1879055534 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x555650379500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a599b700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x55565037c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 -- 172.21.15.5:0/1879055534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 msgr2=0x555650378fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650378fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 -- 172.21.15.5:0/1879055534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 msgr2=0x55565037c0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x55565037c0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a619c700 1 -- 172.21.15.5:0/1879055534 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555650118250 con 0x55565037cad0 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.364+0000 7f19a599b700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x55565037c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.365+0000 7f19a619c700 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x555650379500 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f199c01e010 tx=0x7f199c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9978c8c42ba8ff45 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.365+0000 7f1994bff700 1 -- 172.21.15.5:0/1879055534 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c016040 con 0x55565037cad0 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.365+0000 7f1994bff700 1 -- 172.21.15.5:0/1879055534 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f199c03d070 con 0x55565037cad0 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.365+0000 7f1994bff700 1 -- 172.21.15.5:0/1879055534 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199c003b30 con 0x55565037cad0 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1879055534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 msgr2=0x555650379500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x555650379500 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f199c01e010 tx=0x7f199c000bc0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1879055534 shutdown_connections 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x555650379500 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x55565037c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1879055534 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650378fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.366+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1879055534 >> 172.21.15.5:0/1879055534 conn(0x555650367da0 msgr2=0x555650374b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.367+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1879055534 shutdown_connections 2023-07-21T19:37:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.367+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1879055534 wait complete. 2023-07-21T19:37:32.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.367+0000 7f19bb61eb80 1 -- start start 2023-07-21T19:37:32.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 Processor -- start 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650383430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x5556503814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650381aa0 con 0x55565037cad0 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650381c10 con 0x5556503712d0 2023-07-21T19:37:32.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555650381dd0 con 0x555650371be0 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.368+0000 7f19bb61eb80 10 client.0 _ll_register_callbacks cb 0x7f1998009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.369+0000 7f19a599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.369+0000 7f19a599b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:34916/0 (socket says 172.21.15.5:34916) 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.369+0000 7f19a599b700 1 -- 172.21.15.5:0/1838998450 learned_addr learned my addr 172.21.15.5:0/1838998450 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a619c700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x5556503814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a519a700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650383430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 -- 172.21.15.5:0/1838998450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 msgr2=0x555650383430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650383430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 -- 172.21.15.5:0/1838998450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 msgr2=0x5556503814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x5556503814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 -- 172.21.15.5:0/1838998450 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1998009280 con 0x5556503712d0 2023-07-21T19:37:32.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19a599b700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55565038efc0 tx=0x7f1998009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=507069dac6ab036a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.370+0000 7f19897fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f199800e040 con 0x5556503712d0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f199801c120 con 0x5556503712d0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f199c003e20 con 0x5556503712d0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f199801c2e0 con 0x5556503712d0 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19bb61eb80 10 client.30308 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:32.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.371+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5556500dac60 con 0x5556503712d0 2023-07-21T19:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f199c003e20 con 0x5556503712d0 2023-07-21T19:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1998007640 con 0x5556503712d0 2023-07-21T19:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 10 client.30308 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 10 client.30308 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:32.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 20 client.30308 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5556503e1380 0x5556503e3790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 4 client.30308 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.372+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55565027f050 con 0x5556503e1380 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.373+0000 7f19a519a700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5556503e1380 0x5556503e3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.373+0000 7f19a519a700 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5556503e1380 0x5556503e3790 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66aeaa960fc9c2ee server_cookie=451500d73781909b in_seq=0 out_seq=0 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.374+0000 7f19897fa700 10 client.30308 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:32.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.374+0000 7f19897fa700 1 -- 172.21.15.5:0/1838998450 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55565027f050 con 0x5556503e1380 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.374+0000 7f19897fa700 10 client.30308 handle_command_reply: tid=0 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.375+0000 7f19bb61eb80 1 client.30308 shutdown 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 msgr2=0x555650382ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55565038efc0 tx=0x7f1998009920 comp rx=0 tx=0).stop 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 shutdown_connections 2023-07-21T19:37:32.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5556503e1380 0x5556503e3790 crc :-1 s=READY pgs=542 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55565037cad0 0x5556503814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5556503712d0 0x555650382ef0 secure :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0x55565038efc0 tx=0x7f1998009920 comp rx=0 tx=0).stop 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 --2- 172.21.15.5:0/1838998450 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555650371be0 0x555650383430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19897fa700 0 client.30308 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 >> 172.21.15.5:0/1838998450 conn(0x555650367da0 msgr2=0x5556503733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.376+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 shutdown_connections 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.377+0000 7f19bb61eb80 1 -- 172.21.15.5:0/1838998450 wait complete. 2023-07-21T19:37:32.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.377+0000 7f19bb61eb80 20 client.30308 trim_cache size 0 max 16384 2023-07-21T19:37:32.365 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:37:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:37:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:37:32.365 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:37:32.366 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 msgr2=0x7f198c0a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 0x7f198c0a3870 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f199c01f000 tx=0x7f199c012f00 comp rx=0 tx=0).stop 2023-07-21T19:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 msgr2=0x7f19a01a2470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f1990000c00 tx=0x7f1990005f10 comp rx=0 tx=0).stop 2023-07-21T19:37:32.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 shutdown_connections 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f19a01049e0 0x7f19a0105ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f198c0a13c0 0x7f198c0a3870 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f19a00fe9d0 0x7f19a01a1f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 --2- 172.21.15.5:0/3420580282 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f19a00ff380 0x7f19a01a2470 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 >> 172.21.15.5:0/3420580282 conn(0x7f19a00fa4b0 msgr2=0x7f19a0100510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 shutdown_connections 2023-07-21T19:37:32.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:37:32.378+0000 7f19a7bff700 1 -- 172.21.15.5:0/3420580282 wait complete. 2023-07-21T19:37:32.390 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:37:33.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:32 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:37:33.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:32 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:37:33.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:32 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:37:34.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:33 smithi101 ceph-mon[128136]: pgmap v950: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 748 op/s 2023-07-21T19:37:34.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:33 smithi005 ceph-mon[126460]: pgmap v950: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 748 op/s 2023-07-21T19:37:34.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:33 smithi156 ceph-mon[128316]: pgmap v950: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 748 op/s 2023-07-21T19:37:36.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:35 smithi101 ceph-mon[128136]: pgmap v951: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.8 MiB/s wr, 862 op/s 2023-07-21T19:37:36.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:35 smithi005 ceph-mon[126460]: pgmap v951: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.8 MiB/s wr, 862 op/s 2023-07-21T19:37:36.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:35 smithi156 ceph-mon[128316]: pgmap v951: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.8 MiB/s wr, 862 op/s 2023-07-21T19:37:36.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 665.0 s, 156.0 tps, lat 209.601 ms stddev 168.751 2023-07-21T19:37:38.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:37 smithi101 ceph-mon[128136]: pgmap v952: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 642 op/s 2023-07-21T19:37:38.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:37 smithi005 ceph-mon[126460]: pgmap v952: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 642 op/s 2023-07-21T19:37:38.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:37 smithi156 ceph-mon[128316]: pgmap v952: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.4 MiB/s wr, 642 op/s 2023-07-21T19:37:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 -- 172.21.15.5:0/60232746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c0ffb00 msgr2=0x7f6b0c0fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 --2- 172.21.15.5:0/60232746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c0fff70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f6afc009990 tx=0x7f6afc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 -- 172.21.15.5:0/60232746 shutdown_connections 2023-07-21T19:37:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 --2- 172.21.15.5:0/60232746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c1004b0 0x7f6b0c10c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 --2- 172.21.15.5:0/60232746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c105b10 0x7f6b0c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 --2- 172.21.15.5:0/60232746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c0fff70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.037+0000 7f6b145ff700 1 -- 172.21.15.5:0/60232746 >> 172.21.15.5:0/60232746 conn(0x7f6b0c0fb600 msgr2=0x7f6b0c0fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.038+0000 7f6b145ff700 1 -- 172.21.15.5:0/60232746 shutdown_connections 2023-07-21T19:37:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.038+0000 7f6b145ff700 1 -- 172.21.15.5:0/60232746 wait complete. 2023-07-21T19:37:39.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.039+0000 7f6b145ff700 1 Processor -- start 2023-07-21T19:37:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 -- start start 2023-07-21T19:37:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1004b0 0x7f6b0c198eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c109c70 con 0x7f6b0c0ffb00 2023-07-21T19:37:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c109de0 con 0x7f6b0c1004b0 2023-07-21T19:37:39.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.040+0000 7f6b145ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c109fa0 con 0x7f6b0c105b10 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59552/0 (socket says 172.21.15.5:59552) 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 -- 172.21.15.5:0/2011786207 learned_addr learned my addr 172.21.15.5:0/2011786207 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b1239b700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1004b0 0x7f6b0c198eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 -- 172.21.15.5:0/2011786207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1004b0 msgr2=0x7f6b0c198eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1004b0 0x7f6b0c198eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 -- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c0ffb00 msgr2=0x7f6b0c198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.041+0000 7f6b12b9c700 1 -- 172.21.15.5:0/2011786207 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc0095f0 con 0x7f6b0c105b10 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b12b9c700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ef30 tx=0x7f6b0400c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b037fe700 1 -- 172.21.15.5:0/2011786207 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04018070 con 0x7f6b0c105b10 2023-07-21T19:37:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b037fe700 1 -- 172.21.15.5:0/2011786207 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6b0400f040 con 0x7f6b0c105b10 2023-07-21T19:37:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c10a220 con 0x7f6b0c105b10 2023-07-21T19:37:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b037fe700 1 -- 172.21.15.5:0/2011786207 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400cd20 con 0x7f6b0c105b10 2023-07-21T19:37:39.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.042+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c19c930 con 0x7f6b0c105b10 2023-07-21T19:37:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.049+0000 7f6b037fe700 1 -- 172.21.15.5:0/2011786207 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f6b04003a00 con 0x7f6b0c105b10 2023-07-21T19:37:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.050+0000 7f6b27fd3b80 1 Processor -- start 2023-07-21T19:37:39.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.050+0000 7f6b27fd3b80 1 -- start start 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee69d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee64040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ed69c50 con 0x55b57ecc06b0 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57edbc9e0 con 0x55b57ee6a2d0 2023-07-21T19:37:39.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57edb5e70 con 0x55b57ee64580 2023-07-21T19:37:39.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee69d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b037fe700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 0x7f6af80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b1239b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee64040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59564/0 (socket says 172.21.15.5:59564) 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.051+0000 7f6b11b9a700 1 -- 172.21.15.5:0/2090973290 learned_addr learned my addr 172.21.15.5:0/2090973290 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b037fe700 1 -- 172.21.15.5:0/2011786207 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6b040cc8d0 con 0x7f6b0c105b10 2023-07-21T19:37:39.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b12b9c700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 0x7f6af80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 -- 172.21.15.5:0/2090973290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 msgr2=0x55b57ee64040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee64040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 -- 172.21.15.5:0/2090973290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 msgr2=0x55b57ee69d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee69d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 -- 172.21.15.5:0/2090973290 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57ebe3d80 con 0x55b57ee64580 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.052+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005910 tx=0x7f6afc005fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=235355555ac1d11a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.053+0000 7f6b00bfe700 1 -- 172.21.15.5:0/2090973290 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01d070 con 0x55b57ee64580 2023-07-21T19:37:39.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.053+0000 7f6b00bfe700 1 -- 172.21.15.5:0/2090973290 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc00f460 con 0x55b57ee64580 2023-07-21T19:37:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.053+0000 7f6b00bfe700 1 -- 172.21.15.5:0/2090973290 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc017650 con 0x55b57ee64580 2023-07-21T19:37:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/2090973290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 msgr2=0x55b57ee649d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b12b9c700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 0x7f6af80a37f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b040090a0 tx=0x7f6b040092e0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f6afc005910 tx=0x7f6afc005fb0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/2090973290 shutdown_connections 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee649d0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee69d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/2090973290 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee64040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/2090973290 >> 172.21.15.5:0/2090973290 conn(0x55b57ee59be0 msgr2=0x55b57ee5ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/2090973290 shutdown_connections 2023-07-21T19:37:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.054+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/2090973290 wait complete. 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.055+0000 7f6b27fd3b80 1 -- start start 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.055+0000 7f6b27fd3b80 1 Processor -- start 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee705b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee6d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee6dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ed55a60 con 0x55b57ecc06b0 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ed55760 con 0x55b57ee6a2d0 2023-07-21T19:37:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ec398c0 con 0x55b57ee64580 2023-07-21T19:37:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.056+0000 7f6b27fd3b80 10 client.0 _ll_register_callbacks cb 0x55b57ed53700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.057+0000 7f6b1239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee6d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.057+0000 7f6b1239b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee6d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59576/0 (socket says 172.21.15.5:59576) 2023-07-21T19:37:39.044 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.057+0000 7f6b1239b700 1 -- 172.21.15.5:0/386547295 learned_addr learned my addr 172.21.15.5:0/386547295 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:39.045 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.057+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee705b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.057+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee6dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 -- 172.21.15.5:0/386547295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 msgr2=0x55b57ee6d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee6d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 -- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 msgr2=0x55b57ee6dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.046 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee6dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 -- 172.21.15.5:0/386547295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c10d240 con 0x55b57ecc06b0 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee705b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0098e0 tx=0x7f6afc005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81b3a16c69897f8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.058+0000 7f6af5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01d070 con 0x55b57ecc06b0 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6afc005230 con 0x55b57ecc06b0 2023-07-21T19:37:39.047 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b57ed0f490 con 0x55b57ecc06b0 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6afc0053f0 con 0x55b57ecc06b0 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6b27fd3b80 10 client.20538 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.059+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55b57ebccc00 con 0x55b57ecc06b0 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc03b080 con 0x55b57ecc06b0 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6afc004a90 con 0x55b57ecc06b0 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 10 client.20538 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 10 client.20538 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:39.048 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 20 client.20538 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee7d720 0x55b57ee7fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 4 client.20538 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b57edaee50 con 0x55b57ee7d720 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.060+0000 7f6b1239b700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee7d720 0x55b57ee7fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.061+0000 7f6b1239b700 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee7d720 0x55b57ee7fb30 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fb145201f9e6187 server_cookie=5ab165e1fe6b7cf4 in_seq=0 out_seq=0 2023-07-21T19:37:39.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.061+0000 7f6af5ffb700 10 client.20538 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6af5ffb700 1 -- 172.21.15.5:0/386547295 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b57edaee50 con 0x55b57ee7d720 2023-07-21T19:37:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6af5ffb700 10 client.20538 handle_command_reply: tid=0 2023-07-21T19:37:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 client.20538 shutdown 2023-07-21T19:37:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 msgr2=0x55b57ee705b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee705b0 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f6afc0098e0 tx=0x7f6afc005ce0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 shutdown_connections 2023-07-21T19:37:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee6a2d0 0x55b57ee6dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee7d720 0x55b57ee7fb30 crc :-1 s=READY pgs=544 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee705b0 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/386547295 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee64580 0x55b57ee6d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 >> 172.21.15.5:0/386547295 conn(0x55b57ee59be0 msgr2=0x55b57ee5e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.062+0000 7f6af5ffb700 0 client.20538 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.063+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 shutdown_connections 2023-07-21T19:37:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.063+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/386547295 wait complete. 2023-07-21T19:37:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.063+0000 7f6b27fd3b80 20 client.20538 trim_cache size 0 max 16384 2023-07-21T19:37:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 Processor -- start 2023-07-21T19:37:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 -- start start 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee5ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee64580 0x55b57ee700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ee6e8d0 con 0x55b57ecc06b0 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ee699f0 con 0x55b57ee64580 2023-07-21T19:37:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.088+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57ee6eeb0 con 0x55b57ee59be0 2023-07-21T19:37:39.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59580/0 (socket says 172.21.15.5:59580) 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 -- 172.21.15.5:0/1725758769 learned_addr learned my addr 172.21.15.5:0/1725758769 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee64580 0x55b57ee700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 -- 172.21.15.5:0/1725758769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee64580 msgr2=0x55b57ee700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee64580 0x55b57ee700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 -- 172.21.15.5:0/1725758769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 msgr2=0x55b57ee5ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee5ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.089+0000 7f6b11b9a700 1 -- 172.21.15.5:0/1725758769 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b57ee84760 con 0x55b57ee59be0 2023-07-21T19:37:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.090+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00fb30 tx=0x7f6afc009290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22be003bc50b2aa2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.090+0000 7f6b013ff700 1 -- 172.21.15.5:0/1725758769 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01d040 con 0x55b57ee59be0 2023-07-21T19:37:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.090+0000 7f6b013ff700 1 -- 172.21.15.5:0/1725758769 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc03e070 con 0x55b57ee59be0 2023-07-21T19:37:39.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.090+0000 7f6b013ff700 1 -- 172.21.15.5:0/1725758769 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc004760 con 0x55b57ee59be0 2023-07-21T19:37:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/1725758769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 msgr2=0x55b57ee691d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00fb30 tx=0x7f6afc009290 comp rx=0 tx=0).stop 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/1725758769 shutdown_connections 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ee64580 0x55b57ee700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee691d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/1725758769 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ecc06b0 0x55b57ee5ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/1725758769 >> 172.21.15.5:0/1725758769 conn(0x55b57ee79770 msgr2=0x55b57ee68ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/1725758769 shutdown_connections 2023-07-21T19:37:39.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.091+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/1725758769 wait complete. 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 -- start start 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 Processor -- start 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ee64580 0x55b57ee736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57eed0710 con 0x55b57ee64580 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57eed0880 con 0x55b57ecc06b0 2023-07-21T19:37:39.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.092+0000 7f6b27fd3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b57eed0a40 con 0x55b57ee59be0 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b27fd3b80 10 client.0 _ll_register_callbacks cb 0x7f6afc009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:41174/0 (socket says 172.21.15.5:41174) 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 -- 172.21.15.5:0/268640980 learned_addr learned my addr 172.21.15.5:0/268640980 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ee64580 0x55b57ee736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b1239b700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 -- 172.21.15.5:0/268640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 msgr2=0x55b57ee73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee73190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 -- 172.21.15.5:0/268640980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ee64580 msgr2=0x55b57ee736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ee64580 0x55b57ee736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.093+0000 7f6b11b9a700 1 -- 172.21.15.5:0/268640980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b57ebccc00 con 0x55b57ecc06b0 2023-07-21T19:37:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.094+0000 7f6b11b9a700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00f7f0 tx=0x7f6afc028400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1dcc5505ac9305f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:39.082 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.094+0000 7f6af5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.094+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6afc01d070 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.094+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f6afc000e40 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.094+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6afc00f460 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.095+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6afc00bda0 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.095+0000 7f6b27fd3b80 10 client.30326 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.095+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55b57ec0a1f0 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.095+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f6afc00f460 con 0x55b57ecc06b0 2023-07-21T19:37:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.095+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f6afc0286c0 con 0x55b57ecc06b0 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 10 client.30326 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 10 client.30326 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 20 client.30326 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee6a2d0 0x55b57eed3d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 4 client.30326 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b57ed70ff0 con 0x55b57ee6a2d0 2023-07-21T19:37:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.096+0000 7f6b1239b700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee6a2d0 0x55b57eed3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.097+0000 7f6af5ffb700 10 client.30326 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.097+0000 7f6b1239b700 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee6a2d0 0x55b57eed3d50 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e17b975b6c4132a server_cookie=2b57bec5edfd26fe in_seq=0 out_seq=0 2023-07-21T19:37:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6af5ffb700 1 -- 172.21.15.5:0/268640980 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b57ed70ff0 con 0x55b57ee6a2d0 2023-07-21T19:37:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6af5ffb700 10 client.30326 handle_command_reply: tid=0 2023-07-21T19:37:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 client.30326 shutdown 2023-07-21T19:37:39.113 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 msgr2=0x55b57ee72c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f6afc00f7f0 tx=0x7f6afc028400 comp rx=0 tx=0).stop 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 shutdown_connections 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b57ee6a2d0 0x55b57eed3d50 crc :-1 s=READY pgs=545 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b57ee64580 0x55b57ee736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b57ecc06b0 0x55b57ee72c50 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 --2- 172.21.15.5:0/268640980 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b57ee59be0 0x55b57ee73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.128+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 >> 172.21.15.5:0/268640980 conn(0x55b57ee79770 msgr2=0x55b57ee69350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.114 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.129+0000 7f6af5ffb700 0 client.30326 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.129+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 shutdown_connections 2023-07-21T19:37:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.129+0000 7f6b27fd3b80 1 -- 172.21.15.5:0/268640980 wait complete. 2023-07-21T19:37:39.115 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.129+0000 7f6b27fd3b80 20 client.30326 trim_cache size 0 max 16384 2023-07-21T19:37:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:37:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:37:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:37:39.116 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 msgr2=0x7f6af80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 0x7f6af80a37f0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6b040090a0 tx=0x7f6b040092e0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 msgr2=0x7f6b0c1094f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400ef30 tx=0x7f6b0400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 shutdown_connections 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f6b0c105b10 0x7f6b0c1094f0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.117 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f6af80a1340 0x7f6af80a37f0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f6b0c0ffb00 0x7f6b0c198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 --2- 172.21.15.5:0/2011786207 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f6b0c1004b0 0x7f6b0c198eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 >> 172.21.15.5:0/2011786207 conn(0x7f6b0c0fb600 msgr2=0x7f6b0c10ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 shutdown_connections 2023-07-21T19:37:39.118 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:39.130+0000 7f6b145ff700 1 -- 172.21.15.5:0/2011786207 wait complete. 2023-07-21T19:37:39.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:38 smithi005 ceph-mon[126460]: pgmap v953: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 889 op/s 2023-07-21T19:37:39.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:38 smithi156 ceph-mon[128316]: pgmap v953: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 889 op/s 2023-07-21T19:37:39.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:38 smithi101 ceph-mon[128136]: pgmap v953: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 8.7 MiB/s wr, 889 op/s 2023-07-21T19:37:40.816 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:40 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:37:40.816 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:40 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:37:40.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:40 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:37:40.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:40 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:37:40.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:40 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:37:40.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:40 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:37:40.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:40 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:37:40.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:40 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:37:40.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:40 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:37:41.817 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:41 smithi101 ceph-mon[128136]: pgmap v954: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 554 op/s 2023-07-21T19:37:41.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:41 smithi005 ceph-mon[126460]: pgmap v954: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 554 op/s 2023-07-21T19:37:41.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:41 smithi156 ceph-mon[128316]: pgmap v954: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 554 op/s 2023-07-21T19:37:41.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 670.0 s, 201.0 tps, lat 59.023 ms stddev 17.092 2023-07-21T19:37:44.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:43 smithi101 ceph-mon[128136]: pgmap v955: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 807 op/s 2023-07-21T19:37:44.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:43 smithi005 ceph-mon[126460]: pgmap v955: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 807 op/s 2023-07-21T19:37:44.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:43 smithi156 ceph-mon[128316]: pgmap v955: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.9 MiB/s wr, 807 op/s 2023-07-21T19:37:46.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:45 smithi101 ceph-mon[128136]: pgmap v956: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 713 op/s 2023-07-21T19:37:46.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:45 smithi005 ceph-mon[126460]: pgmap v956: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 713 op/s 2023-07-21T19:37:46.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:45 smithi156 ceph-mon[128316]: pgmap v956: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.6 MiB/s wr, 713 op/s 2023-07-21T19:37:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 675.0 s, 207.6 tps, lat 58.546 ms stddev 28.954 2023-07-21T19:37:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:47 smithi101 ceph-mon[128136]: pgmap v957: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 515 op/s 2023-07-21T19:37:48.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:47 smithi005 ceph-mon[126460]: pgmap v957: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 515 op/s 2023-07-21T19:37:48.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:47 smithi156 ceph-mon[128316]: pgmap v957: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 4.8 MiB/s wr, 515 op/s 2023-07-21T19:37:49.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:48 smithi005 ceph-mon[126460]: pgmap v958: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 807 op/s 2023-07-21T19:37:49.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:48 smithi156 ceph-mon[128316]: pgmap v958: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 807 op/s 2023-07-21T19:37:49.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:48 smithi101 ceph-mon[128136]: pgmap v958: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.6 MiB/s wr, 807 op/s 2023-07-21T19:37:51.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 680.0 s, 172.6 tps, lat 64.980 ms stddev 22.896 2023-07-21T19:37:52.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:51 smithi101 ceph-mon[128136]: pgmap v959: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T19:37:52.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:51 smithi005 ceph-mon[126460]: pgmap v959: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T19:37:52.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:51 smithi156 ceph-mon[128316]: pgmap v959: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 560 op/s 2023-07-21T19:37:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/3726333440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806c910 msgr2=0x7ff5b806cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/3726333440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806c910 0x7ff5b806cd80 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac009990 tx=0x7ff5ac009ca0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/3726333440 shutdown_connections 2023-07-21T19:37:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/3726333440 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b806d3a0 0x7ff5b811b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/3726333440 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b810e2a0 0x7ff5b810e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/3726333440 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806c910 0x7ff5b806cd80 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/3726333440 >> 172.21.15.5:0/3726333440 conn(0x7ff5b806bd70 msgr2=0x7ff5b8070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/3726333440 shutdown_connections 2023-07-21T19:37:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.355+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/3726333440 wait complete. 2023-07-21T19:37:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.356+0000 7ff5bd1ff700 1 Processor -- start 2023-07-21T19:37:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 -- start start 2023-07-21T19:37:52.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b806c910 0x7ff5b81aa930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b810e2a0 0x7ff5b81a4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b81a4e00 con 0x7ff5b810e2a0 2023-07-21T19:37:52.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b81a4f70 con 0x7ff5b806d3a0 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.357+0000 7ff5bd1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5b81a5130 con 0x7ff5b806c910 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.358+0000 7ff5b659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.358+0000 7ff5b659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42362/0 (socket says 172.21.15.5:42362) 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.358+0000 7ff5b659c700 1 -- 172.21.15.5:0/532767633 learned_addr learned my addr 172.21.15.5:0/532767633 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.358+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b806c910 0x7ff5b81aa930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.358+0000 7ff5b759e700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b810e2a0 0x7ff5b81a4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 -- 172.21.15.5:0/532767633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b806c910 msgr2=0x7ff5b81aa930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b806c910 0x7ff5b81aa930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 -- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b810e2a0 msgr2=0x7ff5b81a4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.345 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b810e2a0 0x7ff5b81a4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 -- 172.21.15.5:0/532767633 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5ac0095f0 con 0x7ff5b806d3a0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.359+0000 7ff5b659c700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b650 tx=0x7ff5ac004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.360+0000 7ff59ffff700 1 -- 172.21.15.5:0/532767633 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac01d070 con 0x7ff5b806d3a0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.360+0000 7ff59ffff700 1 -- 172.21.15.5:0/532767633 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5ac005030 con 0x7ff5b806d3a0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.360+0000 7ff59ffff700 1 -- 172.21.15.5:0/532767633 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5ac004a40 con 0x7ff5b806d3a0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.360+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5b81a53b0 con 0x7ff5b806d3a0 2023-07-21T19:37:52.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.360+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5b81a55d0 con 0x7ff5b806d3a0 2023-07-21T19:37:52.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.365+0000 7ff59ffff700 1 -- 172.21.15.5:0/532767633 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff5ac004c60 con 0x7ff5b806d3a0 2023-07-21T19:37:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.366+0000 7ff5d0aecb80 1 Processor -- start 2023-07-21T19:37:52.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.366+0000 7ff5d0aecb80 1 -- start start 2023-07-21T19:37:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac77fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac77a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac67fc50 con 0x5638ac5d66b0 2023-07-21T19:37:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac6d29e0 con 0x5638ac77a580 2023-07-21T19:37:52.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.367+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac6cbe70 con 0x5638ac7802d0 2023-07-21T19:37:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54574/0 (socket says 172.21.15.5:54574) 2023-07-21T19:37:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1305671054 learned_addr learned my addr 172.21.15.5:0/1305671054 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:52.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b659c700 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac77a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b759e700 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac77fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1305671054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 msgr2=0x5638ac77a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac77a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1305671054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 msgr2=0x5638ac77fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac77fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff59ffff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 0x7ff5a00a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1305671054 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ac4f9d80 con 0x5638ac7802d0 2023-07-21T19:37:52.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.368+0000 7ff59ffff700 1 -- 172.21.15.5:0/532767633 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff5ac0cde90 con 0x7ff5b806d3a0 2023-07-21T19:37:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.369+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009cf0 tx=0x7ff5a80103d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=912fd1b77025e603 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.369+0000 7ff59d7fa700 1 -- 172.21.15.5:0/1305671054 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8010d00 con 0x5638ac7802d0 2023-07-21T19:37:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.369+0000 7ff59d7fa700 1 -- 172.21.15.5:0/1305671054 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5a8015070 con 0x5638ac7802d0 2023-07-21T19:37:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.369+0000 7ff59d7fa700 1 -- 172.21.15.5:0/1305671054 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a801a5e0 con 0x5638ac7802d0 2023-07-21T19:37:52.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.369+0000 7ff5b759e700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 0x7ff5a00a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5b759e700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 0x7ff5a00a37b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e10 tx=0x7ff5b000b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1305671054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 msgr2=0x5638ac77a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009cf0 tx=0x7ff5a80103d0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1305671054 shutdown_connections 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac77a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.358 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac77fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1305671054 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac77a040 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1305671054 >> 172.21.15.5:0/1305671054 conn(0x5638ac76fbe0 msgr2=0x5638ac774f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1305671054 shutdown_connections 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.370+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1305671054 wait complete. 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.371+0000 7ff5d0aecb80 1 -- start start 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.371+0000 7ff5d0aecb80 1 Processor -- start 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac7838d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.359 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac783e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac66ba60 con 0x5638ac5d66b0 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac66b760 con 0x5638ac77a580 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac54f8c0 con 0x5638ac7802d0 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5d0aecb80 10 client.0 _ll_register_callbacks cb 0x5638ac669700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5b759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5b759e700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54584/0 (socket says 172.21.15.5:54584) 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5b759e700 1 -- 172.21.15.5:0/4028352871 learned_addr learned my addr 172.21.15.5:0/4028352871 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5b659c700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac783e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.372+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac7838d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 -- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 msgr2=0x5638ac783e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac783e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 -- 172.21.15.5:0/4028352871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 msgr2=0x5638ac7838d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac7838d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 -- 172.21.15.5:0/4028352871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8009920 con 0x5638ac7802d0 2023-07-21T19:37:52.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b659c700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac783e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff5b759e700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0016990 tx=0x7ff5b0004b40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=36aeed35d1f3ef00 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff59aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.373+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5b000e040 con 0x5638ac7802d0 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.374+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff5b0015820 con 0x5638ac7802d0 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.374+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5638ac625490 con 0x5638ac7802d0 2023-07-21T19:37:52.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.374+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5b0015a20 con 0x5638ac7802d0 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.375+0000 7ff5d0aecb80 10 client.30262 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.375+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5638ac4e2c00 con 0x5638ac7802d0 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5638ac625490 con 0x5638ac7802d0 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5b0023c50 con 0x5638ac7802d0 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 10 client.30262 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 10 client.30262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:52.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 20 client.30262 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac793c70 0x5638ac796080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 4 client.30262 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:37:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.376+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5638ac6c4e50 con 0x5638ac793c70 2023-07-21T19:37:52.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.377+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac793c70 0x5638ac796080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.379+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac793c70 0x5638ac796080 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cd39f393fa7e959 server_cookie=928f6bb9c197b3a5 in_seq=0 out_seq=0 2023-07-21T19:37:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.379+0000 7ff59aff5700 10 client.30262 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.380+0000 7ff59aff5700 1 -- 172.21.15.5:0/4028352871 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5638ac6c4e50 con 0x5638ac793c70 2023-07-21T19:37:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.380+0000 7ff59aff5700 10 client.30262 handle_command_reply: tid=0 2023-07-21T19:37:52.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.380+0000 7ff5d0aecb80 1 client.30262 shutdown 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.380+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 msgr2=0x5638ac784350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.380+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0016990 tx=0x7ff5b0004b40 comp rx=0 tx=0).stop 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 shutdown_connections 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac7802d0 0x5638ac784350 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac793c70 0x5638ac796080 crc :-1 s=READY pgs=546 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac7838d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/4028352871 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77a580 0x5638ac783e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff59aff5700 0 client.30262 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 >> 172.21.15.5:0/4028352871 conn(0x5638ac76fbe0 msgr2=0x5638ac7744e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 shutdown_connections 2023-07-21T19:37:52.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.381+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/4028352871 wait complete. 2023-07-21T19:37:52.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.382+0000 7ff5d0aecb80 20 client.30262 trim_cache size 0 max 16384 2023-07-21T19:37:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.406+0000 7ff5d0aecb80 1 Processor -- start 2023-07-21T19:37:52.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.406+0000 7ff5d0aecb80 1 -- start start 2023-07-21T19:37:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.406+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.406+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac76fbe0 0x5638ac784180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.406+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac774c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac775fb0 con 0x5638ac5d66b0 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac7715f0 con 0x5638ac77e2a0 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac776720 con 0x5638ac76fbe0 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:42390/0 (socket says 172.21.15.5:42390) 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1448863913 learned_addr learned my addr 172.21.15.5:0/1448863913 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b759e700 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac774c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.393 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b659c700 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac76fbe0 0x5638ac784180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1448863913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac76fbe0 msgr2=0x5638ac784180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac76fbe0 0x5638ac784180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1448863913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 msgr2=0x5638ac774c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac774c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.407+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/1448863913 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5638ac79ac70 con 0x5638ac77e2a0 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.408+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009cf0 tx=0x7ff5a8009c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1026303eb5ba3fc8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.408+0000 7ff59dffb700 1 -- 172.21.15.5:0/1448863913 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8020050 con 0x5638ac77e2a0 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.408+0000 7ff59dffb700 1 -- 172.21.15.5:0/1448863913 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5a8021040 con 0x5638ac77e2a0 2023-07-21T19:37:52.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.408+0000 7ff59dffb700 1 -- 172.21.15.5:0/1448863913 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8011960 con 0x5638ac77e2a0 2023-07-21T19:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.409+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1448863913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 msgr2=0x5638ac77a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8009cf0 tx=0x7ff5a8009c10 comp rx=0 tx=0).stop 2023-07-21T19:37:52.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1448863913 shutdown_connections 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac5d66b0 0x5638ac774c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac77a040 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/1448863913 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac76fbe0 0x5638ac784180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1448863913 >> 172.21.15.5:0/1448863913 conn(0x5638ac77ded0 msgr2=0x5638ac77f140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1448863913 shutdown_connections 2023-07-21T19:37:52.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.410+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/1448863913 wait complete. 2023-07-21T19:37:52.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.411+0000 7ff5d0aecb80 1 -- start start 2023-07-21T19:37:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.411+0000 7ff5d0aecb80 1 Processor -- start 2023-07-21T19:37:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 0x5638ac789370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac78a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac78a9b0 con 0x5638ac76fbe0 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac78ab20 con 0x5638ac77e2a0 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5638ac78ace0 con 0x5638ac5d66b0 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.412+0000 7ff5d0aecb80 10 client.0 _ll_register_callbacks cb 0x7ff5b0007f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:37:52.400 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54604/0 (socket says 172.21.15.5:54604) 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/3146153228 learned_addr learned my addr 172.21.15.5:0/3146153228 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b759e700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac78a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b659c700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 0x5638ac789370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 msgr2=0x5638ac78a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac78a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/3146153228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 msgr2=0x5638ac789370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:37:52.401 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 0x5638ac789370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.413+0000 7ff5b6d9d700 1 -- 172.21.15.5:0/3146153228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5638ac4f9d80 con 0x5638ac5d66b0 2023-07-21T19:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff5b659c700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 0x5638ac789370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff5b6d9d700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8010b30 tx=0x7ff5a80095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2bee8f2946380d36 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:37:52.402 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff59aff5700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:37:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5a8020040 con 0x5638ac5d66b0 2023-07-21T19:37:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(137..137 src has 1..137) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff5a8017070 con 0x5638ac5d66b0 2023-07-21T19:37:52.403 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.414+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff5a8015070 con 0x5638ac5d66b0 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff5a8027bd0 con 0x5638ac5d66b0 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff5d0aecb80 10 client.30268 fetch_fsmap learned FSMap version 72 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5638ac5201f0 con 0x5638ac5d66b0 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff5a8015070 con 0x5638ac5d66b0 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff5a802e030 con 0x5638ac5d66b0 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff5d0aecb80 10 client.30268 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:37:52.404 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.415+0000 7ff5d0aecb80 10 client.30268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.416+0000 7ff5d0aecb80 20 client.30268 populate_metadata read hostname 'smithi005' 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.416+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac77a580 0x5638ac7ea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.416+0000 7ff5d0aecb80 4 client.30268 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.416+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5638ac686ff0 con 0x5638ac77a580 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.416+0000 7ff5b659c700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac77a580 0x5638ac7ea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.417+0000 7ff59aff5700 10 client.30268 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:52.405 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.417+0000 7ff5b659c700 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac77a580 0x5638ac7ea9a0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2bf18cf0cef27f64 server_cookie=db2a055edae89bc2 in_seq=0 out_seq=0 2023-07-21T19:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff59aff5700 1 -- 172.21.15.5:0/3146153228 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5638ac686ff0 con 0x5638ac77a580 2023-07-21T19:37:52.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff59aff5700 10 client.30268 handle_command_reply: tid=0 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 client.30268 shutdown 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 msgr2=0x5638ac788e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7ff5a8010b30 tx=0x7ff5a80095c0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 shutdown_connections 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5638ac77e2a0 0x5638ac78a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5638ac5d66b0 0x5638ac788e30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5638ac77a580 0x5638ac7ea9a0 crc :-1 s=READY pgs=547 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff59aff5700 0 client.30268 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 --2- 172.21.15.5:0/3146153228 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5638ac76fbe0 0x5638ac789370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 >> 172.21.15.5:0/3146153228 conn(0x5638ac77ded0 msgr2=0x5638ac785a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 shutdown_connections 2023-07-21T19:37:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.448+0000 7ff5d0aecb80 1 -- 172.21.15.5:0/3146153228 wait complete. 2023-07-21T19:37:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.449+0000 7ff5d0aecb80 20 client.30268 trim_cache size 0 max 16384 2023-07-21T19:37:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:37:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:37:52.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:37:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:37:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.449+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 msgr2=0x7ff5a00a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.449+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 0x7ff5a00a37b0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7ff5b0009e10 tx=0x7ff5b000b040 comp rx=0 tx=0).stop 2023-07-21T19:37:52.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.449+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 msgr2=0x7ff5b81aae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.449+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7ff5ac00b650 tx=0x7ff5ac004730 comp rx=0 tx=0).stop 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 shutdown_connections 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff5a00a1300 0x7ff5a00a37b0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff5b810e2a0 0x7ff5b81a4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff5b806c910 0x7ff5b81aa930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 --2- 172.21.15.5:0/532767633 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff5b806d3a0 0x7ff5b81aae70 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:37:52.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 >> 172.21.15.5:0/532767633 conn(0x7ff5b806bd70 msgr2=0x7ff5b806f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:37:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 shutdown_connections 2023-07-21T19:37:52.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:37:52.450+0000 7ff5bd1ff700 1 -- 172.21.15.5:0/532767633 wait complete. 2023-07-21T19:37:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:53 smithi101 ceph-mon[128136]: pgmap v960: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-07-21T19:37:54.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:53 smithi005 ceph-mon[126460]: pgmap v960: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-07-21T19:37:54.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:53 smithi156 ceph-mon[128316]: pgmap v960: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 7.9 MiB/s wr, 840 op/s 2023-07-21T19:37:56.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:55 smithi101 ceph-mon[128136]: pgmap v961: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:56.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:55 smithi005 ceph-mon[126460]: pgmap v961: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:56.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:55 smithi156 ceph-mon[128316]: pgmap v961: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 685.0 s, 186.0 tps, lat 2302.721 ms stddev 6658.530 2023-07-21T19:37:58.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:57 smithi101 ceph-mon[128136]: pgmap v962: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:58.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:57 smithi005 ceph-mon[126460]: pgmap v962: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:58.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:57 smithi156 ceph-mon[128316]: pgmap v962: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.4 MiB/s wr, 571 op/s 2023-07-21T19:37:59.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:37:58 smithi005 ceph-mon[126460]: pgmap v963: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 845 op/s 2023-07-21T19:37:59.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:37:58 smithi156 ceph-mon[128316]: pgmap v963: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 845 op/s 2023-07-21T19:37:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:37:58 smithi101 ceph-mon[128136]: pgmap v963: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 8.1 MiB/s wr, 845 op/s 2023-07-21T19:38:01.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 690.0 s, 22.8 tps, lat 3551.073 ms stddev 1238.756 2023-07-21T19:38:02.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:01 smithi101 ceph-mon[128136]: pgmap v964: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 554 op/s 2023-07-21T19:38:02.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:01 smithi005 ceph-mon[126460]: pgmap v964: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 554 op/s 2023-07-21T19:38:02.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:01 smithi156 ceph-mon[128316]: pgmap v964: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 554 op/s 2023-07-21T19:38:02.391 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:38:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1598586127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 msgr2=0x7fb2b410c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.797 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1598586127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b410c610 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40099c0 tx=0x7fb2a4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1598586127 shutdown_connections 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1598586127 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2b4106a70 0x7fb2b4112f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1598586127 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b410c610 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1598586127 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 0x7fb2b4106340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1598586127 >> 172.21.15.5:0/1598586127 conn(0x7fb2b40759e0 msgr2=0x7fb2b4077df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.798 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.812+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1598586127 shutdown_connections 2023-07-21T19:38:02.799 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.813+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1598586127 wait complete. 2023-07-21T19:38:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.814+0000 7fb2ba1ff700 1 Processor -- start 2023-07-21T19:38:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 -- start start 2023-07-21T19:38:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2b4106a70 0x7fb2b40719b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.800 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 0x7fb2b406a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406b0f0 con 0x7fb2b410c240 2023-07-21T19:38:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406b260 con 0x7fb2b4106a70 2023-07-21T19:38:02.801 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.815+0000 7fb2ba1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b406b420 con 0x7fb2b410cb50 2023-07-21T19:38:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.802 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58740/0 (socket says 172.21.15.5:58740) 2023-07-21T19:38:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 -- 172.21.15.5:0/1408106192 learned_addr learned my addr 172.21.15.5:0/1408106192 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 0x7fb2b406a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.803 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 -- 172.21.15.5:0/1408106192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 msgr2=0x7fb2b406a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 0x7fb2b406a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 -- 172.21.15.5:0/1408106192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2b4106a70 msgr2=0x7fb2b40719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.817+0000 7fb2abfff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2b4106a70 0x7fb2b40719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.818+0000 7fb2abfff700 1 -- 172.21.15.5:0/1408106192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a40095f0 con 0x7fb2b410c240 2023-07-21T19:38:02.804 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.818+0000 7fb2abfff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b520 tx=0x7fb29c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.818+0000 7fb2b17fa700 1 -- 172.21.15.5:0/1408106192 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c011830 con 0x7fb2b410c240 2023-07-21T19:38:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.819+0000 7fb2b17fa700 1 -- 172.21.15.5:0/1408106192 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb29c011e20 con 0x7fb2b410c240 2023-07-21T19:38:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.819+0000 7fb2b17fa700 1 -- 172.21.15.5:0/1408106192 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29c010d20 con 0x7fb2b410c240 2023-07-21T19:38:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.819+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b406b700 con 0x7fb2b410c240 2023-07-21T19:38:02.805 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.819+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b406f780 con 0x7fb2b410c240 2023-07-21T19:38:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.827+0000 7fb2b17fa700 1 -- 172.21.15.5:0/1408106192 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb29c011990 con 0x7fb2b410c240 2023-07-21T19:38:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.830+0000 7fb2cdb4fb80 1 Processor -- start 2023-07-21T19:38:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.834+0000 7fb2cdb4fb80 1 -- start start 2023-07-21T19:38:02.820 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c24251eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2414acb0 con 0x560c242442d0 2023-07-21T19:38:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2419da40 con 0x560c2424fad0 2023-07-21T19:38:02.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.835+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c24196ed0 con 0x560c24244be0 2023-07-21T19:38:02.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.836+0000 7fb2b17fa700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 0x7fb2940a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.837+0000 7fb2b17fa700 1 -- 172.21.15.5:0/1408106192 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb29c0cc690 con 0x7fb2b410c240 2023-07-21T19:38:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.838+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.838+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58748/0 (socket says 172.21.15.5:58748) 2023-07-21T19:38:02.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.838+0000 7fb2b3fff700 1 -- 172.21.15.5:0/3935387333 learned_addr learned my addr 172.21.15.5:0/3935387333 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:02.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.838+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.839+0000 7fb2b3fff700 1 -- 172.21.15.5:0/3935387333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 msgr2=0x560c2424f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.839+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.825 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.839+0000 7fb2b3fff700 1 -- 172.21.15.5:0/3935387333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 msgr2=0x560c24251eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.839+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c24251eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.839+0000 7fb2b3fff700 1 -- 172.21.15.5:0/3935387333 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c23fc4de0 con 0x560c242442d0 2023-07-21T19:38:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.840+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c520 tx=0x7fb2ac00c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d677d543535beb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.840+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 0x7fb2940a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.840+0000 7fb2a9ffb700 1 -- 172.21.15.5:0/3935387333 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac01c030 con 0x560c242442d0 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.840+0000 7fb2a9ffb700 1 -- 172.21.15.5:0/3935387333 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb2ac00ce30 con 0x560c242442d0 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.840+0000 7fb2a9ffb700 1 -- 172.21.15.5:0/3935387333 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0163a0 con 0x560c242442d0 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.841+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 0x7fb2940a38b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c3f0 tx=0x7fb2ac01d000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3935387333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 msgr2=0x560c242446a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c520 tx=0x7fb2ac00c550 comp rx=0 tx=0).stop 2023-07-21T19:38:02.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3935387333 shutdown_connections 2023-07-21T19:38:02.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c24251eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c242446a0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3935387333 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.842+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3935387333 >> 172.21.15.5:0/3935387333 conn(0x560c2423ada0 msgr2=0x560c242406a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.843+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3935387333 shutdown_connections 2023-07-21T19:38:02.829 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.843+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3935387333 wait complete. 2023-07-21T19:38:02.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.843+0000 7fb2cdb4fb80 1 -- start start 2023-07-21T19:38:02.830 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.844+0000 7fb2cdb4fb80 1 Processor -- start 2023-07-21T19:38:02.831 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c2424c880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c2424d300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c24136ac0 con 0x560c242442d0 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c241367c0 con 0x560c2424fad0 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2401a920 con 0x560c24244be0 2023-07-21T19:38:02.832 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.845+0000 7fb2cdb4fb80 10 client.0 _ll_register_callbacks cb 0x560c24134760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.847+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.847+0000 7fb2abfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41290/0 (socket says 172.21.15.5:41290) 2023-07-21T19:38:02.833 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.847+0000 7fb2abfff700 1 -- 172.21.15.5:0/1484547353 learned_addr learned my addr 172.21.15.5:0/1484547353 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.847+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c2424c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.848+0000 7fb2b37fe700 1 -- 172.21.15.5:0/1484547353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 msgr2=0x560c2424cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.848+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.835 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.848+0000 7fb2b37fe700 1 -- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 msgr2=0x560c2424d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.848+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c2424d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.848+0000 7fb2b37fe700 1 -- 172.21.15.5:0/1484547353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac009d30 con 0x560c242442d0 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.849+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c2424c880 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40098e0 tx=0x7fb2a4005790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57469ef15dc0e66e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.849+0000 7fb2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.849+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a400bba0 con 0x560c242442d0 2023-07-21T19:38:02.836 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.849+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb2a400bd00 con 0x560c242442d0 2023-07-21T19:38:02.837 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.850+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560c240f04f0 con 0x560c242442d0 2023-07-21T19:38:02.840 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.855+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2a400bec0 con 0x560c242442d0 2023-07-21T19:38:02.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.855+0000 7fb2cdb4fb80 10 client.20568 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:02.841 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.855+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x560c23fadc60 con 0x560c242442d0 2023-07-21T19:38:02.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.859+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb2a402f080 con 0x560c242442d0 2023-07-21T19:38:02.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.859+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2a4004800 con 0x560c242442d0 2023-07-21T19:38:02.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 10 client.20568 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:02.846 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 10 client.20568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:02.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 20 client.20568 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:02.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2425f520 0x560c24261930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 4 client.20568 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:02.847 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.861+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560c2418feb0 con 0x560c2425f520 2023-07-21T19:38:02.852 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.865+0000 7fb2abfff700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2425f520 0x560c24261930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.857 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.872+0000 7fb2abfff700 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2425f520 0x560c24261930 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a670b54dc97ac4 server_cookie=2218abef7b1de801 in_seq=0 out_seq=0 2023-07-21T19:38:02.858 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.873+0000 7fb2a2ffd700 10 client.20568 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:02.863 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.878+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/1484547353 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x560c2418feb0 con 0x560c2425f520 2023-07-21T19:38:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.878+0000 7fb2a2ffd700 10 client.20568 handle_command_reply: tid=0 2023-07-21T19:38:02.864 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.879+0000 7fb2cdb4fb80 1 client.20568 shutdown 2023-07-21T19:38:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.882+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 msgr2=0x560c2424c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.882+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c2424c880 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40098e0 tx=0x7fb2a4005790 comp rx=0 tx=0).stop 2023-07-21T19:38:02.880 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 shutdown_connections 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c2424fad0 0x560c2424d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2425f520 0x560c24261930 crc :-1 s=READY pgs=548 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c242442d0 0x560c2424c880 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/1484547353 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c2424cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.885+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 >> 172.21.15.5:0/1484547353 conn(0x560c2423ada0 msgr2=0x560c24240230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.881 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.888+0000 7fb2a2ffd700 0 client.20568 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.888+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 shutdown_connections 2023-07-21T19:38:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.888+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/1484547353 wait complete. 2023-07-21T19:38:02.882 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.889+0000 7fb2cdb4fb80 20 client.20568 trim_cache size 0 max 16384 2023-07-21T19:38:02.897 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:02 smithi005 ceph-mon[126460]: osdmap e138: 12 total, 12 up, 12 in 2023-07-21T19:38:02.913 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.925+0000 7fb2cdb4fb80 1 Processor -- start 2023-07-21T19:38:02.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 -- start start 2023-07-21T19:38:02.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c242442d0 0x560c24247b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c24244be0 0x560c242480b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c24241d50 con 0x560c2423ada0 2023-07-21T19:38:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c24245570 con 0x560c24244be0 2023-07-21T19:38:02.920 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.929+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c2423f490 con 0x560c242442d0 2023-07-21T19:38:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58768/0 (socket says 172.21.15.5:58768) 2023-07-21T19:38:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 -- 172.21.15.5:0/3100358222 learned_addr learned my addr 172.21.15.5:0/3100358222 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:02.925 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 -- 172.21.15.5:0/3100358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c242442d0 msgr2=0x560c24247b70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c242442d0 0x560c24247b70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 -- 172.21.15.5:0/3100358222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c24244be0 msgr2=0x560c242480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c24244be0 0x560c242480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.934+0000 7fb2b37fe700 1 -- 172.21.15.5:0/3100358222 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560c23feb250 con 0x560c2423ada0 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.935+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4004b90 tx=0x7fb2a400b720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4158a87b41d8ee2e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.935+0000 7fb2aa7fc700 1 -- 172.21.15.5:0/3100358222 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a402a3d0 con 0x560c2423ada0 2023-07-21T19:38:02.926 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.936+0000 7fb2aa7fc700 1 -- 172.21.15.5:0/3100358222 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x560c240f04f0 con 0x560c2423ada0 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.936+0000 7fb2aa7fc700 1 -- 172.21.15.5:0/3100358222 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4004620 con 0x560c2423ada0 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3100358222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 msgr2=0x560c242564f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4004b90 tx=0x7fb2a400b720 comp rx=0 tx=0).stop 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3100358222 shutdown_connections 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c24244be0 0x560c242480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242564f0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/3100358222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c242442d0 0x560c24247b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.937+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3100358222 >> 172.21.15.5:0/3100358222 conn(0x560c2423c600 msgr2=0x560c2424d040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.940+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3100358222 shutdown_connections 2023-07-21T19:38:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.940+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/3100358222 wait complete. 2023-07-21T19:38:02.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.940+0000 7fb2cdb4fb80 1 -- start start 2023-07-21T19:38:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.941+0000 7fb2cdb4fb80 1 Processor -- start 2023-07-21T19:38:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242ac770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c242442d0 0x560c242accb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c242a7a80 con 0x560c2423ada0 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c242a7bf0 con 0x560c242442d0 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560c242a7db0 con 0x560c24244be0 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.942+0000 7fb2cdb4fb80 10 client.0 _ll_register_callbacks cb 0x7fb2a4009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.943+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.943+0000 7fb2b3fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41304/0 (socket says 172.21.15.5:41304) 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.943+0000 7fb2b3fff700 1 -- 172.21.15.5:0/2501056525 learned_addr learned my addr 172.21.15.5:0/2501056525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:02.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.943+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242ac770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.944+0000 7fb2b3fff700 1 -- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c242442d0 msgr2=0x560c242accb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.944+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c242442d0 0x560c242accb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.944+0000 7fb2b3fff700 1 -- 172.21.15.5:0/2501056525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 msgr2=0x560c242ac770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:02.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.944+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242ac770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.944+0000 7fb2b3fff700 1 -- 172.21.15.5:0/2501056525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560c23fadc60 con 0x560c24244be0 2023-07-21T19:38:02.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.946+0000 7fb2b3fff700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000f80 tx=0x7fb2ac00a610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e0ef5a058048261 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:02.932 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.946+0000 7fb2a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.949+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac009a80 con 0x560c24244be0 2023-07-21T19:38:02.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.949+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fb2ac01d070 con 0x560c24244be0 2023-07-21T19:38:02.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.953+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb2ac00ce30 con 0x560c24244be0 2023-07-21T19:38:02.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.956+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb2ac00a830 con 0x560c24244be0 2023-07-21T19:38:02.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.957+0000 7fb2cdb4fb80 10 client.30280 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:02.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.957+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x560c23fc32b0 con 0x560c24244be0 2023-07-21T19:38:02.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.958+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb2ac00ce30 con 0x560c24244be0 2023-07-21T19:38:02.943 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.958+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb2ac004090 con 0x560c24244be0 2023-07-21T19:38:02.944 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 10 client.30280 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 10 client.30280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 20 client.30280 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:02.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2424fad0 0x560c242c75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 4 client.30280 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:38:02.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.959+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x560c24152050 con 0x560c2424fad0 2023-07-21T19:38:02.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.962+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2424fad0 0x560c242c75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.966+0000 7fb2b37fe700 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2424fad0 0x560c242c75a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9faf856afa31a1c server_cookie=90a31547c30a5eb8 in_seq=0 out_seq=0 2023-07-21T19:38:02.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.966+0000 7fb2a2ffd700 10 client.30280 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.968+0000 7fb2a2ffd700 1 -- 172.21.15.5:0/2501056525 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x560c24152050 con 0x560c2424fad0 2023-07-21T19:38:02.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.968+0000 7fb2a2ffd700 10 client.30280 handle_command_reply: tid=0 2023-07-21T19:38:02.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.969+0000 7fb2cdb4fb80 1 client.30280 shutdown 2023-07-21T19:38:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.969+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 msgr2=0x560c242a7370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.956 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.969+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac000f80 tx=0x7fb2ac00a610 comp rx=0 tx=0).stop 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 shutdown_connections 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560c24244be0 0x560c242a7370 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560c2424fad0 0x560c242c75a0 crc :-1 s=READY pgs=549 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560c2423ada0 0x560c242ac770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 --2- 172.21.15.5:0/2501056525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560c242442d0 0x560c242accb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.957 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 >> 172.21.15.5:0/2501056525 conn(0x560c2423c600 msgr2=0x560c24247b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.970+0000 7fb2a2ffd700 0 client.30280 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:02.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.971+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 shutdown_connections 2023-07-21T19:38:02.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.971+0000 7fb2cdb4fb80 1 -- 172.21.15.5:0/2501056525 wait complete. 2023-07-21T19:38:02.958 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.971+0000 7fb2cdb4fb80 20 client.30280 trim_cache size 0 max 16384 2023-07-21T19:38:02.958 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:38:02.959 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:38:02.959 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:38:02.959 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:38:02.959 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:38:02.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.973+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 msgr2=0x7fb2940a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.973+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 0x7fb2940a38b0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00c3f0 tx=0x7fb2ac01d000 comp rx=0 tx=0).stop 2023-07-21T19:38:02.960 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.973+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 msgr2=0x7fb2b4071f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:02.961 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.973+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7fb29c00b520 tx=0x7fb29c00b8e0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 shutdown_connections 2023-07-21T19:38:02.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb2b410cb50 0x7fb2b406a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb2b4106a70 0x7fb2b40719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb2940a1400 0x7fb2940a38b0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 --2- 172.21.15.5:0/1408106192 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb2b410c240 0x7fb2b4071f30 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:02.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.976+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 >> 172.21.15.5:0/1408106192 conn(0x7fb2b40759e0 msgr2=0x7fb2b4109f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:02.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.982+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 shutdown_connections 2023-07-21T19:38:02.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:02.982+0000 7fb2ba1ff700 1 -- 172.21.15.5:0/1408106192 wait complete. 2023-07-21T19:38:03.002 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:38:03.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:02 smithi101 ceph-mon[128136]: osdmap e138: 12 total, 12 up, 12 in 2023-07-21T19:38:03.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:02 smithi156 ceph-mon[128316]: osdmap e138: 12 total, 12 up, 12 in 2023-07-21T19:38:04.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:03 smithi101 ceph-mon[128136]: pgmap v966: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:04.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:03 smithi005 ceph-mon[126460]: pgmap v966: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:04.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:03 smithi156 ceph-mon[128316]: pgmap v966: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:06.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:05 smithi101 ceph-mon[128136]: pgmap v967: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:06.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:05 smithi005 ceph-mon[126460]: pgmap v967: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:06.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:05 smithi156 ceph-mon[128316]: pgmap v967: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:06.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 -- 172.21.15.5:0/1761291684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 msgr2=0x7f153c10b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 --2- 172.21.15.5:0/1761291684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c10b860 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f153400b0a0 tx=0x7f153400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 -- 172.21.15.5:0/1761291684 shutdown_connections 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 --2- 172.21.15.5:0/1761291684 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c10b860 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 --2- 172.21.15.5:0/1761291684 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c104a30 0x7f153c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 --2- 172.21.15.5:0/1761291684 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c0fea30 0x7f153c0feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.229+0000 7f1540bff700 1 -- 172.21.15.5:0/1761291684 >> 172.21.15.5:0/1761291684 conn(0x7f153c074760 msgr2=0x7f153c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.230+0000 7f1540bff700 1 -- 172.21.15.5:0/1761291684 shutdown_connections 2023-07-21T19:38:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.230+0000 7f1540bff700 1 -- 172.21.15.5:0/1761291684 wait complete. 2023-07-21T19:38:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.231+0000 7f1540bff700 1 Processor -- start 2023-07-21T19:38:06.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.231+0000 7f1540bff700 1 -- start start 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 0x7f153c197dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 0x7f153c19c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19cfb0 con 0x7f153c104a30 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19d120 con 0x7f153c0fea30 2023-07-21T19:38:06.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1540bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153c19d2e0 con 0x7f153c0ff3e0 2023-07-21T19:38:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f153a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 0x7f153c197dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f153a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 0x7f153c197dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38802/0 (socket says 172.21.15.5:38802) 2023-07-21T19:38:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f153a59c700 1 -- 172.21.15.5:0/1219351423 learned_addr learned my addr 172.21.15.5:0/1219351423 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:06.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f153ad9d700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 0x7f153c19c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.232+0000 7f1539d9b700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 -- 172.21.15.5:0/1219351423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 msgr2=0x7f153c198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 -- 172.21.15.5:0/1219351423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 msgr2=0x7f153c197dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 0x7f153c197dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 -- 172.21.15.5:0/1219351423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1534009d00 con 0x7f153c104a30 2023-07-21T19:38:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.233+0000 7f153ad9d700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 0x7f153c19c970 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1534009fd0 tx=0x7f1534007930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.234+0000 7f152b7fe700 1 -- 172.21.15.5:0/1219351423 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153400e040 con 0x7f153c104a30 2023-07-21T19:38:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.234+0000 7f152b7fe700 1 -- 172.21.15.5:0/1219351423 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153400bdc0 con 0x7f153c104a30 2023-07-21T19:38:06.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.234+0000 7f152b7fe700 1 -- 172.21.15.5:0/1219351423 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534005040 con 0x7f153c104a30 2023-07-21T19:38:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.234+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f153c19d560 con 0x7f153c104a30 2023-07-21T19:38:06.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.234+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c19d7b0 con 0x7f153c104a30 2023-07-21T19:38:06.224 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.239+0000 7f152b7fe700 1 -- 172.21.15.5:0/1219351423 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f153401d480 con 0x7f153c104a30 2023-07-21T19:38:06.225 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.240+0000 7f15544c1b80 1 Processor -- start 2023-07-21T19:38:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 -- start start 2023-07-21T19:38:06.226 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f152b7fe700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 0x7f15240a37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e3736b0 0x56451e51cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e517040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 0x56451e5179d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e41cc50 con 0x56451e51d2d0 2023-07-21T19:38:06.227 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e46f9e0 con 0x56451e3736b0 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f15544c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e468e70 con 0x56451e517580 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.241+0000 7f152b7fe700 1 -- 172.21.15.5:0/1219351423 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f15340cdc40 con 0x7f153c104a30 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e3736b0 0x56451e51cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f1539d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e517040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 0x56451e5179d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f1539d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e517040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58806/0 (socket says 172.21.15.5:58806) 2023-07-21T19:38:06.228 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f1539d9b700 1 -- 172.21.15.5:0/3040706525 learned_addr learned my addr 172.21.15.5:0/3040706525 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f1539d9b700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 0x7f15240a37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 -- 172.21.15.5:0/3040706525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e3736b0 msgr2=0x56451e51cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e3736b0 0x56451e51cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 -- 172.21.15.5:0/3040706525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 msgr2=0x56451e517040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e517040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.242+0000 7f153ad9d700 1 -- 172.21.15.5:0/3040706525 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56451e296d80 con 0x56451e517580 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.243+0000 7f153ad9d700 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 0x56451e5179d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f15340cb310 tx=0x7f15340099e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a09b93f209c7e39 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.243+0000 7f1528ff9700 1 -- 172.21.15.5:0/3040706525 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1534003c00 con 0x56451e517580 2023-07-21T19:38:06.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.243+0000 7f1528ff9700 1 -- 172.21.15.5:0/3040706525 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153409ad20 con 0x56451e517580 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.243+0000 7f1528ff9700 1 -- 172.21.15.5:0/3040706525 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153409af80 con 0x56451e517580 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.243+0000 7f1539d9b700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 0x7f15240a37e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56451e513b40 tx=0x7f1530009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 -- 172.21.15.5:0/3040706525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 msgr2=0x56451e5179d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 0x56451e5179d0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f15340cb310 tx=0x7f15340099e0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 -- 172.21.15.5:0/3040706525 shutdown_connections 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e517580 0x56451e5179d0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e3736b0 0x56451e51cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3040706525 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e517040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.244+0000 7f15544c1b80 1 -- 172.21.15.5:0/3040706525 >> 172.21.15.5:0/3040706525 conn(0x56451e50cbe0 msgr2=0x56451e511f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.245+0000 7f15544c1b80 1 -- 172.21.15.5:0/3040706525 shutdown_connections 2023-07-21T19:38:06.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.245+0000 7f15544c1b80 1 -- 172.21.15.5:0/3040706525 wait complete. 2023-07-21T19:38:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.245+0000 7f15544c1b80 1 -- start start 2023-07-21T19:38:06.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.245+0000 7f15544c1b80 1 Processor -- start 2023-07-21T19:38:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e5264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e526a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e520720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e408a60 con 0x56451e51d2d0 2023-07-21T19:38:06.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e408760 con 0x56451e517580 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e2ec8c0 con 0x56451e3736b0 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.246+0000 7f15544c1b80 10 client.0 _ll_register_callbacks cb 0x56451e406700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.247+0000 7f1539d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e526a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.247+0000 7f1539d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e526a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:38816/0 (socket says 172.21.15.5:38816) 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.247+0000 7f1539d9b700 1 -- 172.21.15.5:0/3174672930 learned_addr learned my addr 172.21.15.5:0/3174672930 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.247+0000 7f153a59c700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e5264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.247+0000 7f153ad9d700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e520720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 -- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 msgr2=0x56451e526a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e526a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 -- 172.21.15.5:0/3174672930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 msgr2=0x56451e520720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e520720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 -- 172.21.15.5:0/3174672930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f153c19d7b0 con 0x56451e3736b0 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f153a59c700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e5264d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f152c005950 tx=0x7f152c012b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf00cd81bf6810c1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f1521ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:06.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c017070 con 0x56451e3736b0 2023-07-21T19:38:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.248+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f152c004830 con 0x56451e3736b0 2023-07-21T19:38:06.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56451e3c2490 con 0x56451e3736b0 2023-07-21T19:38:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f152c0049f0 con 0x56451e3736b0 2023-07-21T19:38:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f15544c1b80 10 client.30292 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56451e27fc00 con 0x56451e3736b0 2023-07-21T19:38:06.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56451e3c2490 con 0x56451e3736b0 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.249+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f152c01c600 con 0x56451e3736b0 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 10 client.30292 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 10 client.30292 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 20 client.30292 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e5309c0 0x56451e532dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 4 client.30292 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:06.239 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56451e461e50 con 0x56451e5309c0 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.250+0000 7f1539d9b700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e5309c0 0x56451e532dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.251+0000 7f1521ffb700 10 client.30292 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.251+0000 7f1539d9b700 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e5309c0 0x56451e532dd0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7be263b8287a0427 server_cookie=358c19f02bc64b3f in_seq=0 out_seq=0 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f1521ffb700 1 -- 172.21.15.5:0/3174672930 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56451e461e50 con 0x56451e5309c0 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f1521ffb700 10 client.30292 handle_command_reply: tid=0 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 client.30292 shutdown 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 msgr2=0x56451e5264d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.240 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e5264d0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f152c005950 tx=0x7f152c012b80 comp rx=0 tx=0).stop 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 shutdown_connections 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e5309c0 0x56451e532dd0 crc :-1 s=READY pgs=550 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e51d2d0 0x56451e520720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e5264d0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3174672930 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e526a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 >> 172.21.15.5:0/3174672930 conn(0x56451e50cbe0 msgr2=0x56451e5114e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f1521ffb700 0 client.30292 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:06.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.252+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 shutdown_connections 2023-07-21T19:38:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.253+0000 7f15544c1b80 1 -- 172.21.15.5:0/3174672930 wait complete. 2023-07-21T19:38:06.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.253+0000 7f15544c1b80 20 client.30292 trim_cache size 0 max 16384 2023-07-21T19:38:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 Processor -- start 2023-07-21T19:38:06.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 -- start start 2023-07-21T19:38:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e517580 0x56451e51c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e51d2d0 0x56451e51e010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e3c2490 con 0x56451e517580 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e512dc0 con 0x56451e51d2d0 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.280+0000 7f15544c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e513b70 con 0x56451e3736b0 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:41350/0 (socket says 172.21.15.5:41350) 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 -- 172.21.15.5:0/3918817550 learned_addr learned my addr 172.21.15.5:0/3918817550 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:06.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f1539d9b700 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e517580 0x56451e51c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153ad9d700 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e51d2d0 0x56451e51e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 -- 172.21.15.5:0/3918817550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e51d2d0 msgr2=0x56451e51e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e51d2d0 0x56451e51e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 -- 172.21.15.5:0/3918817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e517580 msgr2=0x56451e51c2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e517580 0x56451e51c2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.281+0000 7f153a59c700 1 -- 172.21.15.5:0/3918817550 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56451e2bd1f0 con 0x56451e3736b0 2023-07-21T19:38:06.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.282+0000 7f153a59c700 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f152c000c00 tx=0x7f152c01cad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dc1f5cdd2d99a2cf server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.282+0000 7f15297fa700 1 -- 172.21.15.5:0/3918817550 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c017040 con 0x56451e3736b0 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.282+0000 7f15297fa700 1 -- 172.21.15.5:0/3918817550 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f152c01cc00 con 0x56451e3736b0 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.282+0000 7f15297fa700 1 -- 172.21.15.5:0/3918817550 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f152c005530 con 0x56451e3736b0 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 -- 172.21.15.5:0/3918817550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 msgr2=0x56451e517040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f152c000c00 tx=0x7f152c01cad0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 -- 172.21.15.5:0/3918817550 shutdown_connections 2023-07-21T19:38:06.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e51d2d0 0x56451e51e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e3736b0 0x56451e517040 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 --2- 172.21.15.5:0/3918817550 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e517580 0x56451e51c2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 -- 172.21.15.5:0/3918817550 >> 172.21.15.5:0/3918817550 conn(0x56451e50cbe0 msgr2=0x56451e51d900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.283+0000 7f15544c1b80 1 -- 172.21.15.5:0/3918817550 shutdown_connections 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.284+0000 7f15544c1b80 1 -- 172.21.15.5:0/3918817550 wait complete. 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.284+0000 7f15544c1b80 1 -- start start 2023-07-21T19:38:06.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 Processor -- start 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e57daa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 0x56451e577900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e577ee0 con 0x56451e3736b0 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e578050 con 0x56451e517580 2023-07-21T19:38:06.273 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56451e578210 con 0x56451e51d2d0 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.285+0000 7f15544c1b80 10 client.0 _ll_register_callbacks cb 0x7f152c009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:58834/0 (socket says 172.21.15.5:58834) 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153a59c700 1 -- 172.21.15.5:0/2805590028 learned_addr learned my addr 172.21.15.5:0/2805590028 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 0x56451e577900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f1539d9b700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e57daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 -- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 msgr2=0x56451e57daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e57daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 -- 172.21.15.5:0/2805590028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 msgr2=0x56451e57d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153ad9d700 1 -- 172.21.15.5:0/2805590028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56451e27fc00 con 0x56451e51d2d0 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.286+0000 7f153a59c700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f153ad9d700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 0x56451e577900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f15340cb340 tx=0x7f153400b3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6bc300b66fbd4f09 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f1521ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:06.275 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f153401d940 con 0x56451e51d2d0 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f1534003c00 con 0x56451e51d2d0 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f153409ad20 con 0x56451e51d2d0 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f153409ed50 con 0x56451e51d2d0 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f15544c1b80 10 client.30298 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.287+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56451e295250 con 0x56451e51d2d0 2023-07-21T19:38:06.276 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f153409ad20 con 0x56451e51d2d0 2023-07-21T19:38:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f153409b010 con 0x56451e51d2d0 2023-07-21T19:38:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f15544c1b80 10 client.30298 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f15544c1b80 10 client.30298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:06.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f15544c1b80 20 client.30298 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.288+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e584ef0 0x56451e5872d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.289+0000 7f15544c1b80 4 client.30298 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.289+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56451e423ff0 con 0x56451e584ef0 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.289+0000 7f153a59c700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e584ef0 0x56451e5872d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.290+0000 7f1521ffb700 10 client.30298 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:06.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.290+0000 7f153a59c700 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e584ef0 0x56451e5872d0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=694ff2f1da6f8293 server_cookie=11718951a416b99a in_seq=0 out_seq=0 2023-07-21T19:38:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f1521ffb700 1 -- 172.21.15.5:0/2805590028 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56451e423ff0 con 0x56451e584ef0 2023-07-21T19:38:06.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f1521ffb700 10 client.30298 handle_command_reply: tid=0 2023-07-21T19:38:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 client.30298 shutdown 2023-07-21T19:38:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 msgr2=0x56451e577900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 0x56451e577900 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f15340cb340 tx=0x7f153400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 shutdown_connections 2023-07-21T19:38:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56451e51d2d0 0x56451e577900 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56451e584ef0 0x56451e5872d0 crc :-1 s=READY pgs=551 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.344+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56451e3736b0 0x56451e57d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f15544c1b80 1 --2- 172.21.15.5:0/2805590028 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56451e517580 0x56451e57daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f1521ffb700 0 client.30298 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:06.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 >> 172.21.15.5:0/2805590028 conn(0x56451e50cbe0 msgr2=0x56451e51c120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 shutdown_connections 2023-07-21T19:38:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f15544c1b80 1 -- 172.21.15.5:0/2805590028 wait complete. 2023-07-21T19:38:06.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.345+0000 7f15544c1b80 20 client.30298 trim_cache size 0 max 16384 2023-07-21T19:38:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:38:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:38:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:38:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:38:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 msgr2=0x7f15240a37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 0x7f15240a37e0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x56451e513b40 tx=0x7f1530009260 comp rx=0 tx=0).stop 2023-07-21T19:38:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 msgr2=0x7f153c19c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 0x7f153c19c970 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f1534009fd0 tx=0x7f1534007930 comp rx=0 tx=0).stop 2023-07-21T19:38:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 shutdown_connections 2023-07-21T19:38:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f15240a1330 0x7f15240a37e0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f153c104a30 0x7f153c19c970 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f153c0fea30 0x7f153c197dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 --2- 172.21.15.5:0/1219351423 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f153c0ff3e0 0x7f153c198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 >> 172.21.15.5:0/1219351423 conn(0x7f153c074760 msgr2=0x7f153c0fcfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 shutdown_connections 2023-07-21T19:38:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:06.346+0000 7f1540bff700 1 -- 172.21.15.5:0/1219351423 wait complete. 2023-07-21T19:38:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 695.0 s, 161.4 tps, lat 766.547 ms stddev 876.904 2023-07-21T19:38:08.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:07 smithi101 ceph-mon[128136]: pgmap v968: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:08.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:07 smithi005 ceph-mon[126460]: pgmap v968: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:08.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:07 smithi156 ceph-mon[128316]: pgmap v968: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 5.0 MiB/s wr, 535 op/s 2023-07-21T19:38:09.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:08 smithi005 ceph-mon[126460]: pgmap v969: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:09.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:08 smithi156 ceph-mon[128316]: pgmap v969: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:09.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:08 smithi101 ceph-mon[128136]: pgmap v969: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:11.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 700.0 s, 174.6 tps, lat 552.098 ms stddev 151.293 2023-07-21T19:38:12.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:11 smithi101 ceph-mon[128136]: pgmap v970: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:12.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:11 smithi005 ceph-mon[126460]: pgmap v970: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:12.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:11 smithi156 ceph-mon[128316]: pgmap v970: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 3.4 MiB/s wr, 368 op/s 2023-07-21T19:38:14.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:13 smithi101 ceph-mon[128136]: pgmap v971: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 595 op/s 2023-07-21T19:38:14.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:13 smithi101 ceph-mon[128136]: osdmap e139: 12 total, 12 up, 12 in 2023-07-21T19:38:14.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:13 smithi005 ceph-mon[126460]: pgmap v971: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 595 op/s 2023-07-21T19:38:14.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:13 smithi005 ceph-mon[126460]: osdmap e139: 12 total, 12 up, 12 in 2023-07-21T19:38:14.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:13 smithi156 ceph-mon[128316]: pgmap v971: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.6 MiB/s wr, 595 op/s 2023-07-21T19:38:14.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:13 smithi156 ceph-mon[128316]: osdmap e139: 12 total, 12 up, 12 in 2023-07-21T19:38:16.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:15 smithi101 ceph-mon[128136]: pgmap v973: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:16.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:15 smithi005 ceph-mon[126460]: pgmap v973: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:16.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:15 smithi156 ceph-mon[128316]: pgmap v973: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:16.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 705.0 s, 172.6 tps, lat 564.799 ms stddev 128.114 2023-07-21T19:38:18.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:17 smithi101 ceph-mon[128136]: pgmap v974: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:18.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:17 smithi005 ceph-mon[126460]: pgmap v974: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:18.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:17 smithi156 ceph-mon[128316]: pgmap v974: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 4.4 MiB/s wr, 445 op/s 2023-07-21T19:38:19.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:18 smithi005 ceph-mon[126460]: pgmap v975: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:19.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:18 smithi156 ceph-mon[128316]: pgmap v975: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:19.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:18 smithi101 ceph-mon[128136]: pgmap v975: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.575+0000 7fb45ffff700 1 -- 172.21.15.5:0/2189936118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580fea30 msgr2=0x7fb4580feea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.575+0000 7fb45ffff700 1 --2- 172.21.15.5:0/2189936118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580fea30 0x7fb4580feea0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7fb448009990 tx=0x7fb448009ca0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 -- 172.21.15.5:0/2189936118 shutdown_connections 2023-07-21T19:38:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 --2- 172.21.15.5:0/2189936118 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4580ff3e0 0x7fb45810b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 --2- 172.21.15.5:0/2189936118 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb458104a30 0x7fb458104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 --2- 172.21.15.5:0/2189936118 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580fea30 0x7fb4580feea0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 -- 172.21.15.5:0/2189936118 >> 172.21.15.5:0/2189936118 conn(0x7fb458074760 msgr2=0x7fb458074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.576+0000 7fb45ffff700 1 -- 172.21.15.5:0/2189936118 shutdown_connections 2023-07-21T19:38:19.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.577+0000 7fb45ffff700 1 -- 172.21.15.5:0/2189936118 wait complete. 2023-07-21T19:38:19.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.578+0000 7fb45ffff700 1 Processor -- start 2023-07-21T19:38:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.579+0000 7fb45ffff700 1 -- start start 2023-07-21T19:38:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.579+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.579+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580ff3e0 0x7fb4581084b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.579+0000 7fb45ffff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4581089f0 0x7fb458108e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.579+0000 7fb45ffff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a8be0 con 0x7fb4580ff3e0 2023-07-21T19:38:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.580+0000 7fb45ffff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a8d50 con 0x7fb4581089f0 2023-07-21T19:38:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.580+0000 7fb45ffff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4581a8f10 con 0x7fb4580fea30 2023-07-21T19:38:19.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.580+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.580+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52874/0 (socket says 172.21.15.5:52874) 2023-07-21T19:38:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.580+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1538181893 learned_addr learned my addr 172.21.15.5:0/1538181893 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45d59a700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580ff3e0 0x7fb4581084b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45e59c700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4581089f0 0x7fb458108e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1538181893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4581089f0 msgr2=0x7fb458108e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4581089f0 0x7fb458108e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580ff3e0 msgr2=0x7fb4581084b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580ff3e0 0x7fb4581084b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.581+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1538181893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4480095f0 con 0x7fb4580fea30 2023-07-21T19:38:19.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e840 tx=0x7fb45400eb50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb44effd700 1 -- 172.21.15.5:0/1538181893 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454004220 con 0x7fb4580fea30 2023-07-21T19:38:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4581a91f0 con 0x7fb4580fea30 2023-07-21T19:38:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581a9700 con 0x7fb4580fea30 2023-07-21T19:38:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb44effd700 1 -- 172.21.15.5:0/1538181893 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb454004380 con 0x7fb4580fea30 2023-07-21T19:38:19.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.582+0000 7fb44effd700 1 -- 172.21.15.5:0/1538181893 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454003b40 con 0x7fb4580fea30 2023-07-21T19:38:19.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.587+0000 7fb44effd700 1 -- 172.21.15.5:0/1538181893 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb454003ca0 con 0x7fb4580fea30 2023-07-21T19:38:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.589+0000 7fb473993b80 1 Processor -- start 2023-07-21T19:38:19.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.589+0000 7fb473993b80 1 -- start start 2023-07-21T19:38:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.589+0000 7fb44effd700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 0x7fb4440a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d782d0 0x564237d72040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d72580 0x564237d729d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237c77c50 con 0x564237d72580 2023-07-21T19:38:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237cca9e0 con 0x564237bce6b0 2023-07-21T19:38:19.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb473993b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237cc3e70 con 0x564237d782d0 2023-07-21T19:38:19.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb44effd700 1 -- 172.21.15.5:0/1538181893 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb454014070 con 0x7fb4580fea30 2023-07-21T19:38:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51884/0 (socket says 172.21.15.5:51884) 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45dd9b700 1 -- 172.21.15.5:0/3132232010 learned_addr learned my addr 172.21.15.5:0/3132232010 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45e59c700 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d72580 0x564237d729d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45d59a700 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d782d0 0x564237d72040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.590+0000 7fb45e59c700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 0x7fb4440a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 -- 172.21.15.5:0/3132232010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d782d0 msgr2=0x564237d72040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d782d0 0x564237d72040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 -- 172.21.15.5:0/3132232010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d72580 msgr2=0x564237d729d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d72580 0x564237d729d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 -- 172.21.15.5:0/3132232010 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564237af1d80 con 0x564237bce6b0 2023-07-21T19:38:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a1d20 tx=0x7fb454020000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=82948affc215ce6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb443fff700 1 -- 172.21.15.5:0/3132232010 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540044f0 con 0x564237bce6b0 2023-07-21T19:38:19.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb45e59c700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 0x7fb4440a3830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb450009900 tx=0x7fb450009650 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb443fff700 1 -- 172.21.15.5:0/3132232010 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb454099e00 con 0x564237bce6b0 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.591+0000 7fb443fff700 1 -- 172.21.15.5:0/3132232010 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb454090d20 con 0x564237bce6b0 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 -- 172.21.15.5:0/3132232010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 msgr2=0x564237d77d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a1d20 tx=0x7fb454020000 comp rx=0 tx=0).stop 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 -- 172.21.15.5:0/3132232010 shutdown_connections 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d72580 0x564237d729d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d77d90 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 --2- 172.21.15.5:0/3132232010 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d782d0 0x564237d72040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.592+0000 7fb473993b80 1 -- 172.21.15.5:0/3132232010 >> 172.21.15.5:0/3132232010 conn(0x564237d67be0 msgr2=0x564237d6cf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.593+0000 7fb473993b80 1 -- 172.21.15.5:0/3132232010 shutdown_connections 2023-07-21T19:38:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.593+0000 7fb473993b80 1 -- 172.21.15.5:0/3132232010 wait complete. 2023-07-21T19:38:19.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.593+0000 7fb473993b80 1 -- start start 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 Processor -- start 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 0x564237d7b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 0x564237d7bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237c63a60 con 0x564237d782d0 2023-07-21T19:38:19.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237c63760 con 0x564237d72580 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237b478c0 con 0x564237bce6b0 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.594+0000 7fb473993b80 10 client.0 _ll_register_callbacks cb 0x564237c61700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52878/0 (socket says 172.21.15.5:52878) 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 -- 172.21.15.5:0/82426159 learned_addr learned my addr 172.21.15.5:0/82426159 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45d59a700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 0x564237d7b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45e59c700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 0x564237d7bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 -- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 msgr2=0x564237d7b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 0x564237d7b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 -- 172.21.15.5:0/82426159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 msgr2=0x564237d7bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 0x564237d7bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45dd9b700 1 -- 172.21.15.5:0/82426159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4581a9700 con 0x564237bce6b0 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45d59a700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 0x564237d7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.595+0000 7fb45e59c700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 0x564237d7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:38:19.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb454096870 tx=0x7fb454005bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=62f12af182b1ac0b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb4417fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540a2250 con 0x564237bce6b0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb45409a270 con 0x564237bce6b0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564237c1d490 con 0x564237bce6b0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4540992f0 con 0x564237bce6b0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb473993b80 10 client.30310 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.596+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564237adac00 con 0x564237bce6b0 2023-07-21T19:38:19.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564237c1d490 con 0x564237bce6b0 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb4540a1dc0 con 0x564237bce6b0 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 10 client.30310 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 10 client.30310 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 20 client.30310 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d8b850 0x564237d8dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 4 client.30310 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.597+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564237cbce50 con 0x564237d8b850 2023-07-21T19:38:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.598+0000 7fb45d59a700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d8b850 0x564237d8dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.598+0000 7fb45d59a700 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d8b850 0x564237d8dc60 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8732c9bc1228a14 server_cookie=484cad5da2e24649 in_seq=0 out_seq=0 2023-07-21T19:38:19.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.598+0000 7fb4417fa700 10 client.30310 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:19.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.599+0000 7fb4417fa700 1 -- 172.21.15.5:0/82426159 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564237cbce50 con 0x564237d8b850 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.599+0000 7fb4417fa700 10 client.30310 handle_command_reply: tid=0 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.599+0000 7fb473993b80 1 client.30310 shutdown 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 msgr2=0x564237d7e4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fb454096870 tx=0x7fb454005bb0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 shutdown_connections 2023-07-21T19:38:19.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d8b850 0x564237d8dc60 crc :-1 s=READY pgs=552 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d782d0 0x564237d7bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237bce6b0 0x564237d7e4f0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb4417fa700 0 client.30310 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 --2- 172.21.15.5:0/82426159 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237d72580 0x564237d7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 >> 172.21.15.5:0/82426159 conn(0x564237d67be0 msgr2=0x564237d6c4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 shutdown_connections 2023-07-21T19:38:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 1 -- 172.21.15.5:0/82426159 wait complete. 2023-07-21T19:38:19.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.600+0000 7fb473993b80 20 client.30310 trim_cache size 0 max 16384 2023-07-21T19:38:19.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.623+0000 7fb473993b80 1 Processor -- start 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.623+0000 7fb473993b80 1 -- start start 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.623+0000 7fb473993b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d82c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb473993b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d7e090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb473993b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d6cc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb473993b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237d69d00 con 0x564237d67be0 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb473993b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237d7cd30 con 0x564237bce6b0 2023-07-21T19:38:19.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb473993b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237d6def0 con 0x564237d72580 2023-07-21T19:38:19.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d82c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d6cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45d59a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d7e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d6cc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52884/0 (socket says 172.21.15.5:52884) 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d82c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36518/0 (socket says 172.21.15.5:36518) 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.624+0000 7fb45e59c700 1 -- 172.21.15.5:0/3195797066 learned_addr learned my addr 172.21.15.5:0/3195797066 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 -- 172.21.15.5:0/3195797066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 msgr2=0x564237d6cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d6cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 -- 172.21.15.5:0/3195797066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 msgr2=0x564237d82c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d82c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 -- 172.21.15.5:0/3195797066 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564237d92850 con 0x564237bce6b0 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb45d59a700 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d7e090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4580fcc80 tx=0x7fb4480052e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d98661f6a4fd2604 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.625+0000 7fb44cff9700 1 -- 172.21.15.5:0/3195797066 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb448027cc0 con 0x564237bce6b0 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.626+0000 7fb44cff9700 1 -- 172.21.15.5:0/3195797066 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb44801e040 con 0x564237bce6b0 2023-07-21T19:38:19.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.626+0000 7fb44cff9700 1 -- 172.21.15.5:0/3195797066 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44800be70 con 0x564237bce6b0 2023-07-21T19:38:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.626+0000 7fb473993b80 1 -- 172.21.15.5:0/3195797066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 msgr2=0x564237d7e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.626+0000 7fb473993b80 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d7e090 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fb4580fcc80 tx=0x7fb4480052e0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 -- 172.21.15.5:0/3195797066 shutdown_connections 2023-07-21T19:38:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d6cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d82c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 --2- 172.21.15.5:0/3195797066 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d7e090 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 -- 172.21.15.5:0/3195797066 >> 172.21.15.5:0/3195797066 conn(0x564237d87770 msgr2=0x564237d79190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 -- 172.21.15.5:0/3195797066 shutdown_connections 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 -- 172.21.15.5:0/3195797066 wait complete. 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.627+0000 7fb473993b80 1 -- start start 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 Processor -- start 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d74320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d74860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237ddf6d0 con 0x564237d67be0 2023-07-21T19:38:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237ddf810 con 0x564237bce6b0 2023-07-21T19:38:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564237ddf9d0 con 0x564237d72580 2023-07-21T19:38:19.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.628+0000 7fb473993b80 10 client.0 _ll_register_callbacks cb 0x7fb448009810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51908/0 (socket says 172.21.15.5:51908) 2023-07-21T19:38:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1779135784 learned_addr learned my addr 172.21.15.5:0/1779135784 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45e59c700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d74860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45d59a700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d74320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1779135784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 msgr2=0x564237d74860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d74860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1779135784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 msgr2=0x564237d74320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d74320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 -- 172.21.15.5:0/1779135784 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb458067c50 con 0x564237bce6b0 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb45dd9b700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a2150 tx=0x7fb45409def0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7b9c2ab525010001 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:19.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.629+0000 7fb4417fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:38:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4540044f0 con 0x564237bce6b0 2023-07-21T19:38:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb4540a2250 con 0x564237bce6b0 2023-07-21T19:38:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb454099e00 con 0x564237bce6b0 2023-07-21T19:38:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4540ca2e0 con 0x564237bce6b0 2023-07-21T19:38:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb473993b80 10 client.30377 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.630+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564237b181f0 con 0x564237bce6b0 2023-07-21T19:38:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb454099e00 con 0x564237bce6b0 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fb454098d60 con 0x564237bce6b0 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 10 client.30377 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 10 client.30377 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 20 client.30377 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d782d0 0x564237de2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 4 client.30377 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:38:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x564237c7eff0 con 0x564237d782d0 2023-07-21T19:38:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.631+0000 7fb45d59a700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d782d0 0x564237de2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.632+0000 7fb45d59a700 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d782d0 0x564237de2c20 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bb5135eeb42fd99 server_cookie=8360e1388d435623 in_seq=0 out_seq=0 2023-07-21T19:38:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.632+0000 7fb4417fa700 10 client.30377 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb4417fa700 1 -- 172.21.15.5:0/1779135784 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x564237c7eff0 con 0x564237d782d0 2023-07-21T19:38:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb4417fa700 10 client.30377 handle_command_reply: tid=0 2023-07-21T19:38:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb473993b80 1 client.30377 shutdown 2023-07-21T19:38:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 msgr2=0x564237d73de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fb4540a2150 tx=0x7fb45409def0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.682+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 shutdown_connections 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564237d72580 0x564237d74860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564237bce6b0 0x564237d73de0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564237d782d0 0x564237de2c20 crc :-1 s=READY pgs=553 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 --2- 172.21.15.5:0/1779135784 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564237d67be0 0x564237d74320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 >> 172.21.15.5:0/1779135784 conn(0x564237d87770 msgr2=0x564237d768f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb4417fa700 0 client.30377 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 shutdown_connections 2023-07-21T19:38:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.683+0000 7fb473993b80 1 -- 172.21.15.5:0/1779135784 wait complete. 2023-07-21T19:38:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.684+0000 7fb473993b80 20 client.30377 trim_cache size 0 max 16384 2023-07-21T19:38:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:38:19.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:38:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:38:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:38:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.684+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 msgr2=0x7fb4440a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.684+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 0x7fb4440a3830 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fb450009900 tx=0x7fb450009650 comp rx=0 tx=0).stop 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.684+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 msgr2=0x7fb45810b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.684+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fb45400e840 tx=0x7fb45400eb50 comp rx=0 tx=0).stop 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 shutdown_connections 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb4581089f0 0x7fb458108e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb4580fea30 0x7fb45810b6b0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb4440a1380 0x7fb4440a3830 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 --2- 172.21.15.5:0/1538181893 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb4580ff3e0 0x7fb4581084b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 >> 172.21.15.5:0/1538181893 conn(0x7fb458074760 msgr2=0x7fb458102720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 shutdown_connections 2023-07-21T19:38:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:19.685+0000 7fb45ffff700 1 -- 172.21.15.5:0/1538181893 wait complete. 2023-07-21T19:38:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 710.0 s, 188.6 tps, lat 551.700 ms stddev 162.153 2023-07-21T19:38:22.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:21 smithi101 ceph-mon[128136]: pgmap v976: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:22.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:21 smithi005 ceph-mon[126460]: pgmap v976: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:22.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:21 smithi156 ceph-mon[128316]: pgmap v976: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.4 MiB/s wr, 582 op/s 2023-07-21T19:38:24.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:23 smithi101 ceph-mon[128136]: pgmap v977: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-07-21T19:38:24.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:23 smithi005 ceph-mon[126460]: pgmap v977: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-07-21T19:38:24.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:23 smithi156 ceph-mon[128316]: pgmap v977: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 6.0 MiB/s wr, 664 op/s 2023-07-21T19:38:26.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:25 smithi101 ceph-mon[128136]: pgmap v978: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 566 op/s 2023-07-21T19:38:26.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:25 smithi005 ceph-mon[126460]: pgmap v978: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 566 op/s 2023-07-21T19:38:26.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:25 smithi156 ceph-mon[128316]: pgmap v978: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.1 MiB/s wr, 566 op/s 2023-07-21T19:38:26.932 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 715.0 s, 196.8 tps, lat 496.902 ms stddev 161.563 2023-07-21T19:38:28.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:27 smithi101 ceph-mon[128136]: pgmap v979: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 554 op/s 2023-07-21T19:38:28.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:27 smithi005 ceph-mon[126460]: pgmap v979: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 554 op/s 2023-07-21T19:38:28.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:27 smithi156 ceph-mon[128316]: pgmap v979: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 5.0 MiB/s wr, 554 op/s 2023-07-21T19:38:29.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:28 smithi005 ceph-mon[126460]: pgmap v980: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 840 op/s 2023-07-21T19:38:29.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:28 smithi156 ceph-mon[128316]: pgmap v980: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 840 op/s 2023-07-21T19:38:29.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:28 smithi101 ceph-mon[128136]: pgmap v980: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 840 op/s 2023-07-21T19:38:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 720.0 s, 242.0 tps, lat 430.474 ms stddev 127.889 2023-07-21T19:38:32.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:31 smithi101 ceph-mon[128136]: pgmap v981: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 591 op/s 2023-07-21T19:38:32.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:31 smithi005 ceph-mon[126460]: pgmap v981: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 591 op/s 2023-07-21T19:38:32.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:31 smithi156 ceph-mon[128316]: pgmap v981: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.3 MiB/s wr, 591 op/s 2023-07-21T19:38:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 -- 172.21.15.5:0/84166838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec06c910 msgr2=0x7f02ec06cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.896 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 --2- 172.21.15.5:0/84166838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec06c910 0x7f02ec06cd80 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f02e001a9a0 tx=0x7f02e001acb0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 -- 172.21.15.5:0/84166838 shutdown_connections 2023-07-21T19:38:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 --2- 172.21.15.5:0/84166838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 0x7f02ec11b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 --2- 172.21.15.5:0/84166838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec10e2a0 0x7f02ec10e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.897 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 --2- 172.21.15.5:0/84166838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec06c910 0x7f02ec06cd80 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.912+0000 7f02f2dff700 1 -- 172.21.15.5:0/84166838 >> 172.21.15.5:0/84166838 conn(0x7f02ec06bd70 msgr2=0x7f02ec070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.913+0000 7f02f2dff700 1 -- 172.21.15.5:0/84166838 shutdown_connections 2023-07-21T19:38:32.898 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.913+0000 7f02f2dff700 1 -- 172.21.15.5:0/84166838 wait complete. 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.914+0000 7f02f2dff700 1 Processor -- start 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.914+0000 7f02f2dff700 1 -- start start 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 0x7f02ec117280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 0x7f02ec11a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.899 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec11af90 con 0x7f02ec06c910 2023-07-21T19:38:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec11b100 con 0x7f02ec06d3a0 2023-07-21T19:38:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.915+0000 7f02f2dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02ec11b2c0 con 0x7f02ec10e2a0 2023-07-21T19:38:32.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38272/0 (socket says 172.21.15.5:38272) 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 0x7f02ec11a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2124981993 learned_addr learned my addr 172.21.15.5:0/2124981993 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02ebfff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 0x7f02ec117280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2124981993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 msgr2=0x7f02ec11a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 0x7f02ec11a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2124981993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 msgr2=0x7f02ec117280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 0x7f02ec117280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2124981993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02e001a600 con 0x7f02ec06c910 2023-07-21T19:38:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.916+0000 7f02f139c700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 0x7f02ec11a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:32.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b530 tx=0x7f02dc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02e9ffb700 1 -- 172.21.15.5:0/2124981993 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc011830 con 0x7f02ec06c910 2023-07-21T19:38:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02ec11b5a0 con 0x7f02ec06c910 2023-07-21T19:38:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02e9ffb700 1 -- 172.21.15.5:0/2124981993 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02dc011e20 con 0x7f02ec06c910 2023-07-21T19:38:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02e9ffb700 1 -- 172.21.15.5:0/2124981993 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc010db0 con 0x7f02ec06c910 2023-07-21T19:38:32.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.917+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02ec1a92b0 con 0x7f02ec06c910 2023-07-21T19:38:32.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.923+0000 7f02e9ffb700 1 -- 172.21.15.5:0/2124981993 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f02dc011990 con 0x7f02ec06c910 2023-07-21T19:38:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.923+0000 7f030673fb80 1 Processor -- start 2023-07-21T19:38:32.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.923+0000 7f030673fb80 1 -- start start 2023-07-21T19:38:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d59fa040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d59fa9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d58ffc50 con 0x5651d5a002d0 2023-07-21T19:38:32.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d59529e0 con 0x5651d59fa580 2023-07-21T19:38:32.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f030673fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d594be70 con 0x5651d58566b0 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f02f139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f02f139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54530/0 (socket says 172.21.15.5:54530) 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.924+0000 7f02f139c700 1 -- 172.21.15.5:0/3207405197 learned_addr learned my addr 172.21.15.5:0/3207405197 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02ebfff700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d59fa9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d59fa040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f139c700 1 -- 172.21.15.5:0/3207405197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 msgr2=0x5651d59fa9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02e9ffb700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 0x7f02d40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f139c700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d59fa9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f139c700 1 -- 172.21.15.5:0/3207405197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 msgr2=0x5651d59fa040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f139c700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d59fa040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f139c700 1 -- 172.21.15.5:0/3207405197 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d5779d80 con 0x5651d58566b0 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d59fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.925+0000 7f02e9ffb700 1 -- 172.21.15.5:0/2124981993 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02dc0cc5a0 con 0x7f02ec06c910 2023-07-21T19:38:32.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.926+0000 7f02f139c700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 0x7f02d40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.926+0000 7f02f139c700 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f02e4007740 tx=0x7f02e4007a50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c22a6b31d520a876 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.926+0000 7f02d2ffd700 1 -- 172.21.15.5:0/3207405197 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e4012930 con 0x5651d58566b0 2023-07-21T19:38:32.912 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.926+0000 7f02d2ffd700 1 -- 172.21.15.5:0/3207405197 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02e400f040 con 0x5651d58566b0 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.926+0000 7f02d2ffd700 1 -- 172.21.15.5:0/3207405197 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02e40208d0 con 0x5651d58566b0 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.927+0000 7f02f139c700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 0x7f02d40a37f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02e401f000 tx=0x7f02e4020000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 -- 172.21.15.5:0/3207405197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 msgr2=0x5651d59ffd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f02e4007740 tx=0x7f02e4007a50 comp rx=0 tx=0).stop 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 -- 172.21.15.5:0/3207405197 shutdown_connections 2023-07-21T19:38:32.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d59fa9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d58566b0 0x5651d59ffd90 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 --2- 172.21.15.5:0/3207405197 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d59fa040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 -- 172.21.15.5:0/3207405197 >> 172.21.15.5:0/3207405197 conn(0x5651d59efbe0 msgr2=0x5651d59f4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 -- 172.21.15.5:0/3207405197 shutdown_connections 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.928+0000 7f030673fb80 1 -- 172.21.15.5:0/3207405197 wait complete. 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.929+0000 7f030673fb80 1 -- start start 2023-07-21T19:38:32.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.929+0000 7f030673fb80 1 Processor -- start 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.929+0000 7f030673fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.929+0000 7f030673fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a037d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f030673fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d5a03e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f030673fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d58eba60 con 0x5651d5a002d0 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f030673fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d58eb760 con 0x5651d58566b0 2023-07-21T19:38:32.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f030673fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d57cf8c0 con 0x5651d59fa580 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f030673fb80 10 client.0 _ll_register_callbacks cb 0x5651d58e9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59276/0 (socket says 172.21.15.5:59276) 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 -- 172.21.15.5:0/4157647400 learned_addr learned my addr 172.21.15.5:0/4157647400 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02ebfff700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a037d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f139c700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d5a03e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 -- 172.21.15.5:0/4157647400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 msgr2=0x5651d5a037d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a037d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 -- 172.21.15.5:0/4157647400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 msgr2=0x5651d5a03e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d5a03e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.930+0000 7f02f0b9b700 1 -- 172.21.15.5:0/4157647400 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02e4009e70 con 0x5651d58566b0 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00bb90 tx=0x7f02dc0cfe40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=beb4ba631ba19c12 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f02cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc099550 con 0x5651d58566b0 2023-07-21T19:38:32.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02dc0104f0 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5651d58a5490 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02dc0992c0 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f030673fb80 10 client.30389 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.931+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5651d5762c00 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5651d58a5490 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f02dc0999e0 con 0x5651d58566b0 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 10 client.30389 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:32.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 10 client.30389 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 20 client.30389 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a13a50 0x5651d5a15e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 4 client.30389 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:32.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5651d5944e50 con 0x5651d5a13a50 2023-07-21T19:38:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.932+0000 7f02ebfff700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a13a50 0x5651d5a15e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.933+0000 7f02ebfff700 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a13a50 0x5651d5a15e60 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f7cb194a8ec5f16 server_cookie=caaec38a04978448 in_seq=0 out_seq=0 2023-07-21T19:38:32.921 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.933+0000 7f02cbfff700 10 client.30389 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.934+0000 7f02cbfff700 1 -- 172.21.15.5:0/4157647400 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5651d5944e50 con 0x5651d5a13a50 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.934+0000 7f02cbfff700 10 client.30389 handle_command_reply: tid=0 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 client.30389 shutdown 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 msgr2=0x5651d5a06550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00bb90 tx=0x7f02dc0cfe40 comp rx=0 tx=0).stop 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 shutdown_connections 2023-07-21T19:38:32.922 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a13a50 0x5651d5a15e60 crc :-1 s=READY pgs=554 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d5a002d0 0x5651d5a03e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d58566b0 0x5651d5a06550 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 --2- 172.21.15.5:0/4157647400 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a037d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 >> 172.21.15.5:0/4157647400 conn(0x5651d59efbe0 msgr2=0x5651d59f44e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f02cbfff700 0 client.30389 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.935+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 shutdown_connections 2023-07-21T19:38:32.923 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.936+0000 7f030673fb80 1 -- 172.21.15.5:0/4157647400 wait complete. 2023-07-21T19:38:32.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.936+0000 7f030673fb80 20 client.30389 trim_cache size 0 max 16384 2023-07-21T19:38:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.958+0000 7f030673fb80 1 Processor -- start 2023-07-21T19:38:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.958+0000 7f030673fb80 1 -- start start 2023-07-21T19:38:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 0x5651d5a060f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d59f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d5a02f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d5944e50 con 0x5651d58566b0 2023-07-21T19:38:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d59ecc00 con 0x5651d59fa580 2023-07-21T19:38:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f030673fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d5a04db0 con 0x5651d59efbe0 2023-07-21T19:38:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.959+0000 7f02f139c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d5a02f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f139c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d5a02f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59290/0 (socket says 172.21.15.5:59290) 2023-07-21T19:38:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f139c700 1 -- 172.21.15.5:0/2636940070 learned_addr learned my addr 172.21.15.5:0/2636940070 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 0x5651d5a060f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02ebfff700 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d59f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2636940070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 msgr2=0x5651d5a02f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d5a02f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2636940070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 msgr2=0x5651d59f4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d59f4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.960+0000 7f02f0b9b700 1 -- 172.21.15.5:0/2636940070 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5651d5a1aa90 con 0x5651d59efbe0 2023-07-21T19:38:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.961+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 0x5651d5a060f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02dc099480 tx=0x7f02dc004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=be1c830b8b4fd5ab server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.961+0000 7f02d37fe700 1 -- 172.21.15.5:0/2636940070 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc099550 con 0x5651d59efbe0 2023-07-21T19:38:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.961+0000 7f02d37fe700 1 -- 172.21.15.5:0/2636940070 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02dc0992c0 con 0x5651d59efbe0 2023-07-21T19:38:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.961+0000 7f02d37fe700 1 -- 172.21.15.5:0/2636940070 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc0a1bd0 con 0x5651d59efbe0 2023-07-21T19:38:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- 172.21.15.5:0/2636940070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 msgr2=0x5651d5a060f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 0x5651d5a060f0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f02dc099480 tx=0x7f02dc004860 comp rx=0 tx=0).stop 2023-07-21T19:38:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- 172.21.15.5:0/2636940070 shutdown_connections 2023-07-21T19:38:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59fa580 0x5651d5a02f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59efbe0 0x5651d5a060f0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 --2- 172.21.15.5:0/2636940070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d59f4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- 172.21.15.5:0/2636940070 >> 172.21.15.5:0/2636940070 conn(0x5651d5a10100 msgr2=0x5651d5a01190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- 172.21.15.5:0/2636940070 shutdown_connections 2023-07-21T19:38:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- 172.21.15.5:0/2636940070 wait complete. 2023-07-21T19:38:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.962+0000 7f030673fb80 1 -- start start 2023-07-21T19:38:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.963+0000 7f030673fb80 1 Processor -- start 2023-07-21T19:38:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.963+0000 7f030673fb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d5a0a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.963+0000 7f030673fb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59efbe0 0x5651d5a08c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f030673fb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a09190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f030673fb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d5a0b270 con 0x5651d58566b0 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f030673fb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d5a096d0 con 0x5651d59efbe0 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f030673fb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5651d5a09890 con 0x5651d59fa580 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f030673fb80 10 client.0 _ll_register_callbacks cb 0x5651d58e9700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f0b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d5a0a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a09190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f139c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a09190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54554/0 (socket says 172.21.15.5:54554) 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f139c700 1 -- 172.21.15.5:0/1862434496 learned_addr learned my addr 172.21.15.5:0/1862434496 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02ebfff700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59efbe0 0x5651d5a08c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f0b9b700 1 -- 172.21.15.5:0/1862434496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 msgr2=0x5651d5a09190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a09190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f0b9b700 1 -- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59efbe0 msgr2=0x5651d5a08c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.964+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59efbe0 0x5651d5a08c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f02f0b9b700 1 -- 172.21.15.5:0/1862434496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5651d5762c00 con 0x5651d58566b0 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f02f0b9b700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d5a0a9a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0b9390 tx=0x7f02dc0b9270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e26f6d3a7c5c14 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f02cbfff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02dc09ade0 con 0x5651d58566b0 2023-07-21T19:38:32.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f02dc0104f0 con 0x5651d58566b0 2023-07-21T19:38:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.965+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f02e400f040 con 0x5651d58566b0 2023-07-21T19:38:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f02dc0997d0 con 0x5651d58566b0 2023-07-21T19:38:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 10 client.20625 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:32.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5651d57a01f0 con 0x5651d58566b0 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f02e400f040 con 0x5651d58566b0 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f02dc0b6060 con 0x5651d58566b0 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 10 client.20625 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 10 client.20625 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 20 client.20625 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:32.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a002d0 0x5651d5a6a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 4 client.20625 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:38:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.966+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5651d5906ff0 con 0x5651d5a002d0 2023-07-21T19:38:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.967+0000 7f02ebfff700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a002d0 0x5651d5a6a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.967+0000 7f02ebfff700 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a002d0 0x5651d5a6a740 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5a73bf8f60a7816 server_cookie=fde6c269fa344737 in_seq=0 out_seq=0 2023-07-21T19:38:32.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.967+0000 7f02cbfff700 10 client.20625 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f02cbfff700 1 -- 172.21.15.5:0/1862434496 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5651d5906ff0 con 0x5651d5a002d0 2023-07-21T19:38:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f02cbfff700 10 client.20625 handle_command_reply: tid=0 2023-07-21T19:38:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 client.20625 shutdown 2023-07-21T19:38:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 msgr2=0x5651d5a0a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d5a0a9a0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f02dc0b9390 tx=0x7f02dc0b9270 comp rx=0 tx=0).stop 2023-07-21T19:38:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 shutdown_connections 2023-07-21T19:38:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5651d59fa580 0x5651d5a09190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5651d5a002d0 0x5651d5a6a740 crc :-1 s=READY pgs=555 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5651d58566b0 0x5651d5a0a9a0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 --2- 172.21.15.5:0/1862434496 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5651d59efbe0 0x5651d5a08c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f02cbfff700 0 client.20625 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:32.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 >> 172.21.15.5:0/1862434496 conn(0x5651d5a10100 msgr2=0x5651d59fe9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 shutdown_connections 2023-07-21T19:38:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:32.999+0000 7f030673fb80 1 -- 172.21.15.5:0/1862434496 wait complete. 2023-07-21T19:38:32.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.000+0000 7f030673fb80 20 client.20625 trim_cache size 0 max 16384 2023-07-21T19:38:32.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:38:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:38:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:38:32.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:38:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.000+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 msgr2=0x7f02d40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.000+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 0x7f02d40a37f0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f02e401f000 tx=0x7f02e4020000 comp rx=0 tx=0).stop 2023-07-21T19:38:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.000+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 msgr2=0x7f02ec116d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.000+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f02dc00b530 tx=0x7f02dc00b840 comp rx=0 tx=0).stop 2023-07-21T19:38:32.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 shutdown_connections 2023-07-21T19:38:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02ec10e2a0 0x7f02ec11a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f02d40a1340 0x7f02d40a37f0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f02ec06c910 0x7f02ec116d20 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 --2- 172.21.15.5:0/2124981993 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f02ec06d3a0 0x7f02ec117280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 >> 172.21.15.5:0/2124981993 conn(0x7f02ec06bd70 msgr2=0x7f02ec0708a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 shutdown_connections 2023-07-21T19:38:32.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:33.001+0000 7f02f2dff700 1 -- 172.21.15.5:0/2124981993 wait complete. 2023-07-21T19:38:33.004 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:38:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 -- 172.21.15.5:0/4101333808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 msgr2=0x7fe9c410e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/4101333808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c410e650 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc009f90 tx=0x7fe9bc00b330 comp rx=0 tx=0).stop 2023-07-21T19:38:33.328 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 -- 172.21.15.5:0/4101333808 shutdown_connections 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/4101333808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406d3a0 0x7fe9c411b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/4101333808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c410e650 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/4101333808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406c9a0 0x7fe9c406ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 -- 172.21.15.5:0/4101333808 >> 172.21.15.5:0/4101333808 conn(0x7fe9c406bf70 msgr2=0x7fe9c4070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 -- 172.21.15.5:0/4101333808 shutdown_connections 2023-07-21T19:38:33.329 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.344+0000 7fe9c9bff700 1 -- 172.21.15.5:0/4101333808 wait complete. 2023-07-21T19:38:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.346+0000 7fe9c9bff700 1 Processor -- start 2023-07-21T19:38:33.330 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.346+0000 7fe9c9bff700 1 -- start start 2023-07-21T19:38:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406d3a0 0x7fe9c41aade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c41a4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.331 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41a4f80 con 0x7fe9c410e280 2023-07-21T19:38:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41a50f0 con 0x7fe9c406d3a0 2023-07-21T19:38:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c9bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c41a52b0 con 0x7fe9c406c9a0 2023-07-21T19:38:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c37fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54578/0 (socket says 172.21.15.5:54578) 2023-07-21T19:38:33.332 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c37fe700 1 -- 172.21.15.5:0/2387065520 learned_addr learned my addr 172.21.15.5:0/2387065520 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406d3a0 0x7fe9c41aade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.333 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.347+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c41a4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 -- 172.21.15.5:0/2387065520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406d3a0 msgr2=0x7fe9c41aade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406d3a0 0x7fe9c41aade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 -- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 msgr2=0x7fe9c41a4910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c41a4910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 -- 172.21.15.5:0/2387065520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9bc009c40 con 0x7fe9c406c9a0 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.348+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc008cd0 tx=0x7fe9bc008db0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.349+0000 7fe9c0ff9700 1 -- 172.21.15.5:0/2387065520 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc00e040 con 0x7fe9c406c9a0 2023-07-21T19:38:33.334 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.349+0000 7fe9c0ff9700 1 -- 172.21.15.5:0/2387065520 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9bc00be00 con 0x7fe9c406c9a0 2023-07-21T19:38:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.349+0000 7fe9c0ff9700 1 -- 172.21.15.5:0/2387065520 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc01b6c0 con 0x7fe9c406c9a0 2023-07-21T19:38:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.349+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c41a5530 con 0x7fe9c406c9a0 2023-07-21T19:38:33.335 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.349+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c41b1890 con 0x7fe9c406c9a0 2023-07-21T19:38:33.337 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.353+0000 7fe9c0ff9700 1 -- 172.21.15.5:0/2387065520 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fe9bc01d990 con 0x7fe9c406c9a0 2023-07-21T19:38:33.338 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.355+0000 7fe9dd45bb80 1 Processor -- start 2023-07-21T19:38:33.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.355+0000 7fe9dd45bb80 1 -- start start 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.355+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe5812d0 0x55f4fe5816a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.355+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe581be0 0x55f4fe58c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe487cb0 con 0x55f4fe5812d0 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe4daa40 con 0x55f4fe581be0 2023-07-21T19:38:33.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe4d3ed0 con 0x55f4fe58cad0 2023-07-21T19:38:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9c0ff9700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 0x7fe9ac0a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.341 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9c0ff9700 1 -- 172.21.15.5:0/2387065520 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe9bc0cde30 con 0x7fe9c406c9a0 2023-07-21T19:38:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54582/0 (socket says 172.21.15.5:54582) 2023-07-21T19:38:33.342 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.356+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/285224892 learned_addr learned my addr 172.21.15.5:0/285224892 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 0x7fe9ac0a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe5812d0 0x55f4fe5816a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/285224892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe581be0 msgr2=0x55f4fe58c590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe581be0 0x55f4fe58c590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/285224892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe5812d0 msgr2=0x55f4fe5816a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe5812d0 0x55f4fe5816a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/285224892 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4fe301de0 con 0x55f4fe58cad0 2023-07-21T19:38:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.357+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400bd10 tx=0x7fe9b400f640 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14559aeaaf055f71 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.358+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 0x7fe9ac0a3830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80095a0 tx=0x7fe9b8009510 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.359+0000 7fe9a9ffb700 1 -- 172.21.15.5:0/285224892 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b400fdf0 con 0x55f4fe58cad0 2023-07-21T19:38:33.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.359+0000 7fe9a9ffb700 1 -- 172.21.15.5:0/285224892 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b400be60 con 0x55f4fe58cad0 2023-07-21T19:38:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.359+0000 7fe9a9ffb700 1 -- 172.21.15.5:0/285224892 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b4011730 con 0x55f4fe58cad0 2023-07-21T19:38:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.360+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/285224892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 msgr2=0x55f4fe58eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.360+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7fe9b400bd10 tx=0x7fe9b400f640 comp rx=0 tx=0).stop 2023-07-21T19:38:33.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/285224892 shutdown_connections 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe58eeb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe5812d0 0x55f4fe5816a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/285224892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe581be0 0x55f4fe58c590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/285224892 >> 172.21.15.5:0/285224892 conn(0x55f4fe577da0 msgr2=0x55f4fe57d6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/285224892 shutdown_connections 2023-07-21T19:38:33.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.361+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/285224892 wait complete. 2023-07-21T19:38:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.362+0000 7fe9dd45bb80 1 -- start start 2023-07-21T19:38:33.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.362+0000 7fe9dd45bb80 1 Processor -- start 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe58cad0 0x55f4fe589f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe473ac0 con 0x55f4fe58cad0 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe4737c0 con 0x55f4fe5812d0 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe357920 con 0x55f4fe581be0 2023-07-21T19:38:33.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.363+0000 7fe9dd45bb80 10 client.0 _ll_register_callbacks cb 0x55f4fe471760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.364+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.364+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:54596/0 (socket says 172.21.15.5:54596) 2023-07-21T19:38:33.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.364+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/1373695783 learned_addr learned my addr 172.21.15.5:0/1373695783 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.364+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe58cad0 0x55f4fe589f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.365+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 msgr2=0x55f4fe58c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.365+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.365+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/1373695783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe58cad0 msgr2=0x55f4fe589f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.365+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe58cad0 0x55f4fe589f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.365+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/1373695783 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9b400b940 con 0x55f4fe581be0 2023-07-21T19:38:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000c00 tx=0x7fe9b400fe20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=30c663a31e55eb72 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b401a8f0 con 0x55f4fe581be0 2023-07-21T19:38:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f4fe42d4f0 con 0x55f4fe581be0 2023-07-21T19:38:33.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe9b401aa50 con 0x55f4fe581be0 2023-07-21T19:38:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9b4011440 con 0x55f4fe581be0 2023-07-21T19:38:33.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.366+0000 7fe9dd45bb80 10 client.30346 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55f4fe2eac60 con 0x55f4fe581be0 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55f4fe42d4f0 con 0x55f4fe581be0 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9b4022070 con 0x55f4fe581be0 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 10 client.30346 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 10 client.30346 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 20 client.30346 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:33.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe59c320 0x55f4fe59e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 4 client.30346 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.367+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f4fe4cceb0 con 0x55f4fe59c320 2023-07-21T19:38:33.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.368+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe59c320 0x55f4fe59e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.369+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe59c320 0x55f4fe59e730 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc7d036174e0ad37 server_cookie=d0cba3188f56153e in_seq=0 out_seq=0 2023-07-21T19:38:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.369+0000 7fe9a2ffd700 10 client.30346 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/1373695783 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f4fe4cceb0 con 0x55f4fe59c320 2023-07-21T19:38:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9a2ffd700 10 client.30346 handle_command_reply: tid=0 2023-07-21T19:38:33.355 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 client.30346 shutdown 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 msgr2=0x55f4fe589810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4000c00 tx=0x7fe9b400fe20 comp rx=0 tx=0).stop 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 shutdown_connections 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe59c320 0x55f4fe59e730 crc :-1 s=READY pgs=556 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe58cad0 0x55f4fe589f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9a2ffd700 0 client.30346 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/1373695783 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe581be0 0x55f4fe589810 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 >> 172.21.15.5:0/1373695783 conn(0x55f4fe577da0 msgr2=0x55f4fe57d290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.370+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 shutdown_connections 2023-07-21T19:38:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.371+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/1373695783 wait complete. 2023-07-21T19:38:33.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.371+0000 7fe9dd45bb80 20 client.30346 trim_cache size 0 max 16384 2023-07-21T19:38:33.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 Processor -- start 2023-07-21T19:38:33.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 -- start start 2023-07-21T19:38:33.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe589510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe574fa0 con 0x55f4fe581be0 2023-07-21T19:38:33.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe577970 con 0x55f4fe5812d0 2023-07-21T19:38:33.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.402+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe582dd0 con 0x55f4fe58cad0 2023-07-21T19:38:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38366/0 (socket says 172.21.15.5:38366) 2023-07-21T19:38:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/2529824891 learned_addr learned my addr 172.21.15.5:0/2529824891 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:33.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe589510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/2529824891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 msgr2=0x55f4fe589510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe589510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/2529824891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 msgr2=0x55f4fe58c0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.403+0000 7fe9c2ffd700 1 -- 172.21.15.5:0/2529824891 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f4fe328250 con 0x55f4fe581be0 2023-07-21T19:38:33.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.404+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005810 tx=0x7fe9b400b5e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c5d82940cdae01f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.404+0000 7fe9aa7fc700 1 -- 172.21.15.5:0/2529824891 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b401adc0 con 0x55f4fe581be0 2023-07-21T19:38:33.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.404+0000 7fe9aa7fc700 1 -- 172.21.15.5:0/2529824891 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b401f040 con 0x55f4fe581be0 2023-07-21T19:38:33.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.404+0000 7fe9aa7fc700 1 -- 172.21.15.5:0/2529824891 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b401a950 con 0x55f4fe581be0 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/2529824891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 msgr2=0x55f4fe588fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fe9b4005810 tx=0x7fe9b400b5e0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/2529824891 shutdown_connections 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe589510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe58c0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/2529824891 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe588fd0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/2529824891 >> 172.21.15.5:0/2529824891 conn(0x55f4fe577da0 msgr2=0x55f4fe5987f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/2529824891 shutdown_connections 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.405+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/2529824891 wait complete. 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.406+0000 7fe9dd45bb80 1 -- start start 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.406+0000 7fe9dd45bb80 1 Processor -- start 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe5916b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe591bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe592170 con 0x55f4fe581be0 2023-07-21T19:38:33.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe5922b0 con 0x55f4fe5812d0 2023-07-21T19:38:33.396 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f4fe592440 con 0x55f4fe58cad0 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.407+0000 7fe9dd45bb80 10 client.0 _ll_register_callbacks cb 0x7fe9b400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:59344/0 (socket says 172.21.15.5:59344) 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 -- 172.21.15.5:0/3423634079 learned_addr learned my addr 172.21.15.5:0/3423634079 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c3fff700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe591bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe5916b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 -- 172.21.15.5:0/3423634079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 msgr2=0x55f4fe591bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe591bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 -- 172.21.15.5:0/3423634079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 msgr2=0x55f4fe5916b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe5916b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.408+0000 7fe9c37fe700 1 -- 172.21.15.5:0/3423634079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f4fe2eac60 con 0x55f4fe5812d0 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.409+0000 7fe9c37fe700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0979a0 tx=0x7fe9bc097270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1aba1f45f7ce394d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.409+0000 7fe9a2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9bc0cb9a0 con 0x55f4fe5812d0 2023-07-21T19:38:33.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fe9bc09ef60 con 0x55f4fe5812d0 2023-07-21T19:38:33.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fe9b400be60 con 0x55f4fe5812d0 2023-07-21T19:38:33.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe9bc0a32f0 con 0x55f4fe5812d0 2023-07-21T19:38:33.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9dd45bb80 10 client.30407 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:33.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55f4fe3002b0 con 0x55f4fe5812d0 2023-07-21T19:38:33.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.410+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fe9b400be60 con 0x55f4fe5812d0 2023-07-21T19:38:33.400 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fe9bc0b9020 con 0x55f4fe5812d0 2023-07-21T19:38:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 10 client.30407 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:33.401 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 10 client.30407 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 20 client.30407 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe5f1360 0x55f4fe5f3770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 4 client.30407 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:38:33.402 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55f4fe48f050 con 0x55f4fe5f1360 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.411+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe5f1360 0x55f4fe5f3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.412+0000 7fe9c2ffd700 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe5f1360 0x55f4fe5f3770 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a140f10dd82248ff server_cookie=160fed6f071ac75e in_seq=0 out_seq=0 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.412+0000 7fe9a2ffd700 10 client.30407 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.412+0000 7fe9a2ffd700 1 -- 172.21.15.5:0/3423634079 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x55f4fe48f050 con 0x55f4fe5f1360 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.412+0000 7fe9a2ffd700 10 client.30407 handle_command_reply: tid=0 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.412+0000 7fe9dd45bb80 1 client.30407 shutdown 2023-07-21T19:38:33.403 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 msgr2=0x55f4fe5933f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc0979a0 tx=0x7fe9bc097270 comp rx=0 tx=0).stop 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 shutdown_connections 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55f4fe58cad0 0x55f4fe591bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55f4fe5812d0 0x55f4fe5933f0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55f4fe5f1360 0x55f4fe5f3770 crc :-1 s=READY pgs=557 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 --2- 172.21.15.5:0/3423634079 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55f4fe581be0 0x55f4fe5916b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.404 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 >> 172.21.15.5:0/3423634079 conn(0x55f4fe577da0 msgr2=0x55f4fe57c8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9a2ffd700 0 client.30407 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 shutdown_connections 2023-07-21T19:38:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 1 -- 172.21.15.5:0/3423634079 wait complete. 2023-07-21T19:38:33.405 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.413+0000 7fe9dd45bb80 20 client.30407 trim_cache size 0 max 16384 2023-07-21T19:38:33.405 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:38:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:38:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:38:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:38:33.406 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:38:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:38:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:38:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:38:33.407 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:38:33.407 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:38:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 msgr2=0x7fe9ac0a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 0x7fe9ac0a3830 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe9b80095a0 tx=0x7fe9b8009510 comp rx=0 tx=0).stop 2023-07-21T19:38:33.408 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 msgr2=0x7fe9c41aa8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fe9bc008cd0 tx=0x7fe9bc008db0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 shutdown_connections 2023-07-21T19:38:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fe9ac0a1380 0x7fe9ac0a3830 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fe9c410e280 0x7fe9c41a4910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.409 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fe9c406c9a0 0x7fe9c41aa8a0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 --2- 172.21.15.5:0/2387065520 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fe9c406d3a0 0x7fe9c41aade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 >> 172.21.15.5:0/2387065520 conn(0x7fe9c406bf70 msgr2=0x7fe9c406f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 shutdown_connections 2023-07-21T19:38:33.410 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:38:33.414+0000 7fe9c9bff700 1 -- 172.21.15.5:0/2387065520 wait complete. 2023-07-21T19:38:33.426 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:38:34.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:33 smithi101 ceph-mon[128136]: pgmap v982: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.4 MiB/s wr, 905 op/s 2023-07-21T19:38:34.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:33 smithi005 ceph-mon[126460]: pgmap v982: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.4 MiB/s wr, 905 op/s 2023-07-21T19:38:34.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:33 smithi156 ceph-mon[128316]: pgmap v982: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 8.4 MiB/s wr, 905 op/s 2023-07-21T19:38:36.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:35 smithi101 ceph-mon[128136]: pgmap v983: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:36.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:35 smithi005 ceph-mon[126460]: pgmap v983: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:36.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:35 smithi156 ceph-mon[128316]: pgmap v983: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:36.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 725.0 s, 249.0 tps, lat 395.321 ms stddev 87.784 2023-07-21T19:38:38.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:37 smithi101 ceph-mon[128136]: pgmap v984: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:38.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:37 smithi005 ceph-mon[126460]: pgmap v984: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:38.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:37 smithi156 ceph-mon[128316]: pgmap v984: 97 pgs: 97 active+clean; 11 GiB data, 18 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 601 op/s 2023-07-21T19:38:39.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:38 smithi156 ceph-mon[128316]: pgmap v985: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 941 op/s 2023-07-21T19:38:39.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:38 smithi101 ceph-mon[128136]: pgmap v985: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 941 op/s 2023-07-21T19:38:39.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:38 smithi005 ceph-mon[126460]: pgmap v985: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.8 MiB/s wr, 941 op/s 2023-07-21T19:38:41.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 730.0 s, 236.2 tps, lat 421.596 ms stddev 84.965 2023-07-21T19:38:42.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:41 smithi101 ceph-mon[128136]: pgmap v986: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 654 op/s 2023-07-21T19:38:42.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:41 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:38:42.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:41 smithi005 ceph-mon[126460]: pgmap v986: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 654 op/s 2023-07-21T19:38:42.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:41 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:38:42.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:41 smithi156 ceph-mon[128316]: pgmap v986: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.2 MiB/s wr, 654 op/s 2023-07-21T19:38:42.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:41 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:38:44.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:43 smithi101 ceph-mon[128136]: pgmap v987: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.6 MiB/s wr, 1.00k op/s 2023-07-21T19:38:44.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:43 smithi005 ceph-mon[126460]: pgmap v987: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.6 MiB/s wr, 1.00k op/s 2023-07-21T19:38:44.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:43 smithi156 ceph-mon[128316]: pgmap v987: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.6 MiB/s wr, 1.00k op/s 2023-07-21T19:38:46.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:45 smithi101 ceph-mon[128136]: pgmap v988: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:46.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:45 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:45 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:45 smithi005 ceph-mon[126460]: pgmap v988: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:46.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:45 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:45 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:45 smithi156 ceph-mon[128316]: pgmap v988: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:46.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:45 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:45 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 -- 172.21.15.5:0/449029301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c0105b60 msgr2=0x7f45c0105f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.214 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 --2- 172.21.15.5:0/449029301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c0105b60 0x7f45c0105f30 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 -- 172.21.15.5:0/449029301 shutdown_connections 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 --2- 172.21.15.5:0/449029301 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c01004e0 0x7f45c010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 --2- 172.21.15.5:0/449029301 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c0105b60 0x7f45c0105f30 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 --2- 172.21.15.5:0/449029301 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c00ffb30 0x7f45c00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.229+0000 7f45c7dff700 1 -- 172.21.15.5:0/449029301 >> 172.21.15.5:0/449029301 conn(0x7f45c00fb650 msgr2=0x7f45c00fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.230+0000 7f45c7dff700 1 -- 172.21.15.5:0/449029301 shutdown_connections 2023-07-21T19:38:46.215 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.230+0000 7f45c7dff700 1 -- 172.21.15.5:0/449029301 wait complete. 2023-07-21T19:38:46.216 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.232+0000 7f45c7dff700 1 Processor -- start 2023-07-21T19:38:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.232+0000 7f45c7dff700 1 -- start start 2023-07-21T19:38:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.232+0000 7f45c7dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c00ffb30 0x7f45c0108f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.217 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c7dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 0x7f45c0109470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c7dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 0x7f45c01a8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c7dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c01a8680 con 0x7f45c00ffb30 2023-07-21T19:38:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c7dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c01a87c0 con 0x7f45c01004e0 2023-07-21T19:38:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c7dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45c01a8980 con 0x7f45c0105b60 2023-07-21T19:38:46.218 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c539a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 0x7f45c0109470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c639c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 0x7f45c01a8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c539a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 0x7f45c0109470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52164/0 (socket says 172.21.15.5:52164) 2023-07-21T19:38:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.233+0000 7f45c539a700 1 -- 172.21.15.5:0/168880364 learned_addr learned my addr 172.21.15.5:0/168880364 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c00ffb30 0x7f45c0108f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.219 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c639c700 1 -- 172.21.15.5:0/168880364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 msgr2=0x7f45c0109470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c639c700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 0x7f45c0109470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c639c700 1 -- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c00ffb30 msgr2=0x7f45c0108f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c639c700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c00ffb30 0x7f45c0108f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.220 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.234+0000 7f45c639c700 1 -- 172.21.15.5:0/168880364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45bc0095f0 con 0x7f45c0105b60 2023-07-21T19:38:46.221 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.236+0000 7f45c639c700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 0x7f45c01a8140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f60 tx=0x7f45b8009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.243+0000 7f45b6ffd700 1 -- 172.21.15.5:0/168880364 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b8018070 con 0x7f45c0105b60 2023-07-21T19:38:46.229 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.243+0000 7f45b6ffd700 1 -- 172.21.15.5:0/168880364 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45b800f040 con 0x7f45c0105b60 2023-07-21T19:38:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.243+0000 7f45b6ffd700 1 -- 172.21.15.5:0/168880364 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b800cca0 con 0x7f45c0105b60 2023-07-21T19:38:46.230 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.245+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45c01a8c00 con 0x7f45c0105b60 2023-07-21T19:38:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.245+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c01a9110 con 0x7f45c0105b60 2023-07-21T19:38:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.252+0000 7f45b6ffd700 1 -- 172.21.15.5:0/168880364 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f45b8007500 con 0x7f45c0105b60 2023-07-21T19:38:46.241 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.257+0000 7f45b6ffd700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 0x7f45ac0a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.242 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.257+0000 7f45b6ffd700 1 -- 172.21.15.5:0/168880364 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f45b80cc090 con 0x7f45c0105b60 2023-07-21T19:38:46.243 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.258+0000 7f45db6cdb80 1 Processor -- start 2023-07-21T19:38:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 -- start start 2023-07-21T19:38:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 0x7f45ac0a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f1d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6ec580 0x56379d6ec9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d5f1c50 con 0x56379d5486b0 2023-07-21T19:38:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d6449e0 con 0x56379d6ec580 2023-07-21T19:38:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.259+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d63de70 con 0x56379d6f22d0 2023-07-21T19:38:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.260+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.260+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53320/0 (socket says 172.21.15.5:53320) 2023-07-21T19:38:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.260+0000 7f45c539a700 1 -- 172.21.15.5:0/3885329323 learned_addr learned my addr 172.21.15.5:0/3885329323 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.260+0000 7f45c639c700 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6ec580 0x56379d6ec9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 -- 172.21.15.5:0/3885329323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6ec580 msgr2=0x56379d6ec9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6ec580 0x56379d6ec9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 -- 172.21.15.5:0/3885329323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 msgr2=0x56379d6f1d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 -- 172.21.15.5:0/3885329323 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56379d46bd80 con 0x56379d6f22d0 2023-07-21T19:38:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c539a700 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009b40 tx=0x7f45b0009e50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=668677a8f21bc343 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.261+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 0x7f45ac0a3940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006010 tx=0x7f45bc0053a0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.262+0000 7f45abfff700 1 -- 172.21.15.5:0/3885329323 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b000e910 con 0x56379d6f22d0 2023-07-21T19:38:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.262+0000 7f45abfff700 1 -- 172.21.15.5:0/3885329323 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45b0013470 con 0x56379d6f22d0 2023-07-21T19:38:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.262+0000 7f45abfff700 1 -- 172.21.15.5:0/3885329323 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0012630 con 0x56379d6f22d0 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3885329323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 msgr2=0x56379d6ec040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009b40 tx=0x7f45b0009e50 comp rx=0 tx=0).stop 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3885329323 shutdown_connections 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6ec580 0x56379d6ec9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f1d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3885329323 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6f22d0 0x56379d6ec040 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.263+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3885329323 >> 172.21.15.5:0/3885329323 conn(0x56379d6e1be0 msgr2=0x56379d6e6f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.264+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3885329323 shutdown_connections 2023-07-21T19:38:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.264+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3885329323 wait complete. 2023-07-21T19:38:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.264+0000 7f45db6cdb80 1 -- start start 2023-07-21T19:38:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.265+0000 7f45db6cdb80 1 Processor -- start 2023-07-21T19:38:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6f5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d5dda60 con 0x56379d5486b0 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d5dd760 con 0x56379d6f22d0 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d4c18c0 con 0x56379d6ec580 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45db6cdb80 10 client.0 _ll_register_callbacks cb 0x56379d5db700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.266+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53336/0 (socket says 172.21.15.5:53336) 2023-07-21T19:38:46.253 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c539a700 1 -- 172.21.15.5:0/3971857279 learned_addr learned my addr 172.21.15.5:0/3971857279 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c639c700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6f5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 -- 172.21.15.5:0/3971857279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 msgr2=0x56379d6f5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 -- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 msgr2=0x56379d6f5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6f5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c5b9b700 1 -- 172.21.15.5:0/3971857279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45b0009770 con 0x56379d5486b0 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.267+0000 7f45c639c700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.268+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f8610 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098e0 tx=0x7f45bc009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27fdf834856d48ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.268+0000 7f45a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.269+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc005160 con 0x56379d5486b0 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.269+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f45bc00be40 con 0x56379d5486b0 2023-07-21T19:38:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.269+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56379d597490 con 0x56379d5486b0 2023-07-21T19:38:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.270+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc004e50 con 0x56379d5486b0 2023-07-21T19:38:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.270+0000 7f45db6cdb80 10 client.20652 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.270+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56379d454c00 con 0x56379d5486b0 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.270+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56379d597490 con 0x56379d5486b0 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45bc028590 con 0x56379d5486b0 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 10 client.20652 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 10 client.20652 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 20 client.20652 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d705bd0 0x56379d707fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 4 client.20652 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.271+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56379d636e50 con 0x56379d705bd0 2023-07-21T19:38:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.272+0000 7f45c539a700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d705bd0 0x56379d707fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.272+0000 7f45c539a700 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d705bd0 0x56379d707fe0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=816d70b06a4ef32d server_cookie=426748250bc71468 in_seq=0 out_seq=0 2023-07-21T19:38:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.272+0000 7f45a97fa700 10 client.20652 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.273+0000 7f45a97fa700 1 -- 172.21.15.5:0/3971857279 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56379d636e50 con 0x56379d705bd0 2023-07-21T19:38:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45a97fa700 10 client.20652 handle_command_reply: tid=0 2023-07-21T19:38:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 client.20652 shutdown 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 msgr2=0x56379d6f8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f8610 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f45bc0098e0 tx=0x7f45bc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 shutdown_connections 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6f5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d705bd0 0x56379d707fe0 crc :-1 s=READY pgs=558 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6f8610 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/3971857279 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 >> 172.21.15.5:0/3971857279 conn(0x56379d6e1be0 msgr2=0x56379d6e64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.274+0000 7f45a97fa700 0 client.20652 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.275+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 shutdown_connections 2023-07-21T19:38:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.275+0000 7f45db6cdb80 1 -- 172.21.15.5:0/3971857279 wait complete. 2023-07-21T19:38:46.262 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.275+0000 7f45db6cdb80 20 client.20652 trim_cache size 0 max 16384 2023-07-21T19:38:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.303+0000 7f45db6cdb80 1 Processor -- start 2023-07-21T19:38:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 -- start start 2023-07-21T19:38:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6ec040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6e6c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d6f19e0 con 0x56379d5486b0 2023-07-21T19:38:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d6e35f0 con 0x56379d6f22d0 2023-07-21T19:38:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.304+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d6f0400 con 0x56379d6ec580 2023-07-21T19:38:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53342/0 (socket says 172.21.15.5:53342) 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c639c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6e6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 -- 172.21.15.5:0/4159110788 learned_addr learned my addr 172.21.15.5:0/4159110788 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6ec040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 -- 172.21.15.5:0/4159110788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 msgr2=0x56379d6e6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6e6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 -- 172.21.15.5:0/4159110788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 msgr2=0x56379d6ec040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6ec040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c539a700 1 -- 172.21.15.5:0/4159110788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56379d4921f0 con 0x56379d6ec580 2023-07-21T19:38:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.305+0000 7f45c639c700 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.306+0000 7f45c539a700 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009fd0 tx=0x7f45b0004860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78fac134c95bbca9 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.306+0000 7f45b4bff700 1 -- 172.21.15.5:0/4159110788 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0027070 con 0x56379d6ec580 2023-07-21T19:38:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.306+0000 7f45b4bff700 1 -- 172.21.15.5:0/4159110788 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45b0023370 con 0x56379d6ec580 2023-07-21T19:38:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.306+0000 7f45b4bff700 1 -- 172.21.15.5:0/4159110788 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45b0023860 con 0x56379d6ec580 2023-07-21T19:38:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 -- 172.21.15.5:0/4159110788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 msgr2=0x56379d6f5840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f45b0009fd0 tx=0x7f45b0004860 comp rx=0 tx=0).stop 2023-07-21T19:38:46.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 -- 172.21.15.5:0/4159110788 shutdown_connections 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d6f22d0 0x56379d6e6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d5486b0 0x56379d6ec040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/4159110788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6f5840 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 -- 172.21.15.5:0/4159110788 >> 172.21.15.5:0/4159110788 conn(0x56379d6e1be0 msgr2=0x56379d6f5130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 -- 172.21.15.5:0/4159110788 shutdown_connections 2023-07-21T19:38:46.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.307+0000 7f45db6cdb80 1 -- 172.21.15.5:0/4159110788 wait complete. 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.308+0000 7f45db6cdb80 1 -- start start 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.308+0000 7f45db6cdb80 1 Processor -- start 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6fb1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d6f22d0 0x56379d6fb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d758b20 con 0x56379d6f22d0 2023-07-21T19:38:46.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d758c90 con 0x56379d5486b0 2023-07-21T19:38:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56379d758e50 con 0x56379d6ec580 2023-07-21T19:38:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.309+0000 7f45db6cdb80 10 client.0 _ll_register_callbacks cb 0x7f45bc00ba20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52216/0 (socket says 172.21.15.5:52216) 2023-07-21T19:38:46.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 -- 172.21.15.5:0/2076049661 learned_addr learned my addr 172.21.15.5:0/2076049661 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c539a700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6fb1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c639c700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d6f22d0 0x56379d6fb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 -- 172.21.15.5:0/2076049661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 msgr2=0x56379d6fb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6fb1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 -- 172.21.15.5:0/2076049661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d6f22d0 msgr2=0x56379d6fb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d6f22d0 0x56379d6fb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c5b9b700 1 -- 172.21.15.5:0/2076049661 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45c01a9110 con 0x56379d5486b0 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.310+0000 7f45c539a700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.311+0000 7f45c5b9b700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc028f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b9a6c1b1ff5c0d4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:46.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.311+0000 7f45a97fa700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:38:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.311+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45bc00be40 con 0x56379d5486b0 2023-07-21T19:38:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.311+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f45bc028430 con 0x56379d5486b0 2023-07-21T19:38:46.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.311+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f45b0013470 con 0x56379d5486b0 2023-07-21T19:38:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.312+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45bc0285f0 con 0x56379d5486b0 2023-07-21T19:38:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.312+0000 7f45db6cdb80 10 client.30422 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.312+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56379d46bd80 con 0x56379d5486b0 2023-07-21T19:38:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f45b0013470 con 0x56379d5486b0 2023-07-21T19:38:46.303 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f45bc00f460 con 0x56379d5486b0 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 10 client.30422 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 10 client.30422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 20 client.30422 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d75a090 0x56379d75c470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 4 client.30422 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:38:46.304 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56379d5f8ff0 con 0x56379d75a090 2023-07-21T19:38:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.313+0000 7f45c539a700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d75a090 0x56379d75c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.315+0000 7f45c539a700 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d75a090 0x56379d75c470 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10753cd34004222c server_cookie=d4141c7a986654f0 in_seq=0 out_seq=0 2023-07-21T19:38:46.305 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.315+0000 7f45a97fa700 10 client.30422 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.339+0000 7f45a97fa700 1 -- 172.21.15.5:0/2076049661 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56379d5f8ff0 con 0x56379d75a090 2023-07-21T19:38:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.339+0000 7f45a97fa700 10 client.30422 handle_command_reply: tid=0 2023-07-21T19:38:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.339+0000 7f45db6cdb80 1 client.30422 shutdown 2023-07-21T19:38:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 msgr2=0x56379d6fac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f45bc009990 tx=0x7f45bc028f50 comp rx=0 tx=0).stop 2023-07-21T19:38:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 shutdown_connections 2023-07-21T19:38:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56379d75a090 0x56379d75c470 crc :-1 s=READY pgs=559 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56379d6f22d0 0x56379d6fb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56379d5486b0 0x56379d6fac90 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 --2- 172.21.15.5:0/2076049661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56379d6ec580 0x56379d6fb1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 >> 172.21.15.5:0/2076049661 conn(0x56379d6e1be0 msgr2=0x56379d6f0a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45a97fa700 0 client.30422 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 shutdown_connections 2023-07-21T19:38:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.340+0000 7f45db6cdb80 1 -- 172.21.15.5:0/2076049661 wait complete. 2023-07-21T19:38:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.341+0000 7f45db6cdb80 20 client.30422 trim_cache size 0 max 16384 2023-07-21T19:38:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:38:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:38:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:38:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:38:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 msgr2=0x7f45ac0a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 0x7f45ac0a3940 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f45bc006010 tx=0x7f45bc0053a0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 msgr2=0x7f45c01a8140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 0x7f45c01a8140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f45b8009f60 tx=0x7f45b8009f90 comp rx=0 tx=0).stop 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 shutdown_connections 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f45c0105b60 0x7f45c01a8140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f45ac0a1490 0x7f45ac0a3940 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f45c00ffb30 0x7f45c0108f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 --2- 172.21.15.5:0/168880364 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f45c01004e0 0x7f45c0109470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 >> 172.21.15.5:0/168880364 conn(0x7f45c00fb650 msgr2=0x7f45c0103650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.342+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 shutdown_connections 2023-07-21T19:38:46.331 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:46.343+0000 7f45c7dff700 1 -- 172.21.15.5:0/168880364 wait complete. 2023-07-21T19:38:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 735.0 s, 202.2 tps, lat 490.979 ms stddev 116.048 2023-07-21T19:38:47.627 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.894 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: pgmap v989: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.895 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.896 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:47 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: pgmap v989: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:47.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:47 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: pgmap v989: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.5 MiB/s wr, 686 op/s 2023-07-21T19:38:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:47 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:48.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:48 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:38:48.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:48 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:38:48.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:48 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:38:48.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:48 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:38:49.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:48 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi156 to 220.9M 2023-07-21T19:38:49.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:48 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi156 to 231678361: error parsing value: Value '231678361' is below minimum 939524096 2023-07-21T19:38:50.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: pgmap v990: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 977 op/s 2023-07-21T19:38:50.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:38:50.070 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:49 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:38:50.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: pgmap v990: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 977 op/s 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:38:50.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:49 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:38:50.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: pgmap v990: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.2 MiB/s wr, 977 op/s 2023-07-21T19:38:50.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:50.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi005 to 222.0M 2023-07-21T19:38:50.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:49 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi005 to 232786534: error parsing value: Value '232786534' is below minimum 939524096 2023-07-21T19:38:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 740.0 s, 237.0 tps, lat 431.391 ms stddev 91.285 2023-07-21T19:38:52.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:51 smithi101 ceph-mon[128136]: pgmap v991: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-07-21T19:38:52.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:51 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:52.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:51 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:52.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:51 smithi005 ceph-mon[126460]: pgmap v991: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-07-21T19:38:52.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:51 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:52.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:51 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:52.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:51 smithi156 ceph-mon[128316]: pgmap v991: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.0 MiB/s wr, 638 op/s 2023-07-21T19:38:52.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:51 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:52.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:51 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:53.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: pgmap v992: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.0 MiB/s wr, 947 op/s 2023-07-21T19:38:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.905 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.906 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:38:53.907 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:38:53.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:38:53.908 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:38:53.909 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:53 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: pgmap v992: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.0 MiB/s wr, 947 op/s 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:38:54.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:38:54.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:38:54.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:53 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: pgmap v992: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.0 MiB/s wr, 947 op/s 2023-07-21T19:38:54.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:54.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.145 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-07-21T19:38:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: Adjusting osd_memory_target on smithi101 to 220.9M 2023-07-21T19:38:54.146 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: Unable to set osd_memory_target on smithi101 to 231653990: error parsing value: Value '231653990' is below minimum 939524096 2023-07-21T19:38:54.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:38:54.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:38:54.147 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:53 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:38:56.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:55 smithi101 ceph-mon[128136]: pgmap v993: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:55 smithi005 ceph-mon[126460]: pgmap v993: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:56.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:55 smithi156 ceph-mon[128316]: pgmap v993: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 745.0 s, 251.4 tps, lat 377.599 ms stddev 59.203 2023-07-21T19:38:58.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:57 smithi101 ceph-mon[128136]: pgmap v994: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:58.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:57 smithi005 ceph-mon[126460]: pgmap v994: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:58.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:57 smithi156 ceph-mon[128316]: pgmap v994: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.7 MiB/s wr, 600 op/s 2023-07-21T19:38:59.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:38:58 smithi156 ceph-mon[128316]: pgmap v995: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 931 op/s 2023-07-21T19:38:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:38:58 smithi101 ceph-mon[128136]: pgmap v995: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 931 op/s 2023-07-21T19:38:59.395 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:38:58 smithi005 ceph-mon[126460]: pgmap v995: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.9 MiB/s wr, 931 op/s 2023-07-21T19:38:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.595+0000 7fdf7cbff700 1 -- 172.21.15.5:0/3729504991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 msgr2=0x7fdf78074e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.595+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/3729504991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf78074e10 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fdf70009990 tx=0x7fdf70009ca0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 -- 172.21.15.5:0/3729504991 shutdown_connections 2023-07-21T19:38:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/3729504991 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 0x7fdf78112e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/3729504991 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf78074e10 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/3729504991 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf78110530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 -- 172.21.15.5:0/3729504991 >> 172.21.15.5:0/3729504991 conn(0x7fdf780fd8d0 msgr2=0x7fdf780ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.597+0000 7fdf7cbff700 1 -- 172.21.15.5:0/3729504991 shutdown_connections 2023-07-21T19:38:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.598+0000 7fdf7cbff700 1 -- 172.21.15.5:0/3729504991 wait complete. 2023-07-21T19:38:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.599+0000 7fdf7cbff700 1 Processor -- start 2023-07-21T19:38:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 -- start start 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf781102f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 0x7fdf78108b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78109340 con 0x7fdf78075350 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf781094b0 con 0x7fdf78074a40 2023-07-21T19:38:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.600+0000 7fdf7cbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf78109670 con 0x7fdf78110a70 2023-07-21T19:38:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48960/0 (socket says 172.21.15.5:48960) 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf781102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf7659c700 1 -- 172.21.15.5:0/2283424185 learned_addr learned my addr 172.21.15.5:0/2283424185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 0x7fdf78108b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 -- 172.21.15.5:0/2283424185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 msgr2=0x7fdf78108b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 0x7fdf78108b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 -- 172.21.15.5:0/2283424185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 msgr2=0x7fdf7810fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf75d9b700 1 -- 172.21.15.5:0/2283424185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf60009520 con 0x7fdf78075350 2023-07-21T19:38:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.601+0000 7fdf7659c700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:59.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf781102f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000e9f0 tx=0x7fdf6000ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf6f7fe700 1 -- 172.21.15.5:0/2283424185 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf6000cb70 con 0x7fdf78075350 2023-07-21T19:38:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf6f7fe700 1 -- 172.21.15.5:0/2283424185 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf6000ee90 con 0x7fdf78075350 2023-07-21T19:38:59.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf700095f0 con 0x7fdf78075350 2023-07-21T19:38:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf6f7fe700 1 -- 172.21.15.5:0/2283424185 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf60003bc0 con 0x7fdf78075350 2023-07-21T19:38:59.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.602+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf7810dae0 con 0x7fdf78075350 2023-07-21T19:38:59.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.609+0000 7fdf6f7fe700 1 -- 172.21.15.5:0/2283424185 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fdf60003e20 con 0x7fdf78075350 2023-07-21T19:38:59.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.610+0000 7fdf904d8b80 1 Processor -- start 2023-07-21T19:38:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.610+0000 7fdf904d8b80 1 -- start start 2023-07-21T19:38:59.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92168040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc921689d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9206dc50 con 0x55fc91fc46b0 2023-07-21T19:38:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc920c09e0 con 0x55fc9216e2d0 2023-07-21T19:38:59.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.611+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc920b9e70 con 0x55fc92168580 2023-07-21T19:38:59.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf76d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf76d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56444/0 (socket says 172.21.15.5:56444) 2023-07-21T19:38:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf76d9d700 1 -- 172.21.15.5:0/4007885152 learned_addr learned my addr 172.21.15.5:0/4007885152 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf6f7fe700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 0x7fdf640a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf7659c700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92168040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf6f7fe700 1 -- 172.21.15.5:0/2283424185 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdf60014070 con 0x7fdf78075350 2023-07-21T19:38:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.612+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 0x7fdf640a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc921689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 -- 172.21.15.5:0/4007885152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 msgr2=0x55fc921689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc921689d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 -- 172.21.15.5:0/4007885152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 msgr2=0x55fc9216dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 -- 172.21.15.5:0/4007885152 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc91ee7d80 con 0x55fc9216e2d0 2023-07-21T19:38:59.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:38:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf7659c700 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92168040 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf700045b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb5d401440c28557 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.613+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 0x7fdf640a37b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdf68013fc0 tx=0x7fdf68004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.614+0000 7fdf6cff9700 1 -- 172.21.15.5:0/4007885152 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001d050 con 0x55fc9216e2d0 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.614+0000 7fdf6cff9700 1 -- 172.21.15.5:0/4007885152 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf7001e040 con 0x55fc9216e2d0 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.614+0000 7fdf6cff9700 1 -- 172.21.15.5:0/4007885152 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70004cf0 con 0x55fc9216e2d0 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 -- 172.21.15.5:0/4007885152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 msgr2=0x55fc92168040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92168040 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fdf70000c00 tx=0x7fdf700045b0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.602 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 -- 172.21.15.5:0/4007885152 shutdown_connections 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc921689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc91fc46b0 0x55fc9216dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/4007885152 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92168040 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 -- 172.21.15.5:0/4007885152 >> 172.21.15.5:0/4007885152 conn(0x55fc9215dbe0 msgr2=0x55fc92162f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 -- 172.21.15.5:0/4007885152 shutdown_connections 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.615+0000 7fdf904d8b80 1 -- 172.21.15.5:0/4007885152 wait complete. 2023-07-21T19:38:59.603 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.616+0000 7fdf904d8b80 1 -- start start 2023-07-21T19:38:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.617+0000 7fdf904d8b80 1 Processor -- start 2023-07-21T19:38:59.604 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.617+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.617+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc92168580 0x55fc921718a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc92171f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc92059a60 con 0x55fc9216e2d0 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc92059760 con 0x55fc92168580 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc91f3d8c0 con 0x55fc91fc46b0 2023-07-21T19:38:59.605 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf904d8b80 10 client.0 _ll_register_callbacks cb 0x55fc92057700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52344/0 (socket says 172.21.15.5:52344) 2023-07-21T19:38:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf7659c700 1 -- 172.21.15.5:0/614161385 learned_addr learned my addr 172.21.15.5:0/614161385 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc92171f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.606 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.618+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc92168580 0x55fc921718a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 -- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc92168580 msgr2=0x55fc921718a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc92168580 0x55fc921718a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 -- 172.21.15.5:0/614161385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 msgr2=0x55fc92171f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc92171f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 -- 172.21.15.5:0/614161385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf68013bf0 con 0x55fc91fc46b0 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.619+0000 7fdf7659c700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf70024ad0 tx=0x7fdf70024b00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6e0f6a71b3f09fb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.620+0000 7fdf5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.621+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001d040 con 0x55fc91fc46b0 2023-07-21T19:38:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.621+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdf7000f660 con 0x55fc91fc46b0 2023-07-21T19:38:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.622+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc92013490 con 0x55fc91fc46b0 2023-07-21T19:38:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.622+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7000bed0 con 0x55fc91fc46b0 2023-07-21T19:38:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.622+0000 7fdf904d8b80 10 client.30376 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.622+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fdf78113880 con 0x55fc91fc46b0 2023-07-21T19:38:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.622+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc92013490 con 0x55fc91fc46b0 2023-07-21T19:38:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf7000f820 con 0x55fc91fc46b0 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 10 client.30376 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 10 client.30376 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 20 client.30376 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc92181630 0x55fc92183a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 4 client.30376 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:38:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.624+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fc920b2e50 con 0x55fc92181630 2023-07-21T19:38:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.625+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc92181630 0x55fc92183a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.626+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc92181630 0x55fc92183a10 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dc840ddfd149156f server_cookie=9795ba92efc38e86 in_seq=0 out_seq=0 2023-07-21T19:38:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.626+0000 7fdf5dffb700 10 client.30376 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.627+0000 7fdf5dffb700 1 -- 172.21.15.5:0/614161385 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fc920b2e50 con 0x55fc92181630 2023-07-21T19:38:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.627+0000 7fdf5dffb700 10 client.30376 handle_command_reply: tid=0 2023-07-21T19:38:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 client.30376 shutdown 2023-07-21T19:38:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 msgr2=0x55fc92174620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fdf70024ad0 tx=0x7fdf70024b00 comp rx=0 tx=0).stop 2023-07-21T19:38:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 shutdown_connections 2023-07-21T19:38:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc92181630 0x55fc92183a10 crc :-1 s=READY pgs=560 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc92171f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92174620 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf5dffb700 0 client.30376 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/614161385 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc92168580 0x55fc921718a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 >> 172.21.15.5:0/614161385 conn(0x55fc9215dbe0 msgr2=0x55fc921624e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.628+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 shutdown_connections 2023-07-21T19:38:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.629+0000 7fdf904d8b80 1 -- 172.21.15.5:0/614161385 wait complete. 2023-07-21T19:38:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.629+0000 7fdf904d8b80 20 client.30376 trim_cache size 0 max 16384 2023-07-21T19:38:59.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.658+0000 7fdf904d8b80 1 Processor -- start 2023-07-21T19:38:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.658+0000 7fdf904d8b80 1 -- start start 2023-07-21T19:38:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc92162c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.644 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc921741c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9216d9f0 con 0x55fc9216e2d0 2023-07-21T19:38:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc92172e80 con 0x55fc91fc46b0 2023-07-21T19:38:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc9216c610 con 0x55fc92168580 2023-07-21T19:38:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49000/0 (socket says 172.21.15.5:49000) 2023-07-21T19:38:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf7659c700 1 -- 172.21.15.5:0/3190532739 learned_addr learned my addr 172.21.15.5:0/3190532739 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.659+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc921741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc92162c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 -- 172.21.15.5:0/3190532739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 msgr2=0x55fc92162c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc92162c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 -- 172.21.15.5:0/3190532739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 msgr2=0x55fc921741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc921741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 -- 172.21.15.5:0/3190532739 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fc921885a0 con 0x55fc91fc46b0 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc92162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.660+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdf70024c20 tx=0x7fdf70005fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d39ea1d758b143f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.661+0000 7fdf6d7fa700 1 -- 172.21.15.5:0/3190532739 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7000fcd0 con 0x55fc91fc46b0 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.661+0000 7fdf6d7fa700 1 -- 172.21.15.5:0/3190532739 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fdf7000fe30 con 0x55fc91fc46b0 2023-07-21T19:38:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.661+0000 7fdf6d7fa700 1 -- 172.21.15.5:0/3190532739 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf70004b90 con 0x55fc91fc46b0 2023-07-21T19:38:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3190532739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 msgr2=0x55fc9216d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fdf70024c20 tx=0x7fdf70005fb0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3190532739 shutdown_connections 2023-07-21T19:38:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc9216e2d0 0x55fc921741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc91fc46b0 0x55fc9216d130 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3190532739 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc92168580 0x55fc92162c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3190532739 >> 172.21.15.5:0/3190532739 conn(0x55fc9215dbe0 msgr2=0x55fc9216ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3190532739 shutdown_connections 2023-07-21T19:38:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.662+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3190532739 wait complete. 2023-07-21T19:38:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.663+0000 7fdf904d8b80 1 -- start start 2023-07-21T19:38:59.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.663+0000 7fdf904d8b80 1 Processor -- start 2023-07-21T19:38:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc92168580 0x55fc92176950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92176e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc921d4500 con 0x55fc92168580 2023-07-21T19:38:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc921d4670 con 0x55fc9216e2d0 2023-07-21T19:38:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fc921d4830 con 0x55fc91fc46b0 2023-07-21T19:38:59.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf904d8b80 10 client.0 _ll_register_callbacks cb 0x7fdf70026070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:38:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf7659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52360/0 (socket says 172.21.15.5:52360) 2023-07-21T19:38:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf7659c700 1 -- 172.21.15.5:0/3669013298 learned_addr learned my addr 172.21.15.5:0/3669013298 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:38:59.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.664+0000 7fdf76d9d700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92176e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc92168580 0x55fc92176950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 -- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 msgr2=0x55fc92176e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92176e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 -- 172.21.15.5:0/3669013298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc92168580 msgr2=0x55fc92176950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:38:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc92168580 0x55fc92176950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 -- 172.21.15.5:0/3669013298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf78113880 con 0x55fc91fc46b0 2023-07-21T19:38:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf7659c700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdf70040040 tx=0x7fdf7000f9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee35d3f167079eca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:38:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.665+0000 7fdf5dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:38:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7001d030 con 0x55fc91fc46b0 2023-07-21T19:38:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fdf70000e40 con 0x55fc91fc46b0 2023-07-21T19:38:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fc92074ff0 con 0x55fc91fc46b0 2023-07-21T19:38:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdf7000e480 con 0x55fc91fc46b0 2023-07-21T19:38:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf904d8b80 10 client.30379 fetch_fsmap learned FSMap version 72 2023-07-21T19:38:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.666+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55fc91ed0c00 con 0x55fc91fc46b0 2023-07-21T19:38:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fc92074ff0 con 0x55fc91fc46b0 2023-07-21T19:38:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fdf70044070 con 0x55fc91fc46b0 2023-07-21T19:38:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 10 client.30379 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:38:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 10 client.30379 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:38:59.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 20 client.30379 populate_metadata read hostname 'smithi005' 2023-07-21T19:38:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc921d5a10 0x55fc921d7df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:38:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 4 client.30379 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:38:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.667+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fc91f08f10 con 0x55fc921d5a10 2023-07-21T19:38:59.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.668+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc921d5a10 0x55fc921d7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:38:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.668+0000 7fdf75d9b700 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc921d5a10 0x55fc921d7df0 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a981c1a865b12af server_cookie=5b6798446ede75a8 in_seq=0 out_seq=0 2023-07-21T19:38:59.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.668+0000 7fdf5dffb700 10 client.30379 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf5dffb700 1 -- 172.21.15.5:0/3669013298 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fc91f08f10 con 0x55fc921d5a10 2023-07-21T19:38:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf5dffb700 10 client.30379 handle_command_reply: tid=0 2023-07-21T19:38:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 client.30379 shutdown 2023-07-21T19:38:59.688 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 msgr2=0x55fc92176410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.689 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fdf70040040 tx=0x7fdf7000f9e0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:38:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:38:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:38:59.690 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 shutdown_connections 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fc9216e2d0 0x55fc92176e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fc91fc46b0 0x55fc92176410 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fc921d5a10 0x55fc921d7df0 crc :-1 s=READY pgs=561 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 --2- 172.21.15.5:0/3669013298 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fc92168580 0x55fc92176950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.703+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 >> 172.21.15.5:0/3669013298 conn(0x55fc9215dbe0 msgr2=0x55fc9216d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.691 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.704+0000 7fdf5dffb700 0 client.30379 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.704+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 shutdown_connections 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.704+0000 7fdf904d8b80 1 -- 172.21.15.5:0/3669013298 wait complete. 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.704+0000 7fdf904d8b80 20 client.30379 trim_cache size 0 max 16384 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.705+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 msgr2=0x7fdf640a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.705+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 0x7fdf640a37b0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fdf68013fc0 tx=0x7fdf68004b40 comp rx=0 tx=0).stop 2023-07-21T19:38:59.692 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.705+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 msgr2=0x7fdf781102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.705+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf781102f0 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fdf6000e9f0 tx=0x7fdf6000ed00 comp rx=0 tx=0).stop 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 shutdown_connections 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fdf78110a70 0x7fdf78108b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fdf78074a40 0x7fdf7810fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fdf640a1300 0x7fdf640a37b0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.693 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 --2- 172.21.15.5:0/2283424185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fdf78075350 0x7fdf781102f0 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:38:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.706+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 >> 172.21.15.5:0/2283424185 conn(0x7fdf780fd8d0 msgr2=0x7fdf780ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:38:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.707+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 shutdown_connections 2023-07-21T19:38:59.694 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:38:59.707+0000 7fdf7cbff700 1 -- 172.21.15.5:0/2283424185 wait complete. 2023-07-21T19:39:01.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 750.0 s, 239.6 tps, lat 439.279 ms stddev 154.610 2023-07-21T19:39:02.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:01 smithi101 ceph-mon[128136]: pgmap v996: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 640 op/s 2023-07-21T19:39:02.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:01 smithi005 ceph-mon[126460]: pgmap v996: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 640 op/s 2023-07-21T19:39:02.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:01 smithi156 ceph-mon[128316]: pgmap v996: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.3 MiB/s wr, 640 op/s 2023-07-21T19:39:03.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:39:03.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 -- 172.21.15.5:0/963876080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b81046c0 msgr2=0x7ff0b8104b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/963876080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b81046c0 0x7ff0b8104b30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009950 tx=0x7ff0ac009c60 comp rx=0 tx=0).stop 2023-07-21T19:39:03.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 -- 172.21.15.5:0/963876080 shutdown_connections 2023-07-21T19:39:03.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/963876080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b8105070 0x7ff0b810b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.761 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/963876080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8103a00 0x7ff0b8103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/963876080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b81046c0 0x7ff0b8104b30 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 -- 172.21.15.5:0/963876080 >> 172.21.15.5:0/963876080 conn(0x7ff0b8075b70 msgr2=0x7ff0b8075f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 -- 172.21.15.5:0/963876080 shutdown_connections 2023-07-21T19:39:03.762 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.776+0000 7ff0bcbff700 1 -- 172.21.15.5:0/963876080 wait complete. 2023-07-21T19:39:03.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.778+0000 7ff0bcbff700 1 Processor -- start 2023-07-21T19:39:03.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.778+0000 7ff0bcbff700 1 -- start start 2023-07-21T19:39:03.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 0x7ff0b8197e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.763 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 0x7ff0b81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8105070 0x7ff0b819c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b819ce50 con 0x7ff0b8105070 2023-07-21T19:39:03.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b819cfc0 con 0x7ff0b81046c0 2023-07-21T19:39:03.764 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0bcbff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b819d180 con 0x7ff0b8103a00 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 0x7ff0b8197e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0b5d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 0x7ff0b81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0b6d9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8105070 0x7ff0b819c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 0x7ff0b8197e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52386/0 (socket says 172.21.15.5:52386) 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.779+0000 7ff0b659c700 1 -- 172.21.15.5:0/1912397955 learned_addr learned my addr 172.21.15.5:0/1912397955 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:03.765 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/1912397955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 msgr2=0x7ff0b8197e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 0x7ff0b8197e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8105070 msgr2=0x7ff0b819c810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8105070 0x7ff0b819c810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/1912397955 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0a80095f0 con 0x7ff0b81046c0 2023-07-21T19:39:03.766 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.780+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 0x7ff0b81983b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009230 tx=0x7ff0ac005660 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.781+0000 7ff0a77fe700 1 -- 172.21.15.5:0/1912397955 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac024070 con 0x7ff0b81046c0 2023-07-21T19:39:03.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.781+0000 7ff0a77fe700 1 -- 172.21.15.5:0/1912397955 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0ac005ce0 con 0x7ff0b81046c0 2023-07-21T19:39:03.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.781+0000 7ff0a77fe700 1 -- 172.21.15.5:0/1912397955 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac016920 con 0x7ff0b81046c0 2023-07-21T19:39:03.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.781+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac009520 con 0x7ff0b81046c0 2023-07-21T19:39:03.767 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.781+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b819d640 con 0x7ff0b81046c0 2023-07-21T19:39:03.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.786+0000 7ff0a77fe700 1 -- 172.21.15.5:0/1912397955 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff0ac004ca0 con 0x7ff0b81046c0 2023-07-21T19:39:03.771 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.787+0000 7ff0d05c0b80 1 Processor -- start 2023-07-21T19:39:03.772 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.787+0000 7ff0d05c0b80 1 -- start start 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a7be0 0x564c457bada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c457bd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c456adcb0 con 0x564c457a7be0 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c45700a40 con 0x564c457a72d0 2023-07-21T19:39:03.773 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c456f9ed0 con 0x564c457bb2e0 2023-07-21T19:39:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.788+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a7be0 0x564c457bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.774 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49034/0 (socket says 172.21.15.5:49034) 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 -- 172.21.15.5:0/2424930670 learned_addr learned my addr 172.21.15.5:0/2424930670 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c457bd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 -- 172.21.15.5:0/2424930670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 msgr2=0x564c457bd6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c457bd6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 -- 172.21.15.5:0/2424930670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a7be0 msgr2=0x564c457bada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.775 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a7be0 0x564c457bada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.789+0000 7ff0b6d9d700 1 -- 172.21.15.5:0/2424930670 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c45527de0 con 0x564c457a72d0 2023-07-21T19:39:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0011670 tx=0x7ff0b0009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7f9b4b575a9e6de server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.776 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0a77fe700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 0x7ff0a00a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0a77fe700 1 -- 172.21.15.5:0/1912397955 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff0ac0cdf20 con 0x7ff0b81046c0 2023-07-21T19:39:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0a4ff9700 1 -- 172.21.15.5:0/2424930670 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b000e040 con 0x564c457a72d0 2023-07-21T19:39:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0a4ff9700 1 -- 172.21.15.5:0/2424930670 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0b000f040 con 0x564c457a72d0 2023-07-21T19:39:03.777 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0a4ff9700 1 -- 172.21.15.5:0/2424930670 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0b0004810 con 0x564c457a72d0 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.790+0000 7ff0b659c700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 0x7ff0a00a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/2424930670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 msgr2=0x564c457a76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7ff0b0011670 tx=0x7ff0b0009920 comp rx=0 tx=0).stop 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/2424930670 shutdown_connections 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0b659c700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 0x7ff0a00a37f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8005d10 tx=0x7ff0a8009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c457bd6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.778 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457a76a0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/2424930670 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a7be0 0x564c457bada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.791+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/2424930670 >> 172.21.15.5:0/2424930670 conn(0x564c4579dda0 msgr2=0x564c457a36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.792+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/2424930670 shutdown_connections 2023-07-21T19:39:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.792+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/2424930670 wait complete. 2023-07-21T19:39:03.779 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.792+0000 7ff0d05c0b80 1 -- start start 2023-07-21T19:39:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.793+0000 7ff0d05c0b80 1 Processor -- start 2023-07-21T19:39:03.780 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457bacf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457bb2e0 0x564c457b8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c45699ac0 con 0x564c457bb2e0 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c456997c0 con 0x564c457a72d0 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4557d920 con 0x564c457a7be0 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0d05c0b80 10 client.0 _ll_register_callbacks cb 0x564c45697760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:03.781 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0b5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0b5d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52398/0 (socket says 172.21.15.5:52398) 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/802383348 learned_addr learned my addr 172.21.15.5:0/802383348 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.794+0000 7ff0b659c700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457bacf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 msgr2=0x564c457bacf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457bacf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/802383348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457bb2e0 msgr2=0x564c457b8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457bb2e0 0x564c457b8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.782 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/802383348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0b0011350 con 0x564c457a7be0 2023-07-21T19:39:03.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009920 tx=0x7ff0ac09ac70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee5e476c25d0afdc server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.795+0000 7ff09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:39:03.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac09b130 con 0x564c457a7be0 2023-07-21T19:39:03.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564c456534f0 con 0x564c457a7be0 2023-07-21T19:39:03.783 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff0ac09f160 con 0x564c457a7be0 2023-07-21T19:39:03.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0ac0a34f0 con 0x564c457a7be0 2023-07-21T19:39:03.784 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff0d05c0b80 10 client.30391 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564c45510c60 con 0x564c457a7be0 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.796+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x564c456534f0 con 0x564c457a7be0 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0ac0a3140 con 0x564c457a7be0 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 10 client.30391 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 10 client.30391 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 20 client.30391 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:03.785 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c457ca830 0x564c457ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 4 client.30391 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564c456f2eb0 con 0x564c457ca830 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.797+0000 7ff0b659c700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c457ca830 0x564c457ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.798+0000 7ff0b659c700 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c457ca830 0x564c457ccc10 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11ebd30c5b2abb26 server_cookie=2495cd80bb76d178 in_seq=0 out_seq=0 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.798+0000 7ff09dffb700 10 client.30391 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff09dffb700 1 -- 172.21.15.5:0/802383348 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x564c456f2eb0 con 0x564c457ca830 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff09dffb700 10 client.30391 handle_command_reply: tid=0 2023-07-21T19:39:03.786 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 client.30391 shutdown 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 msgr2=0x564c457b7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009920 tx=0x7ff0ac09ac70 comp rx=0 tx=0).stop 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 shutdown_connections 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c457ca830 0x564c457ccc10 crc :-1 s=READY pgs=562 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457bb2e0 0x564c457b8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff09dffb700 0 client.30391 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a72d0 0x564c457bacf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.787 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/802383348 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a7be0 0x564c457b7fe0 secure :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009920 tx=0x7ff0ac09ac70 comp rx=0 tx=0).stop 2023-07-21T19:39:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.799+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 >> 172.21.15.5:0/802383348 conn(0x564c4579dda0 msgr2=0x564c457a3230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.800+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 shutdown_connections 2023-07-21T19:39:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.800+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/802383348 wait complete. 2023-07-21T19:39:03.788 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.800+0000 7ff0d05c0b80 20 client.30391 trim_cache size 0 max 16384 2023-07-21T19:39:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.824+0000 7ff0d05c0b80 1 Processor -- start 2023-07-21T19:39:03.809 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 -- start start 2023-07-21T19:39:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457bb2e0 0x564c457c6f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a38c0 0x564c457ba920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c457a8540 con 0x564c457a72d0 2023-07-21T19:39:03.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c457b9550 con 0x564c457bb2e0 2023-07-21T19:39:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.825+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4579afa0 con 0x564c457a38c0 2023-07-21T19:39:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56524/0 (socket says 172.21.15.5:56524) 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 -- 172.21.15.5:0/1275503764 learned_addr learned my addr 172.21.15.5:0/1275503764 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457bb2e0 0x564c457c6f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a38c0 0x564c457ba920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 -- 172.21.15.5:0/1275503764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a38c0 msgr2=0x564c457ba920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a38c0 0x564c457ba920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 -- 172.21.15.5:0/1275503764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457bb2e0 msgr2=0x564c457c6f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.812 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457bb2e0 0x564c457c6f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.826+0000 7ff0b659c700 1 -- 172.21.15.5:0/1275503764 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564c4554e250 con 0x564c457a72d0 2023-07-21T19:39:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.827+0000 7ff0b659c700 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x564c457cd520 tx=0x7ff0a8017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74307ffee7863c23 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.827+0000 7ff0a57fa700 1 -- 172.21.15.5:0/1275503764 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8025040 con 0x564c457a72d0 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.827+0000 7ff0a57fa700 1 -- 172.21.15.5:0/1275503764 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0a8004440 con 0x564c457a72d0 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.827+0000 7ff0a57fa700 1 -- 172.21.15.5:0/1275503764 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a8017070 con 0x564c457a72d0 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/1275503764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 msgr2=0x564c457b77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x564c457cd520 tx=0x7ff0a8017000 comp rx=0 tx=0).stop 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/1275503764 shutdown_connections 2023-07-21T19:39:03.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457a38c0 0x564c457ba920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457b77a0 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/1275503764 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457bb2e0 0x564c457c6f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/1275503764 >> 172.21.15.5:0/1275503764 conn(0x564c4579dda0 msgr2=0x564c457d1570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/1275503764 shutdown_connections 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.828+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/1275503764 wait complete. 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.829+0000 7ff0d05c0b80 1 -- start start 2023-07-21T19:39:03.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.829+0000 7ff0d05c0b80 1 Processor -- start 2023-07-21T19:39:03.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 0x564c457c0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457c1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c4581e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4581ec10 con 0x564c457a72d0 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4581ed50 con 0x564c457a38c0 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564c4581ef10 con 0x564c457bb2e0 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0d05c0b80 10 client.0 _ll_register_callbacks cb 0x7ff0ac097490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 0x564c457c0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0b659c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 0x564c457c0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:49064/0 (socket says 172.21.15.5:49064) 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0b659c700 1 -- 172.21.15.5:0/4605069 learned_addr learned my addr 172.21.15.5:0/4605069 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c4581e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.830+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/4605069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 msgr2=0x564c4581e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c4581e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 msgr2=0x564c457c0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 0x564c457c0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 -- 172.21.15.5:0/4605069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0a802a950 con 0x564c457a72d0 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b6d9d700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c4581e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff0b5d9b700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457c1080 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x564c457b9070 tx=0x7ff0ac0d14c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b10fb19c368a9ff server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.831+0000 7ff09dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0ac09a820 con 0x564c457a72d0 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(139..139 src has 1..139) v4 ==== 8513+0+0 (secure 0 0 0) 0x7ff0ac09b130 con 0x564c457a72d0 2023-07-21T19:39:03.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff0b000f040 con 0x564c457a72d0 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff0ac0a34f0 con 0x564c457a72d0 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff0d05c0b80 10 client.20691 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.832+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x564c455262b0 con 0x564c457a72d0 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff0ac0cb020 con 0x564c457a72d0 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff0ac09a090 con 0x564c457a72d0 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 10 client.20691 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:03.821 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 10 client.20691 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 20 client.20691 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c45820170 0x564c45822550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 4 client.20691 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564c456b5050 con 0x564c45820170 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.833+0000 7ff0b659c700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c45820170 0x564c45822550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.834+0000 7ff09dffb700 10 client.20691 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:03.822 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.834+0000 7ff0b659c700 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c45820170 0x564c45822550 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3bfb1d82d6b151b1 server_cookie=3c458e7884039974 in_seq=0 out_seq=0 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.834+0000 7ff09dffb700 1 -- 172.21.15.5:0/4605069 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564c456b5050 con 0x564c45820170 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.834+0000 7ff09dffb700 10 client.20691 handle_command_reply: tid=0 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.834+0000 7ff0d05c0b80 1 client.20691 shutdown 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 msgr2=0x564c457c1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457c1080 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x564c457b9070 tx=0x7ff0ac0d14c0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 shutdown_connections 2023-07-21T19:39:03.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564c457bb2e0 0x564c4581e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564c457a38c0 0x564c457c0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564c45820170 0x564c45822550 crc :-1 s=READY pgs=563 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 --2- 172.21.15.5:0/4605069 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564c457a72d0 0x564c457c1080 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 >> 172.21.15.5:0/4605069 conn(0x564c4579dda0 msgr2=0x564c457c6f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff09dffb700 0 client.20691 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.835+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 shutdown_connections 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.836+0000 7ff0d05c0b80 1 -- 172.21.15.5:0/4605069 wait complete. 2023-07-21T19:39:03.824 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.836+0000 7ff0d05c0b80 20 client.20691 trim_cache size 0 max 16384 2023-07-21T19:39:03.825 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:39:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:39:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:39:03.825 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:39:03.826 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:39:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 msgr2=0x7ff0a00a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 0x7ff0a00a37f0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff0a8005d10 tx=0x7ff0a8009310 comp rx=0 tx=0).stop 2023-07-21T19:39:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 msgr2=0x7ff0b81983b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 0x7ff0b81983b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7ff0ac009230 tx=0x7ff0ac005660 comp rx=0 tx=0).stop 2023-07-21T19:39:03.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 shutdown_connections 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff0a00a1340 0x7ff0a00a37f0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff0b8105070 0x7ff0b819c810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff0b8103a00 0x7ff0b8197e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 --2- 172.21.15.5:0/1912397955 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff0b81046c0 0x7ff0b81983b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 >> 172.21.15.5:0/1912397955 conn(0x7ff0b8075b70 msgr2=0x7ff0b80fc330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 shutdown_connections 2023-07-21T19:39:03.828 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:03.837+0000 7ff0bcbff700 1 -- 172.21.15.5:0/1912397955 wait complete. 2023-07-21T19:39:03.848 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:03 smithi005 ceph-mon[126460]: pgmap v997: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:39:03.852 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:39:04.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:03 smithi101 ceph-mon[128136]: pgmap v997: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:39:04.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:03 smithi156 ceph-mon[128316]: pgmap v997: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 10 MiB/s wr, 1.03k op/s 2023-07-21T19:39:06.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:05 smithi101 ceph-mon[128136]: pgmap v998: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:06.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:05 smithi005 ceph-mon[126460]: pgmap v998: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:06.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:05 smithi156 ceph-mon[128316]: pgmap v998: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 755.0 s, 188.0 tps, lat 504.505 ms stddev 168.872 2023-07-21T19:39:08.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:07 smithi101 ceph-mon[128136]: pgmap v999: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:07 smithi101 ceph-mon[128136]: osdmap e140: 12 total, 12 up, 12 in 2023-07-21T19:39:08.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:07 smithi005 ceph-mon[126460]: pgmap v999: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:08.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:07 smithi005 ceph-mon[126460]: osdmap e140: 12 total, 12 up, 12 in 2023-07-21T19:39:08.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:07 smithi156 ceph-mon[128316]: pgmap v999: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.0 MiB/s wr, 722 op/s 2023-07-21T19:39:08.158 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:07 smithi156 ceph-mon[128316]: osdmap e140: 12 total, 12 up, 12 in 2023-07-21T19:39:09.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:08 smithi101 ceph-mon[128136]: pgmap v1001: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:09.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:08 smithi005 ceph-mon[126460]: pgmap v1001: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:09.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:08 smithi156 ceph-mon[128316]: pgmap v1001: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:11.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 760.0 s, 82.6 tps, lat 1262.663 ms stddev 897.263 2023-07-21T19:39:12.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:11 smithi101 ceph-mon[128136]: pgmap v1002: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:12.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:11 smithi005 ceph-mon[126460]: pgmap v1002: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:12.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:11 smithi156 ceph-mon[128316]: pgmap v1002: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 7.0 MiB/s wr, 729 op/s 2023-07-21T19:39:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 -- 172.21.15.5:0/1703352867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 msgr2=0x7ff308072c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/1703352867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308072c20 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ff304009990 tx=0x7ff304009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 -- 172.21.15.5:0/1703352867 shutdown_connections 2023-07-21T19:39:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/1703352867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30810e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.931 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/1703352867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308072c20 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/1703352867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff30810be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 -- 172.21.15.5:0/1703352867 >> 172.21.15.5:0/1703352867 conn(0x7ff3080fb5f0 msgr2=0x7ff3080fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 -- 172.21.15.5:0/1703352867 shutdown_connections 2023-07-21T19:39:12.932 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.946+0000 7ff30f5ff700 1 -- 172.21.15.5:0/1703352867 wait complete. 2023-07-21T19:39:12.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.947+0000 7ff30f5ff700 1 Processor -- start 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.947+0000 7ff30f5ff700 1 -- start start 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff308198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30819cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819d1d0 con 0x7ff308073160 2023-07-21T19:39:12.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819d340 con 0x7ff308072850 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30f5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30819d500 con 0x7ff30810c3b0 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff308198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:36028/0 (socket says 172.21.15.5:36028) 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30d39b700 1 -- 172.21.15.5:0/199897826 learned_addr learned my addr 172.21.15.5:0/199897826 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff308198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56566/0 (socket says 172.21.15.5:56566) 2023-07-21T19:39:12.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30db9c700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30819cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30d39b700 1 -- 172.21.15.5:0/199897826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 msgr2=0x7ff30819cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.948+0000 7ff30d39b700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30819cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30d39b700 1 -- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 msgr2=0x7ff308198520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30d39b700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff308198520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30d39b700 1 -- 172.21.15.5:0/199897826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040095f0 con 0x7ff308072850 2023-07-21T19:39:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30db9c700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30d39b700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff304005740 tx=0x7ff304005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff2fe7fc700 1 -- 172.21.15.5:0/199897826 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30401d070 con 0x7ff308072850 2023-07-21T19:39:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff2fe7fc700 1 -- 172.21.15.5:0/199897826 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff304022470 con 0x7ff308072850 2023-07-21T19:39:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff30819d780 con 0x7ff308072850 2023-07-21T19:39:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.949+0000 7ff2fe7fc700 1 -- 172.21.15.5:0/199897826 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff304017650 con 0x7ff308072850 2023-07-21T19:39:12.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.950+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081a5020 con 0x7ff308072850 2023-07-21T19:39:12.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.953+0000 7ff2fe7fc700 1 -- 172.21.15.5:0/199897826 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7ff3040225e0 con 0x7ff308072850 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.955+0000 7ff322ec7b80 1 Processor -- start 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.955+0000 7ff322ec7b80 1 -- start start 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.955+0000 7ff2fe7fc700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 0x7ff2f40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff2fe7fc700 1 -- 172.21.15.5:0/199897826 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff3040ce2c0 con 0x7ff308072850 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 0x55da75c221a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1e3f0 0x55da75c226e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75b1ac50 con 0x55da75c1e3f0 2023-07-21T19:39:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75b6d9e0 con 0x55da75c1df80 2023-07-21T19:39:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75b66e70 con 0x55da75a716b0 2023-07-21T19:39:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.942 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57086/0 (socket says 172.21.15.5:57086) 2023-07-21T19:39:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30db9c700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 0x7ff2f40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30d39b700 1 -- 172.21.15.5:0/342503164 learned_addr learned my addr 172.21.15.5:0/342503164 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 0x55da75c221a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.956+0000 7ff30db9c700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1e3f0 0x55da75c226e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 -- 172.21.15.5:0/342503164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 msgr2=0x55da75c1da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 -- 172.21.15.5:0/342503164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1e3f0 msgr2=0x55da75c226e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1e3f0 0x55da75c226e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 -- 172.21.15.5:0/342503164 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da75994d80 con 0x55da75c1df80 2023-07-21T19:39:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30d39b700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 0x55da75c221a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009c80 tx=0x7ff2f8009f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ac0326a7d5b3e232 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff30db9c700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 0x7ff2f40a37f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff300009550 tx=0x7ff3000090f0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff2f37fe700 1 -- 172.21.15.5:0/342503164 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8011cb0 con 0x55da75c1df80 2023-07-21T19:39:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff2f37fe700 1 -- 172.21.15.5:0/342503164 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2f8011e10 con 0x55da75c1df80 2023-07-21T19:39:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.957+0000 7ff2f37fe700 1 -- 172.21.15.5:0/342503164 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8005280 con 0x55da75c1df80 2023-07-21T19:39:12.947 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 -- 172.21.15.5:0/342503164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 msgr2=0x55da75c221a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 0x55da75c221a0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8009c80 tx=0x7ff2f8009f90 comp rx=0 tx=0).stop 2023-07-21T19:39:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 -- 172.21.15.5:0/342503164 shutdown_connections 2023-07-21T19:39:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1e3f0 0x55da75c226e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c1da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/342503164 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1df80 0x55da75c221a0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.958+0000 7ff322ec7b80 1 -- 172.21.15.5:0/342503164 >> 172.21.15.5:0/342503164 conn(0x55da75c0abe0 msgr2=0x55da75c0ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.959+0000 7ff322ec7b80 1 -- 172.21.15.5:0/342503164 shutdown_connections 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.959+0000 7ff322ec7b80 1 -- 172.21.15.5:0/342503164 wait complete. 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.959+0000 7ff322ec7b80 1 -- start start 2023-07-21T19:39:12.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.959+0000 7ff322ec7b80 1 Processor -- start 2023-07-21T19:39:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 0x55da75c29d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 0x55da75c276d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75b06a60 con 0x55da75c1df80 2023-07-21T19:39:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75b06760 con 0x55da75a716b0 2023-07-21T19:39:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da759ea8c0 con 0x55da75c1e3f0 2023-07-21T19:39:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.960+0000 7ff322ec7b80 10 client.0 _ll_register_callbacks cb 0x55da75b04700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56596/0 (socket says 172.21.15.5:56596) 2023-07-21T19:39:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 0x55da75c29d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30cb9a700 1 -- 172.21.15.5:0/2214023613 learned_addr learned my addr 172.21.15.5:0/2214023613 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30db9c700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 0x55da75c276d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 -- 172.21.15.5:0/2214023613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 msgr2=0x55da75c276d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 0x55da75c276d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 -- 172.21.15.5:0/2214023613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 msgr2=0x55da75c27110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 -- 172.21.15.5:0/2214023613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3081a5020 con 0x55da75a716b0 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30db9c700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 0x55da75c276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff30d39b700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 0x55da75c29d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff304097e10 tx=0x7ff30409aee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16bae1cfb3930f18 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.961+0000 7ff2f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:39:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30409b3a0 con 0x55da75a716b0 2023-07-21T19:39:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff30409f3d0 con 0x55da75a716b0 2023-07-21T19:39:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da75ac0490 con 0x55da75a716b0 2023-07-21T19:39:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff3040a3760 con 0x55da75a716b0 2023-07-21T19:39:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff322ec7b80 10 client.30467 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7ff2f80098e0 con 0x55da75a716b0 2023-07-21T19:39:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55da75ac0490 con 0x55da75a716b0 2023-07-21T19:39:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff3040a32d0 con 0x55da75a716b0 2023-07-21T19:39:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff322ec7b80 10 client.30467 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.962+0000 7ff322ec7b80 10 client.30467 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.963+0000 7ff322ec7b80 20 client.30467 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.963+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c36fd0 0x55da75c393e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.963+0000 7ff322ec7b80 4 client.30467 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.963+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da75b5fe50 con 0x55da75c36fd0 2023-07-21T19:39:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.963+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c36fd0 0x55da75c393e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.964+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c36fd0 0x55da75c393e0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53948994ab581ec1 server_cookie=3f8910eb81482b0b in_seq=0 out_seq=0 2023-07-21T19:39:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.964+0000 7ff2f0ff9700 10 client.30467 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.964+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2214023613 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55da75b5fe50 con 0x55da75c36fd0 2023-07-21T19:39:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.964+0000 7ff2f0ff9700 10 client.30467 handle_command_reply: tid=0 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.964+0000 7ff322ec7b80 1 client.30467 shutdown 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 msgr2=0x55da75c29d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 0x55da75c29d80 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7ff304097e10 tx=0x7ff30409aee0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 shutdown_connections 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75c1e3f0 0x55da75c276d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75a716b0 0x55da75c29d80 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c36fd0 0x55da75c393e0 crc :-1 s=READY pgs=564 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2214023613 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c27110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 >> 172.21.15.5:0/2214023613 conn(0x55da75c0abe0 msgr2=0x55da75c0f4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff2f0ff9700 0 client.30467 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 shutdown_connections 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2214023613 wait complete. 2023-07-21T19:39:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.965+0000 7ff322ec7b80 20 client.30467 trim_cache size 0 max 16384 2023-07-21T19:39:12.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.989+0000 7ff322ec7b80 1 Processor -- start 2023-07-21T19:39:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.990+0000 7ff322ec7b80 1 -- start start 2023-07-21T19:39:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.990+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.991+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c2e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.991+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c14740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.991+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c217a0 con 0x55da75c1df80 2023-07-21T19:39:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.991+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c1ea20 con 0x55da75c1e3f0 2023-07-21T19:39:12.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.991+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c280a0 con 0x55da75a716b0 2023-07-21T19:39:12.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.992+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.992+0000 7ff30d39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57098/0 (socket says 172.21.15.5:57098) 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.992+0000 7ff30d39b700 1 -- 172.21.15.5:0/1431762237 learned_addr learned my addr 172.21.15.5:0/1431762237 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.993+0000 7ff30d39b700 1 -- 172.21.15.5:0/1431762237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 msgr2=0x55da75c14740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.993+0000 7ff30d39b700 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c14740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.993+0000 7ff30d39b700 1 -- 172.21.15.5:0/1431762237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 msgr2=0x55da75c2e500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.993+0000 7ff30d39b700 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c2e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.993+0000 7ff30d39b700 1 -- 172.21.15.5:0/1431762237 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da7597dc00 con 0x55da75a716b0 2023-07-21T19:39:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.994+0000 7ff30d39b700 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a3230 tx=0x7ff304099950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=42910e5b78498c2 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.994+0000 7ff2f3fff700 1 -- 172.21.15.5:0/1431762237 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff30409a890 con 0x55da75a716b0 2023-07-21T19:39:12.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.994+0000 7ff2f3fff700 1 -- 172.21.15.5:0/1431762237 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff30409f3d0 con 0x55da75a716b0 2023-07-21T19:39:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.994+0000 7ff2f3fff700 1 -- 172.21.15.5:0/1431762237 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3040a10d0 con 0x55da75a716b0 2023-07-21T19:39:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.995+0000 7ff322ec7b80 1 -- 172.21.15.5:0/1431762237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 msgr2=0x55da75c225e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:12.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.995+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7ff3040a3230 tx=0x7ff304099950 comp rx=0 tx=0).stop 2023-07-21T19:39:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.996+0000 7ff322ec7b80 1 -- 172.21.15.5:0/1431762237 shutdown_connections 2023-07-21T19:39:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.996+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c14740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.996+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c225e0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.996+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/1431762237 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c2e500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.996+0000 7ff322ec7b80 1 -- 172.21.15.5:0/1431762237 >> 172.21.15.5:0/1431762237 conn(0x55da75c0abe0 msgr2=0x55da75c14030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.997+0000 7ff322ec7b80 1 -- 172.21.15.5:0/1431762237 shutdown_connections 2023-07-21T19:39:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.997+0000 7ff322ec7b80 1 -- 172.21.15.5:0/1431762237 wait complete. 2023-07-21T19:39:12.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.998+0000 7ff322ec7b80 1 -- start start 2023-07-21T19:39:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.998+0000 7ff322ec7b80 1 Processor -- start 2023-07-21T19:39:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c2e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c25e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c263d0 con 0x55da75c1df80 2023-07-21T19:39:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c26540 con 0x55da75c1e3f0 2023-07-21T19:39:12.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da75c26700 con 0x55da75a716b0 2023-07-21T19:39:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:12.999+0000 7ff322ec7b80 10 client.0 _ll_register_callbacks cb 0x7ff3040860b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:12.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.000+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.000+0000 7ff30cb9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56614/0 (socket says 172.21.15.5:56614) 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.000+0000 7ff30cb9a700 1 -- 172.21.15.5:0/2374750277 learned_addr learned my addr 172.21.15.5:0/2374750277 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.001+0000 7ff30cb9a700 1 -- 172.21.15.5:0/2374750277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 msgr2=0x55da75c2e510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.001+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c2e510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.001+0000 7ff30cb9a700 1 -- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 msgr2=0x55da75c25e00 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.001+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c25e00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:12.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.001+0000 7ff30cb9a700 1 -- 172.21.15.5:0/2374750277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80098e0 con 0x55da75c1df80 2023-07-21T19:39:12.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.002+0000 7ff30cb9a700 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x55da75c28650 tx=0x7ff2f8004b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b099e7a9ce2d912 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:12.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.006+0000 7ff2f0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:39:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.006+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8029b60 con 0x55da75c1df80 2023-07-21T19:39:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.006+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 8557+0+0 (secure 0 0 0) 0x7ff2f8029cc0 con 0x55da75c1df80 2023-07-21T19:39:12.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.006+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff2f8011e10 con 0x55da75c1df80 2023-07-21T19:39:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2f8029ea0 con 0x55da75c1df80 2023-07-21T19:39:12.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 10 client.20706 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55da759bb1f0 con 0x55da75c1df80 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7ff2f8011e10 con 0x55da75c1df80 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7ff2f8031070 con 0x55da75c1df80 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 10 client.20706 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 10 client.20706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 20 client.20706 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:12.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c9c5f0 0x55da75c9ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 4 client.20706 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:39:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.007+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55da75b21ff0 con 0x55da75c9c5f0 2023-07-21T19:39:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.008+0000 7ff30d39b700 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c9c5f0 0x55da75c9ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:12.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.010+0000 7ff30d39b700 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c9c5f0 0x55da75c9ea00 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce2c6d6ed4dad30f server_cookie=6b9fb6e8d6c5d0fe in_seq=0 out_seq=0 2023-07-21T19:39:12.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.010+0000 7ff2f0ff9700 10 client.20706 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.051+0000 7ff2f0ff9700 1 -- 172.21.15.5:0/2374750277 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55da75b21ff0 con 0x55da75c9c5f0 2023-07-21T19:39:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.051+0000 7ff2f0ff9700 10 client.20706 handle_command_reply: tid=0 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.051+0000 7ff322ec7b80 1 client.20706 shutdown 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 msgr2=0x55da75c258c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x55da75c28650 tx=0x7ff2f8004b70 comp rx=0 tx=0).stop 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 shutdown_connections 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55da75c1e3f0 0x55da75c25e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55da75a716b0 0x55da75c2e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55da75c9c5f0 0x55da75c9ea00 crc :-1 s=READY pgs=565 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 --2- 172.21.15.5:0/2374750277 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55da75c1df80 0x55da75c258c0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.052+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 >> 172.21.15.5:0/2374750277 conn(0x55da75c0abe0 msgr2=0x55da75c21d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.053+0000 7ff2f0ff9700 0 client.20706 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.054+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 shutdown_connections 2023-07-21T19:39:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.054+0000 7ff322ec7b80 1 -- 172.21.15.5:0/2374750277 wait complete. 2023-07-21T19:39:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.054+0000 7ff322ec7b80 20 client.20706 trim_cache size 0 max 16384 2023-07-21T19:39:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:39:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:39:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:39:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:39:13.041 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 msgr2=0x7ff2f40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 0x7ff2f40a37f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7ff300009550 tx=0x7ff3000090f0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 msgr2=0x7ff308197fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7ff304005740 tx=0x7ff304005770 comp rx=0 tx=0).stop 2023-07-21T19:39:13.042 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 shutdown_connections 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7ff30810c3b0 0x7ff30819cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7ff308072850 0x7ff308197fe0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7ff2f40a1340 0x7ff2f40a37f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 --2- 172.21.15.5:0/199897826 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7ff308073160 0x7ff308198520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.055+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 >> 172.21.15.5:0/199897826 conn(0x7ff3080fb5f0 msgr2=0x7ff30810e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.056+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 shutdown_connections 2023-07-21T19:39:13.043 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:13.056+0000 7ff30f5ff700 1 -- 172.21.15.5:0/199897826 wait complete. 2023-07-21T19:39:14.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:13 smithi101 ceph-mon[128136]: pgmap v1003: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-07-21T19:39:14.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:13 smithi005 ceph-mon[126460]: pgmap v1003: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-07-21T19:39:14.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:13 smithi156 ceph-mon[128316]: pgmap v1003: 97 pgs: 6 active+clean+snaptrim_wait, 2 active+clean+snaptrim, 89 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.0 MiB/s wr, 400 op/s 2023-07-21T19:39:16.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:15 smithi101 ceph-mon[128136]: pgmap v1004: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:16.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:15 smithi005 ceph-mon[126460]: pgmap v1004: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:16.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:15 smithi156 ceph-mon[128316]: pgmap v1004: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:16.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 765.0 s, 207.8 tps, lat 481.425 ms stddev 100.842 2023-07-21T19:39:18.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:17 smithi101 ceph-mon[128136]: pgmap v1005: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:18.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:17 smithi005 ceph-mon[126460]: pgmap v1005: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:18.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:17 smithi156 ceph-mon[128316]: pgmap v1005: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 4.1 MiB/s wr, 421 op/s 2023-07-21T19:39:19.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:18 smithi101 ceph-mon[128136]: osdmap e141: 12 total, 12 up, 12 in 2023-07-21T19:39:19.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:18 smithi101 ceph-mon[128136]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:19.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:18 smithi005 ceph-mon[126460]: osdmap e141: 12 total, 12 up, 12 in 2023-07-21T19:39:19.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:18 smithi005 ceph-mon[126460]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:19.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:18 smithi156 ceph-mon[128316]: osdmap e141: 12 total, 12 up, 12 in 2023-07-21T19:39:19.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:18 smithi156 ceph-mon[128316]: pgmap v1007: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:21.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 770.0 s, 221.6 tps, lat 451.211 ms stddev 162.034 2023-07-21T19:39:22.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:21 smithi101 ceph-mon[128136]: pgmap v1008: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:22.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:21 smithi005 ceph-mon[126460]: pgmap v1008: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:22.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:21 smithi156 ceph-mon[128316]: pgmap v1008: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:24.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:23 smithi101 ceph-mon[128136]: pgmap v1009: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.7 MiB/s wr, 818 op/s 2023-07-21T19:39:24.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:23 smithi005 ceph-mon[126460]: pgmap v1009: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.7 MiB/s wr, 818 op/s 2023-07-21T19:39:24.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:23 smithi156 ceph-mon[128316]: pgmap v1009: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.7 MiB/s wr, 818 op/s 2023-07-21T19:39:26.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:25 smithi101 ceph-mon[128136]: pgmap v1010: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:26.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:25 smithi005 ceph-mon[126460]: pgmap v1010: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:26.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:25 smithi156 ceph-mon[128316]: pgmap v1010: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 -- 172.21.15.5:0/1475176406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c067ee0 msgr2=0x7f463c068350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 --2- 172.21.15.5:0/1475176406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c067ee0 0x7f463c068350 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f4630009990 tx=0x7f4630009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 -- 172.21.15.5:0/1475176406 shutdown_connections 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 --2- 172.21.15.5:0/1475176406 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c1024f0 0x7f463c102940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 --2- 172.21.15.5:0/1475176406 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c104a60 0x7f463c0679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 --2- 172.21.15.5:0/1475176406 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c067ee0 0x7f463c068350 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 -- 172.21.15.5:0/1475176406 >> 172.21.15.5:0/1475176406 conn(0x7f463c074760 msgr2=0x7f463c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 -- 172.21.15.5:0/1475176406 shutdown_connections 2023-07-21T19:39:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.293+0000 7f4640bff700 1 -- 172.21.15.5:0/1475176406 wait complete. 2023-07-21T19:39:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.295+0000 7f4640bff700 1 Processor -- start 2023-07-21T19:39:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.295+0000 7f4640bff700 1 -- start start 2023-07-21T19:39:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 0x7f463c19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c104a60 0x7f463c1a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1a12d0 con 0x7f463c067ee0 2023-07-21T19:39:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1a1440 con 0x7f463c104a60 2023-07-21T19:39:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4640bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f463c1a1600 con 0x7f463c1024f0 2023-07-21T19:39:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42868/0 (socket says 172.21.15.5:42868) 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f4639d9b700 1 -- 172.21.15.5:0/1706252693 learned_addr learned my addr 172.21.15.5:0/1706252693 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.296+0000 7f463ad9d700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c104a60 0x7f463c1a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 -- 172.21.15.5:0/1706252693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c104a60 msgr2=0x7f463c1a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c104a60 0x7f463c1a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 -- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 msgr2=0x7f463c19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f463a59c700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 0x7f463c19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 0x7f463c19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 -- 172.21.15.5:0/1706252693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46300095f0 con 0x7f463c1024f0 2023-07-21T19:39:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f463a59c700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 0x7f463c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:39:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4639d9b700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4630005740 tx=0x7f4630005770 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f462b7fe700 1 -- 172.21.15.5:0/1706252693 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463001d070 con 0x7f463c1024f0 2023-07-21T19:39:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f462b7fe700 1 -- 172.21.15.5:0/1706252693 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4630022470 con 0x7f463c1024f0 2023-07-21T19:39:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f462b7fe700 1 -- 172.21.15.5:0/1706252693 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4630017650 con 0x7f463c1024f0 2023-07-21T19:39:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.297+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f463c1a1880 con 0x7f463c1024f0 2023-07-21T19:39:26.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.298+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f463c1a9120 con 0x7f463c1024f0 2023-07-21T19:39:26.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.303+0000 7f462b7fe700 1 -- 172.21.15.5:0/1706252693 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f463000fb00 con 0x7f463c1024f0 2023-07-21T19:39:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.304+0000 7f46545bbb80 1 Processor -- start 2023-07-21T19:39:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 -- start start 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56210232d6b0 0x5621024d6d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d1580 0x5621024d19d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621023d6c50 con 0x5621024d1580 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621024299e0 con 0x56210232d6b0 2023-07-21T19:39:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.305+0000 7f46545bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562102422e70 con 0x5621024d72d0 2023-07-21T19:39:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f463a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d1580 0x5621024d19d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f463a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42870/0 (socket says 172.21.15.5:42870) 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f463a59c700 1 -- 172.21.15.5:0/3830928199 learned_addr learned my addr 172.21.15.5:0/3830928199 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f462b7fe700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 0x7f46240a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f463ad9d700 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56210232d6b0 0x5621024d6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.306+0000 7f462b7fe700 1 -- 172.21.15.5:0/1706252693 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f46300ce210 con 0x7f463c1024f0 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463ad9d700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 0x7f46240a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 -- 172.21.15.5:0/3830928199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56210232d6b0 msgr2=0x5621024d6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56210232d6b0 0x5621024d6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 -- 172.21.15.5:0/3830928199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d1580 msgr2=0x5621024d19d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d1580 0x5621024d19d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 -- 172.21.15.5:0/3830928199 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562102250d80 con 0x5621024d72d0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.307+0000 7f463a59c700 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f462c009cc0 tx=0x7f462c010370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=46a688a8a4130946 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.308+0000 7f463ad9d700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 0x7f46240a3870 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4634009ec0 tx=0x7f4634004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.308+0000 7f4628ff9700 1 -- 172.21.15.5:0/3830928199 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c004230 con 0x5621024d72d0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.308+0000 7f4628ff9700 1 -- 172.21.15.5:0/3830928199 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462c010d90 con 0x5621024d72d0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.308+0000 7f4628ff9700 1 -- 172.21.15.5:0/3830928199 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c003ce0 con 0x5621024d72d0 2023-07-21T19:39:26.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 -- 172.21.15.5:0/3830928199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 msgr2=0x5621024d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f462c009cc0 tx=0x7f462c010370 comp rx=0 tx=0).stop 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 -- 172.21.15.5:0/3830928199 shutdown_connections 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d1580 0x5621024d19d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56210232d6b0 0x5621024d6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3830928199 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d72d0 0x5621024d1040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 -- 172.21.15.5:0/3830928199 >> 172.21.15.5:0/3830928199 conn(0x5621024c6be0 msgr2=0x5621024cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 -- 172.21.15.5:0/3830928199 shutdown_connections 2023-07-21T19:39:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.309+0000 7f46545bbb80 1 -- 172.21.15.5:0/3830928199 wait complete. 2023-07-21T19:39:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.310+0000 7f46545bbb80 1 -- start start 2023-07-21T19:39:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 Processor -- start 2023-07-21T19:39:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024db020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621023c2a60 con 0x56210232d6b0 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621023c2760 con 0x5621024d72d0 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621022a68c0 con 0x5621024d1580 2023-07-21T19:39:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.311+0000 7f46545bbb80 10 client.0 _ll_register_callbacks cb 0x5621023c0700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f463a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f463a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:49500/0 (socket says 172.21.15.5:49500) 2023-07-21T19:39:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f463a59c700 1 -- 172.21.15.5:0/2242336939 learned_addr learned my addr 172.21.15.5:0/2242336939 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:42876/0 (socket says 172.21.15.5:42876) 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.312+0000 7f463ad9d700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024db020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f463a59c700 1 -- 172.21.15.5:0/2242336939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 msgr2=0x5621024da8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f463a59c700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f463a59c700 1 -- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 msgr2=0x5621024db020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f463a59c700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024db020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f463a59c700 1 -- 172.21.15.5:0/2242336939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f462c009920 con 0x56210232d6b0 2023-07-21T19:39:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.313+0000 7f4639d9b700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.314+0000 7f463a59c700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f462c0036c0 tx=0x7f462c012660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf32ca8a425d296 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.314+0000 7f4621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:39:26.299 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.314+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f462c005ce0 con 0x56210232d6b0 2023-07-21T19:39:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.314+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f462c016070 con 0x56210232d6b0 2023-07-21T19:39:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.314+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56210237c490 con 0x56210232d6b0 2023-07-21T19:39:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.315+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f462c003720 con 0x56210232d6b0 2023-07-21T19:39:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.315+0000 7f46545bbb80 10 client.20715 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:26.300 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.316+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x562102239c00 con 0x56210232d6b0 2023-07-21T19:39:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.316+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56210237c490 con 0x56210232d6b0 2023-07-21T19:39:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f462c004860 con 0x56210232d6b0 2023-07-21T19:39:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 10 client.20715 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 10 client.20715 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:26.301 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 20 client.20715 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621024eaae0 0x5621024ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 4 client.20715 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.317+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56210241be50 con 0x5621024eaae0 2023-07-21T19:39:26.302 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.318+0000 7f4639d9b700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621024eaae0 0x5621024ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.329+0000 7f4639d9b700 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621024eaae0 0x5621024ecef0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2a441d31142c1d server_cookie=ecf30075d77e494c in_seq=0 out_seq=0 2023-07-21T19:39:26.313 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.330+0000 7f4621ffb700 10 client.20715 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.330+0000 7f4621ffb700 1 -- 172.21.15.5:0/2242336939 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56210241be50 con 0x5621024eaae0 2023-07-21T19:39:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.331+0000 7f4621ffb700 10 client.20715 handle_command_reply: tid=0 2023-07-21T19:39:26.314 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.331+0000 7f46545bbb80 1 client.20715 shutdown 2023-07-21T19:39:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.331+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 msgr2=0x5621024dd660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.315 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.331+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f462c0036c0 tx=0x7f462c012660 comp rx=0 tx=0).stop 2023-07-21T19:39:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 shutdown_connections 2023-07-21T19:39:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024db020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5621024eaae0 0x5621024ecef0 crc :-1 s=READY pgs=566 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024dd660 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.316 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 --2- 172.21.15.5:0/2242336939 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 >> 172.21.15.5:0/2242336939 conn(0x5621024c6be0 msgr2=0x5621024cb4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.332+0000 7f4621ffb700 0 client.20715 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:26.317 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.333+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 shutdown_connections 2023-07-21T19:39:26.320 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.337+0000 7f46545bbb80 1 -- 172.21.15.5:0/2242336939 wait complete. 2023-07-21T19:39:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.344+0000 7f46545bbb80 20 client.20715 trim_cache size 0 max 16384 2023-07-21T19:39:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 Processor -- start 2023-07-21T19:39:26.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 -- start start 2023-07-21T19:39:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024cbc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621024c3bb0 con 0x56210232d6b0 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621024db8c0 con 0x5621024d72d0 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.379+0000 7f46545bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621024d76a0 con 0x5621024d1580 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f463a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f463a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55366/0 (socket says 172.21.15.5:55366) 2023-07-21T19:39:26.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f463ad9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024cbc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f463a59c700 1 -- 172.21.15.5:0/22800779 learned_addr learned my addr 172.21.15.5:0/22800779 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:26.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f4639d9b700 1 -- 172.21.15.5:0/22800779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 msgr2=0x5621024cbc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f4639d9b700 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024cbc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f4639d9b700 1 -- 172.21.15.5:0/22800779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 msgr2=0x5621024d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.380+0000 7f4639d9b700 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f4639d9b700 1 -- 172.21.15.5:0/22800779 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621024f1ae0 con 0x5621024d1580 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f463a59c700 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f463ad9d700 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:26.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f4639d9b700 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f46300cb900 tx=0x7f4630009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d831e98809037b0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f46297fa700 1 -- 172.21.15.5:0/22800779 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463009ae50 con 0x5621024d1580 2023-07-21T19:39:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f46297fa700 1 -- 172.21.15.5:0/22800779 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f463009ac70 con 0x5621024d1580 2023-07-21T19:39:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.381+0000 7f46297fa700 1 -- 172.21.15.5:0/22800779 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46300225e0 con 0x5621024d1580 2023-07-21T19:39:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 -- 172.21.15.5:0/22800779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 msgr2=0x5621024da890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da890 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f46300cb900 tx=0x7f4630009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 -- 172.21.15.5:0/22800779 shutdown_connections 2023-07-21T19:39:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d72d0 0x5621024cbc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56210232d6b0 0x5621024d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 --2- 172.21.15.5:0/22800779 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5621024d1580 0x5621024da890 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.382+0000 7f46545bbb80 1 -- 172.21.15.5:0/22800779 >> 172.21.15.5:0/22800779 conn(0x5621024c6be0 msgr2=0x5621024da180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.383+0000 7f46545bbb80 1 -- 172.21.15.5:0/22800779 shutdown_connections 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.383+0000 7f46545bbb80 1 -- 172.21.15.5:0/22800779 wait complete. 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.383+0000 7f46545bbb80 1 -- start start 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 Processor -- start 2023-07-21T19:39:26.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56210232d6b0 0x5621024dfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d72d0 0x5621024e0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621024e22b0 con 0x5621024d72d0 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210253d960 con 0x5621024d1580 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.384+0000 7f46545bbb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56210253db20 con 0x56210232d6b0 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f46545bbb80 10 client.0 _ll_register_callbacks cb 0x7f4634009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f463a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56210232d6b0 0x5621024dfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35730/0 (socket says 172.21.15.5:35730) 2023-07-21T19:39:26.371 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 -- 172.21.15.5:0/3068669996 learned_addr learned my addr 172.21.15.5:0/3068669996 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f463ad9d700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d72d0 0x5621024e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 -- 172.21.15.5:0/3068669996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56210232d6b0 msgr2=0x5621024dfc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.372 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56210232d6b0 0x5621024dfc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 -- 172.21.15.5:0/3068669996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d72d0 msgr2=0x5621024e0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.385+0000 7f4639d9b700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d72d0 0x5621024e0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f4639d9b700 1 -- 172.21.15.5:0/3068669996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562102239c00 con 0x5621024d1580 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f4639d9b700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f46300cb8d0 tx=0x7f463009b1a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfff16f708801c2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f4621ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f463009f170 con 0x5621024d1580 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f463009b310 con 0x5621024d1580 2023-07-21T19:39:26.373 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.386+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f462c010d90 con 0x5621024d1580 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f463009ae50 con 0x5621024d1580 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 10 client.30485 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f463c1a9120 con 0x5621024d1580 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f462c010d90 con 0x5621024d1580 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f463009a800 con 0x5621024d1580 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 10 client.30485 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:26.374 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 10 client.30485 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 20 client.30485 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56210253eea0 0x5621025412b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 4 client.30485 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.387+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5621023ddff0 con 0x56210253eea0 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.388+0000 7f463a59c700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56210253eea0 0x5621025412b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.388+0000 7f4621ffb700 10 client.30485 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:26.375 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.388+0000 7f463a59c700 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56210253eea0 0x5621025412b0 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d99730b8c3a63029 server_cookie=b9a1266a9f40a788 in_seq=0 out_seq=0 2023-07-21T19:39:26.409 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f4621ffb700 1 -- 172.21.15.5:0/3068669996 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5621023ddff0 con 0x56210253eea0 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f4621ffb700 10 client.30485 handle_command_reply: tid=0 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 client.30485 shutdown 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 msgr2=0x5621024e01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f46300cb8d0 tx=0x7f463009b1a0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 shutdown_connections 2023-07-21T19:39:26.410 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56210253eea0 0x5621025412b0 crc :-1 s=READY pgs=567 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5621024d72d0 0x5621024e0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56210232d6b0 0x5621024dfc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f4621ffb700 0 client.30485 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 --2- 172.21.15.5:0/3068669996 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5621024d1580 0x5621024e01d0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.411 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.426+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 >> 172.21.15.5:0/3068669996 conn(0x5621024c6be0 msgr2=0x5621024d5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.427+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 shutdown_connections 2023-07-21T19:39:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.427+0000 7f46545bbb80 1 -- 172.21.15.5:0/3068669996 wait complete. 2023-07-21T19:39:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.427+0000 7f46545bbb80 20 client.30485 trim_cache size 0 max 16384 2023-07-21T19:39:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:39:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:39:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:39:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:39:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.427+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 msgr2=0x7f46240a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.427+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 0x7f46240a3870 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f4634009ec0 tx=0x7f4634004b40 comp rx=0 tx=0).stop 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 msgr2=0x7f463c19c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f4630005740 tx=0x7f4630005770 comp rx=0 tx=0).stop 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 shutdown_connections 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f463c104a60 0x7f463c1a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46240a13c0 0x7f46240a3870 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f463c067ee0 0x7f463c19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 --2- 172.21.15.5:0/1706252693 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f463c1024f0 0x7f463c19c620 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 >> 172.21.15.5:0/1706252693 conn(0x7f463c074760 msgr2=0x7f463c1014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 shutdown_connections 2023-07-21T19:39:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:26.428+0000 7f4640bff700 1 -- 172.21.15.5:0/1706252693 wait complete. 2023-07-21T19:39:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 775.0 s, 229.8 tps, lat 438.157 ms stddev 143.793 2023-07-21T19:39:28.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:27 smithi101 ceph-mon[128136]: pgmap v1011: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:28.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:27 smithi005 ceph-mon[126460]: pgmap v1011: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:28.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:27 smithi156 ceph-mon[128316]: pgmap v1011: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.8 MiB/s wr, 843 op/s 2023-07-21T19:39:29.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:28 smithi101 ceph-mon[128136]: pgmap v1012: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 790 op/s 2023-07-21T19:39:29.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:28 smithi005 ceph-mon[126460]: pgmap v1012: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 790 op/s 2023-07-21T19:39:29.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:28 smithi156 ceph-mon[128316]: pgmap v1012: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.5 MiB/s wr, 790 op/s 2023-07-21T19:39:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 780.0 s, 232.2 tps, lat 422.411 ms stddev 86.374 2023-07-21T19:39:32.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:31 smithi101 ceph-mon[128136]: pgmap v1013: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 706 op/s 2023-07-21T19:39:32.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:31 smithi005 ceph-mon[126460]: pgmap v1013: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 706 op/s 2023-07-21T19:39:32.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:31 smithi156 ceph-mon[128316]: pgmap v1013: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.7 MiB/s wr, 706 op/s 2023-07-21T19:39:33.854 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:39:34.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:33 smithi101 ceph-mon[128136]: pgmap v1014: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.3 MiB/s wr, 973 op/s 2023-07-21T19:39:34.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:33 smithi005 ceph-mon[126460]: pgmap v1014: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.3 MiB/s wr, 973 op/s 2023-07-21T19:39:34.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 -- 172.21.15.5:0/3832890949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba00fe9d0 msgr2=0x7f5ba00fee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/3832890949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba00fee40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009990 tx=0x7f5b94009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 -- 172.21.15.5:0/3832890949 shutdown_connections 2023-07-21T19:39:34.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/3832890949 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 0x7f5ba010b810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/3832890949 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba01049e0 0x7f5ba0104db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/3832890949 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba00fee40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 -- 172.21.15.5:0/3832890949 >> 172.21.15.5:0/3832890949 conn(0x7f5ba00fa4b0 msgr2=0x7f5ba00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 -- 172.21.15.5:0/3832890949 shutdown_connections 2023-07-21T19:39:34.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.169+0000 7f5ba4dff700 1 -- 172.21.15.5:0/3832890949 wait complete. 2023-07-21T19:39:34.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:33 smithi156 ceph-mon[128316]: pgmap v1014: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 9.3 MiB/s wr, 973 op/s 2023-07-21T19:39:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.170+0000 7f5ba4dff700 1 Processor -- start 2023-07-21T19:39:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.170+0000 7f5ba4dff700 1 -- start start 2023-07-21T19:39:34.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba0197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 0x7f5ba01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 0x7f5ba019c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019cd50 con 0x7f5ba00fe9d0 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019cec0 con 0x7f5ba00ff380 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5ba4dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ba019d080 con 0x7f5ba01049e0 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.171+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba0197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 0x7f5ba019c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba0197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55386/0 (socket says 172.21.15.5:55386) 2023-07-21T19:39:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9e59c700 1 -- 172.21.15.5:0/790030278 learned_addr learned my addr 172.21.15.5:0/790030278 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:34.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 0x7f5ba01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 -- 172.21.15.5:0/790030278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 msgr2=0x7f5ba01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 0x7f5ba01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 -- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 msgr2=0x7f5ba0197d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba0197d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 -- 172.21.15.5:0/790030278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b940095f0 con 0x7f5ba01049e0 2023-07-21T19:39:34.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.172+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 0x7f5ba019c710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800efc0 tx=0x7f5b9800c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.173+0000 7f5b8f7fe700 1 -- 172.21.15.5:0/790030278 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b98018070 con 0x7f5ba01049e0 2023-07-21T19:39:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.173+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ba019d300 con 0x7f5ba01049e0 2023-07-21T19:39:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.173+0000 7f5b8f7fe700 1 -- 172.21.15.5:0/790030278 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b9800f040 con 0x7f5ba01049e0 2023-07-21T19:39:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.173+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01a4ba0 con 0x7f5ba01049e0 2023-07-21T19:39:34.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.173+0000 7f5b8f7fe700 1 -- 172.21.15.5:0/790030278 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9800cba0 con 0x7f5ba01049e0 2023-07-21T19:39:34.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.177+0000 7f5b8f7fe700 1 -- 172.21.15.5:0/790030278 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5b98003a00 con 0x7f5ba01049e0 2023-07-21T19:39:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.178+0000 7f5bb8780b80 1 Processor -- start 2023-07-21T19:39:34.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 -- start start 2023-07-21T19:39:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5b8f7fe700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 0x7f5b880a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5b8f7fe700 1 -- 172.21.15.5:0/790030278 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5b980cc6b0 con 0x7f5ba01049e0 2023-07-21T19:39:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd41f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.164 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd42ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd325cb0 con 0x5578bd42aad0 2023-07-21T19:39:34.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd378a40 con 0x5578bd41fbe0 2023-07-21T19:39:34.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.179+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd371ed0 con 0x5578bd41f2d0 2023-07-21T19:39:34.166 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35756/0 (socket says 172.21.15.5:35756) 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd42ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/1041971355 learned_addr learned my addr 172.21.15.5:0/1041971355 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 0x7f5b880a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd41f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/1041971355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 msgr2=0x5578bd41f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd41f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.167 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/1041971355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 msgr2=0x5578bd42ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd42ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/1041971355 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578bd19fde0 con 0x5578bd41fbe0 2023-07-21T19:39:34.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.180+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd42ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:34.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.181+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000c00 tx=0x7f5b9400bc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e15c37c4553523f4 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.168 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.181+0000 7f5b8cff9700 1 -- 172.21.15.5:0/1041971355 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401d070 con 0x5578bd41fbe0 2023-07-21T19:39:34.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.181+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 0x7f5b880a37f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009990 tx=0x7f5b94024000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.182+0000 7f5b8cff9700 1 -- 172.21.15.5:0/1041971355 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b94005a00 con 0x5578bd41fbe0 2023-07-21T19:39:34.169 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.182+0000 7f5b8cff9700 1 -- 172.21.15.5:0/1041971355 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b94004a90 con 0x5578bd41fbe0 2023-07-21T19:39:34.170 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.182+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1041971355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 msgr2=0x5578bd42a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.182+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5b94000c00 tx=0x7f5b9400bc60 comp rx=0 tx=0).stop 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1041971355 shutdown_connections 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd42ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd41f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1041971355 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd42a590 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1041971355 >> 172.21.15.5:0/1041971355 conn(0x5578bd415da0 msgr2=0x5578bd41b6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.171 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1041971355 shutdown_connections 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.183+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1041971355 wait complete. 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.184+0000 7f5bb8780b80 1 -- start start 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.184+0000 7f5bb8780b80 1 Processor -- start 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd42a490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd427d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.172 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd311ac0 con 0x5578bd42aad0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd3117c0 con 0x5578bd41fbe0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd1f5920 con 0x5578bd41f2d0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5bb8780b80 10 client.0 _ll_register_callbacks cb 0x5578bd30f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd427d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35768/0 (socket says 172.21.15.5:35768) 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/3018342060 learned_addr learned my addr 172.21.15.5:0/3018342060 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:34.173 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd427d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:55410/0 (socket says 172.21.15.5:55410) 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd42a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/3018342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 msgr2=0x5578bd42a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd42a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/3018342060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 msgr2=0x5578bd427d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd427d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.185+0000 7f5b9dd9b700 1 -- 172.21.15.5:0/3018342060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba01a4ba0 con 0x5578bd41fbe0 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.186+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b940098e0 tx=0x7f5b94004bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=219e6a23481b3ce6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.174 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.186+0000 7f5b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:39:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.186+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b9401d040 con 0x5578bd41fbe0 2023-07-21T19:39:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.186+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5b94004d00 con 0x5578bd41fbe0 2023-07-21T19:39:34.175 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.186+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5578bd2cb4f0 con 0x5578bd41fbe0 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b94004580 con 0x5578bd41fbe0 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 10 client.30497 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f5b94020040 con 0x5578bd41fbe0 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5578bd2cb4f0 con 0x5578bd41fbe0 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b94032070 con 0x5578bd41fbe0 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 10 client.30497 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:34.177 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 10 client.30497 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 20 client.30497 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd43a010 0x5578bd43c420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 4 client.30497 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.187+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5578bd36aeb0 con 0x5578bd43a010 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.188+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd43a010 0x5578bd43c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.189+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd43a010 0x5578bd43c420 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e921690f7c6f0e40 server_cookie=c50863f53f969e6a in_seq=0 out_seq=0 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.189+0000 7f5b85ffb700 10 client.30497 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:34.178 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5b85ffb700 1 -- 172.21.15.5:0/3018342060 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5578bd36aeb0 con 0x5578bd43a010 2023-07-21T19:39:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5b85ffb700 10 client.30497 handle_command_reply: tid=0 2023-07-21T19:39:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 client.30497 shutdown 2023-07-21T19:39:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 msgr2=0x5578bd427710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f5b940098e0 tx=0x7f5b94004bd0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.179 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 shutdown_connections 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd43a010 0x5578bd43c420 crc :-1 s=READY pgs=568 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd427d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd42a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/3018342060 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd427710 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5b85ffb700 0 client.30497 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.190+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 >> 172.21.15.5:0/3018342060 conn(0x5578bd415da0 msgr2=0x5578bd41b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.180 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.191+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 shutdown_connections 2023-07-21T19:39:34.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.191+0000 7f5bb8780b80 1 -- 172.21.15.5:0/3018342060 wait complete. 2023-07-21T19:39:34.181 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.191+0000 7f5bb8780b80 20 client.30497 trim_cache size 0 max 16384 2023-07-21T19:39:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 Processor -- start 2023-07-21T19:39:34.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 -- start start 2023-07-21T19:39:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd422f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd426ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd42a860 con 0x5578bd42aad0 2023-07-21T19:39:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd43ca80 con 0x5578bd41fbe0 2023-07-21T19:39:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd425c70 con 0x5578bd41f2d0 2023-07-21T19:39:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.216+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:52272/0 (socket says 172.21.15.5:52272) 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 -- 172.21.15.5:0/2170496746 learned_addr learned my addr 172.21.15.5:0/2170496746 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd422f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd426ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 -- 172.21.15.5:0/2170496746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 msgr2=0x5578bd422f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd422f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 -- 172.21.15.5:0/2170496746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 msgr2=0x5578bd426ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd426ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 -- 172.21.15.5:0/2170496746 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5578bd188c60 con 0x5578bd41f2d0 2023-07-21T19:39:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd426ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:39:34.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.217+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5578bd43ccf0 tx=0x7f5b90004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cb1fc5687936ea8d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.218+0000 7f5b8d7fa700 1 -- 172.21.15.5:0/2170496746 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90016030 con 0x5578bd41f2d0 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.218+0000 7f5b8d7fa700 1 -- 172.21.15.5:0/2170496746 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b90017040 con 0x5578bd41f2d0 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.218+0000 7f5b8d7fa700 1 -- 172.21.15.5:0/2170496746 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b900044b0 con 0x5578bd41f2d0 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 -- 172.21.15.5:0/2170496746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 msgr2=0x5578bd4229f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x5578bd43ccf0 tx=0x7f5b90004750 comp rx=0 tx=0).stop 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 -- 172.21.15.5:0/2170496746 shutdown_connections 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd42aad0 0x5578bd426ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd41f2d0 0x5578bd4229f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/2170496746 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41fbe0 0x5578bd422f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 -- 172.21.15.5:0/2170496746 >> 172.21.15.5:0/2170496746 conn(0x5578bd415da0 msgr2=0x5578bd42a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 -- 172.21.15.5:0/2170496746 shutdown_connections 2023-07-21T19:39:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.219+0000 7f5bb8780b80 1 -- 172.21.15.5:0/2170496746 wait complete. 2023-07-21T19:39:34.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.220+0000 7f5bb8780b80 1 -- start start 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.220+0000 7f5bb8780b80 1 Processor -- start 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.220+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.220+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 0x5578bd42f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.220+0000 7f5bb8780b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd42aad0 0x5578bd42fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd4304c0 con 0x5578bd41fbe0 2023-07-21T19:39:34.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd430630 con 0x5578bd41f2d0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5bb8780b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5578bd48dea0 con 0x5578bd42aad0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5bb8780b80 10 client.0 _ll_register_callbacks cb 0x7f5b980954b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35784/0 (socket says 172.21.15.5:35784) 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 -- 172.21.15.5:0/1545718866 learned_addr learned my addr 172.21.15.5:0/1545718866 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9ed9d700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd42aad0 0x5578bd42fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 0x5578bd42f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.215 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 -- 172.21.15.5:0/1545718866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd42aad0 msgr2=0x5578bd42fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd42aad0 0x5578bd42fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 -- 172.21.15.5:0/1545718866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 msgr2=0x5578bd42f870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 0x5578bd42f870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9e59c700 1 -- 172.21.15.5:0/1545718866 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ba019d300 con 0x5578bd41f2d0 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.221+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 0x5578bd42f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5b9e59c700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b90005cb0 tx=0x7f5b90011bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5fc5062d814ff8b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:34.216 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5b85ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90016030 con 0x5578bd41f2d0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5b9001fe40 con 0x5578bd41f2d0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5b94005a00 con 0x5578bd41f2d0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.222+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b90011e60 con 0x5578bd41f2d0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 10 client.30503 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7f5b9001fab0 con 0x5578bd41f2d0 2023-07-21T19:39:34.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5b94005a00 con 0x5578bd41f2d0 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f5b9001d3f0 con 0x5578bd41f2d0 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 10 client.30503 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 10 client.30503 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 20 client.30503 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd48f430 0x5578bd491840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 4 client.30503 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:39:34.218 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5578bd32d050 con 0x5578bd48f430 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.223+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd48f430 0x5578bd491840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.224+0000 7f5b9dd9b700 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd48f430 0x5578bd491840 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e5a083a583cb132 server_cookie=f3dfbdef7ce24205 in_seq=0 out_seq=0 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.224+0000 7f5b85ffb700 10 client.30503 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5b85ffb700 1 -- 172.21.15.5:0/1545718866 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5578bd32d050 con 0x5578bd48f430 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5b85ffb700 10 client.30503 handle_command_reply: tid=0 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 client.30503 shutdown 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 msgr2=0x5578bd431540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.219 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f5b90005cb0 tx=0x7f5b90011bc0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 shutdown_connections 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5578bd42aad0 0x5578bd42fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5578bd41f2d0 0x5578bd431540 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5578bd48f430 0x5578bd491840 crc :-1 s=READY pgs=569 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 --2- 172.21.15.5:0/1545718866 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5578bd41fbe0 0x5578bd42f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5b85ffb700 0 client.30503 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:34.220 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 >> 172.21.15.5:0/1545718866 conn(0x5578bd415da0 msgr2=0x5578bd4212e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 shutdown_connections 2023-07-21T19:39:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.225+0000 7f5bb8780b80 1 -- 172.21.15.5:0/1545718866 wait complete. 2023-07-21T19:39:34.221 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5bb8780b80 20 client.30503 trim_cache size 0 max 16384 2023-07-21T19:39:34.221 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:39:34.222 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:39:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:39:34.223 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:39:34.223 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:39:34.223 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 msgr2=0x7f5b880a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 0x7f5b880a37f0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f5b94009990 tx=0x7f5b94024000 comp rx=0 tx=0).stop 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 msgr2=0x7f5ba019c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 0x7f5ba019c710 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f5b9800efc0 tx=0x7f5b9800c3a0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 shutdown_connections 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5ba01049e0 0x7f5ba019c710 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.224 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f5b880a1340 0x7f5b880a37f0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f5ba00fe9d0 0x7f5ba0197d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 --2- 172.21.15.5:0/790030278 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f5ba00ff380 0x7f5ba01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.226+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 >> 172.21.15.5:0/790030278 conn(0x7f5ba00fa4b0 msgr2=0x7f5ba00fbaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.227+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 shutdown_connections 2023-07-21T19:39:34.225 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:39:34.227+0000 7f5ba4dff700 1 -- 172.21.15.5:0/790030278 wait complete. 2023-07-21T19:39:34.233 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:39:36.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:35 smithi101 ceph-mon[128136]: pgmap v1015: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 684 op/s 2023-07-21T19:39:36.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:35 smithi005 ceph-mon[126460]: pgmap v1015: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 684 op/s 2023-07-21T19:39:36.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:35 smithi156 ceph-mon[128316]: pgmap v1015: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 684 op/s 2023-07-21T19:39:36.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 785.0 s, 232.4 tps, lat 423.942 ms stddev 108.836 2023-07-21T19:39:38.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:37 smithi101 ceph-mon[128136]: pgmap v1016: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 645 op/s 2023-07-21T19:39:38.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:37 smithi005 ceph-mon[126460]: pgmap v1016: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 645 op/s 2023-07-21T19:39:38.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:37 smithi156 ceph-mon[128316]: pgmap v1016: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 645 op/s 2023-07-21T19:39:39.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:38 smithi101 ceph-mon[128136]: pgmap v1017: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.0 MiB/s wr, 931 op/s 2023-07-21T19:39:39.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:38 smithi005 ceph-mon[126460]: pgmap v1017: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.0 MiB/s wr, 931 op/s 2023-07-21T19:39:39.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:38 smithi156 ceph-mon[128316]: pgmap v1017: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 9.0 MiB/s wr, 931 op/s 2023-07-21T19:39:39.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 -- 172.21.15.5:0/2656486218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 msgr2=0x7f3618114d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 --2- 172.21.15.5:0/2656486218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f3618114d10 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f361000b0a0 tx=0x7f361000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 -- 172.21.15.5:0/2656486218 shutdown_connections 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 --2- 172.21.15.5:0/2656486218 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f3618114d10 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 --2- 172.21.15.5:0/2656486218 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 0x7f361806c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 --2- 172.21.15.5:0/2656486218 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f361806cd70 0x7f361806d1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 -- 172.21.15.5:0/2656486218 >> 172.21.15.5:0/2656486218 conn(0x7f361806bd70 msgr2=0x7f3618070f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 -- 172.21.15.5:0/2656486218 shutdown_connections 2023-07-21T19:39:39.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.661+0000 7f36201ff700 1 -- 172.21.15.5:0/2656486218 wait complete. 2023-07-21T19:39:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.663+0000 7f36201ff700 1 Processor -- start 2023-07-21T19:39:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.663+0000 7f36201ff700 1 -- start start 2023-07-21T19:39:39.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f361806cd70 0x7f361819e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 0x7f361819e900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181a35b0 con 0x7f361806cd70 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181a3720 con 0x7f3618108210 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f36201ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36181a38e0 con 0x7f3618112930 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f361e79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f361e79c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47862/0 (socket says 172.21.15.5:47862) 2023-07-21T19:39:39.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f361e79c700 1 -- 172.21.15.5:0/268270838 learned_addr learned my addr 172.21.15.5:0/268270838 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.664+0000 7f361d79a700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 0x7f361819e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.665+0000 7f361e79c700 1 -- 172.21.15.5:0/268270838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 msgr2=0x7f361819e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.665+0000 7f361e79c700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 0x7f361819e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.665+0000 7f361e79c700 1 -- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f361806cd70 msgr2=0x7f361819e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.665+0000 7f361e79c700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f361806cd70 0x7f361819e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.665+0000 7f361e79c700 1 -- 172.21.15.5:0/268270838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3610009d00 con 0x7f3618112930 2023-07-21T19:39:39.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f361e79c700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f361000d010 tx=0x7f361000bd90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f360effd700 1 -- 172.21.15.5:0/268270838 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f361000e040 con 0x7f3618112930 2023-07-21T19:39:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f360effd700 1 -- 172.21.15.5:0/268270838 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3610003e10 con 0x7f3618112930 2023-07-21T19:39:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f360effd700 1 -- 172.21.15.5:0/268270838 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36100052a0 con 0x7f3618112930 2023-07-21T19:39:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36181a3b60 con 0x7f3618112930 2023-07-21T19:39:39.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.666+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3618078950 con 0x7f3618112930 2023-07-21T19:39:39.657 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.672+0000 7f360effd700 1 -- 172.21.15.5:0/268270838 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3610005400 con 0x7f3618112930 2023-07-21T19:39:39.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.674+0000 7f3633a57b80 1 Processor -- start 2023-07-21T19:39:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.675+0000 7f3633a57b80 1 -- start start 2023-07-21T19:39:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f65e76b0 0x5596f678ede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f679fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6690c50 con 0x5596f65e76b0 2023-07-21T19:39:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f66e39e0 con 0x5596f678b040 2023-07-21T19:39:39.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.676+0000 7f3633a57b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f66dce70 con 0x5596f678b9f0 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f360effd700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 0x7f36040a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55578/0 (socket says 172.21.15.5:55578) 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 -- 172.21.15.5:0/246831417 learned_addr learned my addr 172.21.15.5:0/246831417 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f360effd700 1 -- 172.21.15.5:0/268270838 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f36100cdef0 con 0x7f3618112930 2023-07-21T19:39:39.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361d79a700 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f65e76b0 0x5596f678ede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 -- 172.21.15.5:0/246831417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 msgr2=0x5596f679fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f679fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 -- 172.21.15.5:0/246831417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f65e76b0 msgr2=0x5596f678ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.677+0000 7f361df9b700 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f65e76b0 0x5596f678ede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.678+0000 7f361df9b700 1 -- 172.21.15.5:0/246831417 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f650ad80 con 0x5596f678b040 2023-07-21T19:39:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.678+0000 7f361df9b700 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3608009cd0 tx=0x7f360800e350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=53a5964669c4e047 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.679+0000 7f3603fff700 1 -- 172.21.15.5:0/246831417 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608009820 con 0x5596f678b040 2023-07-21T19:39:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.679+0000 7f3603fff700 1 -- 172.21.15.5:0/246831417 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3608004cc0 con 0x5596f678b040 2023-07-21T19:39:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.679+0000 7f3603fff700 1 -- 172.21.15.5:0/246831417 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080056f0 con 0x5596f678b040 2023-07-21T19:39:39.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.679+0000 7f361e79c700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 0x7f36040a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.680+0000 7f3633a57b80 1 -- 172.21.15.5:0/246831417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 msgr2=0x5596f678b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.680+0000 7f3633a57b80 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f3608009cd0 tx=0x7f360800e350 comp rx=0 tx=0).stop 2023-07-21T19:39:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.680+0000 7f361e79c700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 0x7f36040a3870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f361009ac50 tx=0x7f361000b3b0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 -- 172.21.15.5:0/246831417 shutdown_connections 2023-07-21T19:39:39.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f679fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f65e76b0 0x5596f678ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 --2- 172.21.15.5:0/246831417 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f678b040 0x5596f678b4b0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 -- 172.21.15.5:0/246831417 >> 172.21.15.5:0/246831417 conn(0x5596f6780be0 msgr2=0x5596f6785f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 -- 172.21.15.5:0/246831417 shutdown_connections 2023-07-21T19:39:39.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.681+0000 7f3633a57b80 1 -- 172.21.15.5:0/246831417 wait complete. 2023-07-21T19:39:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.682+0000 7f3633a57b80 1 -- start start 2023-07-21T19:39:39.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.682+0000 7f3633a57b80 1 Processor -- start 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f679fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b9f0 0x5596f679d580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f667ca60 con 0x5596f678b9f0 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f667c760 con 0x5596f65e76b0 2023-07-21T19:39:39.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f65608c0 con 0x5596f678b040 2023-07-21T19:39:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.683+0000 7f3633a57b80 10 client.0 _ll_register_callbacks cb 0x5596f667a700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:39.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:47880/0 (socket says 172.21.15.5:47880) 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 -- 172.21.15.5:0/3847575791 learned_addr learned my addr 172.21.15.5:0/3847575791 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361d79a700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f679fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361d79a700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b9f0 0x5596f679d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 -- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 msgr2=0x5596f679fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.670 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f679fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 -- 172.21.15.5:0/3847575791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b9f0 msgr2=0x5596f679d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b9f0 0x5596f679d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361df9b700 1 -- 172.21.15.5:0/3847575791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3608009980 con 0x5596f678b040 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.684+0000 7f361d79a700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f679fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f361df9b700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3608009ca0 tx=0x7f3608005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5337162bbe4648ad server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f36017fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3608009820 con 0x5596f678b040 2023-07-21T19:39:39.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596f6636490 con 0x5596f678b040 2023-07-21T19:39:39.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3608014070 con 0x5596f678b040 2023-07-21T19:39:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3608005130 con 0x5596f678b040 2023-07-21T19:39:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f3633a57b80 10 client.30457 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.685+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5596f64f3c00 con 0x5596f678b040 2023-07-21T19:39:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3608030020 con 0x5596f678b040 2023-07-21T19:39:39.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3608003680 con 0x5596f678b040 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 10 client.30457 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 10 client.30457 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 20 client.30457 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f67ad280 0x5596f67af690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 4 client.30457 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596f66d5e50 con 0x5596f67ad280 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.686+0000 7f361d79a700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f67ad280 0x5596f67af690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.687+0000 7f361d79a700 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f67ad280 0x5596f67af690 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bfe849fb570428f server_cookie=2eb16258c49a6a46 in_seq=0 out_seq=0 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.687+0000 7f36017fa700 10 client.30457 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f36017fa700 1 -- 172.21.15.5:0/3847575791 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596f66d5e50 con 0x5596f67ad280 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f36017fa700 10 client.30457 handle_command_reply: tid=0 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f3633a57b80 1 client.30457 shutdown 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 msgr2=0x5596f679ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f3608009ca0 tx=0x7f3608005650 comp rx=0 tx=0).stop 2023-07-21T19:39:39.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 shutdown_connections 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.688+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f67ad280 0x5596f67af690 crc :-1 s=READY pgs=570 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b9f0 0x5596f679d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f679fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3847575791 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b040 0x5596f679ce40 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 >> 172.21.15.5:0/3847575791 conn(0x5596f6780be0 msgr2=0x5596f67854e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f36017fa700 0 client.30457 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 shutdown_connections 2023-07-21T19:39:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 1 -- 172.21.15.5:0/3847575791 wait complete. 2023-07-21T19:39:39.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.689+0000 7f3633a57b80 20 client.30457 trim_cache size 0 max 16384 2023-07-21T19:39:39.696 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.712+0000 7f3633a57b80 1 Processor -- start 2023-07-21T19:39:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.712+0000 7f3633a57b80 1 -- start start 2023-07-21T19:39:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.712+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a8bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f678a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.697 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f3633a57b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6787890 con 0x5596f678b040 2023-07-21T19:39:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f3633a57b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f677dc00 con 0x5596f65e76b0 2023-07-21T19:39:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f3633a57b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f678c1e0 con 0x5596f678b9f0 2023-07-21T19:39:39.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f361d79a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f361d79a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55596/0 (socket says 172.21.15.5:55596) 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f361d79a700 1 -- 172.21.15.5:0/3246576382 learned_addr learned my addr 172.21.15.5:0/3246576382 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f361d79a700 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f678a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.713+0000 7f361df9b700 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a8bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 -- 172.21.15.5:0/3246576382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 msgr2=0x5596f678a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f678a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 -- 172.21.15.5:0/3246576382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 msgr2=0x5596f67a8bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a8bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 -- 172.21.15.5:0/3246576382 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596f65311f0 con 0x5596f65e76b0 2023-07-21T19:39:39.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f361d79a700 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3614005d60 tx=0x7f361400e330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fbe098c6bd4179 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.714+0000 7f360cff9700 1 -- 172.21.15.5:0/3246576382 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614014cd0 con 0x5596f65e76b0 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.715+0000 7f360cff9700 1 -- 172.21.15.5:0/3246576382 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3614024020 con 0x5596f65e76b0 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.715+0000 7f360cff9700 1 -- 172.21.15.5:0/3246576382 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3614005560 con 0x5596f65e76b0 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.715+0000 7f3633a57b80 1 -- 172.21.15.5:0/3246576382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 msgr2=0x5596f67a4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.715+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f3614005d60 tx=0x7f361400e330 comp rx=0 tx=0).stop 2023-07-21T19:39:39.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 -- 172.21.15.5:0/3246576382 shutdown_connections 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f678a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a4430 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 --2- 172.21.15.5:0/3246576382 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a8bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 -- 172.21.15.5:0/3246576382 >> 172.21.15.5:0/3246576382 conn(0x5596f6780be0 msgr2=0x5596f67b3fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 -- 172.21.15.5:0/3246576382 shutdown_connections 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 -- 172.21.15.5:0/3246576382 wait complete. 2023-07-21T19:39:39.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.716+0000 7f3633a57b80 1 -- start start 2023-07-21T19:39:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 Processor -- start 2023-07-21T19:39:39.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f6800100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6800640 con 0x5596f678b040 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f6800780 con 0x5596f65e76b0 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596f68008f0 con 0x5596f678b9f0 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.717+0000 7f3633a57b80 10 client.0 _ll_register_callbacks cb 0x7f3608009ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:39.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59440/0 (socket says 172.21.15.5:59440) 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 -- 172.21.15.5:0/1804394638 learned_addr learned my addr 172.21.15.5:0/1804394638 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361d79a700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361d79a700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f6800100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 -- 172.21.15.5:0/1804394638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 msgr2=0x5596f6800100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f6800100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 -- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 msgr2=0x5596f67a2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:39.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361df9b700 1 -- 172.21.15.5:0/1804394638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f361401c3f0 con 0x5596f678b040 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.718+0000 7f361d79a700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f6800100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f361df9b700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x5596f679e3f0 tx=0x7f360800ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1eae5c3e179b06b1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f36017fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36080097b0 con 0x5596f678b040 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3608014040 con 0x5596f678b040 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3608004cc0 con 0x5596f678b040 2023-07-21T19:39:39.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3608003cb0 con 0x5596f678b040 2023-07-21T19:39:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f3633a57b80 10 client.20754 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.719+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5596f6509250 con 0x5596f678b040 2023-07-21T19:39:39.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3608042020 con 0x5596f678b040 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f3608033040 con 0x5596f678b040 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 10 client.20754 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 10 client.20754 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 20 client.20754 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f6801cf0 0x5596f6804100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 4 client.20754 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:39:39.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.720+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5596f6697ff0 con 0x5596f6801cf0 2023-07-21T19:39:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.721+0000 7f361d79a700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f6801cf0 0x5596f6804100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.722+0000 7f361d79a700 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f6801cf0 0x5596f6804100 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcf638809e1ab14 server_cookie=b563cb05ffbf88fc in_seq=0 out_seq=0 2023-07-21T19:39:39.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.722+0000 7f36017fa700 10 client.20754 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.742+0000 7f36017fa700 1 -- 172.21.15.5:0/1804394638 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5596f6697ff0 con 0x5596f6801cf0 2023-07-21T19:39:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.742+0000 7f36017fa700 10 client.20754 handle_command_reply: tid=0 2023-07-21T19:39:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.742+0000 7f3633a57b80 1 client.20754 shutdown 2023-07-21T19:39:39.727 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 msgr2=0x5596f67a27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x5596f679e3f0 tx=0x7f360800ed30 comp rx=0 tx=0).stop 2023-07-21T19:39:39.728 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 shutdown_connections 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596f678b9f0 0x5596f6800100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596f65e76b0 0x5596f67a2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596f6801cf0 0x5596f6804100 crc :-1 s=READY pgs=571 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 --2- 172.21.15.5:0/1804394638 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596f678b040 0x5596f67a27d0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 >> 172.21.15.5:0/1804394638 conn(0x5596f6780be0 msgr2=0x5596f67a8bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f36017fa700 0 client.20754 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:39.729 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 shutdown_connections 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.743+0000 7f3633a57b80 1 -- 172.21.15.5:0/1804394638 wait complete. 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.744+0000 7f3633a57b80 20 client.20754 trim_cache size 0 max 16384 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:39:39.730 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.744+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 msgr2=0x7f36040a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.744+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 0x7f36040a3870 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f361009ac50 tx=0x7f361000b3b0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.744+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 msgr2=0x7f36181a2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:39.731 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.744+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f361000d010 tx=0x7f361000bd90 comp rx=0 tx=0).stop 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 shutdown_connections 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3618112930 0x7f36181a2f70 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f36040a13c0 0x7f36040a3870 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f361806cd70 0x7f361819e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 --2- 172.21.15.5:0/268270838 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3618108210 0x7f361819e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 >> 172.21.15.5:0/268270838 conn(0x7f361806bd70 msgr2=0x7f361810a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:39.732 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 shutdown_connections 2023-07-21T19:39:39.733 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:39.745+0000 7f36201ff700 1 -- 172.21.15.5:0/268270838 wait complete. 2023-07-21T19:39:41.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 790.0 s, 236.4 tps, lat 435.524 ms stddev 134.152 2023-07-21T19:39:42.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:41 smithi101 ceph-mon[128136]: pgmap v1018: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-21T19:39:42.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:41 smithi005 ceph-mon[126460]: pgmap v1018: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-21T19:39:42.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:41 smithi156 ceph-mon[128316]: pgmap v1018: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 5.8 MiB/s wr, 597 op/s 2023-07-21T19:39:44.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:43 smithi101 ceph-mon[128136]: pgmap v1019: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 832 op/s 2023-07-21T19:39:44.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:43 smithi005 ceph-mon[126460]: pgmap v1019: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 832 op/s 2023-07-21T19:39:44.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:43 smithi156 ceph-mon[128316]: pgmap v1019: 97 pgs: 97 active+clean; 11 GiB data, 19 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 7.9 MiB/s wr, 832 op/s 2023-07-21T19:39:46.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:45 smithi101 ceph-mon[128136]: pgmap v1020: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2023-07-21T19:39:46.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:45 smithi005 ceph-mon[126460]: pgmap v1020: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2023-07-21T19:39:46.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:45 smithi156 ceph-mon[128316]: pgmap v1020: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 5.8 MiB/s wr, 602 op/s 2023-07-21T19:39:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 795.0 s, 246.2 tps, lat 405.069 ms stddev 71.709 2023-07-21T19:39:48.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:47 smithi101 ceph-mon[128136]: pgmap v1021: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:48.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:47 smithi005 ceph-mon[126460]: pgmap v1021: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:48.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:47 smithi156 ceph-mon[128316]: pgmap v1021: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.3 MiB/s wr, 558 op/s 2023-07-21T19:39:49.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:48 smithi101 ceph-mon[128136]: pgmap v1022: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 873 op/s 2023-07-21T19:39:49.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:48 smithi005 ceph-mon[126460]: pgmap v1022: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 873 op/s 2023-07-21T19:39:49.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:48 smithi156 ceph-mon[128316]: pgmap v1022: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 873 op/s 2023-07-21T19:39:51.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 800.0 s, 244.4 tps, lat 409.595 ms stddev 73.773 2023-07-21T19:39:52.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:51 smithi101 ceph-mon[128136]: pgmap v1023: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 586 op/s 2023-07-21T19:39:52.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:51 smithi005 ceph-mon[126460]: pgmap v1023: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 586 op/s 2023-07-21T19:39:52.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:51 smithi156 ceph-mon[128316]: pgmap v1023: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.7 MiB/s wr, 586 op/s 2023-07-21T19:39:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.982+0000 7fcdf0fff700 1 -- 172.21.15.5:0/671367180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 msgr2=0x7fcdec110530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:52.967 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.982+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/671367180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec110530 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fcde0009990 tx=0x7fcde0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 -- 172.21.15.5:0/671367180 shutdown_connections 2023-07-21T19:39:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/671367180 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec110a70 0x7fcdec112e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/671367180 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec074a40 0x7fcdec074e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.968 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/671367180 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec110530 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 -- 172.21.15.5:0/671367180 >> 172.21.15.5:0/671367180 conn(0x7fcdec0fd8d0 msgr2=0x7fcdec0ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 -- 172.21.15.5:0/671367180 shutdown_connections 2023-07-21T19:39:52.969 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.983+0000 7fcdf0fff700 1 -- 172.21.15.5:0/671367180 wait complete. 2023-07-21T19:39:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.984+0000 7fcdf0fff700 1 Processor -- start 2023-07-21T19:39:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 -- start start 2023-07-21T19:39:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec074a40 0x7fcdec19c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.970 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec110a70 0x7fcdec1a0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a13c0 con 0x7fcdec074a40 2023-07-21T19:39:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a1530 con 0x7fcdec075350 2023-07-21T19:39:52.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.985+0000 7fcdf0fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdec1a16f0 con 0x7fcdec110a70 2023-07-21T19:39:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58910/0 (socket says 172.21.15.5:58910) 2023-07-21T19:39:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 -- 172.21.15.5:0/1764026686 learned_addr learned my addr 172.21.15.5:0/1764026686 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:52.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcdead9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec110a70 0x7fcdec1a0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcdea59c700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec074a40 0x7fcdec19c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 -- 172.21.15.5:0/1764026686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec110a70 msgr2=0x7fcdec1a0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec110a70 0x7fcdec1a0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 -- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec074a40 msgr2=0x7fcdec19c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec074a40 0x7fcdec19c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.986+0000 7fcde9d9b700 1 -- 172.21.15.5:0/1764026686 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde00095f0 con 0x7fcdec075350 2023-07-21T19:39:52.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcddb7fe700 1 -- 172.21.15.5:0/1764026686 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde001d070 con 0x7fcdec075350 2023-07-21T19:39:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdec1a1970 con 0x7fcdec075350 2023-07-21T19:39:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcddb7fe700 1 -- 172.21.15.5:0/1764026686 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcde0022470 con 0x7fcdec075350 2023-07-21T19:39:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcddb7fe700 1 -- 172.21.15.5:0/1764026686 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde0017650 con 0x7fcdec075350 2023-07-21T19:39:52.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.987+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1a9210 con 0x7fcdec075350 2023-07-21T19:39:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.992+0000 7fcddb7fe700 1 -- 172.21.15.5:0/1764026686 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcde00225e0 con 0x7fcdec075350 2023-07-21T19:39:52.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.993+0000 7fce04a0db80 1 Processor -- start 2023-07-21T19:39:52.978 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 -- start start 2023-07-21T19:39:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e2fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e2a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e2a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44d2fc50 con 0x563b44e302d0 2023-07-21T19:39:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44d829e0 con 0x563b44e2a580 2023-07-21T19:39:52.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.994+0000 7fce04a0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44d7be70 con 0x563b44c866b0 2023-07-21T19:39:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.995+0000 7fcde9d9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e2a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.995+0000 7fcdead9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e2fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.995+0000 7fcdead9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e2fd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53140/0 (socket says 172.21.15.5:53140) 2023-07-21T19:39:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.995+0000 7fcdead9d700 1 -- 172.21.15.5:0/2048746019 learned_addr learned my addr 172.21.15.5:0/2048746019 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:52.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.995+0000 7fcddb7fe700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 0x7fcdd40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcddb7fe700 1 -- 172.21.15.5:0/1764026686 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcde00ce160 con 0x7fcdec075350 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 -- 172.21.15.5:0/2048746019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 msgr2=0x563b44e2fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e2fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 -- 172.21.15.5:0/2048746019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 msgr2=0x563b44e2a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e2a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 -- 172.21.15.5:0/2048746019 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b44ba9d80 con 0x563b44e2a580 2023-07-21T19:39:52.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.996+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e2a9d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcde00cb850 tx=0x7fcde0009ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70fcec0815e60e74 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.997+0000 7fcdead9d700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 0x7fcdd40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.997+0000 7fcdd8ff9700 1 -- 172.21.15.5:0/2048746019 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde00975a0 con 0x563b44e2a580 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.997+0000 7fcdd8ff9700 1 -- 172.21.15.5:0/2048746019 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcde009b030 con 0x563b44e2a580 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.997+0000 7fcdd8ff9700 1 -- 172.21.15.5:0/2048746019 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde009f290 con 0x563b44e2a580 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 -- 172.21.15.5:0/2048746019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 msgr2=0x563b44e2a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fcdead9d700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 0x7fcdd40a3870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009fd0 tx=0x7fcde4009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e2a9d0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fcde00cb850 tx=0x7fcde0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 -- 172.21.15.5:0/2048746019 shutdown_connections 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e2a9d0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e2fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 --2- 172.21.15.5:0/2048746019 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e2a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 -- 172.21.15.5:0/2048746019 >> 172.21.15.5:0/2048746019 conn(0x563b44e1fbe0 msgr2=0x563b44e24f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 -- 172.21.15.5:0/2048746019 shutdown_connections 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.998+0000 7fce04a0db80 1 -- 172.21.15.5:0/2048746019 wait complete. 2023-07-21T19:39:52.985 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.999+0000 7fce04a0db80 1 -- start start 2023-07-21T19:39:52.986 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:52.999+0000 7fce04a0db80 1 Processor -- start 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44e2a580 0x563b44e398c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e33640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44d1ba60 con 0x563b44e302d0 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44d1b760 con 0x563b44c866b0 2023-07-21T19:39:52.987 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44bff8c0 con 0x563b44e2a580 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.000+0000 7fce04a0db80 10 client.0 _ll_register_callbacks cb 0x563b44d19700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58930/0 (socket says 172.21.15.5:58930) 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 -- 172.21.15.5:0/582131325 learned_addr learned my addr 172.21.15.5:0/582131325 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44e2a580 0x563b44e398c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdead9d700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e33640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.988 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 -- 172.21.15.5:0/582131325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44e2a580 msgr2=0x563b44e398c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44e2a580 0x563b44e398c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 -- 172.21.15.5:0/582131325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 msgr2=0x563b44e33640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e33640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.001+0000 7fcdea59c700 1 -- 172.21.15.5:0/582131325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdec1a9210 con 0x563b44c866b0 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.002+0000 7fcdea59c700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011660 tx=0x7fcddc011970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c64616fa2e07ee22 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.002+0000 7fcdd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.002+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc012860 con 0x563b44c866b0 2023-07-21T19:39:52.989 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.002+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563b44cd5490 con 0x563b44c866b0 2023-07-21T19:39:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.002+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcddc011e80 con 0x563b44c866b0 2023-07-21T19:39:52.990 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.003+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcddc0129c0 con 0x563b44c866b0 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.003+0000 7fce04a0db80 10 client.30530 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.003+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563b44b92c00 con 0x563b44c866b0 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x563b44cd5490 con 0x563b44c866b0 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcddc00fc50 con 0x563b44c866b0 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 10 client.30530 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:52.991 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 10 client.30530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 20 client.30530 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e43830 0x563b44e45c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 4 client.30530 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.004+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563b44d74e50 con 0x563b44e43830 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.005+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e43830 0x563b44e45c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.005+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e43830 0x563b44e45c40 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1601f04d5aaf785 server_cookie=7b36f55cbd734a84 in_seq=0 out_seq=0 2023-07-21T19:39:52.992 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.005+0000 7fcdd1ffb700 10 client.30530 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.006+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/582131325 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563b44d74e50 con 0x563b44e43830 2023-07-21T19:39:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.006+0000 7fcdd1ffb700 10 client.30530 handle_command_reply: tid=0 2023-07-21T19:39:52.993 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.006+0000 7fce04a0db80 1 client.30530 shutdown 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 msgr2=0x563b44e39380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011660 tx=0x7fcddc011970 comp rx=0 tx=0).stop 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 shutdown_connections 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e43830 0x563b44e45c40 crc :-1 s=READY pgs=572 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e33640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44c866b0 0x563b44e39380 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.994 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 --2- 172.21.15.5:0/582131325 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44e2a580 0x563b44e398c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 >> 172.21.15.5:0/582131325 conn(0x563b44e1fbe0 msgr2=0x563b44e23c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.007+0000 7fcdd1ffb700 0 client.30530 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.008+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 shutdown_connections 2023-07-21T19:39:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.008+0000 7fce04a0db80 1 -- 172.21.15.5:0/582131325 wait complete. 2023-07-21T19:39:52.995 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.008+0000 7fce04a0db80 20 client.30530 trim_cache size 0 max 16384 2023-07-21T19:39:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 Processor -- start 2023-07-21T19:39:53.019 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 -- start start 2023-07-21T19:39:53.020 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e24c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e2a580 0x563b44e2c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e34bb0 con 0x563b44e2a580 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e31ba0 con 0x563b44e302d0 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.035+0000 7fce04a0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e2e3b0 con 0x563b44c866b0 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:58934/0 (socket says 172.21.15.5:58934) 2023-07-21T19:39:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 -- 172.21.15.5:0/4027038567 learned_addr learned my addr 172.21.15.5:0/4027038567 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:53.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdea59c700 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e24c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 -- 172.21.15.5:0/4027038567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 msgr2=0x563b44e24c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e24c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.036+0000 7fcdead9d700 1 -- 172.21.15.5:0/4027038567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e2a580 msgr2=0x563b44e2c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdead9d700 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e2a580 0x563b44e2c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdead9d700 1 -- 172.21.15.5:0/4027038567 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563b44bd01f0 con 0x563b44e302d0 2023-07-21T19:39:53.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdead9d700 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcde40178c0 tx=0x7fcde4004b40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3baee52dc00c6aa server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdd97fa700 1 -- 172.21.15.5:0/4027038567 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4007ea0 con 0x563b44e302d0 2023-07-21T19:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdd97fa700 1 -- 172.21.15.5:0/4027038567 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcde400f040 con 0x563b44e302d0 2023-07-21T19:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.037+0000 7fcdd97fa700 1 -- 172.21.15.5:0/4027038567 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde400e040 con 0x563b44e302d0 2023-07-21T19:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.038+0000 7fce04a0db80 1 -- 172.21.15.5:0/4027038567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 msgr2=0x563b44e2cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:53.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.038+0000 7fce04a0db80 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fcde40178c0 tx=0x7fcde4004b40 comp rx=0 tx=0).stop 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 -- 172.21.15.5:0/4027038567 shutdown_connections 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e302d0 0x563b44e2cce0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e24c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 --2- 172.21.15.5:0/4027038567 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e2a580 0x563b44e2c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 -- 172.21.15.5:0/4027038567 >> 172.21.15.5:0/4027038567 conn(0x563b44e1fbe0 msgr2=0x563b44e310a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:53.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 -- 172.21.15.5:0/4027038567 shutdown_connections 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 -- 172.21.15.5:0/4027038567 wait complete. 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.039+0000 7fce04a0db80 1 -- start start 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 Processor -- start 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e37830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e37d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e38300 con 0x563b44e302d0 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e38440 con 0x563b44e2a580 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x563b44e38600 con 0x563b44c866b0 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.040+0000 7fce04a0db80 10 client.0 _ll_register_callbacks cb 0x7fcddc009c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:53154/0 (socket says 172.21.15.5:53154) 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 -- 172.21.15.5:0/1761311867 learned_addr learned my addr 172.21.15.5:0/1761311867 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:39:53.028 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e37830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdead9d700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e37d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 -- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 msgr2=0x563b44e37830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e37830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 -- 172.21.15.5:0/1761311867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 msgr2=0x563b44e37d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e37d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.041+0000 7fcdea59c700 1 -- 172.21.15.5:0/1761311867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcde40175a0 con 0x563b44c866b0 2023-07-21T19:39:53.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.042+0000 7fcdea59c700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011df0 tx=0x7fcddc005ce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6fbb106b057b3ac8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.042+0000 7fcdd1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.042+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcddc005240 con 0x563b44c866b0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.042+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcddc000e40 con 0x563b44c866b0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.042+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fcde009b030 con 0x563b44c866b0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcddc0053a0 con 0x563b44c866b0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 10 client.30481 fetch_fsmap learned FSMap version 72 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x563b44ba8250 con 0x563b44c866b0 2023-07-21T19:39:53.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcddc038080 con 0x563b44c866b0 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fcddc032030 con 0x563b44c866b0 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 10 client.30481 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 10 client.30481 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 20 client.30481 populate_metadata read hostname 'smithi005' 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e98bc0 0x563b44e9afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 4 client.30481 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:39:53.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.043+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563b44d36ff0 con 0x563b44e98bc0 2023-07-21T19:39:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.044+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e98bc0 0x563b44e9afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:39:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.044+0000 7fcde9d9b700 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e98bc0 0x563b44e9afd0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38572cd16522e626 server_cookie=72d0aeedf9d818d0 in_seq=0 out_seq=0 2023-07-21T19:39:53.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.044+0000 7fcdd1ffb700 10 client.30481 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:53.049 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.065+0000 7fcdd1ffb700 1 -- 172.21.15.5:0/1761311867 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563b44d36ff0 con 0x563b44e98bc0 2023-07-21T19:39:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.065+0000 7fcdd1ffb700 10 client.30481 handle_command_reply: tid=0 2023-07-21T19:39:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.065+0000 7fce04a0db80 1 client.30481 shutdown 2023-07-21T19:39:53.050 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 msgr2=0x563b44e395b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fcddc011df0 tx=0x7fcddc005ce0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 shutdown_connections 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x563b44e98bc0 0x563b44e9afd0 crc :-1 s=READY pgs=573 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x563b44e302d0 0x563b44e37d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x563b44c866b0 0x563b44e395b0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.051 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 --2- 172.21.15.5:0/1761311867 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x563b44e2a580 0x563b44e37830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 >> 172.21.15.5:0/1761311867 conn(0x563b44e1fbe0 msgr2=0x563b44e2e7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fcdd1ffb700 0 client.30481 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:39:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 shutdown_connections 2023-07-21T19:39:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.066+0000 7fce04a0db80 1 -- 172.21.15.5:0/1761311867 wait complete. 2023-07-21T19:39:53.052 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.067+0000 7fce04a0db80 20 client.30481 trim_cache size 0 max 16384 2023-07-21T19:39:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:39:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:39:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:39:53.053 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 msgr2=0x7fcdd40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 0x7fcdd40a3870 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fcde4009fd0 tx=0x7fcde4009040 comp rx=0 tx=0).stop 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 msgr2=0x7fcdec19c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fcde0000c00 tx=0x7fcde0004730 comp rx=0 tx=0).stop 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 shutdown_connections 2023-07-21T19:39:53.054 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fcdec110a70 0x7fcdec1a0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcdd40a13c0 0x7fcdd40a3870 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fcdec074a40 0x7fcdec19c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 --2- 172.21.15.5:0/1764026686 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fcdec075350 0x7fcdec19c710 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 >> 172.21.15.5:0/1764026686 conn(0x7fcdec0fd8d0 msgr2=0x7fcdec1116d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 shutdown_connections 2023-07-21T19:39:53.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:39:53.068+0000 7fcdf0fff700 1 -- 172.21.15.5:0/1764026686 wait complete. 2023-07-21T19:39:54.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:53 smithi101 ceph-mon[128136]: pgmap v1024: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 887 op/s 2023-07-21T19:39:54.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:53 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:39:54.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:53 smithi005 ceph-mon[126460]: pgmap v1024: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 887 op/s 2023-07-21T19:39:54.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:53 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:39:54.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:53 smithi156 ceph-mon[128316]: pgmap v1024: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.4 MiB/s wr, 887 op/s 2023-07-21T19:39:54.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:53 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:39:55.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:54 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:54 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:55.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:54 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:55 smithi005 ceph-mon[126460]: pgmap v1025: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 692 op/s 2023-07-21T19:39:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:55 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:39:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:55 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:39:56.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:55 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:56.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:55 smithi156 ceph-mon[128316]: pgmap v1025: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 692 op/s 2023-07-21T19:39:56.208 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:55 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:39:56.209 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:55 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:39:56.209 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:55 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:56.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:55 smithi101 ceph-mon[128136]: pgmap v1025: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.9 MiB/s wr, 692 op/s 2023-07-21T19:39:56.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:55 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:39:56.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:55 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:39:56.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:55 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:39:56.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 805.0 s, 232.0 tps, lat 430.183 ms stddev 68.662 2023-07-21T19:39:58.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:57 smithi005 ceph-mon[126460]: pgmap v1026: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2023-07-21T19:39:58.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:57 smithi156 ceph-mon[128316]: pgmap v1026: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2023-07-21T19:39:58.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:57 smithi101 ceph-mon[128136]: pgmap v1026: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.3 MiB/s wr, 655 op/s 2023-07-21T19:39:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:39:58 smithi101 ceph-mon[128136]: pgmap v1027: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 980 op/s 2023-07-21T19:39:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:39:58 smithi005 ceph-mon[126460]: pgmap v1027: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 980 op/s 2023-07-21T19:39:59.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:39:58 smithi156 ceph-mon[128316]: pgmap v1027: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 9.4 MiB/s wr, 980 op/s 2023-07-21T19:40:00.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:00 smithi101 ceph-mon[128136]: overall HEALTH_OK 2023-07-21T19:40:00.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:00 smithi005 ceph-mon[126460]: overall HEALTH_OK 2023-07-21T19:40:00.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:00 smithi156 ceph-mon[128316]: overall HEALTH_OK 2023-07-21T19:40:01.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:01 smithi101 ceph-mon[128136]: pgmap v1028: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2023-07-21T19:40:01.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:01 smithi005 ceph-mon[126460]: pgmap v1028: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2023-07-21T19:40:01.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:01 smithi156 ceph-mon[128316]: pgmap v1028: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.4 MiB/s wr, 665 op/s 2023-07-21T19:40:01.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 810.0 s, 216.0 tps, lat 462.684 ms stddev 151.852 2023-07-21T19:40:04.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:03 smithi101 ceph-mon[128136]: pgmap v1029: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-07-21T19:40:04.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:03 smithi005 ceph-mon[126460]: pgmap v1029: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-07-21T19:40:04.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:03 smithi156 ceph-mon[128316]: pgmap v1029: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 9.3 MiB/s wr, 976 op/s 2023-07-21T19:40:04.235 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:40:04.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 -- 172.21.15.5:0/4173066892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39241015b0 msgr2=0x7f3924109aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 --2- 172.21.15.5:0/4173066892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39241015b0 0x7f3924109aa0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f39140098f0 tx=0x7f3914009c00 comp rx=0 tx=0).stop 2023-07-21T19:40:04.623 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 -- 172.21.15.5:0/4173066892 shutdown_connections 2023-07-21T19:40:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 --2- 172.21.15.5:0/4173066892 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f392410a070 0x7f392410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 --2- 172.21.15.5:0/4173066892 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 --2- 172.21.15.5:0/4173066892 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f39241015b0 0x7f3924109aa0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 -- 172.21.15.5:0/4173066892 >> 172.21.15.5:0/4173066892 conn(0x7f39240fa4b0 msgr2=0x7f39240fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.624 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.638+0000 7f392bdff700 1 -- 172.21.15.5:0/4173066892 shutdown_connections 2023-07-21T19:40:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.639+0000 7f392bdff700 1 -- 172.21.15.5:0/4173066892 wait complete. 2023-07-21T19:40:04.625 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.640+0000 7f392bdff700 1 Processor -- start 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.640+0000 7f392bdff700 1 -- start start 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924109610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 0x7f3924107680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 0x7f3924107bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924108100 con 0x7f39241015b0 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924108270 con 0x7f392410a070 2023-07-21T19:40:04.626 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392bdff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3924108430 con 0x7f3924100ca0 2023-07-21T19:40:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924109610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392a39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 0x7f3924107bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924109610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:46608/0 (socket says 172.21.15.5:46608) 2023-07-21T19:40:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f3929b9b700 1 -- 172.21.15.5:0/915350626 learned_addr learned my addr 172.21.15.5:0/915350626 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:04.627 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.641+0000 7f392939a700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 0x7f3924107680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392939a700 1 -- 172.21.15.5:0/915350626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 msgr2=0x7f3924109610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392939a700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924109610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392939a700 1 -- 172.21.15.5:0/915350626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 msgr2=0x7f3924107bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392939a700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 0x7f3924107bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.628 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392939a700 1 -- 172.21.15.5:0/915350626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3914009520 con 0x7f39241015b0 2023-07-21T19:40:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.642+0000 7f392a39c700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 0x7f3924107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.643+0000 7f392939a700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 0x7f3924107680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3914009810 tx=0x7f391400fa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.643+0000 7f391affd700 1 -- 172.21.15.5:0/915350626 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391401c070 con 0x7f39241015b0 2023-07-21T19:40:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.643+0000 7f391affd700 1 -- 172.21.15.5:0/915350626 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f391400ba70 con 0x7f39241015b0 2023-07-21T19:40:04.629 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.643+0000 7f391affd700 1 -- 172.21.15.5:0/915350626 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39140038e0 con 0x7f39241015b0 2023-07-21T19:40:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.645+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39241a89b0 con 0x7f39241015b0 2023-07-21T19:40:04.630 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.645+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39241a8d80 con 0x7f39241015b0 2023-07-21T19:40:04.636 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.652+0000 7f391affd700 1 -- 172.21.15.5:0/915350626 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3914020030 con 0x7f39241015b0 2023-07-21T19:40:04.637 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.653+0000 7f393f6abb80 1 Processor -- start 2023-07-21T19:40:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.653+0000 7f393f6abb80 1 -- start start 2023-07-21T19:40:04.638 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d12d0 0x5596a56d16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56dcad0 0x5596a56deeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a55d7cb0 con 0x5596a56d1be0 2023-07-21T19:40:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a562aa40 con 0x5596a56d12d0 2023-07-21T19:40:04.639 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.654+0000 7f393f6abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a5623ed0 con 0x5596a56dcad0 2023-07-21T19:40:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50160/0 (socket says 172.21.15.5:50160) 2023-07-21T19:40:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 -- 172.21.15.5:0/1042749441 learned_addr learned my addr 172.21.15.5:0/1042749441 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 -- 172.21.15.5:0/1042749441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56dcad0 msgr2=0x5596a56deeb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:40:04.640 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56dcad0 0x5596a56deeb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 -- 172.21.15.5:0/1042749441 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d12d0 msgr2=0x5596a56d16a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:40:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d12d0 0x5596a56d16a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.655+0000 7f3929b9b700 1 -- 172.21.15.5:0/1042749441 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a5451de0 con 0x5596a56d1be0 2023-07-21T19:40:04.641 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.656+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3920009e30 tx=0x7f3920011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bad0715eaa6a92d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.656+0000 7f391affd700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 0x7f39100a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.656+0000 7f391affd700 1 -- 172.21.15.5:0/915350626 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f39140ce060 con 0x7f39241015b0 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.656+0000 7f390ffff700 1 -- 172.21.15.5:0/1042749441 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3920019630 con 0x5596a56d1be0 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.657+0000 7f390ffff700 1 -- 172.21.15.5:0/1042749441 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3920019c20 con 0x5596a56d1be0 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.657+0000 7f390ffff700 1 -- 172.21.15.5:0/1042749441 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392000eb90 con 0x5596a56d1be0 2023-07-21T19:40:04.642 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.657+0000 7f392a39c700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 0x7f39100a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 -- 172.21.15.5:0/1042749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 msgr2=0x5596a56dc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f3920009e30 tx=0x7f3920011720 comp rx=0 tx=0).stop 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 -- 172.21.15.5:0/1042749441 shutdown_connections 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56dcad0 0x5596a56deeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d12d0 0x5596a56d16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.643 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1042749441 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56dc590 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.658+0000 7f393f6abb80 1 -- 172.21.15.5:0/1042749441 >> 172.21.15.5:0/1042749441 conn(0x5596a56c7da0 msgr2=0x5596a56cd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.659+0000 7f392a39c700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 0x7f39100a3940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f391c003e10 tx=0x7f391c00b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.659+0000 7f393f6abb80 1 -- 172.21.15.5:0/1042749441 shutdown_connections 2023-07-21T19:40:04.644 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.660+0000 7f393f6abb80 1 -- 172.21.15.5:0/1042749441 wait complete. 2023-07-21T19:40:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.661+0000 7f393f6abb80 1 -- start start 2023-07-21T19:40:04.645 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.661+0000 7f393f6abb80 1 Processor -- start 2023-07-21T19:40:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56d9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.646 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a55c3ac0 con 0x5596a56dcad0 2023-07-21T19:40:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a55c37c0 con 0x5596a56d1be0 2023-07-21T19:40:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a54a7920 con 0x5596a56d12d0 2023-07-21T19:40:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f393f6abb80 10 client.0 _ll_register_callbacks cb 0x5596a55c1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:04.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.662+0000 7f392a39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:50176/0 (socket says 172.21.15.5:50176) 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 -- 172.21.15.5:0/2154237873 learned_addr learned my addr 172.21.15.5:0/2154237873 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392939a700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56d9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 -- 172.21.15.5:0/2154237873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 msgr2=0x5596a56d9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 -- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 msgr2=0x5596a56d9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56d9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.663+0000 7f392a39c700 1 -- 172.21.15.5:0/2154237873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3920009a60 con 0x5596a56dcad0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.664+0000 7f392a39c700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ece0 tx=0x7f391c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2649e97963d7128c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.664+0000 7f390d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.664+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f391c017d90 con 0x5596a56dcad0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.664+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f391c01c040 con 0x5596a56dcad0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.664+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5596a557d4f0 con 0x5596a56dcad0 2023-07-21T19:40:04.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f391c02a440 con 0x5596a56dcad0 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f393f6abb80 10 client.20778 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5596a543ac60 con 0x5596a56dcad0 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5596a557d4f0 con 0x5596a56dcad0 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f391c0154c0 con 0x5596a56dcad0 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f393f6abb80 10 client.20778 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.665+0000 7f393f6abb80 10 client.20778 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.666+0000 7f393f6abb80 20 client.20778 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:04.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.666+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56fd560 0x5596a56ff970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.666+0000 7f393f6abb80 4 client.20778 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.666+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5596a561ceb0 con 0x5596a56fd560 2023-07-21T19:40:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.666+0000 7f3929b9b700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56fd560 0x5596a56ff970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.667+0000 7f3929b9b700 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56fd560 0x5596a56ff970 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a679752846141ac server_cookie=dd1aa46f8bdc4baa in_seq=0 out_seq=0 2023-07-21T19:40:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.667+0000 7f390d7fa700 10 client.20778 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.668+0000 7f390d7fa700 1 -- 172.21.15.5:0/2154237873 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5596a561ceb0 con 0x5596a56fd560 2023-07-21T19:40:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.668+0000 7f390d7fa700 10 client.20778 handle_command_reply: tid=0 2023-07-21T19:40:04.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.668+0000 7f393f6abb80 1 client.20778 shutdown 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 msgr2=0x5596a56da2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f391c00ece0 tx=0x7f391c004ad0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 shutdown_connections 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56fd560 0x5596a56ff970 crc :-1 s=READY pgs=574 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56dcad0 0x5596a56da2c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 --2- 172.21.15.5:0/2154237873 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56d9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 >> 172.21.15.5:0/2154237873 conn(0x5596a56c7da0 msgr2=0x5596a56cd230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f390d7fa700 0 client.20778 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 shutdown_connections 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 1 -- 172.21.15.5:0/2154237873 wait complete. 2023-07-21T19:40:04.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.669+0000 7f393f6abb80 20 client.20778 trim_cache size 0 max 16384 2023-07-21T19:40:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.692+0000 7f393f6abb80 1 Processor -- start 2023-07-21T19:40:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 -- start start 2023-07-21T19:40:04.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56d6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56cead0 con 0x5596a56d1be0 2023-07-21T19:40:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56d2540 con 0x5596a56c7da0 2023-07-21T19:40:04.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f393f6abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56c4f80 con 0x5596a56d12d0 2023-07-21T19:40:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f392a39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56d6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33904/0 (socket says 172.21.15.5:33904) 2023-07-21T19:40:04.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.693+0000 7f3929b9b700 1 -- 172.21.15.5:0/1816968972 learned_addr learned my addr 172.21.15.5:0/1816968972 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f392939a700 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 -- 172.21.15.5:0/1816968972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 msgr2=0x5596a56d9f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 -- 172.21.15.5:0/1816968972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 msgr2=0x5596a56d6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56d6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 -- 172.21.15.5:0/1816968972 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5596a57045a0 con 0x5596a56c7da0 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f392a39c700 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56d6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:04.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3929b9b700 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f392001c040 tx=0x7f3920005ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd6947249fd94f2d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.694+0000 7f3918bff700 1 -- 172.21.15.5:0/1816968972 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f392002e070 con 0x5596a56c7da0 2023-07-21T19:40:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.695+0000 7f3918bff700 1 -- 172.21.15.5:0/1816968972 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f39200194f0 con 0x5596a56c7da0 2023-07-21T19:40:04.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.695+0000 7f3918bff700 1 -- 172.21.15.5:0/1816968972 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39200052a0 con 0x5596a56c7da0 2023-07-21T19:40:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 -- 172.21.15.5:0/1816968972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 msgr2=0x5596a56d9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f392001c040 tx=0x7f3920005ad0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 -- 172.21.15.5:0/1816968972 shutdown_connections 2023-07-21T19:40:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56d1be0 0x5596a56d6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56c7da0 0x5596a56d9000 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 --2- 172.21.15.5:0/1816968972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56d9f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 -- 172.21.15.5:0/1816968972 >> 172.21.15.5:0/1816968972 conn(0x5596a56f9000 msgr2=0x5596a56d9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 -- 172.21.15.5:0/1816968972 shutdown_connections 2023-07-21T19:40:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.696+0000 7f393f6abb80 1 -- 172.21.15.5:0/1816968972 wait complete. 2023-07-21T19:40:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.697+0000 7f393f6abb80 1 -- start start 2023-07-21T19:40:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.697+0000 7f393f6abb80 1 Processor -- start 2023-07-21T19:40:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56c7da0 0x5596a56f41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56f4730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56f26b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56f2dc0 con 0x5596a56c7da0 2023-07-21T19:40:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56f2f30 con 0x5596a56d1be0 2023-07-21T19:40:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596a56f30f0 con 0x5596a56d12d0 2023-07-21T19:40:04.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f393f6abb80 10 client.0 _ll_register_callbacks cb 0x5596a55c1760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392a39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56f26b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f3929b9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56c7da0 0x5596a56f41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392a39c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56f26b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33912/0 (socket says 172.21.15.5:33912) 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392a39c700 1 -- 172.21.15.5:0/370137008 learned_addr learned my addr 172.21.15.5:0/370137008 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56f4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 -- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 msgr2=0x5596a56f26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56f26b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 -- 172.21.15.5:0/370137008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56c7da0 msgr2=0x5596a56f41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56c7da0 0x5596a56f41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.698+0000 7f392939a700 1 -- 172.21.15.5:0/370137008 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5596a543ac60 con 0x5596a56d12d0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f392939a700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56f4730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f39140cb750 tx=0x7f391409f0c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f6edbc8c88383589 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f390d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3914005340 con 0x5596a56d12d0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f391409b160 con 0x5596a56d12d0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f3920019c20 con 0x5596a56d12d0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3914005740 con 0x5596a56d12d0 2023-07-21T19:40:04.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f393f6abb80 10 client.30496 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.699+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5596a5451de0 con 0x5596a56d12d0 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3920019c20 con 0x5596a56d12d0 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f391409b3c0 con 0x5596a56d12d0 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 10 client.30496 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 10 client.30496 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 20 client.30496 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56dcad0 0x5596a5754470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:04.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 4 client.30496 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5596a55df050 con 0x5596a56dcad0 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.700+0000 7f3929b9b700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56dcad0 0x5596a5754470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.701+0000 7f3929b9b700 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56dcad0 0x5596a5754470 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7147ae446ce729be server_cookie=68c8452cc54b7816 in_seq=0 out_seq=0 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.701+0000 7f390d7fa700 10 client.30496 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f390d7fa700 1 -- 172.21.15.5:0/370137008 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5596a55df050 con 0x5596a56dcad0 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f390d7fa700 10 client.30496 handle_command_reply: tid=0 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 client.30496 shutdown 2023-07-21T19:40:04.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 msgr2=0x5596a56f4730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56f4730 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f39140cb750 tx=0x7f391409f0c0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 shutdown_connections 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5596a56d1be0 0x5596a56f26b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5596a56dcad0 0x5596a5754470 crc :-1 s=READY pgs=575 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5596a56c7da0 0x5596a56f41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f390d7fa700 0 client.30496 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:04.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 --2- 172.21.15.5:0/370137008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5596a56d12d0 0x5596a56f4730 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 >> 172.21.15.5:0/370137008 conn(0x5596a56f9000 msgr2=0x5596a56da000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 shutdown_connections 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.702+0000 7f393f6abb80 1 -- 172.21.15.5:0/370137008 wait complete. 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.703+0000 7f393f6abb80 20 client.30496 trim_cache size 0 max 16384 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:40:04.693 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:40:04.694 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:40:04.695 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:40:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.703+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 msgr2=0x7f39100a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.703+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 0x7f39100a3940 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f391c003e10 tx=0x7f391c00b040 comp rx=0 tx=0).stop 2023-07-21T19:40:04.695 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.703+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 msgr2=0x7f3924107680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.703+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 0x7f3924107680 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f3914009810 tx=0x7f391400fa00 comp rx=0 tx=0).stop 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 shutdown_connections 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f392410a070 0x7f3924107bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3924100ca0 0x7f3924109610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f39100a1490 0x7f39100a3940 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 --2- 172.21.15.5:0/915350626 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f39241015b0 0x7f3924107680 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 >> 172.21.15.5:0/915350626 conn(0x7f39240fa4b0 msgr2=0x7f3924110070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:04.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 shutdown_connections 2023-07-21T19:40:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:04.704+0000 7f392bdff700 1 -- 172.21.15.5:0/915350626 wait complete. 2023-07-21T19:40:04.714 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:40:06.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:05 smithi101 ceph-mon[128136]: pgmap v1030: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 709 op/s 2023-07-21T19:40:06.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:05 smithi005 ceph-mon[126460]: pgmap v1030: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 709 op/s 2023-07-21T19:40:06.155 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:05 smithi156 ceph-mon[128316]: pgmap v1030: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 6.7 MiB/s wr, 709 op/s 2023-07-21T19:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 -- 172.21.15.5:0/4088188098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 msgr2=0x7f0378102180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 --2- 172.21.15.5:0/4088188098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f0378102180 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f036c009990 tx=0x7f036c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 -- 172.21.15.5:0/4088188098 shutdown_connections 2023-07-21T19:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 --2- 172.21.15.5:0/4088188098 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378106df0 0x7f037810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 --2- 172.21.15.5:0/4088188098 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f0378102180 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 --2- 172.21.15.5:0/4088188098 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378102750 0x7f0378106710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.351+0000 7f037dd9d700 1 -- 172.21.15.5:0/4088188098 >> 172.21.15.5:0/4088188098 conn(0x7f03780fd620 msgr2=0x7f03780ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.352+0000 7f037dd9d700 1 -- 172.21.15.5:0/4088188098 shutdown_connections 2023-07-21T19:40:06.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.352+0000 7f037dd9d700 1 -- 172.21.15.5:0/4088188098 wait complete. 2023-07-21T19:40:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.353+0000 7f037dd9d700 1 Processor -- start 2023-07-21T19:40:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.353+0000 7f037dd9d700 1 -- start start 2023-07-21T19:40:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378102750 0x7f037810b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378106df0 0x7f037810ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810bfb0 con 0x7f0378102750 2023-07-21T19:40:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810c120 con 0x7f0378101db0 2023-07-21T19:40:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037dd9d700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f037810c2e0 con 0x7f0378106df0 2023-07-21T19:40:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037cd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f0377fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378102750 0x7f037810b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037cd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33934/0 (socket says 172.21.15.5:33934) 2023-07-21T19:40:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037cd9b700 1 -- 172.21.15.5:0/1490252332 learned_addr learned my addr 172.21.15.5:0/1490252332 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:06.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.354+0000 7f037d59c700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378106df0 0x7f037810ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.355+0000 7f037cd9b700 1 -- 172.21.15.5:0/1490252332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378106df0 msgr2=0x7f037810ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.342 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.355+0000 7f037cd9b700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378106df0 0x7f037810ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.355+0000 7f037cd9b700 1 -- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378102750 msgr2=0x7f037810b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.355+0000 7f037cd9b700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378102750 0x7f037810b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.355+0000 7f037cd9b700 1 -- 172.21.15.5:0/1490252332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0095f0 con 0x7f0378101db0 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f037cd9b700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f036c000c00 tx=0x7f036c009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f0375ffb700 1 -- 172.21.15.5:0/1490252332 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c01d070 con 0x7f0378101db0 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f0375ffb700 1 -- 172.21.15.5:0/1490252332 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f036c021410 con 0x7f0378101db0 2023-07-21T19:40:06.343 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03781a8920 con 0x7f0378101db0 2023-07-21T19:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f0375ffb700 1 -- 172.21.15.5:0/1490252332 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036c017650 con 0x7f0378101db0 2023-07-21T19:40:06.344 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.356+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03781a8df0 con 0x7f0378101db0 2023-07-21T19:40:06.346 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.362+0000 7f0375ffb700 1 -- 172.21.15.5:0/1490252332 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f036c021580 con 0x7f0378101db0 2023-07-21T19:40:06.347 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.363+0000 7f03929a4b80 1 Processor -- start 2023-07-21T19:40:06.348 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 -- start start 2023-07-21T19:40:06.349 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.350 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 0x55d2718651a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b8c0 0x55d271865700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27176ac50 con 0x55d27186b470 2023-07-21T19:40:06.351 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2717bd9e0 con 0x55d2716c16b0 2023-07-21T19:40:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.364+0000 7f03929a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2717b6e70 con 0x55d27186b8c0 2023-07-21T19:40:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f037d59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.352 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f037d59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33936/0 (socket says 172.21.15.5:33936) 2023-07-21T19:40:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f037d59c700 1 -- 172.21.15.5:0/3656930327 learned_addr learned my addr 172.21.15.5:0/3656930327 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f0377fff700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b8c0 0x55d271865700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.353 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f037cd9b700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 0x55d2718651a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.365+0000 7f037d59c700 1 -- 172.21.15.5:0/3656930327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b8c0 msgr2=0x55d271865700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037d59c700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b8c0 0x55d271865700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.354 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037d59c700 1 -- 172.21.15.5:0/3656930327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 msgr2=0x55d2718651a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037d59c700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 0x55d2718651a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037d59c700 1 -- 172.21.15.5:0/3656930327 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d2715e4d80 con 0x55d2716c16b0 2023-07-21T19:40:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037cd9b700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 0x55d2718651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:06.355 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f037d59c700 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0370003fa0 tx=0x7f0370009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab87d66417fbe20b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.356 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f0375ffb700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 0x7f03680a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.360 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f0362ffd700 1 -- 172.21.15.5:0/3656930327 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370019070 con 0x55d2716c16b0 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f0362ffd700 1 -- 172.21.15.5:0/3656930327 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f037000f040 con 0x55d2716c16b0 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.366+0000 7f0362ffd700 1 -- 172.21.15.5:0/3656930327 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03700079a0 con 0x55d2716c16b0 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.367+0000 7f0375ffb700 1 -- 172.21.15.5:0/1490252332 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f036c0ce0d0 con 0x7f0378101db0 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.367+0000 7f037cd9b700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 0x7f03680a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.367+0000 7f03929a4b80 1 -- 172.21.15.5:0/3656930327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 msgr2=0x55d27186af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.367+0000 7f03929a4b80 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f0370003fa0 tx=0x7f0370009ba0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.361 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 -- 172.21.15.5:0/3656930327 shutdown_connections 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b8c0 0x55d271865700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d2716c16b0 0x55d27186af30 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 --2- 172.21.15.5:0/3656930327 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b470 0x55d2718651a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 -- 172.21.15.5:0/3656930327 >> 172.21.15.5:0/3656930327 conn(0x55d27185ad40 msgr2=0x55d271860340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 -- 172.21.15.5:0/3656930327 shutdown_connections 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.368+0000 7f03929a4b80 1 -- 172.21.15.5:0/3656930327 wait complete. 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.369+0000 7f037cd9b700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 0x7f03680a3870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f036c0976d0 tx=0x7f036c009ca0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.362 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.369+0000 7f03929a4b80 1 -- start start 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.369+0000 7f03929a4b80 1 Processor -- start 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2716c16b0 0x55d2718716a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b8c0 0x55d27186eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271756a60 con 0x55d2716c16b0 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271756760 con 0x55d27186b8c0 2023-07-21T19:40:06.363 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27163a8c0 con 0x55d27186b470 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f03929a4b80 10 client.0 _ll_register_callbacks cb 0x55d271754700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f0377fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f0377fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51024/0 (socket says 172.21.15.5:51024) 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f0377fff700 1 -- 172.21.15.5:0/1381998909 learned_addr learned my addr 172.21.15.5:0/1381998909 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.370+0000 7f037d59c700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2716c16b0 0x55d2718716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f037d59c700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b8c0 0x55d27186eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.364 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 -- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b8c0 msgr2=0x55d27186eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b8c0 0x55d27186eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 -- 172.21.15.5:0/1381998909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2716c16b0 msgr2=0x55d2718716a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2716c16b0 0x55d2718716a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 -- 172.21.15.5:0/1381998909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0370003bd0 con 0x55d27186b470 2023-07-21T19:40:06.365 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0377fff700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0364011730 tx=0x7f0364011a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=54f7ec376d8c0c4c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.371+0000 7f0357fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364016070 con 0x55d27186b470 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d271710490 con 0x55d27186b470 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0364004830 con 0x55d27186b470 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03640049f0 con 0x55d27186b470 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f03929a4b80 10 client.30508 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:06.366 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55d2715cdc00 con 0x55d27186b470 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d271710490 con 0x55d27186b470 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.372+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0364021070 con 0x55d27186b470 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 10 client.30508 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 10 client.30508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 20 client.30508 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d27187ed30 0x55d271881140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.367 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 4 client.30508 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d2717afe50 con 0x55d27187ed30 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.373+0000 7f037d59c700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d27187ed30 0x55d271881140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.374+0000 7f037d59c700 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d27187ed30 0x55d271881140 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dd31785040f31df server_cookie=4403c88a18b7e241 in_seq=0 out_seq=0 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.374+0000 7f0357fff700 10 client.30508 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f0357fff700 1 -- 172.21.15.5:0/1381998909 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d2717afe50 con 0x55d27187ed30 2023-07-21T19:40:06.368 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f0357fff700 10 client.30508 handle_command_reply: tid=0 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 client.30508 shutdown 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 msgr2=0x55d27186e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f0364011730 tx=0x7f0364011a40 comp rx=0 tx=0).stop 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 shutdown_connections 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b8c0 0x55d27186eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d27187ed30 0x55d271881140 crc :-1 s=READY pgs=576 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d2716c16b0 0x55d2718716a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.369 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1381998909 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d27186b470 0x55d27186e990 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 >> 172.21.15.5:0/1381998909 conn(0x55d27185ad40 msgr2=0x55d27185f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.375+0000 7f0357fff700 0 client.30508 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.376+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 shutdown_connections 2023-07-21T19:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.376+0000 7f03929a4b80 1 -- 172.21.15.5:0/1381998909 wait complete. 2023-07-21T19:40:06.370 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.376+0000 7f03929a4b80 20 client.30508 trim_cache size 0 max 16384 2023-07-21T19:40:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.398+0000 7f03929a4b80 1 Processor -- start 2023-07-21T19:40:06.383 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 -- start start 2023-07-21T19:40:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d2718679a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d27187aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.384 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271710490 con 0x55d27186b8c0 2023-07-21T19:40:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271861170 con 0x55d27186b470 2023-07-21T19:40:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.399+0000 7f03929a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d27186cef0 con 0x55d2716c16b0 2023-07-21T19:40:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.385 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f037d59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d27187aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50336/0 (socket says 172.21.15.5:50336) 2023-07-21T19:40:06.386 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 -- 172.21.15.5:0/1232762072 learned_addr learned my addr 172.21.15.5:0/1232762072 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:06.387 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f037d59c700 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d2718679a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 -- 172.21.15.5:0/1232762072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 msgr2=0x55d2718679a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d2718679a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 -- 172.21.15.5:0/1232762072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 msgr2=0x55d27187aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.388 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d27187aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 -- 172.21.15.5:0/1232762072 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d27160b1f0 con 0x55d27186b470 2023-07-21T19:40:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.400+0000 7f0377fff700 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0364005a80 tx=0x7f036401abc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=26cd4c71c576d868 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.401+0000 7f03637fe700 1 -- 172.21.15.5:0/1232762072 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364016070 con 0x55d27186b470 2023-07-21T19:40:06.389 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.401+0000 7f03637fe700 1 -- 172.21.15.5:0/1232762072 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0364019450 con 0x55d27186b470 2023-07-21T19:40:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.401+0000 7f03637fe700 1 -- 172.21.15.5:0/1232762072 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03640057b0 con 0x55d27186b470 2023-07-21T19:40:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 -- 172.21.15.5:0/1232762072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 msgr2=0x55d271871240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0364005a80 tx=0x7f036401abc0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.390 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 -- 172.21.15.5:0/1232762072 shutdown_connections 2023-07-21T19:40:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d27187aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.391 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d2718679a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 --2- 172.21.15.5:0/1232762072 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d271871240 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 -- 172.21.15.5:0/1232762072 >> 172.21.15.5:0/1232762072 conn(0x55d27185ad40 msgr2=0x55d27186c310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 -- 172.21.15.5:0/1232762072 shutdown_connections 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.402+0000 7f03929a4b80 1 -- 172.21.15.5:0/1232762072 wait complete. 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.403+0000 7f03929a4b80 1 -- start start 2023-07-21T19:40:06.392 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.403+0000 7f03929a4b80 1 Processor -- start 2023-07-21T19:40:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.403+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.394 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d2718742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d271874810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271874ee0 con 0x55d27186b8c0 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d271875050 con 0x55d27186b470 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d2718d2c00 con 0x55d2716c16b0 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f03929a4b80 10 client.0 _ll_register_callbacks cb 0x7f0364009e10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f037d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.395 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f037d59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:51036/0 (socket says 172.21.15.5:51036) 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f037d59c700 1 -- 172.21.15.5:0/2870242435 learned_addr learned my addr 172.21.15.5:0/2870242435 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.404+0000 7f037d59c700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d271874810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 -- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 msgr2=0x55d2718742d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d2718742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 -- 172.21.15.5:0/2870242435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 msgr2=0x55d271874810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d271874810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.396 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 -- 172.21.15.5:0/2870242435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d2715cdc00 con 0x55d2716c16b0 2023-07-21T19:40:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f037d59c700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0370009160 tx=0x7f0370009ba0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95200ab200be91f6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f0357fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0370019040 con 0x55d2716c16b0 2023-07-21T19:40:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(141..141 src has 1..141) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0370007610 con 0x55d2716c16b0 2023-07-21T19:40:06.397 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.405+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f037000f040 con 0x55d2716c16b0 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.406+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03700297f0 con 0x55d2716c16b0 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.406+0000 7f03929a4b80 10 client.30514 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.406+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55d2715e3250 con 0x55d2716c16b0 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.406+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f037000f040 con 0x55d2716c16b0 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.406+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f0370007bb0 con 0x55d2716c16b0 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 10 client.30514 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:06.398 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 10 client.30514 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 20 client.30514 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2718d4140 0x55d2718d6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 4 client.30514 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d271771ff0 con 0x55d2718d4140 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.407+0000 7f0377fff700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2718d4140 0x55d2718d6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.408+0000 7f0377fff700 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2718d4140 0x55d2718d6550 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=868de248ffd119aa server_cookie=416b7e546ab8c9fb in_seq=0 out_seq=0 2023-07-21T19:40:06.399 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.408+0000 7f0357fff700 10 client.30514 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.447+0000 7f0357fff700 1 -- 172.21.15.5:0/2870242435 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d271771ff0 con 0x55d2718d4140 2023-07-21T19:40:06.432 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.447+0000 7f0357fff700 10 client.30514 handle_command_reply: tid=0 2023-07-21T19:40:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.447+0000 7f03929a4b80 1 client.30514 shutdown 2023-07-21T19:40:06.433 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 msgr2=0x55d271875ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f0370009160 tx=0x7f0370009ba0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.434 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 shutdown_connections 2023-07-21T19:40:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d2718d4140 0x55d2718d6550 crc :-1 s=READY pgs=577 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d27186b8c0 0x55d271874810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.435 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d2716c16b0 0x55d271875ee0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 --2- 172.21.15.5:0/2870242435 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d27186b470 0x55d2718742d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 >> 172.21.15.5:0/2870242435 conn(0x55d27185ad40 msgr2=0x55d27187aed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f0357fff700 0 client.30514 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:06.436 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 shutdown_connections 2023-07-21T19:40:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.448+0000 7f03929a4b80 1 -- 172.21.15.5:0/2870242435 wait complete. 2023-07-21T19:40:06.437 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:40:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:40:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:40:06.438 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:40:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.449+0000 7f03929a4b80 20 client.30514 trim_cache size 0 max 16384 2023-07-21T19:40:06.439 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.449+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 msgr2=0x7f03680a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.449+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 0x7f03680a3870 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f036c0976d0 tx=0x7f036c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.449+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 msgr2=0x7f037810e520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.449+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f036c000c00 tx=0x7f036c009580 comp rx=0 tx=0).stop 2023-07-21T19:40:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 shutdown_connections 2023-07-21T19:40:06.440 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f0378106df0 0x7f037810ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0378101db0 0x7f037810e520 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f03680a13c0 0x7f03680a3870 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 --2- 172.21.15.5:0/1490252332 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f0378102750 0x7f037810b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 >> 172.21.15.5:0/1490252332 conn(0x7f03780fd620 msgr2=0x7f03781056f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 shutdown_connections 2023-07-21T19:40:06.441 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:06.450+0000 7f037dd9d700 1 -- 172.21.15.5:0/1490252332 wait complete. 2023-07-21T19:40:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 815.0 s, 182.2 tps, lat 523.027 ms stddev 158.438 2023-07-21T19:40:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:07 smithi101 ceph-mon[128136]: pgmap v1031: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 668 op/s 2023-07-21T19:40:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:07 smithi101 ceph-mon[128136]: osdmap e142: 12 total, 12 up, 12 in 2023-07-21T19:40:08.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:07 smithi005 ceph-mon[126460]: pgmap v1031: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 668 op/s 2023-07-21T19:40:08.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:07 smithi005 ceph-mon[126460]: osdmap e142: 12 total, 12 up, 12 in 2023-07-21T19:40:08.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:07 smithi156 ceph-mon[128316]: pgmap v1031: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 6.2 MiB/s wr, 668 op/s 2023-07-21T19:40:08.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:07 smithi156 ceph-mon[128316]: osdmap e142: 12 total, 12 up, 12 in 2023-07-21T19:40:09.317 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:08 smithi101 ceph-mon[128136]: pgmap v1033: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:09.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:08 smithi005 ceph-mon[126460]: pgmap v1033: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:09.408 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:08 smithi156 ceph-mon[128316]: pgmap v1033: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:11.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 820.0 s, 89.8 tps, lat 1170.341 ms stddev 1069.234 2023-07-21T19:40:12.069 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:11 smithi101 ceph-mon[128136]: pgmap v1034: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:12.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:11 smithi005 ceph-mon[126460]: pgmap v1034: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:12.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:11 smithi156 ceph-mon[128316]: pgmap v1034: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.5 MiB/s wr, 682 op/s 2023-07-21T19:40:14.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:13 smithi101 ceph-mon[128136]: pgmap v1035: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-07-21T19:40:14.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:13 smithi005 ceph-mon[126460]: pgmap v1035: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-07-21T19:40:14.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:13 smithi156 ceph-mon[128316]: pgmap v1035: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 438 op/s 2023-07-21T19:40:15.730 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:15 smithi156 ceph-mon[128316]: pgmap v1036: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:16.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:15 smithi101 ceph-mon[128136]: pgmap v1036: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:16.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:15 smithi005 ceph-mon[126460]: pgmap v1036: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 825.0 s, 216.6 tps, lat 466.816 ms stddev 137.985 2023-07-21T19:40:18.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:17 smithi101 ceph-mon[128136]: pgmap v1037: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:18.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:17 smithi005 ceph-mon[126460]: pgmap v1037: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:18.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:17 smithi156 ceph-mon[128316]: pgmap v1037: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.5 MiB/s wr, 433 op/s 2023-07-21T19:40:19.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:18 smithi101 ceph-mon[128136]: osdmap e143: 12 total, 12 up, 12 in 2023-07-21T19:40:19.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:18 smithi101 ceph-mon[128136]: pgmap v1039: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2023-07-21T19:40:19.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:18 smithi005 ceph-mon[126460]: osdmap e143: 12 total, 12 up, 12 in 2023-07-21T19:40:19.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:18 smithi005 ceph-mon[126460]: pgmap v1039: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2023-07-21T19:40:19.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:18 smithi156 ceph-mon[128316]: osdmap e143: 12 total, 12 up, 12 in 2023-07-21T19:40:19.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:18 smithi156 ceph-mon[128316]: pgmap v1039: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 4.8 MiB/s wr, 478 op/s 2023-07-21T19:40:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.665+0000 7fc8aedff700 1 -- 172.21.15.5:0/3873326641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80682b0 msgr2=0x7fc8a810c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.665+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3873326641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80682b0 0x7fc8a810c000 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fc894009990 tx=0x7fc894009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.649 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 -- 172.21.15.5:0/3873326641 shutdown_connections 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3873326641 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a810c540 0x7fc8a810e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3873326641 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a80679a0 0x7fc8a8067d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3873326641 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80682b0 0x7fc8a810c000 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 -- 172.21.15.5:0/3873326641 >> 172.21.15.5:0/3873326641 conn(0x7fc8a8074760 msgr2=0x7fc8a8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 -- 172.21.15.5:0/3873326641 shutdown_connections 2023-07-21T19:40:19.650 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.666+0000 7fc8aedff700 1 -- 172.21.15.5:0/3873326641 wait complete. 2023-07-21T19:40:19.651 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.667+0000 7fc8aedff700 1 Processor -- start 2023-07-21T19:40:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.667+0000 7fc8aedff700 1 -- start start 2023-07-21T19:40:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 0x7fc8a8108a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.652 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a810c540 0x7fc8a8108fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81095b0 con 0x7fc8a810c540 2023-07-21T19:40:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a8109720 con 0x7fc8a80679a0 2023-07-21T19:40:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8aedff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a81098e0 con 0x7fc8a80682b0 2023-07-21T19:40:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8acb9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.653 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 0x7fc8a8108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8acb9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35382/0 (socket says 172.21.15.5:35382) 2023-07-21T19:40:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 0x7fc8a8108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57210/0 (socket says 172.21.15.5:57210) 2023-07-21T19:40:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3257710177 learned_addr learned my addr 172.21.15.5:0/3257710177 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:19.654 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.668+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a810c540 0x7fc8a8108fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3257710177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 msgr2=0x7fc8a8108a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 0x7fc8a8108a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a810c540 msgr2=0x7fc8a8108fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a810c540 0x7fc8a8108fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.655 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3257710177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc89c009520 con 0x7fc8a80679a0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eb30 tx=0x7fc89c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8a5ffb700 1 -- 172.21.15.5:0/3257710177 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c009880 con 0x7fc8a80679a0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8a5ffb700 1 -- 172.21.15.5:0/3257710177 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c0044b0 con 0x7fc8a80679a0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8a5ffb700 1 -- 172.21.15.5:0/3257710177 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c003df0 con 0x7fc8a80679a0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.669+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8940095f0 con 0x7fc8a80679a0 2023-07-21T19:40:19.656 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.670+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a81a8eb0 con 0x7fc8a80679a0 2023-07-21T19:40:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.674+0000 7fc8a5ffb700 1 -- 172.21.15.5:0/3257710177 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc89c003f50 con 0x7fc8a80679a0 2023-07-21T19:40:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.675+0000 7fc8c27d8b80 1 Processor -- start 2023-07-21T19:40:19.658 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.675+0000 7fc8c27d8b80 1 -- start start 2023-07-21T19:40:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.659 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de886e9c50 con 0x55de886406b0 2023-07-21T19:40:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de8873c9e0 con 0x55de887e4580 2023-07-21T19:40:19.660 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de88735e70 con 0x55de887ea2d0 2023-07-21T19:40:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8ad39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8acb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35390/0 (socket says 172.21.15.5:35390) 2023-07-21T19:40:19.661 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.676+0000 7fc8a7fff700 1 -- 172.21.15.5:0/3674502788 learned_addr learned my addr 172.21.15.5:0/3674502788 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3674502788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 msgr2=0x55de887e49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3674502788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 msgr2=0x55de887e9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 -- 172.21.15.5:0/3674502788 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de88563d80 con 0x55de887ea2d0 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:19.662 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887e4040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ca790 tx=0x7fc89c0d0d70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=72b1720274a3fa1d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.677+0000 7fc8a5ffb700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 0x7fc8980a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.678+0000 7fc892ffd700 1 -- 172.21.15.5:0/3674502788 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c096450 con 0x55de887ea2d0 2023-07-21T19:40:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.678+0000 7fc892ffd700 1 -- 172.21.15.5:0/3674502788 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c099ee0 con 0x55de887ea2d0 2023-07-21T19:40:19.663 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.678+0000 7fc8a5ffb700 1 -- 172.21.15.5:0/3257710177 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc89c014070 con 0x7fc8a80679a0 2023-07-21T19:40:19.664 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.678+0000 7fc892ffd700 1 -- 172.21.15.5:0/3674502788 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c099a70 con 0x55de887ea2d0 2023-07-21T19:40:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.678+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 0x7fc8980a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/3674502788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 msgr2=0x55de887e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887e4040 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fc89c0ca790 tx=0x7fc89c0d0d70 comp rx=0 tx=0).stop 2023-07-21T19:40:19.665 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/3674502788 shutdown_connections 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887e49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/3674502788 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887e4040 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/3674502788 >> 172.21.15.5:0/3674502788 conn(0x55de887d9be0 msgr2=0x55de887def10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 0x7fc8980a3870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80733d0 tx=0x7fc8a0009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/3674502788 shutdown_connections 2023-07-21T19:40:19.666 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.679+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/3674502788 wait complete. 2023-07-21T19:40:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.680+0000 7fc8c27d8b80 1 -- start start 2023-07-21T19:40:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.680+0000 7fc8c27d8b80 1 Processor -- start 2023-07-21T19:40:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887f0580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.667 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de886d5a60 con 0x55de886406b0 2023-07-21T19:40:19.668 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de886d5760 con 0x55de887e4580 2023-07-21T19:40:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de885b98c0 con 0x55de887ea2d0 2023-07-21T19:40:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.681+0000 7fc8c27d8b80 10 client.0 _ll_register_callbacks cb 0x55de886d3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57218/0 (socket says 172.21.15.5:57218) 2023-07-21T19:40:19.669 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1549687184 learned_addr learned my addr 172.21.15.5:0/1549687184 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887f0580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.671 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 msgr2=0x55de887ed870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887ed870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1549687184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 msgr2=0x55de887f0580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887f0580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1549687184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a0009900 con 0x55de887ea2d0 2023-07-21T19:40:19.672 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.682+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0018e40 tx=0x7fc8a0004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=88ddb4b7039239d5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc887fff700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0016c90 con 0x55de887ea2d0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55de8868f490 con 0x55de887ea2d0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc8a0009c30 con 0x55de887ea2d0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a0016df0 con 0x55de887ea2d0 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc8c27d8b80 10 client.30526 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:19.673 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.683+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x7fc8a81a8eb0 con 0x55de887ea2d0 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55de8868f490 con 0x55de887ea2d0 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc8a0023c50 con 0x55de887ea2d0 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 10 client.30526 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 10 client.30526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 20 client.30526 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:19.674 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887fdc90 0x55de888000a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 4 client.30526 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.684+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55de8872ee50 con 0x55de887fdc90 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.685+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887fdc90 0x55de888000a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.685+0000 7fc887fff700 10 client.30526 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.685+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887fdc90 0x55de888000a0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a941ac4e082a735a server_cookie=f93625b2bef55082 in_seq=0 out_seq=0 2023-07-21T19:40:19.675 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.686+0000 7fc887fff700 1 -- 172.21.15.5:0/1549687184 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55de8872ee50 con 0x55de887fdc90 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.686+0000 7fc887fff700 10 client.30526 handle_command_reply: tid=0 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.686+0000 7fc8c27d8b80 1 client.30526 shutdown 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.686+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 msgr2=0x55de887edef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.686+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fc8a0018e40 tx=0x7fc8a0004ad0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 shutdown_connections 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887ea2d0 0x55de887edef0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.676 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887fdc90 0x55de888000a0 crc :-1 s=READY pgs=578 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887f0580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc887fff700 0 client.30526 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1549687184 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887ed870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 >> 172.21.15.5:0/1549687184 conn(0x55de887d9be0 msgr2=0x55de887ddc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 shutdown_connections 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1549687184 wait complete. 2023-07-21T19:40:19.677 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.687+0000 7fc8c27d8b80 20 client.30526 trim_cache size 0 max 16384 2023-07-21T19:40:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 Processor -- start 2023-07-21T19:40:19.698 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 -- start start 2023-07-21T19:40:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887dec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.699 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887dfff0 con 0x55de886406b0 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887dbd00 con 0x55de887e4580 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.715+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887e99c0 con 0x55de887d9be0 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57222/0 (socket says 172.21.15.5:57222) 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 -- 172.21.15.5:0/2603240708 learned_addr learned my addr 172.21.15.5:0/2603240708 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:19.700 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8a7fff700 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 -- 172.21.15.5:0/2603240708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 msgr2=0x55de887dec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887dec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 -- 172.21.15.5:0/2603240708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 msgr2=0x55de887e9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.701 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 -- 172.21.15.5:0/2603240708 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55de88804cd0 con 0x55de887d9be0 2023-07-21T19:40:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.716+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc89c099e10 tx=0x7fc89c004620 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6278d7d5cd625177 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.717+0000 7fc8937fe700 1 -- 172.21.15.5:0/2603240708 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c0d0400 con 0x55de887d9be0 2023-07-21T19:40:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.717+0000 7fc8937fe700 1 -- 172.21.15.5:0/2603240708 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c099a70 con 0x55de887d9be0 2023-07-21T19:40:19.702 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.717+0000 7fc8937fe700 1 -- 172.21.15.5:0/2603240708 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc89c09a110 con 0x55de887d9be0 2023-07-21T19:40:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/2603240708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 msgr2=0x55de887e4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fc89c099e10 tx=0x7fc89c004620 comp rx=0 tx=0).stop 2023-07-21T19:40:19.703 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/2603240708 shutdown_connections 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887e4580 0x55de887dec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de887d9be0 0x55de887e4040 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/2603240708 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de886406b0 0x55de887e9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.718+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/2603240708 >> 172.21.15.5:0/2603240708 conn(0x55de887f0510 msgr2=0x55de887e89f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.719+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/2603240708 shutdown_connections 2023-07-21T19:40:19.704 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.719+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/2603240708 wait complete. 2023-07-21T19:40:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.719+0000 7fc8c27d8b80 1 -- start start 2023-07-21T19:40:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 Processor -- start 2023-07-21T19:40:19.705 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de886406b0 0x55de8884ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 0x55de8884b270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887f3190 con 0x55de887e4580 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887f3300 con 0x55de887d9be0 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55de887f34c0 con 0x55de886406b0 2023-07-21T19:40:19.706 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.720+0000 7fc8c27d8b80 10 client.0 _ll_register_callbacks cb 0x7fc8a0018810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.721+0000 7fc8ad39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.721+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 0x55de8884b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.721+0000 7fc8a7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 0x55de8884b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35416/0 (socket says 172.21.15.5:35416) 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.721+0000 7fc8a7fff700 1 -- 172.21.15.5:0/1401770915 learned_addr learned my addr 172.21.15.5:0/1401770915 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.721+0000 7fc8ad39c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:56756/0 (socket says 172.21.15.5:56756) 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de886406b0 0x55de8884ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1401770915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de886406b0 msgr2=0x55de8884ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.707 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de886406b0 0x55de8884ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 msgr2=0x55de8884b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 0x55de8884b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 -- 172.21.15.5:0/1401770915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55de88563d80 con 0x55de887e4580 2023-07-21T19:40:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc8ad39c700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00050f0 tx=0x7fc8a0009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f972d6895efe3241 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:19.708 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.722+0000 7fc887fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:40:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.723+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a0039070 con 0x55de887e4580 2023-07-21T19:40:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.723+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc8a0004d60 con 0x55de887e4580 2023-07-21T19:40:19.709 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.723+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc89c099ee0 con 0x55de887e4580 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.723+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc8a0016a90 con 0x55de887e4580 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 10 client.20820 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55de8854cc00 con 0x55de887e4580 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc89c099ee0 con 0x55de887e4580 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc8a003c030 con 0x55de887e4580 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 10 client.20820 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 10 client.20820 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:19.710 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 20 client.20820 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.724+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887ea2d0 0x55de88854bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.725+0000 7fc8c27d8b80 4 client.20820 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:40:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.725+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55de886f0ff0 con 0x55de887ea2d0 2023-07-21T19:40:19.711 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.726+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887ea2d0 0x55de88854bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.727+0000 7fc8acb9b700 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887ea2d0 0x55de88854bb0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f4405b49897101c server_cookie=436fa79f210e1cbb in_seq=0 out_seq=0 2023-07-21T19:40:19.712 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.727+0000 7fc887fff700 10 client.20820 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.777+0000 7fc887fff700 1 -- 172.21.15.5:0/1401770915 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55de886f0ff0 con 0x55de887ea2d0 2023-07-21T19:40:19.760 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.777+0000 7fc887fff700 10 client.20820 handle_command_reply: tid=0 2023-07-21T19:40:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.777+0000 7fc8c27d8b80 1 client.20820 shutdown 2023-07-21T19:40:19.761 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.777+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 msgr2=0x55de887f2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00050f0 tx=0x7fc8a0009e60 comp rx=0 tx=0).stop 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 shutdown_connections 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55de887ea2d0 0x55de88854bb0 crc :-1 s=READY pgs=579 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55de887e4580 0x55de887f2c50 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55de886406b0 0x55de8884ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.762 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 --2- 172.21.15.5:0/1401770915 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55de887d9be0 0x55de8884b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 >> 172.21.15.5:0/1401770915 conn(0x55de887f0510 msgr2=0x55de887e9280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc887fff700 0 client.20820 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 shutdown_connections 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 1 -- 172.21.15.5:0/1401770915 wait complete. 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.778+0000 7fc8c27d8b80 20 client.20820 trim_cache size 0 max 16384 2023-07-21T19:40:19.763 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:40:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:40:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:40:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:40:19.764 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.779+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 msgr2=0x7fc8980a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.779+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 0x7fc8980a3870 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fc8a80733d0 tx=0x7fc8a0009040 comp rx=0 tx=0).stop 2023-07-21T19:40:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.779+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 msgr2=0x7fc8a810bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.779+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fc89c00eb30 tx=0x7fc89c00ee40 comp rx=0 tx=0).stop 2023-07-21T19:40:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.779+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 shutdown_connections 2023-07-21T19:40:19.765 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc8980a13c0 0x7fc8980a3870 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc8a810c540 0x7fc8a8108fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc8a80679a0 0x7fc8a810bc60 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 --2- 172.21.15.5:0/3257710177 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc8a80682b0 0x7fc8a8108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 >> 172.21.15.5:0/3257710177 conn(0x7fc8a8074760 msgr2=0x7fc8a80fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 shutdown_connections 2023-07-21T19:40:19.766 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:19.780+0000 7fc8aedff700 1 -- 172.21.15.5:0/3257710177 wait complete. 2023-07-21T19:40:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 830.0 s, 222.8 tps, lat 439.937 ms stddev 131.439 2023-07-21T19:40:22.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:21 smithi101 ceph-mon[128136]: pgmap v1040: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-07-21T19:40:22.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:21 smithi005 ceph-mon[126460]: pgmap v1040: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-07-21T19:40:22.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:21 smithi156 ceph-mon[128316]: pgmap v1040: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.1 MiB/s wr, 527 op/s 2023-07-21T19:40:24.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:23 smithi101 ceph-mon[128136]: pgmap v1041: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 752 op/s 2023-07-21T19:40:24.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:23 smithi005 ceph-mon[126460]: pgmap v1041: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 752 op/s 2023-07-21T19:40:24.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:23 smithi156 ceph-mon[128316]: pgmap v1041: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 752 op/s 2023-07-21T19:40:26.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:25 smithi101 ceph-mon[128136]: pgmap v1042: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:26.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:25 smithi005 ceph-mon[126460]: pgmap v1042: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:26.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:25 smithi156 ceph-mon[128316]: pgmap v1042: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 835.0 s, 223.8 tps, lat 448.450 ms stddev 154.424 2023-07-21T19:40:28.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:27 smithi101 ceph-mon[128136]: pgmap v1043: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:28.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:27 smithi005 ceph-mon[126460]: pgmap v1043: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:28.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:27 smithi156 ceph-mon[128316]: pgmap v1043: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 7.5 MiB/s wr, 785 op/s 2023-07-21T19:40:29.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:28 smithi101 ceph-mon[128136]: pgmap v1044: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 734 op/s 2023-07-21T19:40:29.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:28 smithi005 ceph-mon[126460]: pgmap v1044: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 734 op/s 2023-07-21T19:40:29.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:28 smithi156 ceph-mon[128316]: pgmap v1044: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 6.9 MiB/s wr, 734 op/s 2023-07-21T19:40:31.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 840.0 s, 71.6 tps, lat 1138.277 ms stddev 1194.388 2023-07-21T19:40:32.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:31 smithi101 ceph-mon[128136]: pgmap v1045: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 687 op/s 2023-07-21T19:40:32.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:31 smithi005 ceph-mon[126460]: pgmap v1045: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 687 op/s 2023-07-21T19:40:32.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:31 smithi156 ceph-mon[128316]: pgmap v1045: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.5 MiB/s wr, 687 op/s 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 -- 172.21.15.5:0/3318539937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 msgr2=0x7f57200fffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 --2- 172.21.15.5:0/3318539937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200fffa0 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f5714009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 -- 172.21.15.5:0/3318539937 shutdown_connections 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 --2- 172.21.15.5:0/3318539937 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 0x7f572010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 --2- 172.21.15.5:0/3318539937 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f5720105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.933 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 --2- 172.21.15.5:0/3318539937 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200fffa0 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 -- 172.21.15.5:0/3318539937 >> 172.21.15.5:0/3318539937 conn(0x7f57200fb5f0 msgr2=0x7f57200fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 -- 172.21.15.5:0/3318539937 shutdown_connections 2023-07-21T19:40:32.934 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.948+0000 7f5724fff700 1 -- 172.21.15.5:0/3318539937 wait complete. 2023-07-21T19:40:32.935 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.949+0000 7f5724fff700 1 Processor -- start 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 -- start start 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 0x7f5720072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f57201a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57201a41c0 con 0x7f57200ffb30 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57201a4300 con 0x7f57201004e0 2023-07-21T19:40:32.936 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f5724fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57201a4470 con 0x7f5720105b60 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f571e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f571e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38206/0 (socket says 172.21.15.5:38206) 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f571e59c700 1 -- 172.21.15.5:0/53113748 learned_addr learned my addr 172.21.15.5:0/53113748 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.950+0000 7f571dd9b700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 0x7f5720072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571ed9d700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f57201a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 -- 172.21.15.5:0/53113748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 msgr2=0x7f57201a3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f57201a3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.937 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 -- 172.21.15.5:0/53113748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 msgr2=0x7f5720072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 0x7f5720072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 -- 172.21.15.5:0/53113748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57140095f0 con 0x7f57200ffb30 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571ed9d700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f57201a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f571e59c700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f571000d980 tx=0x7f571000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f570f7fe700 1 -- 172.21.15.5:0/53113748 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710004220 con 0x7f57200ffb30 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f570f7fe700 1 -- 172.21.15.5:0/53113748 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5710007640 con 0x7f57200ffb30 2023-07-21T19:40:32.938 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57201a4750 con 0x7f57200ffb30 2023-07-21T19:40:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f570f7fe700 1 -- 172.21.15.5:0/53113748 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100055e0 con 0x7f57200ffb30 2023-07-21T19:40:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.951+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57201a4cb0 con 0x7f57200ffb30 2023-07-21T19:40:32.939 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.955+0000 7f570f7fe700 1 -- 172.21.15.5:0/53113748 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f5710004750 con 0x7f57200ffb30 2023-07-21T19:40:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 Processor -- start 2023-07-21T19:40:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 -- start start 2023-07-21T19:40:32.940 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f570f7fe700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 0x7f57080a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 0x55fccb364d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb3652d0 0x55fccb35f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb35f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f570f7fe700 1 -- 172.21.15.5:0/53113748 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57100ccae0 con 0x7f57200ffb30 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb264c50 con 0x55fccb1bb6b0 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb2b79e0 con 0x55fccb35f580 2023-07-21T19:40:32.941 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.956+0000 7f5738a07b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb2b0e70 con 0x55fccb3652d0 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571ed9d700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 0x7f57080a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb35f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 0x55fccb364d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571ed9d700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb35f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57686/0 (socket says 172.21.15.5:57686) 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb3652d0 0x55fccb35f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.943 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571ed9d700 1 -- 172.21.15.5:0/2010840988 learned_addr learned my addr 172.21.15.5:0/2010840988 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 -- 172.21.15.5:0/2010840988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb3652d0 msgr2=0x55fccb35f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb3652d0 0x55fccb35f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 -- 172.21.15.5:0/2010840988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 msgr2=0x55fccb35f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb35f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571e59c700 1 -- 172.21.15.5:0/2010840988 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fccb0ded80 con 0x55fccb1bb6b0 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.957+0000 7f571ed9d700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 0x7f57080a37f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5718009fd0 tx=0x7f5718011440 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.944 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f571e59c700 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 0x55fccb364d90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f57100ca1d0 tx=0x7f5710005740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a669d2f842e951 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f570cff9700 1 -- 172.21.15.5:0/2010840988 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000f3c0 con 0x55fccb1bb6b0 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f570cff9700 1 -- 172.21.15.5:0/2010840988 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5710099b50 con 0x55fccb1bb6b0 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f570cff9700 1 -- 172.21.15.5:0/2010840988 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57100992a0 con 0x55fccb1bb6b0 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f5738a07b80 1 -- 172.21.15.5:0/2010840988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 msgr2=0x55fccb364d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.958+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 0x55fccb364d90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7f57100ca1d0 tx=0x7f5710005740 comp rx=0 tx=0).stop 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 -- 172.21.15.5:0/2010840988 shutdown_connections 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb35f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.945 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb1bb6b0 0x55fccb364d90 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2010840988 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb3652d0 0x55fccb35f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 -- 172.21.15.5:0/2010840988 >> 172.21.15.5:0/2010840988 conn(0x55fccb354be0 msgr2=0x55fccb359f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 -- 172.21.15.5:0/2010840988 shutdown_connections 2023-07-21T19:40:32.946 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 -- 172.21.15.5:0/2010840988 wait complete. 2023-07-21T19:40:32.948 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 -- start start 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.959+0000 7f5738a07b80 1 Processor -- start 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb35f580 0x55fccb368e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb3652d0 0x55fccb369340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb250a60 con 0x55fccb3652d0 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb250760 con 0x55fccb1bb6b0 2023-07-21T19:40:32.949 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb1348c0 con 0x55fccb35f580 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f5738a07b80 10 client.0 _ll_register_callbacks cb 0x55fccb24e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f571e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f571ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb3652d0 0x55fccb369340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f571e59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57698/0 (socket says 172.21.15.5:57698) 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f571e59c700 1 -- 172.21.15.5:0/2382650934 learned_addr learned my addr 172.21.15.5:0/2382650934 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.960+0000 7f571dd9b700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb35f580 0x55fccb368e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 -- 172.21.15.5:0/2382650934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb35f580 msgr2=0x55fccb368e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb35f580 0x55fccb368e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 -- 172.21.15.5:0/2382650934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb3652d0 msgr2=0x55fccb369340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb3652d0 0x55fccb369340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 -- 172.21.15.5:0/2382650934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57201a4cb0 con 0x55fccb1bb6b0 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f571e59c700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f57100ca230 tx=0x7f57100b2460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf1e9d70096d3751 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5705ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710099250 con 0x55fccb1bb6b0 2023-07-21T19:40:32.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fccb20a490 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f5710099db0 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5710099f70 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.961+0000 7f5738a07b80 10 client.30593 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55fccb0c7c00 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55fccb20a490 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f571007e050 con 0x55fccb1bb6b0 2023-07-21T19:40:32.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 10 client.30593 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 10 client.30593 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 20 client.30593 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb378630 0x55fccb37aa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 4 client.30593 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fccb2a9e50 con 0x55fccb378630 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.962+0000 7f571dd9b700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb378630 0x55fccb37aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.963+0000 7f5705ffb700 10 client.30593 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:32.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.963+0000 7f571dd9b700 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb378630 0x55fccb37aa10 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af82a11f943872d3 server_cookie=1550c5e37e9cf5c7 in_seq=0 out_seq=0 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5705ffb700 1 -- 172.21.15.5:0/2382650934 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55fccb2a9e50 con 0x55fccb378630 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5705ffb700 10 client.30593 handle_command_reply: tid=0 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 client.30593 shutdown 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 msgr2=0x55fccb3688c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f57100ca230 tx=0x7f57100b2460 comp rx=0 tx=0).stop 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 shutdown_connections 2023-07-21T19:40:32.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb378630 0x55fccb37aa10 crc :-1 s=READY pgs=580 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb3652d0 0x55fccb369340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3688c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5705ffb700 0 client.30593 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 --2- 172.21.15.5:0/2382650934 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb35f580 0x55fccb368e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.964+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 >> 172.21.15.5:0/2382650934 conn(0x55fccb354be0 msgr2=0x55fccb3594e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.965+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 shutdown_connections 2023-07-21T19:40:32.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.965+0000 7f5738a07b80 1 -- 172.21.15.5:0/2382650934 wait complete. 2023-07-21T19:40:32.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.965+0000 7f5738a07b80 20 client.30593 trim_cache size 0 max 16384 2023-07-21T19:40:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.987+0000 7f5738a07b80 1 Processor -- start 2023-07-21T19:40:32.971 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.987+0000 7f5738a07b80 1 -- start start 2023-07-21T19:40:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb354be0 0x55fccb3642e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb359c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.972 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb20a490 con 0x55fccb354be0 2023-07-21T19:40:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb325060 con 0x55fccb35f580 2023-07-21T19:40:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f5738a07b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb35ba70 con 0x55fccb1bb6b0 2023-07-21T19:40:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f571dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.973 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f571e59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb354be0 0x55fccb3642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.988+0000 7f571dd9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59740/0 (socket says 172.21.15.5:59740) 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 -- 172.21.15.5:0/278603611 learned_addr learned my addr 172.21.15.5:0/278603611 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571ed9d700 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb359c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 -- 172.21.15.5:0/278603611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 msgr2=0x55fccb359c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb359c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 -- 172.21.15.5:0/278603611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb354be0 msgr2=0x55fccb3642e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.974 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb354be0 0x55fccb3642e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 -- 172.21.15.5:0/278603611 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fccb369480 con 0x55fccb1bb6b0 2023-07-21T19:40:32.975 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.989+0000 7f571dd9b700 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f571400b260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aa891812f2f56a9c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.990+0000 7f570d7fa700 1 -- 172.21.15.5:0/278603611 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571402a070 con 0x55fccb1bb6b0 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.990+0000 7f570d7fa700 1 -- 172.21.15.5:0/278603611 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f571402e070 con 0x55fccb1bb6b0 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.990+0000 7f570d7fa700 1 -- 172.21.15.5:0/278603611 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571401d040 con 0x55fccb1bb6b0 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- 172.21.15.5:0/278603611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 msgr2=0x55fccb36fb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f5714009990 tx=0x7f571400b260 comp rx=0 tx=0).stop 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- 172.21.15.5:0/278603611 shutdown_connections 2023-07-21T19:40:32.976 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb35f580 0x55fccb359c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb354be0 0x55fccb3642e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 --2- 172.21.15.5:0/278603611 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb1bb6b0 0x55fccb36fb60 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- 172.21.15.5:0/278603611 >> 172.21.15.5:0/278603611 conn(0x55fccb374660 msgr2=0x55fccb365a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- 172.21.15.5:0/278603611 shutdown_connections 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- 172.21.15.5:0/278603611 wait complete. 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.991+0000 7f5738a07b80 1 -- start start 2023-07-21T19:40:32.977 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 Processor -- start 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3c5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 0x55fccb3c6360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb3c09f0 con 0x55fccb35f580 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb3c0b60 con 0x55fccb1bb6b0 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fccb3c0d20 con 0x55fccb354be0 2023-07-21T19:40:32.979 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.992+0000 7f5738a07b80 10 client.0 _ll_register_callbacks cb 0x7f5718011930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.993+0000 7f571ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.993+0000 7f571ed9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:38228/0 (socket says 172.21.15.5:38228) 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.993+0000 7f571ed9d700 1 -- 172.21.15.5:0/1071056145 learned_addr learned my addr 172.21.15.5:0/1071056145 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.993+0000 7f571dd9b700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 0x55fccb3c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.993+0000 7f571e59c700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3c5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 -- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 msgr2=0x55fccb3c5e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.980 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3c5e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 -- 172.21.15.5:0/1071056145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 msgr2=0x55fccb3c0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 -- 172.21.15.5:0/1071056145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f572010d220 con 0x55fccb354be0 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571ed9d700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f571dd9b700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 0x55fccb3c6360 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f57140052b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f37f871f2505b7dd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f5705ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571402a050 con 0x55fccb354be0 2023-07-21T19:40:32.981 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5710099b50 con 0x55fccb354be0 2023-07-21T19:40:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.994+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f57140056b0 con 0x55fccb354be0 2023-07-21T19:40:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f571401d040 con 0x55fccb354be0 2023-07-21T19:40:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 10 client.30547 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:32.982 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55fccb1051f0 con 0x55fccb354be0 2023-07-21T19:40:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f5710099b50 con 0x55fccb354be0 2023-07-21T19:40:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f571403c070 con 0x55fccb354be0 2023-07-21T19:40:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 10 client.30547 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 10 client.30547 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:32.983 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 20 client.30547 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb3652d0 0x55fccb3cf760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 4 client.30547 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.995+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55fccb26bff0 con 0x55fccb3652d0 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.996+0000 7f571e59c700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb3652d0 0x55fccb3cf760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.997+0000 7f571e59c700 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb3652d0 0x55fccb3cf760 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=24273deb880810f5 server_cookie=5c8418efd6f36adf in_seq=0 out_seq=0 2023-07-21T19:40:32.984 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:32.997+0000 7f5705ffb700 10 client.30547 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5705ffb700 1 -- 172.21.15.5:0/1071056145 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55fccb26bff0 con 0x55fccb3652d0 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5705ffb700 10 client.30547 handle_command_reply: tid=0 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 client.30547 shutdown 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 msgr2=0x55fccb3c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 0x55fccb3c6360 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f57140098e0 tx=0x7f57140052b0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.029 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 shutdown_connections 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55fccb3652d0 0x55fccb3cf760 crc :-1 s=READY pgs=581 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55fccb35f580 0x55fccb3c0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55fccb1bb6b0 0x55fccb3c5e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 --2- 172.21.15.5:0/1071056145 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55fccb354be0 0x55fccb3c6360 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5705ffb700 0 client.30547 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.044+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 >> 172.21.15.5:0/1071056145 conn(0x55fccb374660 msgr2=0x55fccb365a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:33.030 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.045+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 shutdown_connections 2023-07-21T19:40:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.045+0000 7f5738a07b80 1 -- 172.21.15.5:0/1071056145 wait complete. 2023-07-21T19:40:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.045+0000 7f5738a07b80 20 client.30547 trim_cache size 0 max 16384 2023-07-21T19:40:33.031 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:40:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:40:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:40:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:40:33.032 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 msgr2=0x7f57080a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 0x7f57080a37f0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5718009fd0 tx=0x7f5718011440 comp rx=0 tx=0).stop 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 msgr2=0x7f57200729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f571000d980 tx=0x7f571000dc90 comp rx=0 tx=0).stop 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 shutdown_connections 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f5720105b60 0x7f57201a3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f57080a1340 0x7f57080a37f0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.033 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f57200ffb30 0x7f57200729f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 --2- 172.21.15.5:0/53113748 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f57201004e0 0x7f5720072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 >> 172.21.15.5:0/53113748 conn(0x7f57200fb5f0 msgr2=0x7f5720103730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 shutdown_connections 2023-07-21T19:40:33.034 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:33.046+0000 7f5724fff700 1 -- 172.21.15.5:0/53113748 wait complete. 2023-07-21T19:40:34.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:33 smithi101 ceph-mon[128136]: pgmap v1046: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 759 op/s 2023-07-21T19:40:34.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:33 smithi005 ceph-mon[126460]: pgmap v1046: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 759 op/s 2023-07-21T19:40:34.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:33 smithi156 ceph-mon[128316]: pgmap v1046: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 7.2 MiB/s wr, 759 op/s 2023-07-21T19:40:34.716 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:40:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.071+0000 7f1b105ff700 1 -- 172.21.15.5:0/2548130802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b080ffb00 msgr2=0x7f1b080fff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.071+0000 7f1b105ff700 1 --2- 172.21.15.5:0/2548130802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b080ffb00 0x7f1b080fff70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f1af8009990 tx=0x7f1af8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 -- 172.21.15.5:0/2548130802 shutdown_connections 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 --2- 172.21.15.5:0/2548130802 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b081004b0 0x7f1b0810c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 --2- 172.21.15.5:0/2548130802 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 0x7f1b08105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 --2- 172.21.15.5:0/2548130802 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b080ffb00 0x7f1b080fff70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 -- 172.21.15.5:0/2548130802 >> 172.21.15.5:0/2548130802 conn(0x7f1b080fb600 msgr2=0x7f1b080fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 -- 172.21.15.5:0/2548130802 shutdown_connections 2023-07-21T19:40:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.072+0000 7f1b105ff700 1 -- 172.21.15.5:0/2548130802 wait complete. 2023-07-21T19:40:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.074+0000 7f1b105ff700 1 Processor -- start 2023-07-21T19:40:35.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.074+0000 7f1b105ff700 1 -- start start 2023-07-21T19:40:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b080ffb00 0x7f1b08197db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 0x7f1b0819c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819cd90 con 0x7f1b08105b10 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819cf00 con 0x7f1b081004b0 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b105ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0819d0c0 con 0x7f1b080ffb00 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b0db9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b0e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b080ffb00 0x7f1b08197db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b0db9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57724/0 (socket says 172.21.15.5:57724) 2023-07-21T19:40:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.075+0000 7f1b0db9a700 1 -- 172.21.15.5:0/657599405 learned_addr learned my addr 172.21.15.5:0/657599405 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 0x7f1b0819c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 -- 172.21.15.5:0/657599405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b080ffb00 msgr2=0x7f1b08197db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b080ffb00 0x7f1b08197db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 -- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 msgr2=0x7f1b0819c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 0x7f1b0819c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 -- 172.21.15.5:0/657599405 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b04009c40 con 0x7f1b081004b0 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.076+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005740 tx=0x7f1af8005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.077+0000 7f1aff7fe700 1 -- 172.21.15.5:0/657599405 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af801d070 con 0x7f1b081004b0 2023-07-21T19:40:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.077+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1af80095f0 con 0x7f1b081004b0 2023-07-21T19:40:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.077+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b081a4a00 con 0x7f1b081004b0 2023-07-21T19:40:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.077+0000 7f1aff7fe700 1 -- 172.21.15.5:0/657599405 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1af8022470 con 0x7f1b081004b0 2023-07-21T19:40:35.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.077+0000 7f1aff7fe700 1 -- 172.21.15.5:0/657599405 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1af8017650 con 0x7f1b081004b0 2023-07-21T19:40:35.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.083+0000 7f1b23e9db80 1 Processor -- start 2023-07-21T19:40:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.083+0000 7f1b23e9db80 1 -- start start 2023-07-21T19:40:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.083+0000 7f1aff7fe700 1 -- 172.21.15.5:0/657599405 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1af800fb00 con 0x7f1b081004b0 2023-07-21T19:40:35.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a6bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 0x564254a6e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425495ecb0 con 0x564254a6c2e0 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642549b1a40 con 0x564254a60ae0 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5642549aaed0 con 0x564254a613f0 2023-07-21T19:40:35.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b0eb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b0eb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57738/0 (socket says 172.21.15.5:57738) 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/945254061 learned_addr learned my addr 172.21.15.5:0/945254061 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.084+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a6bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 0x564254a6e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/945254061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 msgr2=0x564254a6bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a6bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/945254061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 msgr2=0x564254a6e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 0x564254a6e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/945254061 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5642547d8de0 con 0x564254a60ae0 2023-07-21T19:40:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 0x564254a6e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:40:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009ca0 tx=0x7f1b000113e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7de27e414a15ccce server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1afcbfe700 1 -- 172.21.15.5:0/945254061 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b000097f0 con 0x564254a60ae0 2023-07-21T19:40:35.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1afcbfe700 1 -- 172.21.15.5:0/945254061 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b00011d10 con 0x564254a60ae0 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.085+0000 7f1afcbfe700 1 -- 172.21.15.5:0/945254061 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b0000f720 con 0x564254a60ae0 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.086+0000 7f1aff7fe700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 0x7f1af40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.086+0000 7f1b23e9db80 1 -- 172.21.15.5:0/945254061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 msgr2=0x564254a60eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.086+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1b00009ca0 tx=0x7f1b000113e0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.086+0000 7f1aff7fe700 1 -- 172.21.15.5:0/657599405 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1af80cdf80 con 0x7f1b081004b0 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 -- 172.21.15.5:0/945254061 shutdown_connections 2023-07-21T19:40:35.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a6c2e0 0x564254a6e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a60eb0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/945254061 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a6bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 -- 172.21.15.5:0/945254061 >> 172.21.15.5:0/945254061 conn(0x564254a4eda0 msgr2=0x564254a546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 0x7f1af40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 -- 172.21.15.5:0/945254061 shutdown_connections 2023-07-21T19:40:35.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.087+0000 7f1b23e9db80 1 -- 172.21.15.5:0/945254061 wait complete. 2023-07-21T19:40:35.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.088+0000 7f1b23e9db80 1 -- start start 2023-07-21T19:40:35.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.088+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 0x7f1af40a37f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1b040075c0 tx=0x7f1b0400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 Processor -- start 2023-07-21T19:40:35.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a60ae0 0x564254a6bc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425494aac0 con 0x564254a60ae0 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425494a7c0 con 0x564254a6c2e0 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.089+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56425482e920 con 0x564254a613f0 2023-07-21T19:40:35.077 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b23e9db80 10 client.0 _ll_register_callbacks cb 0x564254948760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0db9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0eb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0eb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57742/0 (socket says 172.21.15.5:57742) 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0db9a700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59764/0 (socket says 172.21.15.5:59764) 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/1636415548 learned_addr learned my addr 172.21.15.5:0/1636415548 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.090+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a60ae0 0x564254a6bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/1636415548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 msgr2=0x564254a68ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.078 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/1636415548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a60ae0 msgr2=0x564254a6bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a60ae0 0x564254a6bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/1636415548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b00009950 con 0x564254a6c2e0 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b0001c040 tx=0x7f1b00009890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=79a0d683f6455329 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.091+0000 7f1af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:40:35.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00005ce0 con 0x564254a6c2e0 2023-07-21T19:40:35.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1b00012660 con 0x564254a6c2e0 2023-07-21T19:40:35.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5642549044f0 con 0x564254a6c2e0 2023-07-21T19:40:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b00012820 con 0x564254a6c2e0 2023-07-21T19:40:35.081 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1b23e9db80 10 client.30611 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.092+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5642547c1c60 con 0x564254a6c2e0 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5642549044f0 con 0x564254a6c2e0 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b00004980 con 0x564254a6c2e0 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 10 client.30611 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 10 client.30611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:35.082 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 20 client.30611 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254a7ba40 0x564254a7de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 4 client.30611 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5642549a3eb0 con 0x564254a7ba40 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.093+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254a7ba40 0x564254a7de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.094+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254a7ba40 0x564254a7de50 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7bc5d6f0042eb894 server_cookie=7a191a1623a06aa in_seq=0 out_seq=0 2023-07-21T19:40:35.083 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.094+0000 7f1af1ffb700 10 client.30611 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:35.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.095+0000 7f1af1ffb700 1 -- 172.21.15.5:0/1636415548 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5642549a3eb0 con 0x564254a7ba40 2023-07-21T19:40:35.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.095+0000 7f1af1ffb700 10 client.30611 handle_command_reply: tid=0 2023-07-21T19:40:35.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.095+0000 7f1b23e9db80 1 client.30611 shutdown 2023-07-21T19:40:35.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 msgr2=0x564254a69400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.084 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1b0001c040 tx=0x7f1b00009890 comp rx=0 tx=0).stop 2023-07-21T19:40:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 shutdown_connections 2023-07-21T19:40:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a69400 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254a7ba40 0x564254a7de50 crc :-1 s=READY pgs=582 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a60ae0 0x564254a6bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.085 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/1636415548 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a613f0 0x564254a68ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 >> 172.21.15.5:0/1636415548 conn(0x564254a4eda0 msgr2=0x564254a54230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1af1ffb700 0 client.30611 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 shutdown_connections 2023-07-21T19:40:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.096+0000 7f1b23e9db80 1 -- 172.21.15.5:0/1636415548 wait complete. 2023-07-21T19:40:35.086 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.097+0000 7f1b23e9db80 20 client.30611 trim_cache size 0 max 16384 2023-07-21T19:40:35.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.122+0000 7f1b23e9db80 1 Processor -- start 2023-07-21T19:40:35.107 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 -- start start 2023-07-21T19:40:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.108 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a6b750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254a55d20 con 0x564254a613f0 2023-07-21T19:40:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254a61d50 con 0x564254a6c2e0 2023-07-21T19:40:35.109 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254a61970 con 0x564254a60ae0 2023-07-21T19:40:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.123+0000 7f1b0e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0e39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:59768/0 (socket says 172.21.15.5:59768) 2023-07-21T19:40:35.110 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0e39b700 1 -- 172.21.15.5:0/4037894646 learned_addr learned my addr 172.21.15.5:0/4037894646 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a6b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/4037894646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 msgr2=0x564254a68660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.111 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/4037894646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 msgr2=0x564254a6b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a6b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/4037894646 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564254a530e0 con 0x564254a6c2e0 2023-07-21T19:40:35.112 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:35.113 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.124+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a641e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b00004d40 tx=0x7f1b00009c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a01bc6707eadfc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.125+0000 7f1afd3ff700 1 -- 172.21.15.5:0/4037894646 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00011e80 con 0x564254a6c2e0 2023-07-21T19:40:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.125+0000 7f1afd3ff700 1 -- 172.21.15.5:0/4037894646 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b00005c80 con 0x564254a6c2e0 2023-07-21T19:40:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.125+0000 7f1afd3ff700 1 -- 172.21.15.5:0/4037894646 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00004970 con 0x564254a6c2e0 2023-07-21T19:40:35.116 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- 172.21.15.5:0/4037894646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 msgr2=0x564254a641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a641e0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1b00004d40 tx=0x7f1b00009c20 comp rx=0 tx=0).stop 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- 172.21.15.5:0/4037894646 shutdown_connections 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a6c2e0 0x564254a641e0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a60ae0 0x564254a68660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/4037894646 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a6b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.117 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- 172.21.15.5:0/4037894646 >> 172.21.15.5:0/4037894646 conn(0x564254a4eda0 msgr2=0x564254a77e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- 172.21.15.5:0/4037894646 shutdown_connections 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- 172.21.15.5:0/4037894646 wait complete. 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.126+0000 7f1b23e9db80 1 -- start start 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.127+0000 7f1b23e9db80 1 Processor -- start 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 0x564254a70790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.118 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254a70ea0 con 0x564254a613f0 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254a71010 con 0x564254a60ae0 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x564254ace930 con 0x564254a6c2e0 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b23e9db80 10 client.0 _ll_register_callbacks cb 0x7f1b00009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b0e39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b0e39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57748/0 (socket says 172.21.15.5:57748) 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b0e39b700 1 -- 172.21.15.5:0/674751326 learned_addr learned my addr 172.21.15.5:0/674751326 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:35.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.128+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 0x564254a70790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0db9a700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 msgr2=0x564254a72e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/674751326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 msgr2=0x564254a701a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a701a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 -- 172.21.15.5:0/674751326 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b04016430 con 0x564254a6c2e0 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:35.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1b0eb9c700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 0x564254a70790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1b000117e0 tx=0x7f1b00009860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=977c17e3ca1afad6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.129+0000 7f1af1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.130+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b00011e80 con 0x564254a6c2e0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.130+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1b00011d10 con 0x564254a6c2e0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.130+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1b00000e40 con 0x564254a6c2e0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1b00004920 con 0x564254a6c2e0 2023-07-21T19:40:35.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 10 client.30565 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5642547c1c60 con 0x564254a6c2e0 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1b00011d10 con 0x564254a6c2e0 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f1b0003d030 con 0x564254a6c2e0 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 10 client.30565 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 10 client.30565 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 20 client.30565 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254acfb90 0x564254ad1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:35.122 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 4 client.30565 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:40:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.131+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x564254966050 con 0x564254acfb90 2023-07-21T19:40:35.123 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.132+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254acfb90 0x564254ad1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.133+0000 7f1b0e39b700 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254acfb90 0x564254ad1f70 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf40ef470339928a server_cookie=e65f6ad06b415036 in_seq=0 out_seq=0 2023-07-21T19:40:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.133+0000 7f1af1ffb700 10 client.30565 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.134+0000 7f1af1ffb700 1 -- 172.21.15.5:0/674751326 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x564254966050 con 0x564254acfb90 2023-07-21T19:40:35.124 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.134+0000 7f1af1ffb700 10 client.30565 handle_command_reply: tid=0 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.134+0000 7f1b23e9db80 1 client.30565 shutdown 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 msgr2=0x564254a70790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 0x564254a70790 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f1b000117e0 tx=0x7f1b00009860 comp rx=0 tx=0).stop 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 shutdown_connections 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x564254a6c2e0 0x564254a70790 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x564254a60ae0 0x564254a72e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x564254acfb90 0x564254ad1f70 crc :-1 s=READY pgs=583 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 --2- 172.21.15.5:0/674751326 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x564254a613f0 0x564254a701a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 >> 172.21.15.5:0/674751326 conn(0x564254a4eda0 msgr2=0x564254a641e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1af1ffb700 0 client.30565 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.135+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 shutdown_connections 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.136+0000 7f1b23e9db80 1 -- 172.21.15.5:0/674751326 wait complete. 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.136+0000 7f1b23e9db80 20 client.30565 trim_cache size 0 max 16384 2023-07-21T19:40:35.126 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:40:35.127 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:40:35.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:40:35.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:40:35.128 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:40:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 msgr2=0x7f1af40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 0x7f1af40a37f0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1b040075c0 tx=0x7f1b0400b040 comp rx=0 tx=0).stop 2023-07-21T19:40:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 msgr2=0x7f1b081982f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f1af8005740 tx=0x7f1af8005770 comp rx=0 tx=0).stop 2023-07-21T19:40:35.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 shutdown_connections 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f1af40a1340 0x7f1af40a37f0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1b08105b10 0x7f1b0819c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1b080ffb00 0x7f1b08197db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 --2- 172.21.15.5:0/657599405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f1b081004b0 0x7f1b081982f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.137+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 >> 172.21.15.5:0/657599405 conn(0x7f1b080fb600 msgr2=0x7f1b080fbbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.138+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 shutdown_connections 2023-07-21T19:40:35.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:40:35.139+0000 7f1b105ff700 1 -- 172.21.15.5:0/657599405 wait complete. 2023-07-21T19:40:35.160 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:40:36.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:35 smithi101 ceph-mon[128136]: pgmap v1047: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2023-07-21T19:40:36.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:35 smithi005 ceph-mon[126460]: pgmap v1047: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2023-07-21T19:40:36.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:35 smithi156 ceph-mon[128316]: pgmap v1047: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 4.3 MiB/s wr, 475 op/s 2023-07-21T19:40:36.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 845.0 s, 241.6 tps, lat 495.914 ms stddev 565.459 2023-07-21T19:40:38.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:37 smithi101 ceph-mon[128136]: pgmap v1048: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2023-07-21T19:40:38.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:37 smithi005 ceph-mon[126460]: pgmap v1048: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2023-07-21T19:40:38.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:37 smithi156 ceph-mon[128316]: pgmap v1048: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 3.8 MiB/s wr, 419 op/s 2023-07-21T19:40:39.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:38 smithi101 ceph-mon[128136]: pgmap v1049: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 662 op/s 2023-07-21T19:40:39.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:38 smithi005 ceph-mon[126460]: pgmap v1049: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 662 op/s 2023-07-21T19:40:39.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:38 smithi156 ceph-mon[128316]: pgmap v1049: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 6.3 MiB/s wr, 662 op/s 2023-07-21T19:40:41.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 850.0 s, 255.2 tps, lat 388.341 ms stddev 63.152 2023-07-21T19:40:42.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:41 smithi101 ceph-mon[128136]: pgmap v1050: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-07-21T19:40:42.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:41 smithi005 ceph-mon[126460]: pgmap v1050: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-07-21T19:40:42.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:41 smithi156 ceph-mon[128316]: pgmap v1050: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 4.2 MiB/s wr, 441 op/s 2023-07-21T19:40:44.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:43 smithi101 ceph-mon[128136]: pgmap v1051: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 691 op/s 2023-07-21T19:40:44.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:43 smithi005 ceph-mon[126460]: pgmap v1051: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 691 op/s 2023-07-21T19:40:44.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:43 smithi156 ceph-mon[128316]: pgmap v1051: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 6.3 MiB/s wr, 691 op/s 2023-07-21T19:40:46.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:45 smithi101 ceph-mon[128136]: pgmap v1052: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 642 op/s 2023-07-21T19:40:46.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:45 smithi005 ceph-mon[126460]: pgmap v1052: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 642 op/s 2023-07-21T19:40:46.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:45 smithi156 ceph-mon[128316]: pgmap v1052: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 642 op/s 2023-07-21T19:40:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 -- 172.21.15.5:0/893018381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 msgr2=0x7faaa8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.231 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 --2- 172.21.15.5:0/893018381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa8067fe0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7faa980099c0 tx=0x7faa98009cd0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 -- 172.21.15.5:0/893018381 shutdown_connections 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 --2- 172.21.15.5:0/893018381 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 --2- 172.21.15.5:0/893018381 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa8067fe0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 --2- 172.21.15.5:0/893018381 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 0x7faaa8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 -- 172.21.15.5:0/893018381 >> 172.21.15.5:0/893018381 conn(0x7faaa8074760 msgr2=0x7faaa8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.232 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 -- 172.21.15.5:0/893018381 shutdown_connections 2023-07-21T19:40:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.247+0000 7faaad9ff700 1 -- 172.21.15.5:0/893018381 wait complete. 2023-07-21T19:40:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.248+0000 7faaad9ff700 1 Processor -- start 2023-07-21T19:40:46.233 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 -- start start 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 0x7faaa819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa8195930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8195fa0 con 0x7faaa8104e50 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa8196110 con 0x7faaa8067c10 2023-07-21T19:40:46.234 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.249+0000 7faaad9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaa81962d0 con 0x7faaa80685b0 2023-07-21T19:40:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.250+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa8195930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.250+0000 7faaa6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.235 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.250+0000 7faaa6ffd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:55496/0 (socket says 172.21.15.5:55496) 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.250+0000 7faaa6ffd700 1 -- 172.21.15.5:0/4178012590 learned_addr learned my addr 172.21.15.5:0/4178012590 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.250+0000 7faaa67fc700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 0x7faaa819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 -- 172.21.15.5:0/4178012590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 msgr2=0x7faaa819c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 0x7faaa819c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 -- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 msgr2=0x7faaa8195930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa8195930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.236 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 -- 172.21.15.5:0/4178012590 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa980095f0 con 0x7faaa8067c10 2023-07-21T19:40:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa77fe700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa8195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:40:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.251+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faa980098e0 tx=0x7faa98004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.252+0000 7faaac9fd700 1 -- 172.21.15.5:0/4178012590 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9801d070 con 0x7faaa8067c10 2023-07-21T19:40:46.237 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.252+0000 7faaac9fd700 1 -- 172.21.15.5:0/4178012590 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa98004c70 con 0x7faaa8067c10 2023-07-21T19:40:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.252+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaa8196550 con 0x7faaa8067c10 2023-07-21T19:40:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.252+0000 7faaac9fd700 1 -- 172.21.15.5:0/4178012590 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa98003c70 con 0x7faaa8067c10 2023-07-21T19:40:46.238 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.252+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa81a2ac0 con 0x7faaa8067c10 2023-07-21T19:40:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.260+0000 7faac1380b80 1 Processor -- start 2023-07-21T19:40:46.244 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.260+0000 7faac1380b80 1 -- start start 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.260+0000 7faac1380b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a87670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.260+0000 7faac1380b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faac1380b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 0x560058a799d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaac9fd700 1 -- 172.21.15.5:0/4178012590 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7faa980050c0 con 0x7faaa8067c10 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faac1380b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005897ec50 con 0x560058a87bb0 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faac1380b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600589d19e0 con 0x560058a79580 2023-07-21T19:40:46.245 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faac1380b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5600589cae70 con 0x5600588d56b0 2023-07-21T19:40:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaa6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.246 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a87670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaa6ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54248/0 (socket says 172.21.15.5:54248) 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaa6ffd700 1 -- 172.21.15.5:0/2986658405 learned_addr learned my addr 172.21.15.5:0/2986658405 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.261+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 0x560058a799d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 -- 172.21.15.5:0/2986658405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 msgr2=0x560058a799d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 0x560058a799d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 -- 172.21.15.5:0/2986658405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 msgr2=0x560058a79040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.247 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 -- 172.21.15.5:0/2986658405 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5600587f8d80 con 0x5600588d56b0 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 0x560058a799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a87670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faaa819c7d0 tx=0x7faaa0007960 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bddfe85876c3e1a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faa8dffb700 1 -- 172.21.15.5:0/2986658405 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0004bb0 con 0x5600588d56b0 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.262+0000 7faa8dffb700 1 -- 172.21.15.5:0/2986658405 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa000f040 con 0x5600588d56b0 2023-07-21T19:40:46.248 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.263+0000 7faa8dffb700 1 -- 172.21.15.5:0/2986658405 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0015cd0 con 0x5600588d56b0 2023-07-21T19:40:46.249 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.263+0000 7faaac9fd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 0x7faa900a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.263+0000 7faaac9fd700 1 -- 172.21.15.5:0/4178012590 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faa980cda40 con 0x7faaa8067c10 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.263+0000 7faac1380b80 1 -- 172.21.15.5:0/2986658405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 msgr2=0x560058a87670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.263+0000 7faac1380b80 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a87670 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7faaa819c7d0 tx=0x7faaa0007960 comp rx=0 tx=0).stop 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 -- 172.21.15.5:0/2986658405 shutdown_connections 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a79580 0x560058a799d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.250 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a87670 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 --2- 172.21.15.5:0/2986658405 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a87bb0 0x560058a79040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 -- 172.21.15.5:0/2986658405 >> 172.21.15.5:0/2986658405 conn(0x560058a6ebe0 msgr2=0x560058a74140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 0x7faa900a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 -- 172.21.15.5:0/2986658405 shutdown_connections 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 -- 172.21.15.5:0/2986658405 wait complete. 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.264+0000 7faac1380b80 1 -- start start 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.265+0000 7faaa6ffd700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 0x7faa900a3940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa9809aad0 tx=0x7faa98009cd0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.251 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.265+0000 7faac1380b80 1 Processor -- start 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.265+0000 7faac1380b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600588d56b0 0x560058a8de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.265+0000 7faac1380b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faac1380b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faac1380b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005896aa60 con 0x5600588d56b0 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faac1380b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005896a760 con 0x560058a87bb0 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faac1380b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56005884e8c0 con 0x560058a79580 2023-07-21T19:40:46.252 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faac1380b80 10 client.0 _ll_register_callbacks cb 0x560058968700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600588d56b0 0x560058a8de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57274/0 (socket says 172.21.15.5:57274) 2023-07-21T19:40:46.254 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 -- 172.21.15.5:0/2806425872 learned_addr learned my addr 172.21.15.5:0/2806425872 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 -- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 msgr2=0x560058a8b790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8b790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 -- 172.21.15.5:0/2806425872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600588d56b0 msgr2=0x560058a8de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600588d56b0 0x560058a8de90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.266+0000 7faaa67fc700 1 -- 172.21.15.5:0/2806425872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaa0009db0 con 0x560058a79580 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.267+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00bf10 tx=0x7faa9c012730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87fed2e03593ebca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.255 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.267+0000 7faa86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.267+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c004a30 con 0x560058a79580 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.267+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faa9c004b90 con 0x560058a79580 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.267+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560058924490 con 0x560058a79580 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.268+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7faa9c010670 con 0x560058a79580 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.268+0000 7faac1380b80 10 client.30577 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.268+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x5600587e1c00 con 0x560058a79580 2023-07-21T19:40:46.256 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.268+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faa9c026080 con 0x560058a79580 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.268+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faa9c020030 con 0x560058a79580 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 10 client.30577 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 10 client.30577 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 20 client.30577 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a9b480 0x560058a9d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 4 client.30577 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5600589c3e50 con 0x560058a9b480 2023-07-21T19:40:46.257 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.269+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a9b480 0x560058a9d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.270+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a9b480 0x560058a9d890 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dde2a6ee9ef4022d server_cookie=2fb57b08286b5c8c in_seq=0 out_seq=0 2023-07-21T19:40:46.258 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.270+0000 7faa86ffd700 10 client.30577 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.270+0000 7faa86ffd700 1 -- 172.21.15.5:0/2806425872 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x7faaa0029780 con 0x560058a9b480 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faa86ffd700 10 client.30577 handle_command_reply: tid=0 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 client.30577 shutdown 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 msgr2=0x560058a8b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00bf10 tx=0x7faa9c012730 comp rx=0 tx=0).stop 2023-07-21T19:40:46.259 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 shutdown_connections 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8b790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a9b480 0x560058a9d890 crc :-1 s=READY pgs=584 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5600588d56b0 0x560058a8de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 --2- 172.21.15.5:0/2806425872 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x560058a79580 0x560058a8b0a0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 >> 172.21.15.5:0/2806425872 conn(0x560058a6ebe0 msgr2=0x560058a72c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faa86ffd700 0 client.30577 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.271+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 shutdown_connections 2023-07-21T19:40:46.260 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.272+0000 7faac1380b80 1 -- 172.21.15.5:0/2806425872 wait complete. 2023-07-21T19:40:46.261 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.272+0000 7faac1380b80 20 client.30577 trim_cache size 0 max 16384 2023-07-21T19:40:46.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.297+0000 7faac1380b80 1 Processor -- start 2023-07-21T19:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.297+0000 7faac1380b80 1 -- start start 2023-07-21T19:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a79040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058a92a10 con 0x560058a79580 2023-07-21T19:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058a8a660 con 0x560058a87bb0 2023-07-21T19:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.298+0000 7faac1380b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058a8a820 con 0x5600588d56b0 2023-07-21T19:40:46.283 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa67fc700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a924d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:54264/0 (socket says 172.21.15.5:54264) 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa67fc700 1 -- 172.21.15.5:0/1493371185 learned_addr learned my addr 172.21.15.5:0/1493371185 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a79040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 -- 172.21.15.5:0/1493371185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 msgr2=0x560058a8d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 -- 172.21.15.5:0/1493371185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 msgr2=0x560058a924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.299+0000 7faaa77fe700 1 -- 172.21.15.5:0/1493371185 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560058a8aeb0 con 0x5600588d56b0 2023-07-21T19:40:46.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.300+0000 7faaa77fe700 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a79040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560058a9e170 tx=0x7faaa0015650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=374e543e11c0cff3 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.300+0000 7faa8e7fc700 1 -- 172.21.15.5:0/1493371185 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa001d660 con 0x5600588d56b0 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.300+0000 7faa8e7fc700 1 -- 172.21.15.5:0/1493371185 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa0025070 con 0x5600588d56b0 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.300+0000 7faa8e7fc700 1 -- 172.21.15.5:0/1493371185 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa00143a0 con 0x5600588d56b0 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.301+0000 7faac1380b80 1 -- 172.21.15.5:0/1493371185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 msgr2=0x560058a79040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.301+0000 7faac1380b80 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a79040 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x560058a9e170 tx=0x7faaa0015650 comp rx=0 tx=0).stop 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.301+0000 7faac1380b80 1 -- 172.21.15.5:0/1493371185 shutdown_connections 2023-07-21T19:40:46.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a8d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a79040 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 --2- 172.21.15.5:0/1493371185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 -- 172.21.15.5:0/1493371185 >> 172.21.15.5:0/1493371185 conn(0x560058a6ebe0 msgr2=0x560058a79950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.287 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 -- 172.21.15.5:0/1493371185 shutdown_connections 2023-07-21T19:40:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 -- 172.21.15.5:0/1493371185 wait complete. 2023-07-21T19:40:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.302+0000 7faac1380b80 1 -- start start 2023-07-21T19:40:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.303+0000 7faac1380b80 1 Processor -- start 2023-07-21T19:40:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a831a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a837d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058a83f10 con 0x560058a79580 2023-07-21T19:40:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058aef340 con 0x560058a87bb0 2023-07-21T19:40:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x560058a7a430 con 0x5600588d56b0 2023-07-21T19:40:46.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.304+0000 7faac1380b80 10 client.0 _ll_register_callbacks cb 0x560058968700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:57292/0 (socket says 172.21.15.5:57292) 2023-07-21T19:40:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 -- 172.21.15.5:0/2005785850 learned_addr learned my addr 172.21.15.5:0/2005785850 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a831a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a837d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 -- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 msgr2=0x560058a837d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a837d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 -- 172.21.15.5:0/2005785850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 msgr2=0x560058a831a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a831a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.305+0000 7faaa77fe700 1 -- 172.21.15.5:0/2005785850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5600587f8d80 con 0x5600588d56b0 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faaa77fe700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faaa00075d0 tx=0x7faaa001cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9f9a95203d012c27 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:46.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faa86ffd700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:40:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa0030070 con 0x5600588d56b0 2023-07-21T19:40:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faaa000f040 con 0x5600588d56b0 2023-07-21T19:40:46.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faaa0016950 con 0x5600588d56b0 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7faaa0003a00 con 0x5600588d56b0 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faac1380b80 10 client.30583 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.306+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56005881f1f0 con 0x5600588d56b0 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faaa0039020 con 0x5600588d56b0 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7faaa001d6f0 con 0x5600588d56b0 2023-07-21T19:40:46.293 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 10 client.30583 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 10 client.30583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 20 client.30583 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a7b670 0x560058a7da50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 4 client.30583 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.307+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x560058985ff0 con 0x560058a7b670 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.308+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a7b670 0x560058a7da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.308+0000 7faaa67fc700 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a7b670 0x560058a7da50 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57bd0bb499a42045 server_cookie=ff3fdac6756ca8d6 in_seq=0 out_seq=0 2023-07-21T19:40:46.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.308+0000 7faa86ffd700 10 client.30583 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:46.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.340+0000 7faa86ffd700 1 -- 172.21.15.5:0/2005785850 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x560058985ff0 con 0x560058a7b670 2023-07-21T19:40:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.340+0000 7faa86ffd700 10 client.30583 handle_command_reply: tid=0 2023-07-21T19:40:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.340+0000 7faac1380b80 1 client.30583 shutdown 2023-07-21T19:40:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 msgr2=0x560058a84a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7faaa00075d0 tx=0x7faaa001cc50 comp rx=0 tx=0).stop 2023-07-21T19:40:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 shutdown_connections 2023-07-21T19:40:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x560058a87bb0 0x560058a837d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5600588d56b0 0x560058a84a10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x560058a7b670 0x560058a7da50 crc :-1 s=READY pgs=585 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 --2- 172.21.15.5:0/2005785850 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x560058a79580 0x560058a831a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faa86ffd700 0 client.30583 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 >> 172.21.15.5:0/2005785850 conn(0x560058a6ebe0 msgr2=0x560058a852a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 shutdown_connections 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.341+0000 7faac1380b80 1 -- 172.21.15.5:0/2005785850 wait complete. 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.342+0000 7faac1380b80 20 client.30583 trim_cache size 0 max 16384 2023-07-21T19:40:46.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:40:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:40:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:40:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:40:46.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 msgr2=0x7faa900a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 0x7faa900a3940 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7faa9809aad0 tx=0x7faa98009cd0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 msgr2=0x7faaa819bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7faa980098e0 tx=0x7faa98004730 comp rx=0 tx=0).stop 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 shutdown_connections 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faa900a1490 0x7faa900a3940 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faaa8104e50 0x7faaa8195930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faaa8067c10 0x7faaa819bad0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 --2- 172.21.15.5:0/4178012590 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faaa80685b0 0x7faaa819c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 >> 172.21.15.5:0/4178012590 conn(0x7faaa8074760 msgr2=0x7faaa80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 shutdown_connections 2023-07-21T19:40:46.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:46.343+0000 7faaad9ff700 1 -- 172.21.15.5:0/4178012590 wait complete. 2023-07-21T19:40:46.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 855.0 s, 251.8 tps, lat 398.286 ms stddev 70.659 2023-07-21T19:40:48.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:47 smithi101 ceph-mon[128136]: pgmap v1053: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 631 op/s 2023-07-21T19:40:48.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:47 smithi005 ceph-mon[126460]: pgmap v1053: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 631 op/s 2023-07-21T19:40:48.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:47 smithi156 ceph-mon[128316]: pgmap v1053: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.2 MiB/s wr, 631 op/s 2023-07-21T19:40:49.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:48 smithi101 ceph-mon[128136]: pgmap v1054: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.1 MiB/s wr, 951 op/s 2023-07-21T19:40:49.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:48 smithi005 ceph-mon[126460]: pgmap v1054: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.1 MiB/s wr, 951 op/s 2023-07-21T19:40:49.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:48 smithi156 ceph-mon[128316]: pgmap v1054: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 9.1 MiB/s wr, 951 op/s 2023-07-21T19:40:51.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 860.0 s, 223.0 tps, lat 443.643 ms stddev 103.468 2023-07-21T19:40:52.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:51 smithi101 ceph-mon[128136]: pgmap v1055: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 753 op/s 2023-07-21T19:40:52.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:51 smithi005 ceph-mon[126460]: pgmap v1055: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 753 op/s 2023-07-21T19:40:52.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:51 smithi156 ceph-mon[128316]: pgmap v1055: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.2 MiB/s wr, 753 op/s 2023-07-21T19:40:54.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:53 smithi101 ceph-mon[128136]: pgmap v1056: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.9 MiB/s wr, 961 op/s 2023-07-21T19:40:54.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:53 smithi005 ceph-mon[126460]: pgmap v1056: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.9 MiB/s wr, 961 op/s 2023-07-21T19:40:54.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:53 smithi156 ceph-mon[128316]: pgmap v1056: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 MiB/s rd, 8.9 MiB/s wr, 961 op/s 2023-07-21T19:40:56.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:55 smithi101 ceph-mon[128136]: pgmap v1057: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 725 op/s 2023-07-21T19:40:56.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:55 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:40:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:55 smithi005 ceph-mon[126460]: pgmap v1057: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 725 op/s 2023-07-21T19:40:56.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:55 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:40:56.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:55 smithi156 ceph-mon[128316]: pgmap v1057: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 6.7 MiB/s wr, 725 op/s 2023-07-21T19:40:56.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:55 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:40:56.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 865.0 s, 210.4 tps, lat 352.133 ms stddev 153.391 2023-07-21T19:40:58.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:57 smithi101 ceph-mon[128136]: pgmap v1058: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2023-07-21T19:40:58.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:57 smithi005 ceph-mon[126460]: pgmap v1058: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2023-07-21T19:40:58.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:57 smithi156 ceph-mon[128316]: pgmap v1058: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 5.9 MiB/s wr, 660 op/s 2023-07-21T19:40:59.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:58 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:40:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:58 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:40:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:58 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:40:59.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:40:58 smithi101 ceph-mon[128136]: pgmap v1059: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 948 op/s 2023-07-21T19:40:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:58 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:40:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:58 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:40:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:58 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:40:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:40:58 smithi005 ceph-mon[126460]: pgmap v1059: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 948 op/s 2023-07-21T19:40:59.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:58 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:40:59.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:58 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:40:59.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:58 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:40:59.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:40:58 smithi156 ceph-mon[128316]: pgmap v1059: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 8.6 MiB/s wr, 948 op/s 2023-07-21T19:40:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 -- 172.21.15.5:0/1590215008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 msgr2=0x7fc530067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 --2- 172.21.15.5:0/1590215008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 0x7fc530067fe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fc520009990 tx=0x7fc520009ca0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 -- 172.21.15.5:0/1590215008 shutdown_connections 2023-07-21T19:40:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 --2- 172.21.15.5:0/1590215008 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc530104e50 0x7fc53010ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 --2- 172.21.15.5:0/1590215008 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 0x7fc530067fe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 --2- 172.21.15.5:0/1590215008 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc5300685b0 0x7fc530104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 -- 172.21.15.5:0/1590215008 >> 172.21.15.5:0/1590215008 conn(0x7fc530074760 msgr2=0x7fc530074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 -- 172.21.15.5:0/1590215008 shutdown_connections 2023-07-21T19:40:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.575+0000 7fc5353ff700 1 -- 172.21.15.5:0/1590215008 wait complete. 2023-07-21T19:40:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.577+0000 7fc5353ff700 1 Processor -- start 2023-07-21T19:40:59.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.577+0000 7fc5353ff700 1 -- start start 2023-07-21T19:40:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.577+0000 7fc5353ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 0x7fc53010d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc5353ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5300685b0 0x7fc53010d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc5353ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc5353ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc53010ddb0 con 0x7fc530104e50 2023-07-21T19:40:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc5353ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5301a8870 con 0x7fc5300685b0 2023-07-21T19:40:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc5353ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5301a89b0 con 0x7fc530067c10 2023-07-21T19:40:59.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc52f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc52f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36002/0 (socket says 172.21.15.5:36002) 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.578+0000 7fc52f7fe700 1 -- 172.21.15.5:0/3958007715 learned_addr learned my addr 172.21.15.5:0/3958007715 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5300685b0 0x7fc53010d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52f7fe700 1 -- 172.21.15.5:0/3958007715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 msgr2=0x7fc53010d010 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 0x7fc53010d010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52f7fe700 1 -- 172.21.15.5:0/3958007715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5300685b0 msgr2=0x7fc53010d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5300685b0 0x7fc53010d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.579+0000 7fc52f7fe700 1 -- 172.21.15.5:0/3958007715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200095f0 con 0x7fc530104e50 2023-07-21T19:40:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.580+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc52800ef30 tx=0x7fc52800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.564 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.580+0000 7fc527fff700 1 -- 172.21.15.5:0/3958007715 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528018070 con 0x7fc530104e50 2023-07-21T19:40:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.580+0000 7fc527fff700 1 -- 172.21.15.5:0/3958007715 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc52800f040 con 0x7fc530104e50 2023-07-21T19:40:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.580+0000 7fc527fff700 1 -- 172.21.15.5:0/3958007715 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc528003d70 con 0x7fc530104e50 2023-07-21T19:40:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.580+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5301a8bb0 con 0x7fc530104e50 2023-07-21T19:40:59.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.581+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5301a90f0 con 0x7fc530104e50 2023-07-21T19:40:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.587+0000 7fc527fff700 1 -- 172.21.15.5:0/3958007715 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc528004ad0 con 0x7fc530104e50 2023-07-21T19:40:59.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.587+0000 7fc548c31b80 1 Processor -- start 2023-07-21T19:40:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 -- start start 2023-07-21T19:40:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc9d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fc49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740ec9c50 con 0x55c740fc4580 2023-07-21T19:40:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740f1c9e0 con 0x55c740fca2d0 2023-07-21T19:40:59.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.588+0000 7fc548c31b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740f15e70 con 0x55c740e206b0 2023-07-21T19:40:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33630/0 (socket says 172.21.15.5:33630) 2023-07-21T19:40:59.573 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 -- 172.21.15.5:0/1748874812 learned_addr learned my addr 172.21.15.5:0/1748874812 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc9d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fc49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 -- 172.21.15.5:0/1748874812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 msgr2=0x55c740fc9d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc9d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 -- 172.21.15.5:0/1748874812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 msgr2=0x55c740fc49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fc49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52effd700 1 -- 172.21.15.5:0/1748874812 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c740d43d80 con 0x55c740fca2d0 2023-07-21T19:40:59.574 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.589+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fc49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc52effd700 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc520005950 tx=0x7fc520005c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dcdf6b5dac899639 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc527fff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 0x7fc51c0a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc5257fa700 1 -- 172.21.15.5:0/1748874812 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52001d070 con 0x55c740fca2d0 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc5257fa700 1 -- 172.21.15.5:0/1748874812 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc520003fd0 con 0x55c740fca2d0 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc5257fa700 1 -- 172.21.15.5:0/1748874812 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52000bba0 con 0x55c740fca2d0 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc527fff700 1 -- 172.21.15.5:0/3958007715 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc5280cc9a0 con 0x7fc530104e50 2023-07-21T19:40:59.575 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.590+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 0x7fc51c0a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 -- 172.21.15.5:0/1748874812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 msgr2=0x55c740fc4040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.576 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc520005950 tx=0x7fc520005c20 comp rx=0 tx=0).stop 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 -- 172.21.15.5:0/1748874812 shutdown_connections 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fc49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc9d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1748874812 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fca2d0 0x55c740fc4040 secure :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fc520005950 tx=0x7fc520005c20 comp rx=0 tx=0).stop 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc548c31b80 1 -- 172.21.15.5:0/1748874812 >> 172.21.15.5:0/1748874812 conn(0x55c740fb9be0 msgr2=0x55c740fbef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.591+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 0x7fc51c0a38b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc518006ea0 tx=0x7fc518006e30 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.592+0000 7fc548c31b80 1 -- 172.21.15.5:0/1748874812 shutdown_connections 2023-07-21T19:40:59.577 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.592+0000 7fc548c31b80 1 -- 172.21.15.5:0/1748874812 wait complete. 2023-07-21T19:40:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.592+0000 7fc548c31b80 1 -- start start 2023-07-21T19:40:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.593+0000 7fc548c31b80 1 Processor -- start 2023-07-21T19:40:59.578 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.593+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fcdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c740fd9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740eb5a60 con 0x55c740fc4580 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740eb5760 con 0x55c740e206b0 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740d998c0 con 0x55c740fce490 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.594+0000 7fc548c31b80 10 client.0 _ll_register_callbacks cb 0x55c740eb3700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:59.579 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33642/0 (socket says 172.21.15.5:33642) 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 -- 172.21.15.5:0/523953677 learned_addr learned my addr 172.21.15.5:0/523953677 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c740fd9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 -- 172.21.15.5:0/523953677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 msgr2=0x55c740fd9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c740fd9ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 -- 172.21.15.5:0/523953677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 msgr2=0x55c740fcdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fcdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.595+0000 7fc52effd700 1 -- 172.21.15.5:0/523953677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5300fed10 con 0x55c740e206b0 2023-07-21T19:40:59.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc52effd700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc520000c00 tx=0x7fc520003e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8ace6ce1c19904a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc5167fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52001d070 con 0x55c740e206b0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc52000b920 con 0x55c740e206b0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c740e6f490 con 0x55c740e206b0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc52000bae0 con 0x55c740e206b0 2023-07-21T19:40:59.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc548c31b80 10 client.30647 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.596+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55c740d2cc00 con 0x55c740e206b0 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55c740e6f490 con 0x55c740e206b0 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc520022bb0 con 0x55c740e206b0 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 10 client.30647 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 10 client.30647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 20 client.30647 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c740feeb80 0x55c740ff0f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 4 client.30647 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:40:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c740f0ee50 con 0x55c740feeb80 2023-07-21T19:40:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.597+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c740feeb80 0x55c740ff0f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.598+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c740feeb80 0x55c740ff0f60 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bdea1e9eb35b06f5 server_cookie=2cd3b0fec8c41658 in_seq=0 out_seq=0 2023-07-21T19:40:59.584 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.598+0000 7fc5167fc700 10 client.30647 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.599+0000 7fc5167fc700 1 -- 172.21.15.5:0/523953677 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c740f0ee50 con 0x55c740feeb80 2023-07-21T19:40:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.599+0000 7fc5167fc700 10 client.30647 handle_command_reply: tid=0 2023-07-21T19:40:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.599+0000 7fc548c31b80 1 client.30647 shutdown 2023-07-21T19:40:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.599+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 msgr2=0x55c740fcda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.599+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fc520000c00 tx=0x7fc520003e60 comp rx=0 tx=0).stop 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 shutdown_connections 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c740fd9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c740fcda10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c740feeb80 0x55c740ff0f60 crc :-1 s=READY pgs=586 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 --2- 172.21.15.5:0/523953677 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c740fcdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 >> 172.21.15.5:0/523953677 conn(0x55c740fb9be0 msgr2=0x55c740fbe4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc5167fc700 0 client.30647 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:59.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 shutdown_connections 2023-07-21T19:40:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 1 -- 172.21.15.5:0/523953677 wait complete. 2023-07-21T19:40:59.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.600+0000 7fc548c31b80 20 client.30647 trim_cache size 0 max 16384 2023-07-21T19:40:59.607 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.624+0000 7fc548c31b80 1 Processor -- start 2023-07-21T19:40:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.624+0000 7fc548c31b80 1 -- start start 2023-07-21T19:40:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc4040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.608 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fce490 0x55c740fd4c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740fca6f0 con 0x55c740fce490 2023-07-21T19:40:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740fc0020 con 0x55c740fc4580 2023-07-21T19:40:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.625+0000 7fc548c31b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c740fcf040 con 0x55c740e206b0 2023-07-21T19:40:59.609 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33654/0 (socket says 172.21.15.5:33654) 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 -- 172.21.15.5:0/2787148956 learned_addr learned my addr 172.21.15.5:0/2787148956 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52effd700 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fce490 0x55c740fd4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 -- 172.21.15.5:0/2787148956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 msgr2=0x55c740fc4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.610 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 -- 172.21.15.5:0/2787148956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fce490 msgr2=0x55c740fd4c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fce490 0x55c740fd4c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.626+0000 7fc52e7fc700 1 -- 172.21.15.5:0/2787148956 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c740d6a1f0 con 0x55c740fc4580 2023-07-21T19:40:59.611 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.627+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc530106300 tx=0x7fc518004860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=541a40fbbed919b6 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.627+0000 7fc525ffb700 1 -- 172.21.15.5:0/2787148956 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51800fe70 con 0x55c740fc4580 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.627+0000 7fc525ffb700 1 -- 172.21.15.5:0/2787148956 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc5180244d0 con 0x55c740fc4580 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.627+0000 7fc525ffb700 1 -- 172.21.15.5:0/2787148956 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51802dc70 con 0x55c740fc4580 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 -- 172.21.15.5:0/2787148956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 msgr2=0x55c740fc92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fc530106300 tx=0x7fc518004860 comp rx=0 tx=0).stop 2023-07-21T19:40:59.612 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 -- 172.21.15.5:0/2787148956 shutdown_connections 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fce490 0x55c740fd4c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740e206b0 0x55c740fc4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 --2- 172.21.15.5:0/2787148956 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740fc4580 0x55c740fc92d0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.628+0000 7fc548c31b80 1 -- 172.21.15.5:0/2787148956 >> 172.21.15.5:0/2787148956 conn(0x55c740fb9be0 msgr2=0x55c740fc8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.629+0000 7fc548c31b80 1 -- 172.21.15.5:0/2787148956 shutdown_connections 2023-07-21T19:40:59.613 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.629+0000 7fc548c31b80 1 -- 172.21.15.5:0/2787148956 wait complete. 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.629+0000 7fc548c31b80 1 -- start start 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.629+0000 7fc548c31b80 1 Processor -- start 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c7410370c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c741041a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741041f50 con 0x55c740fc4580 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c741042090 con 0x55c740e206b0 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c7410421d0 con 0x55c740fce490 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc548c31b80 10 client.0 _ll_register_callbacks cb 0x7fc520003a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc52effd700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33662/0 (socket says 172.21.15.5:33662) 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc52effd700 1 -- 172.21.15.5:0/1796727897 learned_addr learned my addr 172.21.15.5:0/1796727897 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc52f7fe700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c741041a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.616 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.630+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c7410370c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 -- 172.21.15.5:0/1796727897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 msgr2=0x55c741041a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c741041a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 -- 172.21.15.5:0/1796727897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 msgr2=0x55c7410370c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c7410370c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 -- 172.21.15.5:0/1796727897 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc51800e040 con 0x55c740e206b0 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c7410370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:40:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc52effd700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc5200216f0 tx=0x7fc520021720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=70076fc7305adfae server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc5167fc700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc52001d040 con 0x55c740e206b0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc520003fd0 con 0x55c740e206b0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.631+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fc520000e40 con 0x55c740e206b0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc5200041f0 con 0x55c740e206b0 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 10 client.30653 fetch_fsmap learned FSMap version 72 2023-07-21T19:40:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x55c740d42250 con 0x55c740e206b0 2023-07-21T19:40:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc520003fd0 con 0x55c740e206b0 2023-07-21T19:40:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7fc520032910 con 0x55c740e206b0 2023-07-21T19:40:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 10 client.30653 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 10 client.30653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 20 client.30653 populate_metadata read hostname 'smithi005' 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c741043520 0x55c741045930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 4 client.30653 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.632+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c740ed0ff0 con 0x55c741043520 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.633+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c741043520 0x55c741045930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:40:59.620 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.633+0000 7fc52e7fc700 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c741043520 0x55c741045930 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a30a1f22cede5be6 server_cookie=b2e04a7d7af68573 in_seq=0 out_seq=0 2023-07-21T19:40:59.621 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.633+0000 7fc5167fc700 10 client.30653 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.655+0000 7fc5167fc700 1 -- 172.21.15.5:0/1796727897 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c740ed0ff0 con 0x55c741043520 2023-07-21T19:40:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.655+0000 7fc5167fc700 10 client.30653 handle_command_reply: tid=0 2023-07-21T19:40:59.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.655+0000 7fc548c31b80 1 client.30653 shutdown 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 msgr2=0x55c741036b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fc5200216f0 tx=0x7fc520021720 comp rx=0 tx=0).stop 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 shutdown_connections 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55c740fce490 0x55c741041a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55c740e206b0 0x55c741036b80 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55c741043520 0x55c741045930 crc :-1 s=READY pgs=587 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 --2- 172.21.15.5:0/1796727897 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55c740fc4580 0x55c7410370c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 >> 172.21.15.5:0/1796727897 conn(0x55c740fb9be0 msgr2=0x55c740fc8bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc5167fc700 0 client.30653 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:40:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 shutdown_connections 2023-07-21T19:40:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.656+0000 7fc548c31b80 1 -- 172.21.15.5:0/1796727897 wait complete. 2023-07-21T19:40:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.657+0000 7fc548c31b80 20 client.30653 trim_cache size 0 max 16384 2023-07-21T19:40:59.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:40:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:40:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:40:59.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:40:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 msgr2=0x7fc51c0a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 0x7fc51c0a38b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc518006ea0 tx=0x7fc518006e30 comp rx=0 tx=0).stop 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 msgr2=0x7fc5301a8330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fc52800ef30 tx=0x7fc52800c450 comp rx=0 tx=0).stop 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 shutdown_connections 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc51c0a1400 0x7fc51c0a38b0 secure :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc518006ea0 tx=0x7fc518006e30 comp rx=0 tx=0).stop 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc530104e50 0x7fc5301a8330 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc530067c10 0x7fc53010d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 --2- 172.21.15.5:0/3958007715 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc5300685b0 0x7fc53010d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:40:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.662+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 >> 172.21.15.5:0/3958007715 conn(0x7fc530074760 msgr2=0x7fc5300fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:40:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.663+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 shutdown_connections 2023-07-21T19:40:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:40:59.663+0000 7fc5353ff700 1 -- 172.21.15.5:0/3958007715 wait complete. 2023-07-21T19:41:01.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 870.0 s, 154.6 tps, lat 71.082 ms stddev 45.267 2023-07-21T19:41:02.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:01 smithi101 ceph-mon[128136]: pgmap v1060: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2023-07-21T19:41:02.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:01 smithi005 ceph-mon[126460]: pgmap v1060: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2023-07-21T19:41:02.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:01 smithi156 ceph-mon[128316]: pgmap v1060: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 5.8 MiB/s wr, 656 op/s 2023-07-21T19:41:04.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:03 smithi101 ceph-mon[128136]: pgmap v1061: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 880 op/s 2023-07-21T19:41:04.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:03 smithi005 ceph-mon[126460]: pgmap v1061: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 880 op/s 2023-07-21T19:41:04.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:03 smithi156 ceph-mon[128316]: pgmap v1061: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 7.6 MiB/s wr, 880 op/s 2023-07-21T19:41:05.161 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:41:05.517 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 -- 172.21.15.5:0/282605825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006c9a0 msgr2=0x7f394006ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 --2- 172.21.15.5:0/282605825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006c9a0 0x7f394006ce10 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3930009990 tx=0x7f3930009ca0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 -- 172.21.15.5:0/282605825 shutdown_connections 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 --2- 172.21.15.5:0/282605825 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394006d3a0 0x7f394011b470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 --2- 172.21.15.5:0/282605825 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394010e280 0x7f394010e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 --2- 172.21.15.5:0/282605825 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006c9a0 0x7f394006ce10 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 -- 172.21.15.5:0/282605825 >> 172.21.15.5:0/282605825 conn(0x7f394006bf70 msgr2=0x7f3940070ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.518 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 -- 172.21.15.5:0/282605825 shutdown_connections 2023-07-21T19:41:05.519 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.534+0000 7f39475ff700 1 -- 172.21.15.5:0/282605825 wait complete. 2023-07-21T19:41:05.523 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.536+0000 7f39475ff700 1 Processor -- start 2023-07-21T19:41:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.536+0000 7f39475ff700 1 -- start start 2023-07-21T19:41:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.538+0000 7f39475ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394006c9a0 0x7f3940113d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.538+0000 7f39475ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006d3a0 0x7f3940114260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.539+0000 7f39475ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.524 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.539+0000 7f39475ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401a84b0 con 0x7f394006c9a0 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.539+0000 7f39475ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401a8620 con 0x7f394006d3a0 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.539+0000 7f39475ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39401a87e0 con 0x7f394010e280 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.540+0000 7f3945b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.540+0000 7f3945b9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39326/0 (socket says 172.21.15.5:39326) 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.540+0000 7f3945b9c700 1 -- 172.21.15.5:0/3645810601 learned_addr learned my addr 172.21.15.5:0/3645810601 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.540+0000 7f394539b700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394006c9a0 0x7f3940113d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.525 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.540+0000 7f3944b9a700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006d3a0 0x7f3940114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.541+0000 7f3945b9c700 1 -- 172.21.15.5:0/3645810601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006d3a0 msgr2=0x7f3940114260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.541+0000 7f3945b9c700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006d3a0 0x7f3940114260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.526 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.541+0000 7f3945b9c700 1 -- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394006c9a0 msgr2=0x7f3940113d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.541+0000 7f3945b9c700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394006c9a0 0x7f3940113d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.541+0000 7f3945b9c700 1 -- 172.21.15.5:0/3645810601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39300095f0 con 0x7f394010e280 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f3945b9c700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f393800b970 tx=0x7f393800bc80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f39367fc700 1 -- 172.21.15.5:0/3645810601 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393800c930 con 0x7f394010e280 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f39367fc700 1 -- 172.21.15.5:0/3645810601 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3938014440 con 0x7f394010e280 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f39367fc700 1 -- 172.21.15.5:0/3645810601 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380075d0 con 0x7f394010e280 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39401a8ac0 con 0x7f394010e280 2023-07-21T19:41:05.527 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.542+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39401a8f80 con 0x7f394010e280 2023-07-21T19:41:05.530 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.547+0000 7f39367fc700 1 -- 172.21.15.5:0/3645810601 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f393800ca90 con 0x7f394010e280 2023-07-21T19:41:05.532 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.549+0000 7f39367fc700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 0x7f392c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.550+0000 7f39367fc700 1 -- 172.21.15.5:0/3645810601 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3938006e40 con 0x7f394010e280 2023-07-21T19:41:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.550+0000 7f395aff4b80 1 Processor -- start 2023-07-21T19:41:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.550+0000 7f394539b700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 0x7f392c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.533 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.550+0000 7f395aff4b80 1 -- start start 2023-07-21T19:41:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062ccc2590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.534 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062ccc4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cbbdcb0 con 0x56062ccb7be0 2023-07-21T19:41:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cc10a40 con 0x56062ccc2ad0 2023-07-21T19:41:05.535 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f395aff4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cc09ed0 con 0x56062ccb72d0 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f3944b9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062ccc2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39336/0 (socket says 172.21.15.5:39336) 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.551+0000 7f394539b700 1 -- 172.21.15.5:0/1779452254 learned_addr learned my addr 172.21.15.5:0/1779452254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 0x7f392c0a3980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f393c005950 tx=0x7f393c00b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 -- 172.21.15.5:0/1779452254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 msgr2=0x56062ccc4eb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-07-21T19:41:05.536 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062ccc4eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 -- 172.21.15.5:0/1779452254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 msgr2=0x56062ccc2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062ccc2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 -- 172.21.15.5:0/1779452254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062ca37de0 con 0x56062ccb72d0 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.552+0000 7f394539b700 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f393c010f70 tx=0x7f393c01b000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=888934e5adbd126f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.553+0000 7f392b7fe700 1 -- 172.21.15.5:0/1779452254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c011db0 con 0x56062ccb72d0 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.553+0000 7f392b7fe700 1 -- 172.21.15.5:0/1779452254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f393c026900 con 0x56062ccb72d0 2023-07-21T19:41:05.537 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.553+0000 7f392b7fe700 1 -- 172.21.15.5:0/1779452254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c01d6d0 con 0x56062ccb72d0 2023-07-21T19:41:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 -- 172.21.15.5:0/1779452254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 msgr2=0x56062ccb76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f393c010f70 tx=0x7f393c01b000 comp rx=0 tx=0).stop 2023-07-21T19:41:05.538 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 -- 172.21.15.5:0/1779452254 shutdown_connections 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062ccc4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccb76a0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1779452254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062ccc2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 -- 172.21.15.5:0/1779452254 >> 172.21.15.5:0/1779452254 conn(0x56062ccadda0 msgr2=0x56062ccb36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 -- 172.21.15.5:0/1779452254 shutdown_connections 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.554+0000 7f395aff4b80 1 -- 172.21.15.5:0/1779452254 wait complete. 2023-07-21T19:41:05.539 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.555+0000 7f395aff4b80 1 -- start start 2023-07-21T19:41:05.540 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.555+0000 7f395aff4b80 1 Processor -- start 2023-07-21T19:41:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbfdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccc0300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.541 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cba9ac0 con 0x56062ccc2ad0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cba97c0 con 0x56062ccb7be0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062ca8d920 con 0x56062ccb72d0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.556+0000 7f395aff4b80 10 client.0 _ll_register_callbacks cb 0x56062cba7760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f3944b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbfdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39340/0 (socket says 172.21.15.5:39340) 2023-07-21T19:41:05.542 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 -- 172.21.15.5:0/2439269972 learned_addr learned my addr 172.21.15.5:0/2439269972 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f3945b9c700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccc0300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 -- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 msgr2=0x56062ccbfdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbfdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 -- 172.21.15.5:0/2439269972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 msgr2=0x56062ccc0300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccc0300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f394539b700 1 -- 172.21.15.5:0/2439269972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f393c010c50 con 0x56062ccb72d0 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.557+0000 7f3945b9c700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccc0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:41:05.543 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f394539b700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f393c011a00 tx=0x7f393c013f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=421c2fda08a3ad6d server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f3928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:41:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393c02a040 con 0x56062ccb72d0 2023-07-21T19:41:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56062cb634f0 con 0x56062ccb72d0 2023-07-21T19:41:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f393c017070 con 0x56062ccb72d0 2023-07-21T19:41:05.544 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f393c011db0 con 0x56062ccb72d0 2023-07-21T19:41:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f395aff4b80 10 client.30604 fetch_fsmap learned FSMap version 72 2023-07-21T19:41:05.545 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.558+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56062ca20c60 con 0x56062ccb72d0 2023-07-21T19:41:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f393c03c020 con 0x56062ccb72d0 2023-07-21T19:41:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f393c004870 con 0x56062ccb72d0 2023-07-21T19:41:05.546 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 10 client.30604 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 10 client.30604 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 20 client.30604 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062ccd23c0 0x56062ccd47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 4 client.30604 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56062cc02eb0 con 0x56062ccd23c0 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.559+0000 7f3944b9a700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062ccd23c0 0x56062ccd47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.560+0000 7f3944b9a700 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062ccd23c0 0x56062ccd47d0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2e9b08eace8a8fd server_cookie=ebe98fd7b6229eea in_seq=0 out_seq=0 2023-07-21T19:41:05.547 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.560+0000 7f3928ff9700 10 client.30604 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f3928ff9700 1 -- 172.21.15.5:0/2439269972 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56062cc02eb0 con 0x56062ccd23c0 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f3928ff9700 10 client.30604 handle_command_reply: tid=0 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 client.30604 shutdown 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 msgr2=0x56062ccbf880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f393c011a00 tx=0x7f393c013f40 comp rx=0 tx=0).stop 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 shutdown_connections 2023-07-21T19:41:05.548 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062ccd23c0 0x56062ccd47d0 crc :-1 s=READY pgs=588 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccc0300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccbf880 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 --2- 172.21.15.5:0/2439269972 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbfdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f3928ff9700 0 client.30604 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 >> 172.21.15.5:0/2439269972 conn(0x56062ccadda0 msgr2=0x56062ccb3290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 shutdown_connections 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.561+0000 7f395aff4b80 1 -- 172.21.15.5:0/2439269972 wait complete. 2023-07-21T19:41:05.549 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.562+0000 7f395aff4b80 20 client.30604 trim_cache size 0 max 16384 2023-07-21T19:41:05.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.585+0000 7f395aff4b80 1 Processor -- start 2023-07-21T19:41:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.585+0000 7f395aff4b80 1 -- start start 2023-07-21T19:41:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccc94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccbd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cc02eb0 con 0x56062ccc2ad0 2023-07-21T19:41:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062ccb78f0 con 0x56062ccb7be0 2023-07-21T19:41:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f395aff4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062ccafec0 con 0x56062ccb72d0 2023-07-21T19:41:05.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccc94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f3944b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccc94f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:39352/0 (socket says 172.21.15.5:39352) 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f3944b9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33684/0 (socket says 172.21.15.5:33684) 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.586+0000 7f394539b700 1 -- 172.21.15.5:0/1245750424 learned_addr learned my addr 172.21.15.5:0/1245750424 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3945b9c700 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccbd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 -- 172.21.15.5:0/1245750424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 msgr2=0x56062ccc94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccc94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 -- 172.21.15.5:0/1245750424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 msgr2=0x56062ccbd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccbd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 -- 172.21.15.5:0/1245750424 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56062ccb20e0 con 0x56062ccb7be0 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3945b9c700 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccbd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f3944b9a700 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3940115330 tx=0x7f3930009510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54098b7aad235db5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f392bfff700 1 -- 172.21.15.5:0/1245750424 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3930028d30 con 0x56062ccb7be0 2023-07-21T19:41:05.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f392bfff700 1 -- 172.21.15.5:0/1245750424 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f393001e040 con 0x56062ccb7be0 2023-07-21T19:41:05.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.587+0000 7f392bfff700 1 -- 172.21.15.5:0/1245750424 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f393000be60 con 0x56062ccb7be0 2023-07-21T19:41:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 -- 172.21.15.5:0/1245750424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 msgr2=0x56062ccbcaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f3940115330 tx=0x7f3930009510 comp rx=0 tx=0).stop 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 -- 172.21.15.5:0/1245750424 shutdown_connections 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccc2ad0 0x56062ccbd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062ccc94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1245750424 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccb7be0 0x56062ccbcaf0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 -- 172.21.15.5:0/1245750424 >> 172.21.15.5:0/1245750424 conn(0x56062ccadda0 msgr2=0x56062ccbab70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 -- 172.21.15.5:0/1245750424 shutdown_connections 2023-07-21T19:41:05.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.588+0000 7f395aff4b80 1 -- 172.21.15.5:0/1245750424 wait complete. 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.589+0000 7f395aff4b80 1 -- start start 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.589+0000 7f395aff4b80 1 Processor -- start 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062cd1a400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062cd1a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cd1efe0 con 0x56062ccb7be0 2023-07-21T19:41:05.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cd1f150 con 0x56062ccc2ad0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56062cd1f310 con 0x56062ccb72d0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f395aff4b80 10 client.0 _ll_register_callbacks cb 0x7f3930009d10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f3945b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f394539b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062cd1a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f3945b9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33692/0 (socket says 172.21.15.5:33692) 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f3945b9c700 1 -- 172.21.15.5:0/1790967000 learned_addr learned my addr 172.21.15.5:0/1790967000 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f3944b9a700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062cd1a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.577 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.590+0000 7f3945b9c700 1 -- 172.21.15.5:0/1790967000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 msgr2=0x56062cd1a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3945b9c700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062cd1a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3945b9c700 1 -- 172.21.15.5:0/1790967000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 msgr2=0x56062cd1a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3945b9c700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062cd1a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3945b9c700 1 -- 172.21.15.5:0/1790967000 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f394011bfa0 con 0x56062ccc2ad0 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f394539b700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062cd1a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3945b9c700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f39380992c0 tx=0x7f3938098fa0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d365a93c22e09a server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:05.578 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39380995c0 con 0x56062ccc2ad0 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(143..143 src has 1..143) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f393809d5f0 con 0x56062ccc2ad0 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f393c026900 con 0x56062ccc2ad0 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=72) v2 ==== 24+0+0 (secure 0 0 0) 0x7f39380a1a10 con 0x56062ccc2ad0 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f395aff4b80 10 client.30668 fetch_fsmap learned FSMap version 72 2023-07-21T19:41:05.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.591+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=72,monmap=4+}) v3 -- 0x56062ca5e250 con 0x56062ccc2ad0 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f393c026900 con 0x56062ccc2ad0 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 72) v1 ==== 7462+0+0 (secure 0 0 0) 0x7f39380a1580 con 0x56062ccc2ad0 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 10 client.30668 fetch_fsmap finished waiting for FSMap version 72 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 10 client.30668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 20 client.30668 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062cd27c90 0x56062cd2a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:05.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 4 client.30668 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.592+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56062cbc5050 con 0x56062cd27c90 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.593+0000 7f394539b700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062cd27c90 0x56062cd2a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.593+0000 7f3928ff9700 10 client.30668 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.593+0000 7f394539b700 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062cd27c90 0x56062cd2a070 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54a30dfee74ce620 server_cookie=bd6ebf436de5a383 in_seq=0 out_seq=0 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.593+0000 7f3928ff9700 1 -- 172.21.15.5:0/1790967000 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56062cbc5050 con 0x56062cd27c90 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.593+0000 7f3928ff9700 10 client.30668 handle_command_reply: tid=0 2023-07-21T19:41:05.582 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 client.30668 shutdown 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 msgr2=0x56062cd1ea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f39380992c0 tx=0x7f3938098fa0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 shutdown_connections 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56062ccc2ad0 0x56062cd1ea30 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56062ccb72d0 0x56062cd1a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56062cd27c90 0x56062cd2a070 crc :-1 s=READY pgs=589 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 --2- 172.21.15.5:0/1790967000 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56062ccb7be0 0x56062cd1a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.583 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f3928ff9700 0 client.30668 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 >> 172.21.15.5:0/1790967000 conn(0x56062ccadda0 msgr2=0x56062ccb93e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 shutdown_connections 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 1 -- 172.21.15.5:0/1790967000 wait complete. 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.594+0000 7f395aff4b80 20 client.30668 trim_cache size 0 max 16384 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:41:05.584 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:41:05.585 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:41:05.586 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:41:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 msgr2=0x7f392c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 0x7f392c0a3980 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f393c005950 tx=0x7f393c00b410 comp rx=0 tx=0).stop 2023-07-21T19:41:05.586 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 msgr2=0x7f39401a7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f393800b970 tx=0x7f393800bc80 comp rx=0 tx=0).stop 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 shutdown_connections 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f394010e280 0x7f39401a7eb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f392c0a14d0 0x7f392c0a3980 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f394006c9a0 0x7f3940113d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 --2- 172.21.15.5:0/3645810601 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f394006d3a0 0x7f3940114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 >> 172.21.15.5:0/3645810601 conn(0x7f394006bf70 msgr2=0x7f394006f180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:05.587 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 shutdown_connections 2023-07-21T19:41:05.588 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:05.595+0000 7f39475ff700 1 -- 172.21.15.5:0/3645810601 wait complete. 2023-07-21T19:41:05.601 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:41:05.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:05 smithi005 ceph-mon[126460]: pgmap v1062: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 663 op/s 2023-07-21T19:41:06.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:05 smithi101 ceph-mon[128136]: pgmap v1062: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 663 op/s 2023-07-21T19:41:06.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:05 smithi156 ceph-mon[128316]: pgmap v1062: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.7 MiB/s wr, 663 op/s 2023-07-21T19:41:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 875.0 s, 151.7 tps, lat 70.310 ms stddev 45.319 2023-07-21T19:41:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:07 smithi101 ceph-mon[128136]: pgmap v1063: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 618 op/s 2023-07-21T19:41:08.066 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:07 smithi101 ceph-mon[128136]: osdmap e144: 12 total, 12 up, 12 in 2023-07-21T19:41:08.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:07 smithi005 ceph-mon[126460]: pgmap v1063: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 618 op/s 2023-07-21T19:41:08.143 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:07 smithi005 ceph-mon[126460]: osdmap e144: 12 total, 12 up, 12 in 2023-07-21T19:41:08.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:07 smithi156 ceph-mon[128316]: pgmap v1063: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 5.4 MiB/s wr, 618 op/s 2023-07-21T19:41:08.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:07 smithi156 ceph-mon[128316]: osdmap e144: 12 total, 12 up, 12 in 2023-07-21T19:41:09.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:09 smithi101 ceph-mon[128136]: pgmap v1065: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T19:41:09.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:09 smithi005 ceph-mon[126460]: pgmap v1065: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T19:41:09.408 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:09 smithi156 ceph-mon[128316]: pgmap v1065: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 5.1 MiB/s wr, 586 op/s 2023-07-21T19:41:11.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 880.0 s, 54.2 tps, lat 209.223 ms stddev 249.083 2023-07-21T19:41:12.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:11 smithi101 ceph-mon[128136]: pgmap v1066: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 565 op/s 2023-07-21T19:41:12.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:11 smithi005 ceph-mon[126460]: pgmap v1066: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 565 op/s 2023-07-21T19:41:12.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:11 smithi156 ceph-mon[128316]: pgmap v1066: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 4.9 MiB/s wr, 565 op/s 2023-07-21T19:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.905+0000 7f4edf5ff700 1 -- 172.21.15.5:0/3922381376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed8067c10 msgr2=0x7f4ed8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.905+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/3922381376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed8067c10 0x7f4ed8067fe0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40099c0 tx=0x7f4ed4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.905+0000 7f4edf5ff700 1 -- 172.21.15.5:0/3922381376 shutdown_connections 2023-07-21T19:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/3922381376 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8104e50 0x7f4ed810ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/3922381376 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed8067c10 0x7f4ed8067fe0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/3922381376 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed80685b0 0x7f4ed8104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 -- 172.21.15.5:0/3922381376 >> 172.21.15.5:0/3922381376 conn(0x7f4ed8074760 msgr2=0x7f4ed8074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 -- 172.21.15.5:0/3922381376 shutdown_connections 2023-07-21T19:41:12.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.906+0000 7f4edf5ff700 1 -- 172.21.15.5:0/3922381376 wait complete. 2023-07-21T19:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.907+0000 7f4edf5ff700 1 Processor -- start 2023-07-21T19:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.907+0000 7f4edf5ff700 1 -- start start 2023-07-21T19:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed80685b0 0x7f4ed8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed8104e50 0x7f4ed819b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819bdd0 con 0x7f4ed8104e50 2023-07-21T19:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819bf40 con 0x7f4ed80685b0 2023-07-21T19:41:12.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.908+0000 7f4edf5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed819c100 con 0x7f4ed8067c10 2023-07-21T19:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58804/0 (socket says 172.21.15.5:58804) 2023-07-21T19:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 -- 172.21.15.5:0/805856432 learned_addr learned my addr 172.21.15.5:0/805856432 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed80685b0 0x7f4ed8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.893 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 -- 172.21.15.5:0/805856432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed80685b0 msgr2=0x7f4ed8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed80685b0 0x7f4ed8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 -- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed8104e50 msgr2=0x7f4ed819b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.909+0000 7f4edd39b700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed8104e50 0x7f4ed819b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.910+0000 7f4edd39b700 1 -- 172.21.15.5:0/805856432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40095f0 con 0x7f4ed8067c10 2023-07-21T19:41:12.894 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.910+0000 7f4edd39b700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005910 tx=0x7f4ed400f740 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.910+0000 7f4ece7fc700 1 -- 172.21.15.5:0/805856432 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed401d070 con 0x7f4ed8067c10 2023-07-21T19:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.911+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed819c380 con 0x7f4ed8067c10 2023-07-21T19:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.911+0000 7f4ece7fc700 1 -- 172.21.15.5:0/805856432 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ed40054e0 con 0x7f4ed8067c10 2023-07-21T19:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.911+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed81a4d40 con 0x7f4ed8067c10 2023-07-21T19:41:12.895 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.911+0000 7f4ece7fc700 1 -- 172.21.15.5:0/805856432 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4004ad0 con 0x7f4ed8067c10 2023-07-21T19:41:12.900 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.917+0000 7f4ece7fc700 1 -- 172.21.15.5:0/805856432 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f4ed4004c30 con 0x7f4ed8067c10 2023-07-21T19:41:12.901 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.918+0000 7f4ef2f49b80 1 Processor -- start 2023-07-21T19:41:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.919+0000 7f4ef2f49b80 1 -- start start 2023-07-21T19:41:12.902 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb589d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb584580 0x55d0bb5849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb489c50 con 0x55d0bb3e06b0 2023-07-21T19:41:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb4dc9e0 con 0x55d0bb584580 2023-07-21T19:41:12.903 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.920+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb4d5e70 con 0x55d0bb58a2d0 2023-07-21T19:41:12.904 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.921+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.921+0000 7f4ece7fc700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 0x7f4ec40a3870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.921+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:58808/0 (socket says 172.21.15.5:58808) 2023-07-21T19:41:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.921+0000 7f4edd39b700 1 -- 172.21.15.5:0/1637352307 learned_addr learned my addr 172.21.15.5:0/1637352307 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.921+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb584580 0x55d0bb5849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.905 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.922+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb589d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.922+0000 7f4ece7fc700 1 -- 172.21.15.5:0/805856432 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f4ed40cda80 con 0x7f4ed8067c10 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.923+0000 7f4edd39b700 1 -- 172.21.15.5:0/1637352307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb584580 msgr2=0x55d0bb5849d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.923+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb584580 0x55d0bb5849d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.923+0000 7f4edd39b700 1 -- 172.21.15.5:0/1637352307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 msgr2=0x55d0bb589d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.923+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb589d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.906 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.923+0000 7f4edd39b700 1 -- 172.21.15.5:0/1637352307 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bb303d80 con 0x55d0bb58a2d0 2023-07-21T19:41:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.924+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40cb1b0 tx=0x7f4ed4009cd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=442d36ad580da189 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.924+0000 7f4ec37fe700 1 -- 172.21.15.5:0/1637352307 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409a4c0 con 0x55d0bb58a2d0 2023-07-21T19:41:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.924+0000 7f4ec37fe700 1 -- 172.21.15.5:0/1637352307 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ed409eb90 con 0x55d0bb58a2d0 2023-07-21T19:41:12.907 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.924+0000 7f4ec37fe700 1 -- 172.21.15.5:0/1637352307 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409a050 con 0x55d0bb58a2d0 2023-07-21T19:41:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.925+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1637352307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 msgr2=0x55d0bb584040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.908 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.925+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40cb1b0 tx=0x7f4ed4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.925+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 0x7f4ec40a3870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1637352307 shutdown_connections 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb584580 0x55d0bb5849d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb589d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1637352307 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb584040 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.909 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1637352307 >> 172.21.15.5:0/1637352307 conn(0x55d0bb579be0 msgr2=0x55d0bb57ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.926+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1637352307 shutdown_connections 2023-07-21T19:41:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.927+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1637352307 wait complete. 2023-07-21T19:41:12.910 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.927+0000 7f4ef2f49b80 1 -- start start 2023-07-21T19:41:12.911 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.928+0000 7f4ef2f49b80 1 Processor -- start 2023-07-21T19:41:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 0x7f4ec40a3870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009fd0 tx=0x7f4ed0009040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb584580 0x55d0bb5938c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.913 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb58d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb475a60 con 0x55d0bb584580 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb475760 con 0x55d0bb3e06b0 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb3598c0 con 0x55d0bb58a2d0 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.929+0000 7f4ef2f49b80 10 client.0 _ll_register_callbacks cb 0x55d0bb473700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.930+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.930+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35764/0 (socket says 172.21.15.5:35764) 2023-07-21T19:41:12.914 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.930+0000 7f4edd39b700 1 -- 172.21.15.5:0/3454121737 learned_addr learned my addr 172.21.15.5:0/3454121737 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.930+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb584580 0x55d0bb5938c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.915 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb58d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 -- 172.21.15.5:0/3454121737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 msgr2=0x55d0bb58d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb58d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 -- 172.21.15.5:0/3454121737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb584580 msgr2=0x55d0bb5938c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb584580 0x55d0bb5938c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 -- 172.21.15.5:0/3454121737 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed81a4d40 con 0x55d0bb3e06b0 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.931+0000 7f4edd39b700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b580 tx=0x7f4ed40cb6b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3eeb495296896ed5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.916 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.932+0000 7f4ec0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.932+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed40b2070 con 0x55d0bb3e06b0 2023-07-21T19:41:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.932+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f4ed40a2f20 con 0x55d0bb3e06b0 2023-07-21T19:41:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.932+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0bb42f490 con 0x55d0bb3e06b0 2023-07-21T19:41:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.933+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed4005650 con 0x55d0bb3e06b0 2023-07-21T19:41:12.917 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.933+0000 7f4ef2f49b80 10 client.30680 fetch_fsmap learned FSMap version 73 2023-07-21T19:41:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.933+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55d0bb2ecc00 con 0x55d0bb3e06b0 2023-07-21T19:41:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0bb42f490 con 0x55d0bb3e06b0 2023-07-21T19:41:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 73) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4ed40a2a90 con 0x55d0bb3e06b0 2023-07-21T19:41:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 10 client.30680 fetch_fsmap finished waiting for FSMap version 73 2023-07-21T19:41:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 10 client.30680 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 20 client.30680 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb5a5bc0 0x55d0bb5a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 4 client.30680 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:12.919 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.935+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d0bb4cee50 con 0x55d0bb5a5bc0 2023-07-21T19:41:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.936+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb5a5bc0 0x55d0bb5a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.941+0000 7f4ec0ff9700 10 client.30680 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.941+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb5a5bc0 0x55d0bb5a7fd0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee20fb0535ffe4d7 server_cookie=96a39130be96713a in_seq=0 out_seq=0 2023-07-21T19:41:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.942+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/3454121737 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55d0bb4cee50 con 0x55d0bb5a5bc0 2023-07-21T19:41:12.925 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.942+0000 7f4ec0ff9700 10 client.30680 handle_command_reply: tid=0 2023-07-21T19:41:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 client.30680 shutdown 2023-07-21T19:41:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 msgr2=0x55d0bb593380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400b580 tx=0x7f4ed40cb6b0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.926 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 shutdown_connections 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb58a2d0 0x55d0bb58d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb3e06b0 0x55d0bb593380 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb5a5bc0 0x55d0bb5a7fd0 crc :-1 s=READY pgs=590 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/3454121737 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb584580 0x55d0bb5938c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ec0ff9700 0 client.30680 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 >> 172.21.15.5:0/3454121737 conn(0x55d0bb579be0 msgr2=0x55d0bb57e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:12.927 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.943+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 shutdown_connections 2023-07-21T19:41:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.944+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/3454121737 wait complete. 2023-07-21T19:41:12.928 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.944+0000 7f4ef2f49b80 20 client.30680 trim_cache size 0 max 16384 2023-07-21T19:41:12.950 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 Processor -- start 2023-07-21T19:41:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 -- start start 2023-07-21T19:41:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 0x55d0bb5892b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb58ce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.951 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb588610 con 0x55d0bb3e06b0 2023-07-21T19:41:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb576b40 con 0x55d0bb579be0 2023-07-21T19:41:12.952 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.967+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb5a8630 con 0x55d0bb588200 2023-07-21T19:41:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edcb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 0x55d0bb5892b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edcb9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35778/0 (socket says 172.21.15.5:35778) 2023-07-21T19:41:12.953 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edcb9a700 1 -- 172.21.15.5:0/4073921807 learned_addr learned my addr 172.21.15.5:0/4073921807 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb58ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 -- 172.21.15.5:0/4073921807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 msgr2=0x55d0bb57ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 -- 172.21.15.5:0/4073921807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 msgr2=0x55d0bb58ce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb58ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.954 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edd39b700 1 -- 172.21.15.5:0/4073921807 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d0bb5acbc0 con 0x55d0bb588200 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.968+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.969+0000 7f4edd39b700 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 0x55d0bb5892b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4097060 tx=0x7f4ed409ac60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba01a8371d91d288 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.969+0000 7f4ec3fff700 1 -- 172.21.15.5:0/4073921807 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409adc0 con 0x55d0bb588200 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.969+0000 7f4ec3fff700 1 -- 172.21.15.5:0/4073921807 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4ed40cb140 con 0x55d0bb588200 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.969+0000 7f4ec3fff700 1 -- 172.21.15.5:0/4073921807 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed409b260 con 0x55d0bb588200 2023-07-21T19:41:12.955 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/4073921807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 msgr2=0x55d0bb5892b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:12.956 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 0x55d0bb5892b0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4097060 tx=0x7f4ed409ac60 comp rx=0 tx=0).stop 2023-07-21T19:41:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/4073921807 shutdown_connections 2023-07-21T19:41:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb58ce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb588200 0x55d0bb5892b0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/4073921807 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb579be0 0x55d0bb57ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.957 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/4073921807 >> 172.21.15.5:0/4073921807 conn(0x55d0bb587e30 msgr2=0x55d0bb593680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.970+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/4073921807 shutdown_connections 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.971+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/4073921807 wait complete. 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.971+0000 7f4ef2f49b80 1 -- start start 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.971+0000 7f4ef2f49b80 1 Processor -- start 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb5997e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb579be0 0x55d0bb5960b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.958 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb596b30 con 0x55d0bb3e06b0 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb596c70 con 0x55d0bb588200 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d0bb596e30 con 0x55d0bb579be0 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.972+0000 7f4ef2f49b80 10 client.0 _ll_register_callbacks cb 0x7f4ed4005c00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4eddb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4eddb9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:35794/0 (socket says 172.21.15.5:35794) 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4eddb9c700 1 -- 172.21.15.5:0/1091658519 learned_addr learned my addr 172.21.15.5:0/1091658519 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:12.959 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb5997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4edcb9a700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb579be0 0x55d0bb5960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4eddb9c700 1 -- 172.21.15.5:0/1091658519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb579be0 msgr2=0x55d0bb5960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.973+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb579be0 0x55d0bb5960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4eddb9c700 1 -- 172.21.15.5:0/1091658519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 msgr2=0x55d0bb5997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb5997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:12.960 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4eddb9c700 1 -- 172.21.15.5:0/1091658519 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d0bb2ecc00 con 0x55d0bb588200 2023-07-21T19:41:12.961 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4eddb9c700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0015d80 tx=0x7f4ed000b040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3fdfb189ee83b4b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4ec0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.974+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed000e030 con 0x55d0bb588200 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d0bb490ff0 con 0x55d0bb588200 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(144..144 src has 1..144) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f4ed001d040 con 0x55d0bb588200 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=73) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4ed0014af0 con 0x55d0bb588200 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ef2f49b80 10 client.30686 fetch_fsmap learned FSMap version 73 2023-07-21T19:41:12.962 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=73,monmap=4+}) v3 -- 0x55d0bb32a1f0 con 0x55d0bb588200 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.975+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55d0bb490ff0 con 0x55d0bb588200 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 73) v1 ==== 7466+0+0 (secure 0 0 0) 0x7f4ed00233e0 con 0x55d0bb588200 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 10 client.30686 fetch_fsmap finished waiting for FSMap version 73 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 10 client.30686 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 20 client.30686 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb584580 0x55d0bb59d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:12.963 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 4 client.30686 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:41:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55d0bb324f10 con 0x55d0bb584580 2023-07-21T19:41:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.976+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb584580 0x55d0bb59d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.977+0000 7f4ec0ff9700 10 client.30686 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:12.964 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:12.977+0000 7f4edd39b700 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb584580 0x55d0bb59d6e0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16ad6c49cc8b844e server_cookie=848fb1a95975295e in_seq=0 out_seq=0 2023-07-21T19:41:13.021 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ec0ff9700 1 -- 172.21.15.5:0/1091658519 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55d0bb324f10 con 0x55d0bb584580 2023-07-21T19:41:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ec0ff9700 10 client.30686 handle_command_reply: tid=0 2023-07-21T19:41:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 client.30686 shutdown 2023-07-21T19:41:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 msgr2=0x55d0bb5965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:13.022 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0015d80 tx=0x7f4ed000b040 comp rx=0 tx=0).stop 2023-07-21T19:41:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 shutdown_connections 2023-07-21T19:41:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55d0bb588200 0x55d0bb5965f0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.038+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55d0bb584580 0x55d0bb59d6e0 crc :-1 s=READY pgs=591 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55d0bb3e06b0 0x55d0bb5997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.023 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 1 --2- 172.21.15.5:0/1091658519 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55d0bb579be0 0x55d0bb5960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ec0ff9700 0 client.30686 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 >> 172.21.15.5:0/1091658519 conn(0x55d0bb587e30 msgr2=0x55d0bb5892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 shutdown_connections 2023-07-21T19:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 1 -- 172.21.15.5:0/1091658519 wait complete. 2023-07-21T19:41:13.024 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.039+0000 7f4ef2f49b80 20 client.30686 trim_cache size 0 max 16384 2023-07-21T19:41:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:41:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:41:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:41:13.025 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.040+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 msgr2=0x7f4ec40a3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.040+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 0x7f4ec40a3870 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f4ed0009fd0 tx=0x7f4ed0009040 comp rx=0 tx=0).stop 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.040+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 msgr2=0x7f4ed8197cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.040+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4005910 tx=0x7f4ed400f740 comp rx=0 tx=0).stop 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 shutdown_connections 2023-07-21T19:41:13.026 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f4ec40a13c0 0x7f4ec40a3870 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4ed8104e50 0x7f4ed819b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4ed8067c10 0x7f4ed8197cd0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 --2- 172.21.15.5:0/805856432 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f4ed80685b0 0x7f4ed8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 >> 172.21.15.5:0/805856432 conn(0x7f4ed8074760 msgr2=0x7f4ed80fc840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 shutdown_connections 2023-07-21T19:41:13.027 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:13.041+0000 7f4edf5ff700 1 -- 172.21.15.5:0/805856432 wait complete. 2023-07-21T19:41:13.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:12 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:13.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:12 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:13.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:12 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:14.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:13 smithi101 ceph-mon[128136]: pgmap v1067: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2023-07-21T19:41:14.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:13 smithi005 ceph-mon[126460]: pgmap v1067: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2023-07-21T19:41:14.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:13 smithi156 ceph-mon[128316]: pgmap v1067: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.0 MiB/s wr, 340 op/s 2023-07-21T19:41:16.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:15 smithi101 ceph-mon[128136]: pgmap v1068: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:16.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:15 smithi005 ceph-mon[126460]: pgmap v1068: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:16.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:15 smithi156 ceph-mon[128316]: pgmap v1068: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:16.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 885.0 s, 141.4 tps, lat 77.583 ms stddev 44.456 2023-07-21T19:41:17.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:16 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:17.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:16 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:17.160 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:16 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:41:18.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:17 smithi101 ceph-mon[128136]: pgmap v1069: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:18.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:17 smithi156 ceph-mon[128316]: pgmap v1069: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:18.157 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:17 smithi005 ceph-mon[126460]: pgmap v1069: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.9 MiB/s wr, 317 op/s 2023-07-21T19:41:19.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:18 smithi101 ceph-mon[128136]: osdmap e145: 12 total, 12 up, 12 in 2023-07-21T19:41:19.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:18 smithi101 ceph-mon[128136]: pgmap v1071: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2023-07-21T19:41:19.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:18 smithi005 ceph-mon[126460]: osdmap e145: 12 total, 12 up, 12 in 2023-07-21T19:41:19.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:18 smithi005 ceph-mon[126460]: pgmap v1071: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2023-07-21T19:41:19.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:18 smithi156 ceph-mon[128316]: osdmap e145: 12 total, 12 up, 12 in 2023-07-21T19:41:19.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:18 smithi156 ceph-mon[128316]: pgmap v1071: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 2.8 MiB/s wr, 351 op/s 2023-07-21T19:41:21.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 890.0 s, 157.2 tps, lat 2442.680 ms stddev 7578.045 2023-07-21T19:41:22.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:21 smithi101 ceph-mon[128136]: pgmap v1072: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2023-07-21T19:41:22.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:21 smithi005 ceph-mon[126460]: pgmap v1072: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2023-07-21T19:41:22.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:21 smithi156 ceph-mon[128316]: pgmap v1072: 97 pgs: 97 active+clean; 11 GiB data, 20 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 3.4 MiB/s wr, 381 op/s 2023-07-21T19:41:24.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:23 smithi101 ceph-mon[128136]: pgmap v1073: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2023-07-21T19:41:24.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:23 smithi005 ceph-mon[126460]: pgmap v1073: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2023-07-21T19:41:24.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:23 smithi156 ceph-mon[128316]: pgmap v1073: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.0 MiB/s wr, 538 op/s 2023-07-21T19:41:26.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:25 smithi101 ceph-mon[128136]: pgmap v1074: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:26.144 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:25 smithi005 ceph-mon[126460]: pgmap v1074: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:26.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:25 smithi156 ceph-mon[128316]: pgmap v1074: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.278+0000 7fa2df9ff700 1 -- 172.21.15.5:0/1173018067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 msgr2=0x7fa2d81067b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.278+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/1173018067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d81067b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fa2d4009990 tx=0x7fa2d4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 -- 172.21.15.5:0/1173018067 shutdown_connections 2023-07-21T19:41:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/1173018067 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 0x7fa2d8112f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.263 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/1173018067 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d810c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/1173018067 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d81067b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 -- 172.21.15.5:0/1173018067 >> 172.21.15.5:0/1173018067 conn(0x7fa2d8075a20 msgr2=0x7fa2d8077e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 -- 172.21.15.5:0/1173018067 shutdown_connections 2023-07-21T19:41:26.264 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.279+0000 7fa2df9ff700 1 -- 172.21.15.5:0/1173018067 wait complete. 2023-07-21T19:41:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.281+0000 7fa2df9ff700 1 Processor -- start 2023-07-21T19:41:26.265 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.281+0000 7fa2df9ff700 1 -- start start 2023-07-21T19:41:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d810e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 0x7fa2d810eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d807fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80803a0 con 0x7fa2d810c240 2023-07-21T19:41:26.266 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d8080510 con 0x7fa2d8106cf0 2023-07-21T19:41:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2df9ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2d80806d0 con 0x7fa2d8106340 2023-07-21T19:41:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.282+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d807fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.267 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d807fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52410/0 (socket says 172.21.15.5:52410) 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/4176682675 learned_addr learned my addr 172.21.15.5:0/4176682675 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d810e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dcf9a700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 0x7fa2d810eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 -- 172.21.15.5:0/4176682675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 msgr2=0x7fa2d810eb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 0x7fa2d810eb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.268 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 -- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 msgr2=0x7fa2d807fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d807fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.269 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.283+0000 7fa2dd79b700 1 -- 172.21.15.5:0/4176682675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d0009c40 con 0x7fa2d8106340 2023-07-21T19:41:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d810e600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000efc0 tx=0x7fa2d000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2ce7fc700 1 -- 172.21.15.5:0/4176682675 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0018070 con 0x7fa2d8106340 2023-07-21T19:41:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2ce7fc700 1 -- 172.21.15.5:0/4176682675 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2d000f040 con 0x7fa2d8106340 2023-07-21T19:41:26.270 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2ce7fc700 1 -- 172.21.15.5:0/4176682675 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0004280 con 0x7fa2d8106340 2023-07-21T19:41:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2d40095f0 con 0x7fa2d8106340 2023-07-21T19:41:26.271 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.284+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8080c80 con 0x7fa2d8106340 2023-07-21T19:41:26.274 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.290+0000 7fa2ce7fc700 1 -- 172.21.15.5:0/4176682675 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fa2d000ca20 con 0x7fa2d8106340 2023-07-21T19:41:26.277 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.292+0000 7fa2f3419b80 1 Processor -- start 2023-07-21T19:41:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.293+0000 7fa2ce7fc700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 0x7fa2c40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.278 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.293+0000 7fa2ce7fc700 1 -- 172.21.15.5:0/4176682675 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa2d00cc820 con 0x7fa2d8106340 2023-07-21T19:41:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 -- start start 2023-07-21T19:41:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 0x7fa2c40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b24d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.279 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04a24c50 con 0x562b0497b6b0 2023-07-21T19:41:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04a779e0 con 0x562b04b252d0 2023-07-21T19:41:26.280 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.294+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04a70e70 con 0x562b04b1f580 2023-07-21T19:41:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:32900/0 (socket says 172.21.15.5:32900) 2023-07-21T19:41:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/2448835463 learned_addr learned my addr 172.21.15.5:0/2448835463 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b24d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.281 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/2448835463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 msgr2=0x562b04b1f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/2448835463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 msgr2=0x562b04b24d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b24d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.296+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/2448835463 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b0489ed80 con 0x562b04b1f580 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.297+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800ba90 tx=0x7fa2c801a000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae2a66c88add0f4c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.282 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.297+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 0x7fa2c40a37f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8014660 tx=0x7fa2c8014a20 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.297+0000 7fa2c37fe700 1 -- 172.21.15.5:0/2448835463 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8024070 con 0x562b04b1f580 2023-07-21T19:41:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.297+0000 7fa2c37fe700 1 -- 172.21.15.5:0/2448835463 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c801a070 con 0x562b04b1f580 2023-07-21T19:41:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.297+0000 7fa2c37fe700 1 -- 172.21.15.5:0/2448835463 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8005280 con 0x562b04b1f580 2023-07-21T19:41:26.284 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.298+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2448835463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 msgr2=0x562b04b1f9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.298+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fa2c800ba90 tx=0x7fa2c801a000 comp rx=0 tx=0).stop 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2448835463 shutdown_connections 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b1f9d0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b24d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2448835463 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b1f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2448835463 >> 172.21.15.5:0/2448835463 conn(0x562b04b14be0 msgr2=0x562b04b19f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.285 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2448835463 shutdown_connections 2023-07-21T19:41:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.299+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2448835463 wait complete. 2023-07-21T19:41:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.300+0000 7fa2f3419b80 1 -- start start 2023-07-21T19:41:26.286 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.300+0000 7fa2f3419b80 1 Processor -- start 2023-07-21T19:41:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b28980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.288 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b28ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04a10a60 con 0x562b04b1f580 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04a10760 con 0x562b04b252d0 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b048f48c0 con 0x562b0497b6b0 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.301+0000 7fa2f3419b80 10 client.0 _ll_register_callbacks cb 0x562b04a0e700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:26.289 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:33846/0 (socket says 172.21.15.5:33846) 2023-07-21T19:41:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/1002198728 learned_addr learned my addr 172.21.15.5:0/1002198728 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b28980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.290 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/1002198728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 msgr2=0x562b04b28980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b28980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/1002198728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 msgr2=0x562b04b28ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b28ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.302+0000 7fa2ddf9c700 1 -- 172.21.15.5:0/1002198728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2c8010490 con 0x562b04b252d0 2023-07-21T19:41:26.291 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.303+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005d90 tx=0x7fa2c80142e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99980ee935983517 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.303+0000 7fa2c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.303+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8024070 con 0x562b04b252d0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.303+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b049ca490 con 0x562b04b252d0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa2c800be50 con 0x562b04b252d0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2c8005850 con 0x562b04b252d0 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2f3419b80 10 client.30695 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:26.292 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562b04887c00 con 0x562b04b252d0 2023-07-21T19:41:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x562b049ca490 con 0x562b04b252d0 2023-07-21T19:41:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.304+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2c80117b0 con 0x562b04b252d0 2023-07-21T19:41:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 10 client.30695 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:26.294 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 10 client.30695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 20 client.30695 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b38a50 0x562b04b3ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 4 client.30695 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b04a69e50 con 0x562b04b38a50 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.305+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b38a50 0x562b04b3ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.306+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b38a50 0x562b04b3ae60 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efa4e417a665bd3e server_cookie=e39edc8416c705c6 in_seq=0 out_seq=0 2023-07-21T19:41:26.295 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.306+0000 7fa2c0ff9700 10 client.30695 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.307+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/1002198728 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x562b04a69e50 con 0x562b04b38a50 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2c0ff9700 10 client.30695 handle_command_reply: tid=0 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 client.30695 shutdown 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 msgr2=0x562b04b29400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8005d90 tx=0x7fa2c80142e0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.296 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 shutdown_connections 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b29400 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b28980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b38a50 0x562b04b3ae60 crc :-1 s=READY pgs=592 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/1002198728 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b28ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 >> 172.21.15.5:0/1002198728 conn(0x562b04b14be0 msgr2=0x562b04b18c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.308+0000 7fa2c0ff9700 0 client.30695 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:26.297 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.309+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 shutdown_connections 2023-07-21T19:41:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.309+0000 7fa2f3419b80 1 -- 172.21.15.5:0/1002198728 wait complete. 2023-07-21T19:41:26.298 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.309+0000 7fa2f3419b80 20 client.30695 trim_cache size 0 max 16384 2023-07-21T19:41:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.340+0000 7fa2f3419b80 1 Processor -- start 2023-07-21T19:41:26.324 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.340+0000 7fa2f3419b80 1 -- start start 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.340+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.340+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b24240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b2b9f0 con 0x562b0497b6b0 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b227d0 con 0x562b04b252d0 2023-07-21T19:41:26.325 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b20020 con 0x562b04b1f580 2023-07-21T19:41:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2dd79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.326 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2dd79b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:52440/0 (socket says 172.21.15.5:52440) 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.341+0000 7fa2dd79b700 1 -- 172.21.15.5:0/2663439224 learned_addr learned my addr 172.21.15.5:0/2663439224 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 -- 172.21.15.5:0/2663439224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 msgr2=0x562b04b24240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b24240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 -- 172.21.15.5:0/2663439224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 msgr2=0x562b04b2fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.327 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 -- 172.21.15.5:0/2663439224 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b04b1fc40 con 0x562b0497b6b0 2023-07-21T19:41:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:41:26.328 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.342+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00090a0 tx=0x7fa2d0003a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc39cfddbe18228f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.343+0000 7fa2c3fff700 1 -- 172.21.15.5:0/2663439224 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d0009370 con 0x562b0497b6b0 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.343+0000 7fa2c3fff700 1 -- 172.21.15.5:0/2663439224 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2d0099920 con 0x562b0497b6b0 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.343+0000 7fa2c3fff700 1 -- 172.21.15.5:0/2663439224 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d00b7a20 con 0x562b0497b6b0 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.344+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2663439224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 msgr2=0x562b04b1f040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.344+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa2d00090a0 tx=0x7fa2d0003a00 comp rx=0 tx=0).stop 2023-07-21T19:41:26.329 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.345+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2663439224 shutdown_connections 2023-07-21T19:41:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.345+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.345+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b0497b6b0 0x562b04b1f040 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.345+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/2663439224 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b04b1f580 0x562b04b24240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.345+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2663439224 >> 172.21.15.5:0/2663439224 conn(0x562b04b14be0 msgr2=0x562b04b23b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.330 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.347+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2663439224 shutdown_connections 2023-07-21T19:41:26.332 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.348+0000 7fa2f3419b80 1 -- 172.21.15.5:0/2663439224 wait complete. 2023-07-21T19:41:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.349+0000 7fa2f3419b80 1 -- start start 2023-07-21T19:41:26.333 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.349+0000 7fa2f3419b80 1 Processor -- start 2023-07-21T19:41:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b2d970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.334 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b8ba50 con 0x562b04b1f580 2023-07-21T19:41:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b8bbc0 con 0x562b04b252d0 2023-07-21T19:41:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b04b8bd80 con 0x562b0497b6b0 2023-07-21T19:41:26.335 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.350+0000 7fa2f3419b80 10 client.0 _ll_register_callbacks cb 0x7fa2d0009ac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:32922/0 (socket says 172.21.15.5:32922) 2023-07-21T19:41:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 -- 172.21.15.5:0/3668271490 learned_addr learned my addr 172.21.15.5:0/3668271490 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:26.336 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2ddf9c700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 -- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 msgr2=0x562b04b2deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2deb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 -- 172.21.15.5:0/3668271490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 msgr2=0x562b04b2d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b2d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 -- 172.21.15.5:0/3668271490 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2d8080c80 con 0x562b0497b6b0 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.351+0000 7fa2dd79b700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c6f0 tx=0x7fa2d00a1d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=573b2683fbd00e3b server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:26.337 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2c0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2d009d9d0 con 0x562b0497b6b0 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fa2d00ca190 con 0x562b0497b6b0 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fa2c801a070 con 0x562b0497b6b0 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2d00ca350 con 0x562b0497b6b0 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2f3419b80 10 client.30643 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:26.338 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.352+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x562b048c51f0 con 0x562b0497b6b0 2023-07-21T19:41:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fa2c801a070 con 0x562b0497b6b0 2023-07-21T19:41:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fa2d0098e10 con 0x562b0497b6b0 2023-07-21T19:41:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 10 client.30643 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:26.339 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 10 client.30643 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 20 client.30643 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b8cfa0 0x562b04b8f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 4 client.30643 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x562b04a2bff0 con 0x562b04b8cfa0 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.353+0000 7fa2dcf9a700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b8cfa0 0x562b04b8f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:26.340 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.355+0000 7fa2dcf9a700 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b8cfa0 0x562b04b8f3b0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8601b26e1ace32cc server_cookie=501d9871cd91bfbb in_seq=0 out_seq=0 2023-07-21T19:41:26.341 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.355+0000 7fa2c0ff9700 10 client.30643 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2c0ff9700 1 -- 172.21.15.5:0/3668271490 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x562b04a2bff0 con 0x562b04b8cfa0 2023-07-21T19:41:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2c0ff9700 10 client.30643 handle_command_reply: tid=0 2023-07-21T19:41:26.376 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 client.30643 shutdown 2023-07-21T19:41:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 msgr2=0x562b04b2d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000c6f0 tx=0x7fa2d00a1d50 comp rx=0 tx=0).stop 2023-07-21T19:41:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 shutdown_connections 2023-07-21T19:41:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x562b04b252d0 0x562b04b2deb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.377 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x562b0497b6b0 0x562b04b2d430 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x562b04b8cfa0 0x562b04b8f3b0 crc :-1 s=READY pgs=593 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 --2- 172.21.15.5:0/3668271490 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x562b04b1f580 0x562b04b2d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 >> 172.21.15.5:0/3668271490 conn(0x562b04b14be0 msgr2=0x562b04b28ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.392+0000 7fa2c0ff9700 0 client.30643 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.393+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 shutdown_connections 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.393+0000 7fa2f3419b80 1 -- 172.21.15.5:0/3668271490 wait complete. 2023-07-21T19:41:26.378 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.393+0000 7fa2f3419b80 20 client.30643 trim_cache size 0 max 16384 2023-07-21T19:41:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:41:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:41:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:41:26.379 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:41:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 msgr2=0x7fa2c40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 0x7fa2c40a37f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8014660 tx=0x7fa2c8014a20 comp rx=0 tx=0).stop 2023-07-21T19:41:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 msgr2=0x7fa2d810e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d810e600 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fa2d000efc0 tx=0x7fa2d000c3a0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.380 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 shutdown_connections 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fa2c40a1340 0x7fa2c40a37f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fa2d810c240 0x7fa2d807fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fa2d8106340 0x7fa2d810e600 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 --2- 172.21.15.5:0/4176682675 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fa2d8106cf0 0x7fa2d810eb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 >> 172.21.15.5:0/4176682675 conn(0x7fa2d8075a20 msgr2=0x7fa2d8109e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 shutdown_connections 2023-07-21T19:41:26.381 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:26.394+0000 7fa2df9ff700 1 -- 172.21.15.5:0/4176682675 wait complete. 2023-07-21T19:41:26.931 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 895.0 s, 203.6 tps, lat 965.890 ms stddev 3536.477 2023-07-21T19:41:28.068 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:27 smithi101 ceph-mon[128136]: pgmap v1075: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:28.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:27 smithi005 ceph-mon[126460]: pgmap v1075: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:28.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:27 smithi156 ceph-mon[128316]: pgmap v1075: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.1 MiB/s wr, 557 op/s 2023-07-21T19:41:29.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:29 smithi101 ceph-mon[128136]: pgmap v1076: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 612 op/s 2023-07-21T19:41:29.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:29 smithi005 ceph-mon[126460]: pgmap v1076: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 612 op/s 2023-07-21T19:41:29.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:29 smithi156 ceph-mon[128316]: pgmap v1076: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.8 MiB/s rd, 6.0 MiB/s wr, 612 op/s 2023-07-21T19:41:31.930 INFO:teuthology.orchestra.run.smithi005.stderr:progress: 900.0 s, 222.2 tps, lat 447.052 ms stddev 108.892 2023-07-21T19:41:32.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:31 smithi101 ceph-mon[128136]: pgmap v1077: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 581 op/s 2023-07-21T19:41:32.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:31 smithi005 ceph-mon[126460]: pgmap v1077: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 581 op/s 2023-07-21T19:41:32.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:31 smithi156 ceph-mon[128316]: pgmap v1077: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.5 MiB/s wr, 581 op/s 2023-07-21T19:41:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:transaction type: 2023-07-21T19:41:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:scaling factor: 500 2023-07-21T19:41:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:query mode: simple 2023-07-21T19:41:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:number of clients: 100 2023-07-21T19:41:32.486 INFO:teuthology.orchestra.run.smithi005.stdout:number of threads: 4 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:duration: 900 s 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:number of transactions actually processed: 244673 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:latency average = 367.755 ms 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:latency stddev = 833.134 ms 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:tps = 271.691427 (including connections establishing) 2023-07-21T19:41:32.487 INFO:teuthology.orchestra.run.smithi005.stdout:tps = 271.697562 (excluding connections establishing) 2023-07-21T19:41:32.494 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo systemctl stop postgresql' 2023-07-21T19:41:34.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:33 smithi101 ceph-mon[128136]: pgmap v1078: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 774 op/s 2023-07-21T19:41:34.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:33 smithi005 ceph-mon[126460]: pgmap v1078: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 774 op/s 2023-07-21T19:41:34.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:33 smithi156 ceph-mon[128316]: pgmap v1078: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 7.3 MiB/s wr, 774 op/s 2023-07-21T19:41:35.602 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:41:36.004 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 -- 172.21.15.5:0/1932608185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154112920 msgr2=0x7f2154114d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 --2- 172.21.15.5:0/1932608185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154112920 0x7f2154114d60 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f21440098f0 tx=0x7f2144009c00 comp rx=0 tx=0).stop 2023-07-21T19:41:36.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 -- 172.21.15.5:0/1932608185 shutdown_connections 2023-07-21T19:41:36.005 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 --2- 172.21.15.5:0/1932608185 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154112920 0x7f2154114d60 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 --2- 172.21.15.5:0/1932608185 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f2154107510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 --2- 172.21.15.5:0/1932608185 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2154107a50 0x7f2154107ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.006 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.020+0000 7f2158bff700 1 -- 172.21.15.5:0/1932608185 >> 172.21.15.5:0/1932608185 conn(0x7f215406c200 msgr2=0x7f215406c600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.007 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.021+0000 7f2158bff700 1 -- 172.21.15.5:0/1932608185 shutdown_connections 2023-07-21T19:41:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.021+0000 7f2158bff700 1 -- 172.21.15.5:0/1932608185 wait complete. 2023-07-21T19:41:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.022+0000 7f2158bff700 1 Processor -- start 2023-07-21T19:41:36.008 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 -- start start 2023-07-21T19:41:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f21541a0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2154112920 0x7f21541a4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541a5500 con 0x7f2154112920 2023-07-21T19:41:36.010 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541a5670 con 0x7f215406d950 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.023+0000 7f2158bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21541a5830 con 0x7f2154107a50 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.024+0000 7f2151d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.024+0000 7f2151d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56260/0 (socket says 172.21.15.5:56260) 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.024+0000 7f2151d9b700 1 -- 172.21.15.5:0/1653672894 learned_addr learned my addr 172.21.15.5:0/1653672894 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.024+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f21541a0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 -- 172.21.15.5:0/1653672894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 msgr2=0x7f21541a0a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.011 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 -- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2154112920 msgr2=0x7f21541a4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2154112920 0x7f21541a4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 -- 172.21.15.5:0/1653672894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2144009520 con 0x7f215406d950 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f2151d9b700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f21541a0520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e9c0 tx=0x7f214c00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.012 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f214b7fe700 1 -- 172.21.15.5:0/1653672894 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00c9c0 con 0x7f215406d950 2023-07-21T19:41:36.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f214b7fe700 1 -- 172.21.15.5:0/1653672894 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f214c010430 con 0x7f215406d950 2023-07-21T19:41:36.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.025+0000 7f214b7fe700 1 -- 172.21.15.5:0/1653672894 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c018680 con 0x7f215406d950 2023-07-21T19:41:36.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.026+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21541a5b10 con 0x7f215406d950 2023-07-21T19:41:36.013 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.026+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154115730 con 0x7f215406d950 2023-07-21T19:41:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.032+0000 7f214b7fe700 1 -- 172.21.15.5:0/1653672894 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f214c0105a0 con 0x7f215406d950 2023-07-21T19:41:36.017 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.033+0000 7f216c5cfb80 1 Processor -- start 2023-07-21T19:41:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 -- start start 2023-07-21T19:41:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bbfeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15ab8cb0 con 0x558c15bb2be0 2023-07-21T19:41:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15b0ba40 con 0x558c15bbdad0 2023-07-21T19:41:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15b04ed0 con 0x558c15bb22d0 2023-07-21T19:41:36.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.034+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56264/0 (socket says 172.21.15.5:56264) 2023-07-21T19:41:36.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 -- 172.21.15.5:0/2275579254 learned_addr learned my addr 172.21.15.5:0/2275579254 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:36.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f215259c700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.020 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bbfeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 -- 172.21.15.5:0/2275579254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 msgr2=0x558c15bbfeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bbfeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 -- 172.21.15.5:0/2275579254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 msgr2=0x558c15bbd590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bbd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.035+0000 7f2152d9d700 1 -- 172.21.15.5:0/2275579254 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c15932de0 con 0x558c15bb22d0 2023-07-21T19:41:36.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.036+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2144009230 tx=0x7f214400bf40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e2cb14297a2f429 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.036+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bbfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:41:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.036+0000 7f2148ff9700 1 -- 172.21.15.5:0/2275579254 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214401c070 con 0x558c15bb22d0 2023-07-21T19:41:36.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.037+0000 7f2148ff9700 1 -- 172.21.15.5:0/2275579254 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f214400f8f0 con 0x558c15bb22d0 2023-07-21T19:41:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.037+0000 7f2148ff9700 1 -- 172.21.15.5:0/2275579254 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144005040 con 0x558c15bb22d0 2023-07-21T19:41:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.037+0000 7f214b7fe700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 0x7f21400a38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f214b7fe700 1 -- 172.21.15.5:0/1653672894 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f214c014070 con 0x7f215406d950 2023-07-21T19:41:36.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2275579254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 msgr2=0x558c15bb26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2144009230 tx=0x7f214400bf40 comp rx=0 tx=0).stop 2023-07-21T19:41:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2275579254 shutdown_connections 2023-07-21T19:41:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bbfeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.027 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bb26a0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2275579254 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bbd590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.038+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2275579254 >> 172.21.15.5:0/2275579254 conn(0x558c15ba8da0 msgr2=0x558c15bae6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.028 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.039+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2275579254 shutdown_connections 2023-07-21T19:41:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.039+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2275579254 wait complete. 2023-07-21T19:41:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.039+0000 7f216c5cfb80 1 -- start start 2023-07-21T19:41:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.039+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 0x7f21400a38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 Processor -- start 2023-07-21T19:41:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb2be0 0x558c15bba810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bbdad0 0x558c15bbaf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15aa4ac0 con 0x558c15bbdad0 2023-07-21T19:41:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15aa47c0 con 0x558c15bb2be0 2023-07-21T19:41:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15988920 con 0x558c15bb22d0 2023-07-21T19:41:36.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f216c5cfb80 10 client.0 _ll_register_callbacks cb 0x558c15aa2760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:36.032 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.040+0000 7f215259c700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 0x7f21400a38b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f214c096950 tx=0x7f214c009f70 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.041+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.041+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56270/0 (socket says 172.21.15.5:56270) 2023-07-21T19:41:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.041+0000 7f2152d9d700 1 -- 172.21.15.5:0/2874090229 learned_addr learned my addr 172.21.15.5:0/2874090229 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.041+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb2be0 0x558c15bba810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.036 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.041+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bbdad0 0x558c15bbaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.042+0000 7f2152d9d700 1 -- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb2be0 msgr2=0x558c15bba810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.042+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb2be0 0x558c15bba810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.042+0000 7f2152d9d700 1 -- 172.21.15.5:0/2874090229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bbdad0 msgr2=0x558c15bbaf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.037 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.042+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bbdad0 0x558c15bbaf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.042+0000 7f2152d9d700 1 -- 172.21.15.5:0/2874090229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2154108b10 con 0x558c15bb22d0 2023-07-21T19:41:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.043+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f214400bbd0 tx=0x7f2144019040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b104edd831250b34 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.038 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.043+0000 7f2139ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:41:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.043+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214401c070 con 0x558c15bb22d0 2023-07-21T19:41:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.043+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f214400f770 con 0x558c15bb22d0 2023-07-21T19:41:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.043+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c15afdeb0 con 0x558c15bb22d0 2023-07-21T19:41:36.039 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f21440051a0 con 0x558c15bb22d0 2023-07-21T19:41:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 10 client.30655 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558c1591bc60 con 0x558c15bb22d0 2023-07-21T19:41:36.040 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c15afdeb0 con 0x558c15bb22d0 2023-07-21T19:41:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2144015020 con 0x558c15bb22d0 2023-07-21T19:41:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 10 client.30655 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 10 client.30655 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:36.041 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 20 client.30655 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15bddf30 0x558c15be0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 4 client.30655 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:36.042 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.044+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558c15ac0050 con 0x558c15bddf30 2023-07-21T19:41:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.045+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15bddf30 0x558c15be0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.045+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15bddf30 0x558c15be0310 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=150208811cf0cf7d server_cookie=82f95326f1a10e84 in_seq=0 out_seq=0 2023-07-21T19:41:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.045+0000 7f2139ffb700 10 client.30655 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:36.043 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.046+0000 7f2139ffb700 1 -- 172.21.15.5:0/2874090229 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558c15ac0050 con 0x558c15bddf30 2023-07-21T19:41:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.046+0000 7f2139ffb700 10 client.30655 handle_command_reply: tid=0 2023-07-21T19:41:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.046+0000 7f216c5cfb80 1 client.30655 shutdown 2023-07-21T19:41:36.044 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 msgr2=0x558c15bbd590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f214400bbd0 tx=0x7f2144019040 comp rx=0 tx=0).stop 2023-07-21T19:41:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 shutdown_connections 2023-07-21T19:41:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15bddf30 0x558c15be0310 crc :-1 s=READY pgs=594 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.045 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bbdad0 0x558c15bbaf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd590 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f2139ffb700 0 client.30655 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:36.046 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2874090229 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb2be0 0x558c15bba810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 >> 172.21.15.5:0/2874090229 conn(0x558c15ba8da0 msgr2=0x558c15bae290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 shutdown_connections 2023-07-21T19:41:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.047+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2874090229 wait complete. 2023-07-21T19:41:36.047 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.048+0000 7f216c5cfb80 20 client.30655 trim_cache size 0 max 16384 2023-07-21T19:41:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.073+0000 7f216c5cfb80 1 Processor -- start 2023-07-21T19:41:36.059 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 -- start start 2023-07-21T19:41:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.060 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bb9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bba530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bafd00 con 0x558c15bb2be0 2023-07-21T19:41:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bb3540 con 0x558c15bbdad0 2023-07-21T19:41:36.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.074+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bb3340 con 0x558c15bb22d0 2023-07-21T19:41:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56272/0 (socket says 172.21.15.5:56272) 2023-07-21T19:41:36.062 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 -- 172.21.15.5:0/1110676808 learned_addr learned my addr 172.21.15.5:0/1110676808 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2151d9b700 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bb9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bba530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 -- 172.21.15.5:0/1110676808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 msgr2=0x558c15bba530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.063 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bba530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 -- 172.21.15.5:0/1110676808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 msgr2=0x558c15bb9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bb9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.064 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.075+0000 7f2152d9d700 1 -- 172.21.15.5:0/1110676808 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558c15959250 con 0x558c15bb22d0 2023-07-21T19:41:36.066 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.076+0000 7f2152d9d700 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2144009810 tx=0x7f2144009400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f538ccb4732f5759 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.076+0000 7f21497fa700 1 -- 172.21.15.5:0/1110676808 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214401c070 con 0x558c15bb22d0 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.076+0000 7f21497fa700 1 -- 172.21.15.5:0/1110676808 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f214401d040 con 0x558c15bb22d0 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.076+0000 7f21497fa700 1 -- 172.21.15.5:0/1110676808 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2144003900 con 0x558c15bb22d0 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 -- 172.21.15.5:0/1110676808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 msgr2=0x558c15bbd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2144009810 tx=0x7f2144009400 comp rx=0 tx=0).stop 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 -- 172.21.15.5:0/1110676808 shutdown_connections 2023-07-21T19:41:36.067 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bbdad0 0x558c15bba530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bb22d0 0x558c15bbd130 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/1110676808 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bb9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 -- 172.21.15.5:0/1110676808 >> 172.21.15.5:0/1110676808 conn(0x558c15ba8da0 msgr2=0x558c15bb5b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 -- 172.21.15.5:0/1110676808 shutdown_connections 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.077+0000 7f216c5cfb80 1 -- 172.21.15.5:0/1110676808 wait complete. 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.078+0000 7f216c5cfb80 1 -- start start 2023-07-21T19:41:36.068 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.078+0000 7f216c5cfb80 1 Processor -- start 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb22d0 0x558c15bc41f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bc24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bc4af0 con 0x558c15bb2be0 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bc2f70 con 0x558c15bb22d0 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x558c15bc3100 con 0x558c15bbdad0 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f216c5cfb80 10 client.0 _ll_register_callbacks cb 0x7f214c0091e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:36.069 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f2152d9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:56284/0 (socket says 172.21.15.5:56284) 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.079+0000 7f2152d9d700 1 -- 172.21.15.5:0/2354877084 learned_addr learned my addr 172.21.15.5:0/2354877084 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bc24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 -- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb22d0 msgr2=0x558c15bc41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb22d0 0x558c15bc41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 -- 172.21.15.5:0/2354877084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 msgr2=0x558c15bc24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:36.070 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bc24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 -- 172.21.15.5:0/2354877084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558c1591bc60 con 0x558c15bbdad0 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bc24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2152d9d700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2144003b40 tx=0x7f2144005c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=52689eda5b8915ca server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.080+0000 7f2139ffb700 10 client.0 ms_handle_connect on v2:172.21.15.156:3300/0 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214401c040 con 0x558c15bbdad0 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mon.1 v2:172.21.15.156:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f214400f770 con 0x558c15bbdad0 2023-07-21T19:41:36.071 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558c15a5e4f0 con 0x558c15bbdad0 2023-07-21T19:41:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f214400bec0 con 0x558c15bbdad0 2023-07-21T19:41:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f216c5cfb80 10 client.30661 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.081+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x558c159312b0 con 0x558c15bbdad0 2023-07-21T19:41:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mon.1 v2:172.21.15.156:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x558c15a5e4f0 con 0x558c15bbdad0 2023-07-21T19:41:36.072 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mon.1 v2:172.21.15.156:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2144047070 con 0x558c15bbdad0 2023-07-21T19:41:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 10 client.30661 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:36.073 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 10 client.30661 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 20 client.30661 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15c32c20 0x558c15c35030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 4 client.30661 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.082+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x558c15953f70 con 0x558c15c32c20 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.083+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15c32c20 0x558c15c35030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.084+0000 7f2151d9b700 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15c32c20 0x558c15c35030 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f117a1a5c890756c server_cookie=307ad73b5fa65eba in_seq=0 out_seq=0 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.084+0000 7f2139ffb700 10 client.30661 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:36.074 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.085+0000 7f2139ffb700 1 -- 172.21.15.5:0/2354877084 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x558c15953f70 con 0x558c15c32c20 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.085+0000 7f2139ffb700 10 client.30661 handle_command_reply: tid=0 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.085+0000 7f216c5cfb80 1 client.30661 shutdown 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 msgr2=0x558c15bc2a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2144003b40 tx=0x7f2144005c00 comp rx=0 tx=0).stop 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 shutdown_connections 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x558c15bbdad0 0x558c15bc2a30 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.075 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x558c15bb22d0 0x558c15bc41f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x558c15c32c20 0x558c15c35030 crc :-1 s=READY pgs=595 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 --2- 172.21.15.5:0/2354877084 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x558c15bb2be0 0x558c15bc24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 >> 172.21.15.5:0/2354877084 conn(0x558c15ba8da0 msgr2=0x558c15bb43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f2139ffb700 0 client.30661 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 shutdown_connections 2023-07-21T19:41:36.076 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 1 -- 172.21.15.5:0/2354877084 wait complete. 2023-07-21T19:41:36.077 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:41:36.077 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:41:36.077 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:41:36.077 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:41:36.077 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:41:36.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:41:36.078 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:41:36.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:41:36.078 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:41:36.078 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.086+0000 7f216c5cfb80 20 client.30661 trim_cache size 0 max 16384 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 msgr2=0x7f21400a38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 0x7f21400a38b0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f214c096950 tx=0x7f214c009f70 comp rx=0 tx=0).stop 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 msgr2=0x7f21541a0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f21541a0520 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f214c00e9c0 tx=0x7f214c00ecd0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.079 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 shutdown_connections 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f21400a1400 0x7f21400a38b0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2154112920 0x7f21541a4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f215406d950 0x7f21541a0520 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 --2- 172.21.15.5:0/1653672894 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2154107a50 0x7f21541a0a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.090+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 >> 172.21.15.5:0/1653672894 conn(0x7f215406c200 msgr2=0x7f215410a830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.091+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 shutdown_connections 2023-07-21T19:41:36.080 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:41:36.091+0000 7f2158bff700 1 -- 172.21.15.5:0/1653672894 wait complete. 2023-07-21T19:41:36.100 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:35 smithi005 ceph-mon[126460]: pgmap v1079: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2023-07-21T19:41:36.102 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:41:36.157 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:35 smithi156 ceph-mon[128316]: pgmap v1079: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2023-07-21T19:41:36.316 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:35 smithi101 ceph-mon[128136]: pgmap v1079: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 5.7 MiB/s wr, 640 op/s 2023-07-21T19:41:38.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:37 smithi005 ceph-mon[126460]: pgmap v1080: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2023-07-21T19:41:38.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:37 smithi156 ceph-mon[128316]: pgmap v1080: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2023-07-21T19:41:38.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:37 smithi101 ceph-mon[128136]: pgmap v1080: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 5.5 MiB/s wr, 610 op/s 2023-07-21T19:41:39.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:39 smithi101 ceph-mon[128136]: pgmap v1081: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2023-07-21T19:41:39.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:39 smithi005 ceph-mon[126460]: pgmap v1081: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2023-07-21T19:41:39.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:39 smithi156 ceph-mon[128316]: pgmap v1081: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 9.1 MiB/s wr, 1.06k op/s 2023-07-21T19:41:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 -- 172.21.15.5:0/3313079537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c067c10 msgr2=0x7f468c067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.547 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 --2- 172.21.15.5:0/3313079537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c067c10 0x7f468c067fe0 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7f467c0099c0 tx=0x7f467c009cd0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 -- 172.21.15.5:0/3313079537 shutdown_connections 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 --2- 172.21.15.5:0/3313079537 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c104e50 0x7f468c10ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 --2- 172.21.15.5:0/3313079537 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c067c10 0x7f468c067fe0 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 --2- 172.21.15.5:0/3313079537 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c104770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 -- 172.21.15.5:0/3313079537 >> 172.21.15.5:0/3313079537 conn(0x7f468c074760 msgr2=0x7f468c074b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.548 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 -- 172.21.15.5:0/3313079537 shutdown_connections 2023-07-21T19:41:39.549 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.563+0000 7f4690fff700 1 -- 172.21.15.5:0/3313079537 wait complete. 2023-07-21T19:41:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.564+0000 7f4690fff700 1 Processor -- start 2023-07-21T19:41:39.550 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.564+0000 7f4690fff700 1 -- start start 2023-07-21T19:41:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c19c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 0x7f468c195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c195fe0 con 0x7f468c104e50 2023-07-21T19:41:39.551 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c196150 con 0x7f468c0685b0 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4690fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f468c196310 con 0x7f468c067c10 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468ad9d700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 0x7f468c195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:50770/0 (socket says 172.21.15.5:50770) 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 -- 172.21.15.5:0/1553531025 learned_addr learned my addr 172.21.15.5:0/1553531025 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4689d9b700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c19c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 -- 172.21.15.5:0/1553531025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 msgr2=0x7f468c19c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c19c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 -- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 msgr2=0x7f468c195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 0x7f468c195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468a59c700 1 -- 172.21.15.5:0/1553531025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f467c0095f0 con 0x7f468c067c10 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f468ad9d700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 0x7f468c195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.565+0000 7f4689d9b700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f468a59c700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f467c005910 tx=0x7f467c00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f46837fe700 1 -- 172.21.15.5:0/1553531025 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c01d070 con 0x7f468c067c10 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f468c196590 con 0x7f468c067c10 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f46837fe700 1 -- 172.21.15.5:0/1553531025 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467c005050 con 0x7f468c067c10 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f46837fe700 1 -- 172.21.15.5:0/1553531025 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c004950 con 0x7f468c067c10 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.566+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1a2b00 con 0x7f468c067c10 2023-07-21T19:41:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.570+0000 7f46837fe700 1 -- 172.21.15.5:0/1553531025 <== mon.1 v2:172.21.15.156:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f467c0051c0 con 0x7f468c067c10 2023-07-21T19:41:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.571+0000 7f46a498cb80 1 Processor -- start 2023-07-21T19:41:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 -- start start 2023-07-21T19:41:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 0x5558d16c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16be580 0x5558d16be9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d15c3c50 con 0x5558d16be580 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16169e0 con 0x5558d16c42d0 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f46a498cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d160fe70 con 0x5558d151a6b0 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f468a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.572+0000 7f468ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 0x5558d16c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468ad9d700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 0x5558d16c3d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.156:3300/0 says I am v2:172.21.15.5:50784/0 (socket says 172.21.15.5:50784) 2023-07-21T19:41:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48158/0 (socket says 172.21.15.5:48158) 2023-07-21T19:41:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468ad9d700 1 -- 172.21.15.5:0/2896100373 learned_addr learned my addr 172.21.15.5:0/2896100373 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f46837fe700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 0x7f46700a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.558 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f4689d9b700 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16be580 0x5558d16be9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.559 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f46837fe700 1 -- 172.21.15.5:0/1553531025 <== mon.1 v2:172.21.15.156:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c0cda80 con 0x7f468c067c10 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468ad9d700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 0x7f46700a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 -- 172.21.15.5:0/2896100373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 msgr2=0x5558d16c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 0x5558d16c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 -- 172.21.15.5:0/2896100373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16be580 msgr2=0x5558d16be9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16be580 0x5558d16be9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 -- 172.21.15.5:0/2896100373 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558d143dd80 con 0x5558d16c42d0 2023-07-21T19:41:39.560 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.573+0000 7f468a59c700 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cb170 tx=0x7f467c009310 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34be792e37128c58 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f468ad9d700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 0x7f46700a37f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4684009ec0 tx=0x7f4684004b40 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f4680ff9700 1 -- 172.21.15.5:0/2896100373 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c09a4e0 con 0x5558d16c42d0 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f4680ff9700 1 -- 172.21.15.5:0/2896100373 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467c09ab80 con 0x5558d16c42d0 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f4680ff9700 1 -- 172.21.15.5:0/2896100373 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c09ade0 con 0x5558d16c42d0 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f46a498cb80 1 -- 172.21.15.5:0/2896100373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 msgr2=0x5558d16be040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.574+0000 7f46a498cb80 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f467c0cb170 tx=0x7f467c009310 comp rx=0 tx=0).stop 2023-07-21T19:41:39.561 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 -- 172.21.15.5:0/2896100373 shutdown_connections 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16be580 0x5558d16be9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d151a6b0 0x5558d16c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 --2- 172.21.15.5:0/2896100373 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16be040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 -- 172.21.15.5:0/2896100373 >> 172.21.15.5:0/2896100373 conn(0x5558d16b3be0 msgr2=0x5558d16b8f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 -- 172.21.15.5:0/2896100373 shutdown_connections 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 -- 172.21.15.5:0/2896100373 wait complete. 2023-07-21T19:41:39.562 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 -- start start 2023-07-21T19:41:39.563 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.575+0000 7f46a498cb80 1 Processor -- start 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 0x5558d16cd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16c42d0 0x5558d16c7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d15afa60 con 0x5558d151a6b0 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d15af760 con 0x5558d16be580 2023-07-21T19:41:39.565 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d14938c0 con 0x5558d16c42d0 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f46a498cb80 10 client.0 _ll_register_callbacks cb 0x5558d15ad700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f468a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f468a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39320/0 (socket says 172.21.15.5:39320) 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f468a59c700 1 -- 172.21.15.5:0/248552080 learned_addr learned my addr 172.21.15.5:0/248552080 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.576+0000 7f4689d9b700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 0x5558d16cd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468ad9d700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16c42d0 0x5558d16c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 -- 172.21.15.5:0/248552080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16c42d0 msgr2=0x5558d16c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16c42d0 0x5558d16c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 -- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 msgr2=0x5558d16cd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 0x5558d16cd810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 -- 172.21.15.5:0/248552080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f468c1a2b00 con 0x5558d151a6b0 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f4689d9b700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 0x5558d16cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f468a59c700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f467c09a880 tx=0x7f467c09a5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcaa9dad58042b5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f466dffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c09a070 con 0x5558d151a6b0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c09a1d0 con 0x5558d151a6b0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.577+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5558d1569490 con 0x5558d151a6b0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c09ade0 con 0x5558d151a6b0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 10 client.20949 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5558d1426c00 con 0x5558d151a6b0 2023-07-21T19:41:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x5558d1569490 con 0x5558d151a6b0 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f467c0a2b40 con 0x5558d151a6b0 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 10 client.20949 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 10 client.20949 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 20 client.20949 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d16d73e0 0x5558d16d97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 4 client.20949 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.578+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5558d1608e50 con 0x5558d16d73e0 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.579+0000 7f4689d9b700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d16d73e0 0x5558d16d97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.579+0000 7f4689d9b700 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d16d73e0 0x5558d16d97f0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76b44f061490abbd server_cookie=cca623122bfbbb9c in_seq=0 out_seq=0 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.579+0000 7f466dffb700 10 client.20949 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f466dffb700 1 -- 172.21.15.5:0/248552080 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5558d1608e50 con 0x5558d16d73e0 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f466dffb700 10 client.20949 handle_command_reply: tid=0 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 client.20949 shutdown 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 msgr2=0x5558d16cd2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.570 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f467c09a880 tx=0x7f467c09a5d0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 shutdown_connections 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16c42d0 0x5558d16c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d16d73e0 0x5558d16d97f0 crc :-1 s=READY pgs=596 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16cd2d0 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 --2- 172.21.15.5:0/248552080 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16be580 0x5558d16cd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f466dffb700 0 client.20949 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:39.571 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.580+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 >> 172.21.15.5:0/248552080 conn(0x5558d16b3be0 msgr2=0x5558d16b84e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.581+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 shutdown_connections 2023-07-21T19:41:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.581+0000 7f46a498cb80 1 -- 172.21.15.5:0/248552080 wait complete. 2023-07-21T19:41:39.572 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.581+0000 7f46a498cb80 20 client.20949 trim_cache size 0 max 16384 2023-07-21T19:41:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.596+0000 7f46a498cb80 1 Processor -- start 2023-07-21T19:41:39.580 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 -- start start 2023-07-21T19:41:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16be040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.581 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16c77a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16b55f0 con 0x5558d151a6b0 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16befd0 con 0x5558d16c42d0 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f46a498cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16b9b20 con 0x5558d16be580 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f468a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16be040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f468a59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16be040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:39330/0 (socket says 172.21.15.5:39330) 2023-07-21T19:41:39.582 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f4689d9b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16c77a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f468a59c700 1 -- 172.21.15.5:0/1193801530 learned_addr learned my addr 172.21.15.5:0/1193801530 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.597+0000 7f468ad9d700 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 -- 172.21.15.5:0/1193801530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 msgr2=0x5558d16c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16c6530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 -- 172.21.15.5:0/1193801530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 msgr2=0x5558d16be040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16be040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.583 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 -- 172.21.15.5:0/1193801530 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5558d14641f0 con 0x5558d16be580 2023-07-21T19:41:39.585 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f4689d9b700 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16c77a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5558d16da100 tx=0x7f4678009260 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a5bc8433603e1b5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f46817fa700 1 -- 172.21.15.5:0/1193801530 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678017040 con 0x5558d16be580 2023-07-21T19:41:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f46817fa700 1 -- 172.21.15.5:0/1193801530 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4678018040 con 0x5558d16be580 2023-07-21T19:41:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.598+0000 7f46817fa700 1 -- 172.21.15.5:0/1193801530 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4678004830 con 0x5558d16be580 2023-07-21T19:41:39.586 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- 172.21.15.5:0/1193801530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 msgr2=0x5558d16c77a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16c77a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x5558d16da100 tx=0x7f4678009260 comp rx=0 tx=0).stop 2023-07-21T19:41:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- 172.21.15.5:0/1193801530 shutdown_connections 2023-07-21T19:41:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d16c42d0 0x5558d16c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.587 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d151a6b0 0x5558d16be040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 --2- 172.21.15.5:0/1193801530 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16c77a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- 172.21.15.5:0/1193801530 >> 172.21.15.5:0/1193801530 conn(0x5558d16b3be0 msgr2=0x5558d16c5e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.588 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- 172.21.15.5:0/1193801530 shutdown_connections 2023-07-21T19:41:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- 172.21.15.5:0/1193801530 wait complete. 2023-07-21T19:41:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.599+0000 7f46a498cb80 1 -- start start 2023-07-21T19:41:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 Processor -- start 2023-07-21T19:41:39.589 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16cb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16c42d0 0x5558d172a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.590 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16cbc30 con 0x5558d16c42d0 2023-07-21T19:41:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d16cbda0 con 0x5558d151a6b0 2023-07-21T19:41:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5558d172a7e0 con 0x5558d16be580 2023-07-21T19:41:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f46a498cb80 10 client.0 _ll_register_callbacks cb 0x7f4678009be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:39.591 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.600+0000 7f468a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:48184/0 (socket says 172.21.15.5:48184) 2023-07-21T19:41:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 -- 172.21.15.5:0/4106258733 learned_addr learned my addr 172.21.15.5:0/4106258733 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f4689d9b700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16cb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.592 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468ad9d700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16c42d0 0x5558d172a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 -- 172.21.15.5:0/4106258733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 msgr2=0x5558d16cb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16cb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.593 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 -- 172.21.15.5:0/4106258733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16c42d0 msgr2=0x5558d172a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16c42d0 0x5558d172a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 -- 172.21.15.5:0/4106258733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4678009e70 con 0x5558d151a6b0 2023-07-21T19:41:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f4689d9b700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:39.594 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f468a59c700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f467c0975f0 tx=0x7f467c0a2d60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a20aa3fb48ff8af5 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f466dffb700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.601+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467c0cdc90 con 0x5558d151a6b0 2023-07-21T19:41:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f467c0a2f40 con 0x5558d151a6b0 2023-07-21T19:41:39.595 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f467c09ab80 con 0x5558d151a6b0 2023-07-21T19:41:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f467c0cb150 con 0x5558d151a6b0 2023-07-21T19:41:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 10 client.30728 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:39.596 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x5558d143c250 con 0x5558d151a6b0 2023-07-21T19:41:39.597 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f467c09ab80 con 0x5558d151a6b0 2023-07-21T19:41:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f467c004ad0 con 0x5558d151a6b0 2023-07-21T19:41:39.598 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 10 client.30728 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 10 client.30728 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 20 client.30728 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:39.599 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d172ba40 0x5558d172de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 4 client.30728 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:41:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.602+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5558d15caff0 con 0x5558d172ba40 2023-07-21T19:41:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.603+0000 7f4689d9b700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d172ba40 0x5558d172de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:39.600 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.603+0000 7f4689d9b700 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d172ba40 0x5558d172de20 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb2bf03cf6a66bff server_cookie=e87a76aa5e234d63 in_seq=0 out_seq=0 2023-07-21T19:41:39.601 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.603+0000 7f466dffb700 10 client.30728 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f466dffb700 1 -- 172.21.15.5:0/4106258733 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5558d15caff0 con 0x5558d172ba40 2023-07-21T19:41:39.635 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f466dffb700 10 client.30728 handle_command_reply: tid=0 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 client.30728 shutdown 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 msgr2=0x5558d16caf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f467c0975f0 tx=0x7f467c0a2d60 comp rx=0 tx=0).stop 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 shutdown_connections 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5558d172ba40 0x5558d172de20 crc :-1 s=READY pgs=597 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.636 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5558d16c42d0 0x5558d172a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5558d151a6b0 0x5558d16caf60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 --2- 172.21.15.5:0/4106258733 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5558d16be580 0x5558d16cb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f466dffb700 0 client.30728 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 >> 172.21.15.5:0/4106258733 conn(0x5558d16b3be0 msgr2=0x5558d16c66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 shutdown_connections 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.651+0000 7f46a498cb80 1 -- 172.21.15.5:0/4106258733 wait complete. 2023-07-21T19:41:39.637 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f46a498cb80 20 client.30728 trim_cache size 0 max 16384 2023-07-21T19:41:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:41:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:41:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:41:39.638 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:41:39.639 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 msgr2=0x7f46700a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 0x7f46700a37f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f4684009ec0 tx=0x7f4684004b40 comp rx=0 tx=0).stop 2023-07-21T19:41:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 msgr2=0x7f468c19bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:39.640 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f467c005910 tx=0x7f467c00f690 comp rx=0 tx=0).stop 2023-07-21T19:41:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 shutdown_connections 2023-07-21T19:41:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f46700a1340 0x7f46700a37f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.641 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f468c104e50 0x7f468c195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f468c067c10 0x7f468c19bb10 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 --2- 172.21.15.5:0/1553531025 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f468c0685b0 0x7f468c19c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 >> 172.21.15.5:0/1553531025 conn(0x7f468c074760 msgr2=0x7f468c1036e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:39.642 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 shutdown_connections 2023-07-21T19:41:39.643 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:39.652+0000 7f4690fff700 1 -- 172.21.15.5:0/1553531025 wait complete. 2023-07-21T19:41:42.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:41 smithi101 ceph-mon[128136]: pgmap v1082: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 7.1 MiB/s wr, 845 op/s 2023-07-21T19:41:42.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:41 smithi005 ceph-mon[126460]: pgmap v1082: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 7.1 MiB/s wr, 845 op/s 2023-07-21T19:41:42.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:41 smithi156 ceph-mon[128316]: pgmap v1082: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 939 KiB/s rd, 7.1 MiB/s wr, 845 op/s 2023-07-21T19:41:44.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:43 smithi101 ceph-mon[128136]: pgmap v1083: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 8.6 MiB/s wr, 1.02k op/s 2023-07-21T19:41:44.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:43 smithi005 ceph-mon[126460]: pgmap v1083: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 8.6 MiB/s wr, 1.02k op/s 2023-07-21T19:41:44.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:43 smithi156 ceph-mon[128316]: pgmap v1083: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 916 KiB/s rd, 8.6 MiB/s wr, 1.02k op/s 2023-07-21T19:41:46.067 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:45 smithi101 ceph-mon[128136]: pgmap v1084: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 6.9 MiB/s wr, 881 op/s 2023-07-21T19:41:46.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:45 smithi005 ceph-mon[126460]: pgmap v1084: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 6.9 MiB/s wr, 881 op/s 2023-07-21T19:41:46.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:45 smithi156 ceph-mon[128316]: pgmap v1084: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 316 KiB/s rd, 6.9 MiB/s wr, 881 op/s 2023-07-21T19:41:48.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:47 smithi101 ceph-mon[128136]: pgmap v1085: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.5 MiB/s wr, 802 op/s 2023-07-21T19:41:48.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:47 smithi005 ceph-mon[126460]: pgmap v1085: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.5 MiB/s wr, 802 op/s 2023-07-21T19:41:48.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:47 smithi156 ceph-mon[128316]: pgmap v1085: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 148 KiB/s rd, 6.5 MiB/s wr, 802 op/s 2023-07-21T19:41:49.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:49 smithi101 ceph-mon[128136]: pgmap v1086: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.8 MiB/s wr, 1.18k op/s 2023-07-21T19:41:49.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:49 smithi005 ceph-mon[126460]: pgmap v1086: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.8 MiB/s wr, 1.18k op/s 2023-07-21T19:41:49.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:49 smithi156 ceph-mon[128316]: pgmap v1086: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 447 KiB/s rd, 9.8 MiB/s wr, 1.18k op/s 2023-07-21T19:41:52.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:51 smithi101 ceph-mon[128136]: pgmap v1087: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 6.3 MiB/s wr, 752 op/s 2023-07-21T19:41:52.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:51 smithi005 ceph-mon[126460]: pgmap v1087: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 6.3 MiB/s wr, 752 op/s 2023-07-21T19:41:52.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:51 smithi156 ceph-mon[128316]: pgmap v1087: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 390 KiB/s rd, 6.3 MiB/s wr, 752 op/s 2023-07-21T19:41:52.798 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 -- 172.21.15.5:0/3377168034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 msgr2=0x7f148410c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 --2- 172.21.15.5:0/3377168034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f148410c920 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f147c009f90 tx=0x7f147c00b330 comp rx=0 tx=0).stop 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 -- 172.21.15.5:0/3377168034 shutdown_connections 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 --2- 172.21.15.5:0/3377168034 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f148410c920 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 --2- 172.21.15.5:0/3377168034 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f1484105b60 0x7f1484105f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 --2- 172.21.15.5:0/3377168034 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f14840ffb30 0x7f14840fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 -- 172.21.15.5:0/3377168034 >> 172.21.15.5:0/3377168034 conn(0x7f14840fb5f0 msgr2=0x7f14840fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.799 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 -- 172.21.15.5:0/3377168034 shutdown_connections 2023-07-21T19:41:52.800 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.815+0000 7f1489dff700 1 -- 172.21.15.5:0/3377168034 wait complete. 2023-07-21T19:41:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.816+0000 7f1489dff700 1 Processor -- start 2023-07-21T19:41:52.801 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 -- start start 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f1484072f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1484105b60 0x7f14841a3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841a4290 con 0x7f14840ffb30 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841a43d0 con 0x7f14841004e0 2023-07-21T19:41:52.802 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1489dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14841a4540 con 0x7f1484105b60 2023-07-21T19:41:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f14837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f14837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59858/0 (socket says 172.21.15.5:59858) 2023-07-21T19:41:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f14837fe700 1 -- 172.21.15.5:0/366190987 learned_addr learned my addr 172.21.15.5:0/366190987 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1482ffd700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f1484072f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.803 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.817+0000 7f1483fff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1484105b60 0x7f14841a3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 -- 172.21.15.5:0/366190987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1484105b60 msgr2=0x7f14841a3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1484105b60 0x7f14841a3d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 -- 172.21.15.5:0/366190987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 msgr2=0x7f1484072f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f1484072f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.804 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 -- 172.21.15.5:0/366190987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14740095f0 con 0x7f14840ffb30 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f14837fe700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f147400ec10 tx=0x7f147400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f1480ff9700 1 -- 172.21.15.5:0/366190987 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147400cd40 con 0x7f14840ffb30 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f147c009c40 con 0x7f14840ffb30 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f1480ff9700 1 -- 172.21.15.5:0/366190987 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f14740044b0 con 0x7f14840ffb30 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f1480ff9700 1 -- 172.21.15.5:0/366190987 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474010430 con 0x7f14840ffb30 2023-07-21T19:41:52.805 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.818+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14841a4bf0 con 0x7f14840ffb30 2023-07-21T19:41:52.806 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.822+0000 7f1480ff9700 1 -- 172.21.15.5:0/366190987 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f1474004000 con 0x7f14840ffb30 2023-07-21T19:41:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 Processor -- start 2023-07-21T19:41:52.807 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 -- start start 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400fbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43fffbc50 con 0x55b4400f6580 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44004e9e0 con 0x55b43ff526b0 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.824+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440047e70 con 0x55b4400fc2d0 2023-07-21T19:41:52.809 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1480ff9700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 0x7f146c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1483fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400fbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400f69d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59868/0 (socket says 172.21.15.5:59868) 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1482ffd700 1 -- 172.21.15.5:0/2049655062 learned_addr learned my addr 172.21.15.5:0/2049655062 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1483fff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 0x7f146c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.810 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f1480ff9700 1 -- 172.21.15.5:0/366190987 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f1474014070 con 0x7f14840ffb30 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 -- 172.21.15.5:0/2049655062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 msgr2=0x55b4400fbd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400fbd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 -- 172.21.15.5:0/2049655062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 msgr2=0x55b4400f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.825+0000 7f14837fe700 1 -- 172.21.15.5:0/2049655062 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b43fe75d80 con 0x55b4400fc2d0 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.826+0000 7f14837fe700 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f6040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f14740ca8e0 tx=0x7f147406c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f743dd7db3f95031 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.826+0000 7f1483fff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 0x7f146c0a37f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f147c007900 tx=0x7f147c006000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.811 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.826+0000 7f1469ffb700 1 -- 172.21.15.5:0/2049655062 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474099da0 con 0x55b4400fc2d0 2023-07-21T19:41:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.826+0000 7f1469ffb700 1 -- 172.21.15.5:0/2049655062 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1474099bc0 con 0x55b4400fc2d0 2023-07-21T19:41:52.812 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.826+0000 7f1469ffb700 1 -- 172.21.15.5:0/2049655062 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1474096970 con 0x55b4400fc2d0 2023-07-21T19:41:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- 172.21.15.5:0/2049655062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 msgr2=0x55b4400f6040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f6040 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f14740ca8e0 tx=0x7f147406c2d0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- 172.21.15.5:0/2049655062 shutdown_connections 2023-07-21T19:41:52.815 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400fbd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/2049655062 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f6040 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- 172.21.15.5:0/2049655062 >> 172.21.15.5:0/2049655062 conn(0x55b4400ebbe0 msgr2=0x55b4400f0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.816 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- 172.21.15.5:0/2049655062 shutdown_connections 2023-07-21T19:41:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- 172.21.15.5:0/2049655062 wait complete. 2023-07-21T19:41:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.827+0000 7f149d6b7b80 1 -- start start 2023-07-21T19:41:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 Processor -- start 2023-07-21T19:41:52.817 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b43ff526b0 0x55b440105540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b440105a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43ffe7a60 con 0x55b4400f6580 2023-07-21T19:41:52.818 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43ffe7760 con 0x55b4400fc2d0 2023-07-21T19:41:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b43fecb8c0 con 0x55b43ff526b0 2023-07-21T19:41:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f149d6b7b80 10 client.0 _ll_register_callbacks cb 0x55b43ffe5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f1483fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.819 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b440105a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f1483fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:52094/0 (socket says 172.21.15.5:52094) 2023-07-21T19:41:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.828+0000 7f1483fff700 1 -- 172.21.15.5:0/3654879846 learned_addr learned my addr 172.21.15.5:0/3654879846 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f14837fe700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b43ff526b0 0x55b440105540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.820 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 -- 172.21.15.5:0/3654879846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b43ff526b0 msgr2=0x55b440105540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b43ff526b0 0x55b440105540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 -- 172.21.15.5:0/3654879846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 msgr2=0x55b440105a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b440105a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.821 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 -- 172.21.15.5:0/3654879846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1474094e70 con 0x55b4400fc2d0 2023-07-21T19:41:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1482ffd700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b440105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1483fff700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f147c015000 tx=0x7f147c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d31808d2518a589 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.829+0000 7f1462ffd700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:41:52.822 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f147c025050 con 0x55b4400fc2d0 2023-07-21T19:41:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b43ffa1490 con 0x55b4400fc2d0 2023-07-21T19:41:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f147c01f850 con 0x55b4400fc2d0 2023-07-21T19:41:52.823 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f147c00e070 con 0x55b4400fc2d0 2023-07-21T19:41:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 10 client.30740 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55b43fe5ec00 con 0x55b4400fc2d0 2023-07-21T19:41:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f147c039080 con 0x55b4400fc2d0 2023-07-21T19:41:52.824 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f147c035810 con 0x55b4400fc2d0 2023-07-21T19:41:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 10 client.30740 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 10 client.30740 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:52.825 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.830+0000 7f149d6b7b80 20 client.30740 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.831+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b44010fb40 0x55b440111f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.831+0000 7f149d6b7b80 4 client.30740 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:41:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.831+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b440040e50 con 0x55b44010fb40 2023-07-21T19:41:52.826 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.831+0000 7f14837fe700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b44010fb40 0x55b440111f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.832+0000 7f1462ffd700 10 client.30740 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:52.827 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.832+0000 7f14837fe700 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b44010fb40 0x55b440111f50 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0e01ffb712a05e2 server_cookie=df064d32b156df00 in_seq=0 out_seq=0 2023-07-21T19:41:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.832+0000 7f1462ffd700 1 -- 172.21.15.5:0/3654879846 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b440040e50 con 0x55b44010fb40 2023-07-21T19:41:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.832+0000 7f1462ffd700 10 client.30740 handle_command_reply: tid=0 2023-07-21T19:41:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 client.30740 shutdown 2023-07-21T19:41:52.829 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 msgr2=0x55b4400ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f147c015000 tx=0x7f147c009920 comp rx=0 tx=0).stop 2023-07-21T19:41:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 shutdown_connections 2023-07-21T19:41:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b4400ff840 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.830 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b43ff526b0 0x55b440105540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b44010fb40 0x55b440111f50 crc :-1 s=READY pgs=598 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3654879846 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b440105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f1462ffd700 0 client.30740 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:52.831 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 >> 172.21.15.5:0/3654879846 conn(0x55b4400ebbe0 msgr2=0x55b4400efc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 shutdown_connections 2023-07-21T19:41:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3654879846 wait complete. 2023-07-21T19:41:52.832 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.833+0000 7f149d6b7b80 20 client.30740 trim_cache size 0 max 16384 2023-07-21T19:41:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 Processor -- start 2023-07-21T19:41:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 -- start start 2023-07-21T19:41:52.838 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400f6040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f0c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.839 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b44010c3b0 con 0x55b4400f6580 2023-07-21T19:41:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4400fc050 con 0x55b43ff526b0 2023-07-21T19:41:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4400fb650 con 0x55b4400fc2d0 2023-07-21T19:41:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.840 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f1482ffd700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59886/0 (socket says 172.21.15.5:59886) 2023-07-21T19:41:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f1482ffd700 1 -- 172.21.15.5:0/1189060751 learned_addr learned my addr 172.21.15.5:0/1189060751 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.849+0000 7f14837fe700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1483fff700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.841 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 -- 172.21.15.5:0/1189060751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 msgr2=0x55b4400f0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 -- 172.21.15.5:0/1189060751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 msgr2=0x55b4400f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.842 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 -- 172.21.15.5:0/1189060751 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b4400ed420 con 0x55b4400f6580 2023-07-21T19:41:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f14837fe700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f1482ffd700 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f14780125e0 tx=0x7f1478009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fede0d844d1e8d5e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f146a7fc700 1 -- 172.21.15.5:0/1189060751 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478017070 con 0x55b4400f6580 2023-07-21T19:41:52.843 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f146a7fc700 1 -- 172.21.15.5:0/1189060751 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1478018040 con 0x55b4400f6580 2023-07-21T19:41:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.850+0000 7f146a7fc700 1 -- 172.21.15.5:0/1189060751 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1478003dc0 con 0x55b4400f6580 2023-07-21T19:41:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 -- 172.21.15.5:0/1189060751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 msgr2=0x55b4400fb110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.844 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f14780125e0 tx=0x7f1478009260 comp rx=0 tx=0).stop 2023-07-21T19:41:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 -- 172.21.15.5:0/1189060751 shutdown_connections 2023-07-21T19:41:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400fc2d0 0x55b4400f0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b43ff526b0 0x55b4400f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.845 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/1189060751 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b4400f6580 0x55b4400fb110 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 -- 172.21.15.5:0/1189060751 >> 172.21.15.5:0/1189060751 conn(0x55b4400ebbe0 msgr2=0x55b4400faa00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 -- 172.21.15.5:0/1189060751 shutdown_connections 2023-07-21T19:41:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.851+0000 7f149d6b7b80 1 -- 172.21.15.5:0/1189060751 wait complete. 2023-07-21T19:41:52.846 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 -- start start 2023-07-21T19:41:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 Processor -- start 2023-07-21T19:41:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.847 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400f6580 0x55b440103850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b440103d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b4401042d0 con 0x55b43ff526b0 2023-07-21T19:41:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440104440 con 0x55b4400fc2d0 2023-07-21T19:41:52.848 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b440104600 con 0x55b4400f6580 2023-07-21T19:41:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.852+0000 7f149d6b7b80 10 client.0 _ll_register_callbacks cb 0x55b43ffe5700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:41:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1483fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b440103d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.849 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:59898/0 (socket says 172.21.15.5:59898) 2023-07-21T19:41:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 -- 172.21.15.5:0/3392134945 learned_addr learned my addr 172.21.15.5:0/3392134945 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:41:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1482ffd700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400f6580 0x55b440103850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 -- 172.21.15.5:0/3392134945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400f6580 msgr2=0x55b440103850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.850 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400f6580 0x55b440103850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 -- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 msgr2=0x55b440103d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b440103d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 -- 172.21.15.5:0/3392134945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b43fe75d80 con 0x55b43ff526b0 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1483fff700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b440103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f14837fe700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f14740caee0 tx=0x7f147400ef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21861a75d04e6c26 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1462ffd700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:41:52.851 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14740a26b0 con 0x55b43ff526b0 2023-07-21T19:41:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f147409a260 con 0x55b43ff526b0 2023-07-21T19:41:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.853+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1474099bc0 con 0x55b43ff526b0 2023-07-21T19:41:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1474096970 con 0x55b43ff526b0 2023-07-21T19:41:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 10 client.20973 fetch_fsmap learned FSMap version 74 2023-07-21T19:41:52.852 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55b43fe9c1f0 con 0x55b43ff526b0 2023-07-21T19:41:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f1474099bc0 con 0x55b43ff526b0 2023-07-21T19:41:52.853 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f1474099750 con 0x55b43ff526b0 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 10 client.20973 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 10 client.20973 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 20 client.20973 populate_metadata read hostname 'smithi005' 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b440164b00 0x55b440166f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 4 client.20973 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55b440002ff0 con 0x55b440164b00 2023-07-21T19:41:52.854 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.854+0000 7f1482ffd700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b440164b00 0x55b440166f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:41:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.855+0000 7f1482ffd700 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b440164b00 0x55b440166f10 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=698c75c162ddc2f2 server_cookie=23c5d68f13fbda54 in_seq=0 out_seq=0 2023-07-21T19:41:52.855 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.855+0000 7f1462ffd700 10 client.20973 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.899+0000 7f1462ffd700 1 -- 172.21.15.5:0/3392134945 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55b440002ff0 con 0x55b440164b00 2023-07-21T19:41:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.899+0000 7f1462ffd700 10 client.20973 handle_command_reply: tid=0 2023-07-21T19:41:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 client.20973 shutdown 2023-07-21T19:41:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 msgr2=0x55b4401055d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.883 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f14740caee0 tx=0x7f147400ef90 comp rx=0 tx=0).stop 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 shutdown_connections 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55b4400fc2d0 0x55b440103d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55b440164b00 0x55b440166f10 crc :-1 s=READY pgs=599 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55b43ff526b0 0x55b4401055d0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 --2- 172.21.15.5:0/3392134945 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55b4400f6580 0x55b440103850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.884 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f1462ffd700 0 client.20973 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:41:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 >> 172.21.15.5:0/3392134945 conn(0x55b4400ebbe0 msgr2=0x55b4400fddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 shutdown_connections 2023-07-21T19:41:52.885 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 1 -- 172.21.15.5:0/3392134945 wait complete. 2023-07-21T19:41:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.900+0000 7f149d6b7b80 20 client.20973 trim_cache size 0 max 16384 2023-07-21T19:41:52.886 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:41:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:41:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:41:52.887 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:41:52.888 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 msgr2=0x7f146c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 0x7f146c0a37f0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f147c007900 tx=0x7f147c006000 comp rx=0 tx=0).stop 2023-07-21T19:41:52.889 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 msgr2=0x7f14840729f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:41:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f147400ec10 tx=0x7f147400ef20 comp rx=0 tx=0).stop 2023-07-21T19:41:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 shutdown_connections 2023-07-21T19:41:52.890 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f1484105b60 0x7f14841a3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f146c0a1340 0x7f146c0a37f0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f14840ffb30 0x7f14840729f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 --2- 172.21.15.5:0/366190987 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f14841004e0 0x7f1484072f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:41:52.891 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 >> 172.21.15.5:0/366190987 conn(0x7f14840fb5f0 msgr2=0x7f14840fcad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:41:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 shutdown_connections 2023-07-21T19:41:52.892 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:41:52.901+0000 7f1489dff700 1 -- 172.21.15.5:0/366190987 wait complete. 2023-07-21T19:41:54.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:53 smithi101 ceph-mon[128136]: pgmap v1088: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2023-07-21T19:41:54.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:53 smithi005 ceph-mon[126460]: pgmap v1088: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2023-07-21T19:41:54.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:53 smithi156 ceph-mon[128316]: pgmap v1088: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 418 KiB/s rd, 6.0 MiB/s wr, 704 op/s 2023-07-21T19:41:56.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:55 smithi101 ceph-mon[128136]: pgmap v1089: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-07-21T19:41:56.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:55 smithi005 ceph-mon[126460]: pgmap v1089: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-07-21T19:41:56.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:55 smithi156 ceph-mon[128316]: pgmap v1089: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 4.2 MiB/s wr, 493 op/s 2023-07-21T19:41:58.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:57 smithi101 ceph-mon[128136]: pgmap v1090: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 406 op/s 2023-07-21T19:41:58.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:57 smithi005 ceph-mon[126460]: pgmap v1090: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 406 op/s 2023-07-21T19:41:58.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:57 smithi156 ceph-mon[128316]: pgmap v1090: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 328 KiB/s rd, 3.5 MiB/s wr, 406 op/s 2023-07-21T19:41:59.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:59 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:41:59.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:41:59 smithi101 ceph-mon[128136]: pgmap v1091: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2023-07-21T19:41:59.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:59 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:41:59.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:41:59 smithi005 ceph-mon[126460]: pgmap v1091: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2023-07-21T19:41:59.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:59 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:41:59.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:41:59 smithi156 ceph-mon[128316]: pgmap v1091: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 564 KiB/s rd, 3.8 MiB/s wr, 428 op/s 2023-07-21T19:42:00.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:00 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:42:00.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:00 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:42:00.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:00 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:42:00.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:00 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:42:00.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:00 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:42:00.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:00 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:42:00.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:00 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:42:00.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:00 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:42:00.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:00 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:42:01.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:01 smithi101 ceph-mon[128136]: pgmap v1092: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 500 KiB/s wr, 54 op/s 2023-07-21T19:42:01.330 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:01 smithi156 ceph-mon[128316]: pgmap v1092: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 500 KiB/s wr, 54 op/s 2023-07-21T19:42:01.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:01 smithi005 ceph-mon[126460]: pgmap v1092: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 265 KiB/s rd, 500 KiB/s wr, 54 op/s 2023-07-21T19:42:04.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:03 smithi101 ceph-mon[128136]: pgmap v1093: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 387 KiB/s wr, 49 op/s 2023-07-21T19:42:04.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:03 smithi005 ceph-mon[126460]: pgmap v1093: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 387 KiB/s wr, 49 op/s 2023-07-21T19:42:04.154 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:03 smithi156 ceph-mon[128316]: pgmap v1093: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 341 KiB/s rd, 387 KiB/s wr, 49 op/s 2023-07-21T19:42:04.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ls -lZaR /tmp/cephfs/postgres/' 2023-07-21T19:42:06.055 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.071+0000 7f2bd37ff700 1 -- 172.21.15.5:0/3460092415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc072830 msgr2=0x7f2bcc072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.071+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/3460092415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc072830 0x7f2bcc072c00 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc8009ca0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 -- 172.21.15.5:0/3460092415 shutdown_connections 2023-07-21T19:42:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/3460092415 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/3460092415 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc072830 0x7f2bcc072c00 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.056 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/3460092415 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc073140 0x7f2bcc10bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 -- 172.21.15.5:0/3460092415 >> 172.21.15.5:0/3460092415 conn(0x7f2bcc0fb650 msgr2=0x7f2bcc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 -- 172.21.15.5:0/3460092415 shutdown_connections 2023-07-21T19:42:06.057 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.072+0000 7f2bd37ff700 1 -- 172.21.15.5:0/3460092415 wait complete. 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.073+0000 7f2bd37ff700 1 Processor -- start 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.073+0000 7f2bd37ff700 1 -- start start 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc072830 0x7f2bcc197da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc19c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc19cf90 con 0x7f2bcc10c410 2023-07-21T19:42:06.058 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc19d100 con 0x7f2bcc073140 2023-07-21T19:42:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd37ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bcc19d2c0 con 0x7f2bcc072830 2023-07-21T19:42:06.059 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd159b700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc072830 0x7f2bcc197da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45260/0 (socket says 172.21.15.5:45260) 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/2181463386 learned_addr learned my addr 172.21.15.5:0/2181463386 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.074+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc19c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/2181463386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc072830 msgr2=0x7f2bcc197da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.060 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc072830 0x7f2bcc197da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 msgr2=0x7f2bcc19c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc19c950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/2181463386 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc4009c40 con 0x7f2bcc073140 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e9a0 tx=0x7f2bbc00ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bc27fc700 1 -- 172.21.15.5:0/2181463386 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc009790 con 0x7f2bcc073140 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.075+0000 7f2bc27fc700 1 -- 172.21.15.5:0/2181463386 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bbc00ce50 con 0x7f2bcc073140 2023-07-21T19:42:06.061 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.076+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bc80095f0 con 0x7f2bcc073140 2023-07-21T19:42:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.076+0000 7f2bc27fc700 1 -- 172.21.15.5:0/2181463386 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc018420 con 0x7f2bcc073140 2023-07-21T19:42:06.062 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.076+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc19d790 con 0x7f2bcc073140 2023-07-21T19:42:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.079+0000 7f2bc27fc700 1 -- 172.21.15.5:0/2181463386 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2bbc010430 con 0x7f2bcc073140 2023-07-21T19:42:06.063 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.080+0000 7f2be705db80 1 Processor -- start 2023-07-21T19:42:06.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:05 smithi101 ceph-mon[128136]: pgmap v1094: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 359 KiB/s wr, 45 op/s 2023-07-21T19:42:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.080+0000 7f2be705db80 1 -- start start 2023-07-21T19:42:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438289d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 0x55a438284040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a4382849d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.065 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438189c50 con 0x55a4380e06b0 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4381dc9e0 con 0x55a438284580 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2be705db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4381d5e70 con 0x55a43828a2d0 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438289d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2bd0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a4382849d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438289d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41402/0 (socket says 172.21.15.5:41402) 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2bd1d9c700 1 -- 172.21.15.5:0/183253005 learned_addr learned my addr 172.21.15.5:0/183253005 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.066 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.081+0000 7f2bd159b700 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 0x55a438284040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/183253005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 msgr2=0x55a438284040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 0x55a438284040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/183253005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 msgr2=0x55a438289d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438289d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.068 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/183253005 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a438003d80 con 0x55a438284580 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd159b700 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 0x55a438284040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bc27fc700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 0x7f2bb80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bc27fc700 1 -- 172.21.15.5:0/2181463386 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2bbc014070 con 0x7f2bcc073140 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a4382849d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ca570 tx=0x7f2bbc0d0f40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9848e1bc50971dbd server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 0x7f2bb80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bb77fe700 1 -- 172.21.15.5:0/183253005 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc096230 con 0x55a438284580 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bb77fe700 1 -- 172.21.15.5:0/183253005 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bbc099cc0 con 0x55a438284580 2023-07-21T19:42:06.069 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.082+0000 7f2bb77fe700 1 -- 172.21.15.5:0/183253005 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc09df20 con 0x55a438284580 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 -- 172.21.15.5:0/183253005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 msgr2=0x55a4382849d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a4382849d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc0ca570 tx=0x7f2bbc0d0f40 comp rx=0 tx=0).stop 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 0x7f2bb80a37f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007850 tx=0x7f2bc400b040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 -- 172.21.15.5:0/183253005 shutdown_connections 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a4382849d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438289d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.070 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 --2- 172.21.15.5:0/183253005 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a43828a2d0 0x55a438284040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 -- 172.21.15.5:0/183253005 >> 172.21.15.5:0/183253005 conn(0x55a438279be0 msgr2=0x55a43827ef10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.083+0000 7f2be705db80 1 -- 172.21.15.5:0/183253005 shutdown_connections 2023-07-21T19:42:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.084+0000 7f2be705db80 1 -- 172.21.15.5:0/183253005 wait complete. 2023-07-21T19:42:06.071 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.084+0000 7f2be705db80 1 -- start start 2023-07-21T19:42:06.073 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 Processor -- start 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a43828d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a43828dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438175a60 con 0x55a43828a2d0 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438175760 con 0x55a4380e06b0 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4380598c0 con 0x55a438284580 2023-07-21T19:42:06.074 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.085+0000 7f2be705db80 10 client.0 _ll_register_callbacks cb 0x55a438173700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a43828dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45270/0 (socket says 172.21.15.5:45270) 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 -- 172.21.15.5:0/2283556474 learned_addr learned my addr 172.21.15.5:0/2283556474 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a43828d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 -- 172.21.15.5:0/2283556474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 msgr2=0x55a43828d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a43828d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.075 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 -- 172.21.15.5:0/2283556474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 msgr2=0x55a43828dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a43828dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 -- 172.21.15.5:0/2283556474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc10f3f0 con 0x55a4380e06b0 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a43828d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.086+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc80057e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c38017ae0f283b52 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2bb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801d070 con 0x55a4380e06b0 2023-07-21T19:42:06.076 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a43812f490 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2bc8004500 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc80046c0 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2be705db80 10 client.30758 fetch_fsmap learned FSMap version 74 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x7f2bcc19d790 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.087+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x55a43812f490 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2bc800f7f0 con 0x55a4380e06b0 2023-07-21T19:42:06.077 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 10 client.30758 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 10 client.30758 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 20 client.30758 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a43829dc10 0x55a4382a0020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 4 client.30758 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a4381cee50 con 0x55a43829dc10 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.088+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a43829dc10 0x55a4382a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.089+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a43829dc10 0x55a4382a0020 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0c58ae08f2ebf17 server_cookie=bad3561da62b0b80 in_seq=0 out_seq=0 2023-07-21T19:42:06.078 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.089+0000 7f2bb4ff9700 10 client.30758 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.089+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/2283556474 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a4381cee50 con 0x55a43829dc10 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.089+0000 7f2bb4ff9700 10 client.30758 handle_command_reply: tid=0 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.089+0000 7f2be705db80 1 client.30758 shutdown 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 msgr2=0x55a4382905c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f2bc8009990 tx=0x7f2bc80057e0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 shutdown_connections 2023-07-21T19:42:06.079 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a43829dc10 0x55a4382a0020 crc :-1 s=READY pgs=600 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a43828dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a4380e06b0 0x55a4382905c0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 --2- 172.21.15.5:0/2283556474 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a43828d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2bb4ff9700 0 client.30758 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 >> 172.21.15.5:0/2283556474 conn(0x55a438279be0 msgr2=0x55a43827e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 shutdown_connections 2023-07-21T19:42:06.080 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 1 -- 172.21.15.5:0/2283556474 wait complete. 2023-07-21T19:42:06.081 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.090+0000 7f2be705db80 20 client.30758 trim_cache size 0 max 16384 2023-07-21T19:42:06.093 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.107+0000 7f2be705db80 1 Processor -- start 2023-07-21T19:42:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.107+0000 7f2be705db80 1 -- start start 2023-07-21T19:42:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a438289130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 0x55a438290160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.094 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43828ee20 con 0x55a4380e06b0 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a43829b240 con 0x55a43828a2d0 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2be705db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a438288610 con 0x55a438284580 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 0x55a438290160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd159b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41424/0 (socket says 172.21.15.5:41424) 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 0x55a438290160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45286/0 (socket says 172.21.15.5:45286) 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd159b700 1 -- 172.21.15.5:0/2191524351 learned_addr learned my addr 172.21.15.5:0/2191524351 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.095 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.108+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a438289130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 -- 172.21.15.5:0/2191524351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 msgr2=0x55a438289130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a438289130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 -- 172.21.15.5:0/2191524351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 msgr2=0x55a438290160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 0x55a438290160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 -- 172.21.15.5:0/2191524351 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a438284c20 con 0x55a4380e06b0 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bd159b700 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098e0 tx=0x7f2bc8005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d416f2d70cfd1e8 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.096 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bb7fff700 1 -- 172.21.15.5:0/2191524351 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801d050 con 0x55a4380e06b0 2023-07-21T19:42:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bb7fff700 1 -- 172.21.15.5:0/2191524351 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bc801e040 con 0x55a4380e06b0 2023-07-21T19:42:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.109+0000 7f2bb7fff700 1 -- 172.21.15.5:0/2191524351 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80235f0 con 0x55a4380e06b0 2023-07-21T19:42:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 -- 172.21.15.5:0/2191524351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 msgr2=0x55a438284040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.097 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f2bc80098e0 tx=0x7f2bc8005cd0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 -- 172.21.15.5:0/2191524351 shutdown_connections 2023-07-21T19:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a43828a2d0 0x55a438290160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a4380e06b0 0x55a438284040 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.099 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 --2- 172.21.15.5:0/2191524351 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a438284580 0x55a438289130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 -- 172.21.15.5:0/2191524351 >> 172.21.15.5:0/2191524351 conn(0x55a438279be0 msgr2=0x55a438288a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 -- 172.21.15.5:0/2191524351 shutdown_connections 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.110+0000 7f2be705db80 1 -- 172.21.15.5:0/2191524351 wait complete. 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 -- start start 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 Processor -- start 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4380e06b0 0x55a438292c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.100 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4382f0af0 con 0x55a43828a2d0 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4382f0c30 con 0x55a438284580 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.111+0000 7f2be705db80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a4382f0d70 con 0x55a4380e06b0 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2be705db80 10 client.0 _ll_register_callbacks cb 0x7f2bc8005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.101 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd0d9a700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45294/0 (socket says 172.21.15.5:45294) 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:41426/0 (socket says 172.21.15.5:41426) 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd0d9a700 1 -- 172.21.15.5:0/296852252 learned_addr learned my addr 172.21.15.5:0/296852252 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd159b700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4380e06b0 0x55a438292c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 -- 172.21.15.5:0/296852252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4380e06b0 msgr2=0x55a438292c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4380e06b0 0x55a438292c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 -- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 msgr2=0x55a438293190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.102 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd1d9c700 1 -- 172.21.15.5:0/296852252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bcc19d790 con 0x55a43828a2d0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.112+0000 7f2bd0d9a700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bd1d9c700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4008010 tx=0x7f2bc4004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d93466e29d4491 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bb4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc4027070 con 0x55a43828a2d0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2bbc099cc0 con 0x55a43828a2d0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2bc4022b80 con 0x55a43828a2d0 2023-07-21T19:42:06.103 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2bc4022d40 con 0x55a43828a2d0 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2be705db80 10 client.20991 fetch_fsmap learned FSMap version 74 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x55a437fecc00 con 0x55a43828a2d0 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.113+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2bc4038020 con 0x55a43828a2d0 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2bc402b3f0 con 0x55a43828a2d0 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 10 client.20991 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 10 client.20991 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:06.104 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 20 client.20991 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a4382f25c0 0x55a4382f49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 4 client.20991 mds_command: new command op to 24614 tid=0[{"prefix": "flush journal"}] 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a438190ff0 con 0x55a4382f25c0 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.114+0000 7f2bd159b700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a4382f25c0 0x55a4382f49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.115+0000 7f2bd159b700 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a4382f25c0 0x55a4382f49d0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54968e1ffe40d29d server_cookie=5f8b8d7587bbaeb7 in_seq=0 out_seq=0 2023-07-21T19:42:06.105 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.115+0000 7f2bb4ff9700 10 client.20991 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.105 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2bb4ff9700 1 -- 172.21.15.5:0/296852252 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a438190ff0 con 0x55a4382f25c0 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2bb4ff9700 10 client.20991 handle_command_reply: tid=0 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 client.20991 shutdown 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 msgr2=0x55a4382936d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4008010 tx=0x7f2bc4004b40 comp rx=0 tx=0).stop 2023-07-21T19:42:06.133 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 shutdown_connections 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x55a4382f25c0 0x55a4382f49d0 crc :-1 s=READY pgs=601 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x55a43828a2d0 0x55a4382936d0 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2bb4ff9700 0 client.20991 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x55a4380e06b0 0x55a438292c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 --2- 172.21.15.5:0/296852252 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x55a438284580 0x55a438293190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.134 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 >> 172.21.15.5:0/296852252 conn(0x55a438279be0 msgr2=0x55a4382892b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.149+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 shutdown_connections 2023-07-21T19:42:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2be705db80 1 -- 172.21.15.5:0/296852252 wait complete. 2023-07-21T19:42:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2be705db80 20 client.20991 trim_cache size 0 max 16384 2023-07-21T19:42:06.135 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:{ 2023-07-21T19:42:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "message": "", 2023-07-21T19:42:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout: "return_code": 0 2023-07-21T19:42:06.136 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stdout:} 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 msgr2=0x7f2bb80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 0x7f2bb80a37f0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4007850 tx=0x7f2bc400b040 comp rx=0 tx=0).stop 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 msgr2=0x7f2bcc1982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.150+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e9a0 tx=0x7f2bbc00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 shutdown_connections 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2bb80a1340 0x7f2bb80a37f0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.137 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2bcc10c410 0x7f2bcc19c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2bcc072830 0x7f2bcc197da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 --2- 172.21.15.5:0/2181463386 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2bcc073140 0x7f2bcc1982e0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 >> 172.21.15.5:0/2181463386 conn(0x7f2bcc0fb650 msgr2=0x7f2bcc10e640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 shutdown_connections 2023-07-21T19:42:06.138 INFO:teuthology.task.background_exec.ubuntu@smithi005.front.sepia.ceph.com.smithi005.stderr:2023-07-21T19:42:06.151+0000 7f2bd37ff700 1 -- 172.21.15.5:0/2181463386 wait complete. 2023-07-21T19:42:06.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:05 smithi005 ceph-mon[126460]: pgmap v1094: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 359 KiB/s wr, 45 op/s 2023-07-21T19:42:06.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:05 smithi156 ceph-mon[128316]: pgmap v1094: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 313 KiB/s rd, 359 KiB/s wr, 45 op/s 2023-07-21T19:42:06.259 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/: 2023-07-21T19:42:06.260 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:42:06.260 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8983165423 Jul 21 19:07 . 2023-07-21T19:42:06.260 INFO:teuthology.orchestra.run.smithi005.stdout:drwxrwxrwt. 3 root root system_u:object_r:postgresql_db_t:s0 8966388207 Jul 21 19:07 .. 2023-07-21T19:42:06.260 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8983177086 Jul 21 19:42 data 2023-07-21T19:42:06.260 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data: 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:total 41 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8983177086 Jul 21 19:42 . 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:drwxr-xr-x. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 8983165423 Jul 21 19:07 .. 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 base 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 30 Jul 21 19:08 current_logfiles 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Jul 21 19:08 global 2023-07-21T19:42:06.261 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 733 Jul 21 19:08 log 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_commit_ts 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_dynshmem 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 4269 Jul 21 19:07 pg_hba.conf 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1636 Jul 21 19:07 pg_ident.conf 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:41 pg_logical 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 pg_multixact 2023-07-21T19:42:06.262 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:08 pg_notify 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_replslot 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_serial 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_snapshots 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Jul 21 19:42 pg_stat 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Jul 21 19:42 pg_stat_tmp 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jul 21 19:38 pg_subtrans 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_tblspc 2023-07-21T19:42:06.263 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 pg_twophase 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:42 pg_wal 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jul 21 19:07 pg_xact 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 88 Jul 21 19:07 postgresql.auto.conf 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 22989 Jul 21 19:07 postgresql.conf 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 51 Jul 21 19:08 postmaster.opts 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:06.264 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base: 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:total 3 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 . 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8983177086 Jul 21 19:42 .. 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 1 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 12346 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7893793927 Jul 21 19:27 12347 2023-07-21T19:42:06.265 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:26 pgsql_tmp 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/1: 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:total 7186 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 . 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 .. 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:42:06.266 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:42:06.267 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:42:06.268 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:42:06.269 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:42:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:07 1247 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1247_fsm 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1247_vm 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:07 1249 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1249_fsm 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1249_vm 2023-07-21T19:42:06.271 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:07 1259 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1259_vm 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:42:06.272 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:42:06.273 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:42:06.274 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:42:06.275 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:42:06.276 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606_vm 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 21 19:07 2608 2023-07-21T19:42:06.277 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2608_fsm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2608_vm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610_fsm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2610_vm 2023-07-21T19:42:06.278 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:42:06.279 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:42:06.280 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 21 19:07 2619 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2619_fsm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2619_vm 2023-07-21T19:42:06.281 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:42:06.282 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 21 19:07 2658 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2659 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2662 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2663 2023-07-21T19:42:06.283 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2664 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2665 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2666 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2667 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:42:06.284 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:07 2673 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:07 2674 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2678 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2679 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:42:06.285 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:42:06.286 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2696 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2703 2023-07-21T19:42:06.287 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2704 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:42:06.288 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:42:06.289 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:42:06.290 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:42:06.291 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:42:06.292 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:42:06.293 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:42:06.294 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:42:06.294 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:42:06.294 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:42:06.294 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:42:06.294 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 3455 2023-07-21T19:42:06.295 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:42:06.296 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:42:06.297 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:42:06.298 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:42:06.299 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:42:06.300 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:42:06.301 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:42:06.302 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:42:06.303 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:42:06.304 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:42:06.304 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:42:06.304 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:42:06.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 -- 172.21.15.5:0/3489677498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 msgr2=0x7f0284101070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 --2- 172.21.15.5:0/3489677498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284101070 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0274009990 tx=0x7f0274009ca0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 -- 172.21.15.5:0/3489677498 shutdown_connections 2023-07-21T19:42:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 --2- 172.21.15.5:0/3489677498 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028410c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 --2- 172.21.15.5:0/3489677498 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284101070 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 --2- 172.21.15.5:0/3489677498 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 0x7f0284109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 -- 172.21.15.5:0/3489677498 >> 172.21.15.5:0/3489677498 conn(0x7f02840fa4b0 msgr2=0x7f02840fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 -- 172.21.15.5:0/3489677498 shutdown_connections 2023-07-21T19:42:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.360+0000 7f0288bff700 1 -- 172.21.15.5:0/3489677498 wait complete. 2023-07-21T19:42:06.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.361+0000 7f0288bff700 1 Processor -- start 2023-07-21T19:42:06.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 -- start start 2023-07-21T19:42:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 0x7f0284198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028419c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028419cd30 con 0x7f028410a070 2023-07-21T19:42:06.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028419cea0 con 0x7f0284100ca0 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.362+0000 7f0288bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f028419d060 con 0x7f02841015b0 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f028259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f028259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:45310/0 (socket says 172.21.15.5:45310) 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f028259c700 1 -- 172.21.15.5:0/4078001851 learned_addr learned my addr 172.21.15.5:0/4078001851 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028419c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0281d9b700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 0x7f0284198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 -- 172.21.15.5:0/4078001851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 msgr2=0x7f0284198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 0x7f0284198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 -- 172.21.15.5:0/4078001851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 msgr2=0x7f0284197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 -- 172.21.15.5:0/4078001851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f027c009c40 con 0x7f028410a070 2023-07-21T19:42:06.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f028259c700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.363+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028419c6f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f027c00efb0 tx=0x7f027c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.364+0000 7f02737fe700 1 -- 172.21.15.5:0/4078001851 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c003a00 con 0x7f028410a070 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.364+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02740095f0 con 0x7f028410a070 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.364+0000 7f02737fe700 1 -- 172.21.15.5:0/4078001851 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027c00f040 con 0x7f028410a070 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.364+0000 7f02737fe700 1 -- 172.21.15.5:0/4078001851 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027c0044d0 con 0x7f028410a070 2023-07-21T19:42:06.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.364+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028419d530 con 0x7f028410a070 2023-07-21T19:42:06.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.367+0000 7f02737fe700 1 -- 172.21.15.5:0/4078001851 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f027c00cbc0 con 0x7f028410a070 2023-07-21T19:42:06.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 Processor -- start 2023-07-21T19:42:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 -- start start 2023-07-21T19:42:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6556a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b662eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b55bcb0 con 0x56246b660ad0 2023-07-21T19:42:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b5aea40 con 0x56246b655be0 2023-07-21T19:42:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b5a7ed0 con 0x56246b6552d0 2023-07-21T19:42:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f02737fe700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 0x7f026c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.369+0000 7f02737fe700 1 -- 172.21.15.5:0/4078001851 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f027c0cc7b0 con 0x7f028410a070 2023-07-21T19:42:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b662eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:43628/0 (socket says 172.21.15.5:43628) 2023-07-21T19:42:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 -- 172.21.15.5:0/3718655151 learned_addr learned my addr 172.21.15.5:0/3718655151 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6556a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 0x7f026c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 -- 172.21.15.5:0/3718655151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 msgr2=0x56246b6556a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6556a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 -- 172.21.15.5:0/3718655151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 msgr2=0x56246b662eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b662eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.370+0000 7f028259c700 1 -- 172.21.15.5:0/3718655151 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56246b3d5de0 con 0x56246b655be0 2023-07-21T19:42:06.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.371+0000 7f028259c700 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f027400bac0 tx=0x7f027400baf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a84ec2577439ada0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.371+0000 7f0282d9d700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 0x7f026c0a37f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f027c0c9fb0 tx=0x7f027c01c000 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.371+0000 7f0270ff9700 1 -- 172.21.15.5:0/3718655151 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401d070 con 0x56246b655be0 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.371+0000 7f0270ff9700 1 -- 172.21.15.5:0/3718655151 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f027401e040 con 0x56246b655be0 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.371+0000 7f0270ff9700 1 -- 172.21.15.5:0/3718655151 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0274004db0 con 0x56246b655be0 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3718655151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 msgr2=0x56246b660590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f027400bac0 tx=0x7f027400baf0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3718655151 shutdown_connections 2023-07-21T19:42:06.358 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b662eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6556a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3718655151 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b660590 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3718655151 >> 172.21.15.5:0/3718655151 conn(0x56246b64bda0 msgr2=0x56246b6516a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3718655151 shutdown_connections 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3718655151 wait complete. 2023-07-21T19:42:06.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.372+0000 7f029c5d3b80 1 -- start start 2023-07-21T19:42:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.373+0000 7f029c5d3b80 1 Processor -- start 2023-07-21T19:42:06.361 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6604a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b65d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b547ac0 con 0x56246b660ad0 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b5477c0 con 0x56246b655be0 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b42b920 con 0x56246b6552d0 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f029c5d3b80 10 client.0 _ll_register_callbacks cb 0x56246b545760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:06.362 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36436/0 (socket says 172.21.15.5:36436) 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 -- 172.21.15.5:0/2559620964 learned_addr learned my addr 172.21.15.5:0/2559620964 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6604a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f028259c700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b65d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 -- 172.21.15.5:0/2559620964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 msgr2=0x56246b6604a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6604a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 -- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 msgr2=0x56246b65d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.363 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.374+0000 7f0281d9b700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b65d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0281d9b700 1 -- 172.21.15.5:0/2559620964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f028410d170 con 0x56246b660ad0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0281d9b700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0278012b90 tx=0x7f02780047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8618b30de6c5ff7e server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0269ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0278004830 con 0x56246b660ad0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56246b5014f0 con 0x56246b660ad0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0278004990 con 0x56246b660ad0 2023-07-21T19:42:06.364 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f027800e870 con 0x56246b660ad0 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f029c5d3b80 10 client.21003 fetch_fsmap learned FSMap version 74 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.375+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x56246b3bec60 con 0x56246b660ad0 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x56246b5014f0 con 0x56246b660ad0 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f027802a890 con 0x56246b660ad0 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 10 client.21003 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 10 client.21003 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 20 client.21003 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:06.365 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6702c0 0x56246b6726d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 4 client.21003 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56246b5a0eb0 con 0x56246b6702c0 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.376+0000 7f028259c700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6702c0 0x56246b6726d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.377+0000 7f028259c700 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6702c0 0x56246b6726d0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8409005a45ca7afe server_cookie=a9514f570d1d2f04 in_seq=0 out_seq=0 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.377+0000 7f0269ffb700 10 client.21003 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f0269ffb700 1 -- 172.21.15.5:0/2559620964 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56246b5a0eb0 con 0x56246b6702c0 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f0269ffb700 10 client.21003 handle_command_reply: tid=0 2023-07-21T19:42:06.366 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 client.21003 shutdown 2023-07-21T19:42:06.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 msgr2=0x56246b65dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f0278012b90 tx=0x7f02780047c0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.367 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 shutdown_connections 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6702c0 0x56246b6726d0 crc :-1 s=READY pgs=602 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b660ad0 0x56246b65dcc0 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b6552d0 0x56246b6604a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f0269ffb700 0 client.21003 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/2559620964 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b655be0 0x56246b65d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 >> 172.21.15.5:0/2559620964 conn(0x56246b64bda0 msgr2=0x56246b651230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 shutdown_connections 2023-07-21T19:42:06.368 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 1 -- 172.21.15.5:0/2559620964 wait complete. 2023-07-21T19:42:06.369 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.378+0000 7f029c5d3b80 20 client.21003 trim_cache size 0 max 16384 2023-07-21T19:42:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 Processor -- start 2023-07-21T19:42:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 -- start start 2023-07-21T19:42:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 0x56246b657a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.380 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b660ad0 0x56246b657fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b656540 con 0x56246b6552d0 2023-07-21T19:42:06.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b66ccc0 con 0x56246b660ad0 2023-07-21T19:42:06.381 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.396+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b6571a0 con 0x56246b655be0 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f0281d9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36442/0 (socket says 172.21.15.5:36442) 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f0281d9b700 1 -- 172.21.15.5:0/1479983242 learned_addr learned my addr 172.21.15.5:0/1479983242 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f0281d9b700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b660ad0 0x56246b657fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 0x56246b657a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.382 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 -- 172.21.15.5:0/1479983242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b660ad0 msgr2=0x56246b657fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b660ad0 0x56246b657fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 -- 172.21.15.5:0/1479983242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 msgr2=0x56246b6674b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 -- 172.21.15.5:0/1479983242 --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56246b3fc250 con 0x56246b655be0 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f0281d9b700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.397+0000 7f028259c700 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 0x56246b657a90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0274009ce0 tx=0x7f027401a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea71f23313e3de5c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.383 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.398+0000 7f02717fa700 1 -- 172.21.15.5:0/1479983242 <== mon.1 v2:172.21.15.156:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401d040 con 0x56246b655be0 2023-07-21T19:42:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.398+0000 7f02717fa700 1 -- 172.21.15.5:0/1479983242 <== mon.1 v2:172.21.15.156:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0274009290 con 0x56246b655be0 2023-07-21T19:42:06.384 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.398+0000 7f02717fa700 1 -- 172.21.15.5:0/1479983242 <== mon.1 v2:172.21.15.156:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027400f6c0 con 0x56246b655be0 2023-07-21T19:42:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- 172.21.15.5:0/1479983242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 msgr2=0x56246b657a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.385 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 0x56246b657a90 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0274009ce0 tx=0x7f027401a040 comp rx=0 tx=0).stop 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- 172.21.15.5:0/1479983242 shutdown_connections 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b660ad0 0x56246b657fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b6552d0 0x56246b6674b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/1479983242 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b655be0 0x56246b657a90 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- 172.21.15.5:0/1479983242 >> 172.21.15.5:0/1479983242 conn(0x56246b64bda0 msgr2=0x56246b657380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- 172.21.15.5:0/1479983242 shutdown_connections 2023-07-21T19:42:06.386 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- 172.21.15.5:0/1479983242 wait complete. 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.399+0000 7f029c5d3b80 1 -- start start 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.400+0000 7f029c5d3b80 1 Processor -- start 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.400+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 0x56246b6bdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.400+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.400+0000 7f029c5d3b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b660ad0 0x56246b6b8250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b6b8830 con 0x56246b655be0 2023-07-21T19:42:06.387 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b6b89a0 con 0x56246b6552d0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f029c5d3b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x56246b6b8b60 con 0x56246b660ad0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f029c5d3b80 10 client.0 _ll_register_callbacks cb 0x56246b545760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36446/0 (socket says 172.21.15.5:36446) 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 -- 172.21.15.5:0/3561267022 learned_addr learned my addr 172.21.15.5:0/3561267022 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 0x56246b6bdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b660ad0 0x56246b6b8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.388 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 -- 172.21.15.5:0/3561267022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b660ad0 msgr2=0x56246b6b8250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b660ad0 0x56246b6b8250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 -- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 msgr2=0x56246b6bdeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 0x56246b6bdeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 -- 172.21.15.5:0/3561267022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56246b3bec60 con 0x56246b655be0 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 0x56246b6bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f028259c700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0274006010 tx=0x7f0274004800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff1d42abb7c2782 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.401+0000 7f0269ffb700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:42:06.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f027401d070 con 0x56246b655be0 2023-07-21T19:42:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f027401e040 con 0x56246b655be0 2023-07-21T19:42:06.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(145..145 src has 1..145) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f027400fc10 con 0x56246b655be0 2023-07-21T19:42:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=74) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0274045460 con 0x56246b655be0 2023-07-21T19:42:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f029c5d3b80 10 client.21009 fetch_fsmap learned FSMap version 74 2023-07-21T19:42:06.391 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=74,monmap=4+}) v3 -- 0x56246b3d5de0 con 0x56246b655be0 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.402+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0274049080 con 0x56246b655be0 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 74) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f027402c6c0 con 0x56246b655be0 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 10 client.21009 fetch_fsmap finished waiting for FSMap version 74 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 10 client.21009 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 20 client.21009 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6c5960 0x56246b6c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:06.392 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 4 client.21009 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x56246b563050 con 0x56246b6c5960 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6c5960 0x56246b6c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f0281d9b700 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6c5960 0x56246b6c7d70 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb915922f8a470e3 server_cookie=c13a5a85b733d1c8 in_seq=0 out_seq=0 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.403+0000 7f0269ffb700 10 client.21009 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f0269ffb700 1 -- 172.21.15.5:0/3561267022 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x56246b563050 con 0x56246b6c5960 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f0269ffb700 10 client.21009 handle_command_reply: tid=0 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 client.21009 shutdown 2023-07-21T19:42:06.393 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 msgr2=0x56246b6be3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f0274006010 tx=0x7f0274004800 comp rx=0 tx=0).stop 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 shutdown_connections 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x56246b660ad0 0x56246b6b8250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x56246b6552d0 0x56246b6bdeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x56246b6c5960 0x56246b6c7d70 crc :-1 s=READY pgs=603 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 --2- 172.21.15.5:0/3561267022 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x56246b655be0 0x56246b6be3f0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 >> 172.21.15.5:0/3561267022 conn(0x56246b64bda0 msgr2=0x56246b651140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f0269ffb700 0 client.21009 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 shutdown_connections 2023-07-21T19:42:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 1 -- 172.21.15.5:0/3561267022 wait complete. 2023-07-21T19:42:06.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.404+0000 7f029c5d3b80 20 client.21009 trim_cache size 0 max 16384 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:42:06.396 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:42:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:42:06.397 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:42:06.397 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:42:06.397 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 msgr2=0x7f026c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 0x7f026c0a37f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f027c0c9fb0 tx=0x7f027c01c000 comp rx=0 tx=0).stop 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 msgr2=0x7f028419c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028419c6f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f027c00efb0 tx=0x7f027c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 shutdown_connections 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f026c0a1340 0x7f026c0a37f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f028410a070 0x7f028419c6f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.398 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f0284100ca0 0x7f0284197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 --2- 172.21.15.5:0/4078001851 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f02841015b0 0x7f0284198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 >> 172.21.15.5:0/4078001851 conn(0x7f02840fa4b0 msgr2=0x7f028410c250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 shutdown_connections 2023-07-21T19:42:06.399 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:06.405+0000 7f0288bff700 1 -- 172.21.15.5:0/4078001851 wait complete. 2023-07-21T19:42:06.408 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:42:07.154 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:42:07.154 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:42:07.154 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:42:07.154 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/12346: 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:total 7186 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7356931 Jul 21 19:07 . 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 .. 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:42:07.155 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:42:07.156 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:42:07.157 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:42:07.158 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:42:07.159 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:42:07.159 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:42:07.159 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:42:07.159 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:42:07.159 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:42:07.160 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:07 1247 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1247_fsm 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1247_vm 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:07 1249 2023-07-21T19:42:07.161 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1249_fsm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1249_vm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:07 1259 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1259_vm 2023-07-21T19:42:07.162 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:42:07.163 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:42:07.164 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:42:07.165 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:42:07.166 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2606_vm 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:42:07.167 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 442368 Jul 21 19:07 2608 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2608_fsm 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2608_vm 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610 2023-07-21T19:42:07.168 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2610_fsm 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2610_vm 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:42:07.169 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:42:07.170 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 131072 Jul 21 19:07 2619 2023-07-21T19:42:07.171 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2619_fsm 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2619_vm 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:42:07.172 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 106496 Jul 21 19:07 2658 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2659 2023-07-21T19:42:07.173 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2662 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2663 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2664 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2665 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2666 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2667 2023-07-21T19:42:07.174 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:07 2673 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:07 2674 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2678 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2679 2023-07-21T19:42:07.175 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:42:07.176 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:42:07.177 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:42:07.177 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:42:07.177 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:42:07.177 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:42:07.178 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:42:07.178 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:42:07.178 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2696 2023-07-21T19:42:07.178 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:42:07.179 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:42:07.179 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:42:07.179 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2703 2023-07-21T19:42:07.179 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2704 2023-07-21T19:42:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:42:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:42:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:42:07.180 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:42:07.181 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:42:07.181 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:42:07.181 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:42:07.181 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:42:07.182 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:42:07.182 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:42:07.182 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:42:07.183 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:42:07.183 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:42:07.183 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:42:07.183 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:42:07.184 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:42:07.184 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:42:07.184 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:42:07.185 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:42:07.185 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:42:07.185 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:42:07.185 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:42:07.186 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:42:07.186 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:42:07.186 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:42:07.187 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:42:07.187 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:42:07.187 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:42:07.187 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:42:07.188 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:42:07.188 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:42:07.188 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:42:07.188 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:42:07.188 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:42:07.189 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:42:07.189 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:42:07.189 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:42:07.189 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:42:07.190 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:42:07.190 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:42:07.190 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:42:07.191 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:42:07.191 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:42:07.191 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:42:07.191 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:42:07.191 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:42:07.192 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:42:07.192 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:42:07.192 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:42:07.192 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:42:07.193 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:42:07.193 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:42:07.193 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:42:07.193 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:42:07.193 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 3455 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:42:07.194 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:42:07.195 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:42:07.196 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:42:07.197 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:42:07.197 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:42:07.197 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:42:07.198 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:42:07.199 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:42:07.200 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:42:07.201 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:42:07.202 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:42:07.203 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:42:07.203 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:42:07.203 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:42:07.203 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:42:07.203 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:42:07.782 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:07 smithi156 ceph-mon[128316]: pgmap v1095: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 43 op/s 2023-07-21T19:42:08.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:07 smithi101 ceph-mon[128136]: pgmap v1095: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 43 op/s 2023-07-21T19:42:08.065 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:07 smithi101 ceph-mon[128136]: osdmap e146: 12 total, 12 up, 12 in 2023-07-21T19:42:08.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:07 smithi005 ceph-mon[126460]: pgmap v1095: 97 pgs: 97 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 312 KiB/s rd, 358 KiB/s wr, 43 op/s 2023-07-21T19:42:08.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:07 smithi005 ceph-mon[126460]: osdmap e146: 12 total, 12 up, 12 in 2023-07-21T19:42:09.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:09 smithi101 ceph-mon[128136]: pgmap v1097: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:09.351 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:07 smithi156 ceph-mon[128316]: osdmap e146: 12 total, 12 up, 12 in 2023-07-21T19:42:09.351 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:09 smithi156 ceph-mon[128316]: pgmap v1097: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:09.394 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:09 smithi005 ceph-mon[126460]: pgmap v1097: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:09.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:42:09.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:42:09.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:42:09.499 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:42:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/12347: 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:total 7708785 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 7893793927 Jul 21 19:27 . 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 .. 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 112 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 113 2023-07-21T19:42:09.501 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 57344 Jul 21 19:07 12182 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12182_fsm 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12182_vm 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12184 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12186 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12187_fsm 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12187_vm 2023-07-21T19:42:09.502 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12189 2023-07-21T19:42:09.503 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12191 2023-07-21T19:42:09.503 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192 2023-07-21T19:42:09.503 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12192_fsm 2023-07-21T19:42:09.503 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12192_vm 2023-07-21T19:42:09.503 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12194 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12196 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12197_fsm 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12197_vm 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12199 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12201 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202 2023-07-21T19:42:09.504 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12202_fsm 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12202_vm 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12204 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12206 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 12207_fsm 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12207_vm 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12209 2023-07-21T19:42:09.505 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12211 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12212 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 12214 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 12216 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 73728 Jul 21 19:09 1247 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:09 1247_fsm 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:09 1247_vm 2023-07-21T19:42:09.506 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 393216 Jul 21 19:26 1249 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:09 1249_fsm 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:09 1249_vm 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 606208 Jul 21 19:07 1255 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1255_fsm 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1255_vm 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 90112 Jul 21 19:41 1259 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1259_fsm 2023-07-21T19:42:09.507 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:09 1259_vm 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1417_vm 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 1418_vm 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 745472 Jul 21 19:41 16387 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:41 16387_fsm 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:41 16387_vm 2023-07-21T19:42:09.508 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Jul 21 19:41 16393 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:41 16393_fsm 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:41 16393_vm 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396.1 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396.2 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396.3 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396.4 2023-07-21T19:42:09.509 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16396.5 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 304439296 Jul 21 19:41 16396.6 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1679360 Jul 21 19:41 16396_fsm 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 212992 Jul 21 19:41 16396_vm 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:41 16397 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 237568 Jul 21 19:41 16399 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1073741824 Jul 21 19:41 16401 2023-07-21T19:42:09.510 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49373184 Jul 21 19:41 16401.1 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 12967936 Jul 21 19:41 16403 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:41 16403_fsm 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 174 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 175 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2187 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2224_vm 2023-07-21T19:42:09.511 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2328_vm 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2336_vm 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2337 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2600 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2600_fsm 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2600_vm 2023-07-21T19:42:09.512 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2601_fsm 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2601_vm 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2602 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2602_fsm 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2602_vm 2023-07-21T19:42:09.513 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2603 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2603_fsm 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2603_vm 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2604_vm 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2605 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2605_fsm 2023-07-21T19:42:09.514 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2605_vm 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:24 2606 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2606_fsm 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:24 2606_vm 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2607_fsm 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2607_vm 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 450560 Jul 21 19:24 2608 2023-07-21T19:42:09.515 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:24 2608_fsm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:24 2608_vm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 286720 Jul 21 19:07 2609 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2609_fsm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2609_vm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:26 2610 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:24 2610_fsm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:24 2610_vm 2023-07-21T19:42:09.516 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2611_vm 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2612_fsm 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2612_vm 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2613_vm 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615 2023-07-21T19:42:09.517 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2615_fsm 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2615_vm 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2616_fsm 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2616_vm 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 122880 Jul 21 19:07 2617 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2617_fsm 2023-07-21T19:42:09.518 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2617_vm 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 98304 Jul 21 19:07 2618 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2618_fsm 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2618_vm 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 172032 Jul 21 19:41 2619 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:41 2619_fsm 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:24 2619_vm 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620 2023-07-21T19:42:09.519 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2620_vm 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2650 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2651 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2652 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2653 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2654 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2655 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2656 2023-07-21T19:42:09.520 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2657 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 114688 Jul 21 19:24 2658 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:24 2659 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2660 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2661 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:26 2662 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:26 2663 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2664 2023-07-21T19:42:09.521 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2665 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2666 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2667 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2668 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2669 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2670 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 311296 Jul 21 19:24 2673 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 376832 Jul 21 19:24 2674 2023-07-21T19:42:09.522 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 180224 Jul 21 19:07 2675 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2678 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:24 2679 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2680 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2681 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2682 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2683 2023-07-21T19:42:09.523 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2684 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2685 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2686 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2687 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:07 2688 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 49152 Jul 21 19:07 2689 2023-07-21T19:42:09.524 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 81920 Jul 21 19:07 2690 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 270336 Jul 21 19:07 2691 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2692 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2693 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:41 2696 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2699 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2701 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2702 2023-07-21T19:42:09.525 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:09 2703 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:09 2704 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2753 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2753_fsm 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2753_vm 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2754 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2755 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2756 2023-07-21T19:42:09.526 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 2757 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2830_vm 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2831 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2832_vm 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2833 2023-07-21T19:42:09.527 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2834_vm 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2835 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2836_vm 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2837 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 417792 Jul 21 19:07 2838 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2838_fsm 2023-07-21T19:42:09.528 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2838_vm 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2839 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2840_fsm 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2840_vm 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2841 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2995_vm 2023-07-21T19:42:09.529 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2996 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3079_fsm 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3079_vm 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3080 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3081 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3085 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118 2023-07-21T19:42:09.530 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3118_vm 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3119 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3164 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3256_vm 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3257 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3258 2023-07-21T19:42:09.531 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3350_vm 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3351 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3379 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3380 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3381_vm 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3394 2023-07-21T19:42:09.532 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3394_fsm 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3394_vm 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3395 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3439_vm 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3440 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 40960 Jul 21 19:26 3455 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3456 2023-07-21T19:42:09.533 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3456_fsm 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3456_vm 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3466_vm 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3467 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3468 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3501_vm 2023-07-21T19:42:09.534 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3502 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3503 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3534 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3541_fsm 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3541_vm 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3542 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3574 2023-07-21T19:42:09.535 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3575 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3576_vm 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3596_vm 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3597 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598 2023-07-21T19:42:09.536 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3598_vm 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3599 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3600_fsm 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3600_vm 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3601_fsm 2023-07-21T19:42:09.537 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3601_vm 2023-07-21T19:42:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602 2023-07-21T19:42:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3602_fsm 2023-07-21T19:42:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3602_vm 2023-07-21T19:42:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3603 2023-07-21T19:42:09.538 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3603_fsm 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3603_vm 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3604 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3605 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3606 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3607 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3608 2023-07-21T19:42:09.539 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 32768 Jul 21 19:07 3609 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3712 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 3764_fsm 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3764_vm 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3766 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 3767 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3997 2023-07-21T19:42:09.540 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 5002 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 548 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 549 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6102_vm 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6104_vm 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106 2023-07-21T19:42:09.541 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6106_vm 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6110 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6111 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6112 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6113 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6117 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 826_vm 2023-07-21T19:42:09.542 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 827 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 828 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 137860 Jul 21 19:08 pg_internal.init 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 3 Jul 21 19:07 PG_VERSION 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/base/pgsql_tmp: 2023-07-21T19:42:09.543 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:09.952 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:26 . 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 6 postgres postgres system_u:object_r:postgresql_db_t:s0 7908507789 Jul 21 19:24 .. 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/global: 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout:total 575 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 587404 Jul 21 19:08 . 2023-07-21T19:42:09.953 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1136 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1136_fsm 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1136_vm 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1137 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1213 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1213_fsm 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1213_vm 2023-07-21T19:42:09.954 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1214 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1214_fsm 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1214_vm 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1232 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 1233 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1260 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1260_fsm 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1260_vm 2023-07-21T19:42:09.955 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1261 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1261_fsm 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1261_vm 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1262 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 1262_fsm 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 1262_vm 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2396 2023-07-21T19:42:09.956 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 24576 Jul 21 19:07 2396_fsm 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2396_vm 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2397 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2671 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2672 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2676 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2677 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2694 2023-07-21T19:42:09.957 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2695 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2697 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 2698 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2846 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2846_vm 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2847 2023-07-21T19:42:09.958 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2964 2023-07-21T19:42:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2964_vm 2023-07-21T19:42:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2965 2023-07-21T19:42:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2966 2023-07-21T19:42:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 2966_vm 2023-07-21T19:42:10.312 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 2967 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3592 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 3592_vm 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 3593 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 4060 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 4060_vm 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 4061 2023-07-21T19:42:10.313 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6000 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6000_vm 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6001 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6002 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6100 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 6100_vm 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6114 2023-07-21T19:42:10.314 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 6115 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:41 pg_control 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 512 Jul 21 19:07 pg_filenode.map 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 21644 Jul 21 19:08 pg_internal.init 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/log: 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:42:10.315 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 805 Jul 21 19:08 . 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 805 Jul 21 19:42 postgresql-Fri.log 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_commit_ts: 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.316 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_dynshmem: 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical: 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:total 3 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:41 . 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 mappings 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:41 replorigin_checkpoint 2023-07-21T19:42:10.318 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 snapshots 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/mappings: 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:41 .. 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.319 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_logical/snapshots: 2023-07-21T19:42:10.320 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.320 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.320 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 8 Jul 21 19:41 .. 2023-07-21T19:42:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.320 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact: 2023-07-21T19:42:10.729 INFO:teuthology.orchestra.run.smithi005.stdout:total 2 2023-07-21T19:42:10.730 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 . 2023-07-21T19:42:10.730 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.730 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 members 2023-07-21T19:42:10.730 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 offsets 2023-07-21T19:42:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/members: 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 .. 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 0000 2023-07-21T19:42:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.732 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_multixact/offsets: 2023-07-21T19:42:10.732 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:42:10.732 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:07 . 2023-07-21T19:42:10.733 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 4 postgres postgres system_u:object_r:postgresql_db_t:s0 16384 Jul 21 19:07 .. 2023-07-21T19:42:10.733 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:09 0000 2023-07-21T19:42:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.734 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_notify: 2023-07-21T19:42:10.734 INFO:teuthology.orchestra.run.smithi005.stdout:total 9 2023-07-21T19:42:10.734 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:08 . 2023-07-21T19:42:10.734 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.735 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 8192 Jul 21 19:08 0000 2023-07-21T19:42:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.735 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_replslot: 2023-07-21T19:42:10.735 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.736 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.736 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.736 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_serial: 2023-07-21T19:42:10.737 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.737 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.737 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.737 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_snapshots: 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat: 2023-07-21T19:42:10.738 INFO:teuthology.orchestra.run.smithi005.stdout:total 14 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 11771 Jul 21 19:42 . 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 1864 Jul 21 19:42 db_0.stat 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 9300 Jul 21 19:42 db_12347.stat 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 607 Jul 21 19:42 global.stat 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.739 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_stat_tmp: 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:42 . 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_subtrans: 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:total 193 2023-07-21T19:42:10.740 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jul 21 19:38 . 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 196608 Jul 21 19:41 0003 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_tblspc: 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_twophase: 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal: 2023-07-21T19:42:10.742 INFO:teuthology.orchestra.run.smithi005.stdout:total 1032194 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1056964608 Jul 21 19:42 . 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 00000001000000020000003E 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 00000001000000020000003F 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000040 2023-07-21T19:42:10.743 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000041 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000042 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000043 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000044 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000045 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000046 2023-07-21T19:42:10.744 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 000000010000000200000047 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 000000010000000200000048 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 000000010000000200000049 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004A 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004B 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004C 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004D 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004E 2023-07-21T19:42:10.745 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:40 00000001000000020000004F 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:41 000000010000000200000050 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:41 000000010000000200000051 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:41 000000010000000200000052 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:41 000000010000000200000053 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:41 000000010000000200000054 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000055 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 000000010000000200000056 2023-07-21T19:42:10.746 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000057 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:34 000000010000000200000058 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000059 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 00000001000000020000005A 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:34 00000001000000020000005B 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 00000001000000020000005C 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 00000001000000020000005D 2023-07-21T19:42:10.747 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:34 00000001000000020000005E 2023-07-21T19:42:10.748 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 00000001000000020000005F 2023-07-21T19:42:10.748 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 000000010000000200000060 2023-07-21T19:42:10.748 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:35 000000010000000200000061 2023-07-21T19:42:10.748 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000062 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:34 000000010000000200000063 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000064 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 000000010000000200000065 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000066 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000067 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000068 2023-07-21T19:42:10.749 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000069 2023-07-21T19:42:10.750 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 00000001000000020000006A 2023-07-21T19:42:10.750 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 00000001000000020000006B 2023-07-21T19:42:10.750 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 00000001000000020000006C 2023-07-21T19:42:10.750 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 00000001000000020000006D 2023-07-21T19:42:10.750 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 00000001000000020000006E 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 00000001000000020000006F 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 000000010000000200000070 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000071 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 000000010000000200000072 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000073 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 000000010000000200000074 2023-07-21T19:42:10.751 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 000000010000000200000075 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:39 000000010000000200000076 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000077 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 000000010000000200000078 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:36 000000010000000200000079 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:37 00000001000000020000007A 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 00000001000000020000007B 2023-07-21T19:42:10.752 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 16777216 Jul 21 19:38 00000001000000020000007C 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 archive_status 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_wal/archive_status: 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:total 1 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 0 Jul 21 19:07 . 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 3 postgres postgres system_u:object_r:postgresql_db_t:s0 1056964608 Jul 21 19:42 .. 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:42:10.753 INFO:teuthology.orchestra.run.smithi005.stdout:/tmp/cephfs/postgres/data/pg_xact: 2023-07-21T19:42:10.754 INFO:teuthology.orchestra.run.smithi005.stdout:total 65 2023-07-21T19:42:10.754 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 2 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jul 21 19:07 . 2023-07-21T19:42:10.754 INFO:teuthology.orchestra.run.smithi005.stdout:drwx------. 20 postgres postgres system_u:object_r:postgresql_db_t:s0 8966388099 Jul 21 19:42 .. 2023-07-21T19:42:10.754 INFO:teuthology.orchestra.run.smithi005.stdout:-rw-------. 1 postgres postgres system_u:object_r:postgresql_db_t:s0 65536 Jul 21 19:41 0000 2023-07-21T19:42:10.755 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-07-21T19:42:10.771 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:42:10.783 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-07-21T19:42:10.824 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-07-21T19:42:10.824 INFO:teuthology.task.exec:Executing custom commands... 2023-07-21T19:42:10.825 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:42:10.825 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-07-21T19:42:10.905 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-07-21T19:42:10.997 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-07-21T19:42:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2728445524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4105b10 msgr2=0x7fc4b4105ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.197 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2728445524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4105b10 0x7fc4b4105ee0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4009990 tx=0x7fc4a4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2728445524 shutdown_connections 2023-07-21T19:42:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2728445524 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 0x7fc4b410c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2728445524 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b4105b10 0x7fc4b4105ee0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2728445524 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b40ffb00 0x7fc4b40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.198 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2728445524 >> 172.21.15.5:0/2728445524 conn(0x7fc4b40fb600 msgr2=0x7fc4b40fda10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2728445524 shutdown_connections 2023-07-21T19:42:11.199 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.213+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2728445524 wait complete. 2023-07-21T19:42:11.200 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.214+0000 7fc4ba5ff700 1 Processor -- start 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 -- start start 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 0x7fc4b41982c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b4105b10 0x7fc4b419c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b419cf70 con 0x7fc4b40ffb00 2023-07-21T19:42:11.201 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b419d0e0 con 0x7fc4b41004b0 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4ba5ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b419d2a0 con 0x7fc4b4105b10 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4b3fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36476/0 (socket says 172.21.15.5:36476) 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4b3fff700 1 -- 172.21.15.5:0/2547672375 learned_addr learned my addr 172.21.15.5:0/2547672375 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4b37fe700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 0x7fc4b41982c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.215+0000 7fc4b8b9c700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b4105b10 0x7fc4b419c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.202 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 -- 172.21.15.5:0/2547672375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b4105b10 msgr2=0x7fc4b419c930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b4105b10 0x7fc4b419c930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 -- 172.21.15.5:0/2547672375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 msgr2=0x7fc4b41982c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 0x7fc4b41982c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 -- 172.21.15.5:0/2547672375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a40095f0 con 0x7fc4b40ffb00 2023-07-21T19:42:11.203 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b3fff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005740 tx=0x7fc4a4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a401d070 con 0x7fc4b40ffb00 2023-07-21T19:42:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fc4a4022470 con 0x7fc4b40ffb00 2023-07-21T19:42:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4017650 con 0x7fc4b40ffb00 2023-07-21T19:42:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b419d520 con 0x7fc4b40ffb00 2023-07-21T19:42:11.204 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.216+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b419d7a0 con 0x7fc4b40ffb00 2023-07-21T19:42:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.217+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b40ff240 con 0x7fc4b40ffb00 2023-07-21T19:42:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.219+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fc4a400fb00 con 0x7fc4b40ffb00 2023-07-21T19:42:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.221+0000 7fc4b17fa700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 0x7fc49c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.205 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.221+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 8557+0+0 (secure 0 0 0) 0x7fc4a40ce160 con 0x7fc4b40ffb00 2023-07-21T19:42:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.221+0000 7fc4b37fe700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 0x7fc49c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.222+0000 7fc4b37fe700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 0x7fc49c0a37f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc4b41995a0 tx=0x7fc4a80073d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:11.208 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.225+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fc4a4097580 con 0x7fc4b40ffb00 2023-07-21T19:42:11.447 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.463+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4b419d7a0 con 0x7fc49c0a1340 2023-07-21T19:42:11.456 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.472+0000 7fc4b17fa700 1 -- 172.21.15.5:0/2547672375 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+336 (secure 0 0 0) 0x7fc4b419d7a0 con 0x7fc49c0a1340 2023-07-21T19:42:11.456 INFO:teuthology.orchestra.run.smithi005.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-07-21T00:00:00", "created": "2023-07-21T19:07:37", "first": "2023-07-21T19:08:00", "last": "2023-07-21T19:42:00", "last_pruned": "2023-07-21T19:42:00", "created_count": 35, "pruned_count": 29, "active": true} 2023-07-21T19:42:11.459 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 msgr2=0x7fc49c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 0x7fc49c0a37f0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc4b41995a0 tx=0x7fc4a80073d0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 msgr2=0x7fc4b4197d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc4a4005740 tx=0x7fc4a4005770 comp rx=0 tx=0).stop 2023-07-21T19:42:11.460 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 shutdown_connections 2023-07-21T19:42:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fc4b4105b10 0x7fc4b419c930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fc49c0a1340 0x7fc49c0a37f0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fc4b40ffb00 0x7fc4b4197d80 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.461 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 --2- 172.21.15.5:0/2547672375 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fc4b41004b0 0x7fc4b41982c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 >> 172.21.15.5:0/2547672375 conn(0x7fc4b40fb600 msgr2=0x7fc4b410ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 shutdown_connections 2023-07-21T19:42:11.462 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.475+0000 7fc4ba5ff700 1 -- 172.21.15.5:0/2547672375 wait complete. 2023-07-21T19:42:11.477 DEBUG:teuthology.orchestra.run.smithi005:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-07-21T19:42:11.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 -- 172.21.15.5:0/3567263760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 msgr2=0x7f7cbc100190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/3567263760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc100190 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7f7cac0099c0 tx=0x7f7cac009cd0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 -- 172.21.15.5:0/3567263760 shutdown_connections 2023-07-21T19:42:11.677 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/3567263760 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc104e00 0x7f7cbc10c9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/3567263760 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc100190 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/3567263760 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc100760 0x7f7cbc104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 -- 172.21.15.5:0/3567263760 >> 172.21.15.5:0/3567263760 conn(0x7f7cbc0fb650 msgr2=0x7f7cbc0fda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 -- 172.21.15.5:0/3567263760 shutdown_connections 2023-07-21T19:42:11.678 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.693+0000 7f7cc09ff700 1 -- 172.21.15.5:0/3567263760 wait complete. 2023-07-21T19:42:11.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.694+0000 7f7cc09ff700 1 Processor -- start 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 -- start start 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 0x7f7cbc19c0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 0x7f7cbc195a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc196070 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1961e0 con 0x7f7cbc100760 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cc09ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cbc1963a0 con 0x7f7cbc104e00 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.695+0000 7f7cb3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 0x7f7cbc19c0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:36498/0 (socket says 172.21.15.5:36498) 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 -- 172.21.15.5:0/1168744107 learned_addr learned my addr 172.21.15.5:0/1168744107 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:11.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cbad9d700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 0x7f7cbc195a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 -- 172.21.15.5:0/1168744107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 msgr2=0x7f7cbc195a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 0x7f7cbc195a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 -- 172.21.15.5:0/1168744107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 msgr2=0x7f7cbc19c0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 0x7f7cbc19c0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 -- 172.21.15.5:0/1168744107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cac0095f0 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cb3fff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 0x7f7cbc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:11.683 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cbad9d700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 0x7f7cbc195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cba59c700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005910 tx=0x7f7cac00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac01d070 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f7cac0054e0 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cbc196620 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.696+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cac004a70 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.697+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cbc1a2b90 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.698+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cbc0ff290 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.700+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f7cac00fb80 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.701+0000 7f7cb37fe700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 0x7f7ca80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:11.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.701+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 8557+0+0 (secure 0 0 0) 0x7f7cac0cdfd0 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.702+0000 7f7cb3fff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 0x7f7ca80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:11.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.702+0000 7f7cb3fff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 0x7f7ca80a3770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0fbb20 tx=0x7f7ca400b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:11.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.703+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7cac0973f0 con 0x7f7cbc0ffdc0 2023-07-21T19:42:11.929 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.945+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 --> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7cbc197190 con 0x7f7ca80a12c0 2023-07-21T19:42:11.938 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.954+0000 7f7cb37fe700 1 -- 172.21.15.5:0/1168744107 <== mgr.25261 v2:172.21.15.5:6800/1124791839 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f7cbc197190 con 0x7f7ca80a12c0 2023-07-21T19:42:11.938 INFO:teuthology.orchestra.run.smithi005.stdout:/ 1M 6M3h 2023-07-21T19:42:11.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.956+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 msgr2=0x7f7ca80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.956+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 0x7f7ca80a3770 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc0fbb20 tx=0x7f7ca400b410 comp rx=0 tx=0).stop 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.956+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 msgr2=0x7f7cbc19bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.956+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f7cac005910 tx=0x7f7cac00f690 comp rx=0 tx=0).stop 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 shutdown_connections 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7cbc104e00 0x7f7cbc195a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f7ca80a12c0 0x7f7ca80a3770 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7cbc0ffdc0 0x7f7cbc19bba0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 --2- 172.21.15.5:0/1168744107 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7cbc100760 0x7f7cbc19c0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 >> 172.21.15.5:0/1168744107 conn(0x7f7cbc0fb650 msgr2=0x7f7cbc0fc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 shutdown_connections 2023-07-21T19:42:11.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:11.957+0000 7f7cc09ff700 1 -- 172.21.15.5:0/1168744107 wait complete. 2023-07-21T19:42:11.958 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2023-07-21T19:42:11.982 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2023-07-21T19:42:12.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:11 smithi101 ceph-mon[128136]: pgmap v1098: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:12.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:11 smithi005 ceph-mon[126460]: pgmap v1098: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:12.150 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:11 smithi156 ceph-mon[128316]: pgmap v1098: 97 pgs: 40 active+clean+snaptrim_wait, 24 active+clean+snaptrim, 33 active+clean; 11 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 171 KiB/s rd, 221 KiB/s wr, 45 op/s 2023-07-21T19:42:13.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:12 smithi101 ceph-mon[128136]: from='client.21015 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:13.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:12 smithi101 ceph-mon[128136]: from='client.21021 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:13.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:12 smithi005 ceph-mon[126460]: from='client.21015 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:13.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:12 smithi005 ceph-mon[126460]: from='client.21021 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:13.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:12 smithi156 ceph-mon[128316]: from='client.21015 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:13.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:12 smithi156 ceph-mon[128316]: from='client.21021 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-07-21T19:42:14.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:13 smithi101 ceph-mon[128136]: pgmap v1099: 97 pgs: 12 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 77 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 129 KiB/s wr, 26 op/s 2023-07-21T19:42:14.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:13 smithi005 ceph-mon[126460]: pgmap v1099: 97 pgs: 12 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 77 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 129 KiB/s wr, 26 op/s 2023-07-21T19:42:14.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:13 smithi156 ceph-mon[128316]: pgmap v1099: 97 pgs: 12 active+clean+snaptrim_wait, 8 active+clean+snaptrim, 77 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 124 KiB/s rd, 129 KiB/s wr, 26 op/s 2023-07-21T19:42:16.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:15 smithi101 ceph-mon[128136]: pgmap v1100: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:16.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:15 smithi005 ceph-mon[126460]: pgmap v1100: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:16.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:15 smithi156 ceph-mon[128316]: pgmap v1100: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:18.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:17 smithi101 ceph-mon[128136]: pgmap v1101: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:18.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:17 smithi005 ceph-mon[126460]: pgmap v1101: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:18.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:17 smithi156 ceph-mon[128316]: pgmap v1101: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 133 KiB/s rd, 136 KiB/s wr, 28 op/s 2023-07-21T19:42:19.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:19 smithi101 ceph-mon[128136]: osdmap e147: 12 total, 12 up, 12 in 2023-07-21T19:42:19.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:19 smithi101 ceph-mon[128136]: pgmap v1103: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:19.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:19 smithi005 ceph-mon[126460]: osdmap e147: 12 total, 12 up, 12 in 2023-07-21T19:42:19.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:19 smithi005 ceph-mon[126460]: pgmap v1103: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:19.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:19 smithi156 ceph-mon[128316]: osdmap e147: 12 total, 12 up, 12 in 2023-07-21T19:42:19.404 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:19 smithi156 ceph-mon[128316]: pgmap v1103: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:20.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:20 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:20.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:20 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:20.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:20 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:21.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:21 smithi101 ceph-mon[128136]: pgmap v1104: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:21.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:21 smithi005 ceph-mon[126460]: pgmap v1104: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:21.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:21 smithi156 ceph-mon[128316]: pgmap v1104: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 77 KiB/s rd, 72 KiB/s wr, 13 op/s 2023-07-21T19:42:23.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:23 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:23.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:23 smithi101 ceph-mon[128136]: pgmap v1105: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-07-21T19:42:23.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:23 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:23.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:23 smithi005 ceph-mon[126460]: pgmap v1105: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-07-21T19:42:23.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:23 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:23.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:23 smithi156 ceph-mon[128316]: pgmap v1105: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 154 KiB/s rd, 154 KiB/s wr, 12 op/s 2023-07-21T19:42:25.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:25 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:25.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:25 smithi101 ceph-mon[128136]: pgmap v1106: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:25.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:25 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:25.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:25 smithi005 ceph-mon[126460]: pgmap v1106: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:25.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:25 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:25.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:25 smithi156 ceph-mon[128316]: pgmap v1106: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:27.814 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:27 smithi101 ceph-mon[128136]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:27.814 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:27 smithi101 ceph-mon[128136]: pgmap v1107: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:27.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:27 smithi005 ceph-mon[126460]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:27.892 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:27 smithi005 ceph-mon[126460]: pgmap v1107: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:27.903 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:27 smithi156 ceph-mon[128316]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 3 up:standby-replay 6 up:standby 2023-07-21T19:42:27.904 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:27 smithi156 ceph-mon[128316]: pgmap v1107: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 147 KiB/s rd, 147 KiB/s wr, 11 op/s 2023-07-21T19:42:29.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:29 smithi101 ceph-mon[128136]: pgmap v1108: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-07-21T19:42:29.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:29 smithi005 ceph-mon[126460]: pgmap v1108: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-07-21T19:42:29.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:29 smithi156 ceph-mon[128316]: pgmap v1108: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 123 KiB/s rd, 121 KiB/s wr, 11 op/s 2023-07-21T19:42:32.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:31 smithi101 ceph-mon[128136]: pgmap v1109: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-07-21T19:42:32.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:31 smithi005 ceph-mon[126460]: pgmap v1109: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-07-21T19:42:32.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:31 smithi156 ceph-mon[128316]: pgmap v1109: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 109 KiB/s rd, 108 KiB/s wr, 9 op/s 2023-07-21T19:42:34.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:33 smithi101 ceph-mon[128136]: pgmap v1110: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-07-21T19:42:34.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:33 smithi005 ceph-mon[126460]: pgmap v1110: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-07-21T19:42:34.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:33 smithi156 ceph-mon[128316]: pgmap v1110: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 112 KiB/s rd, 110 KiB/s wr, 12 op/s 2023-07-21T19:42:36.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:35 smithi101 ceph-mon[128136]: pgmap v1111: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2023-07-21T19:42:36.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:35 smithi005 ceph-mon[126460]: pgmap v1111: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2023-07-21T19:42:36.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:35 smithi156 ceph-mon[128316]: pgmap v1111: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 8 op/s 2023-07-21T19:42:36.409 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:42:36.647 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 -- 172.21.15.5:0/1990647222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d8105ae0 msgr2=0x7f11d8105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 --2- 172.21.15.5:0/1990647222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d8105ae0 0x7f11d8105eb0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f11cc009990 tx=0x7f11cc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 -- 172.21.15.5:0/1990647222 shutdown_connections 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 --2- 172.21.15.5:0/1990647222 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 0x7f11d810c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 --2- 172.21.15.5:0/1990647222 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d8105ae0 0x7f11d8105eb0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 --2- 172.21.15.5:0/1990647222 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d80ffad0 0x7f11d80fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.648 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 -- 172.21.15.5:0/1990647222 >> 172.21.15.5:0/1990647222 conn(0x7f11d80fb5f0 msgr2=0x7f11d80fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 -- 172.21.15.5:0/1990647222 shutdown_connections 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.661+0000 7f11defff700 1 -- 172.21.15.5:0/1990647222 wait complete. 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.662+0000 7f11defff700 1 Processor -- start 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.662+0000 7f11defff700 1 -- start start 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 0x7f11d8072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 0x7f11d81a3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.649 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d8073580 con 0x7f11d80ffad0 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81a4240 con 0x7f11d8100480 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11defff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d81a4380 con 0x7f11d8105ae0 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32856/0 (socket says 172.21.15.5:32856) 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 -- 172.21.15.5:0/2720261736 learned_addr learned my addr 172.21.15.5:0/2720261736 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 0x7f11d8072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 -- 172.21.15.5:0/2720261736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 msgr2=0x7f11d81a3d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.650 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 0x7f11d81a3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 0x7f11d81a3d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 -- 172.21.15.5:0/2720261736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 msgr2=0x7f11d8072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 0x7f11d8072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dcd9b700 1 -- 172.21.15.5:0/2720261736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d0009c40 con 0x7f11d80ffad0 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.663+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 0x7f11d81a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.651 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11d5ffb700 1 -- 172.21.15.5:0/2720261736 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc01d070 con 0x7f11d80ffad0 2023-07-21T19:42:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11cc0095f0 con 0x7f11d80ffad0 2023-07-21T19:42:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11d5ffb700 1 -- 172.21.15.5:0/2720261736 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11cc022470 con 0x7f11d80ffad0 2023-07-21T19:42:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11d5ffb700 1 -- 172.21.15.5:0/2720261736 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11cc017650 con 0x7f11d80ffad0 2023-07-21T19:42:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.664+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11d81a4810 con 0x7f11d80ffad0 2023-07-21T19:42:36.652 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.669+0000 7f11d5ffb700 1 -- 172.21.15.5:0/2720261736 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f11cc0225e0 con 0x7f11d80ffad0 2023-07-21T19:42:36.653 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 Processor -- start 2023-07-21T19:42:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 -- start start 2023-07-21T19:42:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11d5ffb700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 0x7f11c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.654 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42d7ad0 0x5565b42d9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b41d2cb0 con 0x5565b42cc2d0 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b4225a40 con 0x5565b42ccbe0 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b421eed0 con 0x5565b42d7ad0 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11d5ffb700 1 -- 172.21.15.5:0/2720261736 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f11cc0ce1f0 con 0x7f11d80ffad0 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.655 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 0x7f11c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:57500/0 (socket says 172.21.15.5:57500) 2023-07-21T19:42:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.670+0000 7f11d7fff700 1 -- 172.21.15.5:0/2528771713 learned_addr learned my addr 172.21.15.5:0/2528771713 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.656 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42d7ad0 0x5565b42d9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 -- 172.21.15.5:0/2528771713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42d7ad0 msgr2=0x5565b42d9eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.658 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42d7ad0 0x5565b42d9eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 -- 172.21.15.5:0/2528771713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 msgr2=0x5565b42cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 -- 172.21.15.5:0/2528771713 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b404cde0 con 0x5565b42ccbe0 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 0x7f11c80a3770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11d8074920 tx=0x7f11d0007ba0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.671+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bd30 tx=0x7f11c40106b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edc12611105379ed server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11c2ffd700 1 -- 172.21.15.5:0/2528771713 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4010e10 con 0x5565b42ccbe0 2023-07-21T19:42:36.659 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11c2ffd700 1 -- 172.21.15.5:0/2528771713 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11c4015070 con 0x5565b42ccbe0 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11c2ffd700 1 -- 172.21.15.5:0/2528771713 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c4003cc0 con 0x5565b42ccbe0 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2528771713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 msgr2=0x5565b42d7590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f11c400bd30 tx=0x7f11c40106b0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2528771713 shutdown_connections 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42d7ad0 0x5565b42d9eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2528771713 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42d7590 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2528771713 >> 172.21.15.5:0/2528771713 conn(0x5565b42c2da0 msgr2=0x5565b42c86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2528771713 shutdown_connections 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.672+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2528771713 wait complete. 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.673+0000 7f11f28e8b80 1 -- start start 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.673+0000 7f11f28e8b80 1 Processor -- start 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42cc2d0 0x5565b42d4860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 0x5565b42d52e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b41beac0 con 0x5565b42ccbe0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b41be7c0 con 0x5565b42d7ad0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b40a2920 con 0x5565b42cc2d0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11f28e8b80 10 client.0 _ll_register_callbacks cb 0x5565b41bc760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11dd59c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 0x5565b42d52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32874/0 (socket says 172.21.15.5:32874) 2023-07-21T19:42:36.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 -- 172.21.15.5:0/262653420 learned_addr learned my addr 172.21.15.5:0/262653420 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42cc2d0 0x5565b42d4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 -- 172.21.15.5:0/262653420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42cc2d0 msgr2=0x5565b42d4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42cc2d0 0x5565b42d4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 -- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 msgr2=0x5565b42d52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 0x5565b42d52e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11d7fff700 1 -- 172.21.15.5:0/262653420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11c400b960 con 0x5565b42ccbe0 2023-07-21T19:42:36.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.674+0000 7f11dd59c700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 0x5565b42d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:42:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11d7fff700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4010eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7282488544c704cb server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:42:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c401f040 con 0x5565b42ccbe0 2023-07-21T19:42:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5565b41784f0 con 0x5565b42ccbe0 2023-07-21T19:42:36.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f11c4017070 con 0x5565b42ccbe0 2023-07-21T19:42:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11c4004510 con 0x5565b42ccbe0 2023-07-21T19:42:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11f28e8b80 10 client.21033 fetch_fsmap learned FSMap version 78 2023-07-21T19:42:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.675+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5565b4035c60 con 0x5565b42ccbe0 2023-07-21T19:42:36.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11c4031020 con 0x5565b42ccbe0 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 78) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f11c40116a0 con 0x5565b42ccbe0 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 10 client.21033 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 10 client.21033 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 20 client.21033 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42e74a0 0x5565b42e98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 4 client.21033 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:42:36.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5565b4217eb0 con 0x5565b42e74a0 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.676+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42e74a0 0x5565b42e98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.677+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42e74a0 0x5565b42e98b0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6620cc1e4df00bac server_cookie=7a4e57a77181d29e in_seq=0 out_seq=0 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.677+0000 7f11b7fff700 10 client.21033 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.677+0000 7f11b7fff700 1 -- 172.21.15.5:0/262653420 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5565b4217eb0 con 0x5565b42e74a0 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.677+0000 7f11b7fff700 10 client.21033 handle_command_reply: tid=0 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.677+0000 7f11f28e8b80 1 client.21033 shutdown 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 msgr2=0x5565b42d4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f11c4000c00 tx=0x7f11c4010eb0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 shutdown_connections 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42d7ad0 0x5565b42d52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42cc2d0 0x5565b42d4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42e74a0 0x5565b42e98b0 crc :-1 s=READY pgs=604 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/262653420 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42ccbe0 0x5565b42d4da0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 >> 172.21.15.5:0/262653420 conn(0x5565b42c2da0 msgr2=0x5565b42c8290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11b7fff700 0 client.21033 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 shutdown_connections 2023-07-21T19:42:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 1 -- 172.21.15.5:0/262653420 wait complete. 2023-07-21T19:42:36.670 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.678+0000 7f11f28e8b80 20 client.21033 trim_cache size 0 max 16384 2023-07-21T19:42:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.695+0000 7f11f28e8b80 1 Processor -- start 2023-07-21T19:42:36.679 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.695+0000 7f11f28e8b80 1 -- start start 2023-07-21T19:42:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42c2da0 0x5565b42d3af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42ccbe0 0x5565b42d1af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.680 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42e4b20 con 0x5565b42cc2d0 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42cd540 con 0x5565b42c2da0 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42cddd0 con 0x5565b42ccbe0 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11dcd9b700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42c2da0 0x5565b42d3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32886/0 (socket says 172.21.15.5:32886) 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 -- 172.21.15.5:0/1001270430 learned_addr learned my addr 172.21.15.5:0/1001270430 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:36.681 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 -- 172.21.15.5:0/1001270430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42ccbe0 msgr2=0x5565b42d1af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42ccbe0 0x5565b42d1af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 -- 172.21.15.5:0/1001270430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42c2da0 msgr2=0x5565b42d3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42c2da0 0x5565b42d3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.696+0000 7f11d7fff700 1 -- 172.21.15.5:0/1001270430 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5565b4073250 con 0x5565b42cc2d0 2023-07-21T19:42:36.682 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11d7fff700 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011ab0 tx=0x7f11c4005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf1bf1352afe48 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.684 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11c37fe700 1 -- 172.21.15.5:0/1001270430 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c401f070 con 0x5565b42cc2d0 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11c37fe700 1 -- 172.21.15.5:0/1001270430 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11c4031030 con 0x5565b42cc2d0 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11c37fe700 1 -- 172.21.15.5:0/1001270430 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c40055e0 con 0x5565b42cc2d0 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11f28e8b80 1 -- 172.21.15.5:0/1001270430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 msgr2=0x5565b42de4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.697+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f11c4011ab0 tx=0x7f11c4005ce0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 -- 172.21.15.5:0/1001270430 shutdown_connections 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42ccbe0 0x5565b42d1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.685 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42c2da0 0x5565b42d3af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/1001270430 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42de4a0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 -- 172.21.15.5:0/1001270430 >> 172.21.15.5:0/1001270430 conn(0x5565b42c4600 msgr2=0x5565b42cfb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 -- 172.21.15.5:0/1001270430 shutdown_connections 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 -- 172.21.15.5:0/1001270430 wait complete. 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 -- start start 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.698+0000 7f11f28e8b80 1 Processor -- start 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42c2da0 0x5565b42ddfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.686 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42dc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42dcf10 con 0x5565b42cc2d0 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42dd080 con 0x5565b42ccbe0 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x5565b42dd240 con 0x5565b42c2da0 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11f28e8b80 10 client.0 _ll_register_callbacks cb 0x7f11cc0092b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:32892/0 (socket says 172.21.15.5:32892) 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 -- 172.21.15.5:0/2285054813 learned_addr learned my addr 172.21.15.5:0/2285054813 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11dd59c700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42dc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42c2da0 0x5565b42ddfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 -- 172.21.15.5:0/2285054813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42c2da0 msgr2=0x5565b42ddfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42c2da0 0x5565b42ddfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 -- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 msgr2=0x5565b42dc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42dc9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.699+0000 7f11d7fff700 1 -- 172.21.15.5:0/2285054813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5565b4035c60 con 0x5565b42cc2d0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11d7fff700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b7c0 tx=0x7f11c4010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9c46bc335fa86f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11b7fff700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11c401f070 con 0x5565b42cc2d0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f11c4015070 con 0x5565b42cc2d0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f11c4000e40 con 0x5565b42cc2d0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f11c4011850 con 0x5565b42cc2d0 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11f28e8b80 10 client.21039 fetch_fsmap learned FSMap version 78 2023-07-21T19:42:36.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.700+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x5565b404b2b0 con 0x5565b42cc2d0 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f11c4015070 con 0x5565b42cc2d0 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 78) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f11c4040710 con 0x5565b42cc2d0 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 10 client.21039 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 10 client.21039 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 20 client.21039 populate_metadata read hostname 'smithi005' 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42d7ad0 0x5565b433e310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:42:36.690 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 4 client.21039 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:42:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x5565b41da050 con 0x5565b42d7ad0 2023-07-21T19:42:36.691 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.701+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42d7ad0 0x5565b433e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11dcd9b700 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42d7ad0 0x5565b433e310 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=851ef4875216181d server_cookie=963ec9ff8f868803 in_seq=0 out_seq=0 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11b7fff700 10 client.21039 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11b7fff700 1 -- 172.21.15.5:0/2285054813 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x5565b41da050 con 0x5565b42d7ad0 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11b7fff700 10 client.21039 handle_command_reply: tid=0 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 client.21039 shutdown 2023-07-21T19:42:36.692 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 msgr2=0x5565b42dc490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f11c400b7c0 tx=0x7f11c4010a00 comp rx=0 tx=0).stop 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 shutdown_connections 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x5565b42ccbe0 0x5565b42dc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x5565b42c2da0 0x5565b42ddfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x5565b42d7ad0 0x5565b433e310 crc :-1 s=READY pgs=605 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 --2- 172.21.15.5:0/2285054813 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x5565b42cc2d0 0x5565b42dc490 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.693 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 >> 172.21.15.5:0/2285054813 conn(0x5565b42c4600 msgr2=0x5565b42d4800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11b7fff700 0 client.21039 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 shutdown_connections 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 1 -- 172.21.15.5:0/2285054813 wait complete. 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.702+0000 7f11f28e8b80 20 client.21039 trim_cache size 0 max 16384 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:42:36.694 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:42:36.695 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:42:36.696 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:42:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 msgr2=0x7f11c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.696 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 0x7f11c80a3770 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f11d8074920 tx=0x7f11d0007ba0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 msgr2=0x7f11d8072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7f11cc000c00 tx=0x7f11cc004730 comp rx=0 tx=0).stop 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 shutdown_connections 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f11d8105ae0 0x7f11d81a3d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f11c80a12c0 0x7f11c80a3770 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f11d80ffad0 0x7f11d8072810 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 --2- 172.21.15.5:0/2720261736 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f11d8100480 0x7f11d8072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:42:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 >> 172.21.15.5:0/2720261736 conn(0x7f11d80fb5f0 msgr2=0x7f11d80fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:42:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 shutdown_connections 2023-07-21T19:42:36.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:42:36.703+0000 7f11defff700 1 -- 172.21.15.5:0/2720261736 wait complete. 2023-07-21T19:42:36.704 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:42:38.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:37 smithi101 ceph-mon[128136]: pgmap v1112: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2023-07-21T19:42:38.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:37 smithi005 ceph-mon[126460]: pgmap v1112: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2023-07-21T19:42:38.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:37 smithi156 ceph-mon[128316]: pgmap v1112: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 4.8 KiB/s wr, 6 op/s 2023-07-21T19:42:39.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:39 smithi101 ceph-mon[128136]: pgmap v1113: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-07-21T19:42:39.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:39 smithi005 ceph-mon[126460]: pgmap v1113: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-07-21T19:42:39.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:39 smithi156 ceph-mon[128316]: pgmap v1113: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 6.4 KiB/s wr, 9 op/s 2023-07-21T19:42:42.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:41 smithi101 ceph-mon[128136]: pgmap v1114: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-07-21T19:42:42.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:41 smithi005 ceph-mon[126460]: pgmap v1114: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-07-21T19:42:42.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:41 smithi156 ceph-mon[128316]: pgmap v1114: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-07-21T19:42:44.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:43 smithi101 ceph-mon[128136]: pgmap v1115: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-07-21T19:42:44.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:43 smithi005 ceph-mon[126460]: pgmap v1115: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-07-21T19:42:44.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:43 smithi156 ceph-mon[128316]: pgmap v1115: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2023-07-21T19:42:46.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:45 smithi101 ceph-mon[128136]: pgmap v1116: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:46.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:45 smithi005 ceph-mon[126460]: pgmap v1116: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:46.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:45 smithi156 ceph-mon[128316]: pgmap v1116: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:48.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:47 smithi101 ceph-mon[128136]: pgmap v1117: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:48.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:47 smithi005 ceph-mon[126460]: pgmap v1117: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:48.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:47 smithi156 ceph-mon[128316]: pgmap v1117: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-07-21T19:42:49.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:49 smithi101 ceph-mon[128136]: pgmap v1118: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-21T19:42:49.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:49 smithi005 ceph-mon[126460]: pgmap v1118: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-21T19:42:49.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:49 smithi156 ceph-mon[128316]: pgmap v1118: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.7 KiB/s wr, 9 op/s 2023-07-21T19:42:52.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:51 smithi101 ceph-mon[128136]: pgmap v1119: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T19:42:52.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:51 smithi005 ceph-mon[126460]: pgmap v1119: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T19:42:52.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:51 smithi156 ceph-mon[128316]: pgmap v1119: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 170 B/s wr, 6 op/s 2023-07-21T19:42:54.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:53 smithi101 ceph-mon[128136]: pgmap v1120: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-21T19:42:54.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:53 smithi005 ceph-mon[126460]: pgmap v1120: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-21T19:42:54.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:53 smithi156 ceph-mon[128316]: pgmap v1120: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 170 B/s wr, 8 op/s 2023-07-21T19:42:56.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:55 smithi101 ceph-mon[128136]: pgmap v1121: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-21T19:42:56.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:55 smithi005 ceph-mon[126460]: pgmap v1121: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-21T19:42:56.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:55 smithi156 ceph-mon[128316]: pgmap v1121: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 6 op/s 2023-07-21T19:42:58.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:57 smithi101 ceph-mon[128136]: pgmap v1122: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:42:58.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:57 smithi005 ceph-mon[126460]: pgmap v1122: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:42:58.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:57 smithi156 ceph-mon[128316]: pgmap v1122: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:42:59.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:42:59 smithi101 ceph-mon[128136]: pgmap v1123: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:42:59.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:42:59 smithi005 ceph-mon[126460]: pgmap v1123: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:42:59.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:42:59 smithi156 ceph-mon[128316]: pgmap v1123: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:00.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:00 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:43:00.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:00 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:43:00.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:00 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:43:01.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:01 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:43:01.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:01 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:43:01.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:01 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:43:01.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:01 smithi101 ceph-mon[128136]: pgmap v1124: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:01.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:01 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:43:01.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:01 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:43:01.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:01 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:43:01.392 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:01 smithi005 ceph-mon[126460]: pgmap v1124: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:01.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:01 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:43:01.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:01 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:43:01.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:01 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:43:01.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:01 smithi156 ceph-mon[128316]: pgmap v1124: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:04.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:03 smithi101 ceph-mon[128136]: pgmap v1125: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:43:04.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:03 smithi005 ceph-mon[126460]: pgmap v1125: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:43:04.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:03 smithi156 ceph-mon[128316]: pgmap v1125: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:43:06.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:05 smithi101 ceph-mon[128136]: pgmap v1126: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:06.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:05 smithi005 ceph-mon[126460]: pgmap v1126: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:06.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:05 smithi156 ceph-mon[128316]: pgmap v1126: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:06.708 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 scrub status 2023-07-21T19:43:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.946+0000 7f09529ff700 1 -- 172.21.15.5:0/4096791341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c100480 msgr2=0x7f094c10c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.946+0000 7f09529ff700 1 --2- 172.21.15.5:0/4096791341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c100480 0x7f094c10c910 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f094400b0a0 tx=0x7f094400b3b0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 -- 172.21.15.5:0/4096791341 shutdown_connections 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 --2- 172.21.15.5:0/4096791341 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c100480 0x7f094c10c910 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 --2- 172.21.15.5:0/4096791341 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c105ae0 0x7f094c105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 --2- 172.21.15.5:0/4096791341 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c0ffad0 0x7f094c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 -- 172.21.15.5:0/4096791341 >> 172.21.15.5:0/4096791341 conn(0x7f094c0fb5f0 msgr2=0x7f094c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 -- 172.21.15.5:0/4096791341 shutdown_connections 2023-07-21T19:43:06.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.947+0000 7f09529ff700 1 -- 172.21.15.5:0/4096791341 wait complete. 2023-07-21T19:43:06.933 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 Processor -- start 2023-07-21T19:43:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 -- start start 2023-07-21T19:43:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 0x7f094c072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 0x7f094c073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.934 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c1a3fc0 con 0x7f094c0ffad0 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c1a4130 con 0x7f094c100480 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f09529ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f094c1a42f0 con 0x7f094c105ae0 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f094bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f0943fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 0x7f094c072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f094bfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57112/0 (socket says 172.21.15.5:57112) 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.948+0000 7f094bfff700 1 -- 172.21.15.5:0/859776009 learned_addr learned my addr 172.21.15.5:0/859776009 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:06.935 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0950f9c700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 0x7f094c073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 -- 172.21.15.5:0/859776009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 msgr2=0x7f094c073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 0x7f094c073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 -- 172.21.15.5:0/859776009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 msgr2=0x7f094c072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 0x7f094c072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 -- 172.21.15.5:0/859776009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944009d00 con 0x7f094c0ffad0 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0950f9c700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 0x7f094c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:43:06.936 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0943fff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 0x7f094c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f094bfff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e9a0 tx=0x7f093c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0949ffb700 1 -- 172.21.15.5:0/859776009 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c009790 con 0x7f094c0ffad0 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0949ffb700 1 -- 172.21.15.5:0/859776009 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f093c00cbd0 con 0x7f094c0ffad0 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f0949ffb700 1 -- 172.21.15.5:0/859776009 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093c018680 con 0x7f094c0ffad0 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.949+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f094c1a45d0 con 0x7f094c0ffad0 2023-07-21T19:43:06.937 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.950+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f094c1a4a90 con 0x7f094c0ffad0 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.953+0000 7f0949ffb700 1 -- 172.21.15.5:0/859776009 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f093c00cd40 con 0x7f094c0ffad0 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f09662e6b80 1 Processor -- start 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f09662e6b80 1 -- start start 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d922846a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f0949ffb700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 0x7f09380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.939 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.954+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92291eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f09662e6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9218acb0 con 0x555d922842d0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f09662e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d921dda40 con 0x555d92284be0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f09662e6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d921d6ed0 con 0x555d9228fad0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0949ffb700 1 -- 172.21.15.5:0/859776009 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f093c014070 con 0x7f094c0ffad0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d922846a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.940 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50974/0 (socket says 172.21.15.5:50974) 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 -- 172.21.15.5:0/3572748383 learned_addr learned my addr 172.21.15.5:0/3572748383 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f094bfff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 0x7f09380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0950f9c700 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92291eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 -- 172.21.15.5:0/3572748383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 msgr2=0x555d92291eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92291eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 -- 172.21.15.5:0/3572748383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 msgr2=0x555d922846a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.941 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d922846a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0943fff700 1 -- 172.21.15.5:0/3572748383 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d92004de0 con 0x555d92284be0 2023-07-21T19:43:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.955+0000 7f0950f9c700 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d922846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:43:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f094bfff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 0x7f09380a3770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f093c0964f0 tx=0x7f093c0109d0 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.942 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f0943fff700 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0934010580 tx=0x7f0934010890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ae5cd77de933086 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f09427fc700 1 -- 172.21.15.5:0/3572748383 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093400bbf0 con 0x555d92284be0 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f09427fc700 1 -- 172.21.15.5:0/3572748383 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0934015070 con 0x555d92284be0 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f09427fc700 1 -- 172.21.15.5:0/3572748383 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934011a80 con 0x555d92284be0 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f09662e6b80 1 -- 172.21.15.5:0/3572748383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 msgr2=0x555d9228f590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.956+0000 7f09662e6b80 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0934010580 tx=0x7f0934010890 comp rx=0 tx=0).stop 2023-07-21T19:43:06.945 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 -- 172.21.15.5:0/3572748383 shutdown_connections 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92291eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d922846a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 --2- 172.21.15.5:0/3572748383 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d92284be0 0x555d9228f590 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 -- 172.21.15.5:0/3572748383 >> 172.21.15.5:0/3572748383 conn(0x555d9227ada0 msgr2=0x555d922806a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 -- 172.21.15.5:0/3572748383 shutdown_connections 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 -- 172.21.15.5:0/3572748383 wait complete. 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 -- start start 2023-07-21T19:43:06.946 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.957+0000 7f09662e6b80 1 Processor -- start 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d9228cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92176ac0 con 0x555d92284be0 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d921767c0 con 0x555d922842d0 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d9205a920 con 0x555d9228fad0 2023-07-21T19:43:06.947 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f09662e6b80 10 client.0 _ll_register_callbacks cb 0x555d92174760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57138/0 (socket says 172.21.15.5:57138) 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50990/0 (socket says 172.21.15.5:50990) 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0943fff700 1 -- 172.21.15.5:0/256119809 learned_addr learned my addr 172.21.15.5:0/256119809 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:06.948 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.958+0000 7f0950f9c700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d9228cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 -- 172.21.15.5:0/256119809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 msgr2=0x555d9228cdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d9228cdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 -- 172.21.15.5:0/256119809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 msgr2=0x555d9228c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 -- 172.21.15.5:0/256119809 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f093400bd50 con 0x555d922842d0 2023-07-21T19:43:06.949 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0943fff700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f0950f9c700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0944009850 tx=0x7f094400b720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ab1d02bb4ed0dc1 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f092b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.101:3300/0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400e040 con 0x555d922842d0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mon.2 v2:172.21.15.101:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0944004a50 con 0x555d922842d0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d921304f0 con 0x555d922842d0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.959+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0944004c10 con 0x555d922842d0 2023-07-21T19:43:06.950 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 10 client.30824 fetch_fsmap learned FSMap version 78 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x555d91fedc60 con 0x555d922842d0 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mon.2 v2:172.21.15.101:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x555d921304f0 con 0x555d922842d0 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mon.2 v2:172.21.15.101:3300/0 5 ==== fsmap(e 78) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f094400b0d0 con 0x555d922842d0 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 10 client.30824 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 10 client.30824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 20 client.30824 populate_metadata read hostname 'smithi005' 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d9229f350 0x555d922a1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.951 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 4 client.30824 mds_command: new command op to 24614 tid=0[{"prefix": "get_command_descriptions"}] 2023-07-21T19:43:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d921cfeb0 con 0x555d9229f350 2023-07-21T19:43:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.960+0000 7f0943fff700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d9229f350 0x555d922a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.961+0000 7f0943fff700 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d9229f350 0x555d922a1760 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1458e6055f94c6c server_cookie=449ef0b3ab1ebb5f in_seq=0 out_seq=0 2023-07-21T19:43:06.952 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.961+0000 7f092b7fe700 10 client.30824 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:43:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f092b7fe700 1 -- 172.21.15.5:0/256119809 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d921cfeb0 con 0x555d9229f350 2023-07-21T19:43:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f092b7fe700 10 client.30824 handle_command_reply: tid=0 2023-07-21T19:43:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 client.30824 shutdown 2023-07-21T19:43:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 msgr2=0x555d9228f4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.953 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0944009850 tx=0x7f094400b720 comp rx=0 tx=0).stop 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 shutdown_connections 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d9228cdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d9228f4c0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d9229f350 0x555d922a1760 crc :-1 s=READY pgs=606 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.962+0000 7f09662e6b80 1 --2- 172.21.15.5:0/256119809 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d9228c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.963+0000 7f092b7fe700 0 client.30824 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:43:06.954 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.963+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 >> 172.21.15.5:0/256119809 conn(0x555d9227ada0 msgr2=0x555d92280290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.963+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 shutdown_connections 2023-07-21T19:43:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.963+0000 7f09662e6b80 1 -- 172.21.15.5:0/256119809 wait complete. 2023-07-21T19:43:06.955 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.963+0000 7f09662e6b80 20 client.30824 trim_cache size 0 max 16384 2023-07-21T19:43:06.963 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.980+0000 7f09662e6b80 1 Processor -- start 2023-07-21T19:43:06.964 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.980+0000 7f09662e6b80 1 -- start start 2023-07-21T19:43:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 0x555d9228bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d9228fad0 0x555d9228c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92285540 con 0x555d922842d0 2023-07-21T19:43:06.965 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92291eb0 con 0x555d9228fad0 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f09662e6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92281c70 con 0x555d92284be0 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57148/0 (socket says 172.21.15.5:57148) 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 -- 172.21.15.5:0/4293952558 learned_addr learned my addr 172.21.15.5:0/4293952558 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d9228fad0 0x555d9228c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0943fff700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 0x555d9228bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 -- 172.21.15.5:0/4293952558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 msgr2=0x555d9228bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.966 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 0x555d9228bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 -- 172.21.15.5:0/4293952558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d9228fad0 msgr2=0x555d9228c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d9228fad0 0x555d9228c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0950f9c700 1 -- 172.21.15.5:0/4293952558 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d9202b250 con 0x555d922842d0 2023-07-21T19:43:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.981+0000 7f0943fff700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 0x555d9228bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-07-21T19:43:06.967 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f0950f9c700 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f094401ce20 tx=0x7f094400afe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd3a22c4f29bab3c server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.969 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f0942ffd700 1 -- 172.21.15.5:0/4293952558 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094400e070 con 0x555d922842d0 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f0942ffd700 1 -- 172.21.15.5:0/4293952558 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f094400f040 con 0x555d922842d0 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f0942ffd700 1 -- 172.21.15.5:0/4293952558 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094401c7a0 con 0x555d922842d0 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 -- 172.21.15.5:0/4293952558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 msgr2=0x555d9228f060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f094401ce20 tx=0x7f094400afe0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 -- 172.21.15.5:0/4293952558 shutdown_connections 2023-07-21T19:43:06.970 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d9228fad0 0x555d9228c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d922842d0 0x555d9228f060 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 --2- 172.21.15.5:0/4293952558 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d92284be0 0x555d9228bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.982+0000 7f09662e6b80 1 -- 172.21.15.5:0/4293952558 >> 172.21.15.5:0/4293952558 conn(0x555d9227ada0 msgr2=0x555d9229b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.983+0000 7f09662e6b80 1 -- 172.21.15.5:0/4293952558 shutdown_connections 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.983+0000 7f09662e6b80 1 -- 172.21.15.5:0/4293952558 wait complete. 2023-07-21T19:43:06.971 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.983+0000 7f09662e6b80 1 -- start start 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.983+0000 7f09662e6b80 1 Processor -- start 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d92295ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92292480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92292b20 con 0x555d92284be0 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92292c90 con 0x555d922842d0 2023-07-21T19:43:06.972 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d92292e50 con 0x555d9228fad0 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f09662e6b80 10 client.0 _ll_register_callbacks cb 0x7f093c005ec0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d92295ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0950f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d92295ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50996/0 (socket says 172.21.15.5:50996) 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57154/0 (socket says 172.21.15.5:57154) 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0950f9c700 1 -- 172.21.15.5:0/2170060661 learned_addr learned my addr 172.21.15.5:0/2170060661 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0950f9c700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92292480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.973 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 -- 172.21.15.5:0/2170060661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 msgr2=0x555d92292480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92292480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 -- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 msgr2=0x555d92295ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d92295ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.984+0000 7f0943fff700 1 -- 172.21.15.5:0/2170060661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d91fedc60 con 0x555d92284be0 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f0943fff700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f09340059d0 tx=0x7f0934010d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74c488e8d2e08e5f server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f092b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.5:3300/0 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f093402e070 con 0x555d92284be0 2023-07-21T19:43:06.974 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f0934015070 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mon.0 v2:172.21.15.5:3300/0 2 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f0934017070 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_get_version_reply(handle=1 version=78) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0934020040 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f09662e6b80 10 client.21057 fetch_fsmap learned FSMap version 78 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.985+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,fsmap=78,monmap=4+}) v3 -- 0x555d920032b0 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mon.0 v2:172.21.15.5:3300/0 4 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f0934036020 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mon.0 v2:172.21.15.5:3300/0 5 ==== fsmap(e 78) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f09340290f0 con 0x555d92284be0 2023-07-21T19:43:06.975 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 10 client.21057 fetch_fsmap finished waiting for FSMap version 78 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 10 client.21057 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 20 client.21057 populate_metadata read hostname 'smithi005' 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d922f41c0 0x555d922f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 4 client.21057 mds_command: new command op to 24614 tid=0[{"prefix": "scrub status"}] 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 --> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555d92192050 con 0x555d922f41c0 2023-07-21T19:43:06.976 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.986+0000 7f0950f9c700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d922f41c0 0x555d922f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f0950f9c700 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d922f41c0 0x555d922f65a0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdef932fff95f27c server_cookie=a41491bf7489d55a in_seq=0 out_seq=0 2023-07-21T19:43:06.977 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f092b7fe700 10 client.21057 ms_handle_connect on v2:172.21.15.101:6838/715074805 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f092b7fe700 1 -- 172.21.15.5:0/2170060661 <== mds.0 v2:172.21.15.101:6838/715074805 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+259 (crc 0 0 0) 0x555d92192050 con 0x555d922f41c0 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f092b7fe700 10 client.21057 handle_command_reply: tid=0 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 client.21057 shutdown 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 msgr2=0x555d92296410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f09340059d0 tx=0x7f0934010d40 comp rx=0 tx=0).stop 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 shutdown_connections 2023-07-21T19:43:06.978 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x555d9228fad0 0x555d92292480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x555d922842d0 0x555d92295ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.101:6838/715074805,v1:172.21.15.101:6839/715074805] conn(0x555d922f41c0 0x555d922f65a0 crc :-1 s=READY pgs=607 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 --2- 172.21.15.5:0/2170060661 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x555d92284be0 0x555d92296410 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f092b7fe700 0 client.21057 ms_handle_reset on v2:172.21.15.101:6838/715074805 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 >> 172.21.15.5:0/2170060661 conn(0x555d9227ada0 msgr2=0x555d922861a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 shutdown_connections 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 1 -- 172.21.15.5:0/2170060661 wait complete. 2023-07-21T19:43:06.979 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.987+0000 7f09662e6b80 20 client.21057 trim_cache size 0 max 16384 2023-07-21T19:43:06.980 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:06.980 INFO:teuthology.orchestra.run.smithi005.stdout: "status": "IDLE (waiting for more scrubs)", 2023-07-21T19:43:06.980 INFO:teuthology.orchestra.run.smithi005.stdout: "scrubs": { 2023-07-21T19:43:06.980 INFO:teuthology.orchestra.run.smithi005.stdout: "5aba8d84-89b5-45d7-8716-e870e6956b63": { 2023-07-21T19:43:06.980 INFO:teuthology.orchestra.run.smithi005.stdout: "path": "/", 2023-07-21T19:43:06.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tag": "5aba8d84-89b5-45d7-8716-e870e6956b63", 2023-07-21T19:43:06.981 INFO:teuthology.orchestra.run.smithi005.stdout: "options": "recursive,force" 2023-07-21T19:43:06.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:06.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:06.981 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 msgr2=0x7f09380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 0x7f09380a3770 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f093c0964f0 tx=0x7f093c0109d0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 msgr2=0x7f094c072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f093c00e9a0 tx=0x7f093c00ecb0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 shutdown_connections 2023-07-21T19:43:06.982 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f094c105ae0 0x7f094c073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f09380a12c0 0x7f09380a3770 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f094c0ffad0 0x7f094c072810 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 --2- 172.21.15.5:0/859776009 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f094c100480 0x7f094c072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 >> 172.21.15.5:0/859776009 conn(0x7f094c0fb5f0 msgr2=0x7f094c0fbbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 shutdown_connections 2023-07-21T19:43:06.983 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:06.988+0000 7f09529ff700 1 -- 172.21.15.5:0/859776009 wait complete. 2023-07-21T19:43:06.989 INFO:tasks.cephfs.filesystem:scrub status for tag:5aba8d84-89b5-45d7-8716-e870e6956b63 - {'path': '/', 'tag': '5aba8d84-89b5-45d7-8716-e870e6956b63', 'options': 'recursive,force'} 2023-07-21T19:43:06.990 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 38, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 55, in do_scrub self._scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 76, in _scrub done = self.fs.wait_until_scrub_complete(tag=tag, sleep=30, timeout=self.scrub_timeout) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephfs/filesystem.py", line 1684, in wait_until_scrub_complete while proceed(): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/contextutil.py", line 134, in __call__ raise MaxWhileTries(error_msg) teuthology.exceptions.MaxWhileTries: reached maximum tries (31) after waiting for 900 seconds 2023-07-21T19:43:06.990 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-21T19:43:06.990 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-07-21T19:43:07.001 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:43:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 -- 172.21.15.5:0/1062920974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914073160 msgr2=0x7f791410bed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 --2- 172.21.15.5:0/1062920974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914073160 0x7f791410bed0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f7908009990 tx=0x7f7908009ca0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 -- 172.21.15.5:0/1062920974 shutdown_connections 2023-07-21T19:43:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 --2- 172.21.15.5:0/1062920974 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f791410c410 0x7f791410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.206 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 --2- 172.21.15.5:0/1062920974 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f7914072850 0x7f7914072c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 --2- 172.21.15.5:0/1062920974 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914073160 0x7f791410bed0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 -- 172.21.15.5:0/1062920974 >> 172.21.15.5:0/1062920974 conn(0x7f79140fb690 msgr2=0x7f79140fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 -- 172.21.15.5:0/1062920974 shutdown_connections 2023-07-21T19:43:07.207 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.222+0000 7f791a1ff700 1 -- 172.21.15.5:0/1062920974 wait complete. 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.223+0000 7f791a1ff700 1 Processor -- start 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.223+0000 7f791a1ff700 1 -- start start 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7914073160 0x7f7914198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f791410c410 0x7f791419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791419cd20 con 0x7f7914072850 2023-07-21T19:43:07.209 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791419ce90 con 0x7f791410c410 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f791a1ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f791419d050 con 0x7f7914073160 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57178/0 (socket says 172.21.15.5:57178) 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 -- 172.21.15.5:0/534942319 learned_addr learned my addr 172.21.15.5:0/534942319 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f7913fff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f791410c410 0x7f791419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f7912ffd700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7914073160 0x7f7914198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.210 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 -- 172.21.15.5:0/534942319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7914073160 msgr2=0x7f7914198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7914073160 0x7f7914198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 -- 172.21.15.5:0/534942319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f791410c410 msgr2=0x7f791419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f791410c410 0x7f791419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.224+0000 7f79137fe700 1 -- 172.21.15.5:0/534942319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79080095f0 con 0x7f7914072850 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f79137fe700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9a0 tx=0x7f790400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904009790 con 0x7f7914072850 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f791419d330 con 0x7f7914072850 2023-07-21T19:43:07.211 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f790400cbd0 con 0x7f7914072850 2023-07-21T19:43:07.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7904018680 con 0x7f7914072850 2023-07-21T19:43:07.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.225+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79141a4b70 con 0x7f7914072850 2023-07-21T19:43:07.212 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.226+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7914067750 con 0x7f7914072850 2023-07-21T19:43:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.228+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f790400cd40 con 0x7f7914072850 2023-07-21T19:43:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.229+0000 7f7910ff9700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 0x7f78fc0a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.213 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.230+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f7904014070 con 0x7f7914072850 2023-07-21T19:43:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.230+0000 7f7912ffd700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 0x7f78fc0a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.214 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.231+0000 7f7912ffd700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 0x7f78fc0a37b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f790800b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:07.217 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.234+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f7904051790 con 0x7f7914072850 2023-07-21T19:43:07.428 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.444+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f79141a4e00 con 0x7f7914072850 2023-07-21T19:43:07.429 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.446+0000 7f7910ff9700 1 -- 172.21.15.5:0/534942319 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v147) v1 ==== 74+0+22992 (secure 0 0 0) 0x7f79040963d0 con 0x7f7914072850 2023-07-21T19:43:07.430 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:43:07.430 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":147,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:42:18.013742+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"146","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":65,"snap_epoch":146,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3558708925":"2023-07-22T19:07:36.084989+0000","172.21.15.5:6801/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/2496466053":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4257470084":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/1985166852":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6800/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":52,"length":1},{"begin":65,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:43:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 msgr2=0x7f78fc0a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.432 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 0x7f78fc0a37b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f7908006010 tx=0x7f790800b260 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 msgr2=0x7f7914197d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f790400e9a0 tx=0x7f790400ecb0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 shutdown_connections 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f791410c410 0x7f791419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f78fc0a1300 0x7f78fc0a37b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f7914072850 0x7f7914197d40 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 --2- 172.21.15.5:0/534942319 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f7914073160 0x7f7914198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.433 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 >> 172.21.15.5:0/534942319 conn(0x7f79140fb690 msgr2=0x7f791410e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 shutdown_connections 2023-07-21T19:43:07.434 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.449+0000 7f791a1ff700 1 -- 172.21.15.5:0/534942319 wait complete. 2023-07-21T19:43:07.452 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '146', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 65, 'snap_epoch': 146, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:43:07.453 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 -- 172.21.15.5:0/3411063997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 msgr2=0x7f8b90105ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 --2- 172.21.15.5:0/3411063997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b90105ec0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0098f0 tx=0x7f8b8c009c00 comp rx=0 tx=0).stop 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 -- 172.21.15.5:0/3411063997 shutdown_connections 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 --2- 172.21.15.5:0/3411063997 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9010c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 --2- 172.21.15.5:0/3411063997 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b90105ec0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.660 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 --2- 172.21.15.5:0/3411063997 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b900fff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 -- 172.21.15.5:0/3411063997 >> 172.21.15.5:0/3411063997 conn(0x7f8b900fb5e0 msgr2=0x7f8b900fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 -- 172.21.15.5:0/3411063997 shutdown_connections 2023-07-21T19:43:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.676+0000 7f8b977ff700 1 -- 172.21.15.5:0/3411063997 wait complete. 2023-07-21T19:43:07.661 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.678+0000 7f8b977ff700 1 Processor -- start 2023-07-21T19:43:07.662 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.678+0000 7f8b977ff700 1 -- start start 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b9019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b901958d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90195f40 con 0x7f8b90100490 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b901960b0 con 0x7f8b90105af0 2023-07-21T19:43:07.663 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b977ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b90196270 con 0x7f8b900ffae0 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57186/0 (socket says 172.21.15.5:57186) 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 -- 172.21.15.5:0/315643643 learned_addr learned my addr 172.21.15.5:0/315643643 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b95d9c700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b901958d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b9559b700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b9019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 -- 172.21.15.5:0/315643643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 msgr2=0x7f8b9019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.664 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b9019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 -- 172.21.15.5:0/315643643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 msgr2=0x7f8b901958d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b901958d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b94d9a700 1 -- 172.21.15.5:0/315643643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b88009d00 con 0x7f8b90100490 2023-07-21T19:43:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.679+0000 7f8b9559b700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b9019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:43:07.665 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b94d9a700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c9b0 tx=0x7f8b8000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80007b00 con 0x7f8b90100490 2023-07-21T19:43:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b8c009520 con 0x7f8b90100490 2023-07-21T19:43:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8b80007c60 con 0x7f8b90100490 2023-07-21T19:43:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80018660 con 0x7f8b90100490 2023-07-21T19:43:07.666 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.680+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901a2670 con 0x7f8b90100490 2023-07-21T19:43:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.681+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b90067290 con 0x7f8b90100490 2023-07-21T19:43:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.683+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8b8000f450 con 0x7f8b90100490 2023-07-21T19:43:07.667 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.684+0000 7f8b867fc700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 0x7f8b7c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.684+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8b800ccff0 con 0x7f8b90100490 2023-07-21T19:43:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.685+0000 7f8b9559b700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 0x7f8b7c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:07.668 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.685+0000 7f8b9559b700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 0x7f8b7c0a3770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009fd0 tx=0x7f8b8c019040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:07.669 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.686+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8b80096470 con 0x7f8b90100490 2023-07-21T19:43:07.879 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.896+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b901a2a30 con 0x7f8b90100490 2023-07-21T19:43:07.889 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.906+0000 7f8b867fc700 1 -- 172.21.15.5:0/315643643 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v147) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b8000fe60 con 0x7f8b90100490 2023-07-21T19:43:07.889 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 msgr2=0x7f8b7c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 0x7f8b7c0a3770 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c009fd0 tx=0x7f8b8c019040 comp rx=0 tx=0).stop 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 msgr2=0x7f8b9019bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f8b8000c9b0 tx=0x7f8b8000ccc0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 shutdown_connections 2023-07-21T19:43:07.892 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8b90105af0 0x7f8b901958d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8b900ffae0 0x7f8b9019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8b7c0a12c0 0x7f8b7c0a3770 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 --2- 172.21.15.5:0/315643643 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8b90100490 0x7f8b9019bda0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 >> 172.21.15.5:0/315643643 conn(0x7f8b900fb5e0 msgr2=0x7f8b900fbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.908+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 shutdown_connections 2023-07-21T19:43:07.893 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:07.909+0000 7f8b977ff700 1 -- 172.21.15.5:0/315643643 wait complete. 2023-07-21T19:43:07.911 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:43:08.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:07 smithi101 ceph-mon[128136]: pgmap v1127: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:08.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/534942319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:43:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 -- 172.21.15.5:0/2041812096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc0ffae0 msgr2=0x7feacc0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.119 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 --2- 172.21.15.5:0/2041812096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc0ffae0 0x7feacc0fff50 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7feac0009990 tx=0x7feac0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 -- 172.21.15.5:0/2041812096 shutdown_connections 2023-07-21T19:43:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 --2- 172.21.15.5:0/2041812096 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc100490 0x7feacc10c920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 --2- 172.21.15.5:0/2041812096 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 0x7feacc105ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.120 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 --2- 172.21.15.5:0/2041812096 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc0ffae0 0x7feacc0fff50 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 -- 172.21.15.5:0/2041812096 >> 172.21.15.5:0/2041812096 conn(0x7feacc0fb5e0 msgr2=0x7feacc0fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 -- 172.21.15.5:0/2041812096 shutdown_connections 2023-07-21T19:43:08.121 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.135+0000 7fead1bff700 1 -- 172.21.15.5:0/2041812096 wait complete. 2023-07-21T19:43:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 Processor -- start 2023-07-21T19:43:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 -- start start 2023-07-21T19:43:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.125 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc100490 0x7feacc19e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 0x7feacc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc1982f0 con 0x7feacc100490 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc198460 con 0x7feacc0ffae0 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.136+0000 7fead1bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feacc198620 con 0x7feacc105af0 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:51052/0 (socket says 172.21.15.5:51052) 2023-07-21T19:43:08.126 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 -- 172.21.15.5:0/2322120468 learned_addr learned my addr 172.21.15.5:0/2322120468 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacaffd700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc100490 0x7feacc19e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacbfff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 0x7feacc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 -- 172.21.15.5:0/2322120468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 msgr2=0x7feacc197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 0x7feacc197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.127 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 -- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc100490 msgr2=0x7feacc19e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc100490 0x7feacc19e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 -- 172.21.15.5:0/2322120468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac00095f0 con 0x7feacc0ffae0 2023-07-21T19:43:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.137+0000 7feacb7fe700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7feabc00d980 tx=0x7feabc00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:08.128 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.138+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc004220 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.138+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feacc198900 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.138+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7feabc007640 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.138+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feabc0055e0 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.138+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feacc1a4b10 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.139+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feacc067290 con 0x7feacc0ffae0 2023-07-21T19:43:08.129 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.141+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7feabc004750 con 0x7feacc0ffae0 2023-07-21T19:43:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.142+0000 7feac8ff9700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 0x7feab40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.142+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7feabc0ccfe0 con 0x7feacc0ffae0 2023-07-21T19:43:08.130 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.143+0000 7feacaffd700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 0x7feab40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.143+0000 7feacaffd700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 0x7feab40a3770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feac0006010 tx=0x7feac0005960 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:08.131 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.147+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7feabc0963d0 con 0x7feacc0ffae0 2023-07-21T19:43:08.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:07 smithi005 ceph-mon[126460]: pgmap v1127: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:08.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/534942319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:43:08.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:07 smithi156 ceph-mon[128316]: pgmap v1127: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:08.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/534942319' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:43:08.339 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.356+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feacc198fe0 con 0x7feacc0ffae0 2023-07-21T19:43:08.340 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.357+0000 7feac8ff9700 1 -- 172.21.15.5:0/2322120468 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v147) v1 ==== 104+0+11 (secure 0 0 0) 0x7feabc0160a0 con 0x7feacc0ffae0 2023-07-21T19:43:08.341 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:43:08.343 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 msgr2=0x7feab40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 0x7feab40a3770 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7feac0006010 tx=0x7feac0005960 comp rx=0 tx=0).stop 2023-07-21T19:43:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 msgr2=0x7feacc19db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.344 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7feabc00d980 tx=0x7feabc00dc90 comp rx=0 tx=0).stop 2023-07-21T19:43:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 shutdown_connections 2023-07-21T19:43:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7feacc105af0 0x7feacc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.345 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7feacc0ffae0 0x7feacc19db40 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7feab40a12c0 0x7feab40a3770 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 --2- 172.21.15.5:0/2322120468 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7feacc100490 0x7feacc19e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 >> 172.21.15.5:0/2322120468 conn(0x7feacc0fb5e0 msgr2=0x7feacc10ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:08.346 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 shutdown_connections 2023-07-21T19:43:08.347 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.359+0000 7fead1bff700 1 -- 172.21.15.5:0/2322120468 wait complete. 2023-07-21T19:43:08.361 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:43:08.568 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 -- 172.21.15.5:0/1631144263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 msgr2=0x7effc0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 --2- 172.21.15.5:0/1631144263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0105eb0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7effbc009990 tx=0x7effbc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 -- 172.21.15.5:0/1631144263 shutdown_connections 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 --2- 172.21.15.5:0/1631144263 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc0100480 0x7effc010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 --2- 172.21.15.5:0/1631144263 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0105eb0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 --2- 172.21.15.5:0/1631144263 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc00ffad0 0x7effc00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.569 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 -- 172.21.15.5:0/1631144263 >> 172.21.15.5:0/1631144263 conn(0x7effc00fb5f0 msgr2=0x7effc00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:08.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 -- 172.21.15.5:0/1631144263 shutdown_connections 2023-07-21T19:43:08.570 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.585+0000 7effc79ff700 1 -- 172.21.15.5:0/1631144263 wait complete. 2023-07-21T19:43:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.586+0000 7effc79ff700 1 Processor -- start 2023-07-21T19:43:08.571 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.586+0000 7effc79ff700 1 -- start start 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.586+0000 7effc79ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc79ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc0100480 0x7effc0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc79ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0073290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc79ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01a3f40 con 0x7effc00ffad0 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc79ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01a40b0 con 0x7effc0105ae0 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc79ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effc01a4270 con 0x7effc0100480 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.572 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57220/0 (socket says 172.21.15.5:57220) 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc5f9c700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0073290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 -- 172.21.15.5:0/121580188 learned_addr learned my addr 172.21.15.5:0/121580188 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc4f9a700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc0100480 0x7effc0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 -- 172.21.15.5:0/121580188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc0100480 msgr2=0x7effc0072d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc0100480 0x7effc0072d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 -- 172.21.15.5:0/121580188 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 msgr2=0x7effc0073290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.573 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0073290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.587+0000 7effc579b700 1 -- 172.21.15.5:0/121580188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb8009c40 con 0x7effc00ffad0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effc579b700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc01d070 con 0x7effc00ffad0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7effbc022470 con 0x7effc00ffad0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effbc0095f0 con 0x7effc00ffad0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effbc017650 con 0x7effc00ffad0 2023-07-21T19:43:08.574 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.588+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effc01a47e0 con 0x7effc00ffad0 2023-07-21T19:43:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.589+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effc0067290 con 0x7effc00ffad0 2023-07-21T19:43:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.591+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7effbc0225e0 con 0x7effc00ffad0 2023-07-21T19:43:08.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.592+0000 7effb67fc700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 0x7effac0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.592+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7effbc0ce140 con 0x7effc00ffad0 2023-07-21T19:43:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.593+0000 7effc4f9a700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 0x7effac0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:08.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.593+0000 7effc4f9a700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 0x7effac0a37f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effc0073fb0 tx=0x7effb000b410 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:08.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.597+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7effbc097580 con 0x7effc00ffad0 2023-07-21T19:43:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.808+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7effc01a4aa0 con 0x7effc00ffad0 2023-07-21T19:43:08.792 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.809+0000 7effb67fc700 1 -- 172.21.15.5:0/121580188 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v147) v1 ==== 100+0+11 (secure 0 0 0) 0x7effbc09aba0 con 0x7effc00ffad0 2023-07-21T19:43:08.793 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 msgr2=0x7effac0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 0x7effac0a37f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7effc0073fb0 tx=0x7effb000b410 comp rx=0 tx=0).stop 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 msgr2=0x7effc0072810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7effbc000c00 tx=0x7effbc004730 comp rx=0 tx=0).stop 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 shutdown_connections 2023-07-21T19:43:08.795 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7effc0105ae0 0x7effc0073290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7effac0a1340 0x7effac0a37f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7effc00ffad0 0x7effc0072810 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 --2- 172.21.15.5:0/121580188 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7effc0100480 0x7effc0072d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 >> 172.21.15.5:0/121580188 conn(0x7effc00fb5f0 msgr2=0x7effc00fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 shutdown_connections 2023-07-21T19:43:08.796 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:08.811+0000 7effc79ff700 1 -- 172.21.15.5:0/121580188 wait complete. 2023-07-21T19:43:08.813 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-07-21T19:43:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 -- 172.21.15.5:0/3365447208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 msgr2=0x7f4740072c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 --2- 172.21.15.5:0/3365447208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740072c00 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f4730009990 tx=0x7f4730009ca0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.018 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 -- 172.21.15.5:0/3365447208 shutdown_connections 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 --2- 172.21.15.5:0/3365447208 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 0x7f474010e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 --2- 172.21.15.5:0/3365447208 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740072c00 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 --2- 172.21.15.5:0/3365447208 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 0x7f474010be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.034+0000 7f4745fff700 1 -- 172.21.15.5:0/3365447208 >> 172.21.15.5:0/3365447208 conn(0x7f47400fb610 msgr2=0x7f47400fda20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.035+0000 7f4745fff700 1 -- 172.21.15.5:0/3365447208 shutdown_connections 2023-07-21T19:43:09.019 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.035+0000 7f4745fff700 1 -- 172.21.15.5:0/3365447208 wait complete. 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.035+0000 7f4745fff700 1 Processor -- start 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 -- start start 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 0x7f4740198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 0x7f474019c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019cd00 con 0x7f4740072830 2023-07-21T19:43:09.021 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019ce70 con 0x7f474010c3d0 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f4745fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f474019d030 con 0x7f4740073140 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f473f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f473f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:57230/0 (socket says 172.21.15.5:57230) 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f473f7fe700 1 -- 172.21.15.5:0/2004219412 learned_addr learned my addr 172.21.15.5:0/2004219412 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f473ffff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 0x7f474019c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.036+0000 7f473effd700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 0x7f4740198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:09.022 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 -- 172.21.15.5:0/2004219412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 msgr2=0x7f4740198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 0x7f4740198260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 -- 172.21.15.5:0/2004219412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 msgr2=0x7f474019c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 0x7f474019c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 -- 172.21.15.5:0/2004219412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47300095f0 con 0x7f4740072830 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473f7fe700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f473001d070 con 0x7f4740072830 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f474019d2b0 con 0x7f4740072830 2023-07-21T19:43:09.023 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.037+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f4730022470 con 0x7f4740072830 2023-07-21T19:43:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.038+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4730017650 con 0x7f4740072830 2023-07-21T19:43:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.038+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47401a4b50 con 0x7f4740072830 2023-07-21T19:43:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.039+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47401075e0 con 0x7f4740072830 2023-07-21T19:43:09.024 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.041+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f47300225e0 con 0x7f4740072830 2023-07-21T19:43:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.042+0000 7f473cff9700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 0x7f47280a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:43:09.025 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.042+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f47300ce120 con 0x7f4740072830 2023-07-21T19:43:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.042+0000 7f473effd700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 0x7f47280a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:43:09.026 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.043+0000 7f473effd700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 0x7f47280a37b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4740199330 tx=0x7f4734009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:43:09.029 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.046+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f4730097560 con 0x7f4740072830 2023-07-21T19:43:09.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/315643643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:08 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2322120468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/315643643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:08 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2322120468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/315643643' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:08 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2322120468' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:43:09.298 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.314+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f4740067300 con 0x7f4740072830 2023-07-21T19:43:09.309 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.326+0000 7f473cff9700 1 -- 172.21.15.5:0/2004219412 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733766 (secure 0 0 0) 0x7f473009ab80 con 0x7f4740072830 2023-07-21T19:43:09.317 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: "epoch": 25, 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: "active_gid": 25261, 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: "active_name": "x", 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addrs": { 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:09.318 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6800", 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1124791839 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v1", 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:6801", 2023-07-21T19:43:09.319 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1124791839 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: "active_addr": "172.21.15.5:6801/1124791839", 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: "active_change": "2023-07-21T19:07:36.085285+0000", 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: "active_mgr_features": 4540138322906710015, 2023-07-21T19:43:09.320 INFO:teuthology.orchestra.run.smithi005.stdout: "available": true, 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "standbys": [ 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25255, 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "y", 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540138322906710015, 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.323 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.324 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.325 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.328 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.330 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.331 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:43:09.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.337 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.338 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.340 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.341 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-07-21T19:43:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-07-21T19:43:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-07-21T19:43:09.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-07-21T19:43:09.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.346 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:43:09.346 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.346 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.347 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.349 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.350 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.351 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-07-21T19:43:09.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-07-21T19:43:09.354 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-07-21T19:43:09.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.355 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.356 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:09.357 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-07-21T19:43:09.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:43:09.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:09.359 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.359 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.359 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-07-21T19:43:09.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.360 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:43:09.361 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.362 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-07-21T19:43:09.363 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.364 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.365 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-07-21T19:43:09.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-07-21T19:43:09.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:09.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.370 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.376 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.376 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-07-21T19:43:09.382 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.383 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.384 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.385 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:09.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.389 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-07-21T19:43:09.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_password": { 2023-07-21T19:43:09.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_password", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web password", 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_user": { 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_user", 2023-07-21T19:43:09.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web user", 2023-07-21T19:43:09.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-07-21T19:43:09.402 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:43:09.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.407 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.408 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-07-21T19:43:09.408 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-07-21T19:43:09.408 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:43:09.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.410 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:43:09.411 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.412 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.413 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.415 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:43:09.416 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.417 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:43:09.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:43:09.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:43:09.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.423 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-07-21T19:43:09.425 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:43:09.427 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.428 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-07-21T19:43:09.429 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.430 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-07-21T19:43:09.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:09.434 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.435 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.436 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.438 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:43:09.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.443 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:43:09.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:43:09.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-07-21T19:43:09.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:43:09.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.453 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.454 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.455 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.456 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.457 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.458 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.459 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.460 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.461 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:43:09.462 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.463 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.464 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.469 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-07-21T19:43:09.470 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.473 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.474 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-07-21T19:43:09.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-07-21T19:43:09.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-07-21T19:43:09.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:43:09.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:43:09.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_password": { 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_password", 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.481 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web password", 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.482 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_user": { 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_user", 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.483 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web user", 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-07-21T19:43:09.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.487 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-07-21T19:43:09.488 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:43:09.489 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.490 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-07-21T19:43:09.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.495 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-07-21T19:43:09.497 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.499 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.500 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.501 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.504 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.506 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.508 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.513 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-07-21T19:43:09.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.520 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.521 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.523 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:43:09.524 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.525 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.527 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.528 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.530 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.531 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.532 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:43:09.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.535 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:43:09.542 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.543 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:43:09.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.545 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.547 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:43:09.551 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.552 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:43:09.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.556 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.557 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:43:09.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:43:09.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.563 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.564 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.565 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.568 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.569 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:43:09.571 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.578 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:43:09.580 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.585 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.586 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.587 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.588 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:43:09.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.590 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.591 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.592 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.593 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.594 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.595 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.596 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-07-21T19:43:09.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-07-21T19:43:09.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.605 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.607 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.608 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.611 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.612 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.613 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.616 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:43:09.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.619 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.620 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.621 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-07-21T19:43:09.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-07-21T19:43:09.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.626 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.627 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-07-21T19:43:09.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:09.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.631 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.633 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.634 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.635 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.636 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.637 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.641 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.642 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.643 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.645 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-07-21T19:43:09.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-07-21T19:43:09.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:43:09.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-07-21T19:43:09.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.657 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.659 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.660 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.661 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.662 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.663 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.664 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.667 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.668 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.669 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.670 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:09.671 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.672 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-07-21T19:43:09.673 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.674 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.675 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:43:09.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.682 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.685 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.686 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.689 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.690 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.691 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.692 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:09.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:43:09.696 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.697 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-07-21T19:43:09.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.700 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.701 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.702 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.704 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.705 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.706 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.707 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.708 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.709 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.710 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.711 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.712 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.714 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.715 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.716 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.717 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.718 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.720 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:43:09.721 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:43:09.722 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.723 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.724 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.725 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.726 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.727 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.730 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.731 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.732 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.733 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.734 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-07-21T19:43:09.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.736 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.738 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.739 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.741 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.742 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.745 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-07-21T19:43:09.747 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.748 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.749 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.750 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.751 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-07-21T19:43:09.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.755 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.761 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.766 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.769 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.771 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.772 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.773 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.774 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.775 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.776 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.784 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.785 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.788 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.789 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.792 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.793 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.794 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.795 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.796 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.799 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.800 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.803 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.805 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.806 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.807 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.808 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.810 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.813 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.818 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.819 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.820 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.821 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.825 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.828 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:43:09.829 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.830 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.831 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:09.832 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.833 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-07-21T19:43:09.834 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:09.836 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.838 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.839 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.840 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.841 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.842 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.843 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.844 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.845 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.846 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.847 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.848 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:09.849 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.850 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-07-21T19:43:09.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.852 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-07-21T19:43:09.853 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:43:09.854 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.855 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.856 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.859 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.860 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.861 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.863 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-07-21T19:43:09.864 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.865 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.866 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.867 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.868 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.869 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.870 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:09.871 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.872 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-07-21T19:43:09.873 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.874 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.876 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:09.877 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.880 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.881 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.882 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.886 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.887 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.888 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.889 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:09.890 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.891 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.892 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-07-21T19:43:09.893 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.894 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.895 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.899 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.900 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.901 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.905 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.906 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.907 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.908 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.909 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.910 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.911 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.912 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.913 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.915 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.916 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.919 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.920 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.921 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.922 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.923 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.924 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.925 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.926 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.927 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.928 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.929 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.931 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.934 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.938 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.940 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.943 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-07-21T19:43:09.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.953 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-07-21T19:43:09.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.962 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.963 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-07-21T19:43:09.964 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.965 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.966 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.967 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.968 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.973 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.974 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.975 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.976 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.977 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.978 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.979 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.981 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.982 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.983 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.988 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.989 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.990 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.991 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:09.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.993 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:09.997 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:09.998 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:09.999 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.000 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.001 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-07-21T19:43:10.002 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.003 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.004 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.005 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.006 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.007 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.008 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.009 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.012 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.013 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.015 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-07-21T19:43:10.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:43:10.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-07-21T19:43:10.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:43:10.019 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-07-21T19:43:10.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-07-21T19:43:10.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:43:10.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-07-21T19:43:10.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:43:10.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.034 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-07-21T19:43:10.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.038 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-07-21T19:43:10.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.041 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.042 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.043 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.044 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.045 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.046 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.047 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.048 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.049 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.050 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.058 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.060 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.061 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.062 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.063 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.064 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.065 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.066 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.067 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.068 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.073 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.074 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.075 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.076 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.077 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.078 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.079 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.080 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.084 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.088 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.089 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.090 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.091 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.092 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.095 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-07-21T19:43:10.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.100 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:10.101 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: "gid": 25258, 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "z", 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: "mgr_features": 4540138322906710015, 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.103 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.104 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.105 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.106 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.109 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.111 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.112 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.113 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-07-21T19:43:10.114 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:43:10.115 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.116 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.117 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.118 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.119 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.120 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.121 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.122 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-07-21T19:43:10.123 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:43:10.124 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.125 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.126 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.127 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.128 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.129 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-07-21T19:43:10.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-07-21T19:43:10.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:43:10.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-07-21T19:43:10.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.140 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-07-21T19:43:10.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:43:10.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.145 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.146 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.147 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.148 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.149 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.150 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.151 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.152 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.153 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.154 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.155 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-07-21T19:43:10.156 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-07-21T19:43:10.157 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:43:10.158 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.159 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.160 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.161 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.162 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.163 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.164 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-07-21T19:43:10.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:43:10.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.170 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_password": { 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_password", 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.171 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web password", 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.172 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_user": { 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_user", 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.173 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web user", 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-07-21T19:43:10.174 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.175 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-07-21T19:43:10.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:43:10.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.179 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.180 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.181 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.182 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.183 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.184 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-07-21T19:43:10.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:43:10.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.192 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:43:10.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:43:10.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:43:10.200 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.202 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-07-21T19:43:10.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-07-21T19:43:10.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.206 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.207 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.208 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.209 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.210 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.211 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.212 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.213 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:43:10.214 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:43:10.215 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.221 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-07-21T19:43:10.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.225 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.226 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.229 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.230 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.231 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.232 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.233 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.241 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-07-21T19:43:10.243 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-07-21T19:43:10.244 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.245 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-07-21T19:43:10.246 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:43:10.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:43:10.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_password": { 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_password", 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web password", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.250 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_user": { 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_user", 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.251 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web user", 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.252 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.253 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-07-21T19:43:10.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-07-21T19:43:10.256 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:43:10.257 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.258 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.259 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.260 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-07-21T19:43:10.263 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.264 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.265 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.266 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.270 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.271 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-07-21T19:43:10.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.274 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.275 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.276 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.277 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.278 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.279 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.281 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.282 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-07-21T19:43:10.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.287 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.288 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.289 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:43:10.290 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.292 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.293 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.294 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.295 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.296 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.297 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.298 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:43:10.299 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.300 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.302 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.303 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.304 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.305 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.306 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.307 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:43:10.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.309 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.310 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.311 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.313 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.314 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:43:10.315 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.316 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:43:10.317 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.321 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.322 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:43:10.324 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.325 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.326 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.327 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.328 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.331 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.332 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:43:10.333 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.334 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.335 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.337 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.338 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.339 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.340 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.341 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:43:10.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.346 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.347 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:43:10.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.354 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.357 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.358 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:43:10.359 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.360 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:43:10.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.364 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.365 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:10.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.368 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.369 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-07-21T19:43:10.370 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.375 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.376 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.377 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.378 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.380 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.381 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.382 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.383 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.385 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.386 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:43:10.388 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.389 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:10.390 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.395 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.396 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:10.397 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.398 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.399 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:10.400 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-07-21T19:43:10.401 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.402 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.404 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.407 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.408 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.409 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.411 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.413 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.414 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-07-21T19:43:10.415 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.416 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.417 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-07-21T19:43:10.418 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.419 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.424 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.425 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.426 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.427 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.429 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.430 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.431 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.432 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.434 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.435 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.436 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.438 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.439 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.440 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.442 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-07-21T19:43:10.443 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-07-21T19:43:10.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:43:10.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-07-21T19:43:10.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.452 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.456 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.459 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.460 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-07-21T19:43:10.461 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.462 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:10.463 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:10.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.467 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.468 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.469 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.470 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.471 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.472 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.473 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.474 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.475 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.475 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.476 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.477 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.478 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.479 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.480 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.482 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.485 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.486 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.487 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.489 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.490 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:43:10.491 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.492 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.493 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.494 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.495 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.496 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.497 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.498 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.500 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.501 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.502 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.504 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-07-21T19:43:10.505 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.506 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.508 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.509 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.510 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.511 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.512 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.515 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-07-21T19:43:10.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:43:10.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-07-21T19:43:10.523 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.524 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.526 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.527 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.528 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.529 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.531 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.532 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.534 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.534 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.534 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.534 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.534 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.535 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.536 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.537 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.538 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.539 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.540 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.541 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.542 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.543 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.544 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.545 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.546 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.547 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.548 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.551 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-07-21T19:43:10.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.556 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.557 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.558 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.559 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.560 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.561 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.562 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.563 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.564 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.565 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-07-21T19:43:10.567 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.568 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.569 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.570 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.571 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.572 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.573 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.574 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.575 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.576 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.577 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.578 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.579 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.580 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.581 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.582 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.583 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.584 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.585 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.589 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.590 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.592 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.593 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.594 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.601 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:43:10.604 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:43:10.605 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.606 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.609 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.612 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.613 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-07-21T19:43:10.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.622 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.623 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:43:10.624 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.626 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.627 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.628 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.631 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.633 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.634 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.635 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.636 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.637 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.638 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:10.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.641 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-07-21T19:43:10.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:10.645 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.647 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:10.648 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.651 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.652 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.653 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.656 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.657 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.658 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-07-21T19:43:10.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:43:10.661 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.662 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.663 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.664 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.665 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.667 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.668 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.669 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.670 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.671 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.672 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.673 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.674 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.675 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.677 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:10.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.679 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:10.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.681 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-07-21T19:43:10.682 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.683 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.684 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.685 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.686 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.687 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.688 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.689 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.690 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.691 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.692 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-07-21T19:43:10.693 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:43:10.694 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.695 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.696 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.697 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.698 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.699 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.700 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.701 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.702 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.703 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-07-21T19:43:10.704 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:43:10.705 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-07-21T19:43:10.706 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.707 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.708 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.709 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.710 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.711 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.712 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.713 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.714 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.715 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.716 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.717 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.718 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.719 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.720 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.721 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.722 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-07-21T19:43:10.723 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.724 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.725 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.726 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.727 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.728 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.729 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.730 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-07-21T19:43:10.731 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-07-21T19:43:10.732 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.733 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.734 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.735 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.736 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.737 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.737 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.737 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.740 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.741 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.742 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-07-21T19:43:10.743 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.744 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.745 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.746 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.747 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.748 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.749 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.750 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.751 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.752 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.753 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.754 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.755 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.756 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.757 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.758 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.759 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.760 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.761 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.762 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.763 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.764 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.765 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.766 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.767 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.768 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.769 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.770 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.771 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.772 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.773 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.774 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-07-21T19:43:10.775 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.776 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.777 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.778 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.779 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.780 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.781 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.782 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.783 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.784 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.785 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.786 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.787 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.788 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.789 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.790 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.791 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.792 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.793 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.794 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.795 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-07-21T19:43:10.796 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.797 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.798 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.799 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.800 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.801 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:43:10.802 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.803 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.804 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.805 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.806 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-07-21T19:43:10.807 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.808 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.809 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.810 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.811 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.812 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.813 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.814 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.815 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.816 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.817 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.818 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.819 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.820 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.821 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.822 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.823 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.824 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-07-21T19:43:10.825 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.826 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.827 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.828 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.829 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.830 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.831 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.832 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.833 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.834 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.835 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.836 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.837 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.838 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.839 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.840 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.841 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.842 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.843 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.844 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.845 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.846 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-07-21T19:43:10.847 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.848 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.849 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.850 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.851 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.852 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.853 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.854 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.855 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.856 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.857 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.858 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.859 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.860 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.861 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.862 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.863 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.864 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.865 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-07-21T19:43:10.866 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.867 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.868 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.869 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.870 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.871 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: "modules": [ 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-07-21T19:43:10.872 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard", 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "iostat", 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "nfs", 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "restful", 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "snap_schedule" 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "available_modules": [ 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.873 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alerts", 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.874 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How frequently to reexamine health status", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.875 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.876 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.877 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.878 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.879 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.880 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.881 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.882 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.883 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.884 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_destination": { 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_destination", 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.885 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email address to send alerts to", 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.886 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_from_name": { 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_from_name", 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "Ceph", 2023-07-21T19:43:10.887 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Email From: name", 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.888 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_host": { 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_host", 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.889 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP server", 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_password": { 2023-07-21T19:43:10.890 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_password", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.891 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Password to authenticate with", 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_port": { 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_port", 2023-07-21T19:43:10.892 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "465", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP port", 2023-07-21T19:43:10.893 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_sender": { 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_sender", 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.894 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SMTP envelope sender", 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.895 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_ssl": { 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_ssl", 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.896 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use SSL to connect to SMTP server", 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.897 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "smtp_user": { 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "smtp_user", 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.898 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "User to authenticate as", 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.899 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "balancer", 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "active": { 2023-07-21T19:43:10.900 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "active", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.901 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically balance PGs across cluster", 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_time": { 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_time", 2023-07-21T19:43:10.902 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0000", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "beginning time of day to automatically balance", 2023-07-21T19:43:10.903 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "begin_weekday": { 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "begin_weekday", 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.904 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:10.905 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_max_iterations": { 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_max_iterations", 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.906 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "25", 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "250", 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.907 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_metrics": { 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_metrics", 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "pgs,objects,bytes", 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.908 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "crush_compat_step": { 2023-07-21T19:43:10.909 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crush_compat_step", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.5", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0.001", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "0.999", 2023-07-21T19:43:10.910 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "aggressiveness of optimization", 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "end_time": { 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_time", 2023-07-21T19:43:10.911 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2359", 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.912 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ending time of day to automatically balance", 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "end_weekday": { 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "end_weekday", 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.913 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "0", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "6", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-07-21T19:43:10.914 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.915 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.916 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.917 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.918 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.919 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.920 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:10.921 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:10.922 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.923 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "min_score": { 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_score", 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.924 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.925 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.926 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "upmap", 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "crush-compat", 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "none", 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap" 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:10.927 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Balancer mode", 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_ids": { 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_ids", 2023-07-21T19:43:10.928 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-07-21T19:43:10.929 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.930 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.931 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_deviation": { 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_deviation", 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.932 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "deviation below which no optimization is attempted", 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.933 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "upmap_max_optimizations": { 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "upmap_max_optimizations", 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.934 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.935 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cephadm", 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_down_multiplier": { 2023-07-21T19:43:10.936 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_down_multiplier", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.937 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_refresh_rate": { 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_refresh_rate", 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.938 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "20", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.939 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "agent_starting_port": { 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "agent_starting_port", 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.940 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4721", 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.941 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_password": { 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_password", 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.942 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web password", 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.943 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "alertmanager_web_user": { 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "alertmanager_web_user", 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.944 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Alertmanager web user", 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_ptrace": { 2023-07-21T19:43:10.945 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_ptrace", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.946 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_interval": { 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_interval", 2023-07-21T19:43:10.947 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to autotune daemon memory", 2023-07-21T19:43:10.948 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "autotune_memory_target_ratio": { 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "autotune_memory_target_ratio", 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:10.949 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0.7", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.950 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "cgroups_split": { 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cgroups_split", 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.951 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.952 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "config_checks_enabled": { 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_checks_enabled", 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.953 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.954 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "config_dashboard": { 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_dashboard", 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.955 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.956 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_alertmanager": { 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_alertmanager", 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-07-21T19:43:10.957 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.958 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_base": { 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_base", 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph", 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.959 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Container image name, without the tag", 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.960 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_elasticsearch": { 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_elasticsearch", 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.961 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "elasticsearch container image", 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.962 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_grafana": { 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_grafana", 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.963 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.964 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_haproxy": { 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_haproxy", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.965 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "HAproxy container image", 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_agent": { 2023-07-21T19:43:10.966 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_agent", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.967 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger agent container image", 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_collector": { 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_collector", 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.968 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger collector container image", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.969 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_jaeger_query": { 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_jaeger_query", 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.970 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Jaeger query container image", 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.971 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_keepalived": { 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_keepalived", 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.972 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Keepalived container image", 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.973 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_loki": { 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_loki", 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-07-21T19:43:10.974 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Loki container image", 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.975 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_node_exporter": { 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_node_exporter", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.976 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_prometheus": { 2023-07-21T19:43:10.977 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_prometheus", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.978 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus container image", 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_promtail": { 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_promtail", 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.979 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Promtail container image", 2023-07-21T19:43:10.980 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "container_image_snmp_gateway": { 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_image_snmp_gateway", 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.981 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "SNMP Gateway container image", 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.982 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "container_init": { 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "container_init", 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.983 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Run podman/docker with `--init`", 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.984 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "daemon_cache_timeout": { 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "daemon_cache_timeout", 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.985 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.986 INFO:teuthology.orchestra.run.smithi005.stdout: "default_cephadm_command_timeout": { 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_cephadm_command_timeout", 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "900", 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.987 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "default_registry": { 2023-07-21T19:43:10.988 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "default_registry", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "docker.io", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.989 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "device_cache_timeout": { 2023-07-21T19:43:10.990 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_cache_timeout", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1800", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.991 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache device inventory", 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "device_enhanced_scan": { 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_enhanced_scan", 2023-07-21T19:43:10.992 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use libstoragemgmt during device scans", 2023-07-21T19:43:10.993 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "facts_cache_timeout": { 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "facts_cache_timeout", 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.994 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "seconds to cache host facts data", 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.995 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "host_check_interval": { 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "host_check_interval", 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:10.996 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to perform a host check", 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:10.997 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:10.998 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:10.999 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.000 INFO:teuthology.orchestra.run.smithi005.stdout: "log_refresh_metadata": { 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_refresh_metadata", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.001 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.002 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.003 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.004 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.005 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.006 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.007 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.008 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf": { 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.009 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.010 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "*", 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.011 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.012 INFO:teuthology.orchestra.run.smithi005.stdout: "max_count_per_host": { 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_count_per_host", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.013 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of daemons per service per host", 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "max_osd_draining_count": { 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_osd_draining_count", 2023-07-21T19:43:11.014 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-07-21T19:43:11.015 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "migration_current": { 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "migration_current", 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.016 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "internal - do not modify", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.017 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "mode": { 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mode", 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.018 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "root", 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm-package", 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "root" 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.019 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "mode for remote execution of cephadm", 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_alerts_path": { 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_alerts_path", 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.020 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-07-21T19:43:11.021 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_password": { 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_password", 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.022 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web password", 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.023 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "prometheus_web_user": { 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus_web_user", 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.024 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Prometheus web user", 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.025 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_insecure": { 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_insecure", 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.026 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.027 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_password": { 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_password", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.028 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_url": { 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_url", 2023-07-21T19:43:11.029 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.030 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "registry_username": { 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "registry_username", 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.031 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.032 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "secure_monitoring_stack": { 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "secure_monitoring_stack", 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.033 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.034 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "service_discovery_port": { 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "service_discovery_port", 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8765", 2023-07-21T19:43:11.035 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "cephadm service discovery port", 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.036 INFO:teuthology.orchestra.run.smithi005.stdout: "ssh_config_file": { 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssh_config_file", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.037 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "use_agent": { 2023-07-21T19:43:11.038 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_agent", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.039 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "use_repo_digest": { 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "use_repo_digest", 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.040 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.041 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_failed_host_check": { 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_failed_host_check", 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.042 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if the host check fails", 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.043 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_daemons": { 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_daemons", 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.044 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.045 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_on_stray_hosts": { 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_on_stray_hosts", 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.046 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.047 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crash", 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.048 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.049 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.050 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.051 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.052 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.053 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.054 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.055 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.056 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.057 INFO:teuthology.orchestra.run.smithi005.stdout: "retain_interval": { 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retain_interval", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "31536000", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.058 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain crashes before pruning them", 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_recent_interval": { 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_recent_interval", 2023-07-21T19:43:11.059 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1209600", 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.060 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "time interval in which to warn about recent crashes", 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.061 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dashboard", 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.062 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.063 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_HOST": { 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_HOST", 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.064 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.065 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.066 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.067 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_ENABLED": { 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_ENABLED", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.068 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-07-21T19:43:11.069 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.070 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "ENABLE_BROWSABLE_API": { 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ENABLE_BROWSABLE_API", 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.071 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.072 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.073 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.074 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.075 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.076 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.077 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.078 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.079 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_NFS": { 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-07-21T19:43:11.080 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.081 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RBD": { 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.082 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.083 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "FEATURE_TOGGLE_RGW": { 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.084 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.085 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.086 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.087 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_PASSWORD": { 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_PASSWORD", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.088 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-07-21T19:43:11.089 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.090 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_URL": { 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_URL", 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.091 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.092 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_API_USERNAME": { 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_API_USERNAME", 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.093 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.094 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.095 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.096 INFO:teuthology.orchestra.run.smithi005.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.097 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-07-21T19:43:11.098 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.099 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "ISSUE_TRACKER_API_KEY": { 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.100 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.101 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_HOST": { 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_HOST", 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.102 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.103 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.104 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.105 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.106 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-07-21T19:43:11.107 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.108 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.109 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.110 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.111 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.112 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.113 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.114 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.115 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-07-21T19:43:11.116 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.117 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_ENABLED": { 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_ENABLED", 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.118 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.119 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.120 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.121 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.122 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.123 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8", 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.124 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "REST_REQUESTS_TIMEOUT": { 2023-07-21T19:43:11.125 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "45", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.126 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ACCESS_KEY": { 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ACCESS_KEY", 2023-07-21T19:43:11.127 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.128 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.129 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "admin", 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.130 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SECRET_KEY": { 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SECRET_KEY", 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.131 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.132 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "RGW_API_SSL_VERIFY": { 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "RGW_API_SSL_VERIFY", 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.133 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-07-21T19:43:11.134 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.135 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-07-21T19:43:11.136 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.137 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.138 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.139 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "cross_origin_url": { 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cross_origin_url", 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.140 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.141 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "crt_file": { 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "crt_file", 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.142 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.143 INFO:teuthology.orchestra.run.smithi005.stdout: "debug": { 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "debug", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.144 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Enable/disable debug options", 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "jwt_token_ttl": { 2023-07-21T19:43:11.145 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "jwt_token_ttl", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "28800", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.146 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.147 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.148 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.149 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.150 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.151 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.152 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.153 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.154 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.155 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.156 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.157 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "motd": { 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "motd", 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.158 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "The message of the day", 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.159 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect_resolve_ip_addr": { 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "redirect_resolve_ip_addr", 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.160 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.161 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.162 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:11.163 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8080", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.164 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.165 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.166 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl_server_port": { 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl_server_port", 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.167 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8443", 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.168 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "redirect", 2023-07-21T19:43:11.169 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "redirect" 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.170 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.171 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.172 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "url_prefix": { 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url_prefix", 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.173 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.174 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.175 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_monitoring": { 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_monitoring", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.176 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "monitor device health metrics", 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.177 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.178 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.179 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.180 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.181 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.182 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.183 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.184 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.185 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "mark_out_threshold": { 2023-07-21T19:43:11.186 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mark_out_threshold", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "2419200", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.187 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "pool_name": { 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pool_name", 2023-07-21T19:43:11.188 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "device_health_metrics", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name of pool in which to store device health metrics", 2023-07-21T19:43:11.189 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "retention_period": { 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "retention_period", 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.190 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15552000", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long to retain device health metrics", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.191 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_frequency": { 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_frequency", 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.192 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to scrape device health metrics", 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.193 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "self_heal": { 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "self_heal", 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.194 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.195 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.196 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how frequently to wake up and check device health", 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "warn_threshold": { 2023-07-21T19:43:11.197 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "warn_threshold", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7257600", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.198 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.199 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "diskprediction_local", 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.200 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.201 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.202 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.203 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.205 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.206 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.207 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.208 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "predict_interval": { 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predict_interval", 2023-07-21T19:43:11.209 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "86400", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.210 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "predictor_model": { 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "predictor_model", 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.211 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "prophetstor", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.212 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.213 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "600", 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.214 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "influx", 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": false, 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "influxdb python module not found", 2023-07-21T19:43:11.215 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "batch_size": { 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "batch_size", 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5000", 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.217 INFO:teuthology.orchestra.run.smithi005.stdout: "database": { 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "database", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "ceph", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.218 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "hostname": { 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "hostname", 2023-07-21T19:43:11.219 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.220 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server hostname", 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.221 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "30", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "5", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.222 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.223 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.224 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.225 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.226 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.227 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.228 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.229 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.230 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.231 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "password": { 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "password", 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.232 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "password of InfluxDB server user", 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.233 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "port": { 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "port", 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.234 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "8086", 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "InfluxDB server port", 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.235 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "ssl": { 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ssl", 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.236 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "false", 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.237 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "threads": { 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threads", 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.238 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "32", 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.239 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "username": { 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "username", 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.240 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "username of InfluxDB server user", 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "verify_ssl": { 2023-07-21T19:43:11.241 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "verify_ssl", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "true", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.242 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.243 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "insights", 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.244 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.245 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.246 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.247 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.248 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.249 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.250 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.251 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.252 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.253 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "iostat", 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.254 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.255 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.256 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.257 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.258 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.259 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.260 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.261 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.262 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.263 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "k8sevents", 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "ceph_event_retention_days": { 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "ceph_event_retention_days", 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.264 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "7", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Days to hold ceph event information within local cache", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.265 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "config_check_secs": { 2023-07-21T19:43:11.266 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "config_check_secs", 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "10", 2023-07-21T19:43:11.267 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.268 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.269 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.270 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.271 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.272 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.273 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.274 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.275 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.276 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.277 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.278 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "localpool", 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "failure_domain": { 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "failure_domain", 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.279 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "host", 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "failure domain for any created local pool", 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-07-21T19:43:11.280 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.281 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.282 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.283 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.284 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.285 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.286 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.287 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.288 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.289 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "min_size": { 2023-07-21T19:43:11.290 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "min_size", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.291 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default min_size for any created local pool", 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "num_rep": { 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "num_rep", 2023-07-21T19:43:11.292 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default replica count for any created local pool", 2023-07-21T19:43:11.293 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_num": { 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_num", 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.294 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "128", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "default pg_num for any created local pool", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.295 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "prefix": { 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prefix", 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.296 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "name prefix for any created local pool", 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.297 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "subtree": { 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "subtree", 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.298 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "rack", 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "CRUSH level for which to create a local pool", 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-07-21T19:43:11.299 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mds_autoscaler", 2023-07-21T19:43:11.300 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.301 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.302 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.303 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.304 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.305 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.306 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.307 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.308 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.309 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.310 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirroring", 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.311 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.312 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.313 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.314 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.315 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.316 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.317 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.318 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.319 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.320 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "nfs", 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.321 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.322 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.323 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.324 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.325 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.326 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.327 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.328 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.329 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.330 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.331 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "fail_fs": { 2023-07-21T19:43:11.332 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "fail_fs", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.333 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.334 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.335 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.336 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.337 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.338 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.339 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.340 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.341 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.342 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.343 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator": { 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "orchestrator", 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.344 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "cephadm", 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "rook", 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "test_orchestrator" 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Orchestrator backend", 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.345 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_perf_query", 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.346 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.347 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.348 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.349 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.350 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.351 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.352 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.353 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.354 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.355 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.356 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "osd_support", 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.357 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.358 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.359 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.360 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.361 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.362 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.363 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.364 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.365 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.366 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.367 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "pg_autoscaler", 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.368 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.369 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.370 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.371 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.372 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.373 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.374 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.375 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.376 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.377 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "noautoscale": { 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "noautoscale", 2023-07-21T19:43:11.378 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.379 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "global autoscale flag", 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:11.380 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.381 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "threshold": { 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "threshold", 2023-07-21T19:43:11.382 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "3.0", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1.0", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "scaling threshold", 2023-07-21T19:43:11.383 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.384 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "progress", 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_pg_recovery_event": { 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_pg_recovery_event", 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.385 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow the module to show pg recovery progress", 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.386 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.387 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.388 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.389 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.390 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.391 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.392 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.393 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.394 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.395 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.396 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.397 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "max_completed_events": { 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_completed_events", 2023-07-21T19:43:11.398 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "50", 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.399 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "number of past completed events to remember", 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "sleep_interval": { 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "sleep_interval", 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.400 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "5", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "how long the module is going to sleep", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.401 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.402 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "prometheus", 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "cache": { 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "cache", 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.403 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.404 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "exclude_perf_counters": { 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "exclude_perf_counters", 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.405 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.406 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.407 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.407 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.407 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.407 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.407 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.408 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.409 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.410 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.411 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.411 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.411 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.411 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.411 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.412 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.412 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.412 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.412 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.413 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.414 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.415 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.416 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.416 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.416 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.416 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.417 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.417 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.417 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.417 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.417 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.418 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools": { 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools", 2023-07-21T19:43:11.419 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.420 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.421 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_stats_pools_refresh_interval": { 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300", 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.422 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.423 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "scrape_interval": { 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "scrape_interval", 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.424 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15.0", 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.425 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "::", 2023-07-21T19:43:11.426 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.427 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "9283", 2023-07-21T19:43:11.428 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.429 INFO:teuthology.orchestra.run.smithi005.stdout: "stale_cache_strategy": { 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stale_cache_strategy", 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "log", 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.430 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_behaviour": { 2023-07-21T19:43:11.431 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_behaviour", 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "default", 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.432 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "default", 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "error" 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.433 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "standby_error_status_code": { 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "standby_error_status_code", 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.434 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "500", 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "400", 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "599", 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.435 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.436 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.437 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.438 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.439 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.440 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.441 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.442 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.443 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.444 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.445 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.446 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_snap_create": { 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_snap_create", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.447 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "mirror_snapshot_schedule": { 2023-07-21T19:43:11.448 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "mirror_snapshot_schedule", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.449 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "trash_purge_schedule": { 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "trash_purge_schedule", 2023-07-21T19:43:11.450 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.451 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.452 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "restful", 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "enable_auth": { 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enable_auth", 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.453 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.454 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "key_file": { 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "key_file", 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.455 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.456 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.457 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.458 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.459 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.460 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.461 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.462 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.463 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.464 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.465 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.466 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "server_addr": { 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_addr", 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.467 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.468 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "server_port": { 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "server_port", 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.469 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.470 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rgw", 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.471 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.472 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.473 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.474 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.475 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.476 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.477 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.478 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.479 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.480 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rook", 2023-07-21T19:43:11.481 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "drive_group_interval": { 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "drive_group_interval", 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.482 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "300.0", 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.483 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.484 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.485 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.486 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.487 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.488 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.489 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.490 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.491 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.492 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "storage_class": { 2023-07-21T19:43:11.493 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "storage_class", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "local", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.494 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.495 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "selftest", 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.496 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.497 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.498 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.499 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.500 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.501 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.502 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.503 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.504 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.505 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "roption1": { 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption1", 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.506 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.507 INFO:teuthology.orchestra.run.smithi005.stdout: "roption2": { 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "roption2", 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "xyz", 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.508 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption1": { 2023-07-21T19:43:11.509 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption1", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.510 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption2": { 2023-07-21T19:43:11.511 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption2", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.512 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption3": { 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption3", 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "float", 2023-07-21T19:43:11.513 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.514 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption4": { 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption4", 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.515 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.516 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption5": { 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption5", 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.517 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.518 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption6": { 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption6", 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.519 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.520 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "rwoption7": { 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rwoption7", 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.521 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "1", 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "42", 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.522 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "testkey": { 2023-07-21T19:43:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testkey", 2023-07-21T19:43:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.523 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.524 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "testlkey": { 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testlkey", 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.525 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.526 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "testnewline": { 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "testnewline", 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.527 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.528 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.529 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "allow_m_granularity": { 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "allow_m_granularity", 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.530 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "allow minute scheduled snapshots", 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.531 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "dump_on_update": { 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "dump_on_update", 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.532 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "dump database to debug log on update", 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.533 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.534 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.535 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.536 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.537 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.538 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.539 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.540 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.541 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.542 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.543 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "stats", 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.544 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.545 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.546 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.547 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.548 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.549 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.550 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.551 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.552 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.553 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "status", 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.554 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.555 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.556 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.557 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.558 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.559 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.560 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.561 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.562 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.563 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.564 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telegraf", 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.565 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "address": { 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "address", 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-07-21T19:43:11.566 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.567 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.568 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "15", 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.569 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.570 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.571 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.572 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.573 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.574 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.575 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.576 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.577 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.578 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.579 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "telemetry", 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.580 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_basic": { 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_basic", 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.581 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share basic cluster information (size, version)", 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_crash": { 2023-07-21T19:43:11.582 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_crash", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.583 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_device": { 2023-07-21T19:43:11.584 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_device", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "True", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.585 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_ident": { 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_ident", 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.586 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.587 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "channel_perf": { 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "channel_perf", 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.588 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Share various performance metrics of a cluster", 2023-07-21T19:43:11.589 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "contact": { 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "contact", 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.590 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.591 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "description": { 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "description", 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.592 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.593 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "device_url": { 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "device_url", 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-07-21T19:43:11.594 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.595 INFO:teuthology.orchestra.run.smithi005.stdout: "enabled": { 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "enabled", 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.596 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.597 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "24", 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "8", 2023-07-21T19:43:11.598 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.599 INFO:teuthology.orchestra.run.smithi005.stdout: "last_opt_revision": { 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "last_opt_revision", 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "1", 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.600 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard": { 2023-07-21T19:43:11.601 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.602 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "leaderboard_description": { 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "leaderboard_description", 2023-07-21T19:43:11.603 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.604 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.605 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.606 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.607 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.608 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.609 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.610 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.611 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.612 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.613 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.614 INFO:teuthology.orchestra.run.smithi005.stdout: "organization": { 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "organization", 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.615 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "proxy": { 2023-07-21T19:43:11.616 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "proxy", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.617 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "url": { 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "url", 2023-07-21T19:43:11.618 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.619 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.620 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "test_orchestrator", 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.621 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.622 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.623 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.624 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.625 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.626 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.627 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.628 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.629 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.630 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.631 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.632 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.633 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.634 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.635 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.636 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.637 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.638 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.639 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.640 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "max_concurrent_clones": { 2023-07-21T19:43:11.641 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "max_concurrent_clones", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "4", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.642 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Number of asynchronous cloner threads", 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "snapshot_clone_delay": { 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snapshot_clone_delay", 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.643 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "0", 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-07-21T19:43:11.644 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix", 2023-07-21T19:43:11.645 INFO:teuthology.orchestra.run.smithi005.stdout: "can_run": true, 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "error_string": "", 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "module_options": { 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "discovery_interval": { 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "discovery_interval", 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "uint", 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.646 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "100", 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "identifier": { 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "identifier", 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.648 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.649 INFO:teuthology.orchestra.run.smithi005.stdout: "interval": { 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "interval", 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "secs", 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "60", 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.650 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "log_level": { 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_level", 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.652 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.653 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster": { 2023-07-21T19:43:11.654 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.655 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_cluster_level": { 2023-07-21T19:43:11.656 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_cluster_level", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "info", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [ 2023-07-21T19:43:11.657 INFO:teuthology.orchestra.run.smithi005.stdout: "", 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "critical", 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "debug", 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "error", 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "info", 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "warning" 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.658 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "log_to_file": { 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "log_to_file", 2023-07-21T19:43:11.659 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "bool", 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 1, 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "False", 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.660 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.661 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_host": { 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_host", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.662 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_port": { 2023-07-21T19:43:11.663 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_port", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "int", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "10051", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.664 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "zabbix_sender": { 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "zabbix_sender", 2023-07-21T19:43:11.665 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "str", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "level": "advanced", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "flags": 0, 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "min": "", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "max": "", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "enum_allowed": [], 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "desc": "", 2023-07-21T19:43:11.666 INFO:teuthology.orchestra.run.smithi005.stdout: "long_desc": "", 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: "tags": [], 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: "see_also": [] 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.667 INFO:teuthology.orchestra.run.smithi005.stdout: "services": { 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "dashboard": "https://172.21.15.5:8443/" 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "always_on_modules": { 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "octopus": [ 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-07-21T19:43:11.668 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-07-21T19:43:11.669 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "pacific": [ 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-07-21T19:43:11.670 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-07-21T19:43:11.671 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "quincy": [ 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-07-21T19:43:11.672 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: ], 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "reef": [ 2023-07-21T19:43:11.673 INFO:teuthology.orchestra.run.smithi005.stdout: "balancer", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "crash", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "devicehealth", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "orchestrator", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "pg_autoscaler", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "progress", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "rbd_support", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "status", 2023-07-21T19:43:11.674 INFO:teuthology.orchestra.run.smithi005.stdout: "telemetry", 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: "volumes" 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: "last_failure_osd_epoch": 88, 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: "active_clients": [ 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "devicehealth", 2023-07-21T19:43:11.675 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2677353589 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.676 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "libcephsqlite", 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.677 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3828252244 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "rbd_support", 2023-07-21T19:43:11.678 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1873060361 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.679 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.680 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 3145130535 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "snap_schedule", 2023-07-21T19:43:11.681 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 1594315300 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.682 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: "name": "volumes", 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: "addrvec": [ 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: { 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: "type": "v2", 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: "addr": "172.21.15.5:0", 2023-07-21T19:43:11.683 INFO:teuthology.orchestra.run.smithi005.stdout: "nonce": 2852339838 2023-07-21T19:43:11.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.684 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.684 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:11.684 INFO:teuthology.orchestra.run.smithi005.stdout: ] 2023-07-21T19:43:11.684 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:11.685 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:09 smithi156 ceph-mon[128316]: pgmap v1128: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-21T19:43:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:09 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/121580188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:43:11.686 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:09 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2004219412' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:43:11.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:09 smithi101 ceph-mon[128136]: pgmap v1128: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-21T19:43:11.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:09 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/121580188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:43:11.686 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:09 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2004219412' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:43:11.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 msgr2=0x7f47280a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:11.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 0x7f47280a37b0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f4740199330 tx=0x7f4734009260 comp rx=0 tx=0).stop 2023-07-21T19:43:11.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 msgr2=0x7f4740197d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:43:11.687 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f4730000c00 tx=0x7f4730004730 comp rx=0 tx=0).stop 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 shutdown_connections 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f474010c3d0 0x7f474019c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f47280a1300 0x7f47280a37b0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f4740072830 0x7f4740197d20 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 --2- 172.21.15.5:0/2004219412 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f4740073140 0x7f4740198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 >> 172.21.15.5:0/2004219412 conn(0x7f47400fb610 msgr2=0x7f474010e670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:43:11.688 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 shutdown_connections 2023-07-21T19:43:11.689 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:43:09.333+0000 7f4745fff700 1 -- 172.21.15.5:0/2004219412 wait complete. 2023-07-21T19:43:11.696 DEBUG:tasks.check_counter:Getting stats from a 2023-07-21T19:43:11.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.a perf dump 2023-07-21T19:43:11.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:09 smithi005 ceph-mon[126460]: pgmap v1128: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 23 op/s 2023-07-21T19:43:11.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:09 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/121580188' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:43:11.698 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:09 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2004219412' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-07-21T19:43:12.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:11 smithi101 ceph-mon[128136]: pgmap v1129: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:12.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:11 smithi101 ceph-mon[128136]: osdmap e148: 12 total, 12 up, 12 in 2023-07-21T19:43:12.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:11 smithi005 ceph-mon[126460]: pgmap v1129: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:12.142 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:11 smithi005 ceph-mon[126460]: osdmap e148: 12 total, 12 up, 12 in 2023-07-21T19:43:12.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:11 smithi156 ceph-mon[128316]: pgmap v1129: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:43:12.156 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:11 smithi156 ceph-mon[128316]: osdmap e148: 12 total, 12 up, 12 in 2023-07-21T19:43:12.876 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:43:13.893 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:13 smithi005 ceph-mon[126460]: pgmap v1131: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 102 B/s wr, 5 op/s 2023-07-21T19:43:14.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:13 smithi156 ceph-mon[128316]: pgmap v1131: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 102 B/s wr, 5 op/s 2023-07-21T19:43:14.318 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:13 smithi101 ceph-mon[128136]: pgmap v1131: 97 pgs: 16 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 53 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 102 B/s wr, 5 op/s 2023-07-21T19:43:14.909 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:14.910 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:14.910 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2023-07-21T19:43:14.910 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2023-07-21T19:43:14.910 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:43:14.910 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.004008238, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000725572, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:14.911 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:14.912 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 612, 2023-07-21T19:43:14.913 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 600, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1536032, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 344400, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.105404088, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.046124807, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.046256212, 2023-07-21T19:43:14.914 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.014032522, 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 600, 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.045245083, 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000075408 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:14.915 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1536032, 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 344400 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:14.916 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 459, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 640, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 383936, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.039009753, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.028770201, 2023-07-21T19:43:14.917 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004188879, 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 456, 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.050883065, 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000111585 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.918 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 383936 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.919 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:14.920 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:14.921 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:14.922 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:14.923 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:14.924 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-07-21T19:43:14.925 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:14.926 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-07-21T19:43:14.927 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 612, 2023-07-21T19:43:14.928 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1447091, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 612, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 612, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1447091, 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-07-21T19:43:14.929 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:14.930 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:14.930 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:14.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:14.930 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:14.930 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:15.355 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-07-21T19:43:15.355 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-07-21T19:43:15.356 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-07-21T19:43:15.356 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-07-21T19:43:15.356 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-07-21T19:43:15.356 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-07-21T19:43:15.356 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-07-21T19:43:15.356 DEBUG:tasks.check_counter:Getting stats from d 2023-07-21T19:43:15.357 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.d perf dump 2023-07-21T19:43:16.091 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:43:16.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:15 smithi005 ceph-mon[126460]: pgmap v1132: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:16.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:15 smithi156 ceph-mon[128316]: pgmap v1132: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:16.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:15 smithi101 ceph-mon[128136]: pgmap v1132: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:17.641 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:17.641 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:17.641 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2023-07-21T19:43:17.641 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2023-07-21T19:43:17.641 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.003436776, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000573657, 2023-07-21T19:43:17.642 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:17.643 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:17.643 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:17.643 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:17.643 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:17.643 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:17.644 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 465, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 640, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 384992, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2023-07-21T19:43:17.645 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.040491538, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.029989591, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004131002, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 462, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.058430897, 2023-07-21T19:43:17.646 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000126473 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.647 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 384992 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 619, 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 607, 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1538000, 2023-07-21T19:43:17.648 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 348112, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.105381228, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.045365811, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.047003027, 2023-07-21T19:43:17.649 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.014371342, 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 607, 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.045101922, 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000074303 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:17.650 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1538000, 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 348112 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-07-21T19:43:17.651 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:17.652 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:17.653 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:17.654 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:17.655 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:17.656 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148731, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 31, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-07-21T19:43:17.657 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:17.658 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:17.659 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 619, 2023-07-21T19:43:17.660 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1448065, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 619, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 619, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1448065, 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-07-21T19:43:17.661 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:17.662 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:17.662 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:17.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:17.662 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:17.662 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:18.061 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:17 smithi005 ceph-mon[126460]: pgmap v1133: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:18.091 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-07-21T19:43:18.092 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-07-21T19:43:18.092 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-07-21T19:43:18.092 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-07-21T19:43:18.092 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-07-21T19:43:18.092 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-07-21T19:43:18.093 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-07-21T19:43:18.093 DEBUG:tasks.check_counter:Getting stats from g 2023-07-21T19:43:18.093 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.g perf dump 2023-07-21T19:43:18.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:17 smithi156 ceph-mon[128316]: pgmap v1133: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:18.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:17 smithi101 ceph-mon[128136]: pgmap v1133: 97 pgs: 22 active+clean+snaptrim, 40 active+clean+snaptrim_wait, 35 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:18.825 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:43:19.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:19 smithi101 ceph-mon[128136]: pgmap v1134: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:19.325 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:19 smithi005 ceph-mon[126460]: pgmap v1134: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:19.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:19 smithi156 ceph-mon[128316]: pgmap v1134: 97 pgs: 2 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 89 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:20.272 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 610, 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 600, 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1533616, 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 345136, 2023-07-21T19:43:20.273 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.106895609, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.044870836, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.046743532, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.014232589, 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:20.274 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 599, 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.052307533, 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000087324 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:20.275 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1533616, 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 345136 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 8, 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 463, 2023-07-21T19:43:20.276 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 3600, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 384736, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.044325318, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.033419501, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.004843341, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:20.277 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 461, 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.040851189, 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000088614 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:20.278 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 3600, 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 384736 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2023-07-21T19:43:20.279 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000691754, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:43:20.280 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000540008, 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:20.281 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-07-21T19:43:20.282 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:20.283 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:20.284 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:20.285 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:20.286 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:20.287 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:20.288 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:20.289 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:20.290 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 614, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1447351, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:20.291 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 614, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 614, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1447351, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:20.292 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:20.293 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:20.293 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:20.824 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-07-21T19:43:20.824 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-07-21T19:43:20.825 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-07-21T19:43:20.825 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-07-21T19:43:20.825 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-07-21T19:43:20.825 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-07-21T19:43:20.825 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-07-21T19:43:20.826 DEBUG:tasks.check_counter:Getting stats from j 2023-07-21T19:43:20.826 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.j perf dump 2023-07-21T19:43:21.563 INFO:teuthology.orchestra.run.smithi005.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.a/config 2023-07-21T19:43:22.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:21 smithi101 ceph-mon[128136]: pgmap v1135: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:22.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:21 smithi005 ceph-mon[126460]: pgmap v1135: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:22.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:21 smithi156 ceph-mon[128316]: pgmap v1135: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 7 op/s 2023-07-21T19:43:24.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:23 smithi101 ceph-mon[128136]: pgmap v1136: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 171 B/s wr, 7 op/s 2023-07-21T19:43:24.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:23 smithi101 ceph-mon[128136]: osdmap e149: 12 total, 12 up, 12 in 2023-07-21T19:43:24.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:23 smithi005 ceph-mon[126460]: pgmap v1136: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 171 B/s wr, 7 op/s 2023-07-21T19:43:24.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:23 smithi005 ceph-mon[126460]: osdmap e149: 12 total, 12 up, 12 in 2023-07-21T19:43:24.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:23 smithi156 ceph-mon[128316]: pgmap v1136: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 171 B/s wr, 7 op/s 2023-07-21T19:43:24.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:23 smithi156 ceph-mon[128316]: osdmap e149: 12 total, 12 up, 12 in 2023-07-21T19:43:25.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:25 smithi005 ceph-mon[126460]: pgmap v1138: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:26.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:25 smithi101 ceph-mon[128136]: pgmap v1138: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:26.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:25 smithi156 ceph-mon[128316]: pgmap v1138: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:26.197 INFO:teuthology.orchestra.run.smithi005.stdout:{ 2023-07-21T19:43:26.197 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 622, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 610, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 1538816, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 349888, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 4, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 0, 2023-07-21T19:43:26.198 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.110072244, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.045477207, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.050026581, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.014600893, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 610, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.043826023, 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000071845 2023-07-21T19:43:26.199 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 1538816, 2023-07-21T19:43:26.200 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 349888 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 4, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 466, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 640, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 385168, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 7, 2023-07-21T19:43:26.201 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.040496882, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.030199839, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.003795268, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 463, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.048752819, 2023-07-21T19:43:26.202 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000105297 2023-07-21T19:43:26.203 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.203 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:26.203 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:26.203 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:26.203 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 640, 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 385168 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_messages": 0, 2023-07-21T19:43:26.204 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages": 2, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_bytes": 0, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_created_connections": 2, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_active_connections": 1, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_total_time": 0.000732188, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_recv_time": 0.000582436, 2023-07-21T19:43:26.205 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:26.206 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "cct": { 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "total_workers": 0, 2023-07-21T19:43:26.207 INFO:teuthology.orchestra.run.smithi005.stdout: "unhealthy_workers": 0 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "mempool": { 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:26.208 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:26.209 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:26.210 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:26.211 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_items": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:26.212 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_bytes": 148714, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_anon_items": 30, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_bytes": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_items": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:26.213 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_bytes": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_items": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "pgmap_items": 0, 2023-07-21T19:43:26.214 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_bytes": 928, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "mds_co_items": 2, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_1_items": 0, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "unittest_2_items": 0 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: }, 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:26.215 INFO:teuthology.orchestra.run.smithi005.stdout: "val": 0, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "max": 104857600, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "get_started": 0, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "get": 622, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "get_sum": 1448455, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "get_or_fail_success": 622, 2023-07-21T19:43:26.216 INFO:teuthology.orchestra.run.smithi005.stdout: "take": 0, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "take_sum": 0, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "put": 622, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "put_sum": 1448455, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "wait": { 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "avgcount": 0, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "sum": 0.000000000, 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: "avgtime": 0.000000000 2023-07-21T19:43:26.217 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:26.218 INFO:teuthology.orchestra.run.smithi005.stdout: } 2023-07-21T19:43:26.218 INFO:teuthology.orchestra.run.smithi005.stdout:} 2023-07-21T19:43:26.666 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-07-21T19:43:26.667 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-07-21T19:43:26.667 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-07-21T19:43:26.667 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-07-21T19:43:26.667 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-07-21T19:43:26.667 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-07-21T19:43:26.668 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-07-21T19:43:26.668 DEBUG:tasks.check_counter:Getting stats from b 2023-07-21T19:43:26.668 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.b perf dump 2023-07-21T19:43:27.883 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:43:28.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:27 smithi101 ceph-mon[128136]: pgmap v1139: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:28.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:27 smithi005 ceph-mon[126460]: pgmap v1139: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:28.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:27 smithi156 ceph-mon[128316]: pgmap v1139: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 7 op/s 2023-07-21T19:43:29.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:29 smithi101 ceph-mon[128136]: pgmap v1140: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:29.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:29 smithi005 ceph-mon[126460]: pgmap v1140: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:29.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:29 smithi156 ceph-mon[128316]: pgmap v1140: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:32.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:31 smithi101 ceph-mon[128136]: pgmap v1141: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:32.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:31 smithi005 ceph-mon[126460]: pgmap v1141: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:32.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:31 smithi156 ceph-mon[128316]: pgmap v1141: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:34.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:33 smithi101 ceph-mon[128136]: pgmap v1142: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:34.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:33 smithi005 ceph-mon[126460]: pgmap v1142: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:34.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:33 smithi156 ceph-mon[128316]: pgmap v1142: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:36.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:35 smithi101 ceph-mon[128136]: pgmap v1143: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:36.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:35 smithi005 ceph-mon[126460]: pgmap v1143: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:36.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:35 smithi156 ceph-mon[128316]: pgmap v1143: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 6 op/s 2023-07-21T19:43:38.017 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-21T19:43:38.017 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:38.018 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 4827053, 2023-07-21T19:43:38.018 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 4827940, 2023-07-21T19:43:38.018 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 788798230, 2023-07-21T19:43:38.018 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 845604731, 2023-07-21T19:43:38.019 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 565, 2023-07-21T19:43:38.019 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:38.019 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 342.440984225, 2023-07-21T19:43:38.019 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 163.141714287, 2023-07-21T19:43:38.020 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 229.367780552, 2023-07-21T19:43:38.020 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.881388907, 2023-07-21T19:43:38.020 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:38.021 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 4827941, 2023-07-21T19:43:38.021 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 93.526001236, 2023-07-21T19:43:38.021 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000019371 2023-07-21T19:43:38.021 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.022 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:38.022 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 9616991, 2023-07-21T19:43:38.022 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.247040679, 2023-07-21T19:43:38.022 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000129 2023-07-21T19:43:38.023 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.023 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 46064, 2023-07-21T19:43:38.023 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 3776 2023-07-21T19:43:38.024 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.024 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:38.024 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 1940567, 2023-07-21T19:43:38.025 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 1949753, 2023-07-21T19:43:38.025 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 312967862, 2023-07-21T19:43:38.025 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 364017193, 2023-07-21T19:43:38.025 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 15, 2023-07-21T19:43:38.026 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 5, 2023-07-21T19:43:38.026 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 81.340566513, 2023-07-21T19:43:38.026 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 34.967067453, 2023-07-21T19:43:38.027 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 53.502308778, 2023-07-21T19:43:38.027 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.341609549, 2023-07-21T19:43:38.027 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:38.028 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1949752, 2023-07-21T19:43:38.028 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 33.661774235, 2023-07-21T19:43:38.028 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000017264 2023-07-21T19:43:38.028 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.029 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:38.029 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 3708071, 2023-07-21T19:43:38.029 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.693390811, 2023-07-21T19:43:38.029 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000186 2023-07-21T19:43:38.030 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.030 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 42832, 2023-07-21T19:43:38.030 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 978800 2023-07-21T19:43:38.030 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 76042, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 90646, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 26049445, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 56815682, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 25, 2023-07-21T19:43:38.031 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 9, 2023-07-21T19:43:38.032 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 7.842032502, 2023-07-21T19:43:38.032 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 2.798316027, 2023-07-21T19:43:38.032 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 3.977525524, 2023-07-21T19:43:38.032 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 1.569654215, 2023-07-21T19:43:38.032 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 90641, 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 9.504361608, 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000104857 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:38.033 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 151156, 2023-07-21T19:43:38.034 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.034575225, 2023-07-21T19:43:38.034 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000228 2023-07-21T19:43:38.034 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.034 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 3040528, 2023-07-21T19:43:38.034 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 826096 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 1, 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-MDSRank": { 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:38.035 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 31089, 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 13.449559696, 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000432614 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:38.036 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 791, 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.042242108, 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000053403 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "mds": { 2023-07-21T19:43:38.037 INFO:teuthology.orchestra.run.smithi101.stdout: "request": 30496, 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "reply": 27269, 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "reply_latency": { 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 27269, 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 422.052964123, 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.015477390 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.038 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_reply": 0, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "forward": 3227, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_complete": 9, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_keys": 1, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_commit": 2629, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_split": 7, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_merge": 0, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes": 1062, 2023-07-21T19:43:38.039 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_top": 541, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_bottom": 508, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pin_tail": 13, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pinned": 271, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_expired": 1634, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_with_caps": 254, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "caps": 255, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "subtrees": 44, 2023-07-21T19:43:38.040 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse": 78382, 2023-07-21T19:43:38.041 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_hit": 67648, 2023-07-21T19:43:38.041 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_forward": 3095, 2023-07-21T19:43:38.041 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_discover": 153, 2023-07-21T19:43:38.041 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:43:38.041 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_remote_ino": 0, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_lock": 4071, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "load_cent": 0, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "q": 0, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "exported": 40, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "exported_inodes": 1400, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "imported": 26, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "imported_inodes": 1033, 2023-07-21T19:43:38.042 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_backtrace_fetch": 2794, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_peer_discover": 0, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rfiles": 1044, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rbytes": 8966388417, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rsnaps": 6, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_fetch": 2794, 2023-07-21T19:43:38.043 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_inodes": 436, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_base_inodes": 21, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dirfrag_rstats": 415, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_file_inodes": 2358, 2023-07-21T19:43:38.044 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_inode_file_caps": 477, 2023-07-21T19:43:38.045 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_revoke": 23107, 2023-07-21T19:43:38.045 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_grant": 16659, 2023-07-21T19:43:38.045 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:38.045 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps": 37082, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps_dirty": 12776, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_cap_release": 245, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "process_request_cap_release": 2575 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.046 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_cache": { 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update": 84, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update_receipt": 65, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_try_discover": 36, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_send_discover": 189, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_handle_discover": 460, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays": 0, 2023-07-21T19:43:38.047 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_created": 394, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_enqueued": 221, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_reintegrated": 921, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_migrated": 73, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:38.048 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_started": 0, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_completed": 0, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_enqueue_scrub": 21, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_exportdir": 41, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_flush": 0, 2023-07-21T19:43:38.049 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragmentdir": 7, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_inodestats": 0 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_log": { 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "evadd": 28530, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "evex": 28468, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "evtrm": 28468, 2023-07-21T19:43:38.050 INFO:teuthology.orchestra.run.smithi101.stdout: "ev": 62, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "evexg": 0, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "evexd": 270, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "segadd": 179, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "segex": 178, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "segtrm": 178, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "seg": 2, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "segexg": 0, 2023-07-21T19:43:38.051 INFO:teuthology.orchestra.run.smithi101.stdout: "segexd": 1, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "expos": 103525477, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "wrpos": 103806644, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "rdpos": 0, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "jlat": { 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 13270, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 54.049752444, 2023-07-21T19:43:38.052 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.004073078 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "replayed": 0 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_mem": { 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "ino": 772, 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "ino+": 2369, 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "ino-": 1597, 2023-07-21T19:43:38.053 INFO:teuthology.orchestra.run.smithi101.stdout: "dir": 123, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "dir+": 190, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "dir-": 67, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "dn": 1062, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "dn+": 3585, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "dn-": 2523, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "cap": 255, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "cap+": 1757, 2023-07-21T19:43:38.054 INFO:teuthology.orchestra.run.smithi101.stdout: "cap-": 1502, 2023-07-21T19:43:38.055 INFO:teuthology.orchestra.run.smithi101.stdout: "rss": 75164, 2023-07-21T19:43:38.055 INFO:teuthology.orchestra.run.smithi101.stdout: "heap": 207156 2023-07-21T19:43:38.055 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.055 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_server": { 2023-07-21T19:43:38.055 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_client_request": 63395, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_server_request": 8903, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_request": 30496, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_session": 171, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_peer_request": 18798, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "req_create_latency": { 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1189, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.990897260, 2023-07-21T19:43:38.056 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001674429 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getattr_latency": { 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1936, 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 11.089993166, 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.005728302 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.057 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "req_link_latency": { 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 172, 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.590284786, 2023-07-21T19:43:38.058 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.009245841 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookup_latency": { 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 12803, 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 189.541684649, 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.014804474 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:38.059 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupino_latency": { 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.060 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupname_latency": { 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.061 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.062 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.063 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.063 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.063 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lssnap_latency": { 2023-07-21T19:43:38.064 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 36, 2023-07-21T19:43:38.064 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.007430708, 2023-07-21T19:43:38.064 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000206408 2023-07-21T19:43:38.064 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.065 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mkdir_latency": { 2023-07-21T19:43:38.065 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 34, 2023-07-21T19:43:38.065 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.043310774, 2023-07-21T19:43:38.066 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001273846 2023-07-21T19:43:38.066 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.066 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mknod_latency": { 2023-07-21T19:43:38.066 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.067 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.067 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.067 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.067 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mksnap_latency": { 2023-07-21T19:43:38.068 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 36, 2023-07-21T19:43:38.068 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.715239533, 2023-07-21T19:43:38.068 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.019867764 2023-07-21T19:43:38.069 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.069 INFO:teuthology.orchestra.run.smithi101.stdout: "req_open_latency": { 2023-07-21T19:43:38.069 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 9466, 2023-07-21T19:43:38.069 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 4.213811167, 2023-07-21T19:43:38.069 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000445152 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: "req_readdir_latency": { 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 163, 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.260540588, 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001598408 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.070 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rename_latency": { 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1201, 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 211.026214425, 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.175708754 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.071 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.072 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmdir_latency": { 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 30, 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.433725019, 2023-07-21T19:43:38.073 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.014457500 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.074 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setattr_latency": { 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1, 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000984214, 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000984214 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.075 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.076 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setlayout_latency": { 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setxattr_latency": { 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 6, 2023-07-21T19:43:38.077 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.018408910, 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.003068151 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: "req_symlink_latency": { 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.078 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: "req_unlink_latency": { 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 196, 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.120438924, 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.005716525 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.079 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.080 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_sessions": { 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: "session_count": 2, 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: "session_add": 5, 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: "session_remove": 3, 2023-07-21T19:43:38.081 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_open": 2, 2023-07-21T19:43:38.082 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_stale": 0, 2023-07-21T19:43:38.082 INFO:teuthology.orchestra.run.smithi101.stdout: "total_load": 905, 2023-07-21T19:43:38.082 INFO:teuthology.orchestra.run.smithi101.stdout: "average_load": 452, 2023-07-21T19:43:38.082 INFO:teuthology.orchestra.run.smithi101.stdout: "avg_session_uptime": 2157 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:38.083 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:38.084 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:38.085 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:38.086 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:38.087 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:38.087 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:38.087 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:38.087 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:38.087 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:38.088 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 873267, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 245, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:38.089 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 167, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:38.090 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 2931334, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 47511, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.091 INFO:teuthology.orchestra.run.smithi101.stdout: "objecter": { 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_active": 0, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_laggy": 0, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send": 22364, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send_bytes": 108211379, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_resend": 31, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_reply": 22364, 2023-07-21T19:43:38.092 INFO:teuthology.orchestra.run.smithi101.stdout: "op_latency": { 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 22364, 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 168.488361881, 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.007533909 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "op_inflight": 0, 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "oplen_avg": { 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 22364, 2023-07-21T19:43:38.093 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 30115 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "op": 22364, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "op_r": 2958, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "op_w": 19406, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "op_rmw": 0, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "op_pg": 0, 2023-07-21T19:43:38.094 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_stat": 2527, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_create": 1391, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_read": 130, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_write": 13379, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writefull": 674, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writesame": 0, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_append": 0, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_zero": 0, 2023-07-21T19:43:38.095 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_truncate": 5, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_delete": 781, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_mapext": 0, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_getxattr": 2825, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_setxattr": 2782, 2023-07-21T19:43:38.096 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_call": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_watch": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_notify": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:38.097 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_other": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_active": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_send": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_resend": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_ping": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_active": 0, 2023-07-21T19:43:38.098 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_send": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_resend": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_active": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_send": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_resend": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_active": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_send": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_resend": 0, 2023-07-21T19:43:38.099 INFO:teuthology.orchestra.run.smithi101.stdout: "command_active": 0, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "command_send": 0, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "command_resend": 0, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "map_epoch": 149, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "map_full": 0, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "map_inc": 65, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_sessions": 12, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_open": 12, 2023-07-21T19:43:38.100 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_close": 0, 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_laggy": 0, 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_wr": 5193, 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_rd": 23, 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_del": 405 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "oft": { 2023-07-21T19:43:38.101 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_objs": 1, 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_updates": 0, 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_removes": 0 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "osd-slow-ops": { 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_ops_count": 36043 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.102 INFO:teuthology.orchestra.run.smithi101.stdout: "purge_queue": { 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops_high_water": 10, 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing": 0, 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_high_water": 10, 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executed": 221, 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:38.103 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 6843660, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 511707697, 2023-07-21T19:43:38.104 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 6843660, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 6843660, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 511707697, 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:38.105 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.106 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.106 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.106 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.106 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.107 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:38.107 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:38.107 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:38.107 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:38.108 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:38.108 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:38.108 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:38.109 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:38.109 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 22364, 2023-07-21T19:43:38.109 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 108234081, 2023-07-21T19:43:38.109 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 22364, 2023-07-21T19:43:38.110 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 108234081, 2023-07-21T19:43:38.110 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:38.110 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.110 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.111 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.111 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.111 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.111 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:38.112 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:38.112 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 1024, 2023-07-21T19:43:38.112 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:38.112 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:38.112 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 22364, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 22364, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 22364, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 22364, 2023-07-21T19:43:38.113 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:38.114 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 221, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 22945, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 221, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:38.115 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 109, 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 22945, 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.116 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle-0x558a8c42e0a0": { 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 28530, 2023-07-21T19:43:38.117 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 99612340, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 28530, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 13270, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 99612340, 2023-07-21T19:43:38.118 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:38.119 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-21T19:43:38.120 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:37 smithi101 ceph-mon[128136]: pgmap v1144: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:38.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:37 smithi005 ceph-mon[126460]: pgmap v1144: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:38.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:37 smithi156 ceph-mon[128316]: pgmap v1144: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:38.542 INFO:tasks.check_counter:Daemon mds.b mds.exported=40 2023-07-21T19:43:38.543 INFO:tasks.check_counter:Daemon mds.b mds.imported=26 2023-07-21T19:43:38.543 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=84 2023-07-21T19:43:38.543 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=65 2023-07-21T19:43:38.543 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-07-21T19:43:38.544 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=36 2023-07-21T19:43:38.544 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=30 2023-07-21T19:43:38.544 DEBUG:tasks.check_counter:Getting stats from e 2023-07-21T19:43:38.544 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.e perf dump 2023-07-21T19:43:39.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:39 smithi101 ceph-mon[128136]: pgmap v1145: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:39.370 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:43:39.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:39 smithi005 ceph-mon[126460]: pgmap v1145: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:39.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:39 smithi156 ceph-mon[128316]: pgmap v1145: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:41.962 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-21T19:43:41.963 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:41.963 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 230, 2023-07-21T19:43:41.963 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 229, 2023-07-21T19:43:41.963 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 203632, 2023-07-21T19:43:41.963 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 131008, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 4, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 0, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.033262280, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.009804888, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.013370131, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.005814326, 2023-07-21T19:43:41.964 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 228, 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.026582280, 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000116588 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:41.965 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 203632, 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 131008 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:41.966 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 5, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 177, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 3120, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 77056, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 3, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 2, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.010079859, 2023-07-21T19:43:41.967 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.006604333, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.001754642, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 176, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.021748449, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000123570 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.968 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 3120, 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 77056 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.969 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 0, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 2, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 0, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 0, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 2, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 1, 2023-07-21T19:43:41.970 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.001165735, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.000000000, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.000971913, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:41.971 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:41.972 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 0, 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-21T19:43:41.973 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:41.974 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:41.975 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:41.976 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:41.977 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 148967, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 29, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:41.978 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:41.979 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 928, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 2, 2023-07-21T19:43:41.980 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:41.981 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 231, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 171095, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 231, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:41.982 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 231, 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 171095, 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:41.983 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-21T19:43:41.984 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:41 smithi101 ceph-mon[128136]: pgmap v1146: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:42.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:41 smithi005 ceph-mon[126460]: pgmap v1146: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:42.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:41 smithi156 ceph-mon[128316]: pgmap v1146: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:42.495 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.e 2023-07-21T19:43:42.495 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.e 2023-07-21T19:43:42.495 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.e 2023-07-21T19:43:42.495 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.e 2023-07-21T19:43:42.496 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2023-07-21T19:43:42.496 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2023-07-21T19:43:42.496 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2023-07-21T19:43:42.496 DEBUG:tasks.check_counter:Getting stats from h 2023-07-21T19:43:42.497 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.h perf dump 2023-07-21T19:43:43.311 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:43:44.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:43 smithi101 ceph-mon[128136]: pgmap v1147: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-21T19:43:44.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:43 smithi005 ceph-mon[126460]: pgmap v1147: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-21T19:43:44.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:43 smithi156 ceph-mon[128316]: pgmap v1147: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 6 op/s 2023-07-21T19:43:45.677 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-21T19:43:45.677 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:45.677 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 4866384, 2023-07-21T19:43:45.677 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 4865494, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 787263813, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 858316738, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 14, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 4, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 345.290831095, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 165.766537215, 2023-07-21T19:43:45.678 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 231.217531648, 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.545777833, 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 4865493, 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 98.184552674, 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000020179 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.679 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 9707497, 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.209741201, 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000124 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 1841664, 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 432944 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.680 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 539358, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 823952, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 170652679, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 405816841, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 53, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:45.681 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 45.567046219, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 20.583862882, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 27.490828638, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.093892130, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 823949, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 50.839539307, 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000061702 2023-07-21T19:43:45.682 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1262199, 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.306650772, 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000242 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.683 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 2016, 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 969952 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 1969387, 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 1973509, 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 391810461, 2023-07-21T19:43:45.684 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 482256248, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 8, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 109.822948589, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 47.136822622, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 74.993967056, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.130417827, 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:45.685 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1973508, 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 79.089279174, 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000040075 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 3852167, 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.684645896, 2023-07-21T19:43:45.686 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000177 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 1, 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-21T19:43:45.687 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-MDSRank": { 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 20583, 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 9.079893986, 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000441135 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.688 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 147, 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.007087291, 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000048212 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.689 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "mds": { 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "request": 195422, 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "reply": 190960, 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "reply_latency": { 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 190960, 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 715.880180257, 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.003748848 2023-07-21T19:43:45.690 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_reply": 0, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "forward": 4458, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_complete": 26, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_keys": 0, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_commit": 35, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_split": 3, 2023-07-21T19:43:45.691 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_merge": 0, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes": 1188, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_top": 0, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_bottom": 98, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pin_tail": 1090, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pinned": 1091, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_expired": 3157, 2023-07-21T19:43:45.692 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_with_caps": 647, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "caps": 647, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "subtrees": 47, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse": 414587, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_hit": 401064, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_forward": 4271, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_discover": 202, 2023-07-21T19:43:45.693 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_dir_fetch": 5, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_remote_ino": 0, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_lock": 5031, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "load_cent": 0, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "q": 0, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "exported": 57, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "exported_inodes": 2581, 2023-07-21T19:43:45.694 INFO:teuthology.orchestra.run.smithi101.stdout: "imported": 49, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "imported_inodes": 2749, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_backtrace_fetch": 4411, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_peer_discover": 0, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rfiles": 1044, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rbytes": 8918784409, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rsnaps": 6, 2023-07-21T19:43:45.695 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_fetch": 4411, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_inodes": 202, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dirfrag_rstats": 202, 2023-07-21T19:43:45.696 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_file_inodes": 4209, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_inode_file_caps": 580, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_revoke": 326501, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_grant": 289046, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:45.697 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps": 340847, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps_dirty": 5973, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_cap_release": 190, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "process_request_cap_release": 1161 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_cache": { 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update": 132, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update_receipt": 108, 2023-07-21T19:43:45.698 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_try_discover": 63, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_send_discover": 256, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_handle_discover": 368, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays": 0, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_created": 371, 2023-07-21T19:43:45.699 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_enqueued": 33, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_reintegrated": 1494, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_migrated": 79, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_started": 0, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_completed": 0, 2023-07-21T19:43:45.700 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_exportdir": 66, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_flush": 0, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragmentdir": 3, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_inodestats": 0 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_log": { 2023-07-21T19:43:45.701 INFO:teuthology.orchestra.run.smithi101.stdout: "evadd": 15882, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "evex": 0, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "evtrm": 0, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "ev": 15882, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "evexg": 0, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "evexd": 0, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "segadd": 56, 2023-07-21T19:43:45.702 INFO:teuthology.orchestra.run.smithi101.stdout: "segex": 0, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "segtrm": 0, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "seg": 56, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "segexg": 0, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "segexd": 0, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "expos": 4194304, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "wrpos": 217464189, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "rdpos": 0, 2023-07-21T19:43:45.703 INFO:teuthology.orchestra.run.smithi101.stdout: "jlat": { 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 8656, 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 41.920483748, 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.004842939 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: "replayed": 0 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.704 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_mem": { 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "ino": 880, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "ino+": 2777, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "ino-": 1897, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "dir": 111, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "dir+": 192, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "dir-": 81, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "dn": 1189, 2023-07-21T19:43:45.705 INFO:teuthology.orchestra.run.smithi101.stdout: "dn+": 4435, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "dn-": 3246, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "cap": 647, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "cap+": 1772, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "cap-": 1125, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "rss": 75368, 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: "heap": 198964 2023-07-21T19:43:45.706 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_server": { 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_client_request": 363918, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_server_request": 7617, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_request": 195422, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_session": 224, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_peer_request": 15476, 2023-07-21T19:43:45.707 INFO:teuthology.orchestra.run.smithi101.stdout: "req_create_latency": { 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 95, 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.173853565, 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001830037 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getattr_latency": { 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 172658, 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 220.759086089, 2023-07-21T19:43:45.708 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001278591 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.709 INFO:teuthology.orchestra.run.smithi101.stdout: "req_link_latency": { 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 339, 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.764482103, 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.005204961 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookup_latency": { 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 11566, 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 300.044725463, 2023-07-21T19:43:45.710 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.025941961 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupino_latency": { 2023-07-21T19:43:45.711 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupname_latency": { 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.712 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:45.713 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.714 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.714 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.714 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.714 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lssnap_latency": { 2023-07-21T19:43:45.714 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mkdir_latency": { 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.715 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mknod_latency": { 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mksnap_latency": { 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.716 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: "req_open_latency": { 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 4214, 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 2.464552241, 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000584848 2023-07-21T19:43:45.717 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "req_readdir_latency": { 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 143, 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.137000409, 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000958044 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rename_latency": { 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1605, 2023-07-21T19:43:45.718 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 189.230954116, 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.117900905 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.719 INFO:teuthology.orchestra.run.smithi101.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmdir_latency": { 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.720 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.721 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setattr_latency": { 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.722 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.723 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setlayout_latency": { 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.724 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setxattr_latency": { 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.725 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "req_symlink_latency": { 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "req_unlink_latency": { 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 340, 2023-07-21T19:43:45.726 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 1.305526271, 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.003839783 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.727 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_sessions": { 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "session_count": 2, 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "session_add": 3, 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "session_remove": 1, 2023-07-21T19:43:45.728 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_open": 2, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_stale": 0, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "total_load": 10277, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "average_load": 5138, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "avg_session_uptime": 2165 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:45.729 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:45.730 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:45.731 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:45.732 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-21T19:43:45.733 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 570415, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 181, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:45.734 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:45.735 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 167, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 3507911, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 38762, 2023-07-21T19:43:45.736 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "objecter": { 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "op_active": 0, 2023-07-21T19:43:45.737 INFO:teuthology.orchestra.run.smithi101.stdout: "op_laggy": 0, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send": 13804, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send_bytes": 213630183, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "op_resend": 27, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "op_reply": 13804, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "op_latency": { 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 13804, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 49.067957605, 2023-07-21T19:43:45.738 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.003554618 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "op_inflight": 0, 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "oplen_avg": { 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 13804, 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 14412 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "op": 13804, 2023-07-21T19:43:45.739 INFO:teuthology.orchestra.run.smithi101.stdout: "op_r": 4508, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "op_w": 9296, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "op_rmw": 0, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "op_pg": 0, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_stat": 0, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_create": 10, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_read": 19, 2023-07-21T19:43:45.740 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_write": 8675, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writefull": 148, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writesame": 0, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_append": 0, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_zero": 0, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_truncate": 1, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_delete": 97, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_mapext": 0, 2023-07-21T19:43:45.741 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_getxattr": 4423, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_setxattr": 20, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_call": 0, 2023-07-21T19:43:45.742 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_watch": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_notify": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_other": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_active": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_send": 0, 2023-07-21T19:43:45.743 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_resend": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_ping": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_active": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_send": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_resend": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_active": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_send": 0, 2023-07-21T19:43:45.744 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_resend": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_active": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_send": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_resend": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "command_active": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "command_send": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "command_resend": 0, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "map_epoch": 149, 2023-07-21T19:43:45.745 INFO:teuthology.orchestra.run.smithi101.stdout: "map_full": 0, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "map_inc": 64, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_sessions": 12, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_open": 12, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_close": 0, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_laggy": 0, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_wr": 660, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_rd": 104, 2023-07-21T19:43:45.746 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_del": 255 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: "oft": { 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_objs": 1, 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_updates": 0, 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_removes": 0 2023-07-21T19:43:45.747 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "osd-slow-ops": { 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_ops_count": 199274 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "purge_queue": { 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops_high_water": 6, 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing": 0, 2023-07-21T19:43:45.748 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_high_water": 6, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executed": 33, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 68, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:45.749 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 7375127, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 685920444, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 7375127, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 7375126, 2023-07-21T19:43:45.750 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 685920376, 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:45.751 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:45.752 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 13804, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 213631575, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 13804, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 213631575, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.753 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 1024, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:45.754 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 13804, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 13804, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 13804, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 13804, 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:45.755 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:45.756 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 33, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 3565, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 33, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 19, 2023-07-21T19:43:45.757 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 3565, 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:45.758 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle-0x556d961440a0": { 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 15882, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 213269885, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 15882, 2023-07-21T19:43:45.759 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 8656, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 213269885, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:45.760 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.761 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:45.761 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-21T19:43:46.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:45 smithi101 ceph-mon[128136]: pgmap v1148: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:46.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:45 smithi005 ceph-mon[126460]: pgmap v1148: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:46.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:45 smithi156 ceph-mon[128316]: pgmap v1148: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:46.243 INFO:tasks.check_counter:Daemon mds.h mds.exported=57 2023-07-21T19:43:46.244 INFO:tasks.check_counter:Daemon mds.h mds.imported=49 2023-07-21T19:43:46.244 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=132 2023-07-21T19:43:46.244 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=108 2023-07-21T19:43:46.244 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-07-21T19:43:46.245 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:43:46.245 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:43:46.245 DEBUG:tasks.check_counter:Getting stats from k 2023-07-21T19:43:46.245 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.k perf dump 2023-07-21T19:43:47.062 INFO:teuthology.orchestra.run.smithi101.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.b/config 2023-07-21T19:43:48.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:47 smithi101 ceph-mon[128136]: pgmap v1149: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:48.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:47 smithi005 ceph-mon[126460]: pgmap v1149: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:48.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:47 smithi156 ceph-mon[128316]: pgmap v1149: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:49.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:49 smithi101 ceph-mon[128136]: pgmap v1150: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:49.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:49 smithi005 ceph-mon[126460]: pgmap v1150: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:49.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:49 smithi156 ceph-mon[128316]: pgmap v1150: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:49.483 INFO:teuthology.orchestra.run.smithi101.stdout:{ 2023-07-21T19:43:49.483 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:49.483 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 1268, 2023-07-21T19:43:49.483 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 1268, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 11916641, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 393562, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 7, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.143178437, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.044480375, 2023-07-21T19:43:49.484 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.046777675, 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.037972347, 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1267, 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.102513926, 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000080910 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.485 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:43:49.486 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 1158, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 1147, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 21343161, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 401470, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 5, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 4, 2023-07-21T19:43:49.487 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.142167861, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.032839993, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.072410765, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.030938918, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 1146, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.113749067, 2023-07-21T19:43:49.488 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000099257 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 264320, 2023-07-21T19:43:49.489 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 111872 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_messages": 298, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages": 440, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_bytes": 8464825, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_bytes": 389433, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_created_connections": 5, 2023-07-21T19:43:49.490 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_active_connections": 4, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_total_time": 0.048754436, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_send_time": 0.013106104, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_recv_time": 0.022681799, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_running_fast_dispatch_time": 0.008967282, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 438, 2023-07-21T19:43:49.491 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.051797956, 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000118260 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.492 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_recv_encrypted_bytes": 160, 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: "msgr_send_encrypted_bytes": 297056 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: "cct": { 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: "total_workers": 1, 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: "unhealthy_workers": 0 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.493 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-MDSRank": { 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 4171, 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.138800692, 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000033277 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.494 INFO:teuthology.orchestra.run.smithi101.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "queue_len": 0, 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "complete_latency": { 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.495 INFO:teuthology.orchestra.run.smithi101.stdout: "mds": { 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "request": 0, 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "reply": 0, 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "reply_latency": { 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.496 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_reply": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "forward": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_complete": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_fetch_keys": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_commit": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_split": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_merge": 0, 2023-07-21T19:43:49.497 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes": 714, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_top": 473, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_bottom": 226, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pin_tail": 15, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_pinned": 38, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_expired": 5, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "inodes_with_caps": 0, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "caps": 0, 2023-07-21T19:43:49.498 INFO:teuthology.orchestra.run.smithi101.stdout: "subtrees": 36, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_hit": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_forward": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_discover": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_remote_ino": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "traverse_lock": 0, 2023-07-21T19:43:49.499 INFO:teuthology.orchestra.run.smithi101.stdout: "load_cent": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "q": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "exported": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "exported_inodes": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "imported": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "imported_inodes": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:49.500 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "openino_peer_discover": 0, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rfiles": 1044, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rbytes": 8966388417, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "root_rsnaps": 6, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:49.501 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "scrub_file_inodes": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:43:49.502 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_cap_release": 0, 2023-07-21T19:43:49.503 INFO:teuthology.orchestra.run.smithi101.stdout: "process_request_cap_release": 0 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_cache": { 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update": 0, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_update_receipt": 0, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_try_discover": 0, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_send_discover": 0, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "dir_handle_discover": 0, 2023-07-21T19:43:49.504 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_created": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_enqueued": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_reintegrated": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "strays_migrated": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:49.505 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_started": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "recovery_completed": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_exportdir": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_flush": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:43:49.506 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "ireq_inodestats": 0 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_log": { 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "evadd": 0, 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "evex": 0, 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "evtrm": 0, 2023-07-21T19:43:49.507 INFO:teuthology.orchestra.run.smithi101.stdout: "ev": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "evexg": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "evexd": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "segadd": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "segex": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "segtrm": 0, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "seg": 2, 2023-07-21T19:43:49.508 INFO:teuthology.orchestra.run.smithi101.stdout: "segexg": 0, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "segexd": 0, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "expos": 103525477, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "wrpos": 0, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "rdpos": 103806532, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "jlat": { 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.509 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: "replayed": 13520 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_mem": { 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: "ino": 330, 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: "ino+": 504, 2023-07-21T19:43:49.510 INFO:teuthology.orchestra.run.smithi101.stdout: "ino-": 174, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dir": 51, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dir+": 60, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dir-": 9, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dn": 714, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dn+": 756, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "dn-": 42, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "cap": 0, 2023-07-21T19:43:49.511 INFO:teuthology.orchestra.run.smithi101.stdout: "cap+": 0, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "cap-": 0, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "rss": 51168, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "heap": 207132 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_server": { 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_client_request": 0, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "dispatch_server_request": 0, 2023-07-21T19:43:49.512 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_request": 0, 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_client_session": 0, 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "handle_peer_request": 0, 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "req_create_latency": { 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.513 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getattr_latency": { 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.514 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: "req_link_latency": { 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.515 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookup_latency": { 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:49.516 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupino_latency": { 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.517 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupname_latency": { 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.518 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.519 INFO:teuthology.orchestra.run.smithi101.stdout: "req_lssnap_latency": { 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mkdir_latency": { 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.520 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mknod_latency": { 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.521 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "req_mksnap_latency": { 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "req_open_latency": { 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.522 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: "req_readdir_latency": { 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.523 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rename_latency": { 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:49.524 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.525 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmdir_latency": { 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:49.526 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.527 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setattr_latency": { 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:49.528 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.529 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setlayout_latency": { 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "req_setxattr_latency": { 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.530 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "req_symlink_latency": { 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.531 INFO:teuthology.orchestra.run.smithi101.stdout: "req_unlink_latency": { 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:49.532 INFO:teuthology.orchestra.run.smithi101.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_sessions": { 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "session_count": 2, 2023-07-21T19:43:49.533 INFO:teuthology.orchestra.run.smithi101.stdout: "session_add": 2, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "session_remove": 0, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_open": 0, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "sessions_stale": 0, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "total_load": 0, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "average_load": 0, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "avg_session_uptime": 4650 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.534 INFO:teuthology.orchestra.run.smithi101.stdout: "mempool": { 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:49.535 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:49.536 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:49.537 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:49.538 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_items": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_bytes": 214375, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_anon_items": 63, 2023-07-21T19:43:49.539 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_bytes": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_items": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:49.540 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_items": 167, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "pgmap_items": 0, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_bytes": 1494313, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "mds_co_items": 38107, 2023-07-21T19:43:49.541 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_1_items": 0, 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "unittest_2_items": 0 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "objecter": { 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "op_active": 0, 2023-07-21T19:43:49.542 INFO:teuthology.orchestra.run.smithi101.stdout: "op_laggy": 0, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send": 2492, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "op_send_bytes": 26, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "op_resend": 3, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "op_reply": 2492, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "op_latency": { 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 2492, 2023-07-21T19:43:49.543 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 4.296910949, 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.001724282 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "op_inflight": 0, 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "oplen_avg": { 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 2492, 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 2493 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.544 INFO:teuthology.orchestra.run.smithi101.stdout: "op": 2492, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "op_r": 2492, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "op_w": 0, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "op_rmw": 0, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "op_pg": 0, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_stat": 1386, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_create": 0, 2023-07-21T19:43:49.545 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_read": 1103, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_write": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writefull": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_writesame": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_append": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_zero": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_truncate": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_delete": 0, 2023-07-21T19:43:49.546 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_mapext": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_getxattr": 2, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_setxattr": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:49.547 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_call": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_watch": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_notify": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "osdop_other": 0, 2023-07-21T19:43:49.548 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_active": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_send": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_resend": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "linger_ping": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_active": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_send": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "poolop_resend": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_active": 0, 2023-07-21T19:43:49.549 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_send": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "poolstat_resend": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_active": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_send": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "statfs_resend": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "command_active": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "command_send": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "command_resend": 0, 2023-07-21T19:43:49.550 INFO:teuthology.orchestra.run.smithi101.stdout: "map_epoch": 149, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "map_full": 0, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "map_inc": 23, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_sessions": 8, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_open": 8, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_session_close": 0, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "osd_laggy": 0, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_wr": 0, 2023-07-21T19:43:49.551 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_rd": 2, 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_del": 0 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "oft": { 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_objs": 0, 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_updates": 0, 2023-07-21T19:43:49.552 INFO:teuthology.orchestra.run.smithi101.stdout: "omap_total_removes": 0 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "osd-slow-ops": { 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "slow_ops_count": 0 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "purge_queue": { 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:43:49.553 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing": 0, 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executing_high_water": 0, 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_executed": 0, 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:49.554 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 2720, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 41465300, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 2720, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:49.555 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 2720, 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 41465300, 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.556 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 104857600, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:49.557 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 2492, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 40764128, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 2492, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 40764128, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.558 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 1024, 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:49.559 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 2492, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 2492, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 2492, 2023-07-21T19:43:49.560 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 2492, 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:49.561 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:49.562 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 0, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 0, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.563 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: }, 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "throttle-write_buf_throttle-0x55ccd2892500": { 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "val": 0, 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "max": 3758096384, 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "get_started": 0, 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "get": 0, 2023-07-21T19:43:49.564 INFO:teuthology.orchestra.run.smithi101.stdout: "get_sum": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "take": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "take_sum": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "put": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "put_sum": 0, 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "wait": { 2023-07-21T19:43:49.565 INFO:teuthology.orchestra.run.smithi101.stdout: "avgcount": 0, 2023-07-21T19:43:49.566 INFO:teuthology.orchestra.run.smithi101.stdout: "sum": 0.000000000, 2023-07-21T19:43:49.566 INFO:teuthology.orchestra.run.smithi101.stdout: "avgtime": 0.000000000 2023-07-21T19:43:49.566 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.566 INFO:teuthology.orchestra.run.smithi101.stdout: } 2023-07-21T19:43:49.566 INFO:teuthology.orchestra.run.smithi101.stdout:} 2023-07-21T19:43:50.013 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2023-07-21T19:43:50.013 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2023-07-21T19:43:50.014 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2023-07-21T19:43:50.014 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=0 2023-07-21T19:43:50.014 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2023-07-21T19:43:50.014 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:43:50.014 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:43:50.015 DEBUG:tasks.check_counter:Getting stats from c 2023-07-21T19:43:50.015 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.c perf dump 2023-07-21T19:43:50.899 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:43:52.046 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:51 smithi156 ceph-mon[128316]: pgmap v1151: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:52.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:51 smithi101 ceph-mon[128136]: pgmap v1151: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:52.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:51 smithi005 ceph-mon[126460]: pgmap v1151: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:52.763 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 5319, 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 5320, 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 77060470, 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1653905, 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 8, 2023-07-21T19:43:52.764 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.598005921, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.173376574, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.226405471, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.148417132, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5318, 2023-07-21T19:43:52.765 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.310772967, 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000058437 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.766 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 2056, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1991, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 65275911, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 771527, 2023-07-21T19:43:52.767 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 5, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.325351727, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.080362776, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.187104160, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.051223001, 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:52.768 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1990, 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.121984259, 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000061298 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.769 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1819856, 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 363744 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1247, 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1711, 2023-07-21T19:43:52.770 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 75176675, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1401768, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 9, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.250849139, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.051052227, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.149123276, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.034815961, 2023-07-21T19:43:52.771 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1709, 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.149634127, 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000087556 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.772 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.773 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 3440, 2023-07-21T19:43:52.773 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 1016128 2023-07-21T19:43:52.773 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:52.774 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13310, 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.360950578, 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000027118 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:52.775 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-07-21T19:43:52.776 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-07-21T19:43:52.777 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 1330, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 480, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 599, 2023-07-21T19:43:52.778 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 251, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 643, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 47, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-07-21T19:43:52.779 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-21T19:43:52.780 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-07-21T19:43:52.781 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:52.782 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:43:52.783 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:43:52.784 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-07-21T19:43:52.785 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-07-21T19:43:52.786 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-07-21T19:43:52.787 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-21T19:43:52.788 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 56, 2023-07-21T19:43:52.789 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 0, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 217463615, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.790 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 15882 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 984, 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 2794, 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 1810, 2023-07-21T19:43:52.791 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 110, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 150, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 40, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 1330, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 4072, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 2742, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-07-21T19:43:52.792 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 56164, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207156 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-07-21T19:43:52.793 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.794 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.795 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.796 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.797 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-21T19:43:52.798 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.799 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-21T19:43:52.800 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.801 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.802 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.803 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.804 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.805 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.806 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:52.807 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.808 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:52.809 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.810 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.811 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.812 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.813 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 2, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 3, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-07-21T19:43:52.814 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 4729 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:52.815 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:52.816 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:52.817 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:52.818 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:52.819 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 242618, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 88, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-21T19:43:52.820 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:52.821 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 3336241, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 7452, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-21T19:43:52.822 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 7869, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 7869, 2023-07-21T19:43:52.823 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7869, 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 9.934864201, 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001262531 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-21T19:43:52.824 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7869, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7870 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 7869, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 7869, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-21T19:43:52.825 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 4516, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 3351, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-21T19:43:52.826 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:52.827 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-21T19:43:52.828 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-21T19:43:52.829 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-21T19:43:52.830 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 149, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 64, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 11, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 11, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-07-21T19:43:52.831 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:52.832 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-21T19:43:52.833 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:52.834 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 8618, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 216696484, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 8618, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:52.835 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 8618, 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 216696484, 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.836 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:52.837 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 7869, 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 213504128, 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 7869, 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 213504128, 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:52.838 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:52.839 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:52.840 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 7869, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 7869, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 7869, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 7869, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.841 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:52.842 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-21T19:43:52.843 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-21T19:43:52.844 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:52.844 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.844 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.844 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.844 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.845 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:52.845 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5616e5a9a500": { 2023-07-21T19:43:52.845 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:52.845 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:52.845 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:52.846 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:52.847 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-21T19:43:53.276 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-07-21T19:43:53.276 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-07-21T19:43:53.277 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-07-21T19:43:53.277 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-07-21T19:43:53.277 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-07-21T19:43:53.277 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:43:53.277 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:43:53.277 DEBUG:tasks.check_counter:Getting stats from f 2023-07-21T19:43:53.278 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.f perf dump 2023-07-21T19:43:54.047 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:43:54.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:53 smithi101 ceph-mon[128136]: pgmap v1152: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-07-21T19:43:54.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:53 smithi005 ceph-mon[126460]: pgmap v1152: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-07-21T19:43:54.152 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:53 smithi156 ceph-mon[128316]: pgmap v1152: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 7 op/s 2023-07-21T19:43:55.848 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-21T19:43:55.848 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:55.849 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1580, 2023-07-21T19:43:55.849 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1580, 2023-07-21T19:43:55.849 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 13252997, 2023-07-21T19:43:55.849 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 490648, 2023-07-21T19:43:55.849 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 9, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.177495898, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.045808411, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.066398700, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.045754027, 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:55.850 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1579, 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.094189057, 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000059651 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.851 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1681, 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 2148, 2023-07-21T19:43:55.852 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 21399632, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1537719, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 10, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.219697717, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.073380508, 2023-07-21T19:43:55.853 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.073214771, 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.050907965, 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 2145, 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.114237862, 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000053257 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.854 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T19:43:55.855 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 1015856 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 5428, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 5363, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 24166280, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 1820438, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 5, 2023-07-21T19:43:55.856 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 4, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.597416339, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.197069052, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.206294073, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.153419897, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 5362, 2023-07-21T19:43:55.857 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.335829872, 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000062631 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.858 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1820128, 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 364336 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-21T19:43:55.859 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-21T19:43:55.860 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-21T19:43:55.860 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.860 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-21T19:43:55.860 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:55.860 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 13388, 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.381982899, 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000028531 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.861 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.862 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 0, 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 0, 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.863 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-21T19:43:55.864 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 929, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 125, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 577, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 227, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 750, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 0, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 0, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 0, 2023-07-21T19:43:55.865 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 13, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 0, 2023-07-21T19:43:55.866 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:55.867 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:55.868 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 0, 2023-07-21T19:43:55.869 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 0, 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 0 2023-07-21T19:43:55.870 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-21T19:43:55.871 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:55.872 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:55.873 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 0, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 0, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-21T19:43:55.874 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 26, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-21T19:43:55.875 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 0, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 58977376, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.876 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 20676 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 403, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 2464, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 2061, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 32, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 83, 2023-07-21T19:43:55.877 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 51, 2023-07-21T19:43:55.878 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 929, 2023-07-21T19:43:55.878 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 3425, 2023-07-21T19:43:55.878 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 2496, 2023-07-21T19:43:55.878 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 0, 2023-07-21T19:43:55.878 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 0, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 0, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 54972, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 207156 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 0, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 0, 2023-07-21T19:43:55.879 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 0, 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 0, 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 0, 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.880 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.881 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.882 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.883 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-21T19:43:55.884 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.885 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-21T19:43:55.886 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.887 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-21T19:43:55.888 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.889 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.890 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.891 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.892 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.893 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.894 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:55.895 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.896 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-21T19:43:55.897 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.898 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.899 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 2, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-21T19:43:55.900 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 0, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 0, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 7286 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:55.901 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:55.902 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:55.903 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:55.904 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:55.905 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 224849, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 75, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-21T19:43:55.906 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:55.907 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 1640629, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 20594, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-21T19:43:55.908 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 7936, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 21, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 3, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 7936, 2023-07-21T19:43:55.909 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7936, 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7.051903703, 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000888596 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-21T19:43:55.910 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7936, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 7937 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 7936, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 7936, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 0, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-21T19:43:55.911 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 4478, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 3456, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-21T19:43:55.912 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 1, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:55.913 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:55.914 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-21T19:43:55.915 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-21T19:43:55.916 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 149, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 64, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 9, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 3, 2023-07-21T19:43:55.917 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 0, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 2, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 0 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 0, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:55.918 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 0 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:55.919 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 0, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 0, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 0, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:55.920 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 8685, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 57996255, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 8685, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:55.921 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 8685, 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 57996255, 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.922 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:55.923 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 7936, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 54796753, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 7936, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 54796753, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.924 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:55.925 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 7936, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 7936, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 7936, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 7936, 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:55.926 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:55.927 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-21T19:43:55.928 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:55.929 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x56439287e500": { 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:55.930 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 0, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 0, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:55.931 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.932 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:55.932 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-21T19:43:56.046 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:55 smithi156 ceph-mon[128316]: pgmap v1153: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:56.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:55 smithi101 ceph-mon[128136]: pgmap v1153: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:56.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:55 smithi005 ceph-mon[126460]: pgmap v1153: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-07-21T19:43:56.331 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-07-21T19:43:56.332 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-07-21T19:43:56.332 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-07-21T19:43:56.332 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-07-21T19:43:56.332 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-07-21T19:43:56.333 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:43:56.333 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:43:56.333 DEBUG:tasks.check_counter:Getting stats from i 2023-07-21T19:43:56.333 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.i perf dump 2023-07-21T19:43:57.067 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:43:57.903 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:57 smithi156 ceph-mon[128316]: pgmap v1154: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:58.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:57 smithi101 ceph-mon[128136]: pgmap v1154: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:58.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:57 smithi005 ceph-mon[126460]: pgmap v1154: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:43:58.645 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-21T19:43:58.646 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:43:58.646 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1986363, 2023-07-21T19:43:58.646 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1982241, 2023-07-21T19:43:58.646 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 404751647, 2023-07-21T19:43:58.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 421819427, 2023-07-21T19:43:58.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 15, 2023-07-21T19:43:58.647 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-07-21T19:43:58.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 105.725052287, 2023-07-21T19:43:58.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 45.900930666, 2023-07-21T19:43:58.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 69.933887633, 2023-07-21T19:43:58.648 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.436130240, 2023-07-21T19:43:58.649 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:58.649 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1982241, 2023-07-21T19:43:58.649 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 52.110942725, 2023-07-21T19:43:58.649 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000026288 2023-07-21T19:43:58.650 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.650 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:58.650 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3802626, 2023-07-21T19:43:58.650 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.573801674, 2023-07-21T19:43:58.651 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000150 2023-07-21T19:43:58.651 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.651 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 0, 2023-07-21T19:43:58.652 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 0 2023-07-21T19:43:58.652 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.652 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:43:58.652 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 579945, 2023-07-21T19:43:58.653 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 884770, 2023-07-21T19:43:58.653 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 183599683, 2023-07-21T19:43:58.653 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 395475240, 2023-07-21T19:43:58.653 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 11, 2023-07-21T19:43:58.654 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:58.654 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 46.781956229, 2023-07-21T19:43:58.654 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 21.372062037, 2023-07-21T19:43:58.655 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 27.612583307, 2023-07-21T19:43:58.655 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.159468205, 2023-07-21T19:43:58.655 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:58.655 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 884769, 2023-07-21T19:43:58.656 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 33.520238211, 2023-07-21T19:43:58.656 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000037885 2023-07-21T19:43:58.656 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.656 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1329141, 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.288833771, 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000217 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1833488, 2023-07-21T19:43:58.657 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 389792 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 1946357, 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 1938018, 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 327731578, 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 321768233, 2023-07-21T19:43:58.658 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 48, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 6, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 82.093763442, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 35.531267719, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 53.742998520, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.092946699, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1938014, 2023-07-21T19:43:58.659 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 27.767134021, 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000014327 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3660462, 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.725090312, 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000198 2023-07-21T19:43:58.660 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 5088, 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 1039072 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 1, 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.661 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-MDSRank": { 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 40207, 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 12.410114399, 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000308655 2023-07-21T19:43:58.662 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.663 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.663 INFO:teuthology.orchestra.run.smithi156.stdout: "finisher-PurgeQueue": { 2023-07-21T19:43:58.663 INFO:teuthology.orchestra.run.smithi156.stdout: "queue_len": 0, 2023-07-21T19:43:58.663 INFO:teuthology.orchestra.run.smithi156.stdout: "complete_latency": { 2023-07-21T19:43:58.663 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 617, 2023-07-21T19:43:58.664 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.024734606, 2023-07-21T19:43:58.664 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000040088 2023-07-21T19:43:58.664 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.665 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.665 INFO:teuthology.orchestra.run.smithi156.stdout: "mds": { 2023-07-21T19:43:58.665 INFO:teuthology.orchestra.run.smithi156.stdout: "request": 231706, 2023-07-21T19:43:58.665 INFO:teuthology.orchestra.run.smithi156.stdout: "reply": 228467, 2023-07-21T19:43:58.666 INFO:teuthology.orchestra.run.smithi156.stdout: "reply_latency": { 2023-07-21T19:43:58.666 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 228467, 2023-07-21T19:43:58.666 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 552.308298763, 2023-07-21T19:43:58.667 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002417453 2023-07-21T19:43:58.667 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.667 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_reply": 0, 2023-07-21T19:43:58.667 INFO:teuthology.orchestra.run.smithi156.stdout: "forward": 3239, 2023-07-21T19:43:58.668 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_complete": 11, 2023-07-21T19:43:58.668 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_fetch_keys": 0, 2023-07-21T19:43:58.668 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_commit": 11, 2023-07-21T19:43:58.668 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_split": 0, 2023-07-21T19:43:58.668 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_merge": 0, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes": 861, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_top": 9, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_bottom": 674, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pin_tail": 178, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_pinned": 847, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_expired": 3047, 2023-07-21T19:43:58.669 INFO:teuthology.orchestra.run.smithi156.stdout: "inodes_with_caps": 250, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "caps": 250, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "subtrees": 48, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse": 479078, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_hit": 469899, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_forward": 3181, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_discover": 274, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_dir_fetch": 5, 2023-07-21T19:43:58.670 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_remote_ino": 1, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "traverse_lock": 2880, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "load_cent": 0, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "q": 0, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "exported": 27, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "exported_inodes": 2443, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "imported": 49, 2023-07-21T19:43:58.671 INFO:teuthology.orchestra.run.smithi156.stdout: "imported_inodes": 2642, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_dir_fetch": 0, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_backtrace_fetch": 14112, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "openino_peer_discover": 1, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rfiles": 1044, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rbytes": 8918784409, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "root_rsnaps": 6, 2023-07-21T19:43:58.672 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_fetch": 14112, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_set_tag": 0, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_backtrace_repaired": 0, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_inotable_repaired": 0, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_inodes": 75, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dir_base_inodes": 0, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_dirfrag_rstats": 75, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "scrub_file_inodes": 14037, 2023-07-21T19:43:58.673 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_inode_file_caps": 336, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_revoke": 335823, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_grant": 291094, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_trunc": 0, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "ceph_cap_op_flush_ack": 0, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps": 339740, 2023-07-21T19:43:58.674 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_caps_dirty": 14444, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_cap_release": 327, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "process_request_cap_release": 910 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_cache": { 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update": 10, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_update_receipt": 346, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_try_discover": 194, 2023-07-21T19:43:58.675 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_send_discover": 549, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "dir_handle_discover": 166, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays": 0, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_delayed": 0, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "num_strays_enqueuing": 0, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_created": 234, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_enqueued": 109, 2023-07-21T19:43:58.676 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_reintegrated": 628, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "strays_migrated": 70, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_processing": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_enqueued": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "num_recovering_prioritized": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_started": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "recovery_completed": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_enqueue_scrub": 0, 2023-07-21T19:43:58.677 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_exportdir": 33, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_flush": 0, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragmentdir": 0, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_fragstats": 0, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "ireq_inodestats": 0 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_log": { 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "evadd": 20676, 2023-07-21T19:43:58.678 INFO:teuthology.orchestra.run.smithi156.stdout: "evex": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "evtrm": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "ev": 20676, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "evexg": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "evexd": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "segadd": 26, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "segex": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "segtrm": 0, 2023-07-21T19:43:58.679 INFO:teuthology.orchestra.run.smithi156.stdout: "seg": 26, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "segexg": 0, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "segexd": 0, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "expos": 4194304, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "wrpos": 58984438, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "rdpos": 0, 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "jlat": { 2023-07-21T19:43:58.680 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 14901, 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 53.814157135, 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.003611446 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: "replayed": 0 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_mem": { 2023-07-21T19:43:58.681 INFO:teuthology.orchestra.run.smithi156.stdout: "ino": 446, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "ino+": 2451, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "ino-": 2005, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir": 116, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir+": 289, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dir-": 173, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dn": 861, 2023-07-21T19:43:58.682 INFO:teuthology.orchestra.run.smithi156.stdout: "dn+": 4018, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "dn-": 3157, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "cap": 250, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "cap+": 1846, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "cap-": 1596, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "rss": 72196, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "heap": 198964 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.683 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_server": { 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_client_request": 394562, 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "dispatch_server_request": 5124, 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_request": 231706, 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_client_session": 125, 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "handle_peer_request": 11441, 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "req_create_latency": { 2023-07-21T19:43:58.684 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 123, 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.252769719, 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002055038 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getattr_latency": { 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 216134, 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 246.217818179, 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.001139190 2023-07-21T19:43:58.685 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getfilelock_latency": { 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "req_link_latency": { 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 125, 2023-07-21T19:43:58.686 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.213480383, 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.009707843 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookup_latency": { 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 7714, 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 146.826191917, 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.019033729 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.687 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookuphash_latency": { 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupino_latency": { 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.688 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupname_latency": { 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.689 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupparent_latency": { 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lookupsnap_latency": { 2023-07-21T19:43:58.690 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "req_lssnap_latency": { 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.691 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mkdir_latency": { 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 1, 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.002117458, 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002117458 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.692 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mknod_latency": { 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "req_mksnap_latency": { 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.693 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: "req_open_latency": { 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3399, 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 1.326279180, 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000390196 2023-07-21T19:43:58.694 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_readdir_latency": { 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 39, 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.024599047, 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000630744 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rename_latency": { 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 805, 2023-07-21T19:43:58.695 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 155.843315863, 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.193594181 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: "req_renamesnap_latency": { 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.696 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_snapdiff_latency": { 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmdir_latency": { 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.697 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmsnap_latency": { 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.698 INFO:teuthology.orchestra.run.smithi156.stdout: "req_rmxattr_latency": { 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setattr_latency": { 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.699 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setdirlayout_latency": { 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.700 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setfilelock_latency": { 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setlayout_latency": { 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.701 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.702 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.702 INFO:teuthology.orchestra.run.smithi156.stdout: "req_setxattr_latency": { 2023-07-21T19:43:58.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.702 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.702 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: "req_symlink_latency": { 2023-07-21T19:43:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.703 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.704 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.704 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.704 INFO:teuthology.orchestra.run.smithi156.stdout: "req_unlink_latency": { 2023-07-21T19:43:58.705 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 127, 2023-07-21T19:43:58.705 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.601727017, 2023-07-21T19:43:58.705 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.004738008 2023-07-21T19:43:58.705 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.706 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_revoke_eviction": 0, 2023-07-21T19:43:58.706 INFO:teuthology.orchestra.run.smithi156.stdout: "cap_acquisition_throttle": 0, 2023-07-21T19:43:58.706 INFO:teuthology.orchestra.run.smithi156.stdout: "req_getvxattr_latency": { 2023-07-21T19:43:58.706 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.707 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.707 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.707 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.708 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.708 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_sessions": { 2023-07-21T19:43:58.708 INFO:teuthology.orchestra.run.smithi156.stdout: "session_count": 1, 2023-07-21T19:43:58.709 INFO:teuthology.orchestra.run.smithi156.stdout: "session_add": 2, 2023-07-21T19:43:58.709 INFO:teuthology.orchestra.run.smithi156.stdout: "session_remove": 1, 2023-07-21T19:43:58.709 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_open": 1, 2023-07-21T19:43:58.710 INFO:teuthology.orchestra.run.smithi156.stdout: "sessions_stale": 0, 2023-07-21T19:43:58.710 INFO:teuthology.orchestra.run.smithi156.stdout: "total_load": 19024, 2023-07-21T19:43:58.710 INFO:teuthology.orchestra.run.smithi156.stdout: "average_load": 19024, 2023-07-21T19:43:58.710 INFO:teuthology.orchestra.run.smithi156.stdout: "avg_session_uptime": 2200 2023-07-21T19:43:58.711 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.711 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-21T19:43:58.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:43:58.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-21T19:43:58.711 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:43:58.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:43:58.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:43:58.712 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:43:58.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:43:58.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:43:58.713 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:43:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:43:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:43:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:43:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:43:58.714 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:43:58.715 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:43:58.716 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:43:58.717 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:43:58.718 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:43:58.718 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 611471, 2023-07-21T19:43:58.718 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 194, 2023-07-21T19:43:58.718 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:43:58.718 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:43:58.719 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 8032, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 167, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 2068052, 2023-07-21T19:43:58.720 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 38381, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "objecter": { 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "op_active": 0, 2023-07-21T19:43:58.721 INFO:teuthology.orchestra.run.smithi156.stdout: "op_laggy": 0, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send": 30010, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_send_bytes": 54978112, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_resend": 26, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_reply": 30010, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "op_latency": { 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 30010, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 65.088122590, 2023-07-21T19:43:58.722 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.002168881 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: "op_inflight": 0, 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: "oplen_avg": { 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 30010, 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 30458 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.723 INFO:teuthology.orchestra.run.smithi156.stdout: "op": 30010, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "op_r": 14206, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "op_w": 15804, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "op_rmw": 0, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "op_pg": 0, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_stat": 0, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_create": 10, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_read": 47, 2023-07-21T19:43:58.724 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_write": 14948, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writefull": 183, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_writesame": 0, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_append": 0, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_zero": 0, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_truncate": 0, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_delete": 372, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_mapext": 0, 2023-07-21T19:43:58.725 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_sparse_read": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_clonerange": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_getxattr": 14112, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_setxattr": 20, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_cmpxattr": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_rmxattr": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_resetxattrs": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_call": 0, 2023-07-21T19:43:58.726 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_watch": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_notify": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_src_cmpxattr": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_pgls_filter": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "osdop_other": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_active": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_send": 0, 2023-07-21T19:43:58.727 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_resend": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "linger_ping": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_active": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_send": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolop_resend": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_active": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_send": 0, 2023-07-21T19:43:58.728 INFO:teuthology.orchestra.run.smithi156.stdout: "poolstat_resend": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_active": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_send": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "statfs_resend": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "command_active": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "command_send": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "command_resend": 0, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "map_epoch": 149, 2023-07-21T19:43:58.729 INFO:teuthology.orchestra.run.smithi156.stdout: "map_full": 0, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "map_inc": 64, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_sessions": 12, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_open": 12, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_session_close": 0, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_laggy": 0, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_wr": 514, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_rd": 58, 2023-07-21T19:43:58.730 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_del": 194 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "oft": { 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_objs": 1, 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_kv_pairs": 0, 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_updates": 0, 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "omap_total_removes": 0 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.731 INFO:teuthology.orchestra.run.smithi156.stdout: "osd-slow-ops": { 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "slow_ops_count": 233604 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "purge_queue": { 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops": 0, 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_ops_high_water": 240, 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing": 0, 2023-07-21T19:43:58.732 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executing_high_water": 8, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_executed": 109, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "pq_item_in_journal": 0 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:58.733 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 4512661, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 509899112, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 4512661, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 4512661, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 509899112, 2023-07-21T19:43:58.734 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_bytes": { 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:58.735 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 30010, 2023-07-21T19:43:58.736 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 54988479, 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 30010, 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 54988479, 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.737 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-objecter_ops": { 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 1024, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 0, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 0, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:58.738 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 0, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 30010, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 30010, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 30010, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 30010, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.739 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle": { 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:58.740 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 109, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 11521, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 109, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 47, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 11521, 2023-07-21T19:43:58.741 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-write_buf_throttle-0x5562185620a0": { 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:43:58.742 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 3758096384, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 20676, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 54790134, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 20676, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:43:58.743 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 14901, 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 54790134, 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.744 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:43:58.745 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-21T19:43:59.159 INFO:tasks.check_counter:Daemon mds.i mds.exported=27 2023-07-21T19:43:59.159 INFO:tasks.check_counter:Daemon mds.i mds.imported=49 2023-07-21T19:43:59.159 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=10 2023-07-21T19:43:59.159 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=346 2023-07-21T19:43:59.160 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-07-21T19:43:59.160 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-07-21T19:43:59.160 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-07-21T19:43:59.160 DEBUG:tasks.check_counter:Getting stats from l 2023-07-21T19:43:59.160 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:81cf785aa7781c4e08381c540f890cb9bc987ef9 shell --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c -- ceph daemon mds.l perf dump 2023-07-21T19:43:59.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:43:59 smithi101 ceph-mon[128136]: pgmap v1155: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:59.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:43:59 smithi005 ceph-mon[126460]: pgmap v1155: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:59.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:43:59 smithi156 ceph-mon[128316]: pgmap v1155: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2023-07-21T19:43:59.883 INFO:teuthology.orchestra.run.smithi156.stderr:Inferring config /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/mon.c/config 2023-07-21T19:44:01.455 INFO:teuthology.orchestra.run.smithi156.stdout:{ 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-0": { 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 630, 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 620, 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 1539744, 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 356288, 2023-07-21T19:44:01.456 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 4, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 0, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.112104601, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.044582683, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.050847575, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.015033209, 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:44:01.457 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 619, 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.040122421, 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000064818 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:44:01.458 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 1539744, 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 356288 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-1": { 2023-07-21T19:44:01.459 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 3, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 474, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 480, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 280896, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 6, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 3, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.030241615, 2023-07-21T19:44:01.460 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.021180170, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.002576670, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 471, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.049440561, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000104969 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.461 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 480, 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 280896 2023-07-21T19:44:01.462 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "AsyncMessenger::Worker-2": { 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_messages": 4, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages": 4, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_bytes": 2960, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_bytes": 448, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_created_connections": 2, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_active_connections": 1, 2023-07-21T19:44:01.463 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_total_time": 0.001115705, 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_send_time": 0.000064301, 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_recv_time": 0.001005222, 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_messages_queue_lat": { 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 3, 2023-07-21T19:44:01.464 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.001186006, 2023-07-21T19:44:01.465 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000395335 2023-07-21T19:44:01.466 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.466 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_handle_ack_lat": { 2023-07-21T19:44:01.466 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:44:01.466 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:44:01.466 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_recv_encrypted_bytes": 2960, 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: "msgr_send_encrypted_bytes": 448 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: "cct": { 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: "total_workers": 0, 2023-07-21T19:44:01.467 INFO:teuthology.orchestra.run.smithi156.stdout: "unhealthy_workers": 0 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "mempool": { 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_bytes": 0, 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "bloom_filter_items": 0, 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_bytes": 0, 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_alloc_items": 0, 2023-07-21T19:44:01.468 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_bytes": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_data_items": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_bytes": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_onode_items": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_bytes": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_meta_items": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_bytes": 0, 2023-07-21T19:44:01.469 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_other_items": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_bytes": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_cache_buffer_items": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_bytes": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_extent_items": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_bytes": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_blob_items": 0, 2023-07-21T19:44:01.470 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_bytes": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_shared_blob_items": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_bytes": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_inline_bl_items": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_bytes": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_fsck_items": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_bytes": 0, 2023-07-21T19:44:01.471 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_txc_items": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_bytes": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_deferred_items": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_bytes": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluestore_writing_items": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_bytes": 0, 2023-07-21T19:44:01.472 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_items": 0, 2023-07-21T19:44:01.473 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_bytes": 0, 2023-07-21T19:44:01.473 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_reader_items": 0, 2023-07-21T19:44:01.473 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_bytes": 0, 2023-07-21T19:44:01.473 INFO:teuthology.orchestra.run.smithi156.stdout: "bluefs_file_writer_items": 0, 2023-07-21T19:44:01.473 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_bytes": 148731, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_anon_items": 31, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_bytes": 0, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "buffer_meta_items": 0, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_bytes": 0, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_items": 0, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_bytes": 0, 2023-07-21T19:44:01.474 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_mapbl_items": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_bytes": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osd_pglog_items": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_bytes": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_items": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_bytes": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "osdmap_mapping_items": 0, 2023-07-21T19:44:01.475 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_bytes": 0, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "pgmap_items": 0, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_bytes": 928, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "mds_co_items": 2, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_bytes": 0, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_1_items": 0, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_bytes": 0, 2023-07-21T19:44:01.476 INFO:teuthology.orchestra.run.smithi156.stdout: "unittest_2_items": 0 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: }, 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "val": 0, 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "max": 104857600, 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "get_started": 0, 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "get": 633, 2023-07-21T19:44:01.477 INFO:teuthology.orchestra.run.smithi156.stdout: "get_sum": 1450625, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_fail": 0, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "get_or_fail_success": 633, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "take": 0, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "take_sum": 0, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "put": 633, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "put_sum": 1450625, 2023-07-21T19:44:01.478 INFO:teuthology.orchestra.run.smithi156.stdout: "wait": { 2023-07-21T19:44:01.479 INFO:teuthology.orchestra.run.smithi156.stdout: "avgcount": 0, 2023-07-21T19:44:01.479 INFO:teuthology.orchestra.run.smithi156.stdout: "sum": 0.000000000, 2023-07-21T19:44:01.479 INFO:teuthology.orchestra.run.smithi156.stdout: "avgtime": 0.000000000 2023-07-21T19:44:01.479 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:44:01.480 INFO:teuthology.orchestra.run.smithi156.stdout: } 2023-07-21T19:44:01.480 INFO:teuthology.orchestra.run.smithi156.stdout:} 2023-07-21T19:44:01.957 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-07-21T19:44:01.957 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-07-21T19:44:01.957 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-07-21T19:44:01.958 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-07-21T19:44:01.958 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-07-21T19:44:01.958 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-07-21T19:44:01.958 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-07-21T19:44:01.959 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-07-21T19:44:01.969 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-07-21T19:44:01.969 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T19:44:01.970 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:44:02.029 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T19:44:02.030 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:44:02.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:01 smithi101 ceph-mon[128136]: pgmap v1156: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:44:02.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:01 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:44:02.114 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:44:02.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:01 smithi005 ceph-mon[126460]: pgmap v1156: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:44:02.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:01 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:44:02.152 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:01 smithi156 ceph-mon[128316]: pgmap v1156: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-07-21T19:44:02.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:01 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-07-21T19:44:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 -- 172.21.15.5:0/1820906978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 msgr2=0x7f65f4104740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 --2- 172.21.15.5:0/1820906978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4104740 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f65e40099c0 tx=0x7f65e4009cd0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.348 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 -- 172.21.15.5:0/1820906978 shutdown_connections 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 --2- 172.21.15.5:0/1820906978 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f4104e20 0x7f65f410c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 --2- 172.21.15.5:0/1820906978 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f40ffde0 0x7f65f41001b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 --2- 172.21.15.5:0/1820906978 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4104740 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 -- 172.21.15.5:0/1820906978 >> 172.21.15.5:0/1820906978 conn(0x7f65f40fb690 msgr2=0x7f65f40fdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 -- 172.21.15.5:0/1820906978 shutdown_connections 2023-07-21T19:44:02.349 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.365+0000 7f65fbfff700 1 -- 172.21.15.5:0/1820906978 wait complete. 2023-07-21T19:44:02.350 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.366+0000 7f65fbfff700 1 Processor -- start 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.366+0000 7f65fbfff700 1 -- start start 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.366+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f40ffde0 0x7f65f4195cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.366+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.366+0000 7f65fbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f4104e20 0x7f65f41995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65fbfff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f4199bc0 con 0x7f65f4100780 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65fbfff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f4199d30 con 0x7f65f4104e20 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65fbfff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65f4199ef0 con 0x7f65f40ffde0 2023-07-21T19:44:02.351 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33488/0 (socket says 172.21.15.5:33488) 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 -- 172.21.15.5:0/3193771485 learned_addr learned my addr 172.21.15.5:0/3193771485 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65fa59c700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f4104e20 0x7f65f41995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f9d9b700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f40ffde0 0x7f65f4195cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 -- 172.21.15.5:0/3193771485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f40ffde0 msgr2=0x7f65f4195cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f40ffde0 0x7f65f4195cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.352 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 -- 172.21.15.5:0/3193771485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f4104e20 msgr2=0x7f65f41995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f4104e20 0x7f65f41995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 -- 172.21.15.5:0/3193771485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40095f0 con 0x7f65f4100780 2023-07-21T19:44:02.353 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.367+0000 7f65f959a700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.368+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e401d070 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.368+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f65e400fe70 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.368+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65f419a170 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.368+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65e40173f0 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.368+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65f41a2b30 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.369+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65f4067750 con 0x7f65f4100780 2023-07-21T19:44:02.354 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.371+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f65e4022650 con 0x7f65f4100780 2023-07-21T19:44:02.356 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.373+0000 7f65eaffd700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 0x7f65e00a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.374+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f65e40ce090 con 0x7f65f4100780 2023-07-21T19:44:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.374+0000 7f65f9d9b700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 0x7f65e00a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.357 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.374+0000 7f65f9d9b700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 0x7f65e00a3830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009c60 tx=0x7f65f0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:02.359 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.376+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f65e40974d0 con 0x7f65f4100780 2023-07-21T19:44:02.575 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.591+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f65f41a2cc0 con 0x7f65f4100780 2023-07-21T19:44:02.576 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.593+0000 7f65eaffd700 1 -- 172.21.15.5:0/3193771485 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+22992 (secure 0 0 0) 0x7f65e409aaf0 con 0x7f65f4100780 2023-07-21T19:44:02.577 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:44:02.577 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":149,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:43:23.036392+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":67,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3558708925":"2023-07-22T19:07:36.084989+0000","172.21.15.5:6801/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/2496466053":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4257470084":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/1985166852":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6800/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":54,"length":1},{"begin":67,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:44:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 msgr2=0x7f65e00a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 0x7f65e00a3830 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f65f0009c60 tx=0x7f65f0009260 comp rx=0 tx=0).stop 2023-07-21T19:44:02.579 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 msgr2=0x7f65f4196210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f65e4000c00 tx=0x7f65e4005f10 comp rx=0 tx=0).stop 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 shutdown_connections 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f65f4104e20 0x7f65f41995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f65f40ffde0 0x7f65f4195cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f65e00a1380 0x7f65e00a3830 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 --2- 172.21.15.5:0/3193771485 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f65f4100780 0x7f65f4196210 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 >> 172.21.15.5:0/3193771485 conn(0x7f65f40fb690 msgr2=0x7f65f40fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:02.580 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 shutdown_connections 2023-07-21T19:44:02.581 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.596+0000 7f65fbfff700 1 -- 172.21.15.5:0/3193771485 wait complete. 2023-07-21T19:44:02.598 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 67, 'snap_epoch': 148, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:44:02.599 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:44:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 -- 172.21.15.5:0/670620759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 msgr2=0x7f20d4072c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 --2- 172.21.15.5:0/670620759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4072c80 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009990 tx=0x7f20c4009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.810 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 -- 172.21.15.5:0/670620759 shutdown_connections 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 --2- 172.21.15.5:0/670620759 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d410e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 --2- 172.21.15.5:0/670620759 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4072c80 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 --2- 172.21.15.5:0/670620759 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d410bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 -- 172.21.15.5:0/670620759 >> 172.21.15.5:0/670620759 conn(0x7f20d40fb5e0 msgr2=0x7f20d40fd9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 -- 172.21.15.5:0/670620759 shutdown_connections 2023-07-21T19:44:02.811 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.827+0000 7f20d9dff700 1 -- 172.21.15.5:0/670620759 wait complete. 2023-07-21T19:44:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 Processor -- start 2023-07-21T19:44:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 -- start start 2023-07-21T19:44:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d41982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.813 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d419c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419cd50 con 0x7f20d40728b0 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419cec0 con 0x7f20d410c320 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.828+0000 7f20d9dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d419d080 con 0x7f20d40731c0 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33502/0 (socket says 172.21.15.5:33502) 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 -- 172.21.15.5:0/207013602 learned_addr learned my addr 172.21.15.5:0/207013602 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:02.814 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d3fff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d419c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d2ffd700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d41982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 -- 172.21.15.5:0/207013602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 msgr2=0x7f20d41982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d41982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 -- 172.21.15.5:0/207013602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 msgr2=0x7f20d419c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d419c710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 -- 172.21.15.5:0/207013602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c40095f0 con 0x7f20d40728b0 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d2ffd700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:44:02.815 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d3fff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:44:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.829+0000 7f20d37fe700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005740 tx=0x7f20c4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.830+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c401d070 con 0x7f20d40728b0 2023-07-21T19:44:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.830+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d419d300 con 0x7f20d40728b0 2023-07-21T19:44:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.830+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f20c4022470 con 0x7f20d40728b0 2023-07-21T19:44:02.816 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.830+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20c4017650 con 0x7f20d40728b0 2023-07-21T19:44:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.830+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d41a4ba0 con 0x7f20d40728b0 2023-07-21T19:44:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.831+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d4107530 con 0x7f20d40728b0 2023-07-21T19:44:02.817 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.833+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f20c400fb00 con 0x7f20d40728b0 2023-07-21T19:44:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.834+0000 7f20d0ff9700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 0x7f20bc0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.834+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f20c40ce6a0 con 0x7f20d40728b0 2023-07-21T19:44:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.835+0000 7f20d2ffd700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 0x7f20bc0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:02.818 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.835+0000 7f20d2ffd700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 0x7f20bc0a37f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c8005e30 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:02.819 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:02.836+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f20c4097ae0 con 0x7f20d40728b0 2023-07-21T19:44:02.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:44:02.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:02 smithi005 ceph-mon[126460]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:44:02.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:02 smithi005 ceph-mon[126460]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:44:02.891 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:02 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3193771485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:03.030 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.047+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f20d41991c0 con 0x7f20d40728b0 2023-07-21T19:44:03.031 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.048+0000 7f20d0ff9700 1 -- 172.21.15.5:0/207013602 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7f20c409b100 con 0x7f20d40728b0 2023-07-21T19:44:03.031 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.050+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 msgr2=0x7f20bc0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.050+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 0x7f20bc0a37f0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f20c8005fd0 tx=0x7f20c8005e30 comp rx=0 tx=0).stop 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.050+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 msgr2=0x7f20d4197d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.050+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f20c4005740 tx=0x7f20c4005770 comp rx=0 tx=0).stop 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.050+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 shutdown_connections 2023-07-21T19:44:03.034 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f20d410c320 0x7f20d419c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f20bc0a1340 0x7f20bc0a37f0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f20d40728b0 0x7f20d4197d70 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 --2- 172.21.15.5:0/207013602 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f20d40731c0 0x7f20d41982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 >> 172.21.15.5:0/207013602 conn(0x7f20d40fb5e0 msgr2=0x7f20d410e480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 shutdown_connections 2023-07-21T19:44:03.035 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.051+0000 7f20d9dff700 1 -- 172.21.15.5:0/207013602 wait complete. 2023-07-21T19:44:03.052 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:44:03.063 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:44:03.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:02 smithi101 ceph-mon[128136]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:44:03.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:02 smithi101 ceph-mon[128136]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:44:03.064 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:02 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3193771485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:03.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-07-21T19:44:03.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:02 smithi156 ceph-mon[128316]: from='mgr.25261 172.21.15.5:0/1116835857' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-07-21T19:44:03.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:02 smithi156 ceph-mon[128316]: from='mgr.25261 ' entity='mgr.x' 2023-07-21T19:44:03.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:02 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3193771485' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 -- 172.21.15.5:0/1113969562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada0105ae0 msgr2=0x7fada0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 --2- 172.21.15.5:0/1113969562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada0105ae0 0x7fada0105eb0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fad900098f0 tx=0x7fad90009c00 comp rx=0 tx=0).stop 2023-07-21T19:44:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 -- 172.21.15.5:0/1113969562 shutdown_connections 2023-07-21T19:44:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 --2- 172.21.15.5:0/1113969562 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0100480 0x7fada010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.260 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 --2- 172.21.15.5:0/1113969562 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada0105ae0 0x7fada0105eb0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 --2- 172.21.15.5:0/1113969562 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada00ffad0 0x7fada00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 -- 172.21.15.5:0/1113969562 >> 172.21.15.5:0/1113969562 conn(0x7fada00fb5f0 msgr2=0x7fada00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 -- 172.21.15.5:0/1113969562 shutdown_connections 2023-07-21T19:44:03.261 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.276+0000 7fada5fff700 1 -- 172.21.15.5:0/1113969562 wait complete. 2023-07-21T19:44:03.262 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.277+0000 7fada5fff700 1 Processor -- start 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 -- start start 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 0x7fada019db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 0x7fada019e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada01982e0 con 0x7fada0105ae0 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0198450 con 0x7fada00ffad0 2023-07-21T19:44:03.263 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fada5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fada0198610 con 0x7fada0100480 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fad9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fad9f7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 0x7fada019db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fad9ffff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33526/0 (socket says 172.21.15.5:33526) 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fad9ffff700 1 -- 172.21.15.5:0/2741988963 learned_addr learned my addr 172.21.15.5:0/2741988963 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.278+0000 7fad9effd700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 0x7fada019e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 -- 172.21.15.5:0/2741988963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 msgr2=0x7fada019e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.264 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 0x7fada019e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 -- 172.21.15.5:0/2741988963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 msgr2=0x7fada019db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 0x7fada019db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 -- 172.21.15.5:0/2741988963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad98009d00 con 0x7fada0105ae0 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9f7fe700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 0x7fada019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9effd700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 0x7fada019e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9ffff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e3c0 tx=0x7fad9800e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad980092d0 con 0x7fada0105ae0 2023-07-21T19:44:03.265 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad90009520 con 0x7fada0105ae0 2023-07-21T19:44:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fad9800f040 con 0x7fada0105ae0 2023-07-21T19:44:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad98007bf0 con 0x7fada0105ae0 2023-07-21T19:44:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.279+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fada01a4ab0 con 0x7fada0105ae0 2023-07-21T19:44:03.266 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.280+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fada0067290 con 0x7fada0105ae0 2023-07-21T19:44:03.267 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.284+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fad98007d50 con 0x7fada0105ae0 2023-07-21T19:44:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.285+0000 7fad9cff9700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 0x7fad880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.268 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.285+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fad980ccf60 con 0x7fada0105ae0 2023-07-21T19:44:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.285+0000 7fad9f7fe700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 0x7fad880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.286+0000 7fad9f7fe700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 0x7fad880a3770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad90009fd0 tx=0x7fad90019040 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:03.271 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.288+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fad980963a0 con 0x7fada0105ae0 2023-07-21T19:44:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.496+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fada0198fd0 con 0x7fada0105ae0 2023-07-21T19:44:03.479 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.496+0000 7fad9cff9700 1 -- 172.21.15.5:0/2741988963 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7fad980999c0 con 0x7fada0105ae0 2023-07-21T19:44:03.480 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:44:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.498+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 msgr2=0x7fad880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.498+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 0x7fad880a3770 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fad90009fd0 tx=0x7fad90019040 comp rx=0 tx=0).stop 2023-07-21T19:44:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.498+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 msgr2=0x7fada0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.498+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fad9800e3c0 tx=0x7fad9800e6d0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.482 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 shutdown_connections 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fad880a12c0 0x7fad880a3770 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fada0105ae0 0x7fada0197b10 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fada00ffad0 0x7fada019db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 --2- 172.21.15.5:0/2741988963 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fada0100480 0x7fada019e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 >> 172.21.15.5:0/2741988963 conn(0x7fada00fb5f0 msgr2=0x7fada010ae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 shutdown_connections 2023-07-21T19:44:03.483 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.499+0000 7fada5fff700 1 -- 172.21.15.5:0/2741988963 wait complete. 2023-07-21T19:44:03.501 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:44:03.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 -- 172.21.15.5:0/3245599150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c105ae0 msgr2=0x7f226c105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 --2- 172.21.15.5:0/3245599150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c105ae0 0x7f226c105eb0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f225c009990 tx=0x7f225c009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 -- 172.21.15.5:0/3245599150 shutdown_connections 2023-07-21T19:44:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 --2- 172.21.15.5:0/3245599150 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 0x7f226c10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 --2- 172.21.15.5:0/3245599150 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c105ae0 0x7f226c105eb0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 --2- 172.21.15.5:0/3245599150 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c0ffad0 0x7f226c0fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 -- 172.21.15.5:0/3245599150 >> 172.21.15.5:0/3245599150 conn(0x7f226c0fb5f0 msgr2=0x7f226c0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 -- 172.21.15.5:0/3245599150 shutdown_connections 2023-07-21T19:44:03.709 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.724+0000 7f2271bff700 1 -- 172.21.15.5:0/3245599150 wait complete. 2023-07-21T19:44:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 Processor -- start 2023-07-21T19:44:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 -- start start 2023-07-21T19:44:03.710 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 0x7f226c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c105ae0 0x7f226c195990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c196000 con 0x7f226c0ffad0 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c196170 con 0x7f226c105ae0 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.726+0000 7f2271bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c196330 con 0x7f226c100480 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.711 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33552/0 (socket says 172.21.15.5:33552) 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 -- 172.21.15.5:0/4070818881 learned_addr learned my addr 172.21.15.5:0/4070818881 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226bfff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c105ae0 0x7f226c195990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f2263fff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 0x7f226c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 -- 172.21.15.5:0/4070818881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 msgr2=0x7f226c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 0x7f226c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.712 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 -- 172.21.15.5:0/4070818881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c105ae0 msgr2=0x7f226c195990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c105ae0 0x7f226c195990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 -- 172.21.15.5:0/4070818881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254009520 con 0x7f226c0ffad0 2023-07-21T19:44:03.713 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.727+0000 7f226b7fe700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f225c005740 tx=0x7f225c005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.728+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c01d070 con 0x7f226c0ffad0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.728+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f225c022470 con 0x7f226c0ffad0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.728+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f225c017650 con 0x7f226c0ffad0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.728+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c0095f0 con 0x7f226c0ffad0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.728+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c1967f0 con 0x7f226c0ffad0 2023-07-21T19:44:03.714 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.729+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c067290 con 0x7f226c0ffad0 2023-07-21T19:44:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.731+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f225c00fb00 con 0x7f226c0ffad0 2023-07-21T19:44:03.715 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.732+0000 7f22697fa700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 0x7f22580a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.732+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f225c0ce2a0 con 0x7f226c0ffad0 2023-07-21T19:44:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.732+0000 7f2263fff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 0x7f22580a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:03.716 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.733+0000 7f2263fff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 0x7f22580a3770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f226c0fdf10 tx=0x7f2254009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:03.719 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.736+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f225c0976e0 con 0x7f226c0ffad0 2023-07-21T19:44:03.927 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.944+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f226c1a29c0 con 0x7f226c0ffad0 2023-07-21T19:44:03.928 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.944+0000 7f22697fa700 1 -- 172.21.15.5:0/4070818881 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7f225c09ad00 con 0x7f226c0ffad0 2023-07-21T19:44:03.928 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:44:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 msgr2=0x7f22580a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 0x7f22580a3770 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f226c0fdf10 tx=0x7f2254009260 comp rx=0 tx=0).stop 2023-07-21T19:44:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 msgr2=0x7f226c19b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f225c005740 tx=0x7f225c005770 comp rx=0 tx=0).stop 2023-07-21T19:44:03.930 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 shutdown_connections 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f226c105ae0 0x7f226c195990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f22580a12c0 0x7f22580a3770 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f226c0ffad0 0x7f226c19b920 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 --2- 172.21.15.5:0/4070818881 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f226c100480 0x7f226c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 >> 172.21.15.5:0/4070818881 conn(0x7f226c0fb5f0 msgr2=0x7f226c0fbc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 shutdown_connections 2023-07-21T19:44:03.931 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:03.947+0000 7f2271bff700 1 -- 172.21.15.5:0/4070818881 wait complete. 2023-07-21T19:44:03.949 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T19:44:04.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 -- 172.21.15.5:0/3669412954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc0ffad0 msgr2=0x7f3abc0fff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/3669412954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc0ffad0 0x7f3abc0fff40 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f3ab0009990 tx=0x7f3ab0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 -- 172.21.15.5:0/3669412954 shutdown_connections 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/3669412954 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc100480 0x7f3abc10c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/3669412954 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc105ae0 0x7f3abc105eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/3669412954 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc0ffad0 0x7f3abc0fff40 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 -- 172.21.15.5:0/3669412954 >> 172.21.15.5:0/3669412954 conn(0x7f3abc0fb5f0 msgr2=0x7f3abc0fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 -- 172.21.15.5:0/3669412954 shutdown_connections 2023-07-21T19:44:04.156 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.171+0000 7f3ac27ff700 1 -- 172.21.15.5:0/3669412954 wait complete. 2023-07-21T19:44:04.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.172+0000 7f3ac27ff700 1 Processor -- start 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.172+0000 7f3ac27ff700 1 -- start start 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 0x7f3abc197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 0x7f3abc198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 0x7f3abc19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19cd30 con 0x7f3abc105ae0 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19cea0 con 0x7f3abc0ffad0 2023-07-21T19:44:04.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac27ff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3abc19d060 con 0x7f3abc100480 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac0d9c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 0x7f3abc19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 0x7f3abc197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3abbfff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 0x7f3abc197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50798/0 (socket says 172.21.15.5:50798) 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3abbfff700 1 -- 172.21.15.5:0/2232779146 learned_addr learned my addr 172.21.15.5:0/2232779146 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3abb7fe700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 0x7f3abc198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac0d9c700 1 -- 172.21.15.5:0/2232779146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 msgr2=0x7f3abc198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.159 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac0d9c700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 0x7f3abc198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac0d9c700 1 -- 172.21.15.5:0/2232779146 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 msgr2=0x7f3abc197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.173+0000 7f3ac0d9c700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 0x7f3abc197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ac0d9c700 1 -- 172.21.15.5:0/2232779146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ab00095f0 con 0x7f3abc105ae0 2023-07-21T19:44:04.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3abb7fe700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 0x7f3abc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ac0d9c700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 0x7f3abc19c6f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400efc0 tx=0x7f3ab400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab4018070 con 0x7f3abc105ae0 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f3ab400f040 con 0x7f3abc105ae0 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3abc19d2e0 con 0x7f3abc105ae0 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ab400cc50 con 0x7f3abc105ae0 2023-07-21T19:44:04.161 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.174+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3abc1a4b80 con 0x7f3abc105ae0 2023-07-21T19:44:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.175+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3abc067290 con 0x7f3abc105ae0 2023-07-21T19:44:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.177+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f3ab4003a00 con 0x7f3abc105ae0 2023-07-21T19:44:04.162 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.179+0000 7f3ab97fa700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 0x7f3aa40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.179+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f3ab40cc8b0 con 0x7f3abc105ae0 2023-07-21T19:44:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.179+0000 7f3abbfff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 0x7f3aa40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.163 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.179+0000 7f3abbfff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 0x7f3aa40a37f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005950 tx=0x7f3aac00b500 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:04.165 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.182+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f3ab4095cf0 con 0x7f3abc105ae0 2023-07-21T19:44:04.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:03 smithi101 ceph-mon[128136]: pgmap v1157: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-21T19:44:04.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:03 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/207013602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:03 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2741988963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:03 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/4070818881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.389 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.406+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3abc1990a0 con 0x7f3abc105ae0 2023-07-21T19:44:04.390 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.406+0000 7f3ab97fa700 1 -- 172.21.15.5:0/2232779146 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v149) v1 ==== 81+0+927 (secure 0 0 0) 0x7f3ab4099310 con 0x7f3abc105ae0 2023-07-21T19:44:04.390 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3558708925 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.390 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1627438252 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.390 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2496466053 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.390 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4257470084 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1985166852 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4175825231 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1627438252 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4029410450 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1672036156 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/474183959 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:04.391 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/697337206 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:04.392 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2485032310 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:04.392 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/474183959 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:04.392 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1317509932 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:04.392 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3754075095 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:04.392 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1672036156 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:04.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:03 smithi005 ceph-mon[126460]: pgmap v1157: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-21T19:44:04.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:03 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/207013602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:03 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2741988963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.393 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:03 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/4070818881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 msgr2=0x7f3aa40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 0x7f3aa40a37f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f3aac005950 tx=0x7f3aac00b500 comp rx=0 tx=0).stop 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 msgr2=0x7f3abc19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 0x7f3abc19c6f0 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f3ab400efc0 tx=0x7f3ab400c3a0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 shutdown_connections 2023-07-21T19:44:04.394 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f3aa40a1340 0x7f3aa40a37f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f3abc105ae0 0x7f3abc19c6f0 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f3abc0ffad0 0x7f3abc197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 --2- 172.21.15.5:0/2232779146 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f3abc100480 0x7f3abc198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 >> 172.21.15.5:0/2232779146 conn(0x7f3abc0fb5f0 msgr2=0x7f3abc10ae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 shutdown_connections 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.409+0000 7f3ac27ff700 1 -- 172.21.15.5:0/2232779146 wait complete. 2023-07-21T19:44:04.395 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2023-07-21T19:44:04.402 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:03 smithi156 ceph-mon[128316]: pgmap v1157: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 44 KiB/s wr, 21 op/s 2023-07-21T19:44:04.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:03 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/207013602' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:03 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2741988963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:03 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/4070818881' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:04.411 DEBUG:teuthology.orchestra.run.smithi005:> set -ex 2023-07-21T19:44:04.411 DEBUG:teuthology.orchestra.run.smithi005:> dd if=/proc/self/mounts of=/dev/stdout 2023-07-21T19:44:04.455 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-07-21T19:44:04.697 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 -- 172.21.15.5:0/235036948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 msgr2=0x7f8ea4105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 --2- 172.21.15.5:0/235036948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4105eb0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8e98009990 tx=0x7f8e98009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 -- 172.21.15.5:0/235036948 shutdown_connections 2023-07-21T19:44:04.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 --2- 172.21.15.5:0/235036948 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea4100480 0x7f8ea410c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.698 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 --2- 172.21.15.5:0/235036948 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4105eb0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 --2- 172.21.15.5:0/235036948 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea40ffad0 0x7f8ea40fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 -- 172.21.15.5:0/235036948 >> 172.21.15.5:0/235036948 conn(0x7f8ea40fb590 msgr2=0x7f8ea40fd9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:04.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 -- 172.21.15.5:0/235036948 shutdown_connections 2023-07-21T19:44:04.699 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.714+0000 7f8eaabff700 1 -- 172.21.15.5:0/235036948 wait complete. 2023-07-21T19:44:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.715+0000 7f8eaabff700 1 Processor -- start 2023-07-21T19:44:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.715+0000 7f8eaabff700 1 -- start start 2023-07-21T19:44:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.715+0000 7f8eaabff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea40ffad0 0x7f8ea419b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.702 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8eaabff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 0x7f8ea419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8eaabff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8eaabff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4195fb0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8eaabff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea4196120 con 0x7f8ea4100480 2023-07-21T19:44:04.703 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8eaabff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ea41962e0 con 0x7f8ea40ffad0 2023-07-21T19:44:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 0x7f8ea419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33594/0 (socket says 172.21.15.5:33594) 2023-07-21T19:44:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea3fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 0x7f8ea419be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50812/0 (socket says 172.21.15.5:50812) 2023-07-21T19:44:04.704 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 -- 172.21.15.5:0/92880089 learned_addr learned my addr 172.21.15.5:0/92880089 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea899b700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea40ffad0 0x7f8ea419b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 -- 172.21.15.5:0/92880089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea40ffad0 msgr2=0x7f8ea419b8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea40ffad0 0x7f8ea419b8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 -- 172.21.15.5:0/92880089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 msgr2=0x7f8ea419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.705 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 0x7f8ea419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.716+0000 7f8ea919c700 1 -- 172.21.15.5:0/92880089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e980095f0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8ea919c700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00ef30 tx=0x7f8e9c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c018070 con 0x7f8ea4105ae0 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ea4196560 con 0x7f8ea4105ae0 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f8e9c00f040 con 0x7f8ea4105ae0 2023-07-21T19:44:04.706 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e9c00cbc0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.717+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ea41a28c0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.718+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ea4067300 con 0x7f8ea4105ae0 2023-07-21T19:44:04.707 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.720+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f8e9c00cd60 con 0x7f8ea4105ae0 2023-07-21T19:44:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.721+0000 7f8ea1ffb700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 0x7f8e940a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.721+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f8e9c0cc780 con 0x7f8ea4105ae0 2023-07-21T19:44:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.722+0000 7f8ea899b700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 0x7f8e940a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.722+0000 7f8ea899b700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 0x7f8e940a37b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e9800b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:04.708 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.725+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f8e9c095bc0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.917 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.934+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ea41a2b80 con 0x7f8ea4105ae0 2023-07-21T19:44:04.919 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.936+0000 7f8ea1ffb700 1 -- 172.21.15.5:0/92880089 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+22992 (secure 0 0 0) 0x7f8e9c0991e0 con 0x7f8ea4105ae0 2023-07-21T19:44:04.919 INFO:teuthology.orchestra.run.smithi005.stdout: 2023-07-21T19:44:04.920 INFO:teuthology.orchestra.run.smithi005.stdout:{"epoch":149,"fsid":"3dc43aa4-27f8-11ee-9b34-001a4aab830c","created":"2023-07-21T18:57:04.739728+0000","modified":"2023-07-21T19:43:23.036392+0000","last_up_change":"2023-07-21T19:04:40.485885+0000","last_in_change":"2023-07-21T19:04:19.906441+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-07-21T19:00:41.817557+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-07-21T19:05:35.444438+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-07-21T19:05:36.667545+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":67,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d6825a11-5019-461a-a2af-a0337a93556d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6802","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6803","nonce":2154692488}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6804","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6805","nonce":2154692488}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6808","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6809","nonce":2154692488}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6806","nonce":2154692488},{"type":"v1","addr":"172.21.15.5:6807","nonce":2154692488}]},"public_addr":"172.21.15.5:6803/2154692488","cluster_addr":"172.21.15.5:6805/2154692488","heartbeat_back_addr":"172.21.15.5:6809/2154692488","heartbeat_front_addr":"172.21.15.5:6807/2154692488","state":["exists","up"]},{"osd":1,"uuid":"22a957d4-0669-466d-88ee-0f05ba0b0909","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6800","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6801","nonce":3147533687}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6802","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6803","nonce":3147533687}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6806","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6807","nonce":3147533687}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6804","nonce":3147533687},{"type":"v1","addr":"172.21.15.101:6805","nonce":3147533687}]},"public_addr":"172.21.15.101:6801/3147533687","cluster_addr":"172.21.15.101:6803/3147533687","heartbeat_back_addr":"172.21.15.101:6807/3147533687","heartbeat_front_addr":"172.21.15.101:6805/3147533687","state":["exists","up"]},{"osd":2,"uuid":"d6ba0786-1e76-44da-b171-81015c102bc4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6800","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6801","nonce":2113778318}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6802","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6803","nonce":2113778318}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6806","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6807","nonce":2113778318}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6804","nonce":2113778318},{"type":"v1","addr":"172.21.15.156:6805","nonce":2113778318}]},"public_addr":"172.21.15.156:6801/2113778318","cluster_addr":"172.21.15.156:6803/2113778318","heartbeat_back_addr":"172.21.15.156:6807/2113778318","heartbeat_front_addr":"172.21.15.156:6805/2113778318","state":["exists","up"]},{"osd":3,"uuid":"01f22961-e7d2-423a-8511-6b3ed03de822","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6810","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6811","nonce":4237119296}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6812","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6813","nonce":4237119296}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6816","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6817","nonce":4237119296}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6814","nonce":4237119296},{"type":"v1","addr":"172.21.15.5:6815","nonce":4237119296}]},"public_addr":"172.21.15.5:6811/4237119296","cluster_addr":"172.21.15.5:6813/4237119296","heartbeat_back_addr":"172.21.15.5:6817/4237119296","heartbeat_front_addr":"172.21.15.5:6815/4237119296","state":["exists","up"]},{"osd":4,"uuid":"157f6702-def5-47b3-a2e1-497954863a97","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6808","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6809","nonce":1960590612}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6810","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6811","nonce":1960590612}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6814","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6815","nonce":1960590612}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6812","nonce":1960590612},{"type":"v1","addr":"172.21.15.101:6813","nonce":1960590612}]},"public_addr":"172.21.15.101:6809/1960590612","cluster_addr":"172.21.15.101:6811/1960590612","heartbeat_back_addr":"172.21.15.101:6815/1960590612","heartbeat_front_addr":"172.21.15.101:6813/1960590612","state":["exists","up"]},{"osd":5,"uuid":"04a4eedc-bd98-4441-8bdb-443010dfc21e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6808","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6809","nonce":1994365713}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6810","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6811","nonce":1994365713}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6814","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6815","nonce":1994365713}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6812","nonce":1994365713},{"type":"v1","addr":"172.21.15.156:6813","nonce":1994365713}]},"public_addr":"172.21.15.156:6809/1994365713","cluster_addr":"172.21.15.156:6811/1994365713","heartbeat_back_addr":"172.21.15.156:6815/1994365713","heartbeat_front_addr":"172.21.15.156:6813/1994365713","state":["exists","up"]},{"osd":6,"uuid":"bd7ecefb-ba21-4052-adc1-17da27ca0595","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6818","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6819","nonce":3271423477}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6820","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6821","nonce":3271423477}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6824","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6825","nonce":3271423477}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6822","nonce":3271423477},{"type":"v1","addr":"172.21.15.5:6823","nonce":3271423477}]},"public_addr":"172.21.15.5:6819/3271423477","cluster_addr":"172.21.15.5:6821/3271423477","heartbeat_back_addr":"172.21.15.5:6825/3271423477","heartbeat_front_addr":"172.21.15.5:6823/3271423477","state":["exists","up"]},{"osd":7,"uuid":"a839fb0f-1bc6-4f9d-a381-079c5f730f63","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6816","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6817","nonce":1417474005}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6818","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6819","nonce":1417474005}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6822","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6823","nonce":1417474005}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6820","nonce":1417474005},{"type":"v1","addr":"172.21.15.101:6821","nonce":1417474005}]},"public_addr":"172.21.15.101:6817/1417474005","cluster_addr":"172.21.15.101:6819/1417474005","heartbeat_back_addr":"172.21.15.101:6823/1417474005","heartbeat_front_addr":"172.21.15.101:6821/1417474005","state":["exists","up"]},{"osd":8,"uuid":"fc535b54-e81e-457d-ac39-57b75be1a082","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6816","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6817","nonce":1480568600}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6818","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6819","nonce":1480568600}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6822","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6823","nonce":1480568600}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6820","nonce":1480568600},{"type":"v1","addr":"172.21.15.156:6821","nonce":1480568600}]},"public_addr":"172.21.15.156:6817/1480568600","cluster_addr":"172.21.15.156:6819/1480568600","heartbeat_back_addr":"172.21.15.156:6823/1480568600","heartbeat_front_addr":"172.21.15.156:6821/1480568600","state":["exists","up"]},{"osd":9,"uuid":"41283e76-6d87-47be-b423-725502de65f4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6826","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6827","nonce":3583557754}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6828","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6829","nonce":3583557754}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6832","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6833","nonce":3583557754}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.5:6830","nonce":3583557754},{"type":"v1","addr":"172.21.15.5:6831","nonce":3583557754}]},"public_addr":"172.21.15.5:6827/3583557754","cluster_addr":"172.21.15.5:6829/3583557754","heartbeat_back_addr":"172.21.15.5:6833/3583557754","heartbeat_front_addr":"172.21.15.5:6831/3583557754","state":["exists","up"]},{"osd":10,"uuid":"ba0f83a2-7456-498d-bf66-564b17ccbce2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6824","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6825","nonce":3222350780}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6826","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6827","nonce":3222350780}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6830","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6831","nonce":3222350780}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.101:6828","nonce":3222350780},{"type":"v1","addr":"172.21.15.101:6829","nonce":3222350780}]},"public_addr":"172.21.15.101:6825/3222350780","cluster_addr":"172.21.15.101:6827/3222350780","heartbeat_back_addr":"172.21.15.101:6831/3222350780","heartbeat_front_addr":"172.21.15.101:6829/3222350780","state":["exists","up"]},{"osd":11,"uuid":"1b9e2433-ce6f-4b3f-9c85-da0c3ebacbb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":86,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6824","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6825","nonce":2474999994}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6826","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6827","nonce":2474999994}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6830","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6831","nonce":2474999994}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.156:6828","nonce":2474999994},{"type":"v1","addr":"172.21.15.156:6829","nonce":2474999994}]},"public_addr":"172.21.15.156:6825/2474999994","cluster_addr":"172.21.15.156:6827/2474999994","heartbeat_back_addr":"172.21.15.156:6831/2474999994","heartbeat_front_addr":"172.21.15.156:6829/2474999994","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T18:59:46.801572+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:13.320493+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:00:39.248862+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:06.177097+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:32.766419+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:01:58.912732+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:25.312514+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:02:51.918498+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:18.552847+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:03:45.096263+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:11.832476+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-07-21T19:04:38.805171+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.4","mappings":[{"from":2,"to":11}]},{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"2.1f","mappings":[{"from":8,"to":1}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.1","mappings":[{"from":4,"to":9}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.5:0/3558708925":"2023-07-22T19:07:36.084989+0000","172.21.15.5:6801/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/2496466053":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4257470084":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/1985166852":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4175825231":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6800/1627438252":"2023-07-22T19:07:36.084989+0000","172.21.15.5:0/4029410450":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/1672036156":"2023-07-22T18:58:07.219133+0000","172.21.15.5:6801/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/697337206":"2023-07-22T18:58:07.219133+0000","172.21.15.5:0/2485032310":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/474183959":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/1317509932":"2023-07-22T18:57:41.027247+0000","172.21.15.5:0/3754075095":"2023-07-22T18:57:41.027247+0000","172.21.15.5:6800/1672036156":"2023-07-22T18:58:07.219133+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":54,"length":1},{"begin":67,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-07-21T19:44:04.921 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 msgr2=0x7f8e940a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 0x7f8e940a37b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f8e98006010 tx=0x7f8e9800b260 comp rx=0 tx=0).stop 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 msgr2=0x7f8ea4195940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8e9c00ef30 tx=0x7f8e9c00c3a0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 shutdown_connections 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f8e940a1300 0x7f8e940a37b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.922 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f8ea4105ae0 0x7f8ea4195940 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f8ea40ffad0 0x7f8ea419b8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 --2- 172.21.15.5:0/92880089 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f8ea4100480 0x7f8ea419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.938+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 >> 172.21.15.5:0/92880089 conn(0x7f8ea40fb590 msgr2=0x7f8ea40fbc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.939+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 shutdown_connections 2023-07-21T19:44:04.923 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:04.939+0000 7f8eaabff700 1 -- 172.21.15.5:0/92880089 wait complete. 2023-07-21T19:44:04.941 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-07-21T19:00:41.817557+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-07-21T19:05:35.444438+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-07-21T19:05:36.667545+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 67, 'snap_epoch': 148, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-07-21T19:44:04.941 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-07-21T19:44:05.148 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 -- 172.21.15.5:0/2713332637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 msgr2=0x7faf9010c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 --2- 172.21.15.5:0/2713332637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf9010c4b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7faf88009f90 tx=0x7faf8800b330 comp rx=0 tx=0).stop 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 -- 172.21.15.5:0/2713332637 shutdown_connections 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 --2- 172.21.15.5:0/2713332637 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf9010c4b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 --2- 172.21.15.5:0/2713332637 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf90100ca0 0x7faf90101070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 --2- 172.21.15.5:0/2713332637 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf901015b0 0x7faf90109aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.149 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 -- 172.21.15.5:0/2713332637 >> 172.21.15.5:0/2713332637 conn(0x7faf900fa4b0 msgr2=0x7faf900fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 -- 172.21.15.5:0/2713332637 shutdown_connections 2023-07-21T19:44:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.165+0000 7faf95dff700 1 -- 172.21.15.5:0/2713332637 wait complete. 2023-07-21T19:44:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.167+0000 7faf95dff700 1 Processor -- start 2023-07-21T19:44:05.150 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.167+0000 7faf95dff700 1 -- start start 2023-07-21T19:44:05.151 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf901015b0 0x7faf9019bf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf90195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90196120 con 0x7faf90100ca0 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90196290 con 0x7faf9010a070 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf95dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf90196450 con 0x7faf901015b0 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.152 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33606/0 (socket says 172.21.15.5:33606) 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 -- 172.21.15.5:0/2294810266 learned_addr learned my addr 172.21.15.5:0/2294810266 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8ffff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf90195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8effd700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf901015b0 0x7faf9019bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 -- 172.21.15.5:0/2294810266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf901015b0 msgr2=0x7faf9019bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf901015b0 0x7faf9019bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 -- 172.21.15.5:0/2294810266 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 msgr2=0x7faf90195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.153 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf90195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.168+0000 7faf8f7fe700 1 -- 172.21.15.5:0/2294810266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf840095f0 con 0x7faf90100ca0 2023-07-21T19:44:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf8f7fe700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9a0 tx=0x7faf8000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80009790 con 0x7faf90100ca0 2023-07-21T19:44:05.154 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf88009c40 con 0x7faf90100ca0 2023-07-21T19:44:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7faf8000cbd0 con 0x7faf90100ca0 2023-07-21T19:44:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf80018680 con 0x7faf90100ca0 2023-07-21T19:44:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.169+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf901a2850 con 0x7faf90100ca0 2023-07-21T19:44:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.170+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf901051f0 con 0x7faf90100ca0 2023-07-21T19:44:05.155 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.172+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7faf8000cd40 con 0x7faf90100ca0 2023-07-21T19:44:05.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.174+0000 7faf8cff9700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 0x7faf780a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.174+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7faf80014070 con 0x7faf90100ca0 2023-07-21T19:44:05.157 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.174+0000 7faf8effd700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 0x7faf780a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.158 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.175+0000 7faf8effd700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 0x7faf780a3770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf9010b430 tx=0x7faf84009310 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:05.160 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.178+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7faf80051790 con 0x7faf90100ca0 2023-07-21T19:44:05.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2232779146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:05.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:05 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/92880089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:05.371 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.388+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7faf90104bd0 con 0x7faf90100ca0 2023-07-21T19:44:05.372 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.389+0000 7faf8cff9700 1 -- 172.21.15.5:0/2294810266 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v149) v1 ==== 93+0+10 (secure 0 0 0) 0x7faf800963d0 con 0x7faf90100ca0 2023-07-21T19:44:05.372 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 1 2023-07-21T19:44:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 msgr2=0x7faf780a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.374 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 0x7faf780a3770 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faf9010b430 tx=0x7faf84009310 comp rx=0 tx=0).stop 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 msgr2=0x7faf9019ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7faf8000e9a0 tx=0x7faf8000ecb0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 shutdown_connections 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7faf9010a070 0x7faf90195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7faf780a12c0 0x7faf780a3770 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7faf90100ca0 0x7faf9019ba40 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.375 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 --2- 172.21.15.5:0/2294810266 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7faf901015b0 0x7faf9019bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 >> 172.21.15.5:0/2294810266 conn(0x7faf900fa4b0 msgr2=0x7faf9010c230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 shutdown_connections 2023-07-21T19:44:05.376 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.391+0000 7faf95dff700 1 -- 172.21.15.5:0/2294810266 wait complete. 2023-07-21T19:44:05.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2232779146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:05.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:05 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/92880089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:05.393 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-07-21T19:44:05.402 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2232779146' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:05.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:05 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/92880089' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-07-21T19:44:05.599 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 -- 172.21.15.5:0/2561354324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df0100b80 msgr2=0x7f2df0100f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 --2- 172.21.15.5:0/2561354324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df0100b80 0x7f2df0100f50 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f2de0009990 tx=0x7f2de0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 -- 172.21.15.5:0/2561354324 shutdown_connections 2023-07-21T19:44:05.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 --2- 172.21.15.5:0/2561354324 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df010a0a0 0x7f2df010c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 --2- 172.21.15.5:0/2561354324 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df0100b80 0x7f2df0100f50 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.600 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 --2- 172.21.15.5:0/2561354324 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0101490 0x7f2df0109b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 -- 172.21.15.5:0/2561354324 >> 172.21.15.5:0/2561354324 conn(0x7f2df00fa4b0 msgr2=0x7f2df00fc8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:05.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 -- 172.21.15.5:0/2561354324 shutdown_connections 2023-07-21T19:44:05.601 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.616+0000 7f2df5fff700 1 -- 172.21.15.5:0/2561354324 wait complete. 2023-07-21T19:44:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.618+0000 7f2df5fff700 1 Processor -- start 2023-07-21T19:44:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.618+0000 7f2df5fff700 1 -- start start 2023-07-21T19:44:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df0101490 0x7f2df01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.602 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 0x7f2df019c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df019cd70 con 0x7f2df010a0a0 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df019cee0 con 0x7f2df0100b80 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2df5fff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2df019d0a0 con 0x7f2df0101490 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:50846/0 (socket says 172.21.15.5:50846) 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 -- 172.21.15.5:0/3806313585 learned_addr learned my addr 172.21.15.5:0/3806313585 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:05.603 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2deffff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 0x7f2df019c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.604 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2deeffd700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df0101490 0x7f2df01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 -- 172.21.15.5:0/3806313585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df0101490 msgr2=0x7f2df01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df0101490 0x7f2df01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 -- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 msgr2=0x7f2df019c730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 0x7f2df019c730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2def7fe700 1 -- 172.21.15.5:0/3806313585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de00095f0 con 0x7f2df0100b80 2023-07-21T19:44:05.605 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.619+0000 7f2deffff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 0x7f2df019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2def7fe700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2de0005910 tx=0x7f2de00047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de001d070 con 0x7f2df0100b80 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7f2de0021410 con 0x7f2df0100b80 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2df019d320 con 0x7f2df0100b80 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2de0017650 con 0x7f2df0100b80 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.620+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2df01a4bc0 con 0x7f2df0100b80 2023-07-21T19:44:05.606 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.621+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2df0067290 con 0x7f2df0100b80 2023-07-21T19:44:05.607 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.623+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7f2de000fb00 con 0x7f2df0100b80 2023-07-21T19:44:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.625+0000 7f2decff9700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 0x7f2dd80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.625+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7f2de00ce480 con 0x7f2df0100b80 2023-07-21T19:44:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.625+0000 7f2deeffd700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 0x7f2dd80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:05.608 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.625+0000 7f2deeffd700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 0x7f2dd80a37f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2df01993a0 tx=0x7f2de4009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:05.612 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.629+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7f2de00978c0 con 0x7f2df0100b80 2023-07-21T19:44:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.840+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2df01a4d50 con 0x7f2df0100b80 2023-07-21T19:44:05.823 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.840+0000 7f2decff9700 1 -- 172.21.15.5:0/3806313585 <== mon.2 v2:172.21.15.101:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v149) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2de009aee0 con 0x7f2df0100b80 2023-07-21T19:44:05.824 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 32 2023-07-21T19:44:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 msgr2=0x7f2dd80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 0x7f2dd80a37f0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2df01993a0 tx=0x7f2de4009260 comp rx=0 tx=0).stop 2023-07-21T19:44:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 msgr2=0x7f2df0197d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f2de0005910 tx=0x7f2de00047b0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.826 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 shutdown_connections 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7f2dd80a1340 0x7f2dd80a37f0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7f2df010a0a0 0x7f2df019c730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7f2df0100b80 0x7f2df0197d90 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 --2- 172.21.15.5:0/3806313585 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7f2df0101490 0x7f2df01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 >> 172.21.15.5:0/3806313585 conn(0x7f2df00fa4b0 msgr2=0x7f2df010c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 shutdown_connections 2023-07-21T19:44:05.827 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:05.843+0000 7f2df5fff700 1 -- 172.21.15.5:0/3806313585 wait complete. 2023-07-21T19:44:05.844 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-07-21T19:44:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 -- 172.21.15.5:0/3490677070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 msgr2=0x7fced0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.050 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 --2- 172.21.15.5:0/3490677070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0105eb0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009990 tx=0x7fcecc009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 -- 172.21.15.5:0/3490677070 shutdown_connections 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 --2- 172.21.15.5:0/3490677070 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced0100480 0x7fced010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 --2- 172.21.15.5:0/3490677070 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0105eb0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 --2- 172.21.15.5:0/3490677070 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced00ffad0 0x7fced00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 -- 172.21.15.5:0/3490677070 >> 172.21.15.5:0/3490677070 conn(0x7fced00fb5f0 msgr2=0x7fced00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 -- 172.21.15.5:0/3490677070 shutdown_connections 2023-07-21T19:44:06.051 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.067+0000 7fced7bff700 1 -- 172.21.15.5:0/3490677070 wait complete. 2023-07-21T19:44:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 Processor -- start 2023-07-21T19:44:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 -- start start 2023-07-21T19:44:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.053 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced0100480 0x7fced019e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0197b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01981c0 con 0x7fced00ffad0 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced0198330 con 0x7fced0105ae0 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.068+0000 7fced7bff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced01984f0 con 0x7fced0100480 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.5:3300/0 says I am v2:172.21.15.5:33642/0 (socket says 172.21.15.5:33642) 2023-07-21T19:44:06.054 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 -- 172.21.15.5:0/2290247083 learned_addr learned my addr 172.21.15.5:0/2290247083 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced619c700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0197b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced519a700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced0100480 0x7fced019e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 -- 172.21.15.5:0/2290247083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced0100480 msgr2=0x7fced019e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.055 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced0100480 0x7fced019e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 -- 172.21.15.5:0/2290247083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 msgr2=0x7fced0197b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0197b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 -- 172.21.15.5:0/2290247083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcec8009c40 con 0x7fced00ffad0 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced619c700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.069+0000 7fced599b700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000c00 tx=0x7fcecc009570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:06.056 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.070+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc01d070 con 0x7fced00ffad0 2023-07-21T19:44:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.070+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcecc0095f0 con 0x7fced00ffad0 2023-07-21T19:44:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.070+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fcecc022470 con 0x7fced00ffad0 2023-07-21T19:44:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.070+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc017650 con 0x7fced00ffad0 2023-07-21T19:44:06.057 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.070+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced01a48f0 con 0x7fced00ffad0 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.071+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced0067290 con 0x7fced00ffad0 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.073+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fcecc00fb00 con 0x7fced00ffad0 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.074+0000 7fcec6ffd700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 0x7fcebc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.074+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fcecc0ce170 con 0x7fced00ffad0 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.074+0000 7fced519a700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 0x7fcebc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.058 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.075+0000 7fced519a700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 0x7fcebc0a3770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fced010cd90 tx=0x7fcec0009260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:06.061 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.078+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fcecc0976c0 con 0x7fced00ffad0 2023-07-21T19:44:06.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:06 smithi005 ceph-mon[126460]: pgmap v1158: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:44:06.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2294810266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:06 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/3806313585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.269 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.286+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fced01a4a80 con 0x7fced00ffad0 2023-07-21T19:44:06.270 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.286+0000 7fcec6ffd700 1 -- 172.21.15.5:0/2290247083 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v149) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcecc09ace0 con 0x7fced00ffad0 2023-07-21T19:44:06.270 INFO:teuthology.orchestra.run.smithi005.stdout:pg_num: 64 2023-07-21T19:44:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 msgr2=0x7fcebc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 0x7fcebc0a3770 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fced010cd90 tx=0x7fcec0009260 comp rx=0 tx=0).stop 2023-07-21T19:44:06.273 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 msgr2=0x7fced019dae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7fcecc000c00 tx=0x7fcecc009570 comp rx=0 tx=0).stop 2023-07-21T19:44:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 shutdown_connections 2023-07-21T19:44:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fced0105ae0 0x7fced0197b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.274 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fcebc0a12c0 0x7fcebc0a3770 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fced00ffad0 0x7fced019dae0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 --2- 172.21.15.5:0/2290247083 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fced0100480 0x7fced019e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.275 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 >> 172.21.15.5:0/2290247083 conn(0x7fced00fb5f0 msgr2=0x7fced010adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 shutdown_connections 2023-07-21T19:44:06.276 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.289+0000 7fced7bff700 1 -- 172.21.15.5:0/2290247083 wait complete. 2023-07-21T19:44:06.291 DEBUG:teuthology.orchestra.run.smithi005:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-07-21T19:44:06.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:06 smithi101 ceph-mon[128136]: pgmap v1158: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:44:06.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2294810266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:06 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/3806313585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.402 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:06 smithi156 ceph-mon[128316]: pgmap v1158: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-07-21T19:44:06.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2294810266' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.403 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:06 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/3806313585' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-07-21T19:44:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 -- 172.21.15.5:0/3387072315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0105ae0 msgr2=0x7fb1c0105eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.496 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/3387072315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0105ae0 0x7fb1c0105eb0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009990 tx=0x7fb1b0009ca0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 -- 172.21.15.5:0/3387072315 shutdown_connections 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/3387072315 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c0100480 0x7fb1c010c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/3387072315 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0105ae0 0x7fb1c0105eb0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/3387072315 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c00ffad0 0x7fb1c00fff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 -- 172.21.15.5:0/3387072315 >> 172.21.15.5:0/3387072315 conn(0x7fb1c00fb5f0 msgr2=0x7fb1c00fda00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:06.497 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 -- 172.21.15.5:0/3387072315 shutdown_connections 2023-07-21T19:44:06.498 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.513+0000 7fb1c5dff700 1 -- 172.21.15.5:0/3387072315 wait complete. 2023-07-21T19:44:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.514+0000 7fb1c5dff700 1 Processor -- start 2023-07-21T19:44:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.514+0000 7fb1c5dff700 1 -- start start 2023-07-21T19:44:06.499 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.514+0000 7fb1c5dff700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1c5dff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 0x7fb1c0072d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1c5dff700 1 --2- >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c0105ae0 0x7fb1c01a3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1c5dff700 1 -- --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c0073580 con 0x7fb1c0100480 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1c5dff700 1 -- --> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01a42a0 con 0x7fb1c00ffad0 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1c5dff700 1 -- --> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c01a43e0 con 0x7fb1c0105ae0 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 --2- >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 0x7fb1c0072d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.500 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1bf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1bf7fe700 1 --2- >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.101:3300/0 says I am v2:172.21.15.5:46622/0 (socket says 172.21.15.5:46622) 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1bf7fe700 1 -- 172.21.15.5:0/1158293073 learned_addr learned my addr 172.21.15.5:0/1158293073 (peer_addr_for_me v2:172.21.15.5:0/0) 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1bffff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c0105ae0 0x7fb1c01a3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 -- 172.21.15.5:0/1158293073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c0105ae0 msgr2=0x7fb1c01a3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c0105ae0 0x7fb1c01a3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 -- 172.21.15.5:0/1158293073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 msgr2=0x7fb1c0072810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.501 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1b7fff700 1 -- 172.21.15.5:0/1158293073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b00095f0 con 0x7fb1c0100480 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.515+0000 7fb1bf7fe700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1b7fff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 0x7fb1c0072d50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b450 tx=0x7fb1a800b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8011830 con 0x7fb1c0100480 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 2 ==== config(25 keys) v1 ==== 1067+0+0 (secure 0 0 0) 0x7fb1a800be60 con 0x7fb1c0100480 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c01a4640 con 0x7fb1c0100480 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1a8010ca0 con 0x7fb1c0100480 2023-07-21T19:44:06.502 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.516+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c01a4b20 con 0x7fb1c0100480 2023-07-21T19:44:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.517+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c0067290 con 0x7fb1c0100480 2023-07-21T19:44:06.503 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.519+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 4 ==== mgrmap(e 25) v1 ==== 135845+0+0 (secure 0 0 0) 0x7fb1a8011990 con 0x7fb1c0100480 2023-07-21T19:44:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.520+0000 7fb1bd7fa700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 0x7fb1ac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-07-21T19:44:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.521+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 8513+0+0 (secure 0 0 0) 0x7fb1a80cc550 con 0x7fb1c0100480 2023-07-21T19:44:06.504 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.521+0000 7fb1bf7fe700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 0x7fb1ac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-07-21T19:44:06.505 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.521+0000 7fb1bf7fe700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 0x7fb1ac0a3770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0006010 tx=0x7fb1b000b260 comp rx=0 tx=0).ready entity=mgr.25261 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-07-21T19:44:06.507 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.524+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+178038 (secure 0 0 0) 0x7fb1a8095a10 con 0x7fb1c0100480 2023-07-21T19:44:06.735 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.752+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 --> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb1c0073d40 con 0x7fb1c0100480 2023-07-21T19:44:06.736 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.753+0000 7fb1bd7fa700 1 -- 172.21.15.5:0/1158293073 <== mon.0 v2:172.21.15.5:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v149) v1 ==== 81+0+927 (secure 0 0 0) 0x7fb1a8099030 con 0x7fb1c0100480 2023-07-21T19:44:06.736 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3558708925 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.736 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1627438252 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2496466053 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4257470084 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1985166852 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4175825231 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1627438252 2023-07-22T19:07:36.084989+0000 2023-07-21T19:44:06.737 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/4029410450 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/1672036156 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6801/474183959 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/697337206 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/2485032310 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/474183959 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/1317509932 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:06.738 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:0/3754075095 2023-07-22T18:57:41.027247+0000 2023-07-21T19:44:06.739 INFO:teuthology.orchestra.run.smithi005.stdout:172.21.15.5:6800/1672036156 2023-07-22T18:58:07.219133+0000 2023-07-21T19:44:06.739 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 msgr2=0x7fb1ac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 0x7fb1ac0a3770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0006010 tx=0x7fb1b000b260 comp rx=0 tx=0).stop 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 msgr2=0x7fb1c0072d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 0x7fb1c0072d50 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7fb1a800b450 tx=0x7fb1a800b760 comp rx=0 tx=0).stop 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 shutdown_connections 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.156:3300/0,v1:172.21.15.156:6789/0] conn(0x7fb1c0105ae0 0x7fb1c01a3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.740 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.101:3300/0,v1:172.21.15.101:6789/0] conn(0x7fb1c00ffad0 0x7fb1c0072810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:6800/1124791839,v1:172.21.15.5:6801/1124791839] conn(0x7fb1ac0a12c0 0x7fb1ac0a3770 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 --2- 172.21.15.5:0/1158293073 >> [v2:172.21.15.5:3300/0,v1:172.21.15.5:6789/0] conn(0x7fb1c0100480 0x7fb1c0072d50 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 >> 172.21.15.5:0/1158293073 conn(0x7fb1c00fb5f0 msgr2=0x7fb1c01035e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 shutdown_connections 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:2023-07-21T19:44:06.755+0000 7fb1c5dff700 1 -- 172.21.15.5:0/1158293073 wait complete. 2023-07-21T19:44:06.741 INFO:teuthology.orchestra.run.smithi005.stderr:listed 16 entries 2023-07-21T19:44:06.756 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi005.front.sepia.ceph.com... 2023-07-21T19:44:06.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:44:06.757 DEBUG:teuthology.orchestra.run.smithi005:> sudo fusermount -u /tmp/cephfs 2023-07-21T19:44:06.829 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi005.stderr:ceph-fuse[174015]: fuse finished with error 0 and tester_r 0 2023-07-21T19:44:07.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/2290247083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:07.141 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:07 smithi005 ceph-mon[126460]: from='client.? 172.21.15.5:0/1158293073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:07.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/2290247083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:07.315 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:07 smithi101 ceph-mon[128136]: from='client.? 172.21.15.5:0/1158293073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:07.409 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/2290247083' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-07-21T19:44:07.409 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:07 smithi156 ceph-mon[128316]: from='client.? 172.21.15.5:0/1158293073' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-07-21T19:44:08.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:08 smithi101 ceph-mon[128136]: pgmap v1159: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:44:08.391 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:08 smithi005 ceph-mon[126460]: pgmap v1159: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:44:08.402 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:08 smithi156 ceph-mon[128316]: pgmap v1159: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 45 KiB/s wr, 20 op/s 2023-07-21T19:44:08.515 INFO:teuthology.orchestra.run:waiting for 300 2023-07-21T19:44:08.516 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T19:44:08.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:44:08.517 DEBUG:teuthology.orchestra.run.smithi005:> (cd /home/ubuntu/cephtest && exec rmdir -- /tmp/cephfs) 2023-07-21T19:44:08.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:44:08.597 DEBUG:teuthology.orchestra.run.smithi005:> ip netns list 2023-07-21T19:44:08.641 INFO:teuthology.orchestra.run.smithi005.stdout:ceph-ns--tmp-cephfs (id: 0) 2023-07-21T19:44:08.643 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:44:08.643 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip netns delete ceph-ns--tmp-cephfs 2023-07-21T19:44:08.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-07-21T19:44:08.756 DEBUG:teuthology.orchestra.run.smithi005:> sudo ip link delete ceph-brx 2023-07-21T19:44:08.871 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-07-21T19:44:08.884 INFO:tasks.cephadm:Teardown begin 2023-07-21T19:44:08.885 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/cephadm.py", line 1677, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-21T19:44:08.885 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:44:08.987 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:44:09.090 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:44:09.179 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-07-21T19:44:09.179 DEBUG:teuthology.orchestra.run.smithi005:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:44:09.227 DEBUG:teuthology.orchestra.run.smithi101:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:44:09.282 DEBUG:teuthology.orchestra.run.smithi156:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-07-21T19:44:09.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:09 smithi101 ceph-mon[128136]: pgmap v1160: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 50 KiB/s wr, 24 op/s 2023-07-21T19:44:09.326 INFO:tasks.cephadm:Stopping all daemons... 2023-07-21T19:44:09.327 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-07-21T19:44:09.327 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a 2023-07-21T19:44:09.389 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:09 smithi005 ceph-mon[126460]: pgmap v1160: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 50 KiB/s wr, 24 op/s 2023-07-21T19:44:09.402 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:09 smithi156 ceph-mon[128316]: pgmap v1160: 97 pgs: 97 active+clean; 10 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 50 KiB/s wr, 24 op/s 2023-07-21T19:44:09.641 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:09 smithi005 systemd[1]: Stopping Ceph mon.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:10.335 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a[126436]: 2023-07-21T19:44:09.927+0000 7ff8384df700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:10.336 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a[126436]: 2023-07-21T19:44:09.927+0000 7ff8384df700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-07-21T19:44:10.640 INFO:journalctl@ceph.mon.a.smithi005.stdout:Jul 21 19:44:10 smithi005 bash[213081]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-a 2023-07-21T19:44:10.841 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.a.service' 2023-07-21T19:44:10.954 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:10.955 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-07-21T19:44:10.955 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-07-21T19:44:10.955 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.b 2023-07-21T19:44:11.313 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:11 smithi101 systemd[1]: Stopping Ceph mon.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:11.927 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-b[128111]: 2023-07-21T19:44:11.653+0000 7f4501c65700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:11.927 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-b[128111]: 2023-07-21T19:44:11.653+0000 7f4501c65700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-07-21T19:44:12.314 INFO:journalctl@ceph.mon.b.smithi101.stdout:Jul 21 19:44:12 smithi101 bash[165713]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-b 2023-07-21T19:44:12.479 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.b.service' 2023-07-21T19:44:12.605 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:12.606 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-07-21T19:44:12.606 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-07-21T19:44:12.606 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.c 2023-07-21T19:44:13.153 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:12 smithi156 systemd[1]: Stopping Ceph mon.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:13.532 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-c[128292]: 2023-07-21T19:44:13.217+0000 7f3fd3cde700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:13.532 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-c[128292]: 2023-07-21T19:44:13.217+0000 7f3fd3cde700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-07-21T19:44:13.903 INFO:journalctl@ceph.mon.c.smithi156.stdout:Jul 21 19:44:13 smithi156 bash[167591]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mon-c 2023-07-21T19:44:14.033 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mon.c.service' 2023-07-21T19:44:14.132 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:14.133 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-07-21T19:44:14.134 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-07-21T19:44:14.135 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x 2023-07-21T19:44:14.641 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:44:14 smithi005 systemd[1]: Stopping Ceph mgr.x for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:14.954 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:44:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:44:14.668+0000 7fca10e3a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:14.954 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:44:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x[126703]: 2023-07-21T19:44:14.668+0000 7fca10e3a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-07-21T19:44:15.391 INFO:journalctl@ceph.mgr.x.smithi005.stdout:Jul 21 19:44:15 smithi005 bash[213225]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-x 2023-07-21T19:44:15.546 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.x.service' 2023-07-21T19:44:15.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:15.643 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-07-21T19:44:15.643 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-07-21T19:44:15.644 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.y 2023-07-21T19:44:16.064 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:44:15 smithi101 systemd[1]: Stopping Ceph mgr.y for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:16.813 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:44:16 smithi101 bash[165861]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-y 2023-07-21T19:44:17.049 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.y.service' 2023-07-21T19:44:17.113 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:44:16 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-07-21T19:44:17.114 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:44:17 smithi101 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.y.service: Failed with result 'exit-code'. 2023-07-21T19:44:17.114 INFO:journalctl@ceph.mgr.y.smithi101.stdout:Jul 21 19:44:17 smithi101 systemd[1]: Stopped Ceph mgr.y for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:44:17.139 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:17.140 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-07-21T19:44:17.140 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-07-21T19:44:17.141 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.z 2023-07-21T19:44:17.652 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:44:17 smithi156 systemd[1]: Stopping Ceph mgr.z for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:18.153 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:44:17 smithi156 bash[167747]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mgr-z 2023-07-21T19:44:18.383 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.z.service' 2023-07-21T19:44:18.446 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:44:18 smithi156 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-07-21T19:44:18.446 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:44:18 smithi156 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mgr.z.service: Failed with result 'exit-code'. 2023-07-21T19:44:18.446 INFO:journalctl@ceph.mgr.z.smithi156.stdout:Jul 21 19:44:18 smithi156 systemd[1]: Stopped Ceph mgr.z for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:44:18.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:18.472 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-07-21T19:44:18.472 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-07-21T19:44:18.473 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.0 2023-07-21T19:44:18.891 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:44:18 smithi005 systemd[1]: Stopping Ceph osd.0 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:19.391 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:44:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T19:44:19.008+0000 7f811c226700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:19.391 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:44:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T19:44:19.008+0000 7f811c226700 -1 osd.0 149 *** Got signal Terminated *** 2023-07-21T19:44:19.392 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:44:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0[138875]: 2023-07-21T19:44:19.008+0000 7f811c226700 -1 osd.0 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:44:25.184 INFO:journalctl@ceph.osd.0.smithi005.stdout:Jul 21 19:44:25 smithi005 bash[213367]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-0 2023-07-21T19:44:28.211 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.0.service' 2023-07-21T19:44:28.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:28.312 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-07-21T19:44:28.313 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-07-21T19:44:28.313 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.1 2023-07-21T19:44:28.814 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:44:28 smithi101 systemd[1]: Stopping Ceph osd.1 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:29.313 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:44:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1[133257]: 2023-07-21T19:44:28.839+0000 7f74b58f9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:29.314 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:44:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1[133257]: 2023-07-21T19:44:28.839+0000 7f74b58f9700 -1 osd.1 149 *** Got signal Terminated *** 2023-07-21T19:44:29.314 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:44:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1[133257]: 2023-07-21T19:44:28.839+0000 7f74b58f9700 -1 osd.1 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:44:37.313 INFO:journalctl@ceph.osd.1.smithi101.stdout:Jul 21 19:44:37 smithi101 bash[166012]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-1 2023-07-21T19:44:39.248 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.1.service' 2023-07-21T19:44:39.337 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:39.337 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-07-21T19:44:39.338 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-07-21T19:44:39.338 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.2 2023-07-21T19:44:39.846 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:44:39 smithi156 systemd[1]: Stopping Ceph osd.2 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:40.152 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:44:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2[134150]: 2023-07-21T19:44:39.862+0000 7f5cf1b2e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:40.153 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:44:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2[134150]: 2023-07-21T19:44:39.862+0000 7f5cf1b2e700 -1 osd.2 149 *** Got signal Terminated *** 2023-07-21T19:44:40.153 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:44:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2[134150]: 2023-07-21T19:44:39.862+0000 7f5cf1b2e700 -1 osd.2 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:44:45.814 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:45.561+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:46.152 INFO:journalctl@ceph.osd.2.smithi156.stdout:Jul 21 19:44:45 smithi156 bash[167896]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-2 2023-07-21T19:44:46.641 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:46 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:46.178+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:46.652 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:46.327+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:46.813 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:46.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:47.153 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:46.773+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:47.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:47 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:47.135+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:47.538 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:47.303+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:47.758 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.2.service' 2023-07-21T19:44:47.813 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:47.565+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:47.849 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:47.850 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-07-21T19:44:47.850 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-07-21T19:44:47.851 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.3 2023-07-21T19:44:48.152 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:47.789+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:48.355 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:48.162+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:48.355 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:47 smithi005 systemd[1]: Stopping Ceph osd.3 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:48.356 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:48.044+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:48.631 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:48.291+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:48.641 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:48.371+0000 7fa05e08a700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:48.641 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:48.371+0000 7fa05e08a700 -1 osd.3 149 *** Got signal Terminated *** 2023-07-21T19:44:48.641 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:48.371+0000 7fa05e08a700 -1 osd.3 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:44:48.641 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:48 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:48.559+0000 7fa059e7f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.269033+0000 front 2023-07-21T19:44:22.269064+0000 (oldest deadline 2023-07-21T19:44:48.169285+0000) 2023-07-21T19:44:48.813 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:48.527+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:48.903 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:48.784+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:48.903 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:48.648+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:49.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:49 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:49.208+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:49.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:49 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:49.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:49.596 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:49.333+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:49.813 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:49.546+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:49.891 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:49 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:49.603+0000 7fa059e7f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.269033+0000 front 2023-07-21T19:44:22.269064+0000 (oldest deadline 2023-07-21T19:44:48.169285+0000) 2023-07-21T19:44:49.903 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:49.613+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:49.903 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:49.828+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:50.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:50 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:50.243+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:50.391 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:50 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:50.004+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:50.588 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:50.333+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:50.814 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:50.544+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:50.843 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:50.605+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:50.891 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:50 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:50.578+0000 7fa059e7f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.269033+0000 front 2023-07-21T19:44:22.269064+0000 (oldest deadline 2023-07-21T19:44:48.169285+0000) 2023-07-21T19:44:51.153 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:50.860+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:51.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:51 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:51.197+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:51.391 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:51 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:51.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:51.572 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:51.319+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:51.814 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:51.552+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:51.865 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:51.588+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:51.891 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:51 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:51.600+0000 7fa059e7f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.269033+0000 front 2023-07-21T19:44:22.269064+0000 (oldest deadline 2023-07-21T19:44:48.169285+0000) 2023-07-21T19:44:52.153 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:51.882+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:52.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:52 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:52.217+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:52.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:52 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:52.027+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:52.602 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:52.294+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:52.873 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:52.619+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:52.891 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:52 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3[144318]: 2023-07-21T19:44:52.562+0000 7fa059e7f700 -1 osd.3 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.269033+0000 front 2023-07-21T19:44:22.269064+0000 (oldest deadline 2023-07-21T19:44:48.169285+0000) 2023-07-21T19:44:53.064 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:52.598+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:53.153 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:52.888+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:53.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:53 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:53.186+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:53.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:53 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:53.062+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:53.590 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:53.303+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:53.903 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:53.607+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:54.064 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:53 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:53.647+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:54.247 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:53.919+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:54.391 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:54 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:54.215+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:54.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:54 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:54.074+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:54.549 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:54.264+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:54.891 INFO:journalctl@ceph.osd.3.smithi005.stdout:Jul 21 19:44:54 smithi005 bash[213643]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-3 2023-07-21T19:44:54.903 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:54.566+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:54.983 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:54 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:54.688+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:55.249 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:54.929+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:55.315 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:55 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:55.000+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:44:55.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:55 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:55.245+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:55.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:55 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:55.115+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:55.575 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:55.266+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:55.901 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:55.591+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:55.901 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:55.591+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:44:56.018 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:55 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:55.720+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:56.153 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:55.917+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:56.154 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:55.917+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:44:56.258 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:56 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:56.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:56.314 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:56 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:56.034+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:44:56.369 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.3.service' 2023-07-21T19:44:56.466 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:44:56.467 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-07-21T19:44:56.467 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-07-21T19:44:56.467 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.4 2023-07-21T19:44:56.624 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:56.235+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:56.641 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:56 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:56.274+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:56.815 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:56 smithi101 systemd[1]: Stopping Ceph osd.4 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:44:56.815 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:56 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:56.766+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:56.815 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:56 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:56.766+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:44:56.903 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:56.640+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:56.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:56.640+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:44:56.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:56.889+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:56.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:56.889+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:44:57.314 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:57.010+0000 7fa68dabe700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:44:57.315 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:57.010+0000 7fa68dabe700 -1 osd.4 149 *** Got signal Terminated *** 2023-07-21T19:44:57.315 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:57.010+0000 7fa68dabe700 -1 osd.4 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:44:57.315 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:57.039+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:44:57.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:57 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:57.232+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:57.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:57 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:57.114+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:57.602 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:57.266+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:57.857 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:57.618+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:57.858 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:57.618+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:44:58.059 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:44:57.634+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:44:58.059 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:57.775+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:58.059 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:57 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:57.775+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:44:58.154 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:57.874+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:58.154 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:57.874+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:44:58.314 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:58 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:58.074+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:44:58.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:58.205+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:58.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:58.205+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:44:58.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:58.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:58.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:58 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:58.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:44:58.644 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:58.256+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:58.645 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:58.256+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:44:58.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:58.660+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:58.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:58.660+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:44:58.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:58.896+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:58.905 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:58.896+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:44:59.065 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:44:58 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:44:58.630+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:44:59.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:58 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:58.734+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:44:59.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:58 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:58.734+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:44:59.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:59.111+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:44:59.392 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:44:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:44:59.111+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:44:59.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:59.228+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:44:59.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:44:59 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:44:59.228+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:44:59.565 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:44:59 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:44:59.100+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:44:59.638 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:59.299+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:44:59.638 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:44:59.299+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:44:59.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:59.654+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:44:59.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:44:59.654+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:44:59.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:59.903+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:44:59.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:44:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:44:59.903+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:00.065 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:44:59 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:44:59.641+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:00.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:59 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:59.768+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:00.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:44:59 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:44:59.768+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:00.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:00.196+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:00.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:00.196+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:00.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:00.140+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:00.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:00 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:00.140+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:00.565 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:45:00 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:45:00.089+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:45:00.593 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:00.291+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:00.594 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:00.291+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:00.856 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:00.609+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:00.856 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:00.609+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:01.065 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:00 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:00.637+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:01.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:00 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:00.763+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:01.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:00 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:00.763+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:01.154 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:00.872+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:01.154 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:00.872+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:01.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:01.237+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:01.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:01.237+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:01.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:01.136+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:01.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:01 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:01.136+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:01.565 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:45:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4[138836]: 2023-07-21T19:45:01.090+0000 7fa6898b3700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.969151+0000 front 2023-07-21T19:44:32.969198+0000 (oldest deadline 2023-07-21T19:44:54.669383+0000) 2023-07-21T19:45:01.634 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:01.337+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:01.634 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:01.337+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:01.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:01.866+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:01.904 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:01.866+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:01.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:01.650+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:01.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:01 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:01.650+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:02.065 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:01.686+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:02.065 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:01.794+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:02.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:01 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:01.794+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:02.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:02.231+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:02.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:02.231+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:02.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:02.098+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:02.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:02 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:02.098+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:02.641 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:02.324+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:02.641 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:02.324+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:02.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:02.657+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:02.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:02.657+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:02.905 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:02.905+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:02.905 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:02 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:02.905+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:02.907 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:02 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:02.671+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:02.908 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:02 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:02.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:02.908 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:02 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:02.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:03.315 INFO:journalctl@ceph.osd.4.smithi101.stdout:Jul 21 19:45:02 smithi101 bash[166294]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-4 2023-07-21T19:45:03.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:03.272+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:03.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:03.272+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:03.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:03.090+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:03.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:03 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:03.090+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:03.636 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:03.287+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:03.636 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:03.287+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:03.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:03.651+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:03.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:03.651+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:03.954 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:03.709+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:03.955 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:03.813+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:03.955 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:03 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:03.813+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:04.249 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:03.939+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:04.249 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:03 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:03.939+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:04.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:04.301+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:04.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:04.301+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:04.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:04.111+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:04.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:04.111+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:04.635 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:04.264+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:04.635 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:04.264+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:04.833 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.4.service' 2023-07-21T19:45:04.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:04.650+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:04.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:04.650+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:04.919 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:04 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:04.669+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:04.919 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:04 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:04.846+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:04.920 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:04 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:04.846+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:04.921 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:04.922 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-07-21T19:45:04.922 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-07-21T19:45:04.923 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.5 2023-07-21T19:45:05.297 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 systemd[1]: Stopping Ceph osd.5 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:05.297 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:04.975+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:05.298 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:04 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:04.975+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:05.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:05.288+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:05.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:05.288+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:05.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:05.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:05.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:05 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:05.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:05.619 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:05.312+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:05.620 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:05.312+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:05.620 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:05.466+0000 7f089e5b9700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:05.620 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:05.466+0000 7f089e5b9700 -1 osd.5 149 *** Got signal Terminated *** 2023-07-21T19:45:05.620 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:05.466+0000 7f089e5b9700 -1 osd.5 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:05.621 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:05.634+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:05.904 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:05.634+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:06.065 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:05 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:05.688+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:06.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:05 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:05.801+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:06.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:05 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:05.801+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:06.297 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:05.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:06.297 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:05.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:06.297 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:05 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:05.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:06.392 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:06.321+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:06.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:06.321+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:06.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:06.122+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:06.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:06 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:06.122+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:06.573 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:06 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:06.311+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:06.573 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:06 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:06.311+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:06.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:06 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:06.588+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:06.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:06 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:06.588+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:07.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:06 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:06.655+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:07.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:06 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:06.655+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:07.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:06 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:06.850+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:07.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:06 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:06.850+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:07.329 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:07.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:07.329 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:07.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:07.330 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:07.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:07.342 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:07.022+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:07.343 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:07.022+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:07.343 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:07.022+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:07.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:07.357+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:07.606 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:07.357+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:07.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:07.344+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:07.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:07.344+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:07.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:07 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:07.344+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:07.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:07.621+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:07.905 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:07 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:07.621+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:08.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:07 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:07.610+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:08.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:07 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:07.610+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:08.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:07 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:07.801+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:08.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:07 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:07.801+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:08.361 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:08.054+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:08.361 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:08.054+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:08.362 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:08.054+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:08.372 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:08.131+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:08.372 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:08.131+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:08.372 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:08.131+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:08.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:08.387+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:08.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:08.387+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:08.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:08.387+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:08.655 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:08.619+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:08.655 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:08.619+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:08.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:08.376+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:08.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:08.376+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:08.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:08 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:08.376+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:09.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:08.619+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:09.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:08.619+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:09.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:08.789+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:09.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:08.789+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:09.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:08 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:08.789+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:09.331 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:09.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:09.332 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:09.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:09.332 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:09.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:09.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:09.347+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:09.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:09.347+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:09.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:09.347+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:09.644 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:09.180+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:09.645 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:09.180+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:09.645 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:09 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:09.180+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:09.655 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:09.571+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.417929+0000 front 2023-07-21T19:44:21.417956+0000 (oldest deadline 2023-07-21T19:44:47.818990+0000) 2023-07-21T19:45:09.655 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5[139706]: 2023-07-21T19:45:09.571+0000 7f08973a8700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.720077+0000 front 2023-07-21T19:44:30.720126+0000 (oldest deadline 2023-07-21T19:44:55.420205+0000) 2023-07-21T19:45:09.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:09.347+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:09.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:09.347+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:09.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:09 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:09.347+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:10.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:09 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:09.639+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:10.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:09 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:09.639+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:10.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:09 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:09.778+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:10.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:09 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:09.778+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:10.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:09 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:09.778+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:10.289 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:10.053+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:10.289 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:10.053+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:10.289 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:10.053+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:10.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:10.364+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:10.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:10.364+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:10.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:10.364+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:10.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:10.135+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:10.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:10.135+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:10.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:10 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:10.135+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:10.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:10.304+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:10.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:10.304+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:10.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:10 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:10.304+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:11.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:10.643+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:11.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:10.643+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:11.066 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:10.763+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:11.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:10.763+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:11.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:10 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:10.763+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:11.298 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:11.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:11.298 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:11.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:11.298 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:11.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:11.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:11.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:11.393 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:11.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:11.394 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:11.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:11.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:11.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:11.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:11.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:11.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:11 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:11.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:11.655 INFO:journalctl@ceph.osd.5.smithi156.stdout:Jul 21 19:45:11 smithi156 bash[168168]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-5 2023-07-21T19:45:11.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:11.311+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:11.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:11.311+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:11.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:11 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:11.311+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:12.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:11.623+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:12.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:11.623+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:12.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:11.807+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:12.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:11.807+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:12.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:11 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:11.807+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:12.328 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:12.085+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:12.329 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:12.085+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:12.329 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:12.085+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:12.341 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:12.048+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:12.341 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:12.048+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:12.341 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:12.048+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:12.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:12.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:12.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:12.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:12.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:12 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:12.342+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:12.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:12.355+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:12.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:12.355+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:12.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:12 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:12.355+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:13.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:12.618+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:13.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:12.618+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:13.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:12.758+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:13.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:12.758+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:13.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:12 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:12.758+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:13.339 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:13.340 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:13.340 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:13.365 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:13.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:13.365 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:13.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:13.365 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:13.100+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:13.431 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.5.service' 2023-07-21T19:45:13.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:13.523 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-07-21T19:45:13.523 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-07-21T19:45:13.524 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.6 2023-07-21T19:45:13.623 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:13.380+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:13.623 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:13.380+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:13.623 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:13 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:13.380+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:13.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:13.353+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:13.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:13.353+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:13.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:13.353+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:13.894 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:13 smithi005 systemd[1]: Stopping Ceph osd.6 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:14.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:13 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:13.624+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:14.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:13 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:13.624+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:14.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:13 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:13.716+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:14.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:13 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:13.716+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:14.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:13 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:13.716+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:14.318 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:14.101+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:14.318 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:14.101+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:14.318 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:14.101+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:14.318 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.051+0000 7fa14ff56700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:14.318 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.051+0000 7fa14ff56700 -1 osd.6 149 *** Got signal Terminated *** 2023-07-21T19:45:14.319 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.051+0000 7fa14ff56700 -1 osd.6 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:14.319 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.332+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:14.344 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.961+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:14.345 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.961+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:14.345 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:13 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:13.961+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:14.643 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.332+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:14.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:14.332+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:14.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:14.359+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:14.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:14.359+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:14.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:14.359+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:15.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:14.608+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:15.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:14.608+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:15.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:14.729+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:15.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:14.729+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:15.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:14 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:14.729+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:15.392 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:14.957+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:15.393 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:14.957+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:15.393 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:14 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:14.957+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:15.393 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:15.115+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:15.115+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:15.115+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:15.315+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:15.315+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:15.315+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:15.394 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:15 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:15.315+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.605829+0000 front 2023-07-21T19:44:51.605769+0000 (oldest deadline 2023-07-21T19:45:14.505746+0000) 2023-07-21T19:45:15.655 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:15.407+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:15.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:15.407+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:15.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:15.407+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:15.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:15.407+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:16.066 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:15.618+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:16.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:15.618+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:16.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:15.618+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:16.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:15.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:16.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:15.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:16.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:15 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:15.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:16.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:16.079+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:16.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:16.079+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:16.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:16.079+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:16.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:16.299+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:16.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:16.299+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:16.396 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:16.299+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:16.396 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:16 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:16.299+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.605829+0000 front 2023-07-21T19:44:51.605769+0000 (oldest deadline 2023-07-21T19:45:14.505746+0000) 2023-07-21T19:45:16.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:15.934+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:16.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:15.934+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:16.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:15 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:15.934+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:16.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:16.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:16.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:16.453+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:16.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:16.453+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:16.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:16.453+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:17.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:16.636+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:17.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:16.636+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:17.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:16.636+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:17.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:16.678+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:17.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:16.678+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:17.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:16 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:16.678+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:17.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:17.033+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:17.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:17.033+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:17.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:17.033+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:17.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:17.264+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:17.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:17.264+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:17.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:17.264+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:17.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:17.264+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.605829+0000 front 2023-07-21T19:44:51.605769+0000 (oldest deadline 2023-07-21T19:45:14.505746+0000) 2023-07-21T19:45:17.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:16.976+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:17.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:16.976+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:17.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:16 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:16.976+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:17.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:17.429+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:17.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:17.429+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:17.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:17.429+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:17.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:17.429+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:18.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:17.625+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:18.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:17.625+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:18.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:17.625+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:18.067 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:17.639+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:18.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:17.639+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:18.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:17.639+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:18.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:17 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:17.639+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:18.279 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:18.034+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:18.279 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:18.034+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:18.279 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:18.034+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:18.280 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:18.034+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:18.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:17.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:18.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:17.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:18.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:17 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:17.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:18.644 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:18.293+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.698804+0000 front 2023-07-21T19:44:23.698832+0000 (oldest deadline 2023-07-21T19:44:45.999131+0000) 2023-07-21T19:45:18.645 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:18.293+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.801408+0000 front 2023-07-21T19:44:31.801391+0000 (oldest deadline 2023-07-21T19:44:57.701458+0000) 2023-07-21T19:45:18.645 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:18.293+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.204535+0000 front 2023-07-21T19:44:42.204571+0000 (oldest deadline 2023-07-21T19:45:06.904769+0000) 2023-07-21T19:45:18.645 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:18 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6[149846]: 2023-07-21T19:45:18.293+0000 7fa14bd4b700 -1 osd.6 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.605829+0000 front 2023-07-21T19:44:51.605769+0000 (oldest deadline 2023-07-21T19:45:14.505746+0000) 2023-07-21T19:45:18.904 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:18.466+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:18.904 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:18.467+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:18.905 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:18.467+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:18.905 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:18.467+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:19.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:18.640+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:19.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:18.640+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:19.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:18.640+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:19.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:18.631+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:19.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:18.631+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:19.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:18.631+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:19.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:18 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:18.631+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:19.156 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:18.918+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:19.156 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:18.918+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:19.156 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:18 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:18.918+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:19.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:19.050+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:19.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:19.050+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:19.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:19.050+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:19.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:19 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:19.050+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:19.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:19.434+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:19.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:19.434+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:19.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:19.434+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:19.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:19.434+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:20.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:19.610+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:20.067 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:19.610+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:20.068 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:19.610+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:20.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:19.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:20.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:19.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:20.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:19.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:20.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:19 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:19.681+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:20.394 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:19.964+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:20.394 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:19.964+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:20.394 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:19 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:19.964+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:20.395 INFO:journalctl@ceph.osd.6.smithi005.stdout:Jul 21 19:45:20 smithi005 bash[213920]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-6 2023-07-21T19:45:20.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:20.039+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:20.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:20.039+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:20.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:20.039+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:20.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:20 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:20.039+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:20.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:20.408+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:20.656 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:20.408+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:20.657 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:20.408+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:20.657 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:20.408+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:20.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:20.567+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:20.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:20.567+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:20.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:20.567+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:20.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:20.652+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:20.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:20.652+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:20.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:20.652+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:20.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:20 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:20.652+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:21.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:21.059+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:21.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:21.059+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:21.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:21.059+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:21.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:21 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:21.059+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:21.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:20.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:21.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:20.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:21.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:20 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:20.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:21.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:21.566+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:21.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:21.566+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:21.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:21.566+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:21.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:21.621+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:21.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:21.621+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:21.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:21.621+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:21.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:21 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:21.621+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:21.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:21.455+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:21.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:21.455+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:21.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:21.455+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:21.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:21.456+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:22.049 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.6.service' 2023-07-21T19:45:22.147 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:22.148 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-07-21T19:45:22.148 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-07-21T19:45:22.149 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.7 2023-07-21T19:45:22.394 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:22.032+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:22.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:22.032+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:22.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:22.032+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:22.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:22 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:22.032+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:22.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:21.992+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:22.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:21.992+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:22.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:21 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:21.992+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:22.515 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 systemd[1]: Stopping Ceph osd.7 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:22.817 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:22.604+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:22.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:22.604+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:22.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:22.604+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:22.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:22.604+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:22.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.529+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:22.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.529+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:22.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.529+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:22.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.665+0000 7f60c5561700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:22.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.665+0000 7f60c5561700 -1 osd.7 149 *** Got signal Terminated *** 2023-07-21T19:45:22.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:22 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:22.665+0000 7f60c5561700 -1 osd.7 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:22.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:22.447+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:22.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:22.447+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:22.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:22.447+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:22.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:22 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:22.447+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:23.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:23.067+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:23.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:23.067+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:23.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:23.067+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:23.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:23 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:23.067+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:23.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:23.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:23.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:23.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:23.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:23.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:23.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:23.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:23.817 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:23.549+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:23.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:23.549+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:23.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:23.549+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:23.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:23.654+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:23.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:23.654+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:23.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:23.654+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:23.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:23.654+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:23.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:23.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:23.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:23.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:23.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:23.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:23.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:23 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:23.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:24.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:24.037+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:24.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:24.037+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:24.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:24.037+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:24.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:24.037+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:24.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:24 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:24.037+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:24.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:24.038+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:24.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:24.038+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:24.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:24.038+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:24.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:24.038+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:24.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:24.532+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:24.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:24.532+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:24.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:24.532+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:24.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:24.532+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.592727+0000 front 2023-07-21T19:44:59.592700+0000 (oldest deadline 2023-07-21T19:45:24.292800+0000) 2023-07-21T19:45:24.818 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:24.618+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:24.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:24.618+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:24.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:24.618+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:24.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:24 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:24.618+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:24.906 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:24.476+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:24.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:24.476+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:24.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:24.476+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:24.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:24.476+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:24.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:24 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:24.476+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:25.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:25.080+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:25.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:25.080+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:25.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:25.080+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:25.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:25.080+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:25.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:25 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:25.080+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:25.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:25.007+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:25.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:25.007+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:25.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:25.007+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:25.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:25.007+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:25.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:25.574+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:25.818 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:25.574+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:25.574+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:25.574+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.592727+0000 front 2023-07-21T19:44:59.592700+0000 (oldest deadline 2023-07-21T19:45:24.292800+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:25.624+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:25.624+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:25.624+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:25.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:25 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:25.624+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:25.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:25.438+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:25.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:25.438+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:25.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:25.438+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:25.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:25.438+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:25.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:25 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:25.438+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:26.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:26.051+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:26.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:26.051+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:26.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:26.051+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:26.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:26.051+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:26.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:26 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:26.051+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:26.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:26.036+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:26.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:26.036+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:26.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:26.036+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:26.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:26.036+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:26.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:26.454+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:26.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:26.454+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:26.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:26.454+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:26.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:26.454+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:26.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:26 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:26.454+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:27.068 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:26.595+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:27.068 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:26.595+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:27.068 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:26.595+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:27.068 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:26.595+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.592727+0000 front 2023-07-21T19:44:59.592700+0000 (oldest deadline 2023-07-21T19:45:24.292800+0000) 2023-07-21T19:45:27.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:26.666+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:27.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:26.666+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:27.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:26.666+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:27.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:26.666+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:27.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:26 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:26.666+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:27.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:27.016+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:27.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:27.016+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:27.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:27.016+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:27.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:27.016+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:27.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:27 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:27.016+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:27.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:27.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:27.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:27.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:27.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:27.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:27.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:27.035+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:27.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:27.441+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:27.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:27.441+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:27.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:27.441+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:27.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:27.441+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:27.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:27 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:27.441+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:28.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:27.698+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:28.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:27.698+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:28.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:27.698+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:28.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:27.698+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:28.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:27.698+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:28.069 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:27.604+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.687144+0000 front 2023-07-21T19:44:31.687283+0000 (oldest deadline 2023-07-21T19:44:57.587311+0000) 2023-07-21T19:45:28.069 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:27.604+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.189404+0000 front 2023-07-21T19:44:42.189523+0000 (oldest deadline 2023-07-21T19:45:06.289091+0000) 2023-07-21T19:45:28.069 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:27.604+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.691074+0000 front 2023-07-21T19:44:52.690875+0000 (oldest deadline 2023-07-21T19:45:14.990972+0000) 2023-07-21T19:45:28.069 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:27 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7[144357]: 2023-07-21T19:45:27.604+0000 7f60bf352700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.592727+0000 front 2023-07-21T19:44:59.592700+0000 (oldest deadline 2023-07-21T19:45:24.292800+0000) 2023-07-21T19:45:28.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:28.065+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:28.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:28.065+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:28.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:28.065+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:28.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:28.065+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:28.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:28 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:28.065+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:28.406 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:28.020+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:28.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:28.020+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:28.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:28.020+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:28.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:28.020+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:28.723 INFO:journalctl@ceph.osd.7.smithi101.stdout:Jul 21 19:45:28 smithi101 bash[166567]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-7 2023-07-21T19:45:28.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:28.486+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:28.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:28.486+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:28.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:28.486+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:28.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:28.486+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:28.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:28 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:28.486+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:29.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:28.735+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:29.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:28.735+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:29.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:28.735+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:29.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:28.735+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:29.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:28 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:28.735+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:29.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:29.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:29.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:29.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:29.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:29.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:29.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:29.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:29.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:29 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:29.042+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:29.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:29.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:29.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:29.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:29.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:29.514+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:29.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:29.514+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:29.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:29.514+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:29.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:29.514+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:29.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:29.514+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:30.034 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:29.771+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:30.034 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:29.771+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:30.034 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:29.771+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:30.034 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:29.771+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:30.034 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:29 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:29.771+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:30.382 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.7.service' 2023-07-21T19:45:30.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:30.081+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:30.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:30.081+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:30.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:30.081+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:30.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:30.081+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:30.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:30 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:30.081+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:30.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.977+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:30.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.977+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:30.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.977+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:30.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:29 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:29.977+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:30.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:30.470 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-07-21T19:45:30.471 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-07-21T19:45:30.471 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.8 2023-07-21T19:45:30.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.501+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:30.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.501+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:30.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.501+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:30.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.501+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:30.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.501+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:30.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 systemd[1]: Stopping Ceph osd.8 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:31.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:30.761+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:31.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:30.761+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:31.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:30.761+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:31.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:30.761+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:31.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:30.761+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:31.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:31.078+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:31.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:31.078+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:31.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:31.078+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:31.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:31.078+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:31.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:31 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:31.078+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:31.407 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.996+0000 7fe54b5eb700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:31.407 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.996+0000 7fe54b5eb700 -1 osd.8 149 *** Got signal Terminated *** 2023-07-21T19:45:31.407 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:30.996+0000 7fe54b5eb700 -1 osd.8 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:31.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:30.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:31.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:30.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:31.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:30.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:31.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:30 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:30.980+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:31.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:31.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:31.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:31.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:31.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:31.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:31.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:31.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:31.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:31.452+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:32.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:31.803+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:32.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:31.803+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:32.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:31.803+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:32.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:31.803+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:32.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:31 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:31.803+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:32.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:32.108+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:32.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:32.108+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:32.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:32.108+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:32.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:32.108+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:32.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:32 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:32.108+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:32.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:31.983+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:32.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:31.983+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:32.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:31.983+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:32.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:31 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:31.983+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:32.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:32.439+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:32.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:32.439+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:32.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:32.439+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:32.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:32.439+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:32.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:32 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:32.439+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:33.318 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:33.319 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:33.319 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:33.319 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:33.319 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:33.319 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:32 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:32.840+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:33.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:33.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:33.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:33.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:33.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:33.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:33.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:33.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:33.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:33 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:33.093+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:33.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.004+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:33.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.004+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:33.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.004+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:33.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.004+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:33.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:33.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:33.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:33.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:33.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:33.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:33.444+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:09.846918+0000 front 2023-07-21T19:45:09.847014+0000 (oldest deadline 2023-07-21T19:45:32.647740+0000) 2023-07-21T19:45:34.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:34.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:34.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:34.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:34.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:34.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:33.792+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:34.395 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:34.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:34.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:34.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:34.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:34.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:34 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:34.086+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:34.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.990+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:34.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.990+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:34.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.990+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:34.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:33 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:33.990+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:34.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:34.449+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:09.846918+0000 front 2023-07-21T19:45:09.847014+0000 (oldest deadline 2023-07-21T19:45:32.647740+0000) 2023-07-21T19:45:35.068 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:35.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:35.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:35.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:35.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:35.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:34 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:34.797+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:35.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:35.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:35.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:35.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:35.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:35.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:35 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:35.075+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:35.407 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:34.954+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:35.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:34.954+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:35.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:34.954+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:35.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:34 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:34.954+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:35.907 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.037018+0000 front 2023-07-21T19:44:22.036948+0000 (oldest deadline 2023-07-21T19:44:46.136852+0000) 2023-07-21T19:45:35.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:33.039626+0000 front 2023-07-21T19:44:33.039716+0000 (oldest deadline 2023-07-21T19:44:57.739927+0000) 2023-07-21T19:45:35.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:44.142215+0000 front 2023-07-21T19:44:44.142246+0000 (oldest deadline 2023-07-21T19:45:08.242421+0000) 2023-07-21T19:45:35.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:51.143359+0000 front 2023-07-21T19:44:51.143446+0000 (oldest deadline 2023-07-21T19:45:15.243596+0000) 2023-07-21T19:45:35.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:58.645966+0000 front 2023-07-21T19:44:58.645900+0000 (oldest deadline 2023-07-21T19:45:23.945859+0000) 2023-07-21T19:45:35.908 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:35 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8[145234]: 2023-07-21T19:45:35.430+0000 7fe544bdb700 -1 osd.8 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:09.846918+0000 front 2023-07-21T19:45:09.847014+0000 (oldest deadline 2023-07-21T19:45:32.647740+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:36.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:35 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:35.760+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:36.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:36.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:36.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:36.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:36.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:36.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.035+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:36.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.002+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:36.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.002+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:36.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.002+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:36.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.002+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:37.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:37.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:37.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:37.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:37.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:37.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:36 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:36.724+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:37.158 INFO:journalctl@ceph.osd.8.smithi156.stdout:Jul 21 19:45:36 smithi156 bash[168447]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-8 2023-07-21T19:45:37.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:37.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:37.159 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:37.159 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.971+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:37.159 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:36 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:36.971+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:37.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:37.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:37.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:37.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:37.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:37.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:36 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:36.987+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.686+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.687+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.687+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.687+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.687+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:38.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:37 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:37.687+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:38.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:38.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:38.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:38.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:38.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:38.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:38 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:37.999+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:38.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:37 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:37.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:38.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:37 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:37.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:38.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:37 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:37.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:38.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:37 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:37.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:38.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:37 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:37.970+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:38.818 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.8.service' 2023-07-21T19:45:38.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:38.909 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-07-21T19:45:38.909 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-07-21T19:45:38.910 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.9 2023-07-21T19:45:39.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:39.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:39.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:39.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:39.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:39.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:38.642+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:39.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:38 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:38.960+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:39.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:38 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:38.960+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:39.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:38 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:38.960+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:39.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:38 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:38.960+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:39.158 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:38 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:38.960+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:39.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:39.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:39.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:39.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:39.398 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:39.398 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.011+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:39.398 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 systemd[1]: Stopping Ceph osd.9 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:39.896 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.439+0000 7f66557aa700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:39.897 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.439+0000 7f66557aa700 -1 osd.9 149 *** Got signal Terminated *** 2023-07-21T19:45:39.897 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.439+0000 7f66557aa700 -1 osd.9 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:40.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:40.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:40.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:40.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:40.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:40.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:39 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:39.613+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:40.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:40.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:40.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:40.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:40.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:40.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:39 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:39.981+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:40.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:39.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:40.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:39.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:40.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:39.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:40.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:39.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:40.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:39 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:39.958+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:41.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:41.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:41.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:41.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:41.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:41.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:40 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:40.597+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:41.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:41.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:41.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:41.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:41.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:41.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:40 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:40.965+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:41.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:41.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:41.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:41.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:41.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:41.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:40 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:40.997+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:42.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:42.069 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:42.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:42.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:42.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:42.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:41 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:41.589+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:42.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:42.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:42.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:42.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:42.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:42.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:41 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:41.982+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:42.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:42.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:42.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:42.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:42.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:42.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:42 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.021+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:42.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:42.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:42.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:42.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:42.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:42.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:42 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:42.564+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:43.396 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:43.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:43.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:43.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:43.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:43.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:42 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:42.935+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:43.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:43.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:43.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:43.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:43.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:43.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:43 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:42.999+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:43.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:43.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:43.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:43.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:43.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:43.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:43 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:43.515+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:23.719064+0000 front 2023-07-21T19:44:23.719000+0000 (oldest deadline 2023-07-21T19:44:47.219142+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:32.421092+0000 front 2023-07-21T19:44:32.421124+0000 (oldest deadline 2023-07-21T19:44:57.121110+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.323837+0000 front 2023-07-21T19:44:42.323697+0000 (oldest deadline 2023-07-21T19:45:07.023467+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:48.724611+0000 front 2023-07-21T19:44:48.724578+0000 (oldest deadline 2023-07-21T19:45:17.525388+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.526184+0000 front 2023-07-21T19:44:57.526200+0000 (oldest deadline 2023-07-21T19:45:23.425983+0000) 2023-07-21T19:45:44.397 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:43 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9[155383]: 2023-07-21T19:45:43.958+0000 7f664dd98700 -1 osd.9 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:07.527645+0000 front 2023-07-21T19:45:07.527670+0000 (oldest deadline 2023-07-21T19:45:33.427546+0000) 2023-07-21T19:45:44.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:44.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:44.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:44.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:44.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:44.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:44.047+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:44.819 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:44.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:44 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:44.547+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:45.408 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:45.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:45.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:45.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:45.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:45.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:45 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:45.023+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:45.647 INFO:journalctl@ceph.osd.9.smithi005.stdout:Jul 21 19:45:45 smithi005 bash[214195]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-9 2023-07-21T19:45:45.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:45.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:45.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:45.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:45.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:45.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:45.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:45 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:45.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:46.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:46.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:46.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:46.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:46.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:46.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:46 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:46.009+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:46.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:46.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:46.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:46.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:46.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:46.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:46.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:46 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:46.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:47.173 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.9.service' 2023-07-21T19:45:47.270 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:47.271 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-07-21T19:45:47.271 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-07-21T19:45:47.271 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.10 2023-07-21T19:45:47.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:47.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:47.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:47.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:47.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:47.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:47 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:47.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:47.776 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 systemd[1]: Stopping Ceph osd.10 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:47.777 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.556+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:47.778 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.786+0000 7f9230bc5700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:47.778 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.786+0000 7f9230bc5700 -1 osd.10 149 *** Got signal Terminated *** 2023-07-21T19:45:47.778 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:47 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:47.786+0000 7f9230bc5700 -1 osd.10 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:48.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:48.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:48.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:48.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:48.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:48.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:48.031+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:48.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:48.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:48.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:48.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:48.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:48.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:48.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:48 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:48.528+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:49.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:49.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:49.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:49.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:49.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:49.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:49 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:49.081+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:49.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:49.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:49.820 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:49.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:49.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:49.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:49.821 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:49 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:49.570+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:50.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:50.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:50.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:50.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:50.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:50.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:50 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:50.121+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:51.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:51.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:51.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:51.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:51.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:51.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:51.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:50 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:50.581+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:51.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:51.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:51.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:51.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:51.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:51.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:51 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:51.075+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:52.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.625+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:52.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.625+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:52.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.625+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:52.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.625+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:52.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.626+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:52.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.626+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:52.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:51 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:51.626+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:52.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:52.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:52.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:52.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:52.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:52.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:52.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:52 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:52.045+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:53.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:21.748287+0000 front 2023-07-21T19:44:21.748277+0000 (oldest deadline 2023-07-21T19:44:45.248260+0000) 2023-07-21T19:45:53.070 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:30.549606+0000 front 2023-07-21T19:44:30.549569+0000 (oldest deadline 2023-07-21T19:44:55.849746+0000) 2023-07-21T19:45:53.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:42.152150+0000 front 2023-07-21T19:44:42.152222+0000 (oldest deadline 2023-07-21T19:45:08.052121+0000) 2023-07-21T19:45:53.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6814 osd.3 since back 2023-07-21T19:44:52.153496+0000 front 2023-07-21T19:44:52.153456+0000 (oldest deadline 2023-07-21T19:45:17.453331+0000) 2023-07-21T19:45:53.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:57.453772+0000 front 2023-07-21T19:44:57.453795+0000 (oldest deadline 2023-07-21T19:45:26.254523+0000) 2023-07-21T19:45:53.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.555725+0000 front 2023-07-21T19:45:08.555639+0000 (oldest deadline 2023-07-21T19:45:32.655595+0000) 2023-07-21T19:45:53.071 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:52 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10[149931]: 2023-07-21T19:45:52.627+0000 7f922d1bb700 -1 osd.10 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:13.756565+0000 front 2023-07-21T19:45:13.756525+0000 (oldest deadline 2023-07-21T19:45:44.357140+0000) 2023-07-21T19:45:53.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:53.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:53.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:54.320 INFO:journalctl@ceph.osd.10.smithi101.stdout:Jul 21 19:45:53 smithi101 bash[166844]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-10 2023-07-21T19:45:54.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:54.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:54.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:54.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:54.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:54.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:54.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:54 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:54.110+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:55.409 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:55.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:55.063+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:55.786 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.10.service' 2023-07-21T19:45:55.874 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:45:55.874 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-07-21T19:45:55.875 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-07-21T19:45:55.875 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.11 2023-07-21T19:45:56.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:55 smithi156 systemd[1]: Stopping Ceph osd.11 for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:45:56.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:56.382 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.099+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.391+0000 7f3bdfc9e700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:45:56.383 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.391+0000 7f3bdfc9e700 -1 osd.11 149 *** Got signal Terminated *** 2023-07-21T19:45:56.384 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:56 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:56.391+0000 7f3bdfc9e700 -1 osd.11 149 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-07-21T19:45:57.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:57.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:57.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:57.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:57.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:57.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:57.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:57 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:57.092+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:58.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:58.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:58.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:58.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:58.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:58.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:58.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:58 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:58.051+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:45:59.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:45:59.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:45:59.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:45:59.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:45:59.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:45:59.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:45:59.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:45:59 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:45:59.012+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:46:00.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:46:00.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:46:00.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.008+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6820 osd.8 since back 2023-07-21T19:45:34.319735+0000 front 2023-07-21T19:45:34.319758+0000 (oldest deadline 2023-07-21T19:45:59.619840+0000) 2023-07-21T19:46:01.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6806 osd.0 since back 2023-07-21T19:44:22.602915+0000 front 2023-07-21T19:44:22.602798+0000 (oldest deadline 2023-07-21T19:44:46.702879+0000) 2023-07-21T19:46:01.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6804 osd.1 since back 2023-07-21T19:44:31.305126+0000 front 2023-07-21T19:44:31.305319+0000 (oldest deadline 2023-07-21T19:44:55.405317+0000) 2023-07-21T19:46:01.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6804 osd.2 since back 2023-07-21T19:44:43.507997+0000 front 2023-07-21T19:44:43.508103+0000 (oldest deadline 2023-07-21T19:45:05.808151+0000) 2023-07-21T19:46:01.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6812 osd.4 since back 2023-07-21T19:44:59.811478+0000 front 2023-07-21T19:44:59.811505+0000 (oldest deadline 2023-07-21T19:45:22.711492+0000) 2023-07-21T19:46:01.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6812 osd.5 since back 2023-07-21T19:45:08.512952+0000 front 2023-07-21T19:45:08.513021+0000 (oldest deadline 2023-07-21T19:45:36.113649+0000) 2023-07-21T19:46:01.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.5:6822 osd.6 since back 2023-07-21T19:45:18.315682+0000 front 2023-07-21T19:45:18.315787+0000 (oldest deadline 2023-07-21T19:45:40.015493+0000) 2023-07-21T19:46:01.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.101:6820 osd.7 since back 2023-07-21T19:45:26.818241+0000 front 2023-07-21T19:45:26.818219+0000 (oldest deadline 2023-07-21T19:45:51.518051+0000) 2023-07-21T19:46:01.411 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11[150731]: 2023-07-21T19:46:00.962+0000 7f3bd8a8d700 -1 osd.11 149 heartbeat_check: no reply from 172.21.15.156:6820 osd.8 since back 2023-07-21T19:45:34.319735+0000 front 2023-07-21T19:45:34.319758+0000 (oldest deadline 2023-07-21T19:45:59.619840+0000) 2023-07-21T19:46:02.410 INFO:journalctl@ceph.osd.11.smithi156.stdout:Jul 21 19:46:02 smithi156 bash[168715]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-osd-11 2023-07-21T19:46:04.083 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@osd.11.service' 2023-07-21T19:46:04.172 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:04.173 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-07-21T19:46:04.173 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-07-21T19:46:04.174 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.a 2023-07-21T19:46:04.649 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:46:04 smithi005 systemd[1]: Stopping Ceph mds.a for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:05.148 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:46:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-a[160827]: 2023-07-21T19:46:04.685+0000 7fc5a50ba700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:05.149 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:46:04 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-a[160827]: 2023-07-21T19:46:04.685+0000 7fc5a50ba700 -1 mds.a *** got signal Terminated *** 2023-07-21T19:46:08.149 INFO:journalctl@ceph.mds.a.smithi005.stdout:Jul 21 19:46:07 smithi005 bash[214472]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-a 2023-07-21T19:46:08.331 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.a.service' 2023-07-21T19:46:08.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:08.427 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-07-21T19:46:08.427 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-07-21T19:46:08.427 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.d 2023-07-21T19:46:08.649 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:46:08 smithi005 systemd[1]: Stopping Ceph mds.d for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:09.399 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:46:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-d[159782]: 2023-07-21T19:46:08.968+0000 7f13c5623700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:09.400 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:46:08 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-d[159782]: 2023-07-21T19:46:08.968+0000 7f13c5623700 -1 mds.d *** got signal Terminated *** 2023-07-21T19:46:13.399 INFO:journalctl@ceph.mds.d.smithi005.stdout:Jul 21 19:46:13 smithi005 bash[214617]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-d 2023-07-21T19:46:13.763 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.d.service' 2023-07-21T19:46:13.854 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:13.855 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-07-21T19:46:13.856 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-07-21T19:46:13.856 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.g 2023-07-21T19:46:14.149 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:46:13 smithi005 systemd[1]: Stopping Ceph mds.g for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:14.649 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:46:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-g[160440]: 2023-07-21T19:46:14.351+0000 7f84cf516700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:14.650 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:46:14 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-g[160440]: 2023-07-21T19:46:14.351+0000 7f84cf516700 -1 mds.g *** got signal Terminated *** 2023-07-21T19:46:16.900 INFO:journalctl@ceph.mds.g.smithi005.stdout:Jul 21 19:46:16 smithi005 bash[214759]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-g 2023-07-21T19:46:17.113 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.g.service' 2023-07-21T19:46:17.203 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:17.203 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-07-21T19:46:17.204 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-07-21T19:46:17.204 DEBUG:teuthology.orchestra.run.smithi005:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.j 2023-07-21T19:46:17.400 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:17 smithi005 systemd[1]: Stopping Ceph mds.j for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:18.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-j[159346]: 2023-07-21T19:46:17.700+0000 7ff91622b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:18.150 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:17 smithi005 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-j[159346]: 2023-07-21T19:46:17.700+0000 7ff91622b700 -1 mds.j *** got signal Terminated *** 2023-07-21T19:46:22.133 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:21 smithi005 bash[214902]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-j 2023-07-21T19:46:22.335 DEBUG:teuthology.orchestra.run.smithi005:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.j.service' 2023-07-21T19:46:22.395 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:22 smithi005 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.j.service: Succeeded. 2023-07-21T19:46:22.395 INFO:journalctl@ceph.mds.j.smithi005.stdout:Jul 21 19:46:22 smithi005 systemd[1]: Stopped Ceph mds.j for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:46:22.423 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:22.424 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-07-21T19:46:22.424 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-07-21T19:46:22.425 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.b 2023-07-21T19:46:22.823 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:46:22 smithi101 systemd[1]: Stopping Ceph mds.b for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:23.323 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:46:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:46:23.008+0000 7fccf5272700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:23.323 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:46:23 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b[153742]: 2023-07-21T19:46:23.008+0000 7fccf5272700 -1 mds.b *** got signal Terminated *** 2023-07-21T19:46:29.824 INFO:journalctl@ceph.mds.b.smithi101.stdout:Jul 21 19:46:29 smithi101 bash[167108]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-b 2023-07-21T19:46:29.981 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.b.service' 2023-07-21T19:46:30.064 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:30.064 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-07-21T19:46:30.065 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-07-21T19:46:30.065 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e 2023-07-21T19:46:30.324 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:46:30 smithi101 systemd[1]: Stopping Ceph mds.e for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:31.074 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:46:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[160963]: 2023-07-21T19:46:30.615+0000 7fa01d300700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:31.074 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:46:30 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e[160963]: 2023-07-21T19:46:30.615+0000 7fa01d300700 -1 mds.e *** got signal Terminated *** 2023-07-21T19:46:32.574 INFO:journalctl@ceph.mds.e.smithi101.stdout:Jul 21 19:46:32 smithi101 bash[167254]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-e 2023-07-21T19:46:32.673 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.e.service' 2023-07-21T19:46:32.758 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:32.759 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-07-21T19:46:32.759 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-07-21T19:46:32.760 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.h 2023-07-21T19:46:33.074 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:46:32 smithi101 systemd[1]: Stopping Ceph mds.h for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:33.574 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:46:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-h[153387]: 2023-07-21T19:46:33.256+0000 7f4e18f5b700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:33.574 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:46:33 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-h[153387]: 2023-07-21T19:46:33.256+0000 7f4e18f5b700 -1 mds.h *** got signal Terminated *** 2023-07-21T19:46:38.324 INFO:journalctl@ceph.mds.h.smithi101.stdout:Jul 21 19:46:37 smithi101 bash[167403]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-h 2023-07-21T19:46:38.375 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.h.service' 2023-07-21T19:46:38.457 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:38.458 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-07-21T19:46:38.458 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-07-21T19:46:38.458 DEBUG:teuthology.orchestra.run.smithi101:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.k 2023-07-21T19:46:38.824 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:46:38 smithi101 systemd[1]: Stopping Ceph mds.k for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:39.325 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:46:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: 2023-07-21T19:46:38.972+0000 7f3c6d766700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:39.325 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:46:38 smithi101 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k[152706]: 2023-07-21T19:46:38.972+0000 7f3c6d766700 -1 mds.k *** got signal Terminated *** 2023-07-21T19:46:43.413 INFO:journalctl@ceph.mds.k.smithi101.stdout:Jul 21 19:46:43 smithi101 bash[167549]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-k 2023-07-21T19:46:43.447 DEBUG:teuthology.orchestra.run.smithi101:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.k.service' 2023-07-21T19:46:43.528 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:43.528 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-07-21T19:46:43.529 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-07-21T19:46:43.529 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.c 2023-07-21T19:46:43.914 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:46:43 smithi156 systemd[1]: Stopping Ceph mds.c for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:44.414 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:46:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-c[154442]: 2023-07-21T19:46:44.109+0000 7f4e83571700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:44.414 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:46:44 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-c[154442]: 2023-07-21T19:46:44.109+0000 7f4e83571700 -1 mds.c *** got signal Terminated *** 2023-07-21T19:46:48.298 INFO:journalctl@ceph.mds.c.smithi156.stdout:Jul 21 19:46:47 smithi156 bash[168996]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-c 2023-07-21T19:46:48.326 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.c.service' 2023-07-21T19:46:48.412 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:48.413 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-07-21T19:46:48.413 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-07-21T19:46:48.414 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.f 2023-07-21T19:46:48.666 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:48 smithi156 systemd[1]: Stopping Ceph mds.f for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:49.164 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-f[154091]: 2023-07-21T19:46:48.917+0000 7f71e3062700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:49.165 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:48 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-f[154091]: 2023-07-21T19:46:48.917+0000 7f71e3062700 -1 mds.f *** got signal Terminated *** 2023-07-21T19:46:52.557 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:52 smithi156 bash[169134]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-f 2023-07-21T19:46:52.810 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:52 smithi156 systemd[1]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.f.service: Succeeded. 2023-07-21T19:46:52.810 INFO:journalctl@ceph.mds.f.smithi156.stdout:Jul 21 19:46:52 smithi156 systemd[1]: Stopped Ceph mds.f for 3dc43aa4-27f8-11ee-9b34-001a4aab830c. 2023-07-21T19:46:52.813 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.f.service' 2023-07-21T19:46:52.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:52.910 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-07-21T19:46:52.910 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-07-21T19:46:52.911 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.i 2023-07-21T19:46:53.165 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:46:53 smithi156 systemd[1]: Stopping Ceph mds.i for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:46:53.778 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:46:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-i[154785]: 2023-07-21T19:46:53.495+0000 7fd98d5d0700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:46:53.778 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:46:53 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-i[154785]: 2023-07-21T19:46:53.495+0000 7fd98d5d0700 -1 mds.i *** got signal Terminated *** 2023-07-21T19:46:59.416 INFO:journalctl@ceph.mds.i.smithi156.stdout:Jul 21 19:46:59 smithi156 bash[169382]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-i 2023-07-21T19:46:59.557 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.i.service' 2023-07-21T19:46:59.649 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:46:59.649 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-07-21T19:46:59.650 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-07-21T19:46:59.650 DEBUG:teuthology.orchestra.run.smithi156:> sudo systemctl stop ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.l 2023-07-21T19:46:59.915 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:46:59 smithi156 systemd[1]: Stopping Ceph mds.l for 3dc43aa4-27f8-11ee-9b34-001a4aab830c... 2023-07-21T19:47:00.665 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:47:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-l[153746]: 2023-07-21T19:47:00.173+0000 7fbca8ba8700 -1 received signal: Terminated from /dev/init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-07-21T19:47:00.666 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:47:00 smithi156 ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-l[153746]: 2023-07-21T19:47:00.173+0000 7fbca8ba8700 -1 mds.l *** got signal Terminated *** 2023-07-21T19:47:04.641 INFO:journalctl@ceph.mds.l.smithi156.stdout:Jul 21 19:47:04 smithi156 bash[169550]: ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c-mds-l 2023-07-21T19:47:05.009 DEBUG:teuthology.orchestra.run.smithi156:> sudo pkill -f 'journalctl -f -n 0 -u ceph-3dc43aa4-27f8-11ee-9b34-001a4aab830c@mds.l.service' 2023-07-21T19:47:05.096 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-07-21T19:47:05.097 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-07-21T19:47:05.098 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:47:08.934 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:47:12.701 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force --keep-logs 2023-07-21T19:47:16.463 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:47:16.544 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:47:16.621 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-07-21T19:47:16.694 INFO:tasks.cephadm:Archiving crash dumps... 2023-07-21T19:47:16.696 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi005/crash 2023-07-21T19:47:16.697 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:47:16.789 INFO:teuthology.orchestra.run.smithi005.stderr:tar: /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:47:16.790 INFO:teuthology.orchestra.run.smithi005.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:47:16.793 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi101/crash 2023-07-21T19:47:16.794 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:47:16.877 INFO:teuthology.orchestra.run.smithi101.stderr:tar: /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:47:16.877 INFO:teuthology.orchestra.run.smithi101.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:47:16.880 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi156/crash 2023-07-21T19:47:16.881 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash -- . 2023-07-21T19:47:16.974 INFO:teuthology.orchestra.run.smithi156.stderr:tar: /var/lib/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/crash: Cannot open: No such file or directory 2023-07-21T19:47:16.974 INFO:teuthology.orchestra.run.smithi156.stderr:tar: Error is not recoverable: exiting now 2023-07-21T19:47:16.977 INFO:tasks.cephadm:Checking cluster log for badness... 2023-07-21T19:47:16.977 DEBUG:teuthology.orchestra.run.smithi005:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2023-07-21T19:47:17.060 INFO:teuthology.orchestra.run.smithi005.stderr:grep: /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph.log: No such file or directory 2023-07-21T19:47:17.116 INFO:tasks.cephadm:Compressing logs... 2023-07-21T19:47:17.116 DEBUG:teuthology.orchestra.run.smithi005:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:47:17.122 DEBUG:teuthology.orchestra.run.smithi101:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:47:17.129 DEBUG:teuthology.orchestra.run.smithi156:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-07-21T19:47:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:47:17.199 INFO:teuthology.orchestra.run.smithi005.stderr:: No such file or directory 2023-07-21T19:47:17.209 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.log 2023-07-21T19:47:17.209 INFO:teuthology.orchestra.run.smithi156.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:47:17.210 INFO:teuthology.orchestra.run.smithi156.stderr:: No such file or directory 2023-07-21T19:47:17.210 INFO:teuthology.orchestra.run.smithi101.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/cephadm.log 2023-07-21T19:47:17.211 INFO:teuthology.orchestra.run.smithi101.stderr:: No such file or directory 2023-07-21T19:47:17.212 INFO:teuthology.orchestra.run.smithi156.stderr:gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.5.log 2023-07-21T19:47:17.212 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.2.log 2023-07-21T19:47:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.0.log 2023-07-21T19:47:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.g.log 2023-07-21T19:47:17.212 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.a.log 2023-07-21T19:47:17.213 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.6.log 2023-07-21T19:47:17.214 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.11.log 2023-07-21T19:47:17.214 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.z.log 2023-07-21T19:47:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.3.log 2023-07-21T19:47:17.215 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.d.log 2023-07-21T19:47:17.215 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.8.log 2023-07-21T19:47:17.216 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.f.log 2023-07-21T19:47:17.216 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.c.log 2023-07-21T19:47:17.217 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.x.log 2023-07-21T19:47:17.218 INFO:teuthology.orchestra.run.smithi101.stderr:gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.10.log 2023-07-21T19:47:17.218 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.9.log 2023-07-21T19:47:17.219 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.i.log 2023-07-21T19:47:17.219 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.c.log 2023-07-21T19:47:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:47:17.220 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.9.log: 94.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.g.log.gz 2023-07-21T19:47:17.221 INFO:teuthology.orchestra.run.smithi005.stderr:gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.j.log 2023-07-21T19:47:17.221 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:47:17.221 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.l.log 2023-07-21T19:47:17.222 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.h.log 2023-07-21T19:47:17.222 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.e.log 2023-07-21T19:47:17.222 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.4.log 2023-07-21T19:47:17.222 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.1.log 2023-07-21T19:47:17.224 INFO:teuthology.orchestra.run.smithi005.stderr: 94.4% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.a.log.gz 2023-07-21T19:47:17.224 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.a.log 2023-07-21T19:47:17.224 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.k.log 2023-07-21T19:47:17.224 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log 2023-07-21T19:47:17.225 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.k.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.7.log 2023-07-21T19:47:17.225 INFO:teuthology.orchestra.run.smithi156.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log: /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.l.log: 93.0% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.z.log.gz 2023-07-21T19:47:17.225 INFO:teuthology.orchestra.run.smithi005.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.j.log: /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.a.log: 94.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.d.log.gz 2023-07-21T19:47:17.226 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.b.log 2023-07-21T19:47:17.226 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.b.log 2023-07-21T19:47:17.227 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.y.log 2023-07-21T19:47:17.228 INFO:teuthology.orchestra.run.smithi156.stderr: 94.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.l.log.gz 2023-07-21T19:47:17.229 INFO:teuthology.orchestra.run.smithi005.stderr: 94.6% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.j.log.gz 2023-07-21T19:47:17.239 INFO:teuthology.orchestra.run.smithi101.stderr:/var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.b.log: /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.y.log: 93.0% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.y.log.gz 2023-07-21T19:47:17.257 INFO:teuthology.orchestra.run.smithi156.stderr: 91.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:47:17.294 INFO:teuthology.orchestra.run.smithi156.stderr: 92.2% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:47:17.516 INFO:teuthology.orchestra.run.smithi101.stderr: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:47:17.749 INFO:teuthology.orchestra.run.smithi101.stderr: 91.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:47:17.965 INFO:teuthology.orchestra.run.smithi005.stderr: 91.5% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-volume.log.gz 2023-07-21T19:47:19.133 INFO:teuthology.orchestra.run.smithi005.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2023-07-21T19:47:19.498 INFO:teuthology.orchestra.run.smithi005.stderr: 89.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mgr.x.log.gz 2023-07-21T19:47:20.481 INFO:teuthology.orchestra.run.smithi156.stderr: 94.9% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.c.log.gz 2023-07-21T19:47:20.613 INFO:teuthology.orchestra.run.smithi156.stderr: 91.6% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.c.log.gz 2023-07-21T19:47:20.811 INFO:teuthology.orchestra.run.smithi156.stderr: 95.3% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.f.log.gz 2023-07-21T19:47:21.148 INFO:teuthology.orchestra.run.smithi101.stderr: 94.9% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.e.log.gz 2023-07-21T19:47:21.438 INFO:teuthology.orchestra.run.smithi101.stderr: 91.9% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.b.log.gz 2023-07-21T19:47:21.626 INFO:teuthology.orchestra.run.smithi101.stderr: 95.2% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.k.log.gz 2023-07-21T19:47:26.811 INFO:teuthology.orchestra.run.smithi005.stderr: 90.4% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mon.a.log.gz 2023-07-21T19:50:37.124 INFO:teuthology.orchestra.run.smithi005.stderr: 91.8% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.9.log.gz 2023-07-21T19:50:44.548 INFO:teuthology.orchestra.run.smithi156.stderr: 91.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.8.log.gz 2023-07-21T19:50:56.861 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.2.log.gz 2023-07-21T19:51:06.248 INFO:teuthology.orchestra.run.smithi005.stderr: 91.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.6.log.gz 2023-07-21T19:51:16.827 INFO:teuthology.orchestra.run.smithi156.stderr: 91.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.11.log.gz 2023-07-21T19:51:22.676 INFO:teuthology.orchestra.run.smithi005.stderr: 91.6% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.0.log.gz 2023-07-21T19:51:29.912 INFO:teuthology.orchestra.run.smithi156.stderr: 91.8% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.5.log.gz 2023-07-21T19:51:40.578 INFO:teuthology.orchestra.run.smithi005.stderr: 91.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.3.log.gz 2023-07-21T19:52:33.834 INFO:teuthology.orchestra.run.smithi101.stderr: 91.8% 92.0% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.10.log.gz -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.4.log.gz 2023-07-21T19:52:33.835 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-07-21T19:52:40.963 INFO:teuthology.orchestra.run.smithi101.stderr: 91.7% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.7.log.gz 2023-07-21T19:52:46.149 INFO:teuthology.orchestra.run.smithi101.stderr: 91.8% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-osd.1.log.gz 2023-07-21T19:53:21.106 INFO:teuthology.orchestra.run.smithi156.stderr: 94.8% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.i.log.gz 2023-07-21T19:53:21.110 INFO:teuthology.orchestra.run.smithi156.stderr: 2023-07-21T19:53:21.111 INFO:teuthology.orchestra.run.smithi156.stderr:real 6m3.916s 2023-07-21T19:53:21.111 INFO:teuthology.orchestra.run.smithi156.stderr:user 7m37.753s 2023-07-21T19:53:21.111 INFO:teuthology.orchestra.run.smithi156.stderr:sys 1m2.460s 2023-07-21T19:54:07.274 INFO:teuthology.orchestra.run.smithi005.stderr: 94.7% -- replaced with /var/log/ceph/ceph-client.0.log.gz 2023-07-21T19:54:07.277 INFO:teuthology.orchestra.run.smithi005.stderr: 2023-07-21T19:54:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:real 6m50.093s 2023-07-21T19:54:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:user 8m40.037s 2023-07-21T19:54:07.278 INFO:teuthology.orchestra.run.smithi005.stderr:sys 1m14.155s 2023-07-21T19:54:30.569 INFO:teuthology.orchestra.run.smithi101.stderr: 97.3% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.b.log.gz 2023-07-21T19:56:38.945 INFO:teuthology.orchestra.run.smithi101.stderr: 95.4% -- replaced with /var/log/ceph/3dc43aa4-27f8-11ee-9b34-001a4aab830c/ceph-mds.h.log.gz 2023-07-21T19:56:38.954 INFO:teuthology.orchestra.run.smithi101.stderr: 2023-07-21T19:56:38.954 INFO:teuthology.orchestra.run.smithi101.stderr:real 9m21.755s 2023-07-21T19:56:38.954 INFO:teuthology.orchestra.run.smithi101.stderr:user 10m27.582s 2023-07-21T19:56:38.954 INFO:teuthology.orchestra.run.smithi101.stderr:sys 1m37.128s 2023-07-21T19:56:38.955 INFO:tasks.cephadm:Archiving logs... 2023-07-21T19:56:38.956 DEBUG:teuthology.misc:Transferring archived files from smithi005:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi005/log 2023-07-21T19:56:38.957 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:57:43.139 DEBUG:teuthology.misc:Transferring archived files from smithi101:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi101/log 2023-07-21T19:57:43.141 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:58:55.282 DEBUG:teuthology.misc:Transferring archived files from smithi156:/var/log/ceph to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi156/log 2023-07-21T19:58:55.283 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /var/log/ceph -- . 2023-07-21T19:59:54.298 INFO:tasks.cephadm:Removing cluster... 2023-07-21T19:59:54.298 DEBUG:teuthology.orchestra.run.smithi005:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force 2023-07-21T19:59:56.780 DEBUG:teuthology.orchestra.run.smithi101:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force 2023-07-21T20:00:00.065 DEBUG:teuthology.orchestra.run.smithi156:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 3dc43aa4-27f8-11ee-9b34-001a4aab830c --force 2023-07-21T20:00:02.840 INFO:tasks.cephadm:Removing cephadm ... 2023-07-21T20:00:02.841 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T20:00:02.890 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T20:00:02.938 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf /home/ubuntu/cephtest/cephadm 2023-07-21T20:00:02.984 INFO:tasks.cephadm:Teardown complete 2023-07-21T20:00:02.985 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-07-21T20:00:02.999 ERROR:teuthology.contextutil:Saw exception from nested tasks Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/contextutil.py", line 32, in nested yield vars File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/task/install/__init__.py", line 626, in task yield File "/home/teuthworker/src/git.ceph.com_teuthology_79491cb69dd37a1eafb93b3471998174bdfa2fd9/teuthology/run_tasks.py", line 192, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 164, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/git.ceph.com_ceph-c_81cf785aa7781c4e08381c540f890cb9bc987ef9/qa/tasks/fwd_scrub.py", line 99, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-21T20:00:03.000 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-07-21T20:00:03.000 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T20:00:03.006 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T20:00:03.011 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-07-21T20:00:03.305 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-07-21T20:00:03.323 INFO:teuthology.task.clock:Checking final clock skew... 2023-07-21T20:00:03.323 DEBUG:teuthology.orchestra.run.smithi005:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T20:00:03.329 DEBUG:teuthology.orchestra.run.smithi101:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T20:00:03.334 DEBUG:teuthology.orchestra.run.smithi156:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-07-21T20:00:03.369 INFO:teuthology.orchestra.run.smithi005.stderr:bash: ntpq: command not found 2023-07-21T20:00:03.373 INFO:teuthology.orchestra.run.smithi101.stderr:bash: ntpq: command not found 2023-07-21T20:00:03.376 INFO:teuthology.orchestra.run.smithi156.stderr:bash: ntpq: command not found 2023-07-21T20:00:03.394 INFO:teuthology.orchestra.run.smithi101.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T20:00:03.394 INFO:teuthology.orchestra.run.smithi101.stdout:=============================================================================== 2023-07-21T20:00:03.394 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 30 +1410us[+1438us] +/- 86ms 2023-07-21T20:00:03.394 INFO:teuthology.orchestra.run.smithi101.stdout:^* hv02.front.sepia.ceph.com 3 6 377 29 -532us[ -503us] +/- 80ms 2023-07-21T20:00:03.394 INFO:teuthology.orchestra.run.smithi101.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 28 -660us[ -660us] +/- 75ms 2023-07-21T20:00:03.395 INFO:teuthology.orchestra.run.smithi101.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T20:00:03.396 INFO:teuthology.orchestra.run.smithi005.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T20:00:03.396 INFO:teuthology.orchestra.run.smithi005.stdout:=============================================================================== 2023-07-21T20:00:03.396 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +1386us[+1386us] +/- 86ms 2023-07-21T20:00:03.396 INFO:teuthology.orchestra.run.smithi005.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -504us[ -446us] +/- 80ms 2023-07-21T20:00:03.396 INFO:teuthology.orchestra.run.smithi005.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 32 -669us[ -612us] +/- 75ms 2023-07-21T20:00:03.397 INFO:teuthology.orchestra.run.smithi005.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T20:00:03.402 INFO:teuthology.orchestra.run.smithi156.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-07-21T20:00:03.402 INFO:teuthology.orchestra.run.smithi156.stdout:=============================================================================== 2023-07-21T20:00:03.403 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +1407us[+1445us] +/- 86ms 2023-07-21T20:00:03.403 INFO:teuthology.orchestra.run.smithi156.stdout:^* hv02.front.sepia.ceph.com 3 6 377 30 -530us[ -492us] +/- 80ms 2023-07-21T20:00:03.403 INFO:teuthology.orchestra.run.smithi156.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 27 -625us[ -625us] +/- 75ms 2023-07-21T20:00:03.403 INFO:teuthology.orchestra.run.smithi156.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-07-21T20:00:03.404 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-07-21T20:00:03.415 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-07-21T20:00:03.416 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-07-21T20:00:03.427 DEBUG:teuthology.orchestra.run.smithi005:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T20:00:03.685 DEBUG:teuthology.orchestra.run.smithi101:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T20:00:03.941 DEBUG:teuthology.orchestra.run.smithi156:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-07-21T20:00:04.202 DEBUG:teuthology.orchestra.run.smithi005:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T20:00:04.280 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:04.282 DEBUG:teuthology.orchestra.run.smithi101:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T20:00:04.357 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:04.358 DEBUG:teuthology.orchestra.run.smithi156:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-07-21T20:00:04.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:04.435 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-07-21T20:00:04.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-07-21T20:00:04.497 INFO:teuthology.task.internal:Duration was 5585.582319 seconds 2023-07-21T20:00:04.498 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-07-21T20:00:04.515 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-07-21T20:00:04.516 DEBUG:teuthology.orchestra.run.smithi005:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T20:00:04.521 DEBUG:teuthology.orchestra.run.smithi101:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T20:00:04.526 DEBUG:teuthology.orchestra.run.smithi156:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-07-21T20:00:04.675 INFO:teuthology.orchestra.run.smithi005.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T20:00:04.680 INFO:teuthology.orchestra.run.smithi101.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T20:00:04.685 INFO:teuthology.orchestra.run.smithi156.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-07-21T20:00:04.993 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-07-21T20:00:04.994 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi005.front.sepia.ceph.com 2023-07-21T20:00:04.994 DEBUG:teuthology.orchestra.run.smithi005:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T20:00:05.061 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi101.front.sepia.ceph.com 2023-07-21T20:00:05.062 DEBUG:teuthology.orchestra.run.smithi101:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T20:00:05.130 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi156.front.sepia.ceph.com 2023-07-21T20:00:05.130 DEBUG:teuthology.orchestra.run.smithi156:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-07-21T20:00:05.199 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-07-21T20:00:05.200 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T20:00:05.205 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T20:00:05.210 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-07-21T20:00:05.347 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-07-21T20:00:05.347 DEBUG:teuthology.orchestra.run.smithi005:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T20:00:05.354 DEBUG:teuthology.orchestra.run.smithi101:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T20:00:05.360 DEBUG:teuthology.orchestra.run.smithi156:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-07-21T20:00:05.538 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-07-21T20:00:05.552 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-07-21T20:00:05.553 DEBUG:teuthology.orchestra.run.smithi005:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T20:00:05.638 DEBUG:teuthology.orchestra.run.smithi101:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T20:00:05.709 DEBUG:teuthology.orchestra.run.smithi156:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-07-21T20:00:05.781 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-07-21T20:00:05.796 DEBUG:teuthology.orchestra.run.smithi005:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:05.802 DEBUG:teuthology.orchestra.run.smithi101:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:05.807 DEBUG:teuthology.orchestra.run.smithi156:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:05.880 INFO:teuthology.orchestra.run.smithi101.stdout:kernel.core_pattern = core 2023-07-21T20:00:05.895 INFO:teuthology.orchestra.run.smithi005.stdout:kernel.core_pattern = core 2023-07-21T20:00:05.898 INFO:teuthology.orchestra.run.smithi156.stdout:kernel.core_pattern = core 2023-07-21T20:00:05.935 DEBUG:teuthology.orchestra.run.smithi005:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:05.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:05.978 DEBUG:teuthology.orchestra.run.smithi101:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:06.020 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:06.021 DEBUG:teuthology.orchestra.run.smithi156:> test -e /home/ubuntu/cephtest/archive/coredump 2023-07-21T20:00:06.063 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-07-21T20:00:06.063 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-07-21T20:00:06.075 INFO:teuthology.task.internal:Transferring archived files... 2023-07-21T20:00:06.076 DEBUG:teuthology.misc:Transferring archived files from smithi005:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi005 2023-07-21T20:00:06.077 DEBUG:teuthology.orchestra.run.smithi005:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T20:00:06.169 DEBUG:teuthology.misc:Transferring archived files from smithi101:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi101 2023-07-21T20:00:06.170 DEBUG:teuthology.orchestra.run.smithi101:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T20:00:06.259 DEBUG:teuthology.misc:Transferring archived files from smithi156:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864/remote/smithi156 2023-07-21T20:00:06.260 DEBUG:teuthology.orchestra.run.smithi156:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-07-21T20:00:06.343 INFO:teuthology.task.internal:Removing archive directory... 2023-07-21T20:00:06.344 DEBUG:teuthology.orchestra.run.smithi005:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T20:00:06.348 DEBUG:teuthology.orchestra.run.smithi101:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T20:00:06.353 DEBUG:teuthology.orchestra.run.smithi156:> rm -rf -- /home/ubuntu/cephtest/archive 2023-07-21T20:00:06.397 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-07-21T20:00:06.411 INFO:teuthology.task.internal:Not uploading archives. 2023-07-21T20:00:06.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-07-21T20:00:06.423 INFO:teuthology.task.internal:Tidying up after the test... 2023-07-21T20:00:06.423 DEBUG:teuthology.orchestra.run.smithi005:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T20:00:06.432 DEBUG:teuthology.orchestra.run.smithi101:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T20:00:06.437 DEBUG:teuthology.orchestra.run.smithi156:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-07-21T20:00:06.472 INFO:teuthology.orchestra.run.smithi005.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 20:00 /home/ubuntu/cephtest 2023-07-21T20:00:06.476 INFO:teuthology.orchestra.run.smithi101.stdout: 524301 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 20:00 /home/ubuntu/cephtest 2023-07-21T20:00:06.480 INFO:teuthology.orchestra.run.smithi156.stdout: 524293 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Jul 21 20:00 /home/ubuntu/cephtest 2023-07-21T20:00:06.484 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-07-21T20:00:06.505 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-07-21T20:00:06.522 DEBUG:teuthology.run_tasks:Exception was not quenched, exiting: RuntimeError: error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds 2023-07-21T20:00:06.522 INFO:teuthology.nuke:Checking targets against current locks 2023-07-21T20:00:06.554 DEBUG:teuthology.nuke:shortname: smithi005 2023-07-21T20:00:06.555 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T20:00:06.581 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi005.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.013420', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCpUxo5ay/ucdkX9rleNMm8Ofp24XRXUtJPOK3Ji3XBpTiUgmmwneaY6316+3CG+k4K0p3IebanuyTvZApjm02/dNCvu8yD3rjEpWVgIuqYH8O3lEdN9DXcCX+7xHJE0m18+ZPsJQVsNrNJxY+5zqyNnBc8jpZq2LStsWwPOKPWhauANRkOBFSGDoLppWZbLx/yJs+R0qoNLLK9LpKQTUnqsapXvbD27OpPahzj/qbNxFPPyY6XrrldZ6qEO+EjJOujFLH3gzv0kJu10DCuzxrpz6ston4RPi1O3GgcHnF4Mqr/W9VwZ94UTMh+k4dIFRVqJEKrcwRHa5kFsvH/WhVx0Ygr/MiV/Q86mmV7JT+/MpzIRTtWyei5L9IBeZIE9cwzj+mOk6SD/t1roIVEGVV/uF2AyFDVBPr6F8oZ822OGpUyyaDsWTitPNdTX4TC3zdSrLc9tyDa1P74eH/ZLeTSf+C6pM/cepOhQaa6XzpDiPSuJUm9HOQxUlEAvcqa7Ac='} 2023-07-21T20:00:06.588 DEBUG:teuthology.nuke:shortname: smithi101 2023-07-21T20:00:06.588 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T20:00:06.614 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi101.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.010484', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T20:00:06.620 DEBUG:teuthology.nuke:shortname: smithi156 2023-07-21T20:00:06.620 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-07-21T20:00:06.649 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi156.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/rishabh-2023-07-21_17:58:19-fs-wip-rishabh-2023Jul13-m-quota-testing-default-smithi/7346864', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-07-21 18:09:09.007635', 'locked_by': 'scheduled_rishabh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-07-21T20:00:06.661 INFO:teuthology.orchestra.console:Power off smithi005 2023-07-21T20:00:06.662 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T20:00:06.682 INFO:teuthology.orchestra.console:Power off smithi101 2023-07-21T20:00:06.683 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T20:00:06.699 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:06.712 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:06.742 INFO:teuthology.orchestra.console:Power off smithi156 2023-07-21T20:00:06.743 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-07-21T20:00:06.767 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:10.727 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:10.756 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:10.794 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:14.844 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi005.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:14.868 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi101.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:14.908 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi156.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-07-21T20:00:15.057 INFO:teuthology.orchestra.console:Power off for smithi005 completed 2023-07-21T20:00:15.083 INFO:teuthology.orchestra.console:Power off for smithi101 completed 2023-07-21T20:00:15.124 INFO:teuthology.orchestra.console:Power off for smithi156 completed 2023-07-21T20:00:15.230 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/fuse objectstore-ec/bluestore-comp omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-workunit/postgres}} duration: 5585.582318544388 failure_reason: 'error during scrub thrashing: reached maximum tries (31) after waiting for 900 seconds' flavor: default mon.a-kernel-sha1: c685da6f56cc77f2cfb9c4221be3960522616caf mon.b-kernel-sha1: c685da6f56cc77f2cfb9c4221be3960522616caf mon.c-kernel-sha1: c685da6f56cc77f2cfb9c4221be3960522616caf owner: scheduled_rishabh@teuthology status: fail success: false 2023-07-21T20:00:15.231 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-07-21T20:00:15.320 INFO:teuthology.run:FAIL